48 Commits
v0.1.9 ... main

Author SHA1 Message Date
5f7503598c feat: govern attachment exceptions and ownership transfers 2026-07-30 17:42:10 +02:00
cd223cbb95 feat: harden campaign delivery and editing 2026-07-30 14:27:04 +02:00
4a120e8009 Complete recipient search and delivery evidence 2026-07-30 05:22:09 +02:00
c769be39da perf(campaign): search share targets server-side 2026-07-30 01:30:13 +02:00
46df12c025 perf(campaign): batch tenant summaries 2026-07-30 01:15:38 +02:00
961d5d1130 refactor(campaign): split review and recipient boundaries 2026-07-30 01:01:26 +02:00
23b9a531d5 fix: remove duplicate Campaign translation key 2026-07-30 00:39:58 +02:00
cc93945f79 feat: integrate reports into Campaign navigation 2026-07-29 22:19:24 +02:00
101f3ccd7d feat: integrate operator queue into Campaign 2026-07-29 21:10:37 +02:00
2199187e8b refactor(api): split campaign workflow routers 2026-07-29 20:07:11 +02:00
dd9592a192 Refactor campaign delivery decision paths 2026-07-29 17:06:15 +02:00
5240749ae1 feat: add governed postbox delivery and report hardening 2026-07-29 14:16:28 +02:00
f11c56e890 fix: make direct campaign pages scrollable 2026-07-28 22:50:11 +02:00
8d0a2608ed Declare campaign route View surfaces 2026-07-28 21:04:55 +02:00
89ae14c032 Integrate campaign delivery with hierarchical mail profiles 2026-07-28 19:33:12 +02:00
38af25ee88 docs: update GovOPlaN repository links 2026-07-27 15:46:51 +02:00
4774a8025c fix(campaign): roll back rejected immediate queues 2026-07-22 20:30:00 +02:00
689dc1fd6b fix(campaign): honor acceptance temp selection 2026-07-22 20:20:31 +02:00
90677348ff docs(campaign): define broker redelivery evidence 2026-07-22 20:19:51 +02:00
06786e86ef test(campaign): prove Celery broker redelivery 2026-07-22 20:19:29 +02:00
6fda123fc3 docs(campaign): distinguish worker process evidence 2026-07-22 18:26:42 +02:00
2fa91bb943 test(campaign): complete delivery fault drills 2026-07-22 18:26:28 +02:00
e3cc476508 fix(campaign): initialize delivery worker runtime 2026-07-22 18:26:11 +02:00
01ef541917 docs(campaign): define target-like mail evidence 2026-07-22 16:49:57 +02:00
d567257311 test(campaign): prove GreenMail delivery journey 2026-07-22 16:49:57 +02:00
3075ef7f5b docs(campaign): document aggregate report baseline 2026-07-22 15:20:04 +02:00
1ab4e91ffd test(campaign): add simple announcement acceptance fixture 2026-07-22 15:18:52 +02:00
735e874bd0 fix(campaign): bound operator queue polling 2026-07-22 09:39:03 +02:00
99d44eeb8d feat(campaign): complete durable operator queue 2026-07-22 09:31:33 +02:00
f095a3e2c7 fix(campaign): sanitize synchronous send results 2026-07-22 09:21:44 +02:00
1225802c5d fix(campaign): suppress overlapping aggregate cells 2026-07-22 09:17:00 +02:00
ac3329cafe fix(campaign): use stable aggregate status filters 2026-07-22 09:11:14 +02:00
4eb651c6ac feat(campaign): filter reports from outcome counts 2026-07-22 09:11:10 +02:00
0b4017c240 refactor(campaign): restore route docstrings 2026-07-22 09:08:12 +02:00
79b576b4bc fix(webui): localize Campaign delivery reporting 2026-07-22 09:04:42 +02:00
b0282ebff2 test(campaign): cover bounded delivery modes 2026-07-22 09:00:35 +02:00
aae4ef5952 chore(release): bump Campaign to 0.1.10 2026-07-22 08:50:53 +02:00
8ee87b7558 feat(campaign): surface aggregate reports 2026-07-22 08:49:38 +02:00
7229fb8e3d fix(campaign): mark excluded delivery as skipped 2026-07-22 08:48:46 +02:00
3487ec7048 test(campaign): enforce aggregate report permissions 2026-07-22 08:46:28 +02:00
22d72f82f5 test(campaign): require export authority for report email 2026-07-22 08:45:00 +02:00
fc36aee6c0 feat(webui): add aggregate Campaign reports 2026-07-22 08:44:15 +02:00
06125cc0e8 feat(campaign): add privacy-safe aggregate reports 2026-07-22 08:44:12 +02:00
21f3014ac5 feat(campaign): add durable operator queue controls 2026-07-22 08:39:44 +02:00
62a68792a4 feat(campaign): persist delivery execution mode 2026-07-22 08:37:58 +02:00
aa4ec66b7b fix(campaign): query reports before pagination 2026-07-22 08:31:43 +02:00
60efd1cb5d feat(campaign): make delivery modes explicit 2026-07-22 08:26:52 +02:00
7e1660344d feat(campaign): bound synchronous delivery 2026-07-22 08:21:42 +02:00
151 changed files with 30295 additions and 6141 deletions

1
.gitignore vendored
View File

@@ -333,6 +333,7 @@ webui/.policy-test-build/
webui/.template-preview-test-build/ webui/.template-preview-test-build/
webui/.import-test-build/ webui/.import-test-build/
webui/.review-preview-test-build/ webui/.review-preview-test-build/
webui/.report-grid-test-build/
# GovOPlaN shared ignore rules from govoplan-core # GovOPlaN shared ignore rules from govoplan-core
# Local WebUI test/build scratch directories # Local WebUI test/build scratch directories

View File

@@ -15,7 +15,8 @@ This repository owns:
- campaign/version/job/issue/send-attempt/append-attempt models and migrations - campaign/version/job/issue/send-attempt/append-attempt models and migrations
- campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths - campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths
- WebUI package `@govoplan/campaign-webui` - WebUI package `@govoplan/campaign-webui`
- route contributions for `/campaigns`, `/campaigns/:campaignId/*`, `/operator`, `/reports`, and `/templates` - route contributions for `/campaigns`, the integrated `/campaigns/queue` view,
`/campaigns/:campaignId/*`, `/reports`, and `/templates`
Core owns the auth facade, RBAC/capability contracts, database/session Core owns the auth facade, RBAC/capability contracts, database/session
primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an
@@ -93,7 +94,8 @@ Platform RBAC and governance rules are documented in `govoplan-core/docs/`.
- [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives. - [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives.
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist. - [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested. - Immediate delivery is bounded to 25 exact eligible recipient jobs by default. Deployments may set `GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0500), and tenants may narrow that ceiling through `campaign_delivery_policy.synchronous_send_max_recipients` in tenant settings.
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
- [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path. - [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path.
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence. - [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.
- [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management. - [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management.

View File

@@ -9,3 +9,5 @@ GOVOPLAN_MAIL_TEST_IMAP_PORT=3143
GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent
GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password
GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45 GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45
GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT=36379
GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS=3

View File

@@ -44,6 +44,111 @@ If the smoke is started immediately after `docker compose up -d`, GreenMail may
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`. login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
## Campaign Acceptance
The transport smoke proves the Mail adapters. The Campaign acceptance runner
proves the public composition: it creates an isolated temporary Core database,
creates a Mail-owned encrypted profile through the API, materializes the
credential-free [`greenmail-delivery`](../../examples/greenmail-delivery/campaign.json)
fixture with only that profile reference, validates/builds it, sends the exact
generated EML through Campaign once, appends it once, and cross-checks Campaign
report/audit state with one unique-subject message in the GreenMail INBOX and
Sent folders. Provider mailbox verification is not a byte-for-byte comparison
after provider-side header or storage transformations.
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
set -a
. ./.env
set +a
/mnt/DATA/git/govoplan/.venv/bin/python run_campaign_acceptance.py \
--evidence /tmp/govoplan-campaign-greenmail-evidence.json
```
The default run also uses controlled loopback protocol endpoints to prove that
an SMTP connection loss before transmission is temporary, an explicit SMTP
authentication rejection is permanent, a final `451` response after DATA is
temporary, one accepted and one refused RCPT command is retained as partial
envelope acceptance, a connection loss after complete DATA is frozen as
`outcome_unknown`, and an IMAP authentication rejection after SMTP acceptance
leaves the send accepted while the append fails. A
second ordinary send must be rejected before another provider effect.
The worker drill queues one job, starts the registered
`govoplan.campaigns.send_email` task body in a dedicated OS process, waits until
the controlled endpoint has received complete DATA, terminates that process,
and starts the same task body in a fresh process. It proves the durable
`sending`/unfinished-attempt boundary is recovered as `outcome_unknown`
without a second SMTP connection or DATA transaction. This is a real process
and task-boundary interruption, but it does not start a Celery daemon, Redis
broker, or broker redelivery; `celery_broker_redelivery` therefore remains
`false` in the evidence.
The bounded JSON contains no endpoint, account, address, credential, profile,
campaign, version, or job identifiers. It records module versions, the fixture
hash, normalized classifications/counts, the Mail-profile boundary, required
audit actions, provider mailbox increments, and coverage flags. Runtime version
declarations identify the exercised composition; they do not claim that the
sources are clean, tagged, signed, or release-provenanced. The evidence names
them `declared_module_versions` and keeps `source_artifact_provenance` false;
exact commit/artifact provenance belongs to the package and release gate.
This runner is restricted to literal loopback IP addresses and the synchronous
Campaign delivery mode. Hostnames such as `localhost` and every non-loopback
address fail before profile creation, avoiding a DNS change between validation
and connection. It is local target-like evidence, not approval of an
institution's SMTP/IMAP service. The controlled post-DATA, temporary-response,
partial-refusal, and task-process interruption drills are local effect-level
proof, not proof of a target provider's behavior or Redis/Celery broker
redelivery. Use `--success-only` only when testing the success journey without
the local failure endpoints.
## Redis/Celery Redelivery Acceptance
Run the maintained broker/worker-loss acceptance separately from the GreenMail
journey:
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
set -a
. ./.env
set +a
/mnt/DATA/git/govoplan/.venv/bin/python run_celery_redelivery_acceptance.py \
--evidence /tmp/govoplan-campaign-celery-redelivery-evidence.json
```
The runner creates a unique Compose project, starts only its loopback-bound,
AOF-enabled Redis service, creates an isolated temporary GovOPlaN database,
and starts a real Celery worker subscribed to `send_email`. A controlled SMTP
server holds the transaction after complete DATA and before the final response.
The runner kills that solo worker with the task still unacknowledged and starts
a replacement worker. After the configured Redis visibility timeout, the same Celery task identity must be redelivered.
The replacement must turn the durable
unfinished attempt into `outcome_unknown`, acknowledge the task, drain the
broker queue/unacked records, and leave the SMTP endpoint at exactly one
connection and one DATA transaction.
Only bounded counts, classifications, and booleans are retained. Worker logs,
task IDs, database identifiers, endpoints, credentials, and raw diagnostics are
kept in the temporary runtime and deleted. The evidence proves the local Redis
transport, real Celery process boundary, runner-supervised replacement, and
Campaign's duplicate-effect guard. It deliberately keeps production daemon supervision,
target-provider behavior, and source-artifact provenance false.
It does not claim that systemd, Kubernetes, another container orchestrator, or
an institution's Redis/SMTP deployment behaves identically.
The default run requires Docker CLI/Compose/daemon access and permission to
pull `redis:7-alpine`; the Celery workers execute from the current Python
environment. The isolated Compose project and volume are removed on exit.
`GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS` defaults to three
seconds only to make this destructive local drill finish promptly; it is not a
production recommendation.
Starting the maintained test bed requires a working Docker CLI, Compose plugin,
daemon/socket access, and permission to pull `greenmail/standalone:2.1.9`. The
Campaign runner needs only the already-running loopback endpoints; it neither
starts Docker nor claims that it did.
## Use With A Campaign ## Use With A Campaign
Use the same settings in a campaign mail profile: Use the same settings in a campaign mail profile:

View File

@@ -15,3 +15,19 @@ services:
- "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025" - "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025"
- "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143" - "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143"
- "127.0.0.1:38080:8080" - "127.0.0.1:38080:8080"
redis:
image: redis:7-alpine
command: ["redis-server", "--appendonly", "yes"]
ports:
- "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379"
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 1s
timeout: 1s
retries: 30
volumes:
- campaign-redis-data:/data
volumes:
campaign-redis-data:

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,856 @@
#!/usr/bin/env python3
"""Prove Redis/Celery redelivery does not repeat an ambiguous SMTP effect.
The default run starts an isolated Redis Compose service, two successive real
Celery worker processes, and a controlled loopback SMTP endpoint. It kills the
first worker after complete DATA but before a final SMTP response. The same
unacknowledged broker task must be delivered to the replacement worker, which
must freeze the unfinished durable attempt as ``outcome_unknown`` without a
second SMTP connection or DATA transaction.
"""
from __future__ import annotations
import argparse
from collections import Counter
from contextlib import contextmanager
from dataclasses import dataclass, replace
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import socket
import subprocess
import sys
import tempfile
import time
from typing import Any, Callable, Iterator, Mapping
from uuid import uuid4
from redis import Redis
from redis.exceptions import RedisError
SCRIPT_ROOT = Path(__file__).resolve().parent
REPOSITORY_ROOT = SCRIPT_ROOT.parents[1]
if str(SCRIPT_ROOT) not in sys.path:
sys.path.insert(0, str(SCRIPT_ROOT))
from run_campaign_acceptance import ( # noqa: E402
AcceptanceError,
DEFAULT_FIXTURE,
EXPECTED_AUDIT_ACTIONS,
TestbedSettings,
_assert_evidence_safe,
_core_package_version,
_durable_state_evidence,
_expect,
_report_evidence,
create_mail_profile,
prepare_campaign_scenario,
required_composition_versions,
smtp_fault_endpoint,
)
EVIDENCE_SCHEMA = "govoplan.campaign.celery-redelivery-acceptance.v1"
MAX_EVIDENCE_BYTES = 128 * 1024
DEFAULT_COMPOSE_FILE = SCRIPT_ROOT / "docker-compose.yml"
TASK_RECEIVED_PATTERN = re.compile(
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] received",
re.IGNORECASE,
)
TASK_SUCCEEDED_PATTERN = re.compile(
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] succeeded",
re.IGNORECASE,
)
WORKER_BOOTSTRAP = r"""
import os
import sys
from govoplan_core.celery_app import celery
visibility_timeout = int(os.environ["GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS"])
celery.conf.broker_transport_options = {
**dict(celery.conf.broker_transport_options or {}),
"polling_interval": 0.25,
"visibility_timeout": visibility_timeout,
}
celery.worker_main(
[
"worker",
"--loglevel=INFO",
"--pool=solo",
"--concurrency=1",
"--queues=send_email",
f"--hostname={sys.argv[1]}@%h",
"--without-gossip",
"--without-mingle",
"--without-heartbeat",
]
)
"""
@dataclass(slots=True)
class WorkerProcess:
process: subprocess.Popen[bytes]
log_path: Path
log_handle: Any
def text(self) -> str:
self.log_handle.flush()
try:
return self.log_path.read_text(encoding="utf-8", errors="replace")
except OSError as exc:
raise AcceptanceError("Celery worker evidence log could not be read") from exc
def received_task_ids(self) -> tuple[str, ...]:
return tuple(TASK_RECEIVED_PATTERN.findall(self.text()))
def succeeded_task_ids(self) -> tuple[str, ...]:
return tuple(TASK_SUCCEEDED_PATTERN.findall(self.text()))
@dataclass(frozen=True, slots=True)
class RedisBrokerState:
queue_depth: int
unacked_hash_count: int
unacked_index_count: int
def as_dict(self) -> dict[str, int]:
return {
"queue_depth": self.queue_depth,
"unacked_hash_count": self.unacked_hash_count,
"unacked_index_count": self.unacked_index_count,
}
def _positive_int(value: str, *, label: str) -> int:
try:
parsed = int(value)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"{label} must be a positive integer") from exc
if parsed <= 0:
raise argparse.ArgumentTypeError(f"{label} must be a positive integer")
return parsed
def _unused_loopback_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as probe:
probe.bind(("127.0.0.1", 0))
return int(probe.getsockname()[1])
def _compose_command(
*, compose_file: Path, project_name: str, operation: str
) -> list[str]:
prefix = [
"docker",
"compose",
"--file",
str(compose_file),
"--project-name",
project_name,
]
if operation == "up":
return [*prefix, "up", "--detach", "redis"]
if operation == "down":
return [*prefix, "down", "--volumes", "--remove-orphans"]
raise AcceptanceError("Unsupported Redis Compose operation")
def _run_compose(
command: list[str],
*,
environment: Mapping[str, str],
timeout_seconds: int,
) -> None:
try:
completed = subprocess.run(
command,
env=dict(environment),
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout_seconds,
check=False,
)
except (OSError, subprocess.TimeoutExpired) as exc:
raise AcceptanceError("Redis Compose lifecycle command failed") from exc
if completed.returncode != 0:
raise AcceptanceError("Redis Compose lifecycle command failed")
def _wait_for_redis(redis_url: str, *, timeout_seconds: int) -> None:
deadline = time.monotonic() + timeout_seconds
client = Redis.from_url(
redis_url,
socket_connect_timeout=1,
socket_timeout=1,
decode_responses=False,
)
try:
while time.monotonic() < deadline:
try:
if client.ping() is True:
return
except RedisError:
pass
time.sleep(0.25)
finally:
client.close()
raise AcceptanceError("Isolated Redis broker did not become ready")
@contextmanager
def isolated_redis_broker(
*,
compose_file: Path,
timeout_seconds: int,
requested_port: int | None = None,
) -> Iterator[str]:
if shutil.which("docker") is None:
raise AcceptanceError("Docker CLI is required to start the isolated Redis broker")
if not compose_file.is_file():
raise AcceptanceError("Redis Compose definition is unavailable")
port = requested_port or _unused_loopback_port()
if port <= 0 or port > 65_535:
raise AcceptanceError("Redis test port is invalid")
project_name = f"govoplan-campaign-redelivery-{uuid4().hex[:12]}"
environment = {
**os.environ,
"GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT": str(port),
}
lifecycle_attempted = False
try:
lifecycle_attempted = True
_run_compose(
_compose_command(
compose_file=compose_file,
project_name=project_name,
operation="up",
),
environment=environment,
timeout_seconds=timeout_seconds,
)
redis_url = f"redis://127.0.0.1:{port}/0"
_wait_for_redis(redis_url, timeout_seconds=timeout_seconds)
yield redis_url
finally:
if lifecycle_attempted:
_run_compose(
_compose_command(
compose_file=compose_file,
project_name=project_name,
operation="down",
),
environment=environment,
timeout_seconds=timeout_seconds,
)
def _start_worker(runtime_root: Path, *, label: str) -> WorkerProcess:
log_path = runtime_root / f"{label}.log"
log_handle = log_path.open("wb")
environment = {**os.environ, "PYTHONUNBUFFERED": "1"}
try:
process = subprocess.Popen(
[sys.executable, "-c", WORKER_BOOTSTRAP, label],
env=environment,
stdin=subprocess.DEVNULL,
stdout=log_handle,
stderr=subprocess.STDOUT,
close_fds=True,
)
except Exception:
log_handle.close()
raise
return WorkerProcess(process=process, log_path=log_path, log_handle=log_handle)
def _wait_for_worker_ready(worker: WorkerProcess, *, timeout_seconds: int) -> None:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before becoming ready")
if " ready." in worker.text():
return
time.sleep(0.2)
raise AcceptanceError("Celery worker did not become ready")
def _wait_for_received_task(
worker: WorkerProcess,
*,
timeout_seconds: int,
expected_task_id: str | None = None,
) -> str:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
received = worker.received_task_ids()
if received:
if len(set(received)) != 1:
raise AcceptanceError("Celery worker received more than one task identity")
task_id = received[0]
if expected_task_id is not None and task_id != expected_task_id:
raise AcceptanceError("Replacement worker received a different broker task")
return task_id
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before receiving the task")
time.sleep(0.2)
raise AcceptanceError("Celery worker did not receive the broker task")
def _wait_for_task_success(
worker: WorkerProcess,
*,
task_id: str,
timeout_seconds: int,
) -> None:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
succeeded = worker.succeeded_task_ids()
if task_id in succeeded:
return
if worker.process.poll() is not None:
raise AcceptanceError("Replacement Celery worker exited before task success")
time.sleep(0.2)
raise AcceptanceError("Redelivered Celery task did not complete")
def _kill_worker(worker: WorkerProcess, *, timeout_seconds: int) -> int:
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before controlled termination")
worker.process.kill()
try:
return_code = worker.process.wait(timeout=timeout_seconds)
except subprocess.TimeoutExpired as exc:
raise AcceptanceError("Celery worker could not be killed") from exc
if return_code == 0:
raise AcceptanceError("Celery worker termination was not forced")
return return_code
def _stop_worker(worker: WorkerProcess, *, timeout_seconds: int) -> None:
if worker.process.poll() is None:
worker.process.terminate()
try:
worker.process.wait(timeout=timeout_seconds)
except subprocess.TimeoutExpired:
worker.process.kill()
worker.process.wait(timeout=timeout_seconds)
worker.log_handle.close()
def _broker_state(redis_url: str) -> RedisBrokerState:
client = Redis.from_url(
redis_url,
socket_connect_timeout=2,
socket_timeout=2,
decode_responses=False,
)
try:
return RedisBrokerState(
queue_depth=int(client.llen("send_email")),
unacked_hash_count=int(client.hlen("unacked")),
unacked_index_count=int(client.zcard("unacked_index")),
)
except RedisError as exc:
raise AcceptanceError("Redis broker state could not be inspected") from exc
finally:
client.close()
def _wait_for_broker_drained(
redis_url: str,
*,
timeout_seconds: int,
) -> RedisBrokerState:
deadline = time.monotonic() + timeout_seconds
last = RedisBrokerState(0, 0, 0)
while time.monotonic() < deadline:
last = _broker_state(redis_url)
if last == RedisBrokerState(0, 0, 0):
return last
time.sleep(0.2)
raise AcceptanceError("Redis broker retained delivery state after recovery")
def _queue_evidence(payload: Mapping[str, Any]) -> dict[str, Any]:
expected = {
"queued_count": 1,
"skipped_count": 0,
"blocked_count": 0,
"enqueued_count": 1,
"delivery_mode": "worker_queue",
"worker_queue_available": True,
"dry_run": False,
}
evidence = {key: payload.get(key) for key in expected}
if evidence != expected:
raise AcceptanceError("Campaign was not durably queued to one Celery task")
return evidence
def execute_redelivery_scenario(
client: Any,
headers: Mapping[str, str],
*,
fixture_path: Path,
settings: TestbedSettings,
endpoint: Any,
redis_url: str,
runtime_root: Path,
snapshot_probe: Callable[[str], tuple[Mapping[str, Any], Mapping[str, Any]]],
audit_probe: Callable[[str, str], Mapping[str, int]],
delivery_probe: Callable[[str, str], Mapping[str, Any]],
) -> dict[str, Any]:
profile_id = create_mail_profile(
client,
headers,
settings,
name="Campaign Redis Celery redelivery drill",
smtp_host=endpoint.host,
smtp_port=endpoint.port,
)
prepared = prepare_campaign_scenario(
client,
headers,
fixture_path=fixture_path,
profile_id=profile_id,
settings=settings,
scenario="celery_broker_redelivery",
snapshot_probe=snapshot_probe,
)
first_worker = _start_worker(runtime_root, label="first-worker")
replacement_worker: WorkerProcess | None = None
try:
_wait_for_worker_ready(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
queued = _expect(
client.post(
f"/api/v1/campaigns/{prepared.campaign_id}/queue",
headers=dict(headers),
json={
"version_id": prepared.version_id,
"include_warnings": True,
"enqueue_celery": True,
"dry_run": False,
},
),
200,
"Celery-redelivery Campaign queue",
)
queue_evidence = _queue_evidence(queued)
first_task_id = _wait_for_received_task(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
if not endpoint.wait_for_data(settings.provider_timeout_seconds):
raise AcceptanceError("Celery worker did not reach complete SMTP DATA")
first_exit_code = _kill_worker(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
endpoint.release_held_connection()
interrupted_state = _durable_state_evidence(
delivery_probe(prepared.campaign_id, prepared.version_id)
)
expected_interrupted = {
"job_count": 1,
"send_status_counts": {"sending": 1},
"attempt_status_counts": {"smtp_in_progress": 1},
"unfinished_attempt_count": 1,
}
if interrupted_state != expected_interrupted:
raise AcceptanceError("Killed worker state was not durably SMTP-in-progress")
replacement_worker = _start_worker(runtime_root, label="replacement-worker")
_wait_for_worker_ready(
replacement_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
redelivered_task_id = _wait_for_received_task(
replacement_worker,
timeout_seconds=settings.provider_timeout_seconds,
expected_task_id=first_task_id,
)
_wait_for_task_success(
replacement_worker,
task_id=redelivered_task_id,
timeout_seconds=settings.provider_timeout_seconds,
)
recovered_state = _durable_state_evidence(
delivery_probe(prepared.campaign_id, prepared.version_id)
)
expected_recovered = {
"job_count": 1,
"send_status_counts": {"outcome_unknown": 1},
"attempt_status_counts": {"outcome_unknown": 1},
"unfinished_attempt_count": 0,
}
if recovered_state != expected_recovered:
raise AcceptanceError("Redelivered task did not freeze the unfinished attempt")
protocol = endpoint.evidence()
expected_protocol = {
"connection_count": 1,
"accepted_rcpt_commands": 1,
"refused_rcpt_commands": 0,
"data_transactions": 1,
}
if protocol != expected_protocol:
raise AcceptanceError("Broker redelivery caused an unexpected SMTP transaction")
broker_after = _wait_for_broker_drained(
redis_url,
timeout_seconds=settings.provider_timeout_seconds,
)
first_received = first_worker.received_task_ids()
replacement_received = replacement_worker.received_task_ids()
if first_received != (first_task_id,) or replacement_received != (
redelivered_task_id,
):
raise AcceptanceError(
"Celery workers did not each receive the broker task exactly once"
)
report = _report_evidence(
_expect(
client.get(
f"/api/v1/campaigns/{prepared.campaign_id}/report",
headers=dict(headers),
params={"version_id": prepared.version_id},
),
200,
"Celery-redelivery Campaign report",
)
)
if report["send_status_counts"] != {"outcome_unknown": 1}:
raise AcceptanceError("Campaign report did not retain outcome_unknown")
audit_actions = dict(
sorted(audit_probe(prepared.campaign_id, prepared.version_id).items())
)
if not {
"campaign.created",
"campaign.validated",
"campaign.messages_built",
"campaign.queued",
}.issubset(audit_actions):
raise AcceptanceError("Celery-redelivery Campaign audit evidence is incomplete")
return {
**prepared.public_evidence(),
"queue": queue_evidence,
"interrupted_durable_state": interrupted_state,
"recovered_durable_state": recovered_state,
"protocol": protocol,
"report": report,
"audit_actions": audit_actions,
"broker": {
"transport": "redis",
"same_task_identity_redelivered": first_task_id
== redelivered_task_id,
"first_worker_received_count": len(first_received),
"replacement_worker_received_count": len(replacement_received),
**broker_after.as_dict(),
},
"supervision": {
"first_worker_killed_after_complete_data": True,
"first_worker_forced_exit": first_exit_code != 0,
"replacement_worker_started": True,
"replacement_worker_completed_redelivery": True,
},
}
finally:
endpoint.release_held_connection()
_stop_worker(first_worker, timeout_seconds=5)
if replacement_worker is not None:
_stop_worker(replacement_worker, timeout_seconds=5)
def _runtime_module_versions(registry: Any) -> dict[str, str]:
versions = {"core": _core_package_version()}
versions.update(
{
manifest.id: manifest.version
for manifest in registry.manifests()
if manifest.id in {"access", "audit", "campaigns", "mail"}
}
)
return versions
def _create_runtime_root() -> Path:
"""Create the isolated runtime under the platform-selected temp root."""
return Path(tempfile.mkdtemp(prefix="govoplan-campaign-celery-redelivery-"))
def _bootstrap_and_run(
*,
settings: TestbedSettings,
fixture_path: Path,
redis_url: str,
visibility_timeout_seconds: int,
) -> dict[str, Any]:
runtime_root = _create_runtime_root()
database = None
try:
os.environ.update(
{
"APP_ENV": "test",
"DATABASE_URL": f"sqlite:///{runtime_root / 'acceptance.db'}",
"FILE_STORAGE_BACKEND": "local",
"FILE_STORAGE_LOCAL_ROOT": str(runtime_root / "files"),
"MOCK_MAILBOX_DIR": str(runtime_root / "mock-mailbox"),
"DEV_BOOTSTRAP_ENABLED": "false",
"CELERY_ENABLED": "true",
"REDIS_URL": redis_url,
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS": str(
visibility_timeout_seconds
),
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true",
}
)
from fastapi.testclient import TestClient
from govoplan_core.db.base import Base
from govoplan_core.db.bootstrap import bootstrap_dev_data
from govoplan_core.db.session import configure_database, set_database
from govoplan_core.settings import Settings, settings as core_settings
from govoplan_core.tenancy.scope import create_scope_tables
isolated_settings = Settings()
for field_name in Settings.model_fields:
setattr(core_settings, field_name, getattr(isolated_settings, field_name))
database = configure_database(os.environ["DATABASE_URL"])
set_database(database)
from govoplan_core.server.app import app
create_scope_tables(database.engine)
Base.metadata.create_all(bind=database.engine)
with database.SessionLocal() as session:
bootstrap_dev_data(
session,
api_key_secret="celery-redelivery-unused-api-key",
user_password="celery-redelivery-admin",
)
def snapshot_probe(
version_id: str,
) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
from govoplan_campaign.backend.db.models import CampaignVersion
with database.SessionLocal() as session:
version = session.get(CampaignVersion, version_id)
if version is None:
raise AcceptanceError("Campaign execution snapshot is unavailable")
raw = version.raw_json if isinstance(version.raw_json, dict) else {}
snapshot = (
version.execution_snapshot
if isinstance(version.execution_snapshot, dict)
else {}
)
return raw, snapshot
def audit_probe(campaign_id: str, version_id: str) -> Mapping[str, int]:
from govoplan_audit.backend.db.models import AuditLog
with database.SessionLocal() as session:
actions = [
row[0]
for row in session.query(AuditLog.action)
.filter(AuditLog.object_id.in_([campaign_id, version_id]))
.all()
if row[0] in EXPECTED_AUDIT_ACTIONS
]
return dict(Counter(actions))
def delivery_probe(campaign_id: str, version_id: str) -> Mapping[str, Any]:
from govoplan_campaign.backend.db.models import CampaignJob, SendAttempt
with database.SessionLocal() as session:
jobs = (
session.query(CampaignJob)
.filter(
CampaignJob.campaign_id == campaign_id,
CampaignJob.campaign_version_id == version_id,
)
.all()
)
job_ids = [job.id for job in jobs]
attempts = (
session.query(SendAttempt)
.filter(SendAttempt.job_id.in_(job_ids))
.all()
if job_ids
else []
)
return {
"job_count": len(jobs),
"send_status_counts": dict(
Counter(job.send_status for job in jobs)
),
"attempt_status_counts": dict(
Counter(attempt.status for attempt in attempts)
),
"unfinished_attempt_count": sum(
1 for attempt in attempts if attempt.finished_at is None
),
}
with TestClient(app) as client:
login = _expect(
client.post(
"/api/v1/auth/login",
json={
"email": "admin@example.local",
"password": "celery-redelivery-admin",
},
),
200,
"Acceptance login",
)
access_token = str(login.get("access_token") or "")
if not access_token:
raise AcceptanceError("Acceptance login returned no access token")
from govoplan_core.core.runtime import get_registry
registry = get_registry()
if registry is None:
raise AcceptanceError("The GovOPlaN module registry is unavailable")
composition_versions = required_composition_versions(
fixture_path,
_runtime_module_versions(registry),
)
with smtp_fault_endpoint("post_data_hold") as endpoint:
scenario = execute_redelivery_scenario(
client,
{"Authorization": f"Bearer {access_token}"},
fixture_path=fixture_path,
settings=settings,
endpoint=endpoint,
redis_url=redis_url,
runtime_root=runtime_root,
snapshot_probe=snapshot_probe,
audit_probe=audit_probe,
delivery_probe=delivery_probe,
)
evidence = {
"schema_version": EVIDENCE_SCHEMA,
"generated_at": datetime.now(timezone.utc).isoformat(),
"fixture_sha256": hashlib.sha256(fixture_path.read_bytes()).hexdigest(),
"declared_module_versions": composition_versions,
"target": {
"kind": "local_redis_celery_controlled_smtp",
"isolated_temporary_database": True,
"redis_started_by_runner": True,
"worker_pool": "solo",
"worker_prefetch_multiplier": 1,
"task_acks_late": True,
"task_reject_on_worker_lost": True,
"visibility_timeout_seconds": visibility_timeout_seconds,
},
"scenario": scenario,
"coverage": {
"redis_broker_delivery": True,
"celery_worker_processes": True,
"forced_worker_loss_after_complete_data": True,
"same_task_broker_redelivery": True,
"durable_outcome_unknown_recovery": True,
"duplicate_smtp_transaction_prevented": True,
"production_daemon_supervisor": False,
"target_provider": False,
"source_artifact_provenance": False,
},
}
_assert_evidence_safe(evidence, settings=settings)
rendered = json.dumps(
evidence,
ensure_ascii=False,
indent=2,
sort_keys=True,
).encode("utf-8") + b"\n"
if len(rendered) > MAX_EVIDENCE_BYTES:
raise AcceptanceError("Celery-redelivery evidence exceeds its size limit")
return evidence
finally:
if database is not None:
database.engine.dispose()
shutil.rmtree(runtime_root, ignore_errors=True)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--fixture", type=Path, default=DEFAULT_FIXTURE)
parser.add_argument("--compose-file", type=Path, default=DEFAULT_COMPOSE_FILE)
parser.add_argument("--redis-port", type=int)
parser.add_argument(
"--visibility-timeout-seconds",
type=lambda value: _positive_int(value, label="visibility timeout"),
default=os.environ.get(
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS",
"3",
),
)
parser.add_argument(
"--timeout-seconds",
type=lambda value: _positive_int(value, label="timeout"),
default=60,
)
parser.add_argument("--evidence", type=Path)
args = parser.parse_args(argv)
try:
settings = TestbedSettings.from_environment()
settings.assert_local_testbed()
settings = replace(
settings,
provider_timeout_seconds=args.timeout_seconds,
)
with isolated_redis_broker(
compose_file=args.compose_file.resolve(),
timeout_seconds=args.timeout_seconds,
requested_port=args.redis_port,
) as redis_url:
evidence = _bootstrap_and_run(
settings=settings,
fixture_path=args.fixture.resolve(),
redis_url=redis_url,
visibility_timeout_seconds=args.visibility_timeout_seconds,
)
rendered = json.dumps(
evidence,
ensure_ascii=False,
indent=2,
sort_keys=True,
) + "\n"
if args.evidence:
args.evidence.parent.mkdir(parents=True, exist_ok=True)
args.evidence.write_text(rendered, encoding="utf-8")
else:
sys.stdout.write(rendered)
return 0
except AcceptanceError as exc:
print(f"Campaign Celery-redelivery acceptance failed: {exc}", file=sys.stderr)
return 1
except Exception as exc:
print(
"Campaign Celery-redelivery acceptance failed unexpectedly "
f"({type(exc).__name__}); inspect local service logs.",
file=sys.stderr,
)
return 1
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,40 @@
# Campaign Access Explanation Coverage
Campaign access explanations are resource-specific evidence. They inherit the
parent Campaign decision only where the child has no independent grant model,
and they must identify that inheritance explicitly.
## Implemented
- Campaign
- Campaign version
- Campaign delivery job / built message
- Computed Campaign report, identified by Campaign, version, and report kind
## Planned Slices
1. Recipient rows and imported recipient-source snapshots
2. Attachment bindings and frozen attachment resolutions
3. Validation issues, review decisions, and attachment-policy overrides
4. Delivery attempts, IMAP append attempts, Postbox attempts, and
reconciliation decisions
5. Campaign shares and ownership-transfer records
6. Import mapping profiles and import executions
7. Reusable Campaign templates and template revisions when the template
library becomes persistent
8. Export packages and protocol/report artifacts
Each child explanation must include:
- the child resource identity and current state;
- the parent Campaign and version where applicable;
- whether access is inherited, independently granted, or further restricted;
- effective owner/share/policy provenance;
- missing-module or unavailable-evidence reasons without leaking the hidden
object;
- a stable resource identifier suitable for audit and support links.
Delivery attempts, review decisions, reports, and exports can contain more
sensitive evidence than the Campaign summary. Their read and diagnostic/export
permissions therefore remain independently enforceable even when the parent
Campaign is readable.

View File

@@ -5,10 +5,12 @@ been validated, built, reviewed, and locked.
## Operating Modes ## Operating Modes
- Local direct send: `CELERY_ENABLED=false`. Queueing stores jobs in the DB, and - Local direct send: `CELERY_ENABLED=false`. **Send now** is available only for
small development runs can be processed with "Send queued now". exact built runs within the effective synchronous limit. It preflights the
complete batch before the first SMTP effect.
- Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing - Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing
publishes delivery tasks, and the Review & Send page polls summary counters. publishes durable delivery tasks, and Review and send polls their persisted
summary counters even after the initiating request has returned.
- Mock send: use only for development review. It does not prove real SMTP/IMAP - Mock send: use only for development review. It does not prove real SMTP/IMAP
credentials or server policy. credentials or server policy.
@@ -26,6 +28,8 @@ been validated, built, reviewed, and locked.
ZIP before using production recipients. ZIP before using production recipients.
- Keep the report page open during tests; it is the operational source of truth - Keep the report page open during tests; it is the operational source of truth
for attempts, outcomes, and reconciliation. for attempts, outcomes, and reconciliation.
- Confirm the effective Send now recipient-job limit. The safe default is 25;
use Queue for workers for ordinary batches or any run above that limit.
## Deliverability Preflight ## Deliverability Preflight
@@ -49,10 +53,15 @@ Before the first live send for a sender domain or mail-server profile:
1. Validate the version with file checks enabled. 1. Validate the version with file checks enabled.
2. Build the version and inspect all blocking review items. 2. Build the version and inspect all blocking review items.
3. Queue only after the selected version is the intended immutable execution 3. Queue only after the selected version is the intended immutable execution
version. version. Select **Queue for workers**, then verify the committed and
4. In local mode, use "Send queued now" for small test runs. published counts.
4. Use **Send now** only if the exact eligible count is non-zero and at or below
the effective deployment/tenant limit shown on the page.
5. In worker mode, verify queue counters move from queued/claimed/sending to a 5. In worker mode, verify queue counters move from queued/claimed/sending to a
terminal SMTP state. terminal SMTP state.
6. If a synchronous request is used, keep Review and send open: it polls the
durable counters while the request runs. A rejection occurs before SMTP and
directs oversized runs to workers.
## Outcome Handling ## Outcome Handling
@@ -95,9 +104,41 @@ bed where possible:
- IMAP append failure after SMTP acceptance. - IMAP append failure after SMTP acceptance.
- Worker restart with queued, claimed, and sending jobs. - Worker restart with queued, claimed, and sending jobs.
For the maintained loopback baseline, run
`dev/mail-testbed/run_campaign_acceptance.py`. It proves the public Campaign
path for SMTP acceptance, IMAP append, repeat-send blocking, an SMTP connection
failure before transmission, an explicit SMTP authentication rejection, an
explicit temporary `451` response after DATA, partial RCPT refusal, a
connection loss after complete DATA, and an IMAP authentication rejection
after SMTP acceptance. Its evidence is an allowlisted classification/count
projection; raw provider diagnostics and transport/account identifiers are
deliberately excluded.
The runner also terminates a dedicated OS process executing the registered
Campaign send task after complete DATA, then invokes the task in a fresh
process. The unfinished durable attempt must become `outcome_unknown` and the
endpoint must observe no second connection or DATA transaction. This covers
the worker task/process boundary but not a broker or daemon.
Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` for the maintained
Redis/Celery delivery and broker redelivery boundary. It starts an isolated
Redis Compose service and real Celery workers, kills the first solo worker after complete DATA while the
late-ack task is unacknowledged, and requires the same task identity to reach a
replacement worker after Redis visibility recovery. Passing evidence also
requires durable `outcome_unknown`, an empty broker queue/unacked set, and
exactly one SMTP connection and DATA transaction. Raw worker logs and task,
database, endpoint, and credential identifiers are never retained.
That second runner proves local runner-supervised process replacement, not the
production process manager. Repeat the worker-loss drill under the selected
systemd, container, Kubernetes, or other production supervisor and the target
Redis/SMTP infrastructure before deployment approval.
## Reporting Checks ## Reporting Checks
- Partial delivery must show accepted, failed, and unknown counts separately. - Partial delivery must show accepted, failed, and unknown counts separately.
- Excluded messages must show SMTP and IMAP as `skipped`, with skipped counts
and filters separate from unattempted or failed delivery.
- Accepted and unknown jobs must not appear in retry selections. - Accepted and unknown jobs must not appear in retry selections.
- Reconciled accepted jobs must remain protected from resend. - Reconciled accepted jobs must remain protected from resend.
- Reconciled not-sent jobs must appear only as explicit retry candidates. - Reconciled not-sent jobs must appear only as explicit retry candidates.

View File

@@ -144,10 +144,14 @@ paths, storage keys, worker claim tokens, and raw provider diagnostics require
the dedicated diagnostic permission and must not leak through ordinary campaign, the dedicated diagnostic permission and must not leak through ordinary campaign,
version, job, or report responses. version, job, or report responses.
The current Campaign Report Web UI additionally requires recipient-read access Campaign now provides a separate aggregate **Reports** surface for readers with
and does not yet hide every action control that the actor lacks. The server `campaigns:report:read` and access to the campaign. It loads only the safe
still authorizes each action, but an aggregate-only reader UI remains open aggregate projections, applies small-cell suppression, and offers no recipient
work; do not promise that experience from `campaigns:report:read` alone. rows, drill-down, filtering, export, or delivery actions. The recipient-aware
**Campaign Report** still requires recipient-read access and does not yet hide
every action control that the actor lacks. The server authorizes each action,
but permission-aware action visibility on that detailed surface remains open
work; do not confuse it with the aggregate reader experience.
### Deliver and resolve outcomes ### Deliver and resolve outcomes
@@ -156,11 +160,13 @@ operator sequence.
At a minimum: At a minimum:
1. Queue only a validated, locked, built version. The current Web UI does not 1. Queue only a validated, locked, built version. Use **Queue for workers** for
expose Queue; use an authorized supporting client or API. ordinary batches; the durable progress remains visible after leaving and
2. Use worker delivery for ordinary batches. Synchronous Send now has no returning to Review and send.
server-enforced job-count bound and must be used only after an operator has 2. Use **Send now** only when the exact persisted eligible build is within the
deliberately confirmed a small controlled run. effective synchronous limit shown by the UI. The default deployment limit
is 25 recipient jobs. The backend repeats the count and preflights every
message and the Mail profile revision before contacting SMTP.
3. Treat `smtp_accepted` as protected from ordinary retry. 3. Treat `smtp_accepted` as protected from ordinary retry.
4. Retry `failed_temporary` explicitly after inspecting the cause. 4. Retry `failed_temporary` explicitly after inspecting the cause.
5. Include `failed_permanent` only after correcting the cause and making a 5. Include `failed_permanent` only after correcting the cause and making a
@@ -179,6 +185,14 @@ Pause stops new eligible work but cannot undo a provider effect already in
progress. Cancel marks work that has not yet produced a protected SMTP outcome; progress. Cancel marks work that has not yet produced a protected SMTP outcome;
it cannot recall accepted mail. it cannot recall accepted mail.
The deployment ceiling is configured with
`GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0 disables Send now; the
accepted range is 0500). A tenant may only narrow that ceiling with
`tenant.settings.campaign_delivery_policy.synchronous_send_max_recipients`.
The effective value and source are returned by the protected delivery-options
API, recorded for successful/rejected synchronous commands, and stated in the
configured handbook topic.
### Test and one-message actions ### Test and one-message actions
The current baseline includes mock send, queue dry-run, synchronous immediate The current baseline includes mock send, queue dry-run, synchronous immediate
@@ -251,6 +265,13 @@ The delivery record should be able to identify:
reconciliation note; and reconciliation note; and
- actor/system trigger, timestamps, policy context, and corrections. - actor/system trigger, timestamps, policy context, and corrections.
An excluded recipient/message is a completed validation decision, not a
pending delivery. Its SMTP and IMAP states are both `skipped`; it is counted and
filterable separately from unattempted, failed, accepted, and append outcomes.
No SMTP or IMAP attempt exists for such a row. If historical data contains
actual transport evidence despite an exclusion marker, that evidence is
preserved for audit and reconciliation rather than relabelled.
## Administration and policy ## Administration and policy
### Roles and permissions ### Roles and permissions
@@ -448,7 +469,7 @@ the current baseline:
- the final audited **test / single send / single resend** semantics; - the final audited **test / single send / single resend** semantics;
- reusable SMTP batch sessions and their measured throughput benefit; - reusable SMTP batch sessions and their measured throughput benefit;
- durable, idempotent Campaign report delivery through a Mail-owned outbox - durable, idempotent Campaign report delivery through a Mail-owned outbox
([`govoplan-mail#17`](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17)); ([`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17));
- a fully packaged one-command Campaign reference composition with production - a fully packaged one-command Campaign reference composition with production
policy presets and target-provider certification; policy presets and target-provider certification;
- function-bound Postbox delivery (stage 2 of the reference program); - function-bound Postbox delivery (stage 2 of the reference program);

View File

@@ -10,7 +10,11 @@ scenario catalogue lives in `examples/README.md`; committed fixture files should
be added under `examples/` only when they validate against the current campaign be added under `examples/` only when they validate against the current campaign
schema and are safe to run in non-production environments. schema and are safe to run in non-production environments.
- simple announcement with one active recipient and no attachments - [`simple-announcement`](../examples/simple-announcement/campaign.json), a
credential-free campaign with one active recipient and no attachments; its
automated acceptance check physically blocks Mail and Files imports, denies
network connections, and validates/builds from an unrelated temporary
workspace
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition - multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
notification fields notification fields
- campaign with global attachments and recipient-specific attachment rules - campaign with global attachments and recipient-specific attachment rules
@@ -22,7 +26,9 @@ schema and are safe to run in non-production environments.
- campaign with blocked recipients or attachment errors that must not be sent - campaign with blocked recipients or attachment errors that must not be sent
- mock delivery campaign that captures SMTP and IMAP append messages in the mail - mock delivery campaign that captures SMTP and IMAP append messages in the mail
development mailbox development mailbox
- real non-production delivery campaign against the GreenMail test bed - [`greenmail-delivery`](../examples/greenmail-delivery/campaign.json), a
credential-free real-delivery Campaign materialized with a temporary
Mail-owned profile by the loopback acceptance runner
## Fixture Rules ## Fixture Rules
@@ -41,12 +47,31 @@ Before tagging a campaign release:
- Review `examples/README.md` and update the scenario catalogue when a release - Review `examples/README.md` and update the scenario catalogue when a release
adds or removes delivery behavior. adds or removes delivery behavior.
- Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'` and
retain its isolated validate/build result as release evidence.
- Run core module permutation tests with campaign installed both with and - Run core module permutation tests with campaign installed both with and
without files/mail. without files/mail.
- Validate and build each maintained example campaign. - Validate and build each maintained example campaign.
- Run the mock delivery example when the mail development mailbox capability is - Run the mock delivery example when the mail development mailbox capability is
enabled. enabled.
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path. - Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
- Run `dev/mail-testbed/run_campaign_acceptance.py` and retain its bounded JSON
projection. It must show one SMTP acceptance, one IMAP append, no duplicate
effect from a repeated ordinary send, matching Campaign report/audit state,
and no resolved transport material in Campaign JSON or its execution
snapshot. Its controlled endpoint evidence must also show explicit SMTP 451,
partial RCPT refusal, post-DATA ambiguity, and task-process interruption
classifications without retaining addresses or provider diagnostics.
- Treat the task-process restart proof separately from the still-open
Redis/Celery broker redelivery and daemon-supervision check; the coverage
projection must keep `celery_broker_redelivery` false.
- Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` as a separate
destructive worker-loss check. Retain its bounded evidence only when the same
broker task is observed at both workers, the durable state is
`outcome_unknown`, broker queue/unacked counts are zero, and the controlled
SMTP endpoint observed one connection and one DATA transaction. This closes
local Redis/Celery redelivery coverage, while production supervisor and
target-provider coverage remain false until separately tested.
- Confirm reusable mail profile selection is revalidated after campaign owner - Confirm reusable mail profile selection is revalidated after campaign owner
transfer. transfer.
- Confirm every inline SMTP/IMAP field is rejected on import/write, omitted - Confirm every inline SMTP/IMAP field is rejected on import/write, omitted

View File

@@ -49,7 +49,7 @@ Non-dry Campaign report email currently fails closed. It must not bypass the
durable job/effect model through a direct SMTP call. Re-enabling it requires the durable job/effect model through a direct SMTP call. Re-enabling it requires the
Mail-owned idempotent outbox, attempt, unknown-outcome, and reconciliation path Mail-owned idempotent outbox, attempt, unknown-outcome, and reconciliation path
tracked in tracked in
[`govoplan-mail#17`](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17). [`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17).
Report generation and dry-run validation remain separate from an external Report generation and dry-run validation remain separate from an external
effect; recipient-level exports require recipient-export authorization. effect; recipient-level exports require recipient-export authorization.

View File

@@ -9,7 +9,7 @@ campaign schema and do not require production data.
| Scenario | Required Modules | Release Check | | Scenario | Required Modules | Release Check |
| --- | --- | --- | | --- | --- | --- |
| `simple-announcement` | core, access, campaigns | Validate and build one active recipient without attachments. | | [`simple-announcement`](simple-announcement/campaign.json) | core, access, campaigns | Validate and build one active recipient without attachments while Mail and Files are absent. |
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. | | `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. | | `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. | | `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
@@ -19,7 +19,7 @@ campaign schema and do not require production data.
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. | | `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. | | `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. | | `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
| `greenmail-delivery` | core, access, campaigns, mail | Send no-attachment, normal attachment, and ZIP attachment variants through `dev/mail-testbed`. | | [`greenmail-delivery`](greenmail-delivery/campaign.json) | core, access, audit, campaigns, mail | Run a credential-free Campaign through a Mail-owned profile, GreenMail SMTP/IMAP, report/audit checks, repeat-send protection, and bounded failure drills. |
## Fixture Rules ## Fixture Rules
@@ -37,9 +37,14 @@ campaign schema and do not require production data.
Before a release tag: Before a release tag:
1. Run module permutation startup checks from core. 1. Run module permutation startup checks from core.
2. Validate every committed example fixture against the current campaign schema. 2. Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'`
3. Build exact messages for each fixture. from this repository. The acceptance test copies each maintained fixture to
4. Run the mock-delivery example when the dev mailbox capability is enabled. an unrelated temporary workspace before using Campaign's public loader,
5. Run `dev/mail-testbed/run_transport_smoke.py`. validator, and message builder.
6. Execute the delivery checklist in 3. Validate every committed example fixture against the current campaign schema.
4. Build exact messages for each fixture.
5. Run the mock-delivery example when the dev mailbox capability is enabled.
6. Run `dev/mail-testbed/run_transport_smoke.py` for low-level transport and attachment variants.
7. Run `dev/mail-testbed/run_campaign_acceptance.py` for the Campaign journey and bounded evidence.
8. Execute the delivery checklist in
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`. `docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.

View File

@@ -0,0 +1,88 @@
{
"version": "1.0",
"campaign": {
"id": "greenmail-delivery",
"name": "GreenMail delivery acceptance",
"description": "Credential-free Campaign fixture for the local SMTP/IMAP acceptance test bed.",
"mode": "test"
},
"fields": [
{
"name": "display_name",
"type": "string",
"label": "Display name",
"required": true
},
{
"name": "acceptance_run",
"type": "string",
"label": "Acceptance run",
"required": true
}
],
"server": {
"mail_profile_id": "00000000-0000-4000-8000-000000000001"
},
"recipients": {
"from": [
{
"email": "campaign-test@govoplan.test",
"name": "GovOPlaN acceptance",
"type": "to"
}
],
"allow_individual_to": true
},
"template": {
"subject": "[GovOPlaN acceptance ${acceptance_run}] Campaign delivery",
"text": "Hello ${display_name},\n\nThis is an isolated GovOPlaN Campaign SMTP/IMAP acceptance message.\n",
"body_mode": "text"
},
"attachments": {
"base_path": ".",
"send_without_attachments_behavior": "continue",
"global": []
},
"entries": {
"inline": [
{
"id": "greenmail-recipient",
"to": [
{
"email": "campaign-test@govoplan.test",
"name": "GreenMail recipient",
"type": "to"
}
],
"fields": {
"display_name": "GreenMail recipient",
"acceptance_run": "fixture"
}
}
]
},
"validation_policy": {
"missing_email": "block",
"template_error": "block"
},
"delivery": {
"rate_limit": {
"messages_per_minute": 60
},
"retry": {
"max_attempts": 3,
"backoff_seconds": [
1,
5,
30
]
},
"imap_append_sent": {
"enabled": true,
"folder": "Sent"
}
},
"status_tracking": {
"enabled": true
}
}

View File

@@ -0,0 +1,22 @@
{
"scenario": "greenmail-delivery",
"campaign_file": "campaign.json",
"required_modules": [
"core",
"access",
"audit",
"campaigns",
"mail"
],
"required_capabilities": [
"mail.campaign_delivery"
],
"transport": "local GreenMail SMTP/IMAP test bed",
"credentials": "local environment only; never copied into Campaign JSON or evidence",
"expected": {
"entries_count": 1,
"built_count": 1,
"smtp_accepted_count": 1,
"imap_appended_count": 1
}
}

View File

@@ -0,0 +1,54 @@
{
"version": "1.0",
"campaign": {
"id": "simple-announcement",
"name": "Simple announcement",
"description": "Credential-free release fixture for Campaign validation and message building.",
"mode": "test"
},
"fields": [
{
"name": "display_name",
"type": "string",
"label": "Display name",
"required": true
}
],
"recipients": {
"from": [
{
"email": "announcements@example.test",
"name": "GovOPlaN Example",
"type": "to"
}
],
"allow_individual_to": true
},
"template": {
"subject": "Planned service maintenance for ${display_name}",
"text": "Hello ${display_name},\n\nThe example service will be unavailable during the announced maintenance window.\n\nThis message was built locally and was not sent.\n",
"body_mode": "text"
},
"attachments": {
"base_path": ".",
"send_without_attachments_behavior": "continue",
"global": []
},
"entries": {
"inline": [
{
"id": "example-recipient",
"to": [
{
"email": "recipient@example.test",
"name": "Example Recipient",
"type": "to"
}
],
"fields": {
"display_name": "Example Recipient"
}
}
]
}
}

View File

@@ -0,0 +1,23 @@
{
"schema_version": 1,
"id": "simple-announcement",
"campaign_file": "campaign.json",
"required_modules": [
"core",
"access",
"campaigns"
],
"absent_optional_modules": [
"files",
"mail"
],
"external_effects": "forbidden",
"expected": {
"campaign_id": "simple-announcement",
"entries_count": 1,
"built_count": 1,
"queueable_count": 1,
"attachment_count": 0,
"subject": "Planned service maintenance for Example Recipient"
}
}

View File

@@ -1,6 +1,6 @@
{ {
"name": "@govoplan/campaign-webui", "name": "@govoplan/campaign-webui",
"version": "0.1.9", "version": "0.1.12",
"private": true, "private": true,
"type": "module", "type": "module",
"main": "webui/src/index.ts", "main": "webui/src/index.ts",
@@ -22,7 +22,7 @@
"read-excel-file": "9.2.0" "read-excel-file": "9.2.0"
}, },
"peerDependencies": { "peerDependencies": {
"@govoplan/core-webui": "^0.1.9", "@govoplan/core-webui": "^0.1.12",
"lucide-react": "^1.23.0", "lucide-react": "^1.23.0",
"react": "^19.0.0", "react": "^19.0.0",
"react-dom": "^19.0.0", "react-dom": "^19.0.0",

View File

@@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "govoplan-campaign" name = "govoplan-campaign"
version = "0.1.9" version = "0.1.12"
description = "GovOPlaN campaigns module with backend and WebUI integration." description = "GovOPlaN campaigns module with backend and WebUI integration."
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
license = { file = "LICENSE" } license = { file = "LICENSE" }
authors = [{ name = "GovOPlaN" }] authors = [{ name = "GovOPlaN" }]
dependencies = [ dependencies = [
"govoplan-core>=0.1.8", "govoplan-core>=0.1.14",
"jsonschema>=4,<5", "jsonschema>=4,<5",
"pydantic>=2,<3", "pydantic>=2,<3",
"SQLAlchemy>=2,<3", "SQLAlchemy>=2,<3",

View File

@@ -53,6 +53,17 @@ class AttachmentIssue(BaseModel):
code: str code: str
message: str message: str
behavior: Behavior | None = None behavior: Behavior | None = None
details: dict[str, Any] = Field(default_factory=dict)
class AttachmentPolicyDecision(BaseModel):
model_config = ConfigDict(extra="forbid")
requirement_policy: Behavior
campaign_policy: Behavior
rule_policy: Behavior | None = None
effective_behavior: Behavior
legacy_drop_normalized: bool = False
class ResolvedAttachment(BaseModel): class ResolvedAttachment(BaseModel):
@@ -82,6 +93,7 @@ class ResolvedAttachment(BaseModel):
zip_entry_names: list[str] = Field(default_factory=list) zip_entry_names: list[str] = Field(default_factory=list)
status: AttachmentMatchStatus status: AttachmentMatchStatus
behavior: Behavior | None = None behavior: Behavior | None = None
missing_policy: AttachmentPolicyDecision | None = None
matches: list[str] = Field(default_factory=list) matches: list[str] = Field(default_factory=list)
issues: list[AttachmentIssue] = Field(default_factory=list) issues: list[AttachmentIssue] = Field(default_factory=list)
@@ -193,12 +205,48 @@ def _rule_allows_multiple(config: AttachmentConfig, rendered_file_filter: str) -
return config.allow_multiple or any(char in rendered_file_filter for char in "*?[") return config.allow_multiple or any(char in rendered_file_filter for char in "*?[")
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior: _MISSING_BEHAVIOR_STRENGTH = {
Behavior.CONTINUE: 0,
Behavior.WARN: 1,
Behavior.ASK: 2,
Behavior.DROP: 2,
Behavior.BLOCK: 3,
}
def _missing_policy_decision(
campaign_config: CampaignConfig,
config: AttachmentConfig,
) -> AttachmentPolicyDecision:
requirement_policy = (
campaign_config.validation_policy.missing_required_attachment
if config.required
else campaign_config.validation_policy.missing_optional_attachment
)
candidates = [
requirement_policy,
campaign_config.attachments.missing_behavior,
]
if config.missing_behavior is not None: if config.missing_behavior is not None:
return config.missing_behavior candidates.append(config.missing_behavior)
if config.required: configured = max(
return campaign_config.validation_policy.missing_required_attachment candidates,
return campaign_config.validation_policy.missing_optional_attachment key=lambda behavior: _MISSING_BEHAVIOR_STRENGTH[behavior],
)
legacy_drop_normalized = configured == Behavior.DROP
if legacy_drop_normalized:
configured = Behavior.BLOCK if config.required else Behavior.ASK
return AttachmentPolicyDecision(
requirement_policy=requirement_policy,
campaign_policy=campaign_config.attachments.missing_behavior,
rule_policy=config.missing_behavior,
effective_behavior=configured,
legacy_drop_normalized=legacy_drop_normalized,
)
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
return _missing_policy_decision(campaign_config, config).effective_behavior
def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior: def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
@@ -356,14 +404,19 @@ def _confine_managed_matches(directory: Path, matches: list[Path]) -> tuple[list
return confined, rejected return confined, rejected
def _issue_for_missing(config: AttachmentConfig, behavior: Behavior) -> AttachmentIssue: def _issue_for_missing(
config: AttachmentConfig,
policy: AttachmentPolicyDecision,
) -> AttachmentIssue:
code = "missing_required_attachment" if config.required else "missing_optional_attachment" code = "missing_required_attachment" if config.required else "missing_optional_attachment"
severity = ResolutionSeverity.ERROR if config.required and behavior == Behavior.BLOCK else ResolutionSeverity.WARNING behavior = policy.effective_behavior
severity = ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
return AttachmentIssue( return AttachmentIssue(
severity=severity, severity=severity,
code=code, code=code,
message=f"No file matched attachment filter {config.file_filter!r}", message=f"No file matched attachment filter {config.file_filter!r}",
behavior=behavior, behavior=behavior,
details={"effective_policy": policy.model_dump(mode="json")},
) )
@@ -377,10 +430,13 @@ def _issue_for_ambiguous(config: AttachmentConfig, behavior: Behavior, match_cou
) )
def _send_without_attachments_behavior(config: CampaignConfig) -> Behavior: def effective_send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
return config.attachments.send_without_attachments_behavior or ( configured = config.attachments.send_without_attachments_behavior or (
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
) )
# Recipient exclusion must be an explicit reviewed action, not an implicit
# consequence of a legacy attachment policy value.
return Behavior.ASK if configured == Behavior.DROP else configured
def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssue: def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssue:
@@ -395,6 +451,7 @@ def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssu
code="missing_attachment_coverage", code="missing_attachment_coverage",
message=messages.get(behavior, "No attachment file was resolved for this message."), message=messages.get(behavior, "No attachment file was resolved for this message."),
behavior=behavior, behavior=behavior,
details={"effective_behavior": behavior.value},
) )
@@ -422,24 +479,13 @@ def _resolve_one_config(
behavior: Behavior | None = None behavior: Behavior | None = None
managed_source = selected_base_path is not None and is_managed_source(selected_base_path.source) managed_source = selected_base_path is not None and is_managed_source(selected_base_path.source)
unsafe_managed_path = False unsafe_managed_path = False
if managed_source: resolution_failed = False
try: try:
if managed_source:
assert_logical_relative_path( assert_logical_relative_path(
rendered_file_filter, rendered_file_filter,
field="rendered managed attachment file_filter", field="rendered managed attachment file_filter",
) )
except CampaignPathSecurityError as exc:
matches = []
unsafe_managed_path = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="unsafe_managed_attachment_path",
message=str(exc),
behavior=Behavior.BLOCK,
)
)
else:
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index) matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
matches, rejected = _confine_managed_matches(directory, matches) matches, rejected = _confine_managed_matches(directory, matches)
if rejected: if rejected:
@@ -453,16 +499,44 @@ def _resolve_one_config(
behavior=Behavior.BLOCK, behavior=Behavior.BLOCK,
) )
) )
else: else:
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index) matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
except CampaignPathSecurityError as exc:
matches = []
unsafe_managed_path = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="unsafe_managed_attachment_path",
message=str(exc),
behavior=Behavior.BLOCK,
)
)
except (OSError, RuntimeError) as exc:
matches = []
resolution_failed = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="attachment_resolution_failed",
message=f"Attachment source could not be read while resolving filter {config.file_filter!r}.",
behavior=Behavior.BLOCK,
details={"error_type": type(exc).__name__},
)
)
missing_policy: AttachmentPolicyDecision | None = None
if unsafe_managed_path: if unsafe_managed_path:
status = AttachmentMatchStatus.MISSING status = AttachmentMatchStatus.MISSING
behavior = Behavior.BLOCK behavior = Behavior.BLOCK
elif resolution_failed:
status = AttachmentMatchStatus.MISSING
behavior = Behavior.BLOCK
elif not matches: elif not matches:
status = AttachmentMatchStatus.MISSING status = AttachmentMatchStatus.MISSING
behavior = _missing_behavior(campaign_config, config) missing_policy = _missing_policy_decision(campaign_config, config)
issues.append(_issue_for_missing(config, behavior)) behavior = missing_policy.effective_behavior
issues.append(_issue_for_missing(config, missing_policy))
elif len(matches) > 1 and not allow_multiple: elif len(matches) > 1 and not allow_multiple:
status = AttachmentMatchStatus.AMBIGUOUS status = AttachmentMatchStatus.AMBIGUOUS
behavior = _ambiguous_behavior(campaign_config, config) behavior = _ambiguous_behavior(campaign_config, config)
@@ -494,6 +568,7 @@ def _resolve_one_config(
zip_entry_name_template=config.zip_entry_name_template, zip_entry_name_template=config.zip_entry_name_template,
status=status, status=status,
behavior=behavior, behavior=behavior,
missing_policy=missing_policy,
matches=[str(path) for path in matches], matches=[str(path) for path in matches],
issues=issues, issues=issues,
) )
@@ -540,7 +615,7 @@ def resolve_entry_attachments(
) )
issues = [issue for item in resolved for issue in item.issues] issues = [issue for item in resolved for issue in item.issues]
missing_coverage_behavior = _send_without_attachments_behavior(config) missing_coverage_behavior = effective_send_without_attachments_behavior(config)
if ( if (
entry.active entry.active
and resolved and resolved

View File

@@ -44,6 +44,7 @@ def _parse_scalar_for_target(target: str, value: Any) -> Any:
"merge_reply_to", "merge_reply_to",
"merge_bounce_to", "merge_bounce_to",
"merge_disposition_notification_to", "merge_disposition_notification_to",
"merge_postbox_targets",
"combine_to", "combine_to",
"combine_cc", "combine_cc",
"combine_bcc", "combine_bcc",

View File

@@ -4,7 +4,15 @@ import copy
from typing import Any from typing import Any
CAMPAIGN_MAIL_SERVER_KEYS = frozenset({"mail_profile_id"}) CAMPAIGN_MAIL_SERVER_KEYS = frozenset(
{
"mail_profile_id",
"smtp_server_id",
"smtp_credential_id",
"imap_server_id",
"imap_credential_id",
}
)
CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset({"created_from", "field_overrides", "opt_ins"}) CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset({"created_from", "field_overrides", "opt_ins"})
CAMPAIGN_OPT_IN_KEYS = frozenset( CAMPAIGN_OPT_IN_KEYS = frozenset(
{"campaign_address_suggestions", "remember_used_addresses", "inline_guidance"} {"campaign_address_suggestions", "remember_used_addresses", "inline_guidance"}
@@ -14,6 +22,7 @@ CAMPAIGN_REVIEW_STATE_KEYS = frozenset(
"build_token", "build_token",
"inspection_complete", "inspection_complete",
"reviewed_message_keys", "reviewed_message_keys",
"issue_decisions",
"updated_at", "updated_at",
"updated_by_user_id", "updated_by_user_id",
} }
@@ -24,8 +33,8 @@ class CampaignMailProfileBoundaryError(ValueError):
"""Raised when campaign JSON owns mail transport configuration. """Raised when campaign JSON owns mail transport configuration.
SMTP/IMAP endpoints and credentials are Mail-module data. Campaign JSON SMTP/IMAP endpoints and credentials are Mail-module data. Campaign JSON
may select one Mail-owned profile, but it must never copy or override that may select Mail-owned profile, server, and credential identifiers, but it
profile's transport configuration. must never copy or override transport configuration.
""" """
@@ -134,6 +143,22 @@ def public_campaign_editor_state(
review_state = _validated_server_review_state(value["review_send"]) review_state = _validated_server_review_state(value["review_send"])
if not include_diagnostics: if not include_diagnostics:
review_state.pop("build_token", None) review_state.pop("build_token", None)
review_state["issue_decisions"] = [
{
key: item[key]
for key in (
"job_id",
"review_key",
"decision",
"reason",
"actor_user_id",
"decided_at",
"issue_codes",
)
if key in item
}
for item in review_state.get("issue_decisions", [])
]
result["review_send"] = review_state result["review_send"] = review_state
except CampaignMailProfileBoundaryError: except CampaignMailProfileBoundaryError:
pass pass
@@ -170,6 +195,102 @@ def _validated_review_actor(value: Any) -> str | None:
return value return value
def _validated_issue_decisions(value: Any) -> list[dict[str, Any]]:
if not isinstance(value, list) or len(value) > 100_000:
raise CampaignMailProfileBoundaryError(
"Campaign review issue decisions are invalid"
)
result: list[dict[str, Any]] = []
allowed_keys = {
"job_id",
"review_key",
"decision",
"reason",
"actor_user_id",
"decided_at",
"build_token",
"message_sha256",
"issue_fingerprint",
"issue_codes",
}
for item in value:
if not isinstance(item, dict) or any(
key not in allowed_keys for key in item
):
raise CampaignMailProfileBoundaryError(
"Campaign review issue decision is invalid"
)
normalized = {
"job_id": _validated_required_string(
item.get("job_id"),
max_length=36,
error="Campaign review decision job is invalid",
),
"review_key": _validated_required_string(
item.get("review_key"),
max_length=512,
error="Campaign review decision key is invalid",
),
"decision": _validated_required_string(
item.get("decision"),
max_length=30,
error="Campaign review decision is invalid",
),
"reason": item.get("reason"),
"actor_user_id": _validated_review_actor(
item.get("actor_user_id")
),
"decided_at": _validated_required_string(
item.get("decided_at"),
max_length=128,
error="Campaign review decision timestamp is invalid",
),
"build_token": _validated_required_string(
item.get("build_token"),
max_length=256,
error="Campaign review decision build token is invalid",
),
"message_sha256": item.get("message_sha256"),
"issue_fingerprint": _validated_required_string(
item.get("issue_fingerprint"),
max_length=64,
error="Campaign review issue fingerprint is invalid",
),
"issue_codes": item.get("issue_codes"),
}
if normalized["decision"] != "accept":
raise CampaignMailProfileBoundaryError(
"Campaign review decision outcome is invalid"
)
if normalized["reason"] is not None and (
not isinstance(normalized["reason"], str)
or len(normalized["reason"]) > 4_000
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision reason is invalid"
)
if normalized["message_sha256"] is not None and (
not isinstance(normalized["message_sha256"], str)
or len(normalized["message_sha256"]) > 64
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision message hash is invalid"
)
if (
not isinstance(normalized["issue_codes"], list)
or len(normalized["issue_codes"]) > 100
or any(
not isinstance(code, str) or not code or len(code) > 100
for code in normalized["issue_codes"]
)
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision issue codes are invalid"
)
result.append(normalized)
return result
def _validated_server_review_state(value: Any) -> dict[str, Any]: def _validated_server_review_state(value: Any) -> dict[str, Any]:
if not isinstance(value, dict) or any( if not isinstance(value, dict) or any(
key not in CAMPAIGN_REVIEW_STATE_KEYS for key in value key not in CAMPAIGN_REVIEW_STATE_KEYS for key in value
@@ -180,6 +301,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
build_token = value.get("build_token") build_token = value.get("build_token")
inspected = value.get("inspection_complete") inspected = value.get("inspection_complete")
keys = value.get("reviewed_message_keys", []) keys = value.get("reviewed_message_keys", [])
issue_decisions = value.get("issue_decisions", [])
updated_at = value.get("updated_at") updated_at = value.get("updated_at")
updated_by = value.get("updated_by_user_id") updated_by = value.get("updated_by_user_id")
validated_build_token = _validated_required_string( validated_build_token = _validated_required_string(
@@ -192,6 +314,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
"Campaign review completion state is invalid" "Campaign review completion state is invalid"
) )
validated_keys = _validated_reviewed_message_keys(keys) validated_keys = _validated_reviewed_message_keys(keys)
validated_decisions = _validated_issue_decisions(issue_decisions)
validated_updated_at = _validated_required_string( validated_updated_at = _validated_required_string(
updated_at, updated_at,
max_length=128, max_length=128,
@@ -202,6 +325,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
"build_token": validated_build_token, "build_token": validated_build_token,
"inspection_complete": inspected, "inspection_complete": inspected,
"reviewed_message_keys": validated_keys, "reviewed_message_keys": validated_keys,
"issue_decisions": validated_decisions,
"updated_at": validated_updated_at, "updated_at": validated_updated_at,
"updated_by_user_id": validated_updated_by, "updated_by_user_id": validated_updated_by,
} }
@@ -218,16 +342,53 @@ def campaign_mail_profile_id(raw_json: dict[str, Any] | None) -> str | None:
return normalized or None return normalized or None
def campaign_mail_resource_ids(
raw_json: dict[str, Any] | None,
) -> dict[str, str | None]:
server = raw_json.get("server") if isinstance(raw_json, dict) else None
if not isinstance(server, dict):
return {
"mail_profile_id": None,
"smtp_server_id": None,
"smtp_credential_id": None,
"imap_server_id": None,
"imap_credential_id": None,
}
return {
key: (
value.strip()
if isinstance((value := server.get(key)), str) and value.strip()
else None
)
for key in CAMPAIGN_MAIL_SERVER_KEYS
}
def campaign_mail_profile_boundary_violations(raw_json: dict[str, Any] | None) -> tuple[str, ...]: def campaign_mail_profile_boundary_violations(raw_json: dict[str, Any] | None) -> tuple[str, ...]:
server = raw_json.get("server") if isinstance(raw_json, dict) else None server = raw_json.get("server") if isinstance(raw_json, dict) else None
if not isinstance(server, dict): if not isinstance(server, dict):
return () return ()
violations = [f"/server/{key}" for key in sorted(server) if key not in CAMPAIGN_MAIL_SERVER_KEYS] violations = [f"/server/{key}" for key in sorted(server) if key not in CAMPAIGN_MAIL_SERVER_KEYS]
if "mail_profile_id" in server: for key in CAMPAIGN_MAIL_SERVER_KEYS:
profile_id = server["mail_profile_id"] if key not in server:
if not isinstance(profile_id, str) or not profile_id.strip(): continue
violations.append("/server/mail_profile_id") value = server[key]
if not isinstance(value, str) or not value.strip():
violations.append(f"/server/{key}")
references = campaign_mail_resource_ids(raw_json)
if references["mail_profile_id"] is None and any(
references[key]
for key in references
if key != "mail_profile_id"
):
violations.append("/server/mail_profile_id")
for protocol in ("smtp", "imap"):
if (
references[f"{protocol}_credential_id"]
and not references[f"{protocol}_server_id"]
):
violations.append(f"/server/{protocol}_server_id")
return tuple(violations) return tuple(violations)
@@ -240,9 +401,9 @@ def assert_campaign_uses_mail_profile_reference(
if violations: if violations:
fields = ", ".join(violations) fields = ", ".join(violations)
raise CampaignMailProfileBoundaryError( raise CampaignMailProfileBoundaryError(
"Campaign JSON may only reference a Mail-module profile through " "Campaign JSON may only reference Mail-owned profiles, servers, and credentials; "
f"server.mail_profile_id; remove campaign-local SMTP/IMAP settings ({fields}), " f"remove campaign-local SMTP/IMAP settings or invalid references ({fields}), "
"select an authorized Mail profile, and save a new campaign version." "select authorized Mail resources, and save a new campaign version."
) )
if require_profile and campaign_mail_profile_id(raw_json) is None: if require_profile and campaign_mail_profile_id(raw_json) is None:
raise CampaignMailProfileBoundaryError( raise CampaignMailProfileBoundaryError(
@@ -254,5 +415,8 @@ def assert_campaign_uses_mail_profile_reference(
def public_campaign_mail_server(raw_json: dict[str, Any] | None) -> dict[str, str]: def public_campaign_mail_server(raw_json: dict[str, Any] | None) -> dict[str, str]:
"""Return the complete public/persisted Campaign-to-Mail contract.""" """Return the complete public/persisted Campaign-to-Mail contract."""
profile_id = campaign_mail_profile_id(raw_json) return {
return {"mail_profile_id": profile_id} if profile_id else {} key: value
for key, value in campaign_mail_resource_ids(raw_json).items()
if value
}

View File

@@ -23,6 +23,8 @@ class FieldType(StrEnum):
DOUBLE = "double" DOUBLE = "double"
DATE = "date" DATE = "date"
PASSWORD = "password" # noqa: S105 # nosec B105 - field type vocabulary. PASSWORD = "password" # noqa: S105 # nosec B105 - field type vocabulary.
ORGANIZATION_UNIT = "organization_unit"
ORGANIZATION_FUNCTION = "organization_function"
class RecipientType(StrEnum): class RecipientType(StrEnum):
@@ -89,6 +91,105 @@ class BuildStatus(StrEnum):
class SendStatus(StrEnum): class SendStatus(StrEnum):
DRAFT = "draft" DRAFT = "draft"
QUEUED = "queued" QUEUED = "queued"
SKIPPED = "skipped"
class DeliveryChannelPolicy(StrEnum):
MAIL = "mail"
POSTBOX = "postbox"
MAIL_AND_POSTBOX = "mail_and_postbox"
MAIL_THEN_POSTBOX = "mail_then_postbox"
POSTBOX_THEN_MAIL = "postbox_then_mail"
@property
def uses_mail(self) -> bool:
return self in {
DeliveryChannelPolicy.MAIL,
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
}
@property
def uses_postbox(self) -> bool:
return self != DeliveryChannelPolicy.MAIL
class PostboxTargetMode(StrEnum):
DIRECT = "direct"
DERIVED = "derived"
class PostboxTargetMatch(StrEnum):
ID = "id"
SLUG = "slug"
class PostboxTargetConfig(StrictModel):
id: str = Field(min_length=1, max_length=120)
mode: PostboxTargetMode = PostboxTargetMode.DIRECT
label: str | None = Field(default=None, max_length=500)
postbox_id: str | None = Field(default=None, max_length=36)
address_key: str | None = Field(default=None, max_length=500)
template_id: str | None = Field(default=None, max_length=36)
organization_unit_id: str | None = Field(default=None, max_length=36)
organization_unit_field: str | None = Field(default=None, max_length=255)
organization_unit_match: PostboxTargetMatch = PostboxTargetMatch.ID
function_id: str | None = Field(default=None, max_length=36)
function_field: str | None = Field(default=None, max_length=255)
function_match: PostboxTargetMatch = PostboxTargetMatch.ID
context_key: str | None = Field(default=None, max_length=255)
context_field: str | None = Field(default=None, max_length=255)
@model_validator(mode="after")
def validate_target_shape(self) -> "PostboxTargetConfig":
direct_values = [self.postbox_id, self.address_key]
if self.mode == PostboxTargetMode.DIRECT:
if sum(bool(value) for value in direct_values) != 1:
raise ValueError(
"A direct Postbox target requires exactly one postbox_id "
"or address_key."
)
if any(
(
self.template_id,
self.organization_unit_id,
self.organization_unit_field,
self.function_id,
self.function_field,
self.context_key,
self.context_field,
)
):
raise ValueError(
"A direct Postbox target cannot contain derived target fields."
)
return self
if any(direct_values):
raise ValueError(
"A derived Postbox target cannot contain postbox_id or address_key."
)
if not self.template_id:
raise ValueError("A derived Postbox target requires template_id.")
if bool(self.organization_unit_id) == bool(self.organization_unit_field):
raise ValueError(
"A derived Postbox target requires exactly one fixed or "
"field-derived organization unit."
)
if bool(self.function_id) == bool(self.function_field):
raise ValueError(
"A derived Postbox target requires exactly one fixed or "
"field-derived function."
)
if self.context_key and self.context_field:
raise ValueError(
"A derived Postbox target may use a fixed context or a context "
"field, not both."
)
return self
class CampaignMeta(StrictModel): class CampaignMeta(StrictModel):
@@ -113,6 +214,10 @@ class MailProfileCapabilities(StrictModel):
class ServerConfig(StrictModel): class ServerConfig(StrictModel):
mail_profile_id: str | None = None mail_profile_id: str | None = None
smtp_server_id: str | None = None
smtp_credential_id: str | None = None
imap_server_id: str | None = None
imap_credential_id: str | None = None
profile_capabilities: MailProfileCapabilities = Field(default_factory=MailProfileCapabilities) profile_capabilities: MailProfileCapabilities = Field(default_factory=MailProfileCapabilities)
@@ -375,7 +480,7 @@ class AttachmentsConfig(StrictModel):
send_without_attachments_behavior: Behavior | None = None send_without_attachments_behavior: Behavior | None = None
zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig) zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig)
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global") global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
missing_behavior: Behavior = Behavior.ASK missing_behavior: Behavior = Behavior.WARN
ambiguous_behavior: Behavior = Behavior.ASK ambiguous_behavior: Behavior = Behavior.ASK
@model_validator(mode="after") @model_validator(mode="after")
@@ -445,6 +550,13 @@ class EntryConfig(StrictModel):
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list) disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
merge_disposition_notification_to: bool = True merge_disposition_notification_to: bool = True
channel_policy: DeliveryChannelPolicy | None = None
postbox_targets: list[PostboxTargetConfig] = Field(
default_factory=list,
max_length=50,
)
merge_postbox_targets: bool = True
attachments: list[AttachmentConfig] = Field(default_factory=list) attachments: list[AttachmentConfig] = Field(default_factory=list)
combine_attachments: bool = True combine_attachments: bool = True
@@ -526,7 +638,7 @@ class EntriesConfig(StrictModel):
class ValidationPolicy(StrictModel): class ValidationPolicy(StrictModel):
missing_required_attachment: Behavior = Behavior.ASK missing_required_attachment: Behavior = Behavior.BLOCK
missing_optional_attachment: Behavior = Behavior.WARN missing_optional_attachment: Behavior = Behavior.WARN
ambiguous_attachment_match: Behavior = Behavior.ASK ambiguous_attachment_match: Behavior = Behavior.ASK
ignore_empty_fields: bool = False ignore_empty_fields: bool = False
@@ -558,7 +670,17 @@ class RetryConfig(StrictModel):
return values return values
class PostboxDeliveryConfig(StrictModel):
targets: list[PostboxTargetConfig] = Field(default_factory=list, max_length=50)
classification: str = Field(default="internal", min_length=1, max_length=50)
unresolved_target: Behavior = Behavior.BLOCK
vacant_target: Behavior = Behavior.WARN
duplicate_target: Behavior = Behavior.WARN
class DeliveryConfig(StrictModel): class DeliveryConfig(StrictModel):
channel_policy: DeliveryChannelPolicy = DeliveryChannelPolicy.MAIL
postbox: PostboxDeliveryConfig = Field(default_factory=PostboxDeliveryConfig)
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig) rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig) imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
retry: RetryConfig = Field(default_factory=RetryConfig) retry: RetryConfig = Field(default_factory=RetryConfig)
@@ -601,3 +723,23 @@ class CampaignConfig(StrictModel):
if path.is_absolute(): if path.is_absolute():
return path return path
return (campaign_file.parent / path).resolve() return (campaign_file.parent / path).resolve()
def effective_delivery_channel_policy(
config: CampaignConfig,
entry: EntryConfig,
) -> DeliveryChannelPolicy:
return entry.channel_policy or config.delivery.channel_policy
def effective_postbox_targets(
config: CampaignConfig,
entry: EntryConfig,
) -> list[PostboxTargetConfig]:
global_targets = list(config.delivery.postbox.targets)
individual_targets = list(entry.postbox_targets)
if not individual_targets:
return global_targets
if entry.merge_postbox_targets:
return [*global_targets, *individual_targets]
return individual_targets

View File

@@ -0,0 +1,321 @@
from __future__ import annotations
from dataclasses import asdict
from typing import Any
from sqlalchemy.orm import Session
from govoplan_core.core.postbox import (
PostboxDeliveryCatalogRef,
PostboxDirectoryEntryRef,
PostboxTargetRef,
)
from govoplan_campaign.backend.campaign.field_values import (
effective_entry_field_values,
)
from govoplan_campaign.backend.campaign.models import (
Behavior,
CampaignConfig,
EntryConfig,
PostboxTargetConfig,
PostboxTargetMatch,
PostboxTargetMode,
effective_postbox_targets,
)
from govoplan_campaign.backend.integrations import postbox_integration
from govoplan_campaign.backend.messages.models import (
MessageIssue,
MessageValidationStatus,
)
def _apply_behavior(
current: MessageValidationStatus,
behavior: Behavior,
) -> MessageValidationStatus:
if behavior == Behavior.BLOCK:
return MessageValidationStatus.BLOCKED
if behavior == Behavior.DROP:
return MessageValidationStatus.EXCLUDED
if behavior == Behavior.ASK and current not in {
MessageValidationStatus.BLOCKED,
MessageValidationStatus.EXCLUDED,
}:
return MessageValidationStatus.NEEDS_REVIEW
if behavior == Behavior.WARN and current == MessageValidationStatus.READY:
return MessageValidationStatus.WARNING
return current
def _issue(
*,
code: str,
message: str,
behavior: Behavior,
) -> MessageIssue:
return MessageIssue(
severity="error" if behavior == Behavior.BLOCK else "warning",
code=code,
message=message,
behavior=behavior.value,
source="postbox",
)
def _field_value(
values: dict[str, Any],
field_name: str | None,
) -> str | None:
if not field_name:
return None
value = values.get(field_name)
if value is None:
return None
text = str(value).strip()
return text or None
def _match_unit(
catalog: PostboxDeliveryCatalogRef,
value: str | None,
match: PostboxTargetMatch,
):
if not value:
return None
return next(
(
unit
for unit in catalog.organization_units
if (unit.id if match == PostboxTargetMatch.ID else unit.slug) == value
),
None,
)
def _match_function(unit, value: str | None, match: PostboxTargetMatch):
if unit is None or not value:
return None
return next(
(
function
for function in unit.functions
if (
function.id
if match == PostboxTargetMatch.ID
else function.slug
)
== value
),
None,
)
def _target_ref(
target: PostboxTargetConfig,
*,
values: dict[str, Any],
catalog: PostboxDeliveryCatalogRef,
) -> tuple[PostboxTargetRef | None, str | None]:
if target.mode == PostboxTargetMode.DIRECT:
return (
PostboxTargetRef(
postbox_id=target.postbox_id,
address_key=target.address_key,
),
None,
)
unit_value = target.organization_unit_id or _field_value(
values,
target.organization_unit_field,
)
unit_match = (
PostboxTargetMatch.ID
if target.organization_unit_id
else target.organization_unit_match
)
unit = _match_unit(catalog, unit_value, unit_match)
if unit is None:
return None, (
f"Organization unit {unit_value!r} could not be resolved by "
f"{unit_match.value}."
)
function_value = target.function_id or _field_value(
values,
target.function_field,
)
function_match = (
PostboxTargetMatch.ID
if target.function_id
else target.function_match
)
function = _match_function(unit, function_value, function_match)
if function is None:
return None, (
f"Organization function {function_value!r} could not be resolved "
f"inside {unit.name!r} by {function_match.value}."
)
context_key = target.context_key or _field_value(
values,
target.context_field,
)
return (
PostboxTargetRef(
template_id=target.template_id,
organization_unit_id=unit.id,
function_id=function.id,
context_key=context_key,
),
None,
)
def _resolved_target_payload(
target: PostboxTargetConfig,
entry: PostboxDirectoryEntryRef,
*,
position: int,
) -> dict[str, Any]:
return {
"target_id": target.id,
"position": position,
"mode": target.mode.value,
"requested": target.model_dump(mode="json", exclude_none=True),
"postbox_id": entry.id,
"address": entry.address,
"address_key": entry.address_key,
"name": entry.name,
"status": entry.status,
"classification": entry.classification,
"organization_unit_id": entry.organization_unit_id,
"organization_unit_name": entry.organization_unit_name,
"function_id": entry.function_id,
"function_name": entry.function_name,
"context_key": entry.context_key,
"template_revision_id": entry.template_revision_id,
"holder_count": entry.holder_count,
"vacant": entry.vacant,
}
def resolve_entry_postbox_targets(
session: Session,
*,
tenant_id: str,
config: CampaignConfig,
entry: EntryConfig,
validation_status: MessageValidationStatus,
materialize: bool,
) -> tuple[
list[dict[str, Any]],
list[MessageIssue],
MessageValidationStatus,
]:
integration = postbox_integration()
policy = config.delivery.postbox
targets = effective_postbox_targets(config, entry)
if not targets:
issue = _issue(
code="postbox_target_missing",
message="Postbox delivery requires at least one target.",
behavior=policy.unresolved_target,
)
return (
[],
[issue],
_apply_behavior(validation_status, policy.unresolved_target),
)
try:
catalog = integration.delivery_catalog(session, tenant_id=tenant_id)
except Exception as exc:
issue = _issue(
code="postbox_unavailable",
message=str(exc),
behavior=Behavior.BLOCK,
)
return [], [issue], MessageValidationStatus.BLOCKED
values = effective_entry_field_values(config, entry)
resolved: list[dict[str, Any]] = []
issues: list[MessageIssue] = []
status = validation_status
seen_postbox_ids: set[str] = set()
for position, target in enumerate(targets):
target_ref, resolution_error = _target_ref(
target,
values=values,
catalog=catalog,
)
if target_ref is None:
issue = _issue(
code="postbox_target_unresolved",
message=resolution_error or "Postbox target could not be resolved.",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
try:
entry_ref = integration.resolve_postbox(
session,
tenant_id=tenant_id,
target=target_ref,
materialize=materialize,
)
except Exception as exc:
issue = _issue(
code="postbox_target_unresolved",
message=f"Postbox target {target.id!r} could not be resolved: {exc}",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
if entry_ref is None:
issue = _issue(
code="postbox_target_unresolved",
message=f"Postbox target {target.id!r} does not exist.",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
if entry_ref.id in seen_postbox_ids:
issue = _issue(
code="postbox_target_duplicate",
message=(
f"Postbox {entry_ref.address!r} is selected more than once; "
"it will receive one message."
),
behavior=policy.duplicate_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.duplicate_target)
continue
seen_postbox_ids.add(entry_ref.id)
resolved.append(
_resolved_target_payload(
target,
entry_ref,
position=position,
)
)
if entry_ref.vacant:
issue = _issue(
code="postbox_target_vacant",
message=(
f"Postbox {entry_ref.address!r} currently has no function "
"holder."
),
behavior=policy.vacant_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.vacant_target)
return resolved, issues, status
def delivery_catalog_payload(catalog: PostboxDeliveryCatalogRef) -> dict[str, Any]:
return asdict(catalog)

View File

@@ -10,7 +10,21 @@ from pydantic import BaseModel, ConfigDict, Field
from .addressing import effective_address_lists from .addressing import effective_address_lists
from .field_values import ignored_entry_field_overrides from .field_values import ignored_entry_field_overrides
from .models import AttachmentConfig, CampaignConfig, EntryConfig, FieldType, SourceType, ZipArchiveConfig, ZipPasswordMode, ZipPasswordScope, ZipRuleMode from .models import (
AttachmentConfig,
CampaignConfig,
DeliveryChannelPolicy,
EntryConfig,
FieldType,
PostboxTargetConfig,
SourceType,
ZipArchiveConfig,
ZipPasswordMode,
ZipPasswordScope,
ZipRuleMode,
effective_delivery_channel_policy,
effective_postbox_targets,
)
from ..attachments.resolver import resolve_campaign_attachments from ..attachments.resolver import resolve_campaign_attachments
@@ -90,6 +104,8 @@ def _mapping_target_known(target: str, field_names: set[str]) -> bool:
"merge_reply_to", "merge_reply_to",
"merge_bounce_to", "merge_bounce_to",
"merge_disposition_notification_to", "merge_disposition_notification_to",
"merge_postbox_targets",
"channel_policy",
"combine_to", "combine_to",
"combine_cc", "combine_cc",
"combine_bcc", "combine_bcc",
@@ -337,10 +353,148 @@ def _global_value_issues(config: CampaignConfig, declared_names: set[str]) -> li
] ]
def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]: def _active_delivery_entries(config: CampaignConfig) -> list[EntryConfig]:
if config.entries.is_inline:
return [
entry
for entry in (config.entries.inline or [])
if entry.active
]
return [config.entries.defaults or EntryConfig()]
def _delivery_policies(config: CampaignConfig) -> set[DeliveryChannelPolicy]:
return {
effective_delivery_channel_policy(config, entry)
for entry in _active_delivery_entries(config)
}
def _postbox_target_field_issues(
config: CampaignConfig,
target: PostboxTargetConfig,
path: str,
) -> list[SemanticIssue]:
definitions = {field.name: field for field in config.fields}
checks = (
(
target.organization_unit_field,
FieldType.ORGANIZATION_UNIT,
"organization unit",
"organization_unit_field",
),
(
target.function_field,
FieldType.ORGANIZATION_FUNCTION,
"organization function",
"function_field",
),
(target.context_field, None, "context", "context_field"),
)
issues: list[SemanticIssue] = [] issues: list[SemanticIssue] = []
for field_name, expected_type, label, key in checks:
if not field_name:
continue
definition = definitions.get(field_name)
if definition is None:
issues.append(
_issue(
Severity.ERROR,
"postbox_target_field_missing",
f"Postbox {label} field {field_name!r} is not declared.",
f"{path}/{key}",
)
)
elif expected_type is not None and definition.type != expected_type:
issues.append(
_issue(
Severity.WARNING,
"postbox_target_field_type",
(
f"Postbox {label} field {field_name!r} should use "
f"field type {expected_type.value!r}."
),
f"{path}/{key}",
)
)
return issues
def _postbox_delivery_issues(
config: CampaignConfig,
*,
postbox_available: bool,
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
policies = _delivery_policies(config)
if not any(policy.uses_postbox for policy in policies):
return issues
if not postbox_available:
issues.append(
_issue(
Severity.ERROR,
"postbox_unavailable",
(
"This campaign uses Postbox delivery, but the Postbox "
"module and its delivery directory are not active."
),
"/delivery/channel_policy",
)
)
for entry_index, entry in enumerate(_active_delivery_entries(config)):
policy = effective_delivery_channel_policy(config, entry)
if not policy.uses_postbox:
continue
targets = effective_postbox_targets(config, entry)
if not targets:
issues.append(
_issue(
Severity.ERROR,
"postbox_target_missing",
"Postbox delivery requires at least one target.",
f"/entries/inline/{entry_index}/postbox_targets",
)
)
continue
seen_ids: set[str] = set()
for target_index, target in enumerate(targets):
target_path = (
f"/entries/inline/{entry_index}/postbox_targets/"
f"{target_index}"
)
if target.id in seen_ids:
issues.append(
_issue(
Severity.WARNING,
"postbox_target_id_duplicate",
f"Postbox target id {target.id!r} is repeated.",
f"{target_path}/id",
)
)
seen_ids.add(target.id)
issues.extend(
_postbox_target_field_issues(config, target, target_path)
)
return issues
def _delivery_issues(
config: CampaignConfig,
*,
postbox_available: bool,
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
policies = _delivery_policies(config)
uses_mail = any(policy.uses_mail for policy in policies)
profile_id = (config.server.mail_profile_id or "").strip() profile_id = (config.server.mail_profile_id or "").strip()
if (config.campaign.mode == "send" or config.delivery.imap_append_sent.enabled) and not profile_id: if (
(
config.campaign.mode == "send"
and uses_mail
or config.delivery.imap_append_sent.enabled
)
and not profile_id
):
issues.append( issues.append(
_issue( _issue(
Severity.ERROR, Severity.ERROR,
@@ -350,7 +504,12 @@ def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
) )
) )
capabilities = config.server.profile_capabilities capabilities = config.server.profile_capabilities
if config.campaign.mode == "send" and profile_id and not capabilities.smtp_available: if (
config.campaign.mode == "send"
and uses_mail
and profile_id
and not capabilities.smtp_available
):
issues.append( issues.append(
_issue( _issue(
Severity.ERROR, Severity.ERROR,
@@ -368,13 +527,22 @@ def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
"/server/mail_profile_id", "/server/mail_profile_id",
) )
) )
issues.extend(
_postbox_delivery_issues(
config,
postbox_available=postbox_available,
)
)
return issues return issues
def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]: def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
"""Require Campaign-owned sender data before a send-mode build.""" """Require Campaign-owned sender data before a send-mode build."""
if config.campaign.mode != "send": if (
config.campaign.mode != "send"
or not any(policy.uses_mail for policy in _delivery_policies(config))
):
return [] return []
if config.entries.is_inline: if config.entries.is_inline:
return [ return [
@@ -385,7 +553,11 @@ def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
f"/entries/inline/{index}/from", f"/entries/inline/{index}/from",
) )
for index, entry in enumerate(config.entries.inline or []) for index, entry in enumerate(config.entries.inline or [])
if entry.active and not effective_address_lists(config, entry)["from"] if (
entry.active
and effective_delivery_channel_policy(config, entry).uses_mail
and not effective_address_lists(config, entry)["from"]
)
] ]
if config.recipients.from_: if config.recipients.from_:
return [] return []
@@ -611,6 +783,7 @@ def validate_campaign_config(
*, *,
campaign_file: str | Path | None = None, campaign_file: str | Path | None = None,
check_files: bool = False, check_files: bool = False,
postbox_available: bool = False,
) -> SemanticReport: ) -> SemanticReport:
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json" campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
issues: list[SemanticIssue] = [] issues: list[SemanticIssue] = []
@@ -622,7 +795,12 @@ def validate_campaign_config(
issues.extend(_global_value_issues(config, declared_names)) issues.extend(_global_value_issues(config, declared_names))
issues.extend(_attachment_path_issues(config)) issues.extend(_attachment_path_issues(config))
issues.extend(_zip_configuration_issues(config)) issues.extend(_zip_configuration_issues(config))
issues.extend(_delivery_issues(config)) issues.extend(
_delivery_issues(
config,
postbox_available=postbox_available,
)
)
issues.extend(_sender_issues(config)) issues.extend(_sender_issues(config))
entries = _entries_validation( entries = _entries_validation(

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
from collections.abc import Iterable, Mapping from collections.abc import Iterable, Mapping
from fastapi import HTTPException
from sqlalchemy import and_, or_ from sqlalchemy import and_, or_
from govoplan_core.core.access import AccessDecisionProvenance, PrincipalRef from govoplan_core.core.access import AccessDecisionProvenance, PrincipalRef
@@ -14,12 +15,61 @@ from govoplan_core.core.campaigns import (
CampaignPolicyContextProvider, CampaignPolicyContextProvider,
CampaignRetentionProvider, CampaignRetentionProvider,
) )
from govoplan_core.core.ownership import (
OwnershipActionDecision,
OwnershipSubjectRef,
OwnershipTransferError,
)
from govoplan_core.security.module_permissions import scopes_grant_compatible from govoplan_core.security.module_permissions import scopes_grant_compatible
from govoplan_campaign.backend.db.models import Campaign, CampaignShare from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignShare,
CampaignVersion,
)
READ_ACTIONS = {"campaigns:campaign:read", "campaign:read"} READ_ACTIONS = {"campaigns:campaign:read", "campaign:read"}
CAMPAIGN_RESOURCE_TYPES = {
"campaign",
"campaign_object",
"campaigns:campaign",
}
CAMPAIGN_VERSION_RESOURCE_TYPES = {
"campaign_version",
"campaigns:version",
}
CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES = {
"campaign_delivery_job",
"campaign_job",
"campaigns:delivery_job",
}
CAMPAIGN_REPORT_RESOURCE_TYPES = {
"campaign_report",
"campaigns:report",
}
def campaign_report_resource_id(
*,
campaign_id: str,
version_id: str,
report_kind: str,
) -> str:
kind = report_kind.strip().lower()
if not campaign_id or not version_id or not kind or ":" in kind:
raise ValueError("Campaign report references require campaign, version, and a bounded kind")
return f"{campaign_id}:{version_id}:{kind}"
def _campaign_report_reference(
resource_id: str,
) -> tuple[str, str, str] | None:
parts = resource_id.split(":", 2)
if len(parts) != 3 or not all(part.strip() for part in parts):
return None
return parts[0], parts[1], parts[2].strip().lower()
class CampaignMailPolicyContextService(CampaignMailPolicyContextProvider): class CampaignMailPolicyContextService(CampaignMailPolicyContextProvider):
@@ -120,20 +170,112 @@ class CampaignAccessService(CampaignAccessProvider):
action: str, action: str,
) -> tuple[AccessDecisionProvenance, ...]: ) -> tuple[AccessDecisionProvenance, ...]:
normalized_type = resource_type.lower().strip() normalized_type = resource_type.lower().strip()
if normalized_type not in {"campaign", "campaign_object", "campaigns:campaign"}: child_item: AccessDecisionProvenance | None = None
return () campaign: Campaign | None
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined] if normalized_type in CAMPAIGN_RESOURCE_TYPES:
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id): campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
return ( elif normalized_type in CAMPAIGN_VERSION_RESOURCE_TYPES:
AccessDecisionProvenance( version = session.get(CampaignVersion, resource_id) # type: ignore[attr-defined]
kind="resource", if version is None:
id=resource_id, return _missing_resource_provenance(
tenant_id=principal.tenant_id, principal,
source="campaigns.not_found", resource_type="campaign_version",
details={"resource_type": "campaign", "found": False}, resource_id=resource_id,
), )
campaign = session.get(Campaign, version.campaign_id) # type: ignore[attr-defined]
child_item = AccessDecisionProvenance(
kind="resource",
id=version.id,
label=f"Version {version.version_number}",
tenant_id=campaign.tenant_id if campaign else principal.tenant_id,
source="campaigns.version",
details={
"resource_type": "campaign_version",
"campaign_id": version.campaign_id,
"version_number": version.version_number,
"workflow_state": version.workflow_state,
"authorization_inherited_from": {
"resource_type": "campaign",
"resource_id": version.campaign_id,
},
},
) )
items = [ elif normalized_type in CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES:
job = session.get(CampaignJob, resource_id) # type: ignore[attr-defined]
if job is None:
return _missing_resource_provenance(
principal,
resource_type="campaign_delivery_job",
resource_id=resource_id,
)
campaign = session.get(Campaign, job.campaign_id) # type: ignore[attr-defined]
child_item = AccessDecisionProvenance(
kind="resource",
id=job.id,
label=job.recipient_email or f"Recipient {job.entry_index}",
tenant_id=job.tenant_id,
source="campaigns.delivery_job",
details={
"resource_type": "campaign_delivery_job",
"campaign_id": job.campaign_id,
"campaign_version_id": job.campaign_version_id,
"queue_status": job.queue_status,
"send_status": job.send_status,
"authorization_inherited_from": {
"resource_type": "campaign",
"resource_id": job.campaign_id,
},
},
)
elif normalized_type in CAMPAIGN_REPORT_RESOURCE_TYPES:
reference = _campaign_report_reference(resource_id)
if reference is None:
return _missing_resource_provenance(
principal,
resource_type="campaign_report",
resource_id=resource_id,
reason="invalid_reference",
)
campaign_id, version_id, report_kind = reference
campaign = session.get(Campaign, campaign_id) # type: ignore[attr-defined]
version = session.get(CampaignVersion, version_id) # type: ignore[attr-defined]
if (
campaign is None
or version is None
or version.campaign_id != campaign.id
):
return _missing_resource_provenance(
principal,
resource_type="campaign_report",
resource_id=resource_id,
)
child_item = AccessDecisionProvenance(
kind="resource",
id=resource_id,
label=report_kind,
tenant_id=campaign.tenant_id,
source="campaigns.report",
details={
"resource_type": "campaign_report",
"campaign_id": campaign.id,
"campaign_version_id": version.id,
"report_kind": report_kind,
"persisted": False,
"authorization_inherited_from": {
"resource_type": "campaign",
"resource_id": campaign.id,
},
},
)
else:
return ()
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id):
return _missing_resource_provenance(
principal,
resource_type=normalized_type,
resource_id=resource_id,
)
items = ([child_item] if child_item is not None else []) + [
AccessDecisionProvenance( AccessDecisionProvenance(
kind="resource", kind="resource",
id=campaign.id, id=campaign.id,
@@ -149,7 +291,11 @@ class CampaignAccessService(CampaignAccessProvider):
] ]
items.extend(_owner_provenance(campaign, principal)) items.extend(_owner_provenance(campaign, principal))
items.extend(_tenant_admin_provenance(principal)) items.extend(_tenant_admin_provenance(principal))
permission_values = {"read", "write"} if action in READ_ACTIONS else {"write"} permission_values = (
{"read", "write"}
if action in READ_ACTIONS or action.strip().lower().endswith(":read")
else {"write"}
)
shares = ( shares = (
session.query(CampaignShare) # type: ignore[attr-defined] session.query(CampaignShare) # type: ignore[attr-defined]
.filter( .filter(
@@ -187,6 +333,283 @@ def access_capability(context: object) -> CampaignAccessService:
return CampaignAccessService() return CampaignAccessService()
class CampaignOwnershipService:
def current_owner(
self,
session: object,
*,
tenant_id: str,
resource_id: str,
) -> OwnershipSubjectRef | None:
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
if campaign is None or campaign.tenant_id != tenant_id:
return None
if campaign.owner_user_id:
return OwnershipSubjectRef(type="user", id=campaign.owner_user_id)
if campaign.owner_group_id:
return OwnershipSubjectRef(type="group", id=campaign.owner_group_id)
return None
def authorize_ownership_action(
self,
session: object,
*,
tenant_id: str,
resource_id: str,
action: str,
actor: OwnershipSubjectRef,
current_owner: OwnershipSubjectRef,
target_owner: OwnershipSubjectRef,
) -> OwnershipActionDecision:
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
if campaign is None or campaign.tenant_id != tenant_id:
return OwnershipActionDecision(False, "Campaign was not found")
if target_owner.type not in {"user", "group"}:
return OwnershipActionDecision(
False,
"Campaign ownership can be assigned only to a user or group",
)
actor_is_current_owner = _actor_controls_owner(actor, current_owner)
actor_is_target = _actor_controls_owner(actor, target_owner)
actor_can_share = scopes_grant_compatible(
actor.scopes,
"campaigns:campaign:share",
)
actor_can_accept_for_group = scopes_grant_compatible(
actor.scopes,
"campaigns:ownership:accept_group",
)
actor_can_recover = scopes_grant_compatible(
actor.scopes,
"campaigns:ownership:recover",
)
actor_can_read = (
scopes_grant_compatible(
actor.scopes,
"campaigns:campaign:read",
)
and CampaignAccessService().can_read_campaign(
session,
tenant_id=tenant_id,
campaign_id=resource_id,
user_id=actor.id,
group_ids=actor.group_ids,
tenant_admin=scopes_grant_compatible(
actor.scopes,
"tenant:*",
),
)
)
if action in {
"propose_transfer",
"request_ownership",
"request_recovery",
}:
target_decision = _valid_campaign_owner_target(
tenant_id=tenant_id,
target_owner=target_owner,
)
if target_decision is not None:
return target_decision
if action == "view_transfer":
allowed = (
_actor_is_owner_member(actor, current_owner)
or _actor_is_owner_member(actor, target_owner)
or actor_can_recover
)
elif action in {
"propose_transfer",
"approve_requested_transfer",
}:
allowed = actor_is_current_owner and actor_can_share
elif action == "cancel_transfer":
allowed = (
(actor_is_current_owner and actor_can_share)
or actor_is_target
)
elif action == "request_ownership":
allowed = actor_can_read and actor_is_target and (
target_owner.type != "group" or actor_can_accept_for_group
)
elif action == "accept_transfer":
allowed = target_owner.type == "user" and actor_is_target
elif action == "accept_group_transfer":
allowed = (
target_owner.type == "group"
and actor_is_target
and actor_can_accept_for_group
)
elif action == "decline_transfer":
allowed = (
actor_is_target
and (
target_owner.type != "group"
or actor_can_accept_for_group
)
) or (
actor_is_current_owner
and actor_can_share
)
elif action in {
"request_recovery",
"approve_recovery",
"execute_recovery",
}:
allowed = actor_can_recover
else:
allowed = False
return OwnershipActionDecision(
allowed=allowed,
reason=None if allowed else f"Campaign ownership action is not allowed: {action}",
)
def apply_owner(
self,
session: object,
*,
tenant_id: str,
resource_id: str,
expected_owner: OwnershipSubjectRef,
target_owner: OwnershipSubjectRef,
actor: OwnershipSubjectRef,
reason: str | None,
) -> None:
del actor, reason
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
if campaign is None or campaign.tenant_id != tenant_id:
raise OwnershipTransferError("Campaign was not found")
if target_owner.type not in {"user", "group"}:
raise OwnershipTransferError(
"Campaign ownership can be assigned only to a user or group"
)
from govoplan_campaign.backend.route_support import (
_clear_current_version_mail_profile_for_owner_transfer,
)
try:
_clear_current_version_mail_profile_for_owner_transfer(
session, # type: ignore[arg-type]
campaign,
)
except HTTPException as exc:
raise OwnershipTransferError(
str(
exc.detail
or "Campaign owner cannot be changed in its current state"
)
) from exc
expected_filter = (
(
Campaign.owner_user_id == expected_owner.id,
Campaign.owner_group_id.is_(None),
)
if expected_owner.type == "user"
else (
Campaign.owner_user_id.is_(None),
Campaign.owner_group_id == expected_owner.id,
)
)
updated = (
session.query(Campaign) # type: ignore[attr-defined]
.filter(
Campaign.id == resource_id,
Campaign.tenant_id == tenant_id,
*expected_filter,
)
.update(
{
Campaign.owner_user_id: (
target_owner.id
if target_owner.type == "user"
else None
),
Campaign.owner_group_id: (
target_owner.id
if target_owner.type == "group"
else None
),
},
synchronize_session=False,
)
)
if updated != 1:
raise OwnershipTransferError(
"Campaign owner changed while the transfer was pending"
)
session.expire(campaign) # type: ignore[attr-defined]
def _valid_campaign_owner_target(
*,
tenant_id: str,
target_owner: OwnershipSubjectRef,
) -> OwnershipActionDecision | None:
from govoplan_campaign.backend.route_support import _access_directory
directory = _access_directory()
if target_owner.type == "user":
item = directory.get_user(target_owner.id)
elif target_owner.type == "group":
item = directory.get_group(target_owner.id)
else:
return OwnershipActionDecision(
False,
"Campaign ownership can be assigned only to a user or group",
)
if (
item is None
or item.tenant_id != tenant_id
or item.status != "active"
):
return OwnershipActionDecision(
False,
f"Target {target_owner.type} is not active in this tenant",
)
return None
def _actor_is_owner_member(
actor: OwnershipSubjectRef,
owner: OwnershipSubjectRef,
) -> bool:
if owner.type == "group":
return owner.id in actor.group_ids
return actor.type == owner.type and actor.id == owner.id
def _actor_controls_owner(
actor: OwnershipSubjectRef,
owner: OwnershipSubjectRef,
) -> bool:
return _actor_is_owner_member(actor, owner)
def _missing_resource_provenance(
principal: PrincipalRef,
*,
resource_type: str,
resource_id: str,
reason: str = "not_found",
) -> tuple[AccessDecisionProvenance, ...]:
return (
AccessDecisionProvenance(
kind="resource",
id=resource_id,
tenant_id=principal.tenant_id,
source="campaigns.not_found",
details={
"resource_type": resource_type,
"found": False,
"reason": reason,
},
),
)
def _owner_provenance(campaign: Campaign, principal: PrincipalRef) -> tuple[AccessDecisionProvenance, ...]: def _owner_provenance(campaign: Campaign, principal: PrincipalRef) -> tuple[AccessDecisionProvenance, ...]:
if campaign.owner_user_id and campaign.owner_user_id == principal.membership_id: if campaign.owner_user_id and campaign.owner_user_id == principal.membership_id:
return ( return (
@@ -279,6 +702,12 @@ class CampaignDeliveryTaskService(CampaignDeliveryTaskProvider):
def delivery_tasks_capability(context: object) -> CampaignDeliveryTaskService: def delivery_tasks_capability(context: object) -> CampaignDeliveryTaskService:
from govoplan_campaign.backend.runtime import configure_runtime
configure_runtime(
registry=getattr(context, "registry", None),
settings=getattr(context, "settings", None),
)
return CampaignDeliveryTaskService() return CampaignDeliveryTaskService()

View File

@@ -21,6 +21,7 @@ from govoplan_campaign.backend.db.models import (
CampaignShare, CampaignShare,
CampaignVersion, CampaignVersion,
ImapAppendAttempt, ImapAppendAttempt,
PostboxDeliveryAttempt,
SendAttempt, SendAttempt,
new_uuid, new_uuid,
) )
@@ -55,7 +56,10 @@ def _record_campaign_changes(session: OrmSession, _flush_context: object, _insta
_record_job_change(session, obj) _record_job_change(session, obj)
elif isinstance(obj, CampaignIssue): elif isinstance(obj, CampaignIssue):
_record_issue_change(session, obj) _record_issue_change(session, obj)
elif isinstance(obj, (SendAttempt, ImapAppendAttempt)): elif isinstance(
obj,
(SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt),
):
_record_attempt_change(session, obj) _record_attempt_change(session, obj)
@@ -182,8 +186,11 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
"validation_status", "validation_status",
"queue_status", "queue_status",
"send_status", "send_status",
"delivery_channel_policy",
"postbox_status",
"imap_status", "imap_status",
"attempt_count", "attempt_count",
"postbox_attempt_count",
"last_error", "last_error",
"queued_at", "queued_at",
"claimed_at", "claimed_at",
@@ -191,6 +198,7 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
"outcome_unknown_at", "outcome_unknown_at",
"sent_at", "sent_at",
"resolved_recipients", "resolved_recipients",
"resolved_postbox_targets",
"resolved_attachments", "resolved_attachments",
"issues_snapshot", "issues_snapshot",
), ),
@@ -217,6 +225,8 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
"validation_status": job.validation_status, "validation_status": job.validation_status,
"queue_status": job.queue_status, "queue_status": job.queue_status,
"send_status": job.send_status, "send_status": job.send_status,
"delivery_channel_policy": job.delivery_channel_policy,
"postbox_status": job.postbox_status,
"imap_status": job.imap_status, "imap_status": job.imap_status,
}, },
) )
@@ -247,10 +257,25 @@ def _record_issue_change(session: OrmSession, issue: CampaignIssue) -> None:
) )
def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppendAttempt) -> None: def _record_attempt_change(
session: OrmSession,
attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt,
) -> None:
operation = _operation_for_object( operation = _operation_for_object(
attempt, attempt,
changed_attrs=("status", "claim_token", "smtp_status_code", "smtp_response", "error_type", "error_message", "folder"), changed_attrs=(
"status",
"claim_token",
"smtp_status_code",
"smtp_response",
"error_type",
"error_message",
"folder",
"provider_delivery_id",
"provider_message_id",
"postbox_id",
"evidence",
),
) )
if operation is None: if operation is None:
return return
@@ -270,7 +295,13 @@ def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppen
payload={ payload={
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}), **(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
"attempt_id": attempt_id, "attempt_id": attempt_id,
"attempt_kind": "imap" if isinstance(attempt, ImapAppendAttempt) else "smtp", "attempt_kind": (
"postbox"
if isinstance(attempt, PostboxDeliveryAttempt)
else "imap"
if isinstance(attempt, ImapAppendAttempt)
else "smtp"
),
"job_id": job.id, "job_id": job.id,
"version_id": job.campaign_version_id, "version_id": job.campaign_version_id,
}, },

View File

@@ -8,6 +8,7 @@ from typing import Any
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.orm import Mapped, mapped_column, relationship
from govoplan_core.core.concurrency import strong_resource_etag
from govoplan_core.db.base import Base, TimestampMixin from govoplan_core.db.base import Base, TimestampMixin
@@ -78,10 +79,14 @@ class JobQueueStatus(StrEnum):
class JobSendStatus(StrEnum): class JobSendStatus(StrEnum):
NOT_QUEUED = "not_queued" NOT_QUEUED = "not_queued"
SKIPPED = "skipped"
QUEUED = "queued" QUEUED = "queued"
CLAIMED = "claimed" CLAIMED = "claimed"
SENDING = "sending" SENDING = "sending"
SMTP_ACCEPTED = "smtp_accepted" SMTP_ACCEPTED = "smtp_accepted"
POSTBOX_ACCEPTED = "postbox_accepted"
DELIVERED = "delivered"
PARTIALLY_ACCEPTED = "partially_accepted"
SENT = "sent" # legacy value retained for existing databases/reports SENT = "sent" # legacy value retained for existing databases/reports
OUTCOME_UNKNOWN = "outcome_unknown" OUTCOME_UNKNOWN = "outcome_unknown"
FAILED_TEMPORARY = "failed_temporary" FAILED_TEMPORARY = "failed_temporary"
@@ -89,6 +94,19 @@ class JobSendStatus(StrEnum):
CANCELLED = "cancelled" CANCELLED = "cancelled"
class JobPostboxStatus(StrEnum):
NOT_REQUESTED = "not_requested"
PENDING = "pending"
DELIVERING = "delivering"
ACCEPTED = "accepted"
ACCEPTED_VACANT = "accepted_vacant"
PARTIALLY_ACCEPTED = "partially_accepted"
REJECTED_TEMPORARY = "rejected_temporary"
REJECTED_PERMANENT = "rejected_permanent"
OUTCOME_UNKNOWN = "outcome_unknown"
SKIPPED = "skipped"
class JobImapStatus(StrEnum): class JobImapStatus(StrEnum):
NOT_REQUESTED = "not_requested" NOT_REQUESTED = "not_requested"
PENDING = "pending" PENDING = "pending"
@@ -171,6 +189,11 @@ class CampaignVersion(Base, TimestampMixin):
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True) campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
version_number: Mapped[int] = mapped_column(Integer, nullable=False) version_number: Mapped[int] = mapped_column(Integer, nullable=False)
edit_revision: Mapped[int] = mapped_column(
Integer,
default=1,
nullable=False,
)
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False) raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False) schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
source_filename: Mapped[str | None] = mapped_column(String(500)) source_filename: Mapped[str | None] = mapped_column(String(500))
@@ -212,9 +235,23 @@ class CampaignVersion(Base, TimestampMixin):
execution_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True) execution_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
execution_snapshot_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True) execution_snapshot_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
execution_snapshot_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) execution_snapshot_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
delivery_mode: Mapped[str | None] = mapped_column(String(30), nullable=True, index=True)
delivery_mode_selected_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
campaign: Mapped[Campaign] = relationship(back_populates="versions") campaign: Mapped[Campaign] = relationship(back_populates="versions")
__mapper_args__ = {
"version_id_col": edit_revision,
}
@property
def strong_etag(self) -> str:
return strong_resource_etag(
"campaign_version",
self.id,
self.edit_revision,
)
@property @property
def mail_profile_migration_required(self) -> bool: def mail_profile_migration_required(self) -> bool:
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_boundary_violations from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_boundary_violations
@@ -246,9 +283,26 @@ class CampaignJob(Base, TimestampMixin):
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True) validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True) queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True) send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
delivery_channel_policy: Mapped[str] = mapped_column(
String(30),
default="mail",
nullable=False,
index=True,
)
postbox_status: Mapped[str] = mapped_column(
String(50),
default=JobPostboxStatus.NOT_REQUESTED.value,
nullable=False,
index=True,
)
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True) imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False) attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
postbox_attempt_count: Mapped[int] = mapped_column(
Integer,
default=0,
nullable=False,
)
last_error: Mapped[str | None] = mapped_column(Text) last_error: Mapped[str | None] = mapped_column(Text)
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
@@ -260,6 +314,11 @@ class CampaignJob(Base, TimestampMixin):
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True) resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
resolved_postbox_targets: Mapped[list[dict[str, Any]]] = mapped_column(
JSON,
default=list,
nullable=False,
)
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
@@ -324,6 +383,139 @@ class SendAttempt(Base, TimestampMixin):
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class CampaignMessageAction(Base, TimestampMixin):
__tablename__ = "campaign_message_actions"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_message_actions_idempotency",
),
Index(
"ix_campaign_message_actions_job_created",
"job_id",
"created_at",
),
Index(
"ix_campaign_message_actions_campaign_kind",
"campaign_id",
"kind",
"status",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
campaign_version_id: Mapped[str] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
job_id: Mapped[str] = mapped_column(
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
kind: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
idempotency_key: Mapped[str] = mapped_column(String(200), nullable=False)
canonical_request_hash: Mapped[str] = mapped_column(String(64), nullable=False)
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
context: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
actor_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
actor_api_key_id: Mapped[str | None] = mapped_column(String(36), nullable=True)
message_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
message_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
recipient_manifest_sha256: Mapped[str] = mapped_column(
String(64),
nullable=False,
)
recipient_count: Mapped[int] = mapped_column(Integer, nullable=False)
prior_send_status: Mapped[str] = mapped_column(String(50), nullable=False)
prior_attempt_count: Mapped[int] = mapped_column(Integer, nullable=False)
final_send_status: Mapped[str | None] = mapped_column(String(50), nullable=True)
status: Mapped[str] = mapped_column(
String(50),
default="initiated",
nullable=False,
index=True,
)
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refusal_summary: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
error_type: Mapped[str | None] = mapped_column(String(120), nullable=True)
error_message: Mapped[str | None] = mapped_column(String(500), nullable=True)
linked_send_attempt_id: Mapped[str | None] = mapped_column(
ForeignKey("send_attempts.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
effect_started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
completed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
class CampaignMessageActionAttempt(Base, TimestampMixin):
__tablename__ = "campaign_message_action_attempts"
__table_args__ = (
UniqueConstraint(
"action_id",
"attempt_number",
name="uq_campaign_message_action_attempt_number",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
action_id: Mapped[str] = mapped_column(
ForeignKey("campaign_message_actions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
status: Mapped[str] = mapped_column(
String(50),
default="initiated",
nullable=False,
index=True,
)
started_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
)
effect_started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
completed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
outcome_code: Mapped[str | None] = mapped_column(String(80), nullable=True)
diagnostic_summary: Mapped[str | None] = mapped_column(
String(500),
nullable=True,
)
class ImapAppendAttempt(Base, TimestampMixin): class ImapAppendAttempt(Base, TimestampMixin):
__tablename__ = "imap_append_attempts" __tablename__ = "imap_append_attempts"
__table_args__ = ( __table_args__ = (
@@ -339,6 +531,79 @@ class ImapAppendAttempt(Base, TimestampMixin):
error_message: Mapped[str | None] = mapped_column(Text) error_message: Mapped[str | None] = mapped_column(Text)
class PostboxDeliveryAttempt(Base, TimestampMixin):
__tablename__ = "campaign_postbox_delivery_attempts"
__table_args__ = (
UniqueConstraint(
"job_id",
"target_key",
"attempt_number",
name="uq_campaign_postbox_attempt_target_number",
),
Index(
"ix_campaign_postbox_attempt_idempotency",
"tenant_id",
"idempotency_key",
),
Index(
"ix_campaign_postbox_attempt_job_status",
"job_id",
"status",
),
)
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
default=new_uuid,
)
tenant_id: Mapped[str] = mapped_column(
String(36),
nullable=False,
index=True,
)
job_id: Mapped[str] = mapped_column(
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
target_key: Mapped[str] = mapped_column(String(64), nullable=False)
target_index: Mapped[int] = mapped_column(Integer, nullable=False)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
status: Mapped[str] = mapped_column(
String(50),
nullable=False,
index=True,
)
target_snapshot: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
provider_delivery_id: Mapped[str | None] = mapped_column(String(36))
provider_message_id: Mapped[str | None] = mapped_column(String(36))
postbox_id: Mapped[str | None] = mapped_column(String(36), index=True)
address: Mapped[str | None] = mapped_column(String(500))
holder_count: Mapped[int | None] = mapped_column(Integer)
vacant: Mapped[bool | None] = mapped_column(Boolean)
duplicate: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
evidence: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
error_type: Mapped[str | None] = mapped_column(String(255))
error_code: Mapped[str | None] = mapped_column(String(100))
error_message: Mapped[str | None] = mapped_column(Text)
started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
)
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
)
__all__ = [ __all__ = [
@@ -356,8 +621,10 @@ __all__ = [
"IssueSeverity", "IssueSeverity",
"JobBuildStatus", "JobBuildStatus",
"JobImapStatus", "JobImapStatus",
"JobPostboxStatus",
"JobQueueStatus", "JobQueueStatus",
"JobSendStatus", "JobSendStatus",
"JobValidationStatus", "JobValidationStatus",
"SendAttempt", "SendAttempt",
"PostboxDeliveryAttempt",
] ]

View File

@@ -0,0 +1,107 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.tenancy.scope import Tenant
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 25
ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 500
SYNCHRONOUS_SEND_MAX_ENV = "GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS"
CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY = "campaign_delivery_policy"
SYNCHRONOUS_SEND_MAX_SETTINGS_KEY = "synchronous_send_max_recipients"
class CampaignDeliveryPolicyError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class SynchronousSendPolicy:
max_recipient_jobs: int
source: str
deployment_max_recipient_jobs: int
tenant_max_recipient_jobs: int | None = None
def as_dict(self) -> dict[str, Any]:
return {
"max_recipient_jobs": self.max_recipient_jobs,
"source": self.source,
"deployment_max_recipient_jobs": self.deployment_max_recipient_jobs,
"tenant_max_recipient_jobs": self.tenant_max_recipient_jobs,
"deployment_setting": SYNCHRONOUS_SEND_MAX_ENV,
"tenant_setting": (
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
}
def effective_synchronous_send_policy(
session: Session,
*,
tenant_id: str,
environ: Mapping[str, str] | None = None,
) -> SynchronousSendPolicy:
env = os.environ if environ is None else environ
deployment_value = _configured_limit(
env.get(SYNCHRONOUS_SEND_MAX_ENV),
source=SYNCHRONOUS_SEND_MAX_ENV,
default=DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
)
tenant = session.get(Tenant, tenant_id)
tenant_raw = _tenant_limit_value(tenant.settings if tenant is not None else None)
if tenant_raw is None:
return SynchronousSendPolicy(
max_recipient_jobs=deployment_value,
source=("deployment" if env.get(SYNCHRONOUS_SEND_MAX_ENV) not in (None, "") else "deployment_default"),
deployment_max_recipient_jobs=deployment_value,
)
tenant_value = _configured_limit(
tenant_raw,
source=(
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
)
effective_value = min(deployment_value, tenant_value)
return SynchronousSendPolicy(
max_recipient_jobs=effective_value,
source="tenant" if tenant_value <= deployment_value else "deployment_ceiling",
deployment_max_recipient_jobs=deployment_value,
tenant_max_recipient_jobs=tenant_value,
)
def _tenant_limit_value(settings: Mapping[str, Any] | None) -> object | None:
if not isinstance(settings, Mapping):
return None
policy = settings.get(CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY)
if not isinstance(policy, Mapping):
return None
return policy.get(SYNCHRONOUS_SEND_MAX_SETTINGS_KEY)
def _configured_limit(value: object, *, source: str, default: int | None = None) -> int:
if value is None or (isinstance(value, str) and not value.strip()):
if default is not None:
return default
raise CampaignDeliveryPolicyError(f"{source} must be configured as an integer")
if isinstance(value, bool):
raise CampaignDeliveryPolicyError(f"{source} must be an integer, not a boolean")
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise CampaignDeliveryPolicyError(f"{source} must be an integer") from exc
if str(parsed) != str(value).strip() and not isinstance(value, int):
raise CampaignDeliveryPolicyError(f"{source} must be an integer")
if parsed < 0 or parsed > ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS:
raise CampaignDeliveryPolicyError(
f"{source} must be between 0 and {ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS}"
)
return parsed

View File

@@ -2,6 +2,11 @@ from __future__ import annotations
from govoplan_core.core.modules import DocumentationCondition, DocumentationContext, DocumentationLink, DocumentationTopic from govoplan_core.core.modules import DocumentationCondition, DocumentationContext, DocumentationLink, DocumentationTopic
from govoplan_campaign.backend.delivery_policy import (
CampaignDeliveryPolicyError,
effective_synchronous_send_policy,
)
_CAMPAIGN_USER_SCOPES = ( _CAMPAIGN_USER_SCOPES = (
"campaigns:campaign:read", "campaigns:campaign:read",
@@ -278,21 +283,24 @@ CAMPAIGN_USER_DOCUMENTATION = (
), ),
steps=( steps=(
"Open Review and send and confirm the selected version, build, recipients, warnings, and review completion.", "Open Review and send and confirm the selected version, build, recipients, warnings, and review completion.",
"Use an authorized supporting client to run the queue dry run and resolve every blocker.", "Select Queue for workers and confirm the exact durable execution that will be committed.",
"Invoke the Queue action through that client so background workers can process the build.", "Remain on Review and send or leave and return later; both views read the same durable job states.",
"Open the campaign Report and monitor queue, SMTP, IMAP, failure, and uncertain-outcome counts.", "Use the delivery controls to pause, resume, cancel unsent work, or retry eligible failures without requeueing accepted or uncertain outcomes.",
), ),
outcome="Eligible jobs queued with an auditable execution snapshot and protected delivery state.", outcome="Eligible jobs queued with an auditable execution snapshot and protected delivery state.",
verification="The Report shows the selected version's jobs as queued or progressing, without changing any accepted job back to retryable.", verification="The Report shows the selected version's jobs as queued or progressing, without changing any accepted job back to retryable.",
related_topic_ids=("campaigns.workflow.complete-review", "campaigns.workflow.retry-and-reconcile"), related_topic_ids=("campaigns.workflow.complete-review", "campaigns.workflow.retry-and-reconcile"),
links=(
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
),
related_modules=("mail", "notifications"), related_modules=("mail", "notifications"),
limitations=("The current Campaign Web UI does not expose the Queue action; use an authorized supporting client or API.",),
), ),
_workflow_topic( _workflow_topic(
topic_id="campaigns.workflow.send-small-controlled-run", topic_id="campaigns.workflow.send-small-controlled-run",
title="Send a campaign immediately", title="Send a campaign immediately",
summary="Run the eligible jobs synchronously only after deliberately confirming that the reviewed campaign is small enough for an interactive request.", summary="Run the eligible jobs synchronously only after deliberately confirming that the reviewed campaign is small enough for an interactive request.",
body="The current synchronous action does not enforce a server-side job-count limit. It must not replace the worker queue for ordinary batches, but it does preserve the same protected SMTP, IMAP, retry, and uncertain-outcome rules.", body="Send now is protected by an effective deployment/tenant recipient-job maximum. The server counts the exact persisted eligible build, rejects an oversized or empty run before SMTP, and preflights every message and the Mail profile revision before the first provider effect.",
order=36, order=36,
audience=("campaign_sender", "campaign_operator"), audience=("campaign_sender", "campaign_operator"),
required_modules=("campaigns", "mail"), required_modules=("campaigns", "mail"),
@@ -308,12 +316,12 @@ CAMPAIGN_USER_DOCUMENTATION = (
help_contexts=("campaign.review-send",), help_contexts=("campaign.review-send",),
prerequisites=( prerequisites=(
"The selected version is validated, locked, built, and reviewed.", "The selected version is validated, locked, built, and reviewed.",
"An operator has deliberately confirmed that this is a small controlled run; the server does not currently enforce that bound.", "The exact eligible recipient-job count is within the effective limit shown on Review and send.",
"Its Mail profile remains available and authorized for the current campaign context.", "Its Mail profile remains available and authorized for the current campaign context.",
), ),
steps=( steps=(
"Open Review and send and inspect the exact build and delivery readiness checks.", "Open Review and send and inspect the exact build and delivery readiness checks.",
"Confirm that this is an intentionally small controlled run; use Queue for an ordinary batch.", "Review the effective synchronous limit and exact eligible count; use Queue for workers when Send now is unavailable or for an ordinary batch.",
"Select Send now and confirm the protected delivery action.", "Select Send now and confirm the protected delivery action.",
"Open Report and inspect each resulting delivery state before attempting any recovery action.", "Open Report and inspect each resulting delivery state before attempting any recovery action.",
), ),
@@ -321,14 +329,36 @@ CAMPAIGN_USER_DOCUMENTATION = (
verification="The Report distinguishes accepted, failed, unattempted, cancelled, and outcome-unknown jobs and retains their attempt history.", verification="The Report distinguishes accepted, failed, unattempted, cancelled, and outcome-unknown jobs and retains their attempt history.",
related_topic_ids=("campaigns.workflow.queue-delivery", "campaigns.workflow.retry-and-reconcile"), related_topic_ids=("campaigns.workflow.queue-delivery", "campaigns.workflow.retry-and-reconcile"),
related_modules=("mail",), related_modules=("mail",),
limitations=("No server-side synchronous job-count limit is currently enforced; use Queue and workers for ordinary batches.",), ),
_workflow_topic(
topic_id="campaigns.workflow.view-aggregate-delivery-report",
title="Review aggregate campaign outcomes",
summary="Inspect privacy-protected Campaign totals without receiving recipient rows, message content, delivery diagnostics, or export authority.",
body="The aggregate Reports view exposes only approved campaign-level business outcomes. Positive cells below the configured threshold are suppressed together with a complementary value or the denominator when needed, so totals cannot be subtracted to recover a small group.",
order=37,
audience=("campaign_aggregate_reader", "campaign_reader", "campaign_manager"),
required_scopes=("campaigns:report:read",),
route="/campaigns/reports",
screen="Reports",
help_contexts=("campaign.report",),
prerequisites=("The campaign is owned by or explicitly shared with you, or you hold tenant-wide authority.",),
steps=(
"Open Reports and select a campaign available to you.",
"Read the stated denominator before comparing accepted, failed, unknown, active, excluded, cancelled, and unattempted outcomes.",
"Treat Suppressed as an intentional privacy boundary rather than zero or missing data.",
"Request separately authorized recipient-level access only when the task genuinely requires individual evidence.",
),
outcome="A business-level Campaign outcome view with small-group and recipient privacy preserved.",
verification="No row, address, message, attachment, diagnostic, filter, drill-down, or export action is available from the aggregate view.",
related_topic_ids=("campaigns.workflow.view-delivery-report",),
links=(DocumentationLink(label="Aggregate Campaign reports", href="/campaigns/reports", kind="runtime"),),
), ),
_workflow_topic( _workflow_topic(
topic_id="campaigns.workflow.view-delivery-report", topic_id="campaigns.workflow.view-delivery-report",
title="Review campaign delivery details", title="Review campaign delivery details",
summary="Inspect delivery totals and recipient-level job evidence in the current Campaign Report UI.", summary="Inspect delivery totals and recipient-level job evidence in the current Campaign Report UI.",
body="The current Report UI requires recipient-read authority in addition to report access. Infrastructure diagnostics remain separately authorized, and every report action is still checked by the server even where the current UI has not yet hidden an unavailable control.", body="The recipient-aware Campaign Report requires campaign-read, report-read, and recipient-read authority. Infrastructure diagnostics remain separately authorized, and the server checks every direct detail route independently of the interface.",
order=37, order=38,
audience=("campaign_reader", "campaign_manager", "campaign_reviewer", "campaign_sender"), audience=("campaign_reader", "campaign_manager", "campaign_reviewer", "campaign_sender"),
required_scopes=("campaigns:campaign:read", "campaigns:report:read", "campaigns:recipient:read"), required_scopes=("campaigns:campaign:read", "campaigns:report:read", "campaigns:recipient:read"),
route="/campaigns/{campaign_id}/report", route="/campaigns/{campaign_id}/report",
@@ -337,21 +367,20 @@ CAMPAIGN_USER_DOCUMENTATION = (
prerequisites=("You may read the selected campaign and its report.",), prerequisites=("You may read the selected campaign and its report.",),
steps=( steps=(
"Open the campaign and select Report.", "Open the campaign and select Report.",
"Review totals for queued, accepted, failed, cancelled, unattempted, and outcome-unknown jobs.", "Review totals for queued, accepted, failed, cancelled, excluded/skipped, unattempted, and outcome-unknown jobs.",
"Inspect the authorized recipient-level rows and attempt history.", "Inspect the authorized recipient-level rows and attempt history.",
"Treat outcome-unknown jobs as unresolved and hand them to an authorized operator for evidence-backed reconciliation.", "Treat outcome-unknown jobs as unresolved and hand them to an authorized operator for evidence-backed reconciliation.",
), ),
outcome="A recipient-aware view of delivery progress and outcomes with diagnostics still separately protected.", outcome="A recipient-aware view of delivery progress and outcomes with diagnostics still separately protected.",
verification="The report totals and job rows match the selected campaign version, and infrastructure diagnostics are absent unless separately authorized.", verification="The report totals and job rows match the selected campaign version, and infrastructure diagnostics are absent unless separately authorized.",
related_topic_ids=("campaigns.workflow.retry-and-reconcile", "campaigns.workflow.export-delivery-report"), related_topic_ids=("campaigns.workflow.view-aggregate-delivery-report", "campaigns.workflow.retry-and-reconcile", "campaigns.workflow.export-delivery-report"),
limitations=("An aggregate-only Report Web UI for readers without recipient-read authority is not implemented yet.",),
), ),
_workflow_topic( _workflow_topic(
topic_id="campaigns.workflow.export-delivery-report", topic_id="campaigns.workflow.export-delivery-report",
title="Export recipient delivery results", title="Export recipient delivery results",
summary="Download an authorized CSV snapshot of recipient-level campaign delivery results for controlled downstream use.", summary="Download an authorized CSV snapshot of recipient-level campaign delivery results for controlled downstream use.",
body="A report export contains personal and delivery evidence. Store, transmit, retain, and delete it according to the campaign's purpose and the applicable export and retention policy.", body="A report export contains personal and delivery evidence. Store, transmit, retain, and delete it according to the campaign's purpose and the applicable export and retention policy.",
order=38, order=39,
audience=("campaign_report_exporter", "campaign_auditor"), audience=("campaign_report_exporter", "campaign_auditor"),
required_scopes=( required_scopes=(
"campaigns:campaign:read", "campaigns:campaign:read",
@@ -481,10 +510,13 @@ def documentation_topics(context: DocumentationContext) -> tuple[DocumentationTo
current_configuration = list(_actor_capabilities(principal, mail_available=mail_available)) current_configuration = list(_actor_capabilities(principal, mail_available=mail_available))
integration_configuration, integration_limitations = _integration_summary(context.registry, principal) integration_configuration, integration_limitations = _integration_summary(context.registry, principal)
current_configuration.extend(integration_configuration) current_configuration.extend(integration_configuration)
delivery_configuration, delivery_limitations = _delivery_policy_summary(context)
current_configuration.extend(delivery_configuration)
limitations = [ limitations = [
"Access to a particular campaign still depends on its owner or sharing rules and on the campaign's current state.", "Access to a particular campaign still depends on its owner or sharing rules and on the campaign's current state.",
"Profile, file, and address pickers re-evaluate the actual resources visible in the selected campaign; this overview does not claim that an eligible item currently exists.", "Profile, file, and address pickers re-evaluate the actual resources visible in the selected campaign; this overview does not claim that an eligible item currently exists.",
*integration_limitations, *integration_limitations,
*delivery_limitations,
] ]
return ( return (
@@ -640,6 +672,24 @@ def _integration_summary(registry: object, principal: object) -> tuple[tuple[str
return tuple(configured), tuple(limitations) return tuple(configured), tuple(limitations)
def _delivery_policy_summary(context: DocumentationContext) -> tuple[tuple[str, ...], tuple[str, ...]]:
session = context.session
tenant_id = str(getattr(context.principal, "tenant_id", "") or "").strip()
if session is None or not tenant_id:
return (), ("The effective synchronous Campaign limit could not be resolved for this documentation request.",)
try:
policy = effective_synchronous_send_policy(session, tenant_id=tenant_id) # type: ignore[arg-type]
except CampaignDeliveryPolicyError as exc:
return (), (f"Synchronous Campaign delivery is disabled until its policy configuration is corrected: {exc}",)
return (
(
"Delivery policy: Send now is limited to "
f"{policy.max_recipient_jobs} eligible recipient job(s) for this tenant. "
"The exact built count and Queue for workers alternative are shown before confirmation."
),
), ()
def _append_if( def _append_if(
target: list[str], target: list[str],
principal: object, principal: object,

View File

@@ -7,11 +7,28 @@ from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from typing import Any, Iterator from typing import Any, Iterator
from govoplan_core.core.postbox import (
CAPABILITY_POSTBOX_DIRECTORY,
CAPABILITY_POSTBOX_DELIVERY,
CAPABILITY_POSTBOX_EVIDENCE,
PostboxDeliveryCatalogRef,
PostboxDeliveryProvider,
PostboxDeliveryReceiptSummaryRef,
PostboxDeliveryRequest,
PostboxDeliveryResult,
PostboxDirectoryEntryRef,
PostboxDirectoryProvider,
PostboxEvidenceProvider,
PostboxTargetRef,
)
from govoplan_campaign.backend.runtime import capability from govoplan_campaign.backend.runtime import capability
FILES_CAPABILITY = "files.campaign_attachments" FILES_CAPABILITY = "files.campaign_attachments"
MAIL_CAPABILITY = "mail.campaign_delivery" MAIL_CAPABILITY = "mail.campaign_delivery"
POSTBOX_CAPABILITY = CAPABILITY_POSTBOX_DELIVERY
POSTBOX_DIRECTORY_CAPABILITY = CAPABILITY_POSTBOX_DIRECTORY
POSTBOX_EVIDENCE_CAPABILITY = CAPABILITY_POSTBOX_EVIDENCE
class OptionalModuleUnavailable(RuntimeError): class OptionalModuleUnavailable(RuntimeError):
@@ -50,6 +67,14 @@ class MailProfileError(OptionalModuleUnavailable):
pass pass
class MailDeliveryCommandError(RuntimeError):
pass
class PostboxDeliveryUnavailable(OptionalModuleUnavailable):
pass
class _PreparedCampaignSnapshot: class _PreparedCampaignSnapshot:
def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None: def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None:
self._directory = directory self._directory = directory
@@ -141,6 +166,12 @@ class MailCampaignIntegration:
def available(self) -> bool: def available(self) -> bool:
return self._delegate is not None return self._delegate is not None
@property
def durable_delivery_available(self) -> bool:
return self._delegate is not None and callable(
getattr(self._delegate, "submit_delivery_command", None)
)
def _require(self) -> Any: def _require(self) -> Any:
if self._delegate is None: if self._delegate is None:
raise MailProfileError("Mail module is not available") raise MailProfileError("Mail module is not available")
@@ -203,15 +234,164 @@ class MailCampaignIntegration:
except getattr(delegate, "MailProfileError", MailProfileError) as exc: except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc raise MailProfileError(str(exc)) from exc
def submit_delivery_command(self, session: Any, **kwargs: Any) -> dict[str, Any]:
delegate = self._require()
method = getattr(delegate, "submit_delivery_command", None)
if not callable(method):
raise MailDeliveryCommandError(
"The installed Mail module does not provide durable delivery commands"
)
try:
return dict(method(session, **kwargs))
except Exception as exc:
raise MailDeliveryCommandError(str(exc)) from exc
def delivery_command_summary(
self,
session: Any,
*,
tenant_id: str,
command_id: str,
) -> dict[str, Any]:
delegate = self._require()
method = getattr(delegate, "delivery_command_summary", None)
if not callable(method):
raise MailDeliveryCommandError(
"The installed Mail module does not provide durable delivery status"
)
try:
return dict(
method(
session,
tenant_id=tenant_id,
command_id=command_id,
)
)
except Exception as exc:
raise MailDeliveryCommandError(str(exc)) from exc
def mock_mailbox(self) -> Any | None: def mock_mailbox(self) -> Any | None:
if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"): if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"):
return None return None
return self._delegate.mock_mailbox() return self._delegate.mock_mailbox()
class PostboxCampaignIntegration:
def __init__(
self,
delivery_delegate: object | None = None,
directory_delegate: object | None = None,
evidence_delegate: object | None = None,
) -> None:
self._delivery_delegate = (
delivery_delegate
if isinstance(delivery_delegate, PostboxDeliveryProvider)
else None
)
self._directory_delegate = (
directory_delegate
if isinstance(directory_delegate, PostboxDirectoryProvider)
else None
)
self._evidence_delegate = (
evidence_delegate
if isinstance(evidence_delegate, PostboxEvidenceProvider)
else None
)
@property
def available(self) -> bool:
return (
self._delivery_delegate is not None
and self._directory_delegate is not None
)
@property
def receipt_evidence_available(self) -> bool:
return self._evidence_delegate is not None
def delivery_catalog(
self,
session: object,
*,
tenant_id: str,
) -> PostboxDeliveryCatalogRef:
if self._directory_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox targets are unavailable because the Postbox module "
"is not active."
)
return self._directory_delegate.delivery_catalog(
session,
tenant_id=tenant_id,
)
def resolve_postbox(
self,
session: object,
*,
tenant_id: str,
target: PostboxTargetRef,
materialize: bool = False,
) -> PostboxDirectoryEntryRef | None:
if self._directory_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox targets are unavailable because the Postbox module "
"is not active."
)
return self._directory_delegate.resolve_postbox(
session,
tenant_id=tenant_id,
target=target,
materialize=materialize,
)
def deliver(
self,
session: object,
request: PostboxDeliveryRequest,
) -> PostboxDeliveryResult:
if self._delivery_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox delivery is unavailable because the Postbox module "
"is not active."
)
return self._delivery_delegate.deliver(session, request)
def delivery_receipt_summaries(
self,
session: object,
*,
tenant_id: str,
delivery_ids: list[str] | tuple[str, ...],
) -> dict[str, PostboxDeliveryReceiptSummaryRef]:
if self._evidence_delegate is None:
return {}
unique_ids = tuple(dict.fromkeys(delivery_ids))
summaries: dict[str, PostboxDeliveryReceiptSummaryRef] = {}
for offset in range(0, len(unique_ids), 500):
summaries.update(
self._evidence_delegate.delivery_receipt_summaries(
session,
tenant_id=tenant_id,
producer_module="campaigns",
delivery_ids=unique_ids[offset : offset + 500],
)
)
return summaries
def files_integration() -> FilesCampaignIntegration: def files_integration() -> FilesCampaignIntegration:
return FilesCampaignIntegration(capability(FILES_CAPABILITY)) return FilesCampaignIntegration(capability(FILES_CAPABILITY))
def mail_integration() -> MailCampaignIntegration: def mail_integration() -> MailCampaignIntegration:
return MailCampaignIntegration(capability(MAIL_CAPABILITY)) return MailCampaignIntegration(capability(MAIL_CAPABILITY))
def postbox_integration() -> PostboxCampaignIntegration:
return PostboxCampaignIntegration(
capability(POSTBOX_CAPABILITY),
capability(POSTBOX_DIRECTORY_CAPABILITY),
capability(POSTBOX_EVIDENCE_CAPABILITY),
)

View File

@@ -11,11 +11,13 @@ from govoplan_core.core.campaigns import (
CAPABILITY_CAMPAIGNS_RETENTION, CAPABILITY_CAMPAIGNS_RETENTION,
) )
from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard
from govoplan_core.core.ownership import OwnershipProviderRegistration
from govoplan_core.core.modules import ( from govoplan_core.core.modules import (
DocumentationCondition, DocumentationCondition,
DocumentationLink, DocumentationLink,
DocumentationTopic, DocumentationTopic,
FrontendModule, FrontendModule,
FrontendRoute,
MigrationSpec, MigrationSpec,
ModuleContext, ModuleContext,
ModuleInterfaceProvider, ModuleInterfaceProvider,
@@ -25,7 +27,14 @@ from govoplan_core.core.modules import (
PermissionDefinition, PermissionDefinition,
RoleTemplate, RoleTemplate,
) )
from govoplan_core.core.views import ViewSurface
from govoplan_core.db.base import Base from govoplan_core.db.base import Base
from govoplan_core.core.postbox import (
CAPABILITY_POSTBOX_DELIVERY,
CAPABILITY_POSTBOX_DIRECTORY,
CAPABILITY_POSTBOX_EVIDENCE,
)
from govoplan_core.core.references import CAPABILITY_ACCESS_REFERENCE_OPTIONS
from govoplan_campaign.backend.change_tracking import register_campaign_change_tracking from govoplan_campaign.backend.change_tracking import register_campaign_change_tracking
from govoplan_campaign.backend.db import models as campaign_models # noqa: F401 - populate Campaign ORM metadata from govoplan_campaign.backend.db import models as campaign_models # noqa: F401 - populate Campaign ORM metadata
from govoplan_campaign.backend.documentation import CAMPAIGN_USER_DOCUMENTATION, documentation_topics from govoplan_campaign.backend.documentation import CAMPAIGN_USER_DOCUMENTATION, documentation_topics
@@ -55,15 +64,17 @@ PERMISSIONS = (
_permission("campaigns:campaign:archive", "Archive campaigns", "Archive campaigns without destroying audit evidence.", "Campaigns"), _permission("campaigns:campaign:archive", "Archive campaigns", "Archive campaigns without destroying audit evidence.", "Campaigns"),
_permission("campaigns:campaign:delete", "Delete campaigns", "Delete draft-only campaigns where retention policy allows it.", "Campaigns"), _permission("campaigns:campaign:delete", "Delete campaigns", "Delete draft-only campaigns where retention policy allows it.", "Campaigns"),
_permission("campaigns:campaign:share", "Share campaigns", "Grant or revoke explicit campaign access.", "Campaigns"), _permission("campaigns:campaign:share", "Share campaigns", "Grant or revoke explicit campaign access.", "Campaigns"),
_permission("campaigns:ownership:accept_group", "Accept group campaign ownership", "Accept or decline campaign ownership on behalf of a group the actor belongs to.", "Campaign governance"),
_permission("campaigns:ownership:recover", "Recover campaign ownership", "Participate in delayed, quorum-backed administrative campaign ownership recovery.", "Campaign governance"),
_permission("campaigns:campaign:validate", "Validate campaigns", "Run technical validation and manage validation locks.", "Campaigns"), _permission("campaigns:campaign:validate", "Validate campaigns", "Run technical validation and manage validation locks.", "Campaigns"),
_permission("campaigns:campaign:build", "Build campaigns", "Build exact messages and attachment evidence.", "Campaigns"), _permission("campaigns:campaign:build", "Build campaigns", "Build exact messages and attachment evidence.", "Campaigns"),
_permission("campaigns:campaign:review", "Complete campaign review", "Record review completion and the exact built messages inspected.", "Campaigns"), _permission("campaigns:campaign:review", "Complete campaign review", "Record review completion and the exact built messages inspected.", "Campaigns"),
_permission("campaigns:campaign:send_test", "Mock-send campaigns", "Use mock delivery and verification tools.", "Campaigns"), _permission("campaigns:campaign:send_test", "Mock-send campaigns", "Use mock delivery and verification tools.", "Campaigns"),
_permission("campaigns:campaign:queue", "Queue campaigns", "Place approved executions into the delivery queue.", "Campaigns"), _permission("campaigns:campaign:queue", "Queue campaigns", "Place approved executions into the delivery queue.", "Campaigns"),
_permission("campaigns:campaign:control", "Control delivery", "Pause, resume or cancel queued and sending jobs.", "Campaigns"), _permission("campaigns:campaign:control", "Control delivery", "Pause, resume or cancel queued and sending jobs.", "Campaigns"),
_permission("campaigns:campaign:send", "Send campaigns", "Start real SMTP delivery.", "Campaigns"), _permission("campaigns:campaign:send", "Send campaigns", "Start real Mail or Postbox delivery.", "Campaigns"),
_permission("campaigns:campaign:retry", "Retry delivery", "Retry failed or unattempted delivery jobs.", "Campaigns"), _permission("campaigns:campaign:retry", "Retry delivery", "Retry failed or unattempted delivery jobs.", "Campaigns"),
_permission("campaigns:campaign:reconcile", "Reconcile delivery", "Resolve outcome-unknown SMTP or IMAP attempts after inspection.", "Campaigns"), _permission("campaigns:campaign:reconcile", "Reconcile delivery", "Resolve outcome-unknown Mail, Postbox, or IMAP attempts after inspection.", "Campaigns"),
_permission("campaigns:diagnostic:read", "View campaign diagnostics", "Inspect worker claims and internal storage locators for campaign delivery troubleshooting.", "Campaign operations"), _permission("campaigns:diagnostic:read", "View campaign diagnostics", "Inspect worker claims and internal storage locators for campaign delivery troubleshooting.", "Campaign operations"),
_permission("campaigns:recipient:read", "View recipients", "Read recipient lists and recipient-specific campaign data.", "Recipients"), _permission("campaigns:recipient:read", "View recipients", "Read recipient lists and recipient-specific campaign data.", "Recipients"),
_permission("campaigns:recipient:write", "Edit recipients", "Create and edit recipient rows and field values.", "Recipients"), _permission("campaigns:recipient:write", "Edit recipients", "Create and edit recipient rows and field values.", "Recipients"),
@@ -74,7 +85,31 @@ PERMISSIONS = (
_permission("campaigns:report:send", "Send reports", "Email campaign reports to configured recipients.", "Reports"), _permission("campaigns:report:send", "Send reports", "Email campaign reports to configured recipients.", "Reports"),
) )
OPERATOR_QUEUE_REQUIRED_ANY = (
"campaigns:campaign:queue",
"campaigns:campaign:retry",
"campaigns:campaign:reconcile",
"campaigns:campaign:control",
)
# Preserve the former /operator route identity for saved View projections.
OPERATOR_QUEUE_SURFACE_ID = "campaigns.route.operator"
# Preserve the former /reports route identity for saved View projections.
REPORTS_SURFACE_ID = "campaigns.route.reports"
REPORTS_REQUIRED_ANY = ("campaigns:report:read",)
CAMPAIGN_MODULE_REQUIRED_ANY = (
"campaigns:campaign:read",
*REPORTS_REQUIRED_ANY,
)
ROLE_TEMPLATES = ( ROLE_TEMPLATES = (
RoleTemplate(
slug="campaign_aggregate_reader",
name="Campaign aggregate reader",
description="View privacy-protected outcome totals without recipient or delivery diagnostics.",
permissions=(
"campaigns:report:read",
),
),
RoleTemplate( RoleTemplate(
slug="campaign_manager", slug="campaign_manager",
name="Campaign manager", name="Campaign manager",
@@ -86,6 +121,7 @@ ROLE_TEMPLATES = (
"campaigns:campaign:copy", "campaigns:campaign:copy",
"campaigns:campaign:validate", "campaigns:campaign:validate",
"campaigns:campaign:build", "campaigns:campaign:build",
"campaigns:ownership:accept_group",
"campaigns:recipient:read", "campaigns:recipient:read",
"campaigns:recipient:write", "campaigns:recipient:write",
"campaigns:recipient:import", "campaigns:recipient:import",
@@ -131,6 +167,26 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
return {"campaigns": session.query(Campaign).filter(Campaign.tenant_id == tenant_id).count()} return {"campaigns": session.query(Campaign).filter(Campaign.tenant_id == tenant_id).count()}
def _tenant_summary_batch(session, tenant_ids) -> dict[str, dict[str, int]]:
from sqlalchemy import func
from govoplan_campaign.backend.db.models import Campaign
ids = tuple(dict.fromkeys(str(tenant_id) for tenant_id in tenant_ids if tenant_id))
if not ids:
return {}
rows = (
session.query(Campaign.tenant_id, func.count(Campaign.id))
.filter(Campaign.tenant_id.in_(ids))
.group_by(Campaign.tenant_id)
.all()
)
return {
tenant_id: {"campaigns": int(count)}
for tenant_id, count in rows
}
def _campaigns_router(context: ModuleContext): def _campaigns_router(context: ModuleContext):
from govoplan_campaign.backend.runtime import configure_runtime from govoplan_campaign.backend.runtime import configure_runtime
@@ -148,9 +204,16 @@ def _campaigns_router(context: ModuleContext):
manifest = ModuleManifest( manifest = ModuleManifest(
id="campaigns", id="campaigns",
name="Campaigns", name="Campaigns",
version="0.1.9", version="0.1.12",
required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR), required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR),
optional_dependencies=("files", "mail", "notifications", "addresses"), optional_capabilities=(CAPABILITY_ACCESS_REFERENCE_OPTIONS,),
optional_dependencies=(
"files",
"mail",
"notifications",
"addresses",
"postbox",
),
provides_interfaces=( provides_interfaces=(
ModuleInterfaceProvider(name="campaigns.access", version="0.1.6"), ModuleInterfaceProvider(name="campaigns.access", version="0.1.6"),
ModuleInterfaceProvider(name="campaigns.delivery_tasks", version="0.1.6"), ModuleInterfaceProvider(name="campaigns.delivery_tasks", version="0.1.6"),
@@ -159,6 +222,12 @@ manifest = ModuleManifest(
ModuleInterfaceProvider(name="campaigns.retention", version="0.1.6"), ModuleInterfaceProvider(name="campaigns.retention", version="0.1.6"),
), ),
requires_interfaces=( requires_interfaces=(
ModuleInterfaceRequirement(
name=CAPABILITY_ACCESS_REFERENCE_OPTIONS,
version_min="0.1.0",
version_max_exclusive="0.2.0",
optional=True,
),
ModuleInterfaceRequirement( ModuleInterfaceRequirement(
name="files.campaign_attachments", name="files.campaign_attachments",
version_min="0.1.0", version_min="0.1.0",
@@ -171,6 +240,12 @@ manifest = ModuleManifest(
version_max_exclusive="0.3.0", version_max_exclusive="0.3.0",
optional=True, optional=True,
), ),
ModuleInterfaceRequirement(
name="mail.delivery_commands",
version_min="0.1.0",
version_max_exclusive="0.2.0",
optional=True,
),
ModuleInterfaceRequirement( ModuleInterfaceRequirement(
name="addresses.lookup", name="addresses.lookup",
version_min="0.1.0", version_min="0.1.0",
@@ -183,49 +258,79 @@ manifest = ModuleManifest(
version_max_exclusive="0.2.0", version_max_exclusive="0.2.0",
optional=True, optional=True,
), ),
ModuleInterfaceRequirement(
name=CAPABILITY_POSTBOX_DELIVERY,
version_min="0.1.1",
version_max_exclusive="0.2.0",
optional=True,
),
ModuleInterfaceRequirement(
name=CAPABILITY_POSTBOX_DIRECTORY,
version_min="0.1.1",
version_max_exclusive="0.2.0",
optional=True,
),
ModuleInterfaceRequirement(
name=CAPABILITY_POSTBOX_EVIDENCE,
version_min="0.1.2",
version_max_exclusive="0.2.0",
optional=True,
),
), ),
permissions=PERMISSIONS, permissions=PERMISSIONS,
route_factory=_campaigns_router, route_factory=_campaigns_router,
role_templates=ROLE_TEMPLATES, role_templates=ROLE_TEMPLATES,
tenant_summary_providers=(_tenant_summary,), tenant_summary_providers=(_tenant_summary,),
tenant_summary_batch_providers=(_tenant_summary_batch,),
nav_items=( nav_items=(
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=("campaigns:campaign:read",), order=20), NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=CAMPAIGN_MODULE_REQUIRED_ANY, order=20),
NavItem(
path="/operator",
label="Operator Queue",
icon="radio-tower",
required_any=(
"campaigns:campaign:queue",
"campaigns:campaign:retry",
"campaigns:campaign:reconcile",
"campaigns:campaign:control",
"campaigns:campaign:send",
),
order=30,
),
NavItem(path="/reports", label="Reports", icon="clipboard-pen-line", required_any=("campaigns:report:read",), order=70),
), ),
frontend=FrontendModule( frontend=FrontendModule(
module_id="campaigns", module_id="campaigns",
package_name="@govoplan/campaign-webui", package_name="@govoplan/campaign-webui",
nav_items=( routes=(
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=("campaigns:campaign:read",), order=20), FrontendRoute(
NavItem( path="/campaigns",
path="/operator", component="CampaignModulePage",
label="Operator Queue", required_any=CAMPAIGN_MODULE_REQUIRED_ANY,
icon="radio-tower", order=20,
required_any=(
"campaigns:campaign:queue",
"campaigns:campaign:retry",
"campaigns:campaign:reconcile",
"campaigns:campaign:control",
"campaigns:campaign:send",
),
order=30,
), ),
NavItem(path="/reports", label="Reports", icon="clipboard-pen-line", required_any=("campaigns:report:read",), order=70), FrontendRoute(
path="/campaigns/queue",
component="OperatorQueuePage",
required_all=("campaigns:campaign:read",),
required_any=OPERATOR_QUEUE_REQUIRED_ANY,
order=21,
surface_id=OPERATOR_QUEUE_SURFACE_ID,
),
FrontendRoute(
path="/campaigns/:campaignId/*",
component="CampaignWorkspace",
required_any=("campaigns:campaign:read",),
order=22,
),
FrontendRoute(
path="/campaigns/reports",
component="AggregateReportsPage",
required_any=REPORTS_REQUIRED_ANY,
order=22,
surface_id=REPORTS_SURFACE_ID,
),
FrontendRoute(path="/templates", component="TemplatesPage", order=90),
),
nav_items=(
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=CAMPAIGN_MODULE_REQUIRED_ANY, order=20),
NavItem(path="/templates", label="Templates", icon="layout-template", order=90), NavItem(path="/templates", label="Templates", icon="layout-template", order=90),
), ),
view_surfaces=(
ViewSurface(
id="campaigns.widget.activity",
module_id="campaigns",
kind="section",
label="Campaign activity widget",
order=50,
),
),
), ),
migration_spec=MigrationSpec( migration_spec=MigrationSpec(
module_id="campaigns", module_id="campaigns",
@@ -242,7 +347,10 @@ manifest = ModuleManifest(
campaign_models.AttachmentBlob, campaign_models.AttachmentBlob,
campaign_models.AttachmentInstance, campaign_models.AttachmentInstance,
campaign_models.SendAttempt, campaign_models.SendAttempt,
campaign_models.CampaignMessageAction,
campaign_models.CampaignMessageActionAttempt,
campaign_models.ImapAppendAttempt, campaign_models.ImapAppendAttempt,
campaign_models.PostboxDeliveryAttempt,
label="Campaigns", label="Campaigns",
), ),
retirement_notes="Destructive retirement drops campaign-owned database tables after the installer captures a database snapshot.", retirement_notes="Destructive retirement drops campaign-owned database tables after the installer captures a database snapshot.",
@@ -258,12 +366,60 @@ manifest = ModuleManifest(
campaign_models.AttachmentBlob, campaign_models.AttachmentBlob,
campaign_models.AttachmentInstance, campaign_models.AttachmentInstance,
campaign_models.SendAttempt, campaign_models.SendAttempt,
campaign_models.CampaignMessageAction,
campaign_models.CampaignMessageActionAttempt,
campaign_models.ImapAppendAttempt, campaign_models.ImapAppendAttempt,
campaign_models.PostboxDeliveryAttempt,
label="Campaigns", label="Campaigns",
), ),
), ),
documentation=( documentation=(
*CAMPAIGN_USER_DOCUMENTATION, *CAMPAIGN_USER_DOCUMENTATION,
DocumentationTopic(
id="campaigns.postbox-delivery",
title="Deliver Campaign messages to Postboxes",
summary="Target one or more exact or organization-derived Postboxes per recipient row, independently or alongside Mail.",
body="Configure campaign-wide targets and optional per-row additions or replacements. Derived targets resolve a published Postbox template with organization unit, function, and optional context values, including values sourced from Campaign fields. Targets are frozen during build. Fallback crosses to the second channel only after a confirmed pre-acceptance rejection; accepted or outcome-unknown effects never trigger fallback.",
layer="available",
documentation_types=("user", "admin"),
audience=("campaign_manager", "campaign_reviewer", "campaign_sender", "campaign_operator"),
order=45,
conditions=(
DocumentationCondition(
required_modules=("campaigns", "postbox"),
any_scopes=("campaigns:recipient:write", "campaigns:campaign:send", "campaigns:campaign:reconcile"),
),
),
links=(
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
DocumentationLink(label="Postboxes", href="/postbox", kind="runtime"),
DocumentationLink(label="Campaign schema", href="/api/v1/campaigns/schema", kind="api"),
),
related_modules=("postbox", "organizations", "idm", "mail", "audit"),
unlocks=("Audited direct, derived, combined, and pre-acceptance fallback delivery to Postboxes.",),
metadata={
"kind": "workflow",
"route": "/campaigns/{campaign_id}/global-settings",
"screen": "Campaign delivery defaults and recipient data",
"prerequisites": [
"Campaign and Postbox are installed and active.",
"At least one exact Postbox or published Postbox template is available.",
],
"steps": [
"Choose Postbox, Mail and Postbox, or an ordered fallback policy.",
"Configure one or more campaign-wide targets.",
"Optionally add or replace targets for individual recipient rows.",
"Validate and build to freeze the resolved Postbox addresses before review.",
"Review, queue, and inspect channel-specific delivery evidence in the report.",
],
"outcome": "Each active row resolves an auditable set of Postbox targets without introducing a hard Campaign dependency on Postbox.",
"verification": "Confirm the frozen target list in the built job and verify accepted, rejected, or outcome-unknown attempts in Campaign reporting.",
"related_topic_ids": [
"campaigns.workflow.prepare-validate-and-build",
"campaigns.workflow.retry-and-reconcile",
],
},
),
DocumentationTopic( DocumentationTopic(
id="campaigns.mail-profile-user-journey", id="campaigns.mail-profile-user-journey",
title="Choose a Mail profile for campaign delivery", title="Choose a Mail profile for campaign delivery",
@@ -360,15 +516,15 @@ manifest = ModuleManifest(
), ),
), ),
links=( links=(
DocumentationLink(label="Campaign operator queue", href="/operator", kind="runtime"), DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
DocumentationLink(label="Campaign reports", href="/reports", kind="runtime"), DocumentationLink(label="Campaign reports", href="/campaigns/reports", kind="runtime"),
DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"), DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"),
), ),
related_modules=("mail", "audit"), related_modules=("mail", "audit"),
unlocks=("Fail-closed recovery without exposing Mail credentials.",), unlocks=("Fail-closed recovery without exposing Mail credentials.",),
metadata={ metadata={
"kind": "reference", "kind": "reference",
"route": "/operator", "route": "/campaigns/queue",
"screen": "Campaign operator queue", "screen": "Campaign operator queue",
"section": "Profile-backed delivery recovery", "section": "Profile-backed delivery recovery",
"related_topic_ids": [ "related_topic_ids": [
@@ -485,8 +641,8 @@ manifest = ModuleManifest(
DocumentationTopic( DocumentationTopic(
id="campaigns.workflow.retry-and-reconcile", id="campaigns.workflow.retry-and-reconcile",
title="Retry only known failures and reconcile uncertain effects", title="Retry only known failures and reconcile uncertain effects",
summary="Keep safe-to-retry failures separate from SMTP or IMAP effects whose outcome is unknown.", summary="Keep safe-to-retry failures separate from Mail, Postbox, or IMAP effects whose outcome is unknown.",
body="A retry creates new attempt evidence and is valid only for an explicitly eligible state. Never blindly retry an unknown SMTP or IMAP effect. Inspect external evidence, reconcile SMTP as accepted or not sent, and reconcile IMAP as appended or not appended; repairing Sent never resends accepted SMTP mail.", body="A retry creates new attempt evidence and is valid only for an explicitly eligible state. Never blindly retry an unknown Mail, Postbox, or IMAP effect. Inspect external evidence and reconcile the affected channel before continuing. Accepted Mail attempts and accepted Postbox targets are immutable during partial retries, and repairing Sent never resends accepted Mail.",
layer="evidence", layer="evidence",
documentation_types=("admin", "user"), documentation_types=("admin", "user"),
audience=("campaign_sender", "campaign_operator"), audience=("campaign_sender", "campaign_operator"),
@@ -498,14 +654,14 @@ manifest = ModuleManifest(
), ),
), ),
links=( links=(
DocumentationLink(label="Campaign operator queue", href="/operator", kind="runtime"), DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"), DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"),
), ),
related_modules=("mail", "audit"), related_modules=("mail", "audit"),
unlocks=("Evidence-backed recovery without accidental duplicate external effects.",), unlocks=("Evidence-backed recovery without accidental duplicate external effects.",),
metadata={ metadata={
"kind": "workflow", "kind": "workflow",
"route": "/operator", "route": "/campaigns/queue",
"screen": "Campaign operator queue", "screen": "Campaign operator queue",
"help_contexts": ["campaign.review-send", "campaign.report", "campaign.audit"], "help_contexts": ["campaign.review-send", "campaign.report", "campaign.audit"],
"prerequisites": [ "prerequisites": [
@@ -545,14 +701,14 @@ manifest = ModuleManifest(
DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"), DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"),
DocumentationLink(label="Reference examples and release checklist", href="govoplan-campaign/docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md", kind="repository"), DocumentationLink(label="Reference examples and release checklist", href="govoplan-campaign/docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md", kind="repository"),
), ),
related_modules=("mail", "files", "addresses", "audit"), related_modules=("mail", "postbox", "files", "addresses", "audit"),
unlocks=("A repeatable, supportable Campaign demonstration rather than an unverified module assembly.",), unlocks=("A repeatable, supportable Campaign demonstration rather than an unverified module assembly.",),
metadata={ metadata={
"kind": "reference", "kind": "reference",
"route": "/campaigns", "route": "/campaigns",
"screen": "Campaign reference composition", "screen": "Campaign reference composition",
"section": "Release, security, integration, and recovery assurance", "section": "Release, security, integration, and recovery assurance",
"verification": "Run the maintained examples, module-permutation tests, migration and restore drills, target SMTP/IMAP checks, version-alignment gate, WebUI/i18n checks, and full security audit for the pinned composition.", "verification": "Run the maintained examples, module-permutation tests, migration and restore drills, target Mail/Postbox/IMAP checks, version-alignment gate, WebUI/i18n checks, and full security audit for the pinned composition.",
"related_topic_ids": [ "related_topic_ids": [
"campaigns.workflow.prepare-validate-and-build", "campaigns.workflow.prepare-validate-and-build",
"campaigns.workflow.complete-review", "campaigns.workflow.complete-review",
@@ -563,6 +719,15 @@ manifest = ModuleManifest(
), ),
), ),
documentation_providers=(documentation_topics,), documentation_providers=(documentation_topics,),
ownership_providers=(
OwnershipProviderRegistration(
resource_type="campaign",
provider=__import__(
"govoplan_campaign.backend.capabilities",
fromlist=["CampaignOwnershipService"],
).CampaignOwnershipService(),
),
),
capability_factories={ capability_factories={
CAPABILITY_CAMPAIGNS_ACCESS: lambda context: __import__( CAPABILITY_CAMPAIGNS_ACCESS: lambda context: __import__(
"govoplan_campaign.backend.capabilities", "govoplan_campaign.backend.capabilities",

View File

@@ -16,6 +16,7 @@ from govoplan_campaign.backend.attachments.resolver import (
EntryAttachmentResolution, EntryAttachmentResolution,
MessageAttachmentStatus, MessageAttachmentStatus,
ResolvedAttachment, ResolvedAttachment,
effective_send_without_attachments_behavior,
resolve_entry_attachments, resolve_entry_attachments,
) )
from govoplan_campaign.backend.campaign.addressing import effective_address_lists, formatted_recipient from govoplan_campaign.backend.campaign.addressing import effective_address_lists, formatted_recipient
@@ -33,6 +34,7 @@ from govoplan_campaign.backend.campaign.models import (
ZipArchiveConfig, ZipArchiveConfig,
ZipPasswordMode, ZipPasswordMode,
ZipPasswordScope, ZipPasswordScope,
effective_delivery_channel_policy,
) )
from govoplan_campaign.backend.campaign.template_values import build_template_values from govoplan_campaign.backend.campaign.template_values import build_template_values
from govoplan_campaign.backend.services.zip_service import create_zip_archive from govoplan_campaign.backend.services.zip_service import create_zip_archive
@@ -171,6 +173,11 @@ def _attachment_summaries(resolution: EntryAttachmentResolution) -> list[Message
label=attachment.label, label=attachment.label,
status=attachment.status.value, status=attachment.status.value,
behavior=attachment.behavior.value if attachment.behavior else None, behavior=attachment.behavior.value if attachment.behavior else None,
missing_policy=(
attachment.missing_policy.model_dump(mode="json")
if attachment.missing_policy
else None
),
required=attachment.required, required=attachment.required,
allow_multiple=attachment.allow_multiple, allow_multiple=attachment.allow_multiple,
zip_enabled=attachment.zip_enabled, zip_enabled=attachment.zip_enabled,
@@ -199,6 +206,7 @@ def _message_issues_from_attachment_resolution(resolution: EntryAttachmentResolu
message=issue.message, message=issue.message,
behavior=issue.behavior.value if issue.behavior else None, behavior=issue.behavior.value if issue.behavior else None,
source="attachments", source="attachments",
details=issue.details,
) )
for issue in resolution.issues for issue in resolution.issues
] ]
@@ -218,12 +226,6 @@ def _append_no_attachment_coverage_issue(issues: list[MessageIssue]) -> None:
) )
def _send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
return config.attachments.send_without_attachments_behavior or (
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
)
def _safe_filename(value: str | None, fallback: str) -> str: def _safe_filename(value: str | None, fallback: str) -> str:
raw = value or fallback raw = value or fallback
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._") safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._")
@@ -425,8 +427,14 @@ def _attach_files(
return attached_count return attached_count
def _imap_initial_status(config: CampaignConfig) -> ImapStatus: def _imap_initial_status(
if config.delivery.imap_append_sent.enabled: config: CampaignConfig,
entry: EntryConfig,
) -> ImapStatus:
if (
effective_delivery_channel_policy(config, entry).uses_mail
and config.delivery.imap_append_sent.enabled
):
return ImapStatus.PENDING return ImapStatus.PENDING
return ImapStatus.NOT_REQUESTED return ImapStatus.NOT_REQUESTED
@@ -513,8 +521,18 @@ def _message_draft(
eml_path: str | None = None, eml_path: str | None = None,
eml_size: int | None = None, eml_size: int | None = None,
) -> MessageDraft: ) -> MessageDraft:
if validation_status == MessageValidationStatus.EXCLUDED:
# Exclusion is a completed validation decision, not a pending delivery.
# Keep both transport projections explicit so reports never imply that
# SMTP or IMAP work is still expected for this row.
send_status = SendStatus.SKIPPED
imap_status = ImapStatus.SKIPPED
if imap_status is None: if imap_status is None:
imap_status = _imap_initial_status(config) if build_status == BuildStatus.BUILT else ImapStatus.SKIPPED imap_status = (
_imap_initial_status(config, entry)
if build_status == BuildStatus.BUILT
else ImapStatus.SKIPPED
)
return MessageDraft( return MessageDraft(
entry_index=entry_index, entry_index=entry_index,
entry_id=entry.id, entry_id=entry.id,
@@ -523,6 +541,10 @@ def _message_draft(
validation_status=validation_status, validation_status=validation_status,
send_status=send_status, send_status=send_status,
imap_status=imap_status, imap_status=imap_status,
delivery_channel_policy=effective_delivery_channel_policy(
config,
entry,
).value,
subject=subject, subject=subject,
from_=_message_address(context.sender), from_=_message_address(context.sender),
from_all=_message_addresses(context.senders), from_all=_message_addresses(context.senders),
@@ -740,7 +762,11 @@ def _build_mime_message(
values=rendered.values, values=rendered.values,
work_dir=work_dir, work_dir=work_dir,
) )
if attachment_count == 0 and context.resolution.attachments and _send_without_attachments_behavior(config) == Behavior.BLOCK: if (
attachment_count == 0
and context.resolution.attachments
and effective_send_without_attachments_behavior(config) == Behavior.BLOCK
):
_append_no_attachment_coverage_issue(context.issues) _append_no_attachment_coverage_issue(context.issues)
return _MimeBuildResult( return _MimeBuildResult(
message=None, message=None,
@@ -764,6 +790,16 @@ def _build_mime_message(
source="attachments", source="attachments",
) )
) )
except OSError as exc:
context.issues.append(
MessageIssue(
severity="error",
code="attachment_unreadable",
message="A resolved attachment could not be read while building the message.",
behavior="block",
source=f"attachments:{type(exc).__name__}",
)
)
except Exception as exc: except Exception as exc:
context.issues.append( context.issues.append(
MessageIssue( MessageIssue(
@@ -803,17 +839,19 @@ def build_entry_message(
if not entry.active: if not entry.active:
return _inactive_entry_message(config=config, entry=entry, entry_index=entry_index, context=context) return _inactive_entry_message(config=config, entry=entry, entry_index=entry_index, context=context)
context.validation_status = _validate_required_sender( channel_policy = effective_delivery_channel_policy(config, entry)
context.senders, if channel_policy.uses_mail:
context.issues, context.validation_status = _validate_required_sender(
context.validation_status, context.senders,
) context.issues,
context.validation_status = _validate_required_recipients( context.validation_status,
config, )
context.recipients, context.validation_status = _validate_required_recipients(
context.issues, config,
context.validation_status, context.recipients,
) context.issues,
context.validation_status,
)
rendered = _render_message_template(config, campaign_file, entry) rendered = _render_message_template(config, campaign_file, entry)
context.validation_status = _validate_rendered_template( context.validation_status = _validate_rendered_template(
config, config,

View File

@@ -33,6 +33,7 @@ class MessageIssue(BaseModel):
message: str message: str
behavior: str | None = None behavior: str | None = None
source: str | None = None source: str | None = None
details: dict[str, object] = Field(default_factory=dict)
class MessageAddress(BaseModel): class MessageAddress(BaseModel):
@@ -49,6 +50,7 @@ class MessageAttachmentSummary(BaseModel):
label: str | None = None label: str | None = None
status: str status: str
behavior: str | None = None behavior: str | None = None
missing_policy: dict[str, object] | None = None
required: bool required: bool
allow_multiple: bool allow_multiple: bool
zip_enabled: bool zip_enabled: bool
@@ -78,6 +80,7 @@ class MessageDraft(BaseModel):
validation_status: MessageValidationStatus validation_status: MessageValidationStatus
send_status: SendStatus send_status: SendStatus
imap_status: ImapStatus imap_status: ImapStatus
delivery_channel_policy: str = "mail"
subject: str | None = None subject: str | None = None
from_: MessageAddress | None = Field(default=None, alias="from") from_: MessageAddress | None = Field(default=None, alias="from")

View File

@@ -0,0 +1,22 @@
"""add audit-proof single-message action ledger
Revision ID: c1a69e4f2b70
Revises: f0a1b2c3d4e5
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
message_actions = import_module(
"govoplan_campaign.backend.migrations.versions.c1a69e4f2b70_campaign_message_actions"
)
revision = message_actions.revision
down_revision = message_actions.down_revision
branch_labels = message_actions.branch_labels
depends_on = message_actions.depends_on
upgrade = message_actions.upgrade
downgrade = message_actions.downgrade

View File

@@ -0,0 +1,30 @@
"""persist the selected Campaign delivery mode
Revision ID: c7a2f91e4b60
Revises: 4d5e6f7a9203
Create Date: 2026-07-22 09:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c7a2f91e4b60"
down_revision = "4d5e6f7a9203"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
def downgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.drop_index("ix_campaign_versions_delivery_mode")
batch.drop_column("delivery_mode_selected_at")
batch.drop_column("delivery_mode")

View File

@@ -0,0 +1,22 @@
"""add monotonic campaign version edit revision
Revision ID: d2b7af503c81
Revises: c1a69e4f2b70
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
edit_revision = import_module(
"govoplan_campaign.backend.migrations.versions.d2b7af503c81_campaign_version_edit_revision"
)
revision = edit_revision.revision
down_revision = edit_revision.down_revision
branch_labels = edit_revision.branch_labels
depends_on = edit_revision.depends_on
upgrade = edit_revision.upgrade
downgrade = edit_revision.downgrade

View File

@@ -0,0 +1,42 @@
"""mark untouched excluded jobs as skipped delivery
Revision ID: d8b3e2c1f4a5
Revises: c7a2f91e4b60
Create Date: 2026-07-22 11:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d8b3e2c1f4a5"
down_revision = "c7a2f91e4b60"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Only normalize rows with no recorded transport effect. Unexpected
# historical delivery evidence must remain intact for audit/reconciliation.
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'skipped', imap_status = 'skipped' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'not_queued' "
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
)
)
def downgrade() -> None:
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'not_queued', imap_status = 'not_requested' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'skipped' "
"AND imap_status = 'skipped'"
)
)

View File

@@ -0,0 +1,32 @@
"""repair a missing IMAP append attempt claim token
Revision ID: e9f0a1b2c3d4
Revises: d8b3e2c1f4a5
Create Date: 2026-07-28 23:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e9f0a1b2c3d4"
down_revision = "d8b3e2c1f4a5"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
if "claim_token" not in columns:
op.add_column(
"imap_append_attempts",
sa.Column("claim_token", sa.String(length=36), nullable=True),
)
def downgrade() -> None:
# The column belongs to revision 3c4d5e6f8192. This repair revision only
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
pass

View File

@@ -0,0 +1,22 @@
"""add governed campaign Postbox delivery
Revision ID: f0a1b2c3d4e5
Revises: e9f0a1b2c3d4
Create Date: 2026-07-29 02:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
_migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"f0a1b2c3d4e5_v0115_postbox_delivery"
)
revision = _migration.revision
down_revision = _migration.down_revision
branch_labels = _migration.branch_labels
depends_on = _migration.depends_on
upgrade = _migration.upgrade
downgrade = _migration.downgrade

View File

@@ -0,0 +1,186 @@
"""add audit-proof single-message action ledger
Revision ID: c1a69e4f2b70
Revises: f0a1b2c3d4e5
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c1a69e4f2b70"
down_revision = "f0a1b2c3d4e5"
branch_labels = None
depends_on = "c91f0a72be34"
def upgrade() -> None:
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_actions" not in tables:
op.create_table(
"campaign_message_actions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("campaign_version_id", sa.String(length=36), nullable=False),
sa.Column("job_id", sa.String(length=36), nullable=False),
sa.Column("kind", sa.String(length=30), nullable=False),
sa.Column("idempotency_key", sa.String(length=200), nullable=False),
sa.Column("canonical_request_hash", sa.String(length=64), nullable=False),
sa.Column("reason", sa.Text(), nullable=True),
sa.Column("context", sa.JSON(), nullable=False),
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
sa.Column("actor_api_key_id", sa.String(length=36), nullable=True),
sa.Column("message_sha256", sa.String(length=64), nullable=False),
sa.Column("message_size_bytes", sa.Integer(), nullable=True),
sa.Column(
"recipient_manifest_sha256",
sa.String(length=64),
nullable=False,
),
sa.Column("recipient_count", sa.Integer(), nullable=False),
sa.Column("prior_send_status", sa.String(length=50), nullable=False),
sa.Column("prior_attempt_count", sa.Integer(), nullable=False),
sa.Column("final_send_status", sa.String(length=50), nullable=True),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("accepted_count", sa.Integer(), nullable=False),
sa.Column("refused_count", sa.Integer(), nullable=False),
sa.Column("refusal_summary", sa.JSON(), nullable=False),
sa.Column("error_type", sa.String(length=120), nullable=True),
sa.Column("error_message", sa.String(length=500), nullable=True),
sa.Column("linked_send_attempt_id", sa.String(length=36), nullable=True),
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["campaign_id"],
["campaigns.id"],
name=op.f(
"fk_campaign_message_actions_campaign_id_campaigns"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["campaign_version_id"],
["campaign_versions.id"],
name=op.f(
"fk_campaign_message_actions_campaign_version_id_campaign_versions"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["job_id"],
["campaign_jobs.id"],
name=op.f(
"fk_campaign_message_actions_job_id_campaign_jobs"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["actor_user_id"],
["access_users.id"],
name=op.f(
"fk_campaign_message_actions_actor_user_id_access_users"
),
ondelete="SET NULL",
),
sa.ForeignKeyConstraint(
["linked_send_attempt_id"],
["send_attempts.id"],
name=op.f(
"fk_campaign_message_actions_linked_send_attempt_id_send_attempts"
),
ondelete="SET NULL",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_message_actions"),
),
sa.UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_message_actions_idempotency",
),
)
op.create_index(
"ix_campaign_message_actions_job_created",
"campaign_message_actions",
["job_id", "created_at"],
unique=False,
)
op.create_index(
"ix_campaign_message_actions_campaign_kind",
"campaign_message_actions",
["campaign_id", "kind", "status"],
unique=False,
)
for column in (
"tenant_id",
"campaign_id",
"campaign_version_id",
"job_id",
"kind",
"actor_user_id",
"status",
"linked_send_attempt_id",
):
op.create_index(
op.f(f"ix_campaign_message_actions_{column}"),
"campaign_message_actions",
[column],
unique=False,
)
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_action_attempts" not in tables:
op.create_table(
"campaign_message_action_attempts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("action_id", sa.String(length=36), nullable=False),
sa.Column("attempt_number", sa.Integer(), nullable=False),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("accepted_count", sa.Integer(), nullable=False),
sa.Column("refused_count", sa.Integer(), nullable=False),
sa.Column("outcome_code", sa.String(length=80), nullable=True),
sa.Column("diagnostic_summary", sa.String(length=500), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["action_id"],
["campaign_message_actions.id"],
name=op.f(
"fk_campaign_message_action_attempts_action_id_campaign_message_actions"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_message_action_attempts"),
),
sa.UniqueConstraint(
"action_id",
"attempt_number",
name="uq_campaign_message_action_attempt_number",
),
)
for column in ("action_id", "status"):
op.create_index(
op.f(f"ix_campaign_message_action_attempts_{column}"),
"campaign_message_action_attempts",
[column],
unique=False,
)
def downgrade() -> None:
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_action_attempts" in tables:
op.drop_table("campaign_message_action_attempts")
if "campaign_message_actions" in tables:
op.drop_table("campaign_message_actions")

View File

@@ -0,0 +1,30 @@
"""persist the selected Campaign delivery mode
Revision ID: c7a2f91e4b60
Revises: 4d5e6f7a9203
Create Date: 2026-07-22 09:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c7a2f91e4b60"
down_revision = "4d5e6f7a9203"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
def downgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.drop_index("ix_campaign_versions_delivery_mode")
batch.drop_column("delivery_mode_selected_at")
batch.drop_column("delivery_mode")

View File

@@ -0,0 +1,45 @@
"""add monotonic campaign version edit revision
Revision ID: d2b7af503c81
Revises: c1a69e4f2b70
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d2b7af503c81"
down_revision = "c1a69e4f2b70"
branch_labels = None
depends_on = "c91f0a72be34"
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
if "edit_revision" not in columns:
with op.batch_alter_table("campaign_versions") as batch_op:
batch_op.add_column(
sa.Column(
"edit_revision",
sa.Integer(),
server_default="1",
nullable=False,
)
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
if "edit_revision" in columns:
with op.batch_alter_table("campaign_versions") as batch_op:
batch_op.drop_column("edit_revision")

View File

@@ -0,0 +1,42 @@
"""mark untouched excluded jobs as skipped delivery
Revision ID: d8b3e2c1f4a5
Revises: c7a2f91e4b60
Create Date: 2026-07-22 11:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d8b3e2c1f4a5"
down_revision = "c7a2f91e4b60"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Only normalize rows with no recorded transport effect. Unexpected
# historical delivery evidence must remain intact for audit/reconciliation.
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'skipped', imap_status = 'skipped' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'not_queued' "
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
)
)
def downgrade() -> None:
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'not_queued', imap_status = 'not_requested' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'skipped' "
"AND imap_status = 'skipped'"
)
)

View File

@@ -0,0 +1,32 @@
"""repair a missing IMAP append attempt claim token
Revision ID: e9f0a1b2c3d4
Revises: d8b3e2c1f4a5
Create Date: 2026-07-28 23:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e9f0a1b2c3d4"
down_revision = "d8b3e2c1f4a5"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
if "claim_token" not in columns:
op.add_column(
"imap_append_attempts",
sa.Column("claim_token", sa.String(length=36), nullable=True),
)
def downgrade() -> None:
# The column belongs to revision 3c4d5e6f8192. This repair revision only
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
pass

View File

@@ -0,0 +1,153 @@
"""add governed campaign Postbox delivery
Revision ID: f0a1b2c3d4e5
Revises: e9f0a1b2c3d4
Create Date: 2026-07-29 02:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "f0a1b2c3d4e5"
down_revision = "e9f0a1b2c3d4"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"campaign_jobs",
sa.Column(
"delivery_channel_policy",
sa.String(length=30),
nullable=False,
server_default="mail",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"postbox_status",
sa.String(length=50),
nullable=False,
server_default="not_requested",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"postbox_attempt_count",
sa.Integer(),
nullable=False,
server_default="0",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"resolved_postbox_targets",
sa.JSON(),
nullable=False,
server_default="[]",
),
)
op.create_index(
op.f("ix_campaign_jobs_delivery_channel_policy"),
"campaign_jobs",
["delivery_channel_policy"],
unique=False,
)
op.create_index(
op.f("ix_campaign_jobs_postbox_status"),
"campaign_jobs",
["postbox_status"],
unique=False,
)
op.create_table(
"campaign_postbox_delivery_attempts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("job_id", sa.String(length=36), nullable=False),
sa.Column("target_key", sa.String(length=64), nullable=False),
sa.Column("target_index", sa.Integer(), nullable=False),
sa.Column("attempt_number", sa.Integer(), nullable=False),
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("target_snapshot", sa.JSON(), nullable=False),
sa.Column("provider_delivery_id", sa.String(length=36), nullable=True),
sa.Column("provider_message_id", sa.String(length=36), nullable=True),
sa.Column("postbox_id", sa.String(length=36), nullable=True),
sa.Column("address", sa.String(length=500), nullable=True),
sa.Column("holder_count", sa.Integer(), nullable=True),
sa.Column("vacant", sa.Boolean(), nullable=True),
sa.Column(
"duplicate",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
sa.Column("evidence", sa.JSON(), nullable=False),
sa.Column("error_type", sa.String(length=255), nullable=True),
sa.Column("error_code", sa.String(length=100), nullable=True),
sa.Column("error_message", sa.Text(), nullable=True),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["job_id"],
["campaign_jobs.id"],
name=op.f(
"fk_campaign_postbox_delivery_attempts_job_id_campaign_jobs"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_postbox_delivery_attempts"),
),
sa.UniqueConstraint(
"job_id",
"target_key",
"attempt_number",
name="uq_campaign_postbox_attempt_target_number",
),
)
for column in ("tenant_id", "job_id", "status", "postbox_id"):
op.create_index(
op.f(f"ix_campaign_postbox_delivery_attempts_{column}"),
"campaign_postbox_delivery_attempts",
[column],
unique=False,
)
op.create_index(
"ix_campaign_postbox_attempt_job_status",
"campaign_postbox_delivery_attempts",
["job_id", "status"],
unique=False,
)
op.create_index(
"ix_campaign_postbox_attempt_idempotency",
"campaign_postbox_delivery_attempts",
["tenant_id", "idempotency_key"],
unique=False,
)
def downgrade() -> None:
op.drop_table("campaign_postbox_delivery_attempts")
op.drop_index(
op.f("ix_campaign_jobs_postbox_status"),
table_name="campaign_jobs",
)
op.drop_index(
op.f("ix_campaign_jobs_delivery_channel_policy"),
table_name="campaign_jobs",
)
op.drop_column("campaign_jobs", "resolved_postbox_targets")
op.drop_column("campaign_jobs", "postbox_attempt_count")
op.drop_column("campaign_jobs", "postbox_status")
op.drop_column("campaign_jobs", "delivery_channel_policy")

View File

@@ -22,6 +22,7 @@ from govoplan_campaign.backend.db.models import (
CampaignVersion, CampaignVersion,
CampaignVersionWorkflowState, CampaignVersionWorkflowState,
JobImapStatus, JobImapStatus,
JobPostboxStatus,
JobQueueStatus, JobQueueStatus,
JobSendStatus, JobSendStatus,
JobValidationStatus, JobValidationStatus,
@@ -30,13 +31,26 @@ from govoplan_campaign.backend.campaign.loader import load_campaign_json, valida
from govoplan_campaign.backend.campaign.mail_profile_boundary import ( from govoplan_campaign.backend.campaign.mail_profile_boundary import (
assert_campaign_uses_mail_profile_reference, assert_campaign_uses_mail_profile_reference,
campaign_mail_profile_id, campaign_mail_profile_id,
campaign_mail_resource_ids,
) )
from govoplan_campaign.backend.campaign.validation import validate_campaign_config from govoplan_campaign.backend.campaign.validation import validate_campaign_config
from govoplan_campaign.backend.campaign.entries import load_campaign_entries
from govoplan_campaign.backend.campaign.postbox_targets import (
resolve_entry_postbox_targets,
)
from govoplan_campaign.backend.messages.builder import build_campaign_messages from govoplan_campaign.backend.messages.builder import build_campaign_messages
from govoplan_campaign.backend.messages.models import MessageDraft from govoplan_campaign.backend.messages.models import MessageDraft
from govoplan_campaign.backend.sending.execution import create_execution_snapshot, profile_transport_revisions from govoplan_campaign.backend.sending.execution import create_execution_snapshot, profile_delivery_summary
from govoplan_campaign.backend.campaign.models import CampaignConfig from govoplan_campaign.backend.campaign.models import (
from govoplan_campaign.backend.integrations import files_integration, mail_integration CampaignConfig,
DeliveryChannelPolicy,
SendStatus,
)
from govoplan_campaign.backend.integrations import (
files_integration,
mail_integration,
postbox_integration,
)
from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths
RUNTIME_DIR = Path(__file__).resolve().parents[3] / "runtime" RUNTIME_DIR = Path(__file__).resolve().parents[3] / "runtime"
@@ -73,6 +87,7 @@ def load_campaign_config_from_json(
materialized = copy.deepcopy(raw_json) materialized = copy.deepcopy(raw_json)
profile_id = campaign_mail_profile_id(raw_json) profile_id = campaign_mail_profile_id(raw_json)
if profile_id: if profile_id:
references = campaign_mail_resource_ids(raw_json)
summary = mail_integration().campaign_profile_delivery_summary( summary = mail_integration().campaign_profile_delivery_summary(
session, session,
tenant_id=tenant_id, tenant_id=tenant_id,
@@ -80,6 +95,10 @@ def load_campaign_config_from_json(
profile_id=profile_id, profile_id=profile_id,
owner_user_id=owner_user_id, owner_user_id=owner_user_id,
owner_group_id=owner_group_id, owner_group_id=owner_group_id,
smtp_server_id=references["smtp_server_id"],
smtp_credential_id=references["smtp_credential_id"],
imap_server_id=references["imap_server_id"],
imap_credential_id=references["imap_credential_id"],
) )
materialized.setdefault("server", {})["profile_capabilities"] = { materialized.setdefault("server", {})["profile_capabilities"] = {
"smtp_available": bool(summary.get("smtp_available")), "smtp_available": bool(summary.get("smtp_available")),
@@ -323,9 +342,19 @@ def validate_campaign_version(
) as prepared: ) as prepared:
managed_raw = load_campaign_json(prepared.path) managed_raw = load_campaign_json(prepared.path)
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id) managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
report = validate_campaign_config(managed_config, campaign_file=prepared.path, check_files=True) report = validate_campaign_config(
managed_config,
campaign_file=prepared.path,
check_files=True,
postbox_available=postbox_integration().available,
)
else: else:
report = validate_campaign_config(config, campaign_file=snapshot_path, check_files=False) report = validate_campaign_config(
config,
campaign_file=snapshot_path,
check_files=False,
postbox_available=postbox_integration().available,
)
report_json = report.model_dump(mode="json") report_json = report.model_dump(mode="json")
report_json.update({"ok": report.ok, "error_count": report.error_count, "warning_count": report.warning_count}) report_json.update({"ok": report.ok, "error_count": report.error_count, "warning_count": report.warning_count})
version.validation_summary = report_json version.validation_summary = report_json
@@ -388,6 +417,7 @@ def _job_from_message(
campaign_id: str, campaign_id: str,
version_id: str, version_id: str,
message: MessageDraft, message: MessageDraft,
resolved_postbox_targets: list[dict[str, Any]] | None = None,
) -> CampaignJob: ) -> CampaignJob:
recipient_email = message.to[0].email if message.to else None recipient_email = message.to[0].email if message.to else None
eml_sha256, message_id_header = _eml_evidence(message.eml_path) eml_sha256, message_id_header = _eml_evidence(message.eml_path)
@@ -406,7 +436,17 @@ def _job_from_message(
build_status=message.build_status.value if hasattr(message.build_status, "value") else str(message.build_status), build_status=message.build_status.value if hasattr(message.build_status, "value") else str(message.build_status),
validation_status=_job_validation_status(message.validation_status.value), validation_status=_job_validation_status(message.validation_status.value),
queue_status=JobQueueStatus.DRAFT.value, queue_status=JobQueueStatus.DRAFT.value,
send_status=JobSendStatus.NOT_QUEUED.value, send_status=(
JobSendStatus.SKIPPED.value
if message.send_status == SendStatus.SKIPPED
else JobSendStatus.NOT_QUEUED.value
),
delivery_channel_policy=message.delivery_channel_policy,
postbox_status=(
JobPostboxStatus.PENDING.value
if DeliveryChannelPolicy(message.delivery_channel_policy).uses_postbox
else JobPostboxStatus.NOT_REQUESTED.value
),
imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value, imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value,
resolved_recipients={ resolved_recipients={
"from": message.from_.model_dump(mode="json") if message.from_ else None, "from": message.from_.model_dump(mode="json") if message.from_ else None,
@@ -418,12 +458,184 @@ def _job_from_message(
"bounce_to": [item.model_dump(mode="json") for item in message.bounce_to], "bounce_to": [item.model_dump(mode="json") for item in message.bounce_to],
"disposition_notification_to": [item.model_dump(mode="json") for item in message.disposition_notification_to], "disposition_notification_to": [item.model_dump(mode="json") for item in message.disposition_notification_to],
}, },
resolved_postbox_targets=resolved_postbox_targets or [],
resolved_attachments=[files_integration().public_attachment_summary_payload(item) for item in message.attachments], resolved_attachments=[files_integration().public_attachment_summary_payload(item) for item in message.attachments],
issues_snapshot=[item.model_dump(mode="json") for item in message.issues], issues_snapshot=[item.model_dump(mode="json") for item in message.issues],
last_error="; ".join(issue.message for issue in message.issues if issue.severity == "error") or None, last_error="; ".join(issue.message for issue in message.issues if issue.severity == "error") or None,
) )
def _resolve_built_postbox_targets(
session: Session,
*,
tenant_id: str,
config: CampaignConfig,
built_messages: list[Any],
entries_by_index: dict[int, Any],
) -> dict[int, list[dict[str, Any]]]:
resolved_by_index: dict[int, list[dict[str, Any]]] = {}
for built in built_messages:
if not DeliveryChannelPolicy(built.draft.delivery_channel_policy).uses_postbox:
continue
entry = entries_by_index.get(built.draft.entry_index)
if entry is None:
raise CampaignPersistenceError("Built recipient row is missing from the campaign input.")
resolved, issues, validation_status = resolve_entry_postbox_targets(
session,
tenant_id=tenant_id,
config=config,
entry=entry,
validation_status=built.draft.validation_status,
materialize=True,
)
built.draft.issues.extend(issues)
built.draft.validation_status = validation_status
resolved_by_index[built.draft.entry_index] = resolved
return resolved_by_index
def _campaign_build_report(result: Any, files: Any) -> dict[str, Any]:
report_json = result.report.model_dump(mode="json", by_alias=True)
for message_payload, message in zip(report_json.get("messages", []), result.report.messages, strict=False):
if isinstance(message_payload, dict):
message_payload["attachments"] = [files.public_attachment_summary_payload(item) for item in message.attachments]
report_json.update({
"built_at": datetime.now(UTC).isoformat(),
"build_token": uuid4().hex,
"built_count": result.report.built_count,
"build_failed_count": result.report.build_failed_count,
"ready_count": result.report.ready_count,
"warning_count": result.report.warning_count,
"needs_review_count": result.report.needs_review_count,
"blocked_count": result.report.blocked_count,
"excluded_count": result.report.excluded_count,
"inactive_count": result.report.inactive_count,
"queueable_count": result.report.queueable_count,
})
return report_json
def _replace_version_jobs(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str,
built_messages: list[Any],
postbox_targets_by_index: dict[int, list[dict[str, Any]]],
) -> list[tuple[CampaignJob, MessageDraft]]:
session.query(CampaignIssue).filter(
CampaignIssue.campaign_version_id == version_id,
CampaignIssue.job_id.is_not(None),
).delete(synchronize_session=False)
session.query(CampaignJob).filter(CampaignJob.campaign_version_id == version_id).delete(synchronize_session=False)
session.flush()
pairs: list[tuple[CampaignJob, MessageDraft]] = []
for built in built_messages:
job = _job_from_message(
tenant_id=tenant_id,
campaign_id=campaign_id,
version_id=version_id,
message=built.draft,
resolved_postbox_targets=postbox_targets_by_index.get(built.draft.entry_index, []),
)
session.add(job)
pairs.append((job, built.draft))
session.flush()
return pairs
def _mail_execution_profile(
session: Session,
*,
version: CampaignVersion,
config: CampaignConfig,
jobs: list[CampaignJob],
) -> tuple[str | None, dict[str, Any]]:
if not any(DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail for job in jobs):
return None, {}
if not config.server.profile_capabilities.smtp_available:
raise CampaignPersistenceError("The selected Mail profile has no SMTP configuration; an execution snapshot cannot be created.")
profile_id = campaign_mail_profile_id(version.raw_json if isinstance(version.raw_json, dict) else {})
if profile_id is None:
raise CampaignPersistenceError("Select an authorized Mail profile before building campaign messages that use Mail.")
summary = profile_delivery_summary(session, version)
if not summary.get("smtp_transport_revision"):
raise CampaignPersistenceError("The selected Mail profile has no SMTP transport revision.")
return profile_id, summary
def _store_execution_snapshot(
session: Session,
*,
version: CampaignVersion,
config: CampaignConfig,
jobs: list[CampaignJob],
build_summary: dict[str, Any],
) -> None:
profile_id, profile = _mail_execution_profile(session, version=version, config=config, jobs=jobs)
snapshot, snapshot_hash = create_execution_snapshot(
version,
mail_profile_id=profile_id,
smtp_server_id=profile.get("smtp_server_id"),
smtp_credential_id=profile.get("smtp_credential_id"),
imap_server_id=profile.get("imap_server_id"),
imap_credential_id=profile.get("imap_credential_id"),
smtp_transport_revision=profile.get("smtp_transport_revision"),
imap_transport_revision=profile.get("imap_transport_revision"),
delivery=config.delivery,
jobs=jobs,
build_summary=build_summary,
)
version.execution_snapshot = snapshot
version.execution_snapshot_hash = snapshot_hash
version.execution_snapshot_at = datetime.now(UTC)
def _store_job_issues(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str,
job_build_pairs: list[tuple[CampaignJob, MessageDraft]],
) -> None:
for job, message in job_build_pairs:
session.add_all([
CampaignIssue(
tenant_id=tenant_id,
campaign_id=campaign_id,
campaign_version_id=version_id,
job_id=job.id,
severity=issue.severity,
code=issue.code,
message=issue.message,
source=issue.source,
behavior=issue.behavior,
)
for issue in message.issues
])
def _apply_campaign_build_state(
campaign: Campaign,
version: CampaignVersion,
*,
needs_review_count: int,
blocked_count: int,
queueable_count: int,
) -> None:
if needs_review_count or blocked_count:
campaign.status = CampaignStatus.NEEDS_REVIEW.value
version.workflow_state = CampaignVersionWorkflowState.APPROVED.value
elif queueable_count > 0:
campaign.status = CampaignStatus.READY_TO_QUEUE.value
version.workflow_state = CampaignVersionWorkflowState.BUILT.value
else:
campaign.status = CampaignStatus.VALIDATED.value
def build_campaign_version( def build_campaign_version(
session: Session, session: Session,
*, *,
@@ -457,97 +669,54 @@ def build_campaign_version(
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id) managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
result = build_campaign_messages(managed_config, campaign_file=prepared.path, output_dir=output_dir, write_eml=write_eml) result = build_campaign_messages(managed_config, campaign_file=prepared.path, output_dir=output_dir, write_eml=write_eml)
files.annotate_built_messages_with_managed_files(result.built_messages, prepared.managed_files_by_local_path) files.annotate_built_messages_with_managed_files(result.built_messages, prepared.managed_files_by_local_path)
report_json = result.report.model_dump(mode="json", by_alias=True) entries_by_index = {
for message_payload, message in zip(report_json.get("messages", []), result.report.messages, strict=False): index: entry
if isinstance(message_payload, dict): for index, entry in enumerate(
message_payload["attachments"] = [files.public_attachment_summary_payload(item) for item in message.attachments] load_campaign_entries(
report_json["built_at"] = datetime.now(UTC).isoformat() managed_config,
report_json["build_token"] = uuid4().hex campaign_file=prepared.path,
report_json.update({ ),
"built_count": result.report.built_count, start=1,
"build_failed_count": result.report.build_failed_count, )
"ready_count": result.report.ready_count, }
"warning_count": result.report.warning_count, resolved_postbox_targets_by_index = _resolve_built_postbox_targets(
"needs_review_count": result.report.needs_review_count, session,
"blocked_count": result.report.blocked_count, tenant_id=tenant_id,
"excluded_count": result.report.excluded_count, config=managed_config,
"inactive_count": result.report.inactive_count, built_messages=result.built_messages,
"queueable_count": result.report.queueable_count, entries_by_index=entries_by_index,
}) )
report_json = _campaign_build_report(result, files)
version.build_summary = report_json version.build_summary = report_json
editor_state = copy.deepcopy(version.editor_state or {}) editor_state = copy.deepcopy(version.editor_state or {})
editor_state.pop("review_send", None) editor_state.pop("review_send", None)
version.editor_state = editor_state version.editor_state = editor_state
# Rebuild jobs for the current version. Later, protect sent jobs from destructive rebuilds. job_build_pairs = _replace_version_jobs(
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id, CampaignIssue.job_id.is_not(None)).delete(synchronize_session=False)
session.query(CampaignJob).filter(CampaignJob.campaign_version_id == version.id).delete(synchronize_session=False)
session.flush()
job_build_pairs: list[tuple[CampaignJob, MessageDraft]] = []
for built in result.built_messages:
job = _job_from_message(
tenant_id=tenant_id,
campaign_id=campaign.id,
version_id=version.id,
message=built.draft,
)
session.add(job)
job_build_pairs.append((job, built.draft))
# Assign all job IDs in one round-trip, then persist exact attachment use
# records in bulk. This avoids one flush plus several metadata queries per
# recipient for large campaigns.
session.flush()
files.record_campaign_attachment_uses_for_jobs(
session, session,
[job for job, _message in job_build_pairs], tenant_id=tenant_id,
stage="built", campaign_id=campaign.id,
version_id=version.id,
built_messages=result.built_messages,
postbox_targets_by_index=resolved_postbox_targets_by_index,
) )
if not managed_config.server.profile_capabilities.smtp_available: jobs = [job for job, _message in job_build_pairs]
raise CampaignPersistenceError("The selected Mail profile has no SMTP configuration; an execution snapshot cannot be created") files.record_campaign_attachment_uses_for_jobs(session, jobs, stage="built")
profile_id = campaign_mail_profile_id(version.raw_json if isinstance(version.raw_json, dict) else {}) _store_execution_snapshot(session, version=version, config=managed_config, jobs=jobs, build_summary=report_json)
if profile_id is None: _store_job_issues(
raise CampaignPersistenceError("Select an authorized Mail profile before building campaign messages") session,
revisions = profile_transport_revisions(session, version) tenant_id=tenant_id,
if not revisions["smtp"]: campaign_id=campaign.id,
raise CampaignPersistenceError("The selected Mail profile has no SMTP transport revision") version_id=version.id,
execution_snapshot, execution_snapshot_hash = create_execution_snapshot( job_build_pairs=job_build_pairs,
)
_apply_campaign_build_state(
campaign,
version, version,
mail_profile_id=profile_id, needs_review_count=result.report.needs_review_count,
smtp_transport_revision=revisions["smtp"], blocked_count=result.report.blocked_count,
imap_transport_revision=revisions["imap"], queueable_count=result.report.queueable_count,
delivery=managed_config.delivery,
jobs=[job for job, _message in job_build_pairs],
build_summary=report_json,
) )
version.execution_snapshot = execution_snapshot
version.execution_snapshot_hash = execution_snapshot_hash
version.execution_snapshot_at = datetime.now(UTC)
for job, message in job_build_pairs:
for issue in message.issues:
session.add(
CampaignIssue(
tenant_id=tenant_id,
campaign_id=campaign.id,
campaign_version_id=version.id,
job_id=job.id,
severity=issue.severity,
code=issue.code,
message=issue.message,
source=issue.source,
behavior=issue.behavior,
)
)
if result.report.needs_review_count or result.report.blocked_count:
campaign.status = CampaignStatus.NEEDS_REVIEW.value
version.workflow_state = CampaignVersionWorkflowState.APPROVED.value
elif result.report.queueable_count > 0:
campaign.status = CampaignStatus.READY_TO_QUEUE.value
version.workflow_state = CampaignVersionWorkflowState.BUILT.value
else:
campaign.status = CampaignStatus.VALIDATED.value
session.add(version) session.add(version)
session.add(campaign) session.add(campaign)

View File

@@ -1,13 +1,20 @@
from __future__ import annotations from __future__ import annotations
import copy import copy
import hashlib
import json
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import Any from typing import Any
from uuid import uuid4 from uuid import uuid4
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from sqlalchemy.orm.exc import StaleDataError
from govoplan_core.core.concurrency import (
RevisionConflictError,
strong_resource_etag,
)
from govoplan_campaign.backend.db.models import ( from govoplan_campaign.backend.db.models import (
Campaign, Campaign,
CampaignIssue, CampaignIssue,
@@ -152,7 +159,7 @@ def minimal_campaign_json(*, external_id: str, name: str, description: str | Non
"send_without_attachments_behavior": "block", "send_without_attachments_behavior": "block",
"global": [], "global": [],
"zip": {"enabled": False, "archives": []}, "zip": {"enabled": False, "archives": []},
"missing_behavior": "ask", "missing_behavior": "warn",
"ambiguous_behavior": "ask", "ambiguous_behavior": "ask",
}, },
"entries": { "entries": {
@@ -170,7 +177,7 @@ def minimal_campaign_json(*, external_id: str, name: str, description: str | Non
}, },
}, },
"validation_policy": { "validation_policy": {
"missing_required_attachment": "ask", "missing_required_attachment": "block",
"missing_optional_attachment": "warn", "missing_optional_attachment": "warn",
"ambiguous_attachment_match": "ask", "ambiguous_attachment_match": "ask",
"unsent_attachment_files": "warn", "unsent_attachment_files": "warn",
@@ -328,6 +335,112 @@ def _apply_campaign_metadata(campaign: Campaign, raw_json: dict[str, Any]) -> No
campaign.external_id = campaign_meta.get("id") or campaign.external_id campaign.external_id = campaign_meta.get("id") or campaign.external_id
def _assert_fork_source_allowed(session: Session, *, campaign: Campaign, source: CampaignVersion) -> None:
if campaign_has_active_working_version(session, campaign):
current = session.get(CampaignVersion, campaign.current_version_id)
current_number = current.version_number if current else "current"
raise LockedCampaignVersionError(
f"Campaign already has active working version #{current_number}. "
"Unlock or continue editing that version instead of creating a parallel draft."
)
if campaign.current_version_id and source.id != campaign.current_version_id:
raise LockedCampaignVersionError(
"Historical versions remain review-only and cannot become a new branch. "
"Create the next working copy from the campaign's current immutable version."
)
def _fork_runtime_json(
session: Session,
*,
tenant_id: str,
campaign: Campaign,
source: CampaignVersion,
raw_json: dict[str, Any] | None,
source_filename: str | None,
source_base_path: str | None,
migrate_legacy_mail_settings: bool,
) -> dict[str, Any]:
source_json = source.raw_json if isinstance(source.raw_json, dict) else {}
requires_migration = bool(campaign_mail_profile_boundary_violations(source_json))
if requires_migration and not migrate_legacy_mail_settings:
raise CampaignPersistenceError(
"This version contains legacy campaign-local SMTP/IMAP settings. Create the editable copy from "
"the Mail settings migration action so the audit record is preserved and the copy uses a Mail profile."
)
base_json = raw_json if raw_json is not None else copy.deepcopy(source_json)
if requires_migration and raw_json is None:
base_json["server"] = public_campaign_mail_server(source_json)
assert_server_safe_campaign_paths(
base_json,
source_filename=source_filename,
source_base_path=source_base_path,
managed_files_available=files_integration().available,
)
runtime_json = normalize_campaign_paths(base_json, source_base_path) if source_base_path else copy.deepcopy(base_json)
assert_campaign_uses_mail_profile_reference(runtime_json)
mail_integration().assert_campaign_mail_policy_allows_json(
session,
tenant_id=tenant_id,
raw_json=runtime_json,
campaign_id=campaign.id,
)
return runtime_json
def _new_forked_campaign_version(
session: Session,
*,
campaign: Campaign,
source: CampaignVersion,
runtime_json: dict[str, Any],
current_flow: str | None,
current_step: str | None,
editor_state: dict[str, Any] | None,
source_filename: str | None,
source_base_path: str | None,
autosave: bool,
) -> CampaignVersion:
return CampaignVersion(
campaign_id=campaign.id,
version_number=_next_version_number(session, campaign.id),
raw_json=runtime_json,
schema_version=str(runtime_json.get("version", source.schema_version or "1.0")),
source_filename=source_filename if source_filename is not None else source.source_filename,
source_base_path=source_base_path if source_base_path is not None else source.source_base_path,
workflow_state=CampaignVersionWorkflowState.EDITING.value,
current_flow=current_flow if current_flow is not None else (source.current_flow or CampaignVersionFlow.MANUAL.value),
current_step=current_step if current_step is not None else source.current_step,
is_complete=False,
editor_state=(
validate_campaign_editor_state(editor_state)
if editor_state is not None
else campaign_editor_state_for_edit(source.editor_state)
),
autosaved_at=datetime.now(UTC) if autosave else None,
)
def _persist_forked_version(
session: Session,
*,
campaign: Campaign,
version: CampaignVersion,
commit: bool,
) -> None:
session.add(version)
session.flush()
_apply_campaign_metadata(campaign, version.raw_json)
campaign.current_version_id = version.id
campaign.status = CampaignStatus.DRAFT.value
session.add(campaign)
if commit:
_write_campaign_snapshot(version)
session.commit()
else:
session.flush()
def fork_campaign_version_for_edit( def fork_campaign_version_for_edit(
session: Session, session: Session,
*, *,
@@ -354,69 +467,30 @@ def fork_campaign_version_for_edit(
source = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id) source = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
campaign = _require_campaign(session, campaign_id) campaign = _require_campaign(session, campaign_id)
if campaign_has_active_working_version(session, campaign): _assert_fork_source_allowed(session, campaign=campaign, source=source)
current = session.get(CampaignVersion, campaign.current_version_id) runtime_json = _fork_runtime_json(
current_number = current.version_number if current else "current" session,
raise LockedCampaignVersionError( tenant_id=tenant_id,
f"Campaign already has active working version #{current_number}. " campaign=campaign,
"Unlock or continue editing that version instead of creating a parallel draft." source=source,
) raw_json=raw_json,
if campaign.current_version_id and source.id != campaign.current_version_id:
raise LockedCampaignVersionError(
"Historical versions remain review-only and cannot become a new branch. "
"Create the next working copy from the campaign's current immutable version."
)
source_json = source.raw_json if isinstance(source.raw_json, dict) else {}
source_requires_mail_migration = bool(campaign_mail_profile_boundary_violations(source_json))
if source_requires_mail_migration and not migrate_legacy_mail_settings:
raise CampaignPersistenceError(
"This version contains legacy campaign-local SMTP/IMAP settings. Create the editable copy from "
"the Mail settings migration action so the audit record is preserved and the copy uses a Mail profile."
)
base_json = raw_json if raw_json is not None else copy.deepcopy(source_json)
if source_requires_mail_migration and raw_json is None:
base_json["server"] = public_campaign_mail_server(source_json)
assert_server_safe_campaign_paths(
base_json,
source_filename=source_filename, source_filename=source_filename,
source_base_path=source_base_path, source_base_path=source_base_path,
managed_files_available=files_integration().available, migrate_legacy_mail_settings=migrate_legacy_mail_settings,
) )
runtime_json = normalize_campaign_paths(base_json, source_base_path) if source_base_path else copy.deepcopy(base_json) new_version = _new_forked_campaign_version(
assert_campaign_uses_mail_profile_reference(runtime_json) session,
mail_integration().assert_campaign_mail_policy_allows_json(session, tenant_id=tenant_id, raw_json=runtime_json, campaign_id=campaign.id) campaign=campaign,
source=source,
new_version = CampaignVersion( runtime_json=runtime_json,
campaign_id=campaign.id, current_flow=current_flow,
version_number=_next_version_number(session, campaign.id), current_step=current_step,
raw_json=runtime_json, editor_state=editor_state,
schema_version=str(runtime_json.get("version", source.schema_version or "1.0")), source_filename=source_filename,
source_filename=source_filename if source_filename is not None else source.source_filename, source_base_path=source_base_path,
source_base_path=source_base_path if source_base_path is not None else source.source_base_path, autosave=autosave,
workflow_state=CampaignVersionWorkflowState.EDITING.value,
current_flow=current_flow if current_flow is not None else (source.current_flow or CampaignVersionFlow.MANUAL.value),
current_step=current_step if current_step is not None else source.current_step,
is_complete=False,
editor_state=(
validate_campaign_editor_state(editor_state)
if editor_state is not None
else campaign_editor_state_for_edit(source.editor_state)
),
autosaved_at=datetime.now(UTC) if autosave else None,
) )
session.add(new_version) _persist_forked_version(session, campaign=campaign, version=new_version, commit=commit)
session.flush()
_apply_campaign_metadata(campaign, runtime_json)
campaign.current_version_id = new_version.id
campaign.status = CampaignStatus.DRAFT.value
session.add(campaign)
if commit:
_write_campaign_snapshot(new_version)
session.commit()
else:
session.flush()
return new_version return new_version
@@ -530,6 +604,113 @@ def unlock_validated_campaign_version(
session.flush() session.flush()
return version return version
def _assert_update_paths_safe(
raw_json: dict[str, Any] | None,
*,
source_filename: str | None,
source_base_path: str | None,
) -> None:
if raw_json is None and source_filename is None and source_base_path is None:
return
assert_server_safe_campaign_paths(
raw_json if raw_json is not None else {},
source_filename=source_filename,
source_base_path=source_base_path,
managed_files_available=files_integration().available,
)
def _updated_runtime_json(
session: Session,
*,
tenant_id: str,
campaign: Campaign,
version: CampaignVersion,
raw_json: dict[str, Any],
source_base_path: str | None,
migrate_legacy_mail_settings: bool,
) -> dict[str, Any]:
runtime_json = normalize_campaign_paths(raw_json, source_base_path) if source_base_path else copy.deepcopy(raw_json)
requires_migration = bool(campaign_mail_profile_boundary_violations(version.raw_json))
if requires_migration and not migrate_legacy_mail_settings:
raise CampaignPersistenceError(
"This version contains legacy campaign-local SMTP/IMAP settings. Select an authorized Mail "
"profile on the Mail settings page and explicitly save the migration; the stored legacy version "
"will not be changed automatically."
)
assert_campaign_uses_mail_profile_reference(runtime_json)
if requires_migration and campaign_mail_profile_id(runtime_json) is None:
raise CampaignPersistenceError(
"Migrating legacy campaign mail settings requires an authorized server.mail_profile_id. "
"Select a Mail profile before saving."
)
mail_integration().assert_campaign_mail_policy_allows_json(
session,
tenant_id=tenant_id,
raw_json=runtime_json,
campaign_id=campaign.id,
)
return runtime_json
def _apply_version_field_updates(
version: CampaignVersion,
*,
current_flow: str | None,
current_step: str | None,
workflow_state: str | None,
is_complete: bool | None,
editor_state: dict[str, Any] | None,
source_filename: str | None,
source_base_path: str | None,
autosave: bool,
) -> None:
updates = (
("current_flow", current_flow),
("current_step", current_step),
("workflow_state", workflow_state),
("is_complete", is_complete),
("source_filename", source_filename),
("source_base_path", source_base_path),
)
for field_name, value in updates:
if value is not None:
setattr(version, field_name, value)
if editor_state is not None:
version.editor_state = validate_campaign_editor_state(editor_state)
if autosave:
version.autosaved_at = datetime.now(UTC)
def _invalidate_version_content(session: Session, *, campaign: Campaign, version: CampaignVersion) -> None:
version.validation_summary = None
version.build_summary = None
clear_execution_snapshot(version)
version.locked_at = None
version.locked_by_user_id = None
if version.workflow_state != CampaignVersionWorkflowState.EDITING.value:
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
campaign.status = CampaignStatus.DRAFT.value
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id).delete(synchronize_session=False)
def _persist_updated_version(
session: Session,
*,
campaign: Campaign,
version: CampaignVersion,
commit: bool,
) -> None:
session.add(version)
session.add(campaign)
session.flush()
if commit:
_write_campaign_snapshot(version)
session.commit()
else:
session.flush()
def update_campaign_version( def update_campaign_version(
session: Session, session: Session,
*, *,
@@ -546,18 +727,31 @@ def update_campaign_version(
source_base_path: str | None = None, source_base_path: str | None = None,
autosave: bool = False, autosave: bool = False,
migrate_legacy_mail_settings: bool = False, migrate_legacy_mail_settings: bool = False,
expected_revision: int | None = None,
commit: bool = True, commit: bool = True,
) -> CampaignVersion: ) -> CampaignVersion:
if raw_json is not None or source_filename is not None or source_base_path is not None: _assert_update_paths_safe(raw_json, source_filename=source_filename, source_base_path=source_base_path)
assert_server_safe_campaign_paths(
raw_json if raw_json is not None else {},
source_filename=source_filename,
source_base_path=source_base_path,
managed_files_available=files_integration().available,
)
version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id) version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
campaign = _require_campaign(session, campaign_id) campaign = _require_campaign(session, campaign_id)
ensure_current_working_version(campaign, version, action="edit") ensure_current_working_version(campaign, version, action="edit")
if (
expected_revision is not None
and version.edit_revision != int(expected_revision)
):
raise RevisionConflictError(
resource_type="campaign_version",
resource_id=version.id,
current_revision=version.edit_revision,
submitted_base_revision=int(expected_revision),
refresh_path=(
f"/api/v1/campaigns/{campaign.id}/versions/{version.id}"
),
current_etag=strong_resource_etag(
"campaign_version",
version.id,
version.edit_revision,
),
)
if is_version_locked(version): if is_version_locked(version):
raise LockedCampaignVersionError( raise LockedCampaignVersionError(
@@ -565,61 +759,70 @@ def update_campaign_version(
) )
if raw_json is not None: if raw_json is not None:
runtime_json = normalize_campaign_paths(raw_json, source_base_path) if source_base_path else copy.deepcopy(raw_json) runtime_json = _updated_runtime_json(
if campaign_mail_profile_boundary_violations(version.raw_json) and not migrate_legacy_mail_settings: session,
raise CampaignPersistenceError( tenant_id=tenant_id,
"This version contains legacy campaign-local SMTP/IMAP settings. Select an authorized Mail " campaign=campaign,
"profile on the Mail settings page and explicitly save the migration; the stored legacy version " version=version,
"will not be changed automatically." raw_json=raw_json,
) source_base_path=source_base_path,
assert_campaign_uses_mail_profile_reference(runtime_json) migrate_legacy_mail_settings=migrate_legacy_mail_settings,
if campaign_mail_profile_boundary_violations(version.raw_json) and campaign_mail_profile_id(runtime_json) is None: )
raise CampaignPersistenceError(
"Migrating legacy campaign mail settings requires an authorized server.mail_profile_id. "
"Select a Mail profile before saving."
)
mail_integration().assert_campaign_mail_policy_allows_json(session, tenant_id=tenant_id, raw_json=runtime_json, campaign_id=campaign.id)
version.raw_json = runtime_json version.raw_json = runtime_json
version.schema_version = str(runtime_json.get("version", version.schema_version or "1.0")) version.schema_version = str(runtime_json.get("version", version.schema_version or "1.0"))
_apply_campaign_metadata(campaign, runtime_json) _apply_campaign_metadata(campaign, runtime_json)
if current_flow is not None: _apply_version_field_updates(
version.current_flow = current_flow version,
if current_step is not None: current_flow=current_flow,
version.current_step = current_step current_step=current_step,
if workflow_state is not None: workflow_state=workflow_state,
version.workflow_state = workflow_state is_complete=is_complete,
if is_complete is not None: editor_state=editor_state,
version.is_complete = is_complete source_filename=source_filename,
if editor_state is not None: source_base_path=source_base_path,
version.editor_state = validate_campaign_editor_state(editor_state) autosave=autosave,
if source_filename is not None: )
version.source_filename = source_filename
if source_base_path is not None:
version.source_base_path = source_base_path
if autosave:
version.autosaved_at = datetime.now(UTC)
# Changes invalidate previous build and validation summaries. # Changes invalidate previous build and validation summaries.
if raw_json is not None: if raw_json is not None:
version.validation_summary = None _invalidate_version_content(session, campaign=campaign, version=version)
version.build_summary = None try:
clear_execution_snapshot(version) _persist_updated_version(
version.locked_at = None session,
version.locked_by_user_id = None campaign=campaign,
if version.workflow_state != CampaignVersionWorkflowState.EDITING.value: version=version,
version.workflow_state = CampaignVersionWorkflowState.EDITING.value commit=commit,
campaign.status = CampaignStatus.DRAFT.value )
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id).delete(synchronize_session=False) except StaleDataError as exc:
session.rollback()
session.add(version) current_revision = (
session.add(campaign) session.query(CampaignVersion.edit_revision)
session.flush() .filter(CampaignVersion.id == version_id)
if commit: .scalar()
_write_campaign_snapshot(version) )
session.commit() if current_revision is None:
else: raise CampaignPersistenceError(
session.flush() f"Campaign version not found: {version_id}"
) from exc
raise RevisionConflictError(
resource_type="campaign_version",
resource_id=version_id,
current_revision=int(current_revision),
submitted_base_revision=int(
expected_revision
if expected_revision is not None
else version.edit_revision
),
refresh_path=(
f"/api/v1/campaigns/{campaign_id}/versions/{version_id}"
),
current_etag=strong_resource_etag(
"campaign_version",
version_id,
int(current_revision),
),
) from exc
return version return version
@@ -631,6 +834,7 @@ def update_campaign_review_state(
version_id: str, version_id: str,
inspection_complete: bool, inspection_complete: bool,
reviewed_message_keys: list[str], reviewed_message_keys: list[str],
issue_decisions: list[dict[str, Any]] | None = None,
user_id: str | None, user_id: str | None,
commit: bool = True, commit: bool = True,
) -> CampaignVersion: ) -> CampaignVersion:
@@ -653,13 +857,22 @@ def update_campaign_review_state(
raise LockedCampaignVersionError("Delivery has started; message review state can no longer be changed.") raise LockedCampaignVersionError("Delivery has started; message review state can no longer be changed.")
build_token = _campaign_review_build_token(version) build_token = _campaign_review_build_token(version)
normalized_reviewed = list(dict.fromkeys(str(value) for value in reviewed_message_keys if str(value).strip())) normalized_reviewed = list(dict.fromkeys(str(value) for value in reviewed_message_keys if str(value).strip()))
normalized_decisions: list[dict[str, Any]] = []
if inspection_complete: if inspection_complete:
normalized_reviewed = _complete_campaign_review_keys(session, version, normalized_reviewed) normalized_reviewed, normalized_decisions = _complete_campaign_review(
session,
version,
normalized_reviewed,
issue_decisions or [],
user_id=user_id,
build_token=build_token,
)
_write_campaign_review_state( _write_campaign_review_state(
version, version,
build_token=build_token, build_token=build_token,
inspection_complete=inspection_complete, inspection_complete=inspection_complete,
reviewed_message_keys=normalized_reviewed, reviewed_message_keys=normalized_reviewed,
issue_decisions=normalized_decisions,
user_id=user_id, user_id=user_id,
) )
session.add(version) session.add(version)
@@ -684,11 +897,15 @@ def _campaign_review_build_token(version: CampaignVersion) -> str:
return build_token return build_token
def _complete_campaign_review_keys( def _complete_campaign_review(
session: Session, session: Session,
version: CampaignVersion, version: CampaignVersion,
reviewed_message_keys: list[str], reviewed_message_keys: list[str],
) -> list[str]: issue_decisions: list[dict[str, Any]],
*,
user_id: str | None,
build_token: str,
) -> tuple[list[str], list[dict[str, Any]]]:
jobs = ( jobs = (
session.query(CampaignJob) session.query(CampaignJob)
.filter(CampaignJob.campaign_version_id == version.id) .filter(CampaignJob.campaign_version_id == version.id)
@@ -703,7 +920,123 @@ def _complete_campaign_review_keys(
raise CampaignPersistenceError( raise CampaignPersistenceError(
"Messages requiring an explicit decision must be opened before review can be completed: " + ", ".join(missing) "Messages requiring an explicit decision must be opened before review can be completed: " + ", ".join(missing)
) )
return list(dict.fromkeys([*reviewed_message_keys, *_bulk_acceptable_review_keys(jobs)])) decisions = _normalize_review_issue_decisions(
jobs,
issue_decisions,
user_id=user_id,
build_token=build_token,
)
return (
list(
dict.fromkeys(
[*reviewed_message_keys, *_bulk_acceptable_review_keys(jobs)]
)
),
decisions,
)
def _normalize_review_issue_decisions(
jobs: list[CampaignJob],
requested: list[dict[str, Any]],
*,
user_id: str | None,
build_token: str,
decided_at: datetime | None = None,
) -> list[dict[str, Any]]:
jobs_by_id = {job.id: job for job in jobs}
requested_by_job: dict[str, dict[str, Any]] = {}
for item in requested:
job_id = str(item.get("job_id") or "").strip()
if not job_id or job_id not in jobs_by_id:
raise CampaignPersistenceError(
"A review decision references a message outside the current build."
)
if jobs_by_id[job_id].validation_status != "needs_review":
raise CampaignPersistenceError(
"Review decisions are accepted only for messages requiring review."
)
if job_id in requested_by_job:
raise CampaignPersistenceError(
"Only one review decision may be recorded per built message."
)
if str(item.get("decision") or "accept") != "accept":
raise CampaignPersistenceError(
"Unsupported campaign review decision."
)
requested_by_job[job_id] = item
timestamp = (decided_at or datetime.now(UTC)).isoformat()
normalized: list[dict[str, Any]] = []
for job in jobs:
reviewable_issues = [
issue
for issue in (job.issues_snapshot or [])
if isinstance(issue, dict)
and str(issue.get("behavior") or "").lower() == "ask"
]
attachment_overrides = [
issue
for issue in reviewable_issues
if str(issue.get("source") or "").startswith("attachments")
]
decision = requested_by_job.get(job.id)
if attachment_overrides and decision is None:
raise CampaignPersistenceError(
"Attachment exceptions require an explicit reason before review can be completed "
f"for message {job.entry_id or job.entry_index}."
)
if decision is None:
continue
reason = str(decision.get("reason") or "").strip()
if attachment_overrides and not reason:
raise CampaignPersistenceError(
"Attachment exception decisions require a reason."
)
evidence_issues = reviewable_issues or [
issue
for issue in (job.issues_snapshot or [])
if isinstance(issue, dict)
]
issue_payload = [
{
"code": str(issue.get("code") or ""),
"behavior": str(issue.get("behavior") or ""),
"source": str(issue.get("source") or ""),
"details": issue.get("details")
if isinstance(issue.get("details"), dict)
else {},
}
for issue in evidence_issues
]
fingerprint = hashlib.sha256(
json.dumps(
issue_payload,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
).hexdigest()
normalized.append(
{
"job_id": job.id,
"review_key": str(job.entry_id or job.entry_index),
"decision": "accept",
"reason": reason or None,
"actor_user_id": user_id,
"decided_at": timestamp,
"build_token": build_token,
"message_sha256": job.eml_sha256,
"issue_fingerprint": fingerprint,
"issue_codes": sorted(
{
str(issue.get("code") or "")
for issue in evidence_issues
if issue.get("code")
}
),
}
)
return normalized
def _required_review_keys(jobs: list[CampaignJob]) -> set[str]: def _required_review_keys(jobs: list[CampaignJob]) -> set[str]:
@@ -728,6 +1061,7 @@ def _write_campaign_review_state(
build_token: str, build_token: str,
inspection_complete: bool, inspection_complete: bool,
reviewed_message_keys: list[str], reviewed_message_keys: list[str],
issue_decisions: list[dict[str, Any]],
user_id: str | None, user_id: str | None,
) -> None: ) -> None:
editor_state = copy.deepcopy(version.editor_state or {}) editor_state = copy.deepcopy(version.editor_state or {})
@@ -735,6 +1069,7 @@ def _write_campaign_review_state(
"build_token": build_token, "build_token": build_token,
"inspection_complete": bool(inspection_complete), "inspection_complete": bool(inspection_complete),
"reviewed_message_keys": reviewed_message_keys, "reviewed_message_keys": reviewed_message_keys,
"issue_decisions": issue_decisions,
"updated_at": datetime.now(UTC).isoformat(), "updated_at": datetime.now(UTC).isoformat(),
"updated_by_user_id": user_id, "updated_by_user_id": user_id,
} }

View File

@@ -0,0 +1,109 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.tenancy.scope import Tenant
DEFAULT_SMALL_CELL_THRESHOLD = 5
MIN_SMALL_CELL_THRESHOLD = 2
MAX_SMALL_CELL_THRESHOLD = 100
SMALL_CELL_THRESHOLD_ENV = "GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD"
CAMPAIGN_REPORT_POLICY_SETTINGS_KEY = "campaign_report_privacy_policy"
SMALL_CELL_THRESHOLD_SETTINGS_KEY = "small_cell_threshold"
class CampaignReportPrivacyPolicyError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class CampaignReportPrivacyPolicy:
small_cell_threshold: int
source: str
deployment_small_cell_threshold: int
tenant_small_cell_threshold: int | None = None
def as_dict(self) -> dict[str, Any]:
return {
"small_cell_threshold": self.small_cell_threshold,
"source": self.source,
"deployment_small_cell_threshold": self.deployment_small_cell_threshold,
"tenant_small_cell_threshold": self.tenant_small_cell_threshold,
"deployment_setting": SMALL_CELL_THRESHOLD_ENV,
"tenant_setting": (
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
),
}
def effective_campaign_report_privacy_policy(
session: Session,
*,
tenant_id: str,
environ: Mapping[str, str] | None = None,
) -> CampaignReportPrivacyPolicy:
env = os.environ if environ is None else environ
deployment_raw = env.get(SMALL_CELL_THRESHOLD_ENV)
deployment_value = _configured_threshold(
deployment_raw,
source=SMALL_CELL_THRESHOLD_ENV,
default=DEFAULT_SMALL_CELL_THRESHOLD,
)
tenant = session.get(Tenant, tenant_id)
tenant_raw = _tenant_threshold_value(tenant.settings if tenant is not None else None)
if tenant_raw is None:
return CampaignReportPrivacyPolicy(
small_cell_threshold=deployment_value,
source="deployment" if deployment_raw not in (None, "") else "deployment_default",
deployment_small_cell_threshold=deployment_value,
)
tenant_value = _configured_threshold(
tenant_raw,
source=(
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
),
)
effective_value = max(deployment_value, tenant_value)
return CampaignReportPrivacyPolicy(
small_cell_threshold=effective_value,
source="tenant" if tenant_value >= deployment_value else "deployment_floor",
deployment_small_cell_threshold=deployment_value,
tenant_small_cell_threshold=tenant_value,
)
def _tenant_threshold_value(settings: Mapping[str, Any] | None) -> object | None:
if not isinstance(settings, Mapping):
return None
policy = settings.get(CAMPAIGN_REPORT_POLICY_SETTINGS_KEY)
if not isinstance(policy, Mapping):
return None
return policy.get(SMALL_CELL_THRESHOLD_SETTINGS_KEY)
def _configured_threshold(value: object, *, source: str, default: int | None = None) -> int:
if value is None or (isinstance(value, str) and not value.strip()):
if default is not None:
return default
raise CampaignReportPrivacyPolicyError(f"{source} must be configured as an integer")
if isinstance(value, bool):
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer, not a boolean")
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer") from exc
if str(parsed) != str(value).strip() and not isinstance(value, int):
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer")
if parsed < MIN_SMALL_CELL_THRESHOLD or parsed > MAX_SMALL_CELL_THRESHOLD:
raise CampaignReportPrivacyPolicyError(
f"{source} must be between {MIN_SMALL_CELL_THRESHOLD} and {MAX_SMALL_CELL_THRESHOLD}"
)
return parsed

View File

@@ -0,0 +1,544 @@
from __future__ import annotations
from collections import Counter
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Literal
from pydantic import BaseModel, ConfigDict
from sqlalchemy import case, func, or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
from govoplan_campaign.backend.report_privacy_policy import (
CampaignReportPrivacyPolicy,
effective_campaign_report_privacy_policy,
)
class AggregateCampaignReportError(RuntimeError):
pass
class AggregateReportCampaign(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
name: str
status: str
class AggregateReportCampaignListItem(AggregateReportCampaign):
updated_at: datetime
class AggregateReportCampaignList(BaseModel):
model_config = ConfigDict(extra="forbid")
campaigns: list[AggregateReportCampaignListItem]
class AggregateCount(BaseModel):
model_config = ConfigDict(extra="forbid")
value: int | None
suppressed: bool = False
class AggregatePopulation(BaseModel):
model_config = ConfigDict(extra="forbid")
denominator: AggregateCount
denominator_definition: str
inactive_source_entries: AggregateCount
excluded_or_blocked_jobs: AggregateCount
class AggregateOutcomeCounts(BaseModel):
model_config = ConfigDict(extra="forbid")
smtp_accepted: AggregateCount
postbox_accepted: AggregateCount
delivered: AggregateCount
partially_accepted: AggregateCount
failed: AggregateCount
outcome_unknown: AggregateCount
queued_or_active: AggregateCount
cancelled: AggregateCount
excluded: AggregateCount
not_attempted: AggregateCount
class AggregateTimeRange(BaseModel):
model_config = ConfigDict(extra="forbid")
first_activity_at: datetime | None
last_activity_at: datetime | None
suppressed: bool
class AggregatePrivacy(BaseModel):
model_config = ConfigDict(extra="forbid")
small_cell_threshold: int
suppression_applied: bool
rule: str
class AggregateCampaignReport(BaseModel):
model_config = ConfigDict(extra="forbid")
generated_at: datetime
campaign: AggregateReportCampaign
version_number: int | None
completion_state: Literal[
"not_started",
"in_progress",
"completed",
"partially_completed",
"incomplete",
"outcome_unknown",
"suppressed",
]
population: AggregatePopulation
outcomes: AggregateOutcomeCounts
time_range: AggregateTimeRange
privacy: AggregatePrivacy
_OUTCOME_KEYS = (
"smtp_accepted",
"postbox_accepted",
"delivered",
"partially_accepted",
"failed",
"outcome_unknown",
"queued_or_active",
"cancelled",
"excluded",
"not_attempted",
)
def generate_aggregate_campaign_report(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str | None = None,
) -> AggregateCampaignReport:
"""Build the deliberately small, recipient-free Campaign report projection."""
campaign = _get_campaign(session, tenant_id=tenant_id, campaign_id=campaign_id)
version = _selected_version(session, campaign, version_id)
facts = _query_aggregate_facts(
session,
tenant_id=tenant_id,
campaign_id=campaign.id,
version=version,
)
policy = effective_campaign_report_privacy_policy(session, tenant_id=tenant_id)
return _build_aggregate_campaign_report(
campaign=campaign,
version=version,
facts=facts,
policy=policy,
)
def _get_campaign(session: Session, *, tenant_id: str, campaign_id: str) -> Campaign:
campaign = (
session.query(Campaign)
.filter(Campaign.tenant_id == tenant_id, Campaign.id == campaign_id)
.one_or_none()
)
if campaign is None:
raise AggregateCampaignReportError("Campaign not found")
return campaign
def _selected_version(
session: Session,
campaign: Campaign,
version_id: str | None,
) -> CampaignVersion | None:
selected_id = version_id or campaign.current_version_id
if not selected_id:
return None
version = session.get(CampaignVersion, selected_id)
if version is None or version.campaign_id != campaign.id:
raise AggregateCampaignReportError("Campaign version not found")
return version
def _query_aggregate_facts(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version: CampaignVersion | None,
) -> _AggregateFacts:
if version is None:
return _AggregateFacts.empty()
smtp_accepted = CampaignJob.send_status.in_(
{"smtp_accepted", "sent"}
)
accepted = CampaignJob.send_status.in_(
{
"smtp_accepted",
"postbox_accepted",
"delivered",
"partially_accepted",
"sent",
}
)
failed = CampaignJob.send_status.in_({"failed_temporary", "failed_permanent"})
unknown = CampaignJob.send_status == "outcome_unknown"
active = CampaignJob.send_status.in_({"queued", "claimed", "sending"})
cancelled = CampaignJob.send_status == "cancelled"
excluded = CampaignJob.send_status == "skipped"
excluded_or_blocked = or_(
CampaignJob.validation_status.in_({"blocked", "excluded", "inactive"}),
CampaignJob.build_status != "built",
)
row = (
session.query(
func.count(CampaignJob.id).label("denominator"),
func.sum(case((smtp_accepted, 1), else_=0)).label(
"smtp_accepted"
),
func.sum(
case(
(
CampaignJob.send_status == "postbox_accepted",
1,
),
else_=0,
)
).label("postbox_accepted"),
func.sum(
case(
(CampaignJob.send_status == "delivered", 1),
else_=0,
)
).label("delivered"),
func.sum(
case(
(
CampaignJob.send_status == "partially_accepted",
1,
),
else_=0,
)
).label("partially_accepted"),
func.sum(case((failed, 1), else_=0)).label("failed"),
func.sum(case((unknown, 1), else_=0)).label("outcome_unknown"),
func.sum(case((active, 1), else_=0)).label("queued_or_active"),
func.sum(case((cancelled, 1), else_=0)).label("cancelled"),
func.sum(case((excluded, 1), else_=0)).label("excluded"),
func.sum(
case((or_(accepted, failed, unknown, active, cancelled, excluded), 0), else_=1)
).label("not_attempted"),
func.sum(case((excluded_or_blocked, 1), else_=0)).label("excluded_or_blocked"),
func.min(CampaignJob.queued_at).label("queued_min"),
func.max(CampaignJob.queued_at).label("queued_max"),
func.min(CampaignJob.smtp_started_at).label("smtp_min"),
func.max(CampaignJob.smtp_started_at).label("smtp_max"),
func.min(CampaignJob.sent_at).label("sent_min"),
func.max(CampaignJob.sent_at).label("sent_max"),
func.min(CampaignJob.outcome_unknown_at).label("unknown_min"),
func.max(CampaignJob.outcome_unknown_at).label("unknown_max"),
)
.filter(
CampaignJob.tenant_id == tenant_id,
CampaignJob.campaign_id == campaign_id,
CampaignJob.campaign_version_id == version.id,
)
.one()
)
values = row._mapping
activity = [
values[key]
for key in (
"queued_min",
"queued_max",
"smtp_min",
"smtp_max",
"sent_min",
"sent_max",
"unknown_min",
"unknown_max",
)
if values[key] is not None
]
return _AggregateFacts(
outcomes={key: int(values[key] or 0) for key in _OUTCOME_KEYS},
denominator=int(values["denominator"] or 0),
excluded_or_blocked=int(values["excluded_or_blocked"] or 0),
first_activity_at=min(activity) if activity else None,
last_activity_at=max(activity) if activity else None,
)
def build_aggregate_campaign_report(
*,
campaign: Campaign,
version: CampaignVersion | None,
jobs: list[CampaignJob],
policy: CampaignReportPrivacyPolicy,
generated_at: datetime | None = None,
) -> AggregateCampaignReport:
return _build_aggregate_campaign_report(
campaign=campaign,
version=version,
facts=_facts_from_jobs(jobs),
policy=policy,
generated_at=generated_at,
)
def _build_aggregate_campaign_report(
*,
campaign: Campaign,
version: CampaignVersion | None,
facts: _AggregateFacts,
policy: CampaignReportPrivacyPolicy,
generated_at: datetime | None = None,
) -> AggregateCampaignReport:
outcome_values = facts.outcomes
outcomes, denominator = _suppress_partition(
outcome_values,
threshold=policy.small_cell_threshold,
)
outcome_suppression_applied = denominator.suppressed or any(
item.suppressed for item in outcomes.values()
)
inactive_entries = _inactive_entry_count(version)
standalone_counts = {
"inactive_source_entries": _suppress_standalone_count(
inactive_entries,
threshold=policy.small_cell_threshold,
),
# This population count overlaps the outcome partition, so exposing it
# can make a suppressed outcome recoverable through subtraction.
"excluded_or_blocked_jobs": (
AggregateCount(value=None, suppressed=True)
if outcome_suppression_applied
else _suppress_standalone_count(
facts.excluded_or_blocked,
threshold=policy.small_cell_threshold,
)
),
}
suppression_applied = denominator.suppressed or any(
item.suppressed for item in (*outcomes.values(), *standalone_counts.values())
)
first_activity = facts.first_activity_at
last_activity = facts.last_activity_at
suppress_time_range = suppression_applied or (
0 < facts.denominator < policy.small_cell_threshold
)
return AggregateCampaignReport(
generated_at=generated_at or datetime.now(timezone.utc),
campaign=AggregateReportCampaign(
id=campaign.id,
name=campaign.name,
status=campaign.status,
),
version_number=version.version_number if version else None,
completion_state=(
"suppressed"
if denominator.suppressed
else _completion_state(outcome_values, facts.denominator)
),
population=AggregatePopulation(
denominator=denominator,
denominator_definition=(
"All persisted recipient delivery jobs for the selected campaign version, "
"including excluded or blocked jobs. Inactive source entries without a job "
"record are excluded and reported separately."
),
inactive_source_entries=standalone_counts["inactive_source_entries"],
excluded_or_blocked_jobs=standalone_counts["excluded_or_blocked_jobs"],
),
outcomes=AggregateOutcomeCounts(**outcomes),
time_range=AggregateTimeRange(
first_activity_at=None if suppress_time_range else first_activity,
last_activity_at=None if suppress_time_range else last_activity,
suppressed=suppress_time_range and first_activity is not None,
),
privacy=AggregatePrivacy(
small_cell_threshold=policy.small_cell_threshold,
suppression_applied=suppression_applied,
rule=(
"Positive counts below the threshold are hidden. At least one additional "
"count or the denominator is hidden when needed to prevent subtraction. "
"Overlapping population counts are hidden whenever outcome suppression applies."
),
),
)
def aggregate_report_campaign_item(campaign: Campaign) -> AggregateReportCampaignListItem:
return AggregateReportCampaignListItem(
id=campaign.id,
name=campaign.name,
status=campaign.status,
updated_at=campaign.updated_at,
)
def _outcome_counts(jobs: list[CampaignJob]) -> dict[str, int]:
counts: Counter[str] = Counter()
for job in jobs:
status = job.send_status
if status in {"smtp_accepted", "sent"}:
counts["smtp_accepted"] += 1
elif status == "postbox_accepted":
counts["postbox_accepted"] += 1
elif status == "delivered":
counts["delivered"] += 1
elif status == "partially_accepted":
counts["partially_accepted"] += 1
elif status in {"failed_temporary", "failed_permanent"}:
counts["failed"] += 1
elif status == "outcome_unknown":
counts["outcome_unknown"] += 1
elif status in {"queued", "claimed", "sending"}:
counts["queued_or_active"] += 1
elif status == "cancelled":
counts["cancelled"] += 1
elif status == "skipped":
counts["excluded"] += 1
else:
counts["not_attempted"] += 1
return {key: counts[key] for key in _OUTCOME_KEYS}
@dataclass(frozen=True, slots=True)
class _AggregateFacts:
outcomes: dict[str, int]
denominator: int
excluded_or_blocked: int
first_activity_at: datetime | None
last_activity_at: datetime | None
@classmethod
def empty(cls) -> _AggregateFacts:
return cls(
outcomes={key: 0 for key in _OUTCOME_KEYS},
denominator=0,
excluded_or_blocked=0,
first_activity_at=None,
last_activity_at=None,
)
def _facts_from_jobs(jobs: list[CampaignJob]) -> _AggregateFacts:
first_activity, last_activity = _activity_range(jobs)
return _AggregateFacts(
outcomes=_outcome_counts(jobs),
denominator=len(jobs),
excluded_or_blocked=sum(
1
for job in jobs
if job.validation_status in {"blocked", "excluded", "inactive"}
or job.build_status != "built"
),
first_activity_at=first_activity,
last_activity_at=last_activity,
)
def _suppress_partition(
counts: dict[str, int],
*,
threshold: int,
) -> tuple[dict[str, AggregateCount], AggregateCount]:
total = sum(counts.values())
suppressed = {key for key, value in counts.items() if 0 < value < threshold}
suppress_denominator = False
if suppressed:
companions = [
(value, key)
for key, value in counts.items()
if key not in suppressed and value > 0
]
if companions:
suppressed.add(max(companions)[1])
else:
suppress_denominator = True
denominator = AggregateCount(
value=None if suppress_denominator else total,
suppressed=suppress_denominator,
)
return (
{
key: AggregateCount(
value=None if key in suppressed else value,
suppressed=key in suppressed,
)
for key, value in counts.items()
},
denominator,
)
def _suppress_standalone_count(value: int, *, threshold: int) -> AggregateCount:
if 0 < value < threshold:
return AggregateCount(value=None, suppressed=True)
return AggregateCount(value=value, suppressed=False)
def _completion_state(
counts: dict[str, int],
total: int,
) -> Literal[
"not_started",
"in_progress",
"completed",
"partially_completed",
"incomplete",
"outcome_unknown",
]:
if total == 0 or counts["not_attempted"] + counts["excluded"] == total:
return "not_started"
if counts["outcome_unknown"]:
return "outcome_unknown"
if counts["queued_or_active"]:
return "in_progress"
fully_accepted = (
counts["smtp_accepted"]
+ counts["postbox_accepted"]
+ counts["delivered"]
)
partially_accepted = counts["partially_accepted"]
if fully_accepted == total:
return "completed"
if fully_accepted or partially_accepted:
return "partially_completed"
return "incomplete"
def _activity_range(jobs: list[CampaignJob]) -> tuple[datetime | None, datetime | None]:
activity = [
value
for job in jobs
for value in (
job.queued_at,
job.smtp_started_at,
job.sent_at,
job.outcome_unknown_at,
)
if value is not None
]
if not activity:
return None, None
return min(activity), max(activity)
def _inactive_entry_count(version: CampaignVersion | None) -> int:
build_summary = version.build_summary if version and isinstance(version.build_summary, dict) else {}
return int(build_summary.get("inactive_count") or build_summary.get("inactive_entries_count") or 0)

File diff suppressed because it is too large Load Diff

View File

@@ -13,7 +13,11 @@ from govoplan_campaign.backend.campaign.models import CampaignConfig
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_id from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_id
from govoplan_campaign.backend.persistence.campaigns import load_version_config from govoplan_campaign.backend.persistence.campaigns import load_version_config
from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
from govoplan_campaign.backend.integrations import SmtpConfigurationError from govoplan_campaign.backend.integrations import (
MailDeliveryCommandError,
SmtpConfigurationError,
mail_integration,
)
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, ensure_execution_snapshot from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, ensure_execution_snapshot
@@ -32,6 +36,9 @@ class CampaignReportEmailResult:
attached_jobs_csv: bool attached_jobs_csv: bool
attached_report_json: bool attached_report_json: bool
accepted_count: int | None = None accepted_count: int | None = None
command_id: str | None = None
delivery_status: str | None = None
duplicate: bool = False
def as_dict(self) -> dict[str, Any]: def as_dict(self) -> dict[str, Any]:
return { return {
@@ -44,6 +51,22 @@ class CampaignReportEmailResult:
"attached_jobs_csv": self.attached_jobs_csv, "attached_jobs_csv": self.attached_jobs_csv,
"attached_report_json": self.attached_report_json, "attached_report_json": self.attached_report_json,
"accepted_count": self.accepted_count, "accepted_count": self.accepted_count,
"command_id": self.command_id,
"delivery_status": self.delivery_status,
"duplicate": self.duplicate,
}
def audit_dict(self) -> dict[str, Any]:
return {
"campaign_id": self.campaign_id,
"version_id": self.version_id,
"recipient_count": len(self.to),
"dry_run": self.dry_run,
"attached_jobs_csv": self.attached_jobs_csv,
"attached_report_json": self.attached_report_json,
"command_id": self.command_id,
"delivery_status": self.delivery_status,
"duplicate": self.duplicate,
} }
@@ -88,8 +111,10 @@ def _text_summary(report: dict[str, Any]) -> str:
f"- Needs attention: {cards['needs_attention']}", f"- Needs attention: {cards['needs_attention']}",
f"- Sent: {cards['sent']}", f"- Sent: {cards['sent']}",
f"- Failed: {cards['failed']}", f"- Failed: {cards['failed']}",
f"- SMTP skipped (excluded): {cards.get('skipped', status.get('send', {}).get('skipped', 0))}",
f"- IMAP appended: {cards['imap_appended']}", f"- IMAP appended: {cards['imap_appended']}",
f"- IMAP failed: {cards['imap_failed']}", f"- IMAP failed: {cards['imap_failed']}",
f"- IMAP skipped: {cards.get('imap_skipped', status.get('imap', {}).get('skipped', 0))}",
"", "",
f"Build status: {status.get('build', {})}", f"Build status: {status.get('build', {})}",
f"Validation status: {status.get('validation', {})}", f"Validation status: {status.get('validation', {})}",
@@ -147,6 +172,8 @@ def send_campaign_report_email(
attach_jobs_csv: bool = False, attach_jobs_csv: bool = False,
attach_report_json: bool = False, attach_report_json: bool = False,
dry_run: bool = False, dry_run: bool = False,
idempotency_key: str | None = None,
created_by_user_id: str | None = None,
) -> CampaignReportEmailResult: ) -> CampaignReportEmailResult:
campaign = session.get(Campaign, campaign_id) campaign = session.get(Campaign, campaign_id)
if not campaign or campaign.tenant_id != tenant_id: if not campaign or campaign.tenant_id != tenant_id:
@@ -173,11 +200,16 @@ def send_campaign_report_email(
) from exc ) from exc
if not snapshot.smtp_transport_revision: if not snapshot.smtp_transport_revision:
raise CampaignReportEmailError("Campaign build evidence has no SMTP transport revision") raise CampaignReportEmailError("Campaign build evidence has no SMTP transport revision")
mail = mail_integration()
if not dry_run: if not dry_run:
raise CampaignReportEmailError( if not mail.durable_delivery_available:
"Report email delivery is disabled until it uses a durable, idempotent Mail-owned outbox with unknown-outcome reconciliation." raise CampaignReportEmailError(
) "Report email delivery requires the durable, idempotent Mail-owned outbox."
)
if not str(idempotency_key or "").strip():
raise CampaignReportEmailError(
"Live report email delivery requires an idempotency key"
)
report = generate_campaign_report( report = generate_campaign_report(
session, session,
tenant_id=tenant_id, tenant_id=tenant_id,
@@ -200,6 +232,48 @@ def send_campaign_report_email(
jobs_csv=jobs_csv, jobs_csv=jobs_csv,
report_json=report_json, report_json=report_json,
) )
if not dry_run:
clean_idempotency_key = str(idempotency_key or "").strip()
from_email, _from_name = _effective_from(config)
if not snapshot.mail_profile_id:
raise CampaignReportEmailError(
"Campaign build evidence has no Mail profile reference"
)
try:
command = mail.submit_delivery_command(
session,
tenant_id=tenant_id,
command_type="campaign_report",
source_module="campaigns",
source_resource_type="campaign",
source_resource_id=campaign.id,
source_version_id=version.id,
idempotency_key=clean_idempotency_key,
profile_id=snapshot.mail_profile_id,
message_bytes=message.as_bytes(),
envelope_from=from_email,
envelope_recipients=to,
from_header=str(message["From"]),
expected_smtp_transport_revision=snapshot.smtp_transport_revision,
smtp_server_id=snapshot.smtp_server_id,
smtp_credential_id=snapshot.smtp_credential_id,
created_by_user_id=created_by_user_id,
)
except MailDeliveryCommandError as exc:
raise CampaignReportEmailError(str(exc)) from exc
return CampaignReportEmailResult(
campaign_id=campaign.id,
version_id=version.id,
to=to,
subject=str(message["Subject"]),
dry_run=False,
sent=False,
attached_jobs_csv=jobs_csv is not None,
attached_report_json=report_json is not None,
command_id=str(command["id"]),
delivery_status=str(command["status"]),
duplicate=bool(command.get("duplicate")),
)
return CampaignReportEmailResult( return CampaignReportEmailResult(
campaign_id=campaign.id, campaign_id=campaign.id,
version_id=version.id, version_id=version.id,

View File

@@ -40,6 +40,51 @@ CAMPAIGN_DIAGNOSTIC_RESPONSE_KEYS = frozenset(
} }
) )
_SEND_NOW_RESULT_KEYS = (
"campaign_id",
"version_id",
"attempted_count",
"sent_count",
"failed_count",
"outcome_unknown_count",
"skipped_count",
"preflight_count",
"delivery_mode",
"dry_run",
)
_SEND_NOW_JOB_RESULT_KEYS = (
"campaign_id",
"version_id",
"job_id",
"status",
"attempt_number",
"dry_run",
"queued_count",
"skipped_count",
"blocked_count",
"enqueued_count",
"delivery_mode",
"worker_queue_available",
)
_SYNCHRONOUS_POLICY_KEYS = (
"max_recipient_jobs",
"source",
"deployment_max_recipient_jobs",
"tenant_max_recipient_jobs",
)
_VALIDATION_SUMMARY_KEYS = ("ok", "error_count", "warning_count")
_BUILD_SUMMARY_KEYS = (
"built_count",
"build_failed_count",
"ready_count",
"warning_count",
"needs_review_count",
"blocked_count",
"excluded_count",
"inactive_count",
"queueable_count",
)
def public_campaign_payload(value: Any, *, include_diagnostics: bool = False) -> Any: def public_campaign_payload(value: Any, *, include_diagnostics: bool = False) -> Any:
"""Return a detached payload without infrastructure-only locators.""" """Return a detached payload without infrastructure-only locators."""
@@ -65,6 +110,7 @@ def public_delivery_result_message(
last_error: Any, last_error: Any,
send_status: Any, send_status: Any,
imap_status: Any, imap_status: Any,
postbox_status: Any = None,
) -> str | None: ) -> str | None:
"""Map persisted provider text to a stable business-safe explanation.""" """Map persisted provider text to a stable business-safe explanation."""
@@ -72,10 +118,22 @@ def public_delivery_result_message(
return None return None
clean_send_status = str(send_status or "") clean_send_status = str(send_status or "")
clean_imap_status = str(imap_status or "") clean_imap_status = str(imap_status or "")
clean_postbox_status = str(postbox_status or "")
if clean_postbox_status == "outcome_unknown":
return "Postbox delivery outcome requires operator reconciliation."
if clean_send_status == "outcome_unknown": if clean_send_status == "outcome_unknown":
return "SMTP delivery outcome requires operator reconciliation." return "Delivery outcome requires operator reconciliation."
if clean_postbox_status in {
"rejected_temporary",
"rejected_permanent",
}:
return "Postbox delivery was rejected; an operator can inspect restricted diagnostics."
if clean_postbox_status == "partially_accepted":
return "Some Postbox targets accepted the message and others rejected it."
if clean_send_status in {"failed_temporary", "failed_permanent"}: if clean_send_status in {"failed_temporary", "failed_permanent"}:
return "SMTP delivery failed; an operator can inspect restricted diagnostics." if clean_postbox_status in {"", "not_requested"}:
return "SMTP delivery failed; an operator can inspect restricted diagnostics."
return "Delivery failed; an operator can inspect restricted diagnostics."
if clean_imap_status in {"outcome_unknown", "appending"}: if clean_imap_status in {"outcome_unknown", "appending"}:
return "Sent-folder append outcome requires operator reconciliation." return "Sent-folder append outcome requires operator reconciliation."
if clean_imap_status in {"failed", "skipped"}: if clean_imap_status in {"failed", "skipped"}:
@@ -83,6 +141,59 @@ def public_delivery_result_message(
return "Delivery recorded a warning; an operator can inspect restricted diagnostics." return "Delivery recorded a warning; an operator can inspect restricted diagnostics."
def public_send_campaign_now_result(
value: dict[str, Any],
*,
validation_summary: dict[str, Any],
build_summary: dict[str, Any],
) -> dict[str, Any]:
"""Project synchronous delivery into its recipient-authorized public contract.
Per-job provider messages are deliberately omitted. They can contain SMTP
diagnostics or refused envelope addresses and belong only in restricted
diagnostics backed by persisted job state.
"""
result = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
policy = value.get("synchronous_send_policy")
result["synchronous_send_policy"] = _selected_payload(
policy if isinstance(policy, dict) else {},
_SYNCHRONOUS_POLICY_KEYS,
)
rows = value.get("results")
if isinstance(rows, list):
result["results"] = [
_selected_payload(row, _SEND_NOW_JOB_RESULT_KEYS)
for row in rows
if isinstance(row, dict)
]
else:
result["results"] = []
result["validation"] = _selected_payload(validation_summary, _VALIDATION_SUMMARY_KEYS)
result["build"] = _selected_payload(build_summary, _BUILD_SUMMARY_KEYS)
return result
def send_campaign_now_audit_details(value: dict[str, Any]) -> dict[str, Any]:
"""Return aggregate-only evidence for a synchronous Campaign send audit."""
details = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
policy = value.get("synchronous_send_policy")
details["synchronous_send_policy"] = _selected_payload(
policy if isinstance(policy, dict) else {},
_SYNCHRONOUS_POLICY_KEYS,
)
return details
def _selected_payload(value: dict[str, Any], keys: tuple[str, ...]) -> dict[str, Any]:
return {
key: copy.deepcopy(value[key])
for key in keys
if key in value
}
def public_campaign_configuration(value: Any) -> Any: def public_campaign_configuration(value: Any) -> Any:
"""Return campaign JSON without infrastructure locators or mail secrets. """Return campaign JSON without infrastructure locators or mail secrets.

View File

@@ -0,0 +1,675 @@
from __future__ import annotations
import copy
import dataclasses
from collections.abc import Callable
from typing import Any
from fastapi import HTTPException, status
from sqlalchemy import and_, exists, or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
CAMPAIGN_MAIL_SERVER_KEYS,
campaign_mail_profile_id,
)
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignIssue,
CampaignJob,
CampaignShare,
CampaignStatus,
CampaignVersion,
CampaignVersionWorkflowState,
RecipientImportMappingProfile,
)
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
from govoplan_campaign.backend.persistence.campaigns import CampaignPersistenceError
from govoplan_campaign.backend.persistence.versions import (
LockedCampaignVersionError,
is_user_locked_version,
is_version_final_locked,
is_version_locked,
update_campaign_version,
)
from govoplan_campaign.backend.schemas import (
CampaignVersionDetailResponse,
CampaignVersionUpdateRequest,
RecipientImportMappingProfilePayload,
)
from govoplan_campaign.backend.sending.execution import (
clear_execution_snapshot,
)
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.auth import ApiPrincipal, has_scope
from govoplan_core.core.access import CAPABILITY_ACCESS_DIRECTORY, AccessDirectory
from govoplan_core.core.concurrency import (
ConcurrencyError,
MissingPreconditionError,
RevisionConflictError,
assert_revision_precondition,
)
from govoplan_core.core.runtime import get_registry
def _capability_payload(value: object) -> dict[str, Any]:
if dataclasses.is_dataclass(value):
return dataclasses.asdict(value)
if isinstance(value, dict):
return dict(value)
payload: dict[str, Any] = {}
for key in (
"contact_id",
"address_book_id",
"display_name",
"email",
"email_label",
"organization",
"role_title",
"tags",
"source_kind",
"source_ref",
"source_revision",
"source_id",
"source_label",
"recipient_count",
"generated_at",
"recipients",
"fields",
"provenance",
):
if hasattr(value, key):
payload[key] = getattr(value, key)
return payload
def _registry_capability(name: str) -> object | None:
registry = get_registry()
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(name)
):
return None
return registry.capability(name)
def _access_directory() -> AccessDirectory:
registry = get_registry()
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_ACCESS_DIRECTORY)
):
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Access directory capability is not configured",
)
capability = registry.require_capability(CAPABILITY_ACCESS_DIRECTORY)
if not isinstance(capability, AccessDirectory):
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Access directory capability is invalid",
)
return capability
def _get_campaign_for_tenant(
session: Session, campaign_id: str, tenant_id: str
) -> Campaign:
campaign = session.get(Campaign, campaign_id)
if not campaign or campaign.tenant_id != tenant_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign not found"
)
return campaign
def _get_version_for_tenant(
session: Session, version_id: str, tenant_id: str
) -> CampaignVersion:
version = session.get(CampaignVersion, version_id)
if not version:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
campaign = session.get(Campaign, version.campaign_id)
if not campaign or campaign.tenant_id != tenant_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
return version
def _principal_group_ids(session: Session, principal: ApiPrincipal) -> set[str]:
del session
return {
group.id
for group in _access_directory().groups_for_user(
principal.user.id, tenant_id=principal.tenant_id
)
}
def _campaign_acl_filter(session: Session, principal: ApiPrincipal):
if has_scope(principal, "tenant:*"):
return None
group_ids = _principal_group_ids(session, principal)
clauses = [Campaign.owner_user_id == principal.user.id]
if group_ids:
clauses.append(Campaign.owner_group_id.in_(group_ids))
share_clauses = [
and_(
CampaignShare.tenant_id == Campaign.tenant_id,
CampaignShare.campaign_id == Campaign.id,
CampaignShare.revoked_at.is_(None),
CampaignShare.target_type == "user",
CampaignShare.target_id == principal.user.id,
)
]
if group_ids:
share_clauses.append(
and_(
CampaignShare.tenant_id == Campaign.tenant_id,
CampaignShare.campaign_id == Campaign.id,
CampaignShare.revoked_at.is_(None),
CampaignShare.target_type == "group",
CampaignShare.target_id.in_(group_ids),
)
)
clauses.append(exists().where(or_(*share_clauses)))
return or_(*clauses)
def _campaign_acl_allows(
session: Session,
campaign: Campaign,
principal: ApiPrincipal,
*,
write: bool = False,
) -> bool:
if has_scope(principal, "tenant:*"):
return True
if campaign.owner_user_id == principal.user.id:
return True
group_ids = _principal_group_ids(session, principal)
if campaign.owner_group_id and campaign.owner_group_id in group_ids:
return True
target_ids = [principal.user.id, *group_ids]
if not target_ids:
return False
query = session.query(CampaignShare).filter(
CampaignShare.tenant_id == campaign.tenant_id,
CampaignShare.campaign_id == campaign.id,
CampaignShare.revoked_at.is_(None),
or_(
CampaignShare.target_type == "user",
CampaignShare.target_type == "group",
),
CampaignShare.target_id.in_(target_ids),
)
shares = query.all()
if not shares:
return False
if not write:
return True
return any(item.permission == "write" for item in shares)
def _require_campaign_acl(
session: Session,
campaign: Campaign,
principal: ApiPrincipal,
*,
write: bool = False,
) -> None:
if not _campaign_acl_allows(session, campaign, principal, write=write):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Campaign is not shared with this principal",
)
def _get_campaign_for_principal(
session: Session, campaign_id: str, principal: ApiPrincipal, *, write: bool = False
) -> Campaign:
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
_require_campaign_acl(session, campaign, principal, write=write)
return campaign
def _require_permission(principal: ApiPrincipal, scope: str) -> None:
if not has_scope(principal, scope):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {scope}"
)
def _campaign_query_for_principal(session: Session, principal: ApiPrincipal):
query = session.query(Campaign).filter(
Campaign.tenant_id == principal.tenant_id, Campaign.status != "deleted"
)
acl_filter = _campaign_acl_filter(session, principal)
if acl_filter is not None:
query = query.filter(acl_filter)
return query
def _get_recipient_import_profile_for_principal(
session: Session, profile_id: str, principal: ApiPrincipal
) -> RecipientImportMappingProfile:
profile = session.get(RecipientImportMappingProfile, profile_id)
if (
not profile
or profile.tenant_id != principal.tenant_id
or profile.owner_user_id != principal.user.id
):
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Recipient import mapping profile not found",
)
return profile
def _apply_recipient_import_profile_payload(
profile: RecipientImportMappingProfile,
payload: RecipientImportMappingProfilePayload,
) -> None:
profile.name = payload.name.strip()
profile.column_count = payload.column_count
profile.headers = list(payload.headers)
profile.normalized_headers = list(payload.normalized_headers)
profile.ordered_header_fingerprint = payload.ordered_header_fingerprint
profile.unordered_header_fingerprint = payload.unordered_header_fingerprint
profile.delimiter = payload.delimiter
profile.header_rows = payload.header_rows
profile.quoted = payload.quoted
profile.value_separators = payload.value_separators
profile.mappings = [mapping.model_dump(mode="json") for mapping in payload.mappings]
def _recipient_sections_changed(
current: dict[str, object] | None, proposed: dict[str, object] | None
) -> bool:
if proposed is None:
return False
current = current or {}
return any(
current.get(key) != proposed.get(key) for key in ("recipients", "entries")
)
def _campaign_mail_profile_id(raw_json: dict[str, object] | None) -> str | None:
return campaign_mail_profile_id(raw_json)
def _require_mail_profile_use_if_needed(
principal: ApiPrincipal, raw_json: dict[str, object] | None
) -> None:
if _campaign_mail_profile_id(raw_json) and not has_scope(
principal, "mail:profile:use"
):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Missing scope: mail:profile:use",
)
def _campaign_response_context(principal: ApiPrincipal) -> dict[str, bool]:
return {"include_diagnostics": has_scope(principal, "campaigns:diagnostic:read")}
def _campaign_version_detail_response(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
mutation: Callable[[], CampaignVersion],
*,
audit_action: str,
details: dict[str, Any] | Callable[[CampaignVersion], dict[str, Any]] | None = None,
validation_error_status: int | None = None,
) -> CampaignVersionDetailResponse:
try:
version = mutation()
audit_details = (
details(version)
if callable(details)
else dict(details or {"campaign_id": campaign_id})
)
audit_from_principal(
session,
principal,
action=audit_action,
object_type="campaign_version",
object_id=version.id,
details=audit_details,
commit=True,
)
_write_current_version_snapshot_if_available(version)
return CampaignVersionDetailResponse.model_validate(
version,
context=_campaign_response_context(principal),
)
except LockedCampaignVersionError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
) from exc
except CampaignPathSecurityError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except CampaignPersistenceError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except RevisionConflictError:
session.rollback()
raise
except Exception as exc:
session.rollback()
if validation_error_status is None:
raise
raise HTTPException(
status_code=validation_error_status, detail=str(exc)
) from exc
def _update_campaign_version_detail_response(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_id: str,
payload: CampaignVersionUpdateRequest,
*,
if_match: str | None,
autosave: bool,
audit_action: str,
) -> CampaignVersionDetailResponse:
_get_campaign_for_principal(session, campaign_id, principal, write=True)
current_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if payload.base_revision is None:
error = MissingPreconditionError(
resource_type="campaign_version",
resource_id=version_id,
)
raise HTTPException(
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
detail=error.as_dict(),
)
try:
assert_revision_precondition(
if_match,
resource_type="campaign_version",
resource_id=version_id,
submitted_base_revision=payload.base_revision,
)
except MissingPreconditionError as exc:
raise HTTPException(
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
detail=exc.as_dict(),
) from exc
except ConcurrencyError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"code": "invalid_precondition",
"message": str(exc),
},
) from exc
if _recipient_sections_changed(current_version.raw_json, payload.campaign_json):
_require_permission(principal, "campaigns:recipient:write")
_require_mail_profile_use_if_needed(principal, payload.campaign_json)
try:
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: update_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
raw_json=payload.campaign_json,
current_flow=payload.current_flow,
current_step=payload.current_step,
workflow_state=payload.workflow_state,
is_complete=payload.is_complete,
editor_state=payload.editor_state,
source_filename=payload.source_filename,
source_base_path=payload.source_base_path,
autosave=autosave,
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
expected_revision=payload.base_revision,
commit=False,
),
audit_action=audit_action,
details=lambda version: {
"campaign_id": campaign_id,
"current_flow": version.current_flow,
"current_step": version.current_step,
"base_revision": payload.base_revision,
"result_revision": version.edit_revision,
"reconciliation_kind": payload.reconciliation_kind,
"resolved_conflict_path_count": len(
payload.resolved_conflict_paths
),
"resolved_conflict_sections": sorted(
{
path.strip("/").split("/", 1)[0][:80]
for path in payload.resolved_conflict_paths[:100]
if path.strip("/")
}
),
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
},
validation_error_status=status.HTTP_422_UNPROCESSABLE_CONTENT,
)
except RevisionConflictError as exc:
session.rollback()
audit_from_principal(
session,
principal,
action="campaign.version_conflict_detected",
object_type="campaign_version",
object_id=version_id,
details={
"campaign_id": campaign_id,
"submitted_base_revision": exc.submitted_base_revision,
"current_revision": exc.current_revision,
"retryable": True,
},
commit=True,
)
raise HTTPException(
status_code=status.HTTP_412_PRECONDITION_FAILED,
detail=exc.as_dict(),
) from exc
def _require_campaign_profile_use_if_needed(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_id: str | None = None,
) -> None:
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
target_version_id = version_id or campaign.current_version_id
if not target_version_id:
return
version = _get_version_for_tenant(session, target_version_id, principal.tenant_id)
if version.campaign_id != campaign.id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
_require_mail_profile_use_if_needed(
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
)
def _require_campaign_versions_profile_use(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_ids: set[str],
) -> None:
"""Authorize every historical version affected by a campaign-wide action."""
for version_id in sorted(version_ids):
_require_campaign_profile_use_if_needed(
session,
principal,
campaign_id,
version_id,
)
def _get_version_for_principal(
session: Session,
version_id: str,
principal: ApiPrincipal,
*,
write: bool = False,
) -> CampaignVersion:
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
campaign = _get_campaign_for_tenant(
session, version.campaign_id, principal.tenant_id
)
_require_campaign_acl(session, campaign, principal, write=write)
return version
def _sync_campaign_metadata_to_current_version(
session: Session, campaign: Campaign
) -> None:
"""Keep editable version JSON aligned with version-independent campaign metadata.
Campaign metadata can be edited from the overview while individual campaign
sections save the current version JSON later. Without this sync, a later
version save can re-apply stale `campaign.name` / `campaign.id` values from
raw_json and make the old overview metadata appear to come back. Audit-safe
or validation-locked versions are left untouched.
"""
if not campaign.current_version_id:
return
version = session.get(CampaignVersion, campaign.current_version_id)
if not version or version.campaign_id != campaign.id or is_version_locked(version):
return
raw_json = copy.deepcopy(
version.raw_json if isinstance(version.raw_json, dict) else {}
)
campaign_section = (
raw_json.get("campaign") if isinstance(raw_json.get("campaign"), dict) else {}
)
raw_json["campaign"] = {
**campaign_section,
"id": campaign.external_id,
"name": campaign.name,
"description": campaign.description or "",
}
version.raw_json = raw_json
session.add(version)
def _clear_current_version_mail_profile_for_owner_transfer(
session: Session, campaign: Campaign
) -> bool:
"""Force explicit profile reselection after campaign ownership changes.
User/group-scoped reusable mail profiles are evaluated against the current
owner. Instead of trying to keep a stale selection across an ownership
transfer, clear the profile from the editable current version and invalidate
validation/build state so the operator has to reselect and revalidate.
"""
if not campaign.current_version_id:
return False
version = session.get(CampaignVersion, campaign.current_version_id)
if not version or version.campaign_id != campaign.id:
return False
raw_json = copy.deepcopy(
version.raw_json if isinstance(version.raw_json, dict) else {}
)
server = (
raw_json.get("server") if isinstance(raw_json.get("server"), dict) else None
)
if not isinstance(server, dict):
return False
profile_id = _campaign_mail_profile_id(raw_json)
if not profile_id:
return False
if is_version_final_locked(version) or is_user_locked_version(version):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Change owner only after creating an editable campaign version; the current version has a selected mail profile and is locked.",
)
next_server = dict(server)
for key in CAMPAIGN_MAIL_SERVER_KEYS:
next_server.pop(key, None)
next_server.pop("profile_id", None)
raw_json["server"] = next_server
version.raw_json = raw_json
version.validation_summary = None
version.build_summary = None
clear_execution_snapshot(version)
version.locked_at = None
version.locked_by_user_id = None
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
version.is_complete = False
editor_state = copy.deepcopy(version.editor_state or {})
editor_state.pop("review_send", None)
version.editor_state = editor_state
session.query(CampaignIssue).filter(
CampaignIssue.campaign_version_id == version.id
).delete(synchronize_session=False)
session.query(CampaignJob).filter(
CampaignJob.campaign_version_id == version.id
).delete(synchronize_session=False)
campaign.status = CampaignStatus.DRAFT.value
session.add(version)
_write_current_version_snapshot_if_available(version)
return True
def _write_current_version_snapshot_if_available(version: CampaignVersion) -> None:
try:
from govoplan_campaign.backend.persistence.campaigns import (
_write_campaign_snapshot,
)
_write_campaign_snapshot(version)
except Exception:
# The database state is authoritative for the WebUI. Snapshot writing is
# best-effort here because ownership changes should not fail due to an
# unavailable local runtime directory.
return
def bounded_query_rows(query, *, limit: int, label: str):
rows = query.limit(limit + 1).all()
if len(rows) > limit:
raise HTTPException(
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail=(
f"{label} exceeds the maximum response size of {limit} rows. "
"Narrow the request or use a paginated/delta endpoint."
),
)
return rows
def job_attempt_rows(query, *, label: str):
return bounded_query_rows(query, limit=1000, label=label)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1 @@
"""Focused HTTP route modules for the campaign API."""

View File

@@ -0,0 +1,398 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from pydantic import BaseModel, Field
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignVersion,
)
from govoplan_core.db.session import get_session
from govoplan_campaign.backend.persistence.campaigns import (
load_campaign_config_from_json,
)
from govoplan_campaign.backend.integrations import (
files_integration,
)
from govoplan_campaign.backend.path_security import (
CampaignPathSecurityError,
assert_server_safe_campaign_paths,
)
from govoplan_campaign.backend.campaign.loader import load_campaign_json
from govoplan_campaign.backend.attachments.resolver import resolve_campaign_attachments
from govoplan_campaign.backend.persistence.versions import (
is_version_final_locked,
is_user_locked_version,
)
from govoplan_campaign.backend.route_support import (
_get_campaign_for_principal,
_get_campaign_for_tenant,
_get_version_for_tenant,
_require_mail_profile_use_if_needed,
_require_permission,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
class CampaignAttachmentPreviewRequest(BaseModel):
include_unmatched: bool = True
include_unlinked_candidates: bool = False
campaign_json: dict[str, object] | None = None
class CampaignAttachmentPreviewResponse(BaseModel):
campaign_id: str
version_id: str
shared_file_count: int
candidate_file_count: int = 0
matched_file_count: int = 0
linked_file_count: int = 0
unlinked_file_count: int = 0
rules: list[dict[str, object]] = Field(default_factory=list)
linkable_files: list[dict[str, object]] = Field(default_factory=list)
unused_shared_files: list[dict[str, object]] = Field(default_factory=list)
class CampaignAttachmentLinkMatchesRequest(BaseModel):
campaign_json: dict[str, object] | None = None
dry_run: bool = False
class CampaignAttachmentLinkMatchesResponse(BaseModel):
campaign_id: str
version_id: str
matched_file_count: int
already_linked_file_count: int
linked_file_count: int
dry_run: bool = False
linked_files: list[dict[str, object]] = Field(default_factory=list)
linkable_files: list[dict[str, object]] = Field(default_factory=list)
def _file_preview(session: Session, asset) -> dict[str, object]:
version, blob = files_integration().current_version_and_blob(session, asset)
return {
"id": asset.id,
"version_id": version.id,
"blob_id": blob.id,
"display_path": asset.display_path,
"filename": asset.filename,
"owner_type": asset.owner_type,
"owner_id": asset.owner_user_id
if asset.owner_type == "user"
else asset.owner_group_id,
"checksum_sha256": blob.checksum_sha256,
"size_bytes": blob.size_bytes,
"content_type": blob.content_type,
"linked_to_campaign": True,
}
def _managed_preview_file(item: dict[str, object]) -> dict[str, object]:
return {
"id": item["asset_id"],
"version_id": item["version_id"],
"blob_id": item["blob_id"],
"display_path": item["display_path"],
"filename": item["filename"],
"owner_type": item["owner_type"],
"owner_id": item["owner_id"],
"checksum_sha256": item["checksum_sha256"],
"size_bytes": item["size_bytes"],
"content_type": item["content_type"],
"linked_to_campaign": bool(item.get("linked_to_campaign", True)),
}
def _attachment_preview_for_version(
session: Session,
principal: ApiPrincipal,
*,
campaign: Campaign,
version: CampaignVersion,
raw: dict[str, object],
include_unmatched: bool,
include_unlinked_candidates: bool,
) -> CampaignAttachmentPreviewResponse:
files = files_integration()
assert_server_safe_campaign_paths(raw, managed_files_available=files.available)
with files.prepared_campaign_snapshot(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign.id,
raw_json=raw,
include_bytes=False,
prefix="govoplan-managed-preview-",
include_unlinked_candidates=include_unlinked_candidates,
user_id=principal.user.id,
is_admin=has_scope(principal, "files:file:admin"),
) as prepared:
prepared_raw = load_campaign_json(prepared.path)
config = load_campaign_config_from_json(
session,
tenant_id=principal.tenant_id,
raw_json=prepared_raw,
campaign_id=campaign.id,
)
report = resolve_campaign_attachments(config, campaign_file=prepared.path)
rules: list[dict[str, object]] = []
matched_asset_ids: set[str] = set()
linked_asset_ids: set[str] = set()
linkable_by_id: dict[str, dict[str, object]] = {}
for entry in report.entries:
for attachment in entry.attachments:
managed_matches = files.managed_match_payloads(
attachment.matches, prepared.managed_files_by_local_path
)
matches: list[dict[str, object]] = []
for item in managed_matches:
asset_id = str(item["asset_id"])
matched_asset_ids.add(asset_id)
if bool(item.get("linked_to_campaign", True)):
linked_asset_ids.add(asset_id)
preview = _managed_preview_file(item)
matches.append(preview)
if not preview["linked_to_campaign"]:
linkable_by_id.setdefault(asset_id, preview)
if not matches:
matches = [
{
"id": "",
"display_path": match,
"filename": match.rsplit("/", 1)[-1].rsplit("\\", 1)[-1],
"owner_type": "legacy",
"owner_id": "",
"linked_to_campaign": True,
}
for match in attachment.matches
]
rules.append(
{
"source": attachment.scope.value,
"entry_index": entry.entry_index,
"entry_id": entry.entry_id,
"index": attachment.index,
"attachment_id": attachment.attachment_id,
"label": attachment.label,
"required": attachment.required,
"pattern": attachment.file_filter,
"base_path_name": attachment.base_path_name,
"base_path": attachment.base_path,
"status": attachment.status.value,
"behavior": attachment.behavior.value
if attachment.behavior
else None,
"zip_included": attachment.zip_enabled,
"zip_mode": attachment.zip_mode.value,
"zip_archive_id": attachment.zip_archive_id,
"zip_filename": attachment.zip_filename,
"matches": matches,
"match_count": len(matches),
"linked_match_count": sum(
1
for match in matches
if bool(match.get("linked_to_campaign", True))
),
"unlinked_match_count": sum(
1
for match in matches
if not bool(match.get("linked_to_campaign", True))
),
"issues": [
issue.model_dump(mode="json") for issue in attachment.issues
],
}
)
unused = [
asset
for asset in prepared.shared_assets
if asset.id not in matched_asset_ids
]
return CampaignAttachmentPreviewResponse(
campaign_id=campaign.id,
version_id=version.id,
shared_file_count=len(prepared.shared_assets),
candidate_file_count=len(
getattr(prepared, "candidate_assets", prepared.shared_assets)
),
matched_file_count=len(matched_asset_ids),
linked_file_count=len(linked_asset_ids),
unlinked_file_count=len(linkable_by_id),
rules=rules,
linkable_files=list(linkable_by_id.values()),
unused_shared_files=[_file_preview(session, asset) for asset in unused]
if include_unmatched
else [],
)
def _link_campaign_attachment_matches(
session: Session,
principal: ApiPrincipal,
*,
campaign: Campaign,
version: CampaignVersion,
raw: dict[str, object],
dry_run: bool = False,
) -> CampaignAttachmentLinkMatchesResponse:
preview = _attachment_preview_for_version(
session,
principal,
campaign=campaign,
version=version,
raw=raw,
include_unmatched=False,
include_unlinked_candidates=True,
)
file_ids = [
str(item.get("id") or "") for item in preview.linkable_files if item.get("id")
]
linked_files: list[dict[str, object]] = []
if file_ids and not dry_run:
files = files_integration()
shares = files.share_assets_with_campaign(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign.id,
file_ids=file_ids,
user_id=principal.user.id,
is_admin=has_scope(principal, "files:file:admin"),
)
share_by_asset_id = {
str(item.get("file_asset_id") or ""): item for item in shares
}
linked_files = [
{**item, "share": share_by_asset_id.get(str(item.get("id") or ""))}
for item in preview.linkable_files
]
return CampaignAttachmentLinkMatchesResponse(
campaign_id=campaign.id,
version_id=version.id,
matched_file_count=preview.matched_file_count,
already_linked_file_count=preview.linked_file_count,
linked_file_count=0 if dry_run else len(file_ids),
dry_run=dry_run,
linked_files=linked_files,
linkable_files=preview.linkable_files,
)
@router.post(
"/{campaign_id}/versions/{version_id}/attachments/preview",
response_model=CampaignAttachmentPreviewResponse,
)
def preview_campaign_attachments(
campaign_id: str,
version_id: str,
payload: CampaignAttachmentPreviewRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
):
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if version.campaign_id != campaign.id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
payload = payload or CampaignAttachmentPreviewRequest()
raw = (
payload.campaign_json
if isinstance(payload.campaign_json, dict)
else version.raw_json
)
raw = raw if isinstance(raw, dict) else {}
_require_mail_profile_use_if_needed(principal, raw)
try:
return _attachment_preview_for_version(
session,
principal,
campaign=campaign,
version=version,
raw=raw,
include_unmatched=payload.include_unmatched,
include_unlinked_candidates=payload.include_unlinked_candidates,
)
except CampaignPathSecurityError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post(
"/{campaign_id}/versions/{version_id}/attachments/link-matches",
response_model=CampaignAttachmentLinkMatchesResponse,
)
def link_campaign_attachment_matches(
campaign_id: str,
version_id: str,
payload: CampaignAttachmentLinkMatchesRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
):
_require_permission(principal, "files:file:share")
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if version.campaign_id != campaign.id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
if is_user_locked_version(version) or is_version_final_locked(version):
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Locked campaign versions cannot link new attachment files",
)
payload = payload or CampaignAttachmentLinkMatchesRequest()
raw = (
payload.campaign_json
if isinstance(payload.campaign_json, dict)
else version.raw_json
)
raw = raw if isinstance(raw, dict) else {}
_require_mail_profile_use_if_needed(principal, raw)
try:
result = _link_campaign_attachment_matches(
session,
principal,
campaign=campaign,
version=version,
raw=raw,
dry_run=payload.dry_run,
)
audit_from_principal(
session,
principal,
action="campaign.attachment_matches_linked"
if not payload.dry_run
else "campaign.attachment_matches_link_previewed",
object_type="campaign_version",
object_id=version_id,
details={
"matched_file_count": result.matched_file_count,
"already_linked_file_count": result.already_linked_file_count,
"linked_file_count": result.linked_file_count,
"dry_run": result.dry_run,
},
commit=True,
)
return result
except HTTPException:
raise
except Exception as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,665 @@
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from govoplan_campaign.backend.schemas import (
AppendSentRequest,
CampaignActionResponse,
CampaignRetryJobsRequest,
CampaignSendJobRequest,
CampaignSendUnattemptedRequest,
CampaignResolveOutcomeRequest,
CampaignDeliveryOptionsResponse,
MockCampaignSendRequest,
MockCampaignSendResponse,
QueueCampaignRequest,
QueueCampaignResponse,
SendCampaignNowRequest,
SendCampaignNowResponse,
)
from govoplan_core.auth import ApiPrincipal, require_any_scope, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_campaign.backend.db.models import (
CampaignJob,
JobImapStatus,
JobQueueStatus,
JobSendStatus,
)
from govoplan_campaign.backend.integrations import (
postbox_integration,
)
from govoplan_core.db.session import get_session
from govoplan_campaign.backend.response_security import (
public_send_campaign_now_result,
send_campaign_now_audit_details,
)
from govoplan_campaign.backend.persistence.campaigns import (
CampaignPersistenceError,
)
from govoplan_campaign.backend.persistence.versions import (
is_user_locked_version,
)
from govoplan_campaign.backend.dev.mock_campaign import (
MockCampaignSendError,
run_mock_campaign_send,
)
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError
from govoplan_campaign.backend.sending.jobs import (
QueueingError,
SynchronousSendRejected,
cancel_campaign_jobs,
enqueue_pending_imap_appends,
pause_campaign_jobs,
queue_campaign_jobs,
queue_failed_jobs_for_retry,
queue_unattempted_jobs,
reconcile_job_outcome,
resume_campaign_jobs,
send_campaign_now,
send_single_campaign_job,
synchronous_send_options,
)
from govoplan_campaign.backend.route_support import (
_get_campaign_for_principal,
_get_campaign_for_tenant,
_get_version_for_tenant,
_require_campaign_profile_use_if_needed,
_require_campaign_versions_profile_use,
_require_mail_profile_use_if_needed,
_require_permission,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
logger = logging.getLogger(__name__)
@router.get(
"/{campaign_id}/delivery-options", response_model=CampaignDeliveryOptionsResponse
)
def campaign_delivery_options(
campaign_id: str,
version_id: str | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(
require_any_scope("campaigns:campaign:send", "campaigns:campaign:queue")
),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
try:
return CampaignDeliveryOptionsResponse(
**synchronous_send_options(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
),
postbox_available=postbox_integration().available,
)
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/queue", response_model=QueueCampaignResponse)
def queue_campaign(
campaign_id: str,
payload: QueueCampaignRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or QueueCampaignRequest()
_require_campaign_profile_use_if_needed(
session, principal, campaign_id, payload.version_id
)
try:
result = queue_campaign_jobs(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=payload.version_id,
include_warnings=payload.include_warnings,
enqueue_celery=payload.enqueue_celery,
dry_run=payload.dry_run,
)
audit_from_principal(
session,
principal,
action="campaign.queued"
if not payload.dry_run
else "campaign.queue_dry_run",
object_type="campaign",
object_id=campaign_id,
details=result.as_dict(),
commit=True,
)
return QueueCampaignResponse(**result.as_dict())
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/jobs/retry", response_model=CampaignActionResponse)
def retry_campaign_jobs(
campaign_id: str,
payload: CampaignRetryJobsRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:retry")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or CampaignRetryJobsRequest()
_require_campaign_profile_use_if_needed(
session, principal, campaign_id, payload.version_id
)
try:
result = queue_failed_jobs_for_retry(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=payload.version_id,
job_ids=payload.job_ids or None,
include_permanent=payload.include_permanent,
force_max_attempts=payload.force_max_attempts,
enqueue_celery=payload.enqueue_celery,
dry_run=payload.dry_run,
)
audit_from_principal(
session,
principal,
action="campaign.jobs_retry_queued"
if not payload.dry_run
else "campaign.jobs_retry_dry_run",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except (QueueingError, ExecutionSnapshotError) as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post(
"/{campaign_id}/jobs/send-unattempted", response_model=CampaignActionResponse
)
def send_unattempted_campaign_jobs(
campaign_id: str,
payload: CampaignSendUnattemptedRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or CampaignSendUnattemptedRequest()
_require_campaign_profile_use_if_needed(
session, principal, campaign_id, payload.version_id
)
try:
result = queue_unattempted_jobs(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=payload.version_id,
job_ids=payload.job_ids or None,
enqueue_celery=payload.enqueue_celery,
dry_run=payload.dry_run,
)
audit_from_principal(
session,
principal,
action="campaign.unattempted_jobs_queued"
if not payload.dry_run
else "campaign.unattempted_jobs_dry_run",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except (QueueingError, ExecutionSnapshotError) as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/jobs/{job_id}/send", response_model=CampaignActionResponse)
def send_single_campaign_job_endpoint(
campaign_id: str,
job_id: str,
payload: CampaignSendJobRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(
require_any_scope(
"campaigns:campaign:send",
"campaigns:campaign:send_test",
)
),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
_require_permission(
principal,
(
"campaigns:campaign:send_test"
if payload.kind == "test"
else "campaigns:campaign:send"
),
)
_require_campaign_profile_use_if_needed(session, principal, campaign_id, None)
try:
result = send_single_campaign_job(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
job_id=job_id,
kind=payload.kind,
idempotency_key=payload.idempotency_key,
actor_user_id=principal.user.id,
actor_api_key_id=getattr(principal, "api_key_id", None),
reason=payload.reason,
action_context=payload.context,
include_warnings=payload.include_warnings,
use_rate_limit=payload.use_rate_limit,
enqueue_imap_task=payload.enqueue_imap_task,
)
audit_from_principal(
session,
principal,
action=f"campaign.message_{payload.kind}",
object_type="campaign_job",
object_id=job_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except (QueueingError, ExecutionSnapshotError) as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception as exc:
logger.exception(
"Unexpected single-message campaign action failure",
extra={
"campaign_id": campaign_id,
"job_id": job_id,
"action_kind": payload.kind,
},
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="The message action failed because of an internal error.",
) from exc
@router.post(
"/{campaign_id}/jobs/{job_id}/resolve-outcome",
response_model=CampaignActionResponse,
)
def resolve_campaign_job_outcome(
campaign_id: str,
job_id: str,
payload: CampaignResolveOutcomeRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:reconcile")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
try:
result = reconcile_job_outcome(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
job_id=job_id,
decision=payload.decision,
note=payload.note,
attempt_id=payload.attempt_id,
commit=False,
)
audit_from_principal(
session,
principal,
action="campaign.job_outcome_reconciled",
object_type="campaign_job",
object_id=job_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except (QueueingError, ExecutionSnapshotError) as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception:
session.rollback()
raise
@router.post("/{campaign_id}/mock-send", response_model=MockCampaignSendResponse)
def mock_send_campaign(
campaign_id: str,
payload: MockCampaignSendRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send_test")),
):
"""Run a fully visible mock delivery flow without mutating campaign state.
The route validates and builds the selected version, then optionally records
mock SMTP deliveries and mock IMAP appends. It never talks to the configured
real SMTP/IMAP servers and it does not mark the version sent/final.
"""
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or MockCampaignSendRequest()
_require_campaign_profile_use_if_needed(
session, principal, campaign_id, payload.version_id
)
try:
result = run_mock_campaign_send(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=payload.version_id,
send=payload.send,
include_warnings=payload.include_warnings,
include_needs_review=payload.include_needs_review,
append_sent=payload.append_sent,
clear_mailbox=payload.clear_mailbox,
check_files=payload.check_files,
)
audit_from_principal(
session,
principal,
action="campaign.mock_send"
if payload.send
else "campaign.mock_send_review",
object_type="campaign",
object_id=campaign_id,
details={
"version_id": result.get("version_id"),
"send_requested": payload.send,
"sent_count": result.get("send", {}).get("sent_count"),
"failed_count": result.get("send", {}).get("failed_count"),
},
commit=True,
)
return MockCampaignSendResponse(result=result)
except MockCampaignSendError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/send-now", response_model=SendCampaignNowResponse)
def send_campaign_now_endpoint(
campaign_id: str,
payload: SendCampaignNowRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
):
"""Preflight and synchronously send a policy-bounded built execution."""
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or SendCampaignNowRequest()
try:
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
version_id = payload.version_id or campaign.current_version_id
if not version_id:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Campaign has no current version",
)
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
_require_mail_profile_use_if_needed(
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
)
validation_result: dict[str, object] | None = (
version.validation_summary
if isinstance(version.validation_summary, dict)
else None
)
build_result: dict[str, object] | None = (
version.build_summary if isinstance(version.build_summary, dict) else None
)
if is_user_locked_version(version):
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="User-locked audit-safe versions cannot be dry-run or sent. Create an editable copy and validate it instead.",
)
if (
not version.locked_at
or not validation_result
or validation_result.get("ok") is not True
):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Campaign version must be validated and locked before dry-run or sending.",
)
if not build_result:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Campaign version must be built before dry-run or sending.",
)
delivery_result = send_campaign_now(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
include_warnings=payload.include_warnings,
dry_run=payload.dry_run,
use_rate_limit=payload.use_rate_limit,
enqueue_imap_task=payload.enqueue_imap_task,
).as_dict()
response_result = public_send_campaign_now_result(
delivery_result,
validation_summary=validation_result,
build_summary=build_result,
)
audit_from_principal(
session,
principal,
action="campaign.sent_now"
if not payload.dry_run
else "campaign.send_now_dry_run",
object_type="campaign",
object_id=campaign_id,
details=send_campaign_now_audit_details(delivery_result),
commit=True,
)
return SendCampaignNowResponse(result=response_result)
except SynchronousSendRejected as exc:
# A synchronous request stages queue state before the all-message
# preflight can run. Rejecting that preflight must not leave work
# eligible for a background worker when no provider effect occurred.
session.rollback()
audit_from_principal(
session,
principal,
action="campaign.send_now_rejected",
object_type="campaign",
object_id=campaign_id,
details={
**exc.audit_details(),
"version_id": payload.version_id,
},
commit=True,
)
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except HTTPException:
raise
except (CampaignPersistenceError, QueueingError) as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/pause", response_model=CampaignActionResponse)
def pause_campaign(
campaign_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
try:
result = pause_campaign_jobs(
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
)
audit_from_principal(
session,
principal,
action="campaign.paused",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/resume", response_model=CampaignActionResponse)
def resume_campaign(
campaign_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
version_ids = {
row[0]
for row in session.query(CampaignJob.campaign_version_id)
.filter(
CampaignJob.tenant_id == principal.tenant_id,
CampaignJob.campaign_id == campaign_id,
CampaignJob.queue_status == JobQueueStatus.PAUSED.value,
)
.distinct()
.all()
}
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
try:
result = resume_campaign_jobs(
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
)
audit_from_principal(
session,
principal,
action="campaign.resumed",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/cancel", response_model=CampaignActionResponse)
def cancel_campaign(
campaign_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
try:
result = cancel_campaign_jobs(
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
)
audit_from_principal(
session,
principal,
action="campaign.cancelled",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/{campaign_id}/append-sent", response_model=CampaignActionResponse)
def append_sent(
campaign_id: str,
payload: AppendSentRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
payload = payload or AppendSentRequest()
version_ids = {
row[0]
for row in session.query(CampaignJob.campaign_version_id)
.filter(
CampaignJob.tenant_id == principal.tenant_id,
CampaignJob.campaign_id == campaign_id,
CampaignJob.send_status.in_(
[JobSendStatus.SMTP_ACCEPTED.value, JobSendStatus.SENT.value]
),
CampaignJob.imap_status.in_(
[JobImapStatus.PENDING.value, JobImapStatus.FAILED.value]
),
)
.distinct()
.all()
}
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
try:
result = enqueue_pending_imap_appends(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
enqueue_celery=payload.enqueue_celery,
run_inline=payload.run_inline,
dry_run=payload.dry_run,
)
audit_from_principal(
session,
principal,
action="campaign.append_sent_enqueued"
if not payload.dry_run
else "campaign.append_sent_dry_run",
object_type="campaign",
object_id=campaign_id,
details=result,
commit=True,
)
return CampaignActionResponse(result=result)
except QueueingError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc

View File

@@ -0,0 +1,508 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy.orm import Session
from govoplan_campaign.backend.schemas import (
CampaignJobsResponse,
CampaignJobsDeltaResponse,
CampaignJobDetailResponse,
CampaignJobDiagnosticsResponse,
)
from govoplan_core.auth import ApiPrincipal, require_scope
from govoplan_core.core.change_sequence import (
decode_sequence_watermark,
encode_sequence_watermark,
sequence_entries_since,
sequence_watermark_is_expired,
)
from govoplan_campaign.backend.change_tracking import (
CAMPAIGNS_MODULE_ID,
CAMPAIGN_JOBS_COLLECTION,
)
from govoplan_campaign.backend.db.models import (
CampaignJob,
CampaignMessageAction,
CampaignMessageActionAttempt,
ImapAppendAttempt,
PostboxDeliveryAttempt,
SendAttempt,
)
from govoplan_campaign.backend.integrations import postbox_integration
from govoplan_core.db.session import get_session
from govoplan_campaign.backend.route_support import (
_get_campaign_for_principal,
_get_campaign_for_tenant,
_require_permission,
job_attempt_rows as _job_attempt_rows,
)
from govoplan_campaign.backend.services.job_queries import (
CampaignJobsQuery,
_campaign_jobs_delta_watermark,
_campaign_jobs_page_response,
_campaign_jobs_query_context,
_job_attempts_payload,
_job_detail_payload,
_job_diagnostics_payload,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
def _postbox_receipts_for_attempts(
session: Session,
*,
tenant_id: str,
attempts: list[PostboxDeliveryAttempt],
):
integration = postbox_integration()
if not integration.receipt_evidence_available:
return None
delivery_ids = [
attempt.provider_delivery_id
for attempt in attempts
if attempt.provider_delivery_id
]
return integration.delivery_receipt_summaries(
session,
tenant_id=tenant_id,
delivery_ids=delivery_ids,
)
@router.get("/{campaign_id}/jobs", response_model=CampaignJobsResponse)
def list_jobs(
campaign_id: str,
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
"""Return a lightweight, paginated job list with server-side filters.
Complete recipients, attachment metadata, issues and attempt history are
available from the separate job-detail endpoint.
"""
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
_campaign_jobs_query_context(
session,
principal,
campaign_id=campaign_id,
version_id=filters.version_id,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
)
)
return _campaign_jobs_page_response(
session,
campaign_id=campaign_id,
version_id=filters.version_id,
base_filters=base_filters,
filtered=filtered,
reviewed_keys=reviewed_keys,
review_metadata=review_metadata,
page=filters.page,
page_size=filters.page_size,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
cursor=filters.cursor,
)
def _campaign_jobs_full_delta_response(
session: Session,
*,
principal: ApiPrincipal,
campaign_id: str,
version_id: str | None,
page: int,
page_size: int,
send_status: list[str] | None,
validation_status: list[str] | None,
imap_status: list[str] | None,
query_text: str | None,
grid_filters: dict[str, str] | None,
sort_by: str,
sort_direction: str,
cursor: str | None = None,
) -> CampaignJobsDeltaResponse:
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
_campaign_jobs_query_context(
session,
principal,
campaign_id=campaign_id,
version_id=version_id,
send_status=send_status,
validation_status=validation_status,
imap_status=imap_status,
query_text=query_text,
grid_filters=grid_filters,
)
)
payload = _campaign_jobs_page_response(
session,
campaign_id=campaign_id,
version_id=version_id,
base_filters=base_filters,
filtered=filtered,
reviewed_keys=reviewed_keys,
review_metadata=review_metadata,
page=page,
page_size=page_size,
send_status=send_status,
validation_status=validation_status,
imap_status=imap_status,
query_text=query_text,
grid_filters=grid_filters,
sort_by=sort_by,
sort_direction=sort_direction,
cursor=cursor,
)
return CampaignJobsDeltaResponse(
**payload.model_dump(),
deleted=[],
watermark=_campaign_jobs_delta_watermark(session, principal.tenant_id),
has_more=False,
full=True,
)
def _job_filter_membership_can_shift(
*,
send_status: list[str] | None,
validation_status: list[str] | None,
imap_status: list[str] | None,
query_text: str | None,
grid_filters: dict[str, str] | None,
sort_by: str,
sort_direction: str,
) -> bool:
return bool(
send_status
or validation_status
or imap_status
or (query_text and query_text.strip())
or grid_filters
or sort_by != "number"
or sort_direction != "asc"
)
@router.get("/{campaign_id}/jobs/delta", response_model=CampaignJobsDeltaResponse)
def list_jobs_delta(
campaign_id: str,
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
since: str | None = None,
limit: int = Query(default=500, ge=1, le=1000),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
if since is None:
return _campaign_jobs_full_delta_response(
session,
principal=principal,
campaign_id=campaign_id,
version_id=filters.version_id,
page=filters.page,
page_size=filters.page_size,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
cursor=filters.cursor,
)
campaign, base_filters, filtered, review_metadata, reviewed_keys = (
_campaign_jobs_query_context(
session,
principal,
campaign_id=campaign_id,
version_id=filters.version_id,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
)
)
try:
since_sequence = decode_sequence_watermark(since)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
) from exc
if sequence_watermark_is_expired(
session,
since=since_sequence,
tenant_id=principal.tenant_id,
module_id=CAMPAIGNS_MODULE_ID,
collections=(CAMPAIGN_JOBS_COLLECTION,),
):
return _campaign_jobs_full_delta_response(
session,
principal=principal,
campaign_id=campaign_id,
version_id=filters.version_id,
page=filters.page,
page_size=filters.page_size,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
cursor=filters.cursor,
)
entries_plus_one = sequence_entries_since(
session,
since=since_sequence,
tenant_id=principal.tenant_id,
module_id=CAMPAIGNS_MODULE_ID,
collections=(CAMPAIGN_JOBS_COLLECTION,),
limit=limit + 1,
)
has_more = len(entries_plus_one) > limit
entries = entries_plus_one[:limit]
relevant_entries = [
entry
for entry in entries
if (entry.payload or {}).get("campaign_id") == campaign.id
and (
not filters.version_id
or (entry.payload or {}).get("version_id") == filters.version_id
)
]
if relevant_entries and (
_job_filter_membership_can_shift(
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
)
or any(entry.operation in {"created", "deleted"} for entry in relevant_entries)
):
return _campaign_jobs_full_delta_response(
session,
principal=principal,
campaign_id=campaign_id,
version_id=filters.version_id,
page=filters.page,
page_size=filters.page_size,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
cursor=filters.cursor,
)
changed_job_ids = {
entry.resource_id
for entry in relevant_entries
if entry.resource_type == "campaign_job" and entry.operation != "deleted"
}
payload = _campaign_jobs_page_response(
session,
campaign_id=campaign_id,
version_id=filters.version_id,
base_filters=base_filters,
filtered=filtered,
reviewed_keys=reviewed_keys,
review_metadata=review_metadata,
page=filters.page,
page_size=filters.page_size,
send_status=filters.send_status,
validation_status=filters.validation_status,
imap_status=filters.imap_status,
query_text=filters.query_text,
grid_filters=filters.grid_filters,
sort_by=filters.sort_by,
sort_direction=filters.sort_direction,
cursor=filters.cursor,
changed_job_ids=changed_job_ids,
)
watermark = (
encode_sequence_watermark(entries[-1].id)
if has_more and entries
else _campaign_jobs_delta_watermark(session, principal.tenant_id)
)
return CampaignJobsDeltaResponse(
**payload.model_dump(),
deleted=[],
watermark=watermark,
has_more=has_more,
full=False,
)
@router.get("/{campaign_id}/jobs/{job_id}", response_model=CampaignJobDetailResponse)
def get_job_detail(
campaign_id: str,
job_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
job = session.get(CampaignJob, job_id)
if (
not job
or job.campaign_id != campaign.id
or job.tenant_id != principal.tenant_id
):
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
)
send_attempts = _job_attempt_rows(
session.query(SendAttempt)
.filter(SendAttempt.job_id == job.id)
.order_by(SendAttempt.attempt_number.asc()),
label="SMTP attempts for this campaign job",
)
imap_attempts = _job_attempt_rows(
session.query(ImapAppendAttempt)
.filter(ImapAppendAttempt.job_id == job.id)
.order_by(ImapAppendAttempt.attempt_number.asc()),
label="IMAP attempts for this campaign job",
)
postbox_attempts = _job_attempt_rows(
session.query(PostboxDeliveryAttempt)
.filter(PostboxDeliveryAttempt.job_id == job.id)
.order_by(
PostboxDeliveryAttempt.target_index.asc(),
PostboxDeliveryAttempt.attempt_number.asc(),
),
label="Postbox attempts for this campaign job",
)
message_actions = _job_attempt_rows(
session.query(CampaignMessageAction)
.filter(CampaignMessageAction.job_id == job.id)
.order_by(CampaignMessageAction.created_at.asc()),
label="Single-message actions for this campaign job",
)
action_ids = [action.id for action in message_actions]
message_action_attempts = (
_job_attempt_rows(
session.query(CampaignMessageActionAttempt)
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
.order_by(CampaignMessageActionAttempt.started_at.asc()),
label="Single-message action attempts for this campaign job",
)
if action_ids
else []
)
return CampaignJobDetailResponse(
job=_job_detail_payload(job),
attempts=_job_attempts_payload(
send_attempts,
imap_attempts,
postbox_attempts,
message_actions,
message_action_attempts,
postbox_receipts=_postbox_receipts_for_attempts(
session,
tenant_id=principal.tenant_id,
attempts=postbox_attempts,
),
),
)
@router.get(
"/{campaign_id}/jobs/{job_id}/diagnostics",
response_model=CampaignJobDiagnosticsResponse,
)
def get_job_diagnostics(
campaign_id: str,
job_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:diagnostic:read")),
):
"""Return infrastructure details only to campaign operators/admins."""
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
job = session.get(CampaignJob, job_id)
if (
not job
or job.campaign_id != campaign.id
or job.tenant_id != principal.tenant_id
):
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
)
send_attempts = _job_attempt_rows(
session.query(SendAttempt)
.filter(SendAttempt.job_id == job.id)
.order_by(SendAttempt.attempt_number.asc()),
label="SMTP diagnostics for this campaign job",
)
imap_attempts = _job_attempt_rows(
session.query(ImapAppendAttempt)
.filter(ImapAppendAttempt.job_id == job.id)
.order_by(ImapAppendAttempt.attempt_number.asc()),
label="IMAP diagnostics for this campaign job",
)
postbox_attempts = _job_attempt_rows(
session.query(PostboxDeliveryAttempt)
.filter(PostboxDeliveryAttempt.job_id == job.id)
.order_by(
PostboxDeliveryAttempt.target_index.asc(),
PostboxDeliveryAttempt.attempt_number.asc(),
),
label="Postbox diagnostics for this campaign job",
)
message_actions = _job_attempt_rows(
session.query(CampaignMessageAction)
.filter(CampaignMessageAction.job_id == job.id)
.order_by(CampaignMessageAction.created_at.asc()),
label="Single-message action diagnostics for this campaign job",
)
action_ids = [action.id for action in message_actions]
message_action_attempts = (
_job_attempt_rows(
session.query(CampaignMessageActionAttempt)
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
.order_by(CampaignMessageActionAttempt.started_at.asc()),
label="Single-message action-attempt diagnostics for this campaign job",
)
if action_ids
else []
)
return _job_diagnostics_payload(
job,
send_attempts,
imap_attempts,
postbox_attempts,
message_actions,
message_action_attempts,
postbox_receipts=_postbox_receipts_for_attempts(
session,
tenant_id=principal.tenant_id,
attempts=postbox_attempts,
),
)

View File

@@ -0,0 +1,257 @@
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, Response, status
from sqlalchemy.orm import Session
from govoplan_campaign.backend.schemas import (
ReportEmailRequest,
ReportEmailResponse,
)
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_campaign.backend.db.models import (
CampaignVersion,
)
from govoplan_core.db.session import get_session
from govoplan_campaign.backend.reports.campaigns import (
CampaignReportError,
generate_campaign_report,
generate_jobs_csv,
)
from govoplan_campaign.backend.reports.emailing import (
CampaignReportEmailError,
send_campaign_report_email,
)
from govoplan_campaign.backend.integrations import (
MailDeliveryCommandError,
MailProfileError,
SmtpConfigurationError,
SmtpSendError,
mail_integration,
)
from govoplan_campaign.backend.route_support import (
_get_campaign_for_principal,
_require_mail_profile_use_if_needed,
_require_permission,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
logger = logging.getLogger(__name__)
def _enqueue_mail_command() -> None:
try:
from govoplan_core.celery_app import celery
from govoplan_core.settings import settings
if settings.celery_enabled:
celery.send_task(
"govoplan.mail.dispatch_outbox",
args=[None, 25],
queue="mail",
)
except Exception:
logger.warning(
"Mail delivery command is durable but immediate worker wake-up failed",
exc_info=True,
)
@router.get("/{campaign_id}/summary")
def campaign_summary(
campaign_id: str,
version_id: str | None = None,
include_jobs: bool = False,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
"""Return dashboard-friendly campaign status counters and summaries."""
_get_campaign_for_principal(session, campaign_id, principal)
if include_jobs:
_require_permission(principal, "campaigns:recipient:read")
try:
return generate_campaign_report(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
include_jobs=include_jobs,
include_recent_failures=include_jobs,
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
)
except CampaignReportError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
@router.get("/{campaign_id}/report")
def campaign_report(
campaign_id: str,
version_id: str | None = None,
include_jobs: bool = False,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
):
"""Return the recipient-level JSON report for one campaign."""
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
try:
return generate_campaign_report(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
include_jobs=include_jobs,
include_recent_failures=include_jobs,
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
)
except CampaignReportError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
@router.get("/{campaign_id}/report/jobs.csv")
def campaign_jobs_csv(
campaign_id: str,
version_id: str | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:report:export")),
):
"""Export per-job campaign status as CSV."""
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:export")
try:
csv_text = generate_jobs_csv(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
)
except CampaignReportError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
return Response(
content=csv_text,
media_type="text/csv; charset=utf-8",
headers={
"Content-Disposition": f'attachment; filename="campaign-{campaign_id}-jobs.csv"'
},
)
@router.post("/{campaign_id}/report/email", response_model=ReportEmailResponse)
def email_campaign_report(
campaign_id: str,
payload: ReportEmailRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:report:send")),
):
"""Generate a campaign report and send it to one or more email addresses."""
campaign = _get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:export")
selected_version_id = payload.version_id or campaign.current_version_id
selected_version = (
session.get(CampaignVersion, selected_version_id)
if selected_version_id
else None
)
if selected_version is not None and selected_version.campaign_id == campaign.id:
_require_mail_profile_use_if_needed(
principal,
selected_version.raw_json
if isinstance(selected_version.raw_json, dict)
else {},
)
try:
result = send_campaign_report_email(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=payload.version_id,
to=payload.to,
include_jobs=payload.include_jobs,
attach_jobs_csv=payload.attach_jobs_csv,
attach_report_json=payload.attach_report_json,
dry_run=payload.dry_run,
idempotency_key=payload.idempotency_key,
created_by_user_id=principal.user.id,
)
audit_from_principal(
session,
principal,
action="report.email_requested"
if not payload.dry_run
else "report.email_dry_run",
object_type="campaign",
object_id=campaign_id,
details=result.audit_dict(),
commit=True,
)
if not payload.dry_run:
_enqueue_mail_command()
return ReportEmailResponse(result=result.as_dict())
except CampaignReportError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except (
CampaignReportEmailError,
MailProfileError,
MailDeliveryCommandError,
SmtpConfigurationError,
SmtpSendError,
) as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception as exc:
logger.error("Campaign report email failed with an unexpected internal error")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Campaign report email could not be completed.",
) from exc
@router.get(
"/{campaign_id}/report/email/{command_id}",
response_model=ReportEmailResponse,
)
def campaign_report_email_status(
campaign_id: str,
command_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
):
_get_campaign_for_principal(session, campaign_id, principal)
try:
result = mail_integration().delivery_command_summary(
session,
tenant_id=principal.tenant_id,
command_id=command_id,
)
except MailDeliveryCommandError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(exc),
) from exc
if (
result.get("source_module") != "campaigns"
or result.get("source_resource_type") != "campaign"
or result.get("source_resource_id") != campaign_id
):
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Campaign report delivery not found",
)
return ReportEmailResponse(result=result)

View File

@@ -0,0 +1,288 @@
from __future__ import annotations
from typing import Literal
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy.orm import Session
from govoplan_core.api.v1.schemas import (
ReferenceOptionListResponse,
ReferenceOptionResponse,
)
from govoplan_campaign.backend.schemas import (
CampaignShareItem,
CampaignShareListResponse,
CampaignShareTargetItem,
CampaignShareTargetsResponse,
CampaignShareUpsertRequest,
CampaignOwnerUpdateRequest,
CampaignResponse,
)
from govoplan_core.auth import ApiPrincipal, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_campaign.backend.db.models import (
CampaignShare,
)
from govoplan_core.db.session import get_session
from govoplan_core.core.references import (
access_scope_reference_page,
access_scope_reference_provider_available,
)
from govoplan_core.core.runtime import get_registry
from govoplan_core.security.time import utc_now
from govoplan_campaign.backend.route_support import (
_access_directory,
_get_campaign_for_principal,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
@router.get(
"/{campaign_id}/share-target-options",
response_model=ReferenceOptionListResponse,
)
def search_campaign_share_targets(
campaign_id: str,
target_type: Literal["user", "group"],
q: str = "",
selected: list[str] = Query(default=[]),
limit: int = Query(default=50, ge=1, le=200),
cursor: str | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
) -> ReferenceOptionListResponse:
_get_campaign_for_principal(session, campaign_id, principal, write=True)
registry = get_registry()
try:
page = access_scope_reference_page(
registry,
principal,
scope_type=target_type,
reference_kind="membership" if target_type == "user" else "group",
query=q,
selected_values=selected,
limit=limit,
cursor=cursor,
administrative=True,
session=session,
)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=str(exc),
) from exc
return ReferenceOptionListResponse(
options=[
ReferenceOptionResponse(**option.to_dict())
for option in page.options
],
provider_available=access_scope_reference_provider_available(registry),
next_cursor=page.next_cursor,
has_more=page.has_more,
)
@router.get("/{campaign_id}/share-targets", response_model=CampaignShareTargetsResponse)
def list_campaign_share_targets(
campaign_id: str,
limit: int = Query(default=500, ge=1, le=1000),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
directory = _access_directory()
users = [
user
for user in directory.users_for_tenant(principal.tenant_id)
if user.status == "active"
]
groups = [
group
for group in directory.groups_for_tenant(principal.tenant_id)
if group.status == "active"
]
if len(users) > limit or len(groups) > limit:
raise HTTPException(
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail=(
f"Campaign share targets exceed the maximum response size of {limit} "
"users or groups. Use a searchable directory selector."
),
)
return CampaignShareTargetsResponse(
users=[
CampaignShareTargetItem(
id=item.id, name=item.display_name or item.email, secondary=item.email
)
for item in users
],
groups=[
CampaignShareTargetItem(id=item.id, name=item.name, secondary=None)
for item in groups
],
)
@router.get("/{campaign_id}/shares", response_model=CampaignShareListResponse)
def list_campaign_shares(
campaign_id: str,
page: int = Query(default=1, ge=1),
page_size: int = Query(default=500, ge=1, le=1000),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
):
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
query = session.query(CampaignShare).filter(
CampaignShare.tenant_id == principal.tenant_id,
CampaignShare.campaign_id == campaign.id,
CampaignShare.revoked_at.is_(None),
)
total = query.order_by(None).count()
pages = max(1, (total + page_size - 1) // page_size)
shares = (
query.order_by(
CampaignShare.target_type.asc(),
CampaignShare.target_id.asc(),
CampaignShare.id.asc(),
)
.offset((page - 1) * page_size)
.limit(page_size)
.all()
)
return CampaignShareListResponse(
shares=[CampaignShareItem.model_validate(item) for item in shares],
total=total,
page=page,
page_size=page_size,
pages=pages,
)
@router.put("/{campaign_id}/owner", response_model=CampaignResponse)
def update_campaign_owner(
campaign_id: str,
payload: CampaignOwnerUpdateRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
):
del payload
_get_campaign_for_principal(
session,
campaign_id,
principal,
write=True,
)
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=(
"Direct campaign owner mutation has been retired. Use the "
"ownership transfer workflow so the target can accept."
),
)
@router.post(
"/{campaign_id}/shares",
response_model=CampaignShareItem,
status_code=status.HTTP_201_CREATED,
)
def upsert_campaign_share(
campaign_id: str,
payload: CampaignShareUpsertRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
):
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
directory = _access_directory()
if payload.target_type == "user":
target = directory.get_user(payload.target_id)
if target is not None and (
target.tenant_id != principal.tenant_id or target.status != "active"
):
target = None
else:
target = directory.get_group(payload.target_id)
if target is not None and (
target.tenant_id != principal.tenant_id or target.status != "active"
):
target = None
if target is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Share target not found"
)
share = (
session.query(CampaignShare)
.filter(
CampaignShare.campaign_id == campaign.id,
CampaignShare.target_type == payload.target_type,
CampaignShare.target_id == payload.target_id,
)
.one_or_none()
)
if share is None:
share = CampaignShare(
tenant_id=principal.tenant_id,
campaign_id=campaign.id,
target_type=payload.target_type,
target_id=payload.target_id,
permission=payload.permission,
created_by_user_id=principal.user.id,
)
else:
share.permission = payload.permission
share.revoked_at = None
session.add(share)
audit_from_principal(
session,
principal,
action="campaign.share_upserted",
object_type="campaign",
object_id=campaign.id,
details=payload.model_dump(),
commit=True,
)
return CampaignShareItem.model_validate(share)
@router.delete(
"/{campaign_id}/shares/{share_id}", status_code=status.HTTP_204_NO_CONTENT
)
def revoke_campaign_share(
campaign_id: str,
share_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
):
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
share = (
session.query(CampaignShare)
.filter(
CampaignShare.id == share_id,
CampaignShare.campaign_id == campaign.id,
CampaignShare.tenant_id == principal.tenant_id,
)
.one_or_none()
)
if share is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign share not found"
)
share.revoked_at = utc_now()
session.add(share)
audit_from_principal(
session,
principal,
action="campaign.share_revoked",
object_type="campaign",
object_id=campaign.id,
details={"share_id": share_id},
commit=True,
)
return None
# Queue / delivery control -------------------------------------------------

View File

@@ -0,0 +1,685 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from govoplan_campaign.backend.schemas import (
BuildCampaignRequest,
CampaignCreateResponse,
CampaignResponse,
CampaignVersionDetailResponse,
CampaignVersionResponse,
CampaignVersionSetStepRequest,
CampaignReviewStateRequest,
CampaignVersionUpdateRequest,
CampaignPartialValidationRequest,
CampaignPartialValidationResponse,
ValidateCampaignRequest,
)
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_campaign.backend.db.models import (
CampaignVersion,
)
from govoplan_core.db.session import get_session
from govoplan_campaign.backend.response_security import (
public_campaign_payload,
)
from govoplan_campaign.backend.persistence.campaigns import (
CampaignPersistenceError,
build_campaign_version,
validate_campaign_version,
)
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
from govoplan_campaign.backend.persistence.versions import (
LockedCampaignVersionError,
fork_campaign_version_for_edit,
is_version_final_locked,
is_user_locked_version,
get_campaign_version_for_tenant,
lock_campaign_version_temporarily,
permanently_lock_campaign_version,
publish_campaign_version,
unlock_user_locked_campaign_version,
unlock_validated_campaign_version,
update_campaign_version,
update_campaign_review_state,
validate_campaign_partial,
)
from govoplan_campaign.backend.route_support import (
_campaign_response_context,
_campaign_version_detail_response,
_get_campaign_for_principal,
_get_campaign_for_tenant,
_get_version_for_principal,
_get_version_for_tenant,
_require_mail_profile_use_if_needed,
_require_permission,
_update_campaign_version_detail_response,
_write_current_version_snapshot_if_available,
bounded_query_rows as _bounded_query_rows,
)
from govoplan_campaign.backend.routes.attachments import (
CampaignAttachmentLinkMatchesResponse,
_link_campaign_attachment_matches,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
@router.get("/{campaign_id}/versions", response_model=list[CampaignVersionResponse])
def list_versions(
campaign_id: str,
limit: int = Query(default=500, ge=1, le=1000),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
_get_campaign_for_principal(session, campaign_id, principal)
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
versions = _bounded_query_rows(
session.query(CampaignVersion)
.filter(CampaignVersion.campaign_id == campaign.id)
.order_by(CampaignVersion.version_number.desc()),
limit=limit,
label="Campaign version history",
)
return [
CampaignVersionResponse.model_validate(
item,
context=_campaign_response_context(principal),
)
for item in versions
]
@router.get(
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
)
def get_version_detail(
campaign_id: str,
version_id: str,
response: Response,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
):
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
try:
version = get_campaign_version_for_tenant(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
)
result = CampaignVersionDetailResponse.model_validate(
version,
context=_campaign_response_context(principal),
)
response.headers["ETag"] = version.strong_etag
return result
except CampaignPersistenceError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
@router.post(
"/{campaign_id}/versions/{version_id}/fork", response_model=CampaignCreateResponse
)
def fork_version_for_edit(
campaign_id: str,
version_id: str,
payload: CampaignVersionUpdateRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:copy")),
):
"""Create the campaign's next and only editable working version.
A new working copy may be created only after the current version is
permanently user-locked or delivery-final. Validation and temporary user
locks must be removed in place instead of creating parallel drafts.
"""
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or CampaignVersionUpdateRequest()
source_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if source_version.campaign_id != campaign_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
effective_json = (
payload.campaign_json
if isinstance(payload.campaign_json, dict)
else source_version.raw_json
)
_require_mail_profile_use_if_needed(
principal,
effective_json if isinstance(effective_json, dict) else {},
)
try:
version = fork_campaign_version_for_edit(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
raw_json=payload.campaign_json,
current_flow=payload.current_flow or "manual",
current_step=payload.current_step,
editor_state=payload.editor_state,
source_filename=payload.source_filename,
source_base_path=payload.source_base_path,
autosave=True,
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
commit=False,
)
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
audit_from_principal(
session,
principal,
action="campaign.version_forked_for_edit",
object_type="campaign_version",
object_id=version.id,
details={
"campaign_id": campaign_id,
"source_version_id": version_id,
"version_number": version.version_number,
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
},
commit=True,
)
_write_current_version_snapshot_if_available(version)
return CampaignCreateResponse(
campaign=CampaignResponse.model_validate(campaign),
version=CampaignVersionResponse.model_validate(
version,
context=_campaign_response_context(principal),
),
)
except LockedCampaignVersionError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
) from exc
except CampaignPathSecurityError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except CampaignPersistenceError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except Exception:
session.rollback()
raise
@router.post(
"/{campaign_id}/versions/{version_id}/unlock-validation",
response_model=CampaignVersionDetailResponse,
)
def unlock_version_validation(
campaign_id: str,
version_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
"""Unlock a successfully validated version before delivery starts.
Unlocking invalidates validation/build state and removes generated jobs for
that version. Sent/final versions cannot be unlocked and must be copied.
"""
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: unlock_validated_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
commit=False,
),
audit_action="campaign.version_validation_unlocked",
)
@router.post(
"/{campaign_id}/versions/{version_id}/lock-temporarily",
response_model=CampaignVersionDetailResponse,
)
def lock_version_temporarily(
campaign_id: str,
version_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: lock_campaign_version_temporarily(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
user_id=principal.user.id,
commit=False,
),
audit_action="campaign.version_user_locked_temporarily",
)
@router.post(
"/{campaign_id}/versions/{version_id}/unlock-user-lock",
response_model=CampaignVersionDetailResponse,
)
def unlock_version_user_lock(
campaign_id: str,
version_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: unlock_user_locked_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
commit=False,
),
audit_action="campaign.version_user_lock_removed",
)
@router.post(
"/{campaign_id}/versions/{version_id}/lock-permanently",
response_model=CampaignVersionDetailResponse,
)
def lock_version_permanently(
campaign_id: str,
version_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: permanently_lock_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
user_id=principal.user.id,
commit=False,
),
audit_action="campaign.version_user_locked_permanently",
)
@router.put(
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
)
def update_version_detail(
campaign_id: str,
version_id: str,
payload: CampaignVersionUpdateRequest,
response: Response,
if_match: str | None = Header(default=None, alias="If-Match"),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
result = _update_campaign_version_detail_response(
session,
principal,
campaign_id,
version_id,
payload,
if_match=if_match,
autosave=False,
audit_action="campaign.version_updated",
)
response.headers["ETag"] = result.strong_etag
return result
@router.post(
"/{campaign_id}/versions/{version_id}/autosave",
response_model=CampaignVersionDetailResponse,
)
def autosave_version(
campaign_id: str,
version_id: str,
payload: CampaignVersionUpdateRequest,
response: Response,
if_match: str | None = Header(default=None, alias="If-Match"),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
result = _update_campaign_version_detail_response(
session,
principal,
campaign_id,
version_id,
payload,
if_match=if_match,
autosave=True,
audit_action="campaign.version_autosaved",
)
response.headers["ETag"] = result.strong_etag
return result
@router.post(
"/{campaign_id}/versions/{version_id}/set-step",
response_model=CampaignVersionDetailResponse,
)
def set_version_step(
campaign_id: str,
version_id: str,
payload: CampaignVersionSetStepRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: update_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
current_flow=payload.current_flow,
current_step=payload.current_step,
autosave=True,
commit=False,
),
audit_action="campaign.version_step_updated",
details={
"campaign_id": campaign_id,
"current_flow": payload.current_flow,
"current_step": payload.current_step,
},
)
@router.post(
"/{campaign_id}/versions/{version_id}/review-state",
response_model=CampaignVersionDetailResponse,
)
def set_version_review_state(
campaign_id: str,
version_id: str,
payload: CampaignReviewStateRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:review")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
try:
version = update_campaign_review_state(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
inspection_complete=payload.inspection_complete,
reviewed_message_keys=payload.reviewed_message_keys,
issue_decisions=[
item.model_dump()
for item in payload.issue_decisions
],
user_id=principal.user.id,
commit=False,
)
audit_from_principal(
session,
principal,
action="campaign.message_review_updated",
object_type="campaign_version",
object_id=version.id,
details={
"campaign_id": campaign_id,
"inspection_complete": payload.inspection_complete,
"reviewed_message_count": len(payload.reviewed_message_keys),
"issue_decision_count": len(payload.issue_decisions),
},
commit=True,
)
return CampaignVersionDetailResponse.model_validate(
version,
context=_campaign_response_context(principal),
)
except LockedCampaignVersionError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
) from exc
except CampaignPersistenceError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except Exception:
session.rollback()
raise
@router.post(
"/{campaign_id}/versions/{version_id}/validate-partial",
response_model=CampaignPartialValidationResponse,
)
def validate_version_partial(
campaign_id: str,
version_id: str,
payload: CampaignPartialValidationRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
try:
version = get_campaign_version_for_tenant(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
)
campaign_json = (
payload.campaign_json
if payload and payload.campaign_json is not None
else version.raw_json
)
result = validate_campaign_partial(
campaign_json, section=payload.section if payload else None
)
audit_from_principal(
session,
principal,
action="campaign.version_partially_validated",
object_type="campaign_version",
object_id=version.id,
details={
"campaign_id": campaign_id,
"section": result.get("section"),
"ok": result.get("ok"),
},
commit=True,
)
return CampaignPartialValidationResponse(**result)
except CampaignPersistenceError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
@router.post(
"/{campaign_id}/versions/{version_id}/publish",
response_model=CampaignVersionDetailResponse,
)
def publish_version(
campaign_id: str,
version_id: str,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
return _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: publish_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
user_id=principal.user.id,
commit=False,
),
audit_action="campaign.version_user_locked_permanently",
)
@router.post("/versions/{version_id}/validate")
def validate_version(
version_id: str,
payload: ValidateCampaignRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
):
_get_version_for_principal(session, version_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
payload = payload or ValidateCampaignRequest()
try:
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
_require_mail_profile_use_if_needed(
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
)
if is_user_locked_version(version) or is_version_final_locked(version):
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="This version has a user lock or final delivery lock and cannot be validated. Remove a temporary lock or create an editable copy.",
)
link_result: CampaignAttachmentLinkMatchesResponse | None = None
if payload.check_files and payload.link_unshared_matches:
_require_permission(principal, "files:file:share")
campaign = _get_campaign_for_tenant(
session, version.campaign_id, principal.tenant_id
)
link_result = _link_campaign_attachment_matches(
session,
principal,
campaign=campaign,
version=version,
raw=version.raw_json if isinstance(version.raw_json, dict) else {},
dry_run=False,
)
audit_from_principal(
session,
principal,
action="campaign.attachment_matches_linked",
object_type="campaign_version",
object_id=version_id,
details={
"matched_file_count": link_result.matched_file_count,
"already_linked_file_count": link_result.already_linked_file_count,
"linked_file_count": link_result.linked_file_count,
"during_validation": True,
},
commit=True,
)
result = validate_campaign_version(
session,
tenant_id=principal.tenant_id,
version_id=version_id,
check_files=payload.check_files,
user_id=principal.user.id,
)
audit_from_principal(
session,
principal,
action="campaign.validated",
object_type="campaign_version",
object_id=version_id,
details={
"check_files": payload.check_files,
"link_unshared_matches": payload.link_unshared_matches,
"linked_file_count": link_result.linked_file_count
if link_result
else 0,
"ok": result.get("ok"),
},
commit=True,
)
return public_campaign_payload(
result,
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
)
except HTTPException:
raise
except CampaignPersistenceError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
@router.post("/versions/{version_id}/build")
def build_version(
version_id: str,
payload: BuildCampaignRequest | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:build")),
):
version = _get_version_for_principal(session, version_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
_require_mail_profile_use_if_needed(
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
)
try:
result = build_campaign_version(
session,
tenant_id=principal.tenant_id,
version_id=version_id,
write_eml=payload.write_eml if payload else True,
)
audit_from_principal(
session,
principal,
action="campaign.messages_built",
object_type="campaign_version",
object_id=version_id,
details={
"write_eml": payload.write_eml if payload else True,
"built_count": result.get("built_count"),
},
commit=True,
)
return public_campaign_payload(
result,
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
)
except CampaignPersistenceError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc

View File

@@ -60,7 +60,9 @@
"integer", "integer",
"double", "double",
"date", "date",
"password" "password",
"organization_unit",
"organization_function"
], ],
"default": "string" "default": "string"
}, },
@@ -92,7 +94,27 @@
"mail_profile_id": { "mail_profile_id": {
"type": "string", "type": "string",
"minLength": 1, "minLength": 1,
"description": "Stable reference to an authorized profile owned by the Mail module. Campaign JSON never stores SMTP/IMAP settings or credentials." "description": "Stable reference to an authorized server envelope owned by the Mail module. Campaign JSON never stores SMTP/IMAP settings or credentials."
},
"smtp_server_id": {
"type": "string",
"minLength": 1,
"description": "Optional explicit Mail-owned SMTP server selection."
},
"smtp_credential_id": {
"type": "string",
"minLength": 1,
"description": "Optional explicit core credential envelope bound to the selected SMTP server."
},
"imap_server_id": {
"type": "string",
"minLength": 1,
"description": "Optional explicit Mail-owned IMAP server selection."
},
"imap_credential_id": {
"type": "string",
"minLength": 1,
"description": "Optional explicit core credential envelope bound to the selected IMAP server."
} }
}, },
"additionalProperties": false "additionalProperties": false
@@ -337,7 +359,7 @@
"continue", "continue",
"warn" "warn"
], ],
"default": "ask" "default": "warn"
}, },
"ambiguous_behavior": { "ambiguous_behavior": {
"type": "string", "type": "string",
@@ -433,7 +455,7 @@
"continue", "continue",
"warn" "warn"
], ],
"default": "ask" "default": "block"
}, },
"missing_optional_attachment": { "missing_optional_attachment": {
"type": "string", "type": "string",
@@ -509,6 +531,12 @@
"delivery": { "delivery": {
"type": "object", "type": "object",
"properties": { "properties": {
"channel_policy": {
"$ref": "#/$defs/delivery_channel_policy"
},
"postbox": {
"$ref": "#/$defs/postbox_delivery"
},
"rate_limit": { "rate_limit": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -626,6 +654,179 @@
}, },
"additionalProperties": false "additionalProperties": false
}, },
"delivery_channel_policy": {
"type": "string",
"enum": [
"mail",
"postbox",
"mail_and_postbox",
"mail_then_postbox",
"postbox_then_mail"
],
"default": "mail"
},
"postbox_target": {
"type": "object",
"required": [
"id",
"mode"
],
"properties": {
"id": {
"type": "string",
"minLength": 1,
"maxLength": 120
},
"mode": {
"type": "string",
"enum": [
"direct",
"derived"
]
},
"label": {
"type": [
"string",
"null"
],
"maxLength": 500
},
"postbox_id": {
"type": [
"string",
"null"
],
"maxLength": 36
},
"address_key": {
"type": [
"string",
"null"
],
"maxLength": 500
},
"template_id": {
"type": [
"string",
"null"
],
"maxLength": 36
},
"organization_unit_id": {
"type": [
"string",
"null"
],
"maxLength": 36
},
"organization_unit_field": {
"type": [
"string",
"null"
],
"maxLength": 255
},
"organization_unit_match": {
"type": "string",
"enum": [
"id",
"slug"
],
"default": "id"
},
"function_id": {
"type": [
"string",
"null"
],
"maxLength": 36
},
"function_field": {
"type": [
"string",
"null"
],
"maxLength": 255
},
"function_match": {
"type": "string",
"enum": [
"id",
"slug"
],
"default": "id"
},
"context_key": {
"type": [
"string",
"null"
],
"maxLength": 255
},
"context_field": {
"type": [
"string",
"null"
],
"maxLength": 255
}
},
"additionalProperties": false
},
"postbox_delivery": {
"type": "object",
"properties": {
"targets": {
"type": "array",
"items": {
"$ref": "#/$defs/postbox_target"
},
"maxItems": 50,
"default": []
},
"classification": {
"type": "string",
"minLength": 1,
"maxLength": 50,
"default": "internal"
},
"unresolved_target": {
"type": "string",
"enum": [
"block",
"ask",
"drop",
"continue",
"warn"
],
"default": "block"
},
"vacant_target": {
"type": "string",
"enum": [
"block",
"ask",
"drop",
"continue",
"warn"
],
"default": "warn"
},
"duplicate_target": {
"type": "string",
"enum": [
"block",
"ask",
"drop",
"continue",
"warn"
],
"default": "warn"
}
},
"additionalProperties": false,
"default": {}
},
"attachment_config": { "attachment_config": {
"type": "object", "type": "object",
"required": [ "required": [
@@ -869,6 +1070,29 @@
"default": true, "default": true,
"description": "Deprecated compatibility alias for merge_disposition_notification_to. New campaign JSON should use merge_*." "description": "Deprecated compatibility alias for merge_disposition_notification_to. New campaign JSON should use merge_*."
}, },
"channel_policy": {
"oneOf": [
{
"$ref": "#/$defs/delivery_channel_policy"
},
{
"type": "null"
}
],
"default": null
},
"postbox_targets": {
"type": "array",
"items": {
"$ref": "#/$defs/postbox_target"
},
"maxItems": 50,
"default": []
},
"merge_postbox_targets": {
"type": "boolean",
"default": true
},
"attachments": { "attachments": {
"type": "array", "type": "array",
"items": { "items": {

View File

@@ -1,9 +1,9 @@
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import datetime
from typing import Any, Literal from typing import Annotated, Any, Literal
from pydantic import BaseModel, ConfigDict, Field, ValidationInfo, field_validator, model_validator from pydantic import BaseModel, BeforeValidator, ConfigDict, Field, ValidationInfo, field_validator, model_validator
from govoplan_core.api.v1.schemas import DeltaDeletedItem from govoplan_core.api.v1.schemas import DeltaDeletedItem
from govoplan_campaign.backend.campaign.mail_profile_boundary import ( from govoplan_campaign.backend.campaign.mail_profile_boundary import (
@@ -58,6 +58,9 @@ class CampaignVersionUpdateRequest(BaseModel):
source_filename: str | None = None source_filename: str | None = None
source_base_path: str | None = None source_base_path: str | None = None
migrate_legacy_mail_settings: bool = False migrate_legacy_mail_settings: bool = False
base_revision: int | None = Field(default=None, ge=1)
reconciliation_kind: Literal["none", "auto_merge", "manual"] = "none"
resolved_conflict_paths: list[str] = Field(default_factory=list, max_length=100)
@field_validator("editor_state") @field_validator("editor_state")
@classmethod @classmethod
@@ -72,11 +75,23 @@ class CampaignVersionSetStepRequest(BaseModel):
current_step: str current_step: str
class CampaignReviewDecisionRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
job_id: str = Field(min_length=1, max_length=36)
decision: Literal["accept"] = "accept"
reason: str | None = Field(default=None, max_length=4_000)
class CampaignReviewStateRequest(BaseModel): class CampaignReviewStateRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
inspection_complete: bool = False inspection_complete: bool = False
reviewed_message_keys: list[str] = Field(default_factory=list) reviewed_message_keys: list[str] = Field(default_factory=list)
issue_decisions: list[CampaignReviewDecisionRequest] = Field(
default_factory=list,
max_length=100_000,
)
class CampaignPartialValidationRequest(BaseModel): class CampaignPartialValidationRequest(BaseModel):
@@ -92,6 +107,8 @@ class CampaignVersionResponse(BaseModel):
id: str id: str
campaign_id: str campaign_id: str
version_number: int version_number: int
edit_revision: int = 1
strong_etag: str = ""
schema_version: str schema_version: str
source_filename: str | None = None source_filename: str | None = None
workflow_state: str = "editing" workflow_state: str = "editing"
@@ -112,6 +129,8 @@ class CampaignVersionResponse(BaseModel):
build_summary: dict[str, Any] | None = None build_summary: dict[str, Any] | None = None
execution_snapshot_hash: str | None = None execution_snapshot_hash: str | None = None
execution_snapshot_at: datetime | None = None execution_snapshot_at: datetime | None = None
delivery_mode: Literal["synchronous", "worker_queue", "database_queue"] | None = None
delivery_mode_selected_at: datetime | None = None
@field_validator("editor_state", mode="before") @field_validator("editor_state", mode="before")
@classmethod @classmethod
@@ -192,6 +211,10 @@ class CampaignDeltaResponse(BaseModel):
watermark: str | None = None watermark: str | None = None
has_more: bool = False has_more: bool = False
full: bool = False full: bool = False
total: int = 0
page: int = 1
page_size: int = 500
pages: int = 1
class CampaignWorkspaceDeltaResponse(CampaignWorkspaceResponse): class CampaignWorkspaceDeltaResponse(CampaignWorkspaceResponse):
@@ -214,6 +237,10 @@ class CampaignShareItem(BaseModel):
class CampaignShareListResponse(BaseModel): class CampaignShareListResponse(BaseModel):
shares: list[CampaignShareItem] shares: list[CampaignShareItem]
total: int = 0
page: int = 1
page_size: int = 500
pages: int = 1
class CampaignShareTargetItem(BaseModel): class CampaignShareTargetItem(BaseModel):
@@ -340,6 +367,13 @@ class CampaignRecipientAddressSourcesResponse(BaseModel):
sources: list[CampaignRecipientAddressSource] = Field(default_factory=list) sources: list[CampaignRecipientAddressSource] = Field(default_factory=list)
class CampaignPostboxCatalogResponse(BaseModel):
available: bool = False
postboxes: list[dict[str, Any]] = Field(default_factory=list)
templates: list[dict[str, Any]] = Field(default_factory=list)
organization_units: list[dict[str, Any]] = Field(default_factory=list)
class CampaignRecipientAddressSourceSnapshotRequest(BaseModel): class CampaignRecipientAddressSourceSnapshotRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
@@ -423,11 +457,21 @@ class CampaignSendUnattemptedRequest(BaseModel):
class CampaignSendJobRequest(BaseModel): class CampaignSendJobRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
kind: Literal["test", "single_send", "single_resend"]
idempotency_key: str = Field(min_length=1, max_length=200)
reason: str | None = Field(default=None, max_length=2000)
context: dict[str, Any] = Field(default_factory=dict)
include_warnings: bool = True include_warnings: bool = True
dry_run: bool = False
use_rate_limit: bool = True use_rate_limit: bool = True
enqueue_imap_task: bool = False enqueue_imap_task: bool = False
@model_validator(mode="after")
def require_resend_reason(self):
self.reason = (self.reason or "").strip() or None
if self.kind == "single_resend" and not self.reason:
raise ValueError("single_resend requires a reason")
return self
class CampaignResolveOutcomeRequest(BaseModel): class CampaignResolveOutcomeRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
@@ -437,8 +481,11 @@ class CampaignResolveOutcomeRequest(BaseModel):
"not_sent", "not_sent",
"imap_appended", "imap_appended",
"imap_not_appended", "imap_not_appended",
"postbox_accepted",
"postbox_not_accepted",
] ]
note: str | None = Field(default=None, max_length=2000) note: str | None = Field(default=None, max_length=2000)
attempt_id: str | None = Field(default=None, max_length=36)
@model_validator(mode="after") @model_validator(mode="after")
def require_reconciliation_evidence(self) -> "CampaignResolveOutcomeRequest": def require_reconciliation_evidence(self) -> "CampaignResolveOutcomeRequest":
@@ -492,6 +539,8 @@ class QueueCampaignResponse(BaseModel):
skipped_count: int skipped_count: int
blocked_count: int blocked_count: int
enqueued_count: int enqueued_count: int
delivery_mode: str = "worker_queue"
worker_queue_available: bool = False
dry_run: bool = False dry_run: bool = False
@@ -512,6 +561,14 @@ class SendCampaignNowResponse(BaseModel):
result: dict[str, Any] result: dict[str, Any]
class CampaignDeliveryOptionsResponse(BaseModel):
campaign_id: str
version_id: str
worker_queue_available: bool
postbox_available: bool = False
synchronous_send: dict[str, Any] = Field(default_factory=dict)
class MockCampaignSendRequest(BaseModel): class MockCampaignSendRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
@@ -540,62 +597,68 @@ class CampaignActionResponse(BaseModel):
result: dict[str, Any] result: dict[str, Any]
def _valid_report_email_domain(domain: str) -> bool:
if not domain or domain.startswith(".") or domain.endswith(".") or ".." in domain:
return False
return all(
label
and not label.startswith("-")
and not label.endswith("-")
and all(character.isalnum() or character == "-" for character in label)
for label in domain.split(".")
)
def _normalize_report_recipient(value: Any) -> str:
if not isinstance(value, str):
raise ValueError("report recipients must be email-address strings")
recipient = value.strip()
if len(recipient) > 320:
raise ValueError("report recipient addresses must be at most 320 characters")
if any(ord(character) < 32 or ord(character) == 127 for character in recipient):
raise ValueError("report recipient addresses must not contain control characters")
if recipient.count("@") != 1:
raise ValueError("report recipients must be email addresses")
local, domain = recipient.split("@", 1)
invalid_local = not local or local.startswith(".") or local.endswith(".") or ".." in local
invalid_address = (
any(character.isspace() for character in recipient)
or any(character in ',;:<>[]()\\"' for character in recipient)
)
if invalid_local or invalid_address or not _valid_report_email_domain(domain):
raise ValueError("report recipients must be email addresses")
return recipient
ReportEmailAddress = Annotated[str, BeforeValidator(_normalize_report_recipient)]
def _deduplicate_report_recipients(value: list[str]) -> list[str]:
recipients: list[str] = []
seen: set[str] = set()
for recipient in value:
key = recipient.casefold()
if key not in seen:
seen.add(key)
recipients.append(recipient)
return recipients
class ReportEmailRequest(BaseModel): class ReportEmailRequest(BaseModel):
model_config = ConfigDict(extra="forbid") model_config = ConfigDict(extra="forbid")
to: list[str] = Field(min_length=1, max_length=50) to: list[ReportEmailAddress] = Field(min_length=1, max_length=50)
version_id: str | None = None version_id: str | None = None
include_jobs: bool = False include_jobs: bool = False
attach_jobs_csv: bool = False attach_jobs_csv: bool = False
attach_report_json: bool = False attach_report_json: bool = False
dry_run: bool = False dry_run: bool = False
idempotency_key: str | None = Field(default=None, min_length=1, max_length=200)
@field_validator("to", mode="before") @field_validator("to")
@classmethod @classmethod
def normalize_and_validate_recipients(cls, value: Any) -> Any: def normalize_and_validate_recipients(cls, value: list[str]) -> list[str]:
if not isinstance(value, list): return _deduplicate_report_recipients(value)
return value
if not 1 <= len(value) <= 50:
raise ValueError("report email requires between 1 and 50 recipients")
recipients: list[str] = []
seen: set[str] = set()
for item in value:
if not isinstance(item, str):
raise ValueError("report recipients must be email-address strings")
recipient = item.strip()
if len(recipient) > 320:
raise ValueError("report recipient addresses must be at most 320 characters")
if any(ord(character) < 32 or ord(character) == 127 for character in recipient):
raise ValueError("report recipient addresses must not contain control characters")
if recipient.count("@") != 1:
raise ValueError("report recipients must be email addresses")
local, domain = recipient.split("@", 1)
if (
not local
or not domain
or any(character.isspace() for character in recipient)
or any(character in ',;:<>[]()\\"' for character in recipient)
or local.startswith(".")
or local.endswith(".")
or ".." in local
or domain.startswith(".")
or domain.endswith(".")
or ".." in domain
or any(
not label
or label.startswith("-")
or label.endswith("-")
or not all(character.isalnum() or character == "-" for character in label)
for label in domain.split(".")
)
):
raise ValueError("report recipients must be email addresses")
key = recipient.casefold()
if key in seen:
continue
seen.add(key)
recipients.append(recipient)
return recipients
class ReportEmailResponse(BaseModel): class ReportEmailResponse(BaseModel):

View File

@@ -9,16 +9,21 @@ from pydantic import BaseModel, ConfigDict
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion, JobValidationStatus from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion, JobValidationStatus
from govoplan_campaign.backend.campaign.models import DeliveryConfig from govoplan_campaign.backend.campaign.models import (
DeliveryChannelPolicy,
DeliveryConfig,
)
from govoplan_campaign.backend.campaign.mail_profile_boundary import ( from govoplan_campaign.backend.campaign.mail_profile_boundary import (
CampaignMailProfileBoundaryError, CampaignMailProfileBoundaryError,
assert_campaign_uses_mail_profile_reference, assert_campaign_uses_mail_profile_reference,
campaign_mail_profile_id, campaign_mail_profile_id,
campaign_mail_resource_ids,
) )
from govoplan_campaign.backend.integrations import MailProfileError, files_integration, mail_integration from govoplan_campaign.backend.integrations import MailProfileError, files_integration, mail_integration
from govoplan_campaign.backend.path_security import CampaignPathSecurityError, assert_server_safe_campaign_paths from govoplan_campaign.backend.path_security import CampaignPathSecurityError, assert_server_safe_campaign_paths
SNAPSHOT_VERSION = "5" SNAPSHOT_VERSION = "7"
SUPPORTED_SNAPSHOT_VERSIONS = {"6", SNAPSHOT_VERSION}
class ExecutionSnapshotError(RuntimeError): class ExecutionSnapshotError(RuntimeError):
@@ -40,7 +45,11 @@ class ExecutionSnapshot(BaseModel):
snapshot_version: str = SNAPSHOT_VERSION snapshot_version: str = SNAPSHOT_VERSION
campaign_version_id: str campaign_version_id: str
campaign_json_sha256: str campaign_json_sha256: str
mail_profile_id: str mail_profile_id: str | None = None
smtp_server_id: str | None = None
smtp_credential_id: str | None = None
imap_server_id: str | None = None
imap_credential_id: str | None = None
created_at: str created_at: str
build_token: str | None = None build_token: str | None = None
built_at: str | None = None built_at: str | None = None
@@ -50,6 +59,8 @@ class ExecutionSnapshot(BaseModel):
effective_policy_sha256: str | None = None effective_policy_sha256: str | None = None
smtp_transport_revision: str | None = None smtp_transport_revision: str | None = None
imap_transport_revision: str | None = None imap_transport_revision: str | None = None
uses_mail: bool = True
uses_postbox: bool = False
delivery: DeliveryConfig delivery: DeliveryConfig
@@ -67,7 +78,7 @@ def snapshot_hash(payload: dict[str, Any]) -> str:
def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict[str, Any]: def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict[str, Any]:
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {} raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
_assert_version_mail_profile_boundary(raw_json) _assert_version_mail_profile_boundary(raw_json, require_profile=True)
mail = mail_integration() mail = mail_integration()
profile_id = campaign_mail_profile_id(raw_json) profile_id = campaign_mail_profile_id(raw_json)
if profile_id is None: # Kept explicit for static typing; the assertion above requires it. if profile_id is None: # Kept explicit for static typing; the assertion above requires it.
@@ -75,12 +86,17 @@ def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict
campaign = session.get(Campaign, version.campaign_id) campaign = session.get(Campaign, version.campaign_id)
if campaign is None: if campaign is None:
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution") raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
references = campaign_mail_resource_ids(raw_json)
try: try:
return mail.campaign_profile_delivery_summary( return mail.campaign_profile_delivery_summary(
session, session,
tenant_id=campaign.tenant_id, tenant_id=campaign.tenant_id,
campaign_id=campaign.id, campaign_id=campaign.id,
profile_id=profile_id, profile_id=profile_id,
smtp_server_id=references["smtp_server_id"],
smtp_credential_id=references["smtp_credential_id"],
imap_server_id=references["imap_server_id"],
imap_credential_id=references["imap_credential_id"],
) )
except MailProfileError as exc: except MailProfileError as exc:
raise ExecutionSnapshotError(str(exc)) from exc raise ExecutionSnapshotError(str(exc)) from exc
@@ -96,27 +112,61 @@ def profile_transport_revisions(session: Session, version: CampaignVersion) -> d
def _assert_snapshot_profile_matches_version(version: CampaignVersion, snapshot: ExecutionSnapshot) -> None: def _assert_snapshot_profile_matches_version(version: CampaignVersion, snapshot: ExecutionSnapshot) -> None:
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {} raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
_assert_version_mail_profile_boundary(raw_json) _assert_version_mail_profile_boundary(
raw_json,
require_profile=snapshot.uses_mail,
)
if not snapshot.uses_mail:
return
if campaign_mail_profile_id(raw_json) != snapshot.mail_profile_id: if campaign_mail_profile_id(raw_json) != snapshot.mail_profile_id:
raise ExecutionSnapshotError( raise ExecutionSnapshotError(
"The campaign's Mail profile reference differs from the built execution snapshot. " "The campaign's Mail profile reference differs from the built execution snapshot. "
"Revalidate and rebuild the campaign before delivery." "Revalidate and rebuild the campaign before delivery."
) )
references = campaign_mail_resource_ids(raw_json)
for key in (
"smtp_server_id",
"smtp_credential_id",
"imap_server_id",
"imap_credential_id",
):
configured = references[key]
if configured and configured != getattr(snapshot, key):
raise ExecutionSnapshotError(
"The campaign's Mail server or credential selection differs from the built "
"execution snapshot. Revalidate and rebuild before delivery."
)
def _assert_version_mail_profile_boundary(raw_json: dict[str, Any]) -> None: def _assert_version_mail_profile_boundary(
raw_json: dict[str, Any],
*,
require_profile: bool,
) -> None:
try: try:
assert_campaign_uses_mail_profile_reference(raw_json, require_profile=True) assert_campaign_uses_mail_profile_reference(
raw_json,
require_profile=require_profile,
)
except CampaignMailProfileBoundaryError as exc: except CampaignMailProfileBoundaryError as exc:
raise ExecutionSnapshotError(str(exc)) from exc raise ExecutionSnapshotError(str(exc)) from exc
def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> str: def _policy_fingerprint(
raw_json: dict[str, Any],
delivery: DeliveryConfig,
*,
snapshot_version: str = SNAPSHOT_VERSION,
) -> str:
delivery_payload = delivery.model_dump(mode="json")
if snapshot_version == "6":
delivery_payload.pop("channel_policy", None)
delivery_payload.pop("postbox", None)
return _sha256( return _sha256(
{ {
"validation_policy": raw_json.get("validation_policy"), "validation_policy": raw_json.get("validation_policy"),
"policy": raw_json.get("policy"), "policy": raw_json.get("policy"),
"delivery": delivery.model_dump(mode="json"), "delivery": delivery_payload,
"attachment_defaults": (raw_json.get("attachments") or {}).get("defaults") "attachment_defaults": (raw_json.get("attachments") or {}).get("defaults")
if isinstance(raw_json.get("attachments"), dict) if isinstance(raw_json.get("attachments"), dict)
else None, else None,
@@ -124,8 +174,12 @@ def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> s
) )
def _job_execution_input_payload(job: CampaignJob) -> dict[str, Any]: def _job_execution_input_payload(
return { job: CampaignJob,
*,
snapshot_version: str = SNAPSHOT_VERSION,
) -> dict[str, Any]:
payload = {
"job_id": job.id, "job_id": job.id,
"entry_index": job.entry_index, "entry_index": job.entry_index,
"entry_id": job.entry_id, "entry_id": job.entry_id,
@@ -140,17 +194,47 @@ def _job_execution_input_payload(job: CampaignJob) -> dict[str, Any]:
"resolved_attachments_sha256": _sha256(job.resolved_attachments or []), "resolved_attachments_sha256": _sha256(job.resolved_attachments or []),
"issues_sha256": _sha256(job.issues_snapshot or []), "issues_sha256": _sha256(job.issues_snapshot or []),
} }
if snapshot_version != "6":
payload.update(
{
"delivery_channel_policy": getattr(
job,
"delivery_channel_policy",
DeliveryChannelPolicy.MAIL.value,
),
"resolved_postbox_targets_sha256": _sha256(
getattr(job, "resolved_postbox_targets", None) or []
),
}
)
return payload
def job_execution_input_hash(job: CampaignJob) -> str: def job_execution_input_hash(
return _sha256(_job_execution_input_payload(job)) job: CampaignJob,
*,
snapshot_version: str = SNAPSHOT_VERSION,
) -> str:
return _sha256(
_job_execution_input_payload(
job,
snapshot_version=snapshot_version,
)
)
def job_manifest_hash(jobs: Iterable[CampaignJob]) -> str: def job_manifest_hash(
jobs: Iterable[CampaignJob],
*,
snapshot_version: str = SNAPSHOT_VERSION,
) -> str:
"""Hash the immutable per-message execution records in stable order.""" """Hash the immutable per-message execution records in stable order."""
payload = [ payload = [
_job_execution_input_payload(job) _job_execution_input_payload(
job,
snapshot_version=snapshot_version,
)
for job in sorted(jobs, key=lambda item: (item.entry_index, item.id)) for job in sorted(jobs, key=lambda item: (item.entry_index, item.id))
] ]
return _sha256(payload) return _sha256(payload)
@@ -159,31 +243,67 @@ def job_manifest_hash(jobs: Iterable[CampaignJob]) -> str:
def create_execution_snapshot( def create_execution_snapshot(
version: CampaignVersion, version: CampaignVersion,
*, *,
mail_profile_id: str, mail_profile_id: str | None,
smtp_transport_revision: str, smtp_transport_revision: str | None,
imap_transport_revision: str | None, imap_transport_revision: str | None,
delivery: DeliveryConfig, delivery: DeliveryConfig,
smtp_server_id: str | None = None,
smtp_credential_id: str | None = None,
imap_server_id: str | None = None,
imap_credential_id: str | None = None,
jobs: Iterable[CampaignJob] = (), jobs: Iterable[CampaignJob] = (),
build_summary: dict[str, Any] | None = None, build_summary: dict[str, Any] | None = None,
) -> tuple[dict[str, Any], str]: ) -> tuple[dict[str, Any], str]:
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {} raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
job_list = list(jobs) job_list = list(jobs)
channel_policies = {
DeliveryChannelPolicy(
getattr(
job,
"delivery_channel_policy",
DeliveryChannelPolicy.MAIL.value,
)
)
for job in job_list
}
uses_mail = any(policy.uses_mail for policy in channel_policies)
uses_postbox = any(policy.uses_postbox for policy in channel_policies)
for job in job_list: for job in job_list:
job.execution_input_sha256 = job_execution_input_hash(job) job.execution_input_sha256 = job_execution_input_hash(
job,
snapshot_version=SNAPSHOT_VERSION,
)
summary = build_summary if isinstance(build_summary, dict) else {} summary = build_summary if isinstance(build_summary, dict) else {}
queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value} queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value}
payload = ExecutionSnapshot( payload = ExecutionSnapshot(
campaign_version_id=version.id, campaign_version_id=version.id,
campaign_json_sha256=_sha256(raw_json), campaign_json_sha256=_sha256(raw_json),
mail_profile_id=mail_profile_id, mail_profile_id=mail_profile_id,
smtp_server_id=smtp_server_id,
smtp_credential_id=smtp_credential_id,
imap_server_id=imap_server_id,
imap_credential_id=imap_credential_id,
build_token=str(summary.get("build_token") or "") or None, build_token=str(summary.get("build_token") or "") or None,
built_at=str(summary.get("built_at") or "") or None, built_at=str(summary.get("built_at") or "") or None,
job_count=len(job_list), job_count=len(job_list),
queueable_job_count=sum(1 for job in job_list if job.validation_status in queueable_statuses), queueable_job_count=sum(1 for job in job_list if job.validation_status in queueable_statuses),
job_manifest_sha256=job_manifest_hash(job_list) if job_list else None, job_manifest_sha256=(
effective_policy_sha256=_policy_fingerprint(raw_json, delivery), job_manifest_hash(
job_list,
snapshot_version=SNAPSHOT_VERSION,
)
if job_list
else None
),
effective_policy_sha256=_policy_fingerprint(
raw_json,
delivery,
snapshot_version=SNAPSHOT_VERSION,
),
smtp_transport_revision=smtp_transport_revision, smtp_transport_revision=smtp_transport_revision,
imap_transport_revision=imap_transport_revision, imap_transport_revision=imap_transport_revision,
uses_mail=uses_mail,
uses_postbox=uses_postbox,
created_at=datetime.now(timezone.utc).isoformat(), created_at=datetime.now(timezone.utc).isoformat(),
delivery=delivery, delivery=delivery,
).model_dump(mode="json") ).model_dump(mode="json")
@@ -207,13 +327,17 @@ def _assert_snapshot_matches_persisted_inputs(
"Campaign inputs changed after this execution snapshot was built. " "Campaign inputs changed after this execution snapshot was built. "
"Revalidate and rebuild the campaign before delivery." "Revalidate and rebuild the campaign before delivery."
) )
if not snapshot.smtp_transport_revision: if snapshot.uses_mail and not snapshot.smtp_transport_revision:
raise ExecutionSnapshotError("Execution snapshot has no SMTP transport revision") raise ExecutionSnapshotError("Execution snapshot has no SMTP transport revision")
if not snapshot.job_manifest_sha256: if not snapshot.job_manifest_sha256:
raise ExecutionSnapshotError("Execution snapshot has no built-job manifest checksum") raise ExecutionSnapshotError("Execution snapshot has no built-job manifest checksum")
if not snapshot.effective_policy_sha256: if not snapshot.effective_policy_sha256:
raise ExecutionSnapshotError("Execution snapshot has no effective-policy checksum") raise ExecutionSnapshotError("Execution snapshot has no effective-policy checksum")
if snapshot.effective_policy_sha256 != _policy_fingerprint(raw_json, snapshot.delivery): if snapshot.effective_policy_sha256 != _policy_fingerprint(
raw_json,
snapshot.delivery,
snapshot_version=snapshot.snapshot_version,
):
raise ExecutionSnapshotError( raise ExecutionSnapshotError(
"Campaign delivery policy changed after the execution snapshot was created. " "Campaign delivery policy changed after the execution snapshot was created. "
"Revalidate and rebuild the campaign before delivery." "Revalidate and rebuild the campaign before delivery."
@@ -224,7 +348,10 @@ def _assert_snapshot_matches_persisted_inputs(
raise ExecutionSnapshotError("Campaign job does not belong to the snapshotted version") raise ExecutionSnapshotError("Campaign job does not belong to the snapshotted version")
if not getattr(effect_job, "execution_input_sha256", None): if not getattr(effect_job, "execution_input_sha256", None):
raise ExecutionSnapshotError("Campaign job has no execution-input checksum; rebuild before delivery") raise ExecutionSnapshotError("Campaign job has no execution-input checksum; rebuild before delivery")
if effect_job.execution_input_sha256 != job_execution_input_hash(effect_job): if effect_job.execution_input_sha256 != job_execution_input_hash(
effect_job,
snapshot_version=snapshot.snapshot_version,
):
raise ExecutionSnapshotError( raise ExecutionSnapshotError(
"Built campaign job inputs changed after the execution snapshot was created. " "Built campaign job inputs changed after the execution snapshot was created. "
"Revalidate and rebuild the campaign before delivery." "Revalidate and rebuild the campaign before delivery."
@@ -246,8 +373,19 @@ def _assert_snapshot_matches_persisted_inputs(
queueable_count = sum(1 for job in jobs if job.validation_status in queueable_statuses) queueable_count = sum(1 for job in jobs if job.validation_status in queueable_statuses)
if ( if (
snapshot.queueable_job_count != queueable_count snapshot.queueable_job_count != queueable_count
or snapshot.job_manifest_sha256 != job_manifest_hash(jobs) or snapshot.job_manifest_sha256
or any(getattr(job, "execution_input_sha256", None) != job_execution_input_hash(job) for job in jobs) != job_manifest_hash(
jobs,
snapshot_version=snapshot.snapshot_version,
)
or any(
getattr(job, "execution_input_sha256", None)
!= job_execution_input_hash(
job,
snapshot_version=snapshot.snapshot_version,
)
for job in jobs
)
): ):
raise ExecutionSnapshotError( raise ExecutionSnapshotError(
"Built campaign job inputs changed after the execution snapshot was created. " "Built campaign job inputs changed after the execution snapshot was created. "
@@ -276,17 +414,20 @@ def ensure_execution_snapshot(
) )
except CampaignPathSecurityError as exc: except CampaignPathSecurityError as exc:
raise ExecutionSnapshotError(str(exc)) from exc raise ExecutionSnapshotError(str(exc)) from exc
_assert_version_mail_profile_boundary(raw_json) _assert_version_mail_profile_boundary(raw_json, require_profile=False)
if isinstance(version.execution_snapshot, dict): if isinstance(version.execution_snapshot, dict):
if str(version.execution_snapshot.get("snapshot_version") or "") != SNAPSHOT_VERSION: stored_version = str(
version.execution_snapshot.get("snapshot_version") or ""
)
if stored_version not in SUPPORTED_SNAPSHOT_VERSIONS:
raise ExecutionSnapshotError( raise ExecutionSnapshotError(
"This campaign has a legacy execution snapshot that may contain campaign-owned transport data. " "This campaign has a legacy execution snapshot that may contain campaign-owned transport data. "
"It is preserved for audit only and cannot be delivered; select a Mail profile, then revalidate " "It is preserved for audit only and cannot be delivered; select a Mail profile, then revalidate "
"and rebuild a new campaign version." "and rebuild a new campaign version."
) )
snapshot = ExecutionSnapshot.model_validate(version.execution_snapshot) snapshot = ExecutionSnapshot.model_validate(version.execution_snapshot)
expected = snapshot_hash(snapshot.model_dump(mode="json")) expected = snapshot_hash(version.execution_snapshot)
if not version.execution_snapshot_hash: if not version.execution_snapshot_hash:
raise ExecutionSnapshotError("Execution snapshot checksum is missing") raise ExecutionSnapshotError("Execution snapshot checksum is missing")
if version.execution_snapshot_hash != expected: if version.execution_snapshot_hash != expected:
@@ -303,11 +444,6 @@ def ensure_execution_snapshot(
from govoplan_campaign.backend.persistence.campaigns import load_version_config from govoplan_campaign.backend.persistence.campaigns import load_version_config
_, _, config = load_version_config(session, version.id) _, _, config = load_version_config(session, version.id)
profile_id = campaign_mail_profile_id(raw_json)
if not config.server.profile_capabilities.smtp_available:
raise ExecutionSnapshotError("The selected Mail profile has no SMTP configuration")
if profile_id is None:
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
jobs = ( jobs = (
session.query(CampaignJob) session.query(CampaignJob)
.filter(CampaignJob.campaign_version_id == version.id) .filter(CampaignJob.campaign_version_id == version.id)
@@ -316,14 +452,33 @@ def ensure_execution_snapshot(
) )
if not jobs: if not jobs:
raise ExecutionSnapshotError("Campaign version has no built jobs; rebuild it before delivery") raise ExecutionSnapshotError("Campaign version has no built jobs; rebuild it before delivery")
revisions = profile_transport_revisions(session, version) uses_mail = any(
if not revisions["smtp"]: DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail
raise ExecutionSnapshotError("The selected Mail profile has no SMTP transport revision") for job in jobs
)
profile_id = campaign_mail_profile_id(raw_json)
summary: dict[str, Any] = {}
if uses_mail:
if not config.server.profile_capabilities.smtp_available:
raise ExecutionSnapshotError(
"The selected Mail profile has no SMTP configuration"
)
if profile_id is None:
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
summary = profile_delivery_summary(session, version)
if not summary.get("smtp_transport_revision"):
raise ExecutionSnapshotError(
"The selected Mail profile has no SMTP transport revision"
)
payload, digest = create_execution_snapshot( payload, digest = create_execution_snapshot(
version, version,
mail_profile_id=profile_id, mail_profile_id=profile_id,
smtp_transport_revision=revisions["smtp"], smtp_server_id=summary.get("smtp_server_id"),
imap_transport_revision=revisions["imap"], smtp_credential_id=summary.get("smtp_credential_id"),
imap_server_id=summary.get("imap_server_id"),
imap_credential_id=summary.get("imap_credential_id"),
smtp_transport_revision=summary.get("smtp_transport_revision"),
imap_transport_revision=summary.get("imap_transport_revision"),
delivery=config.delivery, delivery=config.delivery,
jobs=jobs, jobs=jobs,
build_summary=version.build_summary if isinstance(version.build_summary, dict) else {}, build_summary=version.build_summary if isinstance(version.build_summary, dict) else {},

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,506 @@
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass, field
from email import policy
from email.parser import BytesParser
from typing import Any
from sqlalchemy import func
from sqlalchemy.orm import Session
from govoplan_core.core.postbox import (
PostboxAttachmentRef,
PostboxDeliveryOutcomeUnknown,
PostboxDeliveryRejected,
PostboxDeliveryRequest,
PostboxParticipantRef,
PostboxTargetRef,
)
from govoplan_campaign.backend.db.models import (
CampaignJob,
JobPostboxStatus,
PostboxDeliveryAttempt,
)
from govoplan_campaign.backend.integrations import (
PostboxDeliveryUnavailable,
postbox_integration,
)
from govoplan_core.security.time import utc_now
ACCEPTED_POSTBOX_ATTEMPT_STATUSES = {
JobPostboxStatus.ACCEPTED.value,
JobPostboxStatus.ACCEPTED_VACANT.value,
}
@dataclass(slots=True)
class PostboxChannelOutcome:
accepted: int = 0
accepted_vacant: int = 0
rejected_temporary: int = 0
rejected_permanent: int = 0
outcome_unknown: int = 0
messages: list[str] = field(default_factory=list)
@property
def accepted_count(self) -> int:
return self.accepted + self.accepted_vacant
@property
def rejected_count(self) -> int:
return self.rejected_temporary + self.rejected_permanent
@property
def all_rejected_before_acceptance(self) -> bool:
return (
self.accepted_count == 0
and self.outcome_unknown == 0
and self.rejected_count > 0
)
@property
def status(self) -> str:
if self.outcome_unknown:
return JobPostboxStatus.OUTCOME_UNKNOWN.value
if self.accepted_count and self.rejected_count:
return JobPostboxStatus.PARTIALLY_ACCEPTED.value
if self.accepted_vacant and not self.accepted:
return JobPostboxStatus.ACCEPTED_VACANT.value
if self.accepted_count:
return JobPostboxStatus.ACCEPTED.value
if self.rejected_temporary:
return JobPostboxStatus.REJECTED_TEMPORARY.value
return JobPostboxStatus.REJECTED_PERMANENT.value
def _target_key(target: dict[str, Any]) -> str:
payload = json.dumps(
target,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
return hashlib.sha256(payload).hexdigest()
def _attempt_number(
session: Session,
*,
job_id: str,
target_key: str,
) -> int:
current = (
session.query(func.max(PostboxDeliveryAttempt.attempt_number))
.filter(
PostboxDeliveryAttempt.job_id == job_id,
PostboxDeliveryAttempt.target_key == target_key,
)
.scalar()
)
return int(current or 0) + 1
def _accepted_attempt(
session: Session,
*,
job_id: str,
target_key: str,
) -> PostboxDeliveryAttempt | None:
return (
session.query(PostboxDeliveryAttempt)
.filter(
PostboxDeliveryAttempt.job_id == job_id,
PostboxDeliveryAttempt.target_key == target_key,
PostboxDeliveryAttempt.status.in_(
list(ACCEPTED_POSTBOX_ATTEMPT_STATUSES)
),
)
.order_by(PostboxDeliveryAttempt.attempt_number.desc())
.first()
)
def _message_body(message_bytes: bytes) -> str | None:
message = BytesParser(policy=policy.default).parsebytes(message_bytes)
body = message.get_body(preferencelist=("plain", "html"))
if body is None:
payload = message.get_payload(decode=True)
if not isinstance(payload, bytes):
return None
return payload.decode(message.get_content_charset() or "utf-8", "replace")
try:
content = body.get_content()
except (LookupError, UnicodeError):
payload = body.get_payload(decode=True)
if not isinstance(payload, bytes):
return None
return payload.decode(body.get_content_charset() or "utf-8", "replace")
return str(content)
def _participants(job: CampaignJob) -> tuple[PostboxParticipantRef, ...]:
recipients = job.resolved_recipients or {}
values: list[PostboxParticipantRef] = []
for key in (
"from_all",
"to",
"cc",
"bcc",
"reply_to",
"bounce_to",
"disposition_notification_to",
):
for item in recipients.get(key) or []:
if not isinstance(item, dict):
continue
address = str(item.get("email") or "").strip() or None
label = str(item.get("name") or "").strip() or None
if address is None and label is None:
continue
values.append(
PostboxParticipantRef(
kind="sender" if key == "from_all" else key,
reference_type="mail_address",
label=label,
address=address,
)
)
return tuple(values)
def _attachments(job: CampaignJob) -> tuple[PostboxAttachmentRef, ...]:
values = [_eml_attachment(job)]
for rule_index, rule in enumerate(job.resolved_attachments or []):
if not isinstance(rule, dict):
continue
managed_matches = rule.get("managed_matches")
if isinstance(managed_matches, list) and managed_matches:
values.extend(_managed_attachments(managed_matches))
continue
matches = rule.get("matches")
if isinstance(matches, list):
values.extend(_campaign_attachments(job, rule_index=rule_index, matches=matches))
return tuple(values)
def _eml_attachment(job: CampaignJob) -> PostboxAttachmentRef:
return PostboxAttachmentRef(
reference_type="campaign_eml",
reference_id=job.id,
name=f"{job.entry_id or job.entry_index}.eml",
media_type="message/rfc822",
size_bytes=job.eml_size_bytes,
digest=job.eml_sha256,
metadata={"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id},
)
MANAGED_ATTACHMENT_METADATA_KEYS = {
"asset_id",
"version_id",
"blob_id",
"display_path",
"relative_path",
"owner_type",
"owner_id",
"source_revision",
}
def _managed_attachment(match: dict[str, Any]) -> PostboxAttachmentRef | None:
reference_id = str(match.get("version_id") or match.get("asset_id") or match.get("blob_id") or "").strip()
if not reference_id:
return None
return PostboxAttachmentRef(
reference_type="file_version" if match.get("version_id") else "file_asset",
reference_id=reference_id,
name=str(match.get("filename") or "").strip() or None,
media_type=str(match.get("content_type")) if match.get("content_type") else None,
size_bytes=int(match["size_bytes"]) if match.get("size_bytes") is not None else None,
digest=str(match.get("checksum_sha256")) if match.get("checksum_sha256") else None,
metadata={key: value for key, value in match.items() if key in MANAGED_ATTACHMENT_METADATA_KEYS},
)
def _managed_attachments(matches: list[Any]) -> list[PostboxAttachmentRef]:
attachments = (_managed_attachment(match) for match in matches if isinstance(match, dict))
return [attachment for attachment in attachments if attachment is not None]
def _campaign_attachments(
job: CampaignJob,
*,
rule_index: int,
matches: list[Any],
) -> list[PostboxAttachmentRef]:
metadata = {"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id, "job_id": job.id}
return [
PostboxAttachmentRef(
reference_type="campaign_attachment",
reference_id=f"{job.id}:{rule_index}:{match_index}",
name=str(match).rsplit("/", 1)[-1] or None,
metadata=metadata,
)
for match_index, match in enumerate(matches)
]
def _sender_label(job: CampaignJob) -> str | None:
recipients = job.resolved_recipients or {}
sender = recipients.get("from")
if not isinstance(sender, dict):
return None
name = str(sender.get("name") or "").strip()
address = str(sender.get("email") or "").strip()
if name and address:
return f"{name} <{address}>"
return address or name or None
def _request(
job: CampaignJob,
target: dict[str, Any],
*,
target_key: str,
body_text: str | None,
classification: str,
) -> PostboxDeliveryRequest:
return PostboxDeliveryRequest(
tenant_id=job.tenant_id,
target=PostboxTargetRef(postbox_id=str(target["postbox_id"])),
producer_module="campaigns",
producer_resource_type="campaign_job",
producer_resource_id=job.id,
idempotency_key=(
f"campaign:{job.campaign_version_id}:{job.id}:postbox:"
f"{target_key}"
),
subject=(job.subject or "").strip() or "(No subject)",
body_text=body_text,
sender_label=_sender_label(job),
classification=classification,
participants=_participants(job),
attachments=_attachments(job),
metadata={
"campaign_id": job.campaign_id,
"campaign_version_id": job.campaign_version_id,
"campaign_job_id": job.id,
"entry_id": job.entry_id,
"entry_index": job.entry_index,
"delivery_channel_policy": job.delivery_channel_policy,
"target_snapshot": target,
},
)
def _record_rejection(
session: Session,
*,
job_id: str,
attempt_id: str,
exc: Exception,
temporary: bool,
) -> None:
session.rollback()
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
job = session.get(CampaignJob, job_id)
if attempt is None or job is None:
raise RuntimeError(
"Postbox rejection could not be written to Campaign evidence."
) from exc
attempt.status = (
JobPostboxStatus.REJECTED_TEMPORARY.value
if temporary
else JobPostboxStatus.REJECTED_PERMANENT.value
)
attempt.error_type = exc.__class__.__name__
attempt.error_code = str(getattr(exc, "code", "") or "") or None
attempt.error_message = str(exc)
attempt.finished_at = utc_now()
session.add(attempt)
session.add(job)
session.commit()
def _record_unknown(
session: Session,
*,
job_id: str,
attempt_id: str,
exc: Exception,
) -> None:
session.rollback()
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
job = session.get(CampaignJob, job_id)
if attempt is None or job is None:
raise RuntimeError(
"Unknown Postbox outcome could not be written to Campaign evidence."
) from exc
attempt.status = JobPostboxStatus.OUTCOME_UNKNOWN.value
attempt.error_type = exc.__class__.__name__
attempt.error_code = str(getattr(exc, "code", "") or "") or None
attempt.error_message = str(exc)
attempt.finished_at = utc_now()
job.postbox_status = JobPostboxStatus.OUTCOME_UNKNOWN.value
session.add(attempt)
session.add(job)
session.commit()
def deliver_campaign_job_to_postboxes(
session: Session,
*,
job: CampaignJob,
message_bytes: bytes,
classification: str,
) -> PostboxChannelOutcome:
outcome = PostboxChannelOutcome()
targets = [
target
for target in (job.resolved_postbox_targets or [])
if isinstance(target, dict) and target.get("postbox_id")
]
if not targets:
outcome.rejected_permanent = 1
outcome.messages.append("No frozen Postbox target is available.")
job.postbox_status = JobPostboxStatus.REJECTED_PERMANENT.value
session.add(job)
session.commit()
return outcome
body_text = _message_body(message_bytes)
for target_index, target in enumerate(targets):
key = _target_key(target)
accepted = _accepted_attempt(
session,
job_id=job.id,
target_key=key,
)
if accepted is not None:
if accepted.vacant:
outcome.accepted_vacant += 1
else:
outcome.accepted += 1
continue
attempt_number = _attempt_number(
session,
job_id=job.id,
target_key=key,
)
request = _request(
job,
target,
target_key=key,
body_text=body_text,
classification=classification,
)
attempt = PostboxDeliveryAttempt(
tenant_id=job.tenant_id,
job_id=job.id,
target_key=key,
target_index=target_index,
attempt_number=attempt_number,
idempotency_key=request.idempotency_key,
status=JobPostboxStatus.DELIVERING.value,
target_snapshot=target,
evidence={},
started_at=utc_now(),
)
job.postbox_attempt_count += 1
job.postbox_status = JobPostboxStatus.DELIVERING.value
session.add(attempt)
session.add(job)
session.commit()
attempt_id = attempt.id
try:
result = postbox_integration().deliver(session, request)
current_attempt = session.get(PostboxDeliveryAttempt, attempt_id)
current_job = session.get(CampaignJob, job.id)
if current_attempt is None or current_job is None:
raise RuntimeError(
"Campaign Postbox attempt disappeared before acceptance."
)
current_attempt.status = (
JobPostboxStatus.ACCEPTED_VACANT.value
if result.vacant
else JobPostboxStatus.ACCEPTED.value
)
current_attempt.provider_delivery_id = result.delivery_id
current_attempt.provider_message_id = result.message_id
current_attempt.postbox_id = result.postbox_id
current_attempt.address = result.address
current_attempt.holder_count = result.holder_count
current_attempt.vacant = result.vacant
current_attempt.duplicate = result.duplicate
current_attempt.evidence = dict(result.evidence)
current_attempt.finished_at = utc_now()
session.add(current_attempt)
session.add(current_job)
session.commit()
if result.vacant:
outcome.accepted_vacant += 1
else:
outcome.accepted += 1
except PostboxDeliveryOutcomeUnknown as exc:
_record_unknown(
session,
job_id=job.id,
attempt_id=attempt_id,
exc=exc,
)
outcome.outcome_unknown += 1
outcome.messages.append(str(exc))
except PostboxDeliveryRejected as exc:
if exc.code == "idempotency_conflict":
_record_unknown(
session,
job_id=job.id,
attempt_id=attempt_id,
exc=exc,
)
outcome.outcome_unknown += 1
else:
_record_rejection(
session,
job_id=job.id,
attempt_id=attempt_id,
exc=exc,
temporary=exc.temporary,
)
if exc.temporary:
outcome.rejected_temporary += 1
else:
outcome.rejected_permanent += 1
outcome.messages.append(str(exc))
except PostboxDeliveryUnavailable as exc:
_record_rejection(
session,
job_id=job.id,
attempt_id=attempt_id,
exc=exc,
temporary=True,
)
outcome.rejected_temporary += 1
outcome.messages.append(str(exc))
except Exception as exc:
_record_unknown(
session,
job_id=job.id,
attempt_id=attempt_id,
exc=exc,
)
outcome.outcome_unknown += 1
outcome.messages.append(str(exc))
current_job = session.get(CampaignJob, job.id)
if current_job is not None:
current_job.postbox_status = outcome.status
session.add(current_job)
session.commit()
return outcome

View File

@@ -0,0 +1,999 @@
from __future__ import annotations
import json
from collections.abc import Mapping, Sequence
from typing import Literal
from fastapi import HTTPException, Query, status
from sqlalchemy import and_, func, or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.schemas import (
CampaignJobsResponse,
CampaignJobDiagnosticsResponse,
)
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
from govoplan_core.core.change_sequence import (
encode_sequence_watermark,
max_sequence_id,
)
from govoplan_core.core.pagination import (
KeysetCursorError,
decode_keyset_cursor,
encode_keyset_cursor,
keyset_query_fingerprint,
)
from govoplan_campaign.backend.change_tracking import (
CAMPAIGNS_MODULE_ID,
CAMPAIGN_JOBS_COLLECTION,
)
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignMessageAction,
CampaignMessageActionAttempt,
CampaignVersion,
ImapAppendAttempt,
JobImapStatus,
JobPostboxStatus,
JobQueueStatus,
JobSendStatus,
JobValidationStatus,
PostboxDeliveryAttempt,
SendAttempt,
)
from govoplan_campaign.backend.response_security import (
public_campaign_payload,
public_delivery_result_message,
)
from govoplan_campaign.backend.route_support import (
_get_campaign_for_principal,
_get_campaign_for_tenant,
_get_version_for_tenant,
_require_permission,
)
CAMPAIGN_JOBS_CURSOR_SCOPE = "campaign.jobs"
def _job_review_key(job: CampaignJob) -> str:
return str(job.entry_id or job.entry_index)
def _job_summary_payload(
job: CampaignJob,
*,
reviewed_keys: set[str] | None = None,
) -> dict[str, object]:
review_key = _job_review_key(job)
return {
"id": job.id,
"campaign_version_id": job.campaign_version_id,
"entry_index": job.entry_index,
"entry_id": job.entry_id,
"recipient_email": job.recipient_email,
"subject": job.subject,
"message_id_header": job.message_id_header,
"build_status": job.build_status,
"validation_status": job.validation_status,
"queue_status": job.queue_status,
"send_status": job.send_status,
"delivery_channel_policy": getattr(job, "delivery_channel_policy", "mail"),
"postbox_status": getattr(job, "postbox_status", "not_requested"),
"imap_status": job.imap_status,
"eml_size_bytes": job.eml_size_bytes,
"eml_sha256": job.eml_sha256,
"attempt_count": job.attempt_count,
"postbox_attempt_count": getattr(job, "postbox_attempt_count", 0),
"postbox_target_count": len(
getattr(job, "resolved_postbox_targets", None) or []
),
"last_error": public_delivery_result_message(
last_error=job.last_error,
send_status=job.send_status,
imap_status=job.imap_status,
postbox_status=getattr(job, "postbox_status", "not_requested"),
),
"queued_at": job.queued_at,
"outcome_unknown_at": job.outcome_unknown_at,
"sent_at": job.sent_at,
"created_at": job.created_at,
"updated_at": job.updated_at,
"issues_count": len(job.issues_snapshot or []),
"attachment_count": len(job.resolved_attachments or []),
"review_key": review_key,
"reviewed": review_key in reviewed_keys if reviewed_keys is not None else False,
"matched_file_count": sum(
len(item.get("matches") or [])
for item in (job.resolved_attachments or [])
if isinstance(item, dict)
),
}
def _job_detail_payload(job: CampaignJob) -> dict[str, object]:
return {
**_job_summary_payload(job),
"message_id_header": job.message_id_header,
"issues": job.issues_snapshot or [],
"attachments": public_campaign_payload(job.resolved_attachments or []),
"resolved_recipients": job.resolved_recipients or {},
"resolved_postbox_targets": getattr(job, "resolved_postbox_targets", None)
or [],
}
def _job_attempts_payload(
send_attempts: list[SendAttempt],
imap_attempts: list[ImapAppendAttempt],
postbox_attempts: Sequence[PostboxDeliveryAttempt] = (),
message_actions: Sequence[CampaignMessageAction] = (),
message_action_attempts: Sequence[CampaignMessageActionAttempt] = (),
*,
postbox_receipts: (
Mapping[str, PostboxDeliveryReceiptSummaryRef] | None
) = None,
include_diagnostics: bool = False,
) -> dict[str, list[dict[str, object]]]:
smtp_payloads: list[dict[str, object]] = []
for attempt in send_attempts:
payload: dict[str, object] = {
"id": attempt.id,
"attempt_number": attempt.attempt_number,
"status": attempt.status,
"smtp_status_code": attempt.smtp_status_code,
"started_at": attempt.started_at,
"finished_at": attempt.finished_at,
}
if include_diagnostics:
payload["claim_token"] = attempt.claim_token
payload["smtp_response"] = attempt.smtp_response
payload["error_type"] = attempt.error_type
payload["error_message"] = attempt.error_message
smtp_payloads.append(payload)
imap_payloads: list[dict[str, object]] = []
for attempt in imap_attempts:
payload = {
"id": attempt.id,
"attempt_number": attempt.attempt_number,
"status": attempt.status,
"folder": attempt.folder,
"created_at": attempt.created_at,
"updated_at": attempt.updated_at,
}
if include_diagnostics:
payload["claim_token"] = attempt.claim_token
payload["error_message"] = attempt.error_message
imap_payloads.append(payload)
postbox_payloads: list[dict[str, object]] = []
for attempt in postbox_attempts:
payload = {
"id": attempt.id,
"target_index": attempt.target_index,
"attempt_number": attempt.attempt_number,
"status": attempt.status,
"postbox_id": attempt.postbox_id,
"address": attempt.address,
"holder_count": attempt.holder_count,
"vacant": attempt.vacant,
"duplicate": attempt.duplicate,
"target": attempt.target_snapshot or {},
"started_at": attempt.started_at,
"finished_at": attempt.finished_at,
"error_code": attempt.error_code,
}
if include_diagnostics:
payload["idempotency_key"] = attempt.idempotency_key
payload["provider_delivery_id"] = attempt.provider_delivery_id
payload["provider_message_id"] = attempt.provider_message_id
payload["evidence"] = attempt.evidence or {}
payload["error_type"] = attempt.error_type
payload["error_message"] = attempt.error_message
if attempt.provider_delivery_id:
receipt_summary = (
postbox_receipts.get(attempt.provider_delivery_id)
if postbox_receipts is not None
else None
)
payload["receipt_summary_status"] = (
"available"
if receipt_summary is not None
else "not_found"
if postbox_receipts is not None
else "unavailable"
)
if receipt_summary is not None:
payload["receipt_summary"] = _postbox_receipt_summary_payload(
receipt_summary
)
postbox_payloads.append(payload)
action_attempts_by_action = {
attempt.action_id: attempt
for attempt in message_action_attempts
}
message_action_payloads: list[dict[str, object]] = []
for action in message_actions:
action_attempt = action_attempts_by_action.get(action.id)
payload = {
"id": action.id,
"kind": action.kind,
"status": action.status,
"reason": action.reason,
"actor_user_id": action.actor_user_id,
"actor_api_key_id": action.actor_api_key_id,
"campaign_version_id": action.campaign_version_id,
"message_sha256": action.message_sha256,
"recipient_manifest_sha256": action.recipient_manifest_sha256,
"recipient_count": action.recipient_count,
"prior_send_status": action.prior_send_status,
"final_send_status": action.final_send_status,
"accepted_count": action.accepted_count,
"refused_count": action.refused_count,
"refusal_summary": action.refusal_summary or {},
"linked_send_attempt_id": action.linked_send_attempt_id,
"created_at": action.created_at,
"effect_started_at": action.effect_started_at,
"completed_at": action.completed_at,
"attempt": (
{
"id": action_attempt.id,
"status": action_attempt.status,
"started_at": action_attempt.started_at,
"effect_started_at": action_attempt.effect_started_at,
"completed_at": action_attempt.completed_at,
"accepted_count": action_attempt.accepted_count,
"refused_count": action_attempt.refused_count,
}
if action_attempt is not None
else None
),
}
if include_diagnostics:
payload["idempotency_key"] = action.idempotency_key
payload["canonical_request_hash"] = action.canonical_request_hash
payload["context"] = action.context or {}
payload["error_type"] = action.error_type
payload["error_message"] = action.error_message
if action_attempt is not None:
payload["attempt"] = {
**dict(payload["attempt"] or {}),
"outcome_code": action_attempt.outcome_code,
"diagnostic_summary": action_attempt.diagnostic_summary,
}
message_action_payloads.append(payload)
return {
"smtp": smtp_payloads,
"imap": imap_payloads,
"postbox": postbox_payloads,
"message_actions": message_action_payloads,
}
def _postbox_receipt_summary_payload(
summary: PostboxDeliveryReceiptSummaryRef,
) -> dict[str, object]:
return {
"delivery_id": summary.delivery_id,
"message_id": summary.message_id,
"postbox_id": summary.postbox_id,
"delivery_status": summary.delivery_status,
"accepted_at": summary.accepted_at,
"current_holder_count": summary.current_holder_count,
"currently_readable": summary.currently_readable,
"message_count": summary.message_count,
"routed_message_count": summary.routed_message_count,
"readable_message_count": summary.readable_message_count,
"read_receipt_count": summary.read_receipt_count,
"acknowledged_receipt_count": summary.acknowledged_receipt_count,
"withdrawn_message_count": summary.withdrawn_message_count,
"expired_message_count": summary.expired_message_count,
"first_read_at": summary.first_read_at,
"last_read_at": summary.last_read_at,
"first_acknowledged_at": summary.first_acknowledged_at,
"last_acknowledged_at": summary.last_acknowledged_at,
"route_status_counts": dict(summary.route_status_counts),
}
def _job_diagnostics_payload(
job: CampaignJob,
send_attempts: list[SendAttempt],
imap_attempts: list[ImapAppendAttempt],
postbox_attempts: Sequence[PostboxDeliveryAttempt] = (),
message_actions: Sequence[CampaignMessageAction] = (),
message_action_attempts: Sequence[CampaignMessageActionAttempt] = (),
*,
postbox_receipts: (
Mapping[str, PostboxDeliveryReceiptSummaryRef] | None
) = None,
) -> CampaignJobDiagnosticsResponse:
return CampaignJobDiagnosticsResponse(
job_id=job.id,
campaign_id=job.campaign_id,
campaign_version_id=job.campaign_version_id,
storage={
"eml_local_path": job.eml_local_path,
"eml_storage_key": job.eml_storage_key,
"eml_size_bytes": job.eml_size_bytes,
"eml_sha256": job.eml_sha256,
},
worker_claim={
"claim_token": job.claim_token,
"claimed_at": job.claimed_at,
"smtp_started_at": job.smtp_started_at,
"outcome_unknown_at": job.outcome_unknown_at,
"last_error": job.last_error,
},
attempts=_job_attempts_payload(
send_attempts,
imap_attempts,
postbox_attempts,
message_actions,
message_action_attempts,
postbox_receipts=postbox_receipts,
include_diagnostics=True,
),
)
def _review_metadata(
session: Session,
version: CampaignVersion | None,
base_filters: list[object],
) -> tuple[dict[str, object], set[str]]:
if version is None:
return _empty_review_metadata(), set()
review_state, state_is_current = _current_review_state(version)
reviewed_keys = _current_reviewed_keys(
review_state, state_is_current=state_is_current
)
counts = _review_metadata_counts(_review_rows(session, base_filters), reviewed_keys)
return {
"inspection_complete": bool(
state_is_current and review_state.get("inspection_complete") is True
),
**counts,
}, reviewed_keys
def _empty_review_metadata() -> dict[str, object]:
return {
"inspection_complete": False,
"blocking_count": 0,
"required_count": 0,
"reviewed_required_count": 0,
"bulk_acceptable_count": 0,
}
def _current_review_state(version: CampaignVersion) -> tuple[dict[str, object], bool]:
build_summary = (
version.build_summary if isinstance(version.build_summary, dict) else {}
)
build_token = str(
build_summary.get("build_token") or build_summary.get("built_at") or ""
)
editor_state = (
version.editor_state if isinstance(version.editor_state, dict) else {}
)
review_state = (
editor_state.get("review_send")
if isinstance(editor_state.get("review_send"), dict)
else {}
)
state_token = str(review_state.get("build_token") or "")
return review_state, bool(build_token and state_token == build_token)
def _current_reviewed_keys(
review_state: dict[str, object], *, state_is_current: bool
) -> set[str]:
return {
str(value)
for value in (review_state.get("reviewed_message_keys") or [])
if state_is_current and str(value).strip()
}
def _review_rows(
session: Session, base_filters: list[object]
) -> list[tuple[object, int, str, str]]:
return (
session.query(
CampaignJob.entry_id,
CampaignJob.entry_index,
CampaignJob.build_status,
CampaignJob.validation_status,
)
.filter(*base_filters)
.all()
)
def _review_metadata_counts(
review_rows: list[tuple[object, int, str, str]], reviewed_keys: set[str]
) -> dict[str, int]:
blocking_count = 0
required_count = 0
reviewed_required_count = 0
bulk_acceptable_count = 0
for entry_id, entry_index, build_status, validation_status in review_rows:
key = str(entry_id or entry_index)
if build_status != "built" or validation_status == "blocked":
blocking_count += 1
if validation_status == "needs_review":
required_count += 1
if key in reviewed_keys:
reviewed_required_count += 1
elif validation_status in {"warning", "excluded"}:
bulk_acceptable_count += 1
return {
"blocking_count": blocking_count,
"required_count": required_count,
"reviewed_required_count": reviewed_required_count,
"bulk_acceptable_count": bulk_acceptable_count,
}
def _status_counts(
session: Session, filters: list[object]
) -> dict[str, dict[str, int]]:
result: dict[str, dict[str, int]] = {}
for field_name in (
"build_status",
"validation_status",
"queue_status",
"send_status",
"postbox_status",
"imap_status",
):
column = getattr(CampaignJob, field_name)
rows = (
session.query(column, func.count(CampaignJob.id))
.filter(*filters)
.group_by(column)
.all()
)
result[field_name.removesuffix("_status")] = {
str(value or "unknown"): int(count) for value, count in rows
}
return result
CAMPAIGN_JOB_GRID_SORT_COLUMNS = {
"number": CampaignJob.entry_index,
"recipient": func.lower(func.coalesce(CampaignJob.recipient_email, "")),
"subject": func.lower(func.coalesce(CampaignJob.subject, "")),
"validation": CampaignJob.validation_status,
"queue": CampaignJob.queue_status,
"send": CampaignJob.send_status,
"postbox": CampaignJob.postbox_status,
"imap": CampaignJob.imap_status,
"attempts": CampaignJob.attempt_count,
"updated": CampaignJob.updated_at,
}
CAMPAIGN_JOB_GRID_LIST_FILTERS = {
"validation": (
CampaignJob.validation_status,
{item.value for item in JobValidationStatus},
),
"queue": (CampaignJob.queue_status, {item.value for item in JobQueueStatus}),
"send": (CampaignJob.send_status, {item.value for item in JobSendStatus}),
"postbox": (
CampaignJob.postbox_status,
{item.value for item in JobPostboxStatus},
),
"imap": (CampaignJob.imap_status, {item.value for item in JobImapStatus}),
}
def _campaign_jobs_grid_filter_expressions(
grid_filters: dict[str, str] | None,
) -> list[object]:
values = grid_filters or {}
expressions: list[object] = []
recipient = values.get("recipient", "").strip()
if recipient:
pattern = _contains_pattern(recipient)
expressions.append(
or_(
CampaignJob.recipient_email.ilike(pattern, escape="\\"),
CampaignJob.entry_id.ilike(pattern, escape="\\"),
)
)
subject = values.get("subject", "").strip()
if subject:
expressions.append(
CampaignJob.subject.ilike(_contains_pattern(subject), escape="\\")
)
evidence = values.get("evidence", "").strip()
if evidence:
pattern = _contains_pattern(evidence)
expressions.append(
or_(
CampaignJob.message_id_header.ilike(pattern, escape="\\"),
CampaignJob.eml_sha256.ilike(pattern, escape="\\"),
)
)
attempts = values.get("attempts", "").strip()
if attempts:
expressions.append(
_campaign_jobs_integer_filter(
CampaignJob.attempt_count, attempts, column_id="attempts"
)
)
for column_id, (column, allowed_values) in CAMPAIGN_JOB_GRID_LIST_FILTERS.items():
raw_value = values.get(column_id, "").strip()
if not raw_value:
continue
selected = _campaign_jobs_list_filter(
raw_value, column_id=column_id, allowed_values=allowed_values
)
expressions.append(column.in_(selected))
return expressions
def _campaign_jobs_list_filter(
raw_value: str, *, column_id: str, allowed_values: set[str]
) -> list[str]:
if raw_value.startswith("list:"):
try:
parsed = json.loads(raw_value[5:])
except json.JSONDecodeError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid {column_id} list filter",
) from exc
if not isinstance(parsed, list) or any(
not isinstance(value, str) for value in parsed
):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid {column_id} list filter",
)
selected = list(
dict.fromkeys(value.strip() for value in parsed if value.strip())
)
else:
selected = list(
dict.fromkeys(
value.strip() for value in raw_value.split(",") if value.strip()
)
)
if len(selected) > 50 or any(value not in allowed_values for value in selected):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid {column_id} list filter",
)
return selected
def _campaign_jobs_integer_filter(column: object, raw_value: str, *, column_id: str):
operator, separator, value = raw_value.partition(":")
if not separator:
operator, value = "eq", operator
if operator not in {"eq", "gt", "gte", "lt", "lte"}:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid {column_id} filter operator",
)
try:
expected = int(value)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail=f"Invalid {column_id} filter value",
) from exc
if operator == "gt":
return column > expected
if operator == "gte":
return column >= expected
if operator == "lt":
return column < expected
if operator == "lte":
return column <= expected
return column == expected
def _contains_pattern(value: str) -> str:
escaped = value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
return f"%{escaped}%"
def _campaign_jobs_ordering(sort_by: str, sort_direction: str) -> list[object]:
column = CAMPAIGN_JOB_GRID_SORT_COLUMNS.get(sort_by)
if column is None:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Unsupported Campaign job sort column",
)
primary = column.desc() if sort_direction == "desc" else column.asc()
return [primary, CampaignJob.id.asc()]
def _campaign_jobs_query_context(
session: Session,
principal: ApiPrincipal,
*,
campaign_id: str,
version_id: str | None,
send_status: list[str] | None,
validation_status: list[str] | None,
imap_status: list[str] | None,
query_text: str | None,
grid_filters: dict[str, str] | None = None,
) -> tuple[Campaign, list[object], list[object], dict[str, object], set[str]]:
_get_campaign_for_principal(session, campaign_id, principal)
_require_permission(principal, "campaigns:recipient:read")
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
base_filters: list[object] = [
CampaignJob.campaign_id == campaign.id,
CampaignJob.tenant_id == principal.tenant_id,
]
selected_version: CampaignVersion | None = None
if version_id:
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if version.campaign_id != campaign.id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Campaign version not found",
)
selected_version = version
base_filters.append(CampaignJob.campaign_version_id == version.id)
review_metadata, reviewed_keys = _review_metadata(
session, selected_version, base_filters
)
filtered = list(base_filters)
if send_status:
filtered.append(CampaignJob.send_status.in_(send_status))
if validation_status:
filtered.append(CampaignJob.validation_status.in_(validation_status))
if imap_status:
filtered.append(CampaignJob.imap_status.in_(imap_status))
if query_text and query_text.strip():
pattern = f"%{query_text.strip()}%"
filtered.append(
or_(
CampaignJob.recipient_email.ilike(pattern),
CampaignJob.subject.ilike(pattern),
CampaignJob.entry_id.ilike(pattern),
)
)
filtered.extend(_campaign_jobs_grid_filter_expressions(grid_filters))
return campaign, base_filters, filtered, review_metadata, reviewed_keys
def _campaign_jobs_page_response(
session: Session,
*,
campaign_id: str,
version_id: str | None,
base_filters: list[object],
filtered: list[object],
reviewed_keys: set[str],
review_metadata: dict[str, object],
page: int,
page_size: int,
send_status: list[str] | None = None,
validation_status: list[str] | None = None,
imap_status: list[str] | None = None,
query_text: str | None = None,
grid_filters: dict[str, str] | None = None,
sort_by: str = "number",
sort_direction: str = "asc",
cursor: str | None = None,
changed_job_ids: set[str] | None = None,
) -> CampaignJobsResponse:
total_unfiltered, total = _campaign_jobs_page_counts(
session, base_filters=base_filters, filtered=filtered
)
pages = (total + page_size - 1) // page_size if total else 0
fingerprint = _campaign_jobs_cursor_fingerprint(
campaign_id=campaign_id,
version_id=version_id,
page_size=page_size,
send_status=send_status,
validation_status=validation_status,
imap_status=imap_status,
query_text=query_text,
grid_filters=grid_filters,
sort_by=sort_by,
sort_direction=sort_direction,
)
ordering = _campaign_jobs_ordering(sort_by, sort_direction)
rows_plus_one, start_cursor = _campaign_jobs_page_rows(
session,
filtered=filtered,
ordering=ordering,
page=page,
page_size=page_size,
sort_by=sort_by,
sort_direction=sort_direction,
cursor=cursor,
fingerprint=fingerprint,
)
jobs = rows_plus_one[:page_size]
next_cursor = _campaign_jobs_next_cursor(
jobs,
rows_plus_one=rows_plus_one,
page_size=page_size,
sort_by=sort_by,
sort_direction=sort_direction,
changed_job_ids=changed_job_ids,
fingerprint=fingerprint,
)
if changed_job_ids is not None:
jobs = [job for job in jobs if job.id in changed_job_ids]
return CampaignJobsResponse(
jobs=[_job_summary_payload(job, reviewed_keys=reviewed_keys) for job in jobs],
page=page,
page_size=page_size,
total=total,
total_unfiltered=total_unfiltered,
pages=pages,
cursor=start_cursor,
next_cursor=next_cursor,
counts=_status_counts(session, base_filters),
filtered_counts=_status_counts(session, filtered),
review=review_metadata,
)
def _campaign_jobs_page_counts(
session: Session,
*,
base_filters: list[object],
filtered: list[object],
) -> tuple[int, int]:
total_unfiltered = int(
session.query(func.count(CampaignJob.id)).filter(*base_filters).scalar() or 0
)
total = int(
session.query(func.count(CampaignJob.id)).filter(*filtered).scalar() or 0
)
return total_unfiltered, total
def _campaign_jobs_page_rows(
session: Session,
*,
filtered: list[object],
ordering: list[object],
page: int,
page_size: int,
sort_by: str,
sort_direction: str,
cursor: str | None,
fingerprint: str,
) -> tuple[list[CampaignJob], str | None]:
if cursor:
page_query = _campaign_jobs_query_after_cursor(
session,
filtered=filtered,
cursor=cursor,
fingerprint=fingerprint,
sort_by=sort_by,
sort_direction=sort_direction,
)
return page_query.order_by(*ordering).limit(page_size + 1).all(), cursor
effective_offset = (page - 1) * page_size
page_query = session.query(CampaignJob).filter(*filtered)
start_cursor = _campaign_jobs_offset_cursor(
page_query,
ordering=ordering,
effective_offset=effective_offset,
sort_by=sort_by,
sort_direction=sort_direction,
fingerprint=fingerprint,
)
rows = (
page_query.order_by(*ordering)
.offset(effective_offset)
.limit(page_size + 1)
.all()
)
return rows, start_cursor
def _campaign_jobs_query_after_cursor(
session: Session,
*,
filtered: list[object],
cursor: str,
fingerprint: str,
sort_by: str,
sort_direction: str,
):
if sort_by != "number" or sort_direction != "asc":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Campaign job cursors require number ascending order",
)
try:
cursor_values = decode_keyset_cursor(
CAMPAIGN_JOBS_CURSOR_SCOPE, cursor, fingerprint=fingerprint
)
if cursor_values is None:
raise KeysetCursorError("Invalid pagination cursor")
except KeysetCursorError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
) from exc
return (
session.query(CampaignJob)
.filter(*filtered)
.filter(_campaign_jobs_cursor_condition(cursor_values))
)
def _campaign_jobs_offset_cursor(
query,
*,
ordering: list[object],
effective_offset: int,
sort_by: str,
sort_direction: str,
fingerprint: str,
) -> str | None:
if effective_offset <= 0 or sort_by != "number" or sort_direction != "asc":
return None
previous_row = (
query.order_by(*ordering).offset(effective_offset - 1).limit(1).first()
)
if previous_row is None:
return None
return _campaign_jobs_cursor_for_row(previous_row, fingerprint=fingerprint)
def _campaign_jobs_next_cursor(
jobs: list[CampaignJob],
*,
rows_plus_one: list[CampaignJob],
page_size: int,
sort_by: str,
sort_direction: str,
changed_job_ids: set[str] | None,
fingerprint: str,
) -> str | None:
cursor_supported = sort_by == "number" and sort_direction == "asc"
if (
changed_job_ids is not None
or not cursor_supported
or len(rows_plus_one) <= page_size
or not jobs
):
return None
return _campaign_jobs_cursor_for_row(jobs[-1], fingerprint=fingerprint)
def _campaign_jobs_cursor_fingerprint(
*,
campaign_id: str,
version_id: str | None,
page_size: int,
send_status: list[str] | None,
validation_status: list[str] | None,
imap_status: list[str] | None,
query_text: str | None,
grid_filters: dict[str, str] | None,
sort_by: str,
sort_direction: str,
) -> str:
return keyset_query_fingerprint(
CAMPAIGN_JOBS_CURSOR_SCOPE,
{
"campaign_id": campaign_id,
"version_id": version_id or "",
"page_size": page_size,
"send_status": sorted(send_status or []),
"validation_status": sorted(validation_status or []),
"imap_status": sorted(imap_status or []),
"query": (query_text or "").strip(),
"grid_filters": sorted((grid_filters or {}).items()),
"order": [f"{sort_by}:{sort_direction}", "id:asc"],
},
)
def _campaign_jobs_cursor_for_row(row: CampaignJob, *, fingerprint: str) -> str:
return encode_keyset_cursor(
CAMPAIGN_JOBS_CURSOR_SCOPE,
fingerprint=fingerprint,
values={"id": row.id, "entry_index": row.entry_index},
)
def _campaign_jobs_cursor_condition(cursor_values: dict[str, object]):
cursor_id = cursor_values.get("id")
cursor_index = cursor_values.get("entry_index")
if not isinstance(cursor_id, str) or not cursor_id:
raise KeysetCursorError("Invalid pagination cursor")
try:
entry_index = int(cursor_index)
except (TypeError, ValueError) as exc:
raise KeysetCursorError("Invalid pagination cursor") from exc
return or_(
CampaignJob.entry_index > entry_index,
and_(CampaignJob.entry_index == entry_index, CampaignJob.id > cursor_id),
)
def _campaign_jobs_delta_watermark(session: Session, tenant_id: str) -> str:
return encode_sequence_watermark(
max_sequence_id(
session,
tenant_id=tenant_id,
module_id=CAMPAIGNS_MODULE_ID,
collections=(CAMPAIGN_JOBS_COLLECTION,),
)
)
class CampaignJobsQuery:
def __init__(
self,
version_id: str | None = None,
page: int = Query(default=1, ge=1),
page_size: int = Query(default=50, ge=1, le=200),
cursor: str | None = Query(default=None),
send_status: list[str] | None = Query(default=None),
validation_status: list[str] | None = Query(default=None),
imap_status: list[str] | None = Query(default=None),
query_text: str | None = Query(default=None, alias="q", max_length=200),
sort_by: Literal[
"number",
"recipient",
"subject",
"validation",
"queue",
"send",
"postbox",
"imap",
"attempts",
"updated",
] = Query(default="number"),
sort_direction: Literal["asc", "desc"] = Query(default="asc"),
filter_recipient: str | None = Query(default=None, max_length=500),
filter_subject: str | None = Query(default=None, max_length=1000),
filter_validation: str | None = Query(default=None, max_length=1000),
filter_queue: str | None = Query(default=None, max_length=1000),
filter_send: str | None = Query(default=None, max_length=1000),
filter_postbox: str | None = Query(default=None, max_length=1000),
filter_imap: str | None = Query(default=None, max_length=1000),
filter_attempts: str | None = Query(default=None, max_length=100),
filter_evidence: str | None = Query(default=None, max_length=500),
) -> None:
self.version_id = version_id
self.page = page
self.page_size = page_size
self.cursor = cursor
self.send_status = send_status
self.validation_status = validation_status
self.imap_status = imap_status
self.query_text = query_text
self.sort_by = sort_by
self.sort_direction = sort_direction
self.grid_filters = {
column_id: value
for column_id, value in {
"recipient": filter_recipient,
"subject": filter_subject,
"validation": filter_validation,
"queue": filter_queue,
"send": filter_send,
"postbox": filter_postbox,
"imap": filter_imap,
"attempts": filter_attempts,
"evidence": filter_evidence,
}.items()
if value is not None and value.strip()
}

View File

@@ -1,14 +1,26 @@
from __future__ import annotations from __future__ import annotations
import unittest import unittest
from unittest.mock import patch
from sqlalchemy import create_engine from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker from sqlalchemy.orm import sessionmaker
from govoplan_access.backend.db.models import Account, Group, User from govoplan_access.backend.db.models import Account, Group, User
from govoplan_campaign.backend.capabilities import CampaignAccessService from govoplan_campaign.backend.capabilities import (
from govoplan_campaign.backend.db.models import Campaign, CampaignShare CampaignAccessService,
CampaignOwnershipService,
campaign_report_resource_id,
)
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignShare,
CampaignVersion,
)
from govoplan_core.core.access import PrincipalRef from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.core.ownership import OwnershipSubjectRef, OwnershipTransferError
from govoplan_core.db.base import Base from govoplan_core.db.base import Base
@@ -69,10 +81,326 @@ class CampaignAccessProviderTests(unittest.TestCase):
self.assertTrue(any(item.kind == "share" and item.id == "share-write" for item in items)) self.assertTrue(any(item.kind == "share" and item.id == "share-write" for item in items))
self.assertFalse(any(item.kind == "share" and item.id == "share-read" for item in items)) self.assertFalse(any(item.kind == "share" and item.id == "share-read" for item in items))
def test_campaign_children_explain_their_parent_campaign_access(self) -> None:
session = _session()
self.addCleanup(_close_session, session)
_seed_access_subjects(session)
campaign = Campaign(
id="campaign-child",
tenant_id=TENANT_ID,
owner_user_id=OTHER_USER_ID,
external_id="child",
name="Child resources",
)
version = CampaignVersion(
id="version-child",
campaign_id=campaign.id,
version_number=1,
raw_json={},
)
job = CampaignJob(
id="job-child",
tenant_id=TENANT_ID,
campaign_id=campaign.id,
campaign_version_id=version.id,
entry_index=1,
recipient_email="recipient@example.test",
)
session.add_all(
[
campaign,
version,
job,
CampaignShare(
id="share-child",
tenant_id=TENANT_ID,
campaign_id=campaign.id,
target_type="group",
target_id=GROUP_ID,
permission="read",
),
]
)
session.commit()
service = CampaignAccessService()
principal = _principal(group_ids={GROUP_ID})
version_items = service.explain_resource_provenance(
session,
principal,
resource_type="campaign_version",
resource_id=version.id,
action="campaigns:version:read",
)
job_items = service.explain_resource_provenance(
session,
principal,
resource_type="campaign_delivery_job",
resource_id=job.id,
action="campaigns:delivery_job:read",
)
report_id = campaign_report_resource_id(
campaign_id=campaign.id,
version_id=version.id,
report_kind="delivery",
)
report_items = service.explain_resource_provenance(
session,
principal,
resource_type="campaign_report",
resource_id=report_id,
action="campaigns:report:read",
)
for items, source in (
(version_items, "campaigns.version"),
(job_items, "campaigns.delivery_job"),
(report_items, "campaigns.report"),
):
child = next(item for item in items if item.source == source)
self.assertEqual(
child.details["authorization_inherited_from"],
{
"resource_type": "campaign",
"resource_id": campaign.id,
},
)
self.assertTrue(
any(
item.source == "campaigns.campaign"
and item.id == campaign.id
for item in items
)
)
self.assertTrue(
any(item.kind == "share" and item.id == "share-child" for item in items)
)
report = next(item for item in report_items if item.source == "campaigns.report")
self.assertEqual(report.details["campaign_version_id"], version.id)
self.assertEqual(report.details["report_kind"], "delivery")
self.assertFalse(report.details["persisted"])
def test_campaign_ownership_provider_requires_group_acceptance_authority(self) -> None:
session = _session()
self.addCleanup(_close_session, session)
_seed_access_subjects(session)
campaign = Campaign(
id="campaign-ownership",
tenant_id=TENANT_ID,
owner_user_id=USER_ID,
external_id="ownership",
name="Ownership",
)
session.add(campaign)
session.commit()
service = CampaignOwnershipService()
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
target_group = OwnershipSubjectRef(type="group", id=GROUP_ID)
ordinary_member = OwnershipSubjectRef(
type="user",
id=OTHER_USER_ID,
group_ids=frozenset({GROUP_ID}),
)
group_manager = OwnershipSubjectRef(
type="user",
id=OTHER_USER_ID,
group_ids=frozenset({GROUP_ID}),
scopes=frozenset({"campaigns:ownership:accept_group"}),
)
denied = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="accept_group_transfer",
actor=ordinary_member,
current_owner=current_owner,
target_owner=target_group,
)
allowed = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="accept_group_transfer",
actor=group_manager,
current_owner=current_owner,
target_owner=target_group,
)
self.assertFalse(denied.allowed)
self.assertTrue(allowed.allowed)
@patch(
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
return_value=None,
)
def test_campaign_ownership_request_requires_existing_read_access(
self,
_target_validation,
) -> None:
session = _session()
self.addCleanup(_close_session, session)
_seed_access_subjects(session)
campaign = Campaign(
id="campaign-request",
tenant_id=TENANT_ID,
owner_user_id=USER_ID,
external_id="request",
name="Request",
)
session.add(campaign)
session.commit()
service = CampaignOwnershipService()
requester = OwnershipSubjectRef(
type="user",
id=OTHER_USER_ID,
scopes=frozenset({"campaigns:campaign:read"}),
)
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
denied = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="request_ownership",
actor=requester,
current_owner=current_owner,
target_owner=requester,
)
session.add(
CampaignShare(
id="share-requester",
tenant_id=TENANT_ID,
campaign_id=campaign.id,
target_type="user",
target_id=OTHER_USER_ID,
permission="read",
)
)
session.commit()
allowed = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="request_ownership",
actor=requester,
current_owner=current_owner,
target_owner=requester,
)
self.assertFalse(denied.allowed)
self.assertTrue(allowed.allowed)
@patch(
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
return_value=None,
)
def test_campaign_owner_proposal_requires_share_authority(
self,
_target_validation,
) -> None:
session = _session()
self.addCleanup(_close_session, session)
_seed_access_subjects(session)
campaign = Campaign(
id="campaign-proposal",
tenant_id=TENANT_ID,
owner_user_id=USER_ID,
external_id="proposal",
name="Proposal",
)
session.add(campaign)
session.commit()
service = CampaignOwnershipService()
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
target_owner = OwnershipSubjectRef(type="user", id=OTHER_USER_ID)
denied = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="propose_transfer",
actor=current_owner,
current_owner=current_owner,
target_owner=target_owner,
)
allowed = service.authorize_ownership_action(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
action="propose_transfer",
actor=OwnershipSubjectRef(
type="user",
id=USER_ID,
scopes=frozenset({"campaigns:campaign:share"}),
),
current_owner=current_owner,
target_owner=target_owner,
)
self.assertFalse(denied.allowed)
self.assertTrue(allowed.allowed)
def test_campaign_ownership_provider_applies_only_against_expected_owner(self) -> None:
session = _session()
self.addCleanup(_close_session, session)
_seed_access_subjects(session)
campaign = Campaign(
id="campaign-owner-apply",
tenant_id=TENANT_ID,
owner_user_id=USER_ID,
external_id="owner-apply",
name="Owner apply",
)
session.add(campaign)
session.commit()
service = CampaignOwnershipService()
service.apply_owner(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
target_owner=OwnershipSubjectRef(type="group", id=GROUP_ID),
actor=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
reason=None,
)
session.flush()
self.assertIsNone(campaign.owner_user_id)
self.assertEqual(campaign.owner_group_id, GROUP_ID)
with self.assertRaisesRegex(
OwnershipTransferError,
"changed while the transfer was pending",
):
service.apply_owner(
session,
tenant_id=TENANT_ID,
resource_id=campaign.id,
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
target_owner=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
actor=OwnershipSubjectRef(type="user", id=USER_ID),
reason=None,
)
def _session(): def _session():
engine = create_engine("sqlite:///:memory:", future=True) engine = create_engine("sqlite:///:memory:", future=True)
Base.metadata.create_all(bind=engine, tables=[Account.__table__, User.__table__, Group.__table__, Campaign.__table__, CampaignShare.__table__]) Base.metadata.create_all(
bind=engine,
tables=[
Account.__table__,
User.__table__,
Group.__table__,
Campaign.__table__,
CampaignShare.__table__,
CampaignVersion.__table__,
CampaignJob.__table__,
ChangeSequenceEntry.__table__,
],
)
return sessionmaker(bind=engine, future=True)() return sessionmaker(bind=engine, future=True)()

View File

@@ -0,0 +1,245 @@
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from types import SimpleNamespace
import pytest
from govoplan_campaign.backend.report_privacy_policy import (
CampaignReportPrivacyPolicy,
CampaignReportPrivacyPolicyError,
DEFAULT_SMALL_CELL_THRESHOLD,
effective_campaign_report_privacy_policy,
)
from govoplan_campaign.backend.reports.aggregate import build_aggregate_campaign_report
class _PolicySession:
def __init__(self, settings: dict[str, object] | None = None) -> None:
self.tenant = SimpleNamespace(settings=settings or {})
def get(self, _model, _id):
return self.tenant
def _policy(threshold: int = 5) -> CampaignReportPrivacyPolicy:
return CampaignReportPrivacyPolicy(
small_cell_threshold=threshold,
source="test",
deployment_small_cell_threshold=threshold,
)
def _campaign() -> SimpleNamespace:
now = datetime(2026, 7, 22, 8, 0, tzinfo=UTC)
return SimpleNamespace(
id="campaign-safe",
tenant_id="tenant-safe",
external_id="must-not-leak-external-id",
name="Semester notification",
description="Business-safe description",
status="partially_completed",
owner_user_id="must-not-leak-owner",
current_version_id="must-not-leak-version-id",
updated_at=now,
)
def _version(*, inactive_count: int = 0) -> SimpleNamespace:
return SimpleNamespace(
id="must-not-leak-version-id",
version_number=7,
build_summary={"inactive_count": inactive_count, "build_token": "must-not-leak-token"},
execution_snapshot={"smtp": {"password": "must-not-leak-secret"}},
raw_json={"entries": [{"email": "must-not-leak@example.test"}]},
)
def _job(index: int, send_status: str, **overrides: object) -> SimpleNamespace:
started = datetime(2026, 7, 22, 8, 0, tzinfo=UTC) + timedelta(minutes=index)
values: dict[str, object] = {
"id": f"job-{index}",
"recipient_email": f"private-{index}@example.test",
"subject": f"Personal subject {index}",
"last_error": "smtp.internal.example provider-secret",
"resolved_attachments": [{"storage_key": f"private/{index}"}],
"send_status": send_status,
"validation_status": "ready",
"build_status": "built",
"queued_at": started,
"smtp_started_at": started,
"sent_at": started if send_status in {"smtp_accepted", "sent"} else None,
"outcome_unknown_at": started if send_status == "outcome_unknown" else None,
}
values.update(overrides)
return SimpleNamespace(**values)
def test_aggregate_projection_has_a_strict_recipient_free_shape() -> None:
jobs = [
*[_job(index, "smtp_accepted") for index in range(10)],
*[_job(index + 10, "failed_permanent") for index in range(5)],
]
payload = build_aggregate_campaign_report(
campaign=_campaign(), # type: ignore[arg-type]
version=_version(inactive_count=5), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=_policy(),
generated_at=datetime(2026, 7, 22, 12, 0, tzinfo=UTC),
).model_dump(mode="json")
assert set(payload) == {
"generated_at",
"campaign",
"version_number",
"completion_state",
"population",
"outcomes",
"time_range",
"privacy",
}
assert set(payload["campaign"]) == {"id", "name", "status"}
assert payload["population"]["denominator"] == {"value": 15, "suppressed": False}
assert payload["outcomes"]["smtp_accepted"] == {"value": 10, "suppressed": False}
assert payload["outcomes"]["failed"] == {"value": 5, "suppressed": False}
assert payload["completion_state"] == "partially_completed"
serialized = repr(payload)
for forbidden in (
"private-0@example.test",
"Personal subject",
"smtp.internal.example",
"provider-secret",
"storage_key",
"must-not-leak",
"Business-safe description",
"job-0",
):
assert forbidden not in serialized
def test_small_cells_use_primary_and_complementary_suppression() -> None:
jobs = [
*[_job(index, "smtp_accepted") for index in range(8)],
_job(8, "failed_permanent"),
]
report = build_aggregate_campaign_report(
campaign=_campaign(), # type: ignore[arg-type]
version=_version(inactive_count=1), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=_policy(5),
)
assert report.population.denominator.value == 9
assert report.outcomes.failed.suppressed is True
assert report.outcomes.failed.value is None
assert report.outcomes.smtp_accepted.suppressed is True
assert report.outcomes.smtp_accepted.value is None
assert report.population.inactive_source_entries.suppressed is True
assert report.time_range.suppressed is True
assert report.privacy.suppression_applied is True
def test_outcome_suppression_also_hides_overlapping_population_count() -> None:
jobs = [
*[
_job(
index,
"smtp_accepted",
validation_status="blocked" if index < 4 else "ready",
)
for index in range(8)
],
_job(8, "skipped", validation_status="excluded"),
]
report = build_aggregate_campaign_report(
campaign=_campaign(), # type: ignore[arg-type]
version=_version(inactive_count=5), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=_policy(5),
)
assert report.outcomes.excluded.suppressed is True
assert report.population.excluded_or_blocked_jobs.model_dump() == {
"value": None,
"suppressed": True,
}
# Source entries without jobs are outside the outcome denominator and do
# not overlap the suppressed partition, so their threshold-safe count stays visible.
assert report.population.inactive_source_entries.model_dump() == {
"value": 5,
"suppressed": False,
}
def test_all_small_cells_also_suppress_the_denominator_and_state() -> None:
jobs = [_job(0, "smtp_accepted"), _job(1, "failed_permanent")]
report = build_aggregate_campaign_report(
campaign=_campaign(), # type: ignore[arg-type]
version=_version(), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=_policy(5),
)
assert report.population.denominator.model_dump() == {"value": None, "suppressed": True}
assert report.completion_state == "suppressed"
assert report.outcomes.smtp_accepted.value is None
assert report.outcomes.failed.value is None
def test_explicitly_skipped_jobs_are_exclusions_not_unattempted_outcomes() -> None:
jobs = [_job(index, "skipped") for index in range(5)]
report = build_aggregate_campaign_report(
campaign=_campaign(), # type: ignore[arg-type]
version=_version(), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=_policy(5),
)
assert report.outcomes.excluded.value == 5
assert report.outcomes.not_attempted.value == 0
assert report.completion_state == "not_started"
def test_report_privacy_policy_defaults_to_five_and_tenant_can_only_strengthen() -> None:
default = effective_campaign_report_privacy_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={},
)
assert default.small_cell_threshold == DEFAULT_SMALL_CELL_THRESHOLD == 5
assert default.source == "deployment_default"
strengthened = effective_campaign_report_privacy_policy(
_PolicySession(
{"campaign_report_privacy_policy": {"small_cell_threshold": 10}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": "5"},
)
assert strengthened.small_cell_threshold == 10
assert strengthened.source == "tenant"
floor = effective_campaign_report_privacy_policy(
_PolicySession(
{"campaign_report_privacy_policy": {"small_cell_threshold": 3}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": "7"},
)
assert floor.small_cell_threshold == 7
assert floor.source == "deployment_floor"
@pytest.mark.parametrize("value", [True, 0, 1, 101, "2.5", "disabled"])
def test_invalid_report_privacy_policy_fails_closed(value: object) -> None:
with pytest.raises(CampaignReportPrivacyPolicyError):
effective_campaign_report_privacy_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": value}, # type: ignore[dict-item]
)

View File

@@ -0,0 +1,202 @@
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import Mock, patch
import pytest
from fastapi import HTTPException
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_access.backend.db.models import Account, Group, User
from govoplan_campaign.backend import router as campaign_api
from govoplan_campaign.backend.routes import campaigns as campaign_routes
from govoplan_campaign.backend.routes import jobs as job_routes
from govoplan_campaign.backend.routes import reports as report_routes
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
from govoplan_campaign.backend.reports.aggregate import (
AggregateCampaignReportError,
generate_aggregate_campaign_report,
)
from govoplan_campaign.backend.schemas import ReportEmailRequest
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.db.base import Base
from govoplan_core.tenancy.scope import Tenant, create_scope_tables
class _Principal:
def __init__(self, *scopes: str, tenant_id: str = "tenant-1") -> None:
self.scopes = set(scopes)
self.tenant_id = tenant_id
self.user = SimpleNamespace(id="reader-1")
def has(self, scope: str) -> bool:
return scope in self.scopes
def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
principal = _Principal("campaigns:report:read")
session = Mock()
campaign = SimpleNamespace(id="campaign-1", tenant_id="tenant-1")
with (
patch.object(report_routes, "_get_campaign_for_principal", return_value=campaign),
pytest.raises(HTTPException) as full_report_denied,
):
report_routes.campaign_report(
"campaign-1",
session=session,
principal=principal, # type: ignore[arg-type]
)
assert full_report_denied.value.status_code == 403
assert "campaigns:recipient:read" in full_report_denied.value.detail
with (
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
pytest.raises(HTTPException) as job_detail_denied,
):
job_routes.get_job_detail(
"campaign-1",
"job-1",
session=session,
principal=principal, # type: ignore[arg-type]
)
assert job_detail_denied.value.status_code == 403
with (
patch.object(report_routes, "_get_campaign_for_principal", return_value=campaign),
pytest.raises(HTTPException) as report_email_denied,
):
report_routes.email_campaign_report(
"campaign-1",
ReportEmailRequest(to=["auditor@example.test"]),
session=session,
principal=_Principal("campaigns:report:send"), # type: ignore[arg-type]
)
assert report_email_denied.value.status_code == 403
assert "campaigns:recipient:export" in report_email_denied.value.detail
with (
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
pytest.raises(HTTPException) as diagnostics_denied,
):
job_routes.get_job_diagnostics(
"campaign-1",
"job-1",
session=session,
principal=_Principal("campaigns:diagnostic:read"), # type: ignore[arg-type]
)
assert diagnostics_denied.value.status_code == 403
assert "campaigns:recipient:read" in diagnostics_denied.value.detail
def test_aggregate_route_uses_only_the_safe_projection() -> None:
principal = _Principal("campaigns:report:read")
session = Mock()
safe_projection = Mock()
with (
patch.object(campaign_routes, "_get_campaign_for_principal") as acl,
patch.object(
campaign_routes,
"generate_aggregate_campaign_report",
return_value=safe_projection,
) as generate,
):
result = campaign_routes.aggregate_campaign_report(
"campaign-1",
session=session,
principal=principal, # type: ignore[arg-type]
)
assert result is safe_projection
acl.assert_called_once_with(session, "campaign-1", principal)
generate.assert_called_once_with(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
version_id=None,
)
@pytest.mark.parametrize("path", ["/campaigns/aggregate-reports", "/campaigns/aggregate-reports/{campaign_id}"])
def test_aggregate_routes_require_report_read_permission(path: str) -> None:
route = next(item for item in campaign_api.router.routes if item.path == path)
dependency = next(item for item in route.dependant.dependencies if item.name == "principal")
with pytest.raises(HTTPException) as denied:
dependency.call(_Principal())
assert denied.value.status_code == 403
principal = _Principal("campaigns:report:read")
assert dependency.call(principal) is principal
def test_aggregate_projection_is_tenant_isolated_and_needs_no_optional_module() -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
create_scope_tables(engine)
Base.metadata.create_all(
engine,
tables=[
Account.__table__,
User.__table__,
Group.__table__,
Campaign.__table__,
CampaignVersion.__table__,
CampaignJob.__table__,
ChangeSequenceEntry.__table__,
],
)
with Session(engine) as session:
session.add(Tenant(id="tenant-1", slug="tenant-1", name="Tenant 1", settings={}))
campaign = Campaign(
id="campaign-1",
tenant_id="tenant-1",
external_id="external-1",
name="Safe aggregate",
description=None,
status="sent",
)
version = CampaignVersion(
id="version-1",
campaign_id=campaign.id,
version_number=1,
raw_json={"mail": {"profile_id": "optional-module-not-loaded"}},
schema_version="5",
build_summary={},
)
campaign.current_version_id = version.id
session.add_all([campaign, version])
for index in range(5):
session.add(CampaignJob(
id=f"job-{index}",
tenant_id="tenant-1",
campaign_id=campaign.id,
campaign_version_id=version.id,
entry_index=index,
recipient_email=f"private-{index}@example.test",
subject="Private",
build_status="built",
validation_status="ready",
queue_status="queued",
send_status="smtp_accepted",
imap_status="not_requested",
))
session.commit()
report = generate_aggregate_campaign_report(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
)
assert report.population.denominator.value == 5
assert report.outcomes.smtp_accepted.value == 5
with pytest.raises(AggregateCampaignReportError):
generate_aggregate_campaign_report(
session,
tenant_id="tenant-2",
campaign_id="campaign-1",
)
engine.dispose()

View File

@@ -100,9 +100,9 @@ class CampaignAttachmentBuildTests(unittest.TestCase):
cases = { cases = {
"block": ("build_failed", "blocked", 0, "block", False), "block": ("build_failed", "blocked", 0, "block", False),
"ask": ("built", "needs_review", 0, "ask", True), "ask": ("built", "needs_review", 0, "ask", True),
"drop": ("built", "excluded", 0, "drop", True), "drop": ("built", "needs_review", 0, "ask", True),
"warn": ("built", "warning", 1, "warn", True), "warn": ("built", "warning", 1, "warn", True),
"continue": ("built", "ready", 1, None, True), "continue": ("built", "warning", 1, None, True),
} }
for behavior, (build_status, validation_status, queueable_count, issue_behavior, has_mime) in cases.items(): for behavior, (build_status, validation_status, queueable_count, issue_behavior, has_mime) in cases.items():
with self.subTest(behavior=behavior): with self.subTest(behavior=behavior):
@@ -123,6 +123,9 @@ class CampaignAttachmentBuildTests(unittest.TestCase):
message = result.report.messages[0] message = result.report.messages[0]
self.assertEqual(message.build_status.value, build_status) self.assertEqual(message.build_status.value, build_status)
self.assertEqual(message.validation_status.value, validation_status) self.assertEqual(message.validation_status.value, validation_status)
if validation_status == "excluded":
self.assertEqual(message.send_status.value, "skipped")
self.assertEqual(message.imap_status.value, "skipped")
coverage_issues = [issue for issue in message.issues if issue.code == "missing_attachment_coverage"] coverage_issues = [issue for issue in message.issues if issue.code == "missing_attachment_coverage"]
if issue_behavior is None: if issue_behavior is None:
self.assertEqual(coverage_issues, []) self.assertEqual(coverage_issues, [])
@@ -130,6 +133,71 @@ class CampaignAttachmentBuildTests(unittest.TestCase):
self.assertEqual(coverage_issues[0].behavior, issue_behavior) self.assertEqual(coverage_issues[0].behavior, issue_behavior)
self.assertEqual(result.built_messages[0].mime is not None, has_mime) self.assertEqual(result.built_messages[0].mime is not None, has_mime)
def test_required_missing_policy_cannot_be_loosened_by_rule(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
campaign_file = root / "campaign.json"
campaign_file.write_text("{}", encoding="utf-8")
config = self._no_attachment_config(
behavior="continue",
configure_missing_rule=True,
)
rule = config.attachments.global_[0]
rule.required = True
rule.missing_behavior = "continue"
config.attachments.missing_behavior = "continue"
result = build_campaign_messages(
config,
campaign_file=campaign_file,
output_dir=root / "out",
write_eml=True,
)
message = result.report.messages[0]
self.assertEqual(message.validation_status.value, "blocked")
issue = next(
item
for item in message.issues
if item.code == "missing_required_attachment"
)
self.assertEqual(issue.behavior, "block")
self.assertEqual(
issue.details["effective_policy"]["requirement_policy"],
"block",
)
self.assertEqual(
message.attachments[0].missing_policy["effective_behavior"],
"block",
)
def test_optional_missing_policy_warns_by_default(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
campaign_file = root / "campaign.json"
campaign_file.write_text("{}", encoding="utf-8")
config = self._no_attachment_config(
behavior="continue",
configure_missing_rule=True,
)
config.attachments.global_[0].missing_behavior = None
result = build_campaign_messages(
config,
campaign_file=campaign_file,
output_dir=root / "out",
write_eml=True,
)
message = result.report.messages[0]
self.assertEqual(message.validation_status.value, "warning")
issue = next(
item
for item in message.issues
if item.code == "missing_optional_attachment"
)
self.assertEqual(issue.behavior, "warn")
def test_missing_pattern_does_not_create_zip_member_or_count_as_attachment(self) -> None: def test_missing_pattern_does_not_create_zip_member_or_count_as_attachment(self) -> None:
with tempfile.TemporaryDirectory() as tmp: with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp) root = Path(tmp)

View File

@@ -6,8 +6,11 @@ from unittest.mock import MagicMock, patch
import pytest import pytest
from fastapi import HTTPException from fastapi import HTTPException
from govoplan_campaign.backend import router from govoplan_campaign.backend import route_support
from govoplan_campaign.backend.routes import campaigns as campaign_routes
from govoplan_campaign.backend.routes import versions as version_routes
from govoplan_campaign.backend.schemas import CampaignUpdateRequest, CampaignVersionUpdateRequest from govoplan_campaign.backend.schemas import CampaignUpdateRequest, CampaignVersionUpdateRequest
from govoplan_core.core.concurrency import strong_resource_etag
def _principal() -> SimpleNamespace: def _principal() -> SimpleNamespace:
@@ -26,6 +29,7 @@ def test_version_update_rolls_back_when_its_audit_record_cannot_be_written() ->
raw_json={}, raw_json={},
current_flow="manual", current_flow="manual",
current_step="recipients", current_step="recipients",
edit_revision=1,
) )
def mutate(*_args, **kwargs): def mutate(*_args, **kwargs):
@@ -34,18 +38,26 @@ def test_version_update_rolls_back_when_its_audit_record_cannot_be_written() ->
return version return version
with ( with (
patch.object(router, "_get_campaign_for_principal"), patch.object(route_support, "_get_campaign_for_principal"),
patch.object(router, "_get_version_for_tenant", return_value=version), patch.object(route_support, "_get_version_for_tenant", return_value=version),
patch.object(router, "update_campaign_version", side_effect=mutate), patch.object(route_support, "update_campaign_version", side_effect=mutate),
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")), patch.object(route_support, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
): ):
with pytest.raises(HTTPException, match="audit unavailable") as captured: with pytest.raises(HTTPException, match="audit unavailable") as captured:
router._update_campaign_version_detail_response( # noqa: SLF001 - transaction regression test route_support._update_campaign_version_detail_response( # noqa: SLF001 - transaction regression test
session, session,
principal, # type: ignore[arg-type] principal, # type: ignore[arg-type]
"campaign-1", "campaign-1",
"version-1", "version-1",
CampaignVersionUpdateRequest(current_step="recipients"), CampaignVersionUpdateRequest(
current_step="recipients",
base_revision=1,
),
if_match=strong_resource_etag(
"campaign_version",
"version-1",
1,
),
autosave=True, autosave=True,
audit_action="campaign.version_autosaved", audit_action="campaign.version_autosaved",
) )
@@ -68,15 +80,15 @@ def test_version_fork_rolls_back_when_its_audit_record_cannot_be_written() -> No
return forked return forked
with ( with (
patch.object(router, "_get_campaign_for_principal", return_value=campaign), patch.object(version_routes, "_get_campaign_for_principal", return_value=campaign),
patch.object(router, "_require_permission"), patch.object(version_routes, "_require_permission"),
patch.object(router, "_get_version_for_tenant", return_value=source), patch.object(version_routes, "_get_version_for_tenant", return_value=source),
patch.object(router, "_get_campaign_for_tenant", return_value=campaign), patch.object(version_routes, "_get_campaign_for_tenant", return_value=campaign),
patch.object(router, "fork_campaign_version_for_edit", side_effect=mutate), patch.object(version_routes, "fork_campaign_version_for_edit", side_effect=mutate),
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")), patch.object(version_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
): ):
with pytest.raises(RuntimeError, match="audit unavailable"): with pytest.raises(RuntimeError, match="audit unavailable"):
router.fork_version_for_edit( version_routes.fork_version_for_edit(
"campaign-1", "campaign-1",
"version-1", "version-1",
CampaignVersionUpdateRequest(), CampaignVersionUpdateRequest(),
@@ -102,12 +114,12 @@ def test_metadata_update_rolls_back_when_its_audit_record_cannot_be_written() ->
) )
with ( with (
patch.object(router, "_get_campaign_for_principal", return_value=campaign), patch.object(campaign_routes, "_get_campaign_for_principal", return_value=campaign),
patch.object(router, "_sync_campaign_metadata_to_current_version"), patch.object(campaign_routes, "_sync_campaign_metadata_to_current_version"),
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")), patch.object(campaign_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
): ):
with pytest.raises(RuntimeError, match="audit unavailable"): with pytest.raises(RuntimeError, match="audit unavailable"):
router.update_campaign_metadata_endpoint( campaign_routes.update_campaign_metadata_endpoint(
"campaign-1", "campaign-1",
CampaignUpdateRequest(name="New name"), CampaignUpdateRequest(name="New name"),
session=session, session=session,

View File

@@ -0,0 +1,168 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from sqlalchemy import Column, String, Table, create_engine
from sqlalchemy.orm import Session, sessionmaker
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignIssue,
CampaignVersion,
)
from govoplan_campaign.backend.persistence.versions import update_campaign_version
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.core.concurrency import RevisionConflictError
from govoplan_core.db.base import Base
class CampaignOptimisticConcurrencyTests(unittest.TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
database_path = Path(self.temp_dir.name) / "campaign.db"
self.engine = create_engine(f"sqlite+pysqlite:///{database_path}")
access_users = Base.metadata.tables.get("access_users")
if access_users is None:
access_users = Table(
"access_users",
Base.metadata,
Column("id", String(36), primary_key=True),
)
access_groups = Base.metadata.tables.get("access_groups")
if access_groups is None:
access_groups = Table(
"access_groups",
Base.metadata,
Column("id", String(36), primary_key=True),
)
Base.metadata.create_all(
self.engine,
tables=[
access_users,
access_groups,
ChangeSequenceEntry.__table__,
Campaign.__table__,
CampaignVersion.__table__,
CampaignIssue.__table__,
],
)
self.SessionLocal = sessionmaker(
bind=self.engine,
class_=Session,
expire_on_commit=False,
)
with self.SessionLocal() as session:
campaign = Campaign(
id="campaign-1",
tenant_id="tenant-1",
external_id="C-1",
name="Campaign",
current_version_id="version-1",
)
version = CampaignVersion(
id="version-1",
campaign_id=campaign.id,
version_number=1,
raw_json={
"version": "1.0",
"campaign": {
"id": "C-1",
"name": "Campaign",
"description": "Base",
},
},
)
session.add_all((campaign, version))
session.commit()
self.addCleanup(self.engine.dispose)
self.addCleanup(self.temp_dir.cleanup)
@staticmethod
def _update(
session: Session,
*,
expected_revision: int,
name: str,
) -> CampaignVersion:
with (
patch(
"govoplan_campaign.backend.persistence.versions._updated_runtime_json",
side_effect=lambda _session, **kwargs: kwargs["raw_json"],
),
patch(
"govoplan_campaign.backend.persistence.versions._write_campaign_snapshot"
),
):
return update_campaign_version(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
version_id="version-1",
raw_json={
"version": "1.0",
"campaign": {
"id": "C-1",
"name": name,
"description": "Base",
},
},
expected_revision=expected_revision,
)
def test_only_one_of_two_writers_can_commit_the_same_revision(self) -> None:
first = self.SessionLocal()
second = self.SessionLocal()
self.addCleanup(first.close)
self.addCleanup(second.close)
first.get(CampaignVersion, "version-1")
second.get(CampaignVersion, "version-1")
saved = self._update(
first,
expected_revision=1,
name="First writer",
)
self.assertEqual(saved.edit_revision, 2)
with self.assertRaises(RevisionConflictError) as raised:
self._update(
second,
expected_revision=1,
name="Second writer",
)
self.assertEqual(raised.exception.current_revision, 2)
self.assertEqual(raised.exception.submitted_base_revision, 1)
with self.SessionLocal() as verification:
current = verification.get(CampaignVersion, "version-1")
assert current is not None
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
self.assertEqual(current.edit_revision, 2)
def test_stale_revision_is_rejected_before_mutation(self) -> None:
with self.SessionLocal() as first:
self._update(
first,
expected_revision=1,
name="First writer",
)
with self.SessionLocal() as stale:
with self.assertRaises(RevisionConflictError):
self._update(
stale,
expected_revision=1,
name="Stale writer",
)
stale.rollback()
with self.SessionLocal() as verification:
current = verification.get(CampaignVersion, "version-1")
assert current is not None
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
if __name__ == "__main__":
unittest.main()

View File

@@ -5,10 +5,14 @@ from types import SimpleNamespace
import unittest import unittest
from unittest.mock import patch from unittest.mock import patch
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
from govoplan_campaign.backend.reports.campaigns import ( from govoplan_campaign.backend.reports.campaigns import (
_campaign_postbox_receipt_summary,
_job_evidence_row, _job_evidence_row,
_latest_by_job_id, _latest_by_job_id,
_load_delivery_info, _load_delivery_info,
_review_decision_summary,
_review_decisions_by_job,
generate_campaign_report, generate_campaign_report,
) )
@@ -82,7 +86,21 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
updated_at=_dt(), updated_at=_dt(),
) )
row = _job_evidence_row(job, latest_smtp=smtp, latest_imap=imap) row = _job_evidence_row(
job,
latest_smtp=smtp,
latest_imap=imap,
review_decision={
"decision": "accept",
"reason": "Recipient confirmed no attachment was expected.",
"actor_user_id": "reviewer-1",
"decided_at": "2026-07-08T12:30:00+00:00",
"review_key": "recipient-1",
"message_sha256": "abc123",
"issue_fingerprint": "def456",
"issue_codes": ["missing_optional_attachment"],
},
)
assert row["campaign_id"] == "campaign-1" assert row["campaign_id"] == "campaign-1"
assert row["campaign_version_id"] == "version-1" assert row["campaign_version_id"] == "version-1"
@@ -101,6 +119,38 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
assert "latest_imap_error_message" not in row assert "latest_imap_error_message" not in row
assert "eml_storage_key" not in row assert "eml_storage_key" not in row
assert "eml_local_path" not in row assert "eml_local_path" not in row
assert row["review_decision"] == "accept"
assert row["review_actor_user_id"] == "reviewer-1"
assert row["review_issue_codes"] == "missing_optional_attachment"
assert "review_message_sha256" not in row
def test_report_uses_only_review_decisions_for_the_current_build() -> None:
decision = {
"job_id": "job-1",
"decision": "accept",
"issue_codes": ["missing_optional_attachment"],
}
version = SimpleNamespace(
build_summary={"build_token": "build-2"},
editor_state={
"review_send": {
"build_token": "build-2",
"inspection_complete": True,
"issue_decisions": [decision],
}
},
)
decisions = _review_decisions_by_job(version)
assert decisions == {"job-1": decision}
assert _review_decision_summary(decisions) == {
"exception_decision_count": 1,
"by_issue_code": {"missing_optional_attachment": 1},
}
version.editor_state["review_send"]["build_token"] = "stale-build"
assert _review_decisions_by_job(version) == {}
def test_latest_by_job_id_keeps_highest_attempt_number() -> None: def test_latest_by_job_id_keeps_highest_attempt_number() -> None:
@@ -161,6 +211,64 @@ def test_aggregate_report_omits_recipient_level_failures_by_default() -> None:
assert payload.call_args.kwargs["include_recent_failures"] is False assert payload.call_args.kwargs["include_recent_failures"] is False
def test_postbox_receipt_report_aggregates_provider_state_without_identities() -> None:
class _Query:
def join(self, *args, **kwargs):
return self
def filter(self, *args, **kwargs):
return self
def all(self):
return [("delivery-1",), ("delivery-2",), ("delivery-1",)]
integration = SimpleNamespace(
receipt_evidence_available=True,
delivery_receipt_summaries=lambda session, **kwargs: {
"delivery-1": PostboxDeliveryReceiptSummaryRef(
delivery_id="delivery-1",
message_id="message-1",
postbox_id="postbox-1",
delivery_status="accepted",
accepted_at=_dt(),
currently_readable=True,
read_receipt_count=2,
acknowledged_receipt_count=1,
routed_message_count=1,
),
"delivery-2": PostboxDeliveryReceiptSummaryRef(
delivery_id="delivery-2",
message_id="message-2",
postbox_id="postbox-2",
delivery_status="accepted_vacant",
accepted_at=_dt(),
currently_readable=False,
expired_message_count=1,
),
},
)
session = SimpleNamespace(query=lambda *args: _Query())
with patch(
"govoplan_campaign.backend.reports.campaigns.postbox_integration",
return_value=integration,
):
report = _campaign_postbox_receipt_summary(
session, # type: ignore[arg-type]
tenant_id="tenant-1",
campaign_id="campaign-1",
version=SimpleNamespace(id="version-1"),
)
assert report["delivery_count"] == 2
assert report["currently_readable_delivery_count"] == 1
assert report["read_delivery_count"] == 1
assert report["acknowledged_delivery_count"] == 1
assert report["routed_message_count"] == 1
assert report["expired_message_count"] == 1
assert "account_id" not in repr(report)
assert "identity_id" not in repr(report)
class CampaignReportProjectionTests(unittest.TestCase): class CampaignReportProjectionTests(unittest.TestCase):
def test_evidence_projection(self) -> None: def test_evidence_projection(self) -> None:
test_job_evidence_row_contains_transport_and_message_evidence() test_job_evidence_row_contains_transport_and_message_evidence()

View File

@@ -0,0 +1,314 @@
from __future__ import annotations
import importlib.util
import json
from pathlib import Path
from types import SimpleNamespace
import sys
import tempfile
from unittest import mock
import pytest
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
RUNNER_PATH = (
REPOSITORY_ROOT
/ "dev"
/ "mail-testbed"
/ "run_celery_redelivery_acceptance.py"
)
COMPOSE_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "docker-compose.yml"
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
TASK_ID = "12345678-1234-4234-8234-123456789abc"
def _load_runner():
spec = importlib.util.spec_from_file_location(
"govoplan_campaign_celery_redelivery_acceptance",
RUNNER_PATH,
)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
runner = _load_runner()
class _Response:
def __init__(self, status_code: int, payload: dict) -> None:
self.status_code = status_code
self._payload = payload
def json(self) -> dict:
return self._payload
class _Client:
def post(self, path: str, **_kwargs) -> _Response:
assert path.endswith("/queue")
return _Response(
200,
{
"queued_count": 1,
"skipped_count": 0,
"blocked_count": 0,
"enqueued_count": 1,
"delivery_mode": "worker_queue",
"worker_queue_available": True,
"dry_run": False,
},
)
def get(self, path: str, **_kwargs) -> _Response:
assert path.endswith("/report")
return _Response(
200,
{
"cards": {
"jobs_total": 1,
"outcome_unknown": 1,
"needs_attention": 1,
},
"status_counts": {
"send": {"outcome_unknown": 1},
"imap": {"pending": 1},
},
},
)
class _Endpoint:
host = "127.0.0.1"
port = 4025
def __init__(self) -> None:
self.release_count = 0
def wait_for_data(self, _timeout_seconds: int) -> bool:
return True
def release_held_connection(self) -> None:
self.release_count += 1
def evidence(self) -> dict[str, int]:
return {
"connection_count": 1,
"accepted_rcpt_commands": 1,
"refused_rcpt_commands": 0,
"data_transactions": 1,
}
def _settings():
return runner.TestbedSettings(
smtp_host="127.0.0.1",
smtp_port=3025,
imap_host="127.0.0.1",
imap_port=3143,
username="campaign-test@govoplan.test",
password="local-test-password",
sender="campaign-test@govoplan.test",
recipient="campaign-test@govoplan.test",
sent_folder="Sent",
provider_timeout_seconds=5,
)
def test_compose_redis_is_isolated_durable_and_health_checked() -> None:
compose = COMPOSE_PATH.read_text(encoding="utf-8")
assert "redis:7-alpine" in compose
assert '"--appendonly", "yes"' in compose
assert "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379" in compose
assert 'test: ["CMD", "redis-cli", "ping"]' in compose
assert "campaign-redis-data:/data" in compose
def test_runbook_keeps_local_redelivery_distinct_from_production_supervision() -> None:
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(
encoding="utf-8"
)
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(
encoding="utf-8"
)
assert "run_celery_redelivery_acceptance.py" in testbed
assert "same Celery task identity must be redelivered" in testbed
assert "production daemon supervision" in testbed
assert "empty broker queue/unacked set" in runbook
assert "production process manager" in runbook
def test_compose_lifecycle_targets_only_isolated_redis_service() -> None:
up = runner._compose_command(
compose_file=COMPOSE_PATH,
project_name="govoplan-campaign-redelivery-test",
operation="up",
)
down = runner._compose_command(
compose_file=COMPOSE_PATH,
project_name="govoplan-campaign-redelivery-test",
operation="down",
)
assert up[-3:] == ["up", "--detach", "redis"]
assert down[-3:] == ["down", "--volumes", "--remove-orphans"]
assert "greenmail" not in up
assert "--project-name" in up
def test_worker_bootstrap_uses_real_late_ack_solo_celery_worker() -> None:
source = runner.WORKER_BOOTSTRAP
assert "celery.worker_main" in source
assert '"--pool=solo"' in source
assert '"--queues=send_email"' in source
assert '"visibility_timeout"' in source
assert '"polling_interval"' in source
assert "send_email.run" not in source
def test_runtime_root_uses_platform_temp_selection() -> None:
with mock.patch(
"govoplan_campaign_celery_redelivery_acceptance.tempfile.mkdtemp",
return_value="/selected-temp/govoplan-campaign-celery-redelivery-test",
) as mkdtemp:
runtime_root = runner._create_runtime_root()
assert runtime_root == Path(
"/selected-temp/govoplan-campaign-celery-redelivery-test"
)
mkdtemp.assert_called_once_with(prefix="govoplan-campaign-celery-redelivery-")
def test_worker_log_projection_matches_redelivered_task_without_retaining_id() -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
log_path = Path(temporary_directory) / "worker.log"
log_path.write_text(
"\n".join(
[
f"Task govoplan.campaigns.send_email[{TASK_ID}] received",
f"Task govoplan.campaigns.send_email[{TASK_ID}] succeeded in 0.1s",
]
),
encoding="utf-8",
)
with log_path.open("ab") as handle:
worker = runner.WorkerProcess(
process=SimpleNamespace(),
log_path=log_path,
log_handle=handle,
)
assert worker.received_task_ids() == (TASK_ID,)
assert worker.succeeded_task_ids() == (TASK_ID,)
def test_queue_projection_fails_closed_if_no_task_was_published() -> None:
with pytest.raises(runner.AcceptanceError, match="one Celery task"):
runner._queue_evidence(
{
"queued_count": 1,
"skipped_count": 0,
"blocked_count": 0,
"enqueued_count": 0,
"delivery_mode": "database_queue",
"worker_queue_available": False,
"dry_run": False,
}
)
def test_redelivery_orchestration_requires_same_task_and_no_second_smtp_effect(
monkeypatch,
) -> None:
first_worker = mock.Mock()
first_worker.received_task_ids.return_value = (TASK_ID,)
replacement_worker = mock.Mock()
replacement_worker.received_task_ids.return_value = (TASK_ID,)
workers = iter([first_worker, replacement_worker])
endpoint = _Endpoint()
durable_states = iter(
[
{
"job_count": 1,
"send_status_counts": {"sending": 1},
"attempt_status_counts": {"smtp_in_progress": 1},
"unfinished_attempt_count": 1,
},
{
"job_count": 1,
"send_status_counts": {"outcome_unknown": 1},
"attempt_status_counts": {"outcome_unknown": 1},
"unfinished_attempt_count": 0,
},
]
)
prepared = SimpleNamespace(
campaign_id="campaign-internal",
version_id="version-internal",
public_evidence=lambda: {
"validation": {"ok": True},
"build": {"built_count": 1},
"campaign_mail_boundary": {
"profile_reference_only": True,
"smtp_revision_frozen": True,
"imap_revision_frozen": True,
"resolved_transport_material_present": False,
},
},
)
monkeypatch.setattr(runner, "create_mail_profile", lambda *args, **kwargs: "profile-internal")
monkeypatch.setattr(runner, "prepare_campaign_scenario", lambda *args, **kwargs: prepared)
monkeypatch.setattr(runner, "_start_worker", lambda *args, **kwargs: next(workers))
monkeypatch.setattr(runner, "_wait_for_worker_ready", lambda *args, **kwargs: None)
received = iter([TASK_ID, TASK_ID])
monkeypatch.setattr(runner, "_wait_for_received_task", lambda *args, **kwargs: next(received))
monkeypatch.setattr(runner, "_wait_for_task_success", lambda *args, **kwargs: None)
monkeypatch.setattr(runner, "_kill_worker", lambda *args, **kwargs: -9)
monkeypatch.setattr(runner, "_stop_worker", lambda *args, **kwargs: None)
monkeypatch.setattr(
runner,
"_wait_for_broker_drained",
lambda *args, **kwargs: runner.RedisBrokerState(0, 0, 0),
)
evidence = runner.execute_redelivery_scenario(
_Client(),
{"Authorization": "not-retained"},
fixture_path=FIXTURE_PATH,
settings=_settings(),
endpoint=endpoint,
redis_url="redis://127.0.0.1:36379/0",
runtime_root=Path("/not-used"),
snapshot_probe=lambda _version_id: ({}, {}),
audit_probe=lambda _campaign_id, _version_id: {
"campaign.created": 1,
"campaign.validated": 1,
"campaign.messages_built": 1,
"campaign.queued": 1,
},
delivery_probe=lambda _campaign_id, _version_id: next(durable_states),
)
assert evidence["broker"] == {
"transport": "redis",
"same_task_identity_redelivered": True,
"first_worker_received_count": 1,
"replacement_worker_received_count": 1,
"queue_depth": 0,
"unacked_hash_count": 0,
"unacked_index_count": 0,
}
assert evidence["protocol"]["connection_count"] == 1
assert evidence["protocol"]["data_transactions"] == 1
assert evidence["recovered_durable_state"]["send_status_counts"] == {
"outcome_unknown": 1
}
assert TASK_ID not in json.dumps(evidence, sort_keys=True)
assert endpoint.release_count >= 1

View File

@@ -1,7 +1,9 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import patch
import pytest import pytest
@@ -12,6 +14,7 @@ from govoplan_core.core.modules import DocumentationContext
@dataclass(frozen=True) @dataclass(frozen=True)
class _Principal: class _Principal:
scopes: frozenset[str] scopes: frozenset[str]
tenant_id: str = "tenant-1"
def has(self, scope: str) -> bool: def has(self, scope: str) -> bool:
namespace, resource, _action = scope.split(":", 2) namespace, resource, _action = scope.split(":", 2)
@@ -181,6 +184,32 @@ def test_runtime_documentation_full_composition_uses_only_user_facing_names() ->
assert "secret" not in rendered.lower() assert "secret" not in rendered.lower()
def test_runtime_documentation_states_the_effective_synchronous_limit() -> None:
session = SimpleNamespace(
get=lambda _model, _id: SimpleNamespace(
settings={
"campaign_delivery_policy": {
"synchronous_send_max_recipients": 12,
}
}
)
)
with patch.dict("os.environ", {"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "25"}):
topic = documentation_topics(
DocumentationContext(
registry=_Registry({"mail.campaign_delivery"}),
principal=_Principal(frozenset({"campaigns:campaign:read", "campaigns:campaign:send"})),
session=session,
documentation_type="user",
)
)[0]
assert any(
"Send now is limited to 12 eligible recipient job(s)" in item
for item in topic.metadata["current_configuration"]
)
def _visible_static_topics( def _visible_static_topics(
scopes: set[str], scopes: set[str],
*, *,
@@ -256,6 +285,12 @@ def test_sender_sees_queue_and_send_only_with_the_mail_contract_and_profile_auth
assert "campaigns.workflow.queue-delivery" in with_mail assert "campaigns.workflow.queue-delivery" in with_mail
assert "campaigns.workflow.send-small-controlled-run" in with_mail assert "campaigns.workflow.send-small-controlled-run" in with_mail
queue_topic = next(
topic for topic in CAMPAIGN_USER_DOCUMENTATION
if topic.id == "campaigns.workflow.queue-delivery"
)
assert any(link.href == "/campaigns/queue" for link in queue_topic.links)
def test_connected_authoring_tasks_require_their_declared_contracts_and_permissions() -> None: def test_connected_authoring_tasks_require_their_declared_contracts_and_permissions() -> None:
attachment_scopes = { attachment_scopes = {
@@ -318,6 +353,30 @@ def test_report_export_and_lifecycle_tasks_are_independently_permission_gated()
assert "campaigns.workflow.export-delivery-report" not in custodian assert "campaigns.workflow.export-delivery-report" not in custodian
def test_aggregate_report_task_never_implies_recipient_detail_or_export_authority() -> None:
visible = _visible_static_topics({"campaigns:report:read"})
assert visible == {"campaigns.workflow.view-aggregate-delivery-report"}
topic = next(
item for item in CAMPAIGN_USER_DOCUMENTATION
if item.id == "campaigns.workflow.view-aggregate-delivery-report"
)
assert topic.metadata["route"] == "/campaigns/reports"
assert "export" in topic.metadata["verification"].lower()
def test_handbook_distinguishes_shipped_aggregate_reports_from_detailed_report_gaps() -> None:
handbook = " ".join(
(
Path(__file__).resolve().parents[1] / "docs" / "CAMPAIGN_HANDBOOK.md"
).read_text(encoding="utf-8").lower().split()
)
assert "aggregate-only reader ui remains open" not in handbook
assert "separate aggregate **reports** surface" in handbook
assert "permission-aware action visibility on that detailed surface remains open work" in handbook
def test_static_campaign_handbook_has_unique_ids_help_contexts_and_no_planned_resend_claim() -> None: def test_static_campaign_handbook_has_unique_ids_help_contexts_and_no_planned_resend_claim() -> None:
from govoplan_campaign.backend.manifest import get_manifest from govoplan_campaign.backend.manifest import get_manifest

View File

@@ -0,0 +1,207 @@
from __future__ import annotations
import json
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
FIXTURE_ROOT = REPOSITORY_ROOT / "examples" / "simple-announcement"
_ISOLATED_ACCEPTANCE_PROGRAM = r"""
from __future__ import annotations
import hashlib
import importlib.abc
import json
import socket
import sys
from email import policy
from email.parser import BytesParser
from pathlib import Path
source_root = Path(sys.argv[1]).resolve()
fixture_root = Path(sys.argv[2]).resolve()
sys.path.insert(0, str(source_root))
class AbsentOptionalModuleFinder(importlib.abc.MetaPathFinder):
absent_roots = {"govoplan_files", "govoplan_mail"}
def find_spec(self, fullname, path=None, target=None):
if fullname.partition(".")[0] in self.absent_roots:
raise ModuleNotFoundError(
f"{fullname} is intentionally absent in the Campaign fixture check",
name=fullname,
)
return None
sys.meta_path.insert(0, AbsentOptionalModuleFinder())
def deny_network(*args, **kwargs):
raise AssertionError("the Campaign validate/build fixture must not open a network connection")
class NoNetworkSocket(socket.socket):
def connect(self, address):
deny_network(address)
def connect_ex(self, address):
deny_network(address)
socket.create_connection = deny_network
socket.socket = NoNetworkSocket
from govoplan_campaign.backend.campaign import ( # noqa: E402
load_campaign_config,
load_campaign_json,
validate_campaign_config,
)
from govoplan_campaign.backend.messages import build_campaign_messages # noqa: E402
metadata = json.loads((fixture_root / "fixture.json").read_text(encoding="utf-8"))
assert metadata["required_modules"] == ["core", "access", "campaigns"]
assert metadata["absent_optional_modules"] == ["files", "mail"]
assert metadata["external_effects"] == "forbidden"
campaign_file = fixture_root / metadata["campaign_file"]
raw_campaign = load_campaign_json(campaign_file)
def iter_keys(value):
if isinstance(value, dict):
for key, nested in value.items():
yield key.casefold()
yield from iter_keys(nested)
elif isinstance(value, list):
for nested in value:
yield from iter_keys(nested)
forbidden_key_fragments = ("credential", "imap", "mail_profile", "password", "secret", "smtp")
assert not [
key
for key in iter_keys(raw_campaign)
if any(fragment in key for fragment in forbidden_key_fragments)
]
config = load_campaign_config(campaign_file)
assert config.server.mail_profile_id is None
assert not config.attachments.global_
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
assert validation.ok
assert validation.error_count == 0
assert validation.warning_count == 0
assert validation.entries_count == metadata["expected"]["entries_count"]
def build(output_name):
return build_campaign_messages(
config,
campaign_file=campaign_file,
output_dir=fixture_root.parent / output_name,
write_eml=True,
)
first = build("build-first")
second = build("build-second")
expected = metadata["expected"]
for result in (first, second):
assert result.report.campaign_id == expected["campaign_id"]
assert result.report.entries_count == expected["entries_count"]
assert result.report.built_count == expected["built_count"]
assert result.report.build_failed_count == 0
assert result.report.queueable_count == expected["queueable_count"]
assert len(result.built_messages) == 1
built = result.built_messages[0]
assert built.mime is not None
assert built.draft.subject == expected["subject"]
assert built.draft.validation_status.value == "ready"
assert built.draft.send_status.value == "draft"
assert built.draft.imap_status.value == "not_requested"
assert built.draft.attachment_count == expected["attachment_count"]
assert not built.draft.attachments
assert not built.draft.issues
assert built.draft.from_ is not None
assert built.draft.from_.email == "announcements@example.test"
assert [address.email for address in built.draft.to] == ["recipient@example.test"]
assert built.mime["Subject"] == expected["subject"]
assert "Hello Example Recipient" in built.mime.get_content()
assert list(built.mime.iter_attachments()) == []
def normalized_eml(path_value):
message = BytesParser(policy=policy.default).parsebytes(Path(path_value).read_bytes())
del message["Date"]
del message["Message-ID"]
return message.as_bytes(policy=policy.default)
first_eml = normalized_eml(first.report.messages[0].eml_path)
second_eml = normalized_eml(second.report.messages[0].eml_path)
assert first_eml == second_eml
assert not any(
name == root or name.startswith(root + ".")
for name in sys.modules
for root in ("govoplan_files", "govoplan_mail")
)
print(json.dumps({
"campaign_id": first.report.campaign_id,
"built_count": first.report.built_count,
"queueable_count": first.report.queueable_count,
"normalized_eml_sha256": hashlib.sha256(first_eml).hexdigest(),
}, sort_keys=True))
"""
class CampaignExampleAcceptanceTests(unittest.TestCase):
def test_simple_announcement_validates_and_builds_without_mail_or_files(self) -> None:
self.assertTrue((FIXTURE_ROOT / "campaign.json").is_file())
self.assertTrue((FIXTURE_ROOT / "fixture.json").is_file())
with tempfile.TemporaryDirectory(prefix="govoplan-campaign-acceptance-", dir="/tmp") as temp_dir:
workspace = Path(temp_dir).resolve()
self.assertNotIn(REPOSITORY_ROOT, workspace.parents)
isolated_fixture = workspace / "simple-announcement"
shutil.copytree(FIXTURE_ROOT, isolated_fixture)
completed = subprocess.run(
[
sys.executable,
"-I",
"-c",
_ISOLATED_ACCEPTANCE_PROGRAM,
str(REPOSITORY_ROOT / "src"),
str(isolated_fixture),
],
cwd=workspace,
check=False,
capture_output=True,
text=True,
timeout=30,
)
self.assertEqual(completed.returncode, 0, completed.stderr or completed.stdout)
evidence = json.loads(completed.stdout)
self.assertEqual(evidence["campaign_id"], "simple-announcement")
self.assertEqual(evidence["built_count"], 1)
self.assertEqual(evidence["queueable_count"], 1)
self.assertRegex(evidence["normalized_eml_sha256"], r"^[0-9a-f]{64}$")
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,67 @@
from __future__ import annotations
from importlib import import_module
from unittest.mock import patch
from sqlalchemy import create_engine, text
from govoplan_campaign.backend.campaign.models import BuildStatus, SendStatus
from govoplan_campaign.backend.messages.models import ImapStatus, MessageDraft, MessageValidationStatus
from govoplan_campaign.backend.persistence.campaigns import _job_from_message
def test_excluded_message_persists_explicit_skipped_transport_states() -> None:
message = MessageDraft(
entry_index=0,
entry_id="excluded-entry",
active=True,
build_status=BuildStatus.BUILT,
validation_status=MessageValidationStatus.EXCLUDED,
send_status=SendStatus.SKIPPED,
imap_status=ImapStatus.SKIPPED,
)
job = _job_from_message(
tenant_id="tenant-1",
campaign_id="campaign-1",
version_id="version-1",
message=message,
)
assert job.send_status == "skipped"
assert job.imap_status == "skipped"
def test_status_migration_only_normalizes_excluded_rows_without_transport_evidence() -> None:
migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"d8b3e2c1f4a5_v0110_excluded_delivery_skipped"
)
engine = create_engine("sqlite+pysqlite:///:memory:")
with engine.begin() as connection:
connection.execute(text(
"CREATE TABLE campaign_jobs ("
"id TEXT PRIMARY KEY, validation_status TEXT NOT NULL, "
"send_status TEXT NOT NULL, imap_status TEXT NOT NULL)"
))
connection.execute(
text(
"INSERT INTO campaign_jobs (id, validation_status, send_status, imap_status) VALUES "
"('untouched', 'excluded', 'not_queued', 'pending'), "
"('evidence', 'excluded', 'smtp_accepted', 'appended'), "
"('queueable', 'ready', 'not_queued', 'pending')"
)
)
with patch.object(migration.op, "get_bind", return_value=connection):
migration.upgrade()
rows = {
row.id: (row.send_status, row.imap_status)
for row in connection.execute(
text("SELECT id, send_status, imap_status FROM campaign_jobs ORDER BY id")
)
}
assert rows["untouched"] == ("skipped", "skipped")
assert rows["evidence"] == ("smtp_accepted", "appended")
assert rows["queueable"] == ("not_queued", "pending")

View File

@@ -94,7 +94,7 @@ def _ensure(session: _Session, version) -> None:
ensure_execution_snapshot(session, version) # type: ignore[arg-type] ensure_execution_snapshot(session, version) # type: ignore[arg-type]
def test_v5_snapshot_requires_its_persisted_checksum() -> None: def test_current_snapshot_requires_its_persisted_checksum() -> None:
job = _job() job = _job()
version = _snapshotted_version(job) version = _snapshotted_version(job)
version.execution_snapshot_hash = None version.execution_snapshot_hash = None

View File

@@ -7,9 +7,11 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
import pytest import pytest
from sqlalchemy import create_engine, text from alembic.migration import MigrationContext
from alembic.operations import Operations
from sqlalchemy import create_engine, inspect, text
from govoplan_campaign.backend import router from govoplan_campaign.backend.routes import delivery as router
from govoplan_campaign.backend.db.models import ( from govoplan_campaign.backend.db.models import (
CampaignJob, CampaignJob,
ImapAppendAttempt, ImapAppendAttempt,
@@ -128,6 +130,10 @@ def test_post_provider_persistence_failure_freezes_imap_retry() -> None:
mail_profile_id="profile-1", mail_profile_id="profile-1",
smtp_transport_revision="smtp-revision", smtp_transport_revision="smtp-revision",
imap_transport_revision="imap-revision", imap_transport_revision="imap-revision",
smtp_server_id=None,
smtp_credential_id=None,
imap_server_id=None,
imap_credential_id=None,
delivery=SimpleNamespace( delivery=SimpleNamespace(
imap_append_sent=SimpleNamespace(enabled=True, folder="Sent"), imap_append_sent=SimpleNamespace(enabled=True, folder="Sent"),
), ),
@@ -218,6 +224,31 @@ def test_imap_claim_migration_preserves_and_renumbers_duplicate_attempts() -> No
] ]
def test_imap_attempt_claim_repair_adds_only_the_missing_column() -> None:
migration = importlib.import_module(
"govoplan_campaign.backend.migrations.versions.e9f0a1b2c3d4_v0114_repair_imap_attempt_claim"
)
engine = create_engine("sqlite+pysqlite:///:memory:")
with engine.begin() as connection:
connection.execute(
text(
"CREATE TABLE imap_append_attempts ("
"id VARCHAR(36) PRIMARY KEY, job_id VARCHAR(36) NOT NULL)"
)
)
context = MigrationContext.configure(connection)
with patch.object(migration, "op", Operations(context)):
migration.upgrade()
migration.upgrade()
columns = {
column["name"]
for column in inspect(connection).get_columns("imap_append_attempts")
}
assert columns == {"id", "job_id", "claim_token"}
@pytest.mark.parametrize("decision", ["smtp_accepted", "not_sent", "imap_appended", "imap_not_appended"]) @pytest.mark.parametrize("decision", ["smtp_accepted", "not_sent", "imap_appended", "imap_not_appended"])
def test_reconciliation_requires_an_evidence_note(decision: str) -> None: def test_reconciliation_requires_an_evidence_note(decision: str) -> None:
with pytest.raises(ValueError, match="evidence note"): with pytest.raises(ValueError, match="evidence note"):
@@ -379,14 +410,14 @@ def test_reconciliation_rolls_back_state_when_audit_fails(
return {"decision": decision, "job_id": "job-1"} return {"decision": decision, "job_id": "job-1"}
with ( with (
patch("govoplan_campaign.backend.router._get_campaign_for_principal"), patch("govoplan_campaign.backend.routes.delivery._get_campaign_for_principal"),
patch("govoplan_campaign.backend.router._require_permission"), patch("govoplan_campaign.backend.routes.delivery._require_permission"),
patch( patch(
"govoplan_campaign.backend.router.reconcile_job_outcome", "govoplan_campaign.backend.routes.delivery.reconcile_job_outcome",
side_effect=mutate_without_commit, side_effect=mutate_without_commit,
), ),
patch( patch(
"govoplan_campaign.backend.router.audit_from_principal", "govoplan_campaign.backend.routes.delivery.audit_from_principal",
side_effect=RuntimeError("audit unavailable"), side_effect=RuntimeError("audit unavailable"),
), ),
): ):

View File

@@ -0,0 +1,149 @@
from __future__ import annotations
import unittest
from fastapi import HTTPException
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_access.backend.db.models import Account, Group, User
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
from govoplan_campaign.backend.services.job_queries import (
_campaign_jobs_grid_filter_expressions,
_campaign_jobs_ordering,
_campaign_jobs_page_response,
)
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.db.base import Base
class CampaignJobListQueryTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(
self.engine,
tables=[
Account.__table__,
User.__table__,
Group.__table__,
Campaign.__table__,
CampaignVersion.__table__,
CampaignJob.__table__,
ChangeSequenceEntry.__table__,
],
)
self.session = Session(self.engine)
rows = [
(0, "ordinary-0@example.test", "General notice", "ready", "draft", "not_queued", "not_requested", 0),
(1, "ordinary-1@example.test", "General notice", "ready", "draft", "not_queued", "not_requested", 1),
(2, "target-c@example.test", "Target notice C", "warning", "queued", "queued", "pending", 2),
(3, "ordinary-3@example.test", "General notice", "blocked", "draft", "failed_permanent", "failed", 3),
(4, "target-b@example.test", "Target notice B", "ready", "draft", "failed_temporary", "not_requested", 4),
(5, "target-a@example.test", "Target notice A", "ready", "draft", "outcome_unknown", "outcome_unknown", 5),
(6, "excluded@example.test", "Excluded notice", "excluded", "draft", "skipped", "skipped", 0),
]
for entry_index, recipient, subject, validation, queue, send, imap, attempts in rows:
self.session.add(CampaignJob(
id=f"job-{entry_index}",
tenant_id="tenant-1",
campaign_id="campaign-1",
campaign_version_id="version-1",
entry_index=entry_index,
entry_id=f"entry-{entry_index}",
recipient_email=recipient,
subject=subject,
message_id_header=f"<message-{entry_index}@example.test>",
eml_sha256=f"sha-{entry_index}",
build_status="built",
validation_status=validation,
queue_status=queue,
send_status=send,
imap_status=imap,
attempt_count=attempts,
resolved_attachments=[],
issues_snapshot=[],
))
self.session.commit()
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_grid_filters_apply_before_pagination_and_report_filtered_totals(self) -> None:
base_filters = [CampaignJob.tenant_id == "tenant-1", CampaignJob.campaign_id == "campaign-1"]
grid_filters = {"recipient": "target"}
filtered = [*base_filters, *_campaign_jobs_grid_filter_expressions(grid_filters)]
page = _campaign_jobs_page_response(
self.session,
campaign_id="campaign-1",
version_id="version-1",
base_filters=base_filters,
filtered=filtered,
reviewed_keys=set(),
review_metadata={},
page=1,
page_size=2,
grid_filters=grid_filters,
sort_by="recipient",
sort_direction="asc",
)
self.assertEqual(page.total, 3)
self.assertEqual(page.total_unfiltered, 7)
self.assertEqual(page.pages, 2)
self.assertEqual(
[row["recipient_email"] for row in page.jobs],
["target-a@example.test", "target-b@example.test"],
)
self.assertIsNone(page.next_cursor)
def test_list_and_integer_filters_share_the_full_backend_query(self) -> None:
expressions = _campaign_jobs_grid_filter_expressions({
"send": 'list:["failed_temporary","outcome_unknown"]',
"attempts": "gte:4",
})
rows = (
self.session.query(CampaignJob)
.filter(*expressions)
.order_by(*_campaign_jobs_ordering("attempts", "desc"))
.all()
)
self.assertEqual([row.id for row in rows], ["job-5", "job-4"])
def test_invalid_list_filters_fail_closed(self) -> None:
with self.assertRaises(HTTPException) as raised:
_campaign_jobs_grid_filter_expressions({"send": 'list:["not-a-status"]'})
self.assertEqual(raised.exception.status_code, 422)
def test_skipped_transport_filters_and_counts_remain_separate(self) -> None:
base_filters = [CampaignJob.tenant_id == "tenant-1", CampaignJob.campaign_id == "campaign-1"]
grid_filters = {"send": 'list:["skipped"]', "imap": 'list:["skipped"]'}
filtered = [*base_filters, *_campaign_jobs_grid_filter_expressions(grid_filters)]
page = _campaign_jobs_page_response(
self.session,
campaign_id="campaign-1",
version_id="version-1",
base_filters=base_filters,
filtered=filtered,
reviewed_keys=set(),
review_metadata={},
page=1,
page_size=20,
grid_filters=grid_filters,
)
self.assertEqual([row["id"] for row in page.jobs], ["job-6"])
self.assertEqual(page.counts["send"]["skipped"], 1)
self.assertEqual(page.counts["send"]["not_queued"], 2)
self.assertEqual(page.counts["imap"]["skipped"], 1)
self.assertEqual(page.filtered_counts["send"], {"skipped": 1})
self.assertEqual(page.filtered_counts["imap"], {"skipped": 1})
if __name__ == "__main__":
unittest.main()

View File

@@ -6,7 +6,8 @@ from unittest.mock import call, patch
import pytest import pytest
from fastapi import HTTPException from fastapi import HTTPException
from govoplan_campaign.backend import router from govoplan_campaign.backend import route_support
from govoplan_campaign.backend.routes import versions as router
from govoplan_campaign.backend.campaign.loader import CampaignSchemaError, validate_against_schema from govoplan_campaign.backend.campaign.loader import CampaignSchemaError, validate_against_schema
from govoplan_campaign.backend.campaign.mail_profile_boundary import ( from govoplan_campaign.backend.campaign.mail_profile_boundary import (
CampaignMailProfileBoundaryError, CampaignMailProfileBoundaryError,
@@ -73,7 +74,7 @@ def test_mail_profile_documentation_is_classified_for_adaptive_views() -> None:
def test_campaign_mail_contract_rejects_every_legacy_server_field(legacy_key: str) -> None: def test_campaign_mail_contract_rejects_every_legacy_server_field(legacy_key: str) -> None:
raw = _campaign_json({"mail_profile_id": "profile-1", legacy_key: {}}) raw = _campaign_json({"mail_profile_id": "profile-1", legacy_key: {}})
with pytest.raises(CampaignMailProfileBoundaryError, match="select an authorized Mail profile"): with pytest.raises(CampaignMailProfileBoundaryError, match="select authorized Mail resources"):
assert_campaign_uses_mail_profile_reference(raw) assert_campaign_uses_mail_profile_reference(raw)
@@ -138,7 +139,7 @@ def test_new_execution_snapshot_stores_reference_and_evidence_not_transport_mate
delivery=DeliveryConfig(), delivery=DeliveryConfig(),
) )
assert payload["snapshot_version"] == "5" assert payload["snapshot_version"] == "7"
assert payload["mail_profile_id"] == "profile-1" assert payload["mail_profile_id"] == "profile-1"
assert "smtp" not in payload assert "smtp" not in payload
assert "imap" not in payload assert "imap" not in payload
@@ -214,7 +215,7 @@ def test_fork_inherited_profile_requires_mail_profile_use_scope() -> None:
patch.object(router, "_get_campaign_for_principal", return_value=campaign), patch.object(router, "_get_campaign_for_principal", return_value=campaign),
patch.object(router, "_require_permission"), patch.object(router, "_require_permission"),
patch.object(router, "_get_version_for_tenant", return_value=source), patch.object(router, "_get_version_for_tenant", return_value=source),
patch.object(router, "has_scope", return_value=False), patch.object(route_support, "has_scope", return_value=False),
patch.object(router, "fork_campaign_version_for_edit") as fork, patch.object(router, "fork_campaign_version_for_edit") as fork,
): ):
with pytest.raises(HTTPException) as captured: with pytest.raises(HTTPException) as captured:
@@ -234,8 +235,8 @@ def test_campaign_wide_effect_authorizes_every_affected_version() -> None:
session = object() session = object()
principal = SimpleNamespace(tenant_id="tenant-1") principal = SimpleNamespace(tenant_id="tenant-1")
with patch.object(router, "_require_campaign_profile_use_if_needed") as require_profile: with patch.object(route_support, "_require_campaign_profile_use_if_needed") as require_profile:
router._require_campaign_versions_profile_use( # noqa: SLF001 - security boundary regression test route_support._require_campaign_versions_profile_use( # noqa: SLF001 - security boundary regression test
session, # type: ignore[arg-type] session, # type: ignore[arg-type]
principal, # type: ignore[arg-type] principal, # type: ignore[arg-type]
"campaign-1", "campaign-1",

View File

@@ -0,0 +1,480 @@
from __future__ import annotations
import importlib.util
import json
import smtplib
import sys
from pathlib import Path
from typing import Any
import pytest
from govoplan_campaign.backend.campaign import load_campaign_config
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
RUNNER_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "run_campaign_acceptance.py"
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
def _load_runner():
spec = importlib.util.spec_from_file_location("govoplan_campaign_greenmail_acceptance", RUNNER_PATH)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
runner = _load_runner()
class _Response:
def __init__(self, status_code: int, payload: dict[str, Any]) -> None:
self.status_code = status_code
self._payload = payload
def json(self) -> dict[str, Any]:
return self._payload
class _AcceptanceClient:
def __init__(self) -> None:
self.campaign_json: dict[str, Any] | None = None
self.send_calls = 0
def post(self, path: str, **kwargs: Any) -> _Response:
if path == "/api/v1/campaigns":
self.campaign_json = kwargs["json"]["config"]
return _Response(
200,
{
"campaign": {"id": "campaign-internal"},
"version": {"id": "version-internal"},
},
)
if path.endswith("/validate"):
return _Response(200, {"ok": True, "error_count": 0, "warning_count": 0})
if path.endswith("/build"):
return _Response(
200,
{
"built_count": 1,
"build_failed_count": 0,
"queueable_count": 1,
},
)
if path.endswith("/send-now"):
self.send_calls += 1
if self.send_calls == 2:
return _Response(422, {"detail": "Already accepted"})
return _Response(
200,
{
"result": {
"attempted_count": 1,
"sent_count": 1,
"failed_count": 0,
"outcome_unknown_count": 0,
"skipped_count": 0,
"delivery_mode": "synchronous",
"results": [{"job_id": "not-retained", "status": "smtp_accepted"}],
}
},
)
if path.endswith("/append-sent"):
return _Response(
200,
{
"result": {
"pending_count": 1,
"processed_count": 1,
"appended_count": 1,
"failed_count": 0,
"skipped_count": 0,
"results": [{"job_id": "not-retained", "status": "appended"}],
}
},
)
raise AssertionError(f"unexpected POST {path}")
def get(self, path: str, **kwargs: Any) -> _Response:
if path.endswith("/report"):
return _Response(
200,
{
"cards": {
"jobs_total": 1,
"sent": 1,
"smtp_accepted": 1,
"failed": 0,
"outcome_unknown": 0,
"retryable": 0,
"needs_attention": 0,
"imap_appended": 1,
"imap_failed": 0,
},
"status_counts": {
"send": {"smtp_accepted": 1},
"imap": {"appended": 1},
},
},
)
raise AssertionError(f"unexpected GET {path}")
def _settings():
return runner.TestbedSettings(
smtp_host="127.0.0.1",
smtp_port=3025,
imap_host="127.0.0.1",
imap_port=3143,
username="campaign-test@govoplan.test",
password="local-test-password",
sender="campaign-test@govoplan.test",
recipient="campaign-test@govoplan.test",
sent_folder="Sent",
provider_timeout_seconds=5,
)
@pytest.mark.parametrize("host", ["localhost", "mail.test", "192.168.1.20", "8.8.8.8"])
def test_testbed_rejects_hostnames_and_non_loopback_addresses(host: str) -> None:
settings = _settings()
rejected = runner.TestbedSettings(
smtp_host=host,
smtp_port=settings.smtp_port,
imap_host=settings.imap_host,
imap_port=settings.imap_port,
username=settings.username,
password=settings.password,
sender=settings.sender,
recipient=settings.recipient,
sent_folder=settings.sent_folder,
provider_timeout_seconds=settings.provider_timeout_seconds,
)
with pytest.raises(runner.AcceptanceError, match="literal loopback|restricted to the loopback"):
rejected.assert_local_testbed()
@pytest.mark.parametrize("host", ["127.0.0.1", "127.8.9.10", "::1"])
def test_testbed_accepts_literal_loopback_and_preserves_it_in_profile(host: str) -> None:
settings = _settings()
accepted = runner.TestbedSettings(
smtp_host=host,
smtp_port=settings.smtp_port,
imap_host=host,
imap_port=settings.imap_port,
username=settings.username,
password=settings.password,
sender=settings.sender,
recipient=settings.recipient,
sent_folder=settings.sent_folder,
provider_timeout_seconds=settings.provider_timeout_seconds,
)
accepted.assert_local_testbed()
profile = runner._profile_payload(accepted, name="Literal loopback")
assert profile["smtp"]["host"] == host
assert profile["imap"]["host"] == host
def test_campaign_acceptance_orchestration_retains_only_profile_reference_and_safe_evidence() -> None:
client = _AcceptanceClient()
def snapshot_probe(_version_id: str):
assert client.campaign_json is not None
return client.campaign_json, {
"mail_profile_id": "profile-1",
"smtp_transport_revision": "opaque-smtp-revision",
"imap_transport_revision": "opaque-imap-revision",
"delivery": {"imap_append_sent": {"enabled": True, "folder": "Sent"}},
}
audit = {
"campaign.created": 1,
"campaign.validated": 1,
"campaign.messages_built": 1,
"campaign.sent_now": 1,
"campaign.send_now_rejected": 1,
"campaign.append_sent_enqueued": 1,
}
evidence, subject = runner.execute_campaign_scenario(
client,
{"Authorization": "not-retained"},
fixture_path=FIXTURE_PATH,
profile_id="profile-1",
settings=_settings(),
scenario="success",
snapshot_probe=snapshot_probe,
audit_probe=lambda _campaign_id, _version_id: audit,
append_sent=True,
repeat_send=True,
)
evidence["provider_verification"] = {
"inbox_increment": 1,
"sent_increment": 1,
"unique_subject_matches_in_inbox": 1,
"unique_subject_matches_in_sent": 1,
}
runner._assert_success_evidence(evidence)
runner._assert_evidence_safe(
{
"schema_version": runner.EVIDENCE_SCHEMA,
"coverage": {
"smtp_acceptance": True,
"partial_envelope_refusal": False,
"post_data_connection_loss_outcome_unknown": False,
"source_artifact_provenance": False,
"worker_restart_interruption": False,
},
"success": evidence,
},
settings=_settings(),
)
assert subject.startswith("[GovOPlaN acceptance ")
assert client.send_calls == 2
assert client.campaign_json is not None
assert client.campaign_json["server"] == {"mail_profile_id": "profile-1"}
assert "credentials" not in json.dumps(client.campaign_json).casefold()
serialized = json.dumps(evidence, sort_keys=True)
assert "not-retained" not in serialized
assert "local-test-password" not in serialized
def test_campaign_boundary_rejects_resolved_transport_material() -> None:
with pytest.raises(runner.AcceptanceError, match="forbidden transport material"):
runner.assert_campaign_boundary(
{"server": {"mail_profile_id": "profile-1"}},
{
"mail_profile_id": "profile-1",
"smtp_transport_revision": "smtp-revision",
"imap_transport_revision": "imap-revision",
"smtp": {"host": "should-not-be-here"},
},
profile_id="profile-1",
)
def test_success_projection_fails_closed_on_inconsistent_campaign_report() -> None:
evidence = {
"send": {
"attempted_count": 1,
"sent_count": 1,
"failed_count": 0,
"outcome_unknown_count": 0,
"skipped_count": 0,
"delivery_mode": "synchronous",
"statuses": {"smtp_accepted": 1},
},
"append_sent": {
"pending_count": 1,
"processed_count": 1,
"appended_count": 1,
"failed_count": 0,
"skipped_count": 0,
"statuses": {"appended": 1},
},
"report": {
"cards": {
"jobs_total": 1,
"sent": 0,
"smtp_accepted": 0,
"failed": 0,
"outcome_unknown": 0,
"needs_attention": 0,
"imap_appended": 0,
"imap_failed": 0,
},
"send_status_counts": {},
"imap_status_counts": {},
},
"provider_verification": {
"inbox_increment": 1,
"sent_increment": 1,
"unique_subject_matches_in_inbox": 1,
"unique_subject_matches_in_sent": 1,
},
"campaign_mail_boundary": {
"profile_reference_only": True,
"smtp_revision_frozen": True,
"imap_revision_frozen": True,
"resolved_transport_material_present": False,
},
}
with pytest.raises(runner.AcceptanceError, match="report does not agree"):
runner._assert_success_evidence(evidence)
def test_evidence_projection_rejects_unknown_status_keys() -> None:
with pytest.raises(runner.AcceptanceError, match="unsupported status"):
runner._send_evidence(
{
"delivery_mode": "synchronous",
"results": [{"status": "provider diagnostic: recipient@example.test"}],
}
)
with pytest.raises(runner.AcceptanceError, match="durable attempt status"):
runner._durable_state_evidence(
{
"job_count": 1,
"send_status_counts": {"sending": 1},
"attempt_status_counts": {"provider-secret": 1},
"unfinished_attempt_count": 1,
}
)
with pytest.raises(runner.AcceptanceError, match="synchronous delivery mode"):
runner._send_evidence(
{
"delivery_mode": "provider diagnostic: recipient@example.test",
"results": [],
}
)
with pytest.raises(runner.AcceptanceError, match="unsupported"):
runner._report_evidence(
{
"cards": {},
"status_counts": {
"send": {"smtp_accepted": 1, "provider-secret": 1},
"imap": {},
},
}
)
def _open_fault_smtp(endpoint):
client = smtplib.SMTP(endpoint.host, endpoint.port, timeout=5)
client.ehlo()
client.login("acceptance-user", "acceptance-password")
return client
def test_explicit_temporary_smtp_response_occurs_after_complete_data() -> None:
with runner.smtp_fault_endpoint("temporary_data_response") as endpoint:
client = _open_fault_smtp(endpoint)
try:
with pytest.raises(smtplib.SMTPDataError) as captured:
client.sendmail(
"sender@example.test",
["recipient@example.test"],
b"Subject: temporary\r\n\r\nmessage",
)
finally:
client.close()
assert captured.value.smtp_code == 451
assert endpoint.evidence() == {
"connection_count": 1,
"accepted_rcpt_commands": 1,
"refused_rcpt_commands": 0,
"data_transactions": 1,
}
def test_partial_recipient_refusal_retains_one_accepted_envelope() -> None:
with runner.smtp_fault_endpoint("partial_recipient_refusal") as endpoint:
client = _open_fault_smtp(endpoint)
try:
refused = client.sendmail(
"sender@example.test",
["accepted@example.test", "refused@example.test"],
b"Subject: partial\r\n\r\nmessage",
)
finally:
client.quit()
assert set(refused) == {"refused@example.test"}
assert endpoint.evidence() == {
"connection_count": 1,
"accepted_rcpt_commands": 1,
"refused_rcpt_commands": 1,
"data_transactions": 1,
}
def test_post_data_disconnect_is_a_real_ambiguous_protocol_boundary() -> None:
with runner.smtp_fault_endpoint("post_data_disconnect") as endpoint:
client = _open_fault_smtp(endpoint)
try:
with pytest.raises(smtplib.SMTPServerDisconnected):
client.sendmail(
"sender@example.test",
["recipient@example.test"],
b"Subject: ambiguous\r\n\r\nmessage",
)
finally:
client.close()
assert endpoint.wait_for_data(1)
assert endpoint.evidence()["data_transactions"] == 1
def test_partial_refusal_fixture_adds_a_second_distinct_recipient() -> None:
raw, _subject = runner.materialize_campaign_fixture(
FIXTURE_PATH,
profile_id="profile-1",
settings=_settings(),
scenario="partial_envelope_refusal",
run_token="0123456789ab",
additional_envelope_recipient=True,
)
recipients = raw["entries"]["inline"][0]["to"]
assert len(recipients) == 2
assert recipients[0]["email"] != recipients[1]["email"]
assert recipients[1]["email"].endswith("@govoplan.test")
def test_fixture_contains_no_transport_credentials() -> None:
raw = json.loads(FIXTURE_PATH.read_text(encoding="utf-8"))
runner._assert_no_forbidden_campaign_keys(raw)
assert raw["server"] == {"mail_profile_id": "00000000-0000-4000-8000-000000000001"}
config = load_campaign_config(FIXTURE_PATH)
assert config.server.mail_profile_id == "00000000-0000-4000-8000-000000000001"
def test_fixture_composition_versions_are_complete_and_exact() -> None:
versions = {
"core": "0.1.13",
"access": "0.1.11",
"audit": "0.1.8",
"campaigns": "0.1.11",
"mail": "0.1.10",
"files": "0.1.9",
}
assert runner.required_composition_versions(FIXTURE_PATH, versions) == {
"core": "0.1.13",
"access": "0.1.11",
"audit": "0.1.8",
"campaigns": "0.1.11",
"mail": "0.1.10",
}
def test_fixture_composition_fails_closed_when_a_required_version_is_missing() -> None:
with pytest.raises(runner.AcceptanceError, match="versions are unavailable"):
runner.required_composition_versions(
FIXTURE_PATH,
{
"core": "0.1.13",
"access": "0.1.11",
"campaigns": "0.1.11",
"mail": "0.1.10",
},
)
def test_testbed_documentation_distinguishes_proven_and_open_failure_drills() -> None:
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(encoding="utf-8")
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(encoding="utf-8")
assert "second ordinary send must be rejected before another provider effect" in testbed
assert "connection loss after complete DATA is frozen" in testbed
assert "dedicated OS process" in testbed
assert "Redis/Celery delivery" in runbook
assert "broker redelivery" in runbook
assert "celery_broker_redelivery" in testbed
assert "raw provider diagnostics" in runbook

View File

@@ -0,0 +1,64 @@
from __future__ import annotations
from govoplan_campaign.backend.manifest import (
CAMPAIGN_MODULE_REQUIRED_ANY,
OPERATOR_QUEUE_REQUIRED_ANY,
OPERATOR_QUEUE_SURFACE_ID,
REPORTS_REQUIRED_ANY,
REPORTS_SURFACE_ID,
get_manifest,
)
from govoplan_core.core.registry import manifest_view_surfaces
def test_operator_queue_is_an_integrated_campaign_view() -> None:
manifest = get_manifest()
assert manifest.frontend is not None
assert "/operator" not in {item.path for item in manifest.nav_items}
assert "/operator" not in {item.path for item in manifest.frontend.nav_items}
routes = {route.path: route for route in manifest.frontend.routes}
assert "/operator" not in routes
queue = routes["/campaigns/queue"]
assert queue.component == "OperatorQueuePage"
assert queue.required_all == ("campaigns:campaign:read",)
assert queue.required_any == OPERATOR_QUEUE_REQUIRED_ANY
assert OPERATOR_QUEUE_SURFACE_ID == "campaigns.route.operator"
assert queue.surface_id == OPERATOR_QUEUE_SURFACE_ID
queue_surfaces = [
surface
for surface in manifest_view_surfaces(manifest)
if surface.id == OPERATOR_QUEUE_SURFACE_ID
]
assert len(queue_surfaces) == 1
assert queue_surfaces[0].description == "/campaigns/queue"
def test_aggregate_reports_are_an_integrated_campaign_view() -> None:
manifest = get_manifest()
assert manifest.frontend is not None
backend_nav = {item.path: item for item in manifest.nav_items}
frontend_nav = {item.path: item for item in manifest.frontend.nav_items}
assert "/reports" not in backend_nav
assert "/reports" not in frontend_nav
assert backend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
assert frontend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
routes = {route.path: route for route in manifest.frontend.routes}
assert "/reports" not in routes
report = routes["/campaigns/reports"]
assert report.component == "AggregateReportsPage"
assert report.required_any == REPORTS_REQUIRED_ANY
assert REPORTS_SURFACE_ID == "campaigns.route.reports"
assert report.surface_id == REPORTS_SURFACE_ID
report_surfaces = [
surface
for surface in manifest_view_surfaces(manifest)
if surface.id == REPORTS_SURFACE_ID
]
assert len(report_surfaces) == 1
assert report_surfaces[0].description == "/campaigns/reports"

View File

@@ -5,7 +5,7 @@ import unittest
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from govoplan_campaign.backend import router from govoplan_campaign.backend.routes import campaigns as router
from govoplan_campaign.backend.campaign.models import CampaignConfig from govoplan_campaign.backend.campaign.models import CampaignConfig
from govoplan_campaign.backend.campaign.validation import validate_campaign_config from govoplan_campaign.backend.campaign.validation import validate_campaign_config
from govoplan_campaign.backend.persistence.versions import validate_campaign_partial from govoplan_campaign.backend.persistence.versions import validate_campaign_partial

View File

@@ -0,0 +1,400 @@
from __future__ import annotations
import unittest
from types import SimpleNamespace
from unittest.mock import patch
import pytest
from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy
from govoplan_campaign.backend.db.models import JobSendStatus
from govoplan_campaign.backend.sending.jobs import (
SendJobResult,
_MailChannelOutcome,
_final_multichannel_status,
_send_claimed_multichannel_job,
)
from govoplan_campaign.backend.sending.postbox_delivery import (
PostboxChannelOutcome,
)
def _context():
return SimpleNamespace(
message_bytes=b"message",
snapshot=SimpleNamespace(
delivery=SimpleNamespace(
postbox=SimpleNamespace(classification="internal")
)
),
)
def _job():
return SimpleNamespace(id="job-1")
class _Session:
def __init__(self, job) -> None:
self.job = job
def get(self, _model, _id):
return self.job
def add(self, _value) -> None:
return None
def commit(self) -> None:
return None
class PostboxFallbackOrchestrationTests(unittest.TestCase):
def test_mail_unknown_never_starts_postbox_fallback(self) -> None:
job = _job()
expected = SendJobResult(
job_id=job.id,
status=JobSendStatus.OUTCOME_UNKNOWN.value,
attempt_number=1,
)
with (
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
return_value=_MailChannelOutcome(outcome_unknown=True),
),
patch(
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
return_value=PostboxChannelOutcome(),
),
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
) as deliver_postbox,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=expected,
),
):
result = _send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
use_rate_limit=False,
enqueue_imap_task=False,
)
self.assertIs(result, expected)
deliver_postbox.assert_not_called()
def test_mail_preacceptance_rejection_starts_postbox_fallback(self) -> None:
job = _job()
postbox_outcome = PostboxChannelOutcome(accepted=1)
with (
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
return_value=_MailChannelOutcome(rejected_permanent=True),
),
patch(
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
return_value=PostboxChannelOutcome(),
),
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
return_value=postbox_outcome,
) as deliver_postbox,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=SendJobResult(
job_id=job.id,
status=JobSendStatus.POSTBOX_ACCEPTED.value,
attempt_number=1,
),
),
):
_send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
use_rate_limit=False,
enqueue_imap_task=False,
)
deliver_postbox.assert_called_once()
def test_accepted_postbox_fallback_prevents_later_mail_retry(self) -> None:
job = _job()
with (
patch(
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
return_value=PostboxChannelOutcome(
accepted=1,
rejected_temporary=1,
),
),
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
) as deliver_mail,
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
return_value=PostboxChannelOutcome(accepted=2),
) as deliver_postbox,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=SendJobResult(
job_id=job.id,
status=JobSendStatus.POSTBOX_ACCEPTED.value,
attempt_number=3,
),
),
):
_send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
use_rate_limit=False,
enqueue_imap_task=False,
)
deliver_mail.assert_not_called()
deliver_postbox.assert_called_once()
def test_unknown_postbox_fallback_prevents_every_later_effect(self) -> None:
job = _job()
prior = PostboxChannelOutcome(outcome_unknown=1)
with (
patch(
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
return_value=prior,
),
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
) as deliver_mail,
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
) as deliver_postbox,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=SendJobResult(
job_id=job.id,
status=JobSendStatus.OUTCOME_UNKNOWN.value,
attempt_number=1,
),
) as finalize,
):
_send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
use_rate_limit=False,
enqueue_imap_task=False,
)
deliver_mail.assert_not_called()
deliver_postbox.assert_not_called()
self.assertIs(finalize.call_args.kwargs["postbox"], prior)
def test_postbox_acceptance_stops_mail_fallback(self) -> None:
job = _job()
with (
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
return_value=PostboxChannelOutcome(
accepted=1,
rejected_permanent=1,
),
),
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
) as deliver_mail,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=SendJobResult(
job_id=job.id,
status=JobSendStatus.PARTIALLY_ACCEPTED.value,
attempt_number=2,
),
),
):
_send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
use_rate_limit=False,
enqueue_imap_task=False,
)
deliver_mail.assert_not_called()
def test_all_postbox_rejections_start_mail_fallback(self) -> None:
job = _job()
with (
patch(
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
return_value=PostboxChannelOutcome(
rejected_temporary=1,
rejected_permanent=1,
),
),
patch(
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
return_value=_MailChannelOutcome(accepted=True),
) as deliver_mail,
patch(
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
return_value=SendJobResult(
job_id=job.id,
status=JobSendStatus.SMTP_ACCEPTED.value,
attempt_number=1,
),
),
):
_send_claimed_multichannel_job(
_Session(job), # type: ignore[arg-type]
job=job, # type: ignore[arg-type]
claim_token="claim-1",
context=_context(), # type: ignore[arg-type]
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
use_rate_limit=False,
enqueue_imap_task=False,
)
deliver_mail.assert_called_once()
def test_explicit_dual_delivery_reports_partial_and_unknown(self) -> None:
self.assertEqual(
JobSendStatus.PARTIALLY_ACCEPTED.value,
_final_multichannel_status(
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
mail=_MailChannelOutcome(accepted=True),
postbox=PostboxChannelOutcome(rejected_permanent=1),
),
)
self.assertEqual(
JobSendStatus.OUTCOME_UNKNOWN.value,
_final_multichannel_status(
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
mail=_MailChannelOutcome(accepted=True),
postbox=PostboxChannelOutcome(outcome_unknown=1),
),
)
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
@pytest.mark.parametrize(
("mail", "postbox"),
[
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome()),
(_MailChannelOutcome(accepted=True), PostboxChannelOutcome(outcome_unknown=1)),
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome(accepted=1)),
],
)
def test_outcome_unknown_has_precedence_for_every_delivery_policy(
policy: DeliveryChannelPolicy,
mail: _MailChannelOutcome,
postbox: PostboxChannelOutcome,
) -> None:
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == JobSendStatus.OUTCOME_UNKNOWN.value
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
@pytest.mark.parametrize(
("mail", "postbox", "expected"),
[
(
_MailChannelOutcome(rejected_permanent=True),
PostboxChannelOutcome(rejected_permanent=1),
JobSendStatus.FAILED_PERMANENT.value,
),
(
_MailChannelOutcome(rejected_temporary=True),
PostboxChannelOutcome(rejected_permanent=1),
JobSendStatus.FAILED_TEMPORARY.value,
),
(
_MailChannelOutcome(rejected_permanent=True),
PostboxChannelOutcome(rejected_temporary=1),
JobSendStatus.FAILED_TEMPORARY.value,
),
],
)
def test_rejection_precedence_is_exhaustive_for_every_delivery_policy(
policy: DeliveryChannelPolicy,
mail: _MailChannelOutcome,
postbox: PostboxChannelOutcome,
expected: str,
) -> None:
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected
@pytest.mark.parametrize(
("policy", "mail", "postbox", "expected"),
[
(
DeliveryChannelPolicy.MAIL,
_MailChannelOutcome(accepted=True),
PostboxChannelOutcome(),
JobSendStatus.SMTP_ACCEPTED.value,
),
(
DeliveryChannelPolicy.POSTBOX,
None,
PostboxChannelOutcome(accepted=1),
JobSendStatus.POSTBOX_ACCEPTED.value,
),
(
DeliveryChannelPolicy.POSTBOX,
None,
PostboxChannelOutcome(accepted=1, rejected_permanent=1),
JobSendStatus.PARTIALLY_ACCEPTED.value,
),
(
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
_MailChannelOutcome(accepted=True),
PostboxChannelOutcome(accepted=1),
JobSendStatus.DELIVERED.value,
),
(
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
_MailChannelOutcome(accepted=True),
PostboxChannelOutcome(rejected_permanent=1),
JobSendStatus.PARTIALLY_ACCEPTED.value,
),
(
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
_MailChannelOutcome(rejected_permanent=True),
PostboxChannelOutcome(accepted=1),
JobSendStatus.PARTIALLY_ACCEPTED.value,
),
(
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
_MailChannelOutcome(rejected_permanent=True),
PostboxChannelOutcome(accepted=1),
JobSendStatus.POSTBOX_ACCEPTED.value,
),
(
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
_MailChannelOutcome(accepted=True),
PostboxChannelOutcome(rejected_permanent=1),
JobSendStatus.PARTIALLY_ACCEPTED.value,
),
],
)
def test_accepted_delivery_decision_table(
policy: DeliveryChannelPolicy,
mail: _MailChannelOutcome | None,
postbox: PostboxChannelOutcome,
expected: str,
) -> None:
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,145 @@
from __future__ import annotations
import unittest
from datetime import UTC, datetime
from govoplan_core.core.postbox import (
PostboxDeliveryCatalogRef,
PostboxDirectoryEntryRef,
PostboxDeliveryRequest,
PostboxDeliveryReceiptSummaryRef,
PostboxDeliveryResult,
PostboxTargetRef,
)
from govoplan_campaign.backend.integrations import (
PostboxCampaignIntegration,
PostboxDeliveryUnavailable,
)
class _PostboxDelivery:
def __init__(self) -> None:
self.requests = []
def deliver(self, session, request):
self.requests.append((session, request))
return PostboxDeliveryResult(
delivery_id="delivery-1",
postbox_id="postbox-1",
message_id="message-1",
address="intake@postbox",
status="accepted",
vacant=False,
holder_count=1,
)
def delivery_catalog(self, session, *, tenant_id):
return PostboxDeliveryCatalogRef()
def list_visible_postboxes(self, session, *, tenant_id, actor):
return ()
def resolve_postbox(
self,
session,
*,
tenant_id,
target,
materialize=False,
):
return PostboxDirectoryEntryRef(
id=target.postbox_id or "postbox-1",
tenant_id=tenant_id,
address="intake@postbox",
address_key="intake",
name="Intake",
status="active",
classification="internal",
)
def link_evidence(self, session, *, tenant_id, message_id, attachment):
del session, tenant_id, message_id, attachment
return None
def delivery_receipt_summaries(
self,
session,
*,
tenant_id,
producer_module,
delivery_ids,
):
del session
assert tenant_id == "tenant-1"
assert producer_module == "campaigns"
return {
delivery_id: PostboxDeliveryReceiptSummaryRef(
delivery_id=delivery_id,
message_id="message-1",
postbox_id="postbox-1",
delivery_status="accepted",
accepted_at=datetime(2026, 7, 30, tzinfo=UTC),
currently_readable=True,
read_receipt_count=1,
)
for delivery_id in delivery_ids
}
class PostboxCampaignIntegrationTests(unittest.TestCase):
def test_optional_delivery_boundary_is_typed_and_explicit(self) -> None:
delegate = _PostboxDelivery()
integration = PostboxCampaignIntegration(delegate, delegate, delegate)
request = PostboxDeliveryRequest(
tenant_id="tenant-1",
target=PostboxTargetRef(postbox_id="postbox-1"),
producer_module="campaigns",
producer_resource_type="campaign_job",
producer_resource_id="job-1",
idempotency_key="campaign-1:job-1:postbox-1",
subject="Decision",
)
result = integration.deliver(object(), request)
self.assertTrue(integration.available)
self.assertTrue(integration.receipt_evidence_available)
self.assertEqual("delivery-1", result.delivery_id)
self.assertEqual(request, delegate.requests[0][1])
summaries = integration.delivery_receipt_summaries(
object(),
tenant_id="tenant-1",
delivery_ids=["delivery-1"],
)
self.assertEqual(1, summaries["delivery-1"].read_receipt_count)
def test_missing_postbox_is_reported_without_importing_module_code(
self,
) -> None:
integration = PostboxCampaignIntegration()
request = PostboxDeliveryRequest(
tenant_id="tenant-1",
target=PostboxTargetRef(postbox_id="postbox-1"),
producer_module="campaigns",
producer_resource_type="campaign_job",
producer_resource_id="job-1",
idempotency_key="campaign-1:job-1:postbox-1",
subject="Decision",
)
self.assertFalse(integration.available)
self.assertFalse(integration.receipt_evidence_available)
self.assertEqual(
{},
integration.delivery_receipt_summaries(
object(),
tenant_id="tenant-1",
delivery_ids=["delivery-1"],
),
)
with self.assertRaises(PostboxDeliveryUnavailable):
integration.deliver(object(), request)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,208 @@
from __future__ import annotations
from unittest.mock import patch
from govoplan_core.core.postbox import (
PostboxDeliveryCatalogRef,
PostboxDeliveryTemplateRef,
PostboxDirectoryEntryRef,
PostboxOrganizationFunctionTargetRef,
PostboxOrganizationUnitTargetRef,
)
from govoplan_campaign.backend.campaign.models import CampaignConfig
from govoplan_campaign.backend.campaign.postbox_targets import (
resolve_entry_postbox_targets,
)
from govoplan_campaign.backend.campaign.validation import (
validate_campaign_config,
)
from govoplan_campaign.backend.messages.models import MessageValidationStatus
def _config() -> CampaignConfig:
return CampaignConfig.model_validate(
{
"version": "1.0",
"campaign": {
"id": "campaign-1",
"name": "Postbox campaign",
"mode": "send",
},
"fields": [
{"name": "org_key", "type": "organization_unit"},
{"name": "function_key", "type": "organization_function"},
{"name": "case_key", "type": "string"},
],
"template": {
"subject": "Decision",
"text": "A decision is available.",
"body_mode": "text",
},
"entries": {
"inline": [
{
"id": "recipient-1",
"fields": {
"org_key": "finance",
"function_key": "caseworker",
"case_key": "case-42",
},
}
]
},
"delivery": {
"channel_policy": "postbox",
"postbox": {
"targets": [
{
"id": "direct",
"mode": "direct",
"address_key": "central-intake",
},
{
"id": "derived",
"mode": "derived",
"template_id": "template-1",
"organization_unit_field": "org_key",
"organization_unit_match": "slug",
"function_field": "function_key",
"function_match": "slug",
"context_field": "case_key",
},
]
},
},
}
)
def _catalog() -> PostboxDeliveryCatalogRef:
return PostboxDeliveryCatalogRef(
templates=(
PostboxDeliveryTemplateRef(
id="template-1",
slug="case-inbox",
name="Case inbox",
description=None,
published_revision_id="revision-1",
function_type_id=None,
scope_kind="tenant",
scope_id=None,
classification="internal",
allow_vacant_delivery=True,
),
),
organization_units=(
PostboxOrganizationUnitTargetRef(
id="unit-1",
slug="finance",
name="Finance",
functions=(
PostboxOrganizationFunctionTargetRef(
id="function-1",
slug="caseworker",
name="Caseworker",
),
),
),
),
)
class _PostboxIntegration:
def __init__(self) -> None:
self.targets: list[tuple[object, bool]] = []
def delivery_catalog(self, _session, *, tenant_id: str):
assert tenant_id == "tenant-1"
return _catalog()
def resolve_postbox(
self,
_session,
*,
tenant_id: str,
target,
materialize: bool,
):
self.targets.append((target, materialize))
if target.address_key == "central-intake":
return PostboxDirectoryEntryRef(
id="postbox-direct",
tenant_id=tenant_id,
address="central-intake@postbox",
address_key="central-intake",
name="Central intake",
status="active",
classification="internal",
holder_count=2,
vacant=False,
)
assert target.template_id == "template-1"
assert target.organization_unit_id == "unit-1"
assert target.function_id == "function-1"
assert target.context_key == "case-42"
return PostboxDirectoryEntryRef(
id="postbox-derived",
tenant_id=tenant_id,
address="finance.caseworker.case-42@postbox",
address_key="finance.caseworker.case-42",
name="Finance caseworker",
status="active",
classification="internal",
organization_unit_id="unit-1",
function_id="function-1",
context_key="case-42",
holder_count=1,
vacant=False,
)
def test_postbox_only_campaign_does_not_require_mail() -> None:
config = _config()
available = validate_campaign_config(config, postbox_available=True)
unavailable = validate_campaign_config(config, postbox_available=False)
assert {
issue.code
for issue in available.issues
if issue.severity.value == "error"
} == set()
assert "missing_mail_profile" not in {
issue.code for issue in unavailable.issues
}
assert "missing_sender" not in {
issue.code for issue in unavailable.issues
}
assert "postbox_unavailable" in {
issue.code for issue in unavailable.issues
}
def test_row_resolves_multiple_direct_and_field_derived_postboxes() -> None:
config = _config()
integration = _PostboxIntegration()
with patch(
"govoplan_campaign.backend.campaign.postbox_targets.postbox_integration",
return_value=integration,
):
targets, issues, status = resolve_entry_postbox_targets(
object(), # type: ignore[arg-type]
tenant_id="tenant-1",
config=config,
entry=config.entries.inline[0], # type: ignore[index]
validation_status=MessageValidationStatus.READY,
materialize=True,
)
assert issues == []
assert status == MessageValidationStatus.READY
assert [target["postbox_id"] for target in targets] == [
"postbox-direct",
"postbox-derived",
]
assert targets[1]["context_key"] == "case-42"
assert len(integration.targets) == 2
assert all(materialize for _target, materialize in integration.targets)

View File

@@ -0,0 +1,216 @@
from __future__ import annotations
import unittest
from unittest.mock import patch
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_access.backend.db.models import Account, Group, User
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignStatus,
CampaignVersion,
JobBuildStatus,
JobQueueStatus,
JobSendStatus,
JobValidationStatus,
)
from govoplan_campaign.backend.sending.jobs import (
QueueingError,
cancel_campaign_jobs,
pause_campaign_jobs,
resume_campaign_jobs,
)
from govoplan_campaign.backend.reports.campaigns import _campaign_report_cards, _version_info
from govoplan_core.core.change_sequence import ChangeSequenceEntry, ChangeSequenceRetentionFloor
from govoplan_core.db.base import Base
class CampaignQueueControlTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(
self.engine,
tables=[
Account.__table__,
User.__table__,
Group.__table__,
ChangeSequenceEntry.__table__,
ChangeSequenceRetentionFloor.__table__,
Campaign.__table__,
CampaignVersion.__table__,
CampaignJob.__table__,
],
)
self.session = Session(self.engine)
self._add_campaign("campaign-1", "tenant-1", "version-1")
self._add_campaign("campaign-2", "tenant-2", "version-2")
self._add_job("queued", queue="queued", send="queued")
self._add_job("paused", queue="paused", send="queued")
self._add_job("failed", queue="draft", send="failed_temporary")
self._add_job("unknown", queue="draft", send="outcome_unknown")
self._add_job("accepted", queue="draft", send="smtp_accepted")
self._add_job("claimed", queue="sending", send="claimed")
self._add_job(
"excluded",
queue="draft",
send="skipped",
validation="excluded",
imap="skipped",
)
self._add_job(
"other-tenant",
tenant_id="tenant-2",
campaign_id="campaign-2",
version_id="version-2",
queue="queued",
send="queued",
)
self.session.commit()
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_pause_and_resume_change_only_valid_jobs_in_the_selected_tenant(self) -> None:
with patch("govoplan_campaign.backend.sending.jobs._emit_campaign_status_notification"):
paused = pause_campaign_jobs(
self.session,
tenant_id="tenant-1",
campaign_id="campaign-1",
)
resumed = resume_campaign_jobs(
self.session,
tenant_id="tenant-1",
campaign_id="campaign-1",
enqueue_celery=False,
)
self.assertEqual(paused["paused_count"], 1)
self.assertEqual(resumed, {
"campaign_id": "campaign-1",
"resumed_count": 2,
"enqueued_count": 0,
})
self.session.expire_all()
self.assertEqual(self.session.get(Campaign, "campaign-1").status, CampaignStatus.QUEUED.value)
version = self.session.get(CampaignVersion, "version-1")
self.assertEqual(version.delivery_mode, "database_queue")
self.assertIsNotNone(version.delivery_mode_selected_at)
self.assertEqual(self.session.get(CampaignJob, "queued").queue_status, JobQueueStatus.QUEUED.value)
self.assertEqual(self.session.get(CampaignJob, "paused").queue_status, JobQueueStatus.QUEUED.value)
self.assertEqual(self.session.get(CampaignJob, "other-tenant").queue_status, JobQueueStatus.QUEUED.value)
def test_cancel_is_auditable_idempotent_state_change_and_protects_started_effects(self) -> None:
with patch("govoplan_campaign.backend.sending.jobs._emit_campaign_status_notification"):
first = cancel_campaign_jobs(
self.session,
tenant_id="tenant-1",
campaign_id="campaign-1",
)
second = cancel_campaign_jobs(
self.session,
tenant_id="tenant-1",
campaign_id="campaign-1",
)
self.assertEqual(first["cancelled_count"], 3)
self.assertEqual(first["protected_count"], 3)
self.assertEqual(first["skipped_count"], 1)
self.assertEqual(second["cancelled_count"], 0)
self.assertEqual(second["protected_count"], 3)
self.assertEqual(second["skipped_count"], 1)
self.session.expire_all()
for job_id in ("queued", "paused", "failed"):
job = self.session.get(CampaignJob, job_id)
self.assertEqual(job.queue_status, JobQueueStatus.CANCELLED.value)
self.assertEqual(job.send_status, JobSendStatus.CANCELLED.value)
self.assertEqual(self.session.get(CampaignJob, "unknown").send_status, JobSendStatus.OUTCOME_UNKNOWN.value)
self.assertEqual(self.session.get(CampaignJob, "accepted").send_status, JobSendStatus.SMTP_ACCEPTED.value)
self.assertEqual(self.session.get(CampaignJob, "claimed").send_status, JobSendStatus.CLAIMED.value)
self.assertEqual(self.session.get(CampaignJob, "excluded").send_status, JobSendStatus.SKIPPED.value)
self.assertEqual(self.session.get(CampaignJob, "excluded").queue_status, JobQueueStatus.DRAFT.value)
self.assertEqual(self.session.get(CampaignJob, "other-tenant").send_status, JobSendStatus.QUEUED.value)
def test_controls_fail_closed_for_a_campaign_owned_by_another_tenant(self) -> None:
for control in (pause_campaign_jobs, resume_campaign_jobs, cancel_campaign_jobs):
with self.subTest(control=control.__name__), self.assertRaises(QueueingError):
control(
self.session,
tenant_id="tenant-1",
campaign_id="campaign-2",
)
def test_queue_projection_exposes_exact_action_counts_and_persisted_mode_only(self) -> None:
version = self.session.get(CampaignVersion, "version-1")
version.delivery_mode = "worker_queue"
version.delivery_mode_selected_at = version.updated_at
jobs = (
self.session.query(CampaignJob)
.filter(CampaignJob.tenant_id == "tenant-1")
.all()
)
cards = _campaign_report_cards(version, jobs)
projected_version = _version_info(version)
self.assertEqual(cards["retryable"], 1)
self.assertEqual(cards["queueable_unattempted"], 0)
self.assertEqual(cards["cancellable"], 3)
self.assertEqual(cards["skipped"], 1)
self.assertEqual(cards["imap_skipped"], 1)
self.assertEqual(projected_version["delivery_mode"], "worker_queue")
self.assertIn("delivery_mode_selected_at", projected_version)
self.assertNotIn("execution_snapshot", projected_version)
def _add_campaign(self, campaign_id: str, tenant_id: str, version_id: str) -> None:
self.session.add(Campaign(
id=campaign_id,
tenant_id=tenant_id,
external_id=f"external-{campaign_id}",
name=campaign_id,
status=CampaignStatus.QUEUED.value,
current_version_id=version_id,
settings={},
mail_profile_policy={},
))
self.session.add(CampaignVersion(
id=version_id,
campaign_id=campaign_id,
version_number=1,
raw_json={},
))
def _add_job(
self,
job_id: str,
*,
queue: str,
send: str,
tenant_id: str = "tenant-1",
campaign_id: str = "campaign-1",
version_id: str = "version-1",
validation: str = JobValidationStatus.READY.value,
imap: str = "not_requested",
) -> None:
self.session.add(CampaignJob(
id=job_id,
tenant_id=tenant_id,
campaign_id=campaign_id,
campaign_version_id=version_id,
entry_index=len(self.session.new),
entry_id=f"entry-{job_id}",
build_status=JobBuildStatus.BUILT.value,
validation_status=validation,
queue_status=queue,
send_status=send,
imap_status=imap,
resolved_attachments=[],
issues_snapshot=[],
))
if __name__ == "__main__":
unittest.main()

View File

@@ -2,12 +2,13 @@ from __future__ import annotations
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import patch from unittest.mock import patch
from email.message import EmailMessage
import pytest import pytest
from fastapi import HTTPException from fastapi import HTTPException
from pydantic import ValidationError from pydantic import ValidationError
from govoplan_campaign.backend import router from govoplan_campaign.backend.routes import reports as router
from govoplan_campaign.backend.reports.emailing import CampaignReportEmailError, send_campaign_report_email from govoplan_campaign.backend.reports.emailing import CampaignReportEmailError, send_campaign_report_email
from govoplan_campaign.backend.schemas import ReportEmailRequest from govoplan_campaign.backend.schemas import ReportEmailRequest
@@ -42,6 +43,16 @@ def test_report_email_recipient_schema_rejects_unsafe_addresses(recipients: list
ReportEmailRequest.model_validate({"to": recipients}) ReportEmailRequest.model_validate({"to": recipients})
def test_report_email_recipient_error_identifies_the_invalid_list_item() -> None:
with pytest.raises(ValidationError) as captured:
ReportEmailRequest.model_validate({
"to": ["first@example.test", "invalid recipient", "third@example.test"],
})
assert captured.value.errors()[0]["loc"] == ("to", 1)
assert "email addresses" in captured.value.errors()[0]["msg"]
def test_report_jobs_csv_attachment_requires_recipient_export_permission() -> None: def test_report_jobs_csv_attachment_requires_recipient_export_permission() -> None:
payload = ReportEmailRequest(to=["recipient@example.test"], attach_jobs_csv=True) payload = ReportEmailRequest(to=["recipient@example.test"], attach_jobs_csv=True)
principal = SimpleNamespace(tenant_id="tenant-1") principal = SimpleNamespace(tenant_id="tenant-1")
@@ -74,7 +85,10 @@ def test_report_jobs_csv_attachment_requires_recipient_export_permission() -> No
def test_report_email_requires_permission_to_use_selected_mail_profile() -> None: def test_report_email_requires_permission_to_use_selected_mail_profile() -> None:
payload = ReportEmailRequest(to=["recipient@example.test"]) payload = ReportEmailRequest(to=["recipient@example.test"])
principal = SimpleNamespace(tenant_id="tenant-1") principal = SimpleNamespace(
tenant_id="tenant-1",
has=lambda scope: scope == "campaigns:recipient:export",
)
campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1") campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1")
version = SimpleNamespace( version = SimpleNamespace(
id="version-1", id="version-1",
@@ -106,7 +120,10 @@ def test_report_email_requires_permission_to_use_selected_mail_profile() -> None
def test_unexpected_report_failure_does_not_leak_internal_details() -> None: def test_unexpected_report_failure_does_not_leak_internal_details() -> None:
payload = ReportEmailRequest(to=["recipient@example.test"]) payload = ReportEmailRequest(to=["recipient@example.test"])
principal = SimpleNamespace(tenant_id="tenant-1") principal = SimpleNamespace(
tenant_id="tenant-1",
has=lambda scope: scope == "campaigns:recipient:export",
)
with ( with (
patch.object( patch.object(
router, router,
@@ -167,3 +184,91 @@ def test_report_send_fails_closed_until_durable_mail_outbox_exists() -> None:
) )
generate_report.assert_not_called() generate_report.assert_not_called()
def test_live_report_submits_exact_message_to_durable_mail_command() -> None:
campaign = SimpleNamespace(
id="campaign-1",
tenant_id="tenant-1",
name="Campaign",
external_id="campaign-1",
)
version = SimpleNamespace(
id="version-1",
campaign_id="campaign-1",
raw_json={"server": {"mail_profile_id": "profile-1"}},
execution_snapshot={"snapshot_version": "7"},
)
config = SimpleNamespace(
server=SimpleNamespace(
profile_capabilities=SimpleNamespace(smtp_available=True)
),
)
snapshot = SimpleNamespace(
mail_profile_id="profile-1",
smtp_server_id="smtp-1",
smtp_credential_id="credential-1",
smtp_transport_revision="revision-1",
)
message = EmailMessage()
message["From"] = "Sender <sender@example.test>"
message["To"] = "recipient@example.test"
message["Subject"] = "Report"
message.set_content("content")
mail = SimpleNamespace(
durable_delivery_available=True,
submit_delivery_command=lambda _session, **_kwargs: {
"id": "command-1",
"status": "pending",
"duplicate": False,
},
)
class Session:
def get(self, _model, _id):
return campaign
with (
patch(
"govoplan_campaign.backend.reports.emailing._selected_version",
return_value=version,
),
patch(
"govoplan_campaign.backend.reports.emailing._load_config",
return_value=config,
),
patch(
"govoplan_campaign.backend.reports.emailing.ensure_execution_snapshot",
return_value=snapshot,
),
patch(
"govoplan_campaign.backend.reports.emailing.generate_campaign_report",
return_value={},
),
patch(
"govoplan_campaign.backend.reports.emailing.build_report_message",
return_value=message,
),
patch(
"govoplan_campaign.backend.reports.emailing._effective_from",
return_value=("sender@example.test", "Sender"),
),
patch(
"govoplan_campaign.backend.reports.emailing.mail_integration",
return_value=mail,
),
):
result = send_campaign_report_email(
Session(), # type: ignore[arg-type]
tenant_id="tenant-1",
campaign_id="campaign-1",
to=["recipient@example.test"],
idempotency_key="request-1",
created_by_user_id="user-1",
)
assert result.command_id == "command-1"
assert result.delivery_status == "pending"
assert result.sent is False
assert result.audit_dict()["recipient_count"] == 1
assert "recipient@example.test" not in repr(result.audit_dict())

View File

@@ -5,8 +5,9 @@ import unittest
from datetime import UTC, datetime from datetime import UTC, datetime
from types import SimpleNamespace from types import SimpleNamespace
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
from govoplan_campaign.backend.response_security import public_campaign_payload from govoplan_campaign.backend.response_security import public_campaign_payload
from govoplan_campaign.backend.router import ( from govoplan_campaign.backend.services.job_queries import (
_job_attempts_payload, _job_attempts_payload,
_job_detail_payload, _job_detail_payload,
_job_diagnostics_payload, _job_diagnostics_payload,
@@ -88,6 +89,30 @@ def _imap_attempt() -> SimpleNamespace:
) )
def _postbox_attempt() -> SimpleNamespace:
return SimpleNamespace(
id="postbox-attempt-1",
target_index=0,
attempt_number=1,
status="accepted",
postbox_id="postbox-1",
address="intake@postbox",
holder_count=1,
vacant=False,
duplicate=False,
target_snapshot={"address": "intake@postbox"},
started_at=_now(),
finished_at=_now(),
error_code=None,
idempotency_key="private-idempotency-key",
provider_delivery_id="delivery-1",
provider_message_id="message-1",
evidence={"target_snapshot": {"address": "intake@postbox"}},
error_type=None,
error_message=None,
)
def test_public_payload_recursively_removes_infrastructure_locators() -> None: def test_public_payload_recursively_removes_infrastructure_locators() -> None:
source = { source = {
"campaign_file": "/tmp/campaign.json", "campaign_file": "/tmp/campaign.json",
@@ -243,6 +268,33 @@ def test_operator_diagnostics_include_claim_and_storage_details() -> None:
assert "provider-secret" in diagnostics["worker_claim"]["last_error"] assert "provider-secret" in diagnostics["worker_claim"]["last_error"]
def test_postbox_receipt_summary_is_aggregate_and_contains_no_holder_identity() -> None:
summary = PostboxDeliveryReceiptSummaryRef(
delivery_id="delivery-1",
message_id="message-1",
postbox_id="postbox-1",
delivery_status="accepted",
accepted_at=_now(),
current_holder_count=2,
currently_readable=True,
read_receipt_count=1,
acknowledged_receipt_count=1,
)
attempts = _job_attempts_payload(
[],
[],
[_postbox_attempt()], # type: ignore[list-item]
postbox_receipts={"delivery-1": summary},
)
payload = attempts["postbox"][0]
assert payload["receipt_summary_status"] == "available"
assert payload["receipt_summary"]["currently_readable"] is True
assert payload["receipt_summary"]["read_receipt_count"] == 1
assert "account_id" not in repr(payload["receipt_summary"])
assert "identity_id" not in repr(payload["receipt_summary"])
def test_diagnostics_permission_is_operator_only_by_default() -> None: def test_diagnostics_permission_is_operator_only_by_default() -> None:
from govoplan_campaign.backend.manifest import PERMISSIONS, ROLE_TEMPLATES from govoplan_campaign.backend.manifest import PERMISSIONS, ROLE_TEMPLATES

View File

@@ -0,0 +1,103 @@
from __future__ import annotations
from datetime import UTC, datetime
from types import SimpleNamespace
import pytest
from govoplan_campaign.backend.persistence.campaigns import (
CampaignPersistenceError,
)
from govoplan_campaign.backend.persistence.versions import (
_normalize_review_issue_decisions,
)
def test_attachment_ask_requires_reason_and_freezes_evidence() -> None:
job = _job(
issues=[
{
"code": "missing_optional_attachment",
"behavior": "ask",
"source": "attachments",
"details": {
"effective_policy": {
"effective_behavior": "ask",
}
},
}
]
)
with pytest.raises(
CampaignPersistenceError,
match="require an explicit reason",
):
_normalize_review_issue_decisions(
[job],
[],
user_id="reviewer-1",
build_token="build-1",
)
decisions = _normalize_review_issue_decisions(
[job],
[
{
"job_id": job.id,
"decision": "accept",
"reason": "Recipient confirmed that no attachment is expected.",
}
],
user_id="reviewer-1",
build_token="build-1",
decided_at=datetime(2026, 7, 30, 12, 0, tzinfo=UTC),
)
assert decisions == [
{
"job_id": "job-1",
"review_key": "entry-1",
"decision": "accept",
"reason": "Recipient confirmed that no attachment is expected.",
"actor_user_id": "reviewer-1",
"decided_at": "2026-07-30T12:00:00+00:00",
"build_token": "build-1",
"message_sha256": "a" * 64,
"issue_fingerprint": decisions[0]["issue_fingerprint"],
"issue_codes": ["missing_optional_attachment"],
}
]
assert len(decisions[0]["issue_fingerprint"]) == 64
def test_attachment_block_cannot_be_overridden_by_review_decision() -> None:
job = _job(
issues=[
{
"code": "missing_required_attachment",
"behavior": "block",
"source": "attachments",
}
]
)
decisions = _normalize_review_issue_decisions(
[job],
[],
user_id="reviewer-1",
build_token="build-1",
)
assert decisions == []
def _job(*, issues: list[dict[str, object]]) -> SimpleNamespace:
return SimpleNamespace(
id="job-1",
entry_id="entry-1",
entry_index=1,
validation_status="needs_review",
issues_snapshot=issues,
eml_sha256="a" * 64,
)

View File

@@ -0,0 +1,63 @@
from __future__ import annotations
from collections import Counter
from govoplan_campaign.backend.router import router
from govoplan_campaign.backend.routes.attachments import router as attachments_router
from govoplan_campaign.backend.routes.campaigns import router as campaigns_router
from govoplan_campaign.backend.routes.delivery import router as delivery_router
from govoplan_campaign.backend.routes.jobs import router as jobs_router
from govoplan_campaign.backend.routes.reports import router as reports_router
from govoplan_campaign.backend.routes.sharing import router as sharing_router
from govoplan_campaign.backend.routes.versions import router as versions_router
def _operation_keys(candidate_router) -> list[tuple[str, str]]:
return [
(method, route.path)
for route in candidate_router.routes
for method in sorted(route.methods or ())
]
def test_campaign_router_composes_every_workflow_operation_once() -> None:
workflow_routers = (
campaigns_router,
versions_router,
jobs_router,
reports_router,
sharing_router,
delivery_router,
attachments_router,
)
expected = [
operation
for workflow_router in workflow_routers
for operation in _operation_keys(workflow_router)
]
actual = _operation_keys(router)
assert actual == expected
assert len(actual) == 64
assert not [operation for operation, count in Counter(actual).items() if count > 1]
def test_key_routes_are_owned_by_their_focused_router() -> None:
expectations = (
(campaigns_router, ("GET", "/campaigns/{campaign_id}/workspace")),
(versions_router, ("POST", "/campaigns/versions/{version_id}/build")),
(jobs_router, ("GET", "/campaigns/{campaign_id}/jobs")),
(reports_router, ("GET", "/campaigns/{campaign_id}/report")),
(sharing_router, ("POST", "/campaigns/{campaign_id}/shares")),
(delivery_router, ("POST", "/campaigns/{campaign_id}/send-now")),
(
attachments_router,
(
"POST",
"/campaigns/{campaign_id}/versions/{version_id}/attachments/preview",
),
),
)
for workflow_router, operation in expectations:
assert operation in _operation_keys(workflow_router)

View File

@@ -0,0 +1,133 @@
from __future__ import annotations
from datetime import UTC, datetime
from types import SimpleNamespace
from unittest.mock import Mock, patch
from govoplan_campaign.backend.routes import delivery as router
from govoplan_campaign.backend.schemas import SendCampaignNowRequest
from govoplan_campaign.backend.sending.jobs import SendCampaignNowResult
def test_send_now_omits_provider_and_recipient_text_from_response_and_audit() -> None:
campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1")
version = SimpleNamespace(
id="version-1",
raw_json={},
locked_at=datetime(2026, 7, 22, tzinfo=UTC),
validation_summary={
"ok": True,
"error_count": 0,
"warning_count": 1,
"issues": [{"message": "provider-secret validation detail"}],
},
build_summary={
"built_count": 2,
"build_failed_count": 0,
"ready_count": 2,
"warning_count": 1,
"messages": [{"recipient": "refused-recipient@example.test"}],
"build_token": "provider-secret-build-token",
},
)
delivery = SendCampaignNowResult(
campaign_id="campaign-1",
version_id="version-1",
attempted_count=2,
sent_count=1,
failed_count=1,
outcome_unknown_count=0,
skipped_count=0,
preflight_count=2,
synchronous_send_policy={
"max_recipient_jobs": 25,
"source": "deployment_default",
"deployment_max_recipient_jobs": 25,
"tenant_max_recipient_jobs": None,
"provider_diagnostic": "provider-secret-policy",
},
results=[
{
"job_id": "job-failed",
"status": "failed",
"attempt_number": 1,
"message": "provider-secret smtp.internal.example",
"recipient_email": "private@example.test",
},
{
"job_id": "job-refused",
"status": "smtp_accepted",
"attempt_number": 1,
"message": (
"SMTP accepted 1/2 envelope recipient(s); refused recipients: "
'{"refused-recipient@example.test": [550, "not allowed"]}'
),
},
],
)
principal = SimpleNamespace(tenant_id="tenant-1", user=SimpleNamespace(id="user-1"), api_key=None)
audit = Mock()
with (
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
patch.object(router, "_require_permission"),
patch.object(router, "_get_campaign_for_tenant", return_value=campaign),
patch.object(router, "_get_version_for_tenant", return_value=version),
patch.object(router, "_require_mail_profile_use_if_needed"),
patch.object(router, "is_user_locked_version", return_value=False),
patch.object(router, "send_campaign_now", return_value=delivery),
patch.object(router, "audit_from_principal", audit),
):
response = router.send_campaign_now_endpoint(
"campaign-1",
SendCampaignNowRequest(),
session=Mock(),
principal=principal, # type: ignore[arg-type]
)
payload = response.model_dump(mode="json")["result"]
assert payload["results"] == [
{"job_id": "job-failed", "status": "failed", "attempt_number": 1},
{"job_id": "job-refused", "status": "smtp_accepted", "attempt_number": 1},
]
assert payload["validation"] == {"ok": True, "error_count": 0, "warning_count": 1}
assert payload["build"] == {
"built_count": 2,
"build_failed_count": 0,
"ready_count": 2,
"warning_count": 1,
}
assert payload["synchronous_send_policy"] == {
"max_recipient_jobs": 25,
"source": "deployment_default",
"deployment_max_recipient_jobs": 25,
"tenant_max_recipient_jobs": None,
}
audit_details = audit.call_args.kwargs["details"]
assert set(audit_details) == {
"campaign_id",
"version_id",
"attempted_count",
"sent_count",
"failed_count",
"outcome_unknown_count",
"skipped_count",
"preflight_count",
"delivery_mode",
"dry_run",
"synchronous_send_policy",
}
assert "results" not in audit_details
assert "validation" not in audit_details
assert "build" not in audit_details
serialized = repr({"response": payload, "audit": audit_details})
for forbidden in (
"provider-secret",
"smtp.internal.example",
"private@example.test",
"refused-recipient@example.test",
"not allowed",
):
assert forbidden not in serialized

View File

@@ -10,6 +10,7 @@ from govoplan_campaign.backend.db.models import (
JobSendStatus, JobSendStatus,
JobValidationStatus, JobValidationStatus,
) )
from govoplan_campaign.backend.capabilities import delivery_tasks_capability
from govoplan_campaign.backend.sending.jobs import ( from govoplan_campaign.backend.sending.jobs import (
SendJobResult, SendJobResult,
_queue_validation_statuses, _queue_validation_statuses,
@@ -48,6 +49,17 @@ def _job(entry_id: str, **overrides):
class CampaignQueueSelectionTests(unittest.TestCase): class CampaignQueueSelectionTests(unittest.TestCase):
def test_delivery_task_capability_configures_module_runtime_for_worker_processes(self):
registry = object()
settings = object()
context = SimpleNamespace(registry=registry, settings=settings)
with patch("govoplan_campaign.backend.runtime.configure_runtime") as configure:
capability = delivery_tasks_capability(context)
self.assertIsNotNone(capability)
configure.assert_called_once_with(registry=registry, settings=settings)
def test_selects_queueable_jobs_without_reclassifying_retry_states(self): def test_selects_queueable_jobs_without_reclassifying_retry_states(self):
skipped_send = _job("1", send_status=JobSendStatus.FAILED_TEMPORARY.value) skipped_send = _job("1", send_status=JobSendStatus.FAILED_TEMPORARY.value)
skipped_queue = _job("2", queue_status=JobQueueStatus.PAUSED.value) skipped_queue = _job("2", queue_status=JobQueueStatus.PAUSED.value)
@@ -129,6 +141,8 @@ class CampaignQueueSelectionTests(unittest.TestCase):
) )
snapshot = SimpleNamespace( snapshot = SimpleNamespace(
mail_profile_id="profile-1", mail_profile_id="profile-1",
smtp_server_id="smtp-server-1",
smtp_credential_id="smtp-credential-1",
smtp_transport_revision="frozen", smtp_transport_revision="frozen",
delivery=SimpleNamespace( delivery=SimpleNamespace(
rate_limit=SimpleNamespace(messages_per_minute=60), rate_limit=SimpleNamespace(messages_per_minute=60),

View File

@@ -0,0 +1,94 @@
from __future__ import annotations
import unittest
from types import SimpleNamespace
from unittest.mock import patch
from govoplan_campaign.backend.routes.sharing import (
search_campaign_share_targets,
)
from govoplan_core.core.references import (
CAPABILITY_ACCESS_REFERENCE_OPTIONS,
ReferenceOption,
ReferenceSearchPage,
)
class _Provider:
def __init__(self) -> None:
self.requests = []
def search_reference_options(self, session, principal, *, request):
del session, principal
self.requests.append(request)
return ReferenceSearchPage(
options=(
ReferenceOption(
value="membership-1",
label="Ada",
kind="membership",
source_module="access",
),
),
next_cursor="offset:1",
has_more=True,
)
class _Registry:
def __init__(self) -> None:
self.provider = _Provider()
def has_capability(self, name):
return name == CAPABILITY_ACCESS_REFERENCE_OPTIONS
def capability(self, name):
return self.provider if self.has_capability(name) else None
class CampaignShareTargetSearchTests(unittest.TestCase):
def test_search_uses_membership_references_and_retains_stale_targets(self) -> None:
registry = _Registry()
principal = SimpleNamespace(
tenant_id="tenant-1",
account_id="account-1",
membership_id="membership-1",
group_ids=frozenset(),
)
with (
patch(
"govoplan_campaign.backend.routes.sharing._get_campaign_for_principal"
),
patch(
"govoplan_campaign.backend.routes.sharing.get_registry",
return_value=registry,
),
):
response = search_campaign_share_targets(
"campaign-1",
"user",
q="Ada",
selected=["removed-membership"],
limit=25,
cursor="offset:0",
session=object(), # type: ignore[arg-type]
principal=principal, # type: ignore[arg-type]
)
request = registry.provider.requests[0]
self.assertEqual("membership", request.kind)
self.assertEqual("tenant-1", request.tenant_id)
self.assertEqual("ada", request.query)
self.assertTrue(request.context["administrative"])
self.assertEqual(
["membership-1", "removed-membership"],
[option.value for option in response.options],
)
self.assertEqual("unavailable", response.options[1].availability)
self.assertTrue(response.has_more)
self.assertEqual("offset:1", response.next_cursor)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,417 @@
from __future__ import annotations
import unittest
from types import SimpleNamespace
from unittest.mock import Mock, patch
from sqlalchemy import Column, String, Table, create_engine
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.pool import StaticPool
from pydantic import ValidationError
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.db.base import Base
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignMessageAction,
CampaignMessageActionAttempt,
CampaignVersion,
JobBuildStatus,
JobQueueStatus,
JobSendStatus,
JobValidationStatus,
SendAttempt,
)
from govoplan_campaign.backend.schemas import CampaignSendJobRequest
from govoplan_campaign.backend.sending.jobs import (
QueueingError,
SendJobResult,
send_single_campaign_job,
)
from govoplan_campaign.backend.integrations import SmtpSendError
class _Mail:
def __init__(self, result: object | None = None, error: Exception | None = None):
self.result = result or SimpleNamespace(
accepted_count=1,
refused_recipients={},
)
self.error = error
self.send = Mock()
def wait_for_rate_limit(self, **_kwargs) -> None:
return None
def send_campaign_email_bytes(self, *_args, **_kwargs):
self.send(*_args, **_kwargs)
if self.error is not None:
raise self.error
return self.result
class CampaignSingleMessageActionTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine(
"sqlite+pysqlite://",
connect_args={"check_same_thread": False},
poolclass=StaticPool,
)
access_users = Base.metadata.tables.get("access_users")
if access_users is None:
access_users = Table(
"access_users",
Base.metadata,
Column("id", String(36), primary_key=True),
)
access_groups = Base.metadata.tables.get("access_groups")
if access_groups is None:
access_groups = Table(
"access_groups",
Base.metadata,
Column("id", String(36), primary_key=True),
)
Base.metadata.create_all(
self.engine,
tables=[
access_users,
access_groups,
ChangeSequenceEntry.__table__,
Campaign.__table__,
CampaignVersion.__table__,
CampaignJob.__table__,
SendAttempt.__table__,
CampaignMessageAction.__table__,
CampaignMessageActionAttempt.__table__,
],
)
self.SessionLocal = sessionmaker(
bind=self.engine,
class_=Session,
expire_on_commit=False,
)
with self.SessionLocal() as session:
campaign = Campaign(
id="campaign-1",
tenant_id="tenant-1",
external_id="campaign-1",
name="Campaign",
current_version_id="version-1",
)
version = CampaignVersion(
id="version-1",
campaign_id=campaign.id,
version_number=1,
raw_json={},
validation_summary={"ok": True},
build_summary={"build_token": "build-1"},
)
job = CampaignJob(
id="job-1",
tenant_id="tenant-1",
campaign_id=campaign.id,
campaign_version_id=version.id,
entry_index=1,
recipient_email="recipient@example.test",
eml_local_path="/tmp/message.eml",
eml_sha256="a" * 64,
eml_size_bytes=42,
build_status=JobBuildStatus.BUILT.value,
validation_status=JobValidationStatus.READY.value,
queue_status=JobQueueStatus.DRAFT.value,
send_status=JobSendStatus.NOT_QUEUED.value,
delivery_channel_policy="mail",
resolved_recipients={
"from": {"email": "sender@example.test"},
"to": [{"email": "recipient@example.test"}],
},
)
session.add_all((campaign, version, job))
session.commit()
self.addCleanup(self.engine.dispose)
@staticmethod
def _delivery_context():
return SimpleNamespace(
snapshot=SimpleNamespace(
mail_profile_id="profile-1",
smtp_server_id=None,
smtp_credential_id=None,
smtp_transport_revision="revision-1",
delivery=SimpleNamespace(
rate_limit=SimpleNamespace(messages_per_minute=60),
imap_append_sent=SimpleNamespace(enabled=False),
),
),
message_bytes=b"Subject: test\r\n\r\ncontent",
envelope_from="sender@example.test",
envelope_recipients=["recipient@example.test"],
)
def _run(
self,
session: Session,
*,
kind: str,
key: str,
mail: _Mail,
reason: str | None = None,
):
with (
patch(
"govoplan_campaign.backend.sending.jobs._validate_single_message_action"
),
patch(
"govoplan_campaign.backend.sending.jobs._send_job_delivery_context",
return_value=self._delivery_context(),
),
patch(
"govoplan_campaign.backend.sending.jobs.mail_integration",
return_value=mail,
),
patch(
"govoplan_campaign.backend.sending.jobs.audit_event"
),
):
return send_single_campaign_job(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
job_id="job-1",
kind=kind,
idempotency_key=key,
actor_user_id=None,
reason=reason,
)
def test_test_action_is_idempotent_and_leaves_official_state_unchanged(self) -> None:
mail = _Mail()
with self.SessionLocal() as session:
job = session.get(CampaignJob, "job-1")
assert job is not None
job.send_status = JobSendStatus.FAILED_TEMPORARY.value
job.attempt_count = 2
session.commit()
first = self._run(
session,
kind="test",
key="test-1",
mail=mail,
)
repeated = self._run(
session,
kind="test",
key="test-1",
mail=mail,
)
session.refresh(job)
self.assertEqual(first["status"], "accepted")
self.assertTrue(repeated["duplicate"])
self.assertEqual(job.send_status, JobSendStatus.FAILED_TEMPORARY.value)
self.assertEqual(job.attempt_count, 2)
self.assertEqual(mail.send.call_count, 1)
with self.assertRaisesRegex(QueueingError, "idempotency key"):
self._run(
session,
kind="single_resend",
key="test-1",
reason="Different command",
mail=mail,
)
def test_failed_resend_preserves_prior_success(self) -> None:
mail = _Mail(
error=SmtpSendError(
"Temporary delivery failure",
temporary=True,
)
)
with self.SessionLocal() as session:
job = session.get(CampaignJob, "job-1")
assert job is not None
job.send_status = JobSendStatus.SMTP_ACCEPTED.value
job.sent_at = job.created_at
session.commit()
result = self._run(
session,
kind="single_resend",
key="resend-failed",
reason="Recipient requested another copy",
mail=mail,
)
session.refresh(job)
self.assertEqual(result["status"], "failed_temporary")
self.assertEqual(job.send_status, JobSendStatus.SMTP_ACCEPTED.value)
self.assertIsNotNone(job.sent_at)
def test_successful_resend_completes_previously_failed_message(self) -> None:
mail = _Mail()
with self.SessionLocal() as session:
job = session.get(CampaignJob, "job-1")
assert job is not None
job.send_status = JobSendStatus.FAILED_PERMANENT.value
job.last_error = "old failure"
session.commit()
with (
patch(
"govoplan_campaign.backend.sending.jobs._validate_single_message_action"
),
patch(
"govoplan_campaign.backend.sending.jobs._send_job_delivery_context",
return_value=self._delivery_context(),
),
patch(
"govoplan_campaign.backend.sending.jobs.mail_integration",
return_value=mail,
),
patch(
"govoplan_campaign.backend.sending.jobs.files_integration"
) as files,
patch(
"govoplan_campaign.backend.sending.jobs._update_campaign_after_job"
),
patch(
"govoplan_campaign.backend.sending.jobs.audit_event"
),
):
result = send_single_campaign_job(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
job_id="job-1",
kind="single_resend",
idempotency_key="resend-success",
actor_user_id=None,
reason="Approved operator resend",
)
session.refresh(job)
self.assertEqual(result["status"], "accepted")
self.assertEqual(job.send_status, JobSendStatus.SMTP_ACCEPTED.value)
self.assertIsNotNone(job.sent_at)
files.return_value.mark_job_attachment_uses_sent.assert_called_once()
def test_unknown_resend_is_frozen_and_replay_does_not_send_again(self) -> None:
mail = _Mail(
error=SmtpSendError(
"Unknown delivery outcome",
outcome_unknown=True,
)
)
with self.SessionLocal() as session:
first = self._run(
session,
kind="single_resend",
key="resend-unknown",
reason="Approved operator resend",
mail=mail,
)
repeated = self._run(
session,
kind="single_resend",
key="resend-unknown",
reason="Approved operator resend",
mail=mail,
)
self.assertEqual(first["status"], "outcome_unknown")
self.assertTrue(repeated["duplicate"])
self.assertEqual(mail.send.call_count, 1)
def test_initial_send_links_the_canonical_attempt_and_replay_is_inert(self) -> None:
with self.SessionLocal() as session:
job = session.get(CampaignJob, "job-1")
assert job is not None
def canonical_send(_session: Session, **_kwargs):
attempt = SendAttempt(
job_id=job.id,
attempt_number=1,
status=JobSendStatus.SMTP_ACCEPTED.value,
started_at=job.created_at,
finished_at=job.created_at,
)
job.send_status = JobSendStatus.SMTP_ACCEPTED.value
job.queue_status = JobQueueStatus.DRAFT.value
job.attempt_count = 1
_session.add_all((attempt, job))
_session.commit()
return SendJobResult(
job_id=job.id,
status=JobSendStatus.SMTP_ACCEPTED.value,
attempt_number=1,
)
with (
patch(
"govoplan_campaign.backend.sending.jobs._validate_single_message_action"
),
patch(
"govoplan_campaign.backend.sending.jobs._send_job_delivery_context",
return_value=self._delivery_context(),
),
patch(
"govoplan_campaign.backend.sending.jobs._prepare_single_job_for_send",
return_value="queued",
),
patch(
"govoplan_campaign.backend.sending.jobs.send_campaign_job",
side_effect=canonical_send,
) as send,
patch(
"govoplan_campaign.backend.sending.jobs._emit_campaign_status_notification"
),
patch(
"govoplan_campaign.backend.sending.jobs.audit_event"
),
):
first = send_single_campaign_job(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
job_id="job-1",
kind="single_send",
idempotency_key="initial-1",
actor_user_id=None,
)
repeated = send_single_campaign_job(
session,
tenant_id="tenant-1",
campaign_id="campaign-1",
job_id="job-1",
kind="single_send",
idempotency_key="initial-1",
actor_user_id=None,
)
self.assertEqual(first["status"], "accepted")
self.assertIsNotNone(first["linked_send_attempt_id"])
self.assertTrue(repeated["duplicate"])
self.assertEqual(send.call_count, 1)
def test_request_requires_explicit_kind_key_and_resend_reason(self) -> None:
with self.assertRaises(ValidationError):
CampaignSendJobRequest.model_validate({})
with self.assertRaises(ValidationError):
CampaignSendJobRequest.model_validate(
{
"kind": "single_resend",
"idempotency_key": "resend-1",
}
)
request = CampaignSendJobRequest.model_validate(
{
"kind": "test",
"idempotency_key": "test-1",
}
)
self.assertEqual(request.kind, "test")
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,406 @@
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import Mock, patch
import pytest
from fastapi import HTTPException
from govoplan_campaign.backend import router as campaign_api
from govoplan_campaign.backend.routes import delivery as router
from govoplan_campaign.backend.delivery_policy import (
CampaignDeliveryPolicyError,
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
effective_synchronous_send_policy,
)
from govoplan_campaign.backend.db.models import (
JobBuildStatus,
JobQueueStatus,
JobSendStatus,
JobValidationStatus,
)
from govoplan_campaign.backend.sending.jobs import (
QueueCampaignResult,
SynchronousSendRejected,
_ensure_synchronous_send_count_allowed,
_preflight_synchronous_send_batch,
queue_campaign_jobs,
send_campaign_now,
synchronous_send_candidate_jobs,
synchronous_send_options,
)
class _PolicySession:
def __init__(self, settings: dict[str, object] | None = None) -> None:
self.tenant = SimpleNamespace(settings=settings or {})
def get(self, _model, _id):
return self.tenant
def _version() -> SimpleNamespace:
return SimpleNamespace(
id="version-1",
locked_at=object(),
published_at=None,
validation_summary={"ok": True},
build_summary={"build_token": "build-1"},
editor_state={
"review_send": {
"build_token": "build-1",
"inspection_complete": True,
"reviewed_message_keys": ["reviewed"],
}
},
)
class _Principal:
def __init__(self, *scopes: str) -> None:
self.scopes = set(scopes)
self.tenant_id = "tenant-1"
self.user = SimpleNamespace(id="user-1")
def has(self, scope: str) -> bool:
return scope in self.scopes
def _job(job_id: str, **overrides: object) -> SimpleNamespace:
values: dict[str, object] = {
"id": job_id,
"entry_id": job_id,
"entry_index": 1,
"build_status": JobBuildStatus.BUILT.value,
"validation_status": JobValidationStatus.READY.value,
"queue_status": JobQueueStatus.DRAFT.value,
"send_status": JobSendStatus.NOT_QUEUED.value,
"eml_local_path": f"{job_id}.eml",
"eml_storage_key": None,
}
values.update(overrides)
return SimpleNamespace(**values)
def test_synchronous_policy_defaults_to_25_and_tenant_can_only_narrow() -> None:
default = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={},
)
assert default.max_recipient_jobs == DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS == 25
assert default.source == "deployment_default"
narrowed = effective_synchronous_send_policy(
_PolicySession(
{"campaign_delivery_policy": {"synchronous_send_max_recipients": 10}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "40"},
)
assert narrowed.max_recipient_jobs == 10
assert narrowed.source == "tenant"
ceiling = effective_synchronous_send_policy(
_PolicySession(
{"campaign_delivery_policy": {"synchronous_send_max_recipients": 100}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "40"},
)
assert ceiling.max_recipient_jobs == 40
assert ceiling.source == "deployment_ceiling"
@pytest.mark.parametrize("value", [True, -1, 501, "2.5", "unbounded"])
def test_invalid_synchronous_policy_fails_closed(value: object) -> None:
with pytest.raises(CampaignDeliveryPolicyError):
effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": value}, # type: ignore[dict-item]
)
def test_candidate_count_uses_exact_built_and_reviewed_job_states() -> None:
jobs = [
_job("ready"),
_job("queued", queue_status=JobQueueStatus.QUEUED.value, send_status=JobSendStatus.QUEUED.value),
_job("reviewed", validation_status=JobValidationStatus.NEEDS_REVIEW.value),
_job("blocked", validation_status=JobValidationStatus.BLOCKED.value),
_job("failed", send_status=JobSendStatus.FAILED_TEMPORARY.value),
_job("missing-eml", eml_local_path=None),
]
candidates = synchronous_send_candidate_jobs(_version(), jobs) # type: ignore[arg-type]
assert [job.id for job in candidates] == ["ready", "queued", "reviewed"]
def test_limit_and_zero_count_reject_before_delivery() -> None:
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "2"},
)
with pytest.raises(SynchronousSendRejected, match="No eligible") as empty:
_ensure_synchronous_send_count_allowed(0, policy=policy)
assert empty.value.reason == "no_eligible_recipient_jobs"
with pytest.raises(SynchronousSendRejected, match="Queue it") as oversized:
_ensure_synchronous_send_count_allowed(3, policy=policy)
assert oversized.value.audit_details()["eligible_recipient_job_count"] == 3
assert oversized.value.audit_details()["synchronous_send_policy"]["max_recipient_jobs"] == 2
def test_exact_synchronous_policy_boundary_is_allowed() -> None:
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "2"},
)
_ensure_synchronous_send_count_allowed(2, policy=policy)
def test_post_queue_growth_is_rejected_before_batch_or_provider_preflight() -> None:
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "2"},
)
campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1")
initial_jobs = [_job("one"), _job("two")]
post_queue_jobs = [
_job(
job_id,
queue_status=JobQueueStatus.QUEUED.value,
send_status=JobSendStatus.QUEUED.value,
)
for job_id in ("one", "two", "concurrent")
]
queued = QueueCampaignResult(
campaign_id="campaign-1",
version_id="version-1",
queued_count=2,
skipped_count=0,
blocked_count=0,
enqueued_count=0,
delivery_mode="synchronous",
)
with (
patch("govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant", return_value=campaign),
patch("govoplan_campaign.backend.sending.jobs._get_current_version", return_value=_version()),
patch("govoplan_campaign.backend.sending.jobs._ensure_version_validated_and_locked"),
patch("govoplan_campaign.backend.sending.jobs._ensure_campaign_execution_snapshot"),
patch("govoplan_campaign.backend.sending.jobs.effective_synchronous_send_policy", return_value=policy),
patch("govoplan_campaign.backend.sending.jobs._campaign_jobs_for_queue", return_value=initial_jobs),
patch(
"govoplan_campaign.backend.sending.jobs.queue_campaign_jobs",
return_value=queued,
) as queue,
patch("govoplan_campaign.backend.sending.jobs._campaign_jobs_for_version", return_value=post_queue_jobs),
patch("govoplan_campaign.backend.sending.jobs._preflight_synchronous_send_batch") as batch_preflight,
):
with pytest.raises(SynchronousSendRejected, match="above the effective") as rejected:
send_campaign_now(
object(), # type: ignore[arg-type]
tenant_id="tenant-1",
campaign_id="campaign-1",
)
assert rejected.value.eligible_count == 3
assert queue.call_args.kwargs["commit_queue"] is False
batch_preflight.assert_not_called()
@pytest.mark.parametrize(
("workers_available", "expected_mode", "expected_enqueued"),
((False, "database_queue", 0), (True, "worker_queue", 1)),
)
def test_asynchronous_mode_matches_actual_worker_availability(
workers_available: bool,
expected_mode: str,
expected_enqueued: int,
) -> None:
campaign = SimpleNamespace(id="campaign-1")
version = _version()
job = _job("one")
with (
patch("govoplan_campaign.backend.sending.jobs._celery_enabled", return_value=workers_available),
patch("govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant", return_value=campaign),
patch("govoplan_campaign.backend.sending.jobs._get_current_version", return_value=version),
patch("govoplan_campaign.backend.sending.jobs._ensure_version_validated_and_locked"),
patch("govoplan_campaign.backend.sending.jobs._ensure_campaign_execution_snapshot"),
patch("govoplan_campaign.backend.sending.jobs._campaign_jobs_for_queue", return_value=[job]),
patch(
"govoplan_campaign.backend.sending.jobs._select_campaign_jobs_for_queue",
return_value=([job], 0, 0),
),
patch("govoplan_campaign.backend.sending.jobs._persist_campaign_queue") as persist,
patch(
"govoplan_campaign.backend.sending.jobs._enqueue_campaign_jobs",
return_value=expected_enqueued,
) as enqueue,
):
result = queue_campaign_jobs(
object(), # type: ignore[arg-type]
tenant_id="tenant-1",
campaign_id="campaign-1",
enqueue_celery=True,
)
assert result.delivery_mode == expected_mode
assert result.worker_queue_available is workers_available
assert result.enqueued_count == expected_enqueued
assert persist.call_args.kwargs["delivery_mode"] == expected_mode
assert persist.call_args.kwargs["commit"] is True
assert enqueue.call_args.kwargs["enabled"] is workers_available
def test_invalid_policy_disables_synchronous_mode_without_hiding_queue_availability() -> None:
campaign = SimpleNamespace(id="campaign-1")
version = _version()
with (
patch("govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant", return_value=campaign),
patch("govoplan_campaign.backend.sending.jobs._get_version_for_campaign", return_value=version),
patch("govoplan_campaign.backend.sending.jobs._campaign_jobs_for_version", return_value=[_job("one")]),
patch("govoplan_campaign.backend.sending.jobs._celery_enabled", return_value=True),
patch(
"govoplan_campaign.backend.sending.jobs.effective_synchronous_send_policy",
side_effect=CampaignDeliveryPolicyError("invalid deployment value"),
),
):
options = synchronous_send_options(
object(), # type: ignore[arg-type]
tenant_id="tenant-1",
campaign_id="campaign-1",
)
assert options["worker_queue_available"] is True
assert options["synchronous_send"]["allowed"] is False
assert options["synchronous_send"]["reason"] == "policy_configuration_invalid"
assert options["synchronous_send"]["policy"] == {}
@pytest.mark.parametrize(
("path", "required_scope"),
(
("/campaigns/{campaign_id}/send-now", "campaigns:campaign:send"),
("/campaigns/{campaign_id}/queue", "campaigns:campaign:queue"),
),
)
def test_delivery_endpoints_require_their_mode_permission_and_recipient_authority(
path: str,
required_scope: str,
) -> None:
route = next(item for item in campaign_api.router.routes if item.path == path)
dependency = next(item for item in route.dependant.dependencies if item.name == "principal")
with pytest.raises(HTTPException) as missing_mode_permission:
dependency.call(_Principal("campaigns:recipient:read"))
assert missing_mode_permission.value.status_code == 403
allowed = _Principal(required_scope, "campaigns:recipient:read")
assert dependency.call(allowed) is allowed
mode_only = _Principal(required_scope)
with (
patch.object(router, "_get_campaign_for_principal"),
pytest.raises(HTTPException) as missing_recipient_authority,
):
if required_scope == "campaigns:campaign:send":
router.send_campaign_now_endpoint(
"campaign-1",
session=Mock(),
principal=mode_only, # type: ignore[arg-type]
)
else:
router.queue_campaign(
"campaign-1",
session=Mock(),
principal=mode_only, # type: ignore[arg-type]
)
assert missing_recipient_authority.value.status_code == 403
assert "campaigns:recipient:read" in missing_recipient_authority.value.detail
def test_batch_preflight_checks_every_message_before_provider_effects() -> None:
jobs = [_job("one"), _job("two")]
contexts = {
"one": SimpleNamespace(snapshot=SimpleNamespace(smtp_transport_revision="revision-1")),
"two": SimpleNamespace(snapshot=SimpleNamespace(smtp_transport_revision="revision-1")),
}
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={},
)
provider = Mock()
with (
patch("govoplan_campaign.backend.sending.jobs._preflight_send_campaign_job", return_value=None) as state_preflight,
patch(
"govoplan_campaign.backend.sending.jobs._send_job_delivery_context",
side_effect=lambda _session, job: contexts[job.id],
) as input_preflight,
patch(
"govoplan_campaign.backend.sending.jobs.profile_delivery_summary",
return_value={"smtp_transport_revision": "revision-1"},
),
patch("govoplan_campaign.backend.sending.jobs.mail_integration", return_value=provider),
):
result = _preflight_synchronous_send_batch(
object(), # type: ignore[arg-type]
version=_version(), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=policy,
)
assert list(result) == ["one", "two"]
assert state_preflight.call_count == 2
assert input_preflight.call_count == 2
provider.send_campaign_email_bytes.assert_not_called()
def test_rejected_synchronous_preflight_rolls_back_staged_queue_before_audit() -> None:
session = Mock()
campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1")
version = SimpleNamespace(
id="version-1",
raw_json={},
locked_at=object(),
validation_summary={"ok": True},
build_summary={"built_count": 1},
)
rejection = SynchronousSendRejected(
"Preflight rejected the staged send.",
reason="batch_preflight_failed",
eligible_count=1,
)
with (
patch.object(router, "_get_campaign_for_principal"),
patch.object(router, "_require_permission"),
patch.object(router, "_get_campaign_for_tenant", return_value=campaign),
patch.object(router, "_get_version_for_tenant", return_value=version),
patch.object(router, "_require_mail_profile_use_if_needed"),
patch.object(router, "is_user_locked_version", return_value=False),
patch.object(router, "send_campaign_now", side_effect=rejection),
patch.object(router, "audit_from_principal") as audit,
pytest.raises(HTTPException) as rejected,
):
router.send_campaign_now_endpoint(
"campaign-1",
session=session,
principal=_Principal(
"campaigns:campaign:send", "campaigns:recipient:read"
), # type: ignore[arg-type]
)
assert rejected.value.status_code == 422
session.rollback.assert_called_once_with()
audit.assert_called_once()
assert audit.call_args.kwargs["action"] == "campaign.send_now_rejected"
assert audit.call_args.kwargs["commit"] is True

Some files were not shown because too many files have changed in this diff Show More