Compare commits
92 Commits
724ca779d6
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 5f7503598c | |||
| cd223cbb95 | |||
| 4a120e8009 | |||
| c769be39da | |||
| 46df12c025 | |||
| 961d5d1130 | |||
| 23b9a531d5 | |||
| cc93945f79 | |||
| 101f3ccd7d | |||
| 2199187e8b | |||
| dd9592a192 | |||
| 5240749ae1 | |||
| f11c56e890 | |||
| 8d0a2608ed | |||
| 89ae14c032 | |||
| 38af25ee88 | |||
| 4774a8025c | |||
| 689dc1fd6b | |||
| 90677348ff | |||
| 06786e86ef | |||
| 6fda123fc3 | |||
| 2fa91bb943 | |||
| e3cc476508 | |||
| 01ef541917 | |||
| d567257311 | |||
| 3075ef7f5b | |||
| 1ab4e91ffd | |||
| 735e874bd0 | |||
| 99d44eeb8d | |||
| f095a3e2c7 | |||
| 1225802c5d | |||
| ac3329cafe | |||
| 4eb651c6ac | |||
| 0b4017c240 | |||
| 79b576b4bc | |||
| b0282ebff2 | |||
| aae4ef5952 | |||
| 8ee87b7558 | |||
| 7229fb8e3d | |||
| 3487ec7048 | |||
| 22d72f82f5 | |||
| fc36aee6c0 | |||
| 06125cc0e8 | |||
| 21f3014ac5 | |||
| 62a68792a4 | |||
| aa4ec66b7b | |||
| 60efd1cb5d | |||
| 7e1660344d | |||
| a8c0750dd7 | |||
| bfbb86564c | |||
| c05bb8e474 | |||
| 99ef25b08f | |||
| 03100b77db | |||
| 0ac903c82d | |||
| 60776803c5 | |||
| 9a709b1264 | |||
| af833ca38c | |||
| 50c509d161 | |||
| 9f4eab07f6 | |||
| 25a69b3fa9 | |||
| 24538c2a99 | |||
| 7d8579194d | |||
| 09c63de813 | |||
| 057e660b17 | |||
| 701c0fe184 | |||
| 2c5519908a | |||
| dc56687af6 | |||
| 2c70c553ac | |||
| 8627d0e135 | |||
| d2adcca7ae | |||
| 002ca4b371 | |||
| 641bead0d8 | |||
| 3c305753d6 | |||
| ef513816f8 | |||
| b7653b58f4 | |||
| ce92499333 | |||
| 04bc6c430e | |||
| 04c214b149 | |||
| 35b3cc151d | |||
| c54fec4cc3 | |||
| 1d291377c0 | |||
| 78f52a36d4 | |||
| 0c4f198802 | |||
| c71fa3dc64 | |||
| 0a6064ec62 | |||
| fce6dd1138 | |||
| 4b9eb79065 | |||
| 7ea5bdb217 | |||
| f755cdf48d | |||
| 8965b27517 | |||
| 12036b1f36 | |||
| ad34365f6c |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -332,6 +332,8 @@ cython_debug/
|
|||||||
webui/.policy-test-build/
|
webui/.policy-test-build/
|
||||||
webui/.template-preview-test-build/
|
webui/.template-preview-test-build/
|
||||||
webui/.import-test-build/
|
webui/.import-test-build/
|
||||||
|
webui/.review-preview-test-build/
|
||||||
|
webui/.report-grid-test-build/
|
||||||
|
|
||||||
# GovOPlaN shared ignore rules from govoplan-core
|
# GovOPlaN shared ignore rules from govoplan-core
|
||||||
# Local WebUI test/build scratch directories
|
# Local WebUI test/build scratch directories
|
||||||
@@ -340,6 +342,7 @@ webui/.import-test-build/
|
|||||||
.policy-test-build/
|
.policy-test-build/
|
||||||
.template-preview-test-build/
|
.template-preview-test-build/
|
||||||
.import-test-build/
|
.import-test-build/
|
||||||
|
.review-preview-test-build/
|
||||||
webui/.component-test-build/
|
webui/.component-test-build/
|
||||||
webui/.module-test-build/
|
webui/.module-test-build/
|
||||||
*.db
|
*.db
|
||||||
|
|||||||
19
README.md
19
README.md
@@ -15,7 +15,8 @@ This repository owns:
|
|||||||
- campaign/version/job/issue/send-attempt/append-attempt models and migrations
|
- campaign/version/job/issue/send-attempt/append-attempt models and migrations
|
||||||
- campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths
|
- campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths
|
||||||
- WebUI package `@govoplan/campaign-webui`
|
- WebUI package `@govoplan/campaign-webui`
|
||||||
- route contributions for `/campaigns`, `/campaigns/:campaignId/*`, `/operator`, `/reports`, and `/templates`
|
- route contributions for `/campaigns`, the integrated `/campaigns/queue` view,
|
||||||
|
`/campaigns/:campaignId/*`, `/reports`, and `/templates`
|
||||||
|
|
||||||
Core owns the auth facade, RBAC/capability contracts, database/session
|
Core owns the auth facade, RBAC/capability contracts, database/session
|
||||||
primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an
|
primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an
|
||||||
@@ -30,8 +31,16 @@ The module has one required runtime dependency:
|
|||||||
|
|
||||||
Files and mail are optional module integrations declared in the campaign manifest:
|
Files and mail are optional module integrations declared in the campaign manifest:
|
||||||
|
|
||||||
- `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Without it, campaigns can still use legacy/local attachment paths where configured.
|
- `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Server/API campaigns require this integration for attachments and never resolve caller-supplied local filesystem paths. Legacy file-oriented loading remains available only to explicitly trusted operator/library workflows.
|
||||||
- `govoplan-mail` enables reusable mail profiles, delivery policy checks, SMTP sending, and IMAP append behavior. Without it, campaigns can still be authored, validated, built, and reported, but real delivery/profile features are unavailable.
|
- `govoplan-mail` owns reusable profiles, encrypted SMTP/IMAP credentials, delivery policy checks, connection tests, and transport execution. Campaign JSON stores only `server.mail_profile_id`; inline transport settings and credentials are rejected. Without Mail, campaigns can still be authored, but profile validation and real delivery are unavailable.
|
||||||
|
|
||||||
|
Public campaign, version, job, and report responses expose business data and
|
||||||
|
delivery evidence, but never process-local paths, storage-backend keys, or
|
||||||
|
worker claim tokens. Operational troubleshooting uses the dedicated job
|
||||||
|
diagnostics endpoint and requires the tenant-level
|
||||||
|
`campaigns:diagnostic:read` permission. The campaign sender role receives this
|
||||||
|
permission; tenant-wide administrator scopes continue to grant it through the
|
||||||
|
standard policy evaluator.
|
||||||
|
|
||||||
Backend optional behavior is accessed through core-provided capabilities, not direct required imports. WebUI optional behavior uses core module metadata/capabilities so campaign pages can build and run without files or mail WebUI packages installed.
|
Backend optional behavior is accessed through core-provided capabilities, not direct required imports. WebUI optional behavior uses core module metadata/capabilities so campaign pages can build and run without files or mail WebUI packages installed.
|
||||||
|
|
||||||
@@ -83,7 +92,11 @@ Platform RBAC and governance rules are documented in `govoplan-core/docs/`.
|
|||||||
|
|
||||||
## Operations
|
## Operations
|
||||||
|
|
||||||
|
- [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives.
|
||||||
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
|
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
|
||||||
|
- Immediate delivery is bounded to 25 exact eligible recipient jobs by default. Deployments may set `GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0–500), and tenants may narrow that ceiling through `campaign_delivery_policy.synchronous_send_max_recipients` in tenant settings.
|
||||||
|
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
|
||||||
|
- [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path.
|
||||||
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.
|
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.
|
||||||
- [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management.
|
- [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management.
|
||||||
- [Example campaigns and release checklist](docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md) defines the maintained example scenarios and release gates.
|
- [Example campaigns and release checklist](docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md) defines the maintained example scenarios and release gates.
|
||||||
|
|||||||
@@ -9,3 +9,5 @@ GOVOPLAN_MAIL_TEST_IMAP_PORT=3143
|
|||||||
GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent
|
GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent
|
||||||
GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password
|
GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password
|
||||||
GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45
|
GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45
|
||||||
|
GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT=36379
|
||||||
|
GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS=3
|
||||||
|
|||||||
@@ -44,6 +44,111 @@ If the smoke is started immediately after `docker compose up -d`, GreenMail may
|
|||||||
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
|
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
|
||||||
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
|
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
|
||||||
|
|
||||||
|
## Campaign Acceptance
|
||||||
|
|
||||||
|
The transport smoke proves the Mail adapters. The Campaign acceptance runner
|
||||||
|
proves the public composition: it creates an isolated temporary Core database,
|
||||||
|
creates a Mail-owned encrypted profile through the API, materializes the
|
||||||
|
credential-free [`greenmail-delivery`](../../examples/greenmail-delivery/campaign.json)
|
||||||
|
fixture with only that profile reference, validates/builds it, sends the exact
|
||||||
|
generated EML through Campaign once, appends it once, and cross-checks Campaign
|
||||||
|
report/audit state with one unique-subject message in the GreenMail INBOX and
|
||||||
|
Sent folders. Provider mailbox verification is not a byte-for-byte comparison
|
||||||
|
after provider-side header or storage transformations.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
|
||||||
|
set -a
|
||||||
|
. ./.env
|
||||||
|
set +a
|
||||||
|
/mnt/DATA/git/govoplan/.venv/bin/python run_campaign_acceptance.py \
|
||||||
|
--evidence /tmp/govoplan-campaign-greenmail-evidence.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The default run also uses controlled loopback protocol endpoints to prove that
|
||||||
|
an SMTP connection loss before transmission is temporary, an explicit SMTP
|
||||||
|
authentication rejection is permanent, a final `451` response after DATA is
|
||||||
|
temporary, one accepted and one refused RCPT command is retained as partial
|
||||||
|
envelope acceptance, a connection loss after complete DATA is frozen as
|
||||||
|
`outcome_unknown`, and an IMAP authentication rejection after SMTP acceptance
|
||||||
|
leaves the send accepted while the append fails. A
|
||||||
|
second ordinary send must be rejected before another provider effect.
|
||||||
|
|
||||||
|
The worker drill queues one job, starts the registered
|
||||||
|
`govoplan.campaigns.send_email` task body in a dedicated OS process, waits until
|
||||||
|
the controlled endpoint has received complete DATA, terminates that process,
|
||||||
|
and starts the same task body in a fresh process. It proves the durable
|
||||||
|
`sending`/unfinished-attempt boundary is recovered as `outcome_unknown`
|
||||||
|
without a second SMTP connection or DATA transaction. This is a real process
|
||||||
|
and task-boundary interruption, but it does not start a Celery daemon, Redis
|
||||||
|
broker, or broker redelivery; `celery_broker_redelivery` therefore remains
|
||||||
|
`false` in the evidence.
|
||||||
|
|
||||||
|
The bounded JSON contains no endpoint, account, address, credential, profile,
|
||||||
|
campaign, version, or job identifiers. It records module versions, the fixture
|
||||||
|
hash, normalized classifications/counts, the Mail-profile boundary, required
|
||||||
|
audit actions, provider mailbox increments, and coverage flags. Runtime version
|
||||||
|
declarations identify the exercised composition; they do not claim that the
|
||||||
|
sources are clean, tagged, signed, or release-provenanced. The evidence names
|
||||||
|
them `declared_module_versions` and keeps `source_artifact_provenance` false;
|
||||||
|
exact commit/artifact provenance belongs to the package and release gate.
|
||||||
|
|
||||||
|
This runner is restricted to literal loopback IP addresses and the synchronous
|
||||||
|
Campaign delivery mode. Hostnames such as `localhost` and every non-loopback
|
||||||
|
address fail before profile creation, avoiding a DNS change between validation
|
||||||
|
and connection. It is local target-like evidence, not approval of an
|
||||||
|
institution's SMTP/IMAP service. The controlled post-DATA, temporary-response,
|
||||||
|
partial-refusal, and task-process interruption drills are local effect-level
|
||||||
|
proof, not proof of a target provider's behavior or Redis/Celery broker
|
||||||
|
redelivery. Use `--success-only` only when testing the success journey without
|
||||||
|
the local failure endpoints.
|
||||||
|
|
||||||
|
## Redis/Celery Redelivery Acceptance
|
||||||
|
|
||||||
|
Run the maintained broker/worker-loss acceptance separately from the GreenMail
|
||||||
|
journey:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
|
||||||
|
set -a
|
||||||
|
. ./.env
|
||||||
|
set +a
|
||||||
|
/mnt/DATA/git/govoplan/.venv/bin/python run_celery_redelivery_acceptance.py \
|
||||||
|
--evidence /tmp/govoplan-campaign-celery-redelivery-evidence.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The runner creates a unique Compose project, starts only its loopback-bound,
|
||||||
|
AOF-enabled Redis service, creates an isolated temporary GovOPlaN database,
|
||||||
|
and starts a real Celery worker subscribed to `send_email`. A controlled SMTP
|
||||||
|
server holds the transaction after complete DATA and before the final response.
|
||||||
|
The runner kills that solo worker with the task still unacknowledged and starts
|
||||||
|
a replacement worker. After the configured Redis visibility timeout, the same Celery task identity must be redelivered.
|
||||||
|
The replacement must turn the durable
|
||||||
|
unfinished attempt into `outcome_unknown`, acknowledge the task, drain the
|
||||||
|
broker queue/unacked records, and leave the SMTP endpoint at exactly one
|
||||||
|
connection and one DATA transaction.
|
||||||
|
|
||||||
|
Only bounded counts, classifications, and booleans are retained. Worker logs,
|
||||||
|
task IDs, database identifiers, endpoints, credentials, and raw diagnostics are
|
||||||
|
kept in the temporary runtime and deleted. The evidence proves the local Redis
|
||||||
|
transport, real Celery process boundary, runner-supervised replacement, and
|
||||||
|
Campaign's duplicate-effect guard. It deliberately keeps production daemon supervision,
|
||||||
|
target-provider behavior, and source-artifact provenance false.
|
||||||
|
It does not claim that systemd, Kubernetes, another container orchestrator, or
|
||||||
|
an institution's Redis/SMTP deployment behaves identically.
|
||||||
|
|
||||||
|
The default run requires Docker CLI/Compose/daemon access and permission to
|
||||||
|
pull `redis:7-alpine`; the Celery workers execute from the current Python
|
||||||
|
environment. The isolated Compose project and volume are removed on exit.
|
||||||
|
`GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS` defaults to three
|
||||||
|
seconds only to make this destructive local drill finish promptly; it is not a
|
||||||
|
production recommendation.
|
||||||
|
|
||||||
|
Starting the maintained test bed requires a working Docker CLI, Compose plugin,
|
||||||
|
daemon/socket access, and permission to pull `greenmail/standalone:2.1.9`. The
|
||||||
|
Campaign runner needs only the already-running loopback endpoints; it neither
|
||||||
|
starts Docker nor claims that it did.
|
||||||
|
|
||||||
## Use With A Campaign
|
## Use With A Campaign
|
||||||
|
|
||||||
Use the same settings in a campaign mail profile:
|
Use the same settings in a campaign mail profile:
|
||||||
|
|||||||
@@ -15,3 +15,19 @@ services:
|
|||||||
- "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025"
|
- "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025"
|
||||||
- "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143"
|
- "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143"
|
||||||
- "127.0.0.1:38080:8080"
|
- "127.0.0.1:38080:8080"
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
command: ["redis-server", "--appendonly", "yes"]
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 1s
|
||||||
|
timeout: 1s
|
||||||
|
retries: 30
|
||||||
|
volumes:
|
||||||
|
- campaign-redis-data:/data
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
campaign-redis-data:
|
||||||
|
|||||||
1935
dev/mail-testbed/run_campaign_acceptance.py
Normal file
1935
dev/mail-testbed/run_campaign_acceptance.py
Normal file
File diff suppressed because it is too large
Load Diff
856
dev/mail-testbed/run_celery_redelivery_acceptance.py
Normal file
856
dev/mail-testbed/run_celery_redelivery_acceptance.py
Normal file
@@ -0,0 +1,856 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Prove Redis/Celery redelivery does not repeat an ambiguous SMTP effect.
|
||||||
|
|
||||||
|
The default run starts an isolated Redis Compose service, two successive real
|
||||||
|
Celery worker processes, and a controlled loopback SMTP endpoint. It kills the
|
||||||
|
first worker after complete DATA but before a final SMTP response. The same
|
||||||
|
unacknowledged broker task must be delivered to the replacement worker, which
|
||||||
|
must freeze the unfinished durable attempt as ``outcome_unknown`` without a
|
||||||
|
second SMTP connection or DATA transaction.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from dataclasses import dataclass, replace
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
from typing import Any, Callable, Iterator, Mapping
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from redis import Redis
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
|
||||||
|
|
||||||
|
SCRIPT_ROOT = Path(__file__).resolve().parent
|
||||||
|
REPOSITORY_ROOT = SCRIPT_ROOT.parents[1]
|
||||||
|
if str(SCRIPT_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(SCRIPT_ROOT))
|
||||||
|
|
||||||
|
from run_campaign_acceptance import ( # noqa: E402
|
||||||
|
AcceptanceError,
|
||||||
|
DEFAULT_FIXTURE,
|
||||||
|
EXPECTED_AUDIT_ACTIONS,
|
||||||
|
TestbedSettings,
|
||||||
|
_assert_evidence_safe,
|
||||||
|
_core_package_version,
|
||||||
|
_durable_state_evidence,
|
||||||
|
_expect,
|
||||||
|
_report_evidence,
|
||||||
|
create_mail_profile,
|
||||||
|
prepare_campaign_scenario,
|
||||||
|
required_composition_versions,
|
||||||
|
smtp_fault_endpoint,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
EVIDENCE_SCHEMA = "govoplan.campaign.celery-redelivery-acceptance.v1"
|
||||||
|
MAX_EVIDENCE_BYTES = 128 * 1024
|
||||||
|
DEFAULT_COMPOSE_FILE = SCRIPT_ROOT / "docker-compose.yml"
|
||||||
|
TASK_RECEIVED_PATTERN = re.compile(
|
||||||
|
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] received",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
TASK_SUCCEEDED_PATTERN = re.compile(
|
||||||
|
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] succeeded",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
WORKER_BOOTSTRAP = r"""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from govoplan_core.celery_app import celery
|
||||||
|
|
||||||
|
visibility_timeout = int(os.environ["GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS"])
|
||||||
|
celery.conf.broker_transport_options = {
|
||||||
|
**dict(celery.conf.broker_transport_options or {}),
|
||||||
|
"polling_interval": 0.25,
|
||||||
|
"visibility_timeout": visibility_timeout,
|
||||||
|
}
|
||||||
|
celery.worker_main(
|
||||||
|
[
|
||||||
|
"worker",
|
||||||
|
"--loglevel=INFO",
|
||||||
|
"--pool=solo",
|
||||||
|
"--concurrency=1",
|
||||||
|
"--queues=send_email",
|
||||||
|
f"--hostname={sys.argv[1]}@%h",
|
||||||
|
"--without-gossip",
|
||||||
|
"--without-mingle",
|
||||||
|
"--without-heartbeat",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class WorkerProcess:
|
||||||
|
process: subprocess.Popen[bytes]
|
||||||
|
log_path: Path
|
||||||
|
log_handle: Any
|
||||||
|
|
||||||
|
def text(self) -> str:
|
||||||
|
self.log_handle.flush()
|
||||||
|
try:
|
||||||
|
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
except OSError as exc:
|
||||||
|
raise AcceptanceError("Celery worker evidence log could not be read") from exc
|
||||||
|
|
||||||
|
def received_task_ids(self) -> tuple[str, ...]:
|
||||||
|
return tuple(TASK_RECEIVED_PATTERN.findall(self.text()))
|
||||||
|
|
||||||
|
def succeeded_task_ids(self) -> tuple[str, ...]:
|
||||||
|
return tuple(TASK_SUCCEEDED_PATTERN.findall(self.text()))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class RedisBrokerState:
|
||||||
|
queue_depth: int
|
||||||
|
unacked_hash_count: int
|
||||||
|
unacked_index_count: int
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, int]:
|
||||||
|
return {
|
||||||
|
"queue_depth": self.queue_depth,
|
||||||
|
"unacked_hash_count": self.unacked_hash_count,
|
||||||
|
"unacked_index_count": self.unacked_index_count,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _positive_int(value: str, *, label: str) -> int:
|
||||||
|
try:
|
||||||
|
parsed = int(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise argparse.ArgumentTypeError(f"{label} must be a positive integer") from exc
|
||||||
|
if parsed <= 0:
|
||||||
|
raise argparse.ArgumentTypeError(f"{label} must be a positive integer")
|
||||||
|
return parsed
|
||||||
|
|
||||||
|
|
||||||
|
def _unused_loopback_port() -> int:
|
||||||
|
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as probe:
|
||||||
|
probe.bind(("127.0.0.1", 0))
|
||||||
|
return int(probe.getsockname()[1])
|
||||||
|
|
||||||
|
|
||||||
|
def _compose_command(
|
||||||
|
*, compose_file: Path, project_name: str, operation: str
|
||||||
|
) -> list[str]:
|
||||||
|
prefix = [
|
||||||
|
"docker",
|
||||||
|
"compose",
|
||||||
|
"--file",
|
||||||
|
str(compose_file),
|
||||||
|
"--project-name",
|
||||||
|
project_name,
|
||||||
|
]
|
||||||
|
if operation == "up":
|
||||||
|
return [*prefix, "up", "--detach", "redis"]
|
||||||
|
if operation == "down":
|
||||||
|
return [*prefix, "down", "--volumes", "--remove-orphans"]
|
||||||
|
raise AcceptanceError("Unsupported Redis Compose operation")
|
||||||
|
|
||||||
|
|
||||||
|
def _run_compose(
|
||||||
|
command: list[str],
|
||||||
|
*,
|
||||||
|
environment: Mapping[str, str],
|
||||||
|
timeout_seconds: int,
|
||||||
|
) -> None:
|
||||||
|
try:
|
||||||
|
completed = subprocess.run(
|
||||||
|
command,
|
||||||
|
env=dict(environment),
|
||||||
|
stdin=subprocess.DEVNULL,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE,
|
||||||
|
timeout=timeout_seconds,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
raise AcceptanceError("Redis Compose lifecycle command failed") from exc
|
||||||
|
if completed.returncode != 0:
|
||||||
|
raise AcceptanceError("Redis Compose lifecycle command failed")
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_for_redis(redis_url: str, *, timeout_seconds: int) -> None:
|
||||||
|
deadline = time.monotonic() + timeout_seconds
|
||||||
|
client = Redis.from_url(
|
||||||
|
redis_url,
|
||||||
|
socket_connect_timeout=1,
|
||||||
|
socket_timeout=1,
|
||||||
|
decode_responses=False,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
try:
|
||||||
|
if client.ping() is True:
|
||||||
|
return
|
||||||
|
except RedisError:
|
||||||
|
pass
|
||||||
|
time.sleep(0.25)
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
raise AcceptanceError("Isolated Redis broker did not become ready")
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def isolated_redis_broker(
|
||||||
|
*,
|
||||||
|
compose_file: Path,
|
||||||
|
timeout_seconds: int,
|
||||||
|
requested_port: int | None = None,
|
||||||
|
) -> Iterator[str]:
|
||||||
|
if shutil.which("docker") is None:
|
||||||
|
raise AcceptanceError("Docker CLI is required to start the isolated Redis broker")
|
||||||
|
if not compose_file.is_file():
|
||||||
|
raise AcceptanceError("Redis Compose definition is unavailable")
|
||||||
|
port = requested_port or _unused_loopback_port()
|
||||||
|
if port <= 0 or port > 65_535:
|
||||||
|
raise AcceptanceError("Redis test port is invalid")
|
||||||
|
project_name = f"govoplan-campaign-redelivery-{uuid4().hex[:12]}"
|
||||||
|
environment = {
|
||||||
|
**os.environ,
|
||||||
|
"GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT": str(port),
|
||||||
|
}
|
||||||
|
lifecycle_attempted = False
|
||||||
|
try:
|
||||||
|
lifecycle_attempted = True
|
||||||
|
_run_compose(
|
||||||
|
_compose_command(
|
||||||
|
compose_file=compose_file,
|
||||||
|
project_name=project_name,
|
||||||
|
operation="up",
|
||||||
|
),
|
||||||
|
environment=environment,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
redis_url = f"redis://127.0.0.1:{port}/0"
|
||||||
|
_wait_for_redis(redis_url, timeout_seconds=timeout_seconds)
|
||||||
|
yield redis_url
|
||||||
|
finally:
|
||||||
|
if lifecycle_attempted:
|
||||||
|
_run_compose(
|
||||||
|
_compose_command(
|
||||||
|
compose_file=compose_file,
|
||||||
|
project_name=project_name,
|
||||||
|
operation="down",
|
||||||
|
),
|
||||||
|
environment=environment,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _start_worker(runtime_root: Path, *, label: str) -> WorkerProcess:
|
||||||
|
log_path = runtime_root / f"{label}.log"
|
||||||
|
log_handle = log_path.open("wb")
|
||||||
|
environment = {**os.environ, "PYTHONUNBUFFERED": "1"}
|
||||||
|
try:
|
||||||
|
process = subprocess.Popen(
|
||||||
|
[sys.executable, "-c", WORKER_BOOTSTRAP, label],
|
||||||
|
env=environment,
|
||||||
|
stdin=subprocess.DEVNULL,
|
||||||
|
stdout=log_handle,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
close_fds=True,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
log_handle.close()
|
||||||
|
raise
|
||||||
|
return WorkerProcess(process=process, log_path=log_path, log_handle=log_handle)
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_for_worker_ready(worker: WorkerProcess, *, timeout_seconds: int) -> None:
|
||||||
|
deadline = time.monotonic() + timeout_seconds
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if worker.process.poll() is not None:
|
||||||
|
raise AcceptanceError("Celery worker exited before becoming ready")
|
||||||
|
if " ready." in worker.text():
|
||||||
|
return
|
||||||
|
time.sleep(0.2)
|
||||||
|
raise AcceptanceError("Celery worker did not become ready")
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_for_received_task(
|
||||||
|
worker: WorkerProcess,
|
||||||
|
*,
|
||||||
|
timeout_seconds: int,
|
||||||
|
expected_task_id: str | None = None,
|
||||||
|
) -> str:
|
||||||
|
deadline = time.monotonic() + timeout_seconds
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
received = worker.received_task_ids()
|
||||||
|
if received:
|
||||||
|
if len(set(received)) != 1:
|
||||||
|
raise AcceptanceError("Celery worker received more than one task identity")
|
||||||
|
task_id = received[0]
|
||||||
|
if expected_task_id is not None and task_id != expected_task_id:
|
||||||
|
raise AcceptanceError("Replacement worker received a different broker task")
|
||||||
|
return task_id
|
||||||
|
if worker.process.poll() is not None:
|
||||||
|
raise AcceptanceError("Celery worker exited before receiving the task")
|
||||||
|
time.sleep(0.2)
|
||||||
|
raise AcceptanceError("Celery worker did not receive the broker task")
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_for_task_success(
|
||||||
|
worker: WorkerProcess,
|
||||||
|
*,
|
||||||
|
task_id: str,
|
||||||
|
timeout_seconds: int,
|
||||||
|
) -> None:
|
||||||
|
deadline = time.monotonic() + timeout_seconds
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
succeeded = worker.succeeded_task_ids()
|
||||||
|
if task_id in succeeded:
|
||||||
|
return
|
||||||
|
if worker.process.poll() is not None:
|
||||||
|
raise AcceptanceError("Replacement Celery worker exited before task success")
|
||||||
|
time.sleep(0.2)
|
||||||
|
raise AcceptanceError("Redelivered Celery task did not complete")
|
||||||
|
|
||||||
|
|
||||||
|
def _kill_worker(worker: WorkerProcess, *, timeout_seconds: int) -> int:
|
||||||
|
if worker.process.poll() is not None:
|
||||||
|
raise AcceptanceError("Celery worker exited before controlled termination")
|
||||||
|
worker.process.kill()
|
||||||
|
try:
|
||||||
|
return_code = worker.process.wait(timeout=timeout_seconds)
|
||||||
|
except subprocess.TimeoutExpired as exc:
|
||||||
|
raise AcceptanceError("Celery worker could not be killed") from exc
|
||||||
|
if return_code == 0:
|
||||||
|
raise AcceptanceError("Celery worker termination was not forced")
|
||||||
|
return return_code
|
||||||
|
|
||||||
|
|
||||||
|
def _stop_worker(worker: WorkerProcess, *, timeout_seconds: int) -> None:
|
||||||
|
if worker.process.poll() is None:
|
||||||
|
worker.process.terminate()
|
||||||
|
try:
|
||||||
|
worker.process.wait(timeout=timeout_seconds)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
worker.process.kill()
|
||||||
|
worker.process.wait(timeout=timeout_seconds)
|
||||||
|
worker.log_handle.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _broker_state(redis_url: str) -> RedisBrokerState:
|
||||||
|
client = Redis.from_url(
|
||||||
|
redis_url,
|
||||||
|
socket_connect_timeout=2,
|
||||||
|
socket_timeout=2,
|
||||||
|
decode_responses=False,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
return RedisBrokerState(
|
||||||
|
queue_depth=int(client.llen("send_email")),
|
||||||
|
unacked_hash_count=int(client.hlen("unacked")),
|
||||||
|
unacked_index_count=int(client.zcard("unacked_index")),
|
||||||
|
)
|
||||||
|
except RedisError as exc:
|
||||||
|
raise AcceptanceError("Redis broker state could not be inspected") from exc
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_for_broker_drained(
|
||||||
|
redis_url: str,
|
||||||
|
*,
|
||||||
|
timeout_seconds: int,
|
||||||
|
) -> RedisBrokerState:
|
||||||
|
deadline = time.monotonic() + timeout_seconds
|
||||||
|
last = RedisBrokerState(0, 0, 0)
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
last = _broker_state(redis_url)
|
||||||
|
if last == RedisBrokerState(0, 0, 0):
|
||||||
|
return last
|
||||||
|
time.sleep(0.2)
|
||||||
|
raise AcceptanceError("Redis broker retained delivery state after recovery")
|
||||||
|
|
||||||
|
|
||||||
|
def _queue_evidence(payload: Mapping[str, Any]) -> dict[str, Any]:
|
||||||
|
expected = {
|
||||||
|
"queued_count": 1,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"blocked_count": 0,
|
||||||
|
"enqueued_count": 1,
|
||||||
|
"delivery_mode": "worker_queue",
|
||||||
|
"worker_queue_available": True,
|
||||||
|
"dry_run": False,
|
||||||
|
}
|
||||||
|
evidence = {key: payload.get(key) for key in expected}
|
||||||
|
if evidence != expected:
|
||||||
|
raise AcceptanceError("Campaign was not durably queued to one Celery task")
|
||||||
|
return evidence
|
||||||
|
|
||||||
|
|
||||||
|
def execute_redelivery_scenario(
|
||||||
|
client: Any,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
*,
|
||||||
|
fixture_path: Path,
|
||||||
|
settings: TestbedSettings,
|
||||||
|
endpoint: Any,
|
||||||
|
redis_url: str,
|
||||||
|
runtime_root: Path,
|
||||||
|
snapshot_probe: Callable[[str], tuple[Mapping[str, Any], Mapping[str, Any]]],
|
||||||
|
audit_probe: Callable[[str, str], Mapping[str, int]],
|
||||||
|
delivery_probe: Callable[[str, str], Mapping[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
profile_id = create_mail_profile(
|
||||||
|
client,
|
||||||
|
headers,
|
||||||
|
settings,
|
||||||
|
name="Campaign Redis Celery redelivery drill",
|
||||||
|
smtp_host=endpoint.host,
|
||||||
|
smtp_port=endpoint.port,
|
||||||
|
)
|
||||||
|
prepared = prepare_campaign_scenario(
|
||||||
|
client,
|
||||||
|
headers,
|
||||||
|
fixture_path=fixture_path,
|
||||||
|
profile_id=profile_id,
|
||||||
|
settings=settings,
|
||||||
|
scenario="celery_broker_redelivery",
|
||||||
|
snapshot_probe=snapshot_probe,
|
||||||
|
)
|
||||||
|
|
||||||
|
first_worker = _start_worker(runtime_root, label="first-worker")
|
||||||
|
replacement_worker: WorkerProcess | None = None
|
||||||
|
try:
|
||||||
|
_wait_for_worker_ready(
|
||||||
|
first_worker,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
queued = _expect(
|
||||||
|
client.post(
|
||||||
|
f"/api/v1/campaigns/{prepared.campaign_id}/queue",
|
||||||
|
headers=dict(headers),
|
||||||
|
json={
|
||||||
|
"version_id": prepared.version_id,
|
||||||
|
"include_warnings": True,
|
||||||
|
"enqueue_celery": True,
|
||||||
|
"dry_run": False,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
200,
|
||||||
|
"Celery-redelivery Campaign queue",
|
||||||
|
)
|
||||||
|
queue_evidence = _queue_evidence(queued)
|
||||||
|
first_task_id = _wait_for_received_task(
|
||||||
|
first_worker,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
if not endpoint.wait_for_data(settings.provider_timeout_seconds):
|
||||||
|
raise AcceptanceError("Celery worker did not reach complete SMTP DATA")
|
||||||
|
first_exit_code = _kill_worker(
|
||||||
|
first_worker,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
endpoint.release_held_connection()
|
||||||
|
|
||||||
|
interrupted_state = _durable_state_evidence(
|
||||||
|
delivery_probe(prepared.campaign_id, prepared.version_id)
|
||||||
|
)
|
||||||
|
expected_interrupted = {
|
||||||
|
"job_count": 1,
|
||||||
|
"send_status_counts": {"sending": 1},
|
||||||
|
"attempt_status_counts": {"smtp_in_progress": 1},
|
||||||
|
"unfinished_attempt_count": 1,
|
||||||
|
}
|
||||||
|
if interrupted_state != expected_interrupted:
|
||||||
|
raise AcceptanceError("Killed worker state was not durably SMTP-in-progress")
|
||||||
|
|
||||||
|
replacement_worker = _start_worker(runtime_root, label="replacement-worker")
|
||||||
|
_wait_for_worker_ready(
|
||||||
|
replacement_worker,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
redelivered_task_id = _wait_for_received_task(
|
||||||
|
replacement_worker,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
expected_task_id=first_task_id,
|
||||||
|
)
|
||||||
|
_wait_for_task_success(
|
||||||
|
replacement_worker,
|
||||||
|
task_id=redelivered_task_id,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
recovered_state = _durable_state_evidence(
|
||||||
|
delivery_probe(prepared.campaign_id, prepared.version_id)
|
||||||
|
)
|
||||||
|
expected_recovered = {
|
||||||
|
"job_count": 1,
|
||||||
|
"send_status_counts": {"outcome_unknown": 1},
|
||||||
|
"attempt_status_counts": {"outcome_unknown": 1},
|
||||||
|
"unfinished_attempt_count": 0,
|
||||||
|
}
|
||||||
|
if recovered_state != expected_recovered:
|
||||||
|
raise AcceptanceError("Redelivered task did not freeze the unfinished attempt")
|
||||||
|
|
||||||
|
protocol = endpoint.evidence()
|
||||||
|
expected_protocol = {
|
||||||
|
"connection_count": 1,
|
||||||
|
"accepted_rcpt_commands": 1,
|
||||||
|
"refused_rcpt_commands": 0,
|
||||||
|
"data_transactions": 1,
|
||||||
|
}
|
||||||
|
if protocol != expected_protocol:
|
||||||
|
raise AcceptanceError("Broker redelivery caused an unexpected SMTP transaction")
|
||||||
|
broker_after = _wait_for_broker_drained(
|
||||||
|
redis_url,
|
||||||
|
timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
first_received = first_worker.received_task_ids()
|
||||||
|
replacement_received = replacement_worker.received_task_ids()
|
||||||
|
if first_received != (first_task_id,) or replacement_received != (
|
||||||
|
redelivered_task_id,
|
||||||
|
):
|
||||||
|
raise AcceptanceError(
|
||||||
|
"Celery workers did not each receive the broker task exactly once"
|
||||||
|
)
|
||||||
|
report = _report_evidence(
|
||||||
|
_expect(
|
||||||
|
client.get(
|
||||||
|
f"/api/v1/campaigns/{prepared.campaign_id}/report",
|
||||||
|
headers=dict(headers),
|
||||||
|
params={"version_id": prepared.version_id},
|
||||||
|
),
|
||||||
|
200,
|
||||||
|
"Celery-redelivery Campaign report",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if report["send_status_counts"] != {"outcome_unknown": 1}:
|
||||||
|
raise AcceptanceError("Campaign report did not retain outcome_unknown")
|
||||||
|
audit_actions = dict(
|
||||||
|
sorted(audit_probe(prepared.campaign_id, prepared.version_id).items())
|
||||||
|
)
|
||||||
|
if not {
|
||||||
|
"campaign.created",
|
||||||
|
"campaign.validated",
|
||||||
|
"campaign.messages_built",
|
||||||
|
"campaign.queued",
|
||||||
|
}.issubset(audit_actions):
|
||||||
|
raise AcceptanceError("Celery-redelivery Campaign audit evidence is incomplete")
|
||||||
|
|
||||||
|
return {
|
||||||
|
**prepared.public_evidence(),
|
||||||
|
"queue": queue_evidence,
|
||||||
|
"interrupted_durable_state": interrupted_state,
|
||||||
|
"recovered_durable_state": recovered_state,
|
||||||
|
"protocol": protocol,
|
||||||
|
"report": report,
|
||||||
|
"audit_actions": audit_actions,
|
||||||
|
"broker": {
|
||||||
|
"transport": "redis",
|
||||||
|
"same_task_identity_redelivered": first_task_id
|
||||||
|
== redelivered_task_id,
|
||||||
|
"first_worker_received_count": len(first_received),
|
||||||
|
"replacement_worker_received_count": len(replacement_received),
|
||||||
|
**broker_after.as_dict(),
|
||||||
|
},
|
||||||
|
"supervision": {
|
||||||
|
"first_worker_killed_after_complete_data": True,
|
||||||
|
"first_worker_forced_exit": first_exit_code != 0,
|
||||||
|
"replacement_worker_started": True,
|
||||||
|
"replacement_worker_completed_redelivery": True,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
endpoint.release_held_connection()
|
||||||
|
_stop_worker(first_worker, timeout_seconds=5)
|
||||||
|
if replacement_worker is not None:
|
||||||
|
_stop_worker(replacement_worker, timeout_seconds=5)
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime_module_versions(registry: Any) -> dict[str, str]:
|
||||||
|
versions = {"core": _core_package_version()}
|
||||||
|
versions.update(
|
||||||
|
{
|
||||||
|
manifest.id: manifest.version
|
||||||
|
for manifest in registry.manifests()
|
||||||
|
if manifest.id in {"access", "audit", "campaigns", "mail"}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return versions
|
||||||
|
|
||||||
|
|
||||||
|
def _create_runtime_root() -> Path:
|
||||||
|
"""Create the isolated runtime under the platform-selected temp root."""
|
||||||
|
|
||||||
|
return Path(tempfile.mkdtemp(prefix="govoplan-campaign-celery-redelivery-"))
|
||||||
|
|
||||||
|
|
||||||
|
def _bootstrap_and_run(
|
||||||
|
*,
|
||||||
|
settings: TestbedSettings,
|
||||||
|
fixture_path: Path,
|
||||||
|
redis_url: str,
|
||||||
|
visibility_timeout_seconds: int,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
runtime_root = _create_runtime_root()
|
||||||
|
database = None
|
||||||
|
try:
|
||||||
|
os.environ.update(
|
||||||
|
{
|
||||||
|
"APP_ENV": "test",
|
||||||
|
"DATABASE_URL": f"sqlite:///{runtime_root / 'acceptance.db'}",
|
||||||
|
"FILE_STORAGE_BACKEND": "local",
|
||||||
|
"FILE_STORAGE_LOCAL_ROOT": str(runtime_root / "files"),
|
||||||
|
"MOCK_MAILBOX_DIR": str(runtime_root / "mock-mailbox"),
|
||||||
|
"DEV_BOOTSTRAP_ENABLED": "false",
|
||||||
|
"CELERY_ENABLED": "true",
|
||||||
|
"REDIS_URL": redis_url,
|
||||||
|
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS": str(
|
||||||
|
visibility_timeout_seconds
|
||||||
|
),
|
||||||
|
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from govoplan_core.db.base import Base
|
||||||
|
from govoplan_core.db.bootstrap import bootstrap_dev_data
|
||||||
|
from govoplan_core.db.session import configure_database, set_database
|
||||||
|
from govoplan_core.settings import Settings, settings as core_settings
|
||||||
|
from govoplan_core.tenancy.scope import create_scope_tables
|
||||||
|
|
||||||
|
isolated_settings = Settings()
|
||||||
|
for field_name in Settings.model_fields:
|
||||||
|
setattr(core_settings, field_name, getattr(isolated_settings, field_name))
|
||||||
|
database = configure_database(os.environ["DATABASE_URL"])
|
||||||
|
set_database(database)
|
||||||
|
|
||||||
|
from govoplan_core.server.app import app
|
||||||
|
|
||||||
|
create_scope_tables(database.engine)
|
||||||
|
Base.metadata.create_all(bind=database.engine)
|
||||||
|
with database.SessionLocal() as session:
|
||||||
|
bootstrap_dev_data(
|
||||||
|
session,
|
||||||
|
api_key_secret="celery-redelivery-unused-api-key",
|
||||||
|
user_password="celery-redelivery-admin",
|
||||||
|
)
|
||||||
|
|
||||||
|
def snapshot_probe(
|
||||||
|
version_id: str,
|
||||||
|
) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
|
||||||
|
from govoplan_campaign.backend.db.models import CampaignVersion
|
||||||
|
|
||||||
|
with database.SessionLocal() as session:
|
||||||
|
version = session.get(CampaignVersion, version_id)
|
||||||
|
if version is None:
|
||||||
|
raise AcceptanceError("Campaign execution snapshot is unavailable")
|
||||||
|
raw = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
snapshot = (
|
||||||
|
version.execution_snapshot
|
||||||
|
if isinstance(version.execution_snapshot, dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
return raw, snapshot
|
||||||
|
|
||||||
|
def audit_probe(campaign_id: str, version_id: str) -> Mapping[str, int]:
|
||||||
|
from govoplan_audit.backend.db.models import AuditLog
|
||||||
|
|
||||||
|
with database.SessionLocal() as session:
|
||||||
|
actions = [
|
||||||
|
row[0]
|
||||||
|
for row in session.query(AuditLog.action)
|
||||||
|
.filter(AuditLog.object_id.in_([campaign_id, version_id]))
|
||||||
|
.all()
|
||||||
|
if row[0] in EXPECTED_AUDIT_ACTIONS
|
||||||
|
]
|
||||||
|
return dict(Counter(actions))
|
||||||
|
|
||||||
|
def delivery_probe(campaign_id: str, version_id: str) -> Mapping[str, Any]:
|
||||||
|
from govoplan_campaign.backend.db.models import CampaignJob, SendAttempt
|
||||||
|
|
||||||
|
with database.SessionLocal() as session:
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.campaign_version_id == version_id,
|
||||||
|
)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
job_ids = [job.id for job in jobs]
|
||||||
|
attempts = (
|
||||||
|
session.query(SendAttempt)
|
||||||
|
.filter(SendAttempt.job_id.in_(job_ids))
|
||||||
|
.all()
|
||||||
|
if job_ids
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"job_count": len(jobs),
|
||||||
|
"send_status_counts": dict(
|
||||||
|
Counter(job.send_status for job in jobs)
|
||||||
|
),
|
||||||
|
"attempt_status_counts": dict(
|
||||||
|
Counter(attempt.status for attempt in attempts)
|
||||||
|
),
|
||||||
|
"unfinished_attempt_count": sum(
|
||||||
|
1 for attempt in attempts if attempt.finished_at is None
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
login = _expect(
|
||||||
|
client.post(
|
||||||
|
"/api/v1/auth/login",
|
||||||
|
json={
|
||||||
|
"email": "admin@example.local",
|
||||||
|
"password": "celery-redelivery-admin",
|
||||||
|
},
|
||||||
|
),
|
||||||
|
200,
|
||||||
|
"Acceptance login",
|
||||||
|
)
|
||||||
|
access_token = str(login.get("access_token") or "")
|
||||||
|
if not access_token:
|
||||||
|
raise AcceptanceError("Acceptance login returned no access token")
|
||||||
|
from govoplan_core.core.runtime import get_registry
|
||||||
|
|
||||||
|
registry = get_registry()
|
||||||
|
if registry is None:
|
||||||
|
raise AcceptanceError("The GovOPlaN module registry is unavailable")
|
||||||
|
composition_versions = required_composition_versions(
|
||||||
|
fixture_path,
|
||||||
|
_runtime_module_versions(registry),
|
||||||
|
)
|
||||||
|
with smtp_fault_endpoint("post_data_hold") as endpoint:
|
||||||
|
scenario = execute_redelivery_scenario(
|
||||||
|
client,
|
||||||
|
{"Authorization": f"Bearer {access_token}"},
|
||||||
|
fixture_path=fixture_path,
|
||||||
|
settings=settings,
|
||||||
|
endpoint=endpoint,
|
||||||
|
redis_url=redis_url,
|
||||||
|
runtime_root=runtime_root,
|
||||||
|
snapshot_probe=snapshot_probe,
|
||||||
|
audit_probe=audit_probe,
|
||||||
|
delivery_probe=delivery_probe,
|
||||||
|
)
|
||||||
|
|
||||||
|
evidence = {
|
||||||
|
"schema_version": EVIDENCE_SCHEMA,
|
||||||
|
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"fixture_sha256": hashlib.sha256(fixture_path.read_bytes()).hexdigest(),
|
||||||
|
"declared_module_versions": composition_versions,
|
||||||
|
"target": {
|
||||||
|
"kind": "local_redis_celery_controlled_smtp",
|
||||||
|
"isolated_temporary_database": True,
|
||||||
|
"redis_started_by_runner": True,
|
||||||
|
"worker_pool": "solo",
|
||||||
|
"worker_prefetch_multiplier": 1,
|
||||||
|
"task_acks_late": True,
|
||||||
|
"task_reject_on_worker_lost": True,
|
||||||
|
"visibility_timeout_seconds": visibility_timeout_seconds,
|
||||||
|
},
|
||||||
|
"scenario": scenario,
|
||||||
|
"coverage": {
|
||||||
|
"redis_broker_delivery": True,
|
||||||
|
"celery_worker_processes": True,
|
||||||
|
"forced_worker_loss_after_complete_data": True,
|
||||||
|
"same_task_broker_redelivery": True,
|
||||||
|
"durable_outcome_unknown_recovery": True,
|
||||||
|
"duplicate_smtp_transaction_prevented": True,
|
||||||
|
"production_daemon_supervisor": False,
|
||||||
|
"target_provider": False,
|
||||||
|
"source_artifact_provenance": False,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
_assert_evidence_safe(evidence, settings=settings)
|
||||||
|
rendered = json.dumps(
|
||||||
|
evidence,
|
||||||
|
ensure_ascii=False,
|
||||||
|
indent=2,
|
||||||
|
sort_keys=True,
|
||||||
|
).encode("utf-8") + b"\n"
|
||||||
|
if len(rendered) > MAX_EVIDENCE_BYTES:
|
||||||
|
raise AcceptanceError("Celery-redelivery evidence exceeds its size limit")
|
||||||
|
return evidence
|
||||||
|
finally:
|
||||||
|
if database is not None:
|
||||||
|
database.engine.dispose()
|
||||||
|
shutil.rmtree(runtime_root, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--fixture", type=Path, default=DEFAULT_FIXTURE)
|
||||||
|
parser.add_argument("--compose-file", type=Path, default=DEFAULT_COMPOSE_FILE)
|
||||||
|
parser.add_argument("--redis-port", type=int)
|
||||||
|
parser.add_argument(
|
||||||
|
"--visibility-timeout-seconds",
|
||||||
|
type=lambda value: _positive_int(value, label="visibility timeout"),
|
||||||
|
default=os.environ.get(
|
||||||
|
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS",
|
||||||
|
"3",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--timeout-seconds",
|
||||||
|
type=lambda value: _positive_int(value, label="timeout"),
|
||||||
|
default=60,
|
||||||
|
)
|
||||||
|
parser.add_argument("--evidence", type=Path)
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
try:
|
||||||
|
settings = TestbedSettings.from_environment()
|
||||||
|
settings.assert_local_testbed()
|
||||||
|
settings = replace(
|
||||||
|
settings,
|
||||||
|
provider_timeout_seconds=args.timeout_seconds,
|
||||||
|
)
|
||||||
|
with isolated_redis_broker(
|
||||||
|
compose_file=args.compose_file.resolve(),
|
||||||
|
timeout_seconds=args.timeout_seconds,
|
||||||
|
requested_port=args.redis_port,
|
||||||
|
) as redis_url:
|
||||||
|
evidence = _bootstrap_and_run(
|
||||||
|
settings=settings,
|
||||||
|
fixture_path=args.fixture.resolve(),
|
||||||
|
redis_url=redis_url,
|
||||||
|
visibility_timeout_seconds=args.visibility_timeout_seconds,
|
||||||
|
)
|
||||||
|
rendered = json.dumps(
|
||||||
|
evidence,
|
||||||
|
ensure_ascii=False,
|
||||||
|
indent=2,
|
||||||
|
sort_keys=True,
|
||||||
|
) + "\n"
|
||||||
|
if args.evidence:
|
||||||
|
args.evidence.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.evidence.write_text(rendered, encoding="utf-8")
|
||||||
|
else:
|
||||||
|
sys.stdout.write(rendered)
|
||||||
|
return 0
|
||||||
|
except AcceptanceError as exc:
|
||||||
|
print(f"Campaign Celery-redelivery acceptance failed: {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
except Exception as exc:
|
||||||
|
print(
|
||||||
|
"Campaign Celery-redelivery acceptance failed unexpectedly "
|
||||||
|
f"({type(exc).__name__}); inspect local service logs.",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
40
docs/ACCESS_EXPLANATION_COVERAGE.md
Normal file
40
docs/ACCESS_EXPLANATION_COVERAGE.md
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
# Campaign Access Explanation Coverage
|
||||||
|
|
||||||
|
Campaign access explanations are resource-specific evidence. They inherit the
|
||||||
|
parent Campaign decision only where the child has no independent grant model,
|
||||||
|
and they must identify that inheritance explicitly.
|
||||||
|
|
||||||
|
## Implemented
|
||||||
|
|
||||||
|
- Campaign
|
||||||
|
- Campaign version
|
||||||
|
- Campaign delivery job / built message
|
||||||
|
- Computed Campaign report, identified by Campaign, version, and report kind
|
||||||
|
|
||||||
|
## Planned Slices
|
||||||
|
|
||||||
|
1. Recipient rows and imported recipient-source snapshots
|
||||||
|
2. Attachment bindings and frozen attachment resolutions
|
||||||
|
3. Validation issues, review decisions, and attachment-policy overrides
|
||||||
|
4. Delivery attempts, IMAP append attempts, Postbox attempts, and
|
||||||
|
reconciliation decisions
|
||||||
|
5. Campaign shares and ownership-transfer records
|
||||||
|
6. Import mapping profiles and import executions
|
||||||
|
7. Reusable Campaign templates and template revisions when the template
|
||||||
|
library becomes persistent
|
||||||
|
8. Export packages and protocol/report artifacts
|
||||||
|
|
||||||
|
Each child explanation must include:
|
||||||
|
|
||||||
|
- the child resource identity and current state;
|
||||||
|
- the parent Campaign and version where applicable;
|
||||||
|
- whether access is inherited, independently granted, or further restricted;
|
||||||
|
- effective owner/share/policy provenance;
|
||||||
|
- missing-module or unavailable-evidence reasons without leaking the hidden
|
||||||
|
object;
|
||||||
|
- a stable resource identifier suitable for audit and support links.
|
||||||
|
|
||||||
|
Delivery attempts, review decisions, reports, and exports can contain more
|
||||||
|
sensitive evidence than the Campaign summary. Their read and diagnostic/export
|
||||||
|
permissions therefore remain independently enforceable even when the parent
|
||||||
|
Campaign is readable.
|
||||||
@@ -5,16 +5,20 @@ been validated, built, reviewed, and locked.
|
|||||||
|
|
||||||
## Operating Modes
|
## Operating Modes
|
||||||
|
|
||||||
- Local direct send: `CELERY_ENABLED=false`. Queueing stores jobs in the DB, and
|
- Local direct send: `CELERY_ENABLED=false`. **Send now** is available only for
|
||||||
small development runs can be processed with "Send queued now".
|
exact built runs within the effective synchronous limit. It preflights the
|
||||||
|
complete batch before the first SMTP effect.
|
||||||
- Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing
|
- Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing
|
||||||
publishes delivery tasks, and the Review & Send page polls summary counters.
|
publishes durable delivery tasks, and Review and send polls their persisted
|
||||||
|
summary counters even after the initiating request has returned.
|
||||||
- Mock send: use only for development review. It does not prove real SMTP/IMAP
|
- Mock send: use only for development review. It does not prove real SMTP/IMAP
|
||||||
credentials or server policy.
|
credentials or server policy.
|
||||||
|
|
||||||
## Before First Live Use
|
## Before First Live Use
|
||||||
|
|
||||||
- Use dedicated non-production SMTP/IMAP credentials.
|
- Use dedicated non-production SMTP/IMAP credentials.
|
||||||
|
- Store and test those credentials in a Mail-module profile. Campaign must
|
||||||
|
contain only the selected `server.mail_profile_id` reference.
|
||||||
- Start the repository test bed in `dev/mail-testbed/` when a local
|
- Start the repository test bed in `dev/mail-testbed/` when a local
|
||||||
production-like SMTP/IMAP server is sufficient.
|
production-like SMTP/IMAP server is sufficient.
|
||||||
- Use a dedicated mailbox/folder for append-to-Sent tests.
|
- Use a dedicated mailbox/folder for append-to-Sent tests.
|
||||||
@@ -24,6 +28,8 @@ been validated, built, reviewed, and locked.
|
|||||||
ZIP before using production recipients.
|
ZIP before using production recipients.
|
||||||
- Keep the report page open during tests; it is the operational source of truth
|
- Keep the report page open during tests; it is the operational source of truth
|
||||||
for attempts, outcomes, and reconciliation.
|
for attempts, outcomes, and reconciliation.
|
||||||
|
- Confirm the effective Send now recipient-job limit. The safe default is 25;
|
||||||
|
use Queue for workers for ordinary batches or any run above that limit.
|
||||||
|
|
||||||
## Deliverability Preflight
|
## Deliverability Preflight
|
||||||
|
|
||||||
@@ -47,10 +53,15 @@ Before the first live send for a sender domain or mail-server profile:
|
|||||||
1. Validate the version with file checks enabled.
|
1. Validate the version with file checks enabled.
|
||||||
2. Build the version and inspect all blocking review items.
|
2. Build the version and inspect all blocking review items.
|
||||||
3. Queue only after the selected version is the intended immutable execution
|
3. Queue only after the selected version is the intended immutable execution
|
||||||
version.
|
version. Select **Queue for workers**, then verify the committed and
|
||||||
4. In local mode, use "Send queued now" for small test runs.
|
published counts.
|
||||||
|
4. Use **Send now** only if the exact eligible count is non-zero and at or below
|
||||||
|
the effective deployment/tenant limit shown on the page.
|
||||||
5. In worker mode, verify queue counters move from queued/claimed/sending to a
|
5. In worker mode, verify queue counters move from queued/claimed/sending to a
|
||||||
terminal SMTP state.
|
terminal SMTP state.
|
||||||
|
6. If a synchronous request is used, keep Review and send open: it polls the
|
||||||
|
durable counters while the request runs. A rejection occurs before SMTP and
|
||||||
|
directs oversized runs to workers.
|
||||||
|
|
||||||
## Outcome Handling
|
## Outcome Handling
|
||||||
|
|
||||||
@@ -64,6 +75,13 @@ Before the first live send for a sender domain or mail-server profile:
|
|||||||
- `claimed` or `sending` that does not progress: treat as a worker interruption.
|
- `claimed` or `sending` that does not progress: treat as a worker interruption.
|
||||||
Re-run worker handling or reconcile if SMTP may already have accepted the
|
Re-run worker handling or reconcile if SMTP may already have accepted the
|
||||||
message.
|
message.
|
||||||
|
- IMAP `appending`: A worker owns the durable append claim. Do not start a
|
||||||
|
second append; if the worker cannot finish, reconcile only after checking the
|
||||||
|
mailbox.
|
||||||
|
- IMAP `outcome_unknown`: Never append automatically. An operator with
|
||||||
|
`campaigns:campaign:reconcile` must record an evidence note and resolve it as
|
||||||
|
`imap_appended` or `imap_not_appended`. Only the latter becomes explicitly
|
||||||
|
retryable.
|
||||||
|
|
||||||
## Reconciliation
|
## Reconciliation
|
||||||
|
|
||||||
@@ -86,9 +104,41 @@ bed where possible:
|
|||||||
- IMAP append failure after SMTP acceptance.
|
- IMAP append failure after SMTP acceptance.
|
||||||
- Worker restart with queued, claimed, and sending jobs.
|
- Worker restart with queued, claimed, and sending jobs.
|
||||||
|
|
||||||
|
For the maintained loopback baseline, run
|
||||||
|
`dev/mail-testbed/run_campaign_acceptance.py`. It proves the public Campaign
|
||||||
|
path for SMTP acceptance, IMAP append, repeat-send blocking, an SMTP connection
|
||||||
|
failure before transmission, an explicit SMTP authentication rejection, an
|
||||||
|
explicit temporary `451` response after DATA, partial RCPT refusal, a
|
||||||
|
connection loss after complete DATA, and an IMAP authentication rejection
|
||||||
|
after SMTP acceptance. Its evidence is an allowlisted classification/count
|
||||||
|
projection; raw provider diagnostics and transport/account identifiers are
|
||||||
|
deliberately excluded.
|
||||||
|
|
||||||
|
The runner also terminates a dedicated OS process executing the registered
|
||||||
|
Campaign send task after complete DATA, then invokes the task in a fresh
|
||||||
|
process. The unfinished durable attempt must become `outcome_unknown` and the
|
||||||
|
endpoint must observe no second connection or DATA transaction. This covers
|
||||||
|
the worker task/process boundary but not a broker or daemon.
|
||||||
|
|
||||||
|
Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` for the maintained
|
||||||
|
Redis/Celery delivery and broker redelivery boundary. It starts an isolated
|
||||||
|
Redis Compose service and real Celery workers, kills the first solo worker after complete DATA while the
|
||||||
|
late-ack task is unacknowledged, and requires the same task identity to reach a
|
||||||
|
replacement worker after Redis visibility recovery. Passing evidence also
|
||||||
|
requires durable `outcome_unknown`, an empty broker queue/unacked set, and
|
||||||
|
exactly one SMTP connection and DATA transaction. Raw worker logs and task,
|
||||||
|
database, endpoint, and credential identifiers are never retained.
|
||||||
|
|
||||||
|
That second runner proves local runner-supervised process replacement, not the
|
||||||
|
production process manager. Repeat the worker-loss drill under the selected
|
||||||
|
systemd, container, Kubernetes, or other production supervisor and the target
|
||||||
|
Redis/SMTP infrastructure before deployment approval.
|
||||||
|
|
||||||
## Reporting Checks
|
## Reporting Checks
|
||||||
|
|
||||||
- Partial delivery must show accepted, failed, and unknown counts separately.
|
- Partial delivery must show accepted, failed, and unknown counts separately.
|
||||||
|
- Excluded messages must show SMTP and IMAP as `skipped`, with skipped counts
|
||||||
|
and filters separate from unattempted or failed delivery.
|
||||||
- Accepted and unknown jobs must not appear in retry selections.
|
- Accepted and unknown jobs must not appear in retry selections.
|
||||||
- Reconciled accepted jobs must remain protected from resend.
|
- Reconciled accepted jobs must remain protected from resend.
|
||||||
- Reconciled not-sent jobs must appear only as explicit retry candidates.
|
- Reconciled not-sent jobs must appear only as explicit retry candidates.
|
||||||
|
|||||||
479
docs/CAMPAIGN_HANDBOOK.md
Normal file
479
docs/CAMPAIGN_HANDBOOK.md
Normal file
@@ -0,0 +1,479 @@
|
|||||||
|
# Campaign Handbook
|
||||||
|
|
||||||
|
## Purpose and status
|
||||||
|
|
||||||
|
This is the canonical, multi-perspective handbook for the Campaign module. It
|
||||||
|
describes the current implementation, the operational contract it relies on,
|
||||||
|
and the remaining work required before Campaign can be presented as GovOPlaN's
|
||||||
|
maintained reference composition.
|
||||||
|
|
||||||
|
Use the section that matches the task at hand:
|
||||||
|
|
||||||
|
| Perspective | Start here |
|
||||||
|
| --- | --- |
|
||||||
|
| Campaign author | [Prepare a campaign](#prepare-a-campaign) |
|
||||||
|
| Reviewer | [Review and complete review](#review-and-complete-review) |
|
||||||
|
| Sender or delivery operator | [Deliver and resolve outcomes](#deliver-and-resolve-outcomes) |
|
||||||
|
| Campaign or tenant administrator | [Administration and policy](#administration-and-policy) |
|
||||||
|
| Platform operator | [Operations and recovery](#operations-and-recovery) |
|
||||||
|
| Integrator or developer | [Composition and integration contracts](#composition-and-integration-contracts) |
|
||||||
|
| Security, privacy, or audit reviewer | [Assurance model](#assurance-model) |
|
||||||
|
| Release reviewer | [Reference-composition acceptance](#reference-composition-acceptance) |
|
||||||
|
|
||||||
|
The shorter task documents remain useful companions:
|
||||||
|
|
||||||
|
- [Campaign delivery runbook](CAMPAIGN_DELIVERY_RUNBOOK.md)
|
||||||
|
- [Mail profile boundary](MAIL_PROFILE_BOUNDARY.md)
|
||||||
|
- [Recipient import guide](RECIPIENT_IMPORT_GUIDE.md)
|
||||||
|
- [Recipient and Addresses boundary](RECIPIENT_ADDRESS_BOUNDARY.md)
|
||||||
|
- [Examples and release checklist](EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md)
|
||||||
|
|
||||||
|
## What Campaign is for
|
||||||
|
|
||||||
|
Campaign turns governed source data into individually built messages and then
|
||||||
|
controls their review, delivery, and evidence. It is intentionally a
|
||||||
|
composition module: it demonstrates how one user journey can use optional Mail,
|
||||||
|
Files, Addresses, and Notifications capabilities alongside Core access/audit
|
||||||
|
infrastructure without copying ownership from those modules. Policy may consume
|
||||||
|
Campaign context through a narrow capability; Campaign does not import Policy.
|
||||||
|
|
||||||
|
Campaign owns:
|
||||||
|
|
||||||
|
- the communication purpose, content, campaign-local fields, and templates;
|
||||||
|
- campaign-local recipient snapshots, exclusions, and personalization;
|
||||||
|
- message and attachment rules for a version;
|
||||||
|
- validation, review, build, queue, and delivery-control state;
|
||||||
|
- the durable jobs and attempts needed to explain delivery outcomes; and
|
||||||
|
- campaign-specific reports, shares, and frozen execution evidence.
|
||||||
|
|
||||||
|
Campaign does not own:
|
||||||
|
|
||||||
|
- SMTP/IMAP profiles, credentials, protocol adapters, or mailbox policy;
|
||||||
|
- long-lived address-book master data, consent lifecycle, or deduplication;
|
||||||
|
- managed file bytes, connector credentials, or external-file provenance;
|
||||||
|
- general-purpose workflow definitions; or
|
||||||
|
- global identity, organization, permission, or retention policy.
|
||||||
|
|
||||||
|
Those boundaries matter in both persistence and UI. A campaign references an
|
||||||
|
authorized Mail profile and managed file versions; it must never become a
|
||||||
|
second secret store, file store, or address directory.
|
||||||
|
|
||||||
|
## Process view
|
||||||
|
|
||||||
|
The supported process is a controlled progression, not a single "send" call:
|
||||||
|
|
||||||
|
```text
|
||||||
|
create/edit
|
||||||
|
-> validate and resolve policy/integrations
|
||||||
|
-> review warnings and blockers
|
||||||
|
-> build exact recipient messages
|
||||||
|
-> complete review and queue
|
||||||
|
-> SMTP attempt per job
|
||||||
|
-> optional IMAP append per accepted job
|
||||||
|
-> report, retry, reconcile, or correct
|
||||||
|
-> archive when no active/uncertain delivery remains
|
||||||
|
```
|
||||||
|
|
||||||
|
Campaign status summarizes the whole campaign. Version workflow state describes
|
||||||
|
the selected immutable/editable version. Each recipient job separately records
|
||||||
|
build, validation, queue, SMTP, and IMAP state. Operators must use the job-level
|
||||||
|
states when deciding whether another external effect is safe.
|
||||||
|
|
||||||
|
Important distinctions:
|
||||||
|
|
||||||
|
- **Validation lock** is the reversible lock created by a successful
|
||||||
|
validation/build path. Editing requires unlocking and invalidates derived
|
||||||
|
evidence as appropriate.
|
||||||
|
- **User lock** is an explicit audit-safe lock. A permanently locked or
|
||||||
|
delivery-final version is not edited in place; create an editable successor.
|
||||||
|
- **SMTP accepted** means the provider accepted the message. It does not prove
|
||||||
|
inbox delivery, reading, or business acknowledgement.
|
||||||
|
- **Outcome unknown** means an attempt may have had an external effect. It must
|
||||||
|
be investigated and reconciled before retry.
|
||||||
|
- **IMAP append** is a separate effect after SMTP acceptance. An append failure
|
||||||
|
is not evidence that sending failed.
|
||||||
|
- **Archive** preserves evidence. Draft-only campaigns without built, locked,
|
||||||
|
or delivery evidence may be deleted where policy allows; evidence-bearing
|
||||||
|
campaigns are archived instead.
|
||||||
|
|
||||||
|
## User tasks
|
||||||
|
|
||||||
|
### Prepare a campaign
|
||||||
|
|
||||||
|
1. Create a campaign and confirm its owner or owning group.
|
||||||
|
2. Define global settings, fields, templates, and recipient data.
|
||||||
|
3. Import one-off recipient data or select a reusable Addresses source when the
|
||||||
|
optional capability is installed. Review source provenance and stale-source
|
||||||
|
warnings; Campaign freezes the selected rows rather than following later
|
||||||
|
directory changes silently.
|
||||||
|
4. Select managed attachments through Files. Server/API campaigns do not accept
|
||||||
|
arbitrary local paths. Preview rules and unmatched files before building.
|
||||||
|
5. Open **Mail settings** and select an available Mail profile. The campaign
|
||||||
|
stores only `server.mail_profile_id`; it never accepts SMTP/IMAP settings,
|
||||||
|
usernames, passwords, or credential references.
|
||||||
|
6. Save the editable version, validate the relevant sections, and resolve every
|
||||||
|
blocking issue. Warnings remain explicit review decisions.
|
||||||
|
7. Build the exact messages and inspect recipient, addressing, template,
|
||||||
|
attachment, and generated-message evidence.
|
||||||
|
|
||||||
|
If a selected optional module is absent, Campaign remains loadable and explains
|
||||||
|
which function is unavailable. It must not fail startup because Mail, Files, or
|
||||||
|
Addresses is not installed.
|
||||||
|
|
||||||
|
### Review and complete review
|
||||||
|
|
||||||
|
The reviewer should verify the immutable candidate that will be delivered, not
|
||||||
|
just the authoring form:
|
||||||
|
|
||||||
|
1. Confirm purpose, owner, selected version, and recipient count.
|
||||||
|
2. Inspect blocking errors, warnings, exclusions, and recipients requiring
|
||||||
|
review.
|
||||||
|
3. Inspect representative and exceptional rendered messages, including From,
|
||||||
|
To/CC/BCC, Reply-To, subject, body, and attachment evidence.
|
||||||
|
4. Confirm the selected Mail profile is authorized for the campaign's current
|
||||||
|
tenant and owner context.
|
||||||
|
5. Confirm attachment behavior when a rule matches no files, ZIP/password
|
||||||
|
behavior, and any recipient-specific files.
|
||||||
|
6. Record review completion and the inspected message keys through the review
|
||||||
|
surface. The current baseline does not persist a distinct approve/reject
|
||||||
|
decision or review reason. If content, recipients, attachment inputs, owner
|
||||||
|
context, or non-secret transport identity changes, revalidate and rebuild.
|
||||||
|
|
||||||
|
Normal readers and reviewers see business state and safe evidence. Process-local
|
||||||
|
paths, storage keys, worker claim tokens, and raw provider diagnostics require
|
||||||
|
the dedicated diagnostic permission and must not leak through ordinary campaign,
|
||||||
|
version, job, or report responses.
|
||||||
|
|
||||||
|
Campaign now provides a separate aggregate **Reports** surface for readers with
|
||||||
|
`campaigns:report:read` and access to the campaign. It loads only the safe
|
||||||
|
aggregate projections, applies small-cell suppression, and offers no recipient
|
||||||
|
rows, drill-down, filtering, export, or delivery actions. The recipient-aware
|
||||||
|
**Campaign Report** still requires recipient-read access and does not yet hide
|
||||||
|
every action control that the actor lacks. The server authorizes each action,
|
||||||
|
but permission-aware action visibility on that detailed surface remains open
|
||||||
|
work; do not confuse it with the aggregate reader experience.
|
||||||
|
|
||||||
|
### Deliver and resolve outcomes
|
||||||
|
|
||||||
|
Use the [delivery runbook](CAMPAIGN_DELIVERY_RUNBOOK.md) for the detailed
|
||||||
|
operator sequence.
|
||||||
|
|
||||||
|
At a minimum:
|
||||||
|
|
||||||
|
1. Queue only a validated, locked, built version. Use **Queue for workers** for
|
||||||
|
ordinary batches; the durable progress remains visible after leaving and
|
||||||
|
returning to Review and send.
|
||||||
|
2. Use **Send now** only when the exact persisted eligible build is within the
|
||||||
|
effective synchronous limit shown by the UI. The default deployment limit
|
||||||
|
is 25 recipient jobs. The backend repeats the count and preflights every
|
||||||
|
message and the Mail profile revision before contacting SMTP.
|
||||||
|
3. Treat `smtp_accepted` as protected from ordinary retry.
|
||||||
|
4. Retry `failed_temporary` explicitly after inspecting the cause.
|
||||||
|
5. Include `failed_permanent` only after correcting the cause and making a
|
||||||
|
conscious override.
|
||||||
|
6. Never retry `outcome_unknown` blindly. Inspect SMTP/provider evidence and
|
||||||
|
reconcile it as **accepted** or **not sent**, with a note identifying the
|
||||||
|
evidence.
|
||||||
|
7. Process append-to-Sent only for SMTP-accepted jobs and investigate append
|
||||||
|
failure independently. Never retry an `outcome_unknown` IMAP append blindly:
|
||||||
|
reconcile mailbox evidence as **appended** or **not appended** with a note.
|
||||||
|
Only the latter becomes explicitly retryable, and neither decision resends
|
||||||
|
the already SMTP-accepted message.
|
||||||
|
8. Archive only after active and uncertain effects are resolved.
|
||||||
|
|
||||||
|
Pause stops new eligible work but cannot undo a provider effect already in
|
||||||
|
progress. Cancel marks work that has not yet produced a protected SMTP outcome;
|
||||||
|
it cannot recall accepted mail.
|
||||||
|
|
||||||
|
The deployment ceiling is configured with
|
||||||
|
`GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0 disables Send now; the
|
||||||
|
accepted range is 0–500). A tenant may only narrow that ceiling with
|
||||||
|
`tenant.settings.campaign_delivery_policy.synchronous_send_max_recipients`.
|
||||||
|
The effective value and source are returned by the protected delivery-options
|
||||||
|
API, recorded for successful/rejected synchronous commands, and stated in the
|
||||||
|
configured handbook topic.
|
||||||
|
|
||||||
|
### Test and one-message actions
|
||||||
|
|
||||||
|
The current baseline includes mock send, queue dry-run, synchronous immediate
|
||||||
|
delivery, sending one selected job, retry selection, and unattempted-job
|
||||||
|
selection. Their audit and state behavior is not yet the final vocabulary
|
||||||
|
accepted for issue `govoplan-campaign#69`.
|
||||||
|
|
||||||
|
The intended distinction is:
|
||||||
|
|
||||||
|
- **Test:** deliver once to the configured test path, audit it, and leave the
|
||||||
|
production job unsent.
|
||||||
|
- **Single send:** send one unsent job, audit it, and mark its production effect
|
||||||
|
complete.
|
||||||
|
- **Single resend:** intentionally send one job again regardless of an earlier
|
||||||
|
failure or acceptance, with distinct authority, warning, reason, and audit.
|
||||||
|
|
||||||
|
Until that slice is implemented and target-tested, do not present the existing
|
||||||
|
buttons as a complete resend policy. Ordinary retry protection remains the safe
|
||||||
|
default.
|
||||||
|
|
||||||
|
## Data and evidence model
|
||||||
|
|
||||||
|
### Versions and snapshots
|
||||||
|
|
||||||
|
Editable campaign JSON is versioned. Build creates recipient jobs and an
|
||||||
|
execution snapshot. A new profile-only snapshot contains the stable Mail
|
||||||
|
profile id, delivery policy/evidence, and opaque Mail-issued transport
|
||||||
|
revisions. It contains no resolved SMTP/IMAP configuration or credentials.
|
||||||
|
|
||||||
|
At delivery time Mail re-authorizes the profile, compares the expected opaque
|
||||||
|
revisions, resolves credentials inside the same Mail-owned operation, and
|
||||||
|
performs the transport effect. Campaign receives only the sanitized result it
|
||||||
|
needs for job state and evidence. This same-call check prevents a
|
||||||
|
validate-then-use race across the module boundary.
|
||||||
|
|
||||||
|
Credential rotation that does not change the non-secret transport identity may
|
||||||
|
continue to satisfy the snapshot. A host, account identity, protocol, sender,
|
||||||
|
or other revisioned transport change stops delivery until the campaign is
|
||||||
|
revalidated and rebuilt.
|
||||||
|
|
||||||
|
### Existing legacy database records
|
||||||
|
|
||||||
|
The current Campaign database may contain versions with inline SMTP/IMAP
|
||||||
|
material. No separate historical Campaign JSON corpus exists. Inline transport
|
||||||
|
material in those rows is treated as inert legacy data and is never interpreted
|
||||||
|
as an executable Mail configuration:
|
||||||
|
|
||||||
|
- public responses remove legacy transport fields and secrets;
|
||||||
|
- validation, build, queue, retry, and delivery fail closed;
|
||||||
|
- an editable version changes to profile-only form only through an explicit
|
||||||
|
Mail-settings save; and
|
||||||
|
- a locked version is preserved and must be forked to an editable successor.
|
||||||
|
|
||||||
|
Normal database backup/restore and access controls cover those rows together
|
||||||
|
with the rest of the current database. There is no separate historical-JSON,
|
||||||
|
backup-scanning, or inline-secret migration program. Restoring an existing
|
||||||
|
legacy row preserves it as inert evidence and does not make it deliverable.
|
||||||
|
|
||||||
|
### Recipient and attachment evidence
|
||||||
|
|
||||||
|
The delivery record should be able to identify:
|
||||||
|
|
||||||
|
- source/import context and the frozen recipient row;
|
||||||
|
- effective addressing and message id;
|
||||||
|
- template inputs and unresolved-placeholder decisions;
|
||||||
|
- managed file/version ids, source provenance, checksums, and ZIP evidence;
|
||||||
|
- generated EML checksum and size;
|
||||||
|
- Mail profile reference and opaque transport revisions;
|
||||||
|
- each SMTP and IMAP attempt, its classification, safe provider response, and
|
||||||
|
reconciliation note; and
|
||||||
|
- actor/system trigger, timestamps, policy context, and corrections.
|
||||||
|
|
||||||
|
An excluded recipient/message is a completed validation decision, not a
|
||||||
|
pending delivery. Its SMTP and IMAP states are both `skipped`; it is counted and
|
||||||
|
filterable separately from unattempted, failed, accepted, and append outcomes.
|
||||||
|
No SMTP or IMAP attempt exists for such a row. If historical data contains
|
||||||
|
actual transport evidence despite an exclusion marker, that evidence is
|
||||||
|
preserved for audit and reconciliation rather than relabelled.
|
||||||
|
|
||||||
|
## Administration and policy
|
||||||
|
|
||||||
|
### Roles and permissions
|
||||||
|
|
||||||
|
The supplied role templates deliberately separate preparation, review, and
|
||||||
|
delivery:
|
||||||
|
|
||||||
|
- **Campaign manager** prepares, validates, and builds campaigns and recipients.
|
||||||
|
- **Campaign reviewer** inspects prepared material and records review
|
||||||
|
completion for the exact messages checked.
|
||||||
|
- **Campaign sender** queues, sends, pauses/resumes/cancels, retries, reconciles,
|
||||||
|
reads reports, and has diagnostic access.
|
||||||
|
|
||||||
|
Administrators may compose narrower roles from the declared permissions. Keep
|
||||||
|
these separations where institutional policy requires four-eyes approval. Mail
|
||||||
|
profile use additionally requires `mail:profile:use`; profile and credential
|
||||||
|
administration remains a Mail permission.
|
||||||
|
|
||||||
|
Campaign access is also constrained by tenant, owner/group context, and explicit
|
||||||
|
shares. A share grants only its declared campaign permission; it does not grant
|
||||||
|
Mail credentials, Files administration, or tenant-wide recipient access.
|
||||||
|
|
||||||
|
### Configuration checklist
|
||||||
|
|
||||||
|
- Install compatible Core and Campaign versions. Access and base audit are Core
|
||||||
|
infrastructure; install optional Mail, Files, Addresses, Notifications, and
|
||||||
|
Policy modules only where the configured journey requires them.
|
||||||
|
- Configure Mail profiles and policy in Mail, not in campaign fixtures or JSON.
|
||||||
|
- Configure Files storage/connectors and attachment permissions in Files.
|
||||||
|
- Define role assignments and separation-of-duty policy.
|
||||||
|
- Configure Redis/Celery for durable batch workers where production volume
|
||||||
|
requires it. Local execution is a development/small-run mode, not horizontal
|
||||||
|
worker coordination.
|
||||||
|
- Set rate limits for the target provider. Mail may use Redis for shared
|
||||||
|
throttling when present and a process-local fallback in development.
|
||||||
|
- Establish retention, deletion, archive, report-export, and diagnostic-access
|
||||||
|
policy before production recipient data is loaded.
|
||||||
|
- Use non-production SMTP/IMAP identities and the maintained examples before
|
||||||
|
allowing a production profile.
|
||||||
|
|
||||||
|
### Owner transfer
|
||||||
|
|
||||||
|
Mail profile visibility can depend on campaign owner or group. Transferring an
|
||||||
|
editable campaign with a selected profile clears/requires reselection and
|
||||||
|
revalidation. A locked or delivery-final version is never silently rewritten;
|
||||||
|
create an editable successor.
|
||||||
|
|
||||||
|
## Operations and recovery
|
||||||
|
|
||||||
|
### Health to observe
|
||||||
|
|
||||||
|
- database and migration health;
|
||||||
|
- compatible module interface versions;
|
||||||
|
- queue publication, worker heartbeat, claim age, and backlog;
|
||||||
|
- SMTP/IMAP profile test result and deployment egress policy;
|
||||||
|
- Mail profile authorization/transport-revision mismatch;
|
||||||
|
- Files resolution and frozen attachment availability;
|
||||||
|
- counts by queue, SMTP, IMAP, and reconciliation state; and
|
||||||
|
- audit and report generation failures.
|
||||||
|
|
||||||
|
### Worker interruption
|
||||||
|
|
||||||
|
A crashed worker can leave a job claimed or sending. Do not infer non-delivery
|
||||||
|
from worker loss. If the SMTP boundary may have been crossed, classify the
|
||||||
|
outcome as unknown and reconcile from external evidence. Only work that is
|
||||||
|
provably unattempted may be returned to an ordinary queue.
|
||||||
|
|
||||||
|
### Retry and reconciliation
|
||||||
|
|
||||||
|
Retries create new attempt evidence; they do not overwrite the previous
|
||||||
|
attempt. Reconciliation is a privileged factual correction supported by an
|
||||||
|
operator note. Repeated automated requests should be idempotent for the same
|
||||||
|
eligible job state; a deliberate resend is a different, not-yet-finalized
|
||||||
|
business action and must never be disguised as a retry.
|
||||||
|
|
||||||
|
### Backups and restoration
|
||||||
|
|
||||||
|
Back up Campaign, Mail, Files, Core/Audit, and shared storage consistently for
|
||||||
|
the composition. A database restore without generated EML/file storage, or file
|
||||||
|
storage without matching metadata, does not reconstruct the evidence chain.
|
||||||
|
After restore, keep outbound delivery paused until queue/attempt state and
|
||||||
|
provider evidence have been reconciled; never let restored accepted jobs send
|
||||||
|
again merely because a queue message was lost.
|
||||||
|
|
||||||
|
### Incident handling
|
||||||
|
|
||||||
|
1. Pause new delivery when duplicate or unknown effects are possible.
|
||||||
|
2. Preserve database, queue, provider, worker, and audit evidence.
|
||||||
|
3. Scope affected campaigns/jobs without exposing recipient content broadly.
|
||||||
|
4. Reconcile uncertain jobs individually or through an approved bounded tool.
|
||||||
|
5. Correct configuration in its owning module, then revalidate/rebuild where
|
||||||
|
revisioned transport inputs changed.
|
||||||
|
6. Record the incident reference and recovery rationale in audit evidence.
|
||||||
|
|
||||||
|
## Composition and integration contracts
|
||||||
|
|
||||||
|
Campaign has one required platform dependency: Core. Optional module behavior
|
||||||
|
is discovered through versioned, Core-mediated capabilities; Campaign must not
|
||||||
|
import optional sibling ORM, services, or WebUI implementation.
|
||||||
|
|
||||||
|
Current principal contracts include:
|
||||||
|
|
||||||
|
| Contract | Direction | Purpose |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `mail.campaign_delivery` 0.2.x | Mail -> Campaign | Summarize a known reference; authorize and revision-gate it; send/append using Mail-owned configuration and credentials; return sanitized results |
|
||||||
|
| `files.campaign_attachments` 0.1.x | Files -> Campaign | Select/materialize governed file versions and preserve campaign usage/evidence |
|
||||||
|
| `addresses.lookup` 0.1.x | Addresses -> Campaign | Optional address suggestions |
|
||||||
|
| `addresses.recipient_source` 0.1.x | Addresses -> Campaign | Optional versioned recipient-source snapshots |
|
||||||
|
| `campaigns.access` 0.1.x | Campaign -> platform | Explain campaign access/existence without exporting ORM objects |
|
||||||
|
| `campaigns.mail_policy_context` 0.1.x | Campaign -> Mail | Resolve campaign tenant/owner context for Mail policy |
|
||||||
|
| `campaigns.delivery_tasks` 0.1.x | Campaign -> workers | Execute narrow queued send/append tasks |
|
||||||
|
| `campaigns.retention` 0.1.x | Campaign -> retention | Apply Campaign-owned retention behavior |
|
||||||
|
|
||||||
|
Breaking payload or ownership changes require an interface-version bump and a
|
||||||
|
release-composition alignment gate. Optional absence must be tested physically,
|
||||||
|
not only hidden in navigation.
|
||||||
|
|
||||||
|
### External API expectations
|
||||||
|
|
||||||
|
- Tenant and campaign access are evaluated for every operation.
|
||||||
|
- Writes require CSRF/auth behavior supplied by Core and the specific declared
|
||||||
|
scope.
|
||||||
|
- Queue/retry/reconcile endpoints operate on persisted state and return safe
|
||||||
|
summaries; initiating HTTP success is not proof of external delivery.
|
||||||
|
- Delta endpoints are optimization surfaces, not a separate source of truth.
|
||||||
|
- Report exports contain permitted business/evidence fields but no credentials,
|
||||||
|
local paths, storage keys, or worker claims.
|
||||||
|
|
||||||
|
## Assurance model
|
||||||
|
|
||||||
|
### Security invariants
|
||||||
|
|
||||||
|
- No new campaign payload, fixture, response, or execution snapshot contains
|
||||||
|
SMTP/IMAP settings or credentials.
|
||||||
|
- Mail resolves credentials and performs transports inside Mail-owned calls.
|
||||||
|
- Every real connector peer is validated and pinned at connection time under
|
||||||
|
deployment-wide private-network policy.
|
||||||
|
- API/server attachment paths use managed Files references; arbitrary and
|
||||||
|
traversal-capable local paths are rejected.
|
||||||
|
- Public responses recursively remove infrastructure locators and secret-like
|
||||||
|
legacy fields.
|
||||||
|
- Accepted and outcome-unknown jobs are protected from ordinary retry.
|
||||||
|
- Diagnostic permission is separate from campaign read/report access.
|
||||||
|
|
||||||
|
### Privacy
|
||||||
|
|
||||||
|
Recipient fields and rendered messages may contain personal or sensitive data.
|
||||||
|
Grant recipient read/export, report export, and diagnostic access separately.
|
||||||
|
Prefer aggregate status for readers who do not need recipient detail. Define
|
||||||
|
purpose, lawful basis, minimization, export control, and retention before the
|
||||||
|
campaign starts; do not use Campaign as a substitute consent or address-master
|
||||||
|
system.
|
||||||
|
|
||||||
|
### Audit and destructive actions
|
||||||
|
|
||||||
|
Material authoring, validation, locking, review, queueing, send, retry,
|
||||||
|
reconciliation, sharing, owner transfer, archive, and permitted deletion actions
|
||||||
|
emit attributable evidence. Audit details must be non-secret and should refer to
|
||||||
|
stable ids rather than repeat message bodies or credentials.
|
||||||
|
|
||||||
|
Draft deletion is allowed only while no audit-relevant build, delivery job, or
|
||||||
|
lock exists. Evidence-bearing campaigns are archived. Destructive module
|
||||||
|
retirement remains a separately confirmed installer operation with backup and
|
||||||
|
retirement evidence.
|
||||||
|
|
||||||
|
## Reference-composition acceptance
|
||||||
|
|
||||||
|
Campaign is ready to serve as the demonstration module only when all of the
|
||||||
|
following are repeatable in a pinned clean installation:
|
||||||
|
|
||||||
|
1. The maintained examples validate and build with Mail/Files present, and
|
||||||
|
Campaign still starts with each optional module absent.
|
||||||
|
2. A user can import recipients, select managed files, choose an authorized Mail
|
||||||
|
profile, validate, review, build, and queue without entering transport ids or
|
||||||
|
secrets manually.
|
||||||
|
3. Campaign JSON and all ordinary APIs reject/omit inline transport material;
|
||||||
|
legacy records are visible as migration-required and cannot execute.
|
||||||
|
4. SMTP success, temporary/permanent failure, connection loss, worker loss,
|
||||||
|
outcome unknown, retry, reconciliation, IMAP success, and IMAP failure have
|
||||||
|
tested, non-duplicating outcomes against the target environment.
|
||||||
|
5. Author, reviewer, sender/operator, reader, and administrator views use the
|
||||||
|
central component system and expose only task-relevant actions.
|
||||||
|
6. Reports and audit can reconstruct recipient/message/file/profile/attempt
|
||||||
|
evidence without exposing secrets or ordinary-reader infrastructure details.
|
||||||
|
7. Clean install, upgrade, backup/restore, module permutations, version
|
||||||
|
alignment, security audit, and target SMTP/IMAP tests pass.
|
||||||
|
8. This handbook and its adaptive Docs topics match the shipped UI wording and
|
||||||
|
distinguish implemented behavior from planned work.
|
||||||
|
|
||||||
|
## Explicitly planned, not yet claimed
|
||||||
|
|
||||||
|
The following are part of the selected reference journey but are not implied by
|
||||||
|
the current baseline:
|
||||||
|
|
||||||
|
- the final audited **test / single send / single resend** semantics;
|
||||||
|
- reusable SMTP batch sessions and their measured throughput benefit;
|
||||||
|
- durable, idempotent Campaign report delivery through a Mail-owned outbox
|
||||||
|
([`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17));
|
||||||
|
- a fully packaged one-command Campaign reference composition with production
|
||||||
|
policy presets and target-provider certification;
|
||||||
|
- function-bound Postbox delivery (stage 2 of the reference program);
|
||||||
|
- generic workflow-driven campaign transitions.
|
||||||
|
|
||||||
|
Each item needs an owning issue, implementation, failure tests, documentation,
|
||||||
|
and release evidence before the wording above can move from planned to current.
|
||||||
@@ -10,26 +10,32 @@ scenario catalogue lives in `examples/README.md`; committed fixture files should
|
|||||||
be added under `examples/` only when they validate against the current campaign
|
be added under `examples/` only when they validate against the current campaign
|
||||||
schema and are safe to run in non-production environments.
|
schema and are safe to run in non-production environments.
|
||||||
|
|
||||||
- simple announcement with one active recipient and no attachments
|
- [`simple-announcement`](../examples/simple-announcement/campaign.json), a
|
||||||
|
credential-free campaign with one active recipient and no attachments; its
|
||||||
|
automated acceptance check physically blocks Mail and Files imports, denies
|
||||||
|
network connections, and validates/builds from an unrelated temporary
|
||||||
|
workspace
|
||||||
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
|
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
|
||||||
notification fields
|
notification fields
|
||||||
- campaign with global attachments and recipient-specific attachment rules
|
- campaign with global attachments and recipient-specific attachment rules
|
||||||
- campaign with password-protected ZIP attachments
|
- campaign with password-protected ZIP attachments
|
||||||
- campaign using a reusable mail profile from the mail module
|
- campaign using a reusable mail profile from the mail module
|
||||||
- campaign using inline SMTP/IMAP settings where policy allows campaign-local
|
- legacy inline SMTP/IMAP campaign rejected with an actionable profile-migration
|
||||||
settings
|
error and no returned transport data
|
||||||
- campaign with validation warnings that may be sent only after explicit review
|
- campaign with validation warnings that may be sent only after explicit review
|
||||||
- campaign with blocked recipients or attachment errors that must not be sent
|
- campaign with blocked recipients or attachment errors that must not be sent
|
||||||
- mock delivery campaign that captures SMTP and IMAP append messages in the mail
|
- mock delivery campaign that captures SMTP and IMAP append messages in the mail
|
||||||
development mailbox
|
development mailbox
|
||||||
- real non-production delivery campaign against the GreenMail test bed
|
- [`greenmail-delivery`](../examples/greenmail-delivery/campaign.json), a
|
||||||
|
credential-free real-delivery Campaign materialized with a temporary
|
||||||
|
Mail-owned profile by the loopback acceptance runner
|
||||||
|
|
||||||
## Fixture Rules
|
## Fixture Rules
|
||||||
|
|
||||||
- Examples must not contain production recipient data or production credentials.
|
- Examples must not contain production recipient data or production credentials.
|
||||||
- Attachment examples should use deterministic small files and checksums.
|
- Attachment examples should use deterministic small files and checksums.
|
||||||
- Secret values must be represented through saved-credential placeholders or
|
- Mail secrets belong only to encrypted Mail profiles and must never appear in
|
||||||
secret references.
|
a campaign fixture, placeholder, or campaign-local secret reference.
|
||||||
- Examples that require optional modules must declare the required modules and
|
- Examples that require optional modules must declare the required modules and
|
||||||
capabilities in their README or fixture metadata.
|
capabilities in their README or fixture metadata.
|
||||||
- Examples must stay valid when files or mail modules are physically absent,
|
- Examples must stay valid when files or mail modules are physically absent,
|
||||||
@@ -41,16 +47,37 @@ Before tagging a campaign release:
|
|||||||
|
|
||||||
- Review `examples/README.md` and update the scenario catalogue when a release
|
- Review `examples/README.md` and update the scenario catalogue when a release
|
||||||
adds or removes delivery behavior.
|
adds or removes delivery behavior.
|
||||||
|
- Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'` and
|
||||||
|
retain its isolated validate/build result as release evidence.
|
||||||
- Run core module permutation tests with campaign installed both with and
|
- Run core module permutation tests with campaign installed both with and
|
||||||
without files/mail.
|
without files/mail.
|
||||||
- Validate and build each maintained example campaign.
|
- Validate and build each maintained example campaign.
|
||||||
- Run the mock delivery example when the mail development mailbox capability is
|
- Run the mock delivery example when the mail development mailbox capability is
|
||||||
enabled.
|
enabled.
|
||||||
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
|
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
|
||||||
|
- Run `dev/mail-testbed/run_campaign_acceptance.py` and retain its bounded JSON
|
||||||
|
projection. It must show one SMTP acceptance, one IMAP append, no duplicate
|
||||||
|
effect from a repeated ordinary send, matching Campaign report/audit state,
|
||||||
|
and no resolved transport material in Campaign JSON or its execution
|
||||||
|
snapshot. Its controlled endpoint evidence must also show explicit SMTP 451,
|
||||||
|
partial RCPT refusal, post-DATA ambiguity, and task-process interruption
|
||||||
|
classifications without retaining addresses or provider diagnostics.
|
||||||
|
- Treat the task-process restart proof separately from the still-open
|
||||||
|
Redis/Celery broker redelivery and daemon-supervision check; the coverage
|
||||||
|
projection must keep `celery_broker_redelivery` false.
|
||||||
|
- Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` as a separate
|
||||||
|
destructive worker-loss check. Retain its bounded evidence only when the same
|
||||||
|
broker task is observed at both workers, the durable state is
|
||||||
|
`outcome_unknown`, broker queue/unacked counts are zero, and the controlled
|
||||||
|
SMTP endpoint observed one connection and one DATA transaction. This closes
|
||||||
|
local Redis/Celery redelivery coverage, while production supervisor and
|
||||||
|
target-provider coverage remain false until separately tested.
|
||||||
- Confirm reusable mail profile selection is revalidated after campaign owner
|
- Confirm reusable mail profile selection is revalidated after campaign owner
|
||||||
transfer.
|
transfer.
|
||||||
- Confirm inline SMTP/IMAP settings are hidden or blocked when policy disables
|
- Confirm every inline SMTP/IMAP field is rejected on import/write, omitted
|
||||||
campaign-local mail settings.
|
from responses, and blocked from legacy execution until explicitly migrated.
|
||||||
|
- Confirm execution snapshots store only the Mail profile reference and
|
||||||
|
opaque Mail-owned transport revisions, not resolved transport material.
|
||||||
- Confirm delivery reports include SMTP outcome, IMAP append outcome, latest
|
- Confirm delivery reports include SMTP outcome, IMAP append outcome, latest
|
||||||
error, generated EML reference, and attachment evidence.
|
error, generated EML reference, and attachment evidence.
|
||||||
- Confirm retries cannot resend messages already accepted by SMTP unless an
|
- Confirm retries cannot resend messages already accepted by SMTP unless an
|
||||||
@@ -71,6 +98,6 @@ Use the Review & Send preflight panel and the delivery runbook together:
|
|||||||
2. Build exact messages.
|
2. Build exact messages.
|
||||||
3. Review warnings, generated recipients, body content, and attachment evidence.
|
3. Review warnings, generated recipients, body content, and attachment evidence.
|
||||||
4. Run mock delivery if available for the release channel.
|
4. Run mock delivery if available for the release channel.
|
||||||
5. Test SMTP and IMAP settings against non-production infrastructure.
|
5. Test the selected Mail profile against non-production infrastructure.
|
||||||
6. Send only after queue, rate limit, and append-to-Sent behavior are understood.
|
6. Send only after queue, rate limit, and append-to-Sent behavior are understood.
|
||||||
7. Reconcile failed, unknown, or pending jobs from the report/audit surfaces.
|
7. Reconcile failed, unknown, or pending jobs from the report/audit surfaces.
|
||||||
|
|||||||
93
docs/MAIL_PROFILE_BOUNDARY.md
Normal file
93
docs/MAIL_PROFILE_BOUNDARY.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
# Campaign and Mail Profile Boundary
|
||||||
|
|
||||||
|
## Product view
|
||||||
|
|
||||||
|
A campaign chooses an authorized Mail profile. It does not define a mail
|
||||||
|
server. The Campaign module owns recipients, content, attachment rules,
|
||||||
|
delivery intent, review state, and delivery evidence. The Mail module owns the
|
||||||
|
SMTP/IMAP endpoints, encrypted credentials, connection tests, profile policy,
|
||||||
|
and runtime transport adapters.
|
||||||
|
|
||||||
|
The persisted campaign contract is therefore deliberately narrow:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"server": {
|
||||||
|
"mail_profile_id": "stable-mail-profile-id"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
No `server.smtp`, `server.imap`, `server.credentials`, credential-inheritance
|
||||||
|
override, or password is valid campaign JSON.
|
||||||
|
|
||||||
|
## User journey
|
||||||
|
|
||||||
|
1. A Mail administrator creates and tests a reusable profile in Mail.
|
||||||
|
2. A campaign author opens **Mail settings** and selects one profile available
|
||||||
|
for the campaign's tenant, owner, and policy context.
|
||||||
|
3. Campaign stores only the stable profile identifier.
|
||||||
|
4. Validation asks Mail to authorize the active profile and returns only
|
||||||
|
availability flags plus opaque Mail-owned transport revisions. Reading that
|
||||||
|
summary does not decrypt credentials.
|
||||||
|
5. Build stores the profile identifier, delivery policy, job manifest, and
|
||||||
|
non-secret transport revisions as execution evidence. It stores no
|
||||||
|
resolved host, username, password, or other transport material.
|
||||||
|
6. A delivery worker invokes one Mail-owned effect operation. Mail re-authorizes
|
||||||
|
the profile, resolves credentials, compares the expected transport
|
||||||
|
revision, checks policy, and sends or appends without returning transport
|
||||||
|
material to Campaign. If the selected profile or its non-secret transport settings
|
||||||
|
changed after build, delivery stops until the campaign is revalidated and
|
||||||
|
rebuilt. A password rotation that leaves the transport identity unchanged
|
||||||
|
does not invalidate the build.
|
||||||
|
7. Mail returns only Campaign-owned envelope addresses, counts, sanitized
|
||||||
|
refusal classifications/status codes, or the selected Sent folder. Raw
|
||||||
|
server banners, provider bytes, host details, and credentials never enter
|
||||||
|
Campaign attempts or public evidence.
|
||||||
|
|
||||||
|
Non-dry Campaign report email currently fails closed. It must not bypass the
|
||||||
|
durable job/effect model through a direct SMTP call. Re-enabling it requires the
|
||||||
|
Mail-owned idempotent outbox, attempt, unknown-outcome, and reconciliation path
|
||||||
|
tracked in
|
||||||
|
[`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17).
|
||||||
|
Report generation and dry-run validation remain separate from an external
|
||||||
|
effect; recipient-level exports require recipient-export authorization.
|
||||||
|
|
||||||
|
Campaign authors need `mail:profile:use` in addition to the relevant Campaign
|
||||||
|
permission. Profile visibility remains governed by Mail policy and campaign
|
||||||
|
owner context.
|
||||||
|
|
||||||
|
## Existing database rows and migration
|
||||||
|
|
||||||
|
The current database can contain campaign versions with inline SMTP/IMAP
|
||||||
|
settings or credentials. There is no separate historical Campaign JSON corpus
|
||||||
|
to import or remediate. GovOPlaN treats those inline fields as inert legacy
|
||||||
|
material and does not delete or rewrite the stored audit rows automatically:
|
||||||
|
|
||||||
|
- API responses omit all legacy transport fields and secrets and expose a
|
||||||
|
`mail_profile_migration_required` marker.
|
||||||
|
- validation, build, queue, retry, and delivery fail closed with an actionable
|
||||||
|
profile-migration error;
|
||||||
|
- unrelated edits cannot silently scrub the legacy fields;
|
||||||
|
- an editable version is migrated only through an explicit Mail-settings save
|
||||||
|
with an authorized profile; and
|
||||||
|
- a locked version remains unchanged. Creating its editable successor records
|
||||||
|
the migration while retaining the locked source as audit evidence.
|
||||||
|
|
||||||
|
Legacy execution snapshots are likewise retained but cannot be used for
|
||||||
|
delivery. Revalidate and rebuild an editable profile-only version. Normal
|
||||||
|
database backup, restore, encryption, and access controls apply to the current
|
||||||
|
database as a whole; the product does not define a separate historical-JSON or
|
||||||
|
inline-secret recovery workflow. A restored legacy row remains inert and
|
||||||
|
fail-closed under the same rules.
|
||||||
|
|
||||||
|
## Operator checks
|
||||||
|
|
||||||
|
Before live delivery, confirm that:
|
||||||
|
|
||||||
|
- the profile is active and still authorized for the campaign owner;
|
||||||
|
- SMTP and optional IMAP profile tests pass;
|
||||||
|
- validation and build occurred after the latest transport-identity change;
|
||||||
|
- append-to-Sent is enabled only when the selected profile has IMAP; and
|
||||||
|
- reports show the profile-bound snapshot revisions and delivery outcomes,
|
||||||
|
never credentials or resolved transport configuration.
|
||||||
@@ -9,25 +9,25 @@ campaign schema and do not require production data.
|
|||||||
|
|
||||||
| Scenario | Required Modules | Release Check |
|
| Scenario | Required Modules | Release Check |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `simple-announcement` | core, access, campaigns | Validate and build one active recipient without attachments. |
|
| [`simple-announcement`](simple-announcement/campaign.json) | core, access, campaigns | Validate and build one active recipient without attachments while Mail and Files are absent. |
|
||||||
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
|
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
|
||||||
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
|
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
|
||||||
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
|
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
|
||||||
| `zip-protected` | core, access, campaigns | Build password-protected AES ZIP output and verify password-source metadata. |
|
| `zip-protected` | core, access, campaigns | Build password-protected AES ZIP output and verify password-source metadata. |
|
||||||
| `mail-profile-send` | core, access, campaigns, mail | Select a reusable mail profile and send through the GreenMail test bed. |
|
| `mail-profile-send` | core, access, campaigns, mail | Select a reusable mail profile and send through the GreenMail test bed. |
|
||||||
| `inline-mail-settings` | core, access, campaigns, mail | Use campaign-local SMTP/IMAP settings only when policy allows it. |
|
| `legacy-inline-mail-rejected` | core, access, campaigns, mail | Confirm legacy campaign-local SMTP/IMAP data fails closed and requires explicit profile migration. |
|
||||||
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
|
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
|
||||||
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
|
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
|
||||||
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
|
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
|
||||||
| `greenmail-delivery` | core, access, campaigns, mail | Send no-attachment, normal attachment, and ZIP attachment variants through `dev/mail-testbed`. |
|
| [`greenmail-delivery`](greenmail-delivery/campaign.json) | core, access, audit, campaigns, mail | Run a credential-free Campaign through a Mail-owned profile, GreenMail SMTP/IMAP, report/audit checks, repeat-send protection, and bounded failure drills. |
|
||||||
|
|
||||||
## Fixture Rules
|
## Fixture Rules
|
||||||
|
|
||||||
- Do not commit real recipients, mail credentials, or production attachment
|
- Do not commit real recipients, mail credentials, or production attachment
|
||||||
names.
|
names.
|
||||||
- Keep attachments deterministic and small.
|
- Keep attachments deterministic and small.
|
||||||
- Store secrets as placeholders, saved-credential references, or local `.env`
|
- Store transport secrets only in encrypted Mail profiles or local `.env`
|
||||||
values consumed by the test bed.
|
values consumed directly by the test bed, never in campaign JSON.
|
||||||
- Declare optional module requirements in fixture metadata.
|
- Declare optional module requirements in fixture metadata.
|
||||||
- Fixtures must not import files or mail modules directly; optional behavior is
|
- Fixtures must not import files or mail modules directly; optional behavior is
|
||||||
discovered through core module metadata and capabilities.
|
discovered through core module metadata and capabilities.
|
||||||
@@ -37,9 +37,14 @@ campaign schema and do not require production data.
|
|||||||
Before a release tag:
|
Before a release tag:
|
||||||
|
|
||||||
1. Run module permutation startup checks from core.
|
1. Run module permutation startup checks from core.
|
||||||
2. Validate every committed example fixture against the current campaign schema.
|
2. Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'`
|
||||||
3. Build exact messages for each fixture.
|
from this repository. The acceptance test copies each maintained fixture to
|
||||||
4. Run the mock-delivery example when the dev mailbox capability is enabled.
|
an unrelated temporary workspace before using Campaign's public loader,
|
||||||
5. Run `dev/mail-testbed/run_transport_smoke.py`.
|
validator, and message builder.
|
||||||
6. Execute the delivery checklist in
|
3. Validate every committed example fixture against the current campaign schema.
|
||||||
|
4. Build exact messages for each fixture.
|
||||||
|
5. Run the mock-delivery example when the dev mailbox capability is enabled.
|
||||||
|
6. Run `dev/mail-testbed/run_transport_smoke.py` for low-level transport and attachment variants.
|
||||||
|
7. Run `dev/mail-testbed/run_campaign_acceptance.py` for the Campaign journey and bounded evidence.
|
||||||
|
8. Execute the delivery checklist in
|
||||||
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.
|
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.
|
||||||
|
|||||||
88
examples/greenmail-delivery/campaign.json
Normal file
88
examples/greenmail-delivery/campaign.json
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "greenmail-delivery",
|
||||||
|
"name": "GreenMail delivery acceptance",
|
||||||
|
"description": "Credential-free Campaign fixture for the local SMTP/IMAP acceptance test bed.",
|
||||||
|
"mode": "test"
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "display_name",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Display name",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "acceptance_run",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Acceptance run",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"server": {
|
||||||
|
"mail_profile_id": "00000000-0000-4000-8000-000000000001"
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": [
|
||||||
|
{
|
||||||
|
"email": "campaign-test@govoplan.test",
|
||||||
|
"name": "GovOPlaN acceptance",
|
||||||
|
"type": "to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_to": true
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "[GovOPlaN acceptance ${acceptance_run}] Campaign delivery",
|
||||||
|
"text": "Hello ${display_name},\n\nThis is an isolated GovOPlaN Campaign SMTP/IMAP acceptance message.\n",
|
||||||
|
"body_mode": "text"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": ".",
|
||||||
|
"send_without_attachments_behavior": "continue",
|
||||||
|
"global": []
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"inline": [
|
||||||
|
{
|
||||||
|
"id": "greenmail-recipient",
|
||||||
|
"to": [
|
||||||
|
{
|
||||||
|
"email": "campaign-test@govoplan.test",
|
||||||
|
"name": "GreenMail recipient",
|
||||||
|
"type": "to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"fields": {
|
||||||
|
"display_name": "GreenMail recipient",
|
||||||
|
"acceptance_run": "fixture"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 60
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
1,
|
||||||
|
5,
|
||||||
|
30
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "Sent"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true
|
||||||
|
}
|
||||||
|
}
|
||||||
22
examples/greenmail-delivery/fixture.json
Normal file
22
examples/greenmail-delivery/fixture.json
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"scenario": "greenmail-delivery",
|
||||||
|
"campaign_file": "campaign.json",
|
||||||
|
"required_modules": [
|
||||||
|
"core",
|
||||||
|
"access",
|
||||||
|
"audit",
|
||||||
|
"campaigns",
|
||||||
|
"mail"
|
||||||
|
],
|
||||||
|
"required_capabilities": [
|
||||||
|
"mail.campaign_delivery"
|
||||||
|
],
|
||||||
|
"transport": "local GreenMail SMTP/IMAP test bed",
|
||||||
|
"credentials": "local environment only; never copied into Campaign JSON or evidence",
|
||||||
|
"expected": {
|
||||||
|
"entries_count": 1,
|
||||||
|
"built_count": 1,
|
||||||
|
"smtp_accepted_count": 1,
|
||||||
|
"imap_appended_count": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
54
examples/simple-announcement/campaign.json
Normal file
54
examples/simple-announcement/campaign.json
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "simple-announcement",
|
||||||
|
"name": "Simple announcement",
|
||||||
|
"description": "Credential-free release fixture for Campaign validation and message building.",
|
||||||
|
"mode": "test"
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "display_name",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Display name",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"recipients": {
|
||||||
|
"from": [
|
||||||
|
{
|
||||||
|
"email": "announcements@example.test",
|
||||||
|
"name": "GovOPlaN Example",
|
||||||
|
"type": "to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_to": true
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Planned service maintenance for ${display_name}",
|
||||||
|
"text": "Hello ${display_name},\n\nThe example service will be unavailable during the announced maintenance window.\n\nThis message was built locally and was not sent.\n",
|
||||||
|
"body_mode": "text"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": ".",
|
||||||
|
"send_without_attachments_behavior": "continue",
|
||||||
|
"global": []
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"inline": [
|
||||||
|
{
|
||||||
|
"id": "example-recipient",
|
||||||
|
"to": [
|
||||||
|
{
|
||||||
|
"email": "recipient@example.test",
|
||||||
|
"name": "Example Recipient",
|
||||||
|
"type": "to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"fields": {
|
||||||
|
"display_name": "Example Recipient"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
23
examples/simple-announcement/fixture.json
Normal file
23
examples/simple-announcement/fixture.json
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"schema_version": 1,
|
||||||
|
"id": "simple-announcement",
|
||||||
|
"campaign_file": "campaign.json",
|
||||||
|
"required_modules": [
|
||||||
|
"core",
|
||||||
|
"access",
|
||||||
|
"campaigns"
|
||||||
|
],
|
||||||
|
"absent_optional_modules": [
|
||||||
|
"files",
|
||||||
|
"mail"
|
||||||
|
],
|
||||||
|
"external_effects": "forbidden",
|
||||||
|
"expected": {
|
||||||
|
"campaign_id": "simple-announcement",
|
||||||
|
"entries_count": 1,
|
||||||
|
"built_count": 1,
|
||||||
|
"queueable_count": 1,
|
||||||
|
"attachment_count": 0,
|
||||||
|
"subject": "Planned service maintenance for Example Recipient"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@govoplan/campaign-webui",
|
"name": "@govoplan/campaign-webui",
|
||||||
"version": "0.1.8",
|
"version": "0.1.12",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "webui/src/index.ts",
|
"main": "webui/src/index.ts",
|
||||||
@@ -22,7 +22,7 @@
|
|||||||
"read-excel-file": "9.2.0"
|
"read-excel-file": "9.2.0"
|
||||||
},
|
},
|
||||||
"peerDependencies": {
|
"peerDependencies": {
|
||||||
"@govoplan/core-webui": "^0.1.8",
|
"@govoplan/core-webui": "^0.1.12",
|
||||||
"lucide-react": "^1.23.0",
|
"lucide-react": "^1.23.0",
|
||||||
"react": "^19.0.0",
|
"react": "^19.0.0",
|
||||||
"react-dom": "^19.0.0",
|
"react-dom": "^19.0.0",
|
||||||
|
|||||||
@@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "govoplan-campaign"
|
name = "govoplan-campaign"
|
||||||
version = "0.1.8"
|
version = "0.1.12"
|
||||||
description = "GovOPlaN campaigns module with backend and WebUI integration."
|
description = "GovOPlaN campaigns module with backend and WebUI integration."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.12"
|
requires-python = ">=3.12"
|
||||||
license = { file = "LICENSE" }
|
license = { file = "LICENSE" }
|
||||||
authors = [{ name = "GovOPlaN" }]
|
authors = [{ name = "GovOPlaN" }]
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"govoplan-core>=0.1.8",
|
"govoplan-core>=0.1.14",
|
||||||
"jsonschema>=4,<5",
|
"jsonschema>=4,<5",
|
||||||
"pydantic>=2,<3",
|
"pydantic>=2,<3",
|
||||||
"SQLAlchemy>=2,<3",
|
"SQLAlchemy>=2,<3",
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import fnmatch
|
import fnmatch
|
||||||
import re
|
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from enum import StrEnum
|
from enum import StrEnum
|
||||||
@@ -13,6 +12,12 @@ from pydantic import BaseModel, ConfigDict, Field
|
|||||||
from govoplan_campaign.backend.campaign.entries import load_campaign_entries
|
from govoplan_campaign.backend.campaign.entries import load_campaign_entries
|
||||||
from govoplan_campaign.backend.campaign.template_values import build_template_values
|
from govoplan_campaign.backend.campaign.template_values import build_template_values
|
||||||
from govoplan_campaign.backend.campaign.models import AttachmentBasePathConfig, AttachmentConfig, Behavior, CampaignConfig, EntryConfig, ZipArchiveConfig, ZipRuleMode
|
from govoplan_campaign.backend.campaign.models import AttachmentBasePathConfig, AttachmentConfig, Behavior, CampaignConfig, EntryConfig, ZipArchiveConfig, ZipRuleMode
|
||||||
|
from govoplan_campaign.backend.path_security import (
|
||||||
|
CampaignPathSecurityError,
|
||||||
|
assert_logical_relative_path,
|
||||||
|
is_managed_source,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.template_rendering import render_template
|
||||||
|
|
||||||
|
|
||||||
class AttachmentScope(StrEnum):
|
class AttachmentScope(StrEnum):
|
||||||
@@ -48,6 +53,17 @@ class AttachmentIssue(BaseModel):
|
|||||||
code: str
|
code: str
|
||||||
message: str
|
message: str
|
||||||
behavior: Behavior | None = None
|
behavior: Behavior | None = None
|
||||||
|
details: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentPolicyDecision(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
requirement_policy: Behavior
|
||||||
|
campaign_policy: Behavior
|
||||||
|
rule_policy: Behavior | None = None
|
||||||
|
effective_behavior: Behavior
|
||||||
|
legacy_drop_normalized: bool = False
|
||||||
|
|
||||||
|
|
||||||
class ResolvedAttachment(BaseModel):
|
class ResolvedAttachment(BaseModel):
|
||||||
@@ -77,6 +93,7 @@ class ResolvedAttachment(BaseModel):
|
|||||||
zip_entry_names: list[str] = Field(default_factory=list)
|
zip_entry_names: list[str] = Field(default_factory=list)
|
||||||
status: AttachmentMatchStatus
|
status: AttachmentMatchStatus
|
||||||
behavior: Behavior | None = None
|
behavior: Behavior | None = None
|
||||||
|
missing_policy: AttachmentPolicyDecision | None = None
|
||||||
matches: list[str] = Field(default_factory=list)
|
matches: list[str] = Field(default_factory=list)
|
||||||
issues: list[AttachmentIssue] = Field(default_factory=list)
|
issues: list[AttachmentIssue] = Field(default_factory=list)
|
||||||
|
|
||||||
@@ -140,43 +157,8 @@ def _resolve_path(campaign_file: str | Path, raw_path: str) -> Path:
|
|||||||
return (campaign_path.parent / path).resolve()
|
return (campaign_path.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
_DOLLAR_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
|
|
||||||
_BRACE_FIELD_PATTERN = re.compile(r"(?<!\\)\{\{\s*(.*?)\s*\}\}")
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_template_key(raw: str) -> str:
|
|
||||||
key = raw.strip()
|
|
||||||
if key.startswith("fields."):
|
|
||||||
key = key.removeprefix("fields.")
|
|
||||||
elif key.startswith("local."):
|
|
||||||
key = "local::" + key.removeprefix("local.")
|
|
||||||
elif key.startswith("global."):
|
|
||||||
key = "global::" + key.removeprefix("global.")
|
|
||||||
|
|
||||||
if key.startswith("local::") or key.startswith("global::"):
|
|
||||||
return key
|
|
||||||
if key.startswith("local:"):
|
|
||||||
return "local::" + key.removeprefix("local:")
|
|
||||||
if key.startswith("global:"):
|
|
||||||
return "global::" + key.removeprefix("global:")
|
|
||||||
return key
|
|
||||||
|
|
||||||
|
|
||||||
def _render_template(template: str, values: dict[str, Any]) -> str:
|
|
||||||
def replace(match: re.Match[str]) -> str:
|
|
||||||
key = _normalize_template_key(match.group(1))
|
|
||||||
if key in values:
|
|
||||||
value = values[key]
|
|
||||||
return "" if value is None else str(value)
|
|
||||||
return match.group(0)
|
|
||||||
|
|
||||||
rendered = _DOLLAR_FIELD_PATTERN.sub(replace, template)
|
|
||||||
rendered = _BRACE_FIELD_PATTERN.sub(replace, rendered)
|
|
||||||
return rendered.replace(r"\${", "${").replace(r"\}", "}")
|
|
||||||
|
|
||||||
|
|
||||||
def _rendered_base_dir(config: AttachmentConfig, values: dict[str, Any]) -> str:
|
def _rendered_base_dir(config: AttachmentConfig, values: dict[str, Any]) -> str:
|
||||||
rendered = _render_template(config.base_dir, values).strip()
|
rendered = render_template(config.base_dir, values, keep_missing=True).strip()
|
||||||
return rendered or "."
|
return rendered or "."
|
||||||
|
|
||||||
|
|
||||||
@@ -223,12 +205,48 @@ def _rule_allows_multiple(config: AttachmentConfig, rendered_file_filter: str) -
|
|||||||
return config.allow_multiple or any(char in rendered_file_filter for char in "*?[")
|
return config.allow_multiple or any(char in rendered_file_filter for char in "*?[")
|
||||||
|
|
||||||
|
|
||||||
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
_MISSING_BEHAVIOR_STRENGTH = {
|
||||||
|
Behavior.CONTINUE: 0,
|
||||||
|
Behavior.WARN: 1,
|
||||||
|
Behavior.ASK: 2,
|
||||||
|
Behavior.DROP: 2,
|
||||||
|
Behavior.BLOCK: 3,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _missing_policy_decision(
|
||||||
|
campaign_config: CampaignConfig,
|
||||||
|
config: AttachmentConfig,
|
||||||
|
) -> AttachmentPolicyDecision:
|
||||||
|
requirement_policy = (
|
||||||
|
campaign_config.validation_policy.missing_required_attachment
|
||||||
|
if config.required
|
||||||
|
else campaign_config.validation_policy.missing_optional_attachment
|
||||||
|
)
|
||||||
|
candidates = [
|
||||||
|
requirement_policy,
|
||||||
|
campaign_config.attachments.missing_behavior,
|
||||||
|
]
|
||||||
if config.missing_behavior is not None:
|
if config.missing_behavior is not None:
|
||||||
return config.missing_behavior
|
candidates.append(config.missing_behavior)
|
||||||
if config.required:
|
configured = max(
|
||||||
return campaign_config.validation_policy.missing_required_attachment
|
candidates,
|
||||||
return campaign_config.validation_policy.missing_optional_attachment
|
key=lambda behavior: _MISSING_BEHAVIOR_STRENGTH[behavior],
|
||||||
|
)
|
||||||
|
legacy_drop_normalized = configured == Behavior.DROP
|
||||||
|
if legacy_drop_normalized:
|
||||||
|
configured = Behavior.BLOCK if config.required else Behavior.ASK
|
||||||
|
return AttachmentPolicyDecision(
|
||||||
|
requirement_policy=requirement_policy,
|
||||||
|
campaign_policy=campaign_config.attachments.missing_behavior,
|
||||||
|
rule_policy=config.missing_behavior,
|
||||||
|
effective_behavior=configured,
|
||||||
|
legacy_drop_normalized=legacy_drop_normalized,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
||||||
|
return _missing_policy_decision(campaign_config, config).effective_behavior
|
||||||
|
|
||||||
|
|
||||||
def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
||||||
@@ -260,7 +278,7 @@ def _attachment_zip_archive(
|
|||||||
def _render_zip_filename(archive: ZipArchiveConfig | None, values: dict[str, Any]) -> str | None:
|
def _render_zip_filename(archive: ZipArchiveConfig | None, values: dict[str, Any]) -> str | None:
|
||||||
if archive is None:
|
if archive is None:
|
||||||
return None
|
return None
|
||||||
rendered = _render_template(archive.name or "attachments.zip", values).strip() or "attachments.zip"
|
rendered = render_template(archive.name or "attachments.zip", values, keep_missing=True).strip() or "attachments.zip"
|
||||||
return rendered if rendered.lower().endswith(".zip") else f"{rendered}.zip"
|
return rendered if rendered.lower().endswith(".zip") else f"{rendered}.zip"
|
||||||
|
|
||||||
|
|
||||||
@@ -369,14 +387,36 @@ def _match_files(directory: Path, file_filter: str, include_subdirs: bool, match
|
|||||||
return sorted(path for path in directory.glob(file_filter) if path.is_file())
|
return sorted(path for path in directory.glob(file_filter) if path.is_file())
|
||||||
|
|
||||||
|
|
||||||
def _issue_for_missing(config: AttachmentConfig, behavior: Behavior) -> AttachmentIssue:
|
def _confine_managed_matches(directory: Path, matches: list[Path]) -> tuple[list[Path], bool]:
|
||||||
|
root = directory.resolve()
|
||||||
|
confined: list[Path] = []
|
||||||
|
rejected = False
|
||||||
|
for match in matches:
|
||||||
|
try:
|
||||||
|
resolved = match.resolve()
|
||||||
|
except (OSError, RuntimeError):
|
||||||
|
rejected = True
|
||||||
|
continue
|
||||||
|
if not resolved.is_relative_to(root):
|
||||||
|
rejected = True
|
||||||
|
continue
|
||||||
|
confined.append(match)
|
||||||
|
return confined, rejected
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_for_missing(
|
||||||
|
config: AttachmentConfig,
|
||||||
|
policy: AttachmentPolicyDecision,
|
||||||
|
) -> AttachmentIssue:
|
||||||
code = "missing_required_attachment" if config.required else "missing_optional_attachment"
|
code = "missing_required_attachment" if config.required else "missing_optional_attachment"
|
||||||
severity = ResolutionSeverity.ERROR if config.required and behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
behavior = policy.effective_behavior
|
||||||
|
severity = ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
||||||
return AttachmentIssue(
|
return AttachmentIssue(
|
||||||
severity=severity,
|
severity=severity,
|
||||||
code=code,
|
code=code,
|
||||||
message=f"No file matched attachment filter {config.file_filter!r}",
|
message=f"No file matched attachment filter {config.file_filter!r}",
|
||||||
behavior=behavior,
|
behavior=behavior,
|
||||||
|
details={"effective_policy": policy.model_dump(mode="json")},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -390,6 +430,31 @@ def _issue_for_ambiguous(config: AttachmentConfig, behavior: Behavior, match_cou
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def effective_send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
|
||||||
|
configured = config.attachments.send_without_attachments_behavior or (
|
||||||
|
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
|
||||||
|
)
|
||||||
|
# Recipient exclusion must be an explicit reviewed action, not an implicit
|
||||||
|
# consequence of a legacy attachment policy value.
|
||||||
|
return Behavior.ASK if configured == Behavior.DROP else configured
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssue:
|
||||||
|
messages = {
|
||||||
|
Behavior.BLOCK: "No attachment file was resolved for this message, and campaign policy blocks sending without attachments.",
|
||||||
|
Behavior.ASK: "No attachment file was resolved for this message. Confirm the message should still be sent.",
|
||||||
|
Behavior.DROP: "No attachment file was resolved for this message. Campaign policy excludes messages without attachments.",
|
||||||
|
Behavior.WARN: "No attachment file was resolved for this message. Campaign policy allows sending with a warning.",
|
||||||
|
}
|
||||||
|
return AttachmentIssue(
|
||||||
|
severity=ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.WARNING,
|
||||||
|
code="missing_attachment_coverage",
|
||||||
|
message=messages.get(behavior, "No attachment file was resolved for this message."),
|
||||||
|
behavior=behavior,
|
||||||
|
details={"effective_behavior": behavior.value},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _resolve_one_config(
|
def _resolve_one_config(
|
||||||
*,
|
*,
|
||||||
campaign_file: str | Path,
|
campaign_file: str | Path,
|
||||||
@@ -401,23 +466,77 @@ def _resolve_one_config(
|
|||||||
match_index: AttachmentMatchIndex | None = None,
|
match_index: AttachmentMatchIndex | None = None,
|
||||||
) -> ResolvedAttachment:
|
) -> ResolvedAttachment:
|
||||||
rendered_base_dir = _rendered_base_dir(config, values)
|
rendered_base_dir = _rendered_base_dir(config, values)
|
||||||
rendered_file_filter = _render_template(config.file_filter, values)
|
rendered_file_filter = render_template(config.file_filter, values, keep_missing=True)
|
||||||
directory, selected_base_path = _resolve_attachment_directory(
|
directory, selected_base_path = _resolve_attachment_directory(
|
||||||
campaign_file=campaign_file,
|
campaign_file=campaign_file,
|
||||||
campaign_config=campaign_config,
|
campaign_config=campaign_config,
|
||||||
attachment_config=config,
|
attachment_config=config,
|
||||||
rendered_base_dir=rendered_base_dir,
|
rendered_base_dir=rendered_base_dir,
|
||||||
)
|
)
|
||||||
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
|
||||||
allow_multiple = _rule_allows_multiple(config, rendered_file_filter)
|
allow_multiple = _rule_allows_multiple(config, rendered_file_filter)
|
||||||
|
|
||||||
issues: list[AttachmentIssue] = []
|
issues: list[AttachmentIssue] = []
|
||||||
behavior: Behavior | None = None
|
behavior: Behavior | None = None
|
||||||
|
managed_source = selected_base_path is not None and is_managed_source(selected_base_path.source)
|
||||||
|
unsafe_managed_path = False
|
||||||
|
resolution_failed = False
|
||||||
|
try:
|
||||||
|
if managed_source:
|
||||||
|
assert_logical_relative_path(
|
||||||
|
rendered_file_filter,
|
||||||
|
field="rendered managed attachment file_filter",
|
||||||
|
)
|
||||||
|
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
||||||
|
matches, rejected = _confine_managed_matches(directory, matches)
|
||||||
|
if rejected:
|
||||||
|
matches = []
|
||||||
|
unsafe_managed_path = True
|
||||||
|
issues.append(
|
||||||
|
AttachmentIssue(
|
||||||
|
severity=ResolutionSeverity.ERROR,
|
||||||
|
code="managed_attachment_path_escape",
|
||||||
|
message="A managed attachment match resolved outside its materialized source directory.",
|
||||||
|
behavior=Behavior.BLOCK,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
||||||
|
except CampaignPathSecurityError as exc:
|
||||||
|
matches = []
|
||||||
|
unsafe_managed_path = True
|
||||||
|
issues.append(
|
||||||
|
AttachmentIssue(
|
||||||
|
severity=ResolutionSeverity.ERROR,
|
||||||
|
code="unsafe_managed_attachment_path",
|
||||||
|
message=str(exc),
|
||||||
|
behavior=Behavior.BLOCK,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except (OSError, RuntimeError) as exc:
|
||||||
|
matches = []
|
||||||
|
resolution_failed = True
|
||||||
|
issues.append(
|
||||||
|
AttachmentIssue(
|
||||||
|
severity=ResolutionSeverity.ERROR,
|
||||||
|
code="attachment_resolution_failed",
|
||||||
|
message=f"Attachment source could not be read while resolving filter {config.file_filter!r}.",
|
||||||
|
behavior=Behavior.BLOCK,
|
||||||
|
details={"error_type": type(exc).__name__},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
if not matches:
|
missing_policy: AttachmentPolicyDecision | None = None
|
||||||
|
if unsafe_managed_path:
|
||||||
status = AttachmentMatchStatus.MISSING
|
status = AttachmentMatchStatus.MISSING
|
||||||
behavior = _missing_behavior(campaign_config, config)
|
behavior = Behavior.BLOCK
|
||||||
issues.append(_issue_for_missing(config, behavior))
|
elif resolution_failed:
|
||||||
|
status = AttachmentMatchStatus.MISSING
|
||||||
|
behavior = Behavior.BLOCK
|
||||||
|
elif not matches:
|
||||||
|
status = AttachmentMatchStatus.MISSING
|
||||||
|
missing_policy = _missing_policy_decision(campaign_config, config)
|
||||||
|
behavior = missing_policy.effective_behavior
|
||||||
|
issues.append(_issue_for_missing(config, missing_policy))
|
||||||
elif len(matches) > 1 and not allow_multiple:
|
elif len(matches) > 1 and not allow_multiple:
|
||||||
status = AttachmentMatchStatus.AMBIGUOUS
|
status = AttachmentMatchStatus.AMBIGUOUS
|
||||||
behavior = _ambiguous_behavior(campaign_config, config)
|
behavior = _ambiguous_behavior(campaign_config, config)
|
||||||
@@ -449,6 +568,7 @@ def _resolve_one_config(
|
|||||||
zip_entry_name_template=config.zip_entry_name_template,
|
zip_entry_name_template=config.zip_entry_name_template,
|
||||||
status=status,
|
status=status,
|
||||||
behavior=behavior,
|
behavior=behavior,
|
||||||
|
missing_policy=missing_policy,
|
||||||
matches=[str(path) for path in matches],
|
matches=[str(path) for path in matches],
|
||||||
issues=issues,
|
issues=issues,
|
||||||
)
|
)
|
||||||
@@ -495,6 +615,15 @@ def resolve_entry_attachments(
|
|||||||
)
|
)
|
||||||
|
|
||||||
issues = [issue for item in resolved for issue in item.issues]
|
issues = [issue for item in resolved for issue in item.issues]
|
||||||
|
missing_coverage_behavior = effective_send_without_attachments_behavior(config)
|
||||||
|
if (
|
||||||
|
entry.active
|
||||||
|
and resolved
|
||||||
|
and missing_coverage_behavior != Behavior.CONTINUE
|
||||||
|
and sum(len(item.matches) for item in resolved) == 0
|
||||||
|
and not any(issue.behavior == Behavior.BLOCK for issue in issues)
|
||||||
|
):
|
||||||
|
issues.append(_issue_for_missing_attachment_coverage(missing_coverage_behavior))
|
||||||
return EntryAttachmentResolution(
|
return EntryAttachmentResolution(
|
||||||
entry_index=entry_index,
|
entry_index=entry_index,
|
||||||
entry_id=entry.id,
|
entry_id=entry.id,
|
||||||
|
|||||||
@@ -44,6 +44,7 @@ def _parse_scalar_for_target(target: str, value: Any) -> Any:
|
|||||||
"merge_reply_to",
|
"merge_reply_to",
|
||||||
"merge_bounce_to",
|
"merge_bounce_to",
|
||||||
"merge_disposition_notification_to",
|
"merge_disposition_notification_to",
|
||||||
|
"merge_postbox_targets",
|
||||||
"combine_to",
|
"combine_to",
|
||||||
"combine_cc",
|
"combine_cc",
|
||||||
"combine_bcc",
|
"combine_bcc",
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ from typing import Any
|
|||||||
|
|
||||||
from jsonschema import Draft202012Validator, FormatChecker
|
from jsonschema import Draft202012Validator, FormatChecker
|
||||||
|
|
||||||
|
from .mail_profile_boundary import assert_campaign_uses_mail_profile_reference
|
||||||
from .models import CampaignConfig
|
from .models import CampaignConfig
|
||||||
|
|
||||||
|
|
||||||
@@ -83,6 +84,7 @@ def load_campaign_config(
|
|||||||
schema_path: str | Path | None = None,
|
schema_path: str | Path | None = None,
|
||||||
) -> CampaignConfig:
|
) -> CampaignConfig:
|
||||||
data = load_campaign_json(path)
|
data = load_campaign_json(path)
|
||||||
|
assert_campaign_uses_mail_profile_reference(data)
|
||||||
if validate_schema:
|
if validate_schema:
|
||||||
validate_against_schema(data, schema_path=schema_path)
|
validate_against_schema(data, schema_path=schema_path)
|
||||||
return CampaignConfig.model_validate(data)
|
return CampaignConfig.model_validate(data)
|
||||||
|
|||||||
422
src/govoplan_campaign/backend/campaign/mail_profile_boundary.py
Normal file
422
src/govoplan_campaign/backend/campaign/mail_profile_boundary.py
Normal file
@@ -0,0 +1,422 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
CAMPAIGN_MAIL_SERVER_KEYS = frozenset(
|
||||||
|
{
|
||||||
|
"mail_profile_id",
|
||||||
|
"smtp_server_id",
|
||||||
|
"smtp_credential_id",
|
||||||
|
"imap_server_id",
|
||||||
|
"imap_credential_id",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset({"created_from", "field_overrides", "opt_ins"})
|
||||||
|
CAMPAIGN_OPT_IN_KEYS = frozenset(
|
||||||
|
{"campaign_address_suggestions", "remember_used_addresses", "inline_guidance"}
|
||||||
|
)
|
||||||
|
CAMPAIGN_REVIEW_STATE_KEYS = frozenset(
|
||||||
|
{
|
||||||
|
"build_token",
|
||||||
|
"inspection_complete",
|
||||||
|
"reviewed_message_keys",
|
||||||
|
"issue_decisions",
|
||||||
|
"updated_at",
|
||||||
|
"updated_by_user_id",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignMailProfileBoundaryError(ValueError):
|
||||||
|
"""Raised when campaign JSON owns mail transport configuration.
|
||||||
|
|
||||||
|
SMTP/IMAP endpoints and credentials are Mail-module data. Campaign JSON
|
||||||
|
may select Mail-owned profile, server, and credential identifiers, but it
|
||||||
|
must never copy or override transport configuration.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_opt_ins(value: Any) -> dict[str, bool]:
|
||||||
|
if not isinstance(value, dict) or any(
|
||||||
|
key not in CAMPAIGN_OPT_IN_KEYS for key in value
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign editor opt_ins contains unsupported fields"
|
||||||
|
)
|
||||||
|
if any(not isinstance(item, bool) for item in value.values()):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign editor opt_ins values must be booleans"
|
||||||
|
)
|
||||||
|
return copy.deepcopy(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_valid_field_override(key: Any, value: Any) -> bool:
|
||||||
|
return (
|
||||||
|
isinstance(key, str)
|
||||||
|
and bool(key.strip())
|
||||||
|
and len(key) <= 256
|
||||||
|
and isinstance(value, bool)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_field_overrides(value: Any) -> dict[str, bool]:
|
||||||
|
if not isinstance(value, dict) or len(value) > 10_000:
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign editor field_overrides must be a bounded object"
|
||||||
|
)
|
||||||
|
if any(not _is_valid_field_override(key, item) for key, item in value.items()):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign editor field_overrides must map short field names to booleans"
|
||||||
|
)
|
||||||
|
return copy.deepcopy(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_required_string(value: Any, *, max_length: int, error: str) -> str:
|
||||||
|
if not isinstance(value, str) or not value.strip() or len(value) > max_length:
|
||||||
|
raise CampaignMailProfileBoundaryError(error)
|
||||||
|
return value.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def validate_campaign_editor_state(
|
||||||
|
value: dict[str, Any] | None,
|
||||||
|
*,
|
||||||
|
allow_server_review_state: bool = False,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Validate the bounded Campaign-owned UI metadata contract.
|
||||||
|
|
||||||
|
Arbitrary editor metadata would be a second, weakly typed persistence and
|
||||||
|
response channel for Mail credentials. Review evidence is server-owned and
|
||||||
|
cannot be supplied through ordinary version create/update requests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if value is None:
|
||||||
|
return {}
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise CampaignMailProfileBoundaryError("Campaign editor state must be an object")
|
||||||
|
allowed = set(CAMPAIGN_CLIENT_EDITOR_STATE_KEYS)
|
||||||
|
if allow_server_review_state:
|
||||||
|
allowed.add("review_send")
|
||||||
|
if any(key not in allowed for key in value):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign editor state contains unsupported or transport-owned fields"
|
||||||
|
)
|
||||||
|
|
||||||
|
result: dict[str, Any] = {}
|
||||||
|
if "created_from" in value:
|
||||||
|
result["created_from"] = _validated_required_string(
|
||||||
|
value["created_from"],
|
||||||
|
max_length=128,
|
||||||
|
error="Campaign editor created_from must be a short string",
|
||||||
|
)
|
||||||
|
if "opt_ins" in value:
|
||||||
|
result["opt_ins"] = _validated_opt_ins(value["opt_ins"])
|
||||||
|
if "field_overrides" in value:
|
||||||
|
result["field_overrides"] = _validated_field_overrides(
|
||||||
|
value["field_overrides"]
|
||||||
|
)
|
||||||
|
if "review_send" in value:
|
||||||
|
result["review_send"] = _validated_server_review_state(value["review_send"])
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def public_campaign_editor_state(
|
||||||
|
value: Any,
|
||||||
|
*,
|
||||||
|
include_diagnostics: bool = False,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Return only known non-secret UI metadata from current or legacy rows."""
|
||||||
|
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return {}
|
||||||
|
result: dict[str, Any] = {}
|
||||||
|
for key in CAMPAIGN_CLIENT_EDITOR_STATE_KEYS:
|
||||||
|
if key not in value:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
result.update(validate_campaign_editor_state({key: value[key]}))
|
||||||
|
except CampaignMailProfileBoundaryError:
|
||||||
|
continue
|
||||||
|
if "review_send" in value:
|
||||||
|
try:
|
||||||
|
review_state = _validated_server_review_state(value["review_send"])
|
||||||
|
if not include_diagnostics:
|
||||||
|
review_state.pop("build_token", None)
|
||||||
|
review_state["issue_decisions"] = [
|
||||||
|
{
|
||||||
|
key: item[key]
|
||||||
|
for key in (
|
||||||
|
"job_id",
|
||||||
|
"review_key",
|
||||||
|
"decision",
|
||||||
|
"reason",
|
||||||
|
"actor_user_id",
|
||||||
|
"decided_at",
|
||||||
|
"issue_codes",
|
||||||
|
)
|
||||||
|
if key in item
|
||||||
|
}
|
||||||
|
for item in review_state.get("issue_decisions", [])
|
||||||
|
]
|
||||||
|
result["review_send"] = review_state
|
||||||
|
except CampaignMailProfileBoundaryError:
|
||||||
|
pass
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def campaign_editor_state_for_edit(value: Any) -> dict[str, Any]:
|
||||||
|
"""Copy only client-owned safe metadata into a new editable version."""
|
||||||
|
|
||||||
|
state = public_campaign_editor_state(value)
|
||||||
|
state.pop("review_send", None)
|
||||||
|
return state
|
||||||
|
|
||||||
|
|
||||||
|
def _is_valid_reviewed_message_key(value: Any) -> bool:
|
||||||
|
return isinstance(value, str) and bool(value.strip()) and len(value) <= 512
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_reviewed_message_keys(value: Any) -> list[str]:
|
||||||
|
if not isinstance(value, list) or len(value) > 100_000:
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign reviewed message keys are invalid"
|
||||||
|
)
|
||||||
|
if any(not _is_valid_reviewed_message_key(key) for key in value):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign reviewed message keys are invalid"
|
||||||
|
)
|
||||||
|
return list(dict.fromkeys(key.strip() for key in value))
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_review_actor(value: Any) -> str | None:
|
||||||
|
if value is not None and (not isinstance(value, str) or len(value) > 256):
|
||||||
|
raise CampaignMailProfileBoundaryError("Campaign review actor is invalid")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_issue_decisions(value: Any) -> list[dict[str, Any]]:
|
||||||
|
if not isinstance(value, list) or len(value) > 100_000:
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review issue decisions are invalid"
|
||||||
|
)
|
||||||
|
result: list[dict[str, Any]] = []
|
||||||
|
allowed_keys = {
|
||||||
|
"job_id",
|
||||||
|
"review_key",
|
||||||
|
"decision",
|
||||||
|
"reason",
|
||||||
|
"actor_user_id",
|
||||||
|
"decided_at",
|
||||||
|
"build_token",
|
||||||
|
"message_sha256",
|
||||||
|
"issue_fingerprint",
|
||||||
|
"issue_codes",
|
||||||
|
}
|
||||||
|
for item in value:
|
||||||
|
if not isinstance(item, dict) or any(
|
||||||
|
key not in allowed_keys for key in item
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review issue decision is invalid"
|
||||||
|
)
|
||||||
|
normalized = {
|
||||||
|
"job_id": _validated_required_string(
|
||||||
|
item.get("job_id"),
|
||||||
|
max_length=36,
|
||||||
|
error="Campaign review decision job is invalid",
|
||||||
|
),
|
||||||
|
"review_key": _validated_required_string(
|
||||||
|
item.get("review_key"),
|
||||||
|
max_length=512,
|
||||||
|
error="Campaign review decision key is invalid",
|
||||||
|
),
|
||||||
|
"decision": _validated_required_string(
|
||||||
|
item.get("decision"),
|
||||||
|
max_length=30,
|
||||||
|
error="Campaign review decision is invalid",
|
||||||
|
),
|
||||||
|
"reason": item.get("reason"),
|
||||||
|
"actor_user_id": _validated_review_actor(
|
||||||
|
item.get("actor_user_id")
|
||||||
|
),
|
||||||
|
"decided_at": _validated_required_string(
|
||||||
|
item.get("decided_at"),
|
||||||
|
max_length=128,
|
||||||
|
error="Campaign review decision timestamp is invalid",
|
||||||
|
),
|
||||||
|
"build_token": _validated_required_string(
|
||||||
|
item.get("build_token"),
|
||||||
|
max_length=256,
|
||||||
|
error="Campaign review decision build token is invalid",
|
||||||
|
),
|
||||||
|
"message_sha256": item.get("message_sha256"),
|
||||||
|
"issue_fingerprint": _validated_required_string(
|
||||||
|
item.get("issue_fingerprint"),
|
||||||
|
max_length=64,
|
||||||
|
error="Campaign review issue fingerprint is invalid",
|
||||||
|
),
|
||||||
|
"issue_codes": item.get("issue_codes"),
|
||||||
|
}
|
||||||
|
if normalized["decision"] != "accept":
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review decision outcome is invalid"
|
||||||
|
)
|
||||||
|
if normalized["reason"] is not None and (
|
||||||
|
not isinstance(normalized["reason"], str)
|
||||||
|
or len(normalized["reason"]) > 4_000
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review decision reason is invalid"
|
||||||
|
)
|
||||||
|
if normalized["message_sha256"] is not None and (
|
||||||
|
not isinstance(normalized["message_sha256"], str)
|
||||||
|
or len(normalized["message_sha256"]) > 64
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review decision message hash is invalid"
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
not isinstance(normalized["issue_codes"], list)
|
||||||
|
or len(normalized["issue_codes"]) > 100
|
||||||
|
or any(
|
||||||
|
not isinstance(code, str) or not code or len(code) > 100
|
||||||
|
for code in normalized["issue_codes"]
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review decision issue codes are invalid"
|
||||||
|
)
|
||||||
|
result.append(normalized)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_server_review_state(value: Any) -> dict[str, Any]:
|
||||||
|
if not isinstance(value, dict) or any(
|
||||||
|
key not in CAMPAIGN_REVIEW_STATE_KEYS for key in value
|
||||||
|
):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review editor state is invalid"
|
||||||
|
)
|
||||||
|
build_token = value.get("build_token")
|
||||||
|
inspected = value.get("inspection_complete")
|
||||||
|
keys = value.get("reviewed_message_keys", [])
|
||||||
|
issue_decisions = value.get("issue_decisions", [])
|
||||||
|
updated_at = value.get("updated_at")
|
||||||
|
updated_by = value.get("updated_by_user_id")
|
||||||
|
validated_build_token = _validated_required_string(
|
||||||
|
build_token,
|
||||||
|
max_length=256,
|
||||||
|
error="Campaign review build token is invalid",
|
||||||
|
)
|
||||||
|
if not isinstance(inspected, bool):
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign review completion state is invalid"
|
||||||
|
)
|
||||||
|
validated_keys = _validated_reviewed_message_keys(keys)
|
||||||
|
validated_decisions = _validated_issue_decisions(issue_decisions)
|
||||||
|
validated_updated_at = _validated_required_string(
|
||||||
|
updated_at,
|
||||||
|
max_length=128,
|
||||||
|
error="Campaign review timestamp is invalid",
|
||||||
|
)
|
||||||
|
validated_updated_by = _validated_review_actor(updated_by)
|
||||||
|
return {
|
||||||
|
"build_token": validated_build_token,
|
||||||
|
"inspection_complete": inspected,
|
||||||
|
"reviewed_message_keys": validated_keys,
|
||||||
|
"issue_decisions": validated_decisions,
|
||||||
|
"updated_at": validated_updated_at,
|
||||||
|
"updated_by_user_id": validated_updated_by,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def campaign_mail_profile_id(raw_json: dict[str, Any] | None) -> str | None:
|
||||||
|
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||||
|
if not isinstance(server, dict):
|
||||||
|
return None
|
||||||
|
value = server.get("mail_profile_id")
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return None
|
||||||
|
normalized = value.strip()
|
||||||
|
return normalized or None
|
||||||
|
|
||||||
|
|
||||||
|
def campaign_mail_resource_ids(
|
||||||
|
raw_json: dict[str, Any] | None,
|
||||||
|
) -> dict[str, str | None]:
|
||||||
|
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||||
|
if not isinstance(server, dict):
|
||||||
|
return {
|
||||||
|
"mail_profile_id": None,
|
||||||
|
"smtp_server_id": None,
|
||||||
|
"smtp_credential_id": None,
|
||||||
|
"imap_server_id": None,
|
||||||
|
"imap_credential_id": None,
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
key: (
|
||||||
|
value.strip()
|
||||||
|
if isinstance((value := server.get(key)), str) and value.strip()
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
for key in CAMPAIGN_MAIL_SERVER_KEYS
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def campaign_mail_profile_boundary_violations(raw_json: dict[str, Any] | None) -> tuple[str, ...]:
|
||||||
|
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||||
|
if not isinstance(server, dict):
|
||||||
|
return ()
|
||||||
|
|
||||||
|
violations = [f"/server/{key}" for key in sorted(server) if key not in CAMPAIGN_MAIL_SERVER_KEYS]
|
||||||
|
for key in CAMPAIGN_MAIL_SERVER_KEYS:
|
||||||
|
if key not in server:
|
||||||
|
continue
|
||||||
|
value = server[key]
|
||||||
|
if not isinstance(value, str) or not value.strip():
|
||||||
|
violations.append(f"/server/{key}")
|
||||||
|
references = campaign_mail_resource_ids(raw_json)
|
||||||
|
if references["mail_profile_id"] is None and any(
|
||||||
|
references[key]
|
||||||
|
for key in references
|
||||||
|
if key != "mail_profile_id"
|
||||||
|
):
|
||||||
|
violations.append("/server/mail_profile_id")
|
||||||
|
for protocol in ("smtp", "imap"):
|
||||||
|
if (
|
||||||
|
references[f"{protocol}_credential_id"]
|
||||||
|
and not references[f"{protocol}_server_id"]
|
||||||
|
):
|
||||||
|
violations.append(f"/server/{protocol}_server_id")
|
||||||
|
return tuple(violations)
|
||||||
|
|
||||||
|
|
||||||
|
def assert_campaign_uses_mail_profile_reference(
|
||||||
|
raw_json: dict[str, Any] | None,
|
||||||
|
*,
|
||||||
|
require_profile: bool = False,
|
||||||
|
) -> None:
|
||||||
|
violations = campaign_mail_profile_boundary_violations(raw_json)
|
||||||
|
if violations:
|
||||||
|
fields = ", ".join(violations)
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign JSON may only reference Mail-owned profiles, servers, and credentials; "
|
||||||
|
f"remove campaign-local SMTP/IMAP settings or invalid references ({fields}), "
|
||||||
|
"select authorized Mail resources, and save a new campaign version."
|
||||||
|
)
|
||||||
|
if require_profile and campaign_mail_profile_id(raw_json) is None:
|
||||||
|
raise CampaignMailProfileBoundaryError(
|
||||||
|
"Campaign delivery requires server.mail_profile_id. Select an authorized, active "
|
||||||
|
"profile from the Mail module and validate the campaign again."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def public_campaign_mail_server(raw_json: dict[str, Any] | None) -> dict[str, str]:
|
||||||
|
"""Return the complete public/persisted Campaign-to-Mail contract."""
|
||||||
|
|
||||||
|
return {
|
||||||
|
key: value
|
||||||
|
for key, value in campaign_mail_resource_ids(raw_json).items()
|
||||||
|
if value
|
||||||
|
}
|
||||||
@@ -6,15 +6,6 @@ from typing import Any, Literal
|
|||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||||
|
|
||||||
from govoplan_core.mail.config import (
|
|
||||||
ImapConfig,
|
|
||||||
ImapServerConfig,
|
|
||||||
SmtpConfig,
|
|
||||||
SmtpServerConfig,
|
|
||||||
TransportCredentials,
|
|
||||||
normalize_split_transport_credentials,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class StrictModel(BaseModel):
|
class StrictModel(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid", populate_by_name=True)
|
model_config = ConfigDict(extra="forbid", populate_by_name=True)
|
||||||
@@ -31,7 +22,9 @@ class FieldType(StrEnum):
|
|||||||
INTEGER = "integer"
|
INTEGER = "integer"
|
||||||
DOUBLE = "double"
|
DOUBLE = "double"
|
||||||
DATE = "date"
|
DATE = "date"
|
||||||
PASSWORD = "password"
|
PASSWORD = "password" # noqa: S105 # nosec B105 - field type vocabulary.
|
||||||
|
ORGANIZATION_UNIT = "organization_unit"
|
||||||
|
ORGANIZATION_FUNCTION = "organization_function"
|
||||||
|
|
||||||
|
|
||||||
class RecipientType(StrEnum):
|
class RecipientType(StrEnum):
|
||||||
@@ -98,6 +91,105 @@ class BuildStatus(StrEnum):
|
|||||||
class SendStatus(StrEnum):
|
class SendStatus(StrEnum):
|
||||||
DRAFT = "draft"
|
DRAFT = "draft"
|
||||||
QUEUED = "queued"
|
QUEUED = "queued"
|
||||||
|
SKIPPED = "skipped"
|
||||||
|
|
||||||
|
|
||||||
|
class DeliveryChannelPolicy(StrEnum):
|
||||||
|
MAIL = "mail"
|
||||||
|
POSTBOX = "postbox"
|
||||||
|
MAIL_AND_POSTBOX = "mail_and_postbox"
|
||||||
|
MAIL_THEN_POSTBOX = "mail_then_postbox"
|
||||||
|
POSTBOX_THEN_MAIL = "postbox_then_mail"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def uses_mail(self) -> bool:
|
||||||
|
return self in {
|
||||||
|
DeliveryChannelPolicy.MAIL,
|
||||||
|
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||||
|
}
|
||||||
|
|
||||||
|
@property
|
||||||
|
def uses_postbox(self) -> bool:
|
||||||
|
return self != DeliveryChannelPolicy.MAIL
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxTargetMode(StrEnum):
|
||||||
|
DIRECT = "direct"
|
||||||
|
DERIVED = "derived"
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxTargetMatch(StrEnum):
|
||||||
|
ID = "id"
|
||||||
|
SLUG = "slug"
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxTargetConfig(StrictModel):
|
||||||
|
id: str = Field(min_length=1, max_length=120)
|
||||||
|
mode: PostboxTargetMode = PostboxTargetMode.DIRECT
|
||||||
|
label: str | None = Field(default=None, max_length=500)
|
||||||
|
|
||||||
|
postbox_id: str | None = Field(default=None, max_length=36)
|
||||||
|
address_key: str | None = Field(default=None, max_length=500)
|
||||||
|
|
||||||
|
template_id: str | None = Field(default=None, max_length=36)
|
||||||
|
organization_unit_id: str | None = Field(default=None, max_length=36)
|
||||||
|
organization_unit_field: str | None = Field(default=None, max_length=255)
|
||||||
|
organization_unit_match: PostboxTargetMatch = PostboxTargetMatch.ID
|
||||||
|
function_id: str | None = Field(default=None, max_length=36)
|
||||||
|
function_field: str | None = Field(default=None, max_length=255)
|
||||||
|
function_match: PostboxTargetMatch = PostboxTargetMatch.ID
|
||||||
|
context_key: str | None = Field(default=None, max_length=255)
|
||||||
|
context_field: str | None = Field(default=None, max_length=255)
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def validate_target_shape(self) -> "PostboxTargetConfig":
|
||||||
|
direct_values = [self.postbox_id, self.address_key]
|
||||||
|
if self.mode == PostboxTargetMode.DIRECT:
|
||||||
|
if sum(bool(value) for value in direct_values) != 1:
|
||||||
|
raise ValueError(
|
||||||
|
"A direct Postbox target requires exactly one postbox_id "
|
||||||
|
"or address_key."
|
||||||
|
)
|
||||||
|
if any(
|
||||||
|
(
|
||||||
|
self.template_id,
|
||||||
|
self.organization_unit_id,
|
||||||
|
self.organization_unit_field,
|
||||||
|
self.function_id,
|
||||||
|
self.function_field,
|
||||||
|
self.context_key,
|
||||||
|
self.context_field,
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise ValueError(
|
||||||
|
"A direct Postbox target cannot contain derived target fields."
|
||||||
|
)
|
||||||
|
return self
|
||||||
|
|
||||||
|
if any(direct_values):
|
||||||
|
raise ValueError(
|
||||||
|
"A derived Postbox target cannot contain postbox_id or address_key."
|
||||||
|
)
|
||||||
|
if not self.template_id:
|
||||||
|
raise ValueError("A derived Postbox target requires template_id.")
|
||||||
|
if bool(self.organization_unit_id) == bool(self.organization_unit_field):
|
||||||
|
raise ValueError(
|
||||||
|
"A derived Postbox target requires exactly one fixed or "
|
||||||
|
"field-derived organization unit."
|
||||||
|
)
|
||||||
|
if bool(self.function_id) == bool(self.function_field):
|
||||||
|
raise ValueError(
|
||||||
|
"A derived Postbox target requires exactly one fixed or "
|
||||||
|
"field-derived function."
|
||||||
|
)
|
||||||
|
if self.context_key and self.context_field:
|
||||||
|
raise ValueError(
|
||||||
|
"A derived Postbox target may use a fixed context or a context "
|
||||||
|
"field, not both."
|
||||||
|
)
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
class CampaignMeta(StrictModel):
|
class CampaignMeta(StrictModel):
|
||||||
@@ -115,37 +207,18 @@ class FieldDefinition(StrictModel):
|
|||||||
can_override: bool = True
|
can_override: bool = True
|
||||||
|
|
||||||
|
|
||||||
class MailServerCredentials(StrictModel):
|
class MailProfileCapabilities(StrictModel):
|
||||||
smtp: TransportCredentials = Field(default_factory=TransportCredentials)
|
smtp_available: bool = False
|
||||||
imap: TransportCredentials = Field(default_factory=TransportCredentials)
|
imap_available: bool = False
|
||||||
|
|
||||||
|
|
||||||
class ServerConfig(StrictModel):
|
class ServerConfig(StrictModel):
|
||||||
mail_profile_id: str | None = None
|
mail_profile_id: str | None = None
|
||||||
inherit_smtp_credentials: bool = True
|
smtp_server_id: str | None = None
|
||||||
inherit_imap_credentials: bool = True
|
smtp_credential_id: str | None = None
|
||||||
smtp: SmtpServerConfig | None = None
|
imap_server_id: str | None = None
|
||||||
imap: ImapServerConfig | None = None
|
imap_credential_id: str | None = None
|
||||||
credentials: MailServerCredentials = Field(default_factory=MailServerCredentials)
|
profile_capabilities: MailProfileCapabilities = Field(default_factory=MailProfileCapabilities)
|
||||||
|
|
||||||
@model_validator(mode="before")
|
|
||||||
@classmethod
|
|
||||||
def normalize_legacy_credentials(cls, value: Any) -> Any:
|
|
||||||
return normalize_split_transport_credentials(value)
|
|
||||||
|
|
||||||
def runtime_smtp_config(self) -> SmtpConfig | None:
|
|
||||||
if self.smtp is None:
|
|
||||||
return None
|
|
||||||
payload = self.smtp.model_dump(mode="json")
|
|
||||||
payload.update(self.credentials.smtp.model_dump(mode="json", exclude_none=True))
|
|
||||||
return SmtpConfig.model_validate(payload)
|
|
||||||
|
|
||||||
def runtime_imap_config(self) -> ImapConfig | None:
|
|
||||||
if self.imap is None:
|
|
||||||
return None
|
|
||||||
payload = self.imap.model_dump(mode="json")
|
|
||||||
payload.update(self.credentials.imap.model_dump(mode="json", exclude_none=True))
|
|
||||||
return ImapConfig.model_validate(payload)
|
|
||||||
|
|
||||||
|
|
||||||
class RecipientConfig(StrictModel):
|
class RecipientConfig(StrictModel):
|
||||||
@@ -404,11 +477,20 @@ class AttachmentsConfig(StrictModel):
|
|||||||
base_paths: list[AttachmentBasePathConfig] = Field(default_factory=list)
|
base_paths: list[AttachmentBasePathConfig] = Field(default_factory=list)
|
||||||
allow_individual: bool = False
|
allow_individual: bool = False
|
||||||
send_without_attachments: bool = True
|
send_without_attachments: bool = True
|
||||||
|
send_without_attachments_behavior: Behavior | None = None
|
||||||
zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig)
|
zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig)
|
||||||
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
|
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
|
||||||
missing_behavior: Behavior = Behavior.ASK
|
missing_behavior: Behavior = Behavior.WARN
|
||||||
ambiguous_behavior: Behavior = Behavior.ASK
|
ambiguous_behavior: Behavior = Behavior.ASK
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def normalize_send_without_attachments_behavior(self) -> "AttachmentsConfig":
|
||||||
|
if self.send_without_attachments_behavior is None:
|
||||||
|
self.send_without_attachments_behavior = Behavior.CONTINUE if self.send_without_attachments else Behavior.BLOCK
|
||||||
|
else:
|
||||||
|
self.send_without_attachments = self.send_without_attachments_behavior != Behavior.BLOCK
|
||||||
|
return self
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def individual_base_path_values(self) -> set[str]:
|
def individual_base_path_values(self) -> set[str]:
|
||||||
return {base_path.path for base_path in self.base_paths if base_path.allow_individual}
|
return {base_path.path for base_path in self.base_paths if base_path.allow_individual}
|
||||||
@@ -468,6 +550,13 @@ class EntryConfig(StrictModel):
|
|||||||
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
|
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
merge_disposition_notification_to: bool = True
|
merge_disposition_notification_to: bool = True
|
||||||
|
|
||||||
|
channel_policy: DeliveryChannelPolicy | None = None
|
||||||
|
postbox_targets: list[PostboxTargetConfig] = Field(
|
||||||
|
default_factory=list,
|
||||||
|
max_length=50,
|
||||||
|
)
|
||||||
|
merge_postbox_targets: bool = True
|
||||||
|
|
||||||
attachments: list[AttachmentConfig] = Field(default_factory=list)
|
attachments: list[AttachmentConfig] = Field(default_factory=list)
|
||||||
combine_attachments: bool = True
|
combine_attachments: bool = True
|
||||||
|
|
||||||
@@ -549,7 +638,7 @@ class EntriesConfig(StrictModel):
|
|||||||
|
|
||||||
|
|
||||||
class ValidationPolicy(StrictModel):
|
class ValidationPolicy(StrictModel):
|
||||||
missing_required_attachment: Behavior = Behavior.ASK
|
missing_required_attachment: Behavior = Behavior.BLOCK
|
||||||
missing_optional_attachment: Behavior = Behavior.WARN
|
missing_optional_attachment: Behavior = Behavior.WARN
|
||||||
ambiguous_attachment_match: Behavior = Behavior.ASK
|
ambiguous_attachment_match: Behavior = Behavior.ASK
|
||||||
ignore_empty_fields: bool = False
|
ignore_empty_fields: bool = False
|
||||||
@@ -581,7 +670,17 @@ class RetryConfig(StrictModel):
|
|||||||
return values
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxDeliveryConfig(StrictModel):
|
||||||
|
targets: list[PostboxTargetConfig] = Field(default_factory=list, max_length=50)
|
||||||
|
classification: str = Field(default="internal", min_length=1, max_length=50)
|
||||||
|
unresolved_target: Behavior = Behavior.BLOCK
|
||||||
|
vacant_target: Behavior = Behavior.WARN
|
||||||
|
duplicate_target: Behavior = Behavior.WARN
|
||||||
|
|
||||||
|
|
||||||
class DeliveryConfig(StrictModel):
|
class DeliveryConfig(StrictModel):
|
||||||
|
channel_policy: DeliveryChannelPolicy = DeliveryChannelPolicy.MAIL
|
||||||
|
postbox: PostboxDeliveryConfig = Field(default_factory=PostboxDeliveryConfig)
|
||||||
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
|
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
|
||||||
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
|
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
|
||||||
retry: RetryConfig = Field(default_factory=RetryConfig)
|
retry: RetryConfig = Field(default_factory=RetryConfig)
|
||||||
@@ -624,3 +723,23 @@ class CampaignConfig(StrictModel):
|
|||||||
if path.is_absolute():
|
if path.is_absolute():
|
||||||
return path
|
return path
|
||||||
return (campaign_file.parent / path).resolve()
|
return (campaign_file.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
|
def effective_delivery_channel_policy(
|
||||||
|
config: CampaignConfig,
|
||||||
|
entry: EntryConfig,
|
||||||
|
) -> DeliveryChannelPolicy:
|
||||||
|
return entry.channel_policy or config.delivery.channel_policy
|
||||||
|
|
||||||
|
|
||||||
|
def effective_postbox_targets(
|
||||||
|
config: CampaignConfig,
|
||||||
|
entry: EntryConfig,
|
||||||
|
) -> list[PostboxTargetConfig]:
|
||||||
|
global_targets = list(config.delivery.postbox.targets)
|
||||||
|
individual_targets = list(entry.postbox_targets)
|
||||||
|
if not individual_targets:
|
||||||
|
return global_targets
|
||||||
|
if entry.merge_postbox_targets:
|
||||||
|
return [*global_targets, *individual_targets]
|
||||||
|
return individual_targets
|
||||||
|
|||||||
321
src/govoplan_campaign/backend/campaign/postbox_targets.py
Normal file
321
src/govoplan_campaign/backend/campaign/postbox_targets.py
Normal file
@@ -0,0 +1,321 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import asdict
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_core.core.postbox import (
|
||||||
|
PostboxDeliveryCatalogRef,
|
||||||
|
PostboxDirectoryEntryRef,
|
||||||
|
PostboxTargetRef,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.campaign.field_values import (
|
||||||
|
effective_entry_field_values,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.campaign.models import (
|
||||||
|
Behavior,
|
||||||
|
CampaignConfig,
|
||||||
|
EntryConfig,
|
||||||
|
PostboxTargetConfig,
|
||||||
|
PostboxTargetMatch,
|
||||||
|
PostboxTargetMode,
|
||||||
|
effective_postbox_targets,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import postbox_integration
|
||||||
|
from govoplan_campaign.backend.messages.models import (
|
||||||
|
MessageIssue,
|
||||||
|
MessageValidationStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_behavior(
|
||||||
|
current: MessageValidationStatus,
|
||||||
|
behavior: Behavior,
|
||||||
|
) -> MessageValidationStatus:
|
||||||
|
if behavior == Behavior.BLOCK:
|
||||||
|
return MessageValidationStatus.BLOCKED
|
||||||
|
if behavior == Behavior.DROP:
|
||||||
|
return MessageValidationStatus.EXCLUDED
|
||||||
|
if behavior == Behavior.ASK and current not in {
|
||||||
|
MessageValidationStatus.BLOCKED,
|
||||||
|
MessageValidationStatus.EXCLUDED,
|
||||||
|
}:
|
||||||
|
return MessageValidationStatus.NEEDS_REVIEW
|
||||||
|
if behavior == Behavior.WARN and current == MessageValidationStatus.READY:
|
||||||
|
return MessageValidationStatus.WARNING
|
||||||
|
return current
|
||||||
|
|
||||||
|
|
||||||
|
def _issue(
|
||||||
|
*,
|
||||||
|
code: str,
|
||||||
|
message: str,
|
||||||
|
behavior: Behavior,
|
||||||
|
) -> MessageIssue:
|
||||||
|
return MessageIssue(
|
||||||
|
severity="error" if behavior == Behavior.BLOCK else "warning",
|
||||||
|
code=code,
|
||||||
|
message=message,
|
||||||
|
behavior=behavior.value,
|
||||||
|
source="postbox",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _field_value(
|
||||||
|
values: dict[str, Any],
|
||||||
|
field_name: str | None,
|
||||||
|
) -> str | None:
|
||||||
|
if not field_name:
|
||||||
|
return None
|
||||||
|
value = values.get(field_name)
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
text = str(value).strip()
|
||||||
|
return text or None
|
||||||
|
|
||||||
|
|
||||||
|
def _match_unit(
|
||||||
|
catalog: PostboxDeliveryCatalogRef,
|
||||||
|
value: str | None,
|
||||||
|
match: PostboxTargetMatch,
|
||||||
|
):
|
||||||
|
if not value:
|
||||||
|
return None
|
||||||
|
return next(
|
||||||
|
(
|
||||||
|
unit
|
||||||
|
for unit in catalog.organization_units
|
||||||
|
if (unit.id if match == PostboxTargetMatch.ID else unit.slug) == value
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _match_function(unit, value: str | None, match: PostboxTargetMatch):
|
||||||
|
if unit is None or not value:
|
||||||
|
return None
|
||||||
|
return next(
|
||||||
|
(
|
||||||
|
function
|
||||||
|
for function in unit.functions
|
||||||
|
if (
|
||||||
|
function.id
|
||||||
|
if match == PostboxTargetMatch.ID
|
||||||
|
else function.slug
|
||||||
|
)
|
||||||
|
== value
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _target_ref(
|
||||||
|
target: PostboxTargetConfig,
|
||||||
|
*,
|
||||||
|
values: dict[str, Any],
|
||||||
|
catalog: PostboxDeliveryCatalogRef,
|
||||||
|
) -> tuple[PostboxTargetRef | None, str | None]:
|
||||||
|
if target.mode == PostboxTargetMode.DIRECT:
|
||||||
|
return (
|
||||||
|
PostboxTargetRef(
|
||||||
|
postbox_id=target.postbox_id,
|
||||||
|
address_key=target.address_key,
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
unit_value = target.organization_unit_id or _field_value(
|
||||||
|
values,
|
||||||
|
target.organization_unit_field,
|
||||||
|
)
|
||||||
|
unit_match = (
|
||||||
|
PostboxTargetMatch.ID
|
||||||
|
if target.organization_unit_id
|
||||||
|
else target.organization_unit_match
|
||||||
|
)
|
||||||
|
unit = _match_unit(catalog, unit_value, unit_match)
|
||||||
|
if unit is None:
|
||||||
|
return None, (
|
||||||
|
f"Organization unit {unit_value!r} could not be resolved by "
|
||||||
|
f"{unit_match.value}."
|
||||||
|
)
|
||||||
|
|
||||||
|
function_value = target.function_id or _field_value(
|
||||||
|
values,
|
||||||
|
target.function_field,
|
||||||
|
)
|
||||||
|
function_match = (
|
||||||
|
PostboxTargetMatch.ID
|
||||||
|
if target.function_id
|
||||||
|
else target.function_match
|
||||||
|
)
|
||||||
|
function = _match_function(unit, function_value, function_match)
|
||||||
|
if function is None:
|
||||||
|
return None, (
|
||||||
|
f"Organization function {function_value!r} could not be resolved "
|
||||||
|
f"inside {unit.name!r} by {function_match.value}."
|
||||||
|
)
|
||||||
|
|
||||||
|
context_key = target.context_key or _field_value(
|
||||||
|
values,
|
||||||
|
target.context_field,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
PostboxTargetRef(
|
||||||
|
template_id=target.template_id,
|
||||||
|
organization_unit_id=unit.id,
|
||||||
|
function_id=function.id,
|
||||||
|
context_key=context_key,
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolved_target_payload(
|
||||||
|
target: PostboxTargetConfig,
|
||||||
|
entry: PostboxDirectoryEntryRef,
|
||||||
|
*,
|
||||||
|
position: int,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"target_id": target.id,
|
||||||
|
"position": position,
|
||||||
|
"mode": target.mode.value,
|
||||||
|
"requested": target.model_dump(mode="json", exclude_none=True),
|
||||||
|
"postbox_id": entry.id,
|
||||||
|
"address": entry.address,
|
||||||
|
"address_key": entry.address_key,
|
||||||
|
"name": entry.name,
|
||||||
|
"status": entry.status,
|
||||||
|
"classification": entry.classification,
|
||||||
|
"organization_unit_id": entry.organization_unit_id,
|
||||||
|
"organization_unit_name": entry.organization_unit_name,
|
||||||
|
"function_id": entry.function_id,
|
||||||
|
"function_name": entry.function_name,
|
||||||
|
"context_key": entry.context_key,
|
||||||
|
"template_revision_id": entry.template_revision_id,
|
||||||
|
"holder_count": entry.holder_count,
|
||||||
|
"vacant": entry.vacant,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_entry_postbox_targets(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
config: CampaignConfig,
|
||||||
|
entry: EntryConfig,
|
||||||
|
validation_status: MessageValidationStatus,
|
||||||
|
materialize: bool,
|
||||||
|
) -> tuple[
|
||||||
|
list[dict[str, Any]],
|
||||||
|
list[MessageIssue],
|
||||||
|
MessageValidationStatus,
|
||||||
|
]:
|
||||||
|
integration = postbox_integration()
|
||||||
|
policy = config.delivery.postbox
|
||||||
|
targets = effective_postbox_targets(config, entry)
|
||||||
|
if not targets:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_missing",
|
||||||
|
message="Postbox delivery requires at least one target.",
|
||||||
|
behavior=policy.unresolved_target,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
[],
|
||||||
|
[issue],
|
||||||
|
_apply_behavior(validation_status, policy.unresolved_target),
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
catalog = integration.delivery_catalog(session, tenant_id=tenant_id)
|
||||||
|
except Exception as exc:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_unavailable",
|
||||||
|
message=str(exc),
|
||||||
|
behavior=Behavior.BLOCK,
|
||||||
|
)
|
||||||
|
return [], [issue], MessageValidationStatus.BLOCKED
|
||||||
|
|
||||||
|
values = effective_entry_field_values(config, entry)
|
||||||
|
resolved: list[dict[str, Any]] = []
|
||||||
|
issues: list[MessageIssue] = []
|
||||||
|
status = validation_status
|
||||||
|
seen_postbox_ids: set[str] = set()
|
||||||
|
for position, target in enumerate(targets):
|
||||||
|
target_ref, resolution_error = _target_ref(
|
||||||
|
target,
|
||||||
|
values=values,
|
||||||
|
catalog=catalog,
|
||||||
|
)
|
||||||
|
if target_ref is None:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_unresolved",
|
||||||
|
message=resolution_error or "Postbox target could not be resolved.",
|
||||||
|
behavior=policy.unresolved_target,
|
||||||
|
)
|
||||||
|
issues.append(issue)
|
||||||
|
status = _apply_behavior(status, policy.unresolved_target)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
entry_ref = integration.resolve_postbox(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
target=target_ref,
|
||||||
|
materialize=materialize,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_unresolved",
|
||||||
|
message=f"Postbox target {target.id!r} could not be resolved: {exc}",
|
||||||
|
behavior=policy.unresolved_target,
|
||||||
|
)
|
||||||
|
issues.append(issue)
|
||||||
|
status = _apply_behavior(status, policy.unresolved_target)
|
||||||
|
continue
|
||||||
|
if entry_ref is None:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_unresolved",
|
||||||
|
message=f"Postbox target {target.id!r} does not exist.",
|
||||||
|
behavior=policy.unresolved_target,
|
||||||
|
)
|
||||||
|
issues.append(issue)
|
||||||
|
status = _apply_behavior(status, policy.unresolved_target)
|
||||||
|
continue
|
||||||
|
if entry_ref.id in seen_postbox_ids:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_duplicate",
|
||||||
|
message=(
|
||||||
|
f"Postbox {entry_ref.address!r} is selected more than once; "
|
||||||
|
"it will receive one message."
|
||||||
|
),
|
||||||
|
behavior=policy.duplicate_target,
|
||||||
|
)
|
||||||
|
issues.append(issue)
|
||||||
|
status = _apply_behavior(status, policy.duplicate_target)
|
||||||
|
continue
|
||||||
|
seen_postbox_ids.add(entry_ref.id)
|
||||||
|
resolved.append(
|
||||||
|
_resolved_target_payload(
|
||||||
|
target,
|
||||||
|
entry_ref,
|
||||||
|
position=position,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if entry_ref.vacant:
|
||||||
|
issue = _issue(
|
||||||
|
code="postbox_target_vacant",
|
||||||
|
message=(
|
||||||
|
f"Postbox {entry_ref.address!r} currently has no function "
|
||||||
|
"holder."
|
||||||
|
),
|
||||||
|
behavior=policy.vacant_target,
|
||||||
|
)
|
||||||
|
issues.append(issue)
|
||||||
|
status = _apply_behavior(status, policy.vacant_target)
|
||||||
|
return resolved, issues, status
|
||||||
|
|
||||||
|
|
||||||
|
def delivery_catalog_payload(catalog: PostboxDeliveryCatalogRef) -> dict[str, Any]:
|
||||||
|
return asdict(catalog)
|
||||||
@@ -8,8 +8,24 @@ from typing import Iterable
|
|||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from .addressing import effective_address_lists
|
||||||
from .field_values import ignored_entry_field_overrides
|
from .field_values import ignored_entry_field_overrides
|
||||||
from .models import AttachmentConfig, CampaignConfig, EntryConfig, FieldType, SourceType, ZipArchiveConfig, ZipPasswordMode, ZipPasswordScope, ZipRuleMode
|
from .models import (
|
||||||
|
AttachmentConfig,
|
||||||
|
CampaignConfig,
|
||||||
|
DeliveryChannelPolicy,
|
||||||
|
EntryConfig,
|
||||||
|
FieldType,
|
||||||
|
PostboxTargetConfig,
|
||||||
|
SourceType,
|
||||||
|
ZipArchiveConfig,
|
||||||
|
ZipPasswordMode,
|
||||||
|
ZipPasswordScope,
|
||||||
|
ZipRuleMode,
|
||||||
|
effective_delivery_channel_policy,
|
||||||
|
effective_postbox_targets,
|
||||||
|
)
|
||||||
|
from ..attachments.resolver import resolve_campaign_attachments
|
||||||
|
|
||||||
|
|
||||||
class Severity(StrEnum):
|
class Severity(StrEnum):
|
||||||
@@ -88,6 +104,8 @@ def _mapping_target_known(target: str, field_names: set[str]) -> bool:
|
|||||||
"merge_reply_to",
|
"merge_reply_to",
|
||||||
"merge_bounce_to",
|
"merge_bounce_to",
|
||||||
"merge_disposition_notification_to",
|
"merge_disposition_notification_to",
|
||||||
|
"merge_postbox_targets",
|
||||||
|
"channel_policy",
|
||||||
"combine_to",
|
"combine_to",
|
||||||
"combine_cc",
|
"combine_cc",
|
||||||
"combine_bcc",
|
"combine_bcc",
|
||||||
@@ -335,54 +353,230 @@ def _global_value_issues(config: CampaignConfig, declared_names: set[str]) -> li
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
def _active_delivery_entries(config: CampaignConfig) -> list[EntryConfig]:
|
||||||
|
if config.entries.is_inline:
|
||||||
|
return [
|
||||||
|
entry
|
||||||
|
for entry in (config.entries.inline or [])
|
||||||
|
if entry.active
|
||||||
|
]
|
||||||
|
return [config.entries.defaults or EntryConfig()]
|
||||||
|
|
||||||
|
|
||||||
|
def _delivery_policies(config: CampaignConfig) -> set[DeliveryChannelPolicy]:
|
||||||
|
return {
|
||||||
|
effective_delivery_channel_policy(config, entry)
|
||||||
|
for entry in _active_delivery_entries(config)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _postbox_target_field_issues(
|
||||||
|
config: CampaignConfig,
|
||||||
|
target: PostboxTargetConfig,
|
||||||
|
path: str,
|
||||||
|
) -> list[SemanticIssue]:
|
||||||
|
definitions = {field.name: field for field in config.fields}
|
||||||
|
checks = (
|
||||||
|
(
|
||||||
|
target.organization_unit_field,
|
||||||
|
FieldType.ORGANIZATION_UNIT,
|
||||||
|
"organization unit",
|
||||||
|
"organization_unit_field",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
target.function_field,
|
||||||
|
FieldType.ORGANIZATION_FUNCTION,
|
||||||
|
"organization function",
|
||||||
|
"function_field",
|
||||||
|
),
|
||||||
|
(target.context_field, None, "context", "context_field"),
|
||||||
|
)
|
||||||
issues: list[SemanticIssue] = []
|
issues: list[SemanticIssue] = []
|
||||||
runtime_imap = config.server.runtime_imap_config()
|
for field_name, expected_type, label, key in checks:
|
||||||
if config.delivery.imap_append_sent.enabled:
|
if not field_name:
|
||||||
issues.extend(_imap_delivery_issues(runtime_imap))
|
continue
|
||||||
runtime_smtp = config.server.runtime_smtp_config()
|
definition = definitions.get(field_name)
|
||||||
if config.campaign.mode == "send" and not runtime_smtp:
|
if definition is None:
|
||||||
issues.append(
|
issues.append(
|
||||||
_issue(
|
_issue(
|
||||||
Severity.ERROR,
|
Severity.ERROR,
|
||||||
"missing_smtp_config",
|
"postbox_target_field_missing",
|
||||||
"campaign mode is 'send', but no server.smtp configuration is present",
|
f"Postbox {label} field {field_name!r} is not declared.",
|
||||||
"/server/smtp",
|
f"{path}/{key}",
|
||||||
|
)
|
||||||
)
|
)
|
||||||
)
|
elif expected_type is not None and definition.type != expected_type:
|
||||||
if runtime_smtp:
|
|
||||||
missing = [name for name in ["host", "port"] if getattr(runtime_smtp, name) in (None, "")]
|
|
||||||
if missing:
|
|
||||||
issues.append(
|
issues.append(
|
||||||
_issue(
|
_issue(
|
||||||
Severity.WARNING,
|
Severity.WARNING,
|
||||||
"incomplete_smtp_config",
|
"postbox_target_field_type",
|
||||||
"SMTP settings are present, but these settings are missing: " + ", ".join(missing),
|
(
|
||||||
"/server/smtp",
|
f"Postbox {label} field {field_name!r} should use "
|
||||||
|
f"field type {expected_type.value!r}."
|
||||||
|
),
|
||||||
|
f"{path}/{key}",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return issues
|
return issues
|
||||||
|
|
||||||
|
|
||||||
def _imap_delivery_issues(runtime_imap: object | None) -> list[SemanticIssue]:
|
def _postbox_delivery_issues(
|
||||||
if runtime_imap is None:
|
config: CampaignConfig,
|
||||||
|
*,
|
||||||
|
postbox_available: bool,
|
||||||
|
) -> list[SemanticIssue]:
|
||||||
|
issues: list[SemanticIssue] = []
|
||||||
|
policies = _delivery_policies(config)
|
||||||
|
if not any(policy.uses_postbox for policy in policies):
|
||||||
|
return issues
|
||||||
|
if not postbox_available:
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"postbox_unavailable",
|
||||||
|
(
|
||||||
|
"This campaign uses Postbox delivery, but the Postbox "
|
||||||
|
"module and its delivery directory are not active."
|
||||||
|
),
|
||||||
|
"/delivery/channel_policy",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for entry_index, entry in enumerate(_active_delivery_entries(config)):
|
||||||
|
policy = effective_delivery_channel_policy(config, entry)
|
||||||
|
if not policy.uses_postbox:
|
||||||
|
continue
|
||||||
|
targets = effective_postbox_targets(config, entry)
|
||||||
|
if not targets:
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"postbox_target_missing",
|
||||||
|
"Postbox delivery requires at least one target.",
|
||||||
|
f"/entries/inline/{entry_index}/postbox_targets",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
for target_index, target in enumerate(targets):
|
||||||
|
target_path = (
|
||||||
|
f"/entries/inline/{entry_index}/postbox_targets/"
|
||||||
|
f"{target_index}"
|
||||||
|
)
|
||||||
|
if target.id in seen_ids:
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"postbox_target_id_duplicate",
|
||||||
|
f"Postbox target id {target.id!r} is repeated.",
|
||||||
|
f"{target_path}/id",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
seen_ids.add(target.id)
|
||||||
|
issues.extend(
|
||||||
|
_postbox_target_field_issues(config, target, target_path)
|
||||||
|
)
|
||||||
|
return issues
|
||||||
|
|
||||||
|
|
||||||
|
def _delivery_issues(
|
||||||
|
config: CampaignConfig,
|
||||||
|
*,
|
||||||
|
postbox_available: bool,
|
||||||
|
) -> list[SemanticIssue]:
|
||||||
|
issues: list[SemanticIssue] = []
|
||||||
|
policies = _delivery_policies(config)
|
||||||
|
uses_mail = any(policy.uses_mail for policy in policies)
|
||||||
|
profile_id = (config.server.mail_profile_id or "").strip()
|
||||||
|
if (
|
||||||
|
(
|
||||||
|
config.campaign.mode == "send"
|
||||||
|
and uses_mail
|
||||||
|
or config.delivery.imap_append_sent.enabled
|
||||||
|
)
|
||||||
|
and not profile_id
|
||||||
|
):
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"missing_mail_profile",
|
||||||
|
"Select an authorized Mail-module profile; campaigns cannot store SMTP/IMAP settings or credentials.",
|
||||||
|
"/server/mail_profile_id",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
capabilities = config.server.profile_capabilities
|
||||||
|
if (
|
||||||
|
config.campaign.mode == "send"
|
||||||
|
and uses_mail
|
||||||
|
and profile_id
|
||||||
|
and not capabilities.smtp_available
|
||||||
|
):
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"mail_profile_without_smtp",
|
||||||
|
"campaign mode is 'send', but the selected Mail profile has no SMTP configuration",
|
||||||
|
"/server/mail_profile_id",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if config.delivery.imap_append_sent.enabled and profile_id and not capabilities.imap_available:
|
||||||
|
issues.append(
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"mail_profile_without_imap",
|
||||||
|
"IMAP append is enabled, but the selected Mail profile has no IMAP configuration",
|
||||||
|
"/server/mail_profile_id",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
issues.extend(
|
||||||
|
_postbox_delivery_issues(
|
||||||
|
config,
|
||||||
|
postbox_available=postbox_available,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return issues
|
||||||
|
|
||||||
|
|
||||||
|
def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||||
|
"""Require Campaign-owned sender data before a send-mode build."""
|
||||||
|
|
||||||
|
if (
|
||||||
|
config.campaign.mode != "send"
|
||||||
|
or not any(policy.uses_mail for policy in _delivery_policies(config))
|
||||||
|
):
|
||||||
|
return []
|
||||||
|
if config.entries.is_inline:
|
||||||
return [
|
return [
|
||||||
_issue(
|
_issue(
|
||||||
Severity.WARNING,
|
Severity.ERROR,
|
||||||
"delivery_imap_enabled_without_server_imap",
|
"missing_sender",
|
||||||
"delivery.imap_append_sent is enabled, but no server.imap configuration is present",
|
"No effective From address is configured; Campaign must resolve the sender before building.",
|
||||||
"/delivery/imap_append_sent/enabled",
|
f"/entries/inline/{index}/from",
|
||||||
|
)
|
||||||
|
for index, entry in enumerate(config.entries.inline or [])
|
||||||
|
if (
|
||||||
|
entry.active
|
||||||
|
and effective_delivery_channel_policy(config, entry).uses_mail
|
||||||
|
and not effective_address_lists(config, entry)["from"]
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
missing = [name for name in ["host", "port", "username", "password"] if getattr(runtime_imap, name) in (None, "")]
|
if config.recipients.from_:
|
||||||
if not missing:
|
return []
|
||||||
|
defaults = config.entries.defaults
|
||||||
|
mapping_can_supply_sender = bool(
|
||||||
|
config.recipients.allow_individual_from
|
||||||
|
and (
|
||||||
|
(defaults is not None and defaults.from_)
|
||||||
|
or "from.email" in (config.entries.mapping or {})
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if mapping_can_supply_sender:
|
||||||
return []
|
return []
|
||||||
return [
|
return [
|
||||||
_issue(
|
_issue(
|
||||||
Severity.ERROR,
|
Severity.ERROR,
|
||||||
"incomplete_imap_config",
|
"missing_sender",
|
||||||
"IMAP append is enabled, but these IMAP settings are missing: " + ", ".join(missing),
|
"Configure recipients.from, or allow and map an individual From address, before building a send-mode campaign.",
|
||||||
"/server/imap",
|
"/recipients/from",
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -507,6 +701,7 @@ def _entries_source_file_issues(config: CampaignConfig, campaign_path: Path, map
|
|||||||
def _file_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
|
def _file_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
|
||||||
issues: list[SemanticIssue] = []
|
issues: list[SemanticIssue] = []
|
||||||
issues.extend(_attachment_file_check_issues(config, campaign_path))
|
issues.extend(_attachment_file_check_issues(config, campaign_path))
|
||||||
|
issues.extend(_attachment_resolution_check_issues(config, campaign_path))
|
||||||
issues.extend(_template_source_file_issues(config, campaign_path))
|
issues.extend(_template_source_file_issues(config, campaign_path))
|
||||||
return issues
|
return issues
|
||||||
|
|
||||||
@@ -536,6 +731,37 @@ def _attachment_file_check_issues(config: CampaignConfig, campaign_path: Path) -
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_resolution_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
|
||||||
|
try:
|
||||||
|
report = resolve_campaign_attachments(config, campaign_file=campaign_path)
|
||||||
|
except Exception as exc:
|
||||||
|
return [
|
||||||
|
_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"attachment_resolution_failed",
|
||||||
|
f"attachment rules could not be resolved: {exc}",
|
||||||
|
"/attachments",
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
issues: list[SemanticIssue] = []
|
||||||
|
for entry in report.entries:
|
||||||
|
entry_path = f"/entries/{entry.entry_id or entry.entry_index}"
|
||||||
|
for issue in entry.issues:
|
||||||
|
if any(issue is attachment_issue for attachment in entry.attachments for attachment_issue in attachment.issues):
|
||||||
|
continue
|
||||||
|
issues.append(_issue(Severity(issue.severity.value), issue.code, issue.message, entry_path))
|
||||||
|
for attachment in entry.attachments:
|
||||||
|
attachment_path = (
|
||||||
|
f"/attachments/global/{attachment.index}"
|
||||||
|
if attachment.scope.value == "global"
|
||||||
|
else f"{entry_path}/attachments/{attachment.index}"
|
||||||
|
)
|
||||||
|
for issue in attachment.issues:
|
||||||
|
issues.append(_issue(Severity(issue.severity.value), issue.code, issue.message, attachment_path))
|
||||||
|
return issues
|
||||||
|
|
||||||
|
|
||||||
def _template_source_file_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
|
def _template_source_file_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
|
||||||
issues: list[SemanticIssue] = []
|
issues: list[SemanticIssue] = []
|
||||||
for schema_path, raw_path in _iter_template_source_paths(config):
|
for schema_path, raw_path in _iter_template_source_paths(config):
|
||||||
@@ -557,6 +783,7 @@ def validate_campaign_config(
|
|||||||
*,
|
*,
|
||||||
campaign_file: str | Path | None = None,
|
campaign_file: str | Path | None = None,
|
||||||
check_files: bool = False,
|
check_files: bool = False,
|
||||||
|
postbox_available: bool = False,
|
||||||
) -> SemanticReport:
|
) -> SemanticReport:
|
||||||
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
|
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
|
||||||
issues: list[SemanticIssue] = []
|
issues: list[SemanticIssue] = []
|
||||||
@@ -568,7 +795,13 @@ def validate_campaign_config(
|
|||||||
issues.extend(_global_value_issues(config, declared_names))
|
issues.extend(_global_value_issues(config, declared_names))
|
||||||
issues.extend(_attachment_path_issues(config))
|
issues.extend(_attachment_path_issues(config))
|
||||||
issues.extend(_zip_configuration_issues(config))
|
issues.extend(_zip_configuration_issues(config))
|
||||||
issues.extend(_delivery_issues(config))
|
issues.extend(
|
||||||
|
_delivery_issues(
|
||||||
|
config,
|
||||||
|
postbox_available=postbox_available,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
issues.extend(_sender_issues(config))
|
||||||
|
|
||||||
entries = _entries_validation(
|
entries = _entries_validation(
|
||||||
config,
|
config,
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from collections.abc import Iterable, Mapping
|
from collections.abc import Iterable, Mapping
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
from sqlalchemy import and_, or_
|
from sqlalchemy import and_, or_
|
||||||
|
|
||||||
from govoplan_core.core.access import AccessDecisionProvenance, PrincipalRef
|
from govoplan_core.core.access import AccessDecisionProvenance, PrincipalRef
|
||||||
@@ -14,12 +15,61 @@ from govoplan_core.core.campaigns import (
|
|||||||
CampaignPolicyContextProvider,
|
CampaignPolicyContextProvider,
|
||||||
CampaignRetentionProvider,
|
CampaignRetentionProvider,
|
||||||
)
|
)
|
||||||
|
from govoplan_core.core.ownership import (
|
||||||
|
OwnershipActionDecision,
|
||||||
|
OwnershipSubjectRef,
|
||||||
|
OwnershipTransferError,
|
||||||
|
)
|
||||||
from govoplan_core.security.module_permissions import scopes_grant_compatible
|
from govoplan_core.security.module_permissions import scopes_grant_compatible
|
||||||
|
|
||||||
from govoplan_campaign.backend.db.models import Campaign, CampaignShare
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignShare,
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
READ_ACTIONS = {"campaigns:campaign:read", "campaign:read"}
|
READ_ACTIONS = {"campaigns:campaign:read", "campaign:read"}
|
||||||
|
CAMPAIGN_RESOURCE_TYPES = {
|
||||||
|
"campaign",
|
||||||
|
"campaign_object",
|
||||||
|
"campaigns:campaign",
|
||||||
|
}
|
||||||
|
CAMPAIGN_VERSION_RESOURCE_TYPES = {
|
||||||
|
"campaign_version",
|
||||||
|
"campaigns:version",
|
||||||
|
}
|
||||||
|
CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES = {
|
||||||
|
"campaign_delivery_job",
|
||||||
|
"campaign_job",
|
||||||
|
"campaigns:delivery_job",
|
||||||
|
}
|
||||||
|
CAMPAIGN_REPORT_RESOURCE_TYPES = {
|
||||||
|
"campaign_report",
|
||||||
|
"campaigns:report",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def campaign_report_resource_id(
|
||||||
|
*,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
report_kind: str,
|
||||||
|
) -> str:
|
||||||
|
kind = report_kind.strip().lower()
|
||||||
|
if not campaign_id or not version_id or not kind or ":" in kind:
|
||||||
|
raise ValueError("Campaign report references require campaign, version, and a bounded kind")
|
||||||
|
return f"{campaign_id}:{version_id}:{kind}"
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_report_reference(
|
||||||
|
resource_id: str,
|
||||||
|
) -> tuple[str, str, str] | None:
|
||||||
|
parts = resource_id.split(":", 2)
|
||||||
|
if len(parts) != 3 or not all(part.strip() for part in parts):
|
||||||
|
return None
|
||||||
|
return parts[0], parts[1], parts[2].strip().lower()
|
||||||
|
|
||||||
|
|
||||||
class CampaignMailPolicyContextService(CampaignMailPolicyContextProvider):
|
class CampaignMailPolicyContextService(CampaignMailPolicyContextProvider):
|
||||||
@@ -120,20 +170,112 @@ class CampaignAccessService(CampaignAccessProvider):
|
|||||||
action: str,
|
action: str,
|
||||||
) -> tuple[AccessDecisionProvenance, ...]:
|
) -> tuple[AccessDecisionProvenance, ...]:
|
||||||
normalized_type = resource_type.lower().strip()
|
normalized_type = resource_type.lower().strip()
|
||||||
if normalized_type not in {"campaign", "campaign_object", "campaigns:campaign"}:
|
child_item: AccessDecisionProvenance | None = None
|
||||||
return ()
|
campaign: Campaign | None
|
||||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
if normalized_type in CAMPAIGN_RESOURCE_TYPES:
|
||||||
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id):
|
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||||
return (
|
elif normalized_type in CAMPAIGN_VERSION_RESOURCE_TYPES:
|
||||||
AccessDecisionProvenance(
|
version = session.get(CampaignVersion, resource_id) # type: ignore[attr-defined]
|
||||||
kind="resource",
|
if version is None:
|
||||||
id=resource_id,
|
return _missing_resource_provenance(
|
||||||
tenant_id=principal.tenant_id,
|
principal,
|
||||||
source="campaigns.not_found",
|
resource_type="campaign_version",
|
||||||
details={"resource_type": "campaign", "found": False},
|
resource_id=resource_id,
|
||||||
),
|
)
|
||||||
|
campaign = session.get(Campaign, version.campaign_id) # type: ignore[attr-defined]
|
||||||
|
child_item = AccessDecisionProvenance(
|
||||||
|
kind="resource",
|
||||||
|
id=version.id,
|
||||||
|
label=f"Version {version.version_number}",
|
||||||
|
tenant_id=campaign.tenant_id if campaign else principal.tenant_id,
|
||||||
|
source="campaigns.version",
|
||||||
|
details={
|
||||||
|
"resource_type": "campaign_version",
|
||||||
|
"campaign_id": version.campaign_id,
|
||||||
|
"version_number": version.version_number,
|
||||||
|
"workflow_state": version.workflow_state,
|
||||||
|
"authorization_inherited_from": {
|
||||||
|
"resource_type": "campaign",
|
||||||
|
"resource_id": version.campaign_id,
|
||||||
|
},
|
||||||
|
},
|
||||||
)
|
)
|
||||||
items = [
|
elif normalized_type in CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES:
|
||||||
|
job = session.get(CampaignJob, resource_id) # type: ignore[attr-defined]
|
||||||
|
if job is None:
|
||||||
|
return _missing_resource_provenance(
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_delivery_job",
|
||||||
|
resource_id=resource_id,
|
||||||
|
)
|
||||||
|
campaign = session.get(Campaign, job.campaign_id) # type: ignore[attr-defined]
|
||||||
|
child_item = AccessDecisionProvenance(
|
||||||
|
kind="resource",
|
||||||
|
id=job.id,
|
||||||
|
label=job.recipient_email or f"Recipient {job.entry_index}",
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
source="campaigns.delivery_job",
|
||||||
|
details={
|
||||||
|
"resource_type": "campaign_delivery_job",
|
||||||
|
"campaign_id": job.campaign_id,
|
||||||
|
"campaign_version_id": job.campaign_version_id,
|
||||||
|
"queue_status": job.queue_status,
|
||||||
|
"send_status": job.send_status,
|
||||||
|
"authorization_inherited_from": {
|
||||||
|
"resource_type": "campaign",
|
||||||
|
"resource_id": job.campaign_id,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
elif normalized_type in CAMPAIGN_REPORT_RESOURCE_TYPES:
|
||||||
|
reference = _campaign_report_reference(resource_id)
|
||||||
|
if reference is None:
|
||||||
|
return _missing_resource_provenance(
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_report",
|
||||||
|
resource_id=resource_id,
|
||||||
|
reason="invalid_reference",
|
||||||
|
)
|
||||||
|
campaign_id, version_id, report_kind = reference
|
||||||
|
campaign = session.get(Campaign, campaign_id) # type: ignore[attr-defined]
|
||||||
|
version = session.get(CampaignVersion, version_id) # type: ignore[attr-defined]
|
||||||
|
if (
|
||||||
|
campaign is None
|
||||||
|
or version is None
|
||||||
|
or version.campaign_id != campaign.id
|
||||||
|
):
|
||||||
|
return _missing_resource_provenance(
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_report",
|
||||||
|
resource_id=resource_id,
|
||||||
|
)
|
||||||
|
child_item = AccessDecisionProvenance(
|
||||||
|
kind="resource",
|
||||||
|
id=resource_id,
|
||||||
|
label=report_kind,
|
||||||
|
tenant_id=campaign.tenant_id,
|
||||||
|
source="campaigns.report",
|
||||||
|
details={
|
||||||
|
"resource_type": "campaign_report",
|
||||||
|
"campaign_id": campaign.id,
|
||||||
|
"campaign_version_id": version.id,
|
||||||
|
"report_kind": report_kind,
|
||||||
|
"persisted": False,
|
||||||
|
"authorization_inherited_from": {
|
||||||
|
"resource_type": "campaign",
|
||||||
|
"resource_id": campaign.id,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
return ()
|
||||||
|
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id):
|
||||||
|
return _missing_resource_provenance(
|
||||||
|
principal,
|
||||||
|
resource_type=normalized_type,
|
||||||
|
resource_id=resource_id,
|
||||||
|
)
|
||||||
|
items = ([child_item] if child_item is not None else []) + [
|
||||||
AccessDecisionProvenance(
|
AccessDecisionProvenance(
|
||||||
kind="resource",
|
kind="resource",
|
||||||
id=campaign.id,
|
id=campaign.id,
|
||||||
@@ -149,7 +291,11 @@ class CampaignAccessService(CampaignAccessProvider):
|
|||||||
]
|
]
|
||||||
items.extend(_owner_provenance(campaign, principal))
|
items.extend(_owner_provenance(campaign, principal))
|
||||||
items.extend(_tenant_admin_provenance(principal))
|
items.extend(_tenant_admin_provenance(principal))
|
||||||
permission_values = {"read", "write"} if action in READ_ACTIONS else {"write"}
|
permission_values = (
|
||||||
|
{"read", "write"}
|
||||||
|
if action in READ_ACTIONS or action.strip().lower().endswith(":read")
|
||||||
|
else {"write"}
|
||||||
|
)
|
||||||
shares = (
|
shares = (
|
||||||
session.query(CampaignShare) # type: ignore[attr-defined]
|
session.query(CampaignShare) # type: ignore[attr-defined]
|
||||||
.filter(
|
.filter(
|
||||||
@@ -187,6 +333,283 @@ def access_capability(context: object) -> CampaignAccessService:
|
|||||||
return CampaignAccessService()
|
return CampaignAccessService()
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignOwnershipService:
|
||||||
|
def current_owner(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
resource_id: str,
|
||||||
|
) -> OwnershipSubjectRef | None:
|
||||||
|
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||||
|
if campaign is None or campaign.tenant_id != tenant_id:
|
||||||
|
return None
|
||||||
|
if campaign.owner_user_id:
|
||||||
|
return OwnershipSubjectRef(type="user", id=campaign.owner_user_id)
|
||||||
|
if campaign.owner_group_id:
|
||||||
|
return OwnershipSubjectRef(type="group", id=campaign.owner_group_id)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def authorize_ownership_action(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
resource_id: str,
|
||||||
|
action: str,
|
||||||
|
actor: OwnershipSubjectRef,
|
||||||
|
current_owner: OwnershipSubjectRef,
|
||||||
|
target_owner: OwnershipSubjectRef,
|
||||||
|
) -> OwnershipActionDecision:
|
||||||
|
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||||
|
if campaign is None or campaign.tenant_id != tenant_id:
|
||||||
|
return OwnershipActionDecision(False, "Campaign was not found")
|
||||||
|
if target_owner.type not in {"user", "group"}:
|
||||||
|
return OwnershipActionDecision(
|
||||||
|
False,
|
||||||
|
"Campaign ownership can be assigned only to a user or group",
|
||||||
|
)
|
||||||
|
|
||||||
|
actor_is_current_owner = _actor_controls_owner(actor, current_owner)
|
||||||
|
actor_is_target = _actor_controls_owner(actor, target_owner)
|
||||||
|
actor_can_share = scopes_grant_compatible(
|
||||||
|
actor.scopes,
|
||||||
|
"campaigns:campaign:share",
|
||||||
|
)
|
||||||
|
actor_can_accept_for_group = scopes_grant_compatible(
|
||||||
|
actor.scopes,
|
||||||
|
"campaigns:ownership:accept_group",
|
||||||
|
)
|
||||||
|
actor_can_recover = scopes_grant_compatible(
|
||||||
|
actor.scopes,
|
||||||
|
"campaigns:ownership:recover",
|
||||||
|
)
|
||||||
|
actor_can_read = (
|
||||||
|
scopes_grant_compatible(
|
||||||
|
actor.scopes,
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
)
|
||||||
|
and CampaignAccessService().can_read_campaign(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=resource_id,
|
||||||
|
user_id=actor.id,
|
||||||
|
group_ids=actor.group_ids,
|
||||||
|
tenant_admin=scopes_grant_compatible(
|
||||||
|
actor.scopes,
|
||||||
|
"tenant:*",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if action in {
|
||||||
|
"propose_transfer",
|
||||||
|
"request_ownership",
|
||||||
|
"request_recovery",
|
||||||
|
}:
|
||||||
|
target_decision = _valid_campaign_owner_target(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
target_owner=target_owner,
|
||||||
|
)
|
||||||
|
if target_decision is not None:
|
||||||
|
return target_decision
|
||||||
|
|
||||||
|
if action == "view_transfer":
|
||||||
|
allowed = (
|
||||||
|
_actor_is_owner_member(actor, current_owner)
|
||||||
|
or _actor_is_owner_member(actor, target_owner)
|
||||||
|
or actor_can_recover
|
||||||
|
)
|
||||||
|
elif action in {
|
||||||
|
"propose_transfer",
|
||||||
|
"approve_requested_transfer",
|
||||||
|
}:
|
||||||
|
allowed = actor_is_current_owner and actor_can_share
|
||||||
|
elif action == "cancel_transfer":
|
||||||
|
allowed = (
|
||||||
|
(actor_is_current_owner and actor_can_share)
|
||||||
|
or actor_is_target
|
||||||
|
)
|
||||||
|
elif action == "request_ownership":
|
||||||
|
allowed = actor_can_read and actor_is_target and (
|
||||||
|
target_owner.type != "group" or actor_can_accept_for_group
|
||||||
|
)
|
||||||
|
elif action == "accept_transfer":
|
||||||
|
allowed = target_owner.type == "user" and actor_is_target
|
||||||
|
elif action == "accept_group_transfer":
|
||||||
|
allowed = (
|
||||||
|
target_owner.type == "group"
|
||||||
|
and actor_is_target
|
||||||
|
and actor_can_accept_for_group
|
||||||
|
)
|
||||||
|
elif action == "decline_transfer":
|
||||||
|
allowed = (
|
||||||
|
actor_is_target
|
||||||
|
and (
|
||||||
|
target_owner.type != "group"
|
||||||
|
or actor_can_accept_for_group
|
||||||
|
)
|
||||||
|
) or (
|
||||||
|
actor_is_current_owner
|
||||||
|
and actor_can_share
|
||||||
|
)
|
||||||
|
elif action in {
|
||||||
|
"request_recovery",
|
||||||
|
"approve_recovery",
|
||||||
|
"execute_recovery",
|
||||||
|
}:
|
||||||
|
allowed = actor_can_recover
|
||||||
|
else:
|
||||||
|
allowed = False
|
||||||
|
|
||||||
|
return OwnershipActionDecision(
|
||||||
|
allowed=allowed,
|
||||||
|
reason=None if allowed else f"Campaign ownership action is not allowed: {action}",
|
||||||
|
)
|
||||||
|
|
||||||
|
def apply_owner(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
resource_id: str,
|
||||||
|
expected_owner: OwnershipSubjectRef,
|
||||||
|
target_owner: OwnershipSubjectRef,
|
||||||
|
actor: OwnershipSubjectRef,
|
||||||
|
reason: str | None,
|
||||||
|
) -> None:
|
||||||
|
del actor, reason
|
||||||
|
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||||
|
if campaign is None or campaign.tenant_id != tenant_id:
|
||||||
|
raise OwnershipTransferError("Campaign was not found")
|
||||||
|
if target_owner.type not in {"user", "group"}:
|
||||||
|
raise OwnershipTransferError(
|
||||||
|
"Campaign ownership can be assigned only to a user or group"
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_clear_current_version_mail_profile_for_owner_transfer,
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
_clear_current_version_mail_profile_for_owner_transfer(
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
campaign,
|
||||||
|
)
|
||||||
|
except HTTPException as exc:
|
||||||
|
raise OwnershipTransferError(
|
||||||
|
str(
|
||||||
|
exc.detail
|
||||||
|
or "Campaign owner cannot be changed in its current state"
|
||||||
|
)
|
||||||
|
) from exc
|
||||||
|
expected_filter = (
|
||||||
|
(
|
||||||
|
Campaign.owner_user_id == expected_owner.id,
|
||||||
|
Campaign.owner_group_id.is_(None),
|
||||||
|
)
|
||||||
|
if expected_owner.type == "user"
|
||||||
|
else (
|
||||||
|
Campaign.owner_user_id.is_(None),
|
||||||
|
Campaign.owner_group_id == expected_owner.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
updated = (
|
||||||
|
session.query(Campaign) # type: ignore[attr-defined]
|
||||||
|
.filter(
|
||||||
|
Campaign.id == resource_id,
|
||||||
|
Campaign.tenant_id == tenant_id,
|
||||||
|
*expected_filter,
|
||||||
|
)
|
||||||
|
.update(
|
||||||
|
{
|
||||||
|
Campaign.owner_user_id: (
|
||||||
|
target_owner.id
|
||||||
|
if target_owner.type == "user"
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
Campaign.owner_group_id: (
|
||||||
|
target_owner.id
|
||||||
|
if target_owner.type == "group"
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
},
|
||||||
|
synchronize_session=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if updated != 1:
|
||||||
|
raise OwnershipTransferError(
|
||||||
|
"Campaign owner changed while the transfer was pending"
|
||||||
|
)
|
||||||
|
session.expire(campaign) # type: ignore[attr-defined]
|
||||||
|
|
||||||
|
|
||||||
|
def _valid_campaign_owner_target(
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
target_owner: OwnershipSubjectRef,
|
||||||
|
) -> OwnershipActionDecision | None:
|
||||||
|
from govoplan_campaign.backend.route_support import _access_directory
|
||||||
|
|
||||||
|
directory = _access_directory()
|
||||||
|
if target_owner.type == "user":
|
||||||
|
item = directory.get_user(target_owner.id)
|
||||||
|
elif target_owner.type == "group":
|
||||||
|
item = directory.get_group(target_owner.id)
|
||||||
|
else:
|
||||||
|
return OwnershipActionDecision(
|
||||||
|
False,
|
||||||
|
"Campaign ownership can be assigned only to a user or group",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
item is None
|
||||||
|
or item.tenant_id != tenant_id
|
||||||
|
or item.status != "active"
|
||||||
|
):
|
||||||
|
return OwnershipActionDecision(
|
||||||
|
False,
|
||||||
|
f"Target {target_owner.type} is not active in this tenant",
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _actor_is_owner_member(
|
||||||
|
actor: OwnershipSubjectRef,
|
||||||
|
owner: OwnershipSubjectRef,
|
||||||
|
) -> bool:
|
||||||
|
if owner.type == "group":
|
||||||
|
return owner.id in actor.group_ids
|
||||||
|
return actor.type == owner.type and actor.id == owner.id
|
||||||
|
|
||||||
|
|
||||||
|
def _actor_controls_owner(
|
||||||
|
actor: OwnershipSubjectRef,
|
||||||
|
owner: OwnershipSubjectRef,
|
||||||
|
) -> bool:
|
||||||
|
return _actor_is_owner_member(actor, owner)
|
||||||
|
|
||||||
|
|
||||||
|
def _missing_resource_provenance(
|
||||||
|
principal: PrincipalRef,
|
||||||
|
*,
|
||||||
|
resource_type: str,
|
||||||
|
resource_id: str,
|
||||||
|
reason: str = "not_found",
|
||||||
|
) -> tuple[AccessDecisionProvenance, ...]:
|
||||||
|
return (
|
||||||
|
AccessDecisionProvenance(
|
||||||
|
kind="resource",
|
||||||
|
id=resource_id,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
source="campaigns.not_found",
|
||||||
|
details={
|
||||||
|
"resource_type": resource_type,
|
||||||
|
"found": False,
|
||||||
|
"reason": reason,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _owner_provenance(campaign: Campaign, principal: PrincipalRef) -> tuple[AccessDecisionProvenance, ...]:
|
def _owner_provenance(campaign: Campaign, principal: PrincipalRef) -> tuple[AccessDecisionProvenance, ...]:
|
||||||
if campaign.owner_user_id and campaign.owner_user_id == principal.membership_id:
|
if campaign.owner_user_id and campaign.owner_user_id == principal.membership_id:
|
||||||
return (
|
return (
|
||||||
@@ -279,6 +702,12 @@ class CampaignDeliveryTaskService(CampaignDeliveryTaskProvider):
|
|||||||
|
|
||||||
|
|
||||||
def delivery_tasks_capability(context: object) -> CampaignDeliveryTaskService:
|
def delivery_tasks_capability(context: object) -> CampaignDeliveryTaskService:
|
||||||
|
from govoplan_campaign.backend.runtime import configure_runtime
|
||||||
|
|
||||||
|
configure_runtime(
|
||||||
|
registry=getattr(context, "registry", None),
|
||||||
|
settings=getattr(context, "settings", None),
|
||||||
|
)
|
||||||
return CampaignDeliveryTaskService()
|
return CampaignDeliveryTaskService()
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from govoplan_campaign.backend.db.models import (
|
|||||||
CampaignShare,
|
CampaignShare,
|
||||||
CampaignVersion,
|
CampaignVersion,
|
||||||
ImapAppendAttempt,
|
ImapAppendAttempt,
|
||||||
|
PostboxDeliveryAttempt,
|
||||||
SendAttempt,
|
SendAttempt,
|
||||||
new_uuid,
|
new_uuid,
|
||||||
)
|
)
|
||||||
@@ -55,7 +56,10 @@ def _record_campaign_changes(session: OrmSession, _flush_context: object, _insta
|
|||||||
_record_job_change(session, obj)
|
_record_job_change(session, obj)
|
||||||
elif isinstance(obj, CampaignIssue):
|
elif isinstance(obj, CampaignIssue):
|
||||||
_record_issue_change(session, obj)
|
_record_issue_change(session, obj)
|
||||||
elif isinstance(obj, (SendAttempt, ImapAppendAttempt)):
|
elif isinstance(
|
||||||
|
obj,
|
||||||
|
(SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt),
|
||||||
|
):
|
||||||
_record_attempt_change(session, obj)
|
_record_attempt_change(session, obj)
|
||||||
|
|
||||||
|
|
||||||
@@ -182,8 +186,11 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
|||||||
"validation_status",
|
"validation_status",
|
||||||
"queue_status",
|
"queue_status",
|
||||||
"send_status",
|
"send_status",
|
||||||
|
"delivery_channel_policy",
|
||||||
|
"postbox_status",
|
||||||
"imap_status",
|
"imap_status",
|
||||||
"attempt_count",
|
"attempt_count",
|
||||||
|
"postbox_attempt_count",
|
||||||
"last_error",
|
"last_error",
|
||||||
"queued_at",
|
"queued_at",
|
||||||
"claimed_at",
|
"claimed_at",
|
||||||
@@ -191,6 +198,7 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
|||||||
"outcome_unknown_at",
|
"outcome_unknown_at",
|
||||||
"sent_at",
|
"sent_at",
|
||||||
"resolved_recipients",
|
"resolved_recipients",
|
||||||
|
"resolved_postbox_targets",
|
||||||
"resolved_attachments",
|
"resolved_attachments",
|
||||||
"issues_snapshot",
|
"issues_snapshot",
|
||||||
),
|
),
|
||||||
@@ -217,6 +225,8 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
|||||||
"validation_status": job.validation_status,
|
"validation_status": job.validation_status,
|
||||||
"queue_status": job.queue_status,
|
"queue_status": job.queue_status,
|
||||||
"send_status": job.send_status,
|
"send_status": job.send_status,
|
||||||
|
"delivery_channel_policy": job.delivery_channel_policy,
|
||||||
|
"postbox_status": job.postbox_status,
|
||||||
"imap_status": job.imap_status,
|
"imap_status": job.imap_status,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
@@ -247,10 +257,25 @@ def _record_issue_change(session: OrmSession, issue: CampaignIssue) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppendAttempt) -> None:
|
def _record_attempt_change(
|
||||||
|
session: OrmSession,
|
||||||
|
attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt,
|
||||||
|
) -> None:
|
||||||
operation = _operation_for_object(
|
operation = _operation_for_object(
|
||||||
attempt,
|
attempt,
|
||||||
changed_attrs=("status", "claim_token", "smtp_status_code", "smtp_response", "error_type", "error_message", "folder"),
|
changed_attrs=(
|
||||||
|
"status",
|
||||||
|
"claim_token",
|
||||||
|
"smtp_status_code",
|
||||||
|
"smtp_response",
|
||||||
|
"error_type",
|
||||||
|
"error_message",
|
||||||
|
"folder",
|
||||||
|
"provider_delivery_id",
|
||||||
|
"provider_message_id",
|
||||||
|
"postbox_id",
|
||||||
|
"evidence",
|
||||||
|
),
|
||||||
)
|
)
|
||||||
if operation is None:
|
if operation is None:
|
||||||
return
|
return
|
||||||
@@ -270,7 +295,13 @@ def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppen
|
|||||||
payload={
|
payload={
|
||||||
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
|
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
|
||||||
"attempt_id": attempt_id,
|
"attempt_id": attempt_id,
|
||||||
"attempt_kind": "imap" if isinstance(attempt, ImapAppendAttempt) else "smtp",
|
"attempt_kind": (
|
||||||
|
"postbox"
|
||||||
|
if isinstance(attempt, PostboxDeliveryAttempt)
|
||||||
|
else "imap"
|
||||||
|
if isinstance(attempt, ImapAppendAttempt)
|
||||||
|
else "smtp"
|
||||||
|
),
|
||||||
"job_id": job.id,
|
"job_id": job.id,
|
||||||
"version_id": job.campaign_version_id,
|
"version_id": job.campaign_version_id,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ from typing import Any
|
|||||||
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint
|
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
|
from govoplan_core.core.concurrency import strong_resource_etag
|
||||||
from govoplan_core.db.base import Base, TimestampMixin
|
from govoplan_core.db.base import Base, TimestampMixin
|
||||||
|
|
||||||
|
|
||||||
@@ -78,10 +79,14 @@ class JobQueueStatus(StrEnum):
|
|||||||
|
|
||||||
class JobSendStatus(StrEnum):
|
class JobSendStatus(StrEnum):
|
||||||
NOT_QUEUED = "not_queued"
|
NOT_QUEUED = "not_queued"
|
||||||
|
SKIPPED = "skipped"
|
||||||
QUEUED = "queued"
|
QUEUED = "queued"
|
||||||
CLAIMED = "claimed"
|
CLAIMED = "claimed"
|
||||||
SENDING = "sending"
|
SENDING = "sending"
|
||||||
SMTP_ACCEPTED = "smtp_accepted"
|
SMTP_ACCEPTED = "smtp_accepted"
|
||||||
|
POSTBOX_ACCEPTED = "postbox_accepted"
|
||||||
|
DELIVERED = "delivered"
|
||||||
|
PARTIALLY_ACCEPTED = "partially_accepted"
|
||||||
SENT = "sent" # legacy value retained for existing databases/reports
|
SENT = "sent" # legacy value retained for existing databases/reports
|
||||||
OUTCOME_UNKNOWN = "outcome_unknown"
|
OUTCOME_UNKNOWN = "outcome_unknown"
|
||||||
FAILED_TEMPORARY = "failed_temporary"
|
FAILED_TEMPORARY = "failed_temporary"
|
||||||
@@ -89,10 +94,25 @@ class JobSendStatus(StrEnum):
|
|||||||
CANCELLED = "cancelled"
|
CANCELLED = "cancelled"
|
||||||
|
|
||||||
|
|
||||||
|
class JobPostboxStatus(StrEnum):
|
||||||
|
NOT_REQUESTED = "not_requested"
|
||||||
|
PENDING = "pending"
|
||||||
|
DELIVERING = "delivering"
|
||||||
|
ACCEPTED = "accepted"
|
||||||
|
ACCEPTED_VACANT = "accepted_vacant"
|
||||||
|
PARTIALLY_ACCEPTED = "partially_accepted"
|
||||||
|
REJECTED_TEMPORARY = "rejected_temporary"
|
||||||
|
REJECTED_PERMANENT = "rejected_permanent"
|
||||||
|
OUTCOME_UNKNOWN = "outcome_unknown"
|
||||||
|
SKIPPED = "skipped"
|
||||||
|
|
||||||
|
|
||||||
class JobImapStatus(StrEnum):
|
class JobImapStatus(StrEnum):
|
||||||
NOT_REQUESTED = "not_requested"
|
NOT_REQUESTED = "not_requested"
|
||||||
PENDING = "pending"
|
PENDING = "pending"
|
||||||
|
APPENDING = "appending"
|
||||||
APPENDED = "appended"
|
APPENDED = "appended"
|
||||||
|
OUTCOME_UNKNOWN = "outcome_unknown"
|
||||||
FAILED = "failed"
|
FAILED = "failed"
|
||||||
SKIPPED = "skipped"
|
SKIPPED = "skipped"
|
||||||
|
|
||||||
@@ -161,8 +181,6 @@ class RecipientImportMappingProfile(Base, TimestampMixin):
|
|||||||
value_separators: Mapped[str] = mapped_column(String(50), default=",;|", nullable=False)
|
value_separators: Mapped[str] = mapped_column(String(50), default=",;|", nullable=False)
|
||||||
mappings: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list, nullable=False)
|
mappings: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list, nullable=False)
|
||||||
|
|
||||||
owner: Mapped["User"] = relationship("User")
|
|
||||||
|
|
||||||
|
|
||||||
class CampaignVersion(Base, TimestampMixin):
|
class CampaignVersion(Base, TimestampMixin):
|
||||||
__tablename__ = "campaign_versions"
|
__tablename__ = "campaign_versions"
|
||||||
@@ -171,6 +189,11 @@ class CampaignVersion(Base, TimestampMixin):
|
|||||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
version_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
version_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
edit_revision: Mapped[int] = mapped_column(
|
||||||
|
Integer,
|
||||||
|
default=1,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
|
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
|
||||||
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
|
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
|
||||||
source_filename: Mapped[str | None] = mapped_column(String(500))
|
source_filename: Mapped[str | None] = mapped_column(String(500))
|
||||||
@@ -212,9 +235,29 @@ class CampaignVersion(Base, TimestampMixin):
|
|||||||
execution_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
execution_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
execution_snapshot_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
execution_snapshot_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
||||||
execution_snapshot_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
execution_snapshot_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
delivery_mode: Mapped[str | None] = mapped_column(String(30), nullable=True, index=True)
|
||||||
|
delivery_mode_selected_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
campaign: Mapped[Campaign] = relationship(back_populates="versions")
|
campaign: Mapped[Campaign] = relationship(back_populates="versions")
|
||||||
|
|
||||||
|
__mapper_args__ = {
|
||||||
|
"version_id_col": edit_revision,
|
||||||
|
}
|
||||||
|
|
||||||
|
@property
|
||||||
|
def strong_etag(self) -> str:
|
||||||
|
return strong_resource_etag(
|
||||||
|
"campaign_version",
|
||||||
|
self.id,
|
||||||
|
self.edit_revision,
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def mail_profile_migration_required(self) -> bool:
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_boundary_violations
|
||||||
|
|
||||||
|
return bool(campaign_mail_profile_boundary_violations(self.raw_json))
|
||||||
|
|
||||||
|
|
||||||
class CampaignJob(Base, TimestampMixin):
|
class CampaignJob(Base, TimestampMixin):
|
||||||
__tablename__ = "campaign_jobs"
|
__tablename__ = "campaign_jobs"
|
||||||
@@ -234,23 +277,48 @@ class CampaignJob(Base, TimestampMixin):
|
|||||||
eml_local_path: Mapped[str | None] = mapped_column(String(1000))
|
eml_local_path: Mapped[str | None] = mapped_column(String(1000))
|
||||||
eml_size_bytes: Mapped[int | None] = mapped_column(Integer)
|
eml_size_bytes: Mapped[int | None] = mapped_column(Integer)
|
||||||
eml_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
eml_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
||||||
|
execution_input_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||||
|
|
||||||
build_status: Mapped[str] = mapped_column(String(50), default=JobBuildStatus.PENDING.value, nullable=False, index=True)
|
build_status: Mapped[str] = mapped_column(String(50), default=JobBuildStatus.PENDING.value, nullable=False, index=True)
|
||||||
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
|
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
|
||||||
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
|
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
|
||||||
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
|
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
|
||||||
|
delivery_channel_policy: Mapped[str] = mapped_column(
|
||||||
|
String(30),
|
||||||
|
default="mail",
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
postbox_status: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
default=JobPostboxStatus.NOT_REQUESTED.value,
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
|
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
|
||||||
|
|
||||||
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
postbox_attempt_count: Mapped[int] = mapped_column(
|
||||||
|
Integer,
|
||||||
|
default=0,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
last_error: Mapped[str | None] = mapped_column(Text)
|
last_error: Mapped[str | None] = mapped_column(Text)
|
||||||
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
|
claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
|
||||||
smtp_started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
smtp_started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
outcome_unknown_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
outcome_unknown_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
imap_claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
imap_claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
|
||||||
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
|
resolved_postbox_targets: Mapped[list[dict[str, Any]]] = mapped_column(
|
||||||
|
JSON,
|
||||||
|
default=list,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||||
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||||
|
|
||||||
@@ -315,17 +383,227 @@ class SendAttempt(Base, TimestampMixin):
|
|||||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignMessageAction(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_message_actions"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"idempotency_key",
|
||||||
|
name="uq_campaign_message_actions_idempotency",
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"ix_campaign_message_actions_job_created",
|
||||||
|
"job_id",
|
||||||
|
"created_at",
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"ix_campaign_message_actions_campaign_kind",
|
||||||
|
"campaign_id",
|
||||||
|
"kind",
|
||||||
|
"status",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||||
|
campaign_id: Mapped[str] = mapped_column(
|
||||||
|
ForeignKey("campaigns.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
campaign_version_id: Mapped[str] = mapped_column(
|
||||||
|
ForeignKey("campaign_versions.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
job_id: Mapped[str] = mapped_column(
|
||||||
|
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
kind: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
|
||||||
|
idempotency_key: Mapped[str] = mapped_column(String(200), nullable=False)
|
||||||
|
canonical_request_hash: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
|
context: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
|
||||||
|
actor_user_id: Mapped[str | None] = mapped_column(
|
||||||
|
ForeignKey("access_users.id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
actor_api_key_id: Mapped[str | None] = mapped_column(String(36), nullable=True)
|
||||||
|
message_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
message_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||||
|
recipient_manifest_sha256: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
recipient_count: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
prior_send_status: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||||
|
prior_attempt_count: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
final_send_status: Mapped[str | None] = mapped_column(String(50), nullable=True)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
default="initiated",
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
refusal_summary: Mapped[dict[str, Any]] = mapped_column(
|
||||||
|
JSON,
|
||||||
|
default=dict,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
error_type: Mapped[str | None] = mapped_column(String(120), nullable=True)
|
||||||
|
error_message: Mapped[str | None] = mapped_column(String(500), nullable=True)
|
||||||
|
linked_send_attempt_id: Mapped[str | None] = mapped_column(
|
||||||
|
ForeignKey("send_attempts.id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
effect_started_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
completed_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignMessageActionAttempt(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_message_action_attempts"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"action_id",
|
||||||
|
"attempt_number",
|
||||||
|
name="uq_campaign_message_action_attempt_number",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
action_id: Mapped[str] = mapped_column(
|
||||||
|
ForeignKey("campaign_message_actions.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
default="initiated",
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
started_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
effect_started_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
completed_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
outcome_code: Mapped[str | None] = mapped_column(String(80), nullable=True)
|
||||||
|
diagnostic_summary: Mapped[str | None] = mapped_column(
|
||||||
|
String(500),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class ImapAppendAttempt(Base, TimestampMixin):
|
class ImapAppendAttempt(Base, TimestampMixin):
|
||||||
__tablename__ = "imap_append_attempts"
|
__tablename__ = "imap_append_attempts"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("job_id", "attempt_number", name="uq_imap_append_attempts_job_attempt"),
|
||||||
|
)
|
||||||
|
|
||||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
job_id: Mapped[str] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=False, index=True)
|
job_id: Mapped[str] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True)
|
||||||
folder: Mapped[str | None] = mapped_column(String(500))
|
folder: Mapped[str | None] = mapped_column(String(500))
|
||||||
status: Mapped[str] = mapped_column(String(50), nullable=False)
|
status: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||||
error_message: Mapped[str | None] = mapped_column(Text)
|
error_message: Mapped[str | None] = mapped_column(Text)
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxDeliveryAttempt(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_postbox_delivery_attempts"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"job_id",
|
||||||
|
"target_key",
|
||||||
|
"attempt_number",
|
||||||
|
name="uq_campaign_postbox_attempt_target_number",
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"ix_campaign_postbox_attempt_idempotency",
|
||||||
|
"tenant_id",
|
||||||
|
"idempotency_key",
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"ix_campaign_postbox_attempt_job_status",
|
||||||
|
"job_id",
|
||||||
|
"status",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
primary_key=True,
|
||||||
|
default=new_uuid,
|
||||||
|
)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
job_id: Mapped[str] = mapped_column(
|
||||||
|
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
target_key: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
target_index: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
target_snapshot: Mapped[dict[str, Any]] = mapped_column(
|
||||||
|
JSON,
|
||||||
|
default=dict,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
provider_delivery_id: Mapped[str | None] = mapped_column(String(36))
|
||||||
|
provider_message_id: Mapped[str | None] = mapped_column(String(36))
|
||||||
|
postbox_id: Mapped[str | None] = mapped_column(String(36), index=True)
|
||||||
|
address: Mapped[str | None] = mapped_column(String(500))
|
||||||
|
holder_count: Mapped[int | None] = mapped_column(Integer)
|
||||||
|
vacant: Mapped[bool | None] = mapped_column(Boolean)
|
||||||
|
duplicate: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||||
|
evidence: Mapped[dict[str, Any]] = mapped_column(
|
||||||
|
JSON,
|
||||||
|
default=dict,
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
error_type: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
error_code: Mapped[str | None] = mapped_column(String(100))
|
||||||
|
error_message: Mapped[str | None] = mapped_column(Text)
|
||||||
|
started_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
)
|
||||||
|
finished_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
@@ -343,8 +621,10 @@ __all__ = [
|
|||||||
"IssueSeverity",
|
"IssueSeverity",
|
||||||
"JobBuildStatus",
|
"JobBuildStatus",
|
||||||
"JobImapStatus",
|
"JobImapStatus",
|
||||||
|
"JobPostboxStatus",
|
||||||
"JobQueueStatus",
|
"JobQueueStatus",
|
||||||
"JobSendStatus",
|
"JobSendStatus",
|
||||||
"JobValidationStatus",
|
"JobValidationStatus",
|
||||||
"SendAttempt",
|
"SendAttempt",
|
||||||
|
"PostboxDeliveryAttempt",
|
||||||
]
|
]
|
||||||
|
|||||||
107
src/govoplan_campaign/backend/delivery_policy.py
Normal file
107
src/govoplan_campaign/backend/delivery_policy.py
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_core.tenancy.scope import Tenant
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 25
|
||||||
|
ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 500
|
||||||
|
SYNCHRONOUS_SEND_MAX_ENV = "GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS"
|
||||||
|
CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY = "campaign_delivery_policy"
|
||||||
|
SYNCHRONOUS_SEND_MAX_SETTINGS_KEY = "synchronous_send_max_recipients"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignDeliveryPolicyError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class SynchronousSendPolicy:
|
||||||
|
max_recipient_jobs: int
|
||||||
|
source: str
|
||||||
|
deployment_max_recipient_jobs: int
|
||||||
|
tenant_max_recipient_jobs: int | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"max_recipient_jobs": self.max_recipient_jobs,
|
||||||
|
"source": self.source,
|
||||||
|
"deployment_max_recipient_jobs": self.deployment_max_recipient_jobs,
|
||||||
|
"tenant_max_recipient_jobs": self.tenant_max_recipient_jobs,
|
||||||
|
"deployment_setting": SYNCHRONOUS_SEND_MAX_ENV,
|
||||||
|
"tenant_setting": (
|
||||||
|
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
|
||||||
|
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def effective_synchronous_send_policy(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
environ: Mapping[str, str] | None = None,
|
||||||
|
) -> SynchronousSendPolicy:
|
||||||
|
env = os.environ if environ is None else environ
|
||||||
|
deployment_value = _configured_limit(
|
||||||
|
env.get(SYNCHRONOUS_SEND_MAX_ENV),
|
||||||
|
source=SYNCHRONOUS_SEND_MAX_ENV,
|
||||||
|
default=DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
|
||||||
|
)
|
||||||
|
tenant = session.get(Tenant, tenant_id)
|
||||||
|
tenant_raw = _tenant_limit_value(tenant.settings if tenant is not None else None)
|
||||||
|
if tenant_raw is None:
|
||||||
|
return SynchronousSendPolicy(
|
||||||
|
max_recipient_jobs=deployment_value,
|
||||||
|
source=("deployment" if env.get(SYNCHRONOUS_SEND_MAX_ENV) not in (None, "") else "deployment_default"),
|
||||||
|
deployment_max_recipient_jobs=deployment_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
tenant_value = _configured_limit(
|
||||||
|
tenant_raw,
|
||||||
|
source=(
|
||||||
|
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
|
||||||
|
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
effective_value = min(deployment_value, tenant_value)
|
||||||
|
return SynchronousSendPolicy(
|
||||||
|
max_recipient_jobs=effective_value,
|
||||||
|
source="tenant" if tenant_value <= deployment_value else "deployment_ceiling",
|
||||||
|
deployment_max_recipient_jobs=deployment_value,
|
||||||
|
tenant_max_recipient_jobs=tenant_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _tenant_limit_value(settings: Mapping[str, Any] | None) -> object | None:
|
||||||
|
if not isinstance(settings, Mapping):
|
||||||
|
return None
|
||||||
|
policy = settings.get(CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY)
|
||||||
|
if not isinstance(policy, Mapping):
|
||||||
|
return None
|
||||||
|
return policy.get(SYNCHRONOUS_SEND_MAX_SETTINGS_KEY)
|
||||||
|
|
||||||
|
|
||||||
|
def _configured_limit(value: object, *, source: str, default: int | None = None) -> int:
|
||||||
|
if value is None or (isinstance(value, str) and not value.strip()):
|
||||||
|
if default is not None:
|
||||||
|
return default
|
||||||
|
raise CampaignDeliveryPolicyError(f"{source} must be configured as an integer")
|
||||||
|
if isinstance(value, bool):
|
||||||
|
raise CampaignDeliveryPolicyError(f"{source} must be an integer, not a boolean")
|
||||||
|
try:
|
||||||
|
parsed = int(value)
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise CampaignDeliveryPolicyError(f"{source} must be an integer") from exc
|
||||||
|
if str(parsed) != str(value).strip() and not isinstance(value, int):
|
||||||
|
raise CampaignDeliveryPolicyError(f"{source} must be an integer")
|
||||||
|
if parsed < 0 or parsed > ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS:
|
||||||
|
raise CampaignDeliveryPolicyError(
|
||||||
|
f"{source} must be between 0 and {ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS}"
|
||||||
|
)
|
||||||
|
return parsed
|
||||||
@@ -14,6 +14,7 @@ from govoplan_campaign.backend.persistence.campaigns import load_campaign_config
|
|||||||
from govoplan_campaign.backend.messages.builder import build_campaign_messages
|
from govoplan_campaign.backend.messages.builder import build_campaign_messages
|
||||||
from govoplan_campaign.backend.messages.models import MessageAddress, MessageDraft, MessageValidationStatus
|
from govoplan_campaign.backend.messages.models import MessageAddress, MessageDraft, MessageValidationStatus
|
||||||
from govoplan_campaign.backend.integrations import files_integration, mail_integration
|
from govoplan_campaign.backend.integrations import files_integration, mail_integration
|
||||||
|
from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths
|
||||||
|
|
||||||
|
|
||||||
class MockCampaignSendError(RuntimeError):
|
class MockCampaignSendError(RuntimeError):
|
||||||
@@ -298,6 +299,237 @@ def _mock_sent_folder(config: Any) -> str:
|
|||||||
return "Sent"
|
return "Sent"
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_campaign_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None,
|
||||||
|
) -> tuple[Campaign, CampaignVersion]:
|
||||||
|
campaign = (
|
||||||
|
session.query(Campaign)
|
||||||
|
.filter(Campaign.id == campaign_id, Campaign.tenant_id == tenant_id)
|
||||||
|
.one_or_none()
|
||||||
|
)
|
||||||
|
if not campaign:
|
||||||
|
raise MockCampaignSendError("Campaign not found or not accessible")
|
||||||
|
wanted_version_id = version_id or campaign.current_version_id
|
||||||
|
if not wanted_version_id:
|
||||||
|
raise MockCampaignSendError("Campaign has no current version")
|
||||||
|
version = session.get(CampaignVersion, wanted_version_id)
|
||||||
|
if not version or version.campaign_id != campaign.id:
|
||||||
|
raise MockCampaignSendError(
|
||||||
|
"Campaign version not found or not part of campaign"
|
||||||
|
)
|
||||||
|
return campaign, version
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_mailbox_for_run(*, send: bool, clear_mailbox: bool) -> Any | None:
|
||||||
|
mailbox = _require_mock_mailbox() if send or clear_mailbox else _mock_mailbox()
|
||||||
|
if clear_mailbox and mailbox is not None:
|
||||||
|
mailbox.clear_records()
|
||||||
|
return mailbox
|
||||||
|
|
||||||
|
|
||||||
|
def _build_mock_campaign_run(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
mailbox: Any | None,
|
||||||
|
send: bool,
|
||||||
|
include_warnings: bool,
|
||||||
|
include_needs_review: bool,
|
||||||
|
append_sent: bool,
|
||||||
|
check_files: bool,
|
||||||
|
) -> tuple[Any, Any, _MockSendBatch]:
|
||||||
|
files = files_integration()
|
||||||
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
raw_json,
|
||||||
|
managed_files_available=files.available,
|
||||||
|
)
|
||||||
|
with files.prepared_campaign_snapshot(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
raw_json=raw_json,
|
||||||
|
include_bytes=True,
|
||||||
|
prefix="govoplan-mock-send-",
|
||||||
|
) as prepared:
|
||||||
|
prepared_raw = load_campaign_json(prepared.path)
|
||||||
|
config = load_campaign_config_from_json(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
raw_json=prepared_raw,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
)
|
||||||
|
validation_report = validate_campaign_config(
|
||||||
|
config,
|
||||||
|
campaign_file=prepared.path,
|
||||||
|
check_files=check_files,
|
||||||
|
)
|
||||||
|
build_result = build_campaign_messages(
|
||||||
|
config,
|
||||||
|
campaign_file=prepared.path,
|
||||||
|
write_eml=False,
|
||||||
|
)
|
||||||
|
files.annotate_built_messages_with_managed_files(
|
||||||
|
build_result.built_messages,
|
||||||
|
prepared.managed_files_by_local_path,
|
||||||
|
)
|
||||||
|
send_batch = _mock_send_batch(
|
||||||
|
config=config,
|
||||||
|
built_messages=build_result.built_messages,
|
||||||
|
mailbox=mailbox,
|
||||||
|
send=send,
|
||||||
|
include_warnings=include_warnings,
|
||||||
|
include_needs_review=include_needs_review,
|
||||||
|
append_sent=append_sent,
|
||||||
|
)
|
||||||
|
return validation_report, build_result, send_batch
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_validation_payload(validation_report: Any) -> dict[str, Any]:
|
||||||
|
payload = validation_report.model_dump(mode="json")
|
||||||
|
payload.update(
|
||||||
|
{
|
||||||
|
"ok": validation_report.ok,
|
||||||
|
"error_count": validation_report.error_count,
|
||||||
|
"warning_count": validation_report.warning_count,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_build_payload(build_result: Any) -> dict[str, Any]:
|
||||||
|
report = build_result.report
|
||||||
|
payload = report.model_dump(mode="json")
|
||||||
|
payload.update(
|
||||||
|
{
|
||||||
|
"built_count": report.built_count,
|
||||||
|
"queueable_count": report.queueable_count,
|
||||||
|
"needs_review_count": report.needs_review_count,
|
||||||
|
"blocked_count": report.blocked_count,
|
||||||
|
"warning_count": report.warning_count,
|
||||||
|
"ready_count": report.ready_count,
|
||||||
|
"messages": [_message_payload(message) for message in report.messages],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_send_steps(
|
||||||
|
*,
|
||||||
|
validation_report: Any,
|
||||||
|
validation_payload: dict[str, Any],
|
||||||
|
build_report: Any,
|
||||||
|
send_batch: _MockSendBatch,
|
||||||
|
send: bool,
|
||||||
|
append_sent: bool,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"key": "validate",
|
||||||
|
"label": "Validate campaign JSON",
|
||||||
|
"status": "ok" if validation_report.ok else "needs_review",
|
||||||
|
"summary": validation_payload,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "build",
|
||||||
|
"label": "Build messages",
|
||||||
|
"status": "ok" if build_report.queueable_count else "needs_review",
|
||||||
|
"summary": {
|
||||||
|
"built": build_report.built_count,
|
||||||
|
"queueable": build_report.queueable_count,
|
||||||
|
"needs_review": build_report.needs_review_count,
|
||||||
|
"blocked": build_report.blocked_count,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "send",
|
||||||
|
"label": "Mock SMTP delivery",
|
||||||
|
"status": (
|
||||||
|
"skipped"
|
||||||
|
if not send
|
||||||
|
else "ok"
|
||||||
|
if send_batch.failed_count == 0
|
||||||
|
else "needs_review"
|
||||||
|
),
|
||||||
|
"summary": {
|
||||||
|
"attempted": send_batch.attempted_count,
|
||||||
|
"sent": send_batch.sent_count,
|
||||||
|
"failed": send_batch.failed_count,
|
||||||
|
"skipped": send_batch.skipped_count,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "imap",
|
||||||
|
"label": "Mock IMAP Sent append",
|
||||||
|
"status": (
|
||||||
|
"skipped"
|
||||||
|
if not send or not append_sent
|
||||||
|
else "ok"
|
||||||
|
if send_batch.imap_failed_count == 0
|
||||||
|
else "needs_review"
|
||||||
|
),
|
||||||
|
"summary": {
|
||||||
|
"appended": send_batch.imap_appended_count,
|
||||||
|
"failed": send_batch.imap_failed_count,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_campaign_send_response(
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
mailbox: Any | None,
|
||||||
|
validation_report: Any,
|
||||||
|
validation_payload: dict[str, Any],
|
||||||
|
build_result: Any,
|
||||||
|
build_payload: dict[str, Any],
|
||||||
|
send_batch: _MockSendBatch,
|
||||||
|
send: bool,
|
||||||
|
include_warnings: bool,
|
||||||
|
include_needs_review: bool,
|
||||||
|
append_sent: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"campaign_id": campaign.id,
|
||||||
|
"version_id": version.id,
|
||||||
|
"version_number": version.version_number,
|
||||||
|
"send_requested": send,
|
||||||
|
"include_warnings": include_warnings,
|
||||||
|
"include_needs_review": include_needs_review,
|
||||||
|
"append_sent": append_sent,
|
||||||
|
"steps": _mock_send_steps(
|
||||||
|
validation_report=validation_report,
|
||||||
|
validation_payload=validation_payload,
|
||||||
|
build_report=build_result.report,
|
||||||
|
send_batch=send_batch,
|
||||||
|
send=send,
|
||||||
|
append_sent=append_sent,
|
||||||
|
),
|
||||||
|
"validation": validation_payload,
|
||||||
|
"build": build_payload,
|
||||||
|
"send": {
|
||||||
|
"attempted_count": send_batch.attempted_count,
|
||||||
|
"sent_count": send_batch.sent_count,
|
||||||
|
"failed_count": send_batch.failed_count,
|
||||||
|
"skipped_count": send_batch.skipped_count,
|
||||||
|
"imap_appended_count": send_batch.imap_appended_count,
|
||||||
|
"imap_failed_count": send_batch.imap_failed_count,
|
||||||
|
"results": send_batch.results,
|
||||||
|
},
|
||||||
|
"mailbox": {
|
||||||
|
"messages": mailbox.list_records(limit=200) if mailbox is not None else []
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def run_mock_campaign_send(
|
def run_mock_campaign_send(
|
||||||
session: Session,
|
session: Session,
|
||||||
*,
|
*,
|
||||||
@@ -319,83 +551,38 @@ def run_mock_campaign_send(
|
|||||||
mailbox only when send=True.
|
mailbox only when send=True.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
campaign = session.query(Campaign).filter(Campaign.id == campaign_id, Campaign.tenant_id == tenant_id).one_or_none()
|
campaign, version = _mock_campaign_version(
|
||||||
if not campaign:
|
|
||||||
raise MockCampaignSendError("Campaign not found or not accessible")
|
|
||||||
wanted_version_id = version_id or campaign.current_version_id
|
|
||||||
if not wanted_version_id:
|
|
||||||
raise MockCampaignSendError("Campaign has no current version")
|
|
||||||
version = session.get(CampaignVersion, wanted_version_id)
|
|
||||||
if not version or version.campaign_id != campaign.id:
|
|
||||||
raise MockCampaignSendError("Campaign version not found or not part of campaign")
|
|
||||||
|
|
||||||
mailbox = _require_mock_mailbox() if send or clear_mailbox else _mock_mailbox()
|
|
||||||
if clear_mailbox and mailbox is not None:
|
|
||||||
mailbox.clear_records()
|
|
||||||
|
|
||||||
files = files_integration()
|
|
||||||
with files.prepared_campaign_snapshot(
|
|
||||||
session,
|
session,
|
||||||
tenant_id=tenant_id,
|
tenant_id=tenant_id,
|
||||||
campaign_id=campaign.id,
|
campaign_id=campaign_id,
|
||||||
raw_json=version.raw_json if isinstance(version.raw_json, dict) else {},
|
version_id=version_id,
|
||||||
include_bytes=True,
|
)
|
||||||
prefix="govoplan-mock-send-",
|
mailbox = _mock_mailbox_for_run(send=send, clear_mailbox=clear_mailbox)
|
||||||
) as prepared:
|
validation_report, build_result, send_batch = _build_mock_campaign_run(
|
||||||
prepared_raw = load_campaign_json(prepared.path)
|
session,
|
||||||
config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=prepared_raw, campaign_id=campaign.id)
|
tenant_id=tenant_id,
|
||||||
validation_report = validate_campaign_config(config, campaign_file=prepared.path, check_files=check_files)
|
campaign=campaign,
|
||||||
build_result = build_campaign_messages(config, campaign_file=prepared.path, write_eml=False)
|
version=version,
|
||||||
files.annotate_built_messages_with_managed_files(build_result.built_messages, prepared.managed_files_by_local_path)
|
mailbox=mailbox,
|
||||||
|
send=send,
|
||||||
send_batch = _mock_send_batch(
|
include_warnings=include_warnings,
|
||||||
config=config,
|
include_needs_review=include_needs_review,
|
||||||
built_messages=build_result.built_messages,
|
append_sent=append_sent,
|
||||||
mailbox=mailbox,
|
check_files=check_files,
|
||||||
send=send,
|
)
|
||||||
include_warnings=include_warnings,
|
validation_payload = _mock_validation_payload(validation_report)
|
||||||
include_needs_review=include_needs_review,
|
build_payload = _mock_build_payload(build_result)
|
||||||
append_sent=append_sent,
|
return _mock_campaign_send_response(
|
||||||
)
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
validation_json = validation_report.model_dump(mode="json")
|
mailbox=mailbox,
|
||||||
validation_json.update({"ok": validation_report.ok, "error_count": validation_report.error_count, "warning_count": validation_report.warning_count})
|
validation_report=validation_report,
|
||||||
build_report = build_result.report
|
validation_payload=validation_payload,
|
||||||
build_json = build_report.model_dump(mode="json")
|
build_result=build_result,
|
||||||
build_json.update({
|
build_payload=build_payload,
|
||||||
"built_count": build_report.built_count,
|
send_batch=send_batch,
|
||||||
"queueable_count": build_report.queueable_count,
|
send=send,
|
||||||
"needs_review_count": build_report.needs_review_count,
|
include_warnings=include_warnings,
|
||||||
"blocked_count": build_report.blocked_count,
|
include_needs_review=include_needs_review,
|
||||||
"warning_count": build_report.warning_count,
|
append_sent=append_sent,
|
||||||
"ready_count": build_report.ready_count,
|
)
|
||||||
"messages": [_message_payload(message) for message in build_report.messages],
|
|
||||||
})
|
|
||||||
|
|
||||||
return {
|
|
||||||
"campaign_id": campaign.id,
|
|
||||||
"version_id": version.id,
|
|
||||||
"version_number": version.version_number,
|
|
||||||
"send_requested": send,
|
|
||||||
"include_warnings": include_warnings,
|
|
||||||
"include_needs_review": include_needs_review,
|
|
||||||
"append_sent": append_sent,
|
|
||||||
"steps": [
|
|
||||||
{"key": "validate", "label": "Validate campaign JSON", "status": "ok" if validation_report.ok else "needs_review", "summary": validation_json},
|
|
||||||
{"key": "build", "label": "Build messages", "status": "ok" if build_report.queueable_count else "needs_review", "summary": {"built": build_report.built_count, "queueable": build_report.queueable_count, "needs_review": build_report.needs_review_count, "blocked": build_report.blocked_count}},
|
|
||||||
{"key": "send", "label": "Mock SMTP delivery", "status": "skipped" if not send else ("ok" if send_batch.failed_count == 0 else "needs_review"), "summary": {"attempted": send_batch.attempted_count, "sent": send_batch.sent_count, "failed": send_batch.failed_count, "skipped": send_batch.skipped_count}},
|
|
||||||
{"key": "imap", "label": "Mock IMAP Sent append", "status": "skipped" if not send or not append_sent else ("ok" if send_batch.imap_failed_count == 0 else "needs_review"), "summary": {"appended": send_batch.imap_appended_count, "failed": send_batch.imap_failed_count}},
|
|
||||||
],
|
|
||||||
"validation": validation_json,
|
|
||||||
"build": build_json,
|
|
||||||
"send": {
|
|
||||||
"attempted_count": send_batch.attempted_count,
|
|
||||||
"sent_count": send_batch.sent_count,
|
|
||||||
"failed_count": send_batch.failed_count,
|
|
||||||
"skipped_count": send_batch.skipped_count,
|
|
||||||
"imap_appended_count": send_batch.imap_appended_count,
|
|
||||||
"imap_failed_count": send_batch.imap_failed_count,
|
|
||||||
"results": send_batch.results,
|
|
||||||
},
|
|
||||||
"mailbox": {"messages": mailbox.list_records(limit=200) if mailbox is not None else []},
|
|
||||||
}
|
|
||||||
|
|||||||
755
src/govoplan_campaign/backend/documentation.py
Normal file
755
src/govoplan_campaign/backend/documentation.py
Normal file
@@ -0,0 +1,755 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from govoplan_core.core.modules import DocumentationCondition, DocumentationContext, DocumentationLink, DocumentationTopic
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.delivery_policy import (
|
||||||
|
CampaignDeliveryPolicyError,
|
||||||
|
effective_synchronous_send_policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_CAMPAIGN_USER_SCOPES = (
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:create",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:copy",
|
||||||
|
"campaigns:campaign:archive",
|
||||||
|
"campaigns:campaign:delete",
|
||||||
|
"campaigns:campaign:share",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:campaign:build",
|
||||||
|
"campaigns:campaign:review",
|
||||||
|
"campaigns:campaign:send_test",
|
||||||
|
"campaigns:campaign:queue",
|
||||||
|
"campaigns:campaign:control",
|
||||||
|
"campaigns:campaign:send",
|
||||||
|
"campaigns:campaign:retry",
|
||||||
|
"campaigns:campaign:reconcile",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
"campaigns:recipient:export",
|
||||||
|
"campaigns:report:read",
|
||||||
|
"campaigns:report:export",
|
||||||
|
"campaigns:report:send",
|
||||||
|
)
|
||||||
|
|
||||||
|
_CAMPAIGN_HELP_CONTEXTS = (
|
||||||
|
"campaigns.list",
|
||||||
|
"campaign.overview",
|
||||||
|
)
|
||||||
|
|
||||||
|
_FILES_INTEGRATION = "files.campaign_attachments"
|
||||||
|
_MAIL_INTEGRATION = "mail.campaign_delivery"
|
||||||
|
_ADDRESSES_LOOKUP_INTEGRATION = "addresses.lookup"
|
||||||
|
_ADDRESSES_SOURCE_INTEGRATION = "addresses.recipient_source"
|
||||||
|
_NOTIFICATIONS_INTEGRATION = "notifications.dispatch"
|
||||||
|
|
||||||
|
|
||||||
|
def _workflow_topic(
|
||||||
|
*,
|
||||||
|
topic_id: str,
|
||||||
|
title: str,
|
||||||
|
summary: str,
|
||||||
|
body: str,
|
||||||
|
order: int,
|
||||||
|
audience: tuple[str, ...],
|
||||||
|
required_scopes: tuple[str, ...],
|
||||||
|
route: str,
|
||||||
|
screen: str,
|
||||||
|
help_contexts: tuple[str, ...],
|
||||||
|
prerequisites: tuple[str, ...],
|
||||||
|
steps: tuple[str, ...],
|
||||||
|
outcome: str,
|
||||||
|
verification: str,
|
||||||
|
related_topic_ids: tuple[str, ...] = (),
|
||||||
|
required_modules: tuple[str, ...] = ("campaigns",),
|
||||||
|
required_capabilities: tuple[str, ...] = (),
|
||||||
|
links: tuple[DocumentationLink, ...] = (DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),),
|
||||||
|
related_modules: tuple[str, ...] = (),
|
||||||
|
limitations: tuple[str, ...] = (),
|
||||||
|
) -> DocumentationTopic:
|
||||||
|
metadata: dict[str, object] = {
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": route,
|
||||||
|
"screen": screen,
|
||||||
|
"help_contexts": list(help_contexts),
|
||||||
|
"prerequisites": list(prerequisites),
|
||||||
|
"steps": list(steps),
|
||||||
|
"outcome": outcome,
|
||||||
|
"verification": verification,
|
||||||
|
"related_topic_ids": list(related_topic_ids),
|
||||||
|
}
|
||||||
|
if limitations:
|
||||||
|
metadata["limitations"] = list(limitations)
|
||||||
|
return DocumentationTopic(
|
||||||
|
id=topic_id,
|
||||||
|
title=title,
|
||||||
|
summary=summary,
|
||||||
|
body=body,
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("user",),
|
||||||
|
audience=audience,
|
||||||
|
order=order,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=required_modules,
|
||||||
|
required_capabilities=required_capabilities,
|
||||||
|
required_scopes=required_scopes,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=links,
|
||||||
|
related_modules=related_modules,
|
||||||
|
unlocks=(outcome,),
|
||||||
|
source_module_id="campaigns",
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
CAMPAIGN_USER_DOCUMENTATION = (
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.create-campaign",
|
||||||
|
title="Create a campaign",
|
||||||
|
summary="Start a governed campaign as an editable draft and complete its purpose and ownership before adding delivery data.",
|
||||||
|
body="A new campaign starts with one editable working version. Creating it does not grant access to Mail profiles, managed files, address sources, or delivery actions; those remain separately authorized.",
|
||||||
|
order=30,
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:create"),
|
||||||
|
route="/campaigns",
|
||||||
|
screen="Campaigns",
|
||||||
|
help_contexts=("campaigns.list",),
|
||||||
|
prerequisites=("You may create campaigns in the active tenant.",),
|
||||||
|
steps=(
|
||||||
|
"Open Campaigns and select New campaign.",
|
||||||
|
"Use the creation wizard to enter a clear name, identifier, and purpose.",
|
||||||
|
"Open the new campaign and confirm its owner before adding recipient or delivery data.",
|
||||||
|
"Continue through the preparation sections and save the editable working version.",
|
||||||
|
),
|
||||||
|
outcome="An owned campaign draft with an editable working version.",
|
||||||
|
verification="The Campaign overview shows the new campaign as a draft and identifies its current working version.",
|
||||||
|
related_topic_ids=("campaigns.workflow.prepare-validate-and-build", "campaigns.workflow.import-recipients"),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.create-editable-successor",
|
||||||
|
title="Create an editable successor",
|
||||||
|
summary="Continue work after a permanent or delivery-final lock without rewriting the preserved version.",
|
||||||
|
body="Create editable copy means creating the campaign's next working version. Validation locks and temporary user locks are removed in place instead; they must not create parallel drafts.",
|
||||||
|
order=31,
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:copy", "campaigns:recipient:read"),
|
||||||
|
route="/campaigns/{campaign_id}",
|
||||||
|
screen="Campaign workspace",
|
||||||
|
help_contexts=("campaign.overview", "campaign.audit"),
|
||||||
|
prerequisites=(
|
||||||
|
"You have write access to the selected campaign.",
|
||||||
|
"Its current version is permanently user-locked or delivery-final.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open the locked current version and review the reason it is read-only.",
|
||||||
|
"Select Create editable copy in the locked-version notice.",
|
||||||
|
"If the notice instead offers an unlock action, unlock that validation or temporary user lock rather than creating a successor.",
|
||||||
|
"Review the new working version, reselect any Mail profile when requested, and validate and build again before delivery.",
|
||||||
|
),
|
||||||
|
outcome="A new editable working version while the source version and its evidence remain unchanged.",
|
||||||
|
verification="The Campaign overview identifies a new current version number and the earlier version remains in history.",
|
||||||
|
related_topic_ids=("campaigns.workflow.prepare-validate-and-build", "campaigns.mail-profile-user-journey"),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.import-recipients",
|
||||||
|
title="Import recipients into a campaign",
|
||||||
|
summary="Turn a text, CSV, or spreadsheet table into reviewed campaign-local recipient rows with source provenance.",
|
||||||
|
body="Import copies valid rows into the editable campaign version. Invalid rows stay visible during preview instead of disappearing, and later changes to the source file do not silently change the saved campaign.",
|
||||||
|
order=33,
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/recipients",
|
||||||
|
screen="Recipient data",
|
||||||
|
help_contexts=("campaign.recipients", "campaign.recipient-data"),
|
||||||
|
prerequisites=(
|
||||||
|
"The current campaign version is editable and you have write access to it.",
|
||||||
|
"The source is tabular and contains only data needed for this campaign.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Recipient data and select Import.",
|
||||||
|
"Upload or paste the table, then confirm its encoding, sheet, header rows, and separator where applicable.",
|
||||||
|
"Map address and campaign fields, choose append or replace, and review every invalid or excluded row.",
|
||||||
|
"Select Import valid rows, inspect the resulting recipient table, and save the campaign page.",
|
||||||
|
),
|
||||||
|
outcome="A campaign-local recipient snapshot with mapping and source evidence.",
|
||||||
|
verification="Reopen Recipient data, confirm the expected row count and addressing, then validate before building messages.",
|
||||||
|
related_topic_ids=("campaigns.workflow.import-address-source", "campaigns.workflow.prepare-validate-and-build"),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.import-address-source",
|
||||||
|
title="Import a recipient source from Addresses",
|
||||||
|
summary="Copy a permitted reusable address book or list into the campaign as a traceable versioned snapshot.",
|
||||||
|
body="Campaign does not follow the address source live. It records the selected source revision and warns when a newer source revision is available, so re-import is always an explicit author action.",
|
||||||
|
order=32,
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
required_modules=("campaigns", "addresses"),
|
||||||
|
required_capabilities=(_ADDRESSES_SOURCE_INTEGRATION,),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/recipients",
|
||||||
|
screen="Recipient data",
|
||||||
|
help_contexts=("campaign.recipients", "campaign.recipient-data"),
|
||||||
|
prerequisites=(
|
||||||
|
"Addresses offers at least one source visible to you.",
|
||||||
|
"The current campaign version is editable and you have write access to it.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Recipient data and select Import address book/list.",
|
||||||
|
"Choose the permitted source and review the returned snapshot and its revision.",
|
||||||
|
"Choose append or replace, import the snapshot, inspect the recipient rows, and save.",
|
||||||
|
"When a stale-source warning appears later, compare the source and re-import deliberately if the campaign should use the new revision.",
|
||||||
|
),
|
||||||
|
outcome="A campaign-local recipient snapshot whose Addresses source and revision can be traced.",
|
||||||
|
verification="Recipient data records the source provenance, and a later source revision does not alter the saved rows until you re-import it.",
|
||||||
|
related_topic_ids=("campaigns.workflow.import-recipients", "campaigns.workflow.prepare-validate-and-build"),
|
||||||
|
related_modules=("addresses",),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.use-managed-attachments",
|
||||||
|
title="Use managed files as campaign attachments",
|
||||||
|
summary="Select governed file versions, preview rule matches, and preserve exactly which files were used for the campaign build.",
|
||||||
|
body="Managed attachments remain owned by Files. Campaign stores governed references and frozen build evidence; it does not copy Files administration authority or accept arbitrary server paths.",
|
||||||
|
order=34,
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
required_modules=("campaigns", "files"),
|
||||||
|
required_capabilities=(_FILES_INTEGRATION,),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"files:file:read",
|
||||||
|
"files:file:share",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/files",
|
||||||
|
screen="Attachments",
|
||||||
|
help_contexts=("campaign.attachments",),
|
||||||
|
prerequisites=(
|
||||||
|
"The current campaign version is editable and you have write access to it.",
|
||||||
|
"The required file versions exist in Files and may be shared with this campaign.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Attachments and choose managed files or patterns from Files.",
|
||||||
|
"Define campaign-wide and recipient-specific rules, including unmatched-file and archive behavior.",
|
||||||
|
"Preview the matches and link the exact managed versions to the campaign.",
|
||||||
|
"Save, validate, and build, then inspect the resolved attachment evidence in Review and send.",
|
||||||
|
),
|
||||||
|
outcome="Governed file versions linked to the campaign and frozen into the exact build evidence.",
|
||||||
|
verification="Confirm the expected filenames, paths, matches, and link state in the Campaign UI. Exact managed versions and checksums remain retained in build evidence for authorized supporting tools.",
|
||||||
|
related_topic_ids=("campaigns.workflow.prepare-validate-and-build",),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||||
|
),
|
||||||
|
related_modules=("files",),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.queue-delivery",
|
||||||
|
title="Queue a campaign for controlled delivery",
|
||||||
|
summary="Place an exact reviewed build into the worker queue while preserving recipient-level state and retry protection.",
|
||||||
|
body="Queueing is a controlled state change, not proof of delivery. Ordinary batches should use background workers, and accepted or outcome-unknown effects remain protected from blind retry.",
|
||||||
|
order=35,
|
||||||
|
audience=("campaign_sender", "campaign_operator"),
|
||||||
|
required_modules=("campaigns", "mail"),
|
||||||
|
required_capabilities=(_MAIL_INTEGRATION,),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:queue",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"mail:profile:use",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/review",
|
||||||
|
screen="Review and send",
|
||||||
|
help_contexts=(),
|
||||||
|
prerequisites=(
|
||||||
|
"The selected version is validated, locked, built, and reviewed.",
|
||||||
|
"Its Mail profile remains available and authorized for the current campaign context.",
|
||||||
|
"You have write access to the selected campaign.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Review and send and confirm the selected version, build, recipients, warnings, and review completion.",
|
||||||
|
"Select Queue for workers and confirm the exact durable execution that will be committed.",
|
||||||
|
"Remain on Review and send or leave and return later; both views read the same durable job states.",
|
||||||
|
"Use the delivery controls to pause, resume, cancel unsent work, or retry eligible failures without requeueing accepted or uncertain outcomes.",
|
||||||
|
),
|
||||||
|
outcome="Eligible jobs queued with an auditable execution snapshot and protected delivery state.",
|
||||||
|
verification="The Report shows the selected version's jobs as queued or progressing, without changing any accepted job back to retryable.",
|
||||||
|
related_topic_ids=("campaigns.workflow.complete-review", "campaigns.workflow.retry-and-reconcile"),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
|
||||||
|
),
|
||||||
|
related_modules=("mail", "notifications"),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.send-small-controlled-run",
|
||||||
|
title="Send a campaign immediately",
|
||||||
|
summary="Run the eligible jobs synchronously only after deliberately confirming that the reviewed campaign is small enough for an interactive request.",
|
||||||
|
body="Send now is protected by an effective deployment/tenant recipient-job maximum. The server counts the exact persisted eligible build, rejects an oversized or empty run before SMTP, and preflights every message and the Mail profile revision before the first provider effect.",
|
||||||
|
order=36,
|
||||||
|
audience=("campaign_sender", "campaign_operator"),
|
||||||
|
required_modules=("campaigns", "mail"),
|
||||||
|
required_capabilities=(_MAIL_INTEGRATION,),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:send",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"mail:profile:use",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/review",
|
||||||
|
screen="Review and send",
|
||||||
|
help_contexts=("campaign.review-send",),
|
||||||
|
prerequisites=(
|
||||||
|
"The selected version is validated, locked, built, and reviewed.",
|
||||||
|
"The exact eligible recipient-job count is within the effective limit shown on Review and send.",
|
||||||
|
"Its Mail profile remains available and authorized for the current campaign context.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Review and send and inspect the exact build and delivery readiness checks.",
|
||||||
|
"Review the effective synchronous limit and exact eligible count; use Queue for workers when Send now is unavailable or for an ordinary batch.",
|
||||||
|
"Select Send now and confirm the protected delivery action.",
|
||||||
|
"Open Report and inspect each resulting delivery state before attempting any recovery action.",
|
||||||
|
),
|
||||||
|
outcome="A synchronous real delivery run with recipient-level attempt and outcome evidence.",
|
||||||
|
verification="The Report distinguishes accepted, failed, unattempted, cancelled, and outcome-unknown jobs and retains their attempt history.",
|
||||||
|
related_topic_ids=("campaigns.workflow.queue-delivery", "campaigns.workflow.retry-and-reconcile"),
|
||||||
|
related_modules=("mail",),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.view-aggregate-delivery-report",
|
||||||
|
title="Review aggregate campaign outcomes",
|
||||||
|
summary="Inspect privacy-protected Campaign totals without receiving recipient rows, message content, delivery diagnostics, or export authority.",
|
||||||
|
body="The aggregate Reports view exposes only approved campaign-level business outcomes. Positive cells below the configured threshold are suppressed together with a complementary value or the denominator when needed, so totals cannot be subtracted to recover a small group.",
|
||||||
|
order=37,
|
||||||
|
audience=("campaign_aggregate_reader", "campaign_reader", "campaign_manager"),
|
||||||
|
required_scopes=("campaigns:report:read",),
|
||||||
|
route="/campaigns/reports",
|
||||||
|
screen="Reports",
|
||||||
|
help_contexts=("campaign.report",),
|
||||||
|
prerequisites=("The campaign is owned by or explicitly shared with you, or you hold tenant-wide authority.",),
|
||||||
|
steps=(
|
||||||
|
"Open Reports and select a campaign available to you.",
|
||||||
|
"Read the stated denominator before comparing accepted, failed, unknown, active, excluded, cancelled, and unattempted outcomes.",
|
||||||
|
"Treat Suppressed as an intentional privacy boundary rather than zero or missing data.",
|
||||||
|
"Request separately authorized recipient-level access only when the task genuinely requires individual evidence.",
|
||||||
|
),
|
||||||
|
outcome="A business-level Campaign outcome view with small-group and recipient privacy preserved.",
|
||||||
|
verification="No row, address, message, attachment, diagnostic, filter, drill-down, or export action is available from the aggregate view.",
|
||||||
|
related_topic_ids=("campaigns.workflow.view-delivery-report",),
|
||||||
|
links=(DocumentationLink(label="Aggregate Campaign reports", href="/campaigns/reports", kind="runtime"),),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.view-delivery-report",
|
||||||
|
title="Review campaign delivery details",
|
||||||
|
summary="Inspect delivery totals and recipient-level job evidence in the current Campaign Report UI.",
|
||||||
|
body="The recipient-aware Campaign Report requires campaign-read, report-read, and recipient-read authority. Infrastructure diagnostics remain separately authorized, and the server checks every direct detail route independently of the interface.",
|
||||||
|
order=38,
|
||||||
|
audience=("campaign_reader", "campaign_manager", "campaign_reviewer", "campaign_sender"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:report:read", "campaigns:recipient:read"),
|
||||||
|
route="/campaigns/{campaign_id}/report",
|
||||||
|
screen="Campaign Report",
|
||||||
|
help_contexts=("campaign.report",),
|
||||||
|
prerequisites=("You may read the selected campaign and its report.",),
|
||||||
|
steps=(
|
||||||
|
"Open the campaign and select Report.",
|
||||||
|
"Review totals for queued, accepted, failed, cancelled, excluded/skipped, unattempted, and outcome-unknown jobs.",
|
||||||
|
"Inspect the authorized recipient-level rows and attempt history.",
|
||||||
|
"Treat outcome-unknown jobs as unresolved and hand them to an authorized operator for evidence-backed reconciliation.",
|
||||||
|
),
|
||||||
|
outcome="A recipient-aware view of delivery progress and outcomes with diagnostics still separately protected.",
|
||||||
|
verification="The report totals and job rows match the selected campaign version, and infrastructure diagnostics are absent unless separately authorized.",
|
||||||
|
related_topic_ids=("campaigns.workflow.view-aggregate-delivery-report", "campaigns.workflow.retry-and-reconcile", "campaigns.workflow.export-delivery-report"),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.export-delivery-report",
|
||||||
|
title="Export recipient delivery results",
|
||||||
|
summary="Download an authorized CSV snapshot of recipient-level campaign delivery results for controlled downstream use.",
|
||||||
|
body="A report export contains personal and delivery evidence. Store, transmit, retain, and delete it according to the campaign's purpose and the applicable export and retention policy.",
|
||||||
|
order=39,
|
||||||
|
audience=("campaign_report_exporter", "campaign_auditor"),
|
||||||
|
required_scopes=(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:report:read",
|
||||||
|
"campaigns:report:export",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:export",
|
||||||
|
),
|
||||||
|
route="/campaigns/{campaign_id}/report",
|
||||||
|
screen="Campaign Report",
|
||||||
|
help_contexts=("campaign.report", "campaign.audit"),
|
||||||
|
prerequisites=(
|
||||||
|
"You may export both campaign reports and recipient data.",
|
||||||
|
"The export has an approved purpose and destination.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open the campaign Report and select the intended version.",
|
||||||
|
"Confirm that recipient-level export is necessary for the task.",
|
||||||
|
"Select Download CSV and store the result only in the approved location.",
|
||||||
|
"Verify the selected version and row counts, then apply the required retention or deletion rule to the downloaded copy.",
|
||||||
|
),
|
||||||
|
outcome="A point-in-time CSV export of authorized campaign job results.",
|
||||||
|
verification="The downloaded file identifies the intended campaign/version and contains business evidence without credentials or ordinary-reader infrastructure details.",
|
||||||
|
related_topic_ids=("campaigns.workflow.view-delivery-report",),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.share-campaign",
|
||||||
|
title="Share a campaign",
|
||||||
|
summary="Grant a user or group explicit view or write access to one campaign without widening their platform permissions.",
|
||||||
|
body="A share can only narrow access to the selected campaign within the recipient's existing role. It never grants Mail profile use, Files authority, tenant-wide recipient access, or a missing Campaign action.",
|
||||||
|
order=39,
|
||||||
|
audience=("campaign_owner", "campaign_access_manager"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:share"),
|
||||||
|
route="/campaigns/{campaign_id}/global-settings",
|
||||||
|
screen="Ownership and sharing",
|
||||||
|
help_contexts=("campaign.overview", "campaign.global-settings"),
|
||||||
|
prerequisites=(
|
||||||
|
"You have write access to the selected campaign.",
|
||||||
|
"The target user or group already has an appropriate Campaign role for the intended actions.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Global settings and find Ownership and sharing.",
|
||||||
|
"Select Share, choose a user or group, and choose Can view or Can edit and operate.",
|
||||||
|
"Save the share and verify the target appears in the sharing table with the intended level.",
|
||||||
|
"Use the row action to revoke the explicit share when it is no longer needed.",
|
||||||
|
),
|
||||||
|
outcome="Explicit campaign access for the selected subject, bounded by that subject's existing permissions.",
|
||||||
|
verification="The sharing table shows the active target and level; revocation removes its explicit access while preserving the audit record.",
|
||||||
|
related_topic_ids=("campaigns.reference.composition-assurance",),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.archive-campaign",
|
||||||
|
title="Archive a campaign",
|
||||||
|
summary="Remove a completed campaign from active work while preserving its versions, outcomes, and audit evidence.",
|
||||||
|
body="Archive only after queued, sending, and outcome-unknown work has been resolved. Archiving preserves evidence and is the correct lifecycle action for any campaign with build, lock, or delivery history.",
|
||||||
|
order=40,
|
||||||
|
audience=("campaign_owner", "campaign_records_manager"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:archive"),
|
||||||
|
route="/campaigns/{campaign_id}",
|
||||||
|
screen="Campaign lifecycle",
|
||||||
|
help_contexts=("campaign.overview", "campaign.report", "campaign.audit"),
|
||||||
|
prerequisites=(
|
||||||
|
"You have write access to the selected campaign.",
|
||||||
|
"No delivery job is queued, sending, or awaiting uncertain-outcome reconciliation.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Open Report and resolve every active or outcome-unknown delivery state.",
|
||||||
|
"Confirm that the campaign should leave active work while its evidence remains retained.",
|
||||||
|
"Invoke the authorized Archive action from a supporting client.",
|
||||||
|
"Reopen or query the campaign and confirm its state is Archived.",
|
||||||
|
),
|
||||||
|
outcome="An archived campaign whose versions, reports, and audit evidence remain preserved.",
|
||||||
|
verification="The campaign state is Archived and its report remains available. Ask an authorized audit reader to verify the platform audit event.",
|
||||||
|
related_topic_ids=("campaigns.workflow.view-delivery-report", "campaigns.workflow.delete-untouched-draft"),
|
||||||
|
limitations=(
|
||||||
|
"The current Campaign Web UI does not yet expose the archive action; use an authorized supporting client or API.",
|
||||||
|
"The Campaign-local Audit page is not integrated yet; audit verification uses the platform audit surface or API.",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
_workflow_topic(
|
||||||
|
topic_id="campaigns.workflow.delete-untouched-draft",
|
||||||
|
title="Delete an untouched campaign draft",
|
||||||
|
summary="Immediately remove a draft that has no protected build, lock, publication, snapshot, or delivery evidence.",
|
||||||
|
body="Deletion is deliberately narrower than archive. It marks an eligible draft deleted and emits an audit record; it cannot erase a campaign that already carries evidence that must be retained.",
|
||||||
|
order=41,
|
||||||
|
audience=("campaign_owner", "campaign_records_manager"),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:delete"),
|
||||||
|
route="/campaigns/{campaign_id}",
|
||||||
|
screen="Campaign lifecycle",
|
||||||
|
help_contexts=("campaign.overview", "campaign.audit"),
|
||||||
|
prerequisites=(
|
||||||
|
"You have write access to the selected campaign.",
|
||||||
|
"The campaign is still a draft and has no jobs, locks, published version, or execution snapshot.",
|
||||||
|
),
|
||||||
|
steps=(
|
||||||
|
"Confirm that the draft is not needed and contains no evidence that should be retained.",
|
||||||
|
"Invoke the authorized Delete action from a supporting client and confirm the destructive action.",
|
||||||
|
"If deletion is refused because protected evidence exists, archive the campaign after resolving any active delivery state.",
|
||||||
|
"Verify that the deleted draft no longer appears in active Campaigns and that the audit event exists.",
|
||||||
|
),
|
||||||
|
outcome="An eligible untouched draft removed from active Campaigns with attributable deletion evidence.",
|
||||||
|
verification="The draft is no longer returned as an active campaign. Ask an authorized audit reader to verify who deleted it and when through the platform audit surface.",
|
||||||
|
related_topic_ids=("campaigns.workflow.archive-campaign",),
|
||||||
|
limitations=(
|
||||||
|
"The current Campaign Web UI does not yet expose the delete action; use an authorized supporting client or API.",
|
||||||
|
"The Campaign-local Audit page is not integrated yet; audit verification uses the platform audit surface or API.",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def documentation_topics(context: DocumentationContext) -> tuple[DocumentationTopic, ...]:
|
||||||
|
"""Describe the current Campaign composition without resolving module data.
|
||||||
|
|
||||||
|
The provider deliberately uses only the actor's permission evaluator and the
|
||||||
|
registry's declared contracts. Resource access, policy decisions, and
|
||||||
|
campaign state remain authoritative at the point where an action is used.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if context.documentation_type != "user":
|
||||||
|
return ()
|
||||||
|
principal = context.principal
|
||||||
|
if principal is None or not _has_any_scope(principal, _CAMPAIGN_USER_SCOPES):
|
||||||
|
return ()
|
||||||
|
|
||||||
|
mail_available = _integration_available(context.registry, _MAIL_INTEGRATION)
|
||||||
|
current_configuration = list(_actor_capabilities(principal, mail_available=mail_available))
|
||||||
|
integration_configuration, integration_limitations = _integration_summary(context.registry, principal)
|
||||||
|
current_configuration.extend(integration_configuration)
|
||||||
|
delivery_configuration, delivery_limitations = _delivery_policy_summary(context)
|
||||||
|
current_configuration.extend(delivery_configuration)
|
||||||
|
limitations = [
|
||||||
|
"Access to a particular campaign still depends on its owner or sharing rules and on the campaign's current state.",
|
||||||
|
"Profile, file, and address pickers re-evaluate the actual resources visible in the selected campaign; this overview does not claim that an eligible item currently exists.",
|
||||||
|
*integration_limitations,
|
||||||
|
*delivery_limitations,
|
||||||
|
]
|
||||||
|
|
||||||
|
return (
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.current-composition",
|
||||||
|
title="Your Campaign role and installed composition",
|
||||||
|
summary="This guide separates actions authorized by your role from optional services connected to Campaign in this installation.",
|
||||||
|
body=_composition_body(current_configuration, limitations),
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("user",),
|
||||||
|
audience=("campaign_user",),
|
||||||
|
order=29,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns",),
|
||||||
|
any_scopes=_CAMPAIGN_USER_SCOPES,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),),
|
||||||
|
related_modules=("mail", "files", "addresses", "notifications"),
|
||||||
|
source_module_id="campaigns",
|
||||||
|
metadata={
|
||||||
|
"kind": "reference",
|
||||||
|
"route": "/campaigns",
|
||||||
|
"screen": "Campaigns",
|
||||||
|
"help_contexts": list(_CAMPAIGN_HELP_CONTEXTS),
|
||||||
|
"current_configuration": current_configuration,
|
||||||
|
"limitations": limitations,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _actor_capabilities(principal: object, *, mail_available: bool) -> tuple[str, ...]:
|
||||||
|
capabilities: list[str] = []
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:read",), "Open campaigns shared with you and inspect their business state.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:create",), "Create new campaigns.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:update",), "Edit eligible working campaign versions.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:copy",), "Create an editable successor from an eligible existing version.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:recipient:read",), "Inspect recipients and recipient-specific campaign data.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:recipient:write",), "Add and edit recipient rows.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:recipient:import",), "Import recipient snapshots.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:validate",), "Validate campaign inputs and resolve blocking issues.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:build",), "Build exact recipient messages for review.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:review",), "Record review completion for an exact build.")
|
||||||
|
if mail_available:
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:send_test",), "Run authorized delivery verification tools.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:queue",), "Queue an eligible reviewed campaign for controlled delivery.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:control",), "Pause, resume, or cancel eligible delivery jobs.")
|
||||||
|
if mail_available:
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:send",), "Start an eligible real delivery run.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:retry",), "Retry delivery jobs whose recorded state permits a retry.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:reconcile",), "Reconcile delivery effects whose outcome is uncertain.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:report:read",), "View authorized campaign delivery reports.")
|
||||||
|
_append_if(
|
||||||
|
capabilities,
|
||||||
|
principal,
|
||||||
|
("campaigns:report:export", "campaigns:recipient:export"),
|
||||||
|
"Export authorized delivery and recipient report data.",
|
||||||
|
require_all=True,
|
||||||
|
)
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:share",), "Manage explicit access to eligible campaigns.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:archive",), "Archive eligible campaigns while retaining their evidence.")
|
||||||
|
_append_if(capabilities, principal, ("campaigns:campaign:delete",), "Delete eligible untouched drafts.")
|
||||||
|
if mail_available:
|
||||||
|
_append_if(
|
||||||
|
capabilities,
|
||||||
|
principal,
|
||||||
|
("campaigns:report:send", "campaigns:report:read", "mail:profile:use"),
|
||||||
|
"Send an authorized campaign report through a Mail profile.",
|
||||||
|
require_all=True,
|
||||||
|
)
|
||||||
|
return tuple(capabilities)
|
||||||
|
|
||||||
|
|
||||||
|
def _integration_summary(registry: object, principal: object) -> tuple[tuple[str, ...], tuple[str, ...]]:
|
||||||
|
configured: list[str] = []
|
||||||
|
limitations: list[str] = []
|
||||||
|
|
||||||
|
mail_available = _integration_available(registry, _MAIL_INTEGRATION)
|
||||||
|
can_select_mail_profile = _has_all_scopes(
|
||||||
|
principal,
|
||||||
|
("campaigns:campaign:read", "campaigns:campaign:update", "mail:profile:use"),
|
||||||
|
)
|
||||||
|
can_deliver_with_mail = _has_scope(principal, "mail:profile:use") and _has_any_scope(
|
||||||
|
principal,
|
||||||
|
("campaigns:campaign:queue", "campaigns:campaign:send", "campaigns:campaign:retry"),
|
||||||
|
)
|
||||||
|
if mail_available and can_select_mail_profile:
|
||||||
|
configured.append("Installed composition: Campaign can open Mail's actor-filtered profile picker while you edit; eligible profiles are resolved in the selected campaign context.")
|
||||||
|
elif mail_available and can_deliver_with_mail:
|
||||||
|
configured.append("Installed composition: Mail-backed Campaign delivery is connected, and the selected profile is re-authorized for each delivery action.")
|
||||||
|
elif mail_available:
|
||||||
|
limitations.append("Mail delivery is configured, but selecting a Mail profile is not included in your current tasks.")
|
||||||
|
else:
|
||||||
|
limitations.append("Mail-backed Campaign delivery is not available in the current composition.")
|
||||||
|
|
||||||
|
files_available = _integration_available(registry, _FILES_INTEGRATION)
|
||||||
|
can_preview_files = _has_all_scopes(
|
||||||
|
principal,
|
||||||
|
("campaigns:campaign:read", "campaigns:recipient:read", "files:file:read"),
|
||||||
|
)
|
||||||
|
can_link_files = _has_all_scopes(
|
||||||
|
principal,
|
||||||
|
(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"files:file:read",
|
||||||
|
"files:file:share",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if files_available and can_link_files:
|
||||||
|
configured.append("Installed composition: Managed file versions can be selected, previewed, and linked as governed campaign attachments when the selected campaign and files pass their resource checks.")
|
||||||
|
elif files_available and can_preview_files:
|
||||||
|
configured.append("Installed composition: Managed campaign attachments can be previewed; linking a version requires campaign-update, validation, and attachment-sharing authority.")
|
||||||
|
elif files_available:
|
||||||
|
limitations.append("Managed attachments are configured, but they are not included in your current Campaign tasks.")
|
||||||
|
else:
|
||||||
|
limitations.append("Managed file attachments are not available in the current composition; campaign-owned uploads remain separate.")
|
||||||
|
|
||||||
|
lookup_available = _integration_available(registry, _ADDRESSES_LOOKUP_INTEGRATION)
|
||||||
|
source_available = _integration_available(registry, _ADDRESSES_SOURCE_INTEGRATION)
|
||||||
|
if lookup_available and _has_scope(principal, "campaigns:recipient:read"):
|
||||||
|
configured.append("Installed composition: Address records can be looked up while preparing recipients.")
|
||||||
|
elif lookup_available:
|
||||||
|
limitations.append("Address lookup is configured, but recipient inspection is not included in your current Campaign tasks.")
|
||||||
|
else:
|
||||||
|
limitations.append("Connected address lookup is not available in the current composition.")
|
||||||
|
can_import_source = _has_all_scopes(
|
||||||
|
principal,
|
||||||
|
(
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if source_available and can_import_source:
|
||||||
|
configured.append("Installed composition: The actor-filtered Addresses source picker can copy a selected source into a campaign as a traceable recipient snapshot.")
|
||||||
|
elif source_available:
|
||||||
|
limitations.append("Connected recipient sources are configured, but source import is not included in your current Campaign tasks.")
|
||||||
|
else:
|
||||||
|
limitations.append("Connected recipient-source snapshots are not available; direct campaign imports remain available when authorized.")
|
||||||
|
|
||||||
|
if _integration_available(registry, _NOTIFICATIONS_INTEGRATION):
|
||||||
|
configured.append("Installed composition: In-app notifications can report important Campaign delivery status changes.")
|
||||||
|
else:
|
||||||
|
limitations.append("Automatic in-app Campaign status notifications are not configured.")
|
||||||
|
|
||||||
|
return tuple(configured), tuple(limitations)
|
||||||
|
|
||||||
|
|
||||||
|
def _delivery_policy_summary(context: DocumentationContext) -> tuple[tuple[str, ...], tuple[str, ...]]:
|
||||||
|
session = context.session
|
||||||
|
tenant_id = str(getattr(context.principal, "tenant_id", "") or "").strip()
|
||||||
|
if session is None or not tenant_id:
|
||||||
|
return (), ("The effective synchronous Campaign limit could not be resolved for this documentation request.",)
|
||||||
|
try:
|
||||||
|
policy = effective_synchronous_send_policy(session, tenant_id=tenant_id) # type: ignore[arg-type]
|
||||||
|
except CampaignDeliveryPolicyError as exc:
|
||||||
|
return (), (f"Synchronous Campaign delivery is disabled until its policy configuration is corrected: {exc}",)
|
||||||
|
return (
|
||||||
|
(
|
||||||
|
"Delivery policy: Send now is limited to "
|
||||||
|
f"{policy.max_recipient_jobs} eligible recipient job(s) for this tenant. "
|
||||||
|
"The exact built count and Queue for workers alternative are shown before confirmation."
|
||||||
|
),
|
||||||
|
), ()
|
||||||
|
|
||||||
|
|
||||||
|
def _append_if(
|
||||||
|
target: list[str],
|
||||||
|
principal: object,
|
||||||
|
scopes: tuple[str, ...],
|
||||||
|
text: str,
|
||||||
|
*,
|
||||||
|
require_all: bool = False,
|
||||||
|
) -> None:
|
||||||
|
allowed = _has_all_scopes(principal, scopes) if require_all else _has_any_scope(principal, scopes)
|
||||||
|
if allowed:
|
||||||
|
target.append(f"Role authorization: {text}")
|
||||||
|
|
||||||
|
|
||||||
|
def _has_scope(principal: object, scope: str) -> bool:
|
||||||
|
checker = getattr(principal, "has", None)
|
||||||
|
if not callable(checker):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(checker(scope))
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _has_any_scope(principal: object, scopes: tuple[str, ...]) -> bool:
|
||||||
|
return any(_has_scope(principal, scope) for scope in scopes)
|
||||||
|
|
||||||
|
|
||||||
|
def _has_all_scopes(principal: object, scopes: tuple[str, ...]) -> bool:
|
||||||
|
return all(_has_scope(principal, scope) for scope in scopes)
|
||||||
|
|
||||||
|
|
||||||
|
def _integration_available(registry: object, interface_name: str) -> bool:
|
||||||
|
return _registry_has_interface(registry, interface_name) and _registry_has_capability(registry, interface_name)
|
||||||
|
|
||||||
|
|
||||||
|
def _registry_has_interface(registry: object, interface_name: str) -> bool:
|
||||||
|
manifests_provider = getattr(registry, "manifests", None)
|
||||||
|
if not callable(manifests_provider):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
manifests = tuple(manifests_provider())
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
return any(
|
||||||
|
getattr(provider, "name", None) == interface_name
|
||||||
|
for manifest in manifests
|
||||||
|
for provider in (getattr(manifest, "provides_interfaces", ()) or ())
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _registry_has_capability(registry: object, capability_name: str) -> bool:
|
||||||
|
capability_checker = getattr(registry, "has_capability", None)
|
||||||
|
if not callable(capability_checker):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(capability_checker(capability_name))
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _composition_body(current_configuration: list[str], limitations: list[str]) -> str:
|
||||||
|
del current_configuration, limitations
|
||||||
|
return "The facts below are calculated for the current actor and installed module contracts. They do not bypass campaign ownership, sharing, state, or operation-time resource and policy checks."
|
||||||
@@ -7,11 +7,28 @@ from contextlib import contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Iterator
|
from typing import Any, Iterator
|
||||||
|
|
||||||
|
from govoplan_core.core.postbox import (
|
||||||
|
CAPABILITY_POSTBOX_DIRECTORY,
|
||||||
|
CAPABILITY_POSTBOX_DELIVERY,
|
||||||
|
CAPABILITY_POSTBOX_EVIDENCE,
|
||||||
|
PostboxDeliveryCatalogRef,
|
||||||
|
PostboxDeliveryProvider,
|
||||||
|
PostboxDeliveryReceiptSummaryRef,
|
||||||
|
PostboxDeliveryRequest,
|
||||||
|
PostboxDeliveryResult,
|
||||||
|
PostboxDirectoryEntryRef,
|
||||||
|
PostboxDirectoryProvider,
|
||||||
|
PostboxEvidenceProvider,
|
||||||
|
PostboxTargetRef,
|
||||||
|
)
|
||||||
from govoplan_campaign.backend.runtime import capability
|
from govoplan_campaign.backend.runtime import capability
|
||||||
|
|
||||||
|
|
||||||
FILES_CAPABILITY = "files.campaign_attachments"
|
FILES_CAPABILITY = "files.campaign_attachments"
|
||||||
MAIL_CAPABILITY = "mail.campaign_delivery"
|
MAIL_CAPABILITY = "mail.campaign_delivery"
|
||||||
|
POSTBOX_CAPABILITY = CAPABILITY_POSTBOX_DELIVERY
|
||||||
|
POSTBOX_DIRECTORY_CAPABILITY = CAPABILITY_POSTBOX_DIRECTORY
|
||||||
|
POSTBOX_EVIDENCE_CAPABILITY = CAPABILITY_POSTBOX_EVIDENCE
|
||||||
|
|
||||||
|
|
||||||
class OptionalModuleUnavailable(RuntimeError):
|
class OptionalModuleUnavailable(RuntimeError):
|
||||||
@@ -34,15 +51,30 @@ class ImapConfigurationError(RuntimeError):
|
|||||||
|
|
||||||
|
|
||||||
class ImapAppendError(RuntimeError):
|
class ImapAppendError(RuntimeError):
|
||||||
def __init__(self, message: str, *, temporary: bool | None = None) -> None:
|
def __init__(
|
||||||
|
self,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
temporary: bool | None = None,
|
||||||
|
outcome_unknown: bool = False,
|
||||||
|
) -> None:
|
||||||
super().__init__(message)
|
super().__init__(message)
|
||||||
self.temporary = temporary
|
self.temporary = temporary
|
||||||
|
self.outcome_unknown = outcome_unknown
|
||||||
|
|
||||||
|
|
||||||
class MailProfileError(OptionalModuleUnavailable):
|
class MailProfileError(OptionalModuleUnavailable):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class MailDeliveryCommandError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxDeliveryUnavailable(OptionalModuleUnavailable):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class _PreparedCampaignSnapshot:
|
class _PreparedCampaignSnapshot:
|
||||||
def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None:
|
def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None:
|
||||||
self._directory = directory
|
self._directory = directory
|
||||||
@@ -50,6 +82,7 @@ class _PreparedCampaignSnapshot:
|
|||||||
self.raw_json = raw_json
|
self.raw_json = raw_json
|
||||||
self.managed_files_by_local_path: dict[str, Any] = {}
|
self.managed_files_by_local_path: dict[str, Any] = {}
|
||||||
self.shared_assets: list[Any] = []
|
self.shared_assets: list[Any] = []
|
||||||
|
self.candidate_assets: list[Any] = []
|
||||||
|
|
||||||
def cleanup(self) -> None:
|
def cleanup(self) -> None:
|
||||||
shutil.rmtree(self._directory, ignore_errors=True)
|
shutil.rmtree(self._directory, ignore_errors=True)
|
||||||
@@ -107,6 +140,11 @@ class FilesCampaignIntegration:
|
|||||||
raise OptionalModuleUnavailable("Files module is not available")
|
raise OptionalModuleUnavailable("Files module is not available")
|
||||||
return self._delegate.current_version_and_blob(session, asset)
|
return self._delegate.current_version_and_blob(session, asset)
|
||||||
|
|
||||||
|
def share_assets_with_campaign(self, session: Any, **kwargs: Any) -> list[dict[str, Any]]:
|
||||||
|
if self._delegate is None:
|
||||||
|
raise OptionalModuleUnavailable("Files module is not available")
|
||||||
|
return self._delegate.share_assets_with_campaign(session, **kwargs)
|
||||||
|
|
||||||
def mark_job_attachment_uses_sent(self, session: Any, job: Any) -> None:
|
def mark_job_attachment_uses_sent(self, session: Any, job: Any) -> None:
|
||||||
if self._delegate is not None:
|
if self._delegate is not None:
|
||||||
self._delegate.mark_job_attachment_uses_sent(session, job)
|
self._delegate.mark_job_attachment_uses_sent(session, job)
|
||||||
@@ -128,22 +166,17 @@ class MailCampaignIntegration:
|
|||||||
def available(self) -> bool:
|
def available(self) -> bool:
|
||||||
return self._delegate is not None
|
return self._delegate is not None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def durable_delivery_available(self) -> bool:
|
||||||
|
return self._delegate is not None and callable(
|
||||||
|
getattr(self._delegate, "submit_delivery_command", None)
|
||||||
|
)
|
||||||
|
|
||||||
def _require(self) -> Any:
|
def _require(self) -> Any:
|
||||||
if self._delegate is None:
|
if self._delegate is None:
|
||||||
raise MailProfileError("Mail module is not available")
|
raise MailProfileError("Mail module is not available")
|
||||||
return self._delegate
|
return self._delegate
|
||||||
|
|
||||||
def materialize_campaign_mail_profile_config(self, session: Any, **kwargs: Any) -> dict[str, Any]:
|
|
||||||
if self._delegate is None:
|
|
||||||
raw_json = kwargs.get("raw_json")
|
|
||||||
if self.mail_profile_id_from_campaign_json(raw_json if isinstance(raw_json, dict) else {}):
|
|
||||||
raise MailProfileError("Campaign mail-server profiles require the mail module")
|
|
||||||
return dict(raw_json) if isinstance(raw_json, dict) else {}
|
|
||||||
try:
|
|
||||||
return self._delegate.materialize_campaign_mail_profile_config(session, **kwargs)
|
|
||||||
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
|
|
||||||
raise MailProfileError(str(exc)) from exc
|
|
||||||
|
|
||||||
def assert_campaign_mail_policy_allows_json(self, session: Any, **kwargs: Any) -> None:
|
def assert_campaign_mail_policy_allows_json(self, session: Any, **kwargs: Any) -> None:
|
||||||
if self._delegate is None:
|
if self._delegate is None:
|
||||||
raw_json = kwargs.get("raw_json")
|
raw_json = kwargs.get("raw_json")
|
||||||
@@ -156,72 +189,86 @@ class MailCampaignIntegration:
|
|||||||
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
|
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
|
||||||
raise MailProfileError(str(exc)) from exc
|
raise MailProfileError(str(exc)) from exc
|
||||||
|
|
||||||
def assert_mail_policy_allows_send(self, session: Any, **kwargs: Any) -> None:
|
|
||||||
delegate = self._require()
|
|
||||||
try:
|
|
||||||
return delegate.assert_mail_policy_allows_send(session, **kwargs)
|
|
||||||
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
|
||||||
raise MailProfileError(str(exc)) from exc
|
|
||||||
|
|
||||||
def mail_profile_id_from_campaign_json(self, raw_json: dict[str, Any]) -> str | None:
|
def mail_profile_id_from_campaign_json(self, raw_json: dict[str, Any]) -> str | None:
|
||||||
if self._delegate is not None:
|
if self._delegate is not None:
|
||||||
return self._delegate.mail_profile_id_from_campaign_json(raw_json)
|
return self._delegate.mail_profile_id_from_campaign_json(raw_json)
|
||||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||||
profile_id = server.get("mail_profile_id") if isinstance(server, dict) else None
|
profile_id = server.get("mail_profile_id") if isinstance(server, dict) else None
|
||||||
if profile_id is None and isinstance(server, dict):
|
|
||||||
profile_id = server.get("profile_id")
|
|
||||||
return str(profile_id).strip() if profile_id else None
|
return str(profile_id).strip() if profile_id else None
|
||||||
|
|
||||||
def ensure_mail_profile_allowed_for_campaign(self, session: Any, **kwargs: Any) -> Any:
|
def campaign_profile_delivery_summary(self, session: Any, **kwargs: Any) -> dict[str, Any]:
|
||||||
delegate = self._require()
|
delegate = self._require()
|
||||||
try:
|
try:
|
||||||
return delegate.ensure_mail_profile_allowed_for_campaign(session, **kwargs)
|
return delegate.campaign_profile_delivery_summary(session, **kwargs)
|
||||||
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
||||||
raise MailProfileError(str(exc)) from exc
|
raise MailProfileError(str(exc)) from exc
|
||||||
|
|
||||||
def smtp_config_from_profile(self, profile: Any) -> Any:
|
|
||||||
return self._require().smtp_config_from_profile(profile)
|
|
||||||
|
|
||||||
def imap_config_from_profile(self, profile: Any) -> Any:
|
|
||||||
return self._require().imap_config_from_profile(profile)
|
|
||||||
|
|
||||||
def effective_profile_credentials_inherited(self, session: Any, **kwargs: Any) -> bool:
|
|
||||||
return self._require().effective_profile_credentials_inherited(session, **kwargs)
|
|
||||||
|
|
||||||
def apply_campaign_credentials(self, profile_payload: dict[str, Any], server: dict[str, Any], protocol: str) -> dict[str, Any]:
|
|
||||||
return self._require().apply_campaign_credentials(profile_payload, server, protocol)
|
|
||||||
|
|
||||||
def wait_for_rate_limit(self, **kwargs: Any) -> None:
|
def wait_for_rate_limit(self, **kwargs: Any) -> None:
|
||||||
if self._delegate is None:
|
if self._delegate is None:
|
||||||
return None
|
return None
|
||||||
return self._delegate.wait_for_rate_limit(**kwargs)
|
return self._delegate.wait_for_rate_limit(**kwargs)
|
||||||
|
|
||||||
def send_email_bytes(self, *args: Any, **kwargs: Any) -> Any:
|
def send_campaign_email_bytes(self, *args: Any, **kwargs: Any) -> Any:
|
||||||
delegate = self._require()
|
delegate = self._require()
|
||||||
try:
|
try:
|
||||||
return delegate.send_email_bytes(*args, **kwargs)
|
return delegate.send_campaign_email_bytes(*args, **kwargs)
|
||||||
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
|
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
|
||||||
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
|
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
|
||||||
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
|
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
|
||||||
raise SmtpConfigurationError(str(exc)) from exc
|
raise SmtpConfigurationError(str(exc)) from exc
|
||||||
|
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
||||||
|
raise MailProfileError(str(exc)) from exc
|
||||||
|
|
||||||
def append_message_to_sent(self, *args: Any, **kwargs: Any) -> Any:
|
def append_campaign_message_to_sent(self, *args: Any, **kwargs: Any) -> Any:
|
||||||
delegate = self._require()
|
delegate = self._require()
|
||||||
try:
|
try:
|
||||||
return delegate.append_message_to_sent(*args, **kwargs)
|
return delegate.append_campaign_message_to_sent(*args, **kwargs)
|
||||||
except getattr(delegate, "ImapAppendError", ImapAppendError) as exc:
|
except getattr(delegate, "ImapAppendError", ImapAppendError) as exc:
|
||||||
raise ImapAppendError(str(exc), temporary=getattr(exc, "temporary", None)) from exc
|
raise ImapAppendError(
|
||||||
|
str(exc),
|
||||||
|
temporary=getattr(exc, "temporary", None),
|
||||||
|
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
|
||||||
|
) from exc
|
||||||
except getattr(delegate, "ImapConfigurationError", ImapConfigurationError) as exc:
|
except getattr(delegate, "ImapConfigurationError", ImapConfigurationError) as exc:
|
||||||
raise ImapConfigurationError(str(exc)) from exc
|
raise ImapConfigurationError(str(exc)) from exc
|
||||||
|
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
||||||
|
raise MailProfileError(str(exc)) from exc
|
||||||
|
|
||||||
def send_email_message(self, *args: Any, **kwargs: Any) -> Any:
|
def submit_delivery_command(self, session: Any, **kwargs: Any) -> dict[str, Any]:
|
||||||
delegate = self._require()
|
delegate = self._require()
|
||||||
|
method = getattr(delegate, "submit_delivery_command", None)
|
||||||
|
if not callable(method):
|
||||||
|
raise MailDeliveryCommandError(
|
||||||
|
"The installed Mail module does not provide durable delivery commands"
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
return delegate.send_email_message(*args, **kwargs)
|
return dict(method(session, **kwargs))
|
||||||
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
|
except Exception as exc:
|
||||||
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
|
raise MailDeliveryCommandError(str(exc)) from exc
|
||||||
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
|
|
||||||
raise SmtpConfigurationError(str(exc)) from exc
|
def delivery_command_summary(
|
||||||
|
self,
|
||||||
|
session: Any,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
command_id: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
delegate = self._require()
|
||||||
|
method = getattr(delegate, "delivery_command_summary", None)
|
||||||
|
if not callable(method):
|
||||||
|
raise MailDeliveryCommandError(
|
||||||
|
"The installed Mail module does not provide durable delivery status"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
return dict(
|
||||||
|
method(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
command_id=command_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
raise MailDeliveryCommandError(str(exc)) from exc
|
||||||
|
|
||||||
def mock_mailbox(self) -> Any | None:
|
def mock_mailbox(self) -> Any | None:
|
||||||
if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"):
|
if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"):
|
||||||
@@ -229,9 +276,122 @@ class MailCampaignIntegration:
|
|||||||
return self._delegate.mock_mailbox()
|
return self._delegate.mock_mailbox()
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxCampaignIntegration:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
delivery_delegate: object | None = None,
|
||||||
|
directory_delegate: object | None = None,
|
||||||
|
evidence_delegate: object | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._delivery_delegate = (
|
||||||
|
delivery_delegate
|
||||||
|
if isinstance(delivery_delegate, PostboxDeliveryProvider)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
self._directory_delegate = (
|
||||||
|
directory_delegate
|
||||||
|
if isinstance(directory_delegate, PostboxDirectoryProvider)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
self._evidence_delegate = (
|
||||||
|
evidence_delegate
|
||||||
|
if isinstance(evidence_delegate, PostboxEvidenceProvider)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def available(self) -> bool:
|
||||||
|
return (
|
||||||
|
self._delivery_delegate is not None
|
||||||
|
and self._directory_delegate is not None
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def receipt_evidence_available(self) -> bool:
|
||||||
|
return self._evidence_delegate is not None
|
||||||
|
|
||||||
|
def delivery_catalog(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
) -> PostboxDeliveryCatalogRef:
|
||||||
|
if self._directory_delegate is None:
|
||||||
|
raise PostboxDeliveryUnavailable(
|
||||||
|
"Postbox targets are unavailable because the Postbox module "
|
||||||
|
"is not active."
|
||||||
|
)
|
||||||
|
return self._directory_delegate.delivery_catalog(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def resolve_postbox(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
target: PostboxTargetRef,
|
||||||
|
materialize: bool = False,
|
||||||
|
) -> PostboxDirectoryEntryRef | None:
|
||||||
|
if self._directory_delegate is None:
|
||||||
|
raise PostboxDeliveryUnavailable(
|
||||||
|
"Postbox targets are unavailable because the Postbox module "
|
||||||
|
"is not active."
|
||||||
|
)
|
||||||
|
return self._directory_delegate.resolve_postbox(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
target=target,
|
||||||
|
materialize=materialize,
|
||||||
|
)
|
||||||
|
|
||||||
|
def deliver(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
request: PostboxDeliveryRequest,
|
||||||
|
) -> PostboxDeliveryResult:
|
||||||
|
if self._delivery_delegate is None:
|
||||||
|
raise PostboxDeliveryUnavailable(
|
||||||
|
"Postbox delivery is unavailable because the Postbox module "
|
||||||
|
"is not active."
|
||||||
|
)
|
||||||
|
return self._delivery_delegate.deliver(session, request)
|
||||||
|
|
||||||
|
def delivery_receipt_summaries(
|
||||||
|
self,
|
||||||
|
session: object,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
delivery_ids: list[str] | tuple[str, ...],
|
||||||
|
) -> dict[str, PostboxDeliveryReceiptSummaryRef]:
|
||||||
|
if self._evidence_delegate is None:
|
||||||
|
return {}
|
||||||
|
unique_ids = tuple(dict.fromkeys(delivery_ids))
|
||||||
|
summaries: dict[str, PostboxDeliveryReceiptSummaryRef] = {}
|
||||||
|
for offset in range(0, len(unique_ids), 500):
|
||||||
|
summaries.update(
|
||||||
|
self._evidence_delegate.delivery_receipt_summaries(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
producer_module="campaigns",
|
||||||
|
delivery_ids=unique_ids[offset : offset + 500],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return summaries
|
||||||
|
|
||||||
|
|
||||||
def files_integration() -> FilesCampaignIntegration:
|
def files_integration() -> FilesCampaignIntegration:
|
||||||
return FilesCampaignIntegration(capability(FILES_CAPABILITY))
|
return FilesCampaignIntegration(capability(FILES_CAPABILITY))
|
||||||
|
|
||||||
|
|
||||||
def mail_integration() -> MailCampaignIntegration:
|
def mail_integration() -> MailCampaignIntegration:
|
||||||
return MailCampaignIntegration(capability(MAIL_CAPABILITY))
|
return MailCampaignIntegration(capability(MAIL_CAPABILITY))
|
||||||
|
|
||||||
|
|
||||||
|
def postbox_integration() -> PostboxCampaignIntegration:
|
||||||
|
return PostboxCampaignIntegration(
|
||||||
|
capability(POSTBOX_CAPABILITY),
|
||||||
|
capability(POSTBOX_DIRECTORY_CAPABILITY),
|
||||||
|
capability(POSTBOX_EVIDENCE_CAPABILITY),
|
||||||
|
)
|
||||||
|
|||||||
@@ -11,8 +11,13 @@ from govoplan_core.core.campaigns import (
|
|||||||
CAPABILITY_CAMPAIGNS_RETENTION,
|
CAPABILITY_CAMPAIGNS_RETENTION,
|
||||||
)
|
)
|
||||||
from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard
|
from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard
|
||||||
|
from govoplan_core.core.ownership import OwnershipProviderRegistration
|
||||||
from govoplan_core.core.modules import (
|
from govoplan_core.core.modules import (
|
||||||
|
DocumentationCondition,
|
||||||
|
DocumentationLink,
|
||||||
|
DocumentationTopic,
|
||||||
FrontendModule,
|
FrontendModule,
|
||||||
|
FrontendRoute,
|
||||||
MigrationSpec,
|
MigrationSpec,
|
||||||
ModuleContext,
|
ModuleContext,
|
||||||
ModuleInterfaceProvider,
|
ModuleInterfaceProvider,
|
||||||
@@ -22,9 +27,17 @@ from govoplan_core.core.modules import (
|
|||||||
PermissionDefinition,
|
PermissionDefinition,
|
||||||
RoleTemplate,
|
RoleTemplate,
|
||||||
)
|
)
|
||||||
|
from govoplan_core.core.views import ViewSurface
|
||||||
from govoplan_core.db.base import Base
|
from govoplan_core.db.base import Base
|
||||||
|
from govoplan_core.core.postbox import (
|
||||||
|
CAPABILITY_POSTBOX_DELIVERY,
|
||||||
|
CAPABILITY_POSTBOX_DIRECTORY,
|
||||||
|
CAPABILITY_POSTBOX_EVIDENCE,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.references import CAPABILITY_ACCESS_REFERENCE_OPTIONS
|
||||||
from govoplan_campaign.backend.change_tracking import register_campaign_change_tracking
|
from govoplan_campaign.backend.change_tracking import register_campaign_change_tracking
|
||||||
from govoplan_campaign.backend.db import models as campaign_models # noqa: F401 - populate Campaign ORM metadata
|
from govoplan_campaign.backend.db import models as campaign_models # noqa: F401 - populate Campaign ORM metadata
|
||||||
|
from govoplan_campaign.backend.documentation import CAMPAIGN_USER_DOCUMENTATION, documentation_topics
|
||||||
|
|
||||||
register_campaign_change_tracking()
|
register_campaign_change_tracking()
|
||||||
|
|
||||||
@@ -51,15 +64,18 @@ PERMISSIONS = (
|
|||||||
_permission("campaigns:campaign:archive", "Archive campaigns", "Archive campaigns without destroying audit evidence.", "Campaigns"),
|
_permission("campaigns:campaign:archive", "Archive campaigns", "Archive campaigns without destroying audit evidence.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:delete", "Delete campaigns", "Delete draft-only campaigns where retention policy allows it.", "Campaigns"),
|
_permission("campaigns:campaign:delete", "Delete campaigns", "Delete draft-only campaigns where retention policy allows it.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:share", "Share campaigns", "Grant or revoke explicit campaign access.", "Campaigns"),
|
_permission("campaigns:campaign:share", "Share campaigns", "Grant or revoke explicit campaign access.", "Campaigns"),
|
||||||
|
_permission("campaigns:ownership:accept_group", "Accept group campaign ownership", "Accept or decline campaign ownership on behalf of a group the actor belongs to.", "Campaign governance"),
|
||||||
|
_permission("campaigns:ownership:recover", "Recover campaign ownership", "Participate in delayed, quorum-backed administrative campaign ownership recovery.", "Campaign governance"),
|
||||||
_permission("campaigns:campaign:validate", "Validate campaigns", "Run technical validation and manage validation locks.", "Campaigns"),
|
_permission("campaigns:campaign:validate", "Validate campaigns", "Run technical validation and manage validation locks.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:build", "Build campaigns", "Build exact messages and attachment evidence.", "Campaigns"),
|
_permission("campaigns:campaign:build", "Build campaigns", "Build exact messages and attachment evidence.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:review", "Approve campaign review", "Approve or reject built messages and review conditions.", "Campaigns"),
|
_permission("campaigns:campaign:review", "Complete campaign review", "Record review completion and the exact built messages inspected.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:send_test", "Mock-send campaigns", "Use mock delivery and verification tools.", "Campaigns"),
|
_permission("campaigns:campaign:send_test", "Mock-send campaigns", "Use mock delivery and verification tools.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:queue", "Queue campaigns", "Place approved executions into the delivery queue.", "Campaigns"),
|
_permission("campaigns:campaign:queue", "Queue campaigns", "Place approved executions into the delivery queue.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:control", "Control delivery", "Pause, resume or cancel queued and sending jobs.", "Campaigns"),
|
_permission("campaigns:campaign:control", "Control delivery", "Pause, resume or cancel queued and sending jobs.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:send", "Send campaigns", "Start real SMTP delivery.", "Campaigns"),
|
_permission("campaigns:campaign:send", "Send campaigns", "Start real Mail or Postbox delivery.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:retry", "Retry delivery", "Retry failed or unattempted delivery jobs.", "Campaigns"),
|
_permission("campaigns:campaign:retry", "Retry delivery", "Retry failed or unattempted delivery jobs.", "Campaigns"),
|
||||||
_permission("campaigns:campaign:reconcile", "Reconcile delivery", "Resolve outcome-unknown SMTP attempts after inspection.", "Campaigns"),
|
_permission("campaigns:campaign:reconcile", "Reconcile delivery", "Resolve outcome-unknown Mail, Postbox, or IMAP attempts after inspection.", "Campaigns"),
|
||||||
|
_permission("campaigns:diagnostic:read", "View campaign diagnostics", "Inspect worker claims and internal storage locators for campaign delivery troubleshooting.", "Campaign operations"),
|
||||||
_permission("campaigns:recipient:read", "View recipients", "Read recipient lists and recipient-specific campaign data.", "Recipients"),
|
_permission("campaigns:recipient:read", "View recipients", "Read recipient lists and recipient-specific campaign data.", "Recipients"),
|
||||||
_permission("campaigns:recipient:write", "Edit recipients", "Create and edit recipient rows and field values.", "Recipients"),
|
_permission("campaigns:recipient:write", "Edit recipients", "Create and edit recipient rows and field values.", "Recipients"),
|
||||||
_permission("campaigns:recipient:import", "Import recipients", "Bulk-import recipient lists.", "Recipients"),
|
_permission("campaigns:recipient:import", "Import recipients", "Bulk-import recipient lists.", "Recipients"),
|
||||||
@@ -69,7 +85,31 @@ PERMISSIONS = (
|
|||||||
_permission("campaigns:report:send", "Send reports", "Email campaign reports to configured recipients.", "Reports"),
|
_permission("campaigns:report:send", "Send reports", "Email campaign reports to configured recipients.", "Reports"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
OPERATOR_QUEUE_REQUIRED_ANY = (
|
||||||
|
"campaigns:campaign:queue",
|
||||||
|
"campaigns:campaign:retry",
|
||||||
|
"campaigns:campaign:reconcile",
|
||||||
|
"campaigns:campaign:control",
|
||||||
|
)
|
||||||
|
# Preserve the former /operator route identity for saved View projections.
|
||||||
|
OPERATOR_QUEUE_SURFACE_ID = "campaigns.route.operator"
|
||||||
|
# Preserve the former /reports route identity for saved View projections.
|
||||||
|
REPORTS_SURFACE_ID = "campaigns.route.reports"
|
||||||
|
REPORTS_REQUIRED_ANY = ("campaigns:report:read",)
|
||||||
|
CAMPAIGN_MODULE_REQUIRED_ANY = (
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
*REPORTS_REQUIRED_ANY,
|
||||||
|
)
|
||||||
|
|
||||||
ROLE_TEMPLATES = (
|
ROLE_TEMPLATES = (
|
||||||
|
RoleTemplate(
|
||||||
|
slug="campaign_aggregate_reader",
|
||||||
|
name="Campaign aggregate reader",
|
||||||
|
description="View privacy-protected outcome totals without recipient or delivery diagnostics.",
|
||||||
|
permissions=(
|
||||||
|
"campaigns:report:read",
|
||||||
|
),
|
||||||
|
),
|
||||||
RoleTemplate(
|
RoleTemplate(
|
||||||
slug="campaign_manager",
|
slug="campaign_manager",
|
||||||
name="Campaign manager",
|
name="Campaign manager",
|
||||||
@@ -81,6 +121,7 @@ ROLE_TEMPLATES = (
|
|||||||
"campaigns:campaign:copy",
|
"campaigns:campaign:copy",
|
||||||
"campaigns:campaign:validate",
|
"campaigns:campaign:validate",
|
||||||
"campaigns:campaign:build",
|
"campaigns:campaign:build",
|
||||||
|
"campaigns:ownership:accept_group",
|
||||||
"campaigns:recipient:read",
|
"campaigns:recipient:read",
|
||||||
"campaigns:recipient:write",
|
"campaigns:recipient:write",
|
||||||
"campaigns:recipient:import",
|
"campaigns:recipient:import",
|
||||||
@@ -90,7 +131,7 @@ ROLE_TEMPLATES = (
|
|||||||
RoleTemplate(
|
RoleTemplate(
|
||||||
slug="campaign_reviewer",
|
slug="campaign_reviewer",
|
||||||
name="Campaign reviewer",
|
name="Campaign reviewer",
|
||||||
description="Inspect and approve prepared campaign messages.",
|
description="Inspect prepared campaign messages and record review completion.",
|
||||||
permissions=(
|
permissions=(
|
||||||
"campaigns:campaign:read",
|
"campaigns:campaign:read",
|
||||||
"campaigns:campaign:validate",
|
"campaigns:campaign:validate",
|
||||||
@@ -111,6 +152,7 @@ ROLE_TEMPLATES = (
|
|||||||
"campaigns:campaign:send",
|
"campaigns:campaign:send",
|
||||||
"campaigns:campaign:retry",
|
"campaigns:campaign:retry",
|
||||||
"campaigns:campaign:reconcile",
|
"campaigns:campaign:reconcile",
|
||||||
|
"campaigns:diagnostic:read",
|
||||||
"campaigns:recipient:read",
|
"campaigns:recipient:read",
|
||||||
"campaigns:report:read",
|
"campaigns:report:read",
|
||||||
"campaigns:report:send",
|
"campaigns:report:send",
|
||||||
@@ -125,6 +167,26 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
|
|||||||
return {"campaigns": session.query(Campaign).filter(Campaign.tenant_id == tenant_id).count()}
|
return {"campaigns": session.query(Campaign).filter(Campaign.tenant_id == tenant_id).count()}
|
||||||
|
|
||||||
|
|
||||||
|
def _tenant_summary_batch(session, tenant_ids) -> dict[str, dict[str, int]]:
|
||||||
|
from sqlalchemy import func
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.db.models import Campaign
|
||||||
|
|
||||||
|
ids = tuple(dict.fromkeys(str(tenant_id) for tenant_id in tenant_ids if tenant_id))
|
||||||
|
if not ids:
|
||||||
|
return {}
|
||||||
|
rows = (
|
||||||
|
session.query(Campaign.tenant_id, func.count(Campaign.id))
|
||||||
|
.filter(Campaign.tenant_id.in_(ids))
|
||||||
|
.group_by(Campaign.tenant_id)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
tenant_id: {"campaigns": int(count)}
|
||||||
|
for tenant_id, count in rows
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _campaigns_router(context: ModuleContext):
|
def _campaigns_router(context: ModuleContext):
|
||||||
from govoplan_campaign.backend.runtime import configure_runtime
|
from govoplan_campaign.backend.runtime import configure_runtime
|
||||||
|
|
||||||
@@ -142,9 +204,16 @@ def _campaigns_router(context: ModuleContext):
|
|||||||
manifest = ModuleManifest(
|
manifest = ModuleManifest(
|
||||||
id="campaigns",
|
id="campaigns",
|
||||||
name="Campaigns",
|
name="Campaigns",
|
||||||
version="0.1.8",
|
version="0.1.12",
|
||||||
required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR),
|
required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR),
|
||||||
optional_dependencies=("files", "mail", "notifications", "addresses"),
|
optional_capabilities=(CAPABILITY_ACCESS_REFERENCE_OPTIONS,),
|
||||||
|
optional_dependencies=(
|
||||||
|
"files",
|
||||||
|
"mail",
|
||||||
|
"notifications",
|
||||||
|
"addresses",
|
||||||
|
"postbox",
|
||||||
|
),
|
||||||
provides_interfaces=(
|
provides_interfaces=(
|
||||||
ModuleInterfaceProvider(name="campaigns.access", version="0.1.6"),
|
ModuleInterfaceProvider(name="campaigns.access", version="0.1.6"),
|
||||||
ModuleInterfaceProvider(name="campaigns.delivery_tasks", version="0.1.6"),
|
ModuleInterfaceProvider(name="campaigns.delivery_tasks", version="0.1.6"),
|
||||||
@@ -153,6 +222,12 @@ manifest = ModuleManifest(
|
|||||||
ModuleInterfaceProvider(name="campaigns.retention", version="0.1.6"),
|
ModuleInterfaceProvider(name="campaigns.retention", version="0.1.6"),
|
||||||
),
|
),
|
||||||
requires_interfaces=(
|
requires_interfaces=(
|
||||||
|
ModuleInterfaceRequirement(
|
||||||
|
name=CAPABILITY_ACCESS_REFERENCE_OPTIONS,
|
||||||
|
version_min="0.1.0",
|
||||||
|
version_max_exclusive="0.2.0",
|
||||||
|
optional=True,
|
||||||
|
),
|
||||||
ModuleInterfaceRequirement(
|
ModuleInterfaceRequirement(
|
||||||
name="files.campaign_attachments",
|
name="files.campaign_attachments",
|
||||||
version_min="0.1.0",
|
version_min="0.1.0",
|
||||||
@@ -161,6 +236,12 @@ manifest = ModuleManifest(
|
|||||||
),
|
),
|
||||||
ModuleInterfaceRequirement(
|
ModuleInterfaceRequirement(
|
||||||
name="mail.campaign_delivery",
|
name="mail.campaign_delivery",
|
||||||
|
version_min="0.2.0",
|
||||||
|
version_max_exclusive="0.3.0",
|
||||||
|
optional=True,
|
||||||
|
),
|
||||||
|
ModuleInterfaceRequirement(
|
||||||
|
name="mail.delivery_commands",
|
||||||
version_min="0.1.0",
|
version_min="0.1.0",
|
||||||
version_max_exclusive="0.2.0",
|
version_max_exclusive="0.2.0",
|
||||||
optional=True,
|
optional=True,
|
||||||
@@ -177,49 +258,79 @@ manifest = ModuleManifest(
|
|||||||
version_max_exclusive="0.2.0",
|
version_max_exclusive="0.2.0",
|
||||||
optional=True,
|
optional=True,
|
||||||
),
|
),
|
||||||
|
ModuleInterfaceRequirement(
|
||||||
|
name=CAPABILITY_POSTBOX_DELIVERY,
|
||||||
|
version_min="0.1.1",
|
||||||
|
version_max_exclusive="0.2.0",
|
||||||
|
optional=True,
|
||||||
|
),
|
||||||
|
ModuleInterfaceRequirement(
|
||||||
|
name=CAPABILITY_POSTBOX_DIRECTORY,
|
||||||
|
version_min="0.1.1",
|
||||||
|
version_max_exclusive="0.2.0",
|
||||||
|
optional=True,
|
||||||
|
),
|
||||||
|
ModuleInterfaceRequirement(
|
||||||
|
name=CAPABILITY_POSTBOX_EVIDENCE,
|
||||||
|
version_min="0.1.2",
|
||||||
|
version_max_exclusive="0.2.0",
|
||||||
|
optional=True,
|
||||||
|
),
|
||||||
),
|
),
|
||||||
permissions=PERMISSIONS,
|
permissions=PERMISSIONS,
|
||||||
route_factory=_campaigns_router,
|
route_factory=_campaigns_router,
|
||||||
role_templates=ROLE_TEMPLATES,
|
role_templates=ROLE_TEMPLATES,
|
||||||
tenant_summary_providers=(_tenant_summary,),
|
tenant_summary_providers=(_tenant_summary,),
|
||||||
|
tenant_summary_batch_providers=(_tenant_summary_batch,),
|
||||||
nav_items=(
|
nav_items=(
|
||||||
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=("campaigns:campaign:read",), order=20),
|
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=CAMPAIGN_MODULE_REQUIRED_ANY, order=20),
|
||||||
NavItem(
|
|
||||||
path="/operator",
|
|
||||||
label="Operator Queue",
|
|
||||||
icon="radio-tower",
|
|
||||||
required_any=(
|
|
||||||
"campaigns:campaign:queue",
|
|
||||||
"campaigns:campaign:retry",
|
|
||||||
"campaigns:campaign:reconcile",
|
|
||||||
"campaigns:campaign:control",
|
|
||||||
"campaigns:campaign:send",
|
|
||||||
),
|
|
||||||
order=30,
|
|
||||||
),
|
|
||||||
NavItem(path="/reports", label="Reports", icon="clipboard-pen-line", required_any=("campaigns:report:read",), order=70),
|
|
||||||
),
|
),
|
||||||
frontend=FrontendModule(
|
frontend=FrontendModule(
|
||||||
module_id="campaigns",
|
module_id="campaigns",
|
||||||
package_name="@govoplan/campaign-webui",
|
package_name="@govoplan/campaign-webui",
|
||||||
nav_items=(
|
routes=(
|
||||||
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=("campaigns:campaign:read",), order=20),
|
FrontendRoute(
|
||||||
NavItem(
|
path="/campaigns",
|
||||||
path="/operator",
|
component="CampaignModulePage",
|
||||||
label="Operator Queue",
|
required_any=CAMPAIGN_MODULE_REQUIRED_ANY,
|
||||||
icon="radio-tower",
|
order=20,
|
||||||
required_any=(
|
|
||||||
"campaigns:campaign:queue",
|
|
||||||
"campaigns:campaign:retry",
|
|
||||||
"campaigns:campaign:reconcile",
|
|
||||||
"campaigns:campaign:control",
|
|
||||||
"campaigns:campaign:send",
|
|
||||||
),
|
|
||||||
order=30,
|
|
||||||
),
|
),
|
||||||
NavItem(path="/reports", label="Reports", icon="clipboard-pen-line", required_any=("campaigns:report:read",), order=70),
|
FrontendRoute(
|
||||||
|
path="/campaigns/queue",
|
||||||
|
component="OperatorQueuePage",
|
||||||
|
required_all=("campaigns:campaign:read",),
|
||||||
|
required_any=OPERATOR_QUEUE_REQUIRED_ANY,
|
||||||
|
order=21,
|
||||||
|
surface_id=OPERATOR_QUEUE_SURFACE_ID,
|
||||||
|
),
|
||||||
|
FrontendRoute(
|
||||||
|
path="/campaigns/:campaignId/*",
|
||||||
|
component="CampaignWorkspace",
|
||||||
|
required_any=("campaigns:campaign:read",),
|
||||||
|
order=22,
|
||||||
|
),
|
||||||
|
FrontendRoute(
|
||||||
|
path="/campaigns/reports",
|
||||||
|
component="AggregateReportsPage",
|
||||||
|
required_any=REPORTS_REQUIRED_ANY,
|
||||||
|
order=22,
|
||||||
|
surface_id=REPORTS_SURFACE_ID,
|
||||||
|
),
|
||||||
|
FrontendRoute(path="/templates", component="TemplatesPage", order=90),
|
||||||
|
),
|
||||||
|
nav_items=(
|
||||||
|
NavItem(path="/campaigns", label="Campaigns", icon="campaign", required_any=CAMPAIGN_MODULE_REQUIRED_ANY, order=20),
|
||||||
NavItem(path="/templates", label="Templates", icon="layout-template", order=90),
|
NavItem(path="/templates", label="Templates", icon="layout-template", order=90),
|
||||||
),
|
),
|
||||||
|
view_surfaces=(
|
||||||
|
ViewSurface(
|
||||||
|
id="campaigns.widget.activity",
|
||||||
|
module_id="campaigns",
|
||||||
|
kind="section",
|
||||||
|
label="Campaign activity widget",
|
||||||
|
order=50,
|
||||||
|
),
|
||||||
|
),
|
||||||
),
|
),
|
||||||
migration_spec=MigrationSpec(
|
migration_spec=MigrationSpec(
|
||||||
module_id="campaigns",
|
module_id="campaigns",
|
||||||
@@ -236,7 +347,10 @@ manifest = ModuleManifest(
|
|||||||
campaign_models.AttachmentBlob,
|
campaign_models.AttachmentBlob,
|
||||||
campaign_models.AttachmentInstance,
|
campaign_models.AttachmentInstance,
|
||||||
campaign_models.SendAttempt,
|
campaign_models.SendAttempt,
|
||||||
|
campaign_models.CampaignMessageAction,
|
||||||
|
campaign_models.CampaignMessageActionAttempt,
|
||||||
campaign_models.ImapAppendAttempt,
|
campaign_models.ImapAppendAttempt,
|
||||||
|
campaign_models.PostboxDeliveryAttempt,
|
||||||
label="Campaigns",
|
label="Campaigns",
|
||||||
),
|
),
|
||||||
retirement_notes="Destructive retirement drops campaign-owned database tables after the installer captures a database snapshot.",
|
retirement_notes="Destructive retirement drops campaign-owned database tables after the installer captures a database snapshot.",
|
||||||
@@ -252,10 +366,368 @@ manifest = ModuleManifest(
|
|||||||
campaign_models.AttachmentBlob,
|
campaign_models.AttachmentBlob,
|
||||||
campaign_models.AttachmentInstance,
|
campaign_models.AttachmentInstance,
|
||||||
campaign_models.SendAttempt,
|
campaign_models.SendAttempt,
|
||||||
|
campaign_models.CampaignMessageAction,
|
||||||
|
campaign_models.CampaignMessageActionAttempt,
|
||||||
campaign_models.ImapAppendAttempt,
|
campaign_models.ImapAppendAttempt,
|
||||||
|
campaign_models.PostboxDeliveryAttempt,
|
||||||
label="Campaigns",
|
label="Campaigns",
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
|
documentation=(
|
||||||
|
*CAMPAIGN_USER_DOCUMENTATION,
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.postbox-delivery",
|
||||||
|
title="Deliver Campaign messages to Postboxes",
|
||||||
|
summary="Target one or more exact or organization-derived Postboxes per recipient row, independently or alongside Mail.",
|
||||||
|
body="Configure campaign-wide targets and optional per-row additions or replacements. Derived targets resolve a published Postbox template with organization unit, function, and optional context values, including values sourced from Campaign fields. Targets are frozen during build. Fallback crosses to the second channel only after a confirmed pre-acceptance rejection; accepted or outcome-unknown effects never trigger fallback.",
|
||||||
|
layer="available",
|
||||||
|
documentation_types=("user", "admin"),
|
||||||
|
audience=("campaign_manager", "campaign_reviewer", "campaign_sender", "campaign_operator"),
|
||||||
|
order=45,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns", "postbox"),
|
||||||
|
any_scopes=("campaigns:recipient:write", "campaigns:campaign:send", "campaigns:campaign:reconcile"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Postboxes", href="/postbox", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign schema", href="/api/v1/campaigns/schema", kind="api"),
|
||||||
|
),
|
||||||
|
related_modules=("postbox", "organizations", "idm", "mail", "audit"),
|
||||||
|
unlocks=("Audited direct, derived, combined, and pre-acceptance fallback delivery to Postboxes.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": "/campaigns/{campaign_id}/global-settings",
|
||||||
|
"screen": "Campaign delivery defaults and recipient data",
|
||||||
|
"prerequisites": [
|
||||||
|
"Campaign and Postbox are installed and active.",
|
||||||
|
"At least one exact Postbox or published Postbox template is available.",
|
||||||
|
],
|
||||||
|
"steps": [
|
||||||
|
"Choose Postbox, Mail and Postbox, or an ordered fallback policy.",
|
||||||
|
"Configure one or more campaign-wide targets.",
|
||||||
|
"Optionally add or replace targets for individual recipient rows.",
|
||||||
|
"Validate and build to freeze the resolved Postbox addresses before review.",
|
||||||
|
"Review, queue, and inspect channel-specific delivery evidence in the report.",
|
||||||
|
],
|
||||||
|
"outcome": "Each active row resolves an auditable set of Postbox targets without introducing a hard Campaign dependency on Postbox.",
|
||||||
|
"verification": "Confirm the frozen target list in the built job and verify accepted, rejected, or outcome-unknown attempts in Campaign reporting.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.workflow.prepare-validate-and-build",
|
||||||
|
"campaigns.workflow.retry-and-reconcile",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.mail-profile-user-journey",
|
||||||
|
title="Choose a Mail profile for campaign delivery",
|
||||||
|
summary="Campaigns reference an authorized Mail profile and never store SMTP/IMAP settings or credentials.",
|
||||||
|
body="Open the campaign Mail settings, select an available profile, test it through Mail, and save. Validation and delivery recheck profile authorization. A changed transport identity requires a new validation and build.",
|
||||||
|
layer="available",
|
||||||
|
documentation_types=("user",),
|
||||||
|
audience=("campaign_manager", "campaign_reviewer", "campaign_sender"),
|
||||||
|
order=46,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns", "mail"),
|
||||||
|
required_scopes=("campaigns:campaign:update", "mail:profile:use"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Mail profiles", href="/settings?section=mail-profiles", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("mail",),
|
||||||
|
unlocks=("Profile-backed SMTP delivery and optional IMAP append-to-Sent.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": "/campaigns/{campaign_id}/mail-settings",
|
||||||
|
"screen": "Campaign Mail settings",
|
||||||
|
"help_contexts": ["campaign.server-settings"],
|
||||||
|
"prerequisites": [
|
||||||
|
"Campaign and Mail are installed.",
|
||||||
|
"You may edit the current campaign version and use at least one authorized Mail profile.",
|
||||||
|
],
|
||||||
|
"steps": [
|
||||||
|
"Open the campaign and go to Mail settings.",
|
||||||
|
"Select an available Mail profile; Campaign stores only its stable identifier.",
|
||||||
|
"Test SMTP and, when configured, IMAP through the Mail module.",
|
||||||
|
"Save, validate, and build the campaign before queueing delivery.",
|
||||||
|
],
|
||||||
|
"outcome": "The editable campaign version references an authorized Mail-owned delivery profile without copying transport settings or credentials.",
|
||||||
|
"verification": "Reopen Mail settings, confirm the selected profile, then run validation and verify that the build completes without profile-drift errors.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.mail-profile-governance",
|
||||||
|
"campaigns.mail-profile-operations",
|
||||||
|
"mail.profile-ownership-and-consumers",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.mail-profile-governance",
|
||||||
|
title="Govern Campaign-to-Mail profile references",
|
||||||
|
summary="Mail owns transport definitions and encrypted credentials; Campaign owns only the selected profile reference and delivery evidence.",
|
||||||
|
body="Grant mail:profile:use to campaign authors, constrain profile availability through Mail policy, and keep effective credential inheritance enabled. Inline transport fields are rejected. Legacy records remain unchanged until an explicit, audited profile migration creates or updates an editable version.",
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("admin",),
|
||||||
|
audience=("tenant_admin", "mail_admin", "campaign_admin"),
|
||||||
|
order=47,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns", "mail"),
|
||||||
|
any_scopes=("mail:profile:write", "admin:policies:read", "system:settings:read"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Mail profiles", href="/settings?section=mail-profiles", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign schema", href="/api/v1/campaigns/schema", kind="api"),
|
||||||
|
DocumentationLink(label="Mail profile boundary", href="govoplan-campaign/docs/MAIL_PROFILE_BOUNDARY.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("mail", "access"),
|
||||||
|
unlocks=("Auditable, reusable transport configuration across campaigns.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "reference",
|
||||||
|
"route": "/campaigns/{campaign_id}/mail-policy",
|
||||||
|
"screen": "Campaign Mail policy",
|
||||||
|
"section": "Profile authorization and credential inheritance",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.mail-profile-user-journey",
|
||||||
|
"campaigns.mail-profile-operations",
|
||||||
|
"mail.profile-ownership-and-consumers",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.mail-profile-operations",
|
||||||
|
title="Operate profile-backed campaign delivery",
|
||||||
|
summary="Workers re-authorize and resolve Mail profiles at execution time while Campaign retains only opaque Mail-owned revisions and outcomes.",
|
||||||
|
body="A legacy snapshot, unauthorized or inactive profile, profile-reference mismatch, or changed SMTP/IMAP transport revision stops delivery. Preserve the record, migrate or correct the profile selection, revalidate, rebuild, and only then queue again. Password-only rotation remains possible without copying secrets into Campaign. Uncertain SMTP and IMAP effects remain blocked until an evidence-backed operator reconciliation.",
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("admin",),
|
||||||
|
audience=("campaign_sender", "campaign_operator", "mail_admin"),
|
||||||
|
order=48,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns", "mail"),
|
||||||
|
any_scopes=("campaigns:diagnostic:read", "campaigns:campaign:reconcile", "mail:profile:test"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign reports", href="/campaigns/reports", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("mail", "audit"),
|
||||||
|
unlocks=("Fail-closed recovery without exposing Mail credentials.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "reference",
|
||||||
|
"route": "/campaigns/queue",
|
||||||
|
"screen": "Campaign operator queue",
|
||||||
|
"section": "Profile-backed delivery recovery",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.mail-profile-user-journey",
|
||||||
|
"campaigns.mail-profile-governance",
|
||||||
|
"mail.profile-ownership-and-consumers",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.workflow.prepare-validate-and-build",
|
||||||
|
title="Prepare, validate, and build a campaign",
|
||||||
|
summary="Turn governed recipient, template, attachment, and Mail-profile inputs into exact built messages for review.",
|
||||||
|
body="Prepare each input in its owning surface, resolve every blocking validation issue, and build exact recipient messages before review. Campaign freezes recipient and attachment evidence for the selected version; later source changes do not silently alter that build.",
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("user",),
|
||||||
|
audience=("campaign_manager", "campaign_author"),
|
||||||
|
order=49,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns",),
|
||||||
|
required_scopes=("campaigns:campaign:update", "campaigns:campaign:validate", "campaigns:campaign:build"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"),
|
||||||
|
DocumentationLink(label="Recipient import guide", href="govoplan-campaign/docs/RECIPIENT_IMPORT_GUIDE.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("addresses", "files", "mail"),
|
||||||
|
unlocks=("A reviewable build whose exact recipient-specific effects can be inspected before delivery.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": "/campaigns/{campaign_id}/global-settings",
|
||||||
|
"screen": "Campaign workspace",
|
||||||
|
"help_contexts": [
|
||||||
|
"campaign.overview",
|
||||||
|
"campaign.settings",
|
||||||
|
"campaign.fields",
|
||||||
|
"campaign.template",
|
||||||
|
"campaign.attachments",
|
||||||
|
"campaign.recipients",
|
||||||
|
"campaign.recipient-data",
|
||||||
|
"campaign.server-settings",
|
||||||
|
"campaign.global-settings",
|
||||||
|
"campaign.review-send",
|
||||||
|
"campaign.json",
|
||||||
|
],
|
||||||
|
"prerequisites": [
|
||||||
|
"The campaign has an owner and a clear communication purpose.",
|
||||||
|
"You may edit, validate, and build the selected campaign version.",
|
||||||
|
"Optional source modules needed by this campaign are installed and authorized.",
|
||||||
|
],
|
||||||
|
"steps": [
|
||||||
|
"Set campaign-wide fields and purpose, then define the recipient fields and templates.",
|
||||||
|
"Import or select recipients and inspect provenance, exclusions, and review-required rows.",
|
||||||
|
"Select managed attachment versions and an authorized Mail profile when those capabilities are used.",
|
||||||
|
"Validate the relevant sections and resolve every blocker without hiding warnings.",
|
||||||
|
"Build the selected version and inspect representative and exceptional rendered messages.",
|
||||||
|
],
|
||||||
|
"outcome": "The selected version has exact built messages and frozen source evidence ready for an independent review.",
|
||||||
|
"verification": "Open Review, confirm the build belongs to the intended version, and inspect counts, warnings, recipients, addressing, rendered content, and attachment evidence.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.workflow.complete-review",
|
||||||
|
"campaigns.mail-profile-user-journey",
|
||||||
|
"files.workflow.import-managed-snapshot",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.workflow.complete-review",
|
||||||
|
title="Inspect built messages and complete review",
|
||||||
|
summary="Review the exact immutable candidate and record which built messages were inspected before delivery is enabled.",
|
||||||
|
body="Review completion records the inspected message keys for the selected build. The current baseline does not persist a separate approve/reject decision or review reason, so do not present completion as a richer decision record. Any material input or non-secret transport-identity change requires validation and a new build.",
|
||||||
|
layer="configured",
|
||||||
|
documentation_types=("user",),
|
||||||
|
audience=("campaign_reviewer",),
|
||||||
|
order=50,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns",),
|
||||||
|
required_scopes=("campaigns:campaign:read", "campaigns:campaign:review"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("files", "mail"),
|
||||||
|
unlocks=("An attributable review-completion record for the exact built messages inspected.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": "/campaigns/{campaign_id}/review",
|
||||||
|
"screen": "Review and send",
|
||||||
|
"help_contexts": ["campaign.review-send"],
|
||||||
|
"prerequisites": [
|
||||||
|
"The selected campaign version is validated and built.",
|
||||||
|
"You may read the campaign and complete its review.",
|
||||||
|
],
|
||||||
|
"steps": [
|
||||||
|
"Confirm the campaign, owner, selected version, recipient count, warnings, and exclusions.",
|
||||||
|
"Inspect representative and exceptional messages, addressing, templates, and attachment evidence.",
|
||||||
|
"Confirm that the selected Mail profile is suitable and authorized for the current context.",
|
||||||
|
"Record review completion for the exact message keys inspected.",
|
||||||
|
],
|
||||||
|
"outcome": "The reviewed build is eligible for a separately authorized queue or send action.",
|
||||||
|
"verification": "Reload Review and confirm completion is tied to the same version and message build; changed inputs must invalidate or supersede it.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.workflow.prepare-validate-and-build",
|
||||||
|
"campaigns.workflow.retry-and-reconcile",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.workflow.retry-and-reconcile",
|
||||||
|
title="Retry only known failures and reconcile uncertain effects",
|
||||||
|
summary="Keep safe-to-retry failures separate from Mail, Postbox, or IMAP effects whose outcome is unknown.",
|
||||||
|
body="A retry creates new attempt evidence and is valid only for an explicitly eligible state. Never blindly retry an unknown Mail, Postbox, or IMAP effect. Inspect external evidence and reconcile the affected channel before continuing. Accepted Mail attempts and accepted Postbox targets are immutable during partial retries, and repairing Sent never resends accepted Mail.",
|
||||||
|
layer="evidence",
|
||||||
|
documentation_types=("admin", "user"),
|
||||||
|
audience=("campaign_sender", "campaign_operator"),
|
||||||
|
order=51,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns",),
|
||||||
|
any_scopes=("campaigns:campaign:retry", "campaigns:campaign:reconcile", "campaigns:diagnostic:read"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
|
||||||
|
DocumentationLink(label="Campaign delivery runbook", href="govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("mail", "audit"),
|
||||||
|
unlocks=("Evidence-backed recovery without accidental duplicate external effects.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "workflow",
|
||||||
|
"route": "/campaigns/queue",
|
||||||
|
"screen": "Campaign operator queue",
|
||||||
|
"help_contexts": ["campaign.review-send", "campaign.report", "campaign.audit"],
|
||||||
|
"prerequisites": [
|
||||||
|
"You may perform the selected retry or reconciliation action.",
|
||||||
|
"Provider, mailbox, worker, and campaign evidence has been preserved.",
|
||||||
|
],
|
||||||
|
"steps": [
|
||||||
|
"Classify the job and latest SMTP and IMAP attempts independently.",
|
||||||
|
"Retry only an explicitly temporary, permanent-with-override, or unattempted eligible state.",
|
||||||
|
"For an unknown effect, inspect provider or mailbox evidence and record the factual reconciliation with a note.",
|
||||||
|
"Verify the resulting protected state before allowing more work for that job.",
|
||||||
|
],
|
||||||
|
"outcome": "Every investigated job is either protected as effected, explicitly retryable, or still visibly unresolved.",
|
||||||
|
"verification": "Confirm the previous attempt remains in history, a retry has a new attempt number, and no accepted SMTP effect was repeated to repair IMAP state.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.mail-profile-operations",
|
||||||
|
"campaigns.reference.composition-assurance",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
DocumentationTopic(
|
||||||
|
id="campaigns.reference.composition-assurance",
|
||||||
|
title="Assure the Campaign reference composition",
|
||||||
|
summary="Release Campaign only with aligned contracts, role-safe surfaces, durable effect evidence, optional-module isolation, and recoverable data.",
|
||||||
|
body="Campaign is a reference composition only when Core, Mail, Files, Addresses, workers, storage, policies, and documentation are tested in the exact installed combination. Normal readers see business state rather than paths, storage keys, worker claims, or raw provider diagnostics; diagnostic and export authority remain separate.",
|
||||||
|
layer="evidence",
|
||||||
|
documentation_types=("admin",),
|
||||||
|
audience=("platform_operator", "security_reviewer", "release_reviewer", "integrator"),
|
||||||
|
order=52,
|
||||||
|
conditions=(
|
||||||
|
DocumentationCondition(
|
||||||
|
required_modules=("campaigns",),
|
||||||
|
any_scopes=("campaigns:diagnostic:read", "campaigns:report:read", "system:settings:read", "admin:modules:read"),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
links=(
|
||||||
|
DocumentationLink(label="Campaign handbook", href="govoplan-campaign/docs/CAMPAIGN_HANDBOOK.md", kind="repository"),
|
||||||
|
DocumentationLink(label="Reference examples and release checklist", href="govoplan-campaign/docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md", kind="repository"),
|
||||||
|
),
|
||||||
|
related_modules=("mail", "postbox", "files", "addresses", "audit"),
|
||||||
|
unlocks=("A repeatable, supportable Campaign demonstration rather than an unverified module assembly.",),
|
||||||
|
metadata={
|
||||||
|
"kind": "reference",
|
||||||
|
"route": "/campaigns",
|
||||||
|
"screen": "Campaign reference composition",
|
||||||
|
"section": "Release, security, integration, and recovery assurance",
|
||||||
|
"verification": "Run the maintained examples, module-permutation tests, migration and restore drills, target Mail/Postbox/IMAP checks, version-alignment gate, WebUI/i18n checks, and full security audit for the pinned composition.",
|
||||||
|
"related_topic_ids": [
|
||||||
|
"campaigns.workflow.prepare-validate-and-build",
|
||||||
|
"campaigns.workflow.complete-review",
|
||||||
|
"campaigns.workflow.retry-and-reconcile",
|
||||||
|
"mail.reference.credentials-egress-retirement",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
),
|
||||||
|
documentation_providers=(documentation_topics,),
|
||||||
|
ownership_providers=(
|
||||||
|
OwnershipProviderRegistration(
|
||||||
|
resource_type="campaign",
|
||||||
|
provider=__import__(
|
||||||
|
"govoplan_campaign.backend.capabilities",
|
||||||
|
fromlist=["CampaignOwnershipService"],
|
||||||
|
).CampaignOwnershipService(),
|
||||||
|
),
|
||||||
|
),
|
||||||
capability_factories={
|
capability_factories={
|
||||||
CAPABILITY_CAMPAIGNS_ACCESS: lambda context: __import__(
|
CAPABILITY_CAMPAIGNS_ACCESS: lambda context: __import__(
|
||||||
"govoplan_campaign.backend.capabilities",
|
"govoplan_campaign.backend.capabilities",
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ from govoplan_campaign.backend.attachments.resolver import (
|
|||||||
EntryAttachmentResolution,
|
EntryAttachmentResolution,
|
||||||
MessageAttachmentStatus,
|
MessageAttachmentStatus,
|
||||||
ResolvedAttachment,
|
ResolvedAttachment,
|
||||||
|
effective_send_without_attachments_behavior,
|
||||||
resolve_entry_attachments,
|
resolve_entry_attachments,
|
||||||
)
|
)
|
||||||
from govoplan_campaign.backend.campaign.addressing import effective_address_lists, formatted_recipient
|
from govoplan_campaign.backend.campaign.addressing import effective_address_lists, formatted_recipient
|
||||||
@@ -33,9 +34,14 @@ from govoplan_campaign.backend.campaign.models import (
|
|||||||
ZipArchiveConfig,
|
ZipArchiveConfig,
|
||||||
ZipPasswordMode,
|
ZipPasswordMode,
|
||||||
ZipPasswordScope,
|
ZipPasswordScope,
|
||||||
|
effective_delivery_channel_policy,
|
||||||
)
|
)
|
||||||
from govoplan_campaign.backend.campaign.template_values import build_template_values
|
from govoplan_campaign.backend.campaign.template_values import build_template_values
|
||||||
from govoplan_campaign.backend.services.zip_service import create_zip_archive
|
from govoplan_campaign.backend.services.zip_service import create_zip_archive
|
||||||
|
from govoplan_campaign.backend.template_rendering import (
|
||||||
|
find_unresolved_placeholders as _find_unresolved_placeholders,
|
||||||
|
render_template as _render_template,
|
||||||
|
)
|
||||||
|
|
||||||
from .models import (
|
from .models import (
|
||||||
CampaignBuildReport,
|
CampaignBuildReport,
|
||||||
@@ -47,28 +53,6 @@ from .models import (
|
|||||||
MessageValidationStatus,
|
MessageValidationStatus,
|
||||||
)
|
)
|
||||||
|
|
||||||
_DOLLAR_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
|
|
||||||
_BRACE_FIELD_PATTERN = re.compile(r"(?<!\\)\{\{\s*(.*?)\s*\}\}")
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_template_key(raw: str) -> str:
|
|
||||||
key = raw.strip()
|
|
||||||
if key.startswith("fields."):
|
|
||||||
key = key.removeprefix("fields.")
|
|
||||||
elif key.startswith("local."):
|
|
||||||
key = "local::" + key.removeprefix("local.")
|
|
||||||
elif key.startswith("global."):
|
|
||||||
key = "global::" + key.removeprefix("global.")
|
|
||||||
|
|
||||||
if key.startswith("local::") or key.startswith("global::"):
|
|
||||||
return key
|
|
||||||
if key.startswith("local:"):
|
|
||||||
return "local::" + key.removeprefix("local:")
|
|
||||||
if key.startswith("global:"):
|
|
||||||
return "global::" + key.removeprefix("global:")
|
|
||||||
return key
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class BuiltMessage:
|
class BuiltMessage:
|
||||||
draft: MessageDraft
|
draft: MessageDraft
|
||||||
@@ -123,29 +107,6 @@ def _read_text(campaign_file: str | Path, raw_path: str | None, encoding: str =
|
|||||||
return path.read_text(encoding=encoding)
|
return path.read_text(encoding=encoding)
|
||||||
|
|
||||||
|
|
||||||
def _render_template(template: str, values: dict[str, Any], *, keep_missing: bool = True) -> str:
|
|
||||||
def replace(match: re.Match[str]) -> str:
|
|
||||||
key = _normalize_template_key(match.group(1))
|
|
||||||
if key in values:
|
|
||||||
value = values[key]
|
|
||||||
return "" if value is None else str(value)
|
|
||||||
return match.group(0) if keep_missing else ""
|
|
||||||
|
|
||||||
rendered = _DOLLAR_FIELD_PATTERN.sub(replace, template)
|
|
||||||
rendered = _BRACE_FIELD_PATTERN.sub(replace, rendered)
|
|
||||||
return rendered.replace(r"\${", "${").replace(r"\}", "}")
|
|
||||||
|
|
||||||
|
|
||||||
def _find_unresolved_placeholders(text: str | None) -> set[str]:
|
|
||||||
if not text:
|
|
||||||
return set()
|
|
||||||
return {
|
|
||||||
_normalize_template_key(match.group(1))
|
|
||||||
for pattern in (_DOLLAR_FIELD_PATTERN, _BRACE_FIELD_PATTERN)
|
|
||||||
for match in pattern.finditer(text)
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _message_address(recipient: RecipientConfig | None) -> MessageAddress | None:
|
def _message_address(recipient: RecipientConfig | None) -> MessageAddress | None:
|
||||||
if recipient is None:
|
if recipient is None:
|
||||||
return None
|
return None
|
||||||
@@ -212,6 +173,11 @@ def _attachment_summaries(resolution: EntryAttachmentResolution) -> list[Message
|
|||||||
label=attachment.label,
|
label=attachment.label,
|
||||||
status=attachment.status.value,
|
status=attachment.status.value,
|
||||||
behavior=attachment.behavior.value if attachment.behavior else None,
|
behavior=attachment.behavior.value if attachment.behavior else None,
|
||||||
|
missing_policy=(
|
||||||
|
attachment.missing_policy.model_dump(mode="json")
|
||||||
|
if attachment.missing_policy
|
||||||
|
else None
|
||||||
|
),
|
||||||
required=attachment.required,
|
required=attachment.required,
|
||||||
allow_multiple=attachment.allow_multiple,
|
allow_multiple=attachment.allow_multiple,
|
||||||
zip_enabled=attachment.zip_enabled,
|
zip_enabled=attachment.zip_enabled,
|
||||||
@@ -240,11 +206,26 @@ def _message_issues_from_attachment_resolution(resolution: EntryAttachmentResolu
|
|||||||
message=issue.message,
|
message=issue.message,
|
||||||
behavior=issue.behavior.value if issue.behavior else None,
|
behavior=issue.behavior.value if issue.behavior else None,
|
||||||
source="attachments",
|
source="attachments",
|
||||||
|
details=issue.details,
|
||||||
)
|
)
|
||||||
for issue in resolution.issues
|
for issue in resolution.issues
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _append_no_attachment_coverage_issue(issues: list[MessageIssue]) -> None:
|
||||||
|
if any(issue.code == "missing_attachment_coverage" for issue in issues):
|
||||||
|
return
|
||||||
|
issues.append(
|
||||||
|
MessageIssue(
|
||||||
|
severity="error",
|
||||||
|
code="missing_attachment_coverage",
|
||||||
|
message="No attachment file was resolved for this message, and sending without attachments is not allowed.",
|
||||||
|
behavior=Behavior.BLOCK.value,
|
||||||
|
source="attachments",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _safe_filename(value: str | None, fallback: str) -> str:
|
def _safe_filename(value: str | None, fallback: str) -> str:
|
||||||
raw = value or fallback
|
raw = value or fallback
|
||||||
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._")
|
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._")
|
||||||
@@ -391,6 +372,7 @@ def _attach_files(
|
|||||||
for attachment in resolution.attachments:
|
for attachment in resolution.attachments:
|
||||||
attachment.message_filenames = []
|
attachment.message_filenames = []
|
||||||
attachment.zip_entry_names = []
|
attachment.zip_entry_names = []
|
||||||
|
attachment.zip_filename = None
|
||||||
|
|
||||||
for attachment in resolution.attachments:
|
for attachment in resolution.attachments:
|
||||||
if attachment.status != AttachmentMatchStatus.OK or not attachment.matches:
|
if attachment.status != AttachmentMatchStatus.OK or not attachment.matches:
|
||||||
@@ -435,6 +417,7 @@ def _attach_files(
|
|||||||
work_dir / "_zip" / f"entry-{entry_index:04d}" / _safe_filename(archive.id, "archive") / filename,
|
work_dir / "_zip" / f"entry-{entry_index:04d}" / _safe_filename(archive.id, "archive") / filename,
|
||||||
members,
|
members,
|
||||||
password,
|
password,
|
||||||
|
archive.method.value,
|
||||||
)
|
)
|
||||||
data, maintype, subtype = _attachment_bytes(archive_path)
|
data, maintype, subtype = _attachment_bytes(archive_path)
|
||||||
message.add_attachment(data, maintype=maintype, subtype=subtype, filename=filename)
|
message.add_attachment(data, maintype=maintype, subtype=subtype, filename=filename)
|
||||||
@@ -444,8 +427,14 @@ def _attach_files(
|
|||||||
|
|
||||||
return attached_count
|
return attached_count
|
||||||
|
|
||||||
def _imap_initial_status(config: CampaignConfig) -> ImapStatus:
|
def _imap_initial_status(
|
||||||
if config.delivery.imap_append_sent.enabled:
|
config: CampaignConfig,
|
||||||
|
entry: EntryConfig,
|
||||||
|
) -> ImapStatus:
|
||||||
|
if (
|
||||||
|
effective_delivery_channel_policy(config, entry).uses_mail
|
||||||
|
and config.delivery.imap_append_sent.enabled
|
||||||
|
):
|
||||||
return ImapStatus.PENDING
|
return ImapStatus.PENDING
|
||||||
return ImapStatus.NOT_REQUESTED
|
return ImapStatus.NOT_REQUESTED
|
||||||
|
|
||||||
@@ -532,8 +521,18 @@ def _message_draft(
|
|||||||
eml_path: str | None = None,
|
eml_path: str | None = None,
|
||||||
eml_size: int | None = None,
|
eml_size: int | None = None,
|
||||||
) -> MessageDraft:
|
) -> MessageDraft:
|
||||||
|
if validation_status == MessageValidationStatus.EXCLUDED:
|
||||||
|
# Exclusion is a completed validation decision, not a pending delivery.
|
||||||
|
# Keep both transport projections explicit so reports never imply that
|
||||||
|
# SMTP or IMAP work is still expected for this row.
|
||||||
|
send_status = SendStatus.SKIPPED
|
||||||
|
imap_status = ImapStatus.SKIPPED
|
||||||
if imap_status is None:
|
if imap_status is None:
|
||||||
imap_status = _imap_initial_status(config) if build_status == BuildStatus.BUILT else ImapStatus.SKIPPED
|
imap_status = (
|
||||||
|
_imap_initial_status(config, entry)
|
||||||
|
if build_status == BuildStatus.BUILT
|
||||||
|
else ImapStatus.SKIPPED
|
||||||
|
)
|
||||||
return MessageDraft(
|
return MessageDraft(
|
||||||
entry_index=entry_index,
|
entry_index=entry_index,
|
||||||
entry_id=entry.id,
|
entry_id=entry.id,
|
||||||
@@ -542,6 +541,10 @@ def _message_draft(
|
|||||||
validation_status=validation_status,
|
validation_status=validation_status,
|
||||||
send_status=send_status,
|
send_status=send_status,
|
||||||
imap_status=imap_status,
|
imap_status=imap_status,
|
||||||
|
delivery_channel_policy=effective_delivery_channel_policy(
|
||||||
|
config,
|
||||||
|
entry,
|
||||||
|
).value,
|
||||||
subject=subject,
|
subject=subject,
|
||||||
from_=_message_address(context.sender),
|
from_=_message_address(context.sender),
|
||||||
from_all=_message_addresses(context.senders),
|
from_all=_message_addresses(context.senders),
|
||||||
@@ -609,6 +612,25 @@ def _validate_required_recipients(
|
|||||||
return MessageValidationStatus.EXCLUDED
|
return MessageValidationStatus.EXCLUDED
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_required_sender(
|
||||||
|
senders: list[RecipientConfig],
|
||||||
|
issues: list[MessageIssue],
|
||||||
|
validation_status: MessageValidationStatus,
|
||||||
|
) -> MessageValidationStatus:
|
||||||
|
if senders:
|
||||||
|
return validation_status
|
||||||
|
issues.append(
|
||||||
|
MessageIssue(
|
||||||
|
severity="error",
|
||||||
|
code="missing_sender",
|
||||||
|
message="No effective From address is configured; Campaign must resolve the sender before building.",
|
||||||
|
behavior="block",
|
||||||
|
source="recipients",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return MessageValidationStatus.BLOCKED
|
||||||
|
|
||||||
|
|
||||||
def _render_message_template(
|
def _render_message_template(
|
||||||
config: CampaignConfig,
|
config: CampaignConfig,
|
||||||
campaign_file: str | Path,
|
campaign_file: str | Path,
|
||||||
@@ -740,6 +762,18 @@ def _build_mime_message(
|
|||||||
values=rendered.values,
|
values=rendered.values,
|
||||||
work_dir=work_dir,
|
work_dir=work_dir,
|
||||||
)
|
)
|
||||||
|
if (
|
||||||
|
attachment_count == 0
|
||||||
|
and context.resolution.attachments
|
||||||
|
and effective_send_without_attachments_behavior(config) == Behavior.BLOCK
|
||||||
|
):
|
||||||
|
_append_no_attachment_coverage_issue(context.issues)
|
||||||
|
return _MimeBuildResult(
|
||||||
|
message=None,
|
||||||
|
build_status=BuildStatus.BUILD_FAILED,
|
||||||
|
validation_status=MessageValidationStatus.BLOCKED,
|
||||||
|
attachment_count=0,
|
||||||
|
)
|
||||||
return _MimeBuildResult(
|
return _MimeBuildResult(
|
||||||
message=message,
|
message=message,
|
||||||
build_status=BuildStatus.BUILT,
|
build_status=BuildStatus.BUILT,
|
||||||
@@ -756,6 +790,16 @@ def _build_mime_message(
|
|||||||
source="attachments",
|
source="attachments",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
except OSError as exc:
|
||||||
|
context.issues.append(
|
||||||
|
MessageIssue(
|
||||||
|
severity="error",
|
||||||
|
code="attachment_unreadable",
|
||||||
|
message="A resolved attachment could not be read while building the message.",
|
||||||
|
behavior="block",
|
||||||
|
source=f"attachments:{type(exc).__name__}",
|
||||||
|
)
|
||||||
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
context.issues.append(
|
context.issues.append(
|
||||||
MessageIssue(
|
MessageIssue(
|
||||||
@@ -795,12 +839,19 @@ def build_entry_message(
|
|||||||
if not entry.active:
|
if not entry.active:
|
||||||
return _inactive_entry_message(config=config, entry=entry, entry_index=entry_index, context=context)
|
return _inactive_entry_message(config=config, entry=entry, entry_index=entry_index, context=context)
|
||||||
|
|
||||||
context.validation_status = _validate_required_recipients(
|
channel_policy = effective_delivery_channel_policy(config, entry)
|
||||||
config,
|
if channel_policy.uses_mail:
|
||||||
context.recipients,
|
context.validation_status = _validate_required_sender(
|
||||||
context.issues,
|
context.senders,
|
||||||
context.validation_status,
|
context.issues,
|
||||||
)
|
context.validation_status,
|
||||||
|
)
|
||||||
|
context.validation_status = _validate_required_recipients(
|
||||||
|
config,
|
||||||
|
context.recipients,
|
||||||
|
context.issues,
|
||||||
|
context.validation_status,
|
||||||
|
)
|
||||||
rendered = _render_message_template(config, campaign_file, entry)
|
rendered = _render_message_template(config, campaign_file, entry)
|
||||||
context.validation_status = _validate_rendered_template(
|
context.validation_status = _validate_rendered_template(
|
||||||
config,
|
config,
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ class MessageIssue(BaseModel):
|
|||||||
message: str
|
message: str
|
||||||
behavior: str | None = None
|
behavior: str | None = None
|
||||||
source: str | None = None
|
source: str | None = None
|
||||||
|
details: dict[str, object] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
class MessageAddress(BaseModel):
|
class MessageAddress(BaseModel):
|
||||||
@@ -49,6 +50,7 @@ class MessageAttachmentSummary(BaseModel):
|
|||||||
label: str | None = None
|
label: str | None = None
|
||||||
status: str
|
status: str
|
||||||
behavior: str | None = None
|
behavior: str | None = None
|
||||||
|
missing_policy: dict[str, object] | None = None
|
||||||
required: bool
|
required: bool
|
||||||
allow_multiple: bool
|
allow_multiple: bool
|
||||||
zip_enabled: bool
|
zip_enabled: bool
|
||||||
@@ -78,6 +80,7 @@ class MessageDraft(BaseModel):
|
|||||||
validation_status: MessageValidationStatus
|
validation_status: MessageValidationStatus
|
||||||
send_status: SendStatus
|
send_status: SendStatus
|
||||||
imap_status: ImapStatus
|
imap_status: ImapStatus
|
||||||
|
delivery_channel_policy: str = "mail"
|
||||||
|
|
||||||
subject: str | None = None
|
subject: str | None = None
|
||||||
from_: MessageAddress | None = Field(default=None, alias="from")
|
from_: MessageAddress | None = Field(default=None, alias="from")
|
||||||
|
|||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""add durable IMAP append claim and attempt idempotency
|
||||||
|
|
||||||
|
Revision ID: 3c4d5e6f8192
|
||||||
|
Revises: 2c3d4e5f7081
|
||||||
|
Create Date: 2026-07-21 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "3c4d5e6f8192"
|
||||||
|
down_revision = "2c3d4e5f7081"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.add_column(sa.Column("imap_claimed_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch.add_column(sa.Column("imap_claim_token", sa.String(length=36), nullable=True))
|
||||||
|
batch.create_index("ix_campaign_jobs_imap_claim_token", ["imap_claim_token"], unique=False)
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.add_column(sa.Column("claim_token", sa.String(length=36), nullable=True))
|
||||||
|
_renumber_attempts()
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.create_unique_constraint(
|
||||||
|
"uq_imap_append_attempts_job_attempt",
|
||||||
|
["job_id", "attempt_number"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.drop_constraint("uq_imap_append_attempts_job_attempt", type_="unique")
|
||||||
|
batch.drop_column("claim_token")
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.drop_index("ix_campaign_jobs_imap_claim_token")
|
||||||
|
batch.drop_column("imap_claim_token")
|
||||||
|
batch.drop_column("imap_claimed_at")
|
||||||
|
|
||||||
|
|
||||||
|
def _renumber_attempts() -> None:
|
||||||
|
"""Make historical attempt numbers unique per job before constraining them."""
|
||||||
|
|
||||||
|
bind = op.get_bind()
|
||||||
|
rows = list(
|
||||||
|
bind.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT id, job_id FROM imap_append_attempts "
|
||||||
|
"ORDER BY job_id, created_at, id"
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
)
|
||||||
|
per_job: dict[str, int] = {}
|
||||||
|
for row in rows:
|
||||||
|
job_id = str(row["job_id"])
|
||||||
|
attempt_number = per_job.get(job_id, 0) + 1
|
||||||
|
per_job[job_id] = attempt_number
|
||||||
|
bind.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE imap_append_attempts SET attempt_number = :attempt_number "
|
||||||
|
"WHERE id = :attempt_id"
|
||||||
|
),
|
||||||
|
{"attempt_number": attempt_number, "attempt_id": row["id"]},
|
||||||
|
)
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
"""seal each campaign delivery job's immutable execution input
|
||||||
|
|
||||||
|
Revision ID: 4d5e6f7a9203
|
||||||
|
Revises: 3c4d5e6f8192
|
||||||
|
Create Date: 2026-07-21 00:00:01.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "4d5e6f7a9203"
|
||||||
|
down_revision = "3c4d5e6f8192"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.add_column(sa.Column("execution_input_sha256", sa.String(length=64), nullable=True))
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.drop_column("execution_input_sha256")
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""add audit-proof single-message action ledger
|
||||||
|
|
||||||
|
Revision ID: c1a69e4f2b70
|
||||||
|
Revises: f0a1b2c3d4e5
|
||||||
|
Create Date: 2026-07-30 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from importlib import import_module
|
||||||
|
|
||||||
|
|
||||||
|
message_actions = import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions.c1a69e4f2b70_campaign_message_actions"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
revision = message_actions.revision
|
||||||
|
down_revision = message_actions.down_revision
|
||||||
|
branch_labels = message_actions.branch_labels
|
||||||
|
depends_on = message_actions.depends_on
|
||||||
|
upgrade = message_actions.upgrade
|
||||||
|
downgrade = message_actions.downgrade
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
"""persist the selected Campaign delivery mode
|
||||||
|
|
||||||
|
Revision ID: c7a2f91e4b60
|
||||||
|
Revises: 4d5e6f7a9203
|
||||||
|
Create Date: 2026-07-22 09:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "c7a2f91e4b60"
|
||||||
|
down_revision = "4d5e6f7a9203"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch:
|
||||||
|
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
|
||||||
|
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch:
|
||||||
|
batch.drop_index("ix_campaign_versions_delivery_mode")
|
||||||
|
batch.drop_column("delivery_mode_selected_at")
|
||||||
|
batch.drop_column("delivery_mode")
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""add monotonic campaign version edit revision
|
||||||
|
|
||||||
|
Revision ID: d2b7af503c81
|
||||||
|
Revises: c1a69e4f2b70
|
||||||
|
Create Date: 2026-07-30 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from importlib import import_module
|
||||||
|
|
||||||
|
|
||||||
|
edit_revision = import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions.d2b7af503c81_campaign_version_edit_revision"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
revision = edit_revision.revision
|
||||||
|
down_revision = edit_revision.down_revision
|
||||||
|
branch_labels = edit_revision.branch_labels
|
||||||
|
depends_on = edit_revision.depends_on
|
||||||
|
upgrade = edit_revision.upgrade
|
||||||
|
downgrade = edit_revision.downgrade
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""mark untouched excluded jobs as skipped delivery
|
||||||
|
|
||||||
|
Revision ID: d8b3e2c1f4a5
|
||||||
|
Revises: c7a2f91e4b60
|
||||||
|
Create Date: 2026-07-22 11:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "d8b3e2c1f4a5"
|
||||||
|
down_revision = "c7a2f91e4b60"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
# Only normalize rows with no recorded transport effect. Unexpected
|
||||||
|
# historical delivery evidence must remain intact for audit/reconciliation.
|
||||||
|
op.get_bind().execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE campaign_jobs "
|
||||||
|
"SET send_status = 'skipped', imap_status = 'skipped' "
|
||||||
|
"WHERE validation_status = 'excluded' "
|
||||||
|
"AND send_status = 'not_queued' "
|
||||||
|
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.get_bind().execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE campaign_jobs "
|
||||||
|
"SET send_status = 'not_queued', imap_status = 'not_requested' "
|
||||||
|
"WHERE validation_status = 'excluded' "
|
||||||
|
"AND send_status = 'skipped' "
|
||||||
|
"AND imap_status = 'skipped'"
|
||||||
|
)
|
||||||
|
)
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""repair a missing IMAP append attempt claim token
|
||||||
|
|
||||||
|
Revision ID: e9f0a1b2c3d4
|
||||||
|
Revises: d8b3e2c1f4a5
|
||||||
|
Create Date: 2026-07-28 23:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "e9f0a1b2c3d4"
|
||||||
|
down_revision = "d8b3e2c1f4a5"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
|
||||||
|
if "claim_token" not in columns:
|
||||||
|
op.add_column(
|
||||||
|
"imap_append_attempts",
|
||||||
|
sa.Column("claim_token", sa.String(length=36), nullable=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# The column belongs to revision 3c4d5e6f8192. This repair revision only
|
||||||
|
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
|
||||||
|
pass
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""add governed campaign Postbox delivery
|
||||||
|
|
||||||
|
Revision ID: f0a1b2c3d4e5
|
||||||
|
Revises: e9f0a1b2c3d4
|
||||||
|
Create Date: 2026-07-29 02:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from importlib import import_module
|
||||||
|
|
||||||
|
|
||||||
|
_migration = import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions."
|
||||||
|
"f0a1b2c3d4e5_v0115_postbox_delivery"
|
||||||
|
)
|
||||||
|
|
||||||
|
revision = _migration.revision
|
||||||
|
down_revision = _migration.down_revision
|
||||||
|
branch_labels = _migration.branch_labels
|
||||||
|
depends_on = _migration.depends_on
|
||||||
|
upgrade = _migration.upgrade
|
||||||
|
downgrade = _migration.downgrade
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""add durable IMAP append claim and attempt idempotency
|
||||||
|
|
||||||
|
Revision ID: 3c4d5e6f8192
|
||||||
|
Revises: 2c3d4e5f7081
|
||||||
|
Create Date: 2026-07-21 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "3c4d5e6f8192"
|
||||||
|
down_revision = "2c3d4e5f7081"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.add_column(sa.Column("imap_claimed_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch.add_column(sa.Column("imap_claim_token", sa.String(length=36), nullable=True))
|
||||||
|
batch.create_index("ix_campaign_jobs_imap_claim_token", ["imap_claim_token"], unique=False)
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.add_column(sa.Column("claim_token", sa.String(length=36), nullable=True))
|
||||||
|
_renumber_attempts()
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.create_unique_constraint(
|
||||||
|
"uq_imap_append_attempts_job_attempt",
|
||||||
|
["job_id", "attempt_number"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("imap_append_attempts") as batch:
|
||||||
|
batch.drop_constraint("uq_imap_append_attempts_job_attempt", type_="unique")
|
||||||
|
batch.drop_column("claim_token")
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.drop_index("ix_campaign_jobs_imap_claim_token")
|
||||||
|
batch.drop_column("imap_claim_token")
|
||||||
|
batch.drop_column("imap_claimed_at")
|
||||||
|
|
||||||
|
|
||||||
|
def _renumber_attempts() -> None:
|
||||||
|
"""Make historical attempt numbers unique per job before constraining them."""
|
||||||
|
|
||||||
|
bind = op.get_bind()
|
||||||
|
rows = list(
|
||||||
|
bind.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT id, job_id FROM imap_append_attempts "
|
||||||
|
"ORDER BY job_id, created_at, id"
|
||||||
|
)
|
||||||
|
).mappings()
|
||||||
|
)
|
||||||
|
per_job: dict[str, int] = {}
|
||||||
|
for row in rows:
|
||||||
|
job_id = str(row["job_id"])
|
||||||
|
attempt_number = per_job.get(job_id, 0) + 1
|
||||||
|
per_job[job_id] = attempt_number
|
||||||
|
bind.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE imap_append_attempts SET attempt_number = :attempt_number "
|
||||||
|
"WHERE id = :attempt_id"
|
||||||
|
),
|
||||||
|
{"attempt_number": attempt_number, "attempt_id": row["id"]},
|
||||||
|
)
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
"""seal each campaign delivery job's immutable execution input
|
||||||
|
|
||||||
|
Revision ID: 4d5e6f7a9203
|
||||||
|
Revises: 3c4d5e6f8192
|
||||||
|
Create Date: 2026-07-21 00:00:01.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "4d5e6f7a9203"
|
||||||
|
down_revision = "3c4d5e6f8192"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.add_column(sa.Column("execution_input_sha256", sa.String(length=64), nullable=True))
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_jobs") as batch:
|
||||||
|
batch.drop_column("execution_input_sha256")
|
||||||
@@ -0,0 +1,186 @@
|
|||||||
|
"""add audit-proof single-message action ledger
|
||||||
|
|
||||||
|
Revision ID: c1a69e4f2b70
|
||||||
|
Revises: f0a1b2c3d4e5
|
||||||
|
Create Date: 2026-07-30 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "c1a69e4f2b70"
|
||||||
|
down_revision = "f0a1b2c3d4e5"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = "c91f0a72be34"
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||||
|
if "campaign_message_actions" not in tables:
|
||||||
|
op.create_table(
|
||||||
|
"campaign_message_actions",
|
||||||
|
sa.Column("id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("campaign_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("campaign_version_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("job_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("kind", sa.String(length=30), nullable=False),
|
||||||
|
sa.Column("idempotency_key", sa.String(length=200), nullable=False),
|
||||||
|
sa.Column("canonical_request_hash", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("reason", sa.Text(), nullable=True),
|
||||||
|
sa.Column("context", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("actor_api_key_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("message_sha256", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("message_size_bytes", sa.Integer(), nullable=True),
|
||||||
|
sa.Column(
|
||||||
|
"recipient_manifest_sha256",
|
||||||
|
sa.String(length=64),
|
||||||
|
nullable=False,
|
||||||
|
),
|
||||||
|
sa.Column("recipient_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("prior_send_status", sa.String(length=50), nullable=False),
|
||||||
|
sa.Column("prior_attempt_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("final_send_status", sa.String(length=50), nullable=True),
|
||||||
|
sa.Column("status", sa.String(length=50), nullable=False),
|
||||||
|
sa.Column("accepted_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("refused_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("refusal_summary", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("error_type", sa.String(length=120), nullable=True),
|
||||||
|
sa.Column("error_message", sa.String(length=500), nullable=True),
|
||||||
|
sa.Column("linked_send_attempt_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["campaign_id"],
|
||||||
|
["campaigns.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_actions_campaign_id_campaigns"
|
||||||
|
),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["campaign_version_id"],
|
||||||
|
["campaign_versions.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_actions_campaign_version_id_campaign_versions"
|
||||||
|
),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["job_id"],
|
||||||
|
["campaign_jobs.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_actions_job_id_campaign_jobs"
|
||||||
|
),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["actor_user_id"],
|
||||||
|
["access_users.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_actions_actor_user_id_access_users"
|
||||||
|
),
|
||||||
|
ondelete="SET NULL",
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["linked_send_attempt_id"],
|
||||||
|
["send_attempts.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_actions_linked_send_attempt_id_send_attempts"
|
||||||
|
),
|
||||||
|
ondelete="SET NULL",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint(
|
||||||
|
"id",
|
||||||
|
name=op.f("pk_campaign_message_actions"),
|
||||||
|
),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"idempotency_key",
|
||||||
|
name="uq_campaign_message_actions_idempotency",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_campaign_message_actions_job_created",
|
||||||
|
"campaign_message_actions",
|
||||||
|
["job_id", "created_at"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_campaign_message_actions_campaign_kind",
|
||||||
|
"campaign_message_actions",
|
||||||
|
["campaign_id", "kind", "status"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
for column in (
|
||||||
|
"tenant_id",
|
||||||
|
"campaign_id",
|
||||||
|
"campaign_version_id",
|
||||||
|
"job_id",
|
||||||
|
"kind",
|
||||||
|
"actor_user_id",
|
||||||
|
"status",
|
||||||
|
"linked_send_attempt_id",
|
||||||
|
):
|
||||||
|
op.create_index(
|
||||||
|
op.f(f"ix_campaign_message_actions_{column}"),
|
||||||
|
"campaign_message_actions",
|
||||||
|
[column],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||||
|
if "campaign_message_action_attempts" not in tables:
|
||||||
|
op.create_table(
|
||||||
|
"campaign_message_action_attempts",
|
||||||
|
sa.Column("id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("action_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("attempt_number", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=50), nullable=False),
|
||||||
|
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("accepted_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("refused_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("outcome_code", sa.String(length=80), nullable=True),
|
||||||
|
sa.Column("diagnostic_summary", sa.String(length=500), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["action_id"],
|
||||||
|
["campaign_message_actions.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_message_action_attempts_action_id_campaign_message_actions"
|
||||||
|
),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint(
|
||||||
|
"id",
|
||||||
|
name=op.f("pk_campaign_message_action_attempts"),
|
||||||
|
),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"action_id",
|
||||||
|
"attempt_number",
|
||||||
|
name="uq_campaign_message_action_attempt_number",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
for column in ("action_id", "status"):
|
||||||
|
op.create_index(
|
||||||
|
op.f(f"ix_campaign_message_action_attempts_{column}"),
|
||||||
|
"campaign_message_action_attempts",
|
||||||
|
[column],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||||
|
if "campaign_message_action_attempts" in tables:
|
||||||
|
op.drop_table("campaign_message_action_attempts")
|
||||||
|
if "campaign_message_actions" in tables:
|
||||||
|
op.drop_table("campaign_message_actions")
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
"""persist the selected Campaign delivery mode
|
||||||
|
|
||||||
|
Revision ID: c7a2f91e4b60
|
||||||
|
Revises: 4d5e6f7a9203
|
||||||
|
Create Date: 2026-07-22 09:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "c7a2f91e4b60"
|
||||||
|
down_revision = "4d5e6f7a9203"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch:
|
||||||
|
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
|
||||||
|
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch:
|
||||||
|
batch.drop_index("ix_campaign_versions_delivery_mode")
|
||||||
|
batch.drop_column("delivery_mode_selected_at")
|
||||||
|
batch.drop_column("delivery_mode")
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
"""add monotonic campaign version edit revision
|
||||||
|
|
||||||
|
Revision ID: d2b7af503c81
|
||||||
|
Revises: c1a69e4f2b70
|
||||||
|
Create Date: 2026-07-30 00:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "d2b7af503c81"
|
||||||
|
down_revision = "c1a69e4f2b70"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = "c91f0a72be34"
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
inspector = sa.inspect(op.get_bind())
|
||||||
|
columns = {
|
||||||
|
column["name"]
|
||||||
|
for column in inspector.get_columns("campaign_versions")
|
||||||
|
}
|
||||||
|
if "edit_revision" not in columns:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column(
|
||||||
|
"edit_revision",
|
||||||
|
sa.Integer(),
|
||||||
|
server_default="1",
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
inspector = sa.inspect(op.get_bind())
|
||||||
|
columns = {
|
||||||
|
column["name"]
|
||||||
|
for column in inspector.get_columns("campaign_versions")
|
||||||
|
}
|
||||||
|
if "edit_revision" in columns:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||||
|
batch_op.drop_column("edit_revision")
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""mark untouched excluded jobs as skipped delivery
|
||||||
|
|
||||||
|
Revision ID: d8b3e2c1f4a5
|
||||||
|
Revises: c7a2f91e4b60
|
||||||
|
Create Date: 2026-07-22 11:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "d8b3e2c1f4a5"
|
||||||
|
down_revision = "c7a2f91e4b60"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
# Only normalize rows with no recorded transport effect. Unexpected
|
||||||
|
# historical delivery evidence must remain intact for audit/reconciliation.
|
||||||
|
op.get_bind().execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE campaign_jobs "
|
||||||
|
"SET send_status = 'skipped', imap_status = 'skipped' "
|
||||||
|
"WHERE validation_status = 'excluded' "
|
||||||
|
"AND send_status = 'not_queued' "
|
||||||
|
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.get_bind().execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE campaign_jobs "
|
||||||
|
"SET send_status = 'not_queued', imap_status = 'not_requested' "
|
||||||
|
"WHERE validation_status = 'excluded' "
|
||||||
|
"AND send_status = 'skipped' "
|
||||||
|
"AND imap_status = 'skipped'"
|
||||||
|
)
|
||||||
|
)
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""repair a missing IMAP append attempt claim token
|
||||||
|
|
||||||
|
Revision ID: e9f0a1b2c3d4
|
||||||
|
Revises: d8b3e2c1f4a5
|
||||||
|
Create Date: 2026-07-28 23:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "e9f0a1b2c3d4"
|
||||||
|
down_revision = "d8b3e2c1f4a5"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
|
||||||
|
if "claim_token" not in columns:
|
||||||
|
op.add_column(
|
||||||
|
"imap_append_attempts",
|
||||||
|
sa.Column("claim_token", sa.String(length=36), nullable=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# The column belongs to revision 3c4d5e6f8192. This repair revision only
|
||||||
|
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
|
||||||
|
pass
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
"""add governed campaign Postbox delivery
|
||||||
|
|
||||||
|
Revision ID: f0a1b2c3d4e5
|
||||||
|
Revises: e9f0a1b2c3d4
|
||||||
|
Create Date: 2026-07-29 02:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "f0a1b2c3d4e5"
|
||||||
|
down_revision = "e9f0a1b2c3d4"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.add_column(
|
||||||
|
"campaign_jobs",
|
||||||
|
sa.Column(
|
||||||
|
"delivery_channel_policy",
|
||||||
|
sa.String(length=30),
|
||||||
|
nullable=False,
|
||||||
|
server_default="mail",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"campaign_jobs",
|
||||||
|
sa.Column(
|
||||||
|
"postbox_status",
|
||||||
|
sa.String(length=50),
|
||||||
|
nullable=False,
|
||||||
|
server_default="not_requested",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"campaign_jobs",
|
||||||
|
sa.Column(
|
||||||
|
"postbox_attempt_count",
|
||||||
|
sa.Integer(),
|
||||||
|
nullable=False,
|
||||||
|
server_default="0",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"campaign_jobs",
|
||||||
|
sa.Column(
|
||||||
|
"resolved_postbox_targets",
|
||||||
|
sa.JSON(),
|
||||||
|
nullable=False,
|
||||||
|
server_default="[]",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_campaign_jobs_delivery_channel_policy"),
|
||||||
|
"campaign_jobs",
|
||||||
|
["delivery_channel_policy"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_campaign_jobs_postbox_status"),
|
||||||
|
"campaign_jobs",
|
||||||
|
["postbox_status"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"campaign_postbox_delivery_attempts",
|
||||||
|
sa.Column("id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("job_id", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("target_key", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("target_index", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("attempt_number", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=50), nullable=False),
|
||||||
|
sa.Column("target_snapshot", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("provider_delivery_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("provider_message_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("postbox_id", sa.String(length=36), nullable=True),
|
||||||
|
sa.Column("address", sa.String(length=500), nullable=True),
|
||||||
|
sa.Column("holder_count", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("vacant", sa.Boolean(), nullable=True),
|
||||||
|
sa.Column(
|
||||||
|
"duplicate",
|
||||||
|
sa.Boolean(),
|
||||||
|
nullable=False,
|
||||||
|
server_default=sa.false(),
|
||||||
|
),
|
||||||
|
sa.Column("evidence", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("error_type", sa.String(length=255), nullable=True),
|
||||||
|
sa.Column("error_code", sa.String(length=100), nullable=True),
|
||||||
|
sa.Column("error_message", sa.Text(), nullable=True),
|
||||||
|
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["job_id"],
|
||||||
|
["campaign_jobs.id"],
|
||||||
|
name=op.f(
|
||||||
|
"fk_campaign_postbox_delivery_attempts_job_id_campaign_jobs"
|
||||||
|
),
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint(
|
||||||
|
"id",
|
||||||
|
name=op.f("pk_campaign_postbox_delivery_attempts"),
|
||||||
|
),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"job_id",
|
||||||
|
"target_key",
|
||||||
|
"attempt_number",
|
||||||
|
name="uq_campaign_postbox_attempt_target_number",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
for column in ("tenant_id", "job_id", "status", "postbox_id"):
|
||||||
|
op.create_index(
|
||||||
|
op.f(f"ix_campaign_postbox_delivery_attempts_{column}"),
|
||||||
|
"campaign_postbox_delivery_attempts",
|
||||||
|
[column],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_campaign_postbox_attempt_job_status",
|
||||||
|
"campaign_postbox_delivery_attempts",
|
||||||
|
["job_id", "status"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_campaign_postbox_attempt_idempotency",
|
||||||
|
"campaign_postbox_delivery_attempts",
|
||||||
|
["tenant_id", "idempotency_key"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("campaign_postbox_delivery_attempts")
|
||||||
|
op.drop_index(
|
||||||
|
op.f("ix_campaign_jobs_postbox_status"),
|
||||||
|
table_name="campaign_jobs",
|
||||||
|
)
|
||||||
|
op.drop_index(
|
||||||
|
op.f("ix_campaign_jobs_delivery_channel_policy"),
|
||||||
|
table_name="campaign_jobs",
|
||||||
|
)
|
||||||
|
op.drop_column("campaign_jobs", "resolved_postbox_targets")
|
||||||
|
op.drop_column("campaign_jobs", "postbox_attempt_count")
|
||||||
|
op.drop_column("campaign_jobs", "postbox_status")
|
||||||
|
op.drop_column("campaign_jobs", "delivery_channel_policy")
|
||||||
170
src/govoplan_campaign/backend/path_security.py
Normal file
170
src/govoplan_campaign/backend/path_security.py
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import PureWindowsPath
|
||||||
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignPathSecurityError(ValueError):
|
||||||
|
"""Raised when server-provided campaign JSON could read local files."""
|
||||||
|
|
||||||
|
|
||||||
|
def _text(value: object) -> str:
|
||||||
|
return value.strip() if isinstance(value, str) else ""
|
||||||
|
|
||||||
|
|
||||||
|
def assert_logical_relative_path(value: object, *, field: str) -> None:
|
||||||
|
raw = _text(value)
|
||||||
|
if not raw:
|
||||||
|
return
|
||||||
|
if "\x00" in raw:
|
||||||
|
raise CampaignPathSecurityError(f"{field} contains a NUL byte")
|
||||||
|
|
||||||
|
normalized = raw.replace("\\", "/")
|
||||||
|
windows_path = PureWindowsPath(raw)
|
||||||
|
if (
|
||||||
|
normalized.startswith("/")
|
||||||
|
or normalized.startswith("~")
|
||||||
|
or windows_path.is_absolute()
|
||||||
|
or bool(windows_path.drive)
|
||||||
|
):
|
||||||
|
raise CampaignPathSecurityError(f"{field} must be a relative managed-file path")
|
||||||
|
if any(part == ".." for part in normalized.split("/")):
|
||||||
|
raise CampaignPathSecurityError(f"{field} must not contain parent-directory traversal")
|
||||||
|
|
||||||
|
|
||||||
|
def is_managed_source(value: object) -> bool:
|
||||||
|
raw = _text(value)
|
||||||
|
if not raw.startswith("managed:"):
|
||||||
|
return False
|
||||||
|
parts = raw.split(":", 2)
|
||||||
|
return len(parts) == 3 and parts[1] in {"user", "group"} and bool(parts[2].strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_rules(raw_json: dict[str, Any]) -> Iterable[tuple[str, dict[str, Any]]]:
|
||||||
|
attachments = raw_json.get("attachments")
|
||||||
|
if isinstance(attachments, dict):
|
||||||
|
global_rules = attachments.get("global")
|
||||||
|
if isinstance(global_rules, list):
|
||||||
|
for index, rule in enumerate(global_rules):
|
||||||
|
if isinstance(rule, dict):
|
||||||
|
yield f"attachments.global[{index}]", rule
|
||||||
|
|
||||||
|
entries = raw_json.get("entries")
|
||||||
|
inline = entries.get("inline") if isinstance(entries, dict) else None
|
||||||
|
if isinstance(inline, list):
|
||||||
|
for entry_index, entry in enumerate(inline):
|
||||||
|
rules = entry.get("attachments") if isinstance(entry, dict) else None
|
||||||
|
if not isinstance(rules, list):
|
||||||
|
continue
|
||||||
|
for rule_index, rule in enumerate(rules):
|
||||||
|
if isinstance(rule, dict):
|
||||||
|
yield f"entries.inline[{entry_index}].attachments[{rule_index}]", rule
|
||||||
|
defaults = entries.get("defaults") if isinstance(entries, dict) else None
|
||||||
|
default_rules = defaults.get("attachments") if isinstance(defaults, dict) else None
|
||||||
|
if isinstance(default_rules, list):
|
||||||
|
for rule_index, rule in enumerate(default_rules):
|
||||||
|
if isinstance(rule, dict):
|
||||||
|
yield f"entries.defaults.attachments[{rule_index}]", rule
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_inline_server_sources(
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
*,
|
||||||
|
source_filename: str | None,
|
||||||
|
source_base_path: str | None,
|
||||||
|
) -> None:
|
||||||
|
if _text(source_filename):
|
||||||
|
raise CampaignPathSecurityError("source_filename is not accepted for server-managed campaigns")
|
||||||
|
if _text(source_base_path):
|
||||||
|
raise CampaignPathSecurityError("source_base_path is not accepted for server-managed campaigns")
|
||||||
|
|
||||||
|
template = raw_json.get("template")
|
||||||
|
template_source = template.get("source") if isinstance(template, dict) else None
|
||||||
|
path_fields = ("subject_path", "text_path", "html_path")
|
||||||
|
if isinstance(template_source, dict) and any(_text(template_source.get(key)) for key in path_fields):
|
||||||
|
raise CampaignPathSecurityError(
|
||||||
|
"template source paths are not accepted for server-managed campaigns; store template content inline"
|
||||||
|
)
|
||||||
|
|
||||||
|
entries = raw_json.get("entries")
|
||||||
|
entries_source = entries.get("source") if isinstance(entries, dict) else None
|
||||||
|
if isinstance(entries_source, dict) and _text(entries_source.get("path")):
|
||||||
|
raise CampaignPathSecurityError(
|
||||||
|
"entries.source.path is not accepted for server-managed campaigns; import recipients into the campaign snapshot"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _managed_attachment_base_paths(attachments: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||||
|
assert_logical_relative_path(attachments.get("base_path"), field="attachments.base_path")
|
||||||
|
raw_base_paths = attachments.get("base_paths")
|
||||||
|
base_paths = raw_base_paths if isinstance(raw_base_paths, list) else []
|
||||||
|
managed: dict[str, dict[str, Any]] = {}
|
||||||
|
for index, item in enumerate(base_paths):
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
assert_logical_relative_path(
|
||||||
|
item.get("path"),
|
||||||
|
field=f"attachments.base_paths[{index}].path",
|
||||||
|
)
|
||||||
|
source_is_managed = is_managed_source(item.get("source"))
|
||||||
|
base_path_id = _text(item.get("id"))
|
||||||
|
if base_path_id and source_is_managed:
|
||||||
|
managed[base_path_id] = item
|
||||||
|
if item.get("unsent_warning") is True and not source_is_managed:
|
||||||
|
raise CampaignPathSecurityError(
|
||||||
|
f"attachments.base_paths[{index}] must use a managed Files source before unsent-file scanning is enabled"
|
||||||
|
)
|
||||||
|
return managed
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_managed_attachment_rules(
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
*,
|
||||||
|
managed_base_paths: dict[str, dict[str, Any]],
|
||||||
|
managed_files_available: bool,
|
||||||
|
) -> None:
|
||||||
|
rules = list(_attachment_rules(raw_json))
|
||||||
|
if not rules:
|
||||||
|
return
|
||||||
|
if not managed_files_available:
|
||||||
|
raise CampaignPathSecurityError("campaign attachments require the Files module in server mode")
|
||||||
|
|
||||||
|
for field, rule in rules:
|
||||||
|
base_path_id = _text(rule.get("base_path_id"))
|
||||||
|
if not base_path_id or base_path_id not in managed_base_paths:
|
||||||
|
raise CampaignPathSecurityError(
|
||||||
|
f"{field}.base_path_id must select an attachments.base_paths entry backed by managed Files"
|
||||||
|
)
|
||||||
|
assert_logical_relative_path(rule.get("base_dir"), field=f"{field}.base_dir")
|
||||||
|
assert_logical_relative_path(rule.get("file_filter"), field=f"{field}.file_filter")
|
||||||
|
|
||||||
|
|
||||||
|
def assert_server_safe_campaign_paths(
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
*,
|
||||||
|
source_filename: str | None = None,
|
||||||
|
source_base_path: str | None = None,
|
||||||
|
managed_files_available: bool,
|
||||||
|
) -> None:
|
||||||
|
"""Reject local-file references at the HTTP/server trust boundary.
|
||||||
|
|
||||||
|
File-oriented campaign loading remains available to trusted operator code
|
||||||
|
through the campaign loader and message builder. Persisted/API campaigns
|
||||||
|
must use inline templates and recipients plus managed Files attachment
|
||||||
|
sources, which are materialized into an isolated snapshot before use.
|
||||||
|
"""
|
||||||
|
|
||||||
|
_assert_inline_server_sources(
|
||||||
|
raw_json,
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
attachments = raw_json.get("attachments")
|
||||||
|
if not isinstance(attachments, dict):
|
||||||
|
return
|
||||||
|
_assert_managed_attachment_rules(
|
||||||
|
raw_json,
|
||||||
|
managed_base_paths=_managed_attachment_base_paths(attachments),
|
||||||
|
managed_files_available=managed_files_available,
|
||||||
|
)
|
||||||
@@ -1,13 +1,14 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
|
import os
|
||||||
|
from datetime import UTC, datetime
|
||||||
from email import policy
|
from email import policy
|
||||||
from email.parser import BytesParser
|
from email.parser import BytesParser
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
import copy
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
@@ -21,17 +22,36 @@ from govoplan_campaign.backend.db.models import (
|
|||||||
CampaignVersion,
|
CampaignVersion,
|
||||||
CampaignVersionWorkflowState,
|
CampaignVersionWorkflowState,
|
||||||
JobImapStatus,
|
JobImapStatus,
|
||||||
|
JobPostboxStatus,
|
||||||
JobQueueStatus,
|
JobQueueStatus,
|
||||||
JobSendStatus,
|
JobSendStatus,
|
||||||
JobValidationStatus,
|
JobValidationStatus,
|
||||||
)
|
)
|
||||||
from govoplan_campaign.backend.campaign.loader import load_campaign_config, load_campaign_json, validate_against_schema
|
from govoplan_campaign.backend.campaign.loader import load_campaign_json, validate_against_schema
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
assert_campaign_uses_mail_profile_reference,
|
||||||
|
campaign_mail_profile_id,
|
||||||
|
campaign_mail_resource_ids,
|
||||||
|
)
|
||||||
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
||||||
|
from govoplan_campaign.backend.campaign.entries import load_campaign_entries
|
||||||
|
from govoplan_campaign.backend.campaign.postbox_targets import (
|
||||||
|
resolve_entry_postbox_targets,
|
||||||
|
)
|
||||||
from govoplan_campaign.backend.messages.builder import build_campaign_messages
|
from govoplan_campaign.backend.messages.builder import build_campaign_messages
|
||||||
from govoplan_campaign.backend.messages.models import MessageDraft
|
from govoplan_campaign.backend.messages.models import MessageDraft
|
||||||
from govoplan_campaign.backend.sending.execution import create_execution_snapshot
|
from govoplan_campaign.backend.sending.execution import create_execution_snapshot, profile_delivery_summary
|
||||||
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
from govoplan_campaign.backend.campaign.models import (
|
||||||
from govoplan_campaign.backend.integrations import files_integration, mail_integration
|
CampaignConfig,
|
||||||
|
DeliveryChannelPolicy,
|
||||||
|
SendStatus,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
files_integration,
|
||||||
|
mail_integration,
|
||||||
|
postbox_integration,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths
|
||||||
|
|
||||||
RUNTIME_DIR = Path(__file__).resolve().parents[3] / "runtime"
|
RUNTIME_DIR = Path(__file__).resolve().parents[3] / "runtime"
|
||||||
CAMPAIGN_SNAPSHOT_DIR = RUNTIME_DIR / "campaign_snapshots"
|
CAMPAIGN_SNAPSHOT_DIR = RUNTIME_DIR / "campaign_snapshots"
|
||||||
@@ -43,8 +63,9 @@ class CampaignPersistenceError(RuntimeError):
|
|||||||
|
|
||||||
|
|
||||||
def _ensure_dirs() -> None:
|
def _ensure_dirs() -> None:
|
||||||
CAMPAIGN_SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
|
for directory in (CAMPAIGN_SNAPSHOT_DIR, BUILD_OUTPUT_DIR):
|
||||||
BUILD_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
directory.mkdir(parents=True, mode=0o700, exist_ok=True)
|
||||||
|
directory.chmod(0o700)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -58,22 +79,49 @@ def load_campaign_config_from_json(
|
|||||||
owner_user_id: str | None = None,
|
owner_user_id: str | None = None,
|
||||||
owner_group_id: str | None = None,
|
owner_group_id: str | None = None,
|
||||||
) -> CampaignConfig:
|
) -> CampaignConfig:
|
||||||
materialized = mail_integration().materialize_campaign_mail_profile_config(
|
# Validate the persisted Campaign-to-Mail contract before asking Mail for a
|
||||||
session,
|
# non-secret capability summary. Campaign never receives resolved transport
|
||||||
tenant_id=tenant_id,
|
# settings, account identities, or credentials.
|
||||||
raw_json=raw_json,
|
assert_campaign_uses_mail_profile_reference(raw_json)
|
||||||
campaign_id=campaign_id,
|
validate_against_schema(raw_json)
|
||||||
owner_user_id=owner_user_id,
|
materialized = copy.deepcopy(raw_json)
|
||||||
owner_group_id=owner_group_id,
|
profile_id = campaign_mail_profile_id(raw_json)
|
||||||
)
|
if profile_id:
|
||||||
validate_against_schema(materialized)
|
references = campaign_mail_resource_ids(raw_json)
|
||||||
|
summary = mail_integration().campaign_profile_delivery_summary(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
profile_id=profile_id,
|
||||||
|
owner_user_id=owner_user_id,
|
||||||
|
owner_group_id=owner_group_id,
|
||||||
|
smtp_server_id=references["smtp_server_id"],
|
||||||
|
smtp_credential_id=references["smtp_credential_id"],
|
||||||
|
imap_server_id=references["imap_server_id"],
|
||||||
|
imap_credential_id=references["imap_credential_id"],
|
||||||
|
)
|
||||||
|
materialized.setdefault("server", {})["profile_capabilities"] = {
|
||||||
|
"smtp_available": bool(summary.get("smtp_available")),
|
||||||
|
"imap_available": bool(summary.get("imap_available")),
|
||||||
|
}
|
||||||
return CampaignConfig.model_validate(materialized)
|
return CampaignConfig.model_validate(materialized)
|
||||||
|
|
||||||
|
|
||||||
def _write_campaign_snapshot(version: CampaignVersion) -> Path:
|
def _write_campaign_snapshot(version: CampaignVersion) -> Path:
|
||||||
_ensure_dirs()
|
_ensure_dirs()
|
||||||
path = CAMPAIGN_SNAPSHOT_DIR / f"{version.id}.json"
|
path = CAMPAIGN_SNAPSHOT_DIR / f"{version.id}.json"
|
||||||
path.write_text(json.dumps(version.raw_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC
|
||||||
|
if hasattr(os, "O_NOFOLLOW"):
|
||||||
|
flags |= os.O_NOFOLLOW
|
||||||
|
descriptor = os.open(path, flags, 0o600)
|
||||||
|
try:
|
||||||
|
os.fchmod(descriptor, 0o600)
|
||||||
|
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
|
||||||
|
descriptor = -1
|
||||||
|
json.dump(version.raw_json, stream, ensure_ascii=False, indent=2)
|
||||||
|
finally:
|
||||||
|
if descriptor >= 0:
|
||||||
|
os.close(descriptor)
|
||||||
return path
|
return path
|
||||||
|
|
||||||
|
|
||||||
@@ -92,13 +140,14 @@ def _resolve_runtime_path(base_path: Path | None, value: str | None) -> str | No
|
|||||||
|
|
||||||
|
|
||||||
def normalize_campaign_paths(raw_json: dict[str, Any], source_base_path: str | Path | None) -> dict[str, Any]:
|
def normalize_campaign_paths(raw_json: dict[str, Any], source_base_path: str | Path | None) -> dict[str, Any]:
|
||||||
"""Return a DB/runtime-safe campaign JSON snapshot.
|
"""Resolve paths for an explicitly trusted, file-oriented import.
|
||||||
|
|
||||||
The CLI naturally resolves relative paths against the campaign.json file.
|
The CLI naturally resolves relative paths against the campaign.json file.
|
||||||
Once the campaign is stored in the database, the JSON snapshot lives in
|
Once the campaign is stored in the database, the JSON snapshot lives in
|
||||||
app/mailer/runtime/campaign_snapshots. To keep existing file-based
|
app/mailer/runtime/campaign_snapshots. To keep existing file-based
|
||||||
campaigns working, relative file paths are normalized to absolute paths at
|
campaigns working, relative file paths are normalized to absolute paths at
|
||||||
import time when a source_base_path is known.
|
import time when a source_base_path is known. HTTP/API callers are rejected
|
||||||
|
by ``assert_server_safe_campaign_paths`` before they can reach this helper.
|
||||||
"""
|
"""
|
||||||
base = Path(source_base_path).expanduser().resolve() if source_base_path else None
|
base = Path(source_base_path).expanduser().resolve() if source_base_path else None
|
||||||
data = copy.deepcopy(raw_json)
|
data = copy.deepcopy(raw_json)
|
||||||
@@ -127,7 +176,14 @@ def create_campaign_version_from_json(
|
|||||||
raw_json: dict[str, Any],
|
raw_json: dict[str, Any],
|
||||||
source_filename: str | None = None,
|
source_filename: str | None = None,
|
||||||
source_base_path: str | None = None,
|
source_base_path: str | None = None,
|
||||||
|
commit: bool = True,
|
||||||
) -> tuple[Campaign, CampaignVersion]:
|
) -> tuple[Campaign, CampaignVersion]:
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
raw_json,
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
managed_files_available=files_integration().available,
|
||||||
|
)
|
||||||
if source_base_path is None and source_filename:
|
if source_base_path is None and source_filename:
|
||||||
source_path = Path(source_filename).expanduser()
|
source_path = Path(source_filename).expanduser()
|
||||||
source_base_path = str(source_path.parent if source_path.suffix else source_path)
|
source_base_path = str(source_path.parent if source_path.suffix else source_path)
|
||||||
@@ -175,8 +231,11 @@ def create_campaign_version_from_json(
|
|||||||
session.flush()
|
session.flush()
|
||||||
campaign.current_version_id = version.id
|
campaign.current_version_id = version.id
|
||||||
session.add(campaign)
|
session.add(campaign)
|
||||||
_write_campaign_snapshot(version)
|
if commit:
|
||||||
session.commit()
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return campaign, version
|
return campaign, version
|
||||||
|
|
||||||
|
|
||||||
@@ -235,8 +294,12 @@ def load_version_config(session: Session, version_id: str):
|
|||||||
campaign = session.get(Campaign, version.campaign_id)
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
if not campaign:
|
if not campaign:
|
||||||
raise CampaignPersistenceError(f"Campaign not found for version: {version_id}")
|
raise CampaignPersistenceError(f"Campaign not found for version: {version_id}")
|
||||||
path = _write_campaign_snapshot(version)
|
|
||||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
raw_json,
|
||||||
|
managed_files_available=files_integration().available,
|
||||||
|
)
|
||||||
|
path = _write_campaign_snapshot(version)
|
||||||
return version, path, load_campaign_config_from_json(session, tenant_id=campaign.tenant_id, raw_json=raw_json, campaign_id=campaign.id)
|
return version, path, load_campaign_config_from_json(session, tenant_id=campaign.tenant_id, raw_json=raw_json, campaign_id=campaign.id)
|
||||||
|
|
||||||
|
|
||||||
@@ -279,9 +342,19 @@ def validate_campaign_version(
|
|||||||
) as prepared:
|
) as prepared:
|
||||||
managed_raw = load_campaign_json(prepared.path)
|
managed_raw = load_campaign_json(prepared.path)
|
||||||
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
|
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
|
||||||
report = validate_campaign_config(managed_config, campaign_file=prepared.path, check_files=True)
|
report = validate_campaign_config(
|
||||||
|
managed_config,
|
||||||
|
campaign_file=prepared.path,
|
||||||
|
check_files=True,
|
||||||
|
postbox_available=postbox_integration().available,
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
report = validate_campaign_config(config, campaign_file=snapshot_path, check_files=False)
|
report = validate_campaign_config(
|
||||||
|
config,
|
||||||
|
campaign_file=snapshot_path,
|
||||||
|
check_files=False,
|
||||||
|
postbox_available=postbox_integration().available,
|
||||||
|
)
|
||||||
report_json = report.model_dump(mode="json")
|
report_json = report.model_dump(mode="json")
|
||||||
report_json.update({"ok": report.ok, "error_count": report.error_count, "warning_count": report.warning_count})
|
report_json.update({"ok": report.ok, "error_count": report.error_count, "warning_count": report.warning_count})
|
||||||
version.validation_summary = report_json
|
version.validation_summary = report_json
|
||||||
@@ -344,6 +417,7 @@ def _job_from_message(
|
|||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
message: MessageDraft,
|
message: MessageDraft,
|
||||||
|
resolved_postbox_targets: list[dict[str, Any]] | None = None,
|
||||||
) -> CampaignJob:
|
) -> CampaignJob:
|
||||||
recipient_email = message.to[0].email if message.to else None
|
recipient_email = message.to[0].email if message.to else None
|
||||||
eml_sha256, message_id_header = _eml_evidence(message.eml_path)
|
eml_sha256, message_id_header = _eml_evidence(message.eml_path)
|
||||||
@@ -362,7 +436,17 @@ def _job_from_message(
|
|||||||
build_status=message.build_status.value if hasattr(message.build_status, "value") else str(message.build_status),
|
build_status=message.build_status.value if hasattr(message.build_status, "value") else str(message.build_status),
|
||||||
validation_status=_job_validation_status(message.validation_status.value),
|
validation_status=_job_validation_status(message.validation_status.value),
|
||||||
queue_status=JobQueueStatus.DRAFT.value,
|
queue_status=JobQueueStatus.DRAFT.value,
|
||||||
send_status=JobSendStatus.NOT_QUEUED.value,
|
send_status=(
|
||||||
|
JobSendStatus.SKIPPED.value
|
||||||
|
if message.send_status == SendStatus.SKIPPED
|
||||||
|
else JobSendStatus.NOT_QUEUED.value
|
||||||
|
),
|
||||||
|
delivery_channel_policy=message.delivery_channel_policy,
|
||||||
|
postbox_status=(
|
||||||
|
JobPostboxStatus.PENDING.value
|
||||||
|
if DeliveryChannelPolicy(message.delivery_channel_policy).uses_postbox
|
||||||
|
else JobPostboxStatus.NOT_REQUESTED.value
|
||||||
|
),
|
||||||
imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value,
|
imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value,
|
||||||
resolved_recipients={
|
resolved_recipients={
|
||||||
"from": message.from_.model_dump(mode="json") if message.from_ else None,
|
"from": message.from_.model_dump(mode="json") if message.from_ else None,
|
||||||
@@ -374,12 +458,184 @@ def _job_from_message(
|
|||||||
"bounce_to": [item.model_dump(mode="json") for item in message.bounce_to],
|
"bounce_to": [item.model_dump(mode="json") for item in message.bounce_to],
|
||||||
"disposition_notification_to": [item.model_dump(mode="json") for item in message.disposition_notification_to],
|
"disposition_notification_to": [item.model_dump(mode="json") for item in message.disposition_notification_to],
|
||||||
},
|
},
|
||||||
|
resolved_postbox_targets=resolved_postbox_targets or [],
|
||||||
resolved_attachments=[files_integration().public_attachment_summary_payload(item) for item in message.attachments],
|
resolved_attachments=[files_integration().public_attachment_summary_payload(item) for item in message.attachments],
|
||||||
issues_snapshot=[item.model_dump(mode="json") for item in message.issues],
|
issues_snapshot=[item.model_dump(mode="json") for item in message.issues],
|
||||||
last_error="; ".join(issue.message for issue in message.issues if issue.severity == "error") or None,
|
last_error="; ".join(issue.message for issue in message.issues if issue.severity == "error") or None,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_built_postbox_targets(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
config: CampaignConfig,
|
||||||
|
built_messages: list[Any],
|
||||||
|
entries_by_index: dict[int, Any],
|
||||||
|
) -> dict[int, list[dict[str, Any]]]:
|
||||||
|
resolved_by_index: dict[int, list[dict[str, Any]]] = {}
|
||||||
|
for built in built_messages:
|
||||||
|
if not DeliveryChannelPolicy(built.draft.delivery_channel_policy).uses_postbox:
|
||||||
|
continue
|
||||||
|
entry = entries_by_index.get(built.draft.entry_index)
|
||||||
|
if entry is None:
|
||||||
|
raise CampaignPersistenceError("Built recipient row is missing from the campaign input.")
|
||||||
|
resolved, issues, validation_status = resolve_entry_postbox_targets(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
config=config,
|
||||||
|
entry=entry,
|
||||||
|
validation_status=built.draft.validation_status,
|
||||||
|
materialize=True,
|
||||||
|
)
|
||||||
|
built.draft.issues.extend(issues)
|
||||||
|
built.draft.validation_status = validation_status
|
||||||
|
resolved_by_index[built.draft.entry_index] = resolved
|
||||||
|
return resolved_by_index
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_build_report(result: Any, files: Any) -> dict[str, Any]:
|
||||||
|
report_json = result.report.model_dump(mode="json", by_alias=True)
|
||||||
|
for message_payload, message in zip(report_json.get("messages", []), result.report.messages, strict=False):
|
||||||
|
if isinstance(message_payload, dict):
|
||||||
|
message_payload["attachments"] = [files.public_attachment_summary_payload(item) for item in message.attachments]
|
||||||
|
report_json.update({
|
||||||
|
"built_at": datetime.now(UTC).isoformat(),
|
||||||
|
"build_token": uuid4().hex,
|
||||||
|
"built_count": result.report.built_count,
|
||||||
|
"build_failed_count": result.report.build_failed_count,
|
||||||
|
"ready_count": result.report.ready_count,
|
||||||
|
"warning_count": result.report.warning_count,
|
||||||
|
"needs_review_count": result.report.needs_review_count,
|
||||||
|
"blocked_count": result.report.blocked_count,
|
||||||
|
"excluded_count": result.report.excluded_count,
|
||||||
|
"inactive_count": result.report.inactive_count,
|
||||||
|
"queueable_count": result.report.queueable_count,
|
||||||
|
})
|
||||||
|
return report_json
|
||||||
|
|
||||||
|
|
||||||
|
def _replace_version_jobs(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
built_messages: list[Any],
|
||||||
|
postbox_targets_by_index: dict[int, list[dict[str, Any]]],
|
||||||
|
) -> list[tuple[CampaignJob, MessageDraft]]:
|
||||||
|
session.query(CampaignIssue).filter(
|
||||||
|
CampaignIssue.campaign_version_id == version_id,
|
||||||
|
CampaignIssue.job_id.is_not(None),
|
||||||
|
).delete(synchronize_session=False)
|
||||||
|
session.query(CampaignJob).filter(CampaignJob.campaign_version_id == version_id).delete(synchronize_session=False)
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
pairs: list[tuple[CampaignJob, MessageDraft]] = []
|
||||||
|
for built in built_messages:
|
||||||
|
job = _job_from_message(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
message=built.draft,
|
||||||
|
resolved_postbox_targets=postbox_targets_by_index.get(built.draft.entry_index, []),
|
||||||
|
)
|
||||||
|
session.add(job)
|
||||||
|
pairs.append((job, built.draft))
|
||||||
|
session.flush()
|
||||||
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
def _mail_execution_profile(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
version: CampaignVersion,
|
||||||
|
config: CampaignConfig,
|
||||||
|
jobs: list[CampaignJob],
|
||||||
|
) -> tuple[str | None, dict[str, Any]]:
|
||||||
|
if not any(DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail for job in jobs):
|
||||||
|
return None, {}
|
||||||
|
if not config.server.profile_capabilities.smtp_available:
|
||||||
|
raise CampaignPersistenceError("The selected Mail profile has no SMTP configuration; an execution snapshot cannot be created.")
|
||||||
|
profile_id = campaign_mail_profile_id(version.raw_json if isinstance(version.raw_json, dict) else {})
|
||||||
|
if profile_id is None:
|
||||||
|
raise CampaignPersistenceError("Select an authorized Mail profile before building campaign messages that use Mail.")
|
||||||
|
summary = profile_delivery_summary(session, version)
|
||||||
|
if not summary.get("smtp_transport_revision"):
|
||||||
|
raise CampaignPersistenceError("The selected Mail profile has no SMTP transport revision.")
|
||||||
|
return profile_id, summary
|
||||||
|
|
||||||
|
|
||||||
|
def _store_execution_snapshot(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
version: CampaignVersion,
|
||||||
|
config: CampaignConfig,
|
||||||
|
jobs: list[CampaignJob],
|
||||||
|
build_summary: dict[str, Any],
|
||||||
|
) -> None:
|
||||||
|
profile_id, profile = _mail_execution_profile(session, version=version, config=config, jobs=jobs)
|
||||||
|
snapshot, snapshot_hash = create_execution_snapshot(
|
||||||
|
version,
|
||||||
|
mail_profile_id=profile_id,
|
||||||
|
smtp_server_id=profile.get("smtp_server_id"),
|
||||||
|
smtp_credential_id=profile.get("smtp_credential_id"),
|
||||||
|
imap_server_id=profile.get("imap_server_id"),
|
||||||
|
imap_credential_id=profile.get("imap_credential_id"),
|
||||||
|
smtp_transport_revision=profile.get("smtp_transport_revision"),
|
||||||
|
imap_transport_revision=profile.get("imap_transport_revision"),
|
||||||
|
delivery=config.delivery,
|
||||||
|
jobs=jobs,
|
||||||
|
build_summary=build_summary,
|
||||||
|
)
|
||||||
|
version.execution_snapshot = snapshot
|
||||||
|
version.execution_snapshot_hash = snapshot_hash
|
||||||
|
version.execution_snapshot_at = datetime.now(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _store_job_issues(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
job_build_pairs: list[tuple[CampaignJob, MessageDraft]],
|
||||||
|
) -> None:
|
||||||
|
for job, message in job_build_pairs:
|
||||||
|
session.add_all([
|
||||||
|
CampaignIssue(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
campaign_version_id=version_id,
|
||||||
|
job_id=job.id,
|
||||||
|
severity=issue.severity,
|
||||||
|
code=issue.code,
|
||||||
|
message=issue.message,
|
||||||
|
source=issue.source,
|
||||||
|
behavior=issue.behavior,
|
||||||
|
)
|
||||||
|
for issue in message.issues
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_campaign_build_state(
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
*,
|
||||||
|
needs_review_count: int,
|
||||||
|
blocked_count: int,
|
||||||
|
queueable_count: int,
|
||||||
|
) -> None:
|
||||||
|
if needs_review_count or blocked_count:
|
||||||
|
campaign.status = CampaignStatus.NEEDS_REVIEW.value
|
||||||
|
version.workflow_state = CampaignVersionWorkflowState.APPROVED.value
|
||||||
|
elif queueable_count > 0:
|
||||||
|
campaign.status = CampaignStatus.READY_TO_QUEUE.value
|
||||||
|
version.workflow_state = CampaignVersionWorkflowState.BUILT.value
|
||||||
|
else:
|
||||||
|
campaign.status = CampaignStatus.VALIDATED.value
|
||||||
|
|
||||||
|
|
||||||
def build_campaign_version(
|
def build_campaign_version(
|
||||||
session: Session,
|
session: Session,
|
||||||
*,
|
*,
|
||||||
@@ -413,91 +669,54 @@ def build_campaign_version(
|
|||||||
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
|
managed_config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=managed_raw, campaign_id=campaign.id)
|
||||||
result = build_campaign_messages(managed_config, campaign_file=prepared.path, output_dir=output_dir, write_eml=write_eml)
|
result = build_campaign_messages(managed_config, campaign_file=prepared.path, output_dir=output_dir, write_eml=write_eml)
|
||||||
files.annotate_built_messages_with_managed_files(result.built_messages, prepared.managed_files_by_local_path)
|
files.annotate_built_messages_with_managed_files(result.built_messages, prepared.managed_files_by_local_path)
|
||||||
report_json = result.report.model_dump(mode="json", by_alias=True)
|
entries_by_index = {
|
||||||
for message_payload, message in zip(report_json.get("messages", []), result.report.messages, strict=False):
|
index: entry
|
||||||
if isinstance(message_payload, dict):
|
for index, entry in enumerate(
|
||||||
message_payload["attachments"] = [files.public_attachment_summary_payload(item) for item in message.attachments]
|
load_campaign_entries(
|
||||||
report_json["built_at"] = datetime.now(UTC).isoformat()
|
managed_config,
|
||||||
report_json["build_token"] = uuid4().hex
|
campaign_file=prepared.path,
|
||||||
report_json.update({
|
),
|
||||||
"built_count": result.report.built_count,
|
start=1,
|
||||||
"build_failed_count": result.report.build_failed_count,
|
)
|
||||||
"ready_count": result.report.ready_count,
|
}
|
||||||
"warning_count": result.report.warning_count,
|
resolved_postbox_targets_by_index = _resolve_built_postbox_targets(
|
||||||
"needs_review_count": result.report.needs_review_count,
|
session,
|
||||||
"blocked_count": result.report.blocked_count,
|
tenant_id=tenant_id,
|
||||||
"excluded_count": result.report.excluded_count,
|
config=managed_config,
|
||||||
"inactive_count": result.report.inactive_count,
|
built_messages=result.built_messages,
|
||||||
"queueable_count": result.report.queueable_count,
|
entries_by_index=entries_by_index,
|
||||||
})
|
)
|
||||||
|
report_json = _campaign_build_report(result, files)
|
||||||
version.build_summary = report_json
|
version.build_summary = report_json
|
||||||
editor_state = copy.deepcopy(version.editor_state or {})
|
editor_state = copy.deepcopy(version.editor_state or {})
|
||||||
editor_state.pop("review_send", None)
|
editor_state.pop("review_send", None)
|
||||||
version.editor_state = editor_state
|
version.editor_state = editor_state
|
||||||
|
|
||||||
# Rebuild jobs for the current version. Later, protect sent jobs from destructive rebuilds.
|
job_build_pairs = _replace_version_jobs(
|
||||||
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id, CampaignIssue.job_id.is_not(None)).delete(synchronize_session=False)
|
|
||||||
session.query(CampaignJob).filter(CampaignJob.campaign_version_id == version.id).delete(synchronize_session=False)
|
|
||||||
session.flush()
|
|
||||||
|
|
||||||
job_build_pairs: list[tuple[CampaignJob, MessageDraft]] = []
|
|
||||||
for built in result.built_messages:
|
|
||||||
job = _job_from_message(
|
|
||||||
tenant_id=tenant_id,
|
|
||||||
campaign_id=campaign.id,
|
|
||||||
version_id=version.id,
|
|
||||||
message=built.draft,
|
|
||||||
)
|
|
||||||
session.add(job)
|
|
||||||
job_build_pairs.append((job, built.draft))
|
|
||||||
|
|
||||||
# Assign all job IDs in one round-trip, then persist exact attachment use
|
|
||||||
# records in bulk. This avoids one flush plus several metadata queries per
|
|
||||||
# recipient for large campaigns.
|
|
||||||
session.flush()
|
|
||||||
files.record_campaign_attachment_uses_for_jobs(
|
|
||||||
session,
|
session,
|
||||||
[job for job, _message in job_build_pairs],
|
tenant_id=tenant_id,
|
||||||
stage="built",
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
built_messages=result.built_messages,
|
||||||
|
postbox_targets_by_index=resolved_postbox_targets_by_index,
|
||||||
)
|
)
|
||||||
runtime_smtp = managed_config.server.runtime_smtp_config()
|
jobs = [job for job, _message in job_build_pairs]
|
||||||
if not runtime_smtp:
|
files.record_campaign_attachment_uses_for_jobs(session, jobs, stage="built")
|
||||||
raise CampaignPersistenceError("Campaign has no SMTP configuration; an execution snapshot cannot be created")
|
_store_execution_snapshot(session, version=version, config=managed_config, jobs=jobs, build_summary=report_json)
|
||||||
execution_snapshot, execution_snapshot_hash = create_execution_snapshot(
|
_store_job_issues(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
job_build_pairs=job_build_pairs,
|
||||||
|
)
|
||||||
|
_apply_campaign_build_state(
|
||||||
|
campaign,
|
||||||
version,
|
version,
|
||||||
smtp=runtime_smtp,
|
needs_review_count=result.report.needs_review_count,
|
||||||
imap=managed_config.server.runtime_imap_config(),
|
blocked_count=result.report.blocked_count,
|
||||||
delivery=managed_config.delivery,
|
queueable_count=result.report.queueable_count,
|
||||||
jobs=[job for job, _message in job_build_pairs],
|
|
||||||
build_summary=report_json,
|
|
||||||
)
|
)
|
||||||
version.execution_snapshot = execution_snapshot
|
|
||||||
version.execution_snapshot_hash = execution_snapshot_hash
|
|
||||||
version.execution_snapshot_at = datetime.now(UTC)
|
|
||||||
for job, message in job_build_pairs:
|
|
||||||
for issue in message.issues:
|
|
||||||
session.add(
|
|
||||||
CampaignIssue(
|
|
||||||
tenant_id=tenant_id,
|
|
||||||
campaign_id=campaign.id,
|
|
||||||
campaign_version_id=version.id,
|
|
||||||
job_id=job.id,
|
|
||||||
severity=issue.severity,
|
|
||||||
code=issue.code,
|
|
||||||
message=issue.message,
|
|
||||||
source=issue.source,
|
|
||||||
behavior=issue.behavior,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
if result.report.needs_review_count or result.report.blocked_count:
|
|
||||||
campaign.status = CampaignStatus.NEEDS_REVIEW.value
|
|
||||||
version.workflow_state = CampaignVersionWorkflowState.APPROVED.value
|
|
||||||
elif result.report.queueable_count > 0:
|
|
||||||
campaign.status = CampaignStatus.READY_TO_QUEUE.value
|
|
||||||
version.workflow_state = CampaignVersionWorkflowState.BUILT.value
|
|
||||||
else:
|
|
||||||
campaign.status = CampaignStatus.VALIDATED.value
|
|
||||||
|
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.add(campaign)
|
session.add(campaign)
|
||||||
|
|||||||
@@ -1,13 +1,20 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import copy
|
import copy
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
from sqlalchemy.orm.exc import StaleDataError
|
||||||
|
|
||||||
|
from govoplan_core.core.concurrency import (
|
||||||
|
RevisionConflictError,
|
||||||
|
strong_resource_etag,
|
||||||
|
)
|
||||||
from govoplan_campaign.backend.db.models import (
|
from govoplan_campaign.backend.db.models import (
|
||||||
Campaign,
|
Campaign,
|
||||||
CampaignIssue,
|
CampaignIssue,
|
||||||
@@ -19,13 +26,22 @@ from govoplan_campaign.backend.db.models import (
|
|||||||
JobSendStatus,
|
JobSendStatus,
|
||||||
)
|
)
|
||||||
from govoplan_campaign.backend.sending.execution import clear_execution_snapshot
|
from govoplan_campaign.backend.sending.execution import clear_execution_snapshot
|
||||||
from govoplan_campaign.backend.integrations import mail_integration
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
campaign_editor_state_for_edit,
|
||||||
|
campaign_mail_profile_boundary_violations,
|
||||||
|
campaign_mail_profile_id,
|
||||||
|
assert_campaign_uses_mail_profile_reference,
|
||||||
|
public_campaign_mail_server,
|
||||||
|
validate_campaign_editor_state,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import files_integration, mail_integration
|
||||||
from govoplan_campaign.backend.persistence.campaigns import (
|
from govoplan_campaign.backend.persistence.campaigns import (
|
||||||
CampaignPersistenceError,
|
CampaignPersistenceError,
|
||||||
_next_version_number,
|
_next_version_number,
|
||||||
_write_campaign_snapshot,
|
_write_campaign_snapshot,
|
||||||
normalize_campaign_paths,
|
normalize_campaign_paths,
|
||||||
)
|
)
|
||||||
|
from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths
|
||||||
|
|
||||||
|
|
||||||
class LockedCampaignVersionError(CampaignPersistenceError):
|
class LockedCampaignVersionError(CampaignPersistenceError):
|
||||||
@@ -105,25 +121,7 @@ def minimal_campaign_json(*, external_id: str, name: str, description: str | Non
|
|||||||
},
|
},
|
||||||
"fields": [],
|
"fields": [],
|
||||||
"global_values": {},
|
"global_values": {},
|
||||||
"server": {
|
"server": {},
|
||||||
"inherit_smtp_credentials": True,
|
|
||||||
"inherit_imap_credentials": True,
|
|
||||||
"smtp": {
|
|
||||||
"host": "",
|
|
||||||
"port": 587,
|
|
||||||
"security": "starttls",
|
|
||||||
},
|
|
||||||
"imap": {
|
|
||||||
"host": "",
|
|
||||||
"port": 993,
|
|
||||||
"security": "tls",
|
|
||||||
"sent_folder": "auto",
|
|
||||||
},
|
|
||||||
"credentials": {
|
|
||||||
"smtp": {"username": "", "password": ""},
|
|
||||||
"imap": {"username": "", "password": ""},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"recipients": {
|
"recipients": {
|
||||||
"from": [],
|
"from": [],
|
||||||
"allow_individual_from": False,
|
"allow_individual_from": False,
|
||||||
@@ -158,9 +156,10 @@ def minimal_campaign_json(*, external_id: str, name: str, description: str | Non
|
|||||||
],
|
],
|
||||||
"allow_individual": True,
|
"allow_individual": True,
|
||||||
"send_without_attachments": False,
|
"send_without_attachments": False,
|
||||||
|
"send_without_attachments_behavior": "block",
|
||||||
"global": [],
|
"global": [],
|
||||||
"zip": {"enabled": False, "archives": []},
|
"zip": {"enabled": False, "archives": []},
|
||||||
"missing_behavior": "ask",
|
"missing_behavior": "warn",
|
||||||
"ambiguous_behavior": "ask",
|
"ambiguous_behavior": "ask",
|
||||||
},
|
},
|
||||||
"entries": {
|
"entries": {
|
||||||
@@ -178,7 +177,7 @@ def minimal_campaign_json(*, external_id: str, name: str, description: str | Non
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
"validation_policy": {
|
"validation_policy": {
|
||||||
"missing_required_attachment": "ask",
|
"missing_required_attachment": "block",
|
||||||
"missing_optional_attachment": "warn",
|
"missing_optional_attachment": "warn",
|
||||||
"ambiguous_attachment_match": "ask",
|
"ambiguous_attachment_match": "ask",
|
||||||
"unsent_attachment_files": "warn",
|
"unsent_attachment_files": "warn",
|
||||||
@@ -216,6 +215,7 @@ def create_minimal_campaign(
|
|||||||
description: str | None = None,
|
description: str | None = None,
|
||||||
current_flow: str = CampaignVersionFlow.CREATE.value,
|
current_flow: str = CampaignVersionFlow.CREATE.value,
|
||||||
current_step: str = "basics",
|
current_step: str = "basics",
|
||||||
|
commit: bool = True,
|
||||||
) -> tuple[Campaign, CampaignVersion]:
|
) -> tuple[Campaign, CampaignVersion]:
|
||||||
existing = session.query(Campaign).filter(Campaign.tenant_id == tenant_id, Campaign.external_id == external_id).one_or_none()
|
existing = session.query(Campaign).filter(Campaign.tenant_id == tenant_id, Campaign.external_id == external_id).one_or_none()
|
||||||
if existing:
|
if existing:
|
||||||
@@ -249,8 +249,11 @@ def create_minimal_campaign(
|
|||||||
session.flush()
|
session.flush()
|
||||||
campaign.current_version_id = version.id
|
campaign.current_version_id = version.id
|
||||||
session.add(campaign)
|
session.add(campaign)
|
||||||
_write_campaign_snapshot(version)
|
if commit:
|
||||||
session.commit()
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return campaign, version
|
return campaign, version
|
||||||
|
|
||||||
|
|
||||||
@@ -332,30 +335,7 @@ def _apply_campaign_metadata(campaign: Campaign, raw_json: dict[str, Any]) -> No
|
|||||||
campaign.external_id = campaign_meta.get("id") or campaign.external_id
|
campaign.external_id = campaign_meta.get("id") or campaign.external_id
|
||||||
|
|
||||||
|
|
||||||
def fork_campaign_version_for_edit(
|
def _assert_fork_source_allowed(session: Session, *, campaign: Campaign, source: CampaignVersion) -> None:
|
||||||
session: Session,
|
|
||||||
*,
|
|
||||||
tenant_id: str,
|
|
||||||
campaign_id: str,
|
|
||||||
version_id: str,
|
|
||||||
raw_json: dict[str, Any] | None = None,
|
|
||||||
current_flow: str | None = None,
|
|
||||||
current_step: str | None = None,
|
|
||||||
editor_state: dict[str, Any] | None = None,
|
|
||||||
source_filename: str | None = None,
|
|
||||||
source_base_path: str | None = None,
|
|
||||||
autosave: bool = True,
|
|
||||||
) -> CampaignVersion:
|
|
||||||
"""Create the next sole working version from immutable campaign history.
|
|
||||||
|
|
||||||
Validation and temporary user locks are still the active working version
|
|
||||||
and must be unlocked in place. A copy is allowed only once the current
|
|
||||||
version is permanently user-locked or delivery-final.
|
|
||||||
"""
|
|
||||||
|
|
||||||
source = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
|
||||||
campaign = _require_campaign(session, campaign_id)
|
|
||||||
|
|
||||||
if campaign_has_active_working_version(session, campaign):
|
if campaign_has_active_working_version(session, campaign):
|
||||||
current = session.get(CampaignVersion, campaign.current_version_id)
|
current = session.get(CampaignVersion, campaign.current_version_id)
|
||||||
current_number = current.version_number if current else "current"
|
current_number = current.version_number if current else "current"
|
||||||
@@ -369,11 +349,59 @@ def fork_campaign_version_for_edit(
|
|||||||
"Create the next working copy from the campaign's current immutable version."
|
"Create the next working copy from the campaign's current immutable version."
|
||||||
)
|
)
|
||||||
|
|
||||||
base_json = raw_json if raw_json is not None else copy.deepcopy(source.raw_json)
|
|
||||||
runtime_json = normalize_campaign_paths(base_json, source_base_path) if source_base_path else copy.deepcopy(base_json)
|
|
||||||
mail_integration().assert_campaign_mail_policy_allows_json(session, tenant_id=tenant_id, raw_json=runtime_json, campaign_id=campaign.id)
|
|
||||||
|
|
||||||
new_version = CampaignVersion(
|
def _fork_runtime_json(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign: Campaign,
|
||||||
|
source: CampaignVersion,
|
||||||
|
raw_json: dict[str, Any] | None,
|
||||||
|
source_filename: str | None,
|
||||||
|
source_base_path: str | None,
|
||||||
|
migrate_legacy_mail_settings: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
source_json = source.raw_json if isinstance(source.raw_json, dict) else {}
|
||||||
|
requires_migration = bool(campaign_mail_profile_boundary_violations(source_json))
|
||||||
|
if requires_migration and not migrate_legacy_mail_settings:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"This version contains legacy campaign-local SMTP/IMAP settings. Create the editable copy from "
|
||||||
|
"the Mail settings migration action so the audit record is preserved and the copy uses a Mail profile."
|
||||||
|
)
|
||||||
|
base_json = raw_json if raw_json is not None else copy.deepcopy(source_json)
|
||||||
|
if requires_migration and raw_json is None:
|
||||||
|
base_json["server"] = public_campaign_mail_server(source_json)
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
base_json,
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
managed_files_available=files_integration().available,
|
||||||
|
)
|
||||||
|
runtime_json = normalize_campaign_paths(base_json, source_base_path) if source_base_path else copy.deepcopy(base_json)
|
||||||
|
assert_campaign_uses_mail_profile_reference(runtime_json)
|
||||||
|
mail_integration().assert_campaign_mail_policy_allows_json(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
raw_json=runtime_json,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
)
|
||||||
|
return runtime_json
|
||||||
|
|
||||||
|
|
||||||
|
def _new_forked_campaign_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
source: CampaignVersion,
|
||||||
|
runtime_json: dict[str, Any],
|
||||||
|
current_flow: str | None,
|
||||||
|
current_step: str | None,
|
||||||
|
editor_state: dict[str, Any] | None,
|
||||||
|
source_filename: str | None,
|
||||||
|
source_base_path: str | None,
|
||||||
|
autosave: bool,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
return CampaignVersion(
|
||||||
campaign_id=campaign.id,
|
campaign_id=campaign.id,
|
||||||
version_number=_next_version_number(session, campaign.id),
|
version_number=_next_version_number(session, campaign.id),
|
||||||
raw_json=runtime_json,
|
raw_json=runtime_json,
|
||||||
@@ -384,18 +412,85 @@ def fork_campaign_version_for_edit(
|
|||||||
current_flow=current_flow if current_flow is not None else (source.current_flow or CampaignVersionFlow.MANUAL.value),
|
current_flow=current_flow if current_flow is not None else (source.current_flow or CampaignVersionFlow.MANUAL.value),
|
||||||
current_step=current_step if current_step is not None else source.current_step,
|
current_step=current_step if current_step is not None else source.current_step,
|
||||||
is_complete=False,
|
is_complete=False,
|
||||||
editor_state=editor_state if editor_state is not None else copy.deepcopy(source.editor_state or {}),
|
editor_state=(
|
||||||
|
validate_campaign_editor_state(editor_state)
|
||||||
|
if editor_state is not None
|
||||||
|
else campaign_editor_state_for_edit(source.editor_state)
|
||||||
|
),
|
||||||
autosaved_at=datetime.now(UTC) if autosave else None,
|
autosaved_at=datetime.now(UTC) if autosave else None,
|
||||||
)
|
)
|
||||||
session.add(new_version)
|
|
||||||
session.flush()
|
|
||||||
|
|
||||||
_apply_campaign_metadata(campaign, runtime_json)
|
|
||||||
campaign.current_version_id = new_version.id
|
def _persist_forked_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
commit: bool,
|
||||||
|
) -> None:
|
||||||
|
session.add(version)
|
||||||
|
session.flush()
|
||||||
|
_apply_campaign_metadata(campaign, version.raw_json)
|
||||||
|
campaign.current_version_id = version.id
|
||||||
campaign.status = CampaignStatus.DRAFT.value
|
campaign.status = CampaignStatus.DRAFT.value
|
||||||
session.add(campaign)
|
session.add(campaign)
|
||||||
_write_campaign_snapshot(new_version)
|
if commit:
|
||||||
session.commit()
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def fork_campaign_version_for_edit(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
raw_json: dict[str, Any] | None = None,
|
||||||
|
current_flow: str | None = None,
|
||||||
|
current_step: str | None = None,
|
||||||
|
editor_state: dict[str, Any] | None = None,
|
||||||
|
source_filename: str | None = None,
|
||||||
|
source_base_path: str | None = None,
|
||||||
|
autosave: bool = True,
|
||||||
|
migrate_legacy_mail_settings: bool = False,
|
||||||
|
commit: bool = True,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
"""Create the next sole working version from immutable campaign history.
|
||||||
|
|
||||||
|
Validation and temporary user locks are still the active working version
|
||||||
|
and must be unlocked in place. A copy is allowed only once the current
|
||||||
|
version is permanently user-locked or delivery-final.
|
||||||
|
"""
|
||||||
|
|
||||||
|
source = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
||||||
|
campaign = _require_campaign(session, campaign_id)
|
||||||
|
|
||||||
|
_assert_fork_source_allowed(session, campaign=campaign, source=source)
|
||||||
|
runtime_json = _fork_runtime_json(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign=campaign,
|
||||||
|
source=source,
|
||||||
|
raw_json=raw_json,
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
migrate_legacy_mail_settings=migrate_legacy_mail_settings,
|
||||||
|
)
|
||||||
|
new_version = _new_forked_campaign_version(
|
||||||
|
session,
|
||||||
|
campaign=campaign,
|
||||||
|
source=source,
|
||||||
|
runtime_json=runtime_json,
|
||||||
|
current_flow=current_flow,
|
||||||
|
current_step=current_step,
|
||||||
|
editor_state=editor_state,
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
autosave=autosave,
|
||||||
|
)
|
||||||
|
_persist_forked_version(session, campaign=campaign, version=new_version, commit=commit)
|
||||||
return new_version
|
return new_version
|
||||||
|
|
||||||
|
|
||||||
@@ -453,6 +548,7 @@ def unlock_validated_campaign_version(
|
|||||||
tenant_id: str,
|
tenant_id: str,
|
||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Unlock a validation snapshot so it can be edited again.
|
"""Unlock a validation snapshot so it can be edited again.
|
||||||
|
|
||||||
@@ -502,9 +598,119 @@ def unlock_validated_campaign_version(
|
|||||||
campaign.status = CampaignStatus.DRAFT.value
|
campaign.status = CampaignStatus.DRAFT.value
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.add(campaign)
|
session.add(campaign)
|
||||||
session.commit()
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
def _assert_update_paths_safe(
|
||||||
|
raw_json: dict[str, Any] | None,
|
||||||
|
*,
|
||||||
|
source_filename: str | None,
|
||||||
|
source_base_path: str | None,
|
||||||
|
) -> None:
|
||||||
|
if raw_json is None and source_filename is None and source_base_path is None:
|
||||||
|
return
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
raw_json if raw_json is not None else {},
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
managed_files_available=files_integration().available,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _updated_runtime_json(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
source_base_path: str | None,
|
||||||
|
migrate_legacy_mail_settings: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
runtime_json = normalize_campaign_paths(raw_json, source_base_path) if source_base_path else copy.deepcopy(raw_json)
|
||||||
|
requires_migration = bool(campaign_mail_profile_boundary_violations(version.raw_json))
|
||||||
|
if requires_migration and not migrate_legacy_mail_settings:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"This version contains legacy campaign-local SMTP/IMAP settings. Select an authorized Mail "
|
||||||
|
"profile on the Mail settings page and explicitly save the migration; the stored legacy version "
|
||||||
|
"will not be changed automatically."
|
||||||
|
)
|
||||||
|
assert_campaign_uses_mail_profile_reference(runtime_json)
|
||||||
|
if requires_migration and campaign_mail_profile_id(runtime_json) is None:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Migrating legacy campaign mail settings requires an authorized server.mail_profile_id. "
|
||||||
|
"Select a Mail profile before saving."
|
||||||
|
)
|
||||||
|
mail_integration().assert_campaign_mail_policy_allows_json(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
raw_json=runtime_json,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
)
|
||||||
|
return runtime_json
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_version_field_updates(
|
||||||
|
version: CampaignVersion,
|
||||||
|
*,
|
||||||
|
current_flow: str | None,
|
||||||
|
current_step: str | None,
|
||||||
|
workflow_state: str | None,
|
||||||
|
is_complete: bool | None,
|
||||||
|
editor_state: dict[str, Any] | None,
|
||||||
|
source_filename: str | None,
|
||||||
|
source_base_path: str | None,
|
||||||
|
autosave: bool,
|
||||||
|
) -> None:
|
||||||
|
updates = (
|
||||||
|
("current_flow", current_flow),
|
||||||
|
("current_step", current_step),
|
||||||
|
("workflow_state", workflow_state),
|
||||||
|
("is_complete", is_complete),
|
||||||
|
("source_filename", source_filename),
|
||||||
|
("source_base_path", source_base_path),
|
||||||
|
)
|
||||||
|
for field_name, value in updates:
|
||||||
|
if value is not None:
|
||||||
|
setattr(version, field_name, value)
|
||||||
|
if editor_state is not None:
|
||||||
|
version.editor_state = validate_campaign_editor_state(editor_state)
|
||||||
|
if autosave:
|
||||||
|
version.autosaved_at = datetime.now(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _invalidate_version_content(session: Session, *, campaign: Campaign, version: CampaignVersion) -> None:
|
||||||
|
version.validation_summary = None
|
||||||
|
version.build_summary = None
|
||||||
|
clear_execution_snapshot(version)
|
||||||
|
version.locked_at = None
|
||||||
|
version.locked_by_user_id = None
|
||||||
|
if version.workflow_state != CampaignVersionWorkflowState.EDITING.value:
|
||||||
|
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
|
||||||
|
campaign.status = CampaignStatus.DRAFT.value
|
||||||
|
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id).delete(synchronize_session=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _persist_updated_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
commit: bool,
|
||||||
|
) -> None:
|
||||||
|
session.add(version)
|
||||||
|
session.add(campaign)
|
||||||
|
session.flush()
|
||||||
|
if commit:
|
||||||
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
|
||||||
def update_campaign_version(
|
def update_campaign_version(
|
||||||
session: Session,
|
session: Session,
|
||||||
*,
|
*,
|
||||||
@@ -520,10 +726,32 @@ def update_campaign_version(
|
|||||||
source_filename: str | None = None,
|
source_filename: str | None = None,
|
||||||
source_base_path: str | None = None,
|
source_base_path: str | None = None,
|
||||||
autosave: bool = False,
|
autosave: bool = False,
|
||||||
|
migrate_legacy_mail_settings: bool = False,
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
|
_assert_update_paths_safe(raw_json, source_filename=source_filename, source_base_path=source_base_path)
|
||||||
version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
||||||
campaign = _require_campaign(session, campaign_id)
|
campaign = _require_campaign(session, campaign_id)
|
||||||
ensure_current_working_version(campaign, version, action="edit")
|
ensure_current_working_version(campaign, version, action="edit")
|
||||||
|
if (
|
||||||
|
expected_revision is not None
|
||||||
|
and version.edit_revision != int(expected_revision)
|
||||||
|
):
|
||||||
|
raise RevisionConflictError(
|
||||||
|
resource_type="campaign_version",
|
||||||
|
resource_id=version.id,
|
||||||
|
current_revision=version.edit_revision,
|
||||||
|
submitted_base_revision=int(expected_revision),
|
||||||
|
refresh_path=(
|
||||||
|
f"/api/v1/campaigns/{campaign.id}/versions/{version.id}"
|
||||||
|
),
|
||||||
|
current_etag=strong_resource_etag(
|
||||||
|
"campaign_version",
|
||||||
|
version.id,
|
||||||
|
version.edit_revision,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
if is_version_locked(version):
|
if is_version_locked(version):
|
||||||
raise LockedCampaignVersionError(
|
raise LockedCampaignVersionError(
|
||||||
@@ -531,46 +759,70 @@ def update_campaign_version(
|
|||||||
)
|
)
|
||||||
|
|
||||||
if raw_json is not None:
|
if raw_json is not None:
|
||||||
runtime_json = normalize_campaign_paths(raw_json, source_base_path) if source_base_path else copy.deepcopy(raw_json)
|
runtime_json = _updated_runtime_json(
|
||||||
mail_integration().assert_campaign_mail_policy_allows_json(session, tenant_id=tenant_id, raw_json=runtime_json, campaign_id=campaign.id)
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
raw_json=raw_json,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
migrate_legacy_mail_settings=migrate_legacy_mail_settings,
|
||||||
|
)
|
||||||
version.raw_json = runtime_json
|
version.raw_json = runtime_json
|
||||||
version.schema_version = str(runtime_json.get("version", version.schema_version or "1.0"))
|
version.schema_version = str(runtime_json.get("version", version.schema_version or "1.0"))
|
||||||
_apply_campaign_metadata(campaign, runtime_json)
|
_apply_campaign_metadata(campaign, runtime_json)
|
||||||
|
|
||||||
if current_flow is not None:
|
_apply_version_field_updates(
|
||||||
version.current_flow = current_flow
|
version,
|
||||||
if current_step is not None:
|
current_flow=current_flow,
|
||||||
version.current_step = current_step
|
current_step=current_step,
|
||||||
if workflow_state is not None:
|
workflow_state=workflow_state,
|
||||||
version.workflow_state = workflow_state
|
is_complete=is_complete,
|
||||||
if is_complete is not None:
|
editor_state=editor_state,
|
||||||
version.is_complete = is_complete
|
source_filename=source_filename,
|
||||||
if editor_state is not None:
|
source_base_path=source_base_path,
|
||||||
version.editor_state = editor_state
|
autosave=autosave,
|
||||||
if source_filename is not None:
|
)
|
||||||
version.source_filename = source_filename
|
|
||||||
if source_base_path is not None:
|
|
||||||
version.source_base_path = source_base_path
|
|
||||||
if autosave:
|
|
||||||
version.autosaved_at = datetime.now(UTC)
|
|
||||||
|
|
||||||
# Changes invalidate previous build and validation summaries.
|
# Changes invalidate previous build and validation summaries.
|
||||||
if raw_json is not None:
|
if raw_json is not None:
|
||||||
version.validation_summary = None
|
_invalidate_version_content(session, campaign=campaign, version=version)
|
||||||
version.build_summary = None
|
try:
|
||||||
clear_execution_snapshot(version)
|
_persist_updated_version(
|
||||||
version.locked_at = None
|
session,
|
||||||
version.locked_by_user_id = None
|
campaign=campaign,
|
||||||
if version.workflow_state != CampaignVersionWorkflowState.EDITING.value:
|
version=version,
|
||||||
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
|
commit=commit,
|
||||||
campaign.status = CampaignStatus.DRAFT.value
|
)
|
||||||
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id).delete(synchronize_session=False)
|
except StaleDataError as exc:
|
||||||
|
session.rollback()
|
||||||
session.add(version)
|
current_revision = (
|
||||||
session.add(campaign)
|
session.query(CampaignVersion.edit_revision)
|
||||||
session.flush()
|
.filter(CampaignVersion.id == version_id)
|
||||||
_write_campaign_snapshot(version)
|
.scalar()
|
||||||
session.commit()
|
)
|
||||||
|
if current_revision is None:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
f"Campaign version not found: {version_id}"
|
||||||
|
) from exc
|
||||||
|
raise RevisionConflictError(
|
||||||
|
resource_type="campaign_version",
|
||||||
|
resource_id=version_id,
|
||||||
|
current_revision=int(current_revision),
|
||||||
|
submitted_base_revision=int(
|
||||||
|
expected_revision
|
||||||
|
if expected_revision is not None
|
||||||
|
else version.edit_revision
|
||||||
|
),
|
||||||
|
refresh_path=(
|
||||||
|
f"/api/v1/campaigns/{campaign_id}/versions/{version_id}"
|
||||||
|
),
|
||||||
|
current_etag=strong_resource_etag(
|
||||||
|
"campaign_version",
|
||||||
|
version_id,
|
||||||
|
int(current_revision),
|
||||||
|
),
|
||||||
|
) from exc
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
@@ -582,7 +834,9 @@ def update_campaign_review_state(
|
|||||||
version_id: str,
|
version_id: str,
|
||||||
inspection_complete: bool,
|
inspection_complete: bool,
|
||||||
reviewed_message_keys: list[str],
|
reviewed_message_keys: list[str],
|
||||||
|
issue_decisions: list[dict[str, Any]] | None = None,
|
||||||
user_id: str | None,
|
user_id: str | None,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Persist review acknowledgement without mutating the locked campaign data.
|
"""Persist review acknowledgement without mutating the locked campaign data.
|
||||||
|
|
||||||
@@ -603,17 +857,29 @@ def update_campaign_review_state(
|
|||||||
raise LockedCampaignVersionError("Delivery has started; message review state can no longer be changed.")
|
raise LockedCampaignVersionError("Delivery has started; message review state can no longer be changed.")
|
||||||
build_token = _campaign_review_build_token(version)
|
build_token = _campaign_review_build_token(version)
|
||||||
normalized_reviewed = list(dict.fromkeys(str(value) for value in reviewed_message_keys if str(value).strip()))
|
normalized_reviewed = list(dict.fromkeys(str(value) for value in reviewed_message_keys if str(value).strip()))
|
||||||
|
normalized_decisions: list[dict[str, Any]] = []
|
||||||
if inspection_complete:
|
if inspection_complete:
|
||||||
normalized_reviewed = _complete_campaign_review_keys(session, version, normalized_reviewed)
|
normalized_reviewed, normalized_decisions = _complete_campaign_review(
|
||||||
|
session,
|
||||||
|
version,
|
||||||
|
normalized_reviewed,
|
||||||
|
issue_decisions or [],
|
||||||
|
user_id=user_id,
|
||||||
|
build_token=build_token,
|
||||||
|
)
|
||||||
_write_campaign_review_state(
|
_write_campaign_review_state(
|
||||||
version,
|
version,
|
||||||
build_token=build_token,
|
build_token=build_token,
|
||||||
inspection_complete=inspection_complete,
|
inspection_complete=inspection_complete,
|
||||||
reviewed_message_keys=normalized_reviewed,
|
reviewed_message_keys=normalized_reviewed,
|
||||||
|
issue_decisions=normalized_decisions,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
)
|
)
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.commit()
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
@@ -631,11 +897,15 @@ def _campaign_review_build_token(version: CampaignVersion) -> str:
|
|||||||
return build_token
|
return build_token
|
||||||
|
|
||||||
|
|
||||||
def _complete_campaign_review_keys(
|
def _complete_campaign_review(
|
||||||
session: Session,
|
session: Session,
|
||||||
version: CampaignVersion,
|
version: CampaignVersion,
|
||||||
reviewed_message_keys: list[str],
|
reviewed_message_keys: list[str],
|
||||||
) -> list[str]:
|
issue_decisions: list[dict[str, Any]],
|
||||||
|
*,
|
||||||
|
user_id: str | None,
|
||||||
|
build_token: str,
|
||||||
|
) -> tuple[list[str], list[dict[str, Any]]]:
|
||||||
jobs = (
|
jobs = (
|
||||||
session.query(CampaignJob)
|
session.query(CampaignJob)
|
||||||
.filter(CampaignJob.campaign_version_id == version.id)
|
.filter(CampaignJob.campaign_version_id == version.id)
|
||||||
@@ -650,7 +920,123 @@ def _complete_campaign_review_keys(
|
|||||||
raise CampaignPersistenceError(
|
raise CampaignPersistenceError(
|
||||||
"Messages requiring an explicit decision must be opened before review can be completed: " + ", ".join(missing)
|
"Messages requiring an explicit decision must be opened before review can be completed: " + ", ".join(missing)
|
||||||
)
|
)
|
||||||
return list(dict.fromkeys([*reviewed_message_keys, *_bulk_acceptable_review_keys(jobs)]))
|
decisions = _normalize_review_issue_decisions(
|
||||||
|
jobs,
|
||||||
|
issue_decisions,
|
||||||
|
user_id=user_id,
|
||||||
|
build_token=build_token,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
list(
|
||||||
|
dict.fromkeys(
|
||||||
|
[*reviewed_message_keys, *_bulk_acceptable_review_keys(jobs)]
|
||||||
|
)
|
||||||
|
),
|
||||||
|
decisions,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_review_issue_decisions(
|
||||||
|
jobs: list[CampaignJob],
|
||||||
|
requested: list[dict[str, Any]],
|
||||||
|
*,
|
||||||
|
user_id: str | None,
|
||||||
|
build_token: str,
|
||||||
|
decided_at: datetime | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
jobs_by_id = {job.id: job for job in jobs}
|
||||||
|
requested_by_job: dict[str, dict[str, Any]] = {}
|
||||||
|
for item in requested:
|
||||||
|
job_id = str(item.get("job_id") or "").strip()
|
||||||
|
if not job_id or job_id not in jobs_by_id:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"A review decision references a message outside the current build."
|
||||||
|
)
|
||||||
|
if jobs_by_id[job_id].validation_status != "needs_review":
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Review decisions are accepted only for messages requiring review."
|
||||||
|
)
|
||||||
|
if job_id in requested_by_job:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Only one review decision may be recorded per built message."
|
||||||
|
)
|
||||||
|
if str(item.get("decision") or "accept") != "accept":
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Unsupported campaign review decision."
|
||||||
|
)
|
||||||
|
requested_by_job[job_id] = item
|
||||||
|
|
||||||
|
timestamp = (decided_at or datetime.now(UTC)).isoformat()
|
||||||
|
normalized: list[dict[str, Any]] = []
|
||||||
|
for job in jobs:
|
||||||
|
reviewable_issues = [
|
||||||
|
issue
|
||||||
|
for issue in (job.issues_snapshot or [])
|
||||||
|
if isinstance(issue, dict)
|
||||||
|
and str(issue.get("behavior") or "").lower() == "ask"
|
||||||
|
]
|
||||||
|
attachment_overrides = [
|
||||||
|
issue
|
||||||
|
for issue in reviewable_issues
|
||||||
|
if str(issue.get("source") or "").startswith("attachments")
|
||||||
|
]
|
||||||
|
decision = requested_by_job.get(job.id)
|
||||||
|
if attachment_overrides and decision is None:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Attachment exceptions require an explicit reason before review can be completed "
|
||||||
|
f"for message {job.entry_id or job.entry_index}."
|
||||||
|
)
|
||||||
|
if decision is None:
|
||||||
|
continue
|
||||||
|
reason = str(decision.get("reason") or "").strip()
|
||||||
|
if attachment_overrides and not reason:
|
||||||
|
raise CampaignPersistenceError(
|
||||||
|
"Attachment exception decisions require a reason."
|
||||||
|
)
|
||||||
|
evidence_issues = reviewable_issues or [
|
||||||
|
issue
|
||||||
|
for issue in (job.issues_snapshot or [])
|
||||||
|
if isinstance(issue, dict)
|
||||||
|
]
|
||||||
|
issue_payload = [
|
||||||
|
{
|
||||||
|
"code": str(issue.get("code") or ""),
|
||||||
|
"behavior": str(issue.get("behavior") or ""),
|
||||||
|
"source": str(issue.get("source") or ""),
|
||||||
|
"details": issue.get("details")
|
||||||
|
if isinstance(issue.get("details"), dict)
|
||||||
|
else {},
|
||||||
|
}
|
||||||
|
for issue in evidence_issues
|
||||||
|
]
|
||||||
|
fingerprint = hashlib.sha256(
|
||||||
|
json.dumps(
|
||||||
|
issue_payload,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
).encode("utf-8")
|
||||||
|
).hexdigest()
|
||||||
|
normalized.append(
|
||||||
|
{
|
||||||
|
"job_id": job.id,
|
||||||
|
"review_key": str(job.entry_id or job.entry_index),
|
||||||
|
"decision": "accept",
|
||||||
|
"reason": reason or None,
|
||||||
|
"actor_user_id": user_id,
|
||||||
|
"decided_at": timestamp,
|
||||||
|
"build_token": build_token,
|
||||||
|
"message_sha256": job.eml_sha256,
|
||||||
|
"issue_fingerprint": fingerprint,
|
||||||
|
"issue_codes": sorted(
|
||||||
|
{
|
||||||
|
str(issue.get("code") or "")
|
||||||
|
for issue in evidence_issues
|
||||||
|
if issue.get("code")
|
||||||
|
}
|
||||||
|
),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
def _required_review_keys(jobs: list[CampaignJob]) -> set[str]:
|
def _required_review_keys(jobs: list[CampaignJob]) -> set[str]:
|
||||||
@@ -675,6 +1061,7 @@ def _write_campaign_review_state(
|
|||||||
build_token: str,
|
build_token: str,
|
||||||
inspection_complete: bool,
|
inspection_complete: bool,
|
||||||
reviewed_message_keys: list[str],
|
reviewed_message_keys: list[str],
|
||||||
|
issue_decisions: list[dict[str, Any]],
|
||||||
user_id: str | None,
|
user_id: str | None,
|
||||||
) -> None:
|
) -> None:
|
||||||
editor_state = copy.deepcopy(version.editor_state or {})
|
editor_state = copy.deepcopy(version.editor_state or {})
|
||||||
@@ -682,6 +1069,7 @@ def _write_campaign_review_state(
|
|||||||
"build_token": build_token,
|
"build_token": build_token,
|
||||||
"inspection_complete": bool(inspection_complete),
|
"inspection_complete": bool(inspection_complete),
|
||||||
"reviewed_message_keys": reviewed_message_keys,
|
"reviewed_message_keys": reviewed_message_keys,
|
||||||
|
"issue_decisions": issue_decisions,
|
||||||
"updated_at": datetime.now(UTC).isoformat(),
|
"updated_at": datetime.now(UTC).isoformat(),
|
||||||
"updated_by_user_id": user_id,
|
"updated_by_user_id": user_id,
|
||||||
}
|
}
|
||||||
@@ -695,6 +1083,7 @@ def lock_campaign_version_temporarily(
|
|||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
user_id: str | None,
|
user_id: str | None,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Apply a reversible user-requested lock without changing workflow state."""
|
"""Apply a reversible user-requested lock without changing workflow state."""
|
||||||
|
|
||||||
@@ -719,7 +1108,10 @@ def lock_campaign_version_temporarily(
|
|||||||
version.user_locked_at = datetime.now(UTC)
|
version.user_locked_at = datetime.now(UTC)
|
||||||
version.user_locked_by_user_id = user_id
|
version.user_locked_by_user_id = user_id
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.commit()
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
@@ -729,6 +1121,7 @@ def unlock_user_locked_campaign_version(
|
|||||||
tenant_id: str,
|
tenant_id: str,
|
||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Remove a reversible user lock without invalidating campaign data."""
|
"""Remove a reversible user lock without invalidating campaign data."""
|
||||||
|
|
||||||
@@ -752,7 +1145,10 @@ def unlock_user_locked_campaign_version(
|
|||||||
version.user_locked_at = None
|
version.user_locked_at = None
|
||||||
version.user_locked_by_user_id = None
|
version.user_locked_by_user_id = None
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.commit()
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
@@ -763,6 +1159,7 @@ def permanently_lock_campaign_version(
|
|||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
user_id: str | None,
|
user_id: str | None,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Apply an irreversible user lock.
|
"""Apply an irreversible user lock.
|
||||||
|
|
||||||
@@ -790,7 +1187,10 @@ def permanently_lock_campaign_version(
|
|||||||
# Retain published_at as a compatibility marker for existing integrations.
|
# Retain published_at as a compatibility marker for existing integrations.
|
||||||
version.published_at = version.published_at or now
|
version.published_at = version.published_at or now
|
||||||
session.add(version)
|
session.add(version)
|
||||||
session.commit()
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
@@ -801,6 +1201,7 @@ def publish_campaign_version(
|
|||||||
campaign_id: str,
|
campaign_id: str,
|
||||||
version_id: str,
|
version_id: str,
|
||||||
user_id: str | None = None,
|
user_id: str | None = None,
|
||||||
|
commit: bool = True,
|
||||||
) -> CampaignVersion:
|
) -> CampaignVersion:
|
||||||
"""Backwards-compatible alias for the permanent user lock."""
|
"""Backwards-compatible alias for the permanent user lock."""
|
||||||
|
|
||||||
@@ -810,6 +1211,7 @@ def publish_campaign_version(
|
|||||||
campaign_id=campaign_id,
|
campaign_id=campaign_id,
|
||||||
version_id=version_id,
|
version_id=version_id,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
|
commit=commit,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -824,6 +1226,7 @@ def validate_campaign_partial(raw_json: dict[str, Any], *, section: str | None =
|
|||||||
_validate_partial_basics(collector, _dict_value(raw_json, "campaign"))
|
_validate_partial_basics(collector, _dict_value(raw_json, "campaign"))
|
||||||
recipients = _dict_value(raw_json, "recipients")
|
recipients = _dict_value(raw_json, "recipients")
|
||||||
_validate_partial_sender(collector, recipients)
|
_validate_partial_sender(collector, recipients)
|
||||||
|
_validate_partial_mail_profile(collector, raw_json)
|
||||||
_validate_partial_recipients(collector, _dict_value(raw_json, "entries"))
|
_validate_partial_recipients(collector, _dict_value(raw_json, "entries"))
|
||||||
_validate_partial_template(collector, _dict_value(raw_json, "template"))
|
_validate_partial_template(collector, _dict_value(raw_json, "template"))
|
||||||
_validate_partial_attachments(collector, _dict_value(raw_json, "attachments"))
|
_validate_partial_attachments(collector, _dict_value(raw_json, "attachments"))
|
||||||
@@ -849,6 +1252,26 @@ def _validate_partial_sender(collector: _PartialValidationCollector, recipients:
|
|||||||
collector.issue("warning", "sender", "recipients.from.email", "missing_sender_email", "Sender email is not configured yet.")
|
collector.issue("warning", "sender", "recipients.from.email", "missing_sender_email", "Sender email is not configured yet.")
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_partial_mail_profile(collector: _PartialValidationCollector, raw_json: dict[str, Any]) -> None:
|
||||||
|
violations = campaign_mail_profile_boundary_violations(raw_json)
|
||||||
|
if violations:
|
||||||
|
collector.issue(
|
||||||
|
"error",
|
||||||
|
"sender",
|
||||||
|
"server",
|
||||||
|
"campaign_local_mail_transport_forbidden",
|
||||||
|
"Campaign-local SMTP/IMAP settings are not supported. Select or migrate to an authorized Mail-module profile.",
|
||||||
|
)
|
||||||
|
elif campaign_mail_profile_id(raw_json) is None:
|
||||||
|
collector.issue(
|
||||||
|
"warning",
|
||||||
|
"sender",
|
||||||
|
"server.mail_profile_id",
|
||||||
|
"missing_mail_profile",
|
||||||
|
"Select an authorized Mail-module profile before validating or delivering this campaign.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _validate_partial_recipients(collector: _PartialValidationCollector, entries: dict[str, Any]) -> None:
|
def _validate_partial_recipients(collector: _PartialValidationCollector, entries: dict[str, Any]) -> None:
|
||||||
has_inline = bool(entries.get("inline"))
|
has_inline = bool(entries.get("inline"))
|
||||||
has_source = isinstance(entries.get("source"), dict)
|
has_source = isinstance(entries.get("source"), dict)
|
||||||
|
|||||||
109
src/govoplan_campaign/backend/report_privacy_policy.py
Normal file
109
src/govoplan_campaign/backend/report_privacy_policy.py
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_core.tenancy.scope import Tenant
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_SMALL_CELL_THRESHOLD = 5
|
||||||
|
MIN_SMALL_CELL_THRESHOLD = 2
|
||||||
|
MAX_SMALL_CELL_THRESHOLD = 100
|
||||||
|
SMALL_CELL_THRESHOLD_ENV = "GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD"
|
||||||
|
CAMPAIGN_REPORT_POLICY_SETTINGS_KEY = "campaign_report_privacy_policy"
|
||||||
|
SMALL_CELL_THRESHOLD_SETTINGS_KEY = "small_cell_threshold"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignReportPrivacyPolicyError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class CampaignReportPrivacyPolicy:
|
||||||
|
small_cell_threshold: int
|
||||||
|
source: str
|
||||||
|
deployment_small_cell_threshold: int
|
||||||
|
tenant_small_cell_threshold: int | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"small_cell_threshold": self.small_cell_threshold,
|
||||||
|
"source": self.source,
|
||||||
|
"deployment_small_cell_threshold": self.deployment_small_cell_threshold,
|
||||||
|
"tenant_small_cell_threshold": self.tenant_small_cell_threshold,
|
||||||
|
"deployment_setting": SMALL_CELL_THRESHOLD_ENV,
|
||||||
|
"tenant_setting": (
|
||||||
|
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
|
||||||
|
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def effective_campaign_report_privacy_policy(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
environ: Mapping[str, str] | None = None,
|
||||||
|
) -> CampaignReportPrivacyPolicy:
|
||||||
|
env = os.environ if environ is None else environ
|
||||||
|
deployment_raw = env.get(SMALL_CELL_THRESHOLD_ENV)
|
||||||
|
deployment_value = _configured_threshold(
|
||||||
|
deployment_raw,
|
||||||
|
source=SMALL_CELL_THRESHOLD_ENV,
|
||||||
|
default=DEFAULT_SMALL_CELL_THRESHOLD,
|
||||||
|
)
|
||||||
|
tenant = session.get(Tenant, tenant_id)
|
||||||
|
tenant_raw = _tenant_threshold_value(tenant.settings if tenant is not None else None)
|
||||||
|
if tenant_raw is None:
|
||||||
|
return CampaignReportPrivacyPolicy(
|
||||||
|
small_cell_threshold=deployment_value,
|
||||||
|
source="deployment" if deployment_raw not in (None, "") else "deployment_default",
|
||||||
|
deployment_small_cell_threshold=deployment_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
tenant_value = _configured_threshold(
|
||||||
|
tenant_raw,
|
||||||
|
source=(
|
||||||
|
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
|
||||||
|
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
effective_value = max(deployment_value, tenant_value)
|
||||||
|
return CampaignReportPrivacyPolicy(
|
||||||
|
small_cell_threshold=effective_value,
|
||||||
|
source="tenant" if tenant_value >= deployment_value else "deployment_floor",
|
||||||
|
deployment_small_cell_threshold=deployment_value,
|
||||||
|
tenant_small_cell_threshold=tenant_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _tenant_threshold_value(settings: Mapping[str, Any] | None) -> object | None:
|
||||||
|
if not isinstance(settings, Mapping):
|
||||||
|
return None
|
||||||
|
policy = settings.get(CAMPAIGN_REPORT_POLICY_SETTINGS_KEY)
|
||||||
|
if not isinstance(policy, Mapping):
|
||||||
|
return None
|
||||||
|
return policy.get(SMALL_CELL_THRESHOLD_SETTINGS_KEY)
|
||||||
|
|
||||||
|
|
||||||
|
def _configured_threshold(value: object, *, source: str, default: int | None = None) -> int:
|
||||||
|
if value is None or (isinstance(value, str) and not value.strip()):
|
||||||
|
if default is not None:
|
||||||
|
return default
|
||||||
|
raise CampaignReportPrivacyPolicyError(f"{source} must be configured as an integer")
|
||||||
|
if isinstance(value, bool):
|
||||||
|
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer, not a boolean")
|
||||||
|
try:
|
||||||
|
parsed = int(value)
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer") from exc
|
||||||
|
if str(parsed) != str(value).strip() and not isinstance(value, int):
|
||||||
|
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer")
|
||||||
|
if parsed < MIN_SMALL_CELL_THRESHOLD or parsed > MAX_SMALL_CELL_THRESHOLD:
|
||||||
|
raise CampaignReportPrivacyPolicyError(
|
||||||
|
f"{source} must be between {MIN_SMALL_CELL_THRESHOLD} and {MAX_SMALL_CELL_THRESHOLD}"
|
||||||
|
)
|
||||||
|
return parsed
|
||||||
544
src/govoplan_campaign/backend/reports/aggregate.py
Normal file
544
src/govoplan_campaign/backend/reports/aggregate.py
Normal file
@@ -0,0 +1,544 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections import Counter
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict
|
||||||
|
from sqlalchemy import case, func, or_
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
|
||||||
|
from govoplan_campaign.backend.report_privacy_policy import (
|
||||||
|
CampaignReportPrivacyPolicy,
|
||||||
|
effective_campaign_report_privacy_policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateCampaignReportError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateReportCampaign(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
status: str
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateReportCampaignListItem(AggregateReportCampaign):
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateReportCampaignList(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaigns: list[AggregateReportCampaignListItem]
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateCount(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
value: int | None
|
||||||
|
suppressed: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class AggregatePopulation(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
denominator: AggregateCount
|
||||||
|
denominator_definition: str
|
||||||
|
inactive_source_entries: AggregateCount
|
||||||
|
excluded_or_blocked_jobs: AggregateCount
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateOutcomeCounts(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
smtp_accepted: AggregateCount
|
||||||
|
postbox_accepted: AggregateCount
|
||||||
|
delivered: AggregateCount
|
||||||
|
partially_accepted: AggregateCount
|
||||||
|
failed: AggregateCount
|
||||||
|
outcome_unknown: AggregateCount
|
||||||
|
queued_or_active: AggregateCount
|
||||||
|
cancelled: AggregateCount
|
||||||
|
excluded: AggregateCount
|
||||||
|
not_attempted: AggregateCount
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateTimeRange(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
first_activity_at: datetime | None
|
||||||
|
last_activity_at: datetime | None
|
||||||
|
suppressed: bool
|
||||||
|
|
||||||
|
|
||||||
|
class AggregatePrivacy(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
small_cell_threshold: int
|
||||||
|
suppression_applied: bool
|
||||||
|
rule: str
|
||||||
|
|
||||||
|
|
||||||
|
class AggregateCampaignReport(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
generated_at: datetime
|
||||||
|
campaign: AggregateReportCampaign
|
||||||
|
version_number: int | None
|
||||||
|
completion_state: Literal[
|
||||||
|
"not_started",
|
||||||
|
"in_progress",
|
||||||
|
"completed",
|
||||||
|
"partially_completed",
|
||||||
|
"incomplete",
|
||||||
|
"outcome_unknown",
|
||||||
|
"suppressed",
|
||||||
|
]
|
||||||
|
population: AggregatePopulation
|
||||||
|
outcomes: AggregateOutcomeCounts
|
||||||
|
time_range: AggregateTimeRange
|
||||||
|
privacy: AggregatePrivacy
|
||||||
|
|
||||||
|
|
||||||
|
_OUTCOME_KEYS = (
|
||||||
|
"smtp_accepted",
|
||||||
|
"postbox_accepted",
|
||||||
|
"delivered",
|
||||||
|
"partially_accepted",
|
||||||
|
"failed",
|
||||||
|
"outcome_unknown",
|
||||||
|
"queued_or_active",
|
||||||
|
"cancelled",
|
||||||
|
"excluded",
|
||||||
|
"not_attempted",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def generate_aggregate_campaign_report(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
) -> AggregateCampaignReport:
|
||||||
|
"""Build the deliberately small, recipient-free Campaign report projection."""
|
||||||
|
|
||||||
|
campaign = _get_campaign(session, tenant_id=tenant_id, campaign_id=campaign_id)
|
||||||
|
version = _selected_version(session, campaign, version_id)
|
||||||
|
facts = _query_aggregate_facts(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version=version,
|
||||||
|
)
|
||||||
|
policy = effective_campaign_report_privacy_policy(session, tenant_id=tenant_id)
|
||||||
|
return _build_aggregate_campaign_report(
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
facts=facts,
|
||||||
|
policy=policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign(session: Session, *, tenant_id: str, campaign_id: str) -> Campaign:
|
||||||
|
campaign = (
|
||||||
|
session.query(Campaign)
|
||||||
|
.filter(Campaign.tenant_id == tenant_id, Campaign.id == campaign_id)
|
||||||
|
.one_or_none()
|
||||||
|
)
|
||||||
|
if campaign is None:
|
||||||
|
raise AggregateCampaignReportError("Campaign not found")
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _selected_version(
|
||||||
|
session: Session,
|
||||||
|
campaign: Campaign,
|
||||||
|
version_id: str | None,
|
||||||
|
) -> CampaignVersion | None:
|
||||||
|
selected_id = version_id or campaign.current_version_id
|
||||||
|
if not selected_id:
|
||||||
|
return None
|
||||||
|
version = session.get(CampaignVersion, selected_id)
|
||||||
|
if version is None or version.campaign_id != campaign.id:
|
||||||
|
raise AggregateCampaignReportError("Campaign version not found")
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _query_aggregate_facts(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version: CampaignVersion | None,
|
||||||
|
) -> _AggregateFacts:
|
||||||
|
if version is None:
|
||||||
|
return _AggregateFacts.empty()
|
||||||
|
|
||||||
|
smtp_accepted = CampaignJob.send_status.in_(
|
||||||
|
{"smtp_accepted", "sent"}
|
||||||
|
)
|
||||||
|
accepted = CampaignJob.send_status.in_(
|
||||||
|
{
|
||||||
|
"smtp_accepted",
|
||||||
|
"postbox_accepted",
|
||||||
|
"delivered",
|
||||||
|
"partially_accepted",
|
||||||
|
"sent",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
failed = CampaignJob.send_status.in_({"failed_temporary", "failed_permanent"})
|
||||||
|
unknown = CampaignJob.send_status == "outcome_unknown"
|
||||||
|
active = CampaignJob.send_status.in_({"queued", "claimed", "sending"})
|
||||||
|
cancelled = CampaignJob.send_status == "cancelled"
|
||||||
|
excluded = CampaignJob.send_status == "skipped"
|
||||||
|
excluded_or_blocked = or_(
|
||||||
|
CampaignJob.validation_status.in_({"blocked", "excluded", "inactive"}),
|
||||||
|
CampaignJob.build_status != "built",
|
||||||
|
)
|
||||||
|
row = (
|
||||||
|
session.query(
|
||||||
|
func.count(CampaignJob.id).label("denominator"),
|
||||||
|
func.sum(case((smtp_accepted, 1), else_=0)).label(
|
||||||
|
"smtp_accepted"
|
||||||
|
),
|
||||||
|
func.sum(
|
||||||
|
case(
|
||||||
|
(
|
||||||
|
CampaignJob.send_status == "postbox_accepted",
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
else_=0,
|
||||||
|
)
|
||||||
|
).label("postbox_accepted"),
|
||||||
|
func.sum(
|
||||||
|
case(
|
||||||
|
(CampaignJob.send_status == "delivered", 1),
|
||||||
|
else_=0,
|
||||||
|
)
|
||||||
|
).label("delivered"),
|
||||||
|
func.sum(
|
||||||
|
case(
|
||||||
|
(
|
||||||
|
CampaignJob.send_status == "partially_accepted",
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
else_=0,
|
||||||
|
)
|
||||||
|
).label("partially_accepted"),
|
||||||
|
func.sum(case((failed, 1), else_=0)).label("failed"),
|
||||||
|
func.sum(case((unknown, 1), else_=0)).label("outcome_unknown"),
|
||||||
|
func.sum(case((active, 1), else_=0)).label("queued_or_active"),
|
||||||
|
func.sum(case((cancelled, 1), else_=0)).label("cancelled"),
|
||||||
|
func.sum(case((excluded, 1), else_=0)).label("excluded"),
|
||||||
|
func.sum(
|
||||||
|
case((or_(accepted, failed, unknown, active, cancelled, excluded), 0), else_=1)
|
||||||
|
).label("not_attempted"),
|
||||||
|
func.sum(case((excluded_or_blocked, 1), else_=0)).label("excluded_or_blocked"),
|
||||||
|
func.min(CampaignJob.queued_at).label("queued_min"),
|
||||||
|
func.max(CampaignJob.queued_at).label("queued_max"),
|
||||||
|
func.min(CampaignJob.smtp_started_at).label("smtp_min"),
|
||||||
|
func.max(CampaignJob.smtp_started_at).label("smtp_max"),
|
||||||
|
func.min(CampaignJob.sent_at).label("sent_min"),
|
||||||
|
func.max(CampaignJob.sent_at).label("sent_max"),
|
||||||
|
func.min(CampaignJob.outcome_unknown_at).label("unknown_min"),
|
||||||
|
func.max(CampaignJob.outcome_unknown_at).label("unknown_max"),
|
||||||
|
)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.campaign_version_id == version.id,
|
||||||
|
)
|
||||||
|
.one()
|
||||||
|
)
|
||||||
|
values = row._mapping
|
||||||
|
activity = [
|
||||||
|
values[key]
|
||||||
|
for key in (
|
||||||
|
"queued_min",
|
||||||
|
"queued_max",
|
||||||
|
"smtp_min",
|
||||||
|
"smtp_max",
|
||||||
|
"sent_min",
|
||||||
|
"sent_max",
|
||||||
|
"unknown_min",
|
||||||
|
"unknown_max",
|
||||||
|
)
|
||||||
|
if values[key] is not None
|
||||||
|
]
|
||||||
|
return _AggregateFacts(
|
||||||
|
outcomes={key: int(values[key] or 0) for key in _OUTCOME_KEYS},
|
||||||
|
denominator=int(values["denominator"] or 0),
|
||||||
|
excluded_or_blocked=int(values["excluded_or_blocked"] or 0),
|
||||||
|
first_activity_at=min(activity) if activity else None,
|
||||||
|
last_activity_at=max(activity) if activity else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_aggregate_campaign_report(
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion | None,
|
||||||
|
jobs: list[CampaignJob],
|
||||||
|
policy: CampaignReportPrivacyPolicy,
|
||||||
|
generated_at: datetime | None = None,
|
||||||
|
) -> AggregateCampaignReport:
|
||||||
|
return _build_aggregate_campaign_report(
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
facts=_facts_from_jobs(jobs),
|
||||||
|
policy=policy,
|
||||||
|
generated_at=generated_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_aggregate_campaign_report(
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion | None,
|
||||||
|
facts: _AggregateFacts,
|
||||||
|
policy: CampaignReportPrivacyPolicy,
|
||||||
|
generated_at: datetime | None = None,
|
||||||
|
) -> AggregateCampaignReport:
|
||||||
|
outcome_values = facts.outcomes
|
||||||
|
outcomes, denominator = _suppress_partition(
|
||||||
|
outcome_values,
|
||||||
|
threshold=policy.small_cell_threshold,
|
||||||
|
)
|
||||||
|
outcome_suppression_applied = denominator.suppressed or any(
|
||||||
|
item.suppressed for item in outcomes.values()
|
||||||
|
)
|
||||||
|
inactive_entries = _inactive_entry_count(version)
|
||||||
|
standalone_counts = {
|
||||||
|
"inactive_source_entries": _suppress_standalone_count(
|
||||||
|
inactive_entries,
|
||||||
|
threshold=policy.small_cell_threshold,
|
||||||
|
),
|
||||||
|
# This population count overlaps the outcome partition, so exposing it
|
||||||
|
# can make a suppressed outcome recoverable through subtraction.
|
||||||
|
"excluded_or_blocked_jobs": (
|
||||||
|
AggregateCount(value=None, suppressed=True)
|
||||||
|
if outcome_suppression_applied
|
||||||
|
else _suppress_standalone_count(
|
||||||
|
facts.excluded_or_blocked,
|
||||||
|
threshold=policy.small_cell_threshold,
|
||||||
|
)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
suppression_applied = denominator.suppressed or any(
|
||||||
|
item.suppressed for item in (*outcomes.values(), *standalone_counts.values())
|
||||||
|
)
|
||||||
|
first_activity = facts.first_activity_at
|
||||||
|
last_activity = facts.last_activity_at
|
||||||
|
suppress_time_range = suppression_applied or (
|
||||||
|
0 < facts.denominator < policy.small_cell_threshold
|
||||||
|
)
|
||||||
|
|
||||||
|
return AggregateCampaignReport(
|
||||||
|
generated_at=generated_at or datetime.now(timezone.utc),
|
||||||
|
campaign=AggregateReportCampaign(
|
||||||
|
id=campaign.id,
|
||||||
|
name=campaign.name,
|
||||||
|
status=campaign.status,
|
||||||
|
),
|
||||||
|
version_number=version.version_number if version else None,
|
||||||
|
completion_state=(
|
||||||
|
"suppressed"
|
||||||
|
if denominator.suppressed
|
||||||
|
else _completion_state(outcome_values, facts.denominator)
|
||||||
|
),
|
||||||
|
population=AggregatePopulation(
|
||||||
|
denominator=denominator,
|
||||||
|
denominator_definition=(
|
||||||
|
"All persisted recipient delivery jobs for the selected campaign version, "
|
||||||
|
"including excluded or blocked jobs. Inactive source entries without a job "
|
||||||
|
"record are excluded and reported separately."
|
||||||
|
),
|
||||||
|
inactive_source_entries=standalone_counts["inactive_source_entries"],
|
||||||
|
excluded_or_blocked_jobs=standalone_counts["excluded_or_blocked_jobs"],
|
||||||
|
),
|
||||||
|
outcomes=AggregateOutcomeCounts(**outcomes),
|
||||||
|
time_range=AggregateTimeRange(
|
||||||
|
first_activity_at=None if suppress_time_range else first_activity,
|
||||||
|
last_activity_at=None if suppress_time_range else last_activity,
|
||||||
|
suppressed=suppress_time_range and first_activity is not None,
|
||||||
|
),
|
||||||
|
privacy=AggregatePrivacy(
|
||||||
|
small_cell_threshold=policy.small_cell_threshold,
|
||||||
|
suppression_applied=suppression_applied,
|
||||||
|
rule=(
|
||||||
|
"Positive counts below the threshold are hidden. At least one additional "
|
||||||
|
"count or the denominator is hidden when needed to prevent subtraction. "
|
||||||
|
"Overlapping population counts are hidden whenever outcome suppression applies."
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_report_campaign_item(campaign: Campaign) -> AggregateReportCampaignListItem:
|
||||||
|
return AggregateReportCampaignListItem(
|
||||||
|
id=campaign.id,
|
||||||
|
name=campaign.name,
|
||||||
|
status=campaign.status,
|
||||||
|
updated_at=campaign.updated_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _outcome_counts(jobs: list[CampaignJob]) -> dict[str, int]:
|
||||||
|
counts: Counter[str] = Counter()
|
||||||
|
for job in jobs:
|
||||||
|
status = job.send_status
|
||||||
|
if status in {"smtp_accepted", "sent"}:
|
||||||
|
counts["smtp_accepted"] += 1
|
||||||
|
elif status == "postbox_accepted":
|
||||||
|
counts["postbox_accepted"] += 1
|
||||||
|
elif status == "delivered":
|
||||||
|
counts["delivered"] += 1
|
||||||
|
elif status == "partially_accepted":
|
||||||
|
counts["partially_accepted"] += 1
|
||||||
|
elif status in {"failed_temporary", "failed_permanent"}:
|
||||||
|
counts["failed"] += 1
|
||||||
|
elif status == "outcome_unknown":
|
||||||
|
counts["outcome_unknown"] += 1
|
||||||
|
elif status in {"queued", "claimed", "sending"}:
|
||||||
|
counts["queued_or_active"] += 1
|
||||||
|
elif status == "cancelled":
|
||||||
|
counts["cancelled"] += 1
|
||||||
|
elif status == "skipped":
|
||||||
|
counts["excluded"] += 1
|
||||||
|
else:
|
||||||
|
counts["not_attempted"] += 1
|
||||||
|
return {key: counts[key] for key in _OUTCOME_KEYS}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class _AggregateFacts:
|
||||||
|
outcomes: dict[str, int]
|
||||||
|
denominator: int
|
||||||
|
excluded_or_blocked: int
|
||||||
|
first_activity_at: datetime | None
|
||||||
|
last_activity_at: datetime | None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def empty(cls) -> _AggregateFacts:
|
||||||
|
return cls(
|
||||||
|
outcomes={key: 0 for key in _OUTCOME_KEYS},
|
||||||
|
denominator=0,
|
||||||
|
excluded_or_blocked=0,
|
||||||
|
first_activity_at=None,
|
||||||
|
last_activity_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _facts_from_jobs(jobs: list[CampaignJob]) -> _AggregateFacts:
|
||||||
|
first_activity, last_activity = _activity_range(jobs)
|
||||||
|
return _AggregateFacts(
|
||||||
|
outcomes=_outcome_counts(jobs),
|
||||||
|
denominator=len(jobs),
|
||||||
|
excluded_or_blocked=sum(
|
||||||
|
1
|
||||||
|
for job in jobs
|
||||||
|
if job.validation_status in {"blocked", "excluded", "inactive"}
|
||||||
|
or job.build_status != "built"
|
||||||
|
),
|
||||||
|
first_activity_at=first_activity,
|
||||||
|
last_activity_at=last_activity,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _suppress_partition(
|
||||||
|
counts: dict[str, int],
|
||||||
|
*,
|
||||||
|
threshold: int,
|
||||||
|
) -> tuple[dict[str, AggregateCount], AggregateCount]:
|
||||||
|
total = sum(counts.values())
|
||||||
|
suppressed = {key for key, value in counts.items() if 0 < value < threshold}
|
||||||
|
suppress_denominator = False
|
||||||
|
if suppressed:
|
||||||
|
companions = [
|
||||||
|
(value, key)
|
||||||
|
for key, value in counts.items()
|
||||||
|
if key not in suppressed and value > 0
|
||||||
|
]
|
||||||
|
if companions:
|
||||||
|
suppressed.add(max(companions)[1])
|
||||||
|
else:
|
||||||
|
suppress_denominator = True
|
||||||
|
denominator = AggregateCount(
|
||||||
|
value=None if suppress_denominator else total,
|
||||||
|
suppressed=suppress_denominator,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
{
|
||||||
|
key: AggregateCount(
|
||||||
|
value=None if key in suppressed else value,
|
||||||
|
suppressed=key in suppressed,
|
||||||
|
)
|
||||||
|
for key, value in counts.items()
|
||||||
|
},
|
||||||
|
denominator,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _suppress_standalone_count(value: int, *, threshold: int) -> AggregateCount:
|
||||||
|
if 0 < value < threshold:
|
||||||
|
return AggregateCount(value=None, suppressed=True)
|
||||||
|
return AggregateCount(value=value, suppressed=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _completion_state(
|
||||||
|
counts: dict[str, int],
|
||||||
|
total: int,
|
||||||
|
) -> Literal[
|
||||||
|
"not_started",
|
||||||
|
"in_progress",
|
||||||
|
"completed",
|
||||||
|
"partially_completed",
|
||||||
|
"incomplete",
|
||||||
|
"outcome_unknown",
|
||||||
|
]:
|
||||||
|
if total == 0 or counts["not_attempted"] + counts["excluded"] == total:
|
||||||
|
return "not_started"
|
||||||
|
if counts["outcome_unknown"]:
|
||||||
|
return "outcome_unknown"
|
||||||
|
if counts["queued_or_active"]:
|
||||||
|
return "in_progress"
|
||||||
|
fully_accepted = (
|
||||||
|
counts["smtp_accepted"]
|
||||||
|
+ counts["postbox_accepted"]
|
||||||
|
+ counts["delivered"]
|
||||||
|
)
|
||||||
|
partially_accepted = counts["partially_accepted"]
|
||||||
|
if fully_accepted == total:
|
||||||
|
return "completed"
|
||||||
|
if fully_accepted or partially_accepted:
|
||||||
|
return "partially_completed"
|
||||||
|
return "incomplete"
|
||||||
|
|
||||||
|
|
||||||
|
def _activity_range(jobs: list[CampaignJob]) -> tuple[datetime | None, datetime | None]:
|
||||||
|
activity = [
|
||||||
|
value
|
||||||
|
for job in jobs
|
||||||
|
for value in (
|
||||||
|
job.queued_at,
|
||||||
|
job.smtp_started_at,
|
||||||
|
job.sent_at,
|
||||||
|
job.outcome_unknown_at,
|
||||||
|
)
|
||||||
|
if value is not None
|
||||||
|
]
|
||||||
|
if not activity:
|
||||||
|
return None, None
|
||||||
|
return min(activity), max(activity)
|
||||||
|
|
||||||
|
|
||||||
|
def _inactive_entry_count(version: CampaignVersion | None) -> int:
|
||||||
|
build_summary = version.build_summary if version and isinstance(version.build_summary, dict) else {}
|
||||||
|
return int(build_summary.get("inactive_count") or build_summary.get("inactive_entries_count") or 0)
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -9,11 +9,16 @@ from typing import Any
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
|
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
|
||||||
from govoplan_campaign.backend.campaign.loader import load_campaign_config
|
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
||||||
from govoplan_campaign.backend.campaign.models import CampaignConfig, SmtpConfig
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_id
|
||||||
from govoplan_campaign.backend.persistence.campaigns import _write_campaign_snapshot
|
from govoplan_campaign.backend.persistence.campaigns import load_version_config
|
||||||
from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
||||||
from govoplan_campaign.backend.integrations import SmtpConfigurationError, mail_integration
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
MailDeliveryCommandError,
|
||||||
|
SmtpConfigurationError,
|
||||||
|
mail_integration,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, ensure_execution_snapshot
|
||||||
|
|
||||||
|
|
||||||
class CampaignReportEmailError(RuntimeError):
|
class CampaignReportEmailError(RuntimeError):
|
||||||
@@ -30,9 +35,10 @@ class CampaignReportEmailResult:
|
|||||||
sent: bool
|
sent: bool
|
||||||
attached_jobs_csv: bool
|
attached_jobs_csv: bool
|
||||||
attached_report_json: bool
|
attached_report_json: bool
|
||||||
smtp_host: str | None = None
|
|
||||||
smtp_port: int | None = None
|
|
||||||
accepted_count: int | None = None
|
accepted_count: int | None = None
|
||||||
|
command_id: str | None = None
|
||||||
|
delivery_status: str | None = None
|
||||||
|
duplicate: bool = False
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
def as_dict(self) -> dict[str, Any]:
|
||||||
return {
|
return {
|
||||||
@@ -44,9 +50,23 @@ class CampaignReportEmailResult:
|
|||||||
"sent": self.sent,
|
"sent": self.sent,
|
||||||
"attached_jobs_csv": self.attached_jobs_csv,
|
"attached_jobs_csv": self.attached_jobs_csv,
|
||||||
"attached_report_json": self.attached_report_json,
|
"attached_report_json": self.attached_report_json,
|
||||||
"smtp_host": self.smtp_host,
|
|
||||||
"smtp_port": self.smtp_port,
|
|
||||||
"accepted_count": self.accepted_count,
|
"accepted_count": self.accepted_count,
|
||||||
|
"command_id": self.command_id,
|
||||||
|
"delivery_status": self.delivery_status,
|
||||||
|
"duplicate": self.duplicate,
|
||||||
|
}
|
||||||
|
|
||||||
|
def audit_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"campaign_id": self.campaign_id,
|
||||||
|
"version_id": self.version_id,
|
||||||
|
"recipient_count": len(self.to),
|
||||||
|
"dry_run": self.dry_run,
|
||||||
|
"attached_jobs_csv": self.attached_jobs_csv,
|
||||||
|
"attached_report_json": self.attached_report_json,
|
||||||
|
"command_id": self.command_id,
|
||||||
|
"delivery_status": self.delivery_status,
|
||||||
|
"duplicate": self.duplicate,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -62,18 +82,15 @@ def _selected_version(
|
|||||||
return version
|
return version
|
||||||
|
|
||||||
|
|
||||||
def _load_config(version: CampaignVersion) -> CampaignConfig:
|
def _load_config(session: Session, version: CampaignVersion) -> CampaignConfig:
|
||||||
snapshot_path = _write_campaign_snapshot(version)
|
_campaign, _version, config = load_version_config(session, version.id)
|
||||||
return load_campaign_config(snapshot_path)
|
return config
|
||||||
|
|
||||||
|
|
||||||
def _effective_from(config: CampaignConfig) -> tuple[str, str | None]:
|
def _effective_from(config: CampaignConfig) -> tuple[str, str | None]:
|
||||||
if config.recipients.from_:
|
if config.recipients.from_:
|
||||||
return config.recipients.from_[0].email, config.recipients.from_[0].name
|
return config.recipients.from_[0].email, config.recipients.from_[0].name
|
||||||
smtp_config = config.server.runtime_smtp_config()
|
raise SmtpConfigurationError("Report email requires a Campaign-owned recipients.from address")
|
||||||
if smtp_config and smtp_config.username and "@" in smtp_config.username:
|
|
||||||
return smtp_config.username, None
|
|
||||||
raise SmtpConfigurationError("Report email requires a recipients.from address or an SMTP username that is an email address")
|
|
||||||
|
|
||||||
|
|
||||||
def _text_summary(report: dict[str, Any]) -> str:
|
def _text_summary(report: dict[str, Any]) -> str:
|
||||||
@@ -94,8 +111,10 @@ def _text_summary(report: dict[str, Any]) -> str:
|
|||||||
f"- Needs attention: {cards['needs_attention']}",
|
f"- Needs attention: {cards['needs_attention']}",
|
||||||
f"- Sent: {cards['sent']}",
|
f"- Sent: {cards['sent']}",
|
||||||
f"- Failed: {cards['failed']}",
|
f"- Failed: {cards['failed']}",
|
||||||
|
f"- SMTP skipped (excluded): {cards.get('skipped', status.get('send', {}).get('skipped', 0))}",
|
||||||
f"- IMAP appended: {cards['imap_appended']}",
|
f"- IMAP appended: {cards['imap_appended']}",
|
||||||
f"- IMAP failed: {cards['imap_failed']}",
|
f"- IMAP failed: {cards['imap_failed']}",
|
||||||
|
f"- IMAP skipped: {cards.get('imap_skipped', status.get('imap', {}).get('skipped', 0))}",
|
||||||
"",
|
"",
|
||||||
f"Build status: {status.get('build', {})}",
|
f"Build status: {status.get('build', {})}",
|
||||||
f"Validation status: {status.get('validation', {})}",
|
f"Validation status: {status.get('validation', {})}",
|
||||||
@@ -150,9 +169,11 @@ def send_campaign_report_email(
|
|||||||
version_id: str | None = None,
|
version_id: str | None = None,
|
||||||
to: list[str],
|
to: list[str],
|
||||||
include_jobs: bool = False,
|
include_jobs: bool = False,
|
||||||
attach_jobs_csv: bool = True,
|
attach_jobs_csv: bool = False,
|
||||||
attach_report_json: bool = False,
|
attach_report_json: bool = False,
|
||||||
dry_run: bool = False,
|
dry_run: bool = False,
|
||||||
|
idempotency_key: str | None = None,
|
||||||
|
created_by_user_id: str | None = None,
|
||||||
) -> CampaignReportEmailResult:
|
) -> CampaignReportEmailResult:
|
||||||
campaign = session.get(Campaign, campaign_id)
|
campaign = session.get(Campaign, campaign_id)
|
||||||
if not campaign or campaign.tenant_id != tenant_id:
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
@@ -161,17 +182,41 @@ def send_campaign_report_email(
|
|||||||
raise CampaignReportEmailError("At least one report recipient is required")
|
raise CampaignReportEmailError("At least one report recipient is required")
|
||||||
|
|
||||||
version = _selected_version(session, campaign, version_id)
|
version = _selected_version(session, campaign, version_id)
|
||||||
config = _load_config(version)
|
config = _load_config(session, version)
|
||||||
smtp_config: SmtpConfig | None = config.server.runtime_smtp_config()
|
if not config.server.profile_capabilities.smtp_available:
|
||||||
if smtp_config is None:
|
|
||||||
raise SmtpConfigurationError("Campaign has no SMTP configuration")
|
raise SmtpConfigurationError("Campaign has no SMTP configuration")
|
||||||
|
profile_id = campaign_mail_profile_id(version.raw_json if isinstance(version.raw_json, dict) else {})
|
||||||
|
if not profile_id:
|
||||||
|
raise SmtpConfigurationError("Campaign has no Mail profile reference")
|
||||||
|
if not isinstance(version.execution_snapshot, dict):
|
||||||
|
raise CampaignReportEmailError(
|
||||||
|
"Report email requires a validated and built campaign version with stored Mail-profile evidence."
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
snapshot = ensure_execution_snapshot(session, version)
|
||||||
|
except ExecutionSnapshotError as exc:
|
||||||
|
raise CampaignReportEmailError(
|
||||||
|
"Report email requires a validated and built campaign version with current Mail-profile evidence."
|
||||||
|
) from exc
|
||||||
|
if not snapshot.smtp_transport_revision:
|
||||||
|
raise CampaignReportEmailError("Campaign build evidence has no SMTP transport revision")
|
||||||
|
mail = mail_integration()
|
||||||
|
if not dry_run:
|
||||||
|
if not mail.durable_delivery_available:
|
||||||
|
raise CampaignReportEmailError(
|
||||||
|
"Report email delivery requires the durable, idempotent Mail-owned outbox."
|
||||||
|
)
|
||||||
|
if not str(idempotency_key or "").strip():
|
||||||
|
raise CampaignReportEmailError(
|
||||||
|
"Live report email delivery requires an idempotency key"
|
||||||
|
)
|
||||||
report = generate_campaign_report(
|
report = generate_campaign_report(
|
||||||
session,
|
session,
|
||||||
tenant_id=tenant_id,
|
tenant_id=tenant_id,
|
||||||
campaign_id=campaign_id,
|
campaign_id=campaign_id,
|
||||||
version_id=version.id,
|
version_id=version.id,
|
||||||
include_jobs=include_jobs,
|
include_jobs=include_jobs,
|
||||||
|
include_recent_failures=include_jobs,
|
||||||
)
|
)
|
||||||
jobs_csv = (
|
jobs_csv = (
|
||||||
generate_jobs_csv(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version.id)
|
generate_jobs_csv(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version.id)
|
||||||
@@ -187,38 +232,55 @@ def send_campaign_report_email(
|
|||||||
jobs_csv=jobs_csv,
|
jobs_csv=jobs_csv,
|
||||||
report_json=report_json,
|
report_json=report_json,
|
||||||
)
|
)
|
||||||
envelope_from, _ = _effective_from(config)
|
if not dry_run:
|
||||||
|
clean_idempotency_key = str(idempotency_key or "").strip()
|
||||||
if dry_run:
|
from_email, _from_name = _effective_from(config)
|
||||||
|
if not snapshot.mail_profile_id:
|
||||||
|
raise CampaignReportEmailError(
|
||||||
|
"Campaign build evidence has no Mail profile reference"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
command = mail.submit_delivery_command(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
command_type="campaign_report",
|
||||||
|
source_module="campaigns",
|
||||||
|
source_resource_type="campaign",
|
||||||
|
source_resource_id=campaign.id,
|
||||||
|
source_version_id=version.id,
|
||||||
|
idempotency_key=clean_idempotency_key,
|
||||||
|
profile_id=snapshot.mail_profile_id,
|
||||||
|
message_bytes=message.as_bytes(),
|
||||||
|
envelope_from=from_email,
|
||||||
|
envelope_recipients=to,
|
||||||
|
from_header=str(message["From"]),
|
||||||
|
expected_smtp_transport_revision=snapshot.smtp_transport_revision,
|
||||||
|
smtp_server_id=snapshot.smtp_server_id,
|
||||||
|
smtp_credential_id=snapshot.smtp_credential_id,
|
||||||
|
created_by_user_id=created_by_user_id,
|
||||||
|
)
|
||||||
|
except MailDeliveryCommandError as exc:
|
||||||
|
raise CampaignReportEmailError(str(exc)) from exc
|
||||||
return CampaignReportEmailResult(
|
return CampaignReportEmailResult(
|
||||||
campaign_id=campaign.id,
|
campaign_id=campaign.id,
|
||||||
version_id=version.id,
|
version_id=version.id,
|
||||||
to=to,
|
to=to,
|
||||||
subject=str(message["Subject"]),
|
subject=str(message["Subject"]),
|
||||||
dry_run=True,
|
dry_run=False,
|
||||||
sent=False,
|
sent=False,
|
||||||
attached_jobs_csv=jobs_csv is not None,
|
attached_jobs_csv=jobs_csv is not None,
|
||||||
attached_report_json=report_json is not None,
|
attached_report_json=report_json is not None,
|
||||||
smtp_host=smtp_config.host,
|
command_id=str(command["id"]),
|
||||||
smtp_port=smtp_config.port,
|
delivery_status=str(command["status"]),
|
||||||
|
duplicate=bool(command.get("duplicate")),
|
||||||
)
|
)
|
||||||
|
|
||||||
result = mail_integration().send_email_message(
|
|
||||||
message,
|
|
||||||
smtp_config=smtp_config,
|
|
||||||
envelope_from=envelope_from,
|
|
||||||
envelope_recipients=to,
|
|
||||||
)
|
|
||||||
return CampaignReportEmailResult(
|
return CampaignReportEmailResult(
|
||||||
campaign_id=campaign.id,
|
campaign_id=campaign.id,
|
||||||
version_id=version.id,
|
version_id=version.id,
|
||||||
to=to,
|
to=to,
|
||||||
subject=str(message["Subject"]),
|
subject=str(message["Subject"]),
|
||||||
dry_run=False,
|
dry_run=True,
|
||||||
sent=True,
|
sent=False,
|
||||||
attached_jobs_csv=jobs_csv is not None,
|
attached_jobs_csv=jobs_csv is not None,
|
||||||
attached_report_json=report_json is not None,
|
attached_report_json=report_json is not None,
|
||||||
smtp_host=result.host,
|
|
||||||
smtp_port=result.port,
|
|
||||||
accepted_count=result.accepted_count,
|
|
||||||
)
|
)
|
||||||
|
|||||||
279
src/govoplan_campaign/backend/response_security.py
Normal file
279
src/govoplan_campaign/backend/response_security.py
Normal file
@@ -0,0 +1,279 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
from pathlib import Path, PureWindowsPath
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import public_campaign_mail_server
|
||||||
|
|
||||||
|
|
||||||
|
# These fields locate process-local or storage-backend resources, or authorize
|
||||||
|
# a worker claim. They are useful for tightly controlled diagnostics but are
|
||||||
|
# not part of the campaign business-data contract.
|
||||||
|
CAMPAIGN_INTERNAL_RESPONSE_KEYS = frozenset(
|
||||||
|
{
|
||||||
|
"campaign_file",
|
||||||
|
"claim_token",
|
||||||
|
"eml_local_path",
|
||||||
|
"eml_path",
|
||||||
|
"eml_storage_key",
|
||||||
|
"local_path",
|
||||||
|
"source_base_path",
|
||||||
|
"storage_bucket",
|
||||||
|
"storage_key",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
CAMPAIGN_DIAGNOSTIC_RESPONSE_KEYS = frozenset(
|
||||||
|
{
|
||||||
|
"background_workers_enabled",
|
||||||
|
"build_token",
|
||||||
|
"celery_enabled",
|
||||||
|
"claimed_at",
|
||||||
|
"effective_policy_sha256",
|
||||||
|
"execution_input_sha256",
|
||||||
|
"imap_claimed_at",
|
||||||
|
"imap_transport_revision",
|
||||||
|
"job_manifest_sha256",
|
||||||
|
"smtp_started_at",
|
||||||
|
"smtp_transport_revision",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
_SEND_NOW_RESULT_KEYS = (
|
||||||
|
"campaign_id",
|
||||||
|
"version_id",
|
||||||
|
"attempted_count",
|
||||||
|
"sent_count",
|
||||||
|
"failed_count",
|
||||||
|
"outcome_unknown_count",
|
||||||
|
"skipped_count",
|
||||||
|
"preflight_count",
|
||||||
|
"delivery_mode",
|
||||||
|
"dry_run",
|
||||||
|
)
|
||||||
|
_SEND_NOW_JOB_RESULT_KEYS = (
|
||||||
|
"campaign_id",
|
||||||
|
"version_id",
|
||||||
|
"job_id",
|
||||||
|
"status",
|
||||||
|
"attempt_number",
|
||||||
|
"dry_run",
|
||||||
|
"queued_count",
|
||||||
|
"skipped_count",
|
||||||
|
"blocked_count",
|
||||||
|
"enqueued_count",
|
||||||
|
"delivery_mode",
|
||||||
|
"worker_queue_available",
|
||||||
|
)
|
||||||
|
_SYNCHRONOUS_POLICY_KEYS = (
|
||||||
|
"max_recipient_jobs",
|
||||||
|
"source",
|
||||||
|
"deployment_max_recipient_jobs",
|
||||||
|
"tenant_max_recipient_jobs",
|
||||||
|
)
|
||||||
|
_VALIDATION_SUMMARY_KEYS = ("ok", "error_count", "warning_count")
|
||||||
|
_BUILD_SUMMARY_KEYS = (
|
||||||
|
"built_count",
|
||||||
|
"build_failed_count",
|
||||||
|
"ready_count",
|
||||||
|
"warning_count",
|
||||||
|
"needs_review_count",
|
||||||
|
"blocked_count",
|
||||||
|
"excluded_count",
|
||||||
|
"inactive_count",
|
||||||
|
"queueable_count",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def public_campaign_payload(value: Any, *, include_diagnostics: bool = False) -> Any:
|
||||||
|
"""Return a detached payload without infrastructure-only locators."""
|
||||||
|
|
||||||
|
if isinstance(value, dict):
|
||||||
|
blocked_keys = CAMPAIGN_INTERNAL_RESPONSE_KEYS
|
||||||
|
if not include_diagnostics:
|
||||||
|
blocked_keys = blocked_keys | CAMPAIGN_DIAGNOSTIC_RESPONSE_KEYS
|
||||||
|
return {
|
||||||
|
key: public_campaign_payload(item, include_diagnostics=include_diagnostics)
|
||||||
|
for key, item in value.items()
|
||||||
|
if key not in blocked_keys
|
||||||
|
}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [public_campaign_payload(item, include_diagnostics=include_diagnostics) for item in value]
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
return tuple(public_campaign_payload(item, include_diagnostics=include_diagnostics) for item in value)
|
||||||
|
return copy.deepcopy(value)
|
||||||
|
|
||||||
|
|
||||||
|
def public_delivery_result_message(
|
||||||
|
*,
|
||||||
|
last_error: Any,
|
||||||
|
send_status: Any,
|
||||||
|
imap_status: Any,
|
||||||
|
postbox_status: Any = None,
|
||||||
|
) -> str | None:
|
||||||
|
"""Map persisted provider text to a stable business-safe explanation."""
|
||||||
|
|
||||||
|
if not last_error:
|
||||||
|
return None
|
||||||
|
clean_send_status = str(send_status or "")
|
||||||
|
clean_imap_status = str(imap_status or "")
|
||||||
|
clean_postbox_status = str(postbox_status or "")
|
||||||
|
if clean_postbox_status == "outcome_unknown":
|
||||||
|
return "Postbox delivery outcome requires operator reconciliation."
|
||||||
|
if clean_send_status == "outcome_unknown":
|
||||||
|
return "Delivery outcome requires operator reconciliation."
|
||||||
|
if clean_postbox_status in {
|
||||||
|
"rejected_temporary",
|
||||||
|
"rejected_permanent",
|
||||||
|
}:
|
||||||
|
return "Postbox delivery was rejected; an operator can inspect restricted diagnostics."
|
||||||
|
if clean_postbox_status == "partially_accepted":
|
||||||
|
return "Some Postbox targets accepted the message and others rejected it."
|
||||||
|
if clean_send_status in {"failed_temporary", "failed_permanent"}:
|
||||||
|
if clean_postbox_status in {"", "not_requested"}:
|
||||||
|
return "SMTP delivery failed; an operator can inspect restricted diagnostics."
|
||||||
|
return "Delivery failed; an operator can inspect restricted diagnostics."
|
||||||
|
if clean_imap_status in {"outcome_unknown", "appending"}:
|
||||||
|
return "Sent-folder append outcome requires operator reconciliation."
|
||||||
|
if clean_imap_status in {"failed", "skipped"}:
|
||||||
|
return "Sent-folder append did not complete; an operator can inspect restricted diagnostics."
|
||||||
|
return "Delivery recorded a warning; an operator can inspect restricted diagnostics."
|
||||||
|
|
||||||
|
|
||||||
|
def public_send_campaign_now_result(
|
||||||
|
value: dict[str, Any],
|
||||||
|
*,
|
||||||
|
validation_summary: dict[str, Any],
|
||||||
|
build_summary: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Project synchronous delivery into its recipient-authorized public contract.
|
||||||
|
|
||||||
|
Per-job provider messages are deliberately omitted. They can contain SMTP
|
||||||
|
diagnostics or refused envelope addresses and belong only in restricted
|
||||||
|
diagnostics backed by persisted job state.
|
||||||
|
"""
|
||||||
|
|
||||||
|
result = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
|
||||||
|
policy = value.get("synchronous_send_policy")
|
||||||
|
result["synchronous_send_policy"] = _selected_payload(
|
||||||
|
policy if isinstance(policy, dict) else {},
|
||||||
|
_SYNCHRONOUS_POLICY_KEYS,
|
||||||
|
)
|
||||||
|
rows = value.get("results")
|
||||||
|
if isinstance(rows, list):
|
||||||
|
result["results"] = [
|
||||||
|
_selected_payload(row, _SEND_NOW_JOB_RESULT_KEYS)
|
||||||
|
for row in rows
|
||||||
|
if isinstance(row, dict)
|
||||||
|
]
|
||||||
|
else:
|
||||||
|
result["results"] = []
|
||||||
|
result["validation"] = _selected_payload(validation_summary, _VALIDATION_SUMMARY_KEYS)
|
||||||
|
result["build"] = _selected_payload(build_summary, _BUILD_SUMMARY_KEYS)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def send_campaign_now_audit_details(value: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Return aggregate-only evidence for a synchronous Campaign send audit."""
|
||||||
|
|
||||||
|
details = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
|
||||||
|
policy = value.get("synchronous_send_policy")
|
||||||
|
details["synchronous_send_policy"] = _selected_payload(
|
||||||
|
policy if isinstance(policy, dict) else {},
|
||||||
|
_SYNCHRONOUS_POLICY_KEYS,
|
||||||
|
)
|
||||||
|
return details
|
||||||
|
|
||||||
|
|
||||||
|
def _selected_payload(value: dict[str, Any], keys: tuple[str, ...]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
key: copy.deepcopy(value[key])
|
||||||
|
for key in keys
|
||||||
|
if key in value
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def public_campaign_configuration(value: Any) -> Any:
|
||||||
|
"""Return campaign JSON without infrastructure locators or mail secrets.
|
||||||
|
|
||||||
|
Password-named business fields are intentionally retained. Only the
|
||||||
|
schema-defined SMTP/IMAP credential paths under ``server`` are secrets.
|
||||||
|
"""
|
||||||
|
|
||||||
|
payload = public_campaign_payload(value)
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return payload
|
||||||
|
if "server" in payload:
|
||||||
|
payload["server"] = public_campaign_mail_server(payload)
|
||||||
|
_sanitize_configuration_paths(payload)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def public_source_filename(value: Any) -> str | None:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
text = str(value).strip()
|
||||||
|
if not text:
|
||||||
|
return text
|
||||||
|
windows_path = PureWindowsPath(text)
|
||||||
|
return windows_path.name if windows_path.is_absolute() or "\\" in text else Path(text).name
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_configuration_paths(payload: dict[str, Any]) -> None:
|
||||||
|
template = payload.get("template")
|
||||||
|
source = template.get("source") if isinstance(template, dict) else None
|
||||||
|
if isinstance(source, dict):
|
||||||
|
for key in ("subject_path", "text_path", "html_path"):
|
||||||
|
if key in source:
|
||||||
|
source[key] = _public_configuration_path(source[key])
|
||||||
|
|
||||||
|
entries = payload.get("entries")
|
||||||
|
entry_source = entries.get("source") if isinstance(entries, dict) else None
|
||||||
|
if isinstance(entry_source, dict) and "path" in entry_source:
|
||||||
|
entry_source["path"] = _public_configuration_path(entry_source["path"])
|
||||||
|
|
||||||
|
attachments = payload.get("attachments")
|
||||||
|
if isinstance(attachments, dict):
|
||||||
|
if "base_path" in attachments:
|
||||||
|
attachments["base_path"] = _public_configuration_path(attachments["base_path"])
|
||||||
|
base_paths = attachments.get("base_paths")
|
||||||
|
if isinstance(base_paths, list):
|
||||||
|
for item in base_paths:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
for key in ("path", "source"):
|
||||||
|
if key in item:
|
||||||
|
item[key] = _public_configuration_path(item[key])
|
||||||
|
_sanitize_attachment_rules(attachments.get("global"))
|
||||||
|
|
||||||
|
if isinstance(entries, dict):
|
||||||
|
inline_entries = entries.get("inline")
|
||||||
|
if isinstance(inline_entries, list):
|
||||||
|
for entry in inline_entries:
|
||||||
|
if isinstance(entry, dict):
|
||||||
|
_sanitize_attachment_rules(entry.get("attachments"))
|
||||||
|
defaults = entries.get("defaults")
|
||||||
|
if isinstance(defaults, dict):
|
||||||
|
_sanitize_attachment_rules(defaults.get("attachments"))
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_attachment_rules(value: Any) -> None:
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return
|
||||||
|
for rule in value:
|
||||||
|
if isinstance(rule, dict) and "base_dir" in rule:
|
||||||
|
rule["base_dir"] = _public_configuration_path(rule["base_dir"])
|
||||||
|
|
||||||
|
|
||||||
|
def _public_configuration_path(value: Any) -> Any:
|
||||||
|
if not isinstance(value, str) or not value.strip():
|
||||||
|
return value
|
||||||
|
text = value.strip()
|
||||||
|
windows_path = PureWindowsPath(text)
|
||||||
|
path = Path(text)
|
||||||
|
if windows_path.is_absolute():
|
||||||
|
return windows_path.name
|
||||||
|
if path.is_absolute() or text.startswith("~"):
|
||||||
|
return path.name
|
||||||
|
return value
|
||||||
@@ -127,7 +127,11 @@ def _apply_eml_retention(
|
|||||||
if job.queue_status in {JobQueueStatus.QUEUED.value, JobQueueStatus.SENDING.value} or job.send_status not in FINAL_EML_SEND_STATUSES:
|
if job.queue_status in {JobQueueStatus.QUEUED.value, JobQueueStatus.SENDING.value} or job.send_status not in FINAL_EML_SEND_STATUSES:
|
||||||
result["skipped_not_final"] += 1
|
result["skipped_not_final"] += 1
|
||||||
continue
|
continue
|
||||||
if job.imap_status == JobImapStatus.PENDING.value:
|
if job.imap_status in {
|
||||||
|
JobImapStatus.PENDING.value,
|
||||||
|
JobImapStatus.APPENDING.value,
|
||||||
|
JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
}:
|
||||||
result["skipped_not_final"] += 1
|
result["skipped_not_final"] += 1
|
||||||
continue
|
continue
|
||||||
result["eligible"] += 1
|
result["eligible"] += 1
|
||||||
|
|||||||
675
src/govoplan_campaign/backend/route_support.py
Normal file
675
src/govoplan_campaign/backend/route_support.py
Normal file
@@ -0,0 +1,675 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import dataclasses
|
||||||
|
from collections.abc import Callable
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import and_, exists, or_
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
CAMPAIGN_MAIL_SERVER_KEYS,
|
||||||
|
campaign_mail_profile_id,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignIssue,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignShare,
|
||||||
|
CampaignStatus,
|
||||||
|
CampaignVersion,
|
||||||
|
CampaignVersionWorkflowState,
|
||||||
|
RecipientImportMappingProfile,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import CampaignPersistenceError
|
||||||
|
from govoplan_campaign.backend.persistence.versions import (
|
||||||
|
LockedCampaignVersionError,
|
||||||
|
is_user_locked_version,
|
||||||
|
is_version_final_locked,
|
||||||
|
is_version_locked,
|
||||||
|
update_campaign_version,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
CampaignVersionDetailResponse,
|
||||||
|
CampaignVersionUpdateRequest,
|
||||||
|
RecipientImportMappingProfilePayload,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.sending.execution import (
|
||||||
|
clear_execution_snapshot,
|
||||||
|
)
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_core.auth import ApiPrincipal, has_scope
|
||||||
|
from govoplan_core.core.access import CAPABILITY_ACCESS_DIRECTORY, AccessDirectory
|
||||||
|
from govoplan_core.core.concurrency import (
|
||||||
|
ConcurrencyError,
|
||||||
|
MissingPreconditionError,
|
||||||
|
RevisionConflictError,
|
||||||
|
assert_revision_precondition,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.runtime import get_registry
|
||||||
|
|
||||||
|
|
||||||
|
def _capability_payload(value: object) -> dict[str, Any]:
|
||||||
|
if dataclasses.is_dataclass(value):
|
||||||
|
return dataclasses.asdict(value)
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return dict(value)
|
||||||
|
payload: dict[str, Any] = {}
|
||||||
|
for key in (
|
||||||
|
"contact_id",
|
||||||
|
"address_book_id",
|
||||||
|
"display_name",
|
||||||
|
"email",
|
||||||
|
"email_label",
|
||||||
|
"organization",
|
||||||
|
"role_title",
|
||||||
|
"tags",
|
||||||
|
"source_kind",
|
||||||
|
"source_ref",
|
||||||
|
"source_revision",
|
||||||
|
"source_id",
|
||||||
|
"source_label",
|
||||||
|
"recipient_count",
|
||||||
|
"generated_at",
|
||||||
|
"recipients",
|
||||||
|
"fields",
|
||||||
|
"provenance",
|
||||||
|
):
|
||||||
|
if hasattr(value, key):
|
||||||
|
payload[key] = getattr(value, key)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def _registry_capability(name: str) -> object | None:
|
||||||
|
registry = get_registry()
|
||||||
|
if (
|
||||||
|
registry is None
|
||||||
|
or not hasattr(registry, "has_capability")
|
||||||
|
or not registry.has_capability(name)
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
return registry.capability(name)
|
||||||
|
|
||||||
|
|
||||||
|
def _access_directory() -> AccessDirectory:
|
||||||
|
registry = get_registry()
|
||||||
|
if (
|
||||||
|
registry is None
|
||||||
|
or not hasattr(registry, "has_capability")
|
||||||
|
or not registry.has_capability(CAPABILITY_ACCESS_DIRECTORY)
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
|
detail="Access directory capability is not configured",
|
||||||
|
)
|
||||||
|
capability = registry.require_capability(CAPABILITY_ACCESS_DIRECTORY)
|
||||||
|
if not isinstance(capability, AccessDirectory):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
|
detail="Access directory capability is invalid",
|
||||||
|
)
|
||||||
|
return capability
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign_for_tenant(
|
||||||
|
session: Session, campaign_id: str, tenant_id: str
|
||||||
|
) -> Campaign:
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign not found"
|
||||||
|
)
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _get_version_for_tenant(
|
||||||
|
session: Session, version_id: str, tenant_id: str
|
||||||
|
) -> CampaignVersion:
|
||||||
|
version = session.get(CampaignVersion, version_id)
|
||||||
|
if not version:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _principal_group_ids(session: Session, principal: ApiPrincipal) -> set[str]:
|
||||||
|
del session
|
||||||
|
return {
|
||||||
|
group.id
|
||||||
|
for group in _access_directory().groups_for_user(
|
||||||
|
principal.user.id, tenant_id=principal.tenant_id
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_acl_filter(session: Session, principal: ApiPrincipal):
|
||||||
|
if has_scope(principal, "tenant:*"):
|
||||||
|
return None
|
||||||
|
group_ids = _principal_group_ids(session, principal)
|
||||||
|
clauses = [Campaign.owner_user_id == principal.user.id]
|
||||||
|
if group_ids:
|
||||||
|
clauses.append(Campaign.owner_group_id.in_(group_ids))
|
||||||
|
share_clauses = [
|
||||||
|
and_(
|
||||||
|
CampaignShare.tenant_id == Campaign.tenant_id,
|
||||||
|
CampaignShare.campaign_id == Campaign.id,
|
||||||
|
CampaignShare.revoked_at.is_(None),
|
||||||
|
CampaignShare.target_type == "user",
|
||||||
|
CampaignShare.target_id == principal.user.id,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
if group_ids:
|
||||||
|
share_clauses.append(
|
||||||
|
and_(
|
||||||
|
CampaignShare.tenant_id == Campaign.tenant_id,
|
||||||
|
CampaignShare.campaign_id == Campaign.id,
|
||||||
|
CampaignShare.revoked_at.is_(None),
|
||||||
|
CampaignShare.target_type == "group",
|
||||||
|
CampaignShare.target_id.in_(group_ids),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
clauses.append(exists().where(or_(*share_clauses)))
|
||||||
|
return or_(*clauses)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_acl_allows(
|
||||||
|
session: Session,
|
||||||
|
campaign: Campaign,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
write: bool = False,
|
||||||
|
) -> bool:
|
||||||
|
if has_scope(principal, "tenant:*"):
|
||||||
|
return True
|
||||||
|
if campaign.owner_user_id == principal.user.id:
|
||||||
|
return True
|
||||||
|
group_ids = _principal_group_ids(session, principal)
|
||||||
|
if campaign.owner_group_id and campaign.owner_group_id in group_ids:
|
||||||
|
return True
|
||||||
|
target_ids = [principal.user.id, *group_ids]
|
||||||
|
if not target_ids:
|
||||||
|
return False
|
||||||
|
query = session.query(CampaignShare).filter(
|
||||||
|
CampaignShare.tenant_id == campaign.tenant_id,
|
||||||
|
CampaignShare.campaign_id == campaign.id,
|
||||||
|
CampaignShare.revoked_at.is_(None),
|
||||||
|
or_(
|
||||||
|
CampaignShare.target_type == "user",
|
||||||
|
CampaignShare.target_type == "group",
|
||||||
|
),
|
||||||
|
CampaignShare.target_id.in_(target_ids),
|
||||||
|
)
|
||||||
|
shares = query.all()
|
||||||
|
if not shares:
|
||||||
|
return False
|
||||||
|
if not write:
|
||||||
|
return True
|
||||||
|
return any(item.permission == "write" for item in shares)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_campaign_acl(
|
||||||
|
session: Session,
|
||||||
|
campaign: Campaign,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
write: bool = False,
|
||||||
|
) -> None:
|
||||||
|
if not _campaign_acl_allows(session, campaign, principal, write=write):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Campaign is not shared with this principal",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign_for_principal(
|
||||||
|
session: Session, campaign_id: str, principal: ApiPrincipal, *, write: bool = False
|
||||||
|
) -> Campaign:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
_require_campaign_acl(session, campaign, principal, write=write)
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _require_permission(principal: ApiPrincipal, scope: str) -> None:
|
||||||
|
if not has_scope(principal, scope):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {scope}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_query_for_principal(session: Session, principal: ApiPrincipal):
|
||||||
|
query = session.query(Campaign).filter(
|
||||||
|
Campaign.tenant_id == principal.tenant_id, Campaign.status != "deleted"
|
||||||
|
)
|
||||||
|
acl_filter = _campaign_acl_filter(session, principal)
|
||||||
|
if acl_filter is not None:
|
||||||
|
query = query.filter(acl_filter)
|
||||||
|
return query
|
||||||
|
|
||||||
|
|
||||||
|
def _get_recipient_import_profile_for_principal(
|
||||||
|
session: Session, profile_id: str, principal: ApiPrincipal
|
||||||
|
) -> RecipientImportMappingProfile:
|
||||||
|
profile = session.get(RecipientImportMappingProfile, profile_id)
|
||||||
|
if (
|
||||||
|
not profile
|
||||||
|
or profile.tenant_id != principal.tenant_id
|
||||||
|
or profile.owner_user_id != principal.user.id
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail="Recipient import mapping profile not found",
|
||||||
|
)
|
||||||
|
return profile
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_recipient_import_profile_payload(
|
||||||
|
profile: RecipientImportMappingProfile,
|
||||||
|
payload: RecipientImportMappingProfilePayload,
|
||||||
|
) -> None:
|
||||||
|
profile.name = payload.name.strip()
|
||||||
|
profile.column_count = payload.column_count
|
||||||
|
profile.headers = list(payload.headers)
|
||||||
|
profile.normalized_headers = list(payload.normalized_headers)
|
||||||
|
profile.ordered_header_fingerprint = payload.ordered_header_fingerprint
|
||||||
|
profile.unordered_header_fingerprint = payload.unordered_header_fingerprint
|
||||||
|
profile.delimiter = payload.delimiter
|
||||||
|
profile.header_rows = payload.header_rows
|
||||||
|
profile.quoted = payload.quoted
|
||||||
|
profile.value_separators = payload.value_separators
|
||||||
|
profile.mappings = [mapping.model_dump(mode="json") for mapping in payload.mappings]
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_sections_changed(
|
||||||
|
current: dict[str, object] | None, proposed: dict[str, object] | None
|
||||||
|
) -> bool:
|
||||||
|
if proposed is None:
|
||||||
|
return False
|
||||||
|
current = current or {}
|
||||||
|
return any(
|
||||||
|
current.get(key) != proposed.get(key) for key in ("recipients", "entries")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_mail_profile_id(raw_json: dict[str, object] | None) -> str | None:
|
||||||
|
return campaign_mail_profile_id(raw_json)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_mail_profile_use_if_needed(
|
||||||
|
principal: ApiPrincipal, raw_json: dict[str, object] | None
|
||||||
|
) -> None:
|
||||||
|
if _campaign_mail_profile_id(raw_json) and not has_scope(
|
||||||
|
principal, "mail:profile:use"
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Missing scope: mail:profile:use",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_response_context(principal: ApiPrincipal) -> dict[str, bool]:
|
||||||
|
return {"include_diagnostics": has_scope(principal, "campaigns:diagnostic:read")}
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_version_detail_response(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
campaign_id: str,
|
||||||
|
mutation: Callable[[], CampaignVersion],
|
||||||
|
*,
|
||||||
|
audit_action: str,
|
||||||
|
details: dict[str, Any] | Callable[[CampaignVersion], dict[str, Any]] | None = None,
|
||||||
|
validation_error_status: int | None = None,
|
||||||
|
) -> CampaignVersionDetailResponse:
|
||||||
|
try:
|
||||||
|
version = mutation()
|
||||||
|
audit_details = (
|
||||||
|
details(version)
|
||||||
|
if callable(details)
|
||||||
|
else dict(details or {"campaign_id": campaign_id})
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action=audit_action,
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details=audit_details,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
_write_current_version_snapshot_if_available(version)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(
|
||||||
|
version,
|
||||||
|
context=_campaign_response_context(principal),
|
||||||
|
)
|
||||||
|
except LockedCampaignVersionError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except CampaignPathSecurityError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except RevisionConflictError:
|
||||||
|
session.rollback()
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
session.rollback()
|
||||||
|
if validation_error_status is None:
|
||||||
|
raise
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=validation_error_status, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _update_campaign_version_detail_response(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest,
|
||||||
|
*,
|
||||||
|
if_match: str | None,
|
||||||
|
autosave: bool,
|
||||||
|
audit_action: str,
|
||||||
|
) -> CampaignVersionDetailResponse:
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
current_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
if payload.base_revision is None:
|
||||||
|
error = MissingPreconditionError(
|
||||||
|
resource_type="campaign_version",
|
||||||
|
resource_id=version_id,
|
||||||
|
)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
|
||||||
|
detail=error.as_dict(),
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
assert_revision_precondition(
|
||||||
|
if_match,
|
||||||
|
resource_type="campaign_version",
|
||||||
|
resource_id=version_id,
|
||||||
|
submitted_base_revision=payload.base_revision,
|
||||||
|
)
|
||||||
|
except MissingPreconditionError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
|
||||||
|
detail=exc.as_dict(),
|
||||||
|
) from exc
|
||||||
|
except ConcurrencyError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail={
|
||||||
|
"code": "invalid_precondition",
|
||||||
|
"message": str(exc),
|
||||||
|
},
|
||||||
|
) from exc
|
||||||
|
if _recipient_sections_changed(current_version.raw_json, payload.campaign_json):
|
||||||
|
_require_permission(principal, "campaigns:recipient:write")
|
||||||
|
_require_mail_profile_use_if_needed(principal, payload.campaign_json)
|
||||||
|
try:
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
raw_json=payload.campaign_json,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
workflow_state=payload.workflow_state,
|
||||||
|
is_complete=payload.is_complete,
|
||||||
|
editor_state=payload.editor_state,
|
||||||
|
source_filename=payload.source_filename,
|
||||||
|
source_base_path=payload.source_base_path,
|
||||||
|
autosave=autosave,
|
||||||
|
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
|
||||||
|
expected_revision=payload.base_revision,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action=audit_action,
|
||||||
|
details=lambda version: {
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"current_flow": version.current_flow,
|
||||||
|
"current_step": version.current_step,
|
||||||
|
"base_revision": payload.base_revision,
|
||||||
|
"result_revision": version.edit_revision,
|
||||||
|
"reconciliation_kind": payload.reconciliation_kind,
|
||||||
|
"resolved_conflict_path_count": len(
|
||||||
|
payload.resolved_conflict_paths
|
||||||
|
),
|
||||||
|
"resolved_conflict_sections": sorted(
|
||||||
|
{
|
||||||
|
path.strip("/").split("/", 1)[0][:80]
|
||||||
|
for path in payload.resolved_conflict_paths[:100]
|
||||||
|
if path.strip("/")
|
||||||
|
}
|
||||||
|
),
|
||||||
|
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
|
||||||
|
},
|
||||||
|
validation_error_status=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
)
|
||||||
|
except RevisionConflictError as exc:
|
||||||
|
session.rollback()
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_conflict_detected",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"submitted_base_revision": exc.submitted_base_revision,
|
||||||
|
"current_revision": exc.current_revision,
|
||||||
|
"retryable": True,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_412_PRECONDITION_FAILED,
|
||||||
|
detail=exc.as_dict(),
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _require_campaign_profile_use_if_needed(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
) -> None:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
target_version_id = version_id or campaign.current_version_id
|
||||||
|
if not target_version_id:
|
||||||
|
return
|
||||||
|
version = _get_version_for_tenant(session, target_version_id, principal.tenant_id)
|
||||||
|
if version.campaign_id != campaign.id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_campaign_versions_profile_use(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
campaign_id: str,
|
||||||
|
version_ids: set[str],
|
||||||
|
) -> None:
|
||||||
|
"""Authorize every historical version affected by a campaign-wide action."""
|
||||||
|
|
||||||
|
for version_id in sorted(version_ids):
|
||||||
|
_require_campaign_profile_use_if_needed(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
version_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_version_for_principal(
|
||||||
|
session: Session,
|
||||||
|
version_id: str,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
write: bool = False,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
campaign = _get_campaign_for_tenant(
|
||||||
|
session, version.campaign_id, principal.tenant_id
|
||||||
|
)
|
||||||
|
_require_campaign_acl(session, campaign, principal, write=write)
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _sync_campaign_metadata_to_current_version(
|
||||||
|
session: Session, campaign: Campaign
|
||||||
|
) -> None:
|
||||||
|
"""Keep editable version JSON aligned with version-independent campaign metadata.
|
||||||
|
|
||||||
|
Campaign metadata can be edited from the overview while individual campaign
|
||||||
|
sections save the current version JSON later. Without this sync, a later
|
||||||
|
version save can re-apply stale `campaign.name` / `campaign.id` values from
|
||||||
|
raw_json and make the old overview metadata appear to come back. Audit-safe
|
||||||
|
or validation-locked versions are left untouched.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not campaign.current_version_id:
|
||||||
|
return
|
||||||
|
|
||||||
|
version = session.get(CampaignVersion, campaign.current_version_id)
|
||||||
|
if not version or version.campaign_id != campaign.id or is_version_locked(version):
|
||||||
|
return
|
||||||
|
|
||||||
|
raw_json = copy.deepcopy(
|
||||||
|
version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
campaign_section = (
|
||||||
|
raw_json.get("campaign") if isinstance(raw_json.get("campaign"), dict) else {}
|
||||||
|
)
|
||||||
|
raw_json["campaign"] = {
|
||||||
|
**campaign_section,
|
||||||
|
"id": campaign.external_id,
|
||||||
|
"name": campaign.name,
|
||||||
|
"description": campaign.description or "",
|
||||||
|
}
|
||||||
|
version.raw_json = raw_json
|
||||||
|
session.add(version)
|
||||||
|
|
||||||
|
|
||||||
|
def _clear_current_version_mail_profile_for_owner_transfer(
|
||||||
|
session: Session, campaign: Campaign
|
||||||
|
) -> bool:
|
||||||
|
"""Force explicit profile reselection after campaign ownership changes.
|
||||||
|
|
||||||
|
User/group-scoped reusable mail profiles are evaluated against the current
|
||||||
|
owner. Instead of trying to keep a stale selection across an ownership
|
||||||
|
transfer, clear the profile from the editable current version and invalidate
|
||||||
|
validation/build state so the operator has to reselect and revalidate.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not campaign.current_version_id:
|
||||||
|
return False
|
||||||
|
|
||||||
|
version = session.get(CampaignVersion, campaign.current_version_id)
|
||||||
|
if not version or version.campaign_id != campaign.id:
|
||||||
|
return False
|
||||||
|
|
||||||
|
raw_json = copy.deepcopy(
|
||||||
|
version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
server = (
|
||||||
|
raw_json.get("server") if isinstance(raw_json.get("server"), dict) else None
|
||||||
|
)
|
||||||
|
if not isinstance(server, dict):
|
||||||
|
return False
|
||||||
|
|
||||||
|
profile_id = _campaign_mail_profile_id(raw_json)
|
||||||
|
if not profile_id:
|
||||||
|
return False
|
||||||
|
|
||||||
|
if is_version_final_locked(version) or is_user_locked_version(version):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail="Change owner only after creating an editable campaign version; the current version has a selected mail profile and is locked.",
|
||||||
|
)
|
||||||
|
|
||||||
|
next_server = dict(server)
|
||||||
|
for key in CAMPAIGN_MAIL_SERVER_KEYS:
|
||||||
|
next_server.pop(key, None)
|
||||||
|
next_server.pop("profile_id", None)
|
||||||
|
raw_json["server"] = next_server
|
||||||
|
|
||||||
|
version.raw_json = raw_json
|
||||||
|
version.validation_summary = None
|
||||||
|
version.build_summary = None
|
||||||
|
clear_execution_snapshot(version)
|
||||||
|
version.locked_at = None
|
||||||
|
version.locked_by_user_id = None
|
||||||
|
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
|
||||||
|
version.is_complete = False
|
||||||
|
|
||||||
|
editor_state = copy.deepcopy(version.editor_state or {})
|
||||||
|
editor_state.pop("review_send", None)
|
||||||
|
version.editor_state = editor_state
|
||||||
|
|
||||||
|
session.query(CampaignIssue).filter(
|
||||||
|
CampaignIssue.campaign_version_id == version.id
|
||||||
|
).delete(synchronize_session=False)
|
||||||
|
session.query(CampaignJob).filter(
|
||||||
|
CampaignJob.campaign_version_id == version.id
|
||||||
|
).delete(synchronize_session=False)
|
||||||
|
campaign.status = CampaignStatus.DRAFT.value
|
||||||
|
session.add(version)
|
||||||
|
_write_current_version_snapshot_if_available(version)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _write_current_version_snapshot_if_available(version: CampaignVersion) -> None:
|
||||||
|
try:
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import (
|
||||||
|
_write_campaign_snapshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
_write_campaign_snapshot(version)
|
||||||
|
except Exception:
|
||||||
|
# The database state is authoritative for the WebUI. Snapshot writing is
|
||||||
|
# best-effort here because ownership changes should not fail due to an
|
||||||
|
# unavailable local runtime directory.
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
def bounded_query_rows(query, *, limit: int, label: str):
|
||||||
|
rows = query.limit(limit + 1).all()
|
||||||
|
if len(rows) > limit:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
|
||||||
|
detail=(
|
||||||
|
f"{label} exceeds the maximum response size of {limit} rows. "
|
||||||
|
"Narrow the request or use a paginated/delta endpoint."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def job_attempt_rows(query, *, label: str):
|
||||||
|
return bounded_query_rows(query, limit=1000, label=label)
|
||||||
File diff suppressed because it is too large
Load Diff
1
src/govoplan_campaign/backend/routes/__init__.py
Normal file
1
src/govoplan_campaign/backend/routes/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""Focused HTTP route modules for the campaign API."""
|
||||||
398
src/govoplan_campaign/backend/routes/attachments.py
Normal file
398
src/govoplan_campaign/backend/routes/attachments.py
Normal file
@@ -0,0 +1,398 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import (
|
||||||
|
load_campaign_config_from_json,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
files_integration,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.path_security import (
|
||||||
|
CampaignPathSecurityError,
|
||||||
|
assert_server_safe_campaign_paths,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.campaign.loader import load_campaign_json
|
||||||
|
from govoplan_campaign.backend.attachments.resolver import resolve_campaign_attachments
|
||||||
|
from govoplan_campaign.backend.persistence.versions import (
|
||||||
|
is_version_final_locked,
|
||||||
|
is_user_locked_version,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_get_campaign_for_tenant,
|
||||||
|
_get_version_for_tenant,
|
||||||
|
_require_mail_profile_use_if_needed,
|
||||||
|
_require_permission,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignAttachmentPreviewRequest(BaseModel):
|
||||||
|
include_unmatched: bool = True
|
||||||
|
include_unlinked_candidates: bool = False
|
||||||
|
campaign_json: dict[str, object] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignAttachmentPreviewResponse(BaseModel):
|
||||||
|
campaign_id: str
|
||||||
|
version_id: str
|
||||||
|
shared_file_count: int
|
||||||
|
candidate_file_count: int = 0
|
||||||
|
matched_file_count: int = 0
|
||||||
|
linked_file_count: int = 0
|
||||||
|
unlinked_file_count: int = 0
|
||||||
|
rules: list[dict[str, object]] = Field(default_factory=list)
|
||||||
|
linkable_files: list[dict[str, object]] = Field(default_factory=list)
|
||||||
|
unused_shared_files: list[dict[str, object]] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignAttachmentLinkMatchesRequest(BaseModel):
|
||||||
|
campaign_json: dict[str, object] | None = None
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignAttachmentLinkMatchesResponse(BaseModel):
|
||||||
|
campaign_id: str
|
||||||
|
version_id: str
|
||||||
|
matched_file_count: int
|
||||||
|
already_linked_file_count: int
|
||||||
|
linked_file_count: int
|
||||||
|
dry_run: bool = False
|
||||||
|
linked_files: list[dict[str, object]] = Field(default_factory=list)
|
||||||
|
linkable_files: list[dict[str, object]] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
def _file_preview(session: Session, asset) -> dict[str, object]:
|
||||||
|
version, blob = files_integration().current_version_and_blob(session, asset)
|
||||||
|
return {
|
||||||
|
"id": asset.id,
|
||||||
|
"version_id": version.id,
|
||||||
|
"blob_id": blob.id,
|
||||||
|
"display_path": asset.display_path,
|
||||||
|
"filename": asset.filename,
|
||||||
|
"owner_type": asset.owner_type,
|
||||||
|
"owner_id": asset.owner_user_id
|
||||||
|
if asset.owner_type == "user"
|
||||||
|
else asset.owner_group_id,
|
||||||
|
"checksum_sha256": blob.checksum_sha256,
|
||||||
|
"size_bytes": blob.size_bytes,
|
||||||
|
"content_type": blob.content_type,
|
||||||
|
"linked_to_campaign": True,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _managed_preview_file(item: dict[str, object]) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"id": item["asset_id"],
|
||||||
|
"version_id": item["version_id"],
|
||||||
|
"blob_id": item["blob_id"],
|
||||||
|
"display_path": item["display_path"],
|
||||||
|
"filename": item["filename"],
|
||||||
|
"owner_type": item["owner_type"],
|
||||||
|
"owner_id": item["owner_id"],
|
||||||
|
"checksum_sha256": item["checksum_sha256"],
|
||||||
|
"size_bytes": item["size_bytes"],
|
||||||
|
"content_type": item["content_type"],
|
||||||
|
"linked_to_campaign": bool(item.get("linked_to_campaign", True)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_preview_for_version(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
raw: dict[str, object],
|
||||||
|
include_unmatched: bool,
|
||||||
|
include_unlinked_candidates: bool,
|
||||||
|
) -> CampaignAttachmentPreviewResponse:
|
||||||
|
files = files_integration()
|
||||||
|
assert_server_safe_campaign_paths(raw, managed_files_available=files.available)
|
||||||
|
with files.prepared_campaign_snapshot(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
raw_json=raw,
|
||||||
|
include_bytes=False,
|
||||||
|
prefix="govoplan-managed-preview-",
|
||||||
|
include_unlinked_candidates=include_unlinked_candidates,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
is_admin=has_scope(principal, "files:file:admin"),
|
||||||
|
) as prepared:
|
||||||
|
prepared_raw = load_campaign_json(prepared.path)
|
||||||
|
config = load_campaign_config_from_json(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
raw_json=prepared_raw,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
)
|
||||||
|
report = resolve_campaign_attachments(config, campaign_file=prepared.path)
|
||||||
|
rules: list[dict[str, object]] = []
|
||||||
|
matched_asset_ids: set[str] = set()
|
||||||
|
linked_asset_ids: set[str] = set()
|
||||||
|
linkable_by_id: dict[str, dict[str, object]] = {}
|
||||||
|
|
||||||
|
for entry in report.entries:
|
||||||
|
for attachment in entry.attachments:
|
||||||
|
managed_matches = files.managed_match_payloads(
|
||||||
|
attachment.matches, prepared.managed_files_by_local_path
|
||||||
|
)
|
||||||
|
matches: list[dict[str, object]] = []
|
||||||
|
for item in managed_matches:
|
||||||
|
asset_id = str(item["asset_id"])
|
||||||
|
matched_asset_ids.add(asset_id)
|
||||||
|
if bool(item.get("linked_to_campaign", True)):
|
||||||
|
linked_asset_ids.add(asset_id)
|
||||||
|
preview = _managed_preview_file(item)
|
||||||
|
matches.append(preview)
|
||||||
|
if not preview["linked_to_campaign"]:
|
||||||
|
linkable_by_id.setdefault(asset_id, preview)
|
||||||
|
if not matches:
|
||||||
|
matches = [
|
||||||
|
{
|
||||||
|
"id": "",
|
||||||
|
"display_path": match,
|
||||||
|
"filename": match.rsplit("/", 1)[-1].rsplit("\\", 1)[-1],
|
||||||
|
"owner_type": "legacy",
|
||||||
|
"owner_id": "",
|
||||||
|
"linked_to_campaign": True,
|
||||||
|
}
|
||||||
|
for match in attachment.matches
|
||||||
|
]
|
||||||
|
rules.append(
|
||||||
|
{
|
||||||
|
"source": attachment.scope.value,
|
||||||
|
"entry_index": entry.entry_index,
|
||||||
|
"entry_id": entry.entry_id,
|
||||||
|
"index": attachment.index,
|
||||||
|
"attachment_id": attachment.attachment_id,
|
||||||
|
"label": attachment.label,
|
||||||
|
"required": attachment.required,
|
||||||
|
"pattern": attachment.file_filter,
|
||||||
|
"base_path_name": attachment.base_path_name,
|
||||||
|
"base_path": attachment.base_path,
|
||||||
|
"status": attachment.status.value,
|
||||||
|
"behavior": attachment.behavior.value
|
||||||
|
if attachment.behavior
|
||||||
|
else None,
|
||||||
|
"zip_included": attachment.zip_enabled,
|
||||||
|
"zip_mode": attachment.zip_mode.value,
|
||||||
|
"zip_archive_id": attachment.zip_archive_id,
|
||||||
|
"zip_filename": attachment.zip_filename,
|
||||||
|
"matches": matches,
|
||||||
|
"match_count": len(matches),
|
||||||
|
"linked_match_count": sum(
|
||||||
|
1
|
||||||
|
for match in matches
|
||||||
|
if bool(match.get("linked_to_campaign", True))
|
||||||
|
),
|
||||||
|
"unlinked_match_count": sum(
|
||||||
|
1
|
||||||
|
for match in matches
|
||||||
|
if not bool(match.get("linked_to_campaign", True))
|
||||||
|
),
|
||||||
|
"issues": [
|
||||||
|
issue.model_dump(mode="json") for issue in attachment.issues
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
unused = [
|
||||||
|
asset
|
||||||
|
for asset in prepared.shared_assets
|
||||||
|
if asset.id not in matched_asset_ids
|
||||||
|
]
|
||||||
|
return CampaignAttachmentPreviewResponse(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
shared_file_count=len(prepared.shared_assets),
|
||||||
|
candidate_file_count=len(
|
||||||
|
getattr(prepared, "candidate_assets", prepared.shared_assets)
|
||||||
|
),
|
||||||
|
matched_file_count=len(matched_asset_ids),
|
||||||
|
linked_file_count=len(linked_asset_ids),
|
||||||
|
unlinked_file_count=len(linkable_by_id),
|
||||||
|
rules=rules,
|
||||||
|
linkable_files=list(linkable_by_id.values()),
|
||||||
|
unused_shared_files=[_file_preview(session, asset) for asset in unused]
|
||||||
|
if include_unmatched
|
||||||
|
else [],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _link_campaign_attachment_matches(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
version: CampaignVersion,
|
||||||
|
raw: dict[str, object],
|
||||||
|
dry_run: bool = False,
|
||||||
|
) -> CampaignAttachmentLinkMatchesResponse:
|
||||||
|
preview = _attachment_preview_for_version(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
raw=raw,
|
||||||
|
include_unmatched=False,
|
||||||
|
include_unlinked_candidates=True,
|
||||||
|
)
|
||||||
|
file_ids = [
|
||||||
|
str(item.get("id") or "") for item in preview.linkable_files if item.get("id")
|
||||||
|
]
|
||||||
|
linked_files: list[dict[str, object]] = []
|
||||||
|
if file_ids and not dry_run:
|
||||||
|
files = files_integration()
|
||||||
|
shares = files.share_assets_with_campaign(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
file_ids=file_ids,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
is_admin=has_scope(principal, "files:file:admin"),
|
||||||
|
)
|
||||||
|
share_by_asset_id = {
|
||||||
|
str(item.get("file_asset_id") or ""): item for item in shares
|
||||||
|
}
|
||||||
|
linked_files = [
|
||||||
|
{**item, "share": share_by_asset_id.get(str(item.get("id") or ""))}
|
||||||
|
for item in preview.linkable_files
|
||||||
|
]
|
||||||
|
return CampaignAttachmentLinkMatchesResponse(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
matched_file_count=preview.matched_file_count,
|
||||||
|
already_linked_file_count=preview.linked_file_count,
|
||||||
|
linked_file_count=0 if dry_run else len(file_ids),
|
||||||
|
dry_run=dry_run,
|
||||||
|
linked_files=linked_files,
|
||||||
|
linkable_files=preview.linkable_files,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/attachments/preview",
|
||||||
|
response_model=CampaignAttachmentPreviewResponse,
|
||||||
|
)
|
||||||
|
def preview_campaign_attachments(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignAttachmentPreviewRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
if version.campaign_id != campaign.id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
|
||||||
|
payload = payload or CampaignAttachmentPreviewRequest()
|
||||||
|
raw = (
|
||||||
|
payload.campaign_json
|
||||||
|
if isinstance(payload.campaign_json, dict)
|
||||||
|
else version.raw_json
|
||||||
|
)
|
||||||
|
raw = raw if isinstance(raw, dict) else {}
|
||||||
|
_require_mail_profile_use_if_needed(principal, raw)
|
||||||
|
try:
|
||||||
|
return _attachment_preview_for_version(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
raw=raw,
|
||||||
|
include_unmatched=payload.include_unmatched,
|
||||||
|
include_unlinked_candidates=payload.include_unlinked_candidates,
|
||||||
|
)
|
||||||
|
except CampaignPathSecurityError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/attachments/link-matches",
|
||||||
|
response_model=CampaignAttachmentLinkMatchesResponse,
|
||||||
|
)
|
||||||
|
def link_campaign_attachment_matches(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignAttachmentLinkMatchesRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||||
|
):
|
||||||
|
_require_permission(principal, "files:file:share")
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
if version.campaign_id != campaign.id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
if is_user_locked_version(version) or is_version_final_locked(version):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail="Locked campaign versions cannot link new attachment files",
|
||||||
|
)
|
||||||
|
payload = payload or CampaignAttachmentLinkMatchesRequest()
|
||||||
|
raw = (
|
||||||
|
payload.campaign_json
|
||||||
|
if isinstance(payload.campaign_json, dict)
|
||||||
|
else version.raw_json
|
||||||
|
)
|
||||||
|
raw = raw if isinstance(raw, dict) else {}
|
||||||
|
_require_mail_profile_use_if_needed(principal, raw)
|
||||||
|
try:
|
||||||
|
result = _link_campaign_attachment_matches(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
raw=raw,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.attachment_matches_linked"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.attachment_matches_link_previewed",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={
|
||||||
|
"matched_file_count": result.matched_file_count,
|
||||||
|
"already_linked_file_count": result.already_linked_file_count,
|
||||||
|
"linked_file_count": result.linked_file_count,
|
||||||
|
"dry_run": result.dry_run,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
1362
src/govoplan_campaign/backend/routes/campaigns.py
Normal file
1362
src/govoplan_campaign/backend/routes/campaigns.py
Normal file
File diff suppressed because it is too large
Load Diff
665
src/govoplan_campaign/backend/routes/delivery.py
Normal file
665
src/govoplan_campaign/backend/routes/delivery.py
Normal file
@@ -0,0 +1,665 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
AppendSentRequest,
|
||||||
|
CampaignActionResponse,
|
||||||
|
CampaignRetryJobsRequest,
|
||||||
|
CampaignSendJobRequest,
|
||||||
|
CampaignSendUnattemptedRequest,
|
||||||
|
CampaignResolveOutcomeRequest,
|
||||||
|
CampaignDeliveryOptionsResponse,
|
||||||
|
MockCampaignSendRequest,
|
||||||
|
MockCampaignSendResponse,
|
||||||
|
QueueCampaignRequest,
|
||||||
|
QueueCampaignResponse,
|
||||||
|
SendCampaignNowRequest,
|
||||||
|
SendCampaignNowResponse,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal, require_any_scope, require_scope
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignJob,
|
||||||
|
JobImapStatus,
|
||||||
|
JobQueueStatus,
|
||||||
|
JobSendStatus,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
postbox_integration,
|
||||||
|
)
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
from govoplan_campaign.backend.response_security import (
|
||||||
|
public_send_campaign_now_result,
|
||||||
|
send_campaign_now_audit_details,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import (
|
||||||
|
CampaignPersistenceError,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.persistence.versions import (
|
||||||
|
is_user_locked_version,
|
||||||
|
)
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.dev.mock_campaign import (
|
||||||
|
MockCampaignSendError,
|
||||||
|
run_mock_campaign_send,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError
|
||||||
|
from govoplan_campaign.backend.sending.jobs import (
|
||||||
|
QueueingError,
|
||||||
|
SynchronousSendRejected,
|
||||||
|
cancel_campaign_jobs,
|
||||||
|
enqueue_pending_imap_appends,
|
||||||
|
pause_campaign_jobs,
|
||||||
|
queue_campaign_jobs,
|
||||||
|
queue_failed_jobs_for_retry,
|
||||||
|
queue_unattempted_jobs,
|
||||||
|
reconcile_job_outcome,
|
||||||
|
resume_campaign_jobs,
|
||||||
|
send_campaign_now,
|
||||||
|
send_single_campaign_job,
|
||||||
|
synchronous_send_options,
|
||||||
|
)
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_get_campaign_for_tenant,
|
||||||
|
_get_version_for_tenant,
|
||||||
|
_require_campaign_profile_use_if_needed,
|
||||||
|
_require_campaign_versions_profile_use,
|
||||||
|
_require_mail_profile_use_if_needed,
|
||||||
|
_require_permission,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/{campaign_id}/delivery-options", response_model=CampaignDeliveryOptionsResponse
|
||||||
|
)
|
||||||
|
def campaign_delivery_options(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(
|
||||||
|
require_any_scope("campaigns:campaign:send", "campaigns:campaign:queue")
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
try:
|
||||||
|
return CampaignDeliveryOptionsResponse(
|
||||||
|
**synchronous_send_options(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
),
|
||||||
|
postbox_available=postbox_integration().available,
|
||||||
|
)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/queue", response_model=QueueCampaignResponse)
|
||||||
|
def queue_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: QueueCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
payload = payload or QueueCampaignRequest()
|
||||||
|
_require_campaign_profile_use_if_needed(
|
||||||
|
session, principal, campaign_id, payload.version_id
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = queue_campaign_jobs(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
include_warnings=payload.include_warnings,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.queued"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.queue_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result.as_dict(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return QueueCampaignResponse(**result.as_dict())
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/jobs/retry", response_model=CampaignActionResponse)
|
||||||
|
def retry_campaign_jobs(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: CampaignRetryJobsRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:retry")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
payload = payload or CampaignRetryJobsRequest()
|
||||||
|
_require_campaign_profile_use_if_needed(
|
||||||
|
session, principal, campaign_id, payload.version_id
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = queue_failed_jobs_for_retry(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
job_ids=payload.job_ids or None,
|
||||||
|
include_permanent=payload.include_permanent,
|
||||||
|
force_max_attempts=payload.force_max_attempts,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.jobs_retry_queued"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.jobs_retry_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/jobs/send-unattempted", response_model=CampaignActionResponse
|
||||||
|
)
|
||||||
|
def send_unattempted_campaign_jobs(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: CampaignSendUnattemptedRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
payload = payload or CampaignSendUnattemptedRequest()
|
||||||
|
_require_campaign_profile_use_if_needed(
|
||||||
|
session, principal, campaign_id, payload.version_id
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = queue_unattempted_jobs(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
job_ids=payload.job_ids or None,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.unattempted_jobs_queued"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.unattempted_jobs_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/jobs/{job_id}/send", response_model=CampaignActionResponse)
|
||||||
|
def send_single_campaign_job_endpoint(
|
||||||
|
campaign_id: str,
|
||||||
|
job_id: str,
|
||||||
|
payload: CampaignSendJobRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(
|
||||||
|
require_any_scope(
|
||||||
|
"campaigns:campaign:send",
|
||||||
|
"campaigns:campaign:send_test",
|
||||||
|
)
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
_require_permission(
|
||||||
|
principal,
|
||||||
|
(
|
||||||
|
"campaigns:campaign:send_test"
|
||||||
|
if payload.kind == "test"
|
||||||
|
else "campaigns:campaign:send"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_require_campaign_profile_use_if_needed(session, principal, campaign_id, None)
|
||||||
|
try:
|
||||||
|
result = send_single_campaign_job(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
job_id=job_id,
|
||||||
|
kind=payload.kind,
|
||||||
|
idempotency_key=payload.idempotency_key,
|
||||||
|
actor_user_id=principal.user.id,
|
||||||
|
actor_api_key_id=getattr(principal, "api_key_id", None),
|
||||||
|
reason=payload.reason,
|
||||||
|
action_context=payload.context,
|
||||||
|
include_warnings=payload.include_warnings,
|
||||||
|
use_rate_limit=payload.use_rate_limit,
|
||||||
|
enqueue_imap_task=payload.enqueue_imap_task,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action=f"campaign.message_{payload.kind}",
|
||||||
|
object_type="campaign_job",
|
||||||
|
object_id=job_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception(
|
||||||
|
"Unexpected single-message campaign action failure",
|
||||||
|
extra={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"job_id": job_id,
|
||||||
|
"action_kind": payload.kind,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
|
detail="The message action failed because of an internal error.",
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/jobs/{job_id}/resolve-outcome",
|
||||||
|
response_model=CampaignActionResponse,
|
||||||
|
)
|
||||||
|
def resolve_campaign_job_outcome(
|
||||||
|
campaign_id: str,
|
||||||
|
job_id: str,
|
||||||
|
payload: CampaignResolveOutcomeRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:reconcile")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
try:
|
||||||
|
result = reconcile_job_outcome(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
job_id=job_id,
|
||||||
|
decision=payload.decision,
|
||||||
|
note=payload.note,
|
||||||
|
attempt_id=payload.attempt_id,
|
||||||
|
commit=False,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.job_outcome_reconciled",
|
||||||
|
object_type="campaign_job",
|
||||||
|
object_id=job_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/mock-send", response_model=MockCampaignSendResponse)
|
||||||
|
def mock_send_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: MockCampaignSendRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send_test")),
|
||||||
|
):
|
||||||
|
"""Run a fully visible mock delivery flow without mutating campaign state.
|
||||||
|
|
||||||
|
The route validates and builds the selected version, then optionally records
|
||||||
|
mock SMTP deliveries and mock IMAP appends. It never talks to the configured
|
||||||
|
real SMTP/IMAP servers and it does not mark the version sent/final.
|
||||||
|
"""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
|
||||||
|
payload = payload or MockCampaignSendRequest()
|
||||||
|
_require_campaign_profile_use_if_needed(
|
||||||
|
session, principal, campaign_id, payload.version_id
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = run_mock_campaign_send(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
send=payload.send,
|
||||||
|
include_warnings=payload.include_warnings,
|
||||||
|
include_needs_review=payload.include_needs_review,
|
||||||
|
append_sent=payload.append_sent,
|
||||||
|
clear_mailbox=payload.clear_mailbox,
|
||||||
|
check_files=payload.check_files,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.mock_send"
|
||||||
|
if payload.send
|
||||||
|
else "campaign.mock_send_review",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details={
|
||||||
|
"version_id": result.get("version_id"),
|
||||||
|
"send_requested": payload.send,
|
||||||
|
"sent_count": result.get("send", {}).get("sent_count"),
|
||||||
|
"failed_count": result.get("send", {}).get("failed_count"),
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return MockCampaignSendResponse(result=result)
|
||||||
|
except MockCampaignSendError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/send-now", response_model=SendCampaignNowResponse)
|
||||||
|
def send_campaign_now_endpoint(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: SendCampaignNowRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
|
||||||
|
):
|
||||||
|
"""Preflight and synchronously send a policy-bounded built execution."""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
|
||||||
|
payload = payload or SendCampaignNowRequest()
|
||||||
|
try:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
version_id = payload.version_id or campaign.current_version_id
|
||||||
|
if not version_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail="Campaign has no current version",
|
||||||
|
)
|
||||||
|
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
validation_result: dict[str, object] | None = (
|
||||||
|
version.validation_summary
|
||||||
|
if isinstance(version.validation_summary, dict)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
build_result: dict[str, object] | None = (
|
||||||
|
version.build_summary if isinstance(version.build_summary, dict) else None
|
||||||
|
)
|
||||||
|
if is_user_locked_version(version):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail="User-locked audit-safe versions cannot be dry-run or sent. Create an editable copy and validate it instead.",
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
not version.locked_at
|
||||||
|
or not validation_result
|
||||||
|
or validation_result.get("ok") is not True
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail="Campaign version must be validated and locked before dry-run or sending.",
|
||||||
|
)
|
||||||
|
if not build_result:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail="Campaign version must be built before dry-run or sending.",
|
||||||
|
)
|
||||||
|
|
||||||
|
delivery_result = send_campaign_now(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
include_warnings=payload.include_warnings,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
use_rate_limit=payload.use_rate_limit,
|
||||||
|
enqueue_imap_task=payload.enqueue_imap_task,
|
||||||
|
).as_dict()
|
||||||
|
response_result = public_send_campaign_now_result(
|
||||||
|
delivery_result,
|
||||||
|
validation_summary=validation_result,
|
||||||
|
build_summary=build_result,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.sent_now"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.send_now_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=send_campaign_now_audit_details(delivery_result),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return SendCampaignNowResponse(result=response_result)
|
||||||
|
except SynchronousSendRejected as exc:
|
||||||
|
# A synchronous request stages queue state before the all-message
|
||||||
|
# preflight can run. Rejecting that preflight must not leave work
|
||||||
|
# eligible for a background worker when no provider effect occurred.
|
||||||
|
session.rollback()
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.send_now_rejected",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details={
|
||||||
|
**exc.audit_details(),
|
||||||
|
"version_id": payload.version_id,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except (CampaignPersistenceError, QueueingError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/pause", response_model=CampaignActionResponse)
|
||||||
|
def pause_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
try:
|
||||||
|
result = pause_campaign_jobs(
|
||||||
|
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.paused",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/resume", response_model=CampaignActionResponse)
|
||||||
|
def resume_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
version_ids = {
|
||||||
|
row[0]
|
||||||
|
for row in session.query(CampaignJob.campaign_version_id)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == principal.tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.queue_status == JobQueueStatus.PAUSED.value,
|
||||||
|
)
|
||||||
|
.distinct()
|
||||||
|
.all()
|
||||||
|
}
|
||||||
|
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
|
||||||
|
try:
|
||||||
|
result = resume_campaign_jobs(
|
||||||
|
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.resumed",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/cancel", response_model=CampaignActionResponse)
|
||||||
|
def cancel_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
try:
|
||||||
|
result = cancel_campaign_jobs(
|
||||||
|
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.cancelled",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/append-sent", response_model=CampaignActionResponse)
|
||||||
|
def append_sent(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: AppendSentRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
payload = payload or AppendSentRequest()
|
||||||
|
version_ids = {
|
||||||
|
row[0]
|
||||||
|
for row in session.query(CampaignJob.campaign_version_id)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == principal.tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.send_status.in_(
|
||||||
|
[JobSendStatus.SMTP_ACCEPTED.value, JobSendStatus.SENT.value]
|
||||||
|
),
|
||||||
|
CampaignJob.imap_status.in_(
|
||||||
|
[JobImapStatus.PENDING.value, JobImapStatus.FAILED.value]
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.distinct()
|
||||||
|
.all()
|
||||||
|
}
|
||||||
|
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
|
||||||
|
try:
|
||||||
|
result = enqueue_pending_imap_appends(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
run_inline=payload.run_inline,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.append_sent_enqueued"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "campaign.append_sent_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
508
src/govoplan_campaign/backend/routes/jobs.py
Normal file
508
src/govoplan_campaign/backend/routes/jobs.py
Normal file
@@ -0,0 +1,508 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
CampaignJobsResponse,
|
||||||
|
CampaignJobsDeltaResponse,
|
||||||
|
CampaignJobDetailResponse,
|
||||||
|
CampaignJobDiagnosticsResponse,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||||
|
from govoplan_core.core.change_sequence import (
|
||||||
|
decode_sequence_watermark,
|
||||||
|
encode_sequence_watermark,
|
||||||
|
sequence_entries_since,
|
||||||
|
sequence_watermark_is_expired,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.change_tracking import (
|
||||||
|
CAMPAIGNS_MODULE_ID,
|
||||||
|
CAMPAIGN_JOBS_COLLECTION,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignJob,
|
||||||
|
CampaignMessageAction,
|
||||||
|
CampaignMessageActionAttempt,
|
||||||
|
ImapAppendAttempt,
|
||||||
|
PostboxDeliveryAttempt,
|
||||||
|
SendAttempt,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import postbox_integration
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_get_campaign_for_tenant,
|
||||||
|
_require_permission,
|
||||||
|
job_attempt_rows as _job_attempt_rows,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.services.job_queries import (
|
||||||
|
CampaignJobsQuery,
|
||||||
|
_campaign_jobs_delta_watermark,
|
||||||
|
_campaign_jobs_page_response,
|
||||||
|
_campaign_jobs_query_context,
|
||||||
|
_job_attempts_payload,
|
||||||
|
_job_detail_payload,
|
||||||
|
_job_diagnostics_payload,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
|
||||||
|
|
||||||
|
def _postbox_receipts_for_attempts(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
attempts: list[PostboxDeliveryAttempt],
|
||||||
|
):
|
||||||
|
integration = postbox_integration()
|
||||||
|
if not integration.receipt_evidence_available:
|
||||||
|
return None
|
||||||
|
delivery_ids = [
|
||||||
|
attempt.provider_delivery_id
|
||||||
|
for attempt in attempts
|
||||||
|
if attempt.provider_delivery_id
|
||||||
|
]
|
||||||
|
return integration.delivery_receipt_summaries(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
delivery_ids=delivery_ids,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/jobs", response_model=CampaignJobsResponse)
|
||||||
|
def list_jobs(
|
||||||
|
campaign_id: str,
|
||||||
|
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
"""Return a lightweight, paginated job list with server-side filters.
|
||||||
|
|
||||||
|
Complete recipients, attachment metadata, issues and attempt history are
|
||||||
|
available from the separate job-detail endpoint.
|
||||||
|
"""
|
||||||
|
|
||||||
|
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||||
|
_campaign_jobs_query_context(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return _campaign_jobs_page_response(
|
||||||
|
session,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
base_filters=base_filters,
|
||||||
|
filtered=filtered,
|
||||||
|
reviewed_keys=reviewed_keys,
|
||||||
|
review_metadata=review_metadata,
|
||||||
|
page=filters.page,
|
||||||
|
page_size=filters.page_size,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
cursor=filters.cursor,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_full_delta_response(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None,
|
||||||
|
page: int,
|
||||||
|
page_size: int,
|
||||||
|
send_status: list[str] | None,
|
||||||
|
validation_status: list[str] | None,
|
||||||
|
imap_status: list[str] | None,
|
||||||
|
query_text: str | None,
|
||||||
|
grid_filters: dict[str, str] | None,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
cursor: str | None = None,
|
||||||
|
) -> CampaignJobsDeltaResponse:
|
||||||
|
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||||
|
_campaign_jobs_query_context(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
send_status=send_status,
|
||||||
|
validation_status=validation_status,
|
||||||
|
imap_status=imap_status,
|
||||||
|
query_text=query_text,
|
||||||
|
grid_filters=grid_filters,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
payload = _campaign_jobs_page_response(
|
||||||
|
session,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
base_filters=base_filters,
|
||||||
|
filtered=filtered,
|
||||||
|
reviewed_keys=reviewed_keys,
|
||||||
|
review_metadata=review_metadata,
|
||||||
|
page=page,
|
||||||
|
page_size=page_size,
|
||||||
|
send_status=send_status,
|
||||||
|
validation_status=validation_status,
|
||||||
|
imap_status=imap_status,
|
||||||
|
query_text=query_text,
|
||||||
|
grid_filters=grid_filters,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
cursor=cursor,
|
||||||
|
)
|
||||||
|
return CampaignJobsDeltaResponse(
|
||||||
|
**payload.model_dump(),
|
||||||
|
deleted=[],
|
||||||
|
watermark=_campaign_jobs_delta_watermark(session, principal.tenant_id),
|
||||||
|
has_more=False,
|
||||||
|
full=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _job_filter_membership_can_shift(
|
||||||
|
*,
|
||||||
|
send_status: list[str] | None,
|
||||||
|
validation_status: list[str] | None,
|
||||||
|
imap_status: list[str] | None,
|
||||||
|
query_text: str | None,
|
||||||
|
grid_filters: dict[str, str] | None,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
) -> bool:
|
||||||
|
return bool(
|
||||||
|
send_status
|
||||||
|
or validation_status
|
||||||
|
or imap_status
|
||||||
|
or (query_text and query_text.strip())
|
||||||
|
or grid_filters
|
||||||
|
or sort_by != "number"
|
||||||
|
or sort_direction != "asc"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/jobs/delta", response_model=CampaignJobsDeltaResponse)
|
||||||
|
def list_jobs_delta(
|
||||||
|
campaign_id: str,
|
||||||
|
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
|
||||||
|
since: str | None = None,
|
||||||
|
limit: int = Query(default=500, ge=1, le=1000),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
if since is None:
|
||||||
|
return _campaign_jobs_full_delta_response(
|
||||||
|
session,
|
||||||
|
principal=principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
page=filters.page,
|
||||||
|
page_size=filters.page_size,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
cursor=filters.cursor,
|
||||||
|
)
|
||||||
|
|
||||||
|
campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||||
|
_campaign_jobs_query_context(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
since_sequence = decode_sequence_watermark(since)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
if sequence_watermark_is_expired(
|
||||||
|
session,
|
||||||
|
since=since_sequence,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
module_id=CAMPAIGNS_MODULE_ID,
|
||||||
|
collections=(CAMPAIGN_JOBS_COLLECTION,),
|
||||||
|
):
|
||||||
|
return _campaign_jobs_full_delta_response(
|
||||||
|
session,
|
||||||
|
principal=principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
page=filters.page,
|
||||||
|
page_size=filters.page_size,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
cursor=filters.cursor,
|
||||||
|
)
|
||||||
|
|
||||||
|
entries_plus_one = sequence_entries_since(
|
||||||
|
session,
|
||||||
|
since=since_sequence,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
module_id=CAMPAIGNS_MODULE_ID,
|
||||||
|
collections=(CAMPAIGN_JOBS_COLLECTION,),
|
||||||
|
limit=limit + 1,
|
||||||
|
)
|
||||||
|
has_more = len(entries_plus_one) > limit
|
||||||
|
entries = entries_plus_one[:limit]
|
||||||
|
relevant_entries = [
|
||||||
|
entry
|
||||||
|
for entry in entries
|
||||||
|
if (entry.payload or {}).get("campaign_id") == campaign.id
|
||||||
|
and (
|
||||||
|
not filters.version_id
|
||||||
|
or (entry.payload or {}).get("version_id") == filters.version_id
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
if relevant_entries and (
|
||||||
|
_job_filter_membership_can_shift(
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
)
|
||||||
|
or any(entry.operation in {"created", "deleted"} for entry in relevant_entries)
|
||||||
|
):
|
||||||
|
return _campaign_jobs_full_delta_response(
|
||||||
|
session,
|
||||||
|
principal=principal,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
page=filters.page,
|
||||||
|
page_size=filters.page_size,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
cursor=filters.cursor,
|
||||||
|
)
|
||||||
|
|
||||||
|
changed_job_ids = {
|
||||||
|
entry.resource_id
|
||||||
|
for entry in relevant_entries
|
||||||
|
if entry.resource_type == "campaign_job" and entry.operation != "deleted"
|
||||||
|
}
|
||||||
|
payload = _campaign_jobs_page_response(
|
||||||
|
session,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=filters.version_id,
|
||||||
|
base_filters=base_filters,
|
||||||
|
filtered=filtered,
|
||||||
|
reviewed_keys=reviewed_keys,
|
||||||
|
review_metadata=review_metadata,
|
||||||
|
page=filters.page,
|
||||||
|
page_size=filters.page_size,
|
||||||
|
send_status=filters.send_status,
|
||||||
|
validation_status=filters.validation_status,
|
||||||
|
imap_status=filters.imap_status,
|
||||||
|
query_text=filters.query_text,
|
||||||
|
grid_filters=filters.grid_filters,
|
||||||
|
sort_by=filters.sort_by,
|
||||||
|
sort_direction=filters.sort_direction,
|
||||||
|
cursor=filters.cursor,
|
||||||
|
changed_job_ids=changed_job_ids,
|
||||||
|
)
|
||||||
|
watermark = (
|
||||||
|
encode_sequence_watermark(entries[-1].id)
|
||||||
|
if has_more and entries
|
||||||
|
else _campaign_jobs_delta_watermark(session, principal.tenant_id)
|
||||||
|
)
|
||||||
|
return CampaignJobsDeltaResponse(
|
||||||
|
**payload.model_dump(),
|
||||||
|
deleted=[],
|
||||||
|
watermark=watermark,
|
||||||
|
has_more=has_more,
|
||||||
|
full=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/jobs/{job_id}", response_model=CampaignJobDetailResponse)
|
||||||
|
def get_job_detail(
|
||||||
|
campaign_id: str,
|
||||||
|
job_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if (
|
||||||
|
not job
|
||||||
|
or job.campaign_id != campaign.id
|
||||||
|
or job.tenant_id != principal.tenant_id
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
|
||||||
|
)
|
||||||
|
send_attempts = _job_attempt_rows(
|
||||||
|
session.query(SendAttempt)
|
||||||
|
.filter(SendAttempt.job_id == job.id)
|
||||||
|
.order_by(SendAttempt.attempt_number.asc()),
|
||||||
|
label="SMTP attempts for this campaign job",
|
||||||
|
)
|
||||||
|
imap_attempts = _job_attempt_rows(
|
||||||
|
session.query(ImapAppendAttempt)
|
||||||
|
.filter(ImapAppendAttempt.job_id == job.id)
|
||||||
|
.order_by(ImapAppendAttempt.attempt_number.asc()),
|
||||||
|
label="IMAP attempts for this campaign job",
|
||||||
|
)
|
||||||
|
postbox_attempts = _job_attempt_rows(
|
||||||
|
session.query(PostboxDeliveryAttempt)
|
||||||
|
.filter(PostboxDeliveryAttempt.job_id == job.id)
|
||||||
|
.order_by(
|
||||||
|
PostboxDeliveryAttempt.target_index.asc(),
|
||||||
|
PostboxDeliveryAttempt.attempt_number.asc(),
|
||||||
|
),
|
||||||
|
label="Postbox attempts for this campaign job",
|
||||||
|
)
|
||||||
|
message_actions = _job_attempt_rows(
|
||||||
|
session.query(CampaignMessageAction)
|
||||||
|
.filter(CampaignMessageAction.job_id == job.id)
|
||||||
|
.order_by(CampaignMessageAction.created_at.asc()),
|
||||||
|
label="Single-message actions for this campaign job",
|
||||||
|
)
|
||||||
|
action_ids = [action.id for action in message_actions]
|
||||||
|
message_action_attempts = (
|
||||||
|
_job_attempt_rows(
|
||||||
|
session.query(CampaignMessageActionAttempt)
|
||||||
|
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
|
||||||
|
.order_by(CampaignMessageActionAttempt.started_at.asc()),
|
||||||
|
label="Single-message action attempts for this campaign job",
|
||||||
|
)
|
||||||
|
if action_ids
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
return CampaignJobDetailResponse(
|
||||||
|
job=_job_detail_payload(job),
|
||||||
|
attempts=_job_attempts_payload(
|
||||||
|
send_attempts,
|
||||||
|
imap_attempts,
|
||||||
|
postbox_attempts,
|
||||||
|
message_actions,
|
||||||
|
message_action_attempts,
|
||||||
|
postbox_receipts=_postbox_receipts_for_attempts(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
attempts=postbox_attempts,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/{campaign_id}/jobs/{job_id}/diagnostics",
|
||||||
|
response_model=CampaignJobDiagnosticsResponse,
|
||||||
|
)
|
||||||
|
def get_job_diagnostics(
|
||||||
|
campaign_id: str,
|
||||||
|
job_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:diagnostic:read")),
|
||||||
|
):
|
||||||
|
"""Return infrastructure details only to campaign operators/admins."""
|
||||||
|
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if (
|
||||||
|
not job
|
||||||
|
or job.campaign_id != campaign.id
|
||||||
|
or job.tenant_id != principal.tenant_id
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
|
||||||
|
)
|
||||||
|
send_attempts = _job_attempt_rows(
|
||||||
|
session.query(SendAttempt)
|
||||||
|
.filter(SendAttempt.job_id == job.id)
|
||||||
|
.order_by(SendAttempt.attempt_number.asc()),
|
||||||
|
label="SMTP diagnostics for this campaign job",
|
||||||
|
)
|
||||||
|
imap_attempts = _job_attempt_rows(
|
||||||
|
session.query(ImapAppendAttempt)
|
||||||
|
.filter(ImapAppendAttempt.job_id == job.id)
|
||||||
|
.order_by(ImapAppendAttempt.attempt_number.asc()),
|
||||||
|
label="IMAP diagnostics for this campaign job",
|
||||||
|
)
|
||||||
|
postbox_attempts = _job_attempt_rows(
|
||||||
|
session.query(PostboxDeliveryAttempt)
|
||||||
|
.filter(PostboxDeliveryAttempt.job_id == job.id)
|
||||||
|
.order_by(
|
||||||
|
PostboxDeliveryAttempt.target_index.asc(),
|
||||||
|
PostboxDeliveryAttempt.attempt_number.asc(),
|
||||||
|
),
|
||||||
|
label="Postbox diagnostics for this campaign job",
|
||||||
|
)
|
||||||
|
message_actions = _job_attempt_rows(
|
||||||
|
session.query(CampaignMessageAction)
|
||||||
|
.filter(CampaignMessageAction.job_id == job.id)
|
||||||
|
.order_by(CampaignMessageAction.created_at.asc()),
|
||||||
|
label="Single-message action diagnostics for this campaign job",
|
||||||
|
)
|
||||||
|
action_ids = [action.id for action in message_actions]
|
||||||
|
message_action_attempts = (
|
||||||
|
_job_attempt_rows(
|
||||||
|
session.query(CampaignMessageActionAttempt)
|
||||||
|
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
|
||||||
|
.order_by(CampaignMessageActionAttempt.started_at.asc()),
|
||||||
|
label="Single-message action-attempt diagnostics for this campaign job",
|
||||||
|
)
|
||||||
|
if action_ids
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
return _job_diagnostics_payload(
|
||||||
|
job,
|
||||||
|
send_attempts,
|
||||||
|
imap_attempts,
|
||||||
|
postbox_attempts,
|
||||||
|
message_actions,
|
||||||
|
message_action_attempts,
|
||||||
|
postbox_receipts=_postbox_receipts_for_attempts(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
attempts=postbox_attempts,
|
||||||
|
),
|
||||||
|
)
|
||||||
257
src/govoplan_campaign/backend/routes/reports.py
Normal file
257
src/govoplan_campaign/backend/routes/reports.py
Normal file
@@ -0,0 +1,257 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Response, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
ReportEmailRequest,
|
||||||
|
ReportEmailResponse,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
from govoplan_campaign.backend.reports.campaigns import (
|
||||||
|
CampaignReportError,
|
||||||
|
generate_campaign_report,
|
||||||
|
generate_jobs_csv,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.reports.emailing import (
|
||||||
|
CampaignReportEmailError,
|
||||||
|
send_campaign_report_email,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
MailDeliveryCommandError,
|
||||||
|
MailProfileError,
|
||||||
|
SmtpConfigurationError,
|
||||||
|
SmtpSendError,
|
||||||
|
mail_integration,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_require_mail_profile_use_if_needed,
|
||||||
|
_require_permission,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _enqueue_mail_command() -> None:
|
||||||
|
try:
|
||||||
|
from govoplan_core.celery_app import celery
|
||||||
|
from govoplan_core.settings import settings
|
||||||
|
|
||||||
|
if settings.celery_enabled:
|
||||||
|
celery.send_task(
|
||||||
|
"govoplan.mail.dispatch_outbox",
|
||||||
|
args=[None, 25],
|
||||||
|
queue="mail",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"Mail delivery command is durable but immediate worker wake-up failed",
|
||||||
|
exc_info=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/summary")
|
||||||
|
def campaign_summary(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
include_jobs: bool = False,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
"""Return dashboard-friendly campaign status counters and summaries."""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
if include_jobs:
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
|
||||||
|
try:
|
||||||
|
return generate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
include_jobs=include_jobs,
|
||||||
|
include_recent_failures=include_jobs,
|
||||||
|
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||||
|
)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/report")
|
||||||
|
def campaign_report(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
include_jobs: bool = False,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
|
||||||
|
):
|
||||||
|
"""Return the recipient-level JSON report for one campaign."""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
|
||||||
|
try:
|
||||||
|
return generate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
include_jobs=include_jobs,
|
||||||
|
include_recent_failures=include_jobs,
|
||||||
|
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||||
|
)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/report/jobs.csv")
|
||||||
|
def campaign_jobs_csv(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:report:export")),
|
||||||
|
):
|
||||||
|
"""Export per-job campaign status as CSV."""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:export")
|
||||||
|
|
||||||
|
try:
|
||||||
|
csv_text = generate_jobs_csv(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||||
|
)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
return Response(
|
||||||
|
content=csv_text,
|
||||||
|
media_type="text/csv; charset=utf-8",
|
||||||
|
headers={
|
||||||
|
"Content-Disposition": f'attachment; filename="campaign-{campaign_id}-jobs.csv"'
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/report/email", response_model=ReportEmailResponse)
|
||||||
|
def email_campaign_report(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: ReportEmailRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:report:send")),
|
||||||
|
):
|
||||||
|
"""Generate a campaign report and send it to one or more email addresses."""
|
||||||
|
campaign = _get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:export")
|
||||||
|
selected_version_id = payload.version_id or campaign.current_version_id
|
||||||
|
selected_version = (
|
||||||
|
session.get(CampaignVersion, selected_version_id)
|
||||||
|
if selected_version_id
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
if selected_version is not None and selected_version.campaign_id == campaign.id:
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal,
|
||||||
|
selected_version.raw_json
|
||||||
|
if isinstance(selected_version.raw_json, dict)
|
||||||
|
else {},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = send_campaign_report_email(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
to=payload.to,
|
||||||
|
include_jobs=payload.include_jobs,
|
||||||
|
attach_jobs_csv=payload.attach_jobs_csv,
|
||||||
|
attach_report_json=payload.attach_report_json,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
idempotency_key=payload.idempotency_key,
|
||||||
|
created_by_user_id=principal.user.id,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="report.email_requested"
|
||||||
|
if not payload.dry_run
|
||||||
|
else "report.email_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result.audit_dict(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
if not payload.dry_run:
|
||||||
|
_enqueue_mail_command()
|
||||||
|
return ReportEmailResponse(result=result.as_dict())
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except (
|
||||||
|
CampaignReportEmailError,
|
||||||
|
MailProfileError,
|
||||||
|
MailDeliveryCommandError,
|
||||||
|
SmtpConfigurationError,
|
||||||
|
SmtpSendError,
|
||||||
|
) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error("Campaign report email failed with an unexpected internal error")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
|
detail="Campaign report email could not be completed.",
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/{campaign_id}/report/email/{command_id}",
|
||||||
|
response_model=ReportEmailResponse,
|
||||||
|
)
|
||||||
|
def campaign_report_email_status(
|
||||||
|
campaign_id: str,
|
||||||
|
command_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
try:
|
||||||
|
result = mail_integration().delivery_command_summary(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
command_id=command_id,
|
||||||
|
)
|
||||||
|
except MailDeliveryCommandError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=str(exc),
|
||||||
|
) from exc
|
||||||
|
if (
|
||||||
|
result.get("source_module") != "campaigns"
|
||||||
|
or result.get("source_resource_type") != "campaign"
|
||||||
|
or result.get("source_resource_id") != campaign_id
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail="Campaign report delivery not found",
|
||||||
|
)
|
||||||
|
return ReportEmailResponse(result=result)
|
||||||
288
src/govoplan_campaign/backend/routes/sharing.py
Normal file
288
src/govoplan_campaign/backend/routes/sharing.py
Normal file
@@ -0,0 +1,288 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_core.api.v1.schemas import (
|
||||||
|
ReferenceOptionListResponse,
|
||||||
|
ReferenceOptionResponse,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
CampaignShareItem,
|
||||||
|
CampaignShareListResponse,
|
||||||
|
CampaignShareTargetItem,
|
||||||
|
CampaignShareTargetsResponse,
|
||||||
|
CampaignShareUpsertRequest,
|
||||||
|
CampaignOwnerUpdateRequest,
|
||||||
|
CampaignResponse,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignShare,
|
||||||
|
)
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
from govoplan_core.core.references import (
|
||||||
|
access_scope_reference_page,
|
||||||
|
access_scope_reference_provider_available,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.runtime import get_registry
|
||||||
|
from govoplan_core.security.time import utc_now
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_access_directory,
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/{campaign_id}/share-target-options",
|
||||||
|
response_model=ReferenceOptionListResponse,
|
||||||
|
)
|
||||||
|
def search_campaign_share_targets(
|
||||||
|
campaign_id: str,
|
||||||
|
target_type: Literal["user", "group"],
|
||||||
|
q: str = "",
|
||||||
|
selected: list[str] = Query(default=[]),
|
||||||
|
limit: int = Query(default=50, ge=1, le=200),
|
||||||
|
cursor: str | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
) -> ReferenceOptionListResponse:
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
registry = get_registry()
|
||||||
|
try:
|
||||||
|
page = access_scope_reference_page(
|
||||||
|
registry,
|
||||||
|
principal,
|
||||||
|
scope_type=target_type,
|
||||||
|
reference_kind="membership" if target_type == "user" else "group",
|
||||||
|
query=q,
|
||||||
|
selected_values=selected,
|
||||||
|
limit=limit,
|
||||||
|
cursor=cursor,
|
||||||
|
administrative=True,
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=str(exc),
|
||||||
|
) from exc
|
||||||
|
return ReferenceOptionListResponse(
|
||||||
|
options=[
|
||||||
|
ReferenceOptionResponse(**option.to_dict())
|
||||||
|
for option in page.options
|
||||||
|
],
|
||||||
|
provider_available=access_scope_reference_provider_available(registry),
|
||||||
|
next_cursor=page.next_cursor,
|
||||||
|
has_more=page.has_more,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/share-targets", response_model=CampaignShareTargetsResponse)
|
||||||
|
def list_campaign_share_targets(
|
||||||
|
campaign_id: str,
|
||||||
|
limit: int = Query(default=500, ge=1, le=1000),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
directory = _access_directory()
|
||||||
|
users = [
|
||||||
|
user
|
||||||
|
for user in directory.users_for_tenant(principal.tenant_id)
|
||||||
|
if user.status == "active"
|
||||||
|
]
|
||||||
|
groups = [
|
||||||
|
group
|
||||||
|
for group in directory.groups_for_tenant(principal.tenant_id)
|
||||||
|
if group.status == "active"
|
||||||
|
]
|
||||||
|
if len(users) > limit or len(groups) > limit:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
|
||||||
|
detail=(
|
||||||
|
f"Campaign share targets exceed the maximum response size of {limit} "
|
||||||
|
"users or groups. Use a searchable directory selector."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return CampaignShareTargetsResponse(
|
||||||
|
users=[
|
||||||
|
CampaignShareTargetItem(
|
||||||
|
id=item.id, name=item.display_name or item.email, secondary=item.email
|
||||||
|
)
|
||||||
|
for item in users
|
||||||
|
],
|
||||||
|
groups=[
|
||||||
|
CampaignShareTargetItem(id=item.id, name=item.name, secondary=None)
|
||||||
|
for item in groups
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/shares", response_model=CampaignShareListResponse)
|
||||||
|
def list_campaign_shares(
|
||||||
|
campaign_id: str,
|
||||||
|
page: int = Query(default=1, ge=1),
|
||||||
|
page_size: int = Query(default=500, ge=1, le=1000),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
):
|
||||||
|
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
query = session.query(CampaignShare).filter(
|
||||||
|
CampaignShare.tenant_id == principal.tenant_id,
|
||||||
|
CampaignShare.campaign_id == campaign.id,
|
||||||
|
CampaignShare.revoked_at.is_(None),
|
||||||
|
)
|
||||||
|
total = query.order_by(None).count()
|
||||||
|
pages = max(1, (total + page_size - 1) // page_size)
|
||||||
|
shares = (
|
||||||
|
query.order_by(
|
||||||
|
CampaignShare.target_type.asc(),
|
||||||
|
CampaignShare.target_id.asc(),
|
||||||
|
CampaignShare.id.asc(),
|
||||||
|
)
|
||||||
|
.offset((page - 1) * page_size)
|
||||||
|
.limit(page_size)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return CampaignShareListResponse(
|
||||||
|
shares=[CampaignShareItem.model_validate(item) for item in shares],
|
||||||
|
total=total,
|
||||||
|
page=page,
|
||||||
|
page_size=page_size,
|
||||||
|
pages=pages,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/{campaign_id}/owner", response_model=CampaignResponse)
|
||||||
|
def update_campaign_owner(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: CampaignOwnerUpdateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
):
|
||||||
|
del payload
|
||||||
|
_get_campaign_for_principal(
|
||||||
|
session,
|
||||||
|
campaign_id,
|
||||||
|
principal,
|
||||||
|
write=True,
|
||||||
|
)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail=(
|
||||||
|
"Direct campaign owner mutation has been retired. Use the "
|
||||||
|
"ownership transfer workflow so the target can accept."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/shares",
|
||||||
|
response_model=CampaignShareItem,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
)
|
||||||
|
def upsert_campaign_share(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: CampaignShareUpsertRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
):
|
||||||
|
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
directory = _access_directory()
|
||||||
|
if payload.target_type == "user":
|
||||||
|
target = directory.get_user(payload.target_id)
|
||||||
|
if target is not None and (
|
||||||
|
target.tenant_id != principal.tenant_id or target.status != "active"
|
||||||
|
):
|
||||||
|
target = None
|
||||||
|
else:
|
||||||
|
target = directory.get_group(payload.target_id)
|
||||||
|
if target is not None and (
|
||||||
|
target.tenant_id != principal.tenant_id or target.status != "active"
|
||||||
|
):
|
||||||
|
target = None
|
||||||
|
if target is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Share target not found"
|
||||||
|
)
|
||||||
|
share = (
|
||||||
|
session.query(CampaignShare)
|
||||||
|
.filter(
|
||||||
|
CampaignShare.campaign_id == campaign.id,
|
||||||
|
CampaignShare.target_type == payload.target_type,
|
||||||
|
CampaignShare.target_id == payload.target_id,
|
||||||
|
)
|
||||||
|
.one_or_none()
|
||||||
|
)
|
||||||
|
if share is None:
|
||||||
|
share = CampaignShare(
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
target_type=payload.target_type,
|
||||||
|
target_id=payload.target_id,
|
||||||
|
permission=payload.permission,
|
||||||
|
created_by_user_id=principal.user.id,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
share.permission = payload.permission
|
||||||
|
share.revoked_at = None
|
||||||
|
session.add(share)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.share_upserted",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign.id,
|
||||||
|
details=payload.model_dump(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignShareItem.model_validate(share)
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete(
|
||||||
|
"/{campaign_id}/shares/{share_id}", status_code=status.HTTP_204_NO_CONTENT
|
||||||
|
)
|
||||||
|
def revoke_campaign_share(
|
||||||
|
campaign_id: str,
|
||||||
|
share_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||||
|
):
|
||||||
|
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
share = (
|
||||||
|
session.query(CampaignShare)
|
||||||
|
.filter(
|
||||||
|
CampaignShare.id == share_id,
|
||||||
|
CampaignShare.campaign_id == campaign.id,
|
||||||
|
CampaignShare.tenant_id == principal.tenant_id,
|
||||||
|
)
|
||||||
|
.one_or_none()
|
||||||
|
)
|
||||||
|
if share is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign share not found"
|
||||||
|
)
|
||||||
|
share.revoked_at = utc_now()
|
||||||
|
session.add(share)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.share_revoked",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign.id,
|
||||||
|
details={"share_id": share_id},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# Queue / delivery control -------------------------------------------------
|
||||||
685
src/govoplan_campaign/backend/routes/versions.py
Normal file
685
src/govoplan_campaign/backend/routes/versions.py
Normal file
@@ -0,0 +1,685 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Response, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
BuildCampaignRequest,
|
||||||
|
CampaignCreateResponse,
|
||||||
|
CampaignResponse,
|
||||||
|
CampaignVersionDetailResponse,
|
||||||
|
CampaignVersionResponse,
|
||||||
|
CampaignVersionSetStepRequest,
|
||||||
|
CampaignReviewStateRequest,
|
||||||
|
CampaignVersionUpdateRequest,
|
||||||
|
CampaignPartialValidationRequest,
|
||||||
|
CampaignPartialValidationResponse,
|
||||||
|
ValidateCampaignRequest,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||||
|
from govoplan_core.audit.logging import audit_from_principal
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
|
from govoplan_core.db.session import get_session
|
||||||
|
from govoplan_campaign.backend.response_security import (
|
||||||
|
public_campaign_payload,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import (
|
||||||
|
CampaignPersistenceError,
|
||||||
|
build_campaign_version,
|
||||||
|
validate_campaign_version,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
|
||||||
|
from govoplan_campaign.backend.persistence.versions import (
|
||||||
|
LockedCampaignVersionError,
|
||||||
|
fork_campaign_version_for_edit,
|
||||||
|
is_version_final_locked,
|
||||||
|
is_user_locked_version,
|
||||||
|
get_campaign_version_for_tenant,
|
||||||
|
lock_campaign_version_temporarily,
|
||||||
|
permanently_lock_campaign_version,
|
||||||
|
publish_campaign_version,
|
||||||
|
unlock_user_locked_campaign_version,
|
||||||
|
unlock_validated_campaign_version,
|
||||||
|
update_campaign_version,
|
||||||
|
update_campaign_review_state,
|
||||||
|
validate_campaign_partial,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_campaign_response_context,
|
||||||
|
_campaign_version_detail_response,
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_get_campaign_for_tenant,
|
||||||
|
_get_version_for_principal,
|
||||||
|
_get_version_for_tenant,
|
||||||
|
_require_mail_profile_use_if_needed,
|
||||||
|
_require_permission,
|
||||||
|
_update_campaign_version_detail_response,
|
||||||
|
_write_current_version_snapshot_if_available,
|
||||||
|
bounded_query_rows as _bounded_query_rows,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.routes.attachments import (
|
||||||
|
CampaignAttachmentLinkMatchesResponse,
|
||||||
|
_link_campaign_attachment_matches,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/versions", response_model=list[CampaignVersionResponse])
|
||||||
|
def list_versions(
|
||||||
|
campaign_id: str,
|
||||||
|
limit: int = Query(default=500, ge=1, le=1000),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
versions = _bounded_query_rows(
|
||||||
|
session.query(CampaignVersion)
|
||||||
|
.filter(CampaignVersion.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignVersion.version_number.desc()),
|
||||||
|
limit=limit,
|
||||||
|
label="Campaign version history",
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
CampaignVersionResponse.model_validate(
|
||||||
|
item,
|
||||||
|
context=_campaign_response_context(principal),
|
||||||
|
)
|
||||||
|
for item in versions
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
|
||||||
|
)
|
||||||
|
def get_version_detail(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
response: Response,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
try:
|
||||||
|
version = get_campaign_version_for_tenant(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
)
|
||||||
|
result = CampaignVersionDetailResponse.model_validate(
|
||||||
|
version,
|
||||||
|
context=_campaign_response_context(principal),
|
||||||
|
)
|
||||||
|
response.headers["ETag"] = version.strong_etag
|
||||||
|
return result
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/fork", response_model=CampaignCreateResponse
|
||||||
|
)
|
||||||
|
def fork_version_for_edit(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:copy")),
|
||||||
|
):
|
||||||
|
"""Create the campaign's next and only editable working version.
|
||||||
|
|
||||||
|
A new working copy may be created only after the current version is
|
||||||
|
permanently user-locked or delivery-final. Validation and temporary user
|
||||||
|
locks must be removed in place instead of creating parallel drafts.
|
||||||
|
"""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
|
||||||
|
payload = payload or CampaignVersionUpdateRequest()
|
||||||
|
source_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
if source_version.campaign_id != campaign_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||||
|
)
|
||||||
|
effective_json = (
|
||||||
|
payload.campaign_json
|
||||||
|
if isinstance(payload.campaign_json, dict)
|
||||||
|
else source_version.raw_json
|
||||||
|
)
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal,
|
||||||
|
effective_json if isinstance(effective_json, dict) else {},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
version = fork_campaign_version_for_edit(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
raw_json=payload.campaign_json,
|
||||||
|
current_flow=payload.current_flow or "manual",
|
||||||
|
current_step=payload.current_step,
|
||||||
|
editor_state=payload.editor_state,
|
||||||
|
source_filename=payload.source_filename,
|
||||||
|
source_base_path=payload.source_base_path,
|
||||||
|
autosave=True,
|
||||||
|
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
|
||||||
|
commit=False,
|
||||||
|
)
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_forked_for_edit",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"source_version_id": version_id,
|
||||||
|
"version_number": version.version_number,
|
||||||
|
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
_write_current_version_snapshot_if_available(version)
|
||||||
|
return CampaignCreateResponse(
|
||||||
|
campaign=CampaignResponse.model_validate(campaign),
|
||||||
|
version=CampaignVersionResponse.model_validate(
|
||||||
|
version,
|
||||||
|
context=_campaign_response_context(principal),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except LockedCampaignVersionError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except CampaignPathSecurityError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/unlock-validation",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def unlock_version_validation(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
"""Unlock a successfully validated version before delivery starts.
|
||||||
|
|
||||||
|
Unlocking invalidates validation/build state and removes generated jobs for
|
||||||
|
that version. Sent/final versions cannot be unlocked and must be copied.
|
||||||
|
"""
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: unlock_validated_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_validation_unlocked",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/lock-temporarily",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def lock_version_temporarily(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: lock_campaign_version_temporarily(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_user_locked_temporarily",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/unlock-user-lock",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def unlock_version_user_lock(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: unlock_user_locked_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_user_lock_removed",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/lock-permanently",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def lock_version_permanently(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: permanently_lock_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_user_locked_permanently",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.put(
|
||||||
|
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
|
||||||
|
)
|
||||||
|
def update_version_detail(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest,
|
||||||
|
response: Response,
|
||||||
|
if_match: str | None = Header(default=None, alias="If-Match"),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
result = _update_campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
version_id,
|
||||||
|
payload,
|
||||||
|
if_match=if_match,
|
||||||
|
autosave=False,
|
||||||
|
audit_action="campaign.version_updated",
|
||||||
|
)
|
||||||
|
response.headers["ETag"] = result.strong_etag
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/autosave",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def autosave_version(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest,
|
||||||
|
response: Response,
|
||||||
|
if_match: str | None = Header(default=None, alias="If-Match"),
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
result = _update_campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
version_id,
|
||||||
|
payload,
|
||||||
|
if_match=if_match,
|
||||||
|
autosave=True,
|
||||||
|
audit_action="campaign.version_autosaved",
|
||||||
|
)
|
||||||
|
response.headers["ETag"] = result.strong_etag
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/set-step",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def set_version_step(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionSetStepRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
autosave=True,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_step_updated",
|
||||||
|
details={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"current_flow": payload.current_flow,
|
||||||
|
"current_step": payload.current_step,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/review-state",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def set_version_review_state(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignReviewStateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:review")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
try:
|
||||||
|
version = update_campaign_review_state(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
inspection_complete=payload.inspection_complete,
|
||||||
|
reviewed_message_keys=payload.reviewed_message_keys,
|
||||||
|
issue_decisions=[
|
||||||
|
item.model_dump()
|
||||||
|
for item in payload.issue_decisions
|
||||||
|
],
|
||||||
|
user_id=principal.user.id,
|
||||||
|
commit=False,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.message_review_updated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"inspection_complete": payload.inspection_complete,
|
||||||
|
"reviewed_message_count": len(payload.reviewed_message_keys),
|
||||||
|
"issue_decision_count": len(payload.issue_decisions),
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(
|
||||||
|
version,
|
||||||
|
context=_campaign_response_context(principal),
|
||||||
|
)
|
||||||
|
except LockedCampaignVersionError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
session.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/validate-partial",
|
||||||
|
response_model=CampaignPartialValidationResponse,
|
||||||
|
)
|
||||||
|
def validate_version_partial(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignPartialValidationRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
try:
|
||||||
|
version = get_campaign_version_for_tenant(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
)
|
||||||
|
campaign_json = (
|
||||||
|
payload.campaign_json
|
||||||
|
if payload and payload.campaign_json is not None
|
||||||
|
else version.raw_json
|
||||||
|
)
|
||||||
|
result = validate_campaign_partial(
|
||||||
|
campaign_json, section=payload.section if payload else None
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_partially_validated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"section": result.get("section"),
|
||||||
|
"ok": result.get("ok"),
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignPartialValidationResponse(**result)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{campaign_id}/versions/{version_id}/publish",
|
||||||
|
response_model=CampaignVersionDetailResponse,
|
||||||
|
)
|
||||||
|
def publish_version(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||||
|
):
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||||
|
return _campaign_version_detail_response(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign_id,
|
||||||
|
lambda: publish_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
commit=False,
|
||||||
|
),
|
||||||
|
audit_action="campaign.version_user_locked_permanently",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/versions/{version_id}/validate")
|
||||||
|
def validate_version(
|
||||||
|
version_id: str,
|
||||||
|
payload: ValidateCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||||
|
):
|
||||||
|
_get_version_for_principal(session, version_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
payload = payload or ValidateCampaignRequest()
|
||||||
|
try:
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
if is_user_locked_version(version) or is_version_final_locked(version):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail="This version has a user lock or final delivery lock and cannot be validated. Remove a temporary lock or create an editable copy.",
|
||||||
|
)
|
||||||
|
link_result: CampaignAttachmentLinkMatchesResponse | None = None
|
||||||
|
if payload.check_files and payload.link_unshared_matches:
|
||||||
|
_require_permission(principal, "files:file:share")
|
||||||
|
campaign = _get_campaign_for_tenant(
|
||||||
|
session, version.campaign_id, principal.tenant_id
|
||||||
|
)
|
||||||
|
link_result = _link_campaign_attachment_matches(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
campaign=campaign,
|
||||||
|
version=version,
|
||||||
|
raw=version.raw_json if isinstance(version.raw_json, dict) else {},
|
||||||
|
dry_run=False,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.attachment_matches_linked",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={
|
||||||
|
"matched_file_count": link_result.matched_file_count,
|
||||||
|
"already_linked_file_count": link_result.already_linked_file_count,
|
||||||
|
"linked_file_count": link_result.linked_file_count,
|
||||||
|
"during_validation": True,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
result = validate_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
version_id=version_id,
|
||||||
|
check_files=payload.check_files,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.validated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={
|
||||||
|
"check_files": payload.check_files,
|
||||||
|
"link_unshared_matches": payload.link_unshared_matches,
|
||||||
|
"linked_file_count": link_result.linked_file_count
|
||||||
|
if link_result
|
||||||
|
else 0,
|
||||||
|
"ok": result.get("ok"),
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return public_campaign_payload(
|
||||||
|
result,
|
||||||
|
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||||
|
)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/versions/{version_id}/build")
|
||||||
|
def build_version(
|
||||||
|
version_id: str,
|
||||||
|
payload: BuildCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:build")),
|
||||||
|
):
|
||||||
|
version = _get_version_for_principal(session, version_id, principal, write=True)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
_require_mail_profile_use_if_needed(
|
||||||
|
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = build_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
version_id=version_id,
|
||||||
|
write_eml=payload.write_eml if payload else True,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.messages_built",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={
|
||||||
|
"write_eml": payload.write_eml if payload else True,
|
||||||
|
"built_count": result.get("built_count"),
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return public_campaign_payload(
|
||||||
|
result,
|
||||||
|
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||||
|
)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||||
|
) from exc
|
||||||
@@ -60,7 +60,9 @@
|
|||||||
"integer",
|
"integer",
|
||||||
"double",
|
"double",
|
||||||
"date",
|
"date",
|
||||||
"password"
|
"password",
|
||||||
|
"organization_unit",
|
||||||
|
"organization_function"
|
||||||
],
|
],
|
||||||
"default": "string"
|
"default": "string"
|
||||||
},
|
},
|
||||||
@@ -90,125 +92,29 @@
|
|||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
"mail_profile_id": {
|
"mail_profile_id": {
|
||||||
"type": "string"
|
"type": "string",
|
||||||
|
"minLength": 1,
|
||||||
|
"description": "Stable reference to an authorized server envelope owned by the Mail module. Campaign JSON never stores SMTP/IMAP settings or credentials."
|
||||||
},
|
},
|
||||||
"inherit_smtp_credentials": {
|
"smtp_server_id": {
|
||||||
"type": "boolean",
|
"type": "string",
|
||||||
"default": true
|
"minLength": 1,
|
||||||
|
"description": "Optional explicit Mail-owned SMTP server selection."
|
||||||
},
|
},
|
||||||
"inherit_imap_credentials": {
|
"smtp_credential_id": {
|
||||||
"type": "boolean",
|
"type": "string",
|
||||||
"default": true
|
"minLength": 1,
|
||||||
|
"description": "Optional explicit core credential envelope bound to the selected SMTP server."
|
||||||
},
|
},
|
||||||
"smtp": {
|
"imap_server_id": {
|
||||||
"type": "object",
|
"type": "string",
|
||||||
"properties": {
|
"minLength": 1,
|
||||||
"host": {
|
"description": "Optional explicit Mail-owned IMAP server selection."
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"port": {
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 1,
|
|
||||||
"maximum": 65535
|
|
||||||
},
|
|
||||||
"username": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"password": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"security": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"plain",
|
|
||||||
"tls",
|
|
||||||
"starttls"
|
|
||||||
],
|
|
||||||
"default": "starttls"
|
|
||||||
},
|
|
||||||
"timeout_seconds": {
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 1,
|
|
||||||
"default": 30
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
},
|
||||||
"imap": {
|
"imap_credential_id": {
|
||||||
"type": "object",
|
"type": "string",
|
||||||
"properties": {
|
"minLength": 1,
|
||||||
"enabled": {
|
"description": "Optional explicit core credential envelope bound to the selected IMAP server."
|
||||||
"type": "boolean",
|
|
||||||
"default": false
|
|
||||||
},
|
|
||||||
"host": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"port": {
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 1,
|
|
||||||
"maximum": 65535
|
|
||||||
},
|
|
||||||
"username": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"password": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"security": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"plain",
|
|
||||||
"tls",
|
|
||||||
"starttls"
|
|
||||||
],
|
|
||||||
"default": "tls"
|
|
||||||
},
|
|
||||||
"sent_folder": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "auto"
|
|
||||||
},
|
|
||||||
"timeout_seconds": {
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 1,
|
|
||||||
"default": 30
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
|
||||||
"credentials": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"smtp": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"username": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"password": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
|
||||||
"imap": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"username": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"password": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false,
|
|
||||||
"default": {
|
|
||||||
"smtp": {},
|
|
||||||
"imap": {}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"additionalProperties": false
|
"additionalProperties": false
|
||||||
@@ -419,7 +325,19 @@
|
|||||||
"send_without_attachments": {
|
"send_without_attachments": {
|
||||||
"type": "boolean",
|
"type": "boolean",
|
||||||
"default": true,
|
"default": true,
|
||||||
"description": "Legacy compatibility flag. Prefer validation_policy and per-config missing_behavior for new campaigns."
|
"description": "Legacy compatibility flag. Use send_without_attachments_behavior for new campaigns."
|
||||||
|
},
|
||||||
|
"send_without_attachments_behavior": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "continue",
|
||||||
|
"description": "Campaign-wide behavior when no attachment file is resolved for an active message."
|
||||||
},
|
},
|
||||||
"zip": {
|
"zip": {
|
||||||
"$ref": "#/$defs/zip_collection_config",
|
"$ref": "#/$defs/zip_collection_config",
|
||||||
@@ -441,7 +359,7 @@
|
|||||||
"continue",
|
"continue",
|
||||||
"warn"
|
"warn"
|
||||||
],
|
],
|
||||||
"default": "ask"
|
"default": "warn"
|
||||||
},
|
},
|
||||||
"ambiguous_behavior": {
|
"ambiguous_behavior": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
@@ -537,7 +455,7 @@
|
|||||||
"continue",
|
"continue",
|
||||||
"warn"
|
"warn"
|
||||||
],
|
],
|
||||||
"default": "ask"
|
"default": "block"
|
||||||
},
|
},
|
||||||
"missing_optional_attachment": {
|
"missing_optional_attachment": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
@@ -613,6 +531,12 @@
|
|||||||
"delivery": {
|
"delivery": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
|
"channel_policy": {
|
||||||
|
"$ref": "#/$defs/delivery_channel_policy"
|
||||||
|
},
|
||||||
|
"postbox": {
|
||||||
|
"$ref": "#/$defs/postbox_delivery"
|
||||||
|
},
|
||||||
"rate_limit": {
|
"rate_limit": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
@@ -730,6 +654,179 @@
|
|||||||
},
|
},
|
||||||
"additionalProperties": false
|
"additionalProperties": false
|
||||||
},
|
},
|
||||||
|
"delivery_channel_policy": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"mail",
|
||||||
|
"postbox",
|
||||||
|
"mail_and_postbox",
|
||||||
|
"mail_then_postbox",
|
||||||
|
"postbox_then_mail"
|
||||||
|
],
|
||||||
|
"default": "mail"
|
||||||
|
},
|
||||||
|
"postbox_target": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"id",
|
||||||
|
"mode"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"id": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1,
|
||||||
|
"maxLength": 120
|
||||||
|
},
|
||||||
|
"mode": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"direct",
|
||||||
|
"derived"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 500
|
||||||
|
},
|
||||||
|
"postbox_id": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 36
|
||||||
|
},
|
||||||
|
"address_key": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 500
|
||||||
|
},
|
||||||
|
"template_id": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 36
|
||||||
|
},
|
||||||
|
"organization_unit_id": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 36
|
||||||
|
},
|
||||||
|
"organization_unit_field": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 255
|
||||||
|
},
|
||||||
|
"organization_unit_match": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"id",
|
||||||
|
"slug"
|
||||||
|
],
|
||||||
|
"default": "id"
|
||||||
|
},
|
||||||
|
"function_id": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 36
|
||||||
|
},
|
||||||
|
"function_field": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 255
|
||||||
|
},
|
||||||
|
"function_match": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"id",
|
||||||
|
"slug"
|
||||||
|
],
|
||||||
|
"default": "id"
|
||||||
|
},
|
||||||
|
"context_key": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 255
|
||||||
|
},
|
||||||
|
"context_field": {
|
||||||
|
"type": [
|
||||||
|
"string",
|
||||||
|
"null"
|
||||||
|
],
|
||||||
|
"maxLength": 255
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"postbox_delivery": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"targets": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/postbox_target"
|
||||||
|
},
|
||||||
|
"maxItems": 50,
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"classification": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1,
|
||||||
|
"maxLength": 50,
|
||||||
|
"default": "internal"
|
||||||
|
},
|
||||||
|
"unresolved_target": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "block"
|
||||||
|
},
|
||||||
|
"vacant_target": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "warn"
|
||||||
|
},
|
||||||
|
"duplicate_target": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "warn"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
"attachment_config": {
|
"attachment_config": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"required": [
|
"required": [
|
||||||
@@ -973,6 +1070,29 @@
|
|||||||
"default": true,
|
"default": true,
|
||||||
"description": "Deprecated compatibility alias for merge_disposition_notification_to. New campaign JSON should use merge_*."
|
"description": "Deprecated compatibility alias for merge_disposition_notification_to. New campaign JSON should use merge_*."
|
||||||
},
|
},
|
||||||
|
"channel_policy": {
|
||||||
|
"oneOf": [
|
||||||
|
{
|
||||||
|
"$ref": "#/$defs/delivery_channel_policy"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "null"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"default": null
|
||||||
|
},
|
||||||
|
"postbox_targets": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/postbox_target"
|
||||||
|
},
|
||||||
|
"maxItems": 50,
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"merge_postbox_targets": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
"attachments": {
|
"attachments": {
|
||||||
"type": "array",
|
"type": "array",
|
||||||
"items": {
|
"items": {
|
||||||
|
|||||||
@@ -1,12 +1,20 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import Any, Literal
|
from typing import Annotated, Any, Literal
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
from pydantic import BaseModel, BeforeValidator, ConfigDict, Field, ValidationInfo, field_validator, model_validator
|
||||||
|
|
||||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||||
from govoplan_core.mail.config import ImapConfig, SmtpConfig
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
public_campaign_editor_state,
|
||||||
|
validate_campaign_editor_state,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.response_security import (
|
||||||
|
public_campaign_configuration,
|
||||||
|
public_campaign_payload,
|
||||||
|
public_source_filename,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class CampaignCreateRequest(BaseModel):
|
class CampaignCreateRequest(BaseModel):
|
||||||
@@ -49,6 +57,15 @@ class CampaignVersionUpdateRequest(BaseModel):
|
|||||||
editor_state: dict[str, Any] | None = None
|
editor_state: dict[str, Any] | None = None
|
||||||
source_filename: str | None = None
|
source_filename: str | None = None
|
||||||
source_base_path: str | None = None
|
source_base_path: str | None = None
|
||||||
|
migrate_legacy_mail_settings: bool = False
|
||||||
|
base_revision: int | None = Field(default=None, ge=1)
|
||||||
|
reconciliation_kind: Literal["none", "auto_merge", "manual"] = "none"
|
||||||
|
resolved_conflict_paths: list[str] = Field(default_factory=list, max_length=100)
|
||||||
|
|
||||||
|
@field_validator("editor_state")
|
||||||
|
@classmethod
|
||||||
|
def validate_editor_state(cls, value: dict[str, Any] | None) -> dict[str, Any] | None:
|
||||||
|
return validate_campaign_editor_state(value) if value is not None else None
|
||||||
|
|
||||||
|
|
||||||
class CampaignVersionSetStepRequest(BaseModel):
|
class CampaignVersionSetStepRequest(BaseModel):
|
||||||
@@ -58,11 +75,23 @@ class CampaignVersionSetStepRequest(BaseModel):
|
|||||||
current_step: str
|
current_step: str
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignReviewDecisionRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
job_id: str = Field(min_length=1, max_length=36)
|
||||||
|
decision: Literal["accept"] = "accept"
|
||||||
|
reason: str | None = Field(default=None, max_length=4_000)
|
||||||
|
|
||||||
|
|
||||||
class CampaignReviewStateRequest(BaseModel):
|
class CampaignReviewStateRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
inspection_complete: bool = False
|
inspection_complete: bool = False
|
||||||
reviewed_message_keys: list[str] = Field(default_factory=list)
|
reviewed_message_keys: list[str] = Field(default_factory=list)
|
||||||
|
issue_decisions: list[CampaignReviewDecisionRequest] = Field(
|
||||||
|
default_factory=list,
|
||||||
|
max_length=100_000,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class CampaignPartialValidationRequest(BaseModel):
|
class CampaignPartialValidationRequest(BaseModel):
|
||||||
@@ -78,9 +107,10 @@ class CampaignVersionResponse(BaseModel):
|
|||||||
id: str
|
id: str
|
||||||
campaign_id: str
|
campaign_id: str
|
||||||
version_number: int
|
version_number: int
|
||||||
|
edit_revision: int = 1
|
||||||
|
strong_etag: str = ""
|
||||||
schema_version: str
|
schema_version: str
|
||||||
source_filename: str | None = None
|
source_filename: str | None = None
|
||||||
source_base_path: str | None = None
|
|
||||||
workflow_state: str = "editing"
|
workflow_state: str = "editing"
|
||||||
current_flow: str = "manual"
|
current_flow: str = "manual"
|
||||||
current_step: str | None = None
|
current_step: str | None = None
|
||||||
@@ -99,10 +129,39 @@ class CampaignVersionResponse(BaseModel):
|
|||||||
build_summary: dict[str, Any] | None = None
|
build_summary: dict[str, Any] | None = None
|
||||||
execution_snapshot_hash: str | None = None
|
execution_snapshot_hash: str | None = None
|
||||||
execution_snapshot_at: datetime | None = None
|
execution_snapshot_at: datetime | None = None
|
||||||
|
delivery_mode: Literal["synchronous", "worker_queue", "database_queue"] | None = None
|
||||||
|
delivery_mode_selected_at: datetime | None = None
|
||||||
|
|
||||||
|
@field_validator("editor_state", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def remove_unsupported_editor_state(cls, value: Any, info: ValidationInfo) -> dict[str, Any]:
|
||||||
|
return public_campaign_editor_state(
|
||||||
|
value,
|
||||||
|
include_diagnostics=bool((info.context or {}).get("include_diagnostics")),
|
||||||
|
)
|
||||||
|
|
||||||
|
@field_validator("source_filename", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def remove_source_directory(cls, value: Any) -> str | None:
|
||||||
|
return public_source_filename(value)
|
||||||
|
|
||||||
|
@field_validator("validation_summary", "build_summary", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def remove_internal_summary_fields(cls, value: Any, info: ValidationInfo) -> Any:
|
||||||
|
return public_campaign_payload(
|
||||||
|
value,
|
||||||
|
include_diagnostics=bool((info.context or {}).get("include_diagnostics")),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class CampaignVersionDetailResponse(CampaignVersionResponse):
|
class CampaignVersionDetailResponse(CampaignVersionResponse):
|
||||||
raw_json: dict[str, Any]
|
raw_json: dict[str, Any]
|
||||||
|
mail_profile_migration_required: bool = False
|
||||||
|
|
||||||
|
@field_validator("raw_json", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def remove_internal_configuration_fields(cls, value: Any) -> Any:
|
||||||
|
return public_campaign_configuration(value)
|
||||||
|
|
||||||
|
|
||||||
class CampaignPartialValidationResponse(BaseModel):
|
class CampaignPartialValidationResponse(BaseModel):
|
||||||
@@ -152,6 +211,10 @@ class CampaignDeltaResponse(BaseModel):
|
|||||||
watermark: str | None = None
|
watermark: str | None = None
|
||||||
has_more: bool = False
|
has_more: bool = False
|
||||||
full: bool = False
|
full: bool = False
|
||||||
|
total: int = 0
|
||||||
|
page: int = 1
|
||||||
|
page_size: int = 500
|
||||||
|
pages: int = 1
|
||||||
|
|
||||||
|
|
||||||
class CampaignWorkspaceDeltaResponse(CampaignWorkspaceResponse):
|
class CampaignWorkspaceDeltaResponse(CampaignWorkspaceResponse):
|
||||||
@@ -174,6 +237,10 @@ class CampaignShareItem(BaseModel):
|
|||||||
|
|
||||||
class CampaignShareListResponse(BaseModel):
|
class CampaignShareListResponse(BaseModel):
|
||||||
shares: list[CampaignShareItem]
|
shares: list[CampaignShareItem]
|
||||||
|
total: int = 0
|
||||||
|
page: int = 1
|
||||||
|
page_size: int = 500
|
||||||
|
pages: int = 1
|
||||||
|
|
||||||
|
|
||||||
class CampaignShareTargetItem(BaseModel):
|
class CampaignShareTargetItem(BaseModel):
|
||||||
@@ -300,6 +367,13 @@ class CampaignRecipientAddressSourcesResponse(BaseModel):
|
|||||||
sources: list[CampaignRecipientAddressSource] = Field(default_factory=list)
|
sources: list[CampaignRecipientAddressSource] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignPostboxCatalogResponse(BaseModel):
|
||||||
|
available: bool = False
|
||||||
|
postboxes: list[dict[str, Any]] = Field(default_factory=list)
|
||||||
|
templates: list[dict[str, Any]] = Field(default_factory=list)
|
||||||
|
organization_units: list[dict[str, Any]] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
class CampaignRecipientAddressSourceSnapshotRequest(BaseModel):
|
class CampaignRecipientAddressSourceSnapshotRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
@@ -351,6 +425,15 @@ class CampaignJobDetailResponse(BaseModel):
|
|||||||
attempts: dict[str, list[dict[str, Any]]] = Field(default_factory=dict)
|
attempts: dict[str, list[dict[str, Any]]] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignJobDiagnosticsResponse(BaseModel):
|
||||||
|
job_id: str
|
||||||
|
campaign_id: str
|
||||||
|
campaign_version_id: str
|
||||||
|
storage: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
worker_claim: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
attempts: dict[str, list[dict[str, Any]]] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
class CampaignRetryJobsRequest(BaseModel):
|
class CampaignRetryJobsRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
@@ -371,17 +454,52 @@ class CampaignSendUnattemptedRequest(BaseModel):
|
|||||||
dry_run: bool = False
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignSendJobRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
kind: Literal["test", "single_send", "single_resend"]
|
||||||
|
idempotency_key: str = Field(min_length=1, max_length=200)
|
||||||
|
reason: str | None = Field(default=None, max_length=2000)
|
||||||
|
context: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
include_warnings: bool = True
|
||||||
|
use_rate_limit: bool = True
|
||||||
|
enqueue_imap_task: bool = False
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def require_resend_reason(self):
|
||||||
|
self.reason = (self.reason or "").strip() or None
|
||||||
|
if self.kind == "single_resend" and not self.reason:
|
||||||
|
raise ValueError("single_resend requires a reason")
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
class CampaignResolveOutcomeRequest(BaseModel):
|
class CampaignResolveOutcomeRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
decision: Literal["smtp_accepted", "not_sent"]
|
decision: Literal[
|
||||||
note: str | None = None
|
"smtp_accepted",
|
||||||
|
"not_sent",
|
||||||
|
"imap_appended",
|
||||||
|
"imap_not_appended",
|
||||||
|
"postbox_accepted",
|
||||||
|
"postbox_not_accepted",
|
||||||
|
]
|
||||||
|
note: str | None = Field(default=None, max_length=2000)
|
||||||
|
attempt_id: str | None = Field(default=None, max_length=36)
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def require_reconciliation_evidence(self) -> "CampaignResolveOutcomeRequest":
|
||||||
|
self.note = (self.note or "").strip()
|
||||||
|
if not self.note:
|
||||||
|
raise ValueError("Reconciliation requires an evidence note")
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
class ValidateCampaignRequest(BaseModel):
|
class ValidateCampaignRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
check_files: bool = False
|
check_files: bool = False
|
||||||
|
link_unshared_matches: bool = False
|
||||||
|
|
||||||
|
|
||||||
class BuildCampaignRequest(BaseModel):
|
class BuildCampaignRequest(BaseModel):
|
||||||
@@ -390,125 +508,6 @@ class BuildCampaignRequest(BaseModel):
|
|||||||
write_eml: bool = True
|
write_eml: bool = True
|
||||||
|
|
||||||
|
|
||||||
class MailSmtpTestRequest(SmtpConfig):
|
|
||||||
"""SMTP settings supplied directly from the WebUI mail settings form."""
|
|
||||||
|
|
||||||
|
|
||||||
class MailImapTestRequest(ImapConfig):
|
|
||||||
"""IMAP settings supplied directly from the WebUI mail settings form."""
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
MailProfileScope = Literal["system", "tenant", "user", "group", "campaign"]
|
|
||||||
|
|
||||||
|
|
||||||
class MailCredentialPolicyPayload(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
inherit: bool | None = None
|
|
||||||
allow_override: bool | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class MailProfilePolicyPayload(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
allowed_profile_ids: list[str] = Field(default_factory=list)
|
|
||||||
allow_user_profiles: bool | None = None
|
|
||||||
allow_group_profiles: bool | None = None
|
|
||||||
allow_campaign_profiles: bool | None = None
|
|
||||||
smtp_credentials: MailCredentialPolicyPayload = Field(default_factory=MailCredentialPolicyPayload)
|
|
||||||
imap_credentials: MailCredentialPolicyPayload = Field(default_factory=MailCredentialPolicyPayload)
|
|
||||||
whitelist: dict[str, list[str]] = Field(default_factory=dict)
|
|
||||||
blacklist: dict[str, list[str]] = Field(default_factory=dict)
|
|
||||||
|
|
||||||
|
|
||||||
class MailProfilePolicyUpdateRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
policy: MailProfilePolicyPayload = Field(default_factory=MailProfilePolicyPayload)
|
|
||||||
|
|
||||||
|
|
||||||
class MailProfilePolicyResponse(BaseModel):
|
|
||||||
scope_type: MailProfileScope
|
|
||||||
scope_id: str | None = None
|
|
||||||
policy: dict[str, Any]
|
|
||||||
effective_policy: dict[str, Any] | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class MailServerProfileCreateRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
name: str = Field(min_length=1, max_length=255)
|
|
||||||
slug: str | None = Field(default=None, max_length=100)
|
|
||||||
description: str | None = None
|
|
||||||
is_active: bool = True
|
|
||||||
scope_type: MailProfileScope = "tenant"
|
|
||||||
scope_id: str | None = None
|
|
||||||
smtp: SmtpConfig
|
|
||||||
imap: ImapConfig | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class MailServerProfileUpdateRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
name: str | None = Field(default=None, max_length=255)
|
|
||||||
slug: str | None = Field(default=None, max_length=100)
|
|
||||||
description: str | None = None
|
|
||||||
is_active: bool | None = None
|
|
||||||
smtp: SmtpConfig | None = None
|
|
||||||
imap: ImapConfig | None = None
|
|
||||||
clear_imap: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
class MailServerProfileResponse(BaseModel):
|
|
||||||
id: str
|
|
||||||
tenant_id: str | None = None
|
|
||||||
scope_type: MailProfileScope = "tenant"
|
|
||||||
scope_id: str | None = None
|
|
||||||
name: str
|
|
||||||
slug: str
|
|
||||||
description: str | None = None
|
|
||||||
is_active: bool
|
|
||||||
smtp: dict[str, Any]
|
|
||||||
imap: dict[str, Any] | None = None
|
|
||||||
smtp_password_configured: bool = False
|
|
||||||
imap_password_configured: bool = False
|
|
||||||
created_at: datetime
|
|
||||||
updated_at: datetime
|
|
||||||
|
|
||||||
|
|
||||||
class MailServerProfileListResponse(BaseModel):
|
|
||||||
profiles: list[MailServerProfileResponse] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class MailConnectionTestResponse(BaseModel):
|
|
||||||
ok: bool
|
|
||||||
protocol: Literal["smtp", "imap"]
|
|
||||||
host: str | None = None
|
|
||||||
port: int | None = None
|
|
||||||
security: str | None = None
|
|
||||||
message: str
|
|
||||||
details: dict[str, Any] = Field(default_factory=dict)
|
|
||||||
|
|
||||||
|
|
||||||
class MailImapFolderResponse(BaseModel):
|
|
||||||
name: str
|
|
||||||
flags: list[str] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class MailImapFolderListResponse(BaseModel):
|
|
||||||
ok: bool
|
|
||||||
protocol: Literal["imap"] = "imap"
|
|
||||||
host: str | None = None
|
|
||||||
port: int | None = None
|
|
||||||
security: str | None = None
|
|
||||||
message: str
|
|
||||||
folders: list[MailImapFolderResponse] = Field(default_factory=list)
|
|
||||||
detected_sent_folder: str | None = None
|
|
||||||
details: dict[str, Any] = Field(default_factory=dict)
|
|
||||||
|
|
||||||
|
|
||||||
class ApiKeyCreateRequest(BaseModel):
|
class ApiKeyCreateRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
@@ -540,6 +539,8 @@ class QueueCampaignResponse(BaseModel):
|
|||||||
skipped_count: int
|
skipped_count: int
|
||||||
blocked_count: int
|
blocked_count: int
|
||||||
enqueued_count: int
|
enqueued_count: int
|
||||||
|
delivery_mode: str = "worker_queue"
|
||||||
|
worker_queue_available: bool = False
|
||||||
dry_run: bool = False
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
@@ -560,6 +561,14 @@ class SendCampaignNowResponse(BaseModel):
|
|||||||
result: dict[str, Any]
|
result: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignDeliveryOptionsResponse(BaseModel):
|
||||||
|
campaign_id: str
|
||||||
|
version_id: str
|
||||||
|
worker_queue_available: bool
|
||||||
|
postbox_available: bool = False
|
||||||
|
synchronous_send: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
class MockCampaignSendRequest(BaseModel):
|
class MockCampaignSendRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
@@ -587,124 +596,70 @@ class AppendSentRequest(BaseModel):
|
|||||||
class CampaignActionResponse(BaseModel):
|
class CampaignActionResponse(BaseModel):
|
||||||
result: dict[str, Any]
|
result: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
def _valid_report_email_domain(domain: str) -> bool:
|
||||||
|
if not domain or domain.startswith(".") or domain.endswith(".") or ".." in domain:
|
||||||
|
return False
|
||||||
|
return all(
|
||||||
|
label
|
||||||
|
and not label.startswith("-")
|
||||||
|
and not label.endswith("-")
|
||||||
|
and all(character.isalnum() or character == "-" for character in label)
|
||||||
|
for label in domain.split(".")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_report_recipient(value: Any) -> str:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise ValueError("report recipients must be email-address strings")
|
||||||
|
recipient = value.strip()
|
||||||
|
if len(recipient) > 320:
|
||||||
|
raise ValueError("report recipient addresses must be at most 320 characters")
|
||||||
|
if any(ord(character) < 32 or ord(character) == 127 for character in recipient):
|
||||||
|
raise ValueError("report recipient addresses must not contain control characters")
|
||||||
|
if recipient.count("@") != 1:
|
||||||
|
raise ValueError("report recipients must be email addresses")
|
||||||
|
local, domain = recipient.split("@", 1)
|
||||||
|
invalid_local = not local or local.startswith(".") or local.endswith(".") or ".." in local
|
||||||
|
invalid_address = (
|
||||||
|
any(character.isspace() for character in recipient)
|
||||||
|
or any(character in ',;:<>[]()\\"' for character in recipient)
|
||||||
|
)
|
||||||
|
if invalid_local or invalid_address or not _valid_report_email_domain(domain):
|
||||||
|
raise ValueError("report recipients must be email addresses")
|
||||||
|
return recipient
|
||||||
|
|
||||||
|
|
||||||
|
ReportEmailAddress = Annotated[str, BeforeValidator(_normalize_report_recipient)]
|
||||||
|
|
||||||
|
|
||||||
|
def _deduplicate_report_recipients(value: list[str]) -> list[str]:
|
||||||
|
recipients: list[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for recipient in value:
|
||||||
|
key = recipient.casefold()
|
||||||
|
if key not in seen:
|
||||||
|
seen.add(key)
|
||||||
|
recipients.append(recipient)
|
||||||
|
return recipients
|
||||||
|
|
||||||
|
|
||||||
class ReportEmailRequest(BaseModel):
|
class ReportEmailRequest(BaseModel):
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
to: list[str]
|
to: list[ReportEmailAddress] = Field(min_length=1, max_length=50)
|
||||||
version_id: str | None = None
|
version_id: str | None = None
|
||||||
include_jobs: bool = False
|
include_jobs: bool = False
|
||||||
attach_jobs_csv: bool = True
|
attach_jobs_csv: bool = False
|
||||||
attach_report_json: bool = False
|
attach_report_json: bool = False
|
||||||
dry_run: bool = False
|
dry_run: bool = False
|
||||||
|
idempotency_key: str | None = Field(default=None, min_length=1, max_length=200)
|
||||||
|
|
||||||
|
@field_validator("to")
|
||||||
|
@classmethod
|
||||||
|
def normalize_and_validate_recipients(cls, value: list[str]) -> list[str]:
|
||||||
|
return _deduplicate_report_recipients(value)
|
||||||
|
|
||||||
|
|
||||||
class ReportEmailResponse(BaseModel):
|
class ReportEmailResponse(BaseModel):
|
||||||
result: dict[str, Any]
|
result: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
class AuditLogItemResponse(BaseModel):
|
|
||||||
model_config = ConfigDict(from_attributes=True)
|
|
||||||
|
|
||||||
id: str
|
|
||||||
tenant_id: str | None = None
|
|
||||||
user_id: str | None = None
|
|
||||||
api_key_id: str | None = None
|
|
||||||
action: str
|
|
||||||
object_type: str | None = None
|
|
||||||
object_id: str | None = None
|
|
||||||
details: dict[str, Any] | None = None
|
|
||||||
created_at: datetime
|
|
||||||
|
|
||||||
|
|
||||||
class AuditLogListResponse(BaseModel):
|
|
||||||
items: list[AuditLogItemResponse]
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
class LoginRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
email: str
|
|
||||||
password: str
|
|
||||||
# Kept optional for backwards compatibility and future tenant-switch login flows.
|
|
||||||
# The WebUI no longer sends it. If omitted, the backend resolves the user by email.
|
|
||||||
tenant_slug: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
class SwitchTenantRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
tenant_id: str
|
|
||||||
|
|
||||||
|
|
||||||
class TenantInfo(BaseModel):
|
|
||||||
id: str
|
|
||||||
slug: str
|
|
||||||
name: str
|
|
||||||
is_active: bool = True
|
|
||||||
default_locale: str = "en"
|
|
||||||
|
|
||||||
|
|
||||||
class TenantMembershipInfo(TenantInfo):
|
|
||||||
roles: list[str] = Field(default_factory=list)
|
|
||||||
is_active: bool = True
|
|
||||||
|
|
||||||
|
|
||||||
class UserInfo(BaseModel):
|
|
||||||
id: str
|
|
||||||
account_id: str
|
|
||||||
email: str
|
|
||||||
# Global account identity used by the title bar and account settings.
|
|
||||||
display_name: str | None = None
|
|
||||||
# Optional tenant-local alias used in tenant administration and ownership.
|
|
||||||
tenant_display_name: str | None = None
|
|
||||||
is_tenant_admin: bool = False
|
|
||||||
password_reset_required: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
class ProfileUpdateRequest(BaseModel):
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
display_name: str | None = Field(default=None, max_length=255)
|
|
||||||
tenant_display_name: str | None = Field(default=None, max_length=255)
|
|
||||||
|
|
||||||
|
|
||||||
class RoleInfo(BaseModel):
|
|
||||||
id: str
|
|
||||||
slug: str
|
|
||||||
name: str
|
|
||||||
permissions: list[str] = Field(default_factory=list)
|
|
||||||
level: Literal["tenant", "system"] = "tenant"
|
|
||||||
|
|
||||||
|
|
||||||
class GroupInfo(BaseModel):
|
|
||||||
id: str
|
|
||||||
slug: str
|
|
||||||
name: str
|
|
||||||
|
|
||||||
|
|
||||||
class LoginResponse(BaseModel):
|
|
||||||
access_token: str
|
|
||||||
token_type: str = "bearer"
|
|
||||||
expires_at: datetime
|
|
||||||
user: UserInfo
|
|
||||||
# Backwards-compatible alias for the active tenant.
|
|
||||||
tenant: TenantInfo
|
|
||||||
active_tenant: TenantInfo
|
|
||||||
tenants: list[TenantMembershipInfo] = Field(default_factory=list)
|
|
||||||
scopes: list[str]
|
|
||||||
roles: list[RoleInfo] = Field(default_factory=list)
|
|
||||||
groups: list[GroupInfo] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class MeResponse(BaseModel):
|
|
||||||
user: UserInfo
|
|
||||||
# Backwards-compatible alias for the active tenant.
|
|
||||||
tenant: TenantInfo
|
|
||||||
active_tenant: TenantInfo
|
|
||||||
tenants: list[TenantMembershipInfo] = Field(default_factory=list)
|
|
||||||
scopes: list[str]
|
|
||||||
roles: list[RoleInfo] = Field(default_factory=list)
|
|
||||||
groups: list[GroupInfo] = Field(default_factory=list)
|
|
||||||
|
|||||||
@@ -9,10 +9,21 @@ from pydantic import BaseModel, ConfigDict
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion, JobValidationStatus
|
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion, JobValidationStatus
|
||||||
from govoplan_campaign.backend.campaign.models import DeliveryConfig, ImapConfig, SmtpConfig
|
from govoplan_campaign.backend.campaign.models import (
|
||||||
from govoplan_campaign.backend.integrations import MailProfileError, mail_integration
|
DeliveryChannelPolicy,
|
||||||
|
DeliveryConfig,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
CampaignMailProfileBoundaryError,
|
||||||
|
assert_campaign_uses_mail_profile_reference,
|
||||||
|
campaign_mail_profile_id,
|
||||||
|
campaign_mail_resource_ids,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import MailProfileError, files_integration, mail_integration
|
||||||
|
from govoplan_campaign.backend.path_security import CampaignPathSecurityError, assert_server_safe_campaign_paths
|
||||||
|
|
||||||
SNAPSHOT_VERSION = "3"
|
SNAPSHOT_VERSION = "7"
|
||||||
|
SUPPORTED_SNAPSHOT_VERSIONS = {"6", SNAPSHOT_VERSION}
|
||||||
|
|
||||||
|
|
||||||
class ExecutionSnapshotError(RuntimeError):
|
class ExecutionSnapshotError(RuntimeError):
|
||||||
@@ -24,9 +35,9 @@ class ExecutionSnapshot(BaseModel):
|
|||||||
|
|
||||||
Rendered messages and attachment evidence remain normalized in
|
Rendered messages and attachment evidence remain normalized in
|
||||||
``CampaignJob`` and ``CampaignAttachmentUse``. This record freezes the
|
``CampaignJob`` and ``CampaignAttachmentUse``. This record freezes the
|
||||||
mutable transport/runtime configuration and cryptographically binds it to
|
Mail-profile reference, delivery policy, and opaque transport revisions and
|
||||||
the build and the exact set of persisted jobs without duplicating all
|
cryptographically binds them to the build and exact persisted jobs. Mail
|
||||||
recipient data into one large JSON value.
|
owns the resolved transport configuration and credentials.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
@@ -34,6 +45,11 @@ class ExecutionSnapshot(BaseModel):
|
|||||||
snapshot_version: str = SNAPSHOT_VERSION
|
snapshot_version: str = SNAPSHOT_VERSION
|
||||||
campaign_version_id: str
|
campaign_version_id: str
|
||||||
campaign_json_sha256: str
|
campaign_json_sha256: str
|
||||||
|
mail_profile_id: str | None = None
|
||||||
|
smtp_server_id: str | None = None
|
||||||
|
smtp_credential_id: str | None = None
|
||||||
|
imap_server_id: str | None = None
|
||||||
|
imap_credential_id: str | None = None
|
||||||
created_at: str
|
created_at: str
|
||||||
build_token: str | None = None
|
build_token: str | None = None
|
||||||
built_at: str | None = None
|
built_at: str | None = None
|
||||||
@@ -41,10 +57,10 @@ class ExecutionSnapshot(BaseModel):
|
|||||||
queueable_job_count: int = 0
|
queueable_job_count: int = 0
|
||||||
job_manifest_sha256: str | None = None
|
job_manifest_sha256: str | None = None
|
||||||
effective_policy_sha256: str | None = None
|
effective_policy_sha256: str | None = None
|
||||||
smtp_config_fingerprint: str | None = None
|
smtp_transport_revision: str | None = None
|
||||||
imap_config_fingerprint: str | None = None
|
imap_transport_revision: str | None = None
|
||||||
smtp: SmtpConfig
|
uses_mail: bool = True
|
||||||
imap: ImapConfig | None = None
|
uses_postbox: bool = False
|
||||||
delivery: DeliveryConfig
|
delivery: DeliveryConfig
|
||||||
|
|
||||||
|
|
||||||
@@ -60,118 +76,97 @@ def snapshot_hash(payload: dict[str, Any]) -> str:
|
|||||||
return _sha256(payload)
|
return _sha256(payload)
|
||||||
|
|
||||||
|
|
||||||
def _transport_fingerprint(config: SmtpConfig | ImapConfig | None) -> str | None:
|
def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict[str, Any]:
|
||||||
if config is None:
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
return None
|
_assert_version_mail_profile_boundary(raw_json, require_profile=True)
|
||||||
payload = config.model_dump(mode="json")
|
|
||||||
# The fingerprint is safe to expose in reports. It identifies the effective
|
|
||||||
# account/transport settings without incorporating or revealing the secret.
|
|
||||||
if "password" in payload:
|
|
||||||
payload["password"] = "<configured>" if payload.get("password") else None
|
|
||||||
return _sha256(payload)
|
|
||||||
|
|
||||||
|
|
||||||
def _redacted_transport_config(config: SmtpConfig | ImapConfig | None) -> SmtpConfig | ImapConfig | None:
|
|
||||||
if config is None:
|
|
||||||
return None
|
|
||||||
payload = config.model_dump(mode="json")
|
|
||||||
payload["password"] = None
|
|
||||||
if isinstance(config, SmtpConfig):
|
|
||||||
return SmtpConfig.model_validate(payload)
|
|
||||||
return ImapConfig.model_validate(payload)
|
|
||||||
|
|
||||||
|
|
||||||
def _transport_password_from_campaign_json(raw_json: dict[str, Any] | None, name: str) -> str | None:
|
|
||||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
|
||||||
credentials = server.get("credentials") if isinstance(server, dict) and isinstance(server.get("credentials"), dict) else None
|
|
||||||
config = credentials.get(name) if isinstance(credentials, dict) and isinstance(credentials.get(name), dict) else None
|
|
||||||
password = config.get("password") if isinstance(config, dict) else None
|
|
||||||
if password is None:
|
|
||||||
legacy_config = server.get(name) if isinstance(server, dict) else None
|
|
||||||
password = legacy_config.get("password") if isinstance(legacy_config, dict) else None
|
|
||||||
if password is None:
|
|
||||||
return None
|
|
||||||
return str(password)
|
|
||||||
|
|
||||||
|
|
||||||
def _server_from_campaign_json(raw_json: dict[str, Any] | None) -> dict[str, Any]:
|
|
||||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
|
||||||
return server if isinstance(server, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def _profile_for_version(session: Session, version: CampaignVersion):
|
|
||||||
mail = mail_integration()
|
mail = mail_integration()
|
||||||
profile_id = mail.mail_profile_id_from_campaign_json(version.raw_json if isinstance(version.raw_json, dict) else {})
|
profile_id = campaign_mail_profile_id(raw_json)
|
||||||
if not profile_id:
|
if profile_id is None: # Kept explicit for static typing; the assertion above requires it.
|
||||||
return None
|
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
|
||||||
campaign = session.get(Campaign, version.campaign_id)
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
if campaign is None:
|
if campaign is None:
|
||||||
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
||||||
|
references = campaign_mail_resource_ids(raw_json)
|
||||||
try:
|
try:
|
||||||
return mail.ensure_mail_profile_allowed_for_campaign(session, tenant_id=campaign.tenant_id, campaign_id=campaign.id, profile_id=profile_id, require_active=True)
|
return mail.campaign_profile_delivery_summary(
|
||||||
|
session,
|
||||||
|
tenant_id=campaign.tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
profile_id=profile_id,
|
||||||
|
smtp_server_id=references["smtp_server_id"],
|
||||||
|
smtp_credential_id=references["smtp_credential_id"],
|
||||||
|
imap_server_id=references["imap_server_id"],
|
||||||
|
imap_credential_id=references["imap_credential_id"],
|
||||||
|
)
|
||||||
except MailProfileError as exc:
|
except MailProfileError as exc:
|
||||||
raise ExecutionSnapshotError(str(exc)) from exc
|
raise ExecutionSnapshotError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
def runtime_smtp_config(session: Session, version: CampaignVersion, snapshot: ExecutionSnapshot) -> SmtpConfig:
|
def profile_transport_revisions(session: Session, version: CampaignVersion) -> dict[str, str | None]:
|
||||||
payload = snapshot.smtp.model_dump(mode="json")
|
summary = profile_delivery_summary(session, version)
|
||||||
if not payload.get("password"):
|
return {
|
||||||
server = _server_from_campaign_json(version.raw_json)
|
"smtp": summary.get("smtp_transport_revision"),
|
||||||
profile = _profile_for_version(session, version)
|
"imap": summary.get("imap_transport_revision"),
|
||||||
if profile is not None:
|
}
|
||||||
campaign = session.get(Campaign, version.campaign_id)
|
|
||||||
if campaign is None:
|
|
||||||
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
|
||||||
mail = mail_integration()
|
|
||||||
profile_payload = mail.smtp_config_from_profile(profile).model_dump(mode="json")
|
|
||||||
if mail.effective_profile_credentials_inherited(session, tenant_id=campaign.tenant_id, campaign_id=campaign.id, server=server, protocol="smtp"):
|
|
||||||
return SmtpConfig.model_validate(profile_payload)
|
|
||||||
return SmtpConfig.model_validate(mail.apply_campaign_credentials(profile_payload, server, "smtp"))
|
|
||||||
payload["password"] = _transport_password_from_campaign_json(version.raw_json, "smtp")
|
|
||||||
return SmtpConfig.model_validate(payload)
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_imap_config(session: Session, version: CampaignVersion, snapshot: ExecutionSnapshot) -> ImapConfig | None:
|
def _assert_snapshot_profile_matches_version(version: CampaignVersion, snapshot: ExecutionSnapshot) -> None:
|
||||||
server = _server_from_campaign_json(version.raw_json)
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
if snapshot.imap is None:
|
_assert_version_mail_profile_boundary(
|
||||||
profile = _profile_for_version(session, version)
|
raw_json,
|
||||||
if profile is None:
|
require_profile=snapshot.uses_mail,
|
||||||
return None
|
)
|
||||||
mail = mail_integration()
|
if not snapshot.uses_mail:
|
||||||
imap = mail.imap_config_from_profile(profile)
|
return
|
||||||
if imap is None:
|
if campaign_mail_profile_id(raw_json) != snapshot.mail_profile_id:
|
||||||
return None
|
raise ExecutionSnapshotError(
|
||||||
campaign = session.get(Campaign, version.campaign_id)
|
"The campaign's Mail profile reference differs from the built execution snapshot. "
|
||||||
if campaign is None:
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
)
|
||||||
imap_payload = imap.model_dump(mode="json")
|
references = campaign_mail_resource_ids(raw_json)
|
||||||
if mail.effective_profile_credentials_inherited(session, tenant_id=campaign.tenant_id, campaign_id=campaign.id, server=server, protocol="imap"):
|
for key in (
|
||||||
return ImapConfig.model_validate(imap_payload)
|
"smtp_server_id",
|
||||||
return ImapConfig.model_validate(mail.apply_campaign_credentials(imap_payload, server, "imap"))
|
"smtp_credential_id",
|
||||||
payload = snapshot.imap.model_dump(mode="json")
|
"imap_server_id",
|
||||||
if not payload.get("password"):
|
"imap_credential_id",
|
||||||
profile = _profile_for_version(session, version)
|
):
|
||||||
if profile is not None:
|
configured = references[key]
|
||||||
mail = mail_integration()
|
if configured and configured != getattr(snapshot, key):
|
||||||
imap = mail.imap_config_from_profile(profile)
|
raise ExecutionSnapshotError(
|
||||||
if imap is not None:
|
"The campaign's Mail server or credential selection differs from the built "
|
||||||
campaign = session.get(Campaign, version.campaign_id)
|
"execution snapshot. Revalidate and rebuild before delivery."
|
||||||
if campaign is None:
|
)
|
||||||
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
|
||||||
imap_payload = imap.model_dump(mode="json")
|
|
||||||
if mail.effective_profile_credentials_inherited(session, tenant_id=campaign.tenant_id, campaign_id=campaign.id, server=server, protocol="imap"):
|
|
||||||
return ImapConfig.model_validate(imap_payload)
|
|
||||||
return ImapConfig.model_validate(mail.apply_campaign_credentials(imap_payload, server, "imap"))
|
|
||||||
payload["password"] = _transport_password_from_campaign_json(version.raw_json, "imap")
|
|
||||||
return ImapConfig.model_validate(payload)
|
|
||||||
|
|
||||||
|
|
||||||
def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> str:
|
def _assert_version_mail_profile_boundary(
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
*,
|
||||||
|
require_profile: bool,
|
||||||
|
) -> None:
|
||||||
|
try:
|
||||||
|
assert_campaign_uses_mail_profile_reference(
|
||||||
|
raw_json,
|
||||||
|
require_profile=require_profile,
|
||||||
|
)
|
||||||
|
except CampaignMailProfileBoundaryError as exc:
|
||||||
|
raise ExecutionSnapshotError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _policy_fingerprint(
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
delivery: DeliveryConfig,
|
||||||
|
*,
|
||||||
|
snapshot_version: str = SNAPSHOT_VERSION,
|
||||||
|
) -> str:
|
||||||
|
delivery_payload = delivery.model_dump(mode="json")
|
||||||
|
if snapshot_version == "6":
|
||||||
|
delivery_payload.pop("channel_policy", None)
|
||||||
|
delivery_payload.pop("postbox", None)
|
||||||
return _sha256(
|
return _sha256(
|
||||||
{
|
{
|
||||||
"validation_policy": raw_json.get("validation_policy"),
|
"validation_policy": raw_json.get("validation_policy"),
|
||||||
"policy": raw_json.get("policy"),
|
"policy": raw_json.get("policy"),
|
||||||
"delivery": delivery.model_dump(mode="json"),
|
"delivery": delivery_payload,
|
||||||
"attachment_defaults": (raw_json.get("attachments") or {}).get("defaults")
|
"attachment_defaults": (raw_json.get("attachments") or {}).get("defaults")
|
||||||
if isinstance(raw_json.get("attachments"), dict)
|
if isinstance(raw_json.get("attachments"), dict)
|
||||||
else None,
|
else None,
|
||||||
@@ -179,70 +174,231 @@ def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> s
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def job_manifest_hash(jobs: Iterable[CampaignJob]) -> str:
|
def _job_execution_input_payload(
|
||||||
"""Hash the immutable per-message execution records in stable order."""
|
job: CampaignJob,
|
||||||
|
*,
|
||||||
payload: list[dict[str, Any]] = []
|
snapshot_version: str = SNAPSHOT_VERSION,
|
||||||
for job in sorted(jobs, key=lambda item: (item.entry_index, item.id)):
|
) -> dict[str, Any]:
|
||||||
payload.append(
|
payload = {
|
||||||
|
"job_id": job.id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"recipient_email": job.recipient_email,
|
||||||
|
"subject": job.subject,
|
||||||
|
"message_id_header": job.message_id_header,
|
||||||
|
"eml_size_bytes": job.eml_size_bytes,
|
||||||
|
"eml_sha256": job.eml_sha256,
|
||||||
|
"build_status": job.build_status,
|
||||||
|
"validation_status": job.validation_status,
|
||||||
|
"resolved_recipients_sha256": _sha256(job.resolved_recipients or {}),
|
||||||
|
"resolved_attachments_sha256": _sha256(job.resolved_attachments or []),
|
||||||
|
"issues_sha256": _sha256(job.issues_snapshot or []),
|
||||||
|
}
|
||||||
|
if snapshot_version != "6":
|
||||||
|
payload.update(
|
||||||
{
|
{
|
||||||
"job_id": job.id,
|
"delivery_channel_policy": getattr(
|
||||||
"entry_index": job.entry_index,
|
job,
|
||||||
"entry_id": job.entry_id,
|
"delivery_channel_policy",
|
||||||
"recipient_email": job.recipient_email,
|
DeliveryChannelPolicy.MAIL.value,
|
||||||
"subject": job.subject,
|
),
|
||||||
"message_id_header": job.message_id_header,
|
"resolved_postbox_targets_sha256": _sha256(
|
||||||
"eml_size_bytes": job.eml_size_bytes,
|
getattr(job, "resolved_postbox_targets", None) or []
|
||||||
"eml_sha256": job.eml_sha256,
|
),
|
||||||
"build_status": job.build_status,
|
|
||||||
"validation_status": job.validation_status,
|
|
||||||
"resolved_recipients_sha256": _sha256(job.resolved_recipients or {}),
|
|
||||||
"resolved_attachments_sha256": _sha256(job.resolved_attachments or []),
|
|
||||||
"issues_sha256": _sha256(job.issues_snapshot or []),
|
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def job_execution_input_hash(
|
||||||
|
job: CampaignJob,
|
||||||
|
*,
|
||||||
|
snapshot_version: str = SNAPSHOT_VERSION,
|
||||||
|
) -> str:
|
||||||
|
return _sha256(
|
||||||
|
_job_execution_input_payload(
|
||||||
|
job,
|
||||||
|
snapshot_version=snapshot_version,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def job_manifest_hash(
|
||||||
|
jobs: Iterable[CampaignJob],
|
||||||
|
*,
|
||||||
|
snapshot_version: str = SNAPSHOT_VERSION,
|
||||||
|
) -> str:
|
||||||
|
"""Hash the immutable per-message execution records in stable order."""
|
||||||
|
|
||||||
|
payload = [
|
||||||
|
_job_execution_input_payload(
|
||||||
|
job,
|
||||||
|
snapshot_version=snapshot_version,
|
||||||
|
)
|
||||||
|
for job in sorted(jobs, key=lambda item: (item.entry_index, item.id))
|
||||||
|
]
|
||||||
return _sha256(payload)
|
return _sha256(payload)
|
||||||
|
|
||||||
|
|
||||||
def create_execution_snapshot(
|
def create_execution_snapshot(
|
||||||
version: CampaignVersion,
|
version: CampaignVersion,
|
||||||
*,
|
*,
|
||||||
smtp: SmtpConfig,
|
mail_profile_id: str | None,
|
||||||
imap: ImapConfig | None,
|
smtp_transport_revision: str | None,
|
||||||
|
imap_transport_revision: str | None,
|
||||||
delivery: DeliveryConfig,
|
delivery: DeliveryConfig,
|
||||||
|
smtp_server_id: str | None = None,
|
||||||
|
smtp_credential_id: str | None = None,
|
||||||
|
imap_server_id: str | None = None,
|
||||||
|
imap_credential_id: str | None = None,
|
||||||
jobs: Iterable[CampaignJob] = (),
|
jobs: Iterable[CampaignJob] = (),
|
||||||
build_summary: dict[str, Any] | None = None,
|
build_summary: dict[str, Any] | None = None,
|
||||||
) -> tuple[dict[str, Any], str]:
|
) -> tuple[dict[str, Any], str]:
|
||||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
job_list = list(jobs)
|
job_list = list(jobs)
|
||||||
|
channel_policies = {
|
||||||
|
DeliveryChannelPolicy(
|
||||||
|
getattr(
|
||||||
|
job,
|
||||||
|
"delivery_channel_policy",
|
||||||
|
DeliveryChannelPolicy.MAIL.value,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for job in job_list
|
||||||
|
}
|
||||||
|
uses_mail = any(policy.uses_mail for policy in channel_policies)
|
||||||
|
uses_postbox = any(policy.uses_postbox for policy in channel_policies)
|
||||||
|
for job in job_list:
|
||||||
|
job.execution_input_sha256 = job_execution_input_hash(
|
||||||
|
job,
|
||||||
|
snapshot_version=SNAPSHOT_VERSION,
|
||||||
|
)
|
||||||
summary = build_summary if isinstance(build_summary, dict) else {}
|
summary = build_summary if isinstance(build_summary, dict) else {}
|
||||||
queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value}
|
queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value}
|
||||||
redacted_smtp = _redacted_transport_config(smtp)
|
|
||||||
redacted_imap = _redacted_transport_config(imap)
|
|
||||||
if not isinstance(redacted_smtp, SmtpConfig):
|
|
||||||
raise ExecutionSnapshotError("Redacted SMTP configuration is invalid.")
|
|
||||||
if redacted_imap is not None and not isinstance(redacted_imap, ImapConfig):
|
|
||||||
raise ExecutionSnapshotError("Redacted IMAP configuration is invalid.")
|
|
||||||
payload = ExecutionSnapshot(
|
payload = ExecutionSnapshot(
|
||||||
campaign_version_id=version.id,
|
campaign_version_id=version.id,
|
||||||
campaign_json_sha256=_sha256(raw_json),
|
campaign_json_sha256=_sha256(raw_json),
|
||||||
|
mail_profile_id=mail_profile_id,
|
||||||
|
smtp_server_id=smtp_server_id,
|
||||||
|
smtp_credential_id=smtp_credential_id,
|
||||||
|
imap_server_id=imap_server_id,
|
||||||
|
imap_credential_id=imap_credential_id,
|
||||||
build_token=str(summary.get("build_token") or "") or None,
|
build_token=str(summary.get("build_token") or "") or None,
|
||||||
built_at=str(summary.get("built_at") or "") or None,
|
built_at=str(summary.get("built_at") or "") or None,
|
||||||
job_count=len(job_list),
|
job_count=len(job_list),
|
||||||
queueable_job_count=sum(1 for job in job_list if job.validation_status in queueable_statuses),
|
queueable_job_count=sum(1 for job in job_list if job.validation_status in queueable_statuses),
|
||||||
job_manifest_sha256=job_manifest_hash(job_list) if job_list else None,
|
job_manifest_sha256=(
|
||||||
effective_policy_sha256=_policy_fingerprint(raw_json, delivery),
|
job_manifest_hash(
|
||||||
smtp_config_fingerprint=_transport_fingerprint(smtp),
|
job_list,
|
||||||
imap_config_fingerprint=_transport_fingerprint(imap),
|
snapshot_version=SNAPSHOT_VERSION,
|
||||||
|
)
|
||||||
|
if job_list
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
effective_policy_sha256=_policy_fingerprint(
|
||||||
|
raw_json,
|
||||||
|
delivery,
|
||||||
|
snapshot_version=SNAPSHOT_VERSION,
|
||||||
|
),
|
||||||
|
smtp_transport_revision=smtp_transport_revision,
|
||||||
|
imap_transport_revision=imap_transport_revision,
|
||||||
|
uses_mail=uses_mail,
|
||||||
|
uses_postbox=uses_postbox,
|
||||||
created_at=datetime.now(timezone.utc).isoformat(),
|
created_at=datetime.now(timezone.utc).isoformat(),
|
||||||
smtp=redacted_smtp,
|
|
||||||
imap=redacted_imap,
|
|
||||||
delivery=delivery,
|
delivery=delivery,
|
||||||
).model_dump(mode="json")
|
).model_dump(mode="json")
|
||||||
return payload, snapshot_hash(payload)
|
return payload, snapshot_hash(payload)
|
||||||
|
|
||||||
|
|
||||||
def ensure_execution_snapshot(session: Session, version: CampaignVersion) -> ExecutionSnapshot:
|
def _assert_snapshot_matches_persisted_inputs(
|
||||||
|
session: Session,
|
||||||
|
version: CampaignVersion,
|
||||||
|
snapshot: ExecutionSnapshot,
|
||||||
|
*,
|
||||||
|
effect_job: CampaignJob | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Fail closed when any build-bound input drifted after snapshot creation."""
|
||||||
|
|
||||||
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
if snapshot.campaign_version_id != version.id:
|
||||||
|
raise ExecutionSnapshotError("Execution snapshot campaign version mismatch")
|
||||||
|
if snapshot.campaign_json_sha256 != _sha256(raw_json):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"Campaign inputs changed after this execution snapshot was built. "
|
||||||
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
|
)
|
||||||
|
if snapshot.uses_mail and not snapshot.smtp_transport_revision:
|
||||||
|
raise ExecutionSnapshotError("Execution snapshot has no SMTP transport revision")
|
||||||
|
if not snapshot.job_manifest_sha256:
|
||||||
|
raise ExecutionSnapshotError("Execution snapshot has no built-job manifest checksum")
|
||||||
|
if not snapshot.effective_policy_sha256:
|
||||||
|
raise ExecutionSnapshotError("Execution snapshot has no effective-policy checksum")
|
||||||
|
if snapshot.effective_policy_sha256 != _policy_fingerprint(
|
||||||
|
raw_json,
|
||||||
|
snapshot.delivery,
|
||||||
|
snapshot_version=snapshot.snapshot_version,
|
||||||
|
):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"Campaign delivery policy changed after the execution snapshot was created. "
|
||||||
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
|
)
|
||||||
|
|
||||||
|
if effect_job is not None:
|
||||||
|
if effect_job.campaign_version_id != version.id:
|
||||||
|
raise ExecutionSnapshotError("Campaign job does not belong to the snapshotted version")
|
||||||
|
if not getattr(effect_job, "execution_input_sha256", None):
|
||||||
|
raise ExecutionSnapshotError("Campaign job has no execution-input checksum; rebuild before delivery")
|
||||||
|
if effect_job.execution_input_sha256 != job_execution_input_hash(
|
||||||
|
effect_job,
|
||||||
|
snapshot_version=snapshot.snapshot_version,
|
||||||
|
):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"Built campaign job inputs changed after the execution snapshot was created. "
|
||||||
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(CampaignJob.campaign_version_id == version.id)
|
||||||
|
.order_by(CampaignJob.entry_index.asc(), CampaignJob.id.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value}
|
||||||
|
if snapshot.job_count != len(jobs):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"Built campaign jobs changed after the execution snapshot was created. "
|
||||||
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
|
)
|
||||||
|
queueable_count = sum(1 for job in jobs if job.validation_status in queueable_statuses)
|
||||||
|
if (
|
||||||
|
snapshot.queueable_job_count != queueable_count
|
||||||
|
or snapshot.job_manifest_sha256
|
||||||
|
!= job_manifest_hash(
|
||||||
|
jobs,
|
||||||
|
snapshot_version=snapshot.snapshot_version,
|
||||||
|
)
|
||||||
|
or any(
|
||||||
|
getattr(job, "execution_input_sha256", None)
|
||||||
|
!= job_execution_input_hash(
|
||||||
|
job,
|
||||||
|
snapshot_version=snapshot.snapshot_version,
|
||||||
|
)
|
||||||
|
for job in jobs
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"Built campaign job inputs changed after the execution snapshot was created. "
|
||||||
|
"Revalidate and rebuild the campaign before delivery."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_execution_snapshot(
|
||||||
|
session: Session,
|
||||||
|
version: CampaignVersion,
|
||||||
|
*,
|
||||||
|
effect_job: CampaignJob | None = None,
|
||||||
|
) -> ExecutionSnapshot:
|
||||||
"""Return a validated snapshot, creating one for pre-migration builds.
|
"""Return a validated snapshot, creating one for pre-migration builds.
|
||||||
|
|
||||||
New builds create the snapshot after persisting their jobs. The fallback is
|
New builds create the snapshot after persisting their jobs. The fallback is
|
||||||
@@ -250,19 +406,44 @@ def ensure_execution_snapshot(session: Session, version: CampaignVersion) -> Exe
|
|||||||
without a manual data migration.
|
without a manual data migration.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||||
|
try:
|
||||||
|
assert_server_safe_campaign_paths(
|
||||||
|
raw_json,
|
||||||
|
managed_files_available=files_integration().available,
|
||||||
|
)
|
||||||
|
except CampaignPathSecurityError as exc:
|
||||||
|
raise ExecutionSnapshotError(str(exc)) from exc
|
||||||
|
_assert_version_mail_profile_boundary(raw_json, require_profile=False)
|
||||||
|
|
||||||
if isinstance(version.execution_snapshot, dict):
|
if isinstance(version.execution_snapshot, dict):
|
||||||
|
stored_version = str(
|
||||||
|
version.execution_snapshot.get("snapshot_version") or ""
|
||||||
|
)
|
||||||
|
if stored_version not in SUPPORTED_SNAPSHOT_VERSIONS:
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"This campaign has a legacy execution snapshot that may contain campaign-owned transport data. "
|
||||||
|
"It is preserved for audit only and cannot be delivered; select a Mail profile, then revalidate "
|
||||||
|
"and rebuild a new campaign version."
|
||||||
|
)
|
||||||
snapshot = ExecutionSnapshot.model_validate(version.execution_snapshot)
|
snapshot = ExecutionSnapshot.model_validate(version.execution_snapshot)
|
||||||
expected = snapshot_hash(snapshot.model_dump(mode="json"))
|
expected = snapshot_hash(version.execution_snapshot)
|
||||||
if version.execution_snapshot_hash and version.execution_snapshot_hash != expected:
|
if not version.execution_snapshot_hash:
|
||||||
|
raise ExecutionSnapshotError("Execution snapshot checksum is missing")
|
||||||
|
if version.execution_snapshot_hash != expected:
|
||||||
raise ExecutionSnapshotError("Execution snapshot checksum mismatch")
|
raise ExecutionSnapshotError("Execution snapshot checksum mismatch")
|
||||||
|
_assert_snapshot_profile_matches_version(version, snapshot)
|
||||||
|
_assert_snapshot_matches_persisted_inputs(
|
||||||
|
session,
|
||||||
|
version,
|
||||||
|
snapshot,
|
||||||
|
effect_job=effect_job,
|
||||||
|
)
|
||||||
return snapshot
|
return snapshot
|
||||||
|
|
||||||
from govoplan_campaign.backend.persistence.campaigns import load_version_config
|
from govoplan_campaign.backend.persistence.campaigns import load_version_config
|
||||||
|
|
||||||
_, _, config = load_version_config(session, version.id)
|
_, _, config = load_version_config(session, version.id)
|
||||||
runtime_smtp = config.server.runtime_smtp_config()
|
|
||||||
if not runtime_smtp:
|
|
||||||
raise ExecutionSnapshotError("Campaign has no SMTP configuration")
|
|
||||||
jobs = (
|
jobs = (
|
||||||
session.query(CampaignJob)
|
session.query(CampaignJob)
|
||||||
.filter(CampaignJob.campaign_version_id == version.id)
|
.filter(CampaignJob.campaign_version_id == version.id)
|
||||||
@@ -271,10 +452,33 @@ def ensure_execution_snapshot(session: Session, version: CampaignVersion) -> Exe
|
|||||||
)
|
)
|
||||||
if not jobs:
|
if not jobs:
|
||||||
raise ExecutionSnapshotError("Campaign version has no built jobs; rebuild it before delivery")
|
raise ExecutionSnapshotError("Campaign version has no built jobs; rebuild it before delivery")
|
||||||
|
uses_mail = any(
|
||||||
|
DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail
|
||||||
|
for job in jobs
|
||||||
|
)
|
||||||
|
profile_id = campaign_mail_profile_id(raw_json)
|
||||||
|
summary: dict[str, Any] = {}
|
||||||
|
if uses_mail:
|
||||||
|
if not config.server.profile_capabilities.smtp_available:
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"The selected Mail profile has no SMTP configuration"
|
||||||
|
)
|
||||||
|
if profile_id is None:
|
||||||
|
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
|
||||||
|
summary = profile_delivery_summary(session, version)
|
||||||
|
if not summary.get("smtp_transport_revision"):
|
||||||
|
raise ExecutionSnapshotError(
|
||||||
|
"The selected Mail profile has no SMTP transport revision"
|
||||||
|
)
|
||||||
payload, digest = create_execution_snapshot(
|
payload, digest = create_execution_snapshot(
|
||||||
version,
|
version,
|
||||||
smtp=runtime_smtp,
|
mail_profile_id=profile_id,
|
||||||
imap=config.server.runtime_imap_config(),
|
smtp_server_id=summary.get("smtp_server_id"),
|
||||||
|
smtp_credential_id=summary.get("smtp_credential_id"),
|
||||||
|
imap_server_id=summary.get("imap_server_id"),
|
||||||
|
imap_credential_id=summary.get("imap_credential_id"),
|
||||||
|
smtp_transport_revision=summary.get("smtp_transport_revision"),
|
||||||
|
imap_transport_revision=summary.get("imap_transport_revision"),
|
||||||
delivery=config.delivery,
|
delivery=config.delivery,
|
||||||
jobs=jobs,
|
jobs=jobs,
|
||||||
build_summary=version.build_summary if isinstance(version.build_summary, dict) else {},
|
build_summary=version.build_summary if isinstance(version.build_summary, dict) else {},
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
506
src/govoplan_campaign/backend/sending/postbox_delivery.py
Normal file
506
src/govoplan_campaign/backend/sending/postbox_delivery.py
Normal file
@@ -0,0 +1,506 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from email import policy
|
||||||
|
from email.parser import BytesParser
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_core.core.postbox import (
|
||||||
|
PostboxAttachmentRef,
|
||||||
|
PostboxDeliveryOutcomeUnknown,
|
||||||
|
PostboxDeliveryRejected,
|
||||||
|
PostboxDeliveryRequest,
|
||||||
|
PostboxParticipantRef,
|
||||||
|
PostboxTargetRef,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignJob,
|
||||||
|
JobPostboxStatus,
|
||||||
|
PostboxDeliveryAttempt,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.integrations import (
|
||||||
|
PostboxDeliveryUnavailable,
|
||||||
|
postbox_integration,
|
||||||
|
)
|
||||||
|
from govoplan_core.security.time import utc_now
|
||||||
|
|
||||||
|
|
||||||
|
ACCEPTED_POSTBOX_ATTEMPT_STATUSES = {
|
||||||
|
JobPostboxStatus.ACCEPTED.value,
|
||||||
|
JobPostboxStatus.ACCEPTED_VACANT.value,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class PostboxChannelOutcome:
|
||||||
|
accepted: int = 0
|
||||||
|
accepted_vacant: int = 0
|
||||||
|
rejected_temporary: int = 0
|
||||||
|
rejected_permanent: int = 0
|
||||||
|
outcome_unknown: int = 0
|
||||||
|
messages: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def accepted_count(self) -> int:
|
||||||
|
return self.accepted + self.accepted_vacant
|
||||||
|
|
||||||
|
@property
|
||||||
|
def rejected_count(self) -> int:
|
||||||
|
return self.rejected_temporary + self.rejected_permanent
|
||||||
|
|
||||||
|
@property
|
||||||
|
def all_rejected_before_acceptance(self) -> bool:
|
||||||
|
return (
|
||||||
|
self.accepted_count == 0
|
||||||
|
and self.outcome_unknown == 0
|
||||||
|
and self.rejected_count > 0
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def status(self) -> str:
|
||||||
|
if self.outcome_unknown:
|
||||||
|
return JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||||
|
if self.accepted_count and self.rejected_count:
|
||||||
|
return JobPostboxStatus.PARTIALLY_ACCEPTED.value
|
||||||
|
if self.accepted_vacant and not self.accepted:
|
||||||
|
return JobPostboxStatus.ACCEPTED_VACANT.value
|
||||||
|
if self.accepted_count:
|
||||||
|
return JobPostboxStatus.ACCEPTED.value
|
||||||
|
if self.rejected_temporary:
|
||||||
|
return JobPostboxStatus.REJECTED_TEMPORARY.value
|
||||||
|
return JobPostboxStatus.REJECTED_PERMANENT.value
|
||||||
|
|
||||||
|
|
||||||
|
def _target_key(target: dict[str, Any]) -> str:
|
||||||
|
payload = json.dumps(
|
||||||
|
target,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
return hashlib.sha256(payload).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _attempt_number(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
job_id: str,
|
||||||
|
target_key: str,
|
||||||
|
) -> int:
|
||||||
|
current = (
|
||||||
|
session.query(func.max(PostboxDeliveryAttempt.attempt_number))
|
||||||
|
.filter(
|
||||||
|
PostboxDeliveryAttempt.job_id == job_id,
|
||||||
|
PostboxDeliveryAttempt.target_key == target_key,
|
||||||
|
)
|
||||||
|
.scalar()
|
||||||
|
)
|
||||||
|
return int(current or 0) + 1
|
||||||
|
|
||||||
|
|
||||||
|
def _accepted_attempt(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
job_id: str,
|
||||||
|
target_key: str,
|
||||||
|
) -> PostboxDeliveryAttempt | None:
|
||||||
|
return (
|
||||||
|
session.query(PostboxDeliveryAttempt)
|
||||||
|
.filter(
|
||||||
|
PostboxDeliveryAttempt.job_id == job_id,
|
||||||
|
PostboxDeliveryAttempt.target_key == target_key,
|
||||||
|
PostboxDeliveryAttempt.status.in_(
|
||||||
|
list(ACCEPTED_POSTBOX_ATTEMPT_STATUSES)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.order_by(PostboxDeliveryAttempt.attempt_number.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _message_body(message_bytes: bytes) -> str | None:
|
||||||
|
message = BytesParser(policy=policy.default).parsebytes(message_bytes)
|
||||||
|
body = message.get_body(preferencelist=("plain", "html"))
|
||||||
|
if body is None:
|
||||||
|
payload = message.get_payload(decode=True)
|
||||||
|
if not isinstance(payload, bytes):
|
||||||
|
return None
|
||||||
|
return payload.decode(message.get_content_charset() or "utf-8", "replace")
|
||||||
|
try:
|
||||||
|
content = body.get_content()
|
||||||
|
except (LookupError, UnicodeError):
|
||||||
|
payload = body.get_payload(decode=True)
|
||||||
|
if not isinstance(payload, bytes):
|
||||||
|
return None
|
||||||
|
return payload.decode(body.get_content_charset() or "utf-8", "replace")
|
||||||
|
return str(content)
|
||||||
|
|
||||||
|
|
||||||
|
def _participants(job: CampaignJob) -> tuple[PostboxParticipantRef, ...]:
|
||||||
|
recipients = job.resolved_recipients or {}
|
||||||
|
values: list[PostboxParticipantRef] = []
|
||||||
|
for key in (
|
||||||
|
"from_all",
|
||||||
|
"to",
|
||||||
|
"cc",
|
||||||
|
"bcc",
|
||||||
|
"reply_to",
|
||||||
|
"bounce_to",
|
||||||
|
"disposition_notification_to",
|
||||||
|
):
|
||||||
|
for item in recipients.get(key) or []:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
address = str(item.get("email") or "").strip() or None
|
||||||
|
label = str(item.get("name") or "").strip() or None
|
||||||
|
if address is None and label is None:
|
||||||
|
continue
|
||||||
|
values.append(
|
||||||
|
PostboxParticipantRef(
|
||||||
|
kind="sender" if key == "from_all" else key,
|
||||||
|
reference_type="mail_address",
|
||||||
|
label=label,
|
||||||
|
address=address,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return tuple(values)
|
||||||
|
|
||||||
|
|
||||||
|
def _attachments(job: CampaignJob) -> tuple[PostboxAttachmentRef, ...]:
|
||||||
|
values = [_eml_attachment(job)]
|
||||||
|
for rule_index, rule in enumerate(job.resolved_attachments or []):
|
||||||
|
if not isinstance(rule, dict):
|
||||||
|
continue
|
||||||
|
managed_matches = rule.get("managed_matches")
|
||||||
|
if isinstance(managed_matches, list) and managed_matches:
|
||||||
|
values.extend(_managed_attachments(managed_matches))
|
||||||
|
continue
|
||||||
|
matches = rule.get("matches")
|
||||||
|
if isinstance(matches, list):
|
||||||
|
values.extend(_campaign_attachments(job, rule_index=rule_index, matches=matches))
|
||||||
|
return tuple(values)
|
||||||
|
|
||||||
|
|
||||||
|
def _eml_attachment(job: CampaignJob) -> PostboxAttachmentRef:
|
||||||
|
return PostboxAttachmentRef(
|
||||||
|
reference_type="campaign_eml",
|
||||||
|
reference_id=job.id,
|
||||||
|
name=f"{job.entry_id or job.entry_index}.eml",
|
||||||
|
media_type="message/rfc822",
|
||||||
|
size_bytes=job.eml_size_bytes,
|
||||||
|
digest=job.eml_sha256,
|
||||||
|
metadata={"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
MANAGED_ATTACHMENT_METADATA_KEYS = {
|
||||||
|
"asset_id",
|
||||||
|
"version_id",
|
||||||
|
"blob_id",
|
||||||
|
"display_path",
|
||||||
|
"relative_path",
|
||||||
|
"owner_type",
|
||||||
|
"owner_id",
|
||||||
|
"source_revision",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _managed_attachment(match: dict[str, Any]) -> PostboxAttachmentRef | None:
|
||||||
|
reference_id = str(match.get("version_id") or match.get("asset_id") or match.get("blob_id") or "").strip()
|
||||||
|
if not reference_id:
|
||||||
|
return None
|
||||||
|
return PostboxAttachmentRef(
|
||||||
|
reference_type="file_version" if match.get("version_id") else "file_asset",
|
||||||
|
reference_id=reference_id,
|
||||||
|
name=str(match.get("filename") or "").strip() or None,
|
||||||
|
media_type=str(match.get("content_type")) if match.get("content_type") else None,
|
||||||
|
size_bytes=int(match["size_bytes"]) if match.get("size_bytes") is not None else None,
|
||||||
|
digest=str(match.get("checksum_sha256")) if match.get("checksum_sha256") else None,
|
||||||
|
metadata={key: value for key, value in match.items() if key in MANAGED_ATTACHMENT_METADATA_KEYS},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _managed_attachments(matches: list[Any]) -> list[PostboxAttachmentRef]:
|
||||||
|
attachments = (_managed_attachment(match) for match in matches if isinstance(match, dict))
|
||||||
|
return [attachment for attachment in attachments if attachment is not None]
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_attachments(
|
||||||
|
job: CampaignJob,
|
||||||
|
*,
|
||||||
|
rule_index: int,
|
||||||
|
matches: list[Any],
|
||||||
|
) -> list[PostboxAttachmentRef]:
|
||||||
|
metadata = {"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id, "job_id": job.id}
|
||||||
|
return [
|
||||||
|
PostboxAttachmentRef(
|
||||||
|
reference_type="campaign_attachment",
|
||||||
|
reference_id=f"{job.id}:{rule_index}:{match_index}",
|
||||||
|
name=str(match).rsplit("/", 1)[-1] or None,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
|
for match_index, match in enumerate(matches)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _sender_label(job: CampaignJob) -> str | None:
|
||||||
|
recipients = job.resolved_recipients or {}
|
||||||
|
sender = recipients.get("from")
|
||||||
|
if not isinstance(sender, dict):
|
||||||
|
return None
|
||||||
|
name = str(sender.get("name") or "").strip()
|
||||||
|
address = str(sender.get("email") or "").strip()
|
||||||
|
if name and address:
|
||||||
|
return f"{name} <{address}>"
|
||||||
|
return address or name or None
|
||||||
|
|
||||||
|
|
||||||
|
def _request(
|
||||||
|
job: CampaignJob,
|
||||||
|
target: dict[str, Any],
|
||||||
|
*,
|
||||||
|
target_key: str,
|
||||||
|
body_text: str | None,
|
||||||
|
classification: str,
|
||||||
|
) -> PostboxDeliveryRequest:
|
||||||
|
return PostboxDeliveryRequest(
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
target=PostboxTargetRef(postbox_id=str(target["postbox_id"])),
|
||||||
|
producer_module="campaigns",
|
||||||
|
producer_resource_type="campaign_job",
|
||||||
|
producer_resource_id=job.id,
|
||||||
|
idempotency_key=(
|
||||||
|
f"campaign:{job.campaign_version_id}:{job.id}:postbox:"
|
||||||
|
f"{target_key}"
|
||||||
|
),
|
||||||
|
subject=(job.subject or "").strip() or "(No subject)",
|
||||||
|
body_text=body_text,
|
||||||
|
sender_label=_sender_label(job),
|
||||||
|
classification=classification,
|
||||||
|
participants=_participants(job),
|
||||||
|
attachments=_attachments(job),
|
||||||
|
metadata={
|
||||||
|
"campaign_id": job.campaign_id,
|
||||||
|
"campaign_version_id": job.campaign_version_id,
|
||||||
|
"campaign_job_id": job.id,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"delivery_channel_policy": job.delivery_channel_policy,
|
||||||
|
"target_snapshot": target,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _record_rejection(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
job_id: str,
|
||||||
|
attempt_id: str,
|
||||||
|
exc: Exception,
|
||||||
|
temporary: bool,
|
||||||
|
) -> None:
|
||||||
|
session.rollback()
|
||||||
|
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if attempt is None or job is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Postbox rejection could not be written to Campaign evidence."
|
||||||
|
) from exc
|
||||||
|
attempt.status = (
|
||||||
|
JobPostboxStatus.REJECTED_TEMPORARY.value
|
||||||
|
if temporary
|
||||||
|
else JobPostboxStatus.REJECTED_PERMANENT.value
|
||||||
|
)
|
||||||
|
attempt.error_type = exc.__class__.__name__
|
||||||
|
attempt.error_code = str(getattr(exc, "code", "") or "") or None
|
||||||
|
attempt.error_message = str(exc)
|
||||||
|
attempt.finished_at = utc_now()
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _record_unknown(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
job_id: str,
|
||||||
|
attempt_id: str,
|
||||||
|
exc: Exception,
|
||||||
|
) -> None:
|
||||||
|
session.rollback()
|
||||||
|
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if attempt is None or job is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Unknown Postbox outcome could not be written to Campaign evidence."
|
||||||
|
) from exc
|
||||||
|
attempt.status = JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||||
|
attempt.error_type = exc.__class__.__name__
|
||||||
|
attempt.error_code = str(getattr(exc, "code", "") or "") or None
|
||||||
|
attempt.error_message = str(exc)
|
||||||
|
attempt.finished_at = utc_now()
|
||||||
|
job.postbox_status = JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def deliver_campaign_job_to_postboxes(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
job: CampaignJob,
|
||||||
|
message_bytes: bytes,
|
||||||
|
classification: str,
|
||||||
|
) -> PostboxChannelOutcome:
|
||||||
|
outcome = PostboxChannelOutcome()
|
||||||
|
targets = [
|
||||||
|
target
|
||||||
|
for target in (job.resolved_postbox_targets or [])
|
||||||
|
if isinstance(target, dict) and target.get("postbox_id")
|
||||||
|
]
|
||||||
|
if not targets:
|
||||||
|
outcome.rejected_permanent = 1
|
||||||
|
outcome.messages.append("No frozen Postbox target is available.")
|
||||||
|
job.postbox_status = JobPostboxStatus.REJECTED_PERMANENT.value
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return outcome
|
||||||
|
|
||||||
|
body_text = _message_body(message_bytes)
|
||||||
|
for target_index, target in enumerate(targets):
|
||||||
|
key = _target_key(target)
|
||||||
|
accepted = _accepted_attempt(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
target_key=key,
|
||||||
|
)
|
||||||
|
if accepted is not None:
|
||||||
|
if accepted.vacant:
|
||||||
|
outcome.accepted_vacant += 1
|
||||||
|
else:
|
||||||
|
outcome.accepted += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
attempt_number = _attempt_number(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
target_key=key,
|
||||||
|
)
|
||||||
|
request = _request(
|
||||||
|
job,
|
||||||
|
target,
|
||||||
|
target_key=key,
|
||||||
|
body_text=body_text,
|
||||||
|
classification=classification,
|
||||||
|
)
|
||||||
|
attempt = PostboxDeliveryAttempt(
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
job_id=job.id,
|
||||||
|
target_key=key,
|
||||||
|
target_index=target_index,
|
||||||
|
attempt_number=attempt_number,
|
||||||
|
idempotency_key=request.idempotency_key,
|
||||||
|
status=JobPostboxStatus.DELIVERING.value,
|
||||||
|
target_snapshot=target,
|
||||||
|
evidence={},
|
||||||
|
started_at=utc_now(),
|
||||||
|
)
|
||||||
|
job.postbox_attempt_count += 1
|
||||||
|
job.postbox_status = JobPostboxStatus.DELIVERING.value
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
attempt_id = attempt.id
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = postbox_integration().deliver(session, request)
|
||||||
|
current_attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||||
|
current_job = session.get(CampaignJob, job.id)
|
||||||
|
if current_attempt is None or current_job is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Campaign Postbox attempt disappeared before acceptance."
|
||||||
|
)
|
||||||
|
current_attempt.status = (
|
||||||
|
JobPostboxStatus.ACCEPTED_VACANT.value
|
||||||
|
if result.vacant
|
||||||
|
else JobPostboxStatus.ACCEPTED.value
|
||||||
|
)
|
||||||
|
current_attempt.provider_delivery_id = result.delivery_id
|
||||||
|
current_attempt.provider_message_id = result.message_id
|
||||||
|
current_attempt.postbox_id = result.postbox_id
|
||||||
|
current_attempt.address = result.address
|
||||||
|
current_attempt.holder_count = result.holder_count
|
||||||
|
current_attempt.vacant = result.vacant
|
||||||
|
current_attempt.duplicate = result.duplicate
|
||||||
|
current_attempt.evidence = dict(result.evidence)
|
||||||
|
current_attempt.finished_at = utc_now()
|
||||||
|
session.add(current_attempt)
|
||||||
|
session.add(current_job)
|
||||||
|
session.commit()
|
||||||
|
if result.vacant:
|
||||||
|
outcome.accepted_vacant += 1
|
||||||
|
else:
|
||||||
|
outcome.accepted += 1
|
||||||
|
except PostboxDeliveryOutcomeUnknown as exc:
|
||||||
|
_record_unknown(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_id=attempt_id,
|
||||||
|
exc=exc,
|
||||||
|
)
|
||||||
|
outcome.outcome_unknown += 1
|
||||||
|
outcome.messages.append(str(exc))
|
||||||
|
except PostboxDeliveryRejected as exc:
|
||||||
|
if exc.code == "idempotency_conflict":
|
||||||
|
_record_unknown(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_id=attempt_id,
|
||||||
|
exc=exc,
|
||||||
|
)
|
||||||
|
outcome.outcome_unknown += 1
|
||||||
|
else:
|
||||||
|
_record_rejection(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_id=attempt_id,
|
||||||
|
exc=exc,
|
||||||
|
temporary=exc.temporary,
|
||||||
|
)
|
||||||
|
if exc.temporary:
|
||||||
|
outcome.rejected_temporary += 1
|
||||||
|
else:
|
||||||
|
outcome.rejected_permanent += 1
|
||||||
|
outcome.messages.append(str(exc))
|
||||||
|
except PostboxDeliveryUnavailable as exc:
|
||||||
|
_record_rejection(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_id=attempt_id,
|
||||||
|
exc=exc,
|
||||||
|
temporary=True,
|
||||||
|
)
|
||||||
|
outcome.rejected_temporary += 1
|
||||||
|
outcome.messages.append(str(exc))
|
||||||
|
except Exception as exc:
|
||||||
|
_record_unknown(
|
||||||
|
session,
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_id=attempt_id,
|
||||||
|
exc=exc,
|
||||||
|
)
|
||||||
|
outcome.outcome_unknown += 1
|
||||||
|
outcome.messages.append(str(exc))
|
||||||
|
|
||||||
|
current_job = session.get(CampaignJob, job.id)
|
||||||
|
if current_job is not None:
|
||||||
|
current_job.postbox_status = outcome.status
|
||||||
|
session.add(current_job)
|
||||||
|
session.commit()
|
||||||
|
return outcome
|
||||||
999
src/govoplan_campaign/backend/services/job_queries.py
Normal file
999
src/govoplan_campaign/backend/services/job_queries.py
Normal file
@@ -0,0 +1,999 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from collections.abc import Mapping, Sequence
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from fastapi import HTTPException, Query, status
|
||||||
|
from sqlalchemy import and_, func, or_
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
CampaignJobsResponse,
|
||||||
|
CampaignJobDiagnosticsResponse,
|
||||||
|
)
|
||||||
|
from govoplan_core.auth import ApiPrincipal
|
||||||
|
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
|
||||||
|
from govoplan_core.core.change_sequence import (
|
||||||
|
encode_sequence_watermark,
|
||||||
|
max_sequence_id,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.pagination import (
|
||||||
|
KeysetCursorError,
|
||||||
|
decode_keyset_cursor,
|
||||||
|
encode_keyset_cursor,
|
||||||
|
keyset_query_fingerprint,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.change_tracking import (
|
||||||
|
CAMPAIGNS_MODULE_ID,
|
||||||
|
CAMPAIGN_JOBS_COLLECTION,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignMessageAction,
|
||||||
|
CampaignMessageActionAttempt,
|
||||||
|
CampaignVersion,
|
||||||
|
ImapAppendAttempt,
|
||||||
|
JobImapStatus,
|
||||||
|
JobPostboxStatus,
|
||||||
|
JobQueueStatus,
|
||||||
|
JobSendStatus,
|
||||||
|
JobValidationStatus,
|
||||||
|
PostboxDeliveryAttempt,
|
||||||
|
SendAttempt,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.response_security import (
|
||||||
|
public_campaign_payload,
|
||||||
|
public_delivery_result_message,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.route_support import (
|
||||||
|
_get_campaign_for_principal,
|
||||||
|
_get_campaign_for_tenant,
|
||||||
|
_get_version_for_tenant,
|
||||||
|
_require_permission,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
CAMPAIGN_JOBS_CURSOR_SCOPE = "campaign.jobs"
|
||||||
|
|
||||||
|
|
||||||
|
def _job_review_key(job: CampaignJob) -> str:
|
||||||
|
return str(job.entry_id or job.entry_index)
|
||||||
|
|
||||||
|
|
||||||
|
def _job_summary_payload(
|
||||||
|
job: CampaignJob,
|
||||||
|
*,
|
||||||
|
reviewed_keys: set[str] | None = None,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
review_key = _job_review_key(job)
|
||||||
|
return {
|
||||||
|
"id": job.id,
|
||||||
|
"campaign_version_id": job.campaign_version_id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"recipient_email": job.recipient_email,
|
||||||
|
"subject": job.subject,
|
||||||
|
"message_id_header": job.message_id_header,
|
||||||
|
"build_status": job.build_status,
|
||||||
|
"validation_status": job.validation_status,
|
||||||
|
"queue_status": job.queue_status,
|
||||||
|
"send_status": job.send_status,
|
||||||
|
"delivery_channel_policy": getattr(job, "delivery_channel_policy", "mail"),
|
||||||
|
"postbox_status": getattr(job, "postbox_status", "not_requested"),
|
||||||
|
"imap_status": job.imap_status,
|
||||||
|
"eml_size_bytes": job.eml_size_bytes,
|
||||||
|
"eml_sha256": job.eml_sha256,
|
||||||
|
"attempt_count": job.attempt_count,
|
||||||
|
"postbox_attempt_count": getattr(job, "postbox_attempt_count", 0),
|
||||||
|
"postbox_target_count": len(
|
||||||
|
getattr(job, "resolved_postbox_targets", None) or []
|
||||||
|
),
|
||||||
|
"last_error": public_delivery_result_message(
|
||||||
|
last_error=job.last_error,
|
||||||
|
send_status=job.send_status,
|
||||||
|
imap_status=job.imap_status,
|
||||||
|
postbox_status=getattr(job, "postbox_status", "not_requested"),
|
||||||
|
),
|
||||||
|
"queued_at": job.queued_at,
|
||||||
|
"outcome_unknown_at": job.outcome_unknown_at,
|
||||||
|
"sent_at": job.sent_at,
|
||||||
|
"created_at": job.created_at,
|
||||||
|
"updated_at": job.updated_at,
|
||||||
|
"issues_count": len(job.issues_snapshot or []),
|
||||||
|
"attachment_count": len(job.resolved_attachments or []),
|
||||||
|
"review_key": review_key,
|
||||||
|
"reviewed": review_key in reviewed_keys if reviewed_keys is not None else False,
|
||||||
|
"matched_file_count": sum(
|
||||||
|
len(item.get("matches") or [])
|
||||||
|
for item in (job.resolved_attachments or [])
|
||||||
|
if isinstance(item, dict)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _job_detail_payload(job: CampaignJob) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
**_job_summary_payload(job),
|
||||||
|
"message_id_header": job.message_id_header,
|
||||||
|
"issues": job.issues_snapshot or [],
|
||||||
|
"attachments": public_campaign_payload(job.resolved_attachments or []),
|
||||||
|
"resolved_recipients": job.resolved_recipients or {},
|
||||||
|
"resolved_postbox_targets": getattr(job, "resolved_postbox_targets", None)
|
||||||
|
or [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _job_attempts_payload(
|
||||||
|
send_attempts: list[SendAttempt],
|
||||||
|
imap_attempts: list[ImapAppendAttempt],
|
||||||
|
postbox_attempts: Sequence[PostboxDeliveryAttempt] = (),
|
||||||
|
message_actions: Sequence[CampaignMessageAction] = (),
|
||||||
|
message_action_attempts: Sequence[CampaignMessageActionAttempt] = (),
|
||||||
|
*,
|
||||||
|
postbox_receipts: (
|
||||||
|
Mapping[str, PostboxDeliveryReceiptSummaryRef] | None
|
||||||
|
) = None,
|
||||||
|
include_diagnostics: bool = False,
|
||||||
|
) -> dict[str, list[dict[str, object]]]:
|
||||||
|
smtp_payloads: list[dict[str, object]] = []
|
||||||
|
for attempt in send_attempts:
|
||||||
|
payload: dict[str, object] = {
|
||||||
|
"id": attempt.id,
|
||||||
|
"attempt_number": attempt.attempt_number,
|
||||||
|
"status": attempt.status,
|
||||||
|
"smtp_status_code": attempt.smtp_status_code,
|
||||||
|
"started_at": attempt.started_at,
|
||||||
|
"finished_at": attempt.finished_at,
|
||||||
|
}
|
||||||
|
if include_diagnostics:
|
||||||
|
payload["claim_token"] = attempt.claim_token
|
||||||
|
payload["smtp_response"] = attempt.smtp_response
|
||||||
|
payload["error_type"] = attempt.error_type
|
||||||
|
payload["error_message"] = attempt.error_message
|
||||||
|
smtp_payloads.append(payload)
|
||||||
|
|
||||||
|
imap_payloads: list[dict[str, object]] = []
|
||||||
|
for attempt in imap_attempts:
|
||||||
|
payload = {
|
||||||
|
"id": attempt.id,
|
||||||
|
"attempt_number": attempt.attempt_number,
|
||||||
|
"status": attempt.status,
|
||||||
|
"folder": attempt.folder,
|
||||||
|
"created_at": attempt.created_at,
|
||||||
|
"updated_at": attempt.updated_at,
|
||||||
|
}
|
||||||
|
if include_diagnostics:
|
||||||
|
payload["claim_token"] = attempt.claim_token
|
||||||
|
payload["error_message"] = attempt.error_message
|
||||||
|
imap_payloads.append(payload)
|
||||||
|
postbox_payloads: list[dict[str, object]] = []
|
||||||
|
for attempt in postbox_attempts:
|
||||||
|
payload = {
|
||||||
|
"id": attempt.id,
|
||||||
|
"target_index": attempt.target_index,
|
||||||
|
"attempt_number": attempt.attempt_number,
|
||||||
|
"status": attempt.status,
|
||||||
|
"postbox_id": attempt.postbox_id,
|
||||||
|
"address": attempt.address,
|
||||||
|
"holder_count": attempt.holder_count,
|
||||||
|
"vacant": attempt.vacant,
|
||||||
|
"duplicate": attempt.duplicate,
|
||||||
|
"target": attempt.target_snapshot or {},
|
||||||
|
"started_at": attempt.started_at,
|
||||||
|
"finished_at": attempt.finished_at,
|
||||||
|
"error_code": attempt.error_code,
|
||||||
|
}
|
||||||
|
if include_diagnostics:
|
||||||
|
payload["idempotency_key"] = attempt.idempotency_key
|
||||||
|
payload["provider_delivery_id"] = attempt.provider_delivery_id
|
||||||
|
payload["provider_message_id"] = attempt.provider_message_id
|
||||||
|
payload["evidence"] = attempt.evidence or {}
|
||||||
|
payload["error_type"] = attempt.error_type
|
||||||
|
payload["error_message"] = attempt.error_message
|
||||||
|
if attempt.provider_delivery_id:
|
||||||
|
receipt_summary = (
|
||||||
|
postbox_receipts.get(attempt.provider_delivery_id)
|
||||||
|
if postbox_receipts is not None
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
payload["receipt_summary_status"] = (
|
||||||
|
"available"
|
||||||
|
if receipt_summary is not None
|
||||||
|
else "not_found"
|
||||||
|
if postbox_receipts is not None
|
||||||
|
else "unavailable"
|
||||||
|
)
|
||||||
|
if receipt_summary is not None:
|
||||||
|
payload["receipt_summary"] = _postbox_receipt_summary_payload(
|
||||||
|
receipt_summary
|
||||||
|
)
|
||||||
|
postbox_payloads.append(payload)
|
||||||
|
action_attempts_by_action = {
|
||||||
|
attempt.action_id: attempt
|
||||||
|
for attempt in message_action_attempts
|
||||||
|
}
|
||||||
|
message_action_payloads: list[dict[str, object]] = []
|
||||||
|
for action in message_actions:
|
||||||
|
action_attempt = action_attempts_by_action.get(action.id)
|
||||||
|
payload = {
|
||||||
|
"id": action.id,
|
||||||
|
"kind": action.kind,
|
||||||
|
"status": action.status,
|
||||||
|
"reason": action.reason,
|
||||||
|
"actor_user_id": action.actor_user_id,
|
||||||
|
"actor_api_key_id": action.actor_api_key_id,
|
||||||
|
"campaign_version_id": action.campaign_version_id,
|
||||||
|
"message_sha256": action.message_sha256,
|
||||||
|
"recipient_manifest_sha256": action.recipient_manifest_sha256,
|
||||||
|
"recipient_count": action.recipient_count,
|
||||||
|
"prior_send_status": action.prior_send_status,
|
||||||
|
"final_send_status": action.final_send_status,
|
||||||
|
"accepted_count": action.accepted_count,
|
||||||
|
"refused_count": action.refused_count,
|
||||||
|
"refusal_summary": action.refusal_summary or {},
|
||||||
|
"linked_send_attempt_id": action.linked_send_attempt_id,
|
||||||
|
"created_at": action.created_at,
|
||||||
|
"effect_started_at": action.effect_started_at,
|
||||||
|
"completed_at": action.completed_at,
|
||||||
|
"attempt": (
|
||||||
|
{
|
||||||
|
"id": action_attempt.id,
|
||||||
|
"status": action_attempt.status,
|
||||||
|
"started_at": action_attempt.started_at,
|
||||||
|
"effect_started_at": action_attempt.effect_started_at,
|
||||||
|
"completed_at": action_attempt.completed_at,
|
||||||
|
"accepted_count": action_attempt.accepted_count,
|
||||||
|
"refused_count": action_attempt.refused_count,
|
||||||
|
}
|
||||||
|
if action_attempt is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
}
|
||||||
|
if include_diagnostics:
|
||||||
|
payload["idempotency_key"] = action.idempotency_key
|
||||||
|
payload["canonical_request_hash"] = action.canonical_request_hash
|
||||||
|
payload["context"] = action.context or {}
|
||||||
|
payload["error_type"] = action.error_type
|
||||||
|
payload["error_message"] = action.error_message
|
||||||
|
if action_attempt is not None:
|
||||||
|
payload["attempt"] = {
|
||||||
|
**dict(payload["attempt"] or {}),
|
||||||
|
"outcome_code": action_attempt.outcome_code,
|
||||||
|
"diagnostic_summary": action_attempt.diagnostic_summary,
|
||||||
|
}
|
||||||
|
message_action_payloads.append(payload)
|
||||||
|
return {
|
||||||
|
"smtp": smtp_payloads,
|
||||||
|
"imap": imap_payloads,
|
||||||
|
"postbox": postbox_payloads,
|
||||||
|
"message_actions": message_action_payloads,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _postbox_receipt_summary_payload(
|
||||||
|
summary: PostboxDeliveryReceiptSummaryRef,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"delivery_id": summary.delivery_id,
|
||||||
|
"message_id": summary.message_id,
|
||||||
|
"postbox_id": summary.postbox_id,
|
||||||
|
"delivery_status": summary.delivery_status,
|
||||||
|
"accepted_at": summary.accepted_at,
|
||||||
|
"current_holder_count": summary.current_holder_count,
|
||||||
|
"currently_readable": summary.currently_readable,
|
||||||
|
"message_count": summary.message_count,
|
||||||
|
"routed_message_count": summary.routed_message_count,
|
||||||
|
"readable_message_count": summary.readable_message_count,
|
||||||
|
"read_receipt_count": summary.read_receipt_count,
|
||||||
|
"acknowledged_receipt_count": summary.acknowledged_receipt_count,
|
||||||
|
"withdrawn_message_count": summary.withdrawn_message_count,
|
||||||
|
"expired_message_count": summary.expired_message_count,
|
||||||
|
"first_read_at": summary.first_read_at,
|
||||||
|
"last_read_at": summary.last_read_at,
|
||||||
|
"first_acknowledged_at": summary.first_acknowledged_at,
|
||||||
|
"last_acknowledged_at": summary.last_acknowledged_at,
|
||||||
|
"route_status_counts": dict(summary.route_status_counts),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _job_diagnostics_payload(
|
||||||
|
job: CampaignJob,
|
||||||
|
send_attempts: list[SendAttempt],
|
||||||
|
imap_attempts: list[ImapAppendAttempt],
|
||||||
|
postbox_attempts: Sequence[PostboxDeliveryAttempt] = (),
|
||||||
|
message_actions: Sequence[CampaignMessageAction] = (),
|
||||||
|
message_action_attempts: Sequence[CampaignMessageActionAttempt] = (),
|
||||||
|
*,
|
||||||
|
postbox_receipts: (
|
||||||
|
Mapping[str, PostboxDeliveryReceiptSummaryRef] | None
|
||||||
|
) = None,
|
||||||
|
) -> CampaignJobDiagnosticsResponse:
|
||||||
|
return CampaignJobDiagnosticsResponse(
|
||||||
|
job_id=job.id,
|
||||||
|
campaign_id=job.campaign_id,
|
||||||
|
campaign_version_id=job.campaign_version_id,
|
||||||
|
storage={
|
||||||
|
"eml_local_path": job.eml_local_path,
|
||||||
|
"eml_storage_key": job.eml_storage_key,
|
||||||
|
"eml_size_bytes": job.eml_size_bytes,
|
||||||
|
"eml_sha256": job.eml_sha256,
|
||||||
|
},
|
||||||
|
worker_claim={
|
||||||
|
"claim_token": job.claim_token,
|
||||||
|
"claimed_at": job.claimed_at,
|
||||||
|
"smtp_started_at": job.smtp_started_at,
|
||||||
|
"outcome_unknown_at": job.outcome_unknown_at,
|
||||||
|
"last_error": job.last_error,
|
||||||
|
},
|
||||||
|
attempts=_job_attempts_payload(
|
||||||
|
send_attempts,
|
||||||
|
imap_attempts,
|
||||||
|
postbox_attempts,
|
||||||
|
message_actions,
|
||||||
|
message_action_attempts,
|
||||||
|
postbox_receipts=postbox_receipts,
|
||||||
|
include_diagnostics=True,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _review_metadata(
|
||||||
|
session: Session,
|
||||||
|
version: CampaignVersion | None,
|
||||||
|
base_filters: list[object],
|
||||||
|
) -> tuple[dict[str, object], set[str]]:
|
||||||
|
if version is None:
|
||||||
|
return _empty_review_metadata(), set()
|
||||||
|
|
||||||
|
review_state, state_is_current = _current_review_state(version)
|
||||||
|
reviewed_keys = _current_reviewed_keys(
|
||||||
|
review_state, state_is_current=state_is_current
|
||||||
|
)
|
||||||
|
counts = _review_metadata_counts(_review_rows(session, base_filters), reviewed_keys)
|
||||||
|
return {
|
||||||
|
"inspection_complete": bool(
|
||||||
|
state_is_current and review_state.get("inspection_complete") is True
|
||||||
|
),
|
||||||
|
**counts,
|
||||||
|
}, reviewed_keys
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_review_metadata() -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"inspection_complete": False,
|
||||||
|
"blocking_count": 0,
|
||||||
|
"required_count": 0,
|
||||||
|
"reviewed_required_count": 0,
|
||||||
|
"bulk_acceptable_count": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _current_review_state(version: CampaignVersion) -> tuple[dict[str, object], bool]:
|
||||||
|
build_summary = (
|
||||||
|
version.build_summary if isinstance(version.build_summary, dict) else {}
|
||||||
|
)
|
||||||
|
build_token = str(
|
||||||
|
build_summary.get("build_token") or build_summary.get("built_at") or ""
|
||||||
|
)
|
||||||
|
editor_state = (
|
||||||
|
version.editor_state if isinstance(version.editor_state, dict) else {}
|
||||||
|
)
|
||||||
|
review_state = (
|
||||||
|
editor_state.get("review_send")
|
||||||
|
if isinstance(editor_state.get("review_send"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
state_token = str(review_state.get("build_token") or "")
|
||||||
|
return review_state, bool(build_token and state_token == build_token)
|
||||||
|
|
||||||
|
|
||||||
|
def _current_reviewed_keys(
|
||||||
|
review_state: dict[str, object], *, state_is_current: bool
|
||||||
|
) -> set[str]:
|
||||||
|
return {
|
||||||
|
str(value)
|
||||||
|
for value in (review_state.get("reviewed_message_keys") or [])
|
||||||
|
if state_is_current and str(value).strip()
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _review_rows(
|
||||||
|
session: Session, base_filters: list[object]
|
||||||
|
) -> list[tuple[object, int, str, str]]:
|
||||||
|
return (
|
||||||
|
session.query(
|
||||||
|
CampaignJob.entry_id,
|
||||||
|
CampaignJob.entry_index,
|
||||||
|
CampaignJob.build_status,
|
||||||
|
CampaignJob.validation_status,
|
||||||
|
)
|
||||||
|
.filter(*base_filters)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _review_metadata_counts(
|
||||||
|
review_rows: list[tuple[object, int, str, str]], reviewed_keys: set[str]
|
||||||
|
) -> dict[str, int]:
|
||||||
|
blocking_count = 0
|
||||||
|
required_count = 0
|
||||||
|
reviewed_required_count = 0
|
||||||
|
bulk_acceptable_count = 0
|
||||||
|
for entry_id, entry_index, build_status, validation_status in review_rows:
|
||||||
|
key = str(entry_id or entry_index)
|
||||||
|
if build_status != "built" or validation_status == "blocked":
|
||||||
|
blocking_count += 1
|
||||||
|
if validation_status == "needs_review":
|
||||||
|
required_count += 1
|
||||||
|
if key in reviewed_keys:
|
||||||
|
reviewed_required_count += 1
|
||||||
|
elif validation_status in {"warning", "excluded"}:
|
||||||
|
bulk_acceptable_count += 1
|
||||||
|
|
||||||
|
return {
|
||||||
|
"blocking_count": blocking_count,
|
||||||
|
"required_count": required_count,
|
||||||
|
"reviewed_required_count": reviewed_required_count,
|
||||||
|
"bulk_acceptable_count": bulk_acceptable_count,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _status_counts(
|
||||||
|
session: Session, filters: list[object]
|
||||||
|
) -> dict[str, dict[str, int]]:
|
||||||
|
result: dict[str, dict[str, int]] = {}
|
||||||
|
for field_name in (
|
||||||
|
"build_status",
|
||||||
|
"validation_status",
|
||||||
|
"queue_status",
|
||||||
|
"send_status",
|
||||||
|
"postbox_status",
|
||||||
|
"imap_status",
|
||||||
|
):
|
||||||
|
column = getattr(CampaignJob, field_name)
|
||||||
|
rows = (
|
||||||
|
session.query(column, func.count(CampaignJob.id))
|
||||||
|
.filter(*filters)
|
||||||
|
.group_by(column)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
result[field_name.removesuffix("_status")] = {
|
||||||
|
str(value or "unknown"): int(count) for value, count in rows
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
CAMPAIGN_JOB_GRID_SORT_COLUMNS = {
|
||||||
|
"number": CampaignJob.entry_index,
|
||||||
|
"recipient": func.lower(func.coalesce(CampaignJob.recipient_email, "")),
|
||||||
|
"subject": func.lower(func.coalesce(CampaignJob.subject, "")),
|
||||||
|
"validation": CampaignJob.validation_status,
|
||||||
|
"queue": CampaignJob.queue_status,
|
||||||
|
"send": CampaignJob.send_status,
|
||||||
|
"postbox": CampaignJob.postbox_status,
|
||||||
|
"imap": CampaignJob.imap_status,
|
||||||
|
"attempts": CampaignJob.attempt_count,
|
||||||
|
"updated": CampaignJob.updated_at,
|
||||||
|
}
|
||||||
|
CAMPAIGN_JOB_GRID_LIST_FILTERS = {
|
||||||
|
"validation": (
|
||||||
|
CampaignJob.validation_status,
|
||||||
|
{item.value for item in JobValidationStatus},
|
||||||
|
),
|
||||||
|
"queue": (CampaignJob.queue_status, {item.value for item in JobQueueStatus}),
|
||||||
|
"send": (CampaignJob.send_status, {item.value for item in JobSendStatus}),
|
||||||
|
"postbox": (
|
||||||
|
CampaignJob.postbox_status,
|
||||||
|
{item.value for item in JobPostboxStatus},
|
||||||
|
),
|
||||||
|
"imap": (CampaignJob.imap_status, {item.value for item in JobImapStatus}),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_grid_filter_expressions(
|
||||||
|
grid_filters: dict[str, str] | None,
|
||||||
|
) -> list[object]:
|
||||||
|
values = grid_filters or {}
|
||||||
|
expressions: list[object] = []
|
||||||
|
recipient = values.get("recipient", "").strip()
|
||||||
|
if recipient:
|
||||||
|
pattern = _contains_pattern(recipient)
|
||||||
|
expressions.append(
|
||||||
|
or_(
|
||||||
|
CampaignJob.recipient_email.ilike(pattern, escape="\\"),
|
||||||
|
CampaignJob.entry_id.ilike(pattern, escape="\\"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
subject = values.get("subject", "").strip()
|
||||||
|
if subject:
|
||||||
|
expressions.append(
|
||||||
|
CampaignJob.subject.ilike(_contains_pattern(subject), escape="\\")
|
||||||
|
)
|
||||||
|
evidence = values.get("evidence", "").strip()
|
||||||
|
if evidence:
|
||||||
|
pattern = _contains_pattern(evidence)
|
||||||
|
expressions.append(
|
||||||
|
or_(
|
||||||
|
CampaignJob.message_id_header.ilike(pattern, escape="\\"),
|
||||||
|
CampaignJob.eml_sha256.ilike(pattern, escape="\\"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
attempts = values.get("attempts", "").strip()
|
||||||
|
if attempts:
|
||||||
|
expressions.append(
|
||||||
|
_campaign_jobs_integer_filter(
|
||||||
|
CampaignJob.attempt_count, attempts, column_id="attempts"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for column_id, (column, allowed_values) in CAMPAIGN_JOB_GRID_LIST_FILTERS.items():
|
||||||
|
raw_value = values.get(column_id, "").strip()
|
||||||
|
if not raw_value:
|
||||||
|
continue
|
||||||
|
selected = _campaign_jobs_list_filter(
|
||||||
|
raw_value, column_id=column_id, allowed_values=allowed_values
|
||||||
|
)
|
||||||
|
expressions.append(column.in_(selected))
|
||||||
|
return expressions
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_list_filter(
|
||||||
|
raw_value: str, *, column_id: str, allowed_values: set[str]
|
||||||
|
) -> list[str]:
|
||||||
|
if raw_value.startswith("list:"):
|
||||||
|
try:
|
||||||
|
parsed = json.loads(raw_value[5:])
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=f"Invalid {column_id} list filter",
|
||||||
|
) from exc
|
||||||
|
if not isinstance(parsed, list) or any(
|
||||||
|
not isinstance(value, str) for value in parsed
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=f"Invalid {column_id} list filter",
|
||||||
|
)
|
||||||
|
selected = list(
|
||||||
|
dict.fromkeys(value.strip() for value in parsed if value.strip())
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
selected = list(
|
||||||
|
dict.fromkeys(
|
||||||
|
value.strip() for value in raw_value.split(",") if value.strip()
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if len(selected) > 50 or any(value not in allowed_values for value in selected):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=f"Invalid {column_id} list filter",
|
||||||
|
)
|
||||||
|
return selected
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_integer_filter(column: object, raw_value: str, *, column_id: str):
|
||||||
|
operator, separator, value = raw_value.partition(":")
|
||||||
|
if not separator:
|
||||||
|
operator, value = "eq", operator
|
||||||
|
if operator not in {"eq", "gt", "gte", "lt", "lte"}:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=f"Invalid {column_id} filter operator",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
expected = int(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail=f"Invalid {column_id} filter value",
|
||||||
|
) from exc
|
||||||
|
if operator == "gt":
|
||||||
|
return column > expected
|
||||||
|
if operator == "gte":
|
||||||
|
return column >= expected
|
||||||
|
if operator == "lt":
|
||||||
|
return column < expected
|
||||||
|
if operator == "lte":
|
||||||
|
return column <= expected
|
||||||
|
return column == expected
|
||||||
|
|
||||||
|
|
||||||
|
def _contains_pattern(value: str) -> str:
|
||||||
|
escaped = value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||||
|
return f"%{escaped}%"
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_ordering(sort_by: str, sort_direction: str) -> list[object]:
|
||||||
|
column = CAMPAIGN_JOB_GRID_SORT_COLUMNS.get(sort_by)
|
||||||
|
if column is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||||
|
detail="Unsupported Campaign job sort column",
|
||||||
|
)
|
||||||
|
primary = column.desc() if sort_direction == "desc" else column.asc()
|
||||||
|
return [primary, CampaignJob.id.asc()]
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_query_context(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None,
|
||||||
|
send_status: list[str] | None,
|
||||||
|
validation_status: list[str] | None,
|
||||||
|
imap_status: list[str] | None,
|
||||||
|
query_text: str | None,
|
||||||
|
grid_filters: dict[str, str] | None = None,
|
||||||
|
) -> tuple[Campaign, list[object], list[object], dict[str, object], set[str]]:
|
||||||
|
_get_campaign_for_principal(session, campaign_id, principal)
|
||||||
|
_require_permission(principal, "campaigns:recipient:read")
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
base_filters: list[object] = [
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.tenant_id == principal.tenant_id,
|
||||||
|
]
|
||||||
|
selected_version: CampaignVersion | None = None
|
||||||
|
if version_id:
|
||||||
|
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||||
|
if version.campaign_id != campaign.id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail="Campaign version not found",
|
||||||
|
)
|
||||||
|
selected_version = version
|
||||||
|
base_filters.append(CampaignJob.campaign_version_id == version.id)
|
||||||
|
|
||||||
|
review_metadata, reviewed_keys = _review_metadata(
|
||||||
|
session, selected_version, base_filters
|
||||||
|
)
|
||||||
|
filtered = list(base_filters)
|
||||||
|
if send_status:
|
||||||
|
filtered.append(CampaignJob.send_status.in_(send_status))
|
||||||
|
if validation_status:
|
||||||
|
filtered.append(CampaignJob.validation_status.in_(validation_status))
|
||||||
|
if imap_status:
|
||||||
|
filtered.append(CampaignJob.imap_status.in_(imap_status))
|
||||||
|
if query_text and query_text.strip():
|
||||||
|
pattern = f"%{query_text.strip()}%"
|
||||||
|
filtered.append(
|
||||||
|
or_(
|
||||||
|
CampaignJob.recipient_email.ilike(pattern),
|
||||||
|
CampaignJob.subject.ilike(pattern),
|
||||||
|
CampaignJob.entry_id.ilike(pattern),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
filtered.extend(_campaign_jobs_grid_filter_expressions(grid_filters))
|
||||||
|
return campaign, base_filters, filtered, review_metadata, reviewed_keys
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_page_response(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None,
|
||||||
|
base_filters: list[object],
|
||||||
|
filtered: list[object],
|
||||||
|
reviewed_keys: set[str],
|
||||||
|
review_metadata: dict[str, object],
|
||||||
|
page: int,
|
||||||
|
page_size: int,
|
||||||
|
send_status: list[str] | None = None,
|
||||||
|
validation_status: list[str] | None = None,
|
||||||
|
imap_status: list[str] | None = None,
|
||||||
|
query_text: str | None = None,
|
||||||
|
grid_filters: dict[str, str] | None = None,
|
||||||
|
sort_by: str = "number",
|
||||||
|
sort_direction: str = "asc",
|
||||||
|
cursor: str | None = None,
|
||||||
|
changed_job_ids: set[str] | None = None,
|
||||||
|
) -> CampaignJobsResponse:
|
||||||
|
total_unfiltered, total = _campaign_jobs_page_counts(
|
||||||
|
session, base_filters=base_filters, filtered=filtered
|
||||||
|
)
|
||||||
|
pages = (total + page_size - 1) // page_size if total else 0
|
||||||
|
fingerprint = _campaign_jobs_cursor_fingerprint(
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
page_size=page_size,
|
||||||
|
send_status=send_status,
|
||||||
|
validation_status=validation_status,
|
||||||
|
imap_status=imap_status,
|
||||||
|
query_text=query_text,
|
||||||
|
grid_filters=grid_filters,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
)
|
||||||
|
ordering = _campaign_jobs_ordering(sort_by, sort_direction)
|
||||||
|
rows_plus_one, start_cursor = _campaign_jobs_page_rows(
|
||||||
|
session,
|
||||||
|
filtered=filtered,
|
||||||
|
ordering=ordering,
|
||||||
|
page=page,
|
||||||
|
page_size=page_size,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
cursor=cursor,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
)
|
||||||
|
jobs = rows_plus_one[:page_size]
|
||||||
|
next_cursor = _campaign_jobs_next_cursor(
|
||||||
|
jobs,
|
||||||
|
rows_plus_one=rows_plus_one,
|
||||||
|
page_size=page_size,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
changed_job_ids=changed_job_ids,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
)
|
||||||
|
if changed_job_ids is not None:
|
||||||
|
jobs = [job for job in jobs if job.id in changed_job_ids]
|
||||||
|
return CampaignJobsResponse(
|
||||||
|
jobs=[_job_summary_payload(job, reviewed_keys=reviewed_keys) for job in jobs],
|
||||||
|
page=page,
|
||||||
|
page_size=page_size,
|
||||||
|
total=total,
|
||||||
|
total_unfiltered=total_unfiltered,
|
||||||
|
pages=pages,
|
||||||
|
cursor=start_cursor,
|
||||||
|
next_cursor=next_cursor,
|
||||||
|
counts=_status_counts(session, base_filters),
|
||||||
|
filtered_counts=_status_counts(session, filtered),
|
||||||
|
review=review_metadata,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_page_counts(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
base_filters: list[object],
|
||||||
|
filtered: list[object],
|
||||||
|
) -> tuple[int, int]:
|
||||||
|
total_unfiltered = int(
|
||||||
|
session.query(func.count(CampaignJob.id)).filter(*base_filters).scalar() or 0
|
||||||
|
)
|
||||||
|
total = int(
|
||||||
|
session.query(func.count(CampaignJob.id)).filter(*filtered).scalar() or 0
|
||||||
|
)
|
||||||
|
return total_unfiltered, total
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_page_rows(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
filtered: list[object],
|
||||||
|
ordering: list[object],
|
||||||
|
page: int,
|
||||||
|
page_size: int,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
cursor: str | None,
|
||||||
|
fingerprint: str,
|
||||||
|
) -> tuple[list[CampaignJob], str | None]:
|
||||||
|
if cursor:
|
||||||
|
page_query = _campaign_jobs_query_after_cursor(
|
||||||
|
session,
|
||||||
|
filtered=filtered,
|
||||||
|
cursor=cursor,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
)
|
||||||
|
return page_query.order_by(*ordering).limit(page_size + 1).all(), cursor
|
||||||
|
|
||||||
|
effective_offset = (page - 1) * page_size
|
||||||
|
page_query = session.query(CampaignJob).filter(*filtered)
|
||||||
|
start_cursor = _campaign_jobs_offset_cursor(
|
||||||
|
page_query,
|
||||||
|
ordering=ordering,
|
||||||
|
effective_offset=effective_offset,
|
||||||
|
sort_by=sort_by,
|
||||||
|
sort_direction=sort_direction,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
)
|
||||||
|
rows = (
|
||||||
|
page_query.order_by(*ordering)
|
||||||
|
.offset(effective_offset)
|
||||||
|
.limit(page_size + 1)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return rows, start_cursor
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_query_after_cursor(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
filtered: list[object],
|
||||||
|
cursor: str,
|
||||||
|
fingerprint: str,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
):
|
||||||
|
if sort_by != "number" or sort_direction != "asc":
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Campaign job cursors require number ascending order",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
cursor_values = decode_keyset_cursor(
|
||||||
|
CAMPAIGN_JOBS_CURSOR_SCOPE, cursor, fingerprint=fingerprint
|
||||||
|
)
|
||||||
|
if cursor_values is None:
|
||||||
|
raise KeysetCursorError("Invalid pagination cursor")
|
||||||
|
except KeysetCursorError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
|
||||||
|
) from exc
|
||||||
|
return (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(*filtered)
|
||||||
|
.filter(_campaign_jobs_cursor_condition(cursor_values))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_offset_cursor(
|
||||||
|
query,
|
||||||
|
*,
|
||||||
|
ordering: list[object],
|
||||||
|
effective_offset: int,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
fingerprint: str,
|
||||||
|
) -> str | None:
|
||||||
|
if effective_offset <= 0 or sort_by != "number" or sort_direction != "asc":
|
||||||
|
return None
|
||||||
|
previous_row = (
|
||||||
|
query.order_by(*ordering).offset(effective_offset - 1).limit(1).first()
|
||||||
|
)
|
||||||
|
if previous_row is None:
|
||||||
|
return None
|
||||||
|
return _campaign_jobs_cursor_for_row(previous_row, fingerprint=fingerprint)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_next_cursor(
|
||||||
|
jobs: list[CampaignJob],
|
||||||
|
*,
|
||||||
|
rows_plus_one: list[CampaignJob],
|
||||||
|
page_size: int,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
changed_job_ids: set[str] | None,
|
||||||
|
fingerprint: str,
|
||||||
|
) -> str | None:
|
||||||
|
cursor_supported = sort_by == "number" and sort_direction == "asc"
|
||||||
|
if (
|
||||||
|
changed_job_ids is not None
|
||||||
|
or not cursor_supported
|
||||||
|
or len(rows_plus_one) <= page_size
|
||||||
|
or not jobs
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
return _campaign_jobs_cursor_for_row(jobs[-1], fingerprint=fingerprint)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_cursor_fingerprint(
|
||||||
|
*,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None,
|
||||||
|
page_size: int,
|
||||||
|
send_status: list[str] | None,
|
||||||
|
validation_status: list[str] | None,
|
||||||
|
imap_status: list[str] | None,
|
||||||
|
query_text: str | None,
|
||||||
|
grid_filters: dict[str, str] | None,
|
||||||
|
sort_by: str,
|
||||||
|
sort_direction: str,
|
||||||
|
) -> str:
|
||||||
|
return keyset_query_fingerprint(
|
||||||
|
CAMPAIGN_JOBS_CURSOR_SCOPE,
|
||||||
|
{
|
||||||
|
"campaign_id": campaign_id,
|
||||||
|
"version_id": version_id or "",
|
||||||
|
"page_size": page_size,
|
||||||
|
"send_status": sorted(send_status or []),
|
||||||
|
"validation_status": sorted(validation_status or []),
|
||||||
|
"imap_status": sorted(imap_status or []),
|
||||||
|
"query": (query_text or "").strip(),
|
||||||
|
"grid_filters": sorted((grid_filters or {}).items()),
|
||||||
|
"order": [f"{sort_by}:{sort_direction}", "id:asc"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_cursor_for_row(row: CampaignJob, *, fingerprint: str) -> str:
|
||||||
|
return encode_keyset_cursor(
|
||||||
|
CAMPAIGN_JOBS_CURSOR_SCOPE,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
values={"id": row.id, "entry_index": row.entry_index},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_cursor_condition(cursor_values: dict[str, object]):
|
||||||
|
cursor_id = cursor_values.get("id")
|
||||||
|
cursor_index = cursor_values.get("entry_index")
|
||||||
|
if not isinstance(cursor_id, str) or not cursor_id:
|
||||||
|
raise KeysetCursorError("Invalid pagination cursor")
|
||||||
|
try:
|
||||||
|
entry_index = int(cursor_index)
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise KeysetCursorError("Invalid pagination cursor") from exc
|
||||||
|
return or_(
|
||||||
|
CampaignJob.entry_index > entry_index,
|
||||||
|
and_(CampaignJob.entry_index == entry_index, CampaignJob.id > cursor_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_jobs_delta_watermark(session: Session, tenant_id: str) -> str:
|
||||||
|
return encode_sequence_watermark(
|
||||||
|
max_sequence_id(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
module_id=CAMPAIGNS_MODULE_ID,
|
||||||
|
collections=(CAMPAIGN_JOBS_COLLECTION,),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignJobsQuery:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
version_id: str | None = None,
|
||||||
|
page: int = Query(default=1, ge=1),
|
||||||
|
page_size: int = Query(default=50, ge=1, le=200),
|
||||||
|
cursor: str | None = Query(default=None),
|
||||||
|
send_status: list[str] | None = Query(default=None),
|
||||||
|
validation_status: list[str] | None = Query(default=None),
|
||||||
|
imap_status: list[str] | None = Query(default=None),
|
||||||
|
query_text: str | None = Query(default=None, alias="q", max_length=200),
|
||||||
|
sort_by: Literal[
|
||||||
|
"number",
|
||||||
|
"recipient",
|
||||||
|
"subject",
|
||||||
|
"validation",
|
||||||
|
"queue",
|
||||||
|
"send",
|
||||||
|
"postbox",
|
||||||
|
"imap",
|
||||||
|
"attempts",
|
||||||
|
"updated",
|
||||||
|
] = Query(default="number"),
|
||||||
|
sort_direction: Literal["asc", "desc"] = Query(default="asc"),
|
||||||
|
filter_recipient: str | None = Query(default=None, max_length=500),
|
||||||
|
filter_subject: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_validation: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_queue: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_send: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_postbox: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_imap: str | None = Query(default=None, max_length=1000),
|
||||||
|
filter_attempts: str | None = Query(default=None, max_length=100),
|
||||||
|
filter_evidence: str | None = Query(default=None, max_length=500),
|
||||||
|
) -> None:
|
||||||
|
self.version_id = version_id
|
||||||
|
self.page = page
|
||||||
|
self.page_size = page_size
|
||||||
|
self.cursor = cursor
|
||||||
|
self.send_status = send_status
|
||||||
|
self.validation_status = validation_status
|
||||||
|
self.imap_status = imap_status
|
||||||
|
self.query_text = query_text
|
||||||
|
self.sort_by = sort_by
|
||||||
|
self.sort_direction = sort_direction
|
||||||
|
self.grid_filters = {
|
||||||
|
column_id: value
|
||||||
|
for column_id, value in {
|
||||||
|
"recipient": filter_recipient,
|
||||||
|
"subject": filter_subject,
|
||||||
|
"validation": filter_validation,
|
||||||
|
"queue": filter_queue,
|
||||||
|
"send": filter_send,
|
||||||
|
"postbox": filter_postbox,
|
||||||
|
"imap": filter_imap,
|
||||||
|
"attempts": filter_attempts,
|
||||||
|
"evidence": filter_evidence,
|
||||||
|
}.items()
|
||||||
|
if value is not None and value.strip()
|
||||||
|
}
|
||||||
@@ -1,8 +1,14 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import binascii
|
||||||
|
from datetime import datetime
|
||||||
|
import secrets
|
||||||
|
import stat
|
||||||
|
import struct
|
||||||
import zipfile
|
import zipfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Iterable
|
from typing import Iterable
|
||||||
|
import zlib
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import pyzipper
|
import pyzipper
|
||||||
@@ -10,6 +16,8 @@ except ImportError: # pragma: no cover
|
|||||||
pyzipper = None
|
pyzipper = None
|
||||||
|
|
||||||
ArchiveMember = tuple[Path, str]
|
ArchiveMember = tuple[Path, str]
|
||||||
|
ZIP_METHOD_AES = "aes"
|
||||||
|
ZIP_METHOD_STANDARD = "zip_standard"
|
||||||
|
|
||||||
|
|
||||||
def _normalized_members(files: Iterable[Path | ArchiveMember]) -> list[ArchiveMember]:
|
def _normalized_members(files: Iterable[Path | ArchiveMember]) -> list[ArchiveMember]:
|
||||||
@@ -33,24 +41,18 @@ def _normalized_members(files: Iterable[Path | ArchiveMember]) -> list[ArchiveMe
|
|||||||
def create_zip_archive(
|
def create_zip_archive(
|
||||||
output_path: Path,
|
output_path: Path,
|
||||||
files: Iterable[Path | ArchiveMember],
|
files: Iterable[Path | ArchiveMember],
|
||||||
password: str = "",
|
password: str = "", # nosec B107 - empty means an unencrypted archive.
|
||||||
|
method: str = ZIP_METHOD_AES,
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Create a ZIP archive, using AES encryption when a password is supplied."""
|
"""Create a ZIP archive, optionally using AES or legacy ZipCrypto encryption."""
|
||||||
|
|
||||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
members = _normalized_members(files)
|
members = _normalized_members(files)
|
||||||
if password:
|
if password:
|
||||||
if pyzipper is None:
|
if method == ZIP_METHOD_STANDARD:
|
||||||
raise RuntimeError("pyzipper is required for writing password-protected ZIP files")
|
_create_zipcrypto_archive(output_path, members, password)
|
||||||
with pyzipper.AESZipFile(
|
return output_path
|
||||||
output_path,
|
_create_aes_archive(output_path, members, password)
|
||||||
"w",
|
|
||||||
compression=pyzipper.ZIP_DEFLATED,
|
|
||||||
encryption=pyzipper.WZ_AES,
|
|
||||||
) as zip_file:
|
|
||||||
zip_file.setpassword(password.encode("utf-8"))
|
|
||||||
for file_path, archive_name in members:
|
|
||||||
zip_file.write(file_path, arcname=archive_name)
|
|
||||||
return output_path
|
return output_path
|
||||||
|
|
||||||
with zipfile.ZipFile(output_path, "w", compression=zipfile.ZIP_DEFLATED) as zip_file:
|
with zipfile.ZipFile(output_path, "w", compression=zipfile.ZIP_DEFLATED) as zip_file:
|
||||||
@@ -59,7 +61,214 @@ def create_zip_archive(
|
|||||||
return output_path
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
def create_encrypted_zip(output_path: Path, files: list[Path], password: str) -> Path:
|
def create_encrypted_zip(output_path: Path, files: list[Path], password: str, method: str = ZIP_METHOD_AES) -> Path:
|
||||||
"""Backward-compatible wrapper for the original per-rule ZIP helper."""
|
"""Backward-compatible wrapper for the original per-rule ZIP helper."""
|
||||||
|
|
||||||
return create_zip_archive(output_path, files, password)
|
return create_zip_archive(output_path, files, password, method)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_aes_archive(output_path: Path, members: list[ArchiveMember], password: str) -> None:
|
||||||
|
if pyzipper is None:
|
||||||
|
raise RuntimeError("pyzipper is required for writing AES password-protected ZIP files")
|
||||||
|
with pyzipper.AESZipFile(
|
||||||
|
output_path,
|
||||||
|
"w",
|
||||||
|
compression=pyzipper.ZIP_DEFLATED,
|
||||||
|
encryption=pyzipper.WZ_AES,
|
||||||
|
) as zip_file:
|
||||||
|
zip_file.setpassword(password.encode("utf-8"))
|
||||||
|
for file_path, archive_name in members:
|
||||||
|
zip_file.write(file_path, arcname=archive_name)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_zipcrypto_archive(output_path: Path, members: list[ArchiveMember], password: str) -> None:
|
||||||
|
entries: list[_CentralDirectoryEntry] = []
|
||||||
|
password_bytes = password.encode("utf-8")
|
||||||
|
with output_path.open("wb") as zip_file:
|
||||||
|
for file_path, archive_name in members:
|
||||||
|
local_header_offset = zip_file.tell()
|
||||||
|
file_data = file_path.read_bytes()
|
||||||
|
crc = binascii.crc32(file_data) & 0xFFFFFFFF
|
||||||
|
compressed_data = _deflate(file_data)
|
||||||
|
encrypted_data = _zipcrypto_encrypt(compressed_data, password_bytes, crc)
|
||||||
|
compressed_size = len(encrypted_data)
|
||||||
|
uncompressed_size = len(file_data)
|
||||||
|
modified_at = datetime.fromtimestamp(file_path.stat().st_mtime)
|
||||||
|
dos_time, dos_date = _dos_timestamp(modified_at)
|
||||||
|
filename_bytes, flags = _filename_bytes(archive_name, encrypted=True)
|
||||||
|
|
||||||
|
zip_file.write(
|
||||||
|
struct.pack(
|
||||||
|
"<IHHHHHIIIHH",
|
||||||
|
0x04034B50,
|
||||||
|
20,
|
||||||
|
flags,
|
||||||
|
zipfile.ZIP_DEFLATED,
|
||||||
|
dos_time,
|
||||||
|
dos_date,
|
||||||
|
crc,
|
||||||
|
compressed_size,
|
||||||
|
uncompressed_size,
|
||||||
|
len(filename_bytes),
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
zip_file.write(filename_bytes)
|
||||||
|
zip_file.write(encrypted_data)
|
||||||
|
entries.append(
|
||||||
|
_CentralDirectoryEntry(
|
||||||
|
filename_bytes=filename_bytes,
|
||||||
|
flags=flags,
|
||||||
|
dos_time=dos_time,
|
||||||
|
dos_date=dos_date,
|
||||||
|
crc=crc,
|
||||||
|
compressed_size=compressed_size,
|
||||||
|
uncompressed_size=uncompressed_size,
|
||||||
|
external_attributes=_external_attributes(file_path),
|
||||||
|
local_header_offset=local_header_offset,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
central_directory_offset = zip_file.tell()
|
||||||
|
for entry in entries:
|
||||||
|
zip_file.write(
|
||||||
|
struct.pack(
|
||||||
|
"<IHHHHHHIIIHHHHHII",
|
||||||
|
0x02014B50,
|
||||||
|
20,
|
||||||
|
20,
|
||||||
|
entry.flags,
|
||||||
|
zipfile.ZIP_DEFLATED,
|
||||||
|
entry.dos_time,
|
||||||
|
entry.dos_date,
|
||||||
|
entry.crc,
|
||||||
|
entry.compressed_size,
|
||||||
|
entry.uncompressed_size,
|
||||||
|
len(entry.filename_bytes),
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
entry.external_attributes,
|
||||||
|
entry.local_header_offset,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
zip_file.write(entry.filename_bytes)
|
||||||
|
|
||||||
|
central_directory_size = zip_file.tell() - central_directory_offset
|
||||||
|
zip_file.write(
|
||||||
|
struct.pack(
|
||||||
|
"<IHHHHIIH",
|
||||||
|
0x06054B50,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
len(entries),
|
||||||
|
len(entries),
|
||||||
|
central_directory_size,
|
||||||
|
central_directory_offset,
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _CentralDirectoryEntry:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
filename_bytes: bytes,
|
||||||
|
flags: int,
|
||||||
|
dos_time: int,
|
||||||
|
dos_date: int,
|
||||||
|
crc: int,
|
||||||
|
compressed_size: int,
|
||||||
|
uncompressed_size: int,
|
||||||
|
external_attributes: int,
|
||||||
|
local_header_offset: int,
|
||||||
|
) -> None:
|
||||||
|
self.filename_bytes = filename_bytes
|
||||||
|
self.flags = flags
|
||||||
|
self.dos_time = dos_time
|
||||||
|
self.dos_date = dos_date
|
||||||
|
self.crc = crc
|
||||||
|
self.compressed_size = compressed_size
|
||||||
|
self.uncompressed_size = uncompressed_size
|
||||||
|
self.external_attributes = external_attributes
|
||||||
|
self.local_header_offset = local_header_offset
|
||||||
|
|
||||||
|
|
||||||
|
def _deflate(data: bytes) -> bytes:
|
||||||
|
compressor = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -zlib.MAX_WBITS)
|
||||||
|
return compressor.compress(data) + compressor.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def _zipcrypto_encrypt(data: bytes, password: bytes, crc: int) -> bytes:
|
||||||
|
cipher = _ZipCrypto(password)
|
||||||
|
header = secrets.token_bytes(11) + bytes([(crc >> 24) & 0xFF])
|
||||||
|
return cipher.encrypt(header + data)
|
||||||
|
|
||||||
|
|
||||||
|
class _ZipCrypto:
|
||||||
|
def __init__(self, password: bytes) -> None:
|
||||||
|
self.key0 = 0x12345678
|
||||||
|
self.key1 = 0x23456789
|
||||||
|
self.key2 = 0x34567890
|
||||||
|
for value in password:
|
||||||
|
self._update_keys(value)
|
||||||
|
|
||||||
|
def encrypt(self, data: bytes) -> bytes:
|
||||||
|
encrypted = bytearray()
|
||||||
|
for value in data:
|
||||||
|
encrypted.append(value ^ self._decrypt_byte())
|
||||||
|
self._update_keys(value)
|
||||||
|
return bytes(encrypted)
|
||||||
|
|
||||||
|
def _decrypt_byte(self) -> int:
|
||||||
|
temp = self.key2 | 2
|
||||||
|
return ((temp * (temp ^ 1)) >> 8) & 0xFF
|
||||||
|
|
||||||
|
def _update_keys(self, value: int) -> None:
|
||||||
|
self.key0 = _zipcrypto_crc32_byte(value, self.key0)
|
||||||
|
self.key1 = (self.key1 + (self.key0 & 0xFF)) & 0xFFFFFFFF
|
||||||
|
self.key1 = (self.key1 * 134775813 + 1) & 0xFFFFFFFF
|
||||||
|
self.key2 = _zipcrypto_crc32_byte((self.key1 >> 24) & 0xFF, self.key2)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_zipcrypto_crc(value: int) -> int:
|
||||||
|
for _ in range(8):
|
||||||
|
if value & 1:
|
||||||
|
value = 0xEDB88320 ^ (value >> 1)
|
||||||
|
else:
|
||||||
|
value >>= 1
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
_ZIPCRYPTO_CRC_TABLE = [_build_zipcrypto_crc(value) for value in range(256)]
|
||||||
|
|
||||||
|
|
||||||
|
def _zipcrypto_crc32_byte(value: int, crc: int) -> int:
|
||||||
|
return ((crc >> 8) ^ _ZIPCRYPTO_CRC_TABLE[(crc ^ value) & 0xFF]) & 0xFFFFFFFF
|
||||||
|
|
||||||
|
|
||||||
|
def _filename_bytes(filename: str, *, encrypted: bool) -> tuple[bytes, int]:
|
||||||
|
flags = 0x1 if encrypted else 0
|
||||||
|
try:
|
||||||
|
return filename.encode("ascii"), flags
|
||||||
|
except UnicodeEncodeError:
|
||||||
|
return filename.encode("utf-8"), flags | 0x800
|
||||||
|
|
||||||
|
|
||||||
|
def _dos_timestamp(value: datetime) -> tuple[int, int]:
|
||||||
|
year = min(max(value.year, 1980), 2107)
|
||||||
|
month = value.month if value.year == year else 1
|
||||||
|
day = value.day if value.year == year else 1
|
||||||
|
dos_time = value.hour << 11 | value.minute << 5 | value.second // 2
|
||||||
|
dos_date = (year - 1980) << 9 | month << 5 | day
|
||||||
|
return dos_time, dos_date
|
||||||
|
|
||||||
|
|
||||||
|
def _external_attributes(path: Path) -> int:
|
||||||
|
try:
|
||||||
|
permissions = stat.S_IMODE(path.stat().st_mode)
|
||||||
|
except OSError:
|
||||||
|
return 0
|
||||||
|
return permissions << 16
|
||||||
|
|||||||
55
src/govoplan_campaign/backend/template_rendering.py
Normal file
55
src/govoplan_campaign/backend/template_rendering.py
Normal file
@@ -0,0 +1,55 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
_DOLLAR_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
|
||||||
|
_BRACE_FIELD_PATTERN = re.compile(r"(?<!\\)\{\{\s*(.*?)\s*\}\}")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_template_key(raw: str) -> str:
|
||||||
|
key = raw.strip()
|
||||||
|
if key.startswith("fields."):
|
||||||
|
key = key.removeprefix("fields.")
|
||||||
|
elif key.startswith("local."):
|
||||||
|
key = "local::" + key.removeprefix("local.")
|
||||||
|
elif key.startswith("global."):
|
||||||
|
key = "global::" + key.removeprefix("global.")
|
||||||
|
|
||||||
|
if key.startswith("local::") or key.startswith("global::"):
|
||||||
|
return key
|
||||||
|
if key.startswith("local:"):
|
||||||
|
return "local::" + key.removeprefix("local:")
|
||||||
|
if key.startswith("global:"):
|
||||||
|
return "global::" + key.removeprefix("global:")
|
||||||
|
return key
|
||||||
|
|
||||||
|
|
||||||
|
def render_template(
|
||||||
|
template: str,
|
||||||
|
values: Mapping[str, Any],
|
||||||
|
*,
|
||||||
|
keep_missing: bool = True,
|
||||||
|
) -> str:
|
||||||
|
def replace(match: re.Match[str]) -> str:
|
||||||
|
key = normalize_template_key(match.group(1))
|
||||||
|
if key in values:
|
||||||
|
value = values[key]
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
return match.group(0) if keep_missing else ""
|
||||||
|
|
||||||
|
rendered = _DOLLAR_FIELD_PATTERN.sub(replace, template)
|
||||||
|
rendered = _BRACE_FIELD_PATTERN.sub(replace, rendered)
|
||||||
|
return rendered.replace(r"\${", "${").replace(r"\}", "}")
|
||||||
|
|
||||||
|
|
||||||
|
def find_unresolved_placeholders(text: str | None) -> set[str]:
|
||||||
|
if not text:
|
||||||
|
return set()
|
||||||
|
return {
|
||||||
|
normalize_template_key(match.group(1))
|
||||||
|
for pattern in (_DOLLAR_FIELD_PATTERN, _BRACE_FIELD_PATTERN)
|
||||||
|
for match in pattern.finditer(text)
|
||||||
|
}
|
||||||
@@ -1,14 +1,26 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import unittest
|
import unittest
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
from sqlalchemy import create_engine
|
from sqlalchemy import create_engine
|
||||||
from sqlalchemy.orm import sessionmaker
|
from sqlalchemy.orm import sessionmaker
|
||||||
|
|
||||||
from govoplan_access.backend.db.models import Account, Group, User
|
from govoplan_access.backend.db.models import Account, Group, User
|
||||||
from govoplan_campaign.backend.capabilities import CampaignAccessService
|
from govoplan_campaign.backend.capabilities import (
|
||||||
from govoplan_campaign.backend.db.models import Campaign, CampaignShare
|
CampaignAccessService,
|
||||||
|
CampaignOwnershipService,
|
||||||
|
campaign_report_resource_id,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignShare,
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
from govoplan_core.core.access import PrincipalRef
|
from govoplan_core.core.access import PrincipalRef
|
||||||
|
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||||
|
from govoplan_core.core.ownership import OwnershipSubjectRef, OwnershipTransferError
|
||||||
from govoplan_core.db.base import Base
|
from govoplan_core.db.base import Base
|
||||||
|
|
||||||
|
|
||||||
@@ -21,6 +33,7 @@ GROUP_ID = "group-1"
|
|||||||
class CampaignAccessProviderTests(unittest.TestCase):
|
class CampaignAccessProviderTests(unittest.TestCase):
|
||||||
def test_campaign_access_provider_explains_owner_share_admin_and_missing_resources(self) -> None:
|
def test_campaign_access_provider_explains_owner_share_admin_and_missing_resources(self) -> None:
|
||||||
session = _session()
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
_seed_access_subjects(session)
|
_seed_access_subjects(session)
|
||||||
owned = Campaign(id="campaign-owned", tenant_id=TENANT_ID, owner_user_id=USER_ID, external_id="owned", name="Owned campaign")
|
owned = Campaign(id="campaign-owned", tenant_id=TENANT_ID, owner_user_id=USER_ID, external_id="owned", name="Owned campaign")
|
||||||
shared = Campaign(id="campaign-shared", tenant_id=TENANT_ID, owner_user_id=OTHER_USER_ID, external_id="shared", name="Shared campaign")
|
shared = Campaign(id="campaign-shared", tenant_id=TENANT_ID, owner_user_id=OTHER_USER_ID, external_id="shared", name="Shared campaign")
|
||||||
@@ -46,6 +59,7 @@ class CampaignAccessProviderTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_campaign_access_provider_requires_write_share_for_write_actions(self) -> None:
|
def test_campaign_access_provider_requires_write_share_for_write_actions(self) -> None:
|
||||||
session = _session()
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
_seed_access_subjects(session)
|
_seed_access_subjects(session)
|
||||||
campaign = Campaign(id="campaign-write", tenant_id=TENANT_ID, owner_user_id=OTHER_USER_ID, external_id="write", name="Write campaign")
|
campaign = Campaign(id="campaign-write", tenant_id=TENANT_ID, owner_user_id=OTHER_USER_ID, external_id="write", name="Write campaign")
|
||||||
session.add_all([
|
session.add_all([
|
||||||
@@ -67,13 +81,335 @@ class CampaignAccessProviderTests(unittest.TestCase):
|
|||||||
self.assertTrue(any(item.kind == "share" and item.id == "share-write" for item in items))
|
self.assertTrue(any(item.kind == "share" and item.id == "share-write" for item in items))
|
||||||
self.assertFalse(any(item.kind == "share" and item.id == "share-read" for item in items))
|
self.assertFalse(any(item.kind == "share" and item.id == "share-read" for item in items))
|
||||||
|
|
||||||
|
def test_campaign_children_explain_their_parent_campaign_access(self) -> None:
|
||||||
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
|
_seed_access_subjects(session)
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-child",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
owner_user_id=OTHER_USER_ID,
|
||||||
|
external_id="child",
|
||||||
|
name="Child resources",
|
||||||
|
)
|
||||||
|
version = CampaignVersion(
|
||||||
|
id="version-child",
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_number=1,
|
||||||
|
raw_json={},
|
||||||
|
)
|
||||||
|
job = CampaignJob(
|
||||||
|
id="job-child",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
campaign_version_id=version.id,
|
||||||
|
entry_index=1,
|
||||||
|
recipient_email="recipient@example.test",
|
||||||
|
)
|
||||||
|
session.add_all(
|
||||||
|
[
|
||||||
|
campaign,
|
||||||
|
version,
|
||||||
|
job,
|
||||||
|
CampaignShare(
|
||||||
|
id="share-child",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
target_type="group",
|
||||||
|
target_id=GROUP_ID,
|
||||||
|
permission="read",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
service = CampaignAccessService()
|
||||||
|
principal = _principal(group_ids={GROUP_ID})
|
||||||
|
version_items = service.explain_resource_provenance(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_version",
|
||||||
|
resource_id=version.id,
|
||||||
|
action="campaigns:version:read",
|
||||||
|
)
|
||||||
|
job_items = service.explain_resource_provenance(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_delivery_job",
|
||||||
|
resource_id=job.id,
|
||||||
|
action="campaigns:delivery_job:read",
|
||||||
|
)
|
||||||
|
report_id = campaign_report_resource_id(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
report_kind="delivery",
|
||||||
|
)
|
||||||
|
report_items = service.explain_resource_provenance(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
resource_type="campaign_report",
|
||||||
|
resource_id=report_id,
|
||||||
|
action="campaigns:report:read",
|
||||||
|
)
|
||||||
|
|
||||||
|
for items, source in (
|
||||||
|
(version_items, "campaigns.version"),
|
||||||
|
(job_items, "campaigns.delivery_job"),
|
||||||
|
(report_items, "campaigns.report"),
|
||||||
|
):
|
||||||
|
child = next(item for item in items if item.source == source)
|
||||||
|
self.assertEqual(
|
||||||
|
child.details["authorization_inherited_from"],
|
||||||
|
{
|
||||||
|
"resource_type": "campaign",
|
||||||
|
"resource_id": campaign.id,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.assertTrue(
|
||||||
|
any(
|
||||||
|
item.source == "campaigns.campaign"
|
||||||
|
and item.id == campaign.id
|
||||||
|
for item in items
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.assertTrue(
|
||||||
|
any(item.kind == "share" and item.id == "share-child" for item in items)
|
||||||
|
)
|
||||||
|
|
||||||
|
report = next(item for item in report_items if item.source == "campaigns.report")
|
||||||
|
self.assertEqual(report.details["campaign_version_id"], version.id)
|
||||||
|
self.assertEqual(report.details["report_kind"], "delivery")
|
||||||
|
self.assertFalse(report.details["persisted"])
|
||||||
|
|
||||||
|
def test_campaign_ownership_provider_requires_group_acceptance_authority(self) -> None:
|
||||||
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
|
_seed_access_subjects(session)
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-ownership",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
owner_user_id=USER_ID,
|
||||||
|
external_id="ownership",
|
||||||
|
name="Ownership",
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
service = CampaignOwnershipService()
|
||||||
|
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||||
|
target_group = OwnershipSubjectRef(type="group", id=GROUP_ID)
|
||||||
|
ordinary_member = OwnershipSubjectRef(
|
||||||
|
type="user",
|
||||||
|
id=OTHER_USER_ID,
|
||||||
|
group_ids=frozenset({GROUP_ID}),
|
||||||
|
)
|
||||||
|
group_manager = OwnershipSubjectRef(
|
||||||
|
type="user",
|
||||||
|
id=OTHER_USER_ID,
|
||||||
|
group_ids=frozenset({GROUP_ID}),
|
||||||
|
scopes=frozenset({"campaigns:ownership:accept_group"}),
|
||||||
|
)
|
||||||
|
|
||||||
|
denied = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="accept_group_transfer",
|
||||||
|
actor=ordinary_member,
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=target_group,
|
||||||
|
)
|
||||||
|
allowed = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="accept_group_transfer",
|
||||||
|
actor=group_manager,
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=target_group,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertFalse(denied.allowed)
|
||||||
|
self.assertTrue(allowed.allowed)
|
||||||
|
|
||||||
|
@patch(
|
||||||
|
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
|
||||||
|
return_value=None,
|
||||||
|
)
|
||||||
|
def test_campaign_ownership_request_requires_existing_read_access(
|
||||||
|
self,
|
||||||
|
_target_validation,
|
||||||
|
) -> None:
|
||||||
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
|
_seed_access_subjects(session)
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-request",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
owner_user_id=USER_ID,
|
||||||
|
external_id="request",
|
||||||
|
name="Request",
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
service = CampaignOwnershipService()
|
||||||
|
requester = OwnershipSubjectRef(
|
||||||
|
type="user",
|
||||||
|
id=OTHER_USER_ID,
|
||||||
|
scopes=frozenset({"campaigns:campaign:read"}),
|
||||||
|
)
|
||||||
|
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||||
|
|
||||||
|
denied = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="request_ownership",
|
||||||
|
actor=requester,
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=requester,
|
||||||
|
)
|
||||||
|
session.add(
|
||||||
|
CampaignShare(
|
||||||
|
id="share-requester",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
target_type="user",
|
||||||
|
target_id=OTHER_USER_ID,
|
||||||
|
permission="read",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
allowed = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="request_ownership",
|
||||||
|
actor=requester,
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=requester,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertFalse(denied.allowed)
|
||||||
|
self.assertTrue(allowed.allowed)
|
||||||
|
|
||||||
|
@patch(
|
||||||
|
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
|
||||||
|
return_value=None,
|
||||||
|
)
|
||||||
|
def test_campaign_owner_proposal_requires_share_authority(
|
||||||
|
self,
|
||||||
|
_target_validation,
|
||||||
|
) -> None:
|
||||||
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
|
_seed_access_subjects(session)
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-proposal",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
owner_user_id=USER_ID,
|
||||||
|
external_id="proposal",
|
||||||
|
name="Proposal",
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
service = CampaignOwnershipService()
|
||||||
|
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||||
|
target_owner = OwnershipSubjectRef(type="user", id=OTHER_USER_ID)
|
||||||
|
|
||||||
|
denied = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="propose_transfer",
|
||||||
|
actor=current_owner,
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=target_owner,
|
||||||
|
)
|
||||||
|
allowed = service.authorize_ownership_action(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
action="propose_transfer",
|
||||||
|
actor=OwnershipSubjectRef(
|
||||||
|
type="user",
|
||||||
|
id=USER_ID,
|
||||||
|
scopes=frozenset({"campaigns:campaign:share"}),
|
||||||
|
),
|
||||||
|
current_owner=current_owner,
|
||||||
|
target_owner=target_owner,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertFalse(denied.allowed)
|
||||||
|
self.assertTrue(allowed.allowed)
|
||||||
|
|
||||||
|
def test_campaign_ownership_provider_applies_only_against_expected_owner(self) -> None:
|
||||||
|
session = _session()
|
||||||
|
self.addCleanup(_close_session, session)
|
||||||
|
_seed_access_subjects(session)
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-owner-apply",
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
owner_user_id=USER_ID,
|
||||||
|
external_id="owner-apply",
|
||||||
|
name="Owner apply",
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
service = CampaignOwnershipService()
|
||||||
|
service.apply_owner(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||||
|
target_owner=OwnershipSubjectRef(type="group", id=GROUP_ID),
|
||||||
|
actor=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
|
||||||
|
reason=None,
|
||||||
|
)
|
||||||
|
session.flush()
|
||||||
|
self.assertIsNone(campaign.owner_user_id)
|
||||||
|
self.assertEqual(campaign.owner_group_id, GROUP_ID)
|
||||||
|
|
||||||
|
with self.assertRaisesRegex(
|
||||||
|
OwnershipTransferError,
|
||||||
|
"changed while the transfer was pending",
|
||||||
|
):
|
||||||
|
service.apply_owner(
|
||||||
|
session,
|
||||||
|
tenant_id=TENANT_ID,
|
||||||
|
resource_id=campaign.id,
|
||||||
|
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||||
|
target_owner=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
|
||||||
|
actor=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||||
|
reason=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _session():
|
def _session():
|
||||||
engine = create_engine("sqlite:///:memory:", future=True)
|
engine = create_engine("sqlite:///:memory:", future=True)
|
||||||
Base.metadata.create_all(bind=engine, tables=[Account.__table__, User.__table__, Group.__table__, Campaign.__table__, CampaignShare.__table__])
|
Base.metadata.create_all(
|
||||||
|
bind=engine,
|
||||||
|
tables=[
|
||||||
|
Account.__table__,
|
||||||
|
User.__table__,
|
||||||
|
Group.__table__,
|
||||||
|
Campaign.__table__,
|
||||||
|
CampaignShare.__table__,
|
||||||
|
CampaignVersion.__table__,
|
||||||
|
CampaignJob.__table__,
|
||||||
|
ChangeSequenceEntry.__table__,
|
||||||
|
],
|
||||||
|
)
|
||||||
return sessionmaker(bind=engine, future=True)()
|
return sessionmaker(bind=engine, future=True)()
|
||||||
|
|
||||||
|
|
||||||
|
def _close_session(session) -> None:
|
||||||
|
engine = session.get_bind()
|
||||||
|
session.close()
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
def _seed_access_subjects(session) -> None:
|
def _seed_access_subjects(session) -> None:
|
||||||
session.add_all([
|
session.add_all([
|
||||||
Account(id="account-1", email="one@example.test", normalized_email="one@example.test"),
|
Account(id="account-1", email="one@example.test", normalized_email="one@example.test"),
|
||||||
|
|||||||
245
tests/test_aggregate_report.py
Normal file
245
tests/test_aggregate_report.py
Normal file
@@ -0,0 +1,245 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.report_privacy_policy import (
|
||||||
|
CampaignReportPrivacyPolicy,
|
||||||
|
CampaignReportPrivacyPolicyError,
|
||||||
|
DEFAULT_SMALL_CELL_THRESHOLD,
|
||||||
|
effective_campaign_report_privacy_policy,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.reports.aggregate import build_aggregate_campaign_report
|
||||||
|
|
||||||
|
|
||||||
|
class _PolicySession:
|
||||||
|
def __init__(self, settings: dict[str, object] | None = None) -> None:
|
||||||
|
self.tenant = SimpleNamespace(settings=settings or {})
|
||||||
|
|
||||||
|
def get(self, _model, _id):
|
||||||
|
return self.tenant
|
||||||
|
|
||||||
|
|
||||||
|
def _policy(threshold: int = 5) -> CampaignReportPrivacyPolicy:
|
||||||
|
return CampaignReportPrivacyPolicy(
|
||||||
|
small_cell_threshold=threshold,
|
||||||
|
source="test",
|
||||||
|
deployment_small_cell_threshold=threshold,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign() -> SimpleNamespace:
|
||||||
|
now = datetime(2026, 7, 22, 8, 0, tzinfo=UTC)
|
||||||
|
return SimpleNamespace(
|
||||||
|
id="campaign-safe",
|
||||||
|
tenant_id="tenant-safe",
|
||||||
|
external_id="must-not-leak-external-id",
|
||||||
|
name="Semester notification",
|
||||||
|
description="Business-safe description",
|
||||||
|
status="partially_completed",
|
||||||
|
owner_user_id="must-not-leak-owner",
|
||||||
|
current_version_id="must-not-leak-version-id",
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _version(*, inactive_count: int = 0) -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(
|
||||||
|
id="must-not-leak-version-id",
|
||||||
|
version_number=7,
|
||||||
|
build_summary={"inactive_count": inactive_count, "build_token": "must-not-leak-token"},
|
||||||
|
execution_snapshot={"smtp": {"password": "must-not-leak-secret"}},
|
||||||
|
raw_json={"entries": [{"email": "must-not-leak@example.test"}]},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _job(index: int, send_status: str, **overrides: object) -> SimpleNamespace:
|
||||||
|
started = datetime(2026, 7, 22, 8, 0, tzinfo=UTC) + timedelta(minutes=index)
|
||||||
|
values: dict[str, object] = {
|
||||||
|
"id": f"job-{index}",
|
||||||
|
"recipient_email": f"private-{index}@example.test",
|
||||||
|
"subject": f"Personal subject {index}",
|
||||||
|
"last_error": "smtp.internal.example provider-secret",
|
||||||
|
"resolved_attachments": [{"storage_key": f"private/{index}"}],
|
||||||
|
"send_status": send_status,
|
||||||
|
"validation_status": "ready",
|
||||||
|
"build_status": "built",
|
||||||
|
"queued_at": started,
|
||||||
|
"smtp_started_at": started,
|
||||||
|
"sent_at": started if send_status in {"smtp_accepted", "sent"} else None,
|
||||||
|
"outcome_unknown_at": started if send_status == "outcome_unknown" else None,
|
||||||
|
}
|
||||||
|
values.update(overrides)
|
||||||
|
return SimpleNamespace(**values)
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_projection_has_a_strict_recipient_free_shape() -> None:
|
||||||
|
jobs = [
|
||||||
|
*[_job(index, "smtp_accepted") for index in range(10)],
|
||||||
|
*[_job(index + 10, "failed_permanent") for index in range(5)],
|
||||||
|
]
|
||||||
|
|
||||||
|
payload = build_aggregate_campaign_report(
|
||||||
|
campaign=_campaign(), # type: ignore[arg-type]
|
||||||
|
version=_version(inactive_count=5), # type: ignore[arg-type]
|
||||||
|
jobs=jobs, # type: ignore[arg-type]
|
||||||
|
policy=_policy(),
|
||||||
|
generated_at=datetime(2026, 7, 22, 12, 0, tzinfo=UTC),
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
assert set(payload) == {
|
||||||
|
"generated_at",
|
||||||
|
"campaign",
|
||||||
|
"version_number",
|
||||||
|
"completion_state",
|
||||||
|
"population",
|
||||||
|
"outcomes",
|
||||||
|
"time_range",
|
||||||
|
"privacy",
|
||||||
|
}
|
||||||
|
assert set(payload["campaign"]) == {"id", "name", "status"}
|
||||||
|
assert payload["population"]["denominator"] == {"value": 15, "suppressed": False}
|
||||||
|
assert payload["outcomes"]["smtp_accepted"] == {"value": 10, "suppressed": False}
|
||||||
|
assert payload["outcomes"]["failed"] == {"value": 5, "suppressed": False}
|
||||||
|
assert payload["completion_state"] == "partially_completed"
|
||||||
|
serialized = repr(payload)
|
||||||
|
for forbidden in (
|
||||||
|
"private-0@example.test",
|
||||||
|
"Personal subject",
|
||||||
|
"smtp.internal.example",
|
||||||
|
"provider-secret",
|
||||||
|
"storage_key",
|
||||||
|
"must-not-leak",
|
||||||
|
"Business-safe description",
|
||||||
|
"job-0",
|
||||||
|
):
|
||||||
|
assert forbidden not in serialized
|
||||||
|
|
||||||
|
|
||||||
|
def test_small_cells_use_primary_and_complementary_suppression() -> None:
|
||||||
|
jobs = [
|
||||||
|
*[_job(index, "smtp_accepted") for index in range(8)],
|
||||||
|
_job(8, "failed_permanent"),
|
||||||
|
]
|
||||||
|
|
||||||
|
report = build_aggregate_campaign_report(
|
||||||
|
campaign=_campaign(), # type: ignore[arg-type]
|
||||||
|
version=_version(inactive_count=1), # type: ignore[arg-type]
|
||||||
|
jobs=jobs, # type: ignore[arg-type]
|
||||||
|
policy=_policy(5),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report.population.denominator.value == 9
|
||||||
|
assert report.outcomes.failed.suppressed is True
|
||||||
|
assert report.outcomes.failed.value is None
|
||||||
|
assert report.outcomes.smtp_accepted.suppressed is True
|
||||||
|
assert report.outcomes.smtp_accepted.value is None
|
||||||
|
assert report.population.inactive_source_entries.suppressed is True
|
||||||
|
assert report.time_range.suppressed is True
|
||||||
|
assert report.privacy.suppression_applied is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_outcome_suppression_also_hides_overlapping_population_count() -> None:
|
||||||
|
jobs = [
|
||||||
|
*[
|
||||||
|
_job(
|
||||||
|
index,
|
||||||
|
"smtp_accepted",
|
||||||
|
validation_status="blocked" if index < 4 else "ready",
|
||||||
|
)
|
||||||
|
for index in range(8)
|
||||||
|
],
|
||||||
|
_job(8, "skipped", validation_status="excluded"),
|
||||||
|
]
|
||||||
|
|
||||||
|
report = build_aggregate_campaign_report(
|
||||||
|
campaign=_campaign(), # type: ignore[arg-type]
|
||||||
|
version=_version(inactive_count=5), # type: ignore[arg-type]
|
||||||
|
jobs=jobs, # type: ignore[arg-type]
|
||||||
|
policy=_policy(5),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report.outcomes.excluded.suppressed is True
|
||||||
|
assert report.population.excluded_or_blocked_jobs.model_dump() == {
|
||||||
|
"value": None,
|
||||||
|
"suppressed": True,
|
||||||
|
}
|
||||||
|
# Source entries without jobs are outside the outcome denominator and do
|
||||||
|
# not overlap the suppressed partition, so their threshold-safe count stays visible.
|
||||||
|
assert report.population.inactive_source_entries.model_dump() == {
|
||||||
|
"value": 5,
|
||||||
|
"suppressed": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_small_cells_also_suppress_the_denominator_and_state() -> None:
|
||||||
|
jobs = [_job(0, "smtp_accepted"), _job(1, "failed_permanent")]
|
||||||
|
|
||||||
|
report = build_aggregate_campaign_report(
|
||||||
|
campaign=_campaign(), # type: ignore[arg-type]
|
||||||
|
version=_version(), # type: ignore[arg-type]
|
||||||
|
jobs=jobs, # type: ignore[arg-type]
|
||||||
|
policy=_policy(5),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report.population.denominator.model_dump() == {"value": None, "suppressed": True}
|
||||||
|
assert report.completion_state == "suppressed"
|
||||||
|
assert report.outcomes.smtp_accepted.value is None
|
||||||
|
assert report.outcomes.failed.value is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_explicitly_skipped_jobs_are_exclusions_not_unattempted_outcomes() -> None:
|
||||||
|
jobs = [_job(index, "skipped") for index in range(5)]
|
||||||
|
|
||||||
|
report = build_aggregate_campaign_report(
|
||||||
|
campaign=_campaign(), # type: ignore[arg-type]
|
||||||
|
version=_version(), # type: ignore[arg-type]
|
||||||
|
jobs=jobs, # type: ignore[arg-type]
|
||||||
|
policy=_policy(5),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report.outcomes.excluded.value == 5
|
||||||
|
assert report.outcomes.not_attempted.value == 0
|
||||||
|
assert report.completion_state == "not_started"
|
||||||
|
|
||||||
|
|
||||||
|
def test_report_privacy_policy_defaults_to_five_and_tenant_can_only_strengthen() -> None:
|
||||||
|
default = effective_campaign_report_privacy_policy(
|
||||||
|
_PolicySession(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
environ={},
|
||||||
|
)
|
||||||
|
assert default.small_cell_threshold == DEFAULT_SMALL_CELL_THRESHOLD == 5
|
||||||
|
assert default.source == "deployment_default"
|
||||||
|
|
||||||
|
strengthened = effective_campaign_report_privacy_policy(
|
||||||
|
_PolicySession(
|
||||||
|
{"campaign_report_privacy_policy": {"small_cell_threshold": 10}}
|
||||||
|
), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": "5"},
|
||||||
|
)
|
||||||
|
assert strengthened.small_cell_threshold == 10
|
||||||
|
assert strengthened.source == "tenant"
|
||||||
|
|
||||||
|
floor = effective_campaign_report_privacy_policy(
|
||||||
|
_PolicySession(
|
||||||
|
{"campaign_report_privacy_policy": {"small_cell_threshold": 3}}
|
||||||
|
), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": "7"},
|
||||||
|
)
|
||||||
|
assert floor.small_cell_threshold == 7
|
||||||
|
assert floor.source == "deployment_floor"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("value", [True, 0, 1, 101, "2.5", "disabled"])
|
||||||
|
def test_invalid_report_privacy_policy_fails_closed(value: object) -> None:
|
||||||
|
with pytest.raises(CampaignReportPrivacyPolicyError):
|
||||||
|
effective_campaign_report_privacy_policy(
|
||||||
|
_PolicySession(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
environ={"GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD": value}, # type: ignore[dict-item]
|
||||||
|
)
|
||||||
202
tests/test_aggregate_report_routes.py
Normal file
202
tests/test_aggregate_report_routes.py
Normal file
@@ -0,0 +1,202 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import Mock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_access.backend.db.models import Account, Group, User
|
||||||
|
from govoplan_campaign.backend import router as campaign_api
|
||||||
|
from govoplan_campaign.backend.routes import campaigns as campaign_routes
|
||||||
|
from govoplan_campaign.backend.routes import jobs as job_routes
|
||||||
|
from govoplan_campaign.backend.routes import reports as report_routes
|
||||||
|
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
|
||||||
|
from govoplan_campaign.backend.reports.aggregate import (
|
||||||
|
AggregateCampaignReportError,
|
||||||
|
generate_aggregate_campaign_report,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.schemas import ReportEmailRequest
|
||||||
|
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||||
|
from govoplan_core.db.base import Base
|
||||||
|
from govoplan_core.tenancy.scope import Tenant, create_scope_tables
|
||||||
|
|
||||||
|
|
||||||
|
class _Principal:
|
||||||
|
def __init__(self, *scopes: str, tenant_id: str = "tenant-1") -> None:
|
||||||
|
self.scopes = set(scopes)
|
||||||
|
self.tenant_id = tenant_id
|
||||||
|
self.user = SimpleNamespace(id="reader-1")
|
||||||
|
|
||||||
|
def has(self, scope: str) -> bool:
|
||||||
|
return scope in self.scopes
|
||||||
|
|
||||||
|
|
||||||
|
def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
|
||||||
|
principal = _Principal("campaigns:report:read")
|
||||||
|
session = Mock()
|
||||||
|
campaign = SimpleNamespace(id="campaign-1", tenant_id="tenant-1")
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(report_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
pytest.raises(HTTPException) as full_report_denied,
|
||||||
|
):
|
||||||
|
report_routes.campaign_report(
|
||||||
|
"campaign-1",
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
assert full_report_denied.value.status_code == 403
|
||||||
|
assert "campaigns:recipient:read" in full_report_denied.value.detail
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
pytest.raises(HTTPException) as job_detail_denied,
|
||||||
|
):
|
||||||
|
job_routes.get_job_detail(
|
||||||
|
"campaign-1",
|
||||||
|
"job-1",
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
assert job_detail_denied.value.status_code == 403
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(report_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
pytest.raises(HTTPException) as report_email_denied,
|
||||||
|
):
|
||||||
|
report_routes.email_campaign_report(
|
||||||
|
"campaign-1",
|
||||||
|
ReportEmailRequest(to=["auditor@example.test"]),
|
||||||
|
session=session,
|
||||||
|
principal=_Principal("campaigns:report:send"), # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
assert report_email_denied.value.status_code == 403
|
||||||
|
assert "campaigns:recipient:export" in report_email_denied.value.detail
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
pytest.raises(HTTPException) as diagnostics_denied,
|
||||||
|
):
|
||||||
|
job_routes.get_job_diagnostics(
|
||||||
|
"campaign-1",
|
||||||
|
"job-1",
|
||||||
|
session=session,
|
||||||
|
principal=_Principal("campaigns:diagnostic:read"), # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
assert diagnostics_denied.value.status_code == 403
|
||||||
|
assert "campaigns:recipient:read" in diagnostics_denied.value.detail
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_route_uses_only_the_safe_projection() -> None:
|
||||||
|
principal = _Principal("campaigns:report:read")
|
||||||
|
session = Mock()
|
||||||
|
safe_projection = Mock()
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(campaign_routes, "_get_campaign_for_principal") as acl,
|
||||||
|
patch.object(
|
||||||
|
campaign_routes,
|
||||||
|
"generate_aggregate_campaign_report",
|
||||||
|
return_value=safe_projection,
|
||||||
|
) as generate,
|
||||||
|
):
|
||||||
|
result = campaign_routes.aggregate_campaign_report(
|
||||||
|
"campaign-1",
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is safe_projection
|
||||||
|
acl.assert_called_once_with(session, "campaign-1", principal)
|
||||||
|
generate.assert_called_once_with(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("path", ["/campaigns/aggregate-reports", "/campaigns/aggregate-reports/{campaign_id}"])
|
||||||
|
def test_aggregate_routes_require_report_read_permission(path: str) -> None:
|
||||||
|
route = next(item for item in campaign_api.router.routes if item.path == path)
|
||||||
|
dependency = next(item for item in route.dependant.dependencies if item.name == "principal")
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as denied:
|
||||||
|
dependency.call(_Principal())
|
||||||
|
assert denied.value.status_code == 403
|
||||||
|
|
||||||
|
principal = _Principal("campaigns:report:read")
|
||||||
|
assert dependency.call(principal) is principal
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_projection_is_tenant_isolated_and_needs_no_optional_module() -> None:
|
||||||
|
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||||
|
create_scope_tables(engine)
|
||||||
|
Base.metadata.create_all(
|
||||||
|
engine,
|
||||||
|
tables=[
|
||||||
|
Account.__table__,
|
||||||
|
User.__table__,
|
||||||
|
Group.__table__,
|
||||||
|
Campaign.__table__,
|
||||||
|
CampaignVersion.__table__,
|
||||||
|
CampaignJob.__table__,
|
||||||
|
ChangeSequenceEntry.__table__,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
with Session(engine) as session:
|
||||||
|
session.add(Tenant(id="tenant-1", slug="tenant-1", name="Tenant 1", settings={}))
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
external_id="external-1",
|
||||||
|
name="Safe aggregate",
|
||||||
|
description=None,
|
||||||
|
status="sent",
|
||||||
|
)
|
||||||
|
version = CampaignVersion(
|
||||||
|
id="version-1",
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_number=1,
|
||||||
|
raw_json={"mail": {"profile_id": "optional-module-not-loaded"}},
|
||||||
|
schema_version="5",
|
||||||
|
build_summary={},
|
||||||
|
)
|
||||||
|
campaign.current_version_id = version.id
|
||||||
|
session.add_all([campaign, version])
|
||||||
|
for index in range(5):
|
||||||
|
session.add(CampaignJob(
|
||||||
|
id=f"job-{index}",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
campaign_version_id=version.id,
|
||||||
|
entry_index=index,
|
||||||
|
recipient_email=f"private-{index}@example.test",
|
||||||
|
subject="Private",
|
||||||
|
build_status="built",
|
||||||
|
validation_status="ready",
|
||||||
|
queue_status="queued",
|
||||||
|
send_status="smtp_accepted",
|
||||||
|
imap_status="not_requested",
|
||||||
|
))
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
report = generate_aggregate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
)
|
||||||
|
assert report.population.denominator.value == 5
|
||||||
|
assert report.outcomes.smtp_accepted.value == 5
|
||||||
|
|
||||||
|
with pytest.raises(AggregateCampaignReportError):
|
||||||
|
generate_aggregate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-2",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
engine.dispose()
|
||||||
280
tests/test_attachment_building.py
Normal file
280
tests/test_attachment_building.py
Normal file
@@ -0,0 +1,280 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
import zipfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
||||||
|
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
||||||
|
from govoplan_campaign.backend.messages.builder import build_campaign_messages
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignAttachmentBuildTests(unittest.TestCase):
|
||||||
|
def _no_attachment_config(
|
||||||
|
self,
|
||||||
|
behavior: str | None = None,
|
||||||
|
legacy_allow: bool | None = None,
|
||||||
|
*,
|
||||||
|
configure_missing_rule: bool = False,
|
||||||
|
) -> CampaignConfig:
|
||||||
|
attachments: dict[str, object] = {
|
||||||
|
"base_path": ".",
|
||||||
|
"global": [],
|
||||||
|
"zip": {"enabled": False, "archives": []},
|
||||||
|
}
|
||||||
|
if configure_missing_rule:
|
||||||
|
attachments["global"] = [
|
||||||
|
{
|
||||||
|
"id": "missing",
|
||||||
|
"label": "Missing attachment",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "missing.pdf",
|
||||||
|
"required": False,
|
||||||
|
"missing_behavior": "continue",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
if behavior is not None:
|
||||||
|
attachments["send_without_attachments_behavior"] = behavior
|
||||||
|
if legacy_allow is not None:
|
||||||
|
attachments["send_without_attachments"] = legacy_allow
|
||||||
|
return CampaignConfig.model_validate({
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {"id": f"no-attachment-{behavior or legacy_allow}", "name": "No attachment policy", "mode": "test"},
|
||||||
|
"fields": [],
|
||||||
|
"global_values": {},
|
||||||
|
"server": {
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"profile_capabilities": {"smtp_available": True},
|
||||||
|
},
|
||||||
|
"recipients": {"from": {"email": "sender@example.org", "type": "to"}, "allow_individual_to": True},
|
||||||
|
"template": {"subject": "Subject", "text": "Body"},
|
||||||
|
"attachments": attachments,
|
||||||
|
"entries": {"inline": [{"id": "recipient-1", "to": [{"email": "recipient@example.org", "type": "to"}]}]},
|
||||||
|
"validation_policy": {"missing_email": "block", "template_error": "block"},
|
||||||
|
"delivery": {"imap_append_sent": {"enabled": False}},
|
||||||
|
})
|
||||||
|
|
||||||
|
def test_send_without_attachments_policy_does_not_block_when_no_rules_are_configured(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = self._no_attachment_config(legacy_allow=False)
|
||||||
|
|
||||||
|
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||||
|
self.assertTrue(validation.ok)
|
||||||
|
self.assertNotIn("missing_attachment_coverage", {issue.code for issue in validation.issues})
|
||||||
|
|
||||||
|
result = build_campaign_messages(config, campaign_file=campaign_file, output_dir=root / "out", write_eml=True)
|
||||||
|
self.assertEqual(result.report.build_failed_count, 0)
|
||||||
|
self.assertEqual(result.report.queueable_count, 1)
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.attachment_count, 0)
|
||||||
|
self.assertIsNotNone(message.eml_path)
|
||||||
|
self.assertNotIn("missing_attachment_coverage", {issue.code for issue in message.issues})
|
||||||
|
self.assertIsNotNone(result.built_messages[0].mime)
|
||||||
|
|
||||||
|
def test_configured_attachment_rule_blocks_generation_when_no_files_resolve(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = self._no_attachment_config(behavior="block", configure_missing_rule=True)
|
||||||
|
|
||||||
|
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||||
|
self.assertFalse(validation.ok)
|
||||||
|
self.assertIn("missing_attachment_coverage", {issue.code for issue in validation.issues})
|
||||||
|
|
||||||
|
result = build_campaign_messages(config, campaign_file=campaign_file, output_dir=root / "out", write_eml=True)
|
||||||
|
self.assertEqual(result.report.build_failed_count, 1)
|
||||||
|
self.assertEqual(result.report.queueable_count, 0)
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.attachment_count, 0)
|
||||||
|
self.assertIsNone(message.eml_path)
|
||||||
|
self.assertIn("missing_attachment_coverage", {issue.code for issue in message.issues})
|
||||||
|
self.assertIsNone(result.built_messages[0].mime)
|
||||||
|
|
||||||
|
def test_send_without_attachments_behavior_modes(self) -> None:
|
||||||
|
cases = {
|
||||||
|
"block": ("build_failed", "blocked", 0, "block", False),
|
||||||
|
"ask": ("built", "needs_review", 0, "ask", True),
|
||||||
|
"drop": ("built", "needs_review", 0, "ask", True),
|
||||||
|
"warn": ("built", "warning", 1, "warn", True),
|
||||||
|
"continue": ("built", "warning", 1, None, True),
|
||||||
|
}
|
||||||
|
for behavior, (build_status, validation_status, queueable_count, issue_behavior, has_mime) in cases.items():
|
||||||
|
with self.subTest(behavior=behavior):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = self._no_attachment_config(behavior=behavior, configure_missing_rule=True)
|
||||||
|
|
||||||
|
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||||
|
if behavior == "block":
|
||||||
|
self.assertFalse(validation.ok)
|
||||||
|
else:
|
||||||
|
self.assertTrue(validation.ok)
|
||||||
|
|
||||||
|
result = build_campaign_messages(config, campaign_file=campaign_file, output_dir=root / "out", write_eml=True)
|
||||||
|
self.assertEqual(result.report.queueable_count, queueable_count)
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.build_status.value, build_status)
|
||||||
|
self.assertEqual(message.validation_status.value, validation_status)
|
||||||
|
if validation_status == "excluded":
|
||||||
|
self.assertEqual(message.send_status.value, "skipped")
|
||||||
|
self.assertEqual(message.imap_status.value, "skipped")
|
||||||
|
coverage_issues = [issue for issue in message.issues if issue.code == "missing_attachment_coverage"]
|
||||||
|
if issue_behavior is None:
|
||||||
|
self.assertEqual(coverage_issues, [])
|
||||||
|
else:
|
||||||
|
self.assertEqual(coverage_issues[0].behavior, issue_behavior)
|
||||||
|
self.assertEqual(result.built_messages[0].mime is not None, has_mime)
|
||||||
|
|
||||||
|
def test_required_missing_policy_cannot_be_loosened_by_rule(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = self._no_attachment_config(
|
||||||
|
behavior="continue",
|
||||||
|
configure_missing_rule=True,
|
||||||
|
)
|
||||||
|
rule = config.attachments.global_[0]
|
||||||
|
rule.required = True
|
||||||
|
rule.missing_behavior = "continue"
|
||||||
|
config.attachments.missing_behavior = "continue"
|
||||||
|
|
||||||
|
result = build_campaign_messages(
|
||||||
|
config,
|
||||||
|
campaign_file=campaign_file,
|
||||||
|
output_dir=root / "out",
|
||||||
|
write_eml=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.validation_status.value, "blocked")
|
||||||
|
issue = next(
|
||||||
|
item
|
||||||
|
for item in message.issues
|
||||||
|
if item.code == "missing_required_attachment"
|
||||||
|
)
|
||||||
|
self.assertEqual(issue.behavior, "block")
|
||||||
|
self.assertEqual(
|
||||||
|
issue.details["effective_policy"]["requirement_policy"],
|
||||||
|
"block",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
message.attachments[0].missing_policy["effective_behavior"],
|
||||||
|
"block",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_optional_missing_policy_warns_by_default(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = self._no_attachment_config(
|
||||||
|
behavior="continue",
|
||||||
|
configure_missing_rule=True,
|
||||||
|
)
|
||||||
|
config.attachments.global_[0].missing_behavior = None
|
||||||
|
|
||||||
|
result = build_campaign_messages(
|
||||||
|
config,
|
||||||
|
campaign_file=campaign_file,
|
||||||
|
output_dir=root / "out",
|
||||||
|
write_eml=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.validation_status.value, "warning")
|
||||||
|
issue = next(
|
||||||
|
item
|
||||||
|
for item in message.issues
|
||||||
|
if item.code == "missing_optional_attachment"
|
||||||
|
)
|
||||||
|
self.assertEqual(issue.behavior, "warn")
|
||||||
|
|
||||||
|
def test_missing_pattern_does_not_create_zip_member_or_count_as_attachment(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
documents = root / "documents"
|
||||||
|
documents.mkdir()
|
||||||
|
(documents / "matched.xlsx").write_bytes(b"matched workbook")
|
||||||
|
campaign_file = root / "campaign.json"
|
||||||
|
campaign_file.write_text("{}", encoding="utf-8")
|
||||||
|
config = CampaignConfig.model_validate({
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {"id": "zip-missing-pattern", "name": "ZIP missing pattern", "mode": "test"},
|
||||||
|
"fields": [],
|
||||||
|
"global_values": {},
|
||||||
|
"server": {
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"profile_capabilities": {"smtp_available": True},
|
||||||
|
},
|
||||||
|
"recipients": {"from": {"email": "sender@example.org", "type": "to"}, "allow_individual_to": True},
|
||||||
|
"template": {"subject": "Subject", "text": "Body"},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": ".",
|
||||||
|
"allow_individual": True,
|
||||||
|
"send_without_attachments": False,
|
||||||
|
"zip": {
|
||||||
|
"enabled": True,
|
||||||
|
"archives": [{"id": "bundle", "name": "recipient-bundle.zip", "standard": True}],
|
||||||
|
},
|
||||||
|
"global": [],
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"inline": [{
|
||||||
|
"id": "recipient-1",
|
||||||
|
"to": [{"email": "recipient@example.org", "type": "to"}],
|
||||||
|
"attachments": [
|
||||||
|
{"id": "matched", "base_dir": "documents", "file_filter": "matched.xlsx", "required": True},
|
||||||
|
{
|
||||||
|
"id": "missing",
|
||||||
|
"base_dir": "documents",
|
||||||
|
"file_filter": "missing.xlsx",
|
||||||
|
"required": False,
|
||||||
|
"missing_behavior": "warn",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}]
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
},
|
||||||
|
"delivery": {"imap_append_sent": {"enabled": False}},
|
||||||
|
})
|
||||||
|
|
||||||
|
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||||
|
self.assertTrue(validation.ok)
|
||||||
|
self.assertIn("missing_optional_attachment", {issue.code for issue in validation.issues})
|
||||||
|
|
||||||
|
result = build_campaign_messages(config, campaign_file=campaign_file, output_dir=root / "out", write_eml=False)
|
||||||
|
self.assertEqual(result.report.built_count, 1)
|
||||||
|
self.assertEqual(result.report.queueable_count, 1)
|
||||||
|
message = result.report.messages[0]
|
||||||
|
self.assertEqual(message.attachment_count, 1)
|
||||||
|
summaries = {attachment.attachment_id: attachment for attachment in message.attachments}
|
||||||
|
self.assertEqual(summaries["matched"].zip_filename, "recipient-bundle.zip")
|
||||||
|
self.assertEqual(summaries["matched"].zip_entry_names, ["matched.xlsx"])
|
||||||
|
self.assertIsNone(summaries["missing"].zip_filename)
|
||||||
|
self.assertEqual(summaries["missing"].zip_entry_names, [])
|
||||||
|
self.assertEqual(summaries["missing"].matches, [])
|
||||||
|
|
||||||
|
mime = result.built_messages[0].mime
|
||||||
|
self.assertIsNotNone(mime)
|
||||||
|
attachments = {part.get_filename(): part.get_payload(decode=True) for part in mime.iter_attachments()}
|
||||||
|
self.assertEqual(set(attachments), {"recipient-bundle.zip"})
|
||||||
|
with zipfile.ZipFile(io.BytesIO(attachments["recipient-bundle.zip"])) as archive:
|
||||||
|
self.assertEqual(archive.namelist(), ["matched.xlsx"])
|
||||||
|
self.assertEqual(archive.read("matched.xlsx"), b"matched workbook")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
131
tests/test_campaign_audit_atomicity.py
Normal file
131
tests/test_campaign_audit_atomicity.py
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
from govoplan_campaign.backend import route_support
|
||||||
|
from govoplan_campaign.backend.routes import campaigns as campaign_routes
|
||||||
|
from govoplan_campaign.backend.routes import versions as version_routes
|
||||||
|
from govoplan_campaign.backend.schemas import CampaignUpdateRequest, CampaignVersionUpdateRequest
|
||||||
|
from govoplan_core.core.concurrency import strong_resource_etag
|
||||||
|
|
||||||
|
|
||||||
|
def _principal() -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
user=SimpleNamespace(id="user-1"),
|
||||||
|
api_key=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_update_rolls_back_when_its_audit_record_cannot_be_written() -> None:
|
||||||
|
session = MagicMock()
|
||||||
|
principal = _principal()
|
||||||
|
version = SimpleNamespace(
|
||||||
|
id="version-1",
|
||||||
|
raw_json={},
|
||||||
|
current_flow="manual",
|
||||||
|
current_step="recipients",
|
||||||
|
edit_revision=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
def mutate(*_args, **kwargs):
|
||||||
|
assert kwargs["commit"] is False
|
||||||
|
session.flush()
|
||||||
|
return version
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(route_support, "_get_campaign_for_principal"),
|
||||||
|
patch.object(route_support, "_get_version_for_tenant", return_value=version),
|
||||||
|
patch.object(route_support, "update_campaign_version", side_effect=mutate),
|
||||||
|
patch.object(route_support, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||||
|
):
|
||||||
|
with pytest.raises(HTTPException, match="audit unavailable") as captured:
|
||||||
|
route_support._update_campaign_version_detail_response( # noqa: SLF001 - transaction regression test
|
||||||
|
session,
|
||||||
|
principal, # type: ignore[arg-type]
|
||||||
|
"campaign-1",
|
||||||
|
"version-1",
|
||||||
|
CampaignVersionUpdateRequest(
|
||||||
|
current_step="recipients",
|
||||||
|
base_revision=1,
|
||||||
|
),
|
||||||
|
if_match=strong_resource_etag(
|
||||||
|
"campaign_version",
|
||||||
|
"version-1",
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
autosave=True,
|
||||||
|
audit_action="campaign.version_autosaved",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert captured.value.status_code == 422
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
session.rollback.assert_called_once_with()
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_fork_rolls_back_when_its_audit_record_cannot_be_written() -> None:
|
||||||
|
session = MagicMock()
|
||||||
|
principal = _principal()
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
source = SimpleNamespace(id="version-1", campaign_id="campaign-1", raw_json={})
|
||||||
|
forked = SimpleNamespace(id="version-2", campaign_id="campaign-1", version_number=2)
|
||||||
|
|
||||||
|
def mutate(*_args, **kwargs):
|
||||||
|
assert kwargs["commit"] is False
|
||||||
|
session.flush()
|
||||||
|
return forked
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(version_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
patch.object(version_routes, "_require_permission"),
|
||||||
|
patch.object(version_routes, "_get_version_for_tenant", return_value=source),
|
||||||
|
patch.object(version_routes, "_get_campaign_for_tenant", return_value=campaign),
|
||||||
|
patch.object(version_routes, "fork_campaign_version_for_edit", side_effect=mutate),
|
||||||
|
patch.object(version_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||||
|
):
|
||||||
|
with pytest.raises(RuntimeError, match="audit unavailable"):
|
||||||
|
version_routes.fork_version_for_edit(
|
||||||
|
"campaign-1",
|
||||||
|
"version-1",
|
||||||
|
CampaignVersionUpdateRequest(),
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
session.rollback.assert_called_once_with()
|
||||||
|
|
||||||
|
|
||||||
|
def test_metadata_update_rolls_back_when_its_audit_record_cannot_be_written() -> None:
|
||||||
|
session = MagicMock()
|
||||||
|
principal = _principal()
|
||||||
|
campaign = SimpleNamespace(
|
||||||
|
id="campaign-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
external_id="C-1",
|
||||||
|
name="Old name",
|
||||||
|
description=None,
|
||||||
|
status="draft",
|
||||||
|
current_version_id=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(campaign_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
patch.object(campaign_routes, "_sync_campaign_metadata_to_current_version"),
|
||||||
|
patch.object(campaign_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||||
|
):
|
||||||
|
with pytest.raises(RuntimeError, match="audit unavailable"):
|
||||||
|
campaign_routes.update_campaign_metadata_endpoint(
|
||||||
|
"campaign-1",
|
||||||
|
CampaignUpdateRequest(name="New name"),
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
session.flush.assert_called_once_with()
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
session.rollback.assert_called_once_with()
|
||||||
168
tests/test_campaign_optimistic_concurrency.py
Normal file
168
tests/test_campaign_optimistic_concurrency.py
Normal file
@@ -0,0 +1,168 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from sqlalchemy import Column, String, Table, create_engine
|
||||||
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignIssue,
|
||||||
|
CampaignVersion,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.persistence.versions import update_campaign_version
|
||||||
|
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||||
|
from govoplan_core.core.concurrency import RevisionConflictError
|
||||||
|
from govoplan_core.db.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignOptimisticConcurrencyTests(unittest.TestCase):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
self.temp_dir = tempfile.TemporaryDirectory()
|
||||||
|
database_path = Path(self.temp_dir.name) / "campaign.db"
|
||||||
|
self.engine = create_engine(f"sqlite+pysqlite:///{database_path}")
|
||||||
|
access_users = Base.metadata.tables.get("access_users")
|
||||||
|
if access_users is None:
|
||||||
|
access_users = Table(
|
||||||
|
"access_users",
|
||||||
|
Base.metadata,
|
||||||
|
Column("id", String(36), primary_key=True),
|
||||||
|
)
|
||||||
|
access_groups = Base.metadata.tables.get("access_groups")
|
||||||
|
if access_groups is None:
|
||||||
|
access_groups = Table(
|
||||||
|
"access_groups",
|
||||||
|
Base.metadata,
|
||||||
|
Column("id", String(36), primary_key=True),
|
||||||
|
)
|
||||||
|
Base.metadata.create_all(
|
||||||
|
self.engine,
|
||||||
|
tables=[
|
||||||
|
access_users,
|
||||||
|
access_groups,
|
||||||
|
ChangeSequenceEntry.__table__,
|
||||||
|
Campaign.__table__,
|
||||||
|
CampaignVersion.__table__,
|
||||||
|
CampaignIssue.__table__,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.SessionLocal = sessionmaker(
|
||||||
|
bind=self.engine,
|
||||||
|
class_=Session,
|
||||||
|
expire_on_commit=False,
|
||||||
|
)
|
||||||
|
with self.SessionLocal() as session:
|
||||||
|
campaign = Campaign(
|
||||||
|
id="campaign-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
external_id="C-1",
|
||||||
|
name="Campaign",
|
||||||
|
current_version_id="version-1",
|
||||||
|
)
|
||||||
|
version = CampaignVersion(
|
||||||
|
id="version-1",
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_number=1,
|
||||||
|
raw_json={
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "C-1",
|
||||||
|
"name": "Campaign",
|
||||||
|
"description": "Base",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
session.add_all((campaign, version))
|
||||||
|
session.commit()
|
||||||
|
self.addCleanup(self.engine.dispose)
|
||||||
|
self.addCleanup(self.temp_dir.cleanup)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _update(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
expected_revision: int,
|
||||||
|
name: str,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.persistence.versions._updated_runtime_json",
|
||||||
|
side_effect=lambda _session, **kwargs: kwargs["raw_json"],
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.persistence.versions._write_campaign_snapshot"
|
||||||
|
),
|
||||||
|
):
|
||||||
|
return update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id="version-1",
|
||||||
|
raw_json={
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "C-1",
|
||||||
|
"name": name,
|
||||||
|
"description": "Base",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
expected_revision=expected_revision,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_only_one_of_two_writers_can_commit_the_same_revision(self) -> None:
|
||||||
|
first = self.SessionLocal()
|
||||||
|
second = self.SessionLocal()
|
||||||
|
self.addCleanup(first.close)
|
||||||
|
self.addCleanup(second.close)
|
||||||
|
first.get(CampaignVersion, "version-1")
|
||||||
|
second.get(CampaignVersion, "version-1")
|
||||||
|
|
||||||
|
saved = self._update(
|
||||||
|
first,
|
||||||
|
expected_revision=1,
|
||||||
|
name="First writer",
|
||||||
|
)
|
||||||
|
self.assertEqual(saved.edit_revision, 2)
|
||||||
|
|
||||||
|
with self.assertRaises(RevisionConflictError) as raised:
|
||||||
|
self._update(
|
||||||
|
second,
|
||||||
|
expected_revision=1,
|
||||||
|
name="Second writer",
|
||||||
|
)
|
||||||
|
self.assertEqual(raised.exception.current_revision, 2)
|
||||||
|
self.assertEqual(raised.exception.submitted_base_revision, 1)
|
||||||
|
|
||||||
|
with self.SessionLocal() as verification:
|
||||||
|
current = verification.get(CampaignVersion, "version-1")
|
||||||
|
assert current is not None
|
||||||
|
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
|
||||||
|
self.assertEqual(current.edit_revision, 2)
|
||||||
|
|
||||||
|
def test_stale_revision_is_rejected_before_mutation(self) -> None:
|
||||||
|
with self.SessionLocal() as first:
|
||||||
|
self._update(
|
||||||
|
first,
|
||||||
|
expected_revision=1,
|
||||||
|
name="First writer",
|
||||||
|
)
|
||||||
|
with self.SessionLocal() as stale:
|
||||||
|
with self.assertRaises(RevisionConflictError):
|
||||||
|
self._update(
|
||||||
|
stale,
|
||||||
|
expected_revision=1,
|
||||||
|
name="Stale writer",
|
||||||
|
)
|
||||||
|
stale.rollback()
|
||||||
|
|
||||||
|
with self.SessionLocal() as verification:
|
||||||
|
current = verification.get(CampaignVersion, "version-1")
|
||||||
|
assert current is not None
|
||||||
|
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -2,8 +2,19 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
from govoplan_campaign.backend.reports.campaigns import _job_evidence_row, _latest_by_job_id
|
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
|
||||||
|
from govoplan_campaign.backend.reports.campaigns import (
|
||||||
|
_campaign_postbox_receipt_summary,
|
||||||
|
_job_evidence_row,
|
||||||
|
_latest_by_job_id,
|
||||||
|
_load_delivery_info,
|
||||||
|
_review_decision_summary,
|
||||||
|
_review_decisions_by_job,
|
||||||
|
generate_campaign_report,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _dt() -> datetime:
|
def _dt() -> datetime:
|
||||||
@@ -75,7 +86,21 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
|
|||||||
updated_at=_dt(),
|
updated_at=_dt(),
|
||||||
)
|
)
|
||||||
|
|
||||||
row = _job_evidence_row(job, latest_smtp=smtp, latest_imap=imap)
|
row = _job_evidence_row(
|
||||||
|
job,
|
||||||
|
latest_smtp=smtp,
|
||||||
|
latest_imap=imap,
|
||||||
|
review_decision={
|
||||||
|
"decision": "accept",
|
||||||
|
"reason": "Recipient confirmed no attachment was expected.",
|
||||||
|
"actor_user_id": "reviewer-1",
|
||||||
|
"decided_at": "2026-07-08T12:30:00+00:00",
|
||||||
|
"review_key": "recipient-1",
|
||||||
|
"message_sha256": "abc123",
|
||||||
|
"issue_fingerprint": "def456",
|
||||||
|
"issue_codes": ["missing_optional_attachment"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
assert row["campaign_id"] == "campaign-1"
|
assert row["campaign_id"] == "campaign-1"
|
||||||
assert row["campaign_version_id"] == "version-1"
|
assert row["campaign_version_id"] == "version-1"
|
||||||
@@ -86,9 +111,46 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
|
|||||||
assert "bundle.zip" in row["attachment_names"]
|
assert "bundle.zip" in row["attachment_names"]
|
||||||
assert "notice.pdf" in row["attachment_names"]
|
assert "notice.pdf" in row["attachment_names"]
|
||||||
assert row["latest_smtp_status_code"] == 250
|
assert row["latest_smtp_status_code"] == 250
|
||||||
assert row["latest_smtp_response"] == "2.0.0 queued"
|
assert "latest_smtp_response" not in row
|
||||||
|
assert "latest_smtp_error_type" not in row
|
||||||
|
assert "latest_smtp_error_message" not in row
|
||||||
assert row["latest_imap_status"] == "appended"
|
assert row["latest_imap_status"] == "appended"
|
||||||
assert row["latest_imap_folder"] == "Sent"
|
assert row["latest_imap_folder"] == "Sent"
|
||||||
|
assert "latest_imap_error_message" not in row
|
||||||
|
assert "eml_storage_key" not in row
|
||||||
|
assert "eml_local_path" not in row
|
||||||
|
assert row["review_decision"] == "accept"
|
||||||
|
assert row["review_actor_user_id"] == "reviewer-1"
|
||||||
|
assert row["review_issue_codes"] == "missing_optional_attachment"
|
||||||
|
assert "review_message_sha256" not in row
|
||||||
|
|
||||||
|
|
||||||
|
def test_report_uses_only_review_decisions_for_the_current_build() -> None:
|
||||||
|
decision = {
|
||||||
|
"job_id": "job-1",
|
||||||
|
"decision": "accept",
|
||||||
|
"issue_codes": ["missing_optional_attachment"],
|
||||||
|
}
|
||||||
|
version = SimpleNamespace(
|
||||||
|
build_summary={"build_token": "build-2"},
|
||||||
|
editor_state={
|
||||||
|
"review_send": {
|
||||||
|
"build_token": "build-2",
|
||||||
|
"inspection_complete": True,
|
||||||
|
"issue_decisions": [decision],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
decisions = _review_decisions_by_job(version)
|
||||||
|
|
||||||
|
assert decisions == {"job-1": decision}
|
||||||
|
assert _review_decision_summary(decisions) == {
|
||||||
|
"exception_decision_count": 1,
|
||||||
|
"by_issue_code": {"missing_optional_attachment": 1},
|
||||||
|
}
|
||||||
|
version.editor_state["review_send"]["build_token"] = "stale-build"
|
||||||
|
assert _review_decisions_by_job(version) == {}
|
||||||
|
|
||||||
|
|
||||||
def test_latest_by_job_id_keeps_highest_attempt_number() -> None:
|
def test_latest_by_job_id_keeps_highest_attempt_number() -> None:
|
||||||
@@ -102,3 +164,118 @@ def test_latest_by_job_id_keeps_highest_attempt_number() -> None:
|
|||||||
|
|
||||||
assert latest["job-1"].attempt_number == 3
|
assert latest["job-1"].attempt_number == 3
|
||||||
assert latest["job-2"].attempt_number == 2
|
assert latest["job-2"].attempt_number == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_legacy_snapshot_never_echoes_validation_details() -> None:
|
||||||
|
version = SimpleNamespace(
|
||||||
|
execution_snapshot={
|
||||||
|
"snapshot_version": "legacy",
|
||||||
|
"smtp": {"host": "smtp.internal.example", "password": "provider-secret"},
|
||||||
|
},
|
||||||
|
execution_snapshot_hash=None,
|
||||||
|
execution_snapshot_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
delivery = _load_delivery_info(version, [])
|
||||||
|
|
||||||
|
assert delivery["load_error"] == "Execution snapshot is invalid; rebuild the selected version before delivery."
|
||||||
|
assert "provider-secret" not in repr(delivery)
|
||||||
|
assert "smtp.internal.example" not in repr(delivery)
|
||||||
|
assert "background_workers_enabled" not in delivery
|
||||||
|
assert "smtp_transport_revision" not in delivery
|
||||||
|
|
||||||
|
operator_delivery = _load_delivery_info(version, [], include_diagnostics=True)
|
||||||
|
assert "background_workers_enabled" in operator_delivery
|
||||||
|
assert "smtp_transport_revision" in operator_delivery
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_report_omits_recipient_level_failures_by_default() -> None:
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
version = SimpleNamespace(id="version-1")
|
||||||
|
with (
|
||||||
|
patch("govoplan_campaign.backend.reports.campaigns._get_campaign", return_value=campaign),
|
||||||
|
patch("govoplan_campaign.backend.reports.campaigns._selected_version", return_value=version),
|
||||||
|
patch("govoplan_campaign.backend.reports.campaigns._report_jobs", return_value=[]),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.reports.campaigns._campaign_report_payload",
|
||||||
|
return_value={"cards": {}},
|
||||||
|
) as payload,
|
||||||
|
):
|
||||||
|
report = generate_campaign_report(
|
||||||
|
object(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report == {"cards": {}}
|
||||||
|
assert payload.call_args.kwargs["include_recent_failures"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_postbox_receipt_report_aggregates_provider_state_without_identities() -> None:
|
||||||
|
class _Query:
|
||||||
|
def join(self, *args, **kwargs):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def filter(self, *args, **kwargs):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return [("delivery-1",), ("delivery-2",), ("delivery-1",)]
|
||||||
|
|
||||||
|
integration = SimpleNamespace(
|
||||||
|
receipt_evidence_available=True,
|
||||||
|
delivery_receipt_summaries=lambda session, **kwargs: {
|
||||||
|
"delivery-1": PostboxDeliveryReceiptSummaryRef(
|
||||||
|
delivery_id="delivery-1",
|
||||||
|
message_id="message-1",
|
||||||
|
postbox_id="postbox-1",
|
||||||
|
delivery_status="accepted",
|
||||||
|
accepted_at=_dt(),
|
||||||
|
currently_readable=True,
|
||||||
|
read_receipt_count=2,
|
||||||
|
acknowledged_receipt_count=1,
|
||||||
|
routed_message_count=1,
|
||||||
|
),
|
||||||
|
"delivery-2": PostboxDeliveryReceiptSummaryRef(
|
||||||
|
delivery_id="delivery-2",
|
||||||
|
message_id="message-2",
|
||||||
|
postbox_id="postbox-2",
|
||||||
|
delivery_status="accepted_vacant",
|
||||||
|
accepted_at=_dt(),
|
||||||
|
currently_readable=False,
|
||||||
|
expired_message_count=1,
|
||||||
|
),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
session = SimpleNamespace(query=lambda *args: _Query())
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.reports.campaigns.postbox_integration",
|
||||||
|
return_value=integration,
|
||||||
|
):
|
||||||
|
report = _campaign_postbox_receipt_summary(
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version=SimpleNamespace(id="version-1"),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert report["delivery_count"] == 2
|
||||||
|
assert report["currently_readable_delivery_count"] == 1
|
||||||
|
assert report["read_delivery_count"] == 1
|
||||||
|
assert report["acknowledged_delivery_count"] == 1
|
||||||
|
assert report["routed_message_count"] == 1
|
||||||
|
assert report["expired_message_count"] == 1
|
||||||
|
assert "account_id" not in repr(report)
|
||||||
|
assert "identity_id" not in repr(report)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignReportProjectionTests(unittest.TestCase):
|
||||||
|
def test_evidence_projection(self) -> None:
|
||||||
|
test_job_evidence_row_contains_transport_and_message_evidence()
|
||||||
|
|
||||||
|
def test_latest_attempt_projection(self) -> None:
|
||||||
|
test_latest_by_job_id_keeps_highest_attempt_number()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|||||||
314
tests/test_celery_redelivery_acceptance.py
Normal file
314
tests/test_celery_redelivery_acceptance.py
Normal file
@@ -0,0 +1,314 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
RUNNER_PATH = (
|
||||||
|
REPOSITORY_ROOT
|
||||||
|
/ "dev"
|
||||||
|
/ "mail-testbed"
|
||||||
|
/ "run_celery_redelivery_acceptance.py"
|
||||||
|
)
|
||||||
|
COMPOSE_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "docker-compose.yml"
|
||||||
|
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
|
||||||
|
TASK_ID = "12345678-1234-4234-8234-123456789abc"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_runner():
|
||||||
|
spec = importlib.util.spec_from_file_location(
|
||||||
|
"govoplan_campaign_celery_redelivery_acceptance",
|
||||||
|
RUNNER_PATH,
|
||||||
|
)
|
||||||
|
assert spec is not None and spec.loader is not None
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules[spec.name] = module
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
runner = _load_runner()
|
||||||
|
|
||||||
|
|
||||||
|
class _Response:
|
||||||
|
def __init__(self, status_code: int, payload: dict) -> None:
|
||||||
|
self.status_code = status_code
|
||||||
|
self._payload = payload
|
||||||
|
|
||||||
|
def json(self) -> dict:
|
||||||
|
return self._payload
|
||||||
|
|
||||||
|
|
||||||
|
class _Client:
|
||||||
|
def post(self, path: str, **_kwargs) -> _Response:
|
||||||
|
assert path.endswith("/queue")
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"queued_count": 1,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"blocked_count": 0,
|
||||||
|
"enqueued_count": 1,
|
||||||
|
"delivery_mode": "worker_queue",
|
||||||
|
"worker_queue_available": True,
|
||||||
|
"dry_run": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def get(self, path: str, **_kwargs) -> _Response:
|
||||||
|
assert path.endswith("/report")
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"cards": {
|
||||||
|
"jobs_total": 1,
|
||||||
|
"outcome_unknown": 1,
|
||||||
|
"needs_attention": 1,
|
||||||
|
},
|
||||||
|
"status_counts": {
|
||||||
|
"send": {"outcome_unknown": 1},
|
||||||
|
"imap": {"pending": 1},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _Endpoint:
|
||||||
|
host = "127.0.0.1"
|
||||||
|
port = 4025
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.release_count = 0
|
||||||
|
|
||||||
|
def wait_for_data(self, _timeout_seconds: int) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def release_held_connection(self) -> None:
|
||||||
|
self.release_count += 1
|
||||||
|
|
||||||
|
def evidence(self) -> dict[str, int]:
|
||||||
|
return {
|
||||||
|
"connection_count": 1,
|
||||||
|
"accepted_rcpt_commands": 1,
|
||||||
|
"refused_rcpt_commands": 0,
|
||||||
|
"data_transactions": 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _settings():
|
||||||
|
return runner.TestbedSettings(
|
||||||
|
smtp_host="127.0.0.1",
|
||||||
|
smtp_port=3025,
|
||||||
|
imap_host="127.0.0.1",
|
||||||
|
imap_port=3143,
|
||||||
|
username="campaign-test@govoplan.test",
|
||||||
|
password="local-test-password",
|
||||||
|
sender="campaign-test@govoplan.test",
|
||||||
|
recipient="campaign-test@govoplan.test",
|
||||||
|
sent_folder="Sent",
|
||||||
|
provider_timeout_seconds=5,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_compose_redis_is_isolated_durable_and_health_checked() -> None:
|
||||||
|
compose = COMPOSE_PATH.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
assert "redis:7-alpine" in compose
|
||||||
|
assert '"--appendonly", "yes"' in compose
|
||||||
|
assert "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379" in compose
|
||||||
|
assert 'test: ["CMD", "redis-cli", "ping"]' in compose
|
||||||
|
assert "campaign-redis-data:/data" in compose
|
||||||
|
|
||||||
|
|
||||||
|
def test_runbook_keeps_local_redelivery_distinct_from_production_supervision() -> None:
|
||||||
|
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "run_celery_redelivery_acceptance.py" in testbed
|
||||||
|
assert "same Celery task identity must be redelivered" in testbed
|
||||||
|
assert "production daemon supervision" in testbed
|
||||||
|
assert "empty broker queue/unacked set" in runbook
|
||||||
|
assert "production process manager" in runbook
|
||||||
|
|
||||||
|
|
||||||
|
def test_compose_lifecycle_targets_only_isolated_redis_service() -> None:
|
||||||
|
up = runner._compose_command(
|
||||||
|
compose_file=COMPOSE_PATH,
|
||||||
|
project_name="govoplan-campaign-redelivery-test",
|
||||||
|
operation="up",
|
||||||
|
)
|
||||||
|
down = runner._compose_command(
|
||||||
|
compose_file=COMPOSE_PATH,
|
||||||
|
project_name="govoplan-campaign-redelivery-test",
|
||||||
|
operation="down",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert up[-3:] == ["up", "--detach", "redis"]
|
||||||
|
assert down[-3:] == ["down", "--volumes", "--remove-orphans"]
|
||||||
|
assert "greenmail" not in up
|
||||||
|
assert "--project-name" in up
|
||||||
|
|
||||||
|
|
||||||
|
def test_worker_bootstrap_uses_real_late_ack_solo_celery_worker() -> None:
|
||||||
|
source = runner.WORKER_BOOTSTRAP
|
||||||
|
|
||||||
|
assert "celery.worker_main" in source
|
||||||
|
assert '"--pool=solo"' in source
|
||||||
|
assert '"--queues=send_email"' in source
|
||||||
|
assert '"visibility_timeout"' in source
|
||||||
|
assert '"polling_interval"' in source
|
||||||
|
assert "send_email.run" not in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_root_uses_platform_temp_selection() -> None:
|
||||||
|
with mock.patch(
|
||||||
|
"govoplan_campaign_celery_redelivery_acceptance.tempfile.mkdtemp",
|
||||||
|
return_value="/selected-temp/govoplan-campaign-celery-redelivery-test",
|
||||||
|
) as mkdtemp:
|
||||||
|
runtime_root = runner._create_runtime_root()
|
||||||
|
|
||||||
|
assert runtime_root == Path(
|
||||||
|
"/selected-temp/govoplan-campaign-celery-redelivery-test"
|
||||||
|
)
|
||||||
|
mkdtemp.assert_called_once_with(prefix="govoplan-campaign-celery-redelivery-")
|
||||||
|
|
||||||
|
|
||||||
|
def test_worker_log_projection_matches_redelivered_task_without_retaining_id() -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as temporary_directory:
|
||||||
|
log_path = Path(temporary_directory) / "worker.log"
|
||||||
|
log_path.write_text(
|
||||||
|
"\n".join(
|
||||||
|
[
|
||||||
|
f"Task govoplan.campaigns.send_email[{TASK_ID}] received",
|
||||||
|
f"Task govoplan.campaigns.send_email[{TASK_ID}] succeeded in 0.1s",
|
||||||
|
]
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
with log_path.open("ab") as handle:
|
||||||
|
worker = runner.WorkerProcess(
|
||||||
|
process=SimpleNamespace(),
|
||||||
|
log_path=log_path,
|
||||||
|
log_handle=handle,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert worker.received_task_ids() == (TASK_ID,)
|
||||||
|
assert worker.succeeded_task_ids() == (TASK_ID,)
|
||||||
|
|
||||||
|
|
||||||
|
def test_queue_projection_fails_closed_if_no_task_was_published() -> None:
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="one Celery task"):
|
||||||
|
runner._queue_evidence(
|
||||||
|
{
|
||||||
|
"queued_count": 1,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"blocked_count": 0,
|
||||||
|
"enqueued_count": 0,
|
||||||
|
"delivery_mode": "database_queue",
|
||||||
|
"worker_queue_available": False,
|
||||||
|
"dry_run": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_redelivery_orchestration_requires_same_task_and_no_second_smtp_effect(
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
first_worker = mock.Mock()
|
||||||
|
first_worker.received_task_ids.return_value = (TASK_ID,)
|
||||||
|
replacement_worker = mock.Mock()
|
||||||
|
replacement_worker.received_task_ids.return_value = (TASK_ID,)
|
||||||
|
workers = iter([first_worker, replacement_worker])
|
||||||
|
endpoint = _Endpoint()
|
||||||
|
durable_states = iter(
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"job_count": 1,
|
||||||
|
"send_status_counts": {"sending": 1},
|
||||||
|
"attempt_status_counts": {"smtp_in_progress": 1},
|
||||||
|
"unfinished_attempt_count": 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"job_count": 1,
|
||||||
|
"send_status_counts": {"outcome_unknown": 1},
|
||||||
|
"attempt_status_counts": {"outcome_unknown": 1},
|
||||||
|
"unfinished_attempt_count": 0,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
prepared = SimpleNamespace(
|
||||||
|
campaign_id="campaign-internal",
|
||||||
|
version_id="version-internal",
|
||||||
|
public_evidence=lambda: {
|
||||||
|
"validation": {"ok": True},
|
||||||
|
"build": {"built_count": 1},
|
||||||
|
"campaign_mail_boundary": {
|
||||||
|
"profile_reference_only": True,
|
||||||
|
"smtp_revision_frozen": True,
|
||||||
|
"imap_revision_frozen": True,
|
||||||
|
"resolved_transport_material_present": False,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(runner, "create_mail_profile", lambda *args, **kwargs: "profile-internal")
|
||||||
|
monkeypatch.setattr(runner, "prepare_campaign_scenario", lambda *args, **kwargs: prepared)
|
||||||
|
monkeypatch.setattr(runner, "_start_worker", lambda *args, **kwargs: next(workers))
|
||||||
|
monkeypatch.setattr(runner, "_wait_for_worker_ready", lambda *args, **kwargs: None)
|
||||||
|
received = iter([TASK_ID, TASK_ID])
|
||||||
|
monkeypatch.setattr(runner, "_wait_for_received_task", lambda *args, **kwargs: next(received))
|
||||||
|
monkeypatch.setattr(runner, "_wait_for_task_success", lambda *args, **kwargs: None)
|
||||||
|
monkeypatch.setattr(runner, "_kill_worker", lambda *args, **kwargs: -9)
|
||||||
|
monkeypatch.setattr(runner, "_stop_worker", lambda *args, **kwargs: None)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
runner,
|
||||||
|
"_wait_for_broker_drained",
|
||||||
|
lambda *args, **kwargs: runner.RedisBrokerState(0, 0, 0),
|
||||||
|
)
|
||||||
|
|
||||||
|
evidence = runner.execute_redelivery_scenario(
|
||||||
|
_Client(),
|
||||||
|
{"Authorization": "not-retained"},
|
||||||
|
fixture_path=FIXTURE_PATH,
|
||||||
|
settings=_settings(),
|
||||||
|
endpoint=endpoint,
|
||||||
|
redis_url="redis://127.0.0.1:36379/0",
|
||||||
|
runtime_root=Path("/not-used"),
|
||||||
|
snapshot_probe=lambda _version_id: ({}, {}),
|
||||||
|
audit_probe=lambda _campaign_id, _version_id: {
|
||||||
|
"campaign.created": 1,
|
||||||
|
"campaign.validated": 1,
|
||||||
|
"campaign.messages_built": 1,
|
||||||
|
"campaign.queued": 1,
|
||||||
|
},
|
||||||
|
delivery_probe=lambda _campaign_id, _version_id: next(durable_states),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert evidence["broker"] == {
|
||||||
|
"transport": "redis",
|
||||||
|
"same_task_identity_redelivered": True,
|
||||||
|
"first_worker_received_count": 1,
|
||||||
|
"replacement_worker_received_count": 1,
|
||||||
|
"queue_depth": 0,
|
||||||
|
"unacked_hash_count": 0,
|
||||||
|
"unacked_index_count": 0,
|
||||||
|
}
|
||||||
|
assert evidence["protocol"]["connection_count"] == 1
|
||||||
|
assert evidence["protocol"]["data_transactions"] == 1
|
||||||
|
assert evidence["recovered_durable_state"]["send_status_counts"] == {
|
||||||
|
"outcome_unknown": 1
|
||||||
|
}
|
||||||
|
assert TASK_ID not in json.dumps(evidence, sort_keys=True)
|
||||||
|
assert endpoint.release_count >= 1
|
||||||
434
tests/test_documentation.py
Normal file
434
tests/test_documentation.py
Normal file
@@ -0,0 +1,434 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.documentation import CAMPAIGN_USER_DOCUMENTATION, documentation_topics
|
||||||
|
from govoplan_core.core.modules import DocumentationContext
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _Principal:
|
||||||
|
scopes: frozenset[str]
|
||||||
|
tenant_id: str = "tenant-1"
|
||||||
|
|
||||||
|
def has(self, scope: str) -> bool:
|
||||||
|
namespace, resource, _action = scope.split(":", 2)
|
||||||
|
return scope in self.scopes or f"{namespace}:{resource}:*" in self.scopes or f"{namespace}:*" in self.scopes or "*:*" in self.scopes
|
||||||
|
|
||||||
|
|
||||||
|
class _Registry:
|
||||||
|
def __init__(self, integrations: set[str], *, interface_only: set[str] | None = None, capability_only: set[str] | None = None) -> None:
|
||||||
|
interfaces = integrations | (interface_only or set())
|
||||||
|
self._capabilities = integrations | (capability_only or set())
|
||||||
|
self._manifests = (
|
||||||
|
SimpleNamespace(
|
||||||
|
provides_interfaces=tuple(SimpleNamespace(name=name) for name in sorted(interfaces)),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def manifests(self):
|
||||||
|
return self._manifests
|
||||||
|
|
||||||
|
def has_capability(self, name: str) -> bool:
|
||||||
|
return name in self._capabilities
|
||||||
|
|
||||||
|
|
||||||
|
def _topics(scopes: set[str], integrations: set[str] | None = None, *, documentation_type: str = "user"):
|
||||||
|
return documentation_topics(
|
||||||
|
DocumentationContext(
|
||||||
|
registry=_Registry(integrations or set()),
|
||||||
|
principal=_Principal(frozenset(scopes)),
|
||||||
|
documentation_type=documentation_type, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_runtime_documentation_provider_is_registered() -> None:
|
||||||
|
from govoplan_campaign.backend.manifest import get_manifest
|
||||||
|
|
||||||
|
assert documentation_topics in get_manifest().documentation_providers
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_documentation_is_user_only_and_requires_a_campaign_task() -> None:
|
||||||
|
assert _topics({"docs:documentation:read"}) == ()
|
||||||
|
assert _topics({"campaigns:campaign:read"}, documentation_type="admin") == ()
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_documentation_reflects_actor_authority_without_exposing_scopes() -> None:
|
||||||
|
topic = _topics(
|
||||||
|
{
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:create",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:campaign:build",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
}
|
||||||
|
)[0]
|
||||||
|
|
||||||
|
configuration = topic.metadata["current_configuration"]
|
||||||
|
assert "Role authorization: Create new campaigns." in configuration
|
||||||
|
assert "Role authorization: Build exact recipient messages for review." in configuration
|
||||||
|
assert not any("queue" in item.lower() for item in configuration)
|
||||||
|
assert not any("campaigns:" in item or "files:" in item or "mail:" in item for item in configuration)
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_documentation_requires_both_interface_and_capability() -> None:
|
||||||
|
integration = "mail.campaign_delivery"
|
||||||
|
scopes = {"campaigns:campaign:read", "campaigns:campaign:update", "mail:profile:use"}
|
||||||
|
|
||||||
|
for registry in (
|
||||||
|
_Registry(set(), interface_only={integration}),
|
||||||
|
_Registry(set(), capability_only={integration}),
|
||||||
|
):
|
||||||
|
topic = documentation_topics(
|
||||||
|
DocumentationContext(registry=registry, principal=_Principal(frozenset(scopes)), documentation_type="user")
|
||||||
|
)[0]
|
||||||
|
assert not any("profile picker" in item for item in topic.metadata["current_configuration"])
|
||||||
|
|
||||||
|
topic = _topics(scopes, {integration})[0]
|
||||||
|
assert any("Mail's actor-filtered profile picker" in item for item in topic.metadata["current_configuration"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_mail_delivery_actions_are_not_presented_without_the_mail_contract() -> None:
|
||||||
|
scopes = {
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:queue",
|
||||||
|
"campaigns:campaign:send",
|
||||||
|
"campaigns:campaign:retry",
|
||||||
|
}
|
||||||
|
|
||||||
|
without_mail = _topics(scopes)[0]
|
||||||
|
with_mail = _topics(scopes, {"mail.campaign_delivery"})[0]
|
||||||
|
|
||||||
|
assert not any("Queue an eligible" in item for item in without_mail.metadata["current_configuration"])
|
||||||
|
assert any("Queue an eligible" in item for item in with_mail.metadata["current_configuration"])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("integrations", "scopes", "expected"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
{"files.campaign_attachments"},
|
||||||
|
{"campaigns:campaign:read", "campaigns:recipient:read", "files:file:read"},
|
||||||
|
"Managed campaign attachments can be previewed",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
{"addresses.lookup"},
|
||||||
|
{"campaigns:campaign:read", "campaigns:recipient:read"},
|
||||||
|
"Address records can be looked up",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
{"addresses.recipient_source"},
|
||||||
|
{
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
},
|
||||||
|
"traceable recipient snapshot",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
{"notifications.dispatch"},
|
||||||
|
{"campaigns:campaign:read"},
|
||||||
|
"status changes",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_runtime_documentation_reflects_optional_composition_permutations(
|
||||||
|
integrations: set[str],
|
||||||
|
scopes: set[str],
|
||||||
|
expected: str,
|
||||||
|
) -> None:
|
||||||
|
topic = _topics(scopes, integrations)[0]
|
||||||
|
assert any(expected in item for item in topic.metadata["current_configuration"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_documentation_full_composition_uses_only_user_facing_names() -> None:
|
||||||
|
integrations = {
|
||||||
|
"mail.campaign_delivery",
|
||||||
|
"files.campaign_attachments",
|
||||||
|
"addresses.lookup",
|
||||||
|
"addresses.recipient_source",
|
||||||
|
"notifications.dispatch",
|
||||||
|
}
|
||||||
|
topic = _topics({"*:*"}, integrations)[0]
|
||||||
|
rendered = "\n".join(
|
||||||
|
(
|
||||||
|
topic.title,
|
||||||
|
topic.summary,
|
||||||
|
topic.body,
|
||||||
|
*topic.metadata["current_configuration"],
|
||||||
|
*topic.metadata["limitations"],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "Mail's actor-filtered profile picker" in rendered
|
||||||
|
assert "Managed file versions" in rendered
|
||||||
|
assert "Address records" in rendered
|
||||||
|
assert "In-app notifications" in rendered
|
||||||
|
assert topic.metadata["help_contexts"] == ["campaigns.list", "campaign.overview"]
|
||||||
|
for technical_name in integrations:
|
||||||
|
assert technical_name not in rendered
|
||||||
|
assert "0.1." not in rendered
|
||||||
|
assert "0.2." not in rendered
|
||||||
|
assert "hostname" not in rendered.lower()
|
||||||
|
assert "secret" not in rendered.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_documentation_states_the_effective_synchronous_limit() -> None:
|
||||||
|
session = SimpleNamespace(
|
||||||
|
get=lambda _model, _id: SimpleNamespace(
|
||||||
|
settings={
|
||||||
|
"campaign_delivery_policy": {
|
||||||
|
"synchronous_send_max_recipients": 12,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with patch.dict("os.environ", {"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "25"}):
|
||||||
|
topic = documentation_topics(
|
||||||
|
DocumentationContext(
|
||||||
|
registry=_Registry({"mail.campaign_delivery"}),
|
||||||
|
principal=_Principal(frozenset({"campaigns:campaign:read", "campaigns:campaign:send"})),
|
||||||
|
session=session,
|
||||||
|
documentation_type="user",
|
||||||
|
)
|
||||||
|
)[0]
|
||||||
|
|
||||||
|
assert any(
|
||||||
|
"Send now is limited to 12 eligible recipient job(s)" in item
|
||||||
|
for item in topic.metadata["current_configuration"]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _visible_static_topics(
|
||||||
|
scopes: set[str],
|
||||||
|
*,
|
||||||
|
modules: set[str] | None = None,
|
||||||
|
capabilities: set[str] | None = None,
|
||||||
|
) -> set[str]:
|
||||||
|
installed = modules or {"campaigns"}
|
||||||
|
available_capabilities = capabilities or set()
|
||||||
|
principal = _Principal(frozenset(scopes))
|
||||||
|
visible: set[str] = set()
|
||||||
|
for topic in CAMPAIGN_USER_DOCUMENTATION:
|
||||||
|
condition = topic.conditions[0]
|
||||||
|
if not set(condition.required_modules).issubset(installed):
|
||||||
|
continue
|
||||||
|
if not set(condition.required_capabilities).issubset(available_capabilities):
|
||||||
|
continue
|
||||||
|
if not all(principal.has(scope) for scope in condition.required_scopes):
|
||||||
|
continue
|
||||||
|
visible.add(topic.id)
|
||||||
|
return visible
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_manager_sees_only_authoring_tasks_from_the_static_handbook() -> None:
|
||||||
|
visible = _visible_static_topics(
|
||||||
|
{
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:create",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:copy",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:campaign:build",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
"campaigns:report:read",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert {
|
||||||
|
"campaigns.workflow.create-campaign",
|
||||||
|
"campaigns.workflow.create-editable-successor",
|
||||||
|
"campaigns.workflow.import-recipients",
|
||||||
|
"campaigns.workflow.view-delivery-report",
|
||||||
|
}.issubset(visible)
|
||||||
|
assert "campaigns.workflow.queue-delivery" not in visible
|
||||||
|
assert "campaigns.workflow.send-small-controlled-run" not in visible
|
||||||
|
assert "campaigns.workflow.export-delivery-report" not in visible
|
||||||
|
assert "campaigns.workflow.share-campaign" not in visible
|
||||||
|
assert "campaigns.workflow.archive-campaign" not in visible
|
||||||
|
assert "campaigns.workflow.delete-untouched-draft" not in visible
|
||||||
|
assert "campaigns.workflow.create-campaign" not in _visible_static_topics({"campaigns:campaign:create"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_sender_sees_queue_and_send_only_with_the_mail_contract_and_profile_authority() -> None:
|
||||||
|
scopes = {
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:queue",
|
||||||
|
"campaigns:campaign:send",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:report:read",
|
||||||
|
"mail:profile:use",
|
||||||
|
}
|
||||||
|
|
||||||
|
without_mail = _visible_static_topics(scopes)
|
||||||
|
with_mail = _visible_static_topics(
|
||||||
|
scopes,
|
||||||
|
modules={"campaigns", "mail"},
|
||||||
|
capabilities={"mail.campaign_delivery"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "campaigns.workflow.queue-delivery" not in without_mail
|
||||||
|
assert "campaigns.workflow.send-small-controlled-run" not in without_mail
|
||||||
|
assert "campaigns.workflow.queue-delivery" in with_mail
|
||||||
|
assert "campaigns.workflow.send-small-controlled-run" in with_mail
|
||||||
|
|
||||||
|
queue_topic = next(
|
||||||
|
topic for topic in CAMPAIGN_USER_DOCUMENTATION
|
||||||
|
if topic.id == "campaigns.workflow.queue-delivery"
|
||||||
|
)
|
||||||
|
assert any(link.href == "/campaigns/queue" for link in queue_topic.links)
|
||||||
|
|
||||||
|
|
||||||
|
def test_connected_authoring_tasks_require_their_declared_contracts_and_permissions() -> None:
|
||||||
|
attachment_scopes = {
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:campaign:validate",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"files:file:read",
|
||||||
|
"files:file:share",
|
||||||
|
}
|
||||||
|
source_scopes = {
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:write",
|
||||||
|
"campaigns:recipient:import",
|
||||||
|
}
|
||||||
|
|
||||||
|
files_visible = _visible_static_topics(
|
||||||
|
attachment_scopes,
|
||||||
|
modules={"campaigns", "files"},
|
||||||
|
capabilities={"files.campaign_attachments"},
|
||||||
|
)
|
||||||
|
addresses_visible = _visible_static_topics(
|
||||||
|
source_scopes,
|
||||||
|
modules={"campaigns", "addresses"},
|
||||||
|
capabilities={"addresses.recipient_source"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "campaigns.workflow.use-managed-attachments" in files_visible
|
||||||
|
assert "campaigns.workflow.import-address-source" in addresses_visible
|
||||||
|
assert "campaigns.workflow.use-managed-attachments" not in _visible_static_topics(attachment_scopes)
|
||||||
|
assert "campaigns.workflow.import-address-source" not in _visible_static_topics(source_scopes)
|
||||||
|
|
||||||
|
|
||||||
|
def test_report_export_and_lifecycle_tasks_are_independently_permission_gated() -> None:
|
||||||
|
exporter = _visible_static_topics(
|
||||||
|
{
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:report:read",
|
||||||
|
"campaigns:report:export",
|
||||||
|
"campaigns:recipient:read",
|
||||||
|
"campaigns:recipient:export",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
custodian = _visible_static_topics(
|
||||||
|
{
|
||||||
|
"campaigns:campaign:read",
|
||||||
|
"campaigns:campaign:share",
|
||||||
|
"campaigns:campaign:archive",
|
||||||
|
"campaigns:campaign:delete",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "campaigns.workflow.export-delivery-report" in exporter
|
||||||
|
assert "campaigns.workflow.view-delivery-report" in exporter
|
||||||
|
assert "campaigns.workflow.share-campaign" in custodian
|
||||||
|
assert "campaigns.workflow.archive-campaign" in custodian
|
||||||
|
assert "campaigns.workflow.delete-untouched-draft" in custodian
|
||||||
|
assert "campaigns.workflow.export-delivery-report" not in custodian
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_report_task_never_implies_recipient_detail_or_export_authority() -> None:
|
||||||
|
visible = _visible_static_topics({"campaigns:report:read"})
|
||||||
|
|
||||||
|
assert visible == {"campaigns.workflow.view-aggregate-delivery-report"}
|
||||||
|
topic = next(
|
||||||
|
item for item in CAMPAIGN_USER_DOCUMENTATION
|
||||||
|
if item.id == "campaigns.workflow.view-aggregate-delivery-report"
|
||||||
|
)
|
||||||
|
assert topic.metadata["route"] == "/campaigns/reports"
|
||||||
|
assert "export" in topic.metadata["verification"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_handbook_distinguishes_shipped_aggregate_reports_from_detailed_report_gaps() -> None:
|
||||||
|
handbook = " ".join(
|
||||||
|
(
|
||||||
|
Path(__file__).resolve().parents[1] / "docs" / "CAMPAIGN_HANDBOOK.md"
|
||||||
|
).read_text(encoding="utf-8").lower().split()
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "aggregate-only reader ui remains open" not in handbook
|
||||||
|
assert "separate aggregate **reports** surface" in handbook
|
||||||
|
assert "permission-aware action visibility on that detailed surface remains open work" in handbook
|
||||||
|
|
||||||
|
|
||||||
|
def test_static_campaign_handbook_has_unique_ids_help_contexts_and_no_planned_resend_claim() -> None:
|
||||||
|
from govoplan_campaign.backend.manifest import get_manifest
|
||||||
|
|
||||||
|
topics = get_manifest().documentation
|
||||||
|
ids = [topic.id for topic in topics]
|
||||||
|
rendered_static = "\n".join(
|
||||||
|
(topic.title + "\n" + topic.summary + "\n" + topic.body).lower()
|
||||||
|
for topic in CAMPAIGN_USER_DOCUMENTATION
|
||||||
|
)
|
||||||
|
known_help_contexts = {
|
||||||
|
"campaigns.list",
|
||||||
|
"campaign.overview",
|
||||||
|
"campaign.settings",
|
||||||
|
"campaign.fields",
|
||||||
|
"campaign.template",
|
||||||
|
"campaign.attachments",
|
||||||
|
"campaign.recipients",
|
||||||
|
"campaign.recipient-data",
|
||||||
|
"campaign.server-settings",
|
||||||
|
"campaign.global-settings",
|
||||||
|
"campaign.review-send",
|
||||||
|
"campaign.report",
|
||||||
|
"campaign.audit",
|
||||||
|
"campaign.json",
|
||||||
|
}
|
||||||
|
|
||||||
|
assert len(ids) == len(set(ids))
|
||||||
|
assert "single resend" not in rendered_static
|
||||||
|
for topic in CAMPAIGN_USER_DOCUMENTATION:
|
||||||
|
assert topic.metadata["kind"] == "workflow"
|
||||||
|
assert topic.metadata["prerequisites"]
|
||||||
|
assert topic.metadata["steps"]
|
||||||
|
assert topic.metadata["outcome"]
|
||||||
|
assert topic.metadata["verification"]
|
||||||
|
assert set(topic.metadata["help_contexts"]).issubset(known_help_contexts)
|
||||||
|
|
||||||
|
existing_user_workflows = {
|
||||||
|
topic.id: topic
|
||||||
|
for topic in topics
|
||||||
|
if topic.id
|
||||||
|
in {
|
||||||
|
"campaigns.mail-profile-user-journey",
|
||||||
|
"campaigns.workflow.prepare-validate-and-build",
|
||||||
|
"campaigns.workflow.complete-review",
|
||||||
|
"campaigns.workflow.retry-and-reconcile",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert set(existing_user_workflows) == {
|
||||||
|
"campaigns.mail-profile-user-journey",
|
||||||
|
"campaigns.workflow.prepare-validate-and-build",
|
||||||
|
"campaigns.workflow.complete-review",
|
||||||
|
"campaigns.workflow.retry-and-reconcile",
|
||||||
|
}
|
||||||
|
for topic in existing_user_workflows.values():
|
||||||
|
assert topic.metadata["help_contexts"]
|
||||||
110
tests/test_editor_state_security.py
Normal file
110
tests/test_editor_state_security.py
Normal file
@@ -0,0 +1,110 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
campaign_editor_state_for_edit,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.schemas import (
|
||||||
|
CampaignVersionResponse,
|
||||||
|
CampaignVersionUpdateRequest,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"editor_state",
|
||||||
|
[
|
||||||
|
{"smtp": {"host": "smtp.example.test", "password": "secret"}},
|
||||||
|
{"transport": {"imap_password": "secret"}},
|
||||||
|
{
|
||||||
|
"review_send": {
|
||||||
|
"build_token": "forged",
|
||||||
|
"inspection_complete": True,
|
||||||
|
"reviewed_message_keys": [],
|
||||||
|
"updated_at": "2026-07-21T00:00:00+00:00",
|
||||||
|
"updated_by_user_id": "user-1",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_version_updates_reject_arbitrary_or_server_owned_editor_state(
|
||||||
|
editor_state: dict[str, object],
|
||||||
|
) -> None:
|
||||||
|
with pytest.raises(ValidationError, match="unsupported or transport-owned"):
|
||||||
|
CampaignVersionUpdateRequest(editor_state=editor_state)
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_response_omits_legacy_secret_bearing_editor_state() -> None:
|
||||||
|
response = CampaignVersionResponse.model_validate(
|
||||||
|
{
|
||||||
|
"id": "version-1",
|
||||||
|
"campaign_id": "campaign-1",
|
||||||
|
"version_number": 1,
|
||||||
|
"schema_version": "1.0",
|
||||||
|
"editor_state": {
|
||||||
|
"opt_ins": {"inline_guidance": True},
|
||||||
|
"smtp": {"host": "smtp.internal.example", "password": "provider-secret"},
|
||||||
|
},
|
||||||
|
"created_at": datetime.now(UTC),
|
||||||
|
"updated_at": datetime.now(UTC),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.editor_state == {"opt_ins": {"inline_guidance": True}}
|
||||||
|
assert "provider-secret" not in repr(response)
|
||||||
|
assert "internal.example" not in repr(response)
|
||||||
|
|
||||||
|
|
||||||
|
def test_review_build_token_is_visible_only_in_operator_diagnostics() -> None:
|
||||||
|
value = {
|
||||||
|
"id": "version-1",
|
||||||
|
"campaign_id": "campaign-1",
|
||||||
|
"version_number": 1,
|
||||||
|
"schema_version": "1.0",
|
||||||
|
"editor_state": {
|
||||||
|
"review_send": {
|
||||||
|
"build_token": "internal-build-token",
|
||||||
|
"inspection_complete": True,
|
||||||
|
"reviewed_message_keys": ["entry-1"],
|
||||||
|
"updated_at": "2026-07-21T00:00:00+00:00",
|
||||||
|
"updated_by_user_id": "reviewer-1",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"created_at": datetime.now(UTC),
|
||||||
|
"updated_at": datetime.now(UTC),
|
||||||
|
}
|
||||||
|
|
||||||
|
public = CampaignVersionResponse.model_validate(value)
|
||||||
|
operator = CampaignVersionResponse.model_validate(
|
||||||
|
value,
|
||||||
|
context={"include_diagnostics": True},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "build_token" not in public.editor_state["review_send"]
|
||||||
|
assert operator.editor_state["review_send"]["build_token"] == "internal-build-token"
|
||||||
|
|
||||||
|
|
||||||
|
def test_fork_copy_keeps_only_client_owned_bounded_metadata() -> None:
|
||||||
|
copied = campaign_editor_state_for_edit(
|
||||||
|
{
|
||||||
|
"created_from": "minimal_campaign",
|
||||||
|
"field_overrides": {"department": False},
|
||||||
|
"review_send": {
|
||||||
|
"build_token": "build-1",
|
||||||
|
"inspection_complete": True,
|
||||||
|
"reviewed_message_keys": ["entry-1"],
|
||||||
|
"updated_at": "2026-07-21T00:00:00+00:00",
|
||||||
|
"updated_by_user_id": "reviewer-1",
|
||||||
|
},
|
||||||
|
"credentials": {"password": "legacy-secret"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert copied == {
|
||||||
|
"created_from": "minimal_campaign",
|
||||||
|
"field_overrides": {"department": False},
|
||||||
|
}
|
||||||
|
assert "legacy-secret" not in repr(copied)
|
||||||
207
tests/test_example_campaigns.py
Normal file
207
tests/test_example_campaigns.py
Normal file
@@ -0,0 +1,207 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
FIXTURE_ROOT = REPOSITORY_ROOT / "examples" / "simple-announcement"
|
||||||
|
|
||||||
|
|
||||||
|
_ISOLATED_ACCEPTANCE_PROGRAM = r"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import importlib.abc
|
||||||
|
import json
|
||||||
|
import socket
|
||||||
|
import sys
|
||||||
|
from email import policy
|
||||||
|
from email.parser import BytesParser
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
source_root = Path(sys.argv[1]).resolve()
|
||||||
|
fixture_root = Path(sys.argv[2]).resolve()
|
||||||
|
sys.path.insert(0, str(source_root))
|
||||||
|
|
||||||
|
|
||||||
|
class AbsentOptionalModuleFinder(importlib.abc.MetaPathFinder):
|
||||||
|
absent_roots = {"govoplan_files", "govoplan_mail"}
|
||||||
|
|
||||||
|
def find_spec(self, fullname, path=None, target=None):
|
||||||
|
if fullname.partition(".")[0] in self.absent_roots:
|
||||||
|
raise ModuleNotFoundError(
|
||||||
|
f"{fullname} is intentionally absent in the Campaign fixture check",
|
||||||
|
name=fullname,
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
sys.meta_path.insert(0, AbsentOptionalModuleFinder())
|
||||||
|
|
||||||
|
|
||||||
|
def deny_network(*args, **kwargs):
|
||||||
|
raise AssertionError("the Campaign validate/build fixture must not open a network connection")
|
||||||
|
|
||||||
|
|
||||||
|
class NoNetworkSocket(socket.socket):
|
||||||
|
def connect(self, address):
|
||||||
|
deny_network(address)
|
||||||
|
|
||||||
|
def connect_ex(self, address):
|
||||||
|
deny_network(address)
|
||||||
|
|
||||||
|
|
||||||
|
socket.create_connection = deny_network
|
||||||
|
socket.socket = NoNetworkSocket
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign import ( # noqa: E402
|
||||||
|
load_campaign_config,
|
||||||
|
load_campaign_json,
|
||||||
|
validate_campaign_config,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.messages import build_campaign_messages # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
metadata = json.loads((fixture_root / "fixture.json").read_text(encoding="utf-8"))
|
||||||
|
assert metadata["required_modules"] == ["core", "access", "campaigns"]
|
||||||
|
assert metadata["absent_optional_modules"] == ["files", "mail"]
|
||||||
|
assert metadata["external_effects"] == "forbidden"
|
||||||
|
|
||||||
|
campaign_file = fixture_root / metadata["campaign_file"]
|
||||||
|
raw_campaign = load_campaign_json(campaign_file)
|
||||||
|
|
||||||
|
|
||||||
|
def iter_keys(value):
|
||||||
|
if isinstance(value, dict):
|
||||||
|
for key, nested in value.items():
|
||||||
|
yield key.casefold()
|
||||||
|
yield from iter_keys(nested)
|
||||||
|
elif isinstance(value, list):
|
||||||
|
for nested in value:
|
||||||
|
yield from iter_keys(nested)
|
||||||
|
|
||||||
|
|
||||||
|
forbidden_key_fragments = ("credential", "imap", "mail_profile", "password", "secret", "smtp")
|
||||||
|
assert not [
|
||||||
|
key
|
||||||
|
for key in iter_keys(raw_campaign)
|
||||||
|
if any(fragment in key for fragment in forbidden_key_fragments)
|
||||||
|
]
|
||||||
|
|
||||||
|
config = load_campaign_config(campaign_file)
|
||||||
|
assert config.server.mail_profile_id is None
|
||||||
|
assert not config.attachments.global_
|
||||||
|
|
||||||
|
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||||
|
assert validation.ok
|
||||||
|
assert validation.error_count == 0
|
||||||
|
assert validation.warning_count == 0
|
||||||
|
assert validation.entries_count == metadata["expected"]["entries_count"]
|
||||||
|
|
||||||
|
|
||||||
|
def build(output_name):
|
||||||
|
return build_campaign_messages(
|
||||||
|
config,
|
||||||
|
campaign_file=campaign_file,
|
||||||
|
output_dir=fixture_root.parent / output_name,
|
||||||
|
write_eml=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
first = build("build-first")
|
||||||
|
second = build("build-second")
|
||||||
|
expected = metadata["expected"]
|
||||||
|
for result in (first, second):
|
||||||
|
assert result.report.campaign_id == expected["campaign_id"]
|
||||||
|
assert result.report.entries_count == expected["entries_count"]
|
||||||
|
assert result.report.built_count == expected["built_count"]
|
||||||
|
assert result.report.build_failed_count == 0
|
||||||
|
assert result.report.queueable_count == expected["queueable_count"]
|
||||||
|
assert len(result.built_messages) == 1
|
||||||
|
|
||||||
|
built = result.built_messages[0]
|
||||||
|
assert built.mime is not None
|
||||||
|
assert built.draft.subject == expected["subject"]
|
||||||
|
assert built.draft.validation_status.value == "ready"
|
||||||
|
assert built.draft.send_status.value == "draft"
|
||||||
|
assert built.draft.imap_status.value == "not_requested"
|
||||||
|
assert built.draft.attachment_count == expected["attachment_count"]
|
||||||
|
assert not built.draft.attachments
|
||||||
|
assert not built.draft.issues
|
||||||
|
assert built.draft.from_ is not None
|
||||||
|
assert built.draft.from_.email == "announcements@example.test"
|
||||||
|
assert [address.email for address in built.draft.to] == ["recipient@example.test"]
|
||||||
|
assert built.mime["Subject"] == expected["subject"]
|
||||||
|
assert "Hello Example Recipient" in built.mime.get_content()
|
||||||
|
assert list(built.mime.iter_attachments()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def normalized_eml(path_value):
|
||||||
|
message = BytesParser(policy=policy.default).parsebytes(Path(path_value).read_bytes())
|
||||||
|
del message["Date"]
|
||||||
|
del message["Message-ID"]
|
||||||
|
return message.as_bytes(policy=policy.default)
|
||||||
|
|
||||||
|
|
||||||
|
first_eml = normalized_eml(first.report.messages[0].eml_path)
|
||||||
|
second_eml = normalized_eml(second.report.messages[0].eml_path)
|
||||||
|
assert first_eml == second_eml
|
||||||
|
assert not any(
|
||||||
|
name == root or name.startswith(root + ".")
|
||||||
|
for name in sys.modules
|
||||||
|
for root in ("govoplan_files", "govoplan_mail")
|
||||||
|
)
|
||||||
|
|
||||||
|
print(json.dumps({
|
||||||
|
"campaign_id": first.report.campaign_id,
|
||||||
|
"built_count": first.report.built_count,
|
||||||
|
"queueable_count": first.report.queueable_count,
|
||||||
|
"normalized_eml_sha256": hashlib.sha256(first_eml).hexdigest(),
|
||||||
|
}, sort_keys=True))
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignExampleAcceptanceTests(unittest.TestCase):
|
||||||
|
def test_simple_announcement_validates_and_builds_without_mail_or_files(self) -> None:
|
||||||
|
self.assertTrue((FIXTURE_ROOT / "campaign.json").is_file())
|
||||||
|
self.assertTrue((FIXTURE_ROOT / "fixture.json").is_file())
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory(prefix="govoplan-campaign-acceptance-", dir="/tmp") as temp_dir:
|
||||||
|
workspace = Path(temp_dir).resolve()
|
||||||
|
self.assertNotIn(REPOSITORY_ROOT, workspace.parents)
|
||||||
|
isolated_fixture = workspace / "simple-announcement"
|
||||||
|
shutil.copytree(FIXTURE_ROOT, isolated_fixture)
|
||||||
|
|
||||||
|
completed = subprocess.run(
|
||||||
|
[
|
||||||
|
sys.executable,
|
||||||
|
"-I",
|
||||||
|
"-c",
|
||||||
|
_ISOLATED_ACCEPTANCE_PROGRAM,
|
||||||
|
str(REPOSITORY_ROOT / "src"),
|
||||||
|
str(isolated_fixture),
|
||||||
|
],
|
||||||
|
cwd=workspace,
|
||||||
|
check=False,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=30,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(completed.returncode, 0, completed.stderr or completed.stdout)
|
||||||
|
evidence = json.loads(completed.stdout)
|
||||||
|
self.assertEqual(evidence["campaign_id"], "simple-announcement")
|
||||||
|
self.assertEqual(evidence["built_count"], 1)
|
||||||
|
self.assertEqual(evidence["queueable_count"], 1)
|
||||||
|
self.assertRegex(evidence["normalized_eml_sha256"], r"^[0-9a-f]{64}$")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
67
tests/test_excluded_delivery_status.py
Normal file
67
tests/test_excluded_delivery_status.py
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from importlib import import_module
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from sqlalchemy import create_engine, text
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.models import BuildStatus, SendStatus
|
||||||
|
from govoplan_campaign.backend.messages.models import ImapStatus, MessageDraft, MessageValidationStatus
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import _job_from_message
|
||||||
|
|
||||||
|
|
||||||
|
def test_excluded_message_persists_explicit_skipped_transport_states() -> None:
|
||||||
|
message = MessageDraft(
|
||||||
|
entry_index=0,
|
||||||
|
entry_id="excluded-entry",
|
||||||
|
active=True,
|
||||||
|
build_status=BuildStatus.BUILT,
|
||||||
|
validation_status=MessageValidationStatus.EXCLUDED,
|
||||||
|
send_status=SendStatus.SKIPPED,
|
||||||
|
imap_status=ImapStatus.SKIPPED,
|
||||||
|
)
|
||||||
|
|
||||||
|
job = _job_from_message(
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id="version-1",
|
||||||
|
message=message,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert job.send_status == "skipped"
|
||||||
|
assert job.imap_status == "skipped"
|
||||||
|
|
||||||
|
|
||||||
|
def test_status_migration_only_normalizes_excluded_rows_without_transport_evidence() -> None:
|
||||||
|
migration = import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions."
|
||||||
|
"d8b3e2c1f4a5_v0110_excluded_delivery_skipped"
|
||||||
|
)
|
||||||
|
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||||
|
with engine.begin() as connection:
|
||||||
|
connection.execute(text(
|
||||||
|
"CREATE TABLE campaign_jobs ("
|
||||||
|
"id TEXT PRIMARY KEY, validation_status TEXT NOT NULL, "
|
||||||
|
"send_status TEXT NOT NULL, imap_status TEXT NOT NULL)"
|
||||||
|
))
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO campaign_jobs (id, validation_status, send_status, imap_status) VALUES "
|
||||||
|
"('untouched', 'excluded', 'not_queued', 'pending'), "
|
||||||
|
"('evidence', 'excluded', 'smtp_accepted', 'appended'), "
|
||||||
|
"('queueable', 'ready', 'not_queued', 'pending')"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with patch.object(migration.op, "get_bind", return_value=connection):
|
||||||
|
migration.upgrade()
|
||||||
|
|
||||||
|
rows = {
|
||||||
|
row.id: (row.send_status, row.imap_status)
|
||||||
|
for row in connection.execute(
|
||||||
|
text("SELECT id, send_status, imap_status FROM campaign_jobs ORDER BY id")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert rows["untouched"] == ("skipped", "skipped")
|
||||||
|
assert rows["evidence"] == ("smtp_accepted", "appended")
|
||||||
|
assert rows["queueable"] == ("not_queued", "pending")
|
||||||
139
tests/test_execution_snapshot_integrity.py
Normal file
139
tests/test_execution_snapshot_integrity.py
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.models import DeliveryConfig
|
||||||
|
from govoplan_campaign.backend.sending.execution import (
|
||||||
|
ExecutionSnapshotError,
|
||||||
|
create_execution_snapshot,
|
||||||
|
ensure_execution_snapshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _raw_campaign() -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {"id": "campaign-1", "name": "Campaign", "mode": "send"},
|
||||||
|
"server": {"mail_profile_id": "profile-1"},
|
||||||
|
"recipients": {"from": [{"email": "sender@example.test"}]},
|
||||||
|
"template": {"subject": "Subject", "text": "Body", "body_mode": "text"},
|
||||||
|
"entries": {"inline": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _job() -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
entry_index=0,
|
||||||
|
entry_id="entry-1",
|
||||||
|
recipient_email="recipient@example.test",
|
||||||
|
subject="Subject",
|
||||||
|
message_id_header="<message@example.test>",
|
||||||
|
eml_size_bytes=123,
|
||||||
|
eml_sha256="eml-digest",
|
||||||
|
build_status="built",
|
||||||
|
validation_status="ready",
|
||||||
|
resolved_recipients={"to": ["recipient@example.test"]},
|
||||||
|
resolved_attachments=[],
|
||||||
|
issues_snapshot=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _Query:
|
||||||
|
def __init__(self, jobs: list[SimpleNamespace]):
|
||||||
|
self.jobs = jobs
|
||||||
|
|
||||||
|
def filter(self, *_args):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def order_by(self, *_args):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return list(self.jobs)
|
||||||
|
|
||||||
|
|
||||||
|
class _Session:
|
||||||
|
def __init__(self, jobs: list[SimpleNamespace]):
|
||||||
|
self.jobs = jobs
|
||||||
|
|
||||||
|
def query(self, _model):
|
||||||
|
return _Query(self.jobs)
|
||||||
|
|
||||||
|
|
||||||
|
def _snapshotted_version(job: SimpleNamespace):
|
||||||
|
version = SimpleNamespace(
|
||||||
|
id="version-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
raw_json=_raw_campaign(),
|
||||||
|
build_summary={"build_token": "build-1", "built_at": "2026-07-21T00:00:00+00:00"},
|
||||||
|
)
|
||||||
|
payload, digest = create_execution_snapshot(
|
||||||
|
version, # type: ignore[arg-type]
|
||||||
|
mail_profile_id="profile-1",
|
||||||
|
smtp_transport_revision="smtp-revision",
|
||||||
|
imap_transport_revision="imap-revision",
|
||||||
|
delivery=DeliveryConfig(),
|
||||||
|
jobs=[job], # type: ignore[list-item]
|
||||||
|
build_summary=version.build_summary,
|
||||||
|
)
|
||||||
|
version.execution_snapshot = payload
|
||||||
|
version.execution_snapshot_hash = digest
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure(session: _Session, version) -> None:
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.sending.execution.files_integration",
|
||||||
|
return_value=SimpleNamespace(available=False),
|
||||||
|
):
|
||||||
|
ensure_execution_snapshot(session, version) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
|
||||||
|
def test_current_snapshot_requires_its_persisted_checksum() -> None:
|
||||||
|
job = _job()
|
||||||
|
version = _snapshotted_version(job)
|
||||||
|
version.execution_snapshot_hash = None
|
||||||
|
|
||||||
|
with pytest.raises(ExecutionSnapshotError, match="checksum is missing"):
|
||||||
|
_ensure(_Session([job]), version)
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_json_drift_is_rejected_before_delivery() -> None:
|
||||||
|
job = _job()
|
||||||
|
version = _snapshotted_version(job)
|
||||||
|
version.raw_json["template"] = {"subject": "Changed", "text": "Body"}
|
||||||
|
|
||||||
|
with pytest.raises(ExecutionSnapshotError, match="Campaign inputs changed"):
|
||||||
|
_ensure(_Session([job]), version)
|
||||||
|
|
||||||
|
|
||||||
|
def test_post_build_recipient_drift_cannot_redirect_delivery() -> None:
|
||||||
|
job = _job()
|
||||||
|
version = _snapshotted_version(job)
|
||||||
|
job.resolved_recipients = {"to": ["attacker@example.test"]}
|
||||||
|
|
||||||
|
with pytest.raises(ExecutionSnapshotError, match="job inputs changed"):
|
||||||
|
_ensure(_Session([job]), version)
|
||||||
|
|
||||||
|
|
||||||
|
def test_effect_check_verifies_only_the_claimed_job_in_constant_time() -> None:
|
||||||
|
job = _job()
|
||||||
|
version = _snapshotted_version(job)
|
||||||
|
session = SimpleNamespace(
|
||||||
|
query=lambda *_args: pytest.fail("per-effect validation must not rescan every campaign job")
|
||||||
|
)
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.sending.execution.files_integration",
|
||||||
|
return_value=SimpleNamespace(available=False),
|
||||||
|
):
|
||||||
|
ensure_execution_snapshot(
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
version,
|
||||||
|
effect_job=job, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
464
tests/test_imap_append_idempotency.py
Normal file
464
tests/test_imap_append_idempotency.py
Normal file
@@ -0,0 +1,464 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
import importlib
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from alembic.migration import MigrationContext
|
||||||
|
from alembic.operations import Operations
|
||||||
|
from sqlalchemy import create_engine, inspect, text
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.routes import delivery as router
|
||||||
|
from govoplan_campaign.backend.db.models import (
|
||||||
|
CampaignJob,
|
||||||
|
ImapAppendAttempt,
|
||||||
|
JobImapStatus,
|
||||||
|
JobSendStatus,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.retention import _apply_eml_retention
|
||||||
|
from govoplan_campaign.backend.schemas import CampaignResolveOutcomeRequest
|
||||||
|
from govoplan_campaign.backend.sending.jobs import (
|
||||||
|
AppendSentResult,
|
||||||
|
QueueingError,
|
||||||
|
_claim_job_for_imap_append,
|
||||||
|
_record_imap_append_success,
|
||||||
|
append_sent_for_job,
|
||||||
|
reconcile_job_outcome,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("imap_status", "expected_status"),
|
||||||
|
[
|
||||||
|
(JobImapStatus.APPENDING.value, "append_in_progress"),
|
||||||
|
(JobImapStatus.OUTCOME_UNKNOWN.value, JobImapStatus.OUTCOME_UNKNOWN.value),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_in_progress_and_unknown_jobs_never_reinvoke_mail_provider(
|
||||||
|
imap_status: str,
|
||||||
|
expected_status: str,
|
||||||
|
) -> None:
|
||||||
|
job = SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
send_status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
imap_status=imap_status,
|
||||||
|
)
|
||||||
|
session = SimpleNamespace(get=lambda _model, _id: job)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs._imap_attempt_count", return_value=1),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs.mail_integration") as mail,
|
||||||
|
):
|
||||||
|
result = append_sent_for_job(
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
job_id="job-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.status == expected_status
|
||||||
|
mail.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_imap_claim_is_a_single_atomic_state_transition() -> None:
|
||||||
|
session = MagicMock()
|
||||||
|
query = session.query.return_value
|
||||||
|
query.filter.return_value.update.return_value = 1
|
||||||
|
job = SimpleNamespace(id="job-1")
|
||||||
|
|
||||||
|
claim_token = _claim_job_for_imap_append(
|
||||||
|
session,
|
||||||
|
job, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert claim_token
|
||||||
|
changes = query.filter.return_value.update.call_args.args[0]
|
||||||
|
assert changes[CampaignJob.imap_status] == JobImapStatus.APPENDING.value
|
||||||
|
assert changes[CampaignJob.imap_claim_token] == claim_token
|
||||||
|
session.commit.assert_called_once_with()
|
||||||
|
session.expire_all.assert_called_once_with()
|
||||||
|
|
||||||
|
|
||||||
|
def test_late_provider_acknowledgement_cannot_overwrite_a_changed_claim() -> None:
|
||||||
|
session = MagicMock()
|
||||||
|
session.query.return_value.filter.return_value.update.return_value = 0
|
||||||
|
job = SimpleNamespace(id="job-1")
|
||||||
|
attempt = SimpleNamespace(id="attempt-1", attempt_number=1)
|
||||||
|
expected = AppendSentResult(
|
||||||
|
job_id="job-1",
|
||||||
|
status=JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
attempt_number=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._imap_result_after_lost_claim",
|
||||||
|
return_value=expected,
|
||||||
|
) as lost_claim,
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs.files_integration") as files,
|
||||||
|
):
|
||||||
|
result = _record_imap_append_success(
|
||||||
|
session,
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
attempt=attempt, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
folder="Sent",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is expected
|
||||||
|
lost_claim.assert_called_once_with(
|
||||||
|
session,
|
||||||
|
job_id="job-1",
|
||||||
|
attempt=attempt,
|
||||||
|
provider_succeeded=True,
|
||||||
|
)
|
||||||
|
files.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_post_provider_persistence_failure_freezes_imap_retry() -> None:
|
||||||
|
job = SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
send_status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
imap_status=JobImapStatus.PENDING.value,
|
||||||
|
)
|
||||||
|
version = SimpleNamespace(id="version-1")
|
||||||
|
snapshot = SimpleNamespace(
|
||||||
|
mail_profile_id="profile-1",
|
||||||
|
smtp_transport_revision="smtp-revision",
|
||||||
|
imap_transport_revision="imap-revision",
|
||||||
|
smtp_server_id=None,
|
||||||
|
smtp_credential_id=None,
|
||||||
|
imap_server_id=None,
|
||||||
|
imap_credential_id=None,
|
||||||
|
delivery=SimpleNamespace(
|
||||||
|
imap_append_sent=SimpleNamespace(enabled=True, folder="Sent"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
attempt = SimpleNamespace(id="attempt-1", attempt_number=1)
|
||||||
|
|
||||||
|
class Session:
|
||||||
|
def get(self, model, _object_id):
|
||||||
|
return version if model.__name__ == "CampaignVersion" else job
|
||||||
|
|
||||||
|
class Mail:
|
||||||
|
def append_campaign_message_to_sent(self, *_args, **_kwargs):
|
||||||
|
return SimpleNamespace(folder="Sent")
|
||||||
|
|
||||||
|
expected = AppendSentResult(
|
||||||
|
job_id="job-1",
|
||||||
|
status=JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
attempt_number=1,
|
||||||
|
)
|
||||||
|
session = Session()
|
||||||
|
with (
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs.ensure_execution_snapshot", return_value=snapshot),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs._load_eml_bytes_for_job", return_value=b"message"),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs._claim_job_for_imap_append", return_value="claim-1"),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs._record_imap_attempt_start", return_value=attempt),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs.mail_integration", return_value=Mail()),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._record_imap_append_success",
|
||||||
|
side_effect=OSError("database unavailable"),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._mark_imap_append_outcome_unknown_after_effect",
|
||||||
|
return_value=expected,
|
||||||
|
) as mark_unknown,
|
||||||
|
):
|
||||||
|
result = append_sent_for_job(
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
job_id="job-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is expected
|
||||||
|
assert "Automatic retry is stopped" in mark_unknown.call_args.kwargs["reason"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_attempt_numbers_are_unique_per_job_in_the_model_contract() -> None:
|
||||||
|
constraints = {
|
||||||
|
constraint.name
|
||||||
|
for constraint in ImapAppendAttempt.__table__.constraints
|
||||||
|
}
|
||||||
|
assert "uq_imap_append_attempts_job_attempt" in constraints
|
||||||
|
|
||||||
|
|
||||||
|
def test_imap_claim_migration_preserves_and_renumbers_duplicate_attempts() -> None:
|
||||||
|
migration = importlib.import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions.3c4d5e6f8192_v019_imap_append_claim"
|
||||||
|
)
|
||||||
|
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||||
|
with engine.begin() as connection:
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"CREATE TABLE imap_append_attempts ("
|
||||||
|
"id VARCHAR(36) PRIMARY KEY, job_id VARCHAR(36) NOT NULL, "
|
||||||
|
"attempt_number INTEGER NOT NULL, created_at DATETIME NOT NULL)"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO imap_append_attempts "
|
||||||
|
"(id, job_id, attempt_number, created_at) VALUES "
|
||||||
|
"('a2', 'job-1', 1, '2026-01-02'), "
|
||||||
|
"('a1', 'job-1', 1, '2026-01-01'), "
|
||||||
|
"('b1', 'job-2', 7, '2026-01-01')"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with patch.object(migration.op, "get_bind", return_value=connection):
|
||||||
|
migration._renumber_attempts()
|
||||||
|
rows = connection.execute(
|
||||||
|
text(
|
||||||
|
"SELECT id, job_id, attempt_number FROM imap_append_attempts "
|
||||||
|
"ORDER BY job_id, attempt_number"
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
|
||||||
|
assert rows == [
|
||||||
|
("a1", "job-1", 1),
|
||||||
|
("a2", "job-1", 2),
|
||||||
|
("b1", "job-2", 1),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_imap_attempt_claim_repair_adds_only_the_missing_column() -> None:
|
||||||
|
migration = importlib.import_module(
|
||||||
|
"govoplan_campaign.backend.migrations.versions.e9f0a1b2c3d4_v0114_repair_imap_attempt_claim"
|
||||||
|
)
|
||||||
|
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||||
|
with engine.begin() as connection:
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"CREATE TABLE imap_append_attempts ("
|
||||||
|
"id VARCHAR(36) PRIMARY KEY, job_id VARCHAR(36) NOT NULL)"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
context = MigrationContext.configure(connection)
|
||||||
|
with patch.object(migration, "op", Operations(context)):
|
||||||
|
migration.upgrade()
|
||||||
|
migration.upgrade()
|
||||||
|
|
||||||
|
columns = {
|
||||||
|
column["name"]
|
||||||
|
for column in inspect(connection).get_columns("imap_append_attempts")
|
||||||
|
}
|
||||||
|
|
||||||
|
assert columns == {"id", "job_id", "claim_token"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("decision", ["smtp_accepted", "not_sent", "imap_appended", "imap_not_appended"])
|
||||||
|
def test_reconciliation_requires_an_evidence_note(decision: str) -> None:
|
||||||
|
with pytest.raises(ValueError, match="evidence note"):
|
||||||
|
CampaignResolveOutcomeRequest(
|
||||||
|
decision=decision, # type: ignore[arg-type]
|
||||||
|
note=" ",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("decision", "expected_status", "attempt_status"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
"imap_appended",
|
||||||
|
JobImapStatus.APPENDED.value,
|
||||||
|
"reconciled_imap_appended",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"imap_not_appended",
|
||||||
|
JobImapStatus.FAILED.value,
|
||||||
|
"reconciled_imap_not_appended",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_imap_reconciliation_preserves_attempt_and_only_not_appended_is_retryable(
|
||||||
|
decision: str,
|
||||||
|
expected_status: str,
|
||||||
|
attempt_status: str,
|
||||||
|
) -> None:
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
job = SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
send_status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
imap_status=JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
imap_claimed_at=datetime.now(timezone.utc),
|
||||||
|
imap_claim_token="claim-1",
|
||||||
|
last_error="unknown",
|
||||||
|
)
|
||||||
|
attempt = SimpleNamespace(
|
||||||
|
id="attempt-1",
|
||||||
|
status=JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
error_message="unknown",
|
||||||
|
)
|
||||||
|
session = MagicMock()
|
||||||
|
session.get.return_value = job
|
||||||
|
session.query.return_value.filter.return_value.order_by.return_value.first.return_value = attempt
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant",
|
||||||
|
return_value=campaign,
|
||||||
|
),
|
||||||
|
patch("govoplan_campaign.backend.sending.jobs.files_integration") as files,
|
||||||
|
):
|
||||||
|
result = reconcile_job_outcome(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
job_id="job-1",
|
||||||
|
decision=decision,
|
||||||
|
note="Mailbox UID evidence checked by operator 42.",
|
||||||
|
commit=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["channel"] == "imap"
|
||||||
|
assert job.imap_status == expected_status
|
||||||
|
assert job.imap_claimed_at is None
|
||||||
|
assert job.imap_claim_token is None
|
||||||
|
assert attempt.status == attempt_status
|
||||||
|
assert attempt.error_message == "Mailbox UID evidence checked by operator 42."
|
||||||
|
assert attempt in session.add.call_args_list[0].args
|
||||||
|
session.flush.assert_called_once_with()
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
if decision == "imap_appended":
|
||||||
|
files().mark_job_attachment_uses_sent.assert_called_once_with(session, job)
|
||||||
|
else:
|
||||||
|
files().mark_job_attachment_uses_sent.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"imap_status",
|
||||||
|
[JobImapStatus.APPENDING.value, JobImapStatus.FAILED.value, JobImapStatus.APPENDED.value],
|
||||||
|
)
|
||||||
|
def test_imap_reconciliation_rejects_live_retryable_or_completed_state(imap_status: str) -> None:
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
job = SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
send_status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
imap_status=imap_status,
|
||||||
|
)
|
||||||
|
session = MagicMock()
|
||||||
|
session.get.return_value = job
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant",
|
||||||
|
return_value=campaign,
|
||||||
|
):
|
||||||
|
with pytest.raises(QueueingError, match="does not require reconciliation"):
|
||||||
|
reconcile_job_outcome(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
job_id="job-1",
|
||||||
|
decision="imap_not_appended",
|
||||||
|
note="Checked mailbox.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("send_status", [JobSendStatus.CLAIMED.value, JobSendStatus.SENDING.value])
|
||||||
|
def test_smtp_reconciliation_rejects_a_live_worker_state(send_status: str) -> None:
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
job = SimpleNamespace(
|
||||||
|
id="job-1",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
send_status=send_status,
|
||||||
|
)
|
||||||
|
session = MagicMock()
|
||||||
|
session.get.return_value = job
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._get_campaign_for_tenant",
|
||||||
|
return_value=campaign,
|
||||||
|
):
|
||||||
|
with pytest.raises(QueueingError, match="does not require reconciliation"):
|
||||||
|
reconcile_job_outcome(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
job_id="job-1",
|
||||||
|
decision="not_sent",
|
||||||
|
note="Checked provider logs.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("decision", "note"),
|
||||||
|
[
|
||||||
|
("smtp_accepted", "SMTP provider evidence checked."),
|
||||||
|
("imap_appended", "Mailbox evidence checked."),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_reconciliation_rolls_back_state_when_audit_fails(
|
||||||
|
decision: str,
|
||||||
|
note: str | None,
|
||||||
|
) -> None:
|
||||||
|
payload = CampaignResolveOutcomeRequest(decision=decision, note=note) # type: ignore[arg-type]
|
||||||
|
principal = SimpleNamespace(tenant_id="tenant-1")
|
||||||
|
session = MagicMock()
|
||||||
|
|
||||||
|
def mutate_without_commit(*_args, **kwargs):
|
||||||
|
assert kwargs["commit"] is False
|
||||||
|
session.flush()
|
||||||
|
return {"decision": decision, "job_id": "job-1"}
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("govoplan_campaign.backend.routes.delivery._get_campaign_for_principal"),
|
||||||
|
patch("govoplan_campaign.backend.routes.delivery._require_permission"),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.routes.delivery.reconcile_job_outcome",
|
||||||
|
side_effect=mutate_without_commit,
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.routes.delivery.audit_from_principal",
|
||||||
|
side_effect=RuntimeError("audit unavailable"),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
with pytest.raises(RuntimeError, match="audit unavailable"):
|
||||||
|
router.resolve_campaign_job_outcome(
|
||||||
|
"campaign-1",
|
||||||
|
"job-1",
|
||||||
|
payload,
|
||||||
|
session=session,
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
session.commit.assert_not_called()
|
||||||
|
session.rollback.assert_called_once_with()
|
||||||
|
|
||||||
|
|
||||||
|
def test_retention_preserves_eml_for_unknown_imap_outcome(tmp_path: Path) -> None:
|
||||||
|
eml_path = tmp_path / "message.eml"
|
||||||
|
eml_path.write_bytes(b"message")
|
||||||
|
job = SimpleNamespace(
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
updated_at=datetime.now(timezone.utc) - timedelta(days=10),
|
||||||
|
queue_status="draft",
|
||||||
|
send_status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
imap_status=JobImapStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
eml_local_path=str(eml_path),
|
||||||
|
eml_storage_key=None,
|
||||||
|
)
|
||||||
|
query = MagicMock()
|
||||||
|
query.filter.return_value.order_by.return_value.all.return_value = [job]
|
||||||
|
session = MagicMock()
|
||||||
|
session.query.return_value = query
|
||||||
|
policy = SimpleNamespace(generated_eml_retention_days=1)
|
||||||
|
|
||||||
|
result = _apply_eml_retention(
|
||||||
|
session,
|
||||||
|
dry_run=False,
|
||||||
|
now=datetime.now(timezone.utc),
|
||||||
|
policy_for_campaign_id=lambda _campaign_id: policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["skipped_not_final"] == 1
|
||||||
|
assert eml_path.exists()
|
||||||
|
session.add.assert_not_called()
|
||||||
149
tests/test_job_list_query.py
Normal file
149
tests/test_job_list_query.py
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from govoplan_access.backend.db.models import Account, Group, User
|
||||||
|
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
|
||||||
|
from govoplan_campaign.backend.services.job_queries import (
|
||||||
|
_campaign_jobs_grid_filter_expressions,
|
||||||
|
_campaign_jobs_ordering,
|
||||||
|
_campaign_jobs_page_response,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||||
|
from govoplan_core.db.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignJobListQueryTests(unittest.TestCase):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
self.engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||||
|
Base.metadata.create_all(
|
||||||
|
self.engine,
|
||||||
|
tables=[
|
||||||
|
Account.__table__,
|
||||||
|
User.__table__,
|
||||||
|
Group.__table__,
|
||||||
|
Campaign.__table__,
|
||||||
|
CampaignVersion.__table__,
|
||||||
|
CampaignJob.__table__,
|
||||||
|
ChangeSequenceEntry.__table__,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.session = Session(self.engine)
|
||||||
|
rows = [
|
||||||
|
(0, "ordinary-0@example.test", "General notice", "ready", "draft", "not_queued", "not_requested", 0),
|
||||||
|
(1, "ordinary-1@example.test", "General notice", "ready", "draft", "not_queued", "not_requested", 1),
|
||||||
|
(2, "target-c@example.test", "Target notice C", "warning", "queued", "queued", "pending", 2),
|
||||||
|
(3, "ordinary-3@example.test", "General notice", "blocked", "draft", "failed_permanent", "failed", 3),
|
||||||
|
(4, "target-b@example.test", "Target notice B", "ready", "draft", "failed_temporary", "not_requested", 4),
|
||||||
|
(5, "target-a@example.test", "Target notice A", "ready", "draft", "outcome_unknown", "outcome_unknown", 5),
|
||||||
|
(6, "excluded@example.test", "Excluded notice", "excluded", "draft", "skipped", "skipped", 0),
|
||||||
|
]
|
||||||
|
for entry_index, recipient, subject, validation, queue, send, imap, attempts in rows:
|
||||||
|
self.session.add(CampaignJob(
|
||||||
|
id=f"job-{entry_index}",
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
campaign_version_id="version-1",
|
||||||
|
entry_index=entry_index,
|
||||||
|
entry_id=f"entry-{entry_index}",
|
||||||
|
recipient_email=recipient,
|
||||||
|
subject=subject,
|
||||||
|
message_id_header=f"<message-{entry_index}@example.test>",
|
||||||
|
eml_sha256=f"sha-{entry_index}",
|
||||||
|
build_status="built",
|
||||||
|
validation_status=validation,
|
||||||
|
queue_status=queue,
|
||||||
|
send_status=send,
|
||||||
|
imap_status=imap,
|
||||||
|
attempt_count=attempts,
|
||||||
|
resolved_attachments=[],
|
||||||
|
issues_snapshot=[],
|
||||||
|
))
|
||||||
|
self.session.commit()
|
||||||
|
|
||||||
|
def tearDown(self) -> None:
|
||||||
|
self.session.close()
|
||||||
|
self.engine.dispose()
|
||||||
|
|
||||||
|
def test_grid_filters_apply_before_pagination_and_report_filtered_totals(self) -> None:
|
||||||
|
base_filters = [CampaignJob.tenant_id == "tenant-1", CampaignJob.campaign_id == "campaign-1"]
|
||||||
|
grid_filters = {"recipient": "target"}
|
||||||
|
filtered = [*base_filters, *_campaign_jobs_grid_filter_expressions(grid_filters)]
|
||||||
|
|
||||||
|
page = _campaign_jobs_page_response(
|
||||||
|
self.session,
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id="version-1",
|
||||||
|
base_filters=base_filters,
|
||||||
|
filtered=filtered,
|
||||||
|
reviewed_keys=set(),
|
||||||
|
review_metadata={},
|
||||||
|
page=1,
|
||||||
|
page_size=2,
|
||||||
|
grid_filters=grid_filters,
|
||||||
|
sort_by="recipient",
|
||||||
|
sort_direction="asc",
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(page.total, 3)
|
||||||
|
self.assertEqual(page.total_unfiltered, 7)
|
||||||
|
self.assertEqual(page.pages, 2)
|
||||||
|
self.assertEqual(
|
||||||
|
[row["recipient_email"] for row in page.jobs],
|
||||||
|
["target-a@example.test", "target-b@example.test"],
|
||||||
|
)
|
||||||
|
self.assertIsNone(page.next_cursor)
|
||||||
|
|
||||||
|
def test_list_and_integer_filters_share_the_full_backend_query(self) -> None:
|
||||||
|
expressions = _campaign_jobs_grid_filter_expressions({
|
||||||
|
"send": 'list:["failed_temporary","outcome_unknown"]',
|
||||||
|
"attempts": "gte:4",
|
||||||
|
})
|
||||||
|
|
||||||
|
rows = (
|
||||||
|
self.session.query(CampaignJob)
|
||||||
|
.filter(*expressions)
|
||||||
|
.order_by(*_campaign_jobs_ordering("attempts", "desc"))
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual([row.id for row in rows], ["job-5", "job-4"])
|
||||||
|
|
||||||
|
def test_invalid_list_filters_fail_closed(self) -> None:
|
||||||
|
with self.assertRaises(HTTPException) as raised:
|
||||||
|
_campaign_jobs_grid_filter_expressions({"send": 'list:["not-a-status"]'})
|
||||||
|
|
||||||
|
self.assertEqual(raised.exception.status_code, 422)
|
||||||
|
|
||||||
|
def test_skipped_transport_filters_and_counts_remain_separate(self) -> None:
|
||||||
|
base_filters = [CampaignJob.tenant_id == "tenant-1", CampaignJob.campaign_id == "campaign-1"]
|
||||||
|
grid_filters = {"send": 'list:["skipped"]', "imap": 'list:["skipped"]'}
|
||||||
|
filtered = [*base_filters, *_campaign_jobs_grid_filter_expressions(grid_filters)]
|
||||||
|
|
||||||
|
page = _campaign_jobs_page_response(
|
||||||
|
self.session,
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id="version-1",
|
||||||
|
base_filters=base_filters,
|
||||||
|
filtered=filtered,
|
||||||
|
reviewed_keys=set(),
|
||||||
|
review_metadata={},
|
||||||
|
page=1,
|
||||||
|
page_size=20,
|
||||||
|
grid_filters=grid_filters,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual([row["id"] for row in page.jobs], ["job-6"])
|
||||||
|
self.assertEqual(page.counts["send"]["skipped"], 1)
|
||||||
|
self.assertEqual(page.counts["send"]["not_queued"], 2)
|
||||||
|
self.assertEqual(page.counts["imap"]["skipped"], 1)
|
||||||
|
self.assertEqual(page.filtered_counts["send"], {"skipped": 1})
|
||||||
|
self.assertEqual(page.filtered_counts["imap"], {"skipped": 1})
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
249
tests/test_mail_profile_boundary.py
Normal file
249
tests/test_mail_profile_boundary.py
Normal file
@@ -0,0 +1,249 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import call, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
from govoplan_campaign.backend import route_support
|
||||||
|
from govoplan_campaign.backend.routes import versions as router
|
||||||
|
from govoplan_campaign.backend.campaign.loader import CampaignSchemaError, validate_against_schema
|
||||||
|
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||||
|
CampaignMailProfileBoundaryError,
|
||||||
|
assert_campaign_uses_mail_profile_reference,
|
||||||
|
campaign_mail_profile_boundary_violations,
|
||||||
|
campaign_mail_profile_id,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.campaign.models import DeliveryConfig
|
||||||
|
from govoplan_campaign.backend.persistence.campaigns import CampaignPersistenceError, load_campaign_config_from_json
|
||||||
|
from govoplan_campaign.backend.persistence.versions import update_campaign_version
|
||||||
|
from govoplan_campaign.backend.integrations import MailCampaignIntegration
|
||||||
|
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, create_execution_snapshot, ensure_execution_snapshot
|
||||||
|
from govoplan_campaign.backend.schemas import CampaignVersionUpdateRequest
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_json(server: dict[str, object] | None = None) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {"id": "campaign-1", "name": "Campaign", "mode": "send"},
|
||||||
|
"server": server or {},
|
||||||
|
"recipients": {"from": [{"email": "sender@example.test"}]},
|
||||||
|
"template": {"subject": "Subject", "text": "Body", "body_mode": "text"},
|
||||||
|
"entries": {"inline": []},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_mail_contract_accepts_only_a_stable_profile_reference() -> None:
|
||||||
|
raw = _campaign_json({"mail_profile_id": " profile-1 "})
|
||||||
|
|
||||||
|
assert_campaign_uses_mail_profile_reference(raw)
|
||||||
|
|
||||||
|
assert campaign_mail_profile_id(raw) == "profile-1"
|
||||||
|
assert campaign_mail_profile_boundary_violations(raw) == ()
|
||||||
|
|
||||||
|
|
||||||
|
def test_mail_profile_documentation_is_classified_for_adaptive_views() -> None:
|
||||||
|
from govoplan_campaign.backend.manifest import get_manifest
|
||||||
|
|
||||||
|
manifest = get_manifest()
|
||||||
|
topics = {topic.id: topic for topic in manifest.documentation}
|
||||||
|
|
||||||
|
workflow = topics["campaigns.mail-profile-user-journey"]
|
||||||
|
assert workflow.metadata["kind"] == "workflow"
|
||||||
|
assert workflow.metadata["route"] == "/campaigns/{campaign_id}/mail-settings"
|
||||||
|
assert workflow.conditions[0].required_scopes == (
|
||||||
|
"campaigns:campaign:update",
|
||||||
|
"mail:profile:use",
|
||||||
|
)
|
||||||
|
assert workflow.metadata["prerequisites"]
|
||||||
|
assert workflow.metadata["steps"]
|
||||||
|
assert workflow.metadata["outcome"]
|
||||||
|
assert workflow.metadata["verification"]
|
||||||
|
assert "campaigns.mail-profile-governance" in workflow.metadata["related_topic_ids"]
|
||||||
|
|
||||||
|
assert topics["campaigns.mail-profile-governance"].metadata["kind"] == "reference"
|
||||||
|
assert topics["campaigns.mail-profile-operations"].metadata["kind"] == "reference"
|
||||||
|
assert topics["campaigns.workflow.prepare-validate-and-build"].metadata["kind"] == "workflow"
|
||||||
|
assert topics["campaigns.workflow.complete-review"].metadata["kind"] == "workflow"
|
||||||
|
assert topics["campaigns.workflow.retry-and-reconcile"].metadata["kind"] == "workflow"
|
||||||
|
assert topics["campaigns.reference.composition-assurance"].metadata["kind"] == "reference"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("legacy_key", ["smtp", "imap", "credentials", "inherit_smtp_credentials", "profile_id"])
|
||||||
|
def test_campaign_mail_contract_rejects_every_legacy_server_field(legacy_key: str) -> None:
|
||||||
|
raw = _campaign_json({"mail_profile_id": "profile-1", legacy_key: {}})
|
||||||
|
|
||||||
|
with pytest.raises(CampaignMailProfileBoundaryError, match="select authorized Mail resources"):
|
||||||
|
assert_campaign_uses_mail_profile_reference(raw)
|
||||||
|
|
||||||
|
|
||||||
|
def test_persisted_schema_rejects_inline_transport_even_without_a_secret() -> None:
|
||||||
|
with pytest.raises(CampaignSchemaError, match="Additional properties are not allowed"):
|
||||||
|
validate_against_schema(_campaign_json({"smtp": {"host": "smtp.example.test"}}))
|
||||||
|
|
||||||
|
|
||||||
|
def test_loader_rejects_inline_transport_before_optional_mail_summary() -> None:
|
||||||
|
integration = SimpleNamespace(campaign_profile_delivery_summary=lambda *_args, **_kwargs: pytest.fail("must not resolve"))
|
||||||
|
with patch("govoplan_campaign.backend.persistence.campaigns.mail_integration", return_value=integration):
|
||||||
|
with pytest.raises(CampaignMailProfileBoundaryError, match="remove campaign-local SMTP/IMAP settings"):
|
||||||
|
load_campaign_config_from_json(
|
||||||
|
object(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
raw_json=_campaign_json({"smtp": {"password": "secret"}}),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_loader_uses_only_non_secret_mail_profile_capabilities() -> None:
|
||||||
|
raw = _campaign_json({"mail_profile_id": "profile-1"})
|
||||||
|
|
||||||
|
def summary(_session, **kwargs):
|
||||||
|
assert kwargs["profile_id"] == "profile-1"
|
||||||
|
return {
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"smtp_available": True,
|
||||||
|
"imap_available": False,
|
||||||
|
"smtp_transport_revision": "opaque-smtp",
|
||||||
|
"imap_transport_revision": None,
|
||||||
|
# Even a broken/malicious provider cannot inject extra material into
|
||||||
|
# Campaign's strict in-memory ServerConfig.
|
||||||
|
"host": "smtp.example.test",
|
||||||
|
"password": "secret",
|
||||||
|
}
|
||||||
|
|
||||||
|
integration = SimpleNamespace(campaign_profile_delivery_summary=summary)
|
||||||
|
with patch("govoplan_campaign.backend.persistence.campaigns.mail_integration", return_value=integration):
|
||||||
|
config = load_campaign_config_from_json(
|
||||||
|
object(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
raw_json=raw,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert raw["server"] == {"mail_profile_id": "profile-1"}
|
||||||
|
assert config.server.mail_profile_id == "profile-1"
|
||||||
|
assert config.server.profile_capabilities.smtp_available is True
|
||||||
|
assert config.server.profile_capabilities.imap_available is False
|
||||||
|
assert "smtp.example.test" not in repr(config.server)
|
||||||
|
assert "secret" not in repr(config.server)
|
||||||
|
|
||||||
|
|
||||||
|
def test_new_execution_snapshot_stores_reference_and_evidence_not_transport_material() -> None:
|
||||||
|
raw = _campaign_json({"mail_profile_id": "profile-1"})
|
||||||
|
version = SimpleNamespace(id="version-1", raw_json=raw)
|
||||||
|
|
||||||
|
payload, _digest = create_execution_snapshot(
|
||||||
|
version, # type: ignore[arg-type]
|
||||||
|
mail_profile_id="profile-1",
|
||||||
|
smtp_transport_revision="opaque-smtp-evidence",
|
||||||
|
imap_transport_revision="opaque-imap-evidence",
|
||||||
|
delivery=DeliveryConfig(),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert payload["snapshot_version"] == "7"
|
||||||
|
assert payload["mail_profile_id"] == "profile-1"
|
||||||
|
assert "smtp" not in payload
|
||||||
|
assert "imap" not in payload
|
||||||
|
assert payload["smtp_transport_revision"] == "opaque-smtp-evidence"
|
||||||
|
assert payload["imap_transport_revision"] == "opaque-imap-evidence"
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_execution_snapshot_is_preserved_but_fails_closed() -> None:
|
||||||
|
version = SimpleNamespace(
|
||||||
|
raw_json=_campaign_json({"mail_profile_id": "profile-1"}),
|
||||||
|
execution_snapshot={"snapshot_version": "3", "smtp": {"host": "legacy.example.test"}},
|
||||||
|
execution_snapshot_hash=None,
|
||||||
|
)
|
||||||
|
with patch(
|
||||||
|
"govoplan_campaign.backend.sending.execution.files_integration",
|
||||||
|
return_value=SimpleNamespace(available=False),
|
||||||
|
):
|
||||||
|
with pytest.raises(ExecutionSnapshotError, match="preserved for audit only"):
|
||||||
|
ensure_execution_snapshot(object(), version) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
assert version.execution_snapshot["smtp"]["host"] == "legacy.example.test"
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_mail_adapter_does_not_expose_raw_transport_helpers() -> None:
|
||||||
|
integration = MailCampaignIntegration(SimpleNamespace())
|
||||||
|
|
||||||
|
for name in (
|
||||||
|
"smtp_config_from_profile",
|
||||||
|
"imap_config_from_profile",
|
||||||
|
"send_email_bytes",
|
||||||
|
"send_email_message",
|
||||||
|
"materialize_campaign_mail_profile_config",
|
||||||
|
):
|
||||||
|
assert not hasattr(integration, name)
|
||||||
|
|
||||||
|
|
||||||
|
def test_editing_a_legacy_record_requires_an_explicit_profile_migration() -> None:
|
||||||
|
legacy_raw = _campaign_json({"smtp": {"host": "smtp.example.test", "password": "secret"}})
|
||||||
|
version = SimpleNamespace(id="version-1", campaign_id="campaign-1", raw_json=legacy_raw)
|
||||||
|
campaign = SimpleNamespace(id="campaign-1", current_version_id="version-1")
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("govoplan_campaign.backend.persistence.versions.get_campaign_version_for_tenant", return_value=version),
|
||||||
|
patch("govoplan_campaign.backend.persistence.versions._require_campaign", return_value=campaign),
|
||||||
|
patch("govoplan_campaign.backend.persistence.versions.ensure_current_working_version"),
|
||||||
|
patch("govoplan_campaign.backend.persistence.versions.is_version_locked", return_value=False),
|
||||||
|
):
|
||||||
|
with pytest.raises(CampaignPersistenceError, match="explicitly save the migration"):
|
||||||
|
update_campaign_version(
|
||||||
|
object(), # type: ignore[arg-type]
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
version_id="version-1",
|
||||||
|
raw_json=_campaign_json({"mail_profile_id": "profile-1"}),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert version.raw_json is legacy_raw
|
||||||
|
assert legacy_raw["server"]["smtp"]["password"] == "secret" # type: ignore[index]
|
||||||
|
|
||||||
|
|
||||||
|
def test_fork_inherited_profile_requires_mail_profile_use_scope() -> None:
|
||||||
|
principal = SimpleNamespace(
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
user=SimpleNamespace(id="user-1"),
|
||||||
|
)
|
||||||
|
campaign = SimpleNamespace(id="campaign-1")
|
||||||
|
source = SimpleNamespace(
|
||||||
|
id="version-1",
|
||||||
|
campaign_id="campaign-1",
|
||||||
|
raw_json={"server": {"mail_profile_id": "profile-1"}},
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||||
|
patch.object(router, "_require_permission"),
|
||||||
|
patch.object(router, "_get_version_for_tenant", return_value=source),
|
||||||
|
patch.object(route_support, "has_scope", return_value=False),
|
||||||
|
patch.object(router, "fork_campaign_version_for_edit") as fork,
|
||||||
|
):
|
||||||
|
with pytest.raises(HTTPException) as captured:
|
||||||
|
router.fork_version_for_edit(
|
||||||
|
"campaign-1",
|
||||||
|
"version-1",
|
||||||
|
CampaignVersionUpdateRequest(),
|
||||||
|
session=object(), # type: ignore[arg-type]
|
||||||
|
principal=principal, # type: ignore[arg-type]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert captured.value.status_code == 403
|
||||||
|
fork.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_wide_effect_authorizes_every_affected_version() -> None:
|
||||||
|
session = object()
|
||||||
|
principal = SimpleNamespace(tenant_id="tenant-1")
|
||||||
|
|
||||||
|
with patch.object(route_support, "_require_campaign_profile_use_if_needed") as require_profile:
|
||||||
|
route_support._require_campaign_versions_profile_use( # noqa: SLF001 - security boundary regression test
|
||||||
|
session, # type: ignore[arg-type]
|
||||||
|
principal, # type: ignore[arg-type]
|
||||||
|
"campaign-1",
|
||||||
|
{"version-2", "version-1"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert require_profile.call_args_list == [
|
||||||
|
call(session, principal, "campaign-1", "version-1"),
|
||||||
|
call(session, principal, "campaign-1", "version-2"),
|
||||||
|
]
|
||||||
480
tests/test_mail_testbed_acceptance.py
Normal file
480
tests/test_mail_testbed_acceptance.py
Normal file
@@ -0,0 +1,480 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import smtplib
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign import load_campaign_config
|
||||||
|
|
||||||
|
|
||||||
|
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
RUNNER_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "run_campaign_acceptance.py"
|
||||||
|
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_runner():
|
||||||
|
spec = importlib.util.spec_from_file_location("govoplan_campaign_greenmail_acceptance", RUNNER_PATH)
|
||||||
|
assert spec is not None and spec.loader is not None
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
sys.modules[spec.name] = module
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
runner = _load_runner()
|
||||||
|
|
||||||
|
|
||||||
|
class _Response:
|
||||||
|
def __init__(self, status_code: int, payload: dict[str, Any]) -> None:
|
||||||
|
self.status_code = status_code
|
||||||
|
self._payload = payload
|
||||||
|
|
||||||
|
def json(self) -> dict[str, Any]:
|
||||||
|
return self._payload
|
||||||
|
|
||||||
|
|
||||||
|
class _AcceptanceClient:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.campaign_json: dict[str, Any] | None = None
|
||||||
|
self.send_calls = 0
|
||||||
|
|
||||||
|
def post(self, path: str, **kwargs: Any) -> _Response:
|
||||||
|
if path == "/api/v1/campaigns":
|
||||||
|
self.campaign_json = kwargs["json"]["config"]
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"campaign": {"id": "campaign-internal"},
|
||||||
|
"version": {"id": "version-internal"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if path.endswith("/validate"):
|
||||||
|
return _Response(200, {"ok": True, "error_count": 0, "warning_count": 0})
|
||||||
|
if path.endswith("/build"):
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"built_count": 1,
|
||||||
|
"build_failed_count": 0,
|
||||||
|
"queueable_count": 1,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if path.endswith("/send-now"):
|
||||||
|
self.send_calls += 1
|
||||||
|
if self.send_calls == 2:
|
||||||
|
return _Response(422, {"detail": "Already accepted"})
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"result": {
|
||||||
|
"attempted_count": 1,
|
||||||
|
"sent_count": 1,
|
||||||
|
"failed_count": 0,
|
||||||
|
"outcome_unknown_count": 0,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"delivery_mode": "synchronous",
|
||||||
|
"results": [{"job_id": "not-retained", "status": "smtp_accepted"}],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if path.endswith("/append-sent"):
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"result": {
|
||||||
|
"pending_count": 1,
|
||||||
|
"processed_count": 1,
|
||||||
|
"appended_count": 1,
|
||||||
|
"failed_count": 0,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"results": [{"job_id": "not-retained", "status": "appended"}],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
raise AssertionError(f"unexpected POST {path}")
|
||||||
|
|
||||||
|
def get(self, path: str, **kwargs: Any) -> _Response:
|
||||||
|
if path.endswith("/report"):
|
||||||
|
return _Response(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"cards": {
|
||||||
|
"jobs_total": 1,
|
||||||
|
"sent": 1,
|
||||||
|
"smtp_accepted": 1,
|
||||||
|
"failed": 0,
|
||||||
|
"outcome_unknown": 0,
|
||||||
|
"retryable": 0,
|
||||||
|
"needs_attention": 0,
|
||||||
|
"imap_appended": 1,
|
||||||
|
"imap_failed": 0,
|
||||||
|
},
|
||||||
|
"status_counts": {
|
||||||
|
"send": {"smtp_accepted": 1},
|
||||||
|
"imap": {"appended": 1},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
raise AssertionError(f"unexpected GET {path}")
|
||||||
|
|
||||||
|
|
||||||
|
def _settings():
|
||||||
|
return runner.TestbedSettings(
|
||||||
|
smtp_host="127.0.0.1",
|
||||||
|
smtp_port=3025,
|
||||||
|
imap_host="127.0.0.1",
|
||||||
|
imap_port=3143,
|
||||||
|
username="campaign-test@govoplan.test",
|
||||||
|
password="local-test-password",
|
||||||
|
sender="campaign-test@govoplan.test",
|
||||||
|
recipient="campaign-test@govoplan.test",
|
||||||
|
sent_folder="Sent",
|
||||||
|
provider_timeout_seconds=5,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("host", ["localhost", "mail.test", "192.168.1.20", "8.8.8.8"])
|
||||||
|
def test_testbed_rejects_hostnames_and_non_loopback_addresses(host: str) -> None:
|
||||||
|
settings = _settings()
|
||||||
|
rejected = runner.TestbedSettings(
|
||||||
|
smtp_host=host,
|
||||||
|
smtp_port=settings.smtp_port,
|
||||||
|
imap_host=settings.imap_host,
|
||||||
|
imap_port=settings.imap_port,
|
||||||
|
username=settings.username,
|
||||||
|
password=settings.password,
|
||||||
|
sender=settings.sender,
|
||||||
|
recipient=settings.recipient,
|
||||||
|
sent_folder=settings.sent_folder,
|
||||||
|
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="literal loopback|restricted to the loopback"):
|
||||||
|
rejected.assert_local_testbed()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("host", ["127.0.0.1", "127.8.9.10", "::1"])
|
||||||
|
def test_testbed_accepts_literal_loopback_and_preserves_it_in_profile(host: str) -> None:
|
||||||
|
settings = _settings()
|
||||||
|
accepted = runner.TestbedSettings(
|
||||||
|
smtp_host=host,
|
||||||
|
smtp_port=settings.smtp_port,
|
||||||
|
imap_host=host,
|
||||||
|
imap_port=settings.imap_port,
|
||||||
|
username=settings.username,
|
||||||
|
password=settings.password,
|
||||||
|
sender=settings.sender,
|
||||||
|
recipient=settings.recipient,
|
||||||
|
sent_folder=settings.sent_folder,
|
||||||
|
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
|
accepted.assert_local_testbed()
|
||||||
|
profile = runner._profile_payload(accepted, name="Literal loopback")
|
||||||
|
assert profile["smtp"]["host"] == host
|
||||||
|
assert profile["imap"]["host"] == host
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_acceptance_orchestration_retains_only_profile_reference_and_safe_evidence() -> None:
|
||||||
|
client = _AcceptanceClient()
|
||||||
|
|
||||||
|
def snapshot_probe(_version_id: str):
|
||||||
|
assert client.campaign_json is not None
|
||||||
|
return client.campaign_json, {
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"smtp_transport_revision": "opaque-smtp-revision",
|
||||||
|
"imap_transport_revision": "opaque-imap-revision",
|
||||||
|
"delivery": {"imap_append_sent": {"enabled": True, "folder": "Sent"}},
|
||||||
|
}
|
||||||
|
|
||||||
|
audit = {
|
||||||
|
"campaign.created": 1,
|
||||||
|
"campaign.validated": 1,
|
||||||
|
"campaign.messages_built": 1,
|
||||||
|
"campaign.sent_now": 1,
|
||||||
|
"campaign.send_now_rejected": 1,
|
||||||
|
"campaign.append_sent_enqueued": 1,
|
||||||
|
}
|
||||||
|
evidence, subject = runner.execute_campaign_scenario(
|
||||||
|
client,
|
||||||
|
{"Authorization": "not-retained"},
|
||||||
|
fixture_path=FIXTURE_PATH,
|
||||||
|
profile_id="profile-1",
|
||||||
|
settings=_settings(),
|
||||||
|
scenario="success",
|
||||||
|
snapshot_probe=snapshot_probe,
|
||||||
|
audit_probe=lambda _campaign_id, _version_id: audit,
|
||||||
|
append_sent=True,
|
||||||
|
repeat_send=True,
|
||||||
|
)
|
||||||
|
evidence["provider_verification"] = {
|
||||||
|
"inbox_increment": 1,
|
||||||
|
"sent_increment": 1,
|
||||||
|
"unique_subject_matches_in_inbox": 1,
|
||||||
|
"unique_subject_matches_in_sent": 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
runner._assert_success_evidence(evidence)
|
||||||
|
runner._assert_evidence_safe(
|
||||||
|
{
|
||||||
|
"schema_version": runner.EVIDENCE_SCHEMA,
|
||||||
|
"coverage": {
|
||||||
|
"smtp_acceptance": True,
|
||||||
|
"partial_envelope_refusal": False,
|
||||||
|
"post_data_connection_loss_outcome_unknown": False,
|
||||||
|
"source_artifact_provenance": False,
|
||||||
|
"worker_restart_interruption": False,
|
||||||
|
},
|
||||||
|
"success": evidence,
|
||||||
|
},
|
||||||
|
settings=_settings(),
|
||||||
|
)
|
||||||
|
assert subject.startswith("[GovOPlaN acceptance ")
|
||||||
|
assert client.send_calls == 2
|
||||||
|
assert client.campaign_json is not None
|
||||||
|
assert client.campaign_json["server"] == {"mail_profile_id": "profile-1"}
|
||||||
|
assert "credentials" not in json.dumps(client.campaign_json).casefold()
|
||||||
|
serialized = json.dumps(evidence, sort_keys=True)
|
||||||
|
assert "not-retained" not in serialized
|
||||||
|
assert "local-test-password" not in serialized
|
||||||
|
|
||||||
|
|
||||||
|
def test_campaign_boundary_rejects_resolved_transport_material() -> None:
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="forbidden transport material"):
|
||||||
|
runner.assert_campaign_boundary(
|
||||||
|
{"server": {"mail_profile_id": "profile-1"}},
|
||||||
|
{
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"smtp_transport_revision": "smtp-revision",
|
||||||
|
"imap_transport_revision": "imap-revision",
|
||||||
|
"smtp": {"host": "should-not-be-here"},
|
||||||
|
},
|
||||||
|
profile_id="profile-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_success_projection_fails_closed_on_inconsistent_campaign_report() -> None:
|
||||||
|
evidence = {
|
||||||
|
"send": {
|
||||||
|
"attempted_count": 1,
|
||||||
|
"sent_count": 1,
|
||||||
|
"failed_count": 0,
|
||||||
|
"outcome_unknown_count": 0,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"delivery_mode": "synchronous",
|
||||||
|
"statuses": {"smtp_accepted": 1},
|
||||||
|
},
|
||||||
|
"append_sent": {
|
||||||
|
"pending_count": 1,
|
||||||
|
"processed_count": 1,
|
||||||
|
"appended_count": 1,
|
||||||
|
"failed_count": 0,
|
||||||
|
"skipped_count": 0,
|
||||||
|
"statuses": {"appended": 1},
|
||||||
|
},
|
||||||
|
"report": {
|
||||||
|
"cards": {
|
||||||
|
"jobs_total": 1,
|
||||||
|
"sent": 0,
|
||||||
|
"smtp_accepted": 0,
|
||||||
|
"failed": 0,
|
||||||
|
"outcome_unknown": 0,
|
||||||
|
"needs_attention": 0,
|
||||||
|
"imap_appended": 0,
|
||||||
|
"imap_failed": 0,
|
||||||
|
},
|
||||||
|
"send_status_counts": {},
|
||||||
|
"imap_status_counts": {},
|
||||||
|
},
|
||||||
|
"provider_verification": {
|
||||||
|
"inbox_increment": 1,
|
||||||
|
"sent_increment": 1,
|
||||||
|
"unique_subject_matches_in_inbox": 1,
|
||||||
|
"unique_subject_matches_in_sent": 1,
|
||||||
|
},
|
||||||
|
"campaign_mail_boundary": {
|
||||||
|
"profile_reference_only": True,
|
||||||
|
"smtp_revision_frozen": True,
|
||||||
|
"imap_revision_frozen": True,
|
||||||
|
"resolved_transport_material_present": False,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="report does not agree"):
|
||||||
|
runner._assert_success_evidence(evidence)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evidence_projection_rejects_unknown_status_keys() -> None:
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="unsupported status"):
|
||||||
|
runner._send_evidence(
|
||||||
|
{
|
||||||
|
"delivery_mode": "synchronous",
|
||||||
|
"results": [{"status": "provider diagnostic: recipient@example.test"}],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="durable attempt status"):
|
||||||
|
runner._durable_state_evidence(
|
||||||
|
{
|
||||||
|
"job_count": 1,
|
||||||
|
"send_status_counts": {"sending": 1},
|
||||||
|
"attempt_status_counts": {"provider-secret": 1},
|
||||||
|
"unfinished_attempt_count": 1,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="synchronous delivery mode"):
|
||||||
|
runner._send_evidence(
|
||||||
|
{
|
||||||
|
"delivery_mode": "provider diagnostic: recipient@example.test",
|
||||||
|
"results": [],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="unsupported"):
|
||||||
|
runner._report_evidence(
|
||||||
|
{
|
||||||
|
"cards": {},
|
||||||
|
"status_counts": {
|
||||||
|
"send": {"smtp_accepted": 1, "provider-secret": 1},
|
||||||
|
"imap": {},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _open_fault_smtp(endpoint):
|
||||||
|
client = smtplib.SMTP(endpoint.host, endpoint.port, timeout=5)
|
||||||
|
client.ehlo()
|
||||||
|
client.login("acceptance-user", "acceptance-password")
|
||||||
|
return client
|
||||||
|
|
||||||
|
|
||||||
|
def test_explicit_temporary_smtp_response_occurs_after_complete_data() -> None:
|
||||||
|
with runner.smtp_fault_endpoint("temporary_data_response") as endpoint:
|
||||||
|
client = _open_fault_smtp(endpoint)
|
||||||
|
try:
|
||||||
|
with pytest.raises(smtplib.SMTPDataError) as captured:
|
||||||
|
client.sendmail(
|
||||||
|
"sender@example.test",
|
||||||
|
["recipient@example.test"],
|
||||||
|
b"Subject: temporary\r\n\r\nmessage",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
assert captured.value.smtp_code == 451
|
||||||
|
assert endpoint.evidence() == {
|
||||||
|
"connection_count": 1,
|
||||||
|
"accepted_rcpt_commands": 1,
|
||||||
|
"refused_rcpt_commands": 0,
|
||||||
|
"data_transactions": 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_partial_recipient_refusal_retains_one_accepted_envelope() -> None:
|
||||||
|
with runner.smtp_fault_endpoint("partial_recipient_refusal") as endpoint:
|
||||||
|
client = _open_fault_smtp(endpoint)
|
||||||
|
try:
|
||||||
|
refused = client.sendmail(
|
||||||
|
"sender@example.test",
|
||||||
|
["accepted@example.test", "refused@example.test"],
|
||||||
|
b"Subject: partial\r\n\r\nmessage",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
client.quit()
|
||||||
|
assert set(refused) == {"refused@example.test"}
|
||||||
|
assert endpoint.evidence() == {
|
||||||
|
"connection_count": 1,
|
||||||
|
"accepted_rcpt_commands": 1,
|
||||||
|
"refused_rcpt_commands": 1,
|
||||||
|
"data_transactions": 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_post_data_disconnect_is_a_real_ambiguous_protocol_boundary() -> None:
|
||||||
|
with runner.smtp_fault_endpoint("post_data_disconnect") as endpoint:
|
||||||
|
client = _open_fault_smtp(endpoint)
|
||||||
|
try:
|
||||||
|
with pytest.raises(smtplib.SMTPServerDisconnected):
|
||||||
|
client.sendmail(
|
||||||
|
"sender@example.test",
|
||||||
|
["recipient@example.test"],
|
||||||
|
b"Subject: ambiguous\r\n\r\nmessage",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
assert endpoint.wait_for_data(1)
|
||||||
|
assert endpoint.evidence()["data_transactions"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_partial_refusal_fixture_adds_a_second_distinct_recipient() -> None:
|
||||||
|
raw, _subject = runner.materialize_campaign_fixture(
|
||||||
|
FIXTURE_PATH,
|
||||||
|
profile_id="profile-1",
|
||||||
|
settings=_settings(),
|
||||||
|
scenario="partial_envelope_refusal",
|
||||||
|
run_token="0123456789ab",
|
||||||
|
additional_envelope_recipient=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
recipients = raw["entries"]["inline"][0]["to"]
|
||||||
|
assert len(recipients) == 2
|
||||||
|
assert recipients[0]["email"] != recipients[1]["email"]
|
||||||
|
assert recipients[1]["email"].endswith("@govoplan.test")
|
||||||
|
|
||||||
|
def test_fixture_contains_no_transport_credentials() -> None:
|
||||||
|
raw = json.loads(FIXTURE_PATH.read_text(encoding="utf-8"))
|
||||||
|
runner._assert_no_forbidden_campaign_keys(raw)
|
||||||
|
assert raw["server"] == {"mail_profile_id": "00000000-0000-4000-8000-000000000001"}
|
||||||
|
config = load_campaign_config(FIXTURE_PATH)
|
||||||
|
assert config.server.mail_profile_id == "00000000-0000-4000-8000-000000000001"
|
||||||
|
|
||||||
|
|
||||||
|
def test_fixture_composition_versions_are_complete_and_exact() -> None:
|
||||||
|
versions = {
|
||||||
|
"core": "0.1.13",
|
||||||
|
"access": "0.1.11",
|
||||||
|
"audit": "0.1.8",
|
||||||
|
"campaigns": "0.1.11",
|
||||||
|
"mail": "0.1.10",
|
||||||
|
"files": "0.1.9",
|
||||||
|
}
|
||||||
|
|
||||||
|
assert runner.required_composition_versions(FIXTURE_PATH, versions) == {
|
||||||
|
"core": "0.1.13",
|
||||||
|
"access": "0.1.11",
|
||||||
|
"audit": "0.1.8",
|
||||||
|
"campaigns": "0.1.11",
|
||||||
|
"mail": "0.1.10",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_fixture_composition_fails_closed_when_a_required_version_is_missing() -> None:
|
||||||
|
with pytest.raises(runner.AcceptanceError, match="versions are unavailable"):
|
||||||
|
runner.required_composition_versions(
|
||||||
|
FIXTURE_PATH,
|
||||||
|
{
|
||||||
|
"core": "0.1.13",
|
||||||
|
"access": "0.1.11",
|
||||||
|
"campaigns": "0.1.11",
|
||||||
|
"mail": "0.1.10",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_testbed_documentation_distinguishes_proven_and_open_failure_drills() -> None:
|
||||||
|
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(encoding="utf-8")
|
||||||
|
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
assert "second ordinary send must be rejected before another provider effect" in testbed
|
||||||
|
assert "connection loss after complete DATA is frozen" in testbed
|
||||||
|
assert "dedicated OS process" in testbed
|
||||||
|
assert "Redis/Celery delivery" in runbook
|
||||||
|
assert "broker redelivery" in runbook
|
||||||
|
assert "celery_broker_redelivery" in testbed
|
||||||
|
assert "raw provider diagnostics" in runbook
|
||||||
64
tests/test_manifest_navigation.py
Normal file
64
tests/test_manifest_navigation.py
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.manifest import (
|
||||||
|
CAMPAIGN_MODULE_REQUIRED_ANY,
|
||||||
|
OPERATOR_QUEUE_REQUIRED_ANY,
|
||||||
|
OPERATOR_QUEUE_SURFACE_ID,
|
||||||
|
REPORTS_REQUIRED_ANY,
|
||||||
|
REPORTS_SURFACE_ID,
|
||||||
|
get_manifest,
|
||||||
|
)
|
||||||
|
from govoplan_core.core.registry import manifest_view_surfaces
|
||||||
|
|
||||||
|
|
||||||
|
def test_operator_queue_is_an_integrated_campaign_view() -> None:
|
||||||
|
manifest = get_manifest()
|
||||||
|
assert manifest.frontend is not None
|
||||||
|
|
||||||
|
assert "/operator" not in {item.path for item in manifest.nav_items}
|
||||||
|
assert "/operator" not in {item.path for item in manifest.frontend.nav_items}
|
||||||
|
|
||||||
|
routes = {route.path: route for route in manifest.frontend.routes}
|
||||||
|
assert "/operator" not in routes
|
||||||
|
queue = routes["/campaigns/queue"]
|
||||||
|
assert queue.component == "OperatorQueuePage"
|
||||||
|
assert queue.required_all == ("campaigns:campaign:read",)
|
||||||
|
assert queue.required_any == OPERATOR_QUEUE_REQUIRED_ANY
|
||||||
|
assert OPERATOR_QUEUE_SURFACE_ID == "campaigns.route.operator"
|
||||||
|
assert queue.surface_id == OPERATOR_QUEUE_SURFACE_ID
|
||||||
|
|
||||||
|
queue_surfaces = [
|
||||||
|
surface
|
||||||
|
for surface in manifest_view_surfaces(manifest)
|
||||||
|
if surface.id == OPERATOR_QUEUE_SURFACE_ID
|
||||||
|
]
|
||||||
|
assert len(queue_surfaces) == 1
|
||||||
|
assert queue_surfaces[0].description == "/campaigns/queue"
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_reports_are_an_integrated_campaign_view() -> None:
|
||||||
|
manifest = get_manifest()
|
||||||
|
assert manifest.frontend is not None
|
||||||
|
|
||||||
|
backend_nav = {item.path: item for item in manifest.nav_items}
|
||||||
|
frontend_nav = {item.path: item for item in manifest.frontend.nav_items}
|
||||||
|
assert "/reports" not in backend_nav
|
||||||
|
assert "/reports" not in frontend_nav
|
||||||
|
assert backend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
|
||||||
|
assert frontend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
|
||||||
|
|
||||||
|
routes = {route.path: route for route in manifest.frontend.routes}
|
||||||
|
assert "/reports" not in routes
|
||||||
|
report = routes["/campaigns/reports"]
|
||||||
|
assert report.component == "AggregateReportsPage"
|
||||||
|
assert report.required_any == REPORTS_REQUIRED_ANY
|
||||||
|
assert REPORTS_SURFACE_ID == "campaigns.route.reports"
|
||||||
|
assert report.surface_id == REPORTS_SURFACE_ID
|
||||||
|
|
||||||
|
report_surfaces = [
|
||||||
|
surface
|
||||||
|
for surface in manifest_view_surfaces(manifest)
|
||||||
|
if surface.id == REPORTS_SURFACE_ID
|
||||||
|
]
|
||||||
|
assert len(report_surfaces) == 1
|
||||||
|
assert report_surfaces[0].description == "/campaigns/reports"
|
||||||
66
tests/test_mock_campaign_helpers.py
Normal file
66
tests/test_mock_campaign_helpers.py
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.dev.mock_campaign import (
|
||||||
|
_MockSendBatch,
|
||||||
|
_mock_send_steps,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MockCampaignStepTests(unittest.TestCase):
|
||||||
|
def test_preview_marks_delivery_steps_as_skipped(self):
|
||||||
|
validation = SimpleNamespace(ok=True)
|
||||||
|
build = SimpleNamespace(
|
||||||
|
built_count=2,
|
||||||
|
queueable_count=2,
|
||||||
|
needs_review_count=0,
|
||||||
|
blocked_count=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
steps = _mock_send_steps(
|
||||||
|
validation_report=validation,
|
||||||
|
validation_payload={"ok": True},
|
||||||
|
build_report=build,
|
||||||
|
send_batch=_MockSendBatch(results=[]),
|
||||||
|
send=False,
|
||||||
|
append_sent=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
[step["status"] for step in steps],
|
||||||
|
["ok", "ok", "skipped", "skipped"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_delivery_failures_require_review(self):
|
||||||
|
validation = SimpleNamespace(ok=False)
|
||||||
|
build = SimpleNamespace(
|
||||||
|
built_count=1,
|
||||||
|
queueable_count=0,
|
||||||
|
needs_review_count=1,
|
||||||
|
blocked_count=0,
|
||||||
|
)
|
||||||
|
batch = _MockSendBatch(
|
||||||
|
results=[],
|
||||||
|
failed_count=1,
|
||||||
|
imap_failed_count=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
steps = _mock_send_steps(
|
||||||
|
validation_report=validation,
|
||||||
|
validation_payload={"ok": False},
|
||||||
|
build_report=build,
|
||||||
|
send_batch=batch,
|
||||||
|
send=True,
|
||||||
|
append_sent=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
[step["status"] for step in steps],
|
||||||
|
["needs_review", "needs_review", "needs_review", "needs_review"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -5,7 +5,7 @@ import unittest
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
from govoplan_campaign.backend import router
|
from govoplan_campaign.backend.routes import campaigns as router
|
||||||
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
||||||
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
||||||
from govoplan_campaign.backend.persistence.versions import validate_campaign_partial
|
from govoplan_campaign.backend.persistence.versions import validate_campaign_partial
|
||||||
@@ -80,6 +80,30 @@ class CampaignPartialValidationTests(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class CampaignSemanticValidationTests(unittest.TestCase):
|
class CampaignSemanticValidationTests(unittest.TestCase):
|
||||||
|
def test_send_mode_requires_campaign_owned_sender_for_each_inline_entry(self) -> None:
|
||||||
|
config = CampaignConfig.model_validate({
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {"id": "campaign-1", "name": "Campaign", "mode": "send"},
|
||||||
|
"server": {
|
||||||
|
"mail_profile_id": "profile-1",
|
||||||
|
"profile_capabilities": {"smtp_available": True},
|
||||||
|
},
|
||||||
|
"recipients": {"allow_individual_from": True},
|
||||||
|
"template": {"subject": "Subject", "text": "Body", "body_mode": "text"},
|
||||||
|
"entries": {
|
||||||
|
"inline": [
|
||||||
|
{"id": "ready", "from": [{"email": "sender@example.local"}]},
|
||||||
|
{"id": "missing"},
|
||||||
|
]
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
report = validate_campaign_config(config)
|
||||||
|
|
||||||
|
missing_sender = [issue for issue in report.issues if issue.code == "missing_sender"]
|
||||||
|
self.assertEqual(1, len(missing_sender))
|
||||||
|
self.assertEqual("/entries/inline/1/from", missing_sender[0].path)
|
||||||
|
|
||||||
def test_semantic_validation_reports_zip_delivery_and_external_mapping_issues(self) -> None:
|
def test_semantic_validation_reports_zip_delivery_and_external_mapping_issues(self) -> None:
|
||||||
with tempfile.TemporaryDirectory() as tmp:
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
root = Path(tmp)
|
root = Path(tmp)
|
||||||
@@ -139,8 +163,7 @@ class CampaignSemanticValidationTests(unittest.TestCase):
|
|||||||
self.assertIn("unknown_global_value", codes)
|
self.assertIn("unknown_global_value", codes)
|
||||||
self.assertIn("zip_global_password_value_missing", codes)
|
self.assertIn("zip_global_password_value_missing", codes)
|
||||||
self.assertIn("zip_archive_unknown", codes)
|
self.assertIn("zip_archive_unknown", codes)
|
||||||
self.assertIn("delivery_imap_enabled_without_server_imap", codes)
|
self.assertIn("missing_mail_profile", codes)
|
||||||
self.assertIn("missing_smtp_config", codes)
|
|
||||||
self.assertIn("unknown_mapping_target", codes)
|
self.assertIn("unknown_mapping_target", codes)
|
||||||
self.assertIn("mapping_target_not_overridable", codes)
|
self.assertIn("mapping_target_not_overridable", codes)
|
||||||
self.assertIn("mapping_columns_missing", codes)
|
self.assertIn("mapping_columns_missing", codes)
|
||||||
|
|||||||
400
tests/test_postbox_delivery_orchestration.py
Normal file
400
tests/test_postbox_delivery_orchestration.py
Normal file
@@ -0,0 +1,400 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy
|
||||||
|
from govoplan_campaign.backend.db.models import JobSendStatus
|
||||||
|
from govoplan_campaign.backend.sending.jobs import (
|
||||||
|
SendJobResult,
|
||||||
|
_MailChannelOutcome,
|
||||||
|
_final_multichannel_status,
|
||||||
|
_send_claimed_multichannel_job,
|
||||||
|
)
|
||||||
|
from govoplan_campaign.backend.sending.postbox_delivery import (
|
||||||
|
PostboxChannelOutcome,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _context():
|
||||||
|
return SimpleNamespace(
|
||||||
|
message_bytes=b"message",
|
||||||
|
snapshot=SimpleNamespace(
|
||||||
|
delivery=SimpleNamespace(
|
||||||
|
postbox=SimpleNamespace(classification="internal")
|
||||||
|
)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _job():
|
||||||
|
return SimpleNamespace(id="job-1")
|
||||||
|
|
||||||
|
|
||||||
|
class _Session:
|
||||||
|
def __init__(self, job) -> None:
|
||||||
|
self.job = job
|
||||||
|
|
||||||
|
def get(self, _model, _id):
|
||||||
|
return self.job
|
||||||
|
|
||||||
|
def add(self, _value) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def commit(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class PostboxFallbackOrchestrationTests(unittest.TestCase):
|
||||||
|
def test_mail_unknown_never_starts_postbox_fallback(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
expected = SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
attempt_number=1,
|
||||||
|
)
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||||
|
return_value=_MailChannelOutcome(outcome_unknown=True),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||||
|
return_value=PostboxChannelOutcome(),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
|
||||||
|
) as deliver_postbox,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=expected,
|
||||||
|
),
|
||||||
|
):
|
||||||
|
result = _send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertIs(result, expected)
|
||||||
|
deliver_postbox.assert_not_called()
|
||||||
|
|
||||||
|
def test_mail_preacceptance_rejection_starts_postbox_fallback(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
postbox_outcome = PostboxChannelOutcome(accepted=1)
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||||
|
return_value=_MailChannelOutcome(rejected_permanent=True),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||||
|
return_value=PostboxChannelOutcome(),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||||
|
return_value=postbox_outcome,
|
||||||
|
) as deliver_postbox,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||||
|
attempt_number=1,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
deliver_postbox.assert_called_once()
|
||||||
|
|
||||||
|
def test_accepted_postbox_fallback_prevents_later_mail_retry(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||||
|
return_value=PostboxChannelOutcome(
|
||||||
|
accepted=1,
|
||||||
|
rejected_temporary=1,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||||
|
) as deliver_mail,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||||
|
return_value=PostboxChannelOutcome(accepted=2),
|
||||||
|
) as deliver_postbox,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||||
|
attempt_number=3,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
deliver_mail.assert_not_called()
|
||||||
|
deliver_postbox.assert_called_once()
|
||||||
|
|
||||||
|
def test_unknown_postbox_fallback_prevents_every_later_effect(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
prior = PostboxChannelOutcome(outcome_unknown=1)
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||||
|
return_value=prior,
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||||
|
) as deliver_mail,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
|
||||||
|
) as deliver_postbox,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
attempt_number=1,
|
||||||
|
),
|
||||||
|
) as finalize,
|
||||||
|
):
|
||||||
|
_send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
deliver_mail.assert_not_called()
|
||||||
|
deliver_postbox.assert_not_called()
|
||||||
|
self.assertIs(finalize.call_args.kwargs["postbox"], prior)
|
||||||
|
|
||||||
|
def test_postbox_acceptance_stops_mail_fallback(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||||
|
return_value=PostboxChannelOutcome(
|
||||||
|
accepted=1,
|
||||||
|
rejected_permanent=1,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||||
|
) as deliver_mail,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
attempt_number=2,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
deliver_mail.assert_not_called()
|
||||||
|
|
||||||
|
def test_all_postbox_rejections_start_mail_fallback(self) -> None:
|
||||||
|
job = _job()
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||||
|
return_value=PostboxChannelOutcome(
|
||||||
|
rejected_temporary=1,
|
||||||
|
rejected_permanent=1,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||||
|
return_value=_MailChannelOutcome(accepted=True),
|
||||||
|
) as deliver_mail,
|
||||||
|
patch(
|
||||||
|
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||||
|
return_value=SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
attempt_number=1,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
_send_claimed_multichannel_job(
|
||||||
|
_Session(job), # type: ignore[arg-type]
|
||||||
|
job=job, # type: ignore[arg-type]
|
||||||
|
claim_token="claim-1",
|
||||||
|
context=_context(), # type: ignore[arg-type]
|
||||||
|
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||||
|
use_rate_limit=False,
|
||||||
|
enqueue_imap_task=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
deliver_mail.assert_called_once()
|
||||||
|
|
||||||
|
def test_explicit_dual_delivery_reports_partial_and_unknown(self) -> None:
|
||||||
|
self.assertEqual(
|
||||||
|
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
_final_multichannel_status(
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
mail=_MailChannelOutcome(accepted=True),
|
||||||
|
postbox=PostboxChannelOutcome(rejected_permanent=1),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||||
|
_final_multichannel_status(
|
||||||
|
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
mail=_MailChannelOutcome(accepted=True),
|
||||||
|
postbox=PostboxChannelOutcome(outcome_unknown=1),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("mail", "postbox"),
|
||||||
|
[
|
||||||
|
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome()),
|
||||||
|
(_MailChannelOutcome(accepted=True), PostboxChannelOutcome(outcome_unknown=1)),
|
||||||
|
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome(accepted=1)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_outcome_unknown_has_precedence_for_every_delivery_policy(
|
||||||
|
policy: DeliveryChannelPolicy,
|
||||||
|
mail: _MailChannelOutcome,
|
||||||
|
postbox: PostboxChannelOutcome,
|
||||||
|
) -> None:
|
||||||
|
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == JobSendStatus.OUTCOME_UNKNOWN.value
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("mail", "postbox", "expected"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
_MailChannelOutcome(rejected_permanent=True),
|
||||||
|
PostboxChannelOutcome(rejected_permanent=1),
|
||||||
|
JobSendStatus.FAILED_PERMANENT.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
_MailChannelOutcome(rejected_temporary=True),
|
||||||
|
PostboxChannelOutcome(rejected_permanent=1),
|
||||||
|
JobSendStatus.FAILED_TEMPORARY.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
_MailChannelOutcome(rejected_permanent=True),
|
||||||
|
PostboxChannelOutcome(rejected_temporary=1),
|
||||||
|
JobSendStatus.FAILED_TEMPORARY.value,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_rejection_precedence_is_exhaustive_for_every_delivery_policy(
|
||||||
|
policy: DeliveryChannelPolicy,
|
||||||
|
mail: _MailChannelOutcome,
|
||||||
|
postbox: PostboxChannelOutcome,
|
||||||
|
expected: str,
|
||||||
|
) -> None:
|
||||||
|
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("policy", "mail", "postbox", "expected"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.MAIL,
|
||||||
|
_MailChannelOutcome(accepted=True),
|
||||||
|
PostboxChannelOutcome(),
|
||||||
|
JobSendStatus.SMTP_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.POSTBOX,
|
||||||
|
None,
|
||||||
|
PostboxChannelOutcome(accepted=1),
|
||||||
|
JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.POSTBOX,
|
||||||
|
None,
|
||||||
|
PostboxChannelOutcome(accepted=1, rejected_permanent=1),
|
||||||
|
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
_MailChannelOutcome(accepted=True),
|
||||||
|
PostboxChannelOutcome(accepted=1),
|
||||||
|
JobSendStatus.DELIVERED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
_MailChannelOutcome(accepted=True),
|
||||||
|
PostboxChannelOutcome(rejected_permanent=1),
|
||||||
|
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||||
|
_MailChannelOutcome(rejected_permanent=True),
|
||||||
|
PostboxChannelOutcome(accepted=1),
|
||||||
|
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||||
|
_MailChannelOutcome(rejected_permanent=True),
|
||||||
|
PostboxChannelOutcome(accepted=1),
|
||||||
|
JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||||
|
_MailChannelOutcome(accepted=True),
|
||||||
|
PostboxChannelOutcome(rejected_permanent=1),
|
||||||
|
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_accepted_delivery_decision_table(
|
||||||
|
policy: DeliveryChannelPolicy,
|
||||||
|
mail: _MailChannelOutcome | None,
|
||||||
|
postbox: PostboxChannelOutcome,
|
||||||
|
expected: str,
|
||||||
|
) -> None:
|
||||||
|
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user