Compare commits
45
Commits
v0.1.10
...
3f3545f080
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3f3545f080 | ||
|
|
d9195a2d2b | ||
|
|
d635f3a5fc | ||
|
|
1d6c745991 | ||
|
|
5df26be074 | ||
|
|
50ce8b0acb | ||
|
|
9da03090a7 | ||
|
|
dd09b06c47 | ||
|
|
c6bbdae2e1 | ||
|
|
bf6e07f307 | ||
|
|
7733265cc8 | ||
|
|
b38597f2be | ||
|
|
4eeba62bbc | ||
|
|
2afdd38128 | ||
|
|
cec3d17bff | ||
|
|
82ddc0c34c | ||
|
|
fa4eb39e0b | ||
|
|
e689fdf495 | ||
|
|
5f7503598c | ||
|
|
cd223cbb95 | ||
|
|
4a120e8009 | ||
|
|
c769be39da | ||
|
|
46df12c025 | ||
|
|
961d5d1130 | ||
|
|
23b9a531d5 | ||
|
|
cc93945f79 | ||
|
|
101f3ccd7d | ||
|
|
2199187e8b | ||
|
|
dd9592a192 | ||
|
|
5240749ae1 | ||
|
|
f11c56e890 | ||
|
|
8d0a2608ed | ||
|
|
89ae14c032 | ||
|
|
38af25ee88 | ||
|
|
4774a8025c | ||
|
|
689dc1fd6b | ||
|
|
90677348ff | ||
|
|
06786e86ef | ||
|
|
6fda123fc3 | ||
|
|
2fa91bb943 | ||
|
|
e3cc476508 | ||
|
|
01ef541917 | ||
|
|
d567257311 | ||
|
|
3075ef7f5b | ||
|
|
1ab4e91ffd |
@@ -1,5 +1,11 @@
|
||||
# GovOPlaN Campaign Codex Guide
|
||||
|
||||
## Documentation Contract
|
||||
|
||||
- Treat documentation as part of every behavior change. Update this module's manifest-driven `DocumentationTopic` contributions for affected user and administrator behavior.
|
||||
- Keep feature content here; `govoplan-docs` projects it without importing Campaign internals.
|
||||
- Maintain a static user/admin baseline and run `/mnt/DATA/git/govoplan/tools/checks/check-manifest-shapes.py` after behavior or manifest changes.
|
||||
|
||||
## Scope
|
||||
|
||||
This repository owns the `campaigns` module: campaign authoring, validation, message building, attachment resolution, queue/review/send control, reports, campaign module manifest, and `@govoplan/campaign-webui`.
|
||||
|
||||
@@ -15,12 +15,32 @@ This repository owns:
|
||||
- campaign/version/job/issue/send-attempt/append-attempt models and migrations
|
||||
- campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths
|
||||
- WebUI package `@govoplan/campaign-webui`
|
||||
- route contributions for `/campaigns`, `/campaigns/:campaignId/*`, `/operator`, `/reports`, and `/templates`
|
||||
- route contributions for `/campaigns`, the integrated `/campaigns/queue` view,
|
||||
`/campaigns/reports`, `/campaigns/:campaignId/*`, and `/templates`
|
||||
|
||||
Core owns the auth facade, RBAC/capability contracts, database/session
|
||||
primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an
|
||||
optional platform module for tenant administration and tenant resolver behavior.
|
||||
Files and mail own their respective storage and transport capabilities.
|
||||
When the optional Reporting module is enabled, Campaign contributes its
|
||||
recipient-free aggregate delivery report through the versioned Core report
|
||||
provider contract. Reporting owns the global `/reports` route. Campaign keeps
|
||||
its module-local `/campaigns/reports` view and does not claim the global route
|
||||
when Reporting is absent.
|
||||
|
||||
Generated EML and printable artifacts are durable execution material, not a
|
||||
node-local runtime cache. Campaign stores EML through Core's shared
|
||||
object-storage contract under opaque Campaign-owned keys. Templates returns a
|
||||
bounded artifact or a Files-managed artifact for printable output. Database job
|
||||
rows retain the expected hashes and provenance. Workers resolve and verify the
|
||||
frozen evidence before delivery. Build failure compensates objects written
|
||||
before database commit. Retention uses a fenced forward-recovery operation and
|
||||
independently verifies both artifact absence and the committed locator update;
|
||||
partial or unobservable cleanup remains visible in Ops.
|
||||
An operator-only, dry-run-first reconciler inventories bounded tenant-prefix
|
||||
pages and removes only old objects that remain unreferenced after an active
|
||||
build-fence check. Applied runs are idempotent, fenced, audited, and preserve
|
||||
database references on every storage failure.
|
||||
|
||||
## Dependencies
|
||||
|
||||
@@ -28,10 +48,16 @@ The module has one required runtime dependency:
|
||||
|
||||
- `govoplan-core` for platform services, auth, RBAC, DB/session lifecycle, migrations, and WebUI shell integration
|
||||
|
||||
Files and mail are optional module integrations declared in the campaign manifest:
|
||||
Files, Mail, Distribution Lists, Templates, Postbox, and Calendar are optional module integrations declared in the campaign manifest:
|
||||
|
||||
- `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Server/API campaigns require this integration for attachments and never resolve caller-supplied local filesystem paths. Legacy file-oriented loading remains available only to explicitly trusted operator/library workflows.
|
||||
- `govoplan-mail` owns reusable profiles, encrypted SMTP/IMAP credentials, delivery policy checks, connection tests, and transport execution. Campaign JSON stores only `server.mail_profile_id`; inline transport settings and credentials are rejected. Without Mail, campaigns can still be authored, but profile validation and real delivery are unavailable.
|
||||
- `govoplan-dist-lists` expands reusable governed audiences. Campaign freezes the exact list revision, provider evidence, candidates, and explicit per-recipient primary/fallback route into its own version.
|
||||
- `govoplan-templates` validates and renders published label, envelope, letter, and list-layout templates for postal or internal-mail delivery. Generated output is hash-bound to its template, inputs, actor, route decisions, and Campaign version.
|
||||
- `govoplan-postbox` resolves exact or organization-derived Postbox targets and records provider acceptance and receipt evidence. It remains optional; Mail-only and print-only campaigns do not require it.
|
||||
- `govoplan-calendar` renders and mirrors individualized VEVENT invitations through the versioned `calendar.invitations` capability. Campaign freezes one METHOD:REQUEST attachment per recipient during build, creates the Calendar mirror only after delivery acceptance, and reads live RSVP state in bounded report batches. Mail may forward METHOD:REPLY parts from an authorized IMAP source. Calendar absence leaves ordinary Campaign authoring and delivery usable.
|
||||
|
||||
Hybrid delivery never treats an opt-in as an implicit duplicate-send instruction. The Campaign author selects one primary route per recipient and may select a supported fallback. A fallback runs only after the first channel rejects before acceptance; accepted or outcome-unknown effects stop cross-channel retry. Printable output is generated once during build, optionally persisted through Files, reviewed with the exact Campaign version, and accepted idempotently per recipient job during delivery.
|
||||
|
||||
Public campaign, version, job, and report responses expose business data and
|
||||
delivery evidence, but never process-local paths, storage-backend keys, or
|
||||
@@ -94,7 +120,7 @@ Platform RBAC and governance rules are documented in `govoplan-core/docs/`.
|
||||
- [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives.
|
||||
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
|
||||
- Immediate delivery is bounded to 25 exact eligible recipient jobs by default. Deployments may set `GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0–500), and tenants may narrow that ceiling through `campaign_delivery_policy.synchronous_send_max_recipients` in tenant settings.
|
||||
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
|
||||
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
|
||||
- [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path.
|
||||
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.
|
||||
- [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management.
|
||||
|
||||
@@ -9,3 +9,5 @@ GOVOPLAN_MAIL_TEST_IMAP_PORT=3143
|
||||
GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent
|
||||
GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password
|
||||
GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45
|
||||
GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT=36379
|
||||
GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS=3
|
||||
|
||||
@@ -44,6 +44,111 @@ If the smoke is started immediately after `docker compose up -d`, GreenMail may
|
||||
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
|
||||
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
|
||||
|
||||
## Campaign Acceptance
|
||||
|
||||
The transport smoke proves the Mail adapters. The Campaign acceptance runner
|
||||
proves the public composition: it creates an isolated temporary Core database,
|
||||
creates a Mail-owned encrypted profile through the API, materializes the
|
||||
credential-free [`greenmail-delivery`](../../examples/greenmail-delivery/campaign.json)
|
||||
fixture with only that profile reference, validates/builds it, sends the exact
|
||||
generated EML through Campaign once, appends it once, and cross-checks Campaign
|
||||
report/audit state with one unique-subject message in the GreenMail INBOX and
|
||||
Sent folders. Provider mailbox verification is not a byte-for-byte comparison
|
||||
after provider-side header or storage transformations.
|
||||
|
||||
```bash
|
||||
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
|
||||
set -a
|
||||
. ./.env
|
||||
set +a
|
||||
/mnt/DATA/git/govoplan/.venv/bin/python run_campaign_acceptance.py \
|
||||
--evidence /tmp/govoplan-campaign-greenmail-evidence.json
|
||||
```
|
||||
|
||||
The default run also uses controlled loopback protocol endpoints to prove that
|
||||
an SMTP connection loss before transmission is temporary, an explicit SMTP
|
||||
authentication rejection is permanent, a final `451` response after DATA is
|
||||
temporary, one accepted and one refused RCPT command is retained as partial
|
||||
envelope acceptance, a connection loss after complete DATA is frozen as
|
||||
`outcome_unknown`, and an IMAP authentication rejection after SMTP acceptance
|
||||
leaves the send accepted while the append fails. A
|
||||
second ordinary send must be rejected before another provider effect.
|
||||
|
||||
The worker drill queues one job, starts the registered
|
||||
`govoplan.campaigns.send_email` task body in a dedicated OS process, waits until
|
||||
the controlled endpoint has received complete DATA, terminates that process,
|
||||
and starts the same task body in a fresh process. It proves the durable
|
||||
`sending`/unfinished-attempt boundary is recovered as `outcome_unknown`
|
||||
without a second SMTP connection or DATA transaction. This is a real process
|
||||
and task-boundary interruption, but it does not start a Celery daemon, Redis
|
||||
broker, or broker redelivery; `celery_broker_redelivery` therefore remains
|
||||
`false` in the evidence.
|
||||
|
||||
The bounded JSON contains no endpoint, account, address, credential, profile,
|
||||
campaign, version, or job identifiers. It records module versions, the fixture
|
||||
hash, normalized classifications/counts, the Mail-profile boundary, required
|
||||
audit actions, provider mailbox increments, and coverage flags. Runtime version
|
||||
declarations identify the exercised composition; they do not claim that the
|
||||
sources are clean, tagged, signed, or release-provenanced. The evidence names
|
||||
them `declared_module_versions` and keeps `source_artifact_provenance` false;
|
||||
exact commit/artifact provenance belongs to the package and release gate.
|
||||
|
||||
This runner is restricted to literal loopback IP addresses and the synchronous
|
||||
Campaign delivery mode. Hostnames such as `localhost` and every non-loopback
|
||||
address fail before profile creation, avoiding a DNS change between validation
|
||||
and connection. It is local target-like evidence, not approval of an
|
||||
institution's SMTP/IMAP service. The controlled post-DATA, temporary-response,
|
||||
partial-refusal, and task-process interruption drills are local effect-level
|
||||
proof, not proof of a target provider's behavior or Redis/Celery broker
|
||||
redelivery. Use `--success-only` only when testing the success journey without
|
||||
the local failure endpoints.
|
||||
|
||||
## Redis/Celery Redelivery Acceptance
|
||||
|
||||
Run the maintained broker/worker-loss acceptance separately from the GreenMail
|
||||
journey:
|
||||
|
||||
```bash
|
||||
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
|
||||
set -a
|
||||
. ./.env
|
||||
set +a
|
||||
/mnt/DATA/git/govoplan/.venv/bin/python run_celery_redelivery_acceptance.py \
|
||||
--evidence /tmp/govoplan-campaign-celery-redelivery-evidence.json
|
||||
```
|
||||
|
||||
The runner creates a unique Compose project, starts only its loopback-bound,
|
||||
AOF-enabled Redis service, creates an isolated temporary GovOPlaN database,
|
||||
and starts a real Celery worker subscribed to `send_email`. A controlled SMTP
|
||||
server holds the transaction after complete DATA and before the final response.
|
||||
The runner kills that solo worker with the task still unacknowledged and starts
|
||||
a replacement worker. After the configured Redis visibility timeout, the same Celery task identity must be redelivered.
|
||||
The replacement must turn the durable
|
||||
unfinished attempt into `outcome_unknown`, acknowledge the task, drain the
|
||||
broker queue/unacked records, and leave the SMTP endpoint at exactly one
|
||||
connection and one DATA transaction.
|
||||
|
||||
Only bounded counts, classifications, and booleans are retained. Worker logs,
|
||||
task IDs, database identifiers, endpoints, credentials, and raw diagnostics are
|
||||
kept in the temporary runtime and deleted. The evidence proves the local Redis
|
||||
transport, real Celery process boundary, runner-supervised replacement, and
|
||||
Campaign's duplicate-effect guard. It deliberately keeps production daemon supervision,
|
||||
target-provider behavior, and source-artifact provenance false.
|
||||
It does not claim that systemd, Kubernetes, another container orchestrator, or
|
||||
an institution's Redis/SMTP deployment behaves identically.
|
||||
|
||||
The default run requires Docker CLI/Compose/daemon access and permission to
|
||||
pull `redis:7-alpine`; the Celery workers execute from the current Python
|
||||
environment. The isolated Compose project and volume are removed on exit.
|
||||
`GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS` defaults to three
|
||||
seconds only to make this destructive local drill finish promptly; it is not a
|
||||
production recommendation.
|
||||
|
||||
Starting the maintained test bed requires a working Docker CLI, Compose plugin,
|
||||
daemon/socket access, and permission to pull `greenmail/standalone:2.1.9`. The
|
||||
Campaign runner needs only the already-running loopback endpoints; it neither
|
||||
starts Docker nor claims that it did.
|
||||
|
||||
## Use With A Campaign
|
||||
|
||||
Use the same settings in a campaign mail profile:
|
||||
|
||||
@@ -15,3 +15,19 @@ services:
|
||||
- "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025"
|
||||
- "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143"
|
||||
- "127.0.0.1:38080:8080"
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
ports:
|
||||
- "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379"
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 1s
|
||||
timeout: 1s
|
||||
retries: 30
|
||||
volumes:
|
||||
- campaign-redis-data:/data
|
||||
|
||||
volumes:
|
||||
campaign-redis-data:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,856 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Prove Redis/Celery redelivery does not repeat an ambiguous SMTP effect.
|
||||
|
||||
The default run starts an isolated Redis Compose service, two successive real
|
||||
Celery worker processes, and a controlled loopback SMTP endpoint. It kills the
|
||||
first worker after complete DATA but before a final SMTP response. The same
|
||||
unacknowledged broker task must be delivered to the replacement worker, which
|
||||
must freeze the unfinished durable attempt as ``outcome_unknown`` without a
|
||||
second SMTP connection or DATA transaction.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from collections import Counter
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass, replace
|
||||
from datetime import datetime, timezone
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from typing import Any, Callable, Iterator, Mapping
|
||||
from uuid import uuid4
|
||||
|
||||
from redis import Redis
|
||||
from redis.exceptions import RedisError
|
||||
|
||||
|
||||
SCRIPT_ROOT = Path(__file__).resolve().parent
|
||||
REPOSITORY_ROOT = SCRIPT_ROOT.parents[1]
|
||||
if str(SCRIPT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPT_ROOT))
|
||||
|
||||
from run_campaign_acceptance import ( # noqa: E402
|
||||
AcceptanceError,
|
||||
DEFAULT_FIXTURE,
|
||||
EXPECTED_AUDIT_ACTIONS,
|
||||
TestbedSettings,
|
||||
_assert_evidence_safe,
|
||||
_core_package_version,
|
||||
_durable_state_evidence,
|
||||
_expect,
|
||||
_report_evidence,
|
||||
create_mail_profile,
|
||||
prepare_campaign_scenario,
|
||||
required_composition_versions,
|
||||
smtp_fault_endpoint,
|
||||
)
|
||||
|
||||
|
||||
EVIDENCE_SCHEMA = "govoplan.campaign.celery-redelivery-acceptance.v1"
|
||||
MAX_EVIDENCE_BYTES = 128 * 1024
|
||||
DEFAULT_COMPOSE_FILE = SCRIPT_ROOT / "docker-compose.yml"
|
||||
TASK_RECEIVED_PATTERN = re.compile(
|
||||
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] received",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
TASK_SUCCEEDED_PATTERN = re.compile(
|
||||
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] succeeded",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
WORKER_BOOTSTRAP = r"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
from govoplan_core.celery_app import celery
|
||||
|
||||
visibility_timeout = int(os.environ["GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS"])
|
||||
celery.conf.broker_transport_options = {
|
||||
**dict(celery.conf.broker_transport_options or {}),
|
||||
"polling_interval": 0.25,
|
||||
"visibility_timeout": visibility_timeout,
|
||||
}
|
||||
celery.worker_main(
|
||||
[
|
||||
"worker",
|
||||
"--loglevel=INFO",
|
||||
"--pool=solo",
|
||||
"--concurrency=1",
|
||||
"--queues=send_email",
|
||||
f"--hostname={sys.argv[1]}@%h",
|
||||
"--without-gossip",
|
||||
"--without-mingle",
|
||||
"--without-heartbeat",
|
||||
]
|
||||
)
|
||||
"""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class WorkerProcess:
|
||||
process: subprocess.Popen[bytes]
|
||||
log_path: Path
|
||||
log_handle: Any
|
||||
|
||||
def text(self) -> str:
|
||||
self.log_handle.flush()
|
||||
try:
|
||||
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
||||
except OSError as exc:
|
||||
raise AcceptanceError("Celery worker evidence log could not be read") from exc
|
||||
|
||||
def received_task_ids(self) -> tuple[str, ...]:
|
||||
return tuple(TASK_RECEIVED_PATTERN.findall(self.text()))
|
||||
|
||||
def succeeded_task_ids(self) -> tuple[str, ...]:
|
||||
return tuple(TASK_SUCCEEDED_PATTERN.findall(self.text()))
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisBrokerState:
|
||||
queue_depth: int
|
||||
unacked_hash_count: int
|
||||
unacked_index_count: int
|
||||
|
||||
def as_dict(self) -> dict[str, int]:
|
||||
return {
|
||||
"queue_depth": self.queue_depth,
|
||||
"unacked_hash_count": self.unacked_hash_count,
|
||||
"unacked_index_count": self.unacked_index_count,
|
||||
}
|
||||
|
||||
|
||||
def _positive_int(value: str, *, label: str) -> int:
|
||||
try:
|
||||
parsed = int(value)
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError(f"{label} must be a positive integer") from exc
|
||||
if parsed <= 0:
|
||||
raise argparse.ArgumentTypeError(f"{label} must be a positive integer")
|
||||
return parsed
|
||||
|
||||
|
||||
def _unused_loopback_port() -> int:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as probe:
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
return int(probe.getsockname()[1])
|
||||
|
||||
|
||||
def _compose_command(
|
||||
*, compose_file: Path, project_name: str, operation: str
|
||||
) -> list[str]:
|
||||
prefix = [
|
||||
"docker",
|
||||
"compose",
|
||||
"--file",
|
||||
str(compose_file),
|
||||
"--project-name",
|
||||
project_name,
|
||||
]
|
||||
if operation == "up":
|
||||
return [*prefix, "up", "--detach", "redis"]
|
||||
if operation == "down":
|
||||
return [*prefix, "down", "--volumes", "--remove-orphans"]
|
||||
raise AcceptanceError("Unsupported Redis Compose operation")
|
||||
|
||||
|
||||
def _run_compose(
|
||||
command: list[str],
|
||||
*,
|
||||
environment: Mapping[str, str],
|
||||
timeout_seconds: int,
|
||||
) -> None:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
env=dict(environment),
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
timeout=timeout_seconds,
|
||||
check=False,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise AcceptanceError("Redis Compose lifecycle command failed") from exc
|
||||
if completed.returncode != 0:
|
||||
raise AcceptanceError("Redis Compose lifecycle command failed")
|
||||
|
||||
|
||||
def _wait_for_redis(redis_url: str, *, timeout_seconds: int) -> None:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
client = Redis.from_url(
|
||||
redis_url,
|
||||
socket_connect_timeout=1,
|
||||
socket_timeout=1,
|
||||
decode_responses=False,
|
||||
)
|
||||
try:
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
if client.ping() is True:
|
||||
return
|
||||
except RedisError:
|
||||
pass
|
||||
time.sleep(0.25)
|
||||
finally:
|
||||
client.close()
|
||||
raise AcceptanceError("Isolated Redis broker did not become ready")
|
||||
|
||||
|
||||
@contextmanager
|
||||
def isolated_redis_broker(
|
||||
*,
|
||||
compose_file: Path,
|
||||
timeout_seconds: int,
|
||||
requested_port: int | None = None,
|
||||
) -> Iterator[str]:
|
||||
if shutil.which("docker") is None:
|
||||
raise AcceptanceError("Docker CLI is required to start the isolated Redis broker")
|
||||
if not compose_file.is_file():
|
||||
raise AcceptanceError("Redis Compose definition is unavailable")
|
||||
port = requested_port or _unused_loopback_port()
|
||||
if port <= 0 or port > 65_535:
|
||||
raise AcceptanceError("Redis test port is invalid")
|
||||
project_name = f"govoplan-campaign-redelivery-{uuid4().hex[:12]}"
|
||||
environment = {
|
||||
**os.environ,
|
||||
"GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT": str(port),
|
||||
}
|
||||
lifecycle_attempted = False
|
||||
try:
|
||||
lifecycle_attempted = True
|
||||
_run_compose(
|
||||
_compose_command(
|
||||
compose_file=compose_file,
|
||||
project_name=project_name,
|
||||
operation="up",
|
||||
),
|
||||
environment=environment,
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
redis_url = f"redis://127.0.0.1:{port}/0"
|
||||
_wait_for_redis(redis_url, timeout_seconds=timeout_seconds)
|
||||
yield redis_url
|
||||
finally:
|
||||
if lifecycle_attempted:
|
||||
_run_compose(
|
||||
_compose_command(
|
||||
compose_file=compose_file,
|
||||
project_name=project_name,
|
||||
operation="down",
|
||||
),
|
||||
environment=environment,
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
|
||||
|
||||
def _start_worker(runtime_root: Path, *, label: str) -> WorkerProcess:
|
||||
log_path = runtime_root / f"{label}.log"
|
||||
log_handle = log_path.open("wb")
|
||||
environment = {**os.environ, "PYTHONUNBUFFERED": "1"}
|
||||
try:
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, "-c", WORKER_BOOTSTRAP, label],
|
||||
env=environment,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=log_handle,
|
||||
stderr=subprocess.STDOUT,
|
||||
close_fds=True,
|
||||
)
|
||||
except Exception:
|
||||
log_handle.close()
|
||||
raise
|
||||
return WorkerProcess(process=process, log_path=log_path, log_handle=log_handle)
|
||||
|
||||
|
||||
def _wait_for_worker_ready(worker: WorkerProcess, *, timeout_seconds: int) -> None:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
while time.monotonic() < deadline:
|
||||
if worker.process.poll() is not None:
|
||||
raise AcceptanceError("Celery worker exited before becoming ready")
|
||||
if " ready." in worker.text():
|
||||
return
|
||||
time.sleep(0.2)
|
||||
raise AcceptanceError("Celery worker did not become ready")
|
||||
|
||||
|
||||
def _wait_for_received_task(
|
||||
worker: WorkerProcess,
|
||||
*,
|
||||
timeout_seconds: int,
|
||||
expected_task_id: str | None = None,
|
||||
) -> str:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
while time.monotonic() < deadline:
|
||||
received = worker.received_task_ids()
|
||||
if received:
|
||||
if len(set(received)) != 1:
|
||||
raise AcceptanceError("Celery worker received more than one task identity")
|
||||
task_id = received[0]
|
||||
if expected_task_id is not None and task_id != expected_task_id:
|
||||
raise AcceptanceError("Replacement worker received a different broker task")
|
||||
return task_id
|
||||
if worker.process.poll() is not None:
|
||||
raise AcceptanceError("Celery worker exited before receiving the task")
|
||||
time.sleep(0.2)
|
||||
raise AcceptanceError("Celery worker did not receive the broker task")
|
||||
|
||||
|
||||
def _wait_for_task_success(
|
||||
worker: WorkerProcess,
|
||||
*,
|
||||
task_id: str,
|
||||
timeout_seconds: int,
|
||||
) -> None:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
while time.monotonic() < deadline:
|
||||
succeeded = worker.succeeded_task_ids()
|
||||
if task_id in succeeded:
|
||||
return
|
||||
if worker.process.poll() is not None:
|
||||
raise AcceptanceError("Replacement Celery worker exited before task success")
|
||||
time.sleep(0.2)
|
||||
raise AcceptanceError("Redelivered Celery task did not complete")
|
||||
|
||||
|
||||
def _kill_worker(worker: WorkerProcess, *, timeout_seconds: int) -> int:
|
||||
if worker.process.poll() is not None:
|
||||
raise AcceptanceError("Celery worker exited before controlled termination")
|
||||
worker.process.kill()
|
||||
try:
|
||||
return_code = worker.process.wait(timeout=timeout_seconds)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise AcceptanceError("Celery worker could not be killed") from exc
|
||||
if return_code == 0:
|
||||
raise AcceptanceError("Celery worker termination was not forced")
|
||||
return return_code
|
||||
|
||||
|
||||
def _stop_worker(worker: WorkerProcess, *, timeout_seconds: int) -> None:
|
||||
if worker.process.poll() is None:
|
||||
worker.process.terminate()
|
||||
try:
|
||||
worker.process.wait(timeout=timeout_seconds)
|
||||
except subprocess.TimeoutExpired:
|
||||
worker.process.kill()
|
||||
worker.process.wait(timeout=timeout_seconds)
|
||||
worker.log_handle.close()
|
||||
|
||||
|
||||
def _broker_state(redis_url: str) -> RedisBrokerState:
|
||||
client = Redis.from_url(
|
||||
redis_url,
|
||||
socket_connect_timeout=2,
|
||||
socket_timeout=2,
|
||||
decode_responses=False,
|
||||
)
|
||||
try:
|
||||
return RedisBrokerState(
|
||||
queue_depth=int(client.llen("send_email")),
|
||||
unacked_hash_count=int(client.hlen("unacked")),
|
||||
unacked_index_count=int(client.zcard("unacked_index")),
|
||||
)
|
||||
except RedisError as exc:
|
||||
raise AcceptanceError("Redis broker state could not be inspected") from exc
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def _wait_for_broker_drained(
|
||||
redis_url: str,
|
||||
*,
|
||||
timeout_seconds: int,
|
||||
) -> RedisBrokerState:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
last = RedisBrokerState(0, 0, 0)
|
||||
while time.monotonic() < deadline:
|
||||
last = _broker_state(redis_url)
|
||||
if last == RedisBrokerState(0, 0, 0):
|
||||
return last
|
||||
time.sleep(0.2)
|
||||
raise AcceptanceError("Redis broker retained delivery state after recovery")
|
||||
|
||||
|
||||
def _queue_evidence(payload: Mapping[str, Any]) -> dict[str, Any]:
|
||||
expected = {
|
||||
"queued_count": 1,
|
||||
"skipped_count": 0,
|
||||
"blocked_count": 0,
|
||||
"enqueued_count": 1,
|
||||
"delivery_mode": "worker_queue",
|
||||
"worker_queue_available": True,
|
||||
"dry_run": False,
|
||||
}
|
||||
evidence = {key: payload.get(key) for key in expected}
|
||||
if evidence != expected:
|
||||
raise AcceptanceError("Campaign was not durably queued to one Celery task")
|
||||
return evidence
|
||||
|
||||
|
||||
def execute_redelivery_scenario(
|
||||
client: Any,
|
||||
headers: Mapping[str, str],
|
||||
*,
|
||||
fixture_path: Path,
|
||||
settings: TestbedSettings,
|
||||
endpoint: Any,
|
||||
redis_url: str,
|
||||
runtime_root: Path,
|
||||
snapshot_probe: Callable[[str], tuple[Mapping[str, Any], Mapping[str, Any]]],
|
||||
audit_probe: Callable[[str, str], Mapping[str, int]],
|
||||
delivery_probe: Callable[[str, str], Mapping[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
profile_id = create_mail_profile(
|
||||
client,
|
||||
headers,
|
||||
settings,
|
||||
name="Campaign Redis Celery redelivery drill",
|
||||
smtp_host=endpoint.host,
|
||||
smtp_port=endpoint.port,
|
||||
)
|
||||
prepared = prepare_campaign_scenario(
|
||||
client,
|
||||
headers,
|
||||
fixture_path=fixture_path,
|
||||
profile_id=profile_id,
|
||||
settings=settings,
|
||||
scenario="celery_broker_redelivery",
|
||||
snapshot_probe=snapshot_probe,
|
||||
)
|
||||
|
||||
first_worker = _start_worker(runtime_root, label="first-worker")
|
||||
replacement_worker: WorkerProcess | None = None
|
||||
try:
|
||||
_wait_for_worker_ready(
|
||||
first_worker,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
queued = _expect(
|
||||
client.post(
|
||||
f"/api/v1/campaigns/{prepared.campaign_id}/queue",
|
||||
headers=dict(headers),
|
||||
json={
|
||||
"version_id": prepared.version_id,
|
||||
"include_warnings": True,
|
||||
"enqueue_celery": True,
|
||||
"dry_run": False,
|
||||
},
|
||||
),
|
||||
200,
|
||||
"Celery-redelivery Campaign queue",
|
||||
)
|
||||
queue_evidence = _queue_evidence(queued)
|
||||
first_task_id = _wait_for_received_task(
|
||||
first_worker,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
if not endpoint.wait_for_data(settings.provider_timeout_seconds):
|
||||
raise AcceptanceError("Celery worker did not reach complete SMTP DATA")
|
||||
first_exit_code = _kill_worker(
|
||||
first_worker,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
endpoint.release_held_connection()
|
||||
|
||||
interrupted_state = _durable_state_evidence(
|
||||
delivery_probe(prepared.campaign_id, prepared.version_id)
|
||||
)
|
||||
expected_interrupted = {
|
||||
"job_count": 1,
|
||||
"send_status_counts": {"sending": 1},
|
||||
"attempt_status_counts": {"smtp_in_progress": 1},
|
||||
"unfinished_attempt_count": 1,
|
||||
}
|
||||
if interrupted_state != expected_interrupted:
|
||||
raise AcceptanceError("Killed worker state was not durably SMTP-in-progress")
|
||||
|
||||
replacement_worker = _start_worker(runtime_root, label="replacement-worker")
|
||||
_wait_for_worker_ready(
|
||||
replacement_worker,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
redelivered_task_id = _wait_for_received_task(
|
||||
replacement_worker,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
expected_task_id=first_task_id,
|
||||
)
|
||||
_wait_for_task_success(
|
||||
replacement_worker,
|
||||
task_id=redelivered_task_id,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
recovered_state = _durable_state_evidence(
|
||||
delivery_probe(prepared.campaign_id, prepared.version_id)
|
||||
)
|
||||
expected_recovered = {
|
||||
"job_count": 1,
|
||||
"send_status_counts": {"outcome_unknown": 1},
|
||||
"attempt_status_counts": {"outcome_unknown": 1},
|
||||
"unfinished_attempt_count": 0,
|
||||
}
|
||||
if recovered_state != expected_recovered:
|
||||
raise AcceptanceError("Redelivered task did not freeze the unfinished attempt")
|
||||
|
||||
protocol = endpoint.evidence()
|
||||
expected_protocol = {
|
||||
"connection_count": 1,
|
||||
"accepted_rcpt_commands": 1,
|
||||
"refused_rcpt_commands": 0,
|
||||
"data_transactions": 1,
|
||||
}
|
||||
if protocol != expected_protocol:
|
||||
raise AcceptanceError("Broker redelivery caused an unexpected SMTP transaction")
|
||||
broker_after = _wait_for_broker_drained(
|
||||
redis_url,
|
||||
timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
first_received = first_worker.received_task_ids()
|
||||
replacement_received = replacement_worker.received_task_ids()
|
||||
if first_received != (first_task_id,) or replacement_received != (
|
||||
redelivered_task_id,
|
||||
):
|
||||
raise AcceptanceError(
|
||||
"Celery workers did not each receive the broker task exactly once"
|
||||
)
|
||||
report = _report_evidence(
|
||||
_expect(
|
||||
client.get(
|
||||
f"/api/v1/campaigns/{prepared.campaign_id}/report",
|
||||
headers=dict(headers),
|
||||
params={"version_id": prepared.version_id},
|
||||
),
|
||||
200,
|
||||
"Celery-redelivery Campaign report",
|
||||
)
|
||||
)
|
||||
if report["send_status_counts"] != {"outcome_unknown": 1}:
|
||||
raise AcceptanceError("Campaign report did not retain outcome_unknown")
|
||||
audit_actions = dict(
|
||||
sorted(audit_probe(prepared.campaign_id, prepared.version_id).items())
|
||||
)
|
||||
if not {
|
||||
"campaign.created",
|
||||
"campaign.validated",
|
||||
"campaign.messages_built",
|
||||
"campaign.queued",
|
||||
}.issubset(audit_actions):
|
||||
raise AcceptanceError("Celery-redelivery Campaign audit evidence is incomplete")
|
||||
|
||||
return {
|
||||
**prepared.public_evidence(),
|
||||
"queue": queue_evidence,
|
||||
"interrupted_durable_state": interrupted_state,
|
||||
"recovered_durable_state": recovered_state,
|
||||
"protocol": protocol,
|
||||
"report": report,
|
||||
"audit_actions": audit_actions,
|
||||
"broker": {
|
||||
"transport": "redis",
|
||||
"same_task_identity_redelivered": first_task_id
|
||||
== redelivered_task_id,
|
||||
"first_worker_received_count": len(first_received),
|
||||
"replacement_worker_received_count": len(replacement_received),
|
||||
**broker_after.as_dict(),
|
||||
},
|
||||
"supervision": {
|
||||
"first_worker_killed_after_complete_data": True,
|
||||
"first_worker_forced_exit": first_exit_code != 0,
|
||||
"replacement_worker_started": True,
|
||||
"replacement_worker_completed_redelivery": True,
|
||||
},
|
||||
}
|
||||
finally:
|
||||
endpoint.release_held_connection()
|
||||
_stop_worker(first_worker, timeout_seconds=5)
|
||||
if replacement_worker is not None:
|
||||
_stop_worker(replacement_worker, timeout_seconds=5)
|
||||
|
||||
|
||||
def _runtime_module_versions(registry: Any) -> dict[str, str]:
|
||||
versions = {"core": _core_package_version()}
|
||||
versions.update(
|
||||
{
|
||||
manifest.id: manifest.version
|
||||
for manifest in registry.manifests()
|
||||
if manifest.id in {"access", "audit", "campaigns", "mail"}
|
||||
}
|
||||
)
|
||||
return versions
|
||||
|
||||
|
||||
def _create_runtime_root() -> Path:
|
||||
"""Create the isolated runtime under the platform-selected temp root."""
|
||||
|
||||
return Path(tempfile.mkdtemp(prefix="govoplan-campaign-celery-redelivery-"))
|
||||
|
||||
|
||||
def _bootstrap_and_run(
|
||||
*,
|
||||
settings: TestbedSettings,
|
||||
fixture_path: Path,
|
||||
redis_url: str,
|
||||
visibility_timeout_seconds: int,
|
||||
) -> dict[str, Any]:
|
||||
runtime_root = _create_runtime_root()
|
||||
database = None
|
||||
try:
|
||||
os.environ.update(
|
||||
{
|
||||
"APP_ENV": "test",
|
||||
"DATABASE_URL": f"sqlite:///{runtime_root / 'acceptance.db'}",
|
||||
"FILE_STORAGE_BACKEND": "local",
|
||||
"FILE_STORAGE_LOCAL_ROOT": str(runtime_root / "files"),
|
||||
"MOCK_MAILBOX_DIR": str(runtime_root / "mock-mailbox"),
|
||||
"DEV_BOOTSTRAP_ENABLED": "false",
|
||||
"CELERY_ENABLED": "true",
|
||||
"REDIS_URL": redis_url,
|
||||
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS": str(
|
||||
visibility_timeout_seconds
|
||||
),
|
||||
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true",
|
||||
}
|
||||
)
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.db.bootstrap import bootstrap_dev_data
|
||||
from govoplan_core.db.session import configure_database, set_database
|
||||
from govoplan_core.settings import Settings, settings as core_settings
|
||||
from govoplan_core.tenancy.scope import create_scope_tables
|
||||
|
||||
isolated_settings = Settings()
|
||||
for field_name in Settings.model_fields:
|
||||
setattr(core_settings, field_name, getattr(isolated_settings, field_name))
|
||||
database = configure_database(os.environ["DATABASE_URL"])
|
||||
set_database(database)
|
||||
|
||||
from govoplan_core.server.app import app
|
||||
|
||||
create_scope_tables(database.engine)
|
||||
Base.metadata.create_all(bind=database.engine)
|
||||
with database.SessionLocal() as session:
|
||||
bootstrap_dev_data(
|
||||
session,
|
||||
api_key_secret="celery-redelivery-unused-api-key",
|
||||
user_password="celery-redelivery-admin",
|
||||
)
|
||||
|
||||
def snapshot_probe(
|
||||
version_id: str,
|
||||
) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
|
||||
from govoplan_campaign.backend.db.models import CampaignVersion
|
||||
|
||||
with database.SessionLocal() as session:
|
||||
version = session.get(CampaignVersion, version_id)
|
||||
if version is None:
|
||||
raise AcceptanceError("Campaign execution snapshot is unavailable")
|
||||
raw = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
snapshot = (
|
||||
version.execution_snapshot
|
||||
if isinstance(version.execution_snapshot, dict)
|
||||
else {}
|
||||
)
|
||||
return raw, snapshot
|
||||
|
||||
def audit_probe(campaign_id: str, version_id: str) -> Mapping[str, int]:
|
||||
from govoplan_audit.backend.db.models import AuditLog
|
||||
|
||||
with database.SessionLocal() as session:
|
||||
actions = [
|
||||
row[0]
|
||||
for row in session.query(AuditLog.action)
|
||||
.filter(AuditLog.object_id.in_([campaign_id, version_id]))
|
||||
.all()
|
||||
if row[0] in EXPECTED_AUDIT_ACTIONS
|
||||
]
|
||||
return dict(Counter(actions))
|
||||
|
||||
def delivery_probe(campaign_id: str, version_id: str) -> Mapping[str, Any]:
|
||||
from govoplan_campaign.backend.db.models import CampaignJob, SendAttempt
|
||||
|
||||
with database.SessionLocal() as session:
|
||||
jobs = (
|
||||
session.query(CampaignJob)
|
||||
.filter(
|
||||
CampaignJob.campaign_id == campaign_id,
|
||||
CampaignJob.campaign_version_id == version_id,
|
||||
)
|
||||
.all()
|
||||
)
|
||||
job_ids = [job.id for job in jobs]
|
||||
attempts = (
|
||||
session.query(SendAttempt)
|
||||
.filter(SendAttempt.job_id.in_(job_ids))
|
||||
.all()
|
||||
if job_ids
|
||||
else []
|
||||
)
|
||||
return {
|
||||
"job_count": len(jobs),
|
||||
"send_status_counts": dict(
|
||||
Counter(job.send_status for job in jobs)
|
||||
),
|
||||
"attempt_status_counts": dict(
|
||||
Counter(attempt.status for attempt in attempts)
|
||||
),
|
||||
"unfinished_attempt_count": sum(
|
||||
1 for attempt in attempts if attempt.finished_at is None
|
||||
),
|
||||
}
|
||||
|
||||
with TestClient(app) as client:
|
||||
login = _expect(
|
||||
client.post(
|
||||
"/api/v1/auth/login",
|
||||
json={
|
||||
"email": "admin@example.local",
|
||||
"password": "celery-redelivery-admin",
|
||||
},
|
||||
),
|
||||
200,
|
||||
"Acceptance login",
|
||||
)
|
||||
access_token = str(login.get("access_token") or "")
|
||||
if not access_token:
|
||||
raise AcceptanceError("Acceptance login returned no access token")
|
||||
from govoplan_core.core.runtime import get_registry
|
||||
|
||||
registry = get_registry()
|
||||
if registry is None:
|
||||
raise AcceptanceError("The GovOPlaN module registry is unavailable")
|
||||
composition_versions = required_composition_versions(
|
||||
fixture_path,
|
||||
_runtime_module_versions(registry),
|
||||
)
|
||||
with smtp_fault_endpoint("post_data_hold") as endpoint:
|
||||
scenario = execute_redelivery_scenario(
|
||||
client,
|
||||
{"Authorization": f"Bearer {access_token}"},
|
||||
fixture_path=fixture_path,
|
||||
settings=settings,
|
||||
endpoint=endpoint,
|
||||
redis_url=redis_url,
|
||||
runtime_root=runtime_root,
|
||||
snapshot_probe=snapshot_probe,
|
||||
audit_probe=audit_probe,
|
||||
delivery_probe=delivery_probe,
|
||||
)
|
||||
|
||||
evidence = {
|
||||
"schema_version": EVIDENCE_SCHEMA,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"fixture_sha256": hashlib.sha256(fixture_path.read_bytes()).hexdigest(),
|
||||
"declared_module_versions": composition_versions,
|
||||
"target": {
|
||||
"kind": "local_redis_celery_controlled_smtp",
|
||||
"isolated_temporary_database": True,
|
||||
"redis_started_by_runner": True,
|
||||
"worker_pool": "solo",
|
||||
"worker_prefetch_multiplier": 1,
|
||||
"task_acks_late": True,
|
||||
"task_reject_on_worker_lost": True,
|
||||
"visibility_timeout_seconds": visibility_timeout_seconds,
|
||||
},
|
||||
"scenario": scenario,
|
||||
"coverage": {
|
||||
"redis_broker_delivery": True,
|
||||
"celery_worker_processes": True,
|
||||
"forced_worker_loss_after_complete_data": True,
|
||||
"same_task_broker_redelivery": True,
|
||||
"durable_outcome_unknown_recovery": True,
|
||||
"duplicate_smtp_transaction_prevented": True,
|
||||
"production_daemon_supervisor": False,
|
||||
"target_provider": False,
|
||||
"source_artifact_provenance": False,
|
||||
},
|
||||
}
|
||||
_assert_evidence_safe(evidence, settings=settings)
|
||||
rendered = json.dumps(
|
||||
evidence,
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
).encode("utf-8") + b"\n"
|
||||
if len(rendered) > MAX_EVIDENCE_BYTES:
|
||||
raise AcceptanceError("Celery-redelivery evidence exceeds its size limit")
|
||||
return evidence
|
||||
finally:
|
||||
if database is not None:
|
||||
database.engine.dispose()
|
||||
shutil.rmtree(runtime_root, ignore_errors=True)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--fixture", type=Path, default=DEFAULT_FIXTURE)
|
||||
parser.add_argument("--compose-file", type=Path, default=DEFAULT_COMPOSE_FILE)
|
||||
parser.add_argument("--redis-port", type=int)
|
||||
parser.add_argument(
|
||||
"--visibility-timeout-seconds",
|
||||
type=lambda value: _positive_int(value, label="visibility timeout"),
|
||||
default=os.environ.get(
|
||||
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS",
|
||||
"3",
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout-seconds",
|
||||
type=lambda value: _positive_int(value, label="timeout"),
|
||||
default=60,
|
||||
)
|
||||
parser.add_argument("--evidence", type=Path)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
settings = TestbedSettings.from_environment()
|
||||
settings.assert_local_testbed()
|
||||
settings = replace(
|
||||
settings,
|
||||
provider_timeout_seconds=args.timeout_seconds,
|
||||
)
|
||||
with isolated_redis_broker(
|
||||
compose_file=args.compose_file.resolve(),
|
||||
timeout_seconds=args.timeout_seconds,
|
||||
requested_port=args.redis_port,
|
||||
) as redis_url:
|
||||
evidence = _bootstrap_and_run(
|
||||
settings=settings,
|
||||
fixture_path=args.fixture.resolve(),
|
||||
redis_url=redis_url,
|
||||
visibility_timeout_seconds=args.visibility_timeout_seconds,
|
||||
)
|
||||
rendered = json.dumps(
|
||||
evidence,
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
) + "\n"
|
||||
if args.evidence:
|
||||
args.evidence.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.evidence.write_text(rendered, encoding="utf-8")
|
||||
else:
|
||||
sys.stdout.write(rendered)
|
||||
return 0
|
||||
except AcceptanceError as exc:
|
||||
print(f"Campaign Celery-redelivery acceptance failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
except Exception as exc:
|
||||
print(
|
||||
"Campaign Celery-redelivery acceptance failed unexpectedly "
|
||||
f"({type(exc).__name__}); inspect local service logs.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,39 @@
|
||||
# Campaign Accessibility Review
|
||||
|
||||
The Campaign WebUI uses Core's semantic `Button`, `Dialog`, `DataGrid`, form,
|
||||
alert, and segmented-control components. Core owns focus trapping, focus return,
|
||||
Escape handling, labels, disabled state, and keyboard behavior for those shared
|
||||
primitives.
|
||||
|
||||
## Repeatable review matrix
|
||||
|
||||
Run this matrix for Campaign overview, wizard, sender and recipients,
|
||||
attachments, template editing, review and send, operator queue, and reports:
|
||||
|
||||
1. Navigate all actions with Tab and Shift+Tab; focus must remain visible and
|
||||
follow the visual reading order.
|
||||
2. Activate buttons and links with Enter, and native buttons with Space.
|
||||
3. Open every dialog, verify initial focus remains within it, close with Escape,
|
||||
and verify focus returns to the opener.
|
||||
4. Use DataGrid sorting, filtering, pagination, row selection, and action menus
|
||||
without a pointer.
|
||||
5. At 200 percent browser zoom and a 320 CSS-pixel viewport, verify that content
|
||||
reflows or scrolls without hiding actions.
|
||||
6. With reduced motion enabled, verify that workflow state does not depend on
|
||||
animation.
|
||||
7. With a screen reader, verify page headings, field labels, validation errors,
|
||||
workflow states, message navigation, and attachment evidence.
|
||||
|
||||
## Automated structural guard
|
||||
|
||||
`npm run test:accessibility-contract` rejects non-semantic click handlers,
|
||||
unlabelled icon-only buttons in the message preview, and Campaign-local modal
|
||||
implementations that bypass Core's `Dialog`. It complements rather than replaces
|
||||
browser and assistive-technology testing.
|
||||
|
||||
## Known boundary
|
||||
|
||||
Translation keys may be visible in source because Core resolves them at runtime.
|
||||
The review must use a built application with current language packages. WCAG
|
||||
conformance is a release-level claim and still requires a bounded manual audit
|
||||
of the release candidate.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Campaign Access Explanation Coverage
|
||||
|
||||
Campaign access explanations are resource-specific evidence. They inherit the
|
||||
parent Campaign decision only where the child has no independent grant model,
|
||||
and they must identify that inheritance explicitly.
|
||||
|
||||
## Implemented
|
||||
|
||||
- Campaign
|
||||
- Campaign version
|
||||
- Campaign delivery job / built message
|
||||
- Computed Campaign report, identified by Campaign, version, and report kind
|
||||
|
||||
## Planned Slices
|
||||
|
||||
1. Recipient rows and imported recipient-source snapshots
|
||||
2. Attachment bindings and frozen attachment resolutions
|
||||
3. Validation issues, review decisions, and attachment-policy overrides
|
||||
4. Delivery attempts, IMAP append attempts, Postbox attempts, and
|
||||
reconciliation decisions
|
||||
5. Campaign shares and ownership-transfer records
|
||||
6. Import mapping profiles and import executions
|
||||
7. Reusable Campaign templates and template revisions when the template
|
||||
library becomes persistent
|
||||
8. Export packages and protocol/report artifacts
|
||||
|
||||
Each child explanation must include:
|
||||
|
||||
- the child resource identity and current state;
|
||||
- the parent Campaign and version where applicable;
|
||||
- whether access is inherited, independently granted, or further restricted;
|
||||
- effective owner/share/policy provenance;
|
||||
- missing-module or unavailable-evidence reasons without leaking the hidden
|
||||
object;
|
||||
- a stable resource identifier suitable for audit and support links.
|
||||
|
||||
Delivery attempts, review decisions, reports, and exports can contain more
|
||||
sensitive evidence than the Campaign summary. Their read and diagnostic/export
|
||||
permissions therefore remain independently enforceable even when the parent
|
||||
Campaign is readable.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Campaign Build Recovery
|
||||
|
||||
Campaign build is fenced per tenant and Campaign version. Before rendering or
|
||||
writing generated artifacts, Campaign commits a Core recovery operation with
|
||||
the canonical source and validation hashes, the runtime fence, and a reserved
|
||||
opaque object prefix. A repeated idempotency key can replay only a verified
|
||||
successful build; it cannot start a second active build.
|
||||
|
||||
Generated EML and bounded print output are written to shared storage and checked
|
||||
for exact size and SHA-256 content. Campaign renews the build fence after that
|
||||
check and before changing jobs, then commits its jobs and execution snapshot,
|
||||
compares the stored object and database manifests, and records verified success.
|
||||
A managed Files output makes the operation forward-recoverable because Campaign
|
||||
cannot undo a Files-owned artifact; object-only builds use explicit
|
||||
compensation.
|
||||
|
||||
If the Campaign database transaction fails, Campaign deletes every object it
|
||||
recorded and verifies absence before recording recovered state. Failed deletion,
|
||||
an unavailable storage check, process loss, or superseded-object cleanup failure
|
||||
leaves a recovery-required operation visible in Ops. Do not retry such an
|
||||
operation as a normal build. Verify its checkpoint chain and reserved prefix,
|
||||
then reconcile it through the owning-module procedure.
|
||||
|
||||
## Orphan inventory and cleanup
|
||||
|
||||
The operator-only endpoint
|
||||
`POST /api/v1/campaigns/operations/artifacts/reconcile` requires
|
||||
`system:settings:write`. It never scans outside
|
||||
`campaign-artifacts/{tenant_id}/`, and one request reads at most `page_size`
|
||||
objects. The default request is a dry run:
|
||||
|
||||
```json
|
||||
{}
|
||||
```
|
||||
|
||||
The response reports each eligible key, size, modification time, age, reason,
|
||||
page totals, and `next_cursor`. Continue with that cursor to inspect the next
|
||||
bounded page. Objects are not eligible when they are referenced by a committed
|
||||
EML or print-output row, belong to an actively fenced build, have no trustworthy
|
||||
modification time, have an invalid build-key shape, or are younger than the
|
||||
grace period. The minimum and default grace period is 24 hours.
|
||||
|
||||
Apply an inspected page with a new idempotency key:
|
||||
|
||||
```json
|
||||
{
|
||||
"apply": true,
|
||||
"idempotency_key": "incident-2026-08-03-page-1",
|
||||
"grace_period_hours": 24,
|
||||
"page_size": 250
|
||||
}
|
||||
```
|
||||
|
||||
An applied run rechecks committed references and active build leases before
|
||||
each bounded deletion batch. A Core distributed lease prevents two nodes from
|
||||
committing the same tenant cleanup concurrently. Every attempted deletion is
|
||||
probed afterward. `recovery_required` means an object was verified to remain;
|
||||
`outcome_unknown` means storage could not prove whether the delete took effect.
|
||||
Use a new idempotency key to retry after the storage problem is corrected. A
|
||||
successful repeated request with the same key returns `already_completed` and
|
||||
does not delete again.
|
||||
|
||||
Cleanup never clears Campaign database references. Audit and recovery evidence
|
||||
records counts and hashed manifests rather than object keys. Exact keys are
|
||||
returned only by this privileged endpoint and must stay in restricted incident
|
||||
records.
|
||||
|
||||
No recovery checkpoint contains message bodies, recipients, credentials, or
|
||||
resolved provider secrets. Object keys remain restricted diagnostics rather
|
||||
than Campaign business data.
|
||||
|
||||
Database rows, `campaign-artifacts/` objects, and the encryption/key service are
|
||||
one coordinated backup and recovery boundary. After any partial restore, pause
|
||||
delivery, run a dry inventory, reconcile Campaign recovery operations in Ops,
|
||||
and verify referenced object hashes before workers resume.
|
||||
@@ -65,6 +65,24 @@ Before the first live send for a sender domain or mail-server profile:
|
||||
|
||||
## Outcome Handling
|
||||
|
||||
Each real Campaign job delivery is represented by a Core recovery-ledger
|
||||
operation before the worker claims the job or invokes Mail, Postbox, or print.
|
||||
Synchronous batches use the same boundary after their batch-wide preflight.
|
||||
Explicit test/resend actions and post-acceptance IMAP appends use separate
|
||||
action-fenced operations. Operations store only opaque IDs, digests, channel
|
||||
policy, and bounded status evidence. A verified acceptance becomes
|
||||
`succeeded`, a definitive pre-effect or provider rejection becomes `rejected`,
|
||||
and uncertain or stranded effects remain `outcome_unknown` or
|
||||
`recovery_required` in Ops. Campaign jobs, message actions, and channel-attempt
|
||||
records remain the business source of truth.
|
||||
|
||||
For SMTP and Sent-folder APPEND, Campaign also passes the stable job/action and
|
||||
attempt identifier into Mail. Mail establishes its own provider-bound recovery
|
||||
fence after profile authorization and policy checks but before network I/O.
|
||||
This nested ownership is intentional: Campaign proves its business transition,
|
||||
while Mail proves the transport effect. Neither layer replays a completed or
|
||||
unknown provider attempt merely to repair the other layer's state.
|
||||
|
||||
- `smtp_accepted`: Do not retry. If IMAP append is enabled and pending, run or
|
||||
enqueue the append action.
|
||||
- `failed_temporary`: Retry explicitly after checking the error and retry count.
|
||||
@@ -104,6 +122,65 @@ bed where possible:
|
||||
- IMAP append failure after SMTP acceptance.
|
||||
- Worker restart with queued, claimed, and sending jobs.
|
||||
|
||||
For the maintained loopback baseline, run
|
||||
`dev/mail-testbed/run_campaign_acceptance.py`. It proves the public Campaign
|
||||
path for SMTP acceptance, IMAP append, repeat-send blocking, an SMTP connection
|
||||
failure before transmission, an explicit SMTP authentication rejection, an
|
||||
explicit temporary `451` response after DATA, partial RCPT refusal, a
|
||||
connection loss after complete DATA, and an IMAP authentication rejection
|
||||
after SMTP acceptance. Its evidence is an allowlisted classification/count
|
||||
projection; raw provider diagnostics and transport/account identifiers are
|
||||
deliberately excluded.
|
||||
|
||||
The runner also terminates a dedicated OS process executing the registered
|
||||
Campaign send task after complete DATA, then invokes the task in a fresh
|
||||
process. The unfinished durable attempt must become `outcome_unknown` and the
|
||||
endpoint must observe no second connection or DATA transaction. This covers
|
||||
the worker task/process boundary but not a broker or daemon.
|
||||
|
||||
Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` for the maintained
|
||||
Redis/Celery delivery and broker redelivery boundary. It starts an isolated
|
||||
Redis Compose service and real Celery workers, kills the first solo worker after complete DATA while the
|
||||
late-ack task is unacknowledged, and requires the same task identity to reach a
|
||||
replacement worker after Redis visibility recovery. Passing evidence also
|
||||
requires durable `outcome_unknown`, an empty broker queue/unacked set, and
|
||||
exactly one SMTP connection and DATA transaction. Raw worker logs and task,
|
||||
database, endpoint, and credential identifiers are never retained.
|
||||
|
||||
That second runner proves local runner-supervised process replacement, not the
|
||||
production process manager. Repeat the worker-loss drill under the selected
|
||||
systemd, container, Kubernetes, or other production supervisor and the target
|
||||
Redis/SMTP infrastructure before deployment approval.
|
||||
|
||||
## Shared Build Artifacts
|
||||
|
||||
Generated EML is stored through Core's configured object-storage backend under
|
||||
opaque Campaign-owned keys. The job records expected byte size, SHA-256 digest,
|
||||
and Message-ID. A worker on another node must retrieve and verify those values
|
||||
before attempting delivery.
|
||||
|
||||
- Do not expose object keys to ordinary campaign users or copy them into
|
||||
business fields.
|
||||
- A build failure deletes objects written before the database transaction can
|
||||
commit.
|
||||
- Retention starts a job-fenced forward-recovery operation before deletion,
|
||||
commits metadata changes in the Campaign-owned boundary, and independently
|
||||
probes the original locator before reporting success. An unavailable backend
|
||||
leaves an outcome-unknown operation; a deletion/metadata mismatch becomes
|
||||
recovery-required in Ops.
|
||||
- A hard process loss between object creation and metadata commit can leave an
|
||||
orphan object. Use the operator-only, dry-run-first Campaign artifact
|
||||
reconciler documented in `CAMPAIGN_BUILD_RECOVERY.md`; it scans one bounded
|
||||
tenant-prefix page, enforces a minimum 24-hour grace period, protects active
|
||||
build fences, and rechecks committed EML and print-output references before
|
||||
deletion.
|
||||
- Restore Campaign rows, object storage, and the encryption key to one
|
||||
coordinated recovery point before resuming workers.
|
||||
|
||||
For a scaled-runtime drill, build on one API replica, consume from another
|
||||
worker, compare the stored evidence, and inject storage failures during build
|
||||
and retention.
|
||||
|
||||
## Reporting Checks
|
||||
|
||||
- Partial delivery must show accepted, failed, and unknown counts separately.
|
||||
|
||||
+121
-16
@@ -30,12 +30,13 @@ The shorter task documents remain useful companions:
|
||||
|
||||
## What Campaign is for
|
||||
|
||||
Campaign turns governed source data into individually built messages and then
|
||||
controls their review, delivery, and evidence. It is intentionally a
|
||||
composition module: it demonstrates how one user journey can use optional Mail,
|
||||
Files, Addresses, and Notifications capabilities alongside Core access/audit
|
||||
infrastructure without copying ownership from those modules. Policy may consume
|
||||
Campaign context through a narrow capability; Campaign does not import Policy.
|
||||
Campaign turns governed source data into individually built messages or
|
||||
printable output and then controls their review, delivery, and evidence. It is
|
||||
intentionally a composition module: it demonstrates how one user journey can
|
||||
use optional Mail, Files, Addresses, Distribution Lists, Templates, Postbox,
|
||||
and Notifications capabilities alongside Core access/audit infrastructure
|
||||
without copying ownership from those modules. Policy may consume Campaign
|
||||
context through a narrow capability; Campaign does not import Policy.
|
||||
|
||||
Campaign owns:
|
||||
|
||||
@@ -66,9 +67,9 @@ The supported process is a controlled progression, not a single "send" call:
|
||||
create/edit
|
||||
-> validate and resolve policy/integrations
|
||||
-> review warnings and blockers
|
||||
-> build exact recipient messages
|
||||
-> build exact recipient messages and/or printable artifacts
|
||||
-> complete review and queue
|
||||
-> SMTP attempt per job
|
||||
-> selected Mail, Postbox, or print effect per job
|
||||
-> optional IMAP append per accepted job
|
||||
-> report, retry, reconcile, or correct
|
||||
-> archive when no active/uncertain delivery remains
|
||||
@@ -135,19 +136,63 @@ just the authoring form:
|
||||
5. Confirm attachment behavior when a rule matches no files, ZIP/password
|
||||
behavior, and any recipient-specific files.
|
||||
6. Record review completion and the inspected message keys through the review
|
||||
surface. The current baseline does not persist a distinct approve/reject
|
||||
decision or review reason. If content, recipients, attachment inputs, owner
|
||||
context, or non-secret transport identity changes, revalidate and rebuild.
|
||||
surface. Validation, build, review, and exception evidence records the actor,
|
||||
timestamp, and immutable build token/message digest where applicable. If
|
||||
content, recipients, attachment inputs, owner context, or non-secret
|
||||
transport identity changes, revalidate and rebuild.
|
||||
|
||||
The Review & Send surface separates three kinds of attention. Critical
|
||||
blockers must be corrected before delivery, individual review items require a
|
||||
recorded message decision, and non-critical group items may be acknowledged
|
||||
together after individual review is complete. Each warning or blocker names
|
||||
the required action, the responsible role, and the workspace to open. The
|
||||
review summary keeps reviewed and remaining counts visible; a completed review
|
||||
acknowledges the group items and remains bound to the current build token.
|
||||
|
||||
This evidence is the Campaign input to separation-of-duties policy. Generic
|
||||
approve/reject chains, delegation, substitutions, escalation, and signatures
|
||||
belong to the optional Approvals capability. Campaign must not claim an
|
||||
approval merely because validation, building, or message review completed.
|
||||
|
||||
When a campaign has an Approval request reference, mock and real delivery
|
||||
resolve `approvals.requests` and require an approved request for the exact
|
||||
`campaign_version` subject and current version id. A missing Approvals module,
|
||||
unknown request, pending/rejected chain, or approval for an older version blocks
|
||||
delivery with an explicit requirement. Campaign stores the approval reference,
|
||||
not Approval tables; changing the campaign version requires a new exact-subject
|
||||
approval.
|
||||
|
||||
Normal readers and reviewers see business state and safe evidence. Process-local
|
||||
paths, storage keys, worker claim tokens, and raw provider diagnostics require
|
||||
the dedicated diagnostic permission and must not leak through ordinary campaign,
|
||||
version, job, or report responses.
|
||||
|
||||
The current Campaign Report Web UI additionally requires recipient-read access
|
||||
and does not yet hide every action control that the actor lacks. The server
|
||||
still authorizes each action, but an aggregate-only reader UI remains open
|
||||
work; do not promise that experience from `campaigns:report:read` alone.
|
||||
Campaign now provides a separate aggregate **Reports** surface for readers with
|
||||
`campaigns:report:read` and access to the campaign. It loads only the safe
|
||||
aggregate projections, applies small-cell suppression, and offers no recipient
|
||||
rows, drill-down, filtering, export, or delivery actions. The recipient-aware
|
||||
**Campaign Report** still requires recipient-read access and does not yet hide
|
||||
every action control that the actor lacks. The server authorizes each action,
|
||||
but permission-aware action visibility on that detailed surface remains open
|
||||
work; do not confuse it with the aggregate reader experience.
|
||||
|
||||
The module-local aggregate surface remains at `/campaigns/reports`. When the
|
||||
optional Reporting module is enabled, Campaign also contributes the same
|
||||
recipient-free projection as the `campaigns/delivery-outcomes` report provider.
|
||||
Reporting owns `/reports`, records the run purpose, effective audience, source
|
||||
campaign/version revision, privacy transformations, retention, actor/time,
|
||||
output hash, and export history, and applies Policy before returning the
|
||||
result. Campaign does not register a fallback `/reports` route when Reporting
|
||||
is absent.
|
||||
|
||||
The detailed Campaign Report includes the selected campaign's effective
|
||||
retention policy, its system/tenant/owner/campaign provenance, and the current
|
||||
evidence state. It distinguishes retained, redacted, expired, partially
|
||||
minimized, unavailable, and not-applicable source JSON, stored report detail,
|
||||
generated EML, and Postbox-copy evidence. When Policy is absent, the report
|
||||
shows the platform defaults and explicitly warns that automated retention
|
||||
enforcement is unavailable. Retention removes or minimizes detail; aggregate
|
||||
counters and audit references may remain so outcomes can still be explained.
|
||||
|
||||
### Deliver and resolve outcomes
|
||||
|
||||
@@ -352,6 +397,12 @@ After restore, keep outbound delivery paused until queue/attempt state and
|
||||
provider evidence have been reconciled; never let restored accepted jobs send
|
||||
again merely because a queue message was lost.
|
||||
|
||||
For unreferenced generated objects after process loss, platform operators first
|
||||
run the bounded Campaign artifact inventory in dry-run mode. Apply only an
|
||||
inspected page with a unique incident idempotency key. The cleanup keeps exact
|
||||
keys out of ordinary Campaign responses, does not clear database references,
|
||||
and leaves storage failures in the Core recovery ledger for explicit retry.
|
||||
|
||||
### Incident handling
|
||||
|
||||
1. Pause new delivery when duplicate or unknown effects are possible.
|
||||
@@ -376,6 +427,8 @@ Current principal contracts include:
|
||||
| `files.campaign_attachments` 0.1.x | Files -> Campaign | Select/materialize governed file versions and preserve campaign usage/evidence |
|
||||
| `addresses.lookup` 0.1.x | Addresses -> Campaign | Optional address suggestions |
|
||||
| `addresses.recipient_source` 0.1.x | Addresses -> Campaign | Optional versioned recipient-source snapshots |
|
||||
| `dist_lists.source` / `dist_lists.expand` 0.1.x | Distribution Lists -> Campaign | Discover, preview, and freeze reusable audiences without importing module internals |
|
||||
| `templates.catalog` / `templates.renderer` 0.1.x | Templates -> Campaign | Select compatible published printable templates and produce deterministic, evidence-bearing artifacts |
|
||||
| `campaigns.access` 0.1.x | Campaign -> platform | Explain campaign access/existence without exporting ORM objects |
|
||||
| `campaigns.mail_policy_context` 0.1.x | Campaign -> Mail | Resolve campaign tenant/owner context for Mail policy |
|
||||
| `campaigns.delivery_tasks` 0.1.x | Campaign -> workers | Execute narrow queued send/append tasks |
|
||||
@@ -385,6 +438,58 @@ Breaking payload or ownership changes require an interface-version bump and a
|
||||
release-composition alignment gate. Optional absence must be tested physically,
|
||||
not only hidden in navigation.
|
||||
|
||||
### Reusable Distribution Lists
|
||||
|
||||
When Distribution Lists is available, Recipient data offers a separate import
|
||||
dialog. The author selects a visible list revision, supplies declared
|
||||
parameters, requests candidate channels, and previews included, excluded,
|
||||
stale, ambiguous, suppressed, policy-blocked, and provider-unavailable results.
|
||||
The final action freezes an idempotent Distribution Lists snapshot and copies
|
||||
the resulting rows into the editable Campaign version.
|
||||
|
||||
Each copied row retains the list and revision IDs, definition and expansion
|
||||
hashes, snapshot ID, source entry IDs, provider references, channel candidates,
|
||||
the explicitly selected primary route, optional fallback, and the decision
|
||||
explanation. Campaign-only fields, attachment rules, review state, and outcomes
|
||||
remain local to Campaign and never mutate the reusable list.
|
||||
|
||||
A later list revision only raises a drift warning. Refresh is deliberate and
|
||||
uses append or replace; saving that changed Campaign version clears prior
|
||||
validation, build, review, and execution state through the normal content
|
||||
invalidation path. Preferred or single usable candidates are preselected
|
||||
visibly; ambiguous rows must be decided before freezing. Postal and
|
||||
internal-mail routes remain active when a compatible published Templates output
|
||||
is selected.
|
||||
|
||||
### Governed hybrid and printable delivery
|
||||
|
||||
Campaign supports Mail, Postbox, printable output, and bounded ordered
|
||||
fallbacks without making any of those provider modules mandatory. Opt-in and
|
||||
channel-preference data are inputs to the visible routing decision; they never
|
||||
silently cause duplicate delivery.
|
||||
|
||||
For a printable route, select a published label, envelope, serial-letter,
|
||||
form-letter, list-layout, or generic template on the Template page. Validation
|
||||
checks the selected revision, output format, and required fields. Build sends
|
||||
one deterministic item collection to `templates.renderer`, records template,
|
||||
input, output, actor, route, and artifact hashes, and stores the resulting
|
||||
artifact through Files when configured. The review stage exposes that exact
|
||||
artifact and its hashes before execution.
|
||||
|
||||
Each recipient job records an idempotent print acceptance attempt for its item
|
||||
in the frozen artifact. `mail_then_print` and `postbox_then_print` invoke print
|
||||
only after a confirmed rejection before acceptance. An accepted or
|
||||
outcome-unknown digital effect never falls through to print because that could
|
||||
produce duplicate delivery. Reports and CSV exports include route provenance,
|
||||
print state, attempts, artifact reference, and hashes.
|
||||
|
||||
Without Templates, Campaign still loads and Mail/Postbox authoring remains
|
||||
available; validation explains why a configured print route cannot proceed.
|
||||
Without Files, Templates may return a bounded artifact instead of a managed
|
||||
file. Campaign copies that payload into shared object storage and exposes it
|
||||
through the Campaign ACL plus `campaigns:recipient:read`; it never redistributes
|
||||
the broader Templates URL. A print-only Campaign does not require Mail or Postbox.
|
||||
|
||||
### External API expectations
|
||||
|
||||
- Tenant and campaign access are evaluated for every operation.
|
||||
@@ -465,7 +570,7 @@ the current baseline:
|
||||
- the final audited **test / single send / single resend** semantics;
|
||||
- reusable SMTP batch sessions and their measured throughput benefit;
|
||||
- durable, idempotent Campaign report delivery through a Mail-owned outbox
|
||||
([`govoplan-mail#17`](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17));
|
||||
([`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17));
|
||||
- a fully packaged one-command Campaign reference composition with production
|
||||
policy presets and target-provider certification;
|
||||
- function-bound Postbox delivery (stage 2 of the reference program);
|
||||
|
||||
@@ -10,7 +10,11 @@ scenario catalogue lives in `examples/README.md`; committed fixture files should
|
||||
be added under `examples/` only when they validate against the current campaign
|
||||
schema and are safe to run in non-production environments.
|
||||
|
||||
- simple announcement with one active recipient and no attachments
|
||||
- [`simple-announcement`](../examples/simple-announcement/campaign.json), a
|
||||
credential-free campaign with one active recipient and no attachments; its
|
||||
automated acceptance check physically blocks Mail and Files imports, denies
|
||||
network connections, and validates/builds from an unrelated temporary
|
||||
workspace
|
||||
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
|
||||
notification fields
|
||||
- campaign with global attachments and recipient-specific attachment rules
|
||||
@@ -22,7 +26,9 @@ schema and are safe to run in non-production environments.
|
||||
- campaign with blocked recipients or attachment errors that must not be sent
|
||||
- mock delivery campaign that captures SMTP and IMAP append messages in the mail
|
||||
development mailbox
|
||||
- real non-production delivery campaign against the GreenMail test bed
|
||||
- [`greenmail-delivery`](../examples/greenmail-delivery/campaign.json), a
|
||||
credential-free real-delivery Campaign materialized with a temporary
|
||||
Mail-owned profile by the loopback acceptance runner
|
||||
|
||||
## Fixture Rules
|
||||
|
||||
@@ -41,12 +47,31 @@ Before tagging a campaign release:
|
||||
|
||||
- Review `examples/README.md` and update the scenario catalogue when a release
|
||||
adds or removes delivery behavior.
|
||||
- Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'` and
|
||||
retain its isolated validate/build result as release evidence.
|
||||
- Run core module permutation tests with campaign installed both with and
|
||||
without files/mail.
|
||||
- Validate and build each maintained example campaign.
|
||||
- Run the mock delivery example when the mail development mailbox capability is
|
||||
enabled.
|
||||
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
|
||||
- Run `dev/mail-testbed/run_campaign_acceptance.py` and retain its bounded JSON
|
||||
projection. It must show one SMTP acceptance, one IMAP append, no duplicate
|
||||
effect from a repeated ordinary send, matching Campaign report/audit state,
|
||||
and no resolved transport material in Campaign JSON or its execution
|
||||
snapshot. Its controlled endpoint evidence must also show explicit SMTP 451,
|
||||
partial RCPT refusal, post-DATA ambiguity, and task-process interruption
|
||||
classifications without retaining addresses or provider diagnostics.
|
||||
- Treat the task-process restart proof separately from the still-open
|
||||
Redis/Celery broker redelivery and daemon-supervision check; the coverage
|
||||
projection must keep `celery_broker_redelivery` false.
|
||||
- Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` as a separate
|
||||
destructive worker-loss check. Retain its bounded evidence only when the same
|
||||
broker task is observed at both workers, the durable state is
|
||||
`outcome_unknown`, broker queue/unacked counts are zero, and the controlled
|
||||
SMTP endpoint observed one connection and one DATA transaction. This closes
|
||||
local Redis/Celery redelivery coverage, while production supervisor and
|
||||
target-provider coverage remain false until separately tested.
|
||||
- Confirm reusable mail profile selection is revalidated after campaign owner
|
||||
transfer.
|
||||
- Confirm every inline SMTP/IMAP field is rejected on import/write, omitted
|
||||
|
||||
@@ -49,7 +49,7 @@ Non-dry Campaign report email currently fails closed. It must not bypass the
|
||||
durable job/effect model through a direct SMTP call. Re-enabling it requires the
|
||||
Mail-owned idempotent outbox, attempt, unknown-outcome, and reconciliation path
|
||||
tracked in
|
||||
[`govoplan-mail#17`](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17).
|
||||
[`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17).
|
||||
Report generation and dry-run validation remain separate from an external
|
||||
effect; recipient-level exports require recipient-export authorization.
|
||||
|
||||
|
||||
@@ -64,6 +64,14 @@ warning and lets the user reopen the import dialog with that source preselected.
|
||||
The user still chooses append or replace; campaign should not silently rewrite
|
||||
recipient rows.
|
||||
|
||||
Distribution Lists is a separate, provider-neutral audience boundary. Campaign
|
||||
uses `dist_lists.source` and `dist_lists.expand` to preview and freeze mixed
|
||||
email, postal, internal-mail, and portal candidates. It stores exact list,
|
||||
revision, source-entry, provider, policy, route, exclusion, and expansion-hash
|
||||
evidence with the Campaign version. Addresses remains the contact/contact-point
|
||||
owner; Distribution Lists remains the reusable audience owner; Campaign owns
|
||||
only its copied recipients, enrichment, route choices, review, and outcomes.
|
||||
|
||||
## Non-Goals For Campaign
|
||||
|
||||
Campaign should not become the global address book. It should not own:
|
||||
|
||||
+12
-7
@@ -9,7 +9,7 @@ campaign schema and do not require production data.
|
||||
|
||||
| Scenario | Required Modules | Release Check |
|
||||
| --- | --- | --- |
|
||||
| `simple-announcement` | core, access, campaigns | Validate and build one active recipient without attachments. |
|
||||
| [`simple-announcement`](simple-announcement/campaign.json) | core, access, campaigns | Validate and build one active recipient without attachments while Mail and Files are absent. |
|
||||
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
|
||||
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
|
||||
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
|
||||
@@ -19,7 +19,7 @@ campaign schema and do not require production data.
|
||||
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
|
||||
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
|
||||
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
|
||||
| `greenmail-delivery` | core, access, campaigns, mail | Send no-attachment, normal attachment, and ZIP attachment variants through `dev/mail-testbed`. |
|
||||
| [`greenmail-delivery`](greenmail-delivery/campaign.json) | core, access, audit, campaigns, mail | Run a credential-free Campaign through a Mail-owned profile, GreenMail SMTP/IMAP, report/audit checks, repeat-send protection, and bounded failure drills. |
|
||||
|
||||
## Fixture Rules
|
||||
|
||||
@@ -37,9 +37,14 @@ campaign schema and do not require production data.
|
||||
Before a release tag:
|
||||
|
||||
1. Run module permutation startup checks from core.
|
||||
2. Validate every committed example fixture against the current campaign schema.
|
||||
3. Build exact messages for each fixture.
|
||||
4. Run the mock-delivery example when the dev mailbox capability is enabled.
|
||||
5. Run `dev/mail-testbed/run_transport_smoke.py`.
|
||||
6. Execute the delivery checklist in
|
||||
2. Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'`
|
||||
from this repository. The acceptance test copies each maintained fixture to
|
||||
an unrelated temporary workspace before using Campaign's public loader,
|
||||
validator, and message builder.
|
||||
3. Validate every committed example fixture against the current campaign schema.
|
||||
4. Build exact messages for each fixture.
|
||||
5. Run the mock-delivery example when the dev mailbox capability is enabled.
|
||||
6. Run `dev/mail-testbed/run_transport_smoke.py` for low-level transport and attachment variants.
|
||||
7. Run `dev/mail-testbed/run_campaign_acceptance.py` for the Campaign journey and bounded evidence.
|
||||
8. Execute the delivery checklist in
|
||||
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "greenmail-delivery",
|
||||
"name": "GreenMail delivery acceptance",
|
||||
"description": "Credential-free Campaign fixture for the local SMTP/IMAP acceptance test bed.",
|
||||
"mode": "test"
|
||||
},
|
||||
"fields": [
|
||||
{
|
||||
"name": "display_name",
|
||||
"type": "string",
|
||||
"label": "Display name",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"name": "acceptance_run",
|
||||
"type": "string",
|
||||
"label": "Acceptance run",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"server": {
|
||||
"mail_profile_id": "00000000-0000-4000-8000-000000000001"
|
||||
},
|
||||
"recipients": {
|
||||
"from": [
|
||||
{
|
||||
"email": "campaign-test@govoplan.test",
|
||||
"name": "GovOPlaN acceptance",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"allow_individual_to": true
|
||||
},
|
||||
"template": {
|
||||
"subject": "[GovOPlaN acceptance ${acceptance_run}] Campaign delivery",
|
||||
"text": "Hello ${display_name},\n\nThis is an isolated GovOPlaN Campaign SMTP/IMAP acceptance message.\n",
|
||||
"body_mode": "text"
|
||||
},
|
||||
"attachments": {
|
||||
"base_path": ".",
|
||||
"send_without_attachments_behavior": "continue",
|
||||
"global": []
|
||||
},
|
||||
"entries": {
|
||||
"inline": [
|
||||
{
|
||||
"id": "greenmail-recipient",
|
||||
"to": [
|
||||
{
|
||||
"email": "campaign-test@govoplan.test",
|
||||
"name": "GreenMail recipient",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"fields": {
|
||||
"display_name": "GreenMail recipient",
|
||||
"acceptance_run": "fixture"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"validation_policy": {
|
||||
"missing_email": "block",
|
||||
"template_error": "block"
|
||||
},
|
||||
"delivery": {
|
||||
"rate_limit": {
|
||||
"messages_per_minute": 60
|
||||
},
|
||||
"retry": {
|
||||
"max_attempts": 3,
|
||||
"backoff_seconds": [
|
||||
1,
|
||||
5,
|
||||
30
|
||||
]
|
||||
},
|
||||
"imap_append_sent": {
|
||||
"enabled": true,
|
||||
"folder": "Sent"
|
||||
}
|
||||
},
|
||||
"status_tracking": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"scenario": "greenmail-delivery",
|
||||
"campaign_file": "campaign.json",
|
||||
"required_modules": [
|
||||
"core",
|
||||
"access",
|
||||
"audit",
|
||||
"campaigns",
|
||||
"mail"
|
||||
],
|
||||
"required_capabilities": [
|
||||
"mail.campaign_delivery"
|
||||
],
|
||||
"transport": "local GreenMail SMTP/IMAP test bed",
|
||||
"credentials": "local environment only; never copied into Campaign JSON or evidence",
|
||||
"expected": {
|
||||
"entries_count": 1,
|
||||
"built_count": 1,
|
||||
"smtp_accepted_count": 1,
|
||||
"imap_appended_count": 1
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "simple-announcement",
|
||||
"name": "Simple announcement",
|
||||
"description": "Credential-free release fixture for Campaign validation and message building.",
|
||||
"mode": "test"
|
||||
},
|
||||
"fields": [
|
||||
{
|
||||
"name": "display_name",
|
||||
"type": "string",
|
||||
"label": "Display name",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"recipients": {
|
||||
"from": [
|
||||
{
|
||||
"email": "announcements@example.test",
|
||||
"name": "GovOPlaN Example",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"allow_individual_to": true
|
||||
},
|
||||
"template": {
|
||||
"subject": "Planned service maintenance for ${display_name}",
|
||||
"text": "Hello ${display_name},\n\nThe example service will be unavailable during the announced maintenance window.\n\nThis message was built locally and was not sent.\n",
|
||||
"body_mode": "text"
|
||||
},
|
||||
"attachments": {
|
||||
"base_path": ".",
|
||||
"send_without_attachments_behavior": "continue",
|
||||
"global": []
|
||||
},
|
||||
"entries": {
|
||||
"inline": [
|
||||
{
|
||||
"id": "example-recipient",
|
||||
"to": [
|
||||
{
|
||||
"email": "recipient@example.test",
|
||||
"name": "Example Recipient",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"fields": {
|
||||
"display_name": "Example Recipient"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"id": "simple-announcement",
|
||||
"campaign_file": "campaign.json",
|
||||
"required_modules": [
|
||||
"core",
|
||||
"access",
|
||||
"campaigns"
|
||||
],
|
||||
"absent_optional_modules": [
|
||||
"files",
|
||||
"mail"
|
||||
],
|
||||
"external_effects": "forbidden",
|
||||
"expected": {
|
||||
"campaign_id": "simple-announcement",
|
||||
"entries_count": 1,
|
||||
"built_count": 1,
|
||||
"queueable_count": 1,
|
||||
"attachment_count": 0,
|
||||
"subject": "Planned service maintenance for Example Recipient"
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@govoplan/campaign-webui",
|
||||
"version": "0.1.10",
|
||||
"version": "0.1.12",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "webui/src/index.ts",
|
||||
|
||||
+2
-2
@@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "govoplan-campaign"
|
||||
version = "0.1.10"
|
||||
version = "0.1.12"
|
||||
description = "GovOPlaN campaigns module with backend and WebUI integration."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { file = "LICENSE" }
|
||||
authors = [{ name = "GovOPlaN" }]
|
||||
dependencies = [
|
||||
"govoplan-core>=0.1.8",
|
||||
"govoplan-core>=0.1.14",
|
||||
"jsonschema>=4,<5",
|
||||
"pydantic>=2,<3",
|
||||
"SQLAlchemy>=2,<3",
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from typing import Mapping
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.approvals import (
|
||||
ApprovalRequestCreateCommand,
|
||||
ApprovalRequestRef,
|
||||
ApprovalStepDefinition,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
ApprovalGateUnavailable,
|
||||
approvals_integration,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.execution import ensure_execution_snapshot
|
||||
|
||||
|
||||
APPROVAL_GATE_KEY = "approval_gate"
|
||||
SUBJECT_MODULE = "campaigns"
|
||||
SUBJECT_TYPE = "campaign_execution"
|
||||
|
||||
|
||||
class CampaignApprovalGateError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _TenantPrincipal:
|
||||
tenant_id: str
|
||||
account_id: str | None = None
|
||||
|
||||
|
||||
def campaign_approval_gate(version: CampaignVersion) -> dict[str, object] | None:
|
||||
state = version.editor_state if isinstance(version.editor_state, dict) else {}
|
||||
gate = state.get(APPROVAL_GATE_KEY)
|
||||
return dict(gate) if isinstance(gate, dict) else None
|
||||
|
||||
|
||||
def request_campaign_approval(
|
||||
session: Session,
|
||||
principal: object,
|
||||
*,
|
||||
campaign: Campaign,
|
||||
version: CampaignVersion,
|
||||
title: str,
|
||||
description: str | None,
|
||||
steps: tuple[ApprovalStepDefinition, ...],
|
||||
idempotency_key: str,
|
||||
template_id: str | None = None,
|
||||
template_revision: int | None = None,
|
||||
unique_actors_across_steps: bool = True,
|
||||
expires_at: datetime | None = None,
|
||||
policy_refs: tuple[str, ...] = (),
|
||||
) -> ApprovalRequestRef:
|
||||
if campaign.tenant_id != str(getattr(principal, "tenant_id", "") or ""):
|
||||
raise CampaignApprovalGateError("Campaign approval tenant mismatch.")
|
||||
if version.campaign_id != campaign.id:
|
||||
raise CampaignApprovalGateError("Campaign approval version mismatch.")
|
||||
snapshot = ensure_execution_snapshot(session, version)
|
||||
digest = str(version.execution_snapshot_hash or "")
|
||||
if len(digest) != 64:
|
||||
raise CampaignApprovalGateError(
|
||||
"Build a valid Campaign execution snapshot before requesting approval."
|
||||
)
|
||||
subject_version = _subject_version(version, snapshot.build_token)
|
||||
evidence_actors = _campaign_evidence_actors(campaign, version)
|
||||
try:
|
||||
request = approvals_integration().create_request(
|
||||
session,
|
||||
principal,
|
||||
command=ApprovalRequestCreateCommand(
|
||||
title=title,
|
||||
description=description,
|
||||
subject_module=SUBJECT_MODULE,
|
||||
subject_type=SUBJECT_TYPE,
|
||||
subject_id=version.id,
|
||||
subject_version=subject_version,
|
||||
subject_digest=digest,
|
||||
steps=steps,
|
||||
separation_of_duties=True,
|
||||
unique_actors_across_steps=unique_actors_across_steps,
|
||||
expires_at=expires_at,
|
||||
policy_refs=policy_refs,
|
||||
evidence_actors=evidence_actors,
|
||||
template_id=template_id,
|
||||
template_revision=template_revision,
|
||||
metadata={
|
||||
"campaign_id": campaign.id,
|
||||
"campaign_version_number": version.version_number,
|
||||
"execution_snapshot_version": snapshot.snapshot_version,
|
||||
},
|
||||
),
|
||||
idempotency_key=idempotency_key,
|
||||
)
|
||||
except ApprovalGateUnavailable as exc:
|
||||
raise CampaignApprovalGateError(str(exc)) from exc
|
||||
state = copy.deepcopy(version.editor_state or {})
|
||||
state[APPROVAL_GATE_KEY] = {
|
||||
"request_id": request.id,
|
||||
"request_revision": request.revision,
|
||||
"subject_version": subject_version,
|
||||
"subject_digest": digest,
|
||||
"requested_at": datetime.now(UTC).isoformat(),
|
||||
"requested_by_user_id": _actor_id(principal),
|
||||
}
|
||||
version.editor_state = state
|
||||
session.add(version)
|
||||
session.flush()
|
||||
return request
|
||||
|
||||
|
||||
def assert_campaign_approval(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
version: CampaignVersion,
|
||||
) -> None:
|
||||
gate = campaign_approval_gate(version)
|
||||
if gate is None:
|
||||
return
|
||||
snapshot = ensure_execution_snapshot(session, version)
|
||||
digest = str(version.execution_snapshot_hash or "")
|
||||
subject_version = _subject_version(version, snapshot.build_token)
|
||||
if (
|
||||
gate.get("subject_digest") != digest
|
||||
or gate.get("subject_version") != subject_version
|
||||
):
|
||||
raise CampaignApprovalGateError(
|
||||
"Campaign execution changed after approval was requested. Request approval for the current build."
|
||||
)
|
||||
request_id = str(gate.get("request_id") or "").strip()
|
||||
if not request_id:
|
||||
raise CampaignApprovalGateError(
|
||||
"Campaign approval gate has no request reference."
|
||||
)
|
||||
try:
|
||||
check = approvals_integration().check_approved(
|
||||
session,
|
||||
_TenantPrincipal(tenant_id=tenant_id),
|
||||
request_id=request_id,
|
||||
subject_module=SUBJECT_MODULE,
|
||||
subject_type=SUBJECT_TYPE,
|
||||
subject_id=version.id,
|
||||
subject_version=subject_version,
|
||||
subject_digest=digest,
|
||||
)
|
||||
except (ApprovalGateUnavailable, LookupError, ValueError) as exc:
|
||||
raise CampaignApprovalGateError(str(exc)) from exc
|
||||
if not check.approved:
|
||||
raise CampaignApprovalGateError(
|
||||
f"Campaign delivery requires Approval request {request_id}, which is {check.state}."
|
||||
)
|
||||
|
||||
|
||||
def campaign_approval_status(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
version: CampaignVersion,
|
||||
) -> dict[str, object]:
|
||||
gate = campaign_approval_gate(version)
|
||||
integration = approvals_integration()
|
||||
if gate is None:
|
||||
return {
|
||||
"configured": False,
|
||||
"available": integration.available,
|
||||
"approved": False,
|
||||
"state": "not_required",
|
||||
"explanation": None,
|
||||
}
|
||||
try:
|
||||
assert_campaign_approval(session, tenant_id=tenant_id, version=version)
|
||||
except CampaignApprovalGateError as exc:
|
||||
return {
|
||||
**gate,
|
||||
"configured": True,
|
||||
"available": integration.available,
|
||||
"approved": False,
|
||||
"state": "unavailable" if not integration.available else "pending",
|
||||
"explanation": str(exc),
|
||||
}
|
||||
return {
|
||||
**gate,
|
||||
"configured": True,
|
||||
"available": True,
|
||||
"approved": True,
|
||||
"state": "approved",
|
||||
"explanation": None,
|
||||
}
|
||||
|
||||
|
||||
def clear_campaign_approval_gate(version: CampaignVersion) -> None:
|
||||
state = copy.deepcopy(version.editor_state or {})
|
||||
state.pop(APPROVAL_GATE_KEY, None)
|
||||
version.editor_state = state
|
||||
|
||||
|
||||
def _campaign_evidence_actors(
|
||||
campaign: Campaign,
|
||||
version: CampaignVersion,
|
||||
) -> Mapping[str, tuple[str, ...]]:
|
||||
validation = (
|
||||
version.validation_summary
|
||||
if isinstance(version.validation_summary, dict)
|
||||
else {}
|
||||
)
|
||||
build = version.build_summary if isinstance(version.build_summary, dict) else {}
|
||||
editor = version.editor_state if isinstance(version.editor_state, dict) else {}
|
||||
review = (
|
||||
editor.get("review_send") if isinstance(editor.get("review_send"), dict) else {}
|
||||
)
|
||||
review_decisions = (
|
||||
review.get("issue_decisions")
|
||||
if isinstance(review.get("issue_decisions"), list)
|
||||
else []
|
||||
)
|
||||
values = {
|
||||
"author": (campaign.created_by_user_id,),
|
||||
"owner": (campaign.owner_user_id,),
|
||||
"validator": (
|
||||
validation.get("validated_by_user_id"),
|
||||
version.locked_by_user_id,
|
||||
),
|
||||
"builder": (build.get("built_by_user_id"),),
|
||||
"reviewer": (
|
||||
review.get("updated_by_user_id"),
|
||||
*(
|
||||
item.get("actor_user_id")
|
||||
for item in review_decisions
|
||||
if isinstance(item, dict)
|
||||
),
|
||||
),
|
||||
}
|
||||
return {
|
||||
role: tuple(dict.fromkeys(str(item) for item in actors if item))
|
||||
for role, actors in values.items()
|
||||
if any(actors)
|
||||
}
|
||||
|
||||
|
||||
def _subject_version(version: CampaignVersion, build_token: str | None) -> str:
|
||||
return str(build_token or f"campaign-version-{version.version_number}")[:120]
|
||||
|
||||
|
||||
def _actor_id(principal: object) -> str | None:
|
||||
for name in ("account_id", "user_id", "identity_id", "membership_id"):
|
||||
value = str(getattr(principal, name, "") or "").strip()
|
||||
if value:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CampaignApprovalGateError",
|
||||
"assert_campaign_approval",
|
||||
"campaign_approval_gate",
|
||||
"campaign_approval_status",
|
||||
"clear_campaign_approval_gate",
|
||||
"request_campaign_approval",
|
||||
]
|
||||
@@ -0,0 +1,78 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from govoplan_core.core.approvals import ApprovalActorSelector, ApprovalStepDefinition
|
||||
|
||||
|
||||
class CampaignApprovalSelectorInput(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
kind: Literal["account", "group", "role", "function_assignment", "any_account"]
|
||||
value: str = Field(min_length=1, max_length=255)
|
||||
label: str | None = Field(default=None, max_length=255)
|
||||
|
||||
|
||||
class CampaignApprovalStepInput(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
key: str = Field(min_length=1, max_length=120)
|
||||
label: str = Field(min_length=1, max_length=255)
|
||||
selectors: list[CampaignApprovalSelectorInput] = Field(min_length=1, max_length=500)
|
||||
required_approvals: int = Field(default=1, ge=1, le=500)
|
||||
rejection_policy: Literal["fail_fast", "collect"] = "fail_fast"
|
||||
due_at: datetime | None = None
|
||||
signature_required: bool = False
|
||||
forbidden_evidence_roles: list[
|
||||
Literal["author", "owner", "validator", "builder", "reviewer"]
|
||||
] = Field(default_factory=list, max_length=5)
|
||||
|
||||
def to_definition(self) -> ApprovalStepDefinition:
|
||||
return ApprovalStepDefinition(
|
||||
key=self.key,
|
||||
label=self.label,
|
||||
selectors=tuple(
|
||||
ApprovalActorSelector(item.kind, item.value, item.label)
|
||||
for item in self.selectors
|
||||
),
|
||||
required_approvals=self.required_approvals,
|
||||
rejection_policy=self.rejection_policy,
|
||||
due_at=self.due_at,
|
||||
signature_required=self.signature_required,
|
||||
forbidden_evidence_roles=tuple(self.forbidden_evidence_roles),
|
||||
)
|
||||
|
||||
|
||||
class CampaignApprovalRequestInput(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
title: str = Field(
|
||||
default="Approve Campaign delivery", min_length=1, max_length=255
|
||||
)
|
||||
description: str | None = Field(default=None, max_length=10_000)
|
||||
steps: list[CampaignApprovalStepInput] = Field(default_factory=list, max_length=100)
|
||||
template_id: str | None = Field(default=None, max_length=36)
|
||||
template_revision: int | None = Field(default=None, ge=1)
|
||||
unique_actors_across_steps: bool = True
|
||||
expires_at: datetime | None = None
|
||||
policy_refs: list[str] = Field(default_factory=list, max_length=500)
|
||||
idempotency_key: str = Field(min_length=1, max_length=160)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_source(self) -> "CampaignApprovalRequestInput":
|
||||
template = self.template_id is not None or self.template_revision is not None
|
||||
if template and (self.template_id is None or self.template_revision is None):
|
||||
raise ValueError("Template id and revision must be supplied together.")
|
||||
if template and self.steps:
|
||||
raise ValueError("Use either an Approval template or inline steps.")
|
||||
if not template and not self.steps:
|
||||
raise ValueError(
|
||||
"At least one Approval step is required without a template."
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
__all__ = ["CampaignApprovalRequestInput"]
|
||||
@@ -0,0 +1,734 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any, Callable
|
||||
|
||||
from sqlalchemy import or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignJob,
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_core.core.object_storage import (
|
||||
StorageBackend,
|
||||
StorageBackendError,
|
||||
StorageObjectInfo,
|
||||
)
|
||||
from govoplan_core.core.recovery import (
|
||||
RecoveryMode,
|
||||
RecoveryOperation,
|
||||
RecoveryPlan,
|
||||
RecoveryStatus,
|
||||
TERMINAL_RECOVERY_STATUSES,
|
||||
)
|
||||
from govoplan_core.core.recovery_runtime import begin_durable_recovery_operation
|
||||
from govoplan_core.core.runtime_coordination import (
|
||||
DistributedLease,
|
||||
RuntimeIdentity,
|
||||
)
|
||||
|
||||
|
||||
CAMPAIGN_ARTIFACT_NAMESPACE = "campaign-artifacts"
|
||||
MINIMUM_GRACE_HOURS = 24
|
||||
MAXIMUM_PAGE_SIZE = 1000
|
||||
_CHECKPOINT_BATCH_SIZE = 25
|
||||
|
||||
SessionFactory = Callable[[], Session]
|
||||
|
||||
|
||||
class CampaignArtifactReconciliationError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ArtifactCandidate:
|
||||
key: str
|
||||
size_bytes: int
|
||||
modified_at: datetime
|
||||
age_seconds: int
|
||||
reason: str = "unreferenced_after_grace_period"
|
||||
disposition: str = "candidate"
|
||||
failure_type: str | None = None
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"key": self.key,
|
||||
"size_bytes": self.size_bytes,
|
||||
"modified_at": self.modified_at.isoformat(),
|
||||
"age_seconds": self.age_seconds,
|
||||
"reason": self.reason,
|
||||
"disposition": self.disposition,
|
||||
"failure_type": self.failure_type,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ArtifactInventory:
|
||||
tenant_prefix: str
|
||||
cursor: str | None
|
||||
next_cursor: str | None
|
||||
scanned_count: int
|
||||
scanned_bytes: int
|
||||
referenced_count: int
|
||||
active_build_count: int
|
||||
young_count: int
|
||||
unknown_age_count: int
|
||||
invalid_shape_count: int
|
||||
candidates: list[ArtifactCandidate]
|
||||
manifest_sha256: str
|
||||
|
||||
@property
|
||||
def candidate_bytes(self) -> int:
|
||||
return sum(candidate.size_bytes for candidate in self.candidates)
|
||||
|
||||
def response(
|
||||
self,
|
||||
*,
|
||||
apply: bool,
|
||||
status: str,
|
||||
recovery_operation_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
deleted = [
|
||||
candidate
|
||||
for candidate in self.candidates
|
||||
if candidate.disposition == "deleted"
|
||||
]
|
||||
failures = [
|
||||
candidate
|
||||
for candidate in self.candidates
|
||||
if candidate.disposition
|
||||
in {"delete_failed", "delete_outcome_unknown"}
|
||||
]
|
||||
return {
|
||||
"apply": apply,
|
||||
"status": status,
|
||||
"recovery_operation_id": recovery_operation_id,
|
||||
"tenant_prefix": self.tenant_prefix,
|
||||
"cursor": self.cursor,
|
||||
"next_cursor": self.next_cursor,
|
||||
"scanned_count": self.scanned_count,
|
||||
"scanned_bytes": self.scanned_bytes,
|
||||
"referenced_count": self.referenced_count,
|
||||
"active_build_count": self.active_build_count,
|
||||
"young_count": self.young_count,
|
||||
"unknown_age_count": self.unknown_age_count,
|
||||
"invalid_shape_count": self.invalid_shape_count,
|
||||
"candidate_count": len(self.candidates),
|
||||
"candidate_bytes": self.candidate_bytes,
|
||||
"deleted_count": len(deleted),
|
||||
"deleted_bytes": sum(candidate.size_bytes for candidate in deleted),
|
||||
"failure_count": len(failures),
|
||||
"manifest_sha256": self.manifest_sha256,
|
||||
"candidates": [candidate.as_dict() for candidate in self.candidates],
|
||||
}
|
||||
|
||||
|
||||
def campaign_artifact_inventory(
|
||||
session: Session,
|
||||
*,
|
||||
storage: StorageBackend,
|
||||
tenant_id: str,
|
||||
grace_period: timedelta,
|
||||
cursor: str | None = None,
|
||||
page_size: int = 250,
|
||||
now: datetime | None = None,
|
||||
) -> ArtifactInventory:
|
||||
observed_at = _as_utc(now or datetime.now(timezone.utc))
|
||||
if grace_period < timedelta(hours=MINIMUM_GRACE_HOURS):
|
||||
raise ValueError(
|
||||
f"Campaign artifact grace period must be at least {MINIMUM_GRACE_HOURS} hours"
|
||||
)
|
||||
bounded_page_size = max(1, min(int(page_size), MAXIMUM_PAGE_SIZE))
|
||||
prefix = _tenant_artifact_prefix(tenant_id)
|
||||
if cursor is not None and not cursor.startswith(prefix):
|
||||
raise ValueError("Campaign artifact cursor is outside the tenant namespace")
|
||||
|
||||
page = storage.list_objects(
|
||||
prefix=prefix,
|
||||
after=cursor,
|
||||
limit=bounded_page_size,
|
||||
)
|
||||
page_keys = {info.key for info in page.objects}
|
||||
referenced_keys = _referenced_artifact_keys(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
prefix=prefix,
|
||||
artifact_keys=page_keys,
|
||||
)
|
||||
active_build_ids = _active_build_ids(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
now=observed_at,
|
||||
)
|
||||
|
||||
candidates: list[ArtifactCandidate] = []
|
||||
referenced_count = 0
|
||||
active_build_count = 0
|
||||
young_count = 0
|
||||
unknown_age_count = 0
|
||||
invalid_shape_count = 0
|
||||
scanned_bytes = 0
|
||||
manifest_rows: list[dict[str, Any]] = []
|
||||
for info in page.objects:
|
||||
scanned_bytes += info.size_bytes
|
||||
build_id = _build_id_for_key(info.key, prefix=prefix)
|
||||
modified_at = _object_modified_at(info)
|
||||
manifest_rows.append(
|
||||
{
|
||||
"key_sha256": _sha256(info.key),
|
||||
"size_bytes": info.size_bytes,
|
||||
"modified_at": modified_at.isoformat() if modified_at else None,
|
||||
}
|
||||
)
|
||||
if build_id is None:
|
||||
invalid_shape_count += 1
|
||||
continue
|
||||
if info.key in referenced_keys:
|
||||
referenced_count += 1
|
||||
continue
|
||||
if build_id in active_build_ids:
|
||||
active_build_count += 1
|
||||
continue
|
||||
if modified_at is None:
|
||||
unknown_age_count += 1
|
||||
continue
|
||||
age = observed_at - modified_at
|
||||
if age < grace_period:
|
||||
young_count += 1
|
||||
continue
|
||||
candidates.append(
|
||||
ArtifactCandidate(
|
||||
key=info.key,
|
||||
size_bytes=info.size_bytes,
|
||||
modified_at=modified_at,
|
||||
age_seconds=max(0, int(age.total_seconds())),
|
||||
)
|
||||
)
|
||||
|
||||
return ArtifactInventory(
|
||||
tenant_prefix=prefix,
|
||||
cursor=cursor,
|
||||
next_cursor=page.next_cursor,
|
||||
scanned_count=len(page.objects),
|
||||
scanned_bytes=scanned_bytes,
|
||||
referenced_count=referenced_count,
|
||||
active_build_count=active_build_count,
|
||||
young_count=young_count,
|
||||
unknown_age_count=unknown_age_count,
|
||||
invalid_shape_count=invalid_shape_count,
|
||||
candidates=candidates,
|
||||
manifest_sha256=_canonical_sha256(manifest_rows),
|
||||
)
|
||||
|
||||
|
||||
def reconcile_campaign_artifacts(
|
||||
session_factory: SessionFactory,
|
||||
*,
|
||||
storage: StorageBackend,
|
||||
identity: RuntimeIdentity,
|
||||
tenant_id: str,
|
||||
apply: bool = False,
|
||||
idempotency_key: str | None = None,
|
||||
grace_period_hours: int = MINIMUM_GRACE_HOURS,
|
||||
cursor: str | None = None,
|
||||
page_size: int = 250,
|
||||
now: datetime | None = None,
|
||||
) -> dict[str, Any]:
|
||||
if apply and not (idempotency_key or "").strip():
|
||||
raise ValueError("Applied Campaign artifact cleanup requires an idempotency key")
|
||||
if grace_period_hours < MINIMUM_GRACE_HOURS:
|
||||
raise ValueError(
|
||||
f"Campaign artifact grace period must be at least {MINIMUM_GRACE_HOURS} hours"
|
||||
)
|
||||
observed_at = _as_utc(now or datetime.now(timezone.utc))
|
||||
grace_period = timedelta(hours=grace_period_hours)
|
||||
if not apply:
|
||||
with session_factory() as session:
|
||||
inventory = campaign_artifact_inventory(
|
||||
session,
|
||||
storage=storage,
|
||||
tenant_id=tenant_id,
|
||||
grace_period=grace_period,
|
||||
cursor=cursor,
|
||||
page_size=page_size,
|
||||
now=observed_at,
|
||||
)
|
||||
return inventory.response(apply=False, status="dry_run")
|
||||
|
||||
prefix = _tenant_artifact_prefix(tenant_id)
|
||||
request = {
|
||||
"tenant_id": tenant_id,
|
||||
"prefix": prefix,
|
||||
"cursor_sha256": _sha256(cursor) if cursor else None,
|
||||
"page_size": max(1, min(int(page_size), MAXIMUM_PAGE_SIZE)),
|
||||
"grace_period_hours": grace_period_hours,
|
||||
}
|
||||
recovery_start = begin_durable_recovery_operation(
|
||||
session_factory,
|
||||
identity=identity,
|
||||
module_id="campaigns",
|
||||
operation_type="artifact-orphan-reconciliation",
|
||||
idempotency_key=str(idempotency_key).strip(),
|
||||
request=request,
|
||||
recovery_plan=RecoveryPlan(
|
||||
mode=RecoveryMode.FORWARD_RECOVERY,
|
||||
preconditions=(
|
||||
"inventory is bounded to one tenant Campaign artifact prefix",
|
||||
"objects younger than the conservative grace period are excluded",
|
||||
),
|
||||
forward_recovery_steps=(
|
||||
"retry only objects still unreferenced by committed Campaign state",
|
||||
),
|
||||
verification_steps=(
|
||||
"probe every attempted object after deletion",
|
||||
"preserve database references without mutation",
|
||||
),
|
||||
),
|
||||
precondition_evidence={
|
||||
"tenant_prefix_sha256": _sha256(prefix),
|
||||
"grace_period_hours": grace_period_hours,
|
||||
"page_size": request["page_size"],
|
||||
},
|
||||
lease_resource_key=f"campaign:artifact-reconcile:{tenant_id}",
|
||||
lease_ttl_seconds=15 * 60,
|
||||
resource_type="campaign_artifact_namespace",
|
||||
resource_id=tenant_id,
|
||||
metadata={"tenant_id": tenant_id},
|
||||
)
|
||||
if recovery_start.replayed:
|
||||
return _replayed_response(
|
||||
prefix=prefix,
|
||||
cursor=cursor,
|
||||
operation_id=recovery_start.operation_id,
|
||||
)
|
||||
operation = recovery_start.operation
|
||||
if operation is None: # pragma: no cover - guarded by replay branch
|
||||
raise CampaignArtifactReconciliationError(
|
||||
"Campaign artifact cleanup authority was not created"
|
||||
)
|
||||
|
||||
try:
|
||||
try:
|
||||
with session_factory() as session:
|
||||
inventory = campaign_artifact_inventory(
|
||||
session,
|
||||
storage=storage,
|
||||
tenant_id=tenant_id,
|
||||
grace_period=grace_period,
|
||||
cursor=cursor,
|
||||
page_size=page_size,
|
||||
now=observed_at,
|
||||
)
|
||||
except Exception as exc:
|
||||
operation.fail(
|
||||
summary="Campaign artifact inventory failed before deletion",
|
||||
evidence={
|
||||
"effect_started": False,
|
||||
"failure_type": type(exc).__name__,
|
||||
},
|
||||
)
|
||||
raise
|
||||
|
||||
operation.checkpoint(
|
||||
kind="artifact-inventory",
|
||||
summary="The bounded Campaign artifact inventory was classified",
|
||||
evidence={
|
||||
"manifest_sha256": inventory.manifest_sha256,
|
||||
"scanned_count": inventory.scanned_count,
|
||||
"candidate_count": len(inventory.candidates),
|
||||
"candidate_bytes": inventory.candidate_bytes,
|
||||
"next_page": inventory.next_cursor is not None,
|
||||
},
|
||||
)
|
||||
|
||||
if inventory.candidates:
|
||||
_apply_inventory(
|
||||
session_factory,
|
||||
storage=storage,
|
||||
operation=operation,
|
||||
inventory=inventory,
|
||||
tenant_id=tenant_id,
|
||||
now=observed_at,
|
||||
)
|
||||
|
||||
failed = [
|
||||
item
|
||||
for item in inventory.candidates
|
||||
if item.disposition == "delete_failed"
|
||||
]
|
||||
unknown = [
|
||||
item
|
||||
for item in inventory.candidates
|
||||
if item.disposition == "delete_outcome_unknown"
|
||||
]
|
||||
evidence = _cleanup_evidence(inventory)
|
||||
if unknown:
|
||||
operation.unresolved(
|
||||
status=RecoveryStatus.OUTCOME_UNKNOWN,
|
||||
summary="Campaign artifact deletion could not be verified",
|
||||
evidence=evidence,
|
||||
failure_summary=(
|
||||
"One or more Campaign artifact deletion outcomes are unknown"
|
||||
),
|
||||
)
|
||||
status = "outcome_unknown"
|
||||
elif failed:
|
||||
operation.unresolved(
|
||||
status=RecoveryStatus.RECOVERY_REQUIRED,
|
||||
summary="Campaign artifact deletion requires a retry",
|
||||
evidence=evidence,
|
||||
failure_summary=(
|
||||
"One or more unreferenced Campaign artifacts remain"
|
||||
),
|
||||
)
|
||||
status = "recovery_required"
|
||||
else:
|
||||
operation.succeed(evidence=evidence)
|
||||
status = "applied"
|
||||
return inventory.response(
|
||||
apply=True,
|
||||
status=status,
|
||||
recovery_operation_id=recovery_start.operation_id,
|
||||
)
|
||||
except Exception:
|
||||
if not operation.closed:
|
||||
try:
|
||||
operation.release_unresolved()
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def _apply_inventory(
|
||||
session_factory: SessionFactory,
|
||||
*,
|
||||
storage: StorageBackend,
|
||||
operation: Any,
|
||||
inventory: ArtifactInventory,
|
||||
tenant_id: str,
|
||||
now: datetime,
|
||||
) -> None:
|
||||
for index in range(0, len(inventory.candidates), _CHECKPOINT_BATCH_SIZE):
|
||||
batch = inventory.candidates[index : index + _CHECKPOINT_BATCH_SIZE]
|
||||
with session_factory() as session:
|
||||
referenced = _referenced_artifact_keys(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
prefix=inventory.tenant_prefix,
|
||||
artifact_keys={candidate.key for candidate in batch},
|
||||
)
|
||||
active_build_ids = _active_build_ids(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
now=now,
|
||||
)
|
||||
operation.checkpoint(
|
||||
kind="artifact-delete-batch-authorized",
|
||||
summary="Deletion authority was renewed for a bounded object batch",
|
||||
evidence={
|
||||
"batch_index": index // _CHECKPOINT_BATCH_SIZE,
|
||||
"batch_count": len(batch),
|
||||
"batch_manifest_sha256": _canonical_sha256(
|
||||
[_sha256(candidate.key) for candidate in batch]
|
||||
),
|
||||
},
|
||||
)
|
||||
for candidate in batch:
|
||||
build_id = _build_id_for_key(
|
||||
candidate.key,
|
||||
prefix=inventory.tenant_prefix,
|
||||
)
|
||||
if candidate.key in referenced or build_id in active_build_ids:
|
||||
candidate.disposition = "protected_before_delete"
|
||||
candidate.reason = "reference_or_active_build_appeared"
|
||||
continue
|
||||
_delete_and_verify(storage, candidate)
|
||||
|
||||
|
||||
def _delete_and_verify(
|
||||
storage: StorageBackend,
|
||||
candidate: ArtifactCandidate,
|
||||
) -> None:
|
||||
delete_failure: Exception | None = None
|
||||
try:
|
||||
storage.delete(candidate.key)
|
||||
except (OSError, StorageBackendError) as exc:
|
||||
delete_failure = exc
|
||||
try:
|
||||
remains = storage.exists(candidate.key)
|
||||
except (OSError, StorageBackendError) as exc:
|
||||
candidate.disposition = "delete_outcome_unknown"
|
||||
candidate.failure_type = type(exc).__name__
|
||||
return
|
||||
if not remains:
|
||||
candidate.disposition = "deleted"
|
||||
return
|
||||
candidate.disposition = "delete_failed"
|
||||
candidate.failure_type = (
|
||||
type(delete_failure).__name__ if delete_failure is not None else None
|
||||
)
|
||||
|
||||
|
||||
def _referenced_artifact_keys(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
prefix: str,
|
||||
artifact_keys: set[str] | None = None,
|
||||
) -> set[str]:
|
||||
if artifact_keys == set():
|
||||
return set()
|
||||
keys: set[str] = set()
|
||||
version_ids = {
|
||||
identity[1]
|
||||
for key in artifact_keys or ()
|
||||
if (identity := _artifact_identity(key, prefix=prefix)) is not None
|
||||
}
|
||||
job_query = session.query(
|
||||
CampaignJob.eml_storage_key,
|
||||
CampaignJob.resolved_print_output,
|
||||
).filter(CampaignJob.tenant_id == tenant_id)
|
||||
if artifact_keys is not None:
|
||||
filters = [CampaignJob.eml_storage_key.in_(artifact_keys)]
|
||||
if version_ids:
|
||||
filters.append(CampaignJob.campaign_version_id.in_(version_ids))
|
||||
job_query = job_query.filter(or_(*filters))
|
||||
job_rows = job_query.yield_per(1000)
|
||||
for eml_storage_key, print_output in job_rows:
|
||||
_add_key(
|
||||
keys,
|
||||
eml_storage_key,
|
||||
prefix=prefix,
|
||||
allowed_keys=artifact_keys,
|
||||
)
|
||||
_add_key(
|
||||
keys,
|
||||
_print_output_storage_key(print_output),
|
||||
prefix=prefix,
|
||||
allowed_keys=artifact_keys,
|
||||
)
|
||||
|
||||
version_query = session.query(CampaignVersion.build_summary).join(
|
||||
Campaign,
|
||||
Campaign.id == CampaignVersion.campaign_id,
|
||||
).filter(Campaign.tenant_id == tenant_id)
|
||||
if artifact_keys is not None:
|
||||
if not version_ids:
|
||||
return keys
|
||||
version_query = version_query.filter(CampaignVersion.id.in_(version_ids))
|
||||
version_rows = version_query.yield_per(500)
|
||||
for (build_summary,) in version_rows:
|
||||
print_output = (
|
||||
build_summary.get("print_output")
|
||||
if isinstance(build_summary, dict)
|
||||
else None
|
||||
)
|
||||
_add_key(
|
||||
keys,
|
||||
_print_output_storage_key(print_output),
|
||||
prefix=prefix,
|
||||
allowed_keys=artifact_keys,
|
||||
)
|
||||
return keys
|
||||
|
||||
|
||||
def _artifact_identity(
|
||||
key: str,
|
||||
*,
|
||||
prefix: str,
|
||||
) -> tuple[str, str, str] | None:
|
||||
if not key.startswith(prefix):
|
||||
return None
|
||||
parts = key[len(prefix) :].split("/")
|
||||
if len(parts) < 4 or any(not part for part in parts[:4]):
|
||||
return None
|
||||
return parts[0], parts[1], parts[2]
|
||||
|
||||
|
||||
def _active_build_ids(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
now: datetime,
|
||||
) -> set[str]:
|
||||
operations = (
|
||||
session.query(RecoveryOperation)
|
||||
.join(
|
||||
CampaignVersion,
|
||||
CampaignVersion.id == RecoveryOperation.resource_id,
|
||||
)
|
||||
.join(Campaign, Campaign.id == CampaignVersion.campaign_id)
|
||||
.filter(
|
||||
Campaign.tenant_id == tenant_id,
|
||||
RecoveryOperation.module_id == "campaigns",
|
||||
RecoveryOperation.operation_type == "build-artifacts",
|
||||
RecoveryOperation.status.not_in(TERMINAL_RECOVERY_STATUSES),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
resource_keys = {
|
||||
operation.lease_resource_key
|
||||
for operation in operations
|
||||
if operation.lease_resource_key
|
||||
}
|
||||
if not resource_keys:
|
||||
return set()
|
||||
installation_ids = {operation.installation_id for operation in operations}
|
||||
leases = (
|
||||
session.query(DistributedLease)
|
||||
.filter(
|
||||
DistributedLease.installation_id.in_(installation_ids),
|
||||
DistributedLease.resource_key.in_(resource_keys),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
leases_by_key = {
|
||||
(lease.installation_id, lease.resource_key): lease for lease in leases
|
||||
}
|
||||
active: set[str] = set()
|
||||
for operation in operations:
|
||||
lease = leases_by_key.get(
|
||||
(operation.installation_id, operation.lease_resource_key or "")
|
||||
)
|
||||
if (
|
||||
lease is not None
|
||||
and lease.holder_node_id == operation.holder_node_id
|
||||
and lease.holder_incarnation == operation.holder_incarnation
|
||||
and lease.fencing_token == operation.fencing_token
|
||||
and _as_utc(lease.expires_at) > now
|
||||
):
|
||||
active.add(operation.id)
|
||||
return active
|
||||
|
||||
|
||||
def _cleanup_evidence(inventory: ArtifactInventory) -> dict[str, Any]:
|
||||
dispositions: dict[str, int] = {}
|
||||
for candidate in inventory.candidates:
|
||||
dispositions[candidate.disposition] = (
|
||||
dispositions.get(candidate.disposition, 0) + 1
|
||||
)
|
||||
verified = not any(
|
||||
key in dispositions for key in ("delete_failed", "delete_outcome_unknown")
|
||||
)
|
||||
return {
|
||||
"verified": verified,
|
||||
"checks": {
|
||||
"candidate_objects": (
|
||||
"absent-or-newly-protected" if verified else "incomplete"
|
||||
),
|
||||
"database_references": "unchanged",
|
||||
"lease_fence": "renewed-before-each-batch",
|
||||
},
|
||||
"inventory_manifest_sha256": inventory.manifest_sha256,
|
||||
"candidate_manifest_sha256": _canonical_sha256(
|
||||
[_sha256(candidate.key) for candidate in inventory.candidates]
|
||||
),
|
||||
"candidate_count": len(inventory.candidates),
|
||||
"candidate_bytes": inventory.candidate_bytes,
|
||||
"dispositions": dispositions,
|
||||
"database_references_mutated": False,
|
||||
}
|
||||
|
||||
|
||||
def _replayed_response(
|
||||
*,
|
||||
prefix: str,
|
||||
cursor: str | None,
|
||||
operation_id: str,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"apply": True,
|
||||
"status": "already_completed",
|
||||
"recovery_operation_id": operation_id,
|
||||
"tenant_prefix": prefix,
|
||||
"cursor": cursor,
|
||||
"next_cursor": None,
|
||||
"scanned_count": 0,
|
||||
"scanned_bytes": 0,
|
||||
"referenced_count": 0,
|
||||
"active_build_count": 0,
|
||||
"young_count": 0,
|
||||
"unknown_age_count": 0,
|
||||
"invalid_shape_count": 0,
|
||||
"candidate_count": 0,
|
||||
"candidate_bytes": 0,
|
||||
"deleted_count": 0,
|
||||
"deleted_bytes": 0,
|
||||
"failure_count": 0,
|
||||
"manifest_sha256": None,
|
||||
"candidates": [],
|
||||
}
|
||||
|
||||
|
||||
def _tenant_artifact_prefix(tenant_id: str) -> str:
|
||||
normalized = str(tenant_id or "").strip()
|
||||
if not normalized or "/" in normalized or normalized in {".", ".."}:
|
||||
raise ValueError("Campaign artifact inventory requires a valid tenant id")
|
||||
return f"{CAMPAIGN_ARTIFACT_NAMESPACE}/{normalized}/"
|
||||
|
||||
|
||||
def _build_id_for_key(key: str, *, prefix: str) -> str | None:
|
||||
identity = _artifact_identity(key, prefix=prefix)
|
||||
return identity[2] if identity is not None else None
|
||||
|
||||
|
||||
def _print_output_storage_key(value: object) -> str | None:
|
||||
if not isinstance(value, dict):
|
||||
return None
|
||||
artifact = value.get("artifact")
|
||||
if not isinstance(artifact, dict):
|
||||
return None
|
||||
key = artifact.get("storage_key")
|
||||
return str(key) if key else None
|
||||
|
||||
|
||||
def _add_key(
|
||||
keys: set[str],
|
||||
value: object,
|
||||
*,
|
||||
prefix: str,
|
||||
allowed_keys: set[str] | None = None,
|
||||
) -> None:
|
||||
if value is None:
|
||||
return
|
||||
key = str(value)
|
||||
if key.startswith(prefix) and (allowed_keys is None or key in allowed_keys):
|
||||
keys.add(key)
|
||||
|
||||
|
||||
def _object_modified_at(info: StorageObjectInfo) -> datetime | None:
|
||||
return _as_utc(info.modified_at) if info.modified_at is not None else None
|
||||
|
||||
|
||||
def _as_utc(value: datetime) -> datetime:
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def _sha256(value: object) -> str:
|
||||
return hashlib.sha256(str(value).encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _canonical_sha256(value: object) -> str:
|
||||
return hashlib.sha256(
|
||||
json.dumps(
|
||||
value,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
).hexdigest()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CAMPAIGN_ARTIFACT_NAMESPACE",
|
||||
"CampaignArtifactReconciliationError",
|
||||
"campaign_artifact_inventory",
|
||||
"reconcile_campaign_artifacts",
|
||||
]
|
||||
@@ -53,6 +53,17 @@ class AttachmentIssue(BaseModel):
|
||||
code: str
|
||||
message: str
|
||||
behavior: Behavior | None = None
|
||||
details: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class AttachmentPolicyDecision(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
requirement_policy: Behavior
|
||||
campaign_policy: Behavior
|
||||
rule_policy: Behavior | None = None
|
||||
effective_behavior: Behavior
|
||||
legacy_drop_normalized: bool = False
|
||||
|
||||
|
||||
class ResolvedAttachment(BaseModel):
|
||||
@@ -82,6 +93,7 @@ class ResolvedAttachment(BaseModel):
|
||||
zip_entry_names: list[str] = Field(default_factory=list)
|
||||
status: AttachmentMatchStatus
|
||||
behavior: Behavior | None = None
|
||||
missing_policy: AttachmentPolicyDecision | None = None
|
||||
matches: list[str] = Field(default_factory=list)
|
||||
issues: list[AttachmentIssue] = Field(default_factory=list)
|
||||
|
||||
@@ -193,12 +205,48 @@ def _rule_allows_multiple(config: AttachmentConfig, rendered_file_filter: str) -
|
||||
return config.allow_multiple or any(char in rendered_file_filter for char in "*?[")
|
||||
|
||||
|
||||
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
||||
_MISSING_BEHAVIOR_STRENGTH = {
|
||||
Behavior.CONTINUE: 0,
|
||||
Behavior.WARN: 1,
|
||||
Behavior.ASK: 2,
|
||||
Behavior.DROP: 2,
|
||||
Behavior.BLOCK: 3,
|
||||
}
|
||||
|
||||
|
||||
def _missing_policy_decision(
|
||||
campaign_config: CampaignConfig,
|
||||
config: AttachmentConfig,
|
||||
) -> AttachmentPolicyDecision:
|
||||
requirement_policy = (
|
||||
campaign_config.validation_policy.missing_required_attachment
|
||||
if config.required
|
||||
else campaign_config.validation_policy.missing_optional_attachment
|
||||
)
|
||||
candidates = [
|
||||
requirement_policy,
|
||||
campaign_config.attachments.missing_behavior,
|
||||
]
|
||||
if config.missing_behavior is not None:
|
||||
return config.missing_behavior
|
||||
if config.required:
|
||||
return campaign_config.validation_policy.missing_required_attachment
|
||||
return campaign_config.validation_policy.missing_optional_attachment
|
||||
candidates.append(config.missing_behavior)
|
||||
configured = max(
|
||||
candidates,
|
||||
key=lambda behavior: _MISSING_BEHAVIOR_STRENGTH[behavior],
|
||||
)
|
||||
legacy_drop_normalized = configured == Behavior.DROP
|
||||
if legacy_drop_normalized:
|
||||
configured = Behavior.BLOCK if config.required else Behavior.ASK
|
||||
return AttachmentPolicyDecision(
|
||||
requirement_policy=requirement_policy,
|
||||
campaign_policy=campaign_config.attachments.missing_behavior,
|
||||
rule_policy=config.missing_behavior,
|
||||
effective_behavior=configured,
|
||||
legacy_drop_normalized=legacy_drop_normalized,
|
||||
)
|
||||
|
||||
|
||||
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
||||
return _missing_policy_decision(campaign_config, config).effective_behavior
|
||||
|
||||
|
||||
def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
|
||||
@@ -356,14 +404,19 @@ def _confine_managed_matches(directory: Path, matches: list[Path]) -> tuple[list
|
||||
return confined, rejected
|
||||
|
||||
|
||||
def _issue_for_missing(config: AttachmentConfig, behavior: Behavior) -> AttachmentIssue:
|
||||
def _issue_for_missing(
|
||||
config: AttachmentConfig,
|
||||
policy: AttachmentPolicyDecision,
|
||||
) -> AttachmentIssue:
|
||||
code = "missing_required_attachment" if config.required else "missing_optional_attachment"
|
||||
severity = ResolutionSeverity.ERROR if config.required and behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
||||
behavior = policy.effective_behavior
|
||||
severity = ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
||||
return AttachmentIssue(
|
||||
severity=severity,
|
||||
code=code,
|
||||
message=f"No file matched attachment filter {config.file_filter!r}",
|
||||
behavior=behavior,
|
||||
details={"effective_policy": policy.model_dump(mode="json")},
|
||||
)
|
||||
|
||||
|
||||
@@ -377,10 +430,13 @@ def _issue_for_ambiguous(config: AttachmentConfig, behavior: Behavior, match_cou
|
||||
)
|
||||
|
||||
|
||||
def _send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
|
||||
return config.attachments.send_without_attachments_behavior or (
|
||||
def effective_send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
|
||||
configured = config.attachments.send_without_attachments_behavior or (
|
||||
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
|
||||
)
|
||||
# Recipient exclusion must be an explicit reviewed action, not an implicit
|
||||
# consequence of a legacy attachment policy value.
|
||||
return Behavior.ASK if configured == Behavior.DROP else configured
|
||||
|
||||
|
||||
def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssue:
|
||||
@@ -395,6 +451,7 @@ def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssu
|
||||
code="missing_attachment_coverage",
|
||||
message=messages.get(behavior, "No attachment file was resolved for this message."),
|
||||
behavior=behavior,
|
||||
details={"effective_behavior": behavior.value},
|
||||
)
|
||||
|
||||
|
||||
@@ -422,24 +479,13 @@ def _resolve_one_config(
|
||||
behavior: Behavior | None = None
|
||||
managed_source = selected_base_path is not None and is_managed_source(selected_base_path.source)
|
||||
unsafe_managed_path = False
|
||||
if managed_source:
|
||||
try:
|
||||
resolution_failed = False
|
||||
try:
|
||||
if managed_source:
|
||||
assert_logical_relative_path(
|
||||
rendered_file_filter,
|
||||
field="rendered managed attachment file_filter",
|
||||
)
|
||||
except CampaignPathSecurityError as exc:
|
||||
matches = []
|
||||
unsafe_managed_path = True
|
||||
issues.append(
|
||||
AttachmentIssue(
|
||||
severity=ResolutionSeverity.ERROR,
|
||||
code="unsafe_managed_attachment_path",
|
||||
message=str(exc),
|
||||
behavior=Behavior.BLOCK,
|
||||
)
|
||||
)
|
||||
else:
|
||||
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
||||
matches, rejected = _confine_managed_matches(directory, matches)
|
||||
if rejected:
|
||||
@@ -453,16 +499,44 @@ def _resolve_one_config(
|
||||
behavior=Behavior.BLOCK,
|
||||
)
|
||||
)
|
||||
else:
|
||||
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
||||
else:
|
||||
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
|
||||
except CampaignPathSecurityError as exc:
|
||||
matches = []
|
||||
unsafe_managed_path = True
|
||||
issues.append(
|
||||
AttachmentIssue(
|
||||
severity=ResolutionSeverity.ERROR,
|
||||
code="unsafe_managed_attachment_path",
|
||||
message=str(exc),
|
||||
behavior=Behavior.BLOCK,
|
||||
)
|
||||
)
|
||||
except (OSError, RuntimeError) as exc:
|
||||
matches = []
|
||||
resolution_failed = True
|
||||
issues.append(
|
||||
AttachmentIssue(
|
||||
severity=ResolutionSeverity.ERROR,
|
||||
code="attachment_resolution_failed",
|
||||
message=f"Attachment source could not be read while resolving filter {config.file_filter!r}.",
|
||||
behavior=Behavior.BLOCK,
|
||||
details={"error_type": type(exc).__name__},
|
||||
)
|
||||
)
|
||||
|
||||
missing_policy: AttachmentPolicyDecision | None = None
|
||||
if unsafe_managed_path:
|
||||
status = AttachmentMatchStatus.MISSING
|
||||
behavior = Behavior.BLOCK
|
||||
elif resolution_failed:
|
||||
status = AttachmentMatchStatus.MISSING
|
||||
behavior = Behavior.BLOCK
|
||||
elif not matches:
|
||||
status = AttachmentMatchStatus.MISSING
|
||||
behavior = _missing_behavior(campaign_config, config)
|
||||
issues.append(_issue_for_missing(config, behavior))
|
||||
missing_policy = _missing_policy_decision(campaign_config, config)
|
||||
behavior = missing_policy.effective_behavior
|
||||
issues.append(_issue_for_missing(config, missing_policy))
|
||||
elif len(matches) > 1 and not allow_multiple:
|
||||
status = AttachmentMatchStatus.AMBIGUOUS
|
||||
behavior = _ambiguous_behavior(campaign_config, config)
|
||||
@@ -494,6 +568,7 @@ def _resolve_one_config(
|
||||
zip_entry_name_template=config.zip_entry_name_template,
|
||||
status=status,
|
||||
behavior=behavior,
|
||||
missing_policy=missing_policy,
|
||||
matches=[str(path) for path in matches],
|
||||
issues=issues,
|
||||
)
|
||||
@@ -540,7 +615,7 @@ def resolve_entry_attachments(
|
||||
)
|
||||
|
||||
issues = [issue for item in resolved for issue in item.issues]
|
||||
missing_coverage_behavior = _send_without_attachments_behavior(config)
|
||||
missing_coverage_behavior = effective_send_without_attachments_behavior(config)
|
||||
if (
|
||||
entry.active
|
||||
and resolved
|
||||
|
||||
@@ -44,6 +44,7 @@ def _parse_scalar_for_target(target: str, value: Any) -> Any:
|
||||
"merge_reply_to",
|
||||
"merge_bounce_to",
|
||||
"merge_disposition_notification_to",
|
||||
"merge_postbox_targets",
|
||||
"combine_to",
|
||||
"combine_cc",
|
||||
"combine_bcc",
|
||||
|
||||
@@ -4,8 +4,18 @@ import copy
|
||||
from typing import Any
|
||||
|
||||
|
||||
CAMPAIGN_MAIL_SERVER_KEYS = frozenset({"mail_profile_id"})
|
||||
CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset({"created_from", "field_overrides", "opt_ins"})
|
||||
CAMPAIGN_MAIL_SERVER_KEYS = frozenset(
|
||||
{
|
||||
"mail_profile_id",
|
||||
"smtp_server_id",
|
||||
"smtp_credential_id",
|
||||
"imap_server_id",
|
||||
"imap_credential_id",
|
||||
}
|
||||
)
|
||||
CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset(
|
||||
{"created_from", "field_overrides", "opt_ins"}
|
||||
)
|
||||
CAMPAIGN_OPT_IN_KEYS = frozenset(
|
||||
{"campaign_address_suggestions", "remember_used_addresses", "inline_guidance"}
|
||||
)
|
||||
@@ -14,18 +24,29 @@ CAMPAIGN_REVIEW_STATE_KEYS = frozenset(
|
||||
"build_token",
|
||||
"inspection_complete",
|
||||
"reviewed_message_keys",
|
||||
"issue_decisions",
|
||||
"updated_at",
|
||||
"updated_by_user_id",
|
||||
}
|
||||
)
|
||||
CAMPAIGN_APPROVAL_GATE_KEYS = frozenset(
|
||||
{
|
||||
"request_id",
|
||||
"request_revision",
|
||||
"subject_version",
|
||||
"subject_digest",
|
||||
"requested_at",
|
||||
"requested_by_user_id",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class CampaignMailProfileBoundaryError(ValueError):
|
||||
"""Raised when campaign JSON owns mail transport configuration.
|
||||
|
||||
SMTP/IMAP endpoints and credentials are Mail-module data. Campaign JSON
|
||||
may select one Mail-owned profile, but it must never copy or override that
|
||||
profile's transport configuration.
|
||||
may select Mail-owned profile, server, and credential identifiers, but it
|
||||
must never copy or override transport configuration.
|
||||
"""
|
||||
|
||||
|
||||
@@ -85,10 +106,13 @@ def validate_campaign_editor_state(
|
||||
if value is None:
|
||||
return {}
|
||||
if not isinstance(value, dict):
|
||||
raise CampaignMailProfileBoundaryError("Campaign editor state must be an object")
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign editor state must be an object"
|
||||
)
|
||||
allowed = set(CAMPAIGN_CLIENT_EDITOR_STATE_KEYS)
|
||||
if allow_server_review_state:
|
||||
allowed.add("review_send")
|
||||
allowed.add("approval_gate")
|
||||
if any(key not in allowed for key in value):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign editor state contains unsupported or transport-owned fields"
|
||||
@@ -104,11 +128,13 @@ def validate_campaign_editor_state(
|
||||
if "opt_ins" in value:
|
||||
result["opt_ins"] = _validated_opt_ins(value["opt_ins"])
|
||||
if "field_overrides" in value:
|
||||
result["field_overrides"] = _validated_field_overrides(
|
||||
value["field_overrides"]
|
||||
)
|
||||
result["field_overrides"] = _validated_field_overrides(value["field_overrides"])
|
||||
if "review_send" in value:
|
||||
result["review_send"] = _validated_server_review_state(value["review_send"])
|
||||
if "approval_gate" in value:
|
||||
result["approval_gate"] = _validated_server_approval_gate(
|
||||
value["approval_gate"]
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@@ -134,9 +160,32 @@ def public_campaign_editor_state(
|
||||
review_state = _validated_server_review_state(value["review_send"])
|
||||
if not include_diagnostics:
|
||||
review_state.pop("build_token", None)
|
||||
review_state["issue_decisions"] = [
|
||||
{
|
||||
key: item[key]
|
||||
for key in (
|
||||
"job_id",
|
||||
"review_key",
|
||||
"decision",
|
||||
"reason",
|
||||
"actor_user_id",
|
||||
"decided_at",
|
||||
"issue_codes",
|
||||
)
|
||||
if key in item
|
||||
}
|
||||
for item in review_state.get("issue_decisions", [])
|
||||
]
|
||||
result["review_send"] = review_state
|
||||
except CampaignMailProfileBoundaryError:
|
||||
pass
|
||||
if "approval_gate" in value:
|
||||
try:
|
||||
result["approval_gate"] = _validated_server_approval_gate(
|
||||
value["approval_gate"]
|
||||
)
|
||||
except CampaignMailProfileBoundaryError:
|
||||
pass
|
||||
return result
|
||||
|
||||
|
||||
@@ -145,9 +194,48 @@ def campaign_editor_state_for_edit(value: Any) -> dict[str, Any]:
|
||||
|
||||
state = public_campaign_editor_state(value)
|
||||
state.pop("review_send", None)
|
||||
state.pop("approval_gate", None)
|
||||
return state
|
||||
|
||||
|
||||
def _validated_server_approval_gate(value: Any) -> dict[str, Any]:
|
||||
if not isinstance(value, dict) or any(
|
||||
key not in CAMPAIGN_APPROVAL_GATE_KEYS for key in value
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError("Campaign approval gate is invalid")
|
||||
required_strings = (
|
||||
"request_id",
|
||||
"subject_version",
|
||||
"subject_digest",
|
||||
"requested_at",
|
||||
)
|
||||
if any(
|
||||
not isinstance(value.get(key), str) or not str(value.get(key)).strip()
|
||||
for key in required_strings
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign approval gate references are invalid"
|
||||
)
|
||||
digest = str(value["subject_digest"])
|
||||
if len(digest) != 64 or any(
|
||||
character not in "0123456789abcdef" for character in digest
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign approval gate digest is invalid"
|
||||
)
|
||||
revision = value.get("request_revision")
|
||||
if not isinstance(revision, int) or revision < 1:
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign approval gate revision is invalid"
|
||||
)
|
||||
requested_by = value.get("requested_by_user_id")
|
||||
if requested_by is not None and not isinstance(requested_by, str):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign approval gate actor is invalid"
|
||||
)
|
||||
return copy.deepcopy(value)
|
||||
|
||||
|
||||
def _is_valid_reviewed_message_key(value: Any) -> bool:
|
||||
return isinstance(value, str) and bool(value.strip()) and len(value) <= 512
|
||||
|
||||
@@ -170,6 +258,98 @@ def _validated_review_actor(value: Any) -> str | None:
|
||||
return value
|
||||
|
||||
|
||||
def _validated_issue_decisions(value: Any) -> list[dict[str, Any]]:
|
||||
if not isinstance(value, list) or len(value) > 100_000:
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review issue decisions are invalid"
|
||||
)
|
||||
result: list[dict[str, Any]] = []
|
||||
allowed_keys = {
|
||||
"job_id",
|
||||
"review_key",
|
||||
"decision",
|
||||
"reason",
|
||||
"actor_user_id",
|
||||
"decided_at",
|
||||
"build_token",
|
||||
"message_sha256",
|
||||
"issue_fingerprint",
|
||||
"issue_codes",
|
||||
}
|
||||
for item in value:
|
||||
if not isinstance(item, dict) or any(key not in allowed_keys for key in item):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review issue decision is invalid"
|
||||
)
|
||||
normalized = {
|
||||
"job_id": _validated_required_string(
|
||||
item.get("job_id"),
|
||||
max_length=36,
|
||||
error="Campaign review decision job is invalid",
|
||||
),
|
||||
"review_key": _validated_required_string(
|
||||
item.get("review_key"),
|
||||
max_length=512,
|
||||
error="Campaign review decision key is invalid",
|
||||
),
|
||||
"decision": _validated_required_string(
|
||||
item.get("decision"),
|
||||
max_length=30,
|
||||
error="Campaign review decision is invalid",
|
||||
),
|
||||
"reason": item.get("reason"),
|
||||
"actor_user_id": _validated_review_actor(item.get("actor_user_id")),
|
||||
"decided_at": _validated_required_string(
|
||||
item.get("decided_at"),
|
||||
max_length=128,
|
||||
error="Campaign review decision timestamp is invalid",
|
||||
),
|
||||
"build_token": _validated_required_string(
|
||||
item.get("build_token"),
|
||||
max_length=256,
|
||||
error="Campaign review decision build token is invalid",
|
||||
),
|
||||
"message_sha256": item.get("message_sha256"),
|
||||
"issue_fingerprint": _validated_required_string(
|
||||
item.get("issue_fingerprint"),
|
||||
max_length=64,
|
||||
error="Campaign review issue fingerprint is invalid",
|
||||
),
|
||||
"issue_codes": item.get("issue_codes"),
|
||||
}
|
||||
if normalized["decision"] != "accept":
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review decision outcome is invalid"
|
||||
)
|
||||
if normalized["reason"] is not None and (
|
||||
not isinstance(normalized["reason"], str)
|
||||
or len(normalized["reason"]) > 4_000
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review decision reason is invalid"
|
||||
)
|
||||
if normalized["message_sha256"] is not None and (
|
||||
not isinstance(normalized["message_sha256"], str)
|
||||
or len(normalized["message_sha256"]) > 64
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review decision message hash is invalid"
|
||||
)
|
||||
if (
|
||||
not isinstance(normalized["issue_codes"], list)
|
||||
or len(normalized["issue_codes"]) > 100
|
||||
or any(
|
||||
not isinstance(code, str) or not code or len(code) > 100
|
||||
for code in normalized["issue_codes"]
|
||||
)
|
||||
):
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign review decision issue codes are invalid"
|
||||
)
|
||||
result.append(normalized)
|
||||
return result
|
||||
|
||||
|
||||
def _validated_server_review_state(value: Any) -> dict[str, Any]:
|
||||
if not isinstance(value, dict) or any(
|
||||
key not in CAMPAIGN_REVIEW_STATE_KEYS for key in value
|
||||
@@ -180,6 +360,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
|
||||
build_token = value.get("build_token")
|
||||
inspected = value.get("inspection_complete")
|
||||
keys = value.get("reviewed_message_keys", [])
|
||||
issue_decisions = value.get("issue_decisions", [])
|
||||
updated_at = value.get("updated_at")
|
||||
updated_by = value.get("updated_by_user_id")
|
||||
validated_build_token = _validated_required_string(
|
||||
@@ -192,6 +373,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
|
||||
"Campaign review completion state is invalid"
|
||||
)
|
||||
validated_keys = _validated_reviewed_message_keys(keys)
|
||||
validated_decisions = _validated_issue_decisions(issue_decisions)
|
||||
validated_updated_at = _validated_required_string(
|
||||
updated_at,
|
||||
max_length=128,
|
||||
@@ -202,6 +384,7 @@ def _validated_server_review_state(value: Any) -> dict[str, Any]:
|
||||
"build_token": validated_build_token,
|
||||
"inspection_complete": inspected,
|
||||
"reviewed_message_keys": validated_keys,
|
||||
"issue_decisions": validated_decisions,
|
||||
"updated_at": validated_updated_at,
|
||||
"updated_by_user_id": validated_updated_by,
|
||||
}
|
||||
@@ -218,16 +401,57 @@ def campaign_mail_profile_id(raw_json: dict[str, Any] | None) -> str | None:
|
||||
return normalized or None
|
||||
|
||||
|
||||
def campaign_mail_profile_boundary_violations(raw_json: dict[str, Any] | None) -> tuple[str, ...]:
|
||||
def campaign_mail_resource_ids(
|
||||
raw_json: dict[str, Any] | None,
|
||||
) -> dict[str, str | None]:
|
||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||
if not isinstance(server, dict):
|
||||
return {
|
||||
"mail_profile_id": None,
|
||||
"smtp_server_id": None,
|
||||
"smtp_credential_id": None,
|
||||
"imap_server_id": None,
|
||||
"imap_credential_id": None,
|
||||
}
|
||||
return {
|
||||
key: (
|
||||
value.strip()
|
||||
if isinstance((value := server.get(key)), str) and value.strip()
|
||||
else None
|
||||
)
|
||||
for key in CAMPAIGN_MAIL_SERVER_KEYS
|
||||
}
|
||||
|
||||
|
||||
def campaign_mail_profile_boundary_violations(
|
||||
raw_json: dict[str, Any] | None,
|
||||
) -> tuple[str, ...]:
|
||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||
if not isinstance(server, dict):
|
||||
return ()
|
||||
|
||||
violations = [f"/server/{key}" for key in sorted(server) if key not in CAMPAIGN_MAIL_SERVER_KEYS]
|
||||
if "mail_profile_id" in server:
|
||||
profile_id = server["mail_profile_id"]
|
||||
if not isinstance(profile_id, str) or not profile_id.strip():
|
||||
violations.append("/server/mail_profile_id")
|
||||
violations = [
|
||||
f"/server/{key}"
|
||||
for key in sorted(server)
|
||||
if key not in CAMPAIGN_MAIL_SERVER_KEYS
|
||||
]
|
||||
for key in CAMPAIGN_MAIL_SERVER_KEYS:
|
||||
if key not in server:
|
||||
continue
|
||||
value = server[key]
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
violations.append(f"/server/{key}")
|
||||
references = campaign_mail_resource_ids(raw_json)
|
||||
if references["mail_profile_id"] is None and any(
|
||||
references[key] for key in references if key != "mail_profile_id"
|
||||
):
|
||||
violations.append("/server/mail_profile_id")
|
||||
for protocol in ("smtp", "imap"):
|
||||
if (
|
||||
references[f"{protocol}_credential_id"]
|
||||
and not references[f"{protocol}_server_id"]
|
||||
):
|
||||
violations.append(f"/server/{protocol}_server_id")
|
||||
return tuple(violations)
|
||||
|
||||
|
||||
@@ -240,9 +464,9 @@ def assert_campaign_uses_mail_profile_reference(
|
||||
if violations:
|
||||
fields = ", ".join(violations)
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
"Campaign JSON may only reference a Mail-module profile through "
|
||||
f"server.mail_profile_id; remove campaign-local SMTP/IMAP settings ({fields}), "
|
||||
"select an authorized Mail profile, and save a new campaign version."
|
||||
"Campaign JSON may only reference Mail-owned profiles, servers, and credentials; "
|
||||
f"remove campaign-local SMTP/IMAP settings or invalid references ({fields}), "
|
||||
"select authorized Mail resources, and save a new campaign version."
|
||||
)
|
||||
if require_profile and campaign_mail_profile_id(raw_json) is None:
|
||||
raise CampaignMailProfileBoundaryError(
|
||||
@@ -254,5 +478,8 @@ def assert_campaign_uses_mail_profile_reference(
|
||||
def public_campaign_mail_server(raw_json: dict[str, Any] | None) -> dict[str, str]:
|
||||
"""Return the complete public/persisted Campaign-to-Mail contract."""
|
||||
|
||||
profile_id = campaign_mail_profile_id(raw_json)
|
||||
return {"mail_profile_id": profile_id} if profile_id else {}
|
||||
return {
|
||||
key: value
|
||||
for key, value in campaign_mail_resource_ids(raw_json).items()
|
||||
if value
|
||||
}
|
||||
|
||||
@@ -23,6 +23,8 @@ class FieldType(StrEnum):
|
||||
DOUBLE = "double"
|
||||
DATE = "date"
|
||||
PASSWORD = "password" # noqa: S105 # nosec B105 - field type vocabulary.
|
||||
ORGANIZATION_UNIT = "organization_unit"
|
||||
ORGANIZATION_FUNCTION = "organization_function"
|
||||
|
||||
|
||||
class RecipientType(StrEnum):
|
||||
@@ -92,6 +94,131 @@ class SendStatus(StrEnum):
|
||||
SKIPPED = "skipped"
|
||||
|
||||
|
||||
class DeliveryChannelPolicy(StrEnum):
|
||||
MAIL = "mail"
|
||||
POSTBOX = "postbox"
|
||||
PRINT = "print"
|
||||
MAIL_AND_POSTBOX = "mail_and_postbox"
|
||||
MAIL_THEN_POSTBOX = "mail_then_postbox"
|
||||
POSTBOX_THEN_MAIL = "postbox_then_mail"
|
||||
MAIL_THEN_PRINT = "mail_then_print"
|
||||
POSTBOX_THEN_PRINT = "postbox_then_print"
|
||||
|
||||
@property
|
||||
def uses_mail(self) -> bool:
|
||||
return self in {
|
||||
DeliveryChannelPolicy.MAIL,
|
||||
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||
DeliveryChannelPolicy.MAIL_THEN_PRINT,
|
||||
}
|
||||
|
||||
@property
|
||||
def uses_postbox(self) -> bool:
|
||||
return self in {
|
||||
DeliveryChannelPolicy.POSTBOX,
|
||||
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||
DeliveryChannelPolicy.POSTBOX_THEN_PRINT,
|
||||
}
|
||||
|
||||
@property
|
||||
def uses_print(self) -> bool:
|
||||
return self in {
|
||||
DeliveryChannelPolicy.PRINT,
|
||||
DeliveryChannelPolicy.MAIL_THEN_PRINT,
|
||||
DeliveryChannelPolicy.POSTBOX_THEN_PRINT,
|
||||
}
|
||||
|
||||
|
||||
class PostboxTargetMode(StrEnum):
|
||||
DIRECT = "direct"
|
||||
DERIVED = "derived"
|
||||
|
||||
|
||||
class PostboxTargetMatch(StrEnum):
|
||||
ID = "id"
|
||||
SLUG = "slug"
|
||||
|
||||
|
||||
class PostboxTargetConfig(StrictModel):
|
||||
id: str = Field(min_length=1, max_length=120)
|
||||
mode: PostboxTargetMode = PostboxTargetMode.DIRECT
|
||||
label: str | None = Field(default=None, max_length=500)
|
||||
|
||||
postbox_id: str | None = Field(default=None, max_length=36)
|
||||
address_key: str | None = Field(default=None, max_length=500)
|
||||
|
||||
template_id: str | None = Field(default=None, max_length=36)
|
||||
organization_unit_id: str | None = Field(default=None, max_length=36)
|
||||
organization_unit_field: str | None = Field(default=None, max_length=255)
|
||||
organization_unit_match: PostboxTargetMatch = PostboxTargetMatch.ID
|
||||
function_id: str | None = Field(default=None, max_length=36)
|
||||
function_field: str | None = Field(default=None, max_length=255)
|
||||
function_match: PostboxTargetMatch = PostboxTargetMatch.ID
|
||||
context_key: str | None = Field(default=None, max_length=255)
|
||||
context_field: str | None = Field(default=None, max_length=255)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_target_shape(self) -> "PostboxTargetConfig":
|
||||
direct_values = [self.postbox_id, self.address_key]
|
||||
if self.mode == PostboxTargetMode.DIRECT:
|
||||
if sum(bool(value) for value in direct_values) != 1:
|
||||
raise ValueError(
|
||||
"A direct Postbox target requires exactly one postbox_id "
|
||||
"or address_key."
|
||||
)
|
||||
if any(
|
||||
(
|
||||
self.template_id,
|
||||
self.organization_unit_id,
|
||||
self.organization_unit_field,
|
||||
self.function_id,
|
||||
self.function_field,
|
||||
self.context_key,
|
||||
self.context_field,
|
||||
)
|
||||
):
|
||||
raise ValueError(
|
||||
"A direct Postbox target cannot contain derived target fields."
|
||||
)
|
||||
return self
|
||||
|
||||
if any(direct_values):
|
||||
raise ValueError(
|
||||
"A derived Postbox target cannot contain postbox_id or address_key."
|
||||
)
|
||||
if not self.template_id:
|
||||
raise ValueError("A derived Postbox target requires template_id.")
|
||||
if bool(self.organization_unit_id) == bool(self.organization_unit_field):
|
||||
raise ValueError(
|
||||
"A derived Postbox target requires exactly one fixed or "
|
||||
"field-derived organization unit."
|
||||
)
|
||||
if bool(self.function_id) == bool(self.function_field):
|
||||
raise ValueError(
|
||||
"A derived Postbox target requires exactly one fixed or "
|
||||
"field-derived function."
|
||||
)
|
||||
if self.context_key and self.context_field:
|
||||
raise ValueError(
|
||||
"A derived Postbox target may use a fixed context or a context "
|
||||
"field, not both."
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
class PrintTargetConfig(StrictModel):
|
||||
channel: Literal["postal", "internal_mail"]
|
||||
target: str = Field(min_length=1, max_length=4000)
|
||||
target_key: str = Field(min_length=1, max_length=500)
|
||||
contact_point_id: str | None = Field(default=None, max_length=36)
|
||||
locale: str | None = Field(default=None, max_length=35)
|
||||
decision_provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignMeta(StrictModel):
|
||||
id: str
|
||||
name: str
|
||||
@@ -114,6 +241,10 @@ class MailProfileCapabilities(StrictModel):
|
||||
|
||||
class ServerConfig(StrictModel):
|
||||
mail_profile_id: str | None = None
|
||||
smtp_server_id: str | None = None
|
||||
smtp_credential_id: str | None = None
|
||||
imap_server_id: str | None = None
|
||||
imap_credential_id: str | None = None
|
||||
profile_capabilities: MailProfileCapabilities = Field(default_factory=MailProfileCapabilities)
|
||||
|
||||
|
||||
@@ -376,7 +507,7 @@ class AttachmentsConfig(StrictModel):
|
||||
send_without_attachments_behavior: Behavior | None = None
|
||||
zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig)
|
||||
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
|
||||
missing_behavior: Behavior = Behavior.ASK
|
||||
missing_behavior: Behavior = Behavior.WARN
|
||||
ambiguous_behavior: Behavior = Behavior.ASK
|
||||
|
||||
@model_validator(mode="after")
|
||||
@@ -446,10 +577,22 @@ class EntryConfig(StrictModel):
|
||||
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
|
||||
merge_disposition_notification_to: bool = True
|
||||
|
||||
channel_policy: DeliveryChannelPolicy | None = None
|
||||
postbox_targets: list[PostboxTargetConfig] = Field(
|
||||
default_factory=list,
|
||||
max_length=50,
|
||||
)
|
||||
merge_postbox_targets: bool = True
|
||||
print_target: PrintTargetConfig | None = None
|
||||
|
||||
attachments: list[AttachmentConfig] = Field(default_factory=list)
|
||||
combine_attachments: bool = True
|
||||
|
||||
fields: dict[str, Any] = Field(default_factory=dict)
|
||||
# Frozen channel candidates, source revisions and the explicit route
|
||||
# decision imported from Distribution Lists. Campaign owns this snapshot;
|
||||
# it never re-resolves the audience during build or delivery.
|
||||
distribution_source: dict[str, Any] = Field(default_factory=dict)
|
||||
last_sent: str | None = None
|
||||
|
||||
|
||||
@@ -465,7 +608,7 @@ class ImportProvenance(StrictModel):
|
||||
id: str
|
||||
imported_at: str
|
||||
mode: Literal["append", "replace"]
|
||||
source_type: Literal["csv", "xlsx", "text", "addresses"]
|
||||
source_type: Literal["csv", "xlsx", "text", "addresses", "distribution_list"]
|
||||
source_id: str | None = None
|
||||
source_label: str | None = None
|
||||
source_revision: str | None = None
|
||||
@@ -527,7 +670,7 @@ class EntriesConfig(StrictModel):
|
||||
|
||||
|
||||
class ValidationPolicy(StrictModel):
|
||||
missing_required_attachment: Behavior = Behavior.ASK
|
||||
missing_required_attachment: Behavior = Behavior.BLOCK
|
||||
missing_optional_attachment: Behavior = Behavior.WARN
|
||||
ambiguous_attachment_match: Behavior = Behavior.ASK
|
||||
ignore_empty_fields: bool = False
|
||||
@@ -559,7 +702,43 @@ class RetryConfig(StrictModel):
|
||||
return values
|
||||
|
||||
|
||||
class PostboxDeliveryConfig(StrictModel):
|
||||
targets: list[PostboxTargetConfig] = Field(default_factory=list, max_length=50)
|
||||
classification: str = Field(default="internal", min_length=1, max_length=50)
|
||||
unresolved_target: Behavior = Behavior.BLOCK
|
||||
vacant_target: Behavior = Behavior.WARN
|
||||
duplicate_target: Behavior = Behavior.WARN
|
||||
|
||||
|
||||
class PrintDeliveryConfig(StrictModel):
|
||||
template_id: str | None = Field(default=None, max_length=36)
|
||||
template_revision: int | None = Field(default=None, ge=1)
|
||||
usage: str = Field(default="campaign_print", min_length=1, max_length=100)
|
||||
output_format: Literal["html", "text"] = "html"
|
||||
profile_id: str | None = Field(default=None, max_length=120)
|
||||
persist_to_files: bool = True
|
||||
|
||||
|
||||
class CalendarInvitationDeliveryConfig(StrictModel):
|
||||
enabled: bool = False
|
||||
calendar_id: str | None = Field(default=None, max_length=36)
|
||||
summary_template: str | None = Field(default=None, max_length=2_000)
|
||||
description_template: str | None = Field(default=None, max_length=20_000)
|
||||
location_template: str | None = Field(default=None, max_length=2_000)
|
||||
start_at_template: str | None = Field(default=None, max_length=1_000)
|
||||
end_at_template: str | None = Field(default=None, max_length=1_000)
|
||||
timezone: str | None = Field(default=None, max_length=100)
|
||||
classification: Literal["PUBLIC", "PRIVATE", "CONFIDENTIAL"] = "PUBLIC"
|
||||
categories: list[str] = Field(default_factory=list, max_length=50)
|
||||
|
||||
|
||||
class DeliveryConfig(StrictModel):
|
||||
channel_policy: DeliveryChannelPolicy = DeliveryChannelPolicy.MAIL
|
||||
postbox: PostboxDeliveryConfig = Field(default_factory=PostboxDeliveryConfig)
|
||||
print: PrintDeliveryConfig = Field(default_factory=PrintDeliveryConfig)
|
||||
calendar_invitation: CalendarInvitationDeliveryConfig = Field(
|
||||
default_factory=CalendarInvitationDeliveryConfig
|
||||
)
|
||||
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
|
||||
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
|
||||
retry: RetryConfig = Field(default_factory=RetryConfig)
|
||||
@@ -602,3 +781,23 @@ class CampaignConfig(StrictModel):
|
||||
if path.is_absolute():
|
||||
return path
|
||||
return (campaign_file.parent / path).resolve()
|
||||
|
||||
|
||||
def effective_delivery_channel_policy(
|
||||
config: CampaignConfig,
|
||||
entry: EntryConfig,
|
||||
) -> DeliveryChannelPolicy:
|
||||
return entry.channel_policy or config.delivery.channel_policy
|
||||
|
||||
|
||||
def effective_postbox_targets(
|
||||
config: CampaignConfig,
|
||||
entry: EntryConfig,
|
||||
) -> list[PostboxTargetConfig]:
|
||||
global_targets = list(config.delivery.postbox.targets)
|
||||
individual_targets = list(entry.postbox_targets)
|
||||
if not individual_targets:
|
||||
return global_targets
|
||||
if entry.merge_postbox_targets:
|
||||
return [*global_targets, *individual_targets]
|
||||
return individual_targets
|
||||
|
||||
@@ -0,0 +1,321 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.postbox import (
|
||||
PostboxDeliveryCatalogRef,
|
||||
PostboxDirectoryEntryRef,
|
||||
PostboxTargetRef,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.field_values import (
|
||||
effective_entry_field_values,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.models import (
|
||||
Behavior,
|
||||
CampaignConfig,
|
||||
EntryConfig,
|
||||
PostboxTargetConfig,
|
||||
PostboxTargetMatch,
|
||||
PostboxTargetMode,
|
||||
effective_postbox_targets,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import postbox_integration
|
||||
from govoplan_campaign.backend.messages.models import (
|
||||
MessageIssue,
|
||||
MessageValidationStatus,
|
||||
)
|
||||
|
||||
|
||||
def _apply_behavior(
|
||||
current: MessageValidationStatus,
|
||||
behavior: Behavior,
|
||||
) -> MessageValidationStatus:
|
||||
if behavior == Behavior.BLOCK:
|
||||
return MessageValidationStatus.BLOCKED
|
||||
if behavior == Behavior.DROP:
|
||||
return MessageValidationStatus.EXCLUDED
|
||||
if behavior == Behavior.ASK and current not in {
|
||||
MessageValidationStatus.BLOCKED,
|
||||
MessageValidationStatus.EXCLUDED,
|
||||
}:
|
||||
return MessageValidationStatus.NEEDS_REVIEW
|
||||
if behavior == Behavior.WARN and current == MessageValidationStatus.READY:
|
||||
return MessageValidationStatus.WARNING
|
||||
return current
|
||||
|
||||
|
||||
def _issue(
|
||||
*,
|
||||
code: str,
|
||||
message: str,
|
||||
behavior: Behavior,
|
||||
) -> MessageIssue:
|
||||
return MessageIssue(
|
||||
severity="error" if behavior == Behavior.BLOCK else "warning",
|
||||
code=code,
|
||||
message=message,
|
||||
behavior=behavior.value,
|
||||
source="postbox",
|
||||
)
|
||||
|
||||
|
||||
def _field_value(
|
||||
values: dict[str, Any],
|
||||
field_name: str | None,
|
||||
) -> str | None:
|
||||
if not field_name:
|
||||
return None
|
||||
value = values.get(field_name)
|
||||
if value is None:
|
||||
return None
|
||||
text = str(value).strip()
|
||||
return text or None
|
||||
|
||||
|
||||
def _match_unit(
|
||||
catalog: PostboxDeliveryCatalogRef,
|
||||
value: str | None,
|
||||
match: PostboxTargetMatch,
|
||||
):
|
||||
if not value:
|
||||
return None
|
||||
return next(
|
||||
(
|
||||
unit
|
||||
for unit in catalog.organization_units
|
||||
if (unit.id if match == PostboxTargetMatch.ID else unit.slug) == value
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _match_function(unit, value: str | None, match: PostboxTargetMatch):
|
||||
if unit is None or not value:
|
||||
return None
|
||||
return next(
|
||||
(
|
||||
function
|
||||
for function in unit.functions
|
||||
if (
|
||||
function.id
|
||||
if match == PostboxTargetMatch.ID
|
||||
else function.slug
|
||||
)
|
||||
== value
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _target_ref(
|
||||
target: PostboxTargetConfig,
|
||||
*,
|
||||
values: dict[str, Any],
|
||||
catalog: PostboxDeliveryCatalogRef,
|
||||
) -> tuple[PostboxTargetRef | None, str | None]:
|
||||
if target.mode == PostboxTargetMode.DIRECT:
|
||||
return (
|
||||
PostboxTargetRef(
|
||||
postbox_id=target.postbox_id,
|
||||
address_key=target.address_key,
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
unit_value = target.organization_unit_id or _field_value(
|
||||
values,
|
||||
target.organization_unit_field,
|
||||
)
|
||||
unit_match = (
|
||||
PostboxTargetMatch.ID
|
||||
if target.organization_unit_id
|
||||
else target.organization_unit_match
|
||||
)
|
||||
unit = _match_unit(catalog, unit_value, unit_match)
|
||||
if unit is None:
|
||||
return None, (
|
||||
f"Organization unit {unit_value!r} could not be resolved by "
|
||||
f"{unit_match.value}."
|
||||
)
|
||||
|
||||
function_value = target.function_id or _field_value(
|
||||
values,
|
||||
target.function_field,
|
||||
)
|
||||
function_match = (
|
||||
PostboxTargetMatch.ID
|
||||
if target.function_id
|
||||
else target.function_match
|
||||
)
|
||||
function = _match_function(unit, function_value, function_match)
|
||||
if function is None:
|
||||
return None, (
|
||||
f"Organization function {function_value!r} could not be resolved "
|
||||
f"inside {unit.name!r} by {function_match.value}."
|
||||
)
|
||||
|
||||
context_key = target.context_key or _field_value(
|
||||
values,
|
||||
target.context_field,
|
||||
)
|
||||
return (
|
||||
PostboxTargetRef(
|
||||
template_id=target.template_id,
|
||||
organization_unit_id=unit.id,
|
||||
function_id=function.id,
|
||||
context_key=context_key,
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _resolved_target_payload(
|
||||
target: PostboxTargetConfig,
|
||||
entry: PostboxDirectoryEntryRef,
|
||||
*,
|
||||
position: int,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"target_id": target.id,
|
||||
"position": position,
|
||||
"mode": target.mode.value,
|
||||
"requested": target.model_dump(mode="json", exclude_none=True),
|
||||
"postbox_id": entry.id,
|
||||
"address": entry.address,
|
||||
"address_key": entry.address_key,
|
||||
"name": entry.name,
|
||||
"status": entry.status,
|
||||
"classification": entry.classification,
|
||||
"organization_unit_id": entry.organization_unit_id,
|
||||
"organization_unit_name": entry.organization_unit_name,
|
||||
"function_id": entry.function_id,
|
||||
"function_name": entry.function_name,
|
||||
"context_key": entry.context_key,
|
||||
"template_revision_id": entry.template_revision_id,
|
||||
"holder_count": entry.holder_count,
|
||||
"vacant": entry.vacant,
|
||||
}
|
||||
|
||||
|
||||
def resolve_entry_postbox_targets(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
config: CampaignConfig,
|
||||
entry: EntryConfig,
|
||||
validation_status: MessageValidationStatus,
|
||||
materialize: bool,
|
||||
) -> tuple[
|
||||
list[dict[str, Any]],
|
||||
list[MessageIssue],
|
||||
MessageValidationStatus,
|
||||
]:
|
||||
integration = postbox_integration()
|
||||
policy = config.delivery.postbox
|
||||
targets = effective_postbox_targets(config, entry)
|
||||
if not targets:
|
||||
issue = _issue(
|
||||
code="postbox_target_missing",
|
||||
message="Postbox delivery requires at least one target.",
|
||||
behavior=policy.unresolved_target,
|
||||
)
|
||||
return (
|
||||
[],
|
||||
[issue],
|
||||
_apply_behavior(validation_status, policy.unresolved_target),
|
||||
)
|
||||
|
||||
try:
|
||||
catalog = integration.delivery_catalog(session, tenant_id=tenant_id)
|
||||
except Exception as exc:
|
||||
issue = _issue(
|
||||
code="postbox_unavailable",
|
||||
message=str(exc),
|
||||
behavior=Behavior.BLOCK,
|
||||
)
|
||||
return [], [issue], MessageValidationStatus.BLOCKED
|
||||
|
||||
values = effective_entry_field_values(config, entry)
|
||||
resolved: list[dict[str, Any]] = []
|
||||
issues: list[MessageIssue] = []
|
||||
status = validation_status
|
||||
seen_postbox_ids: set[str] = set()
|
||||
for position, target in enumerate(targets):
|
||||
target_ref, resolution_error = _target_ref(
|
||||
target,
|
||||
values=values,
|
||||
catalog=catalog,
|
||||
)
|
||||
if target_ref is None:
|
||||
issue = _issue(
|
||||
code="postbox_target_unresolved",
|
||||
message=resolution_error or "Postbox target could not be resolved.",
|
||||
behavior=policy.unresolved_target,
|
||||
)
|
||||
issues.append(issue)
|
||||
status = _apply_behavior(status, policy.unresolved_target)
|
||||
continue
|
||||
try:
|
||||
entry_ref = integration.resolve_postbox(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
target=target_ref,
|
||||
materialize=materialize,
|
||||
)
|
||||
except Exception as exc:
|
||||
issue = _issue(
|
||||
code="postbox_target_unresolved",
|
||||
message=f"Postbox target {target.id!r} could not be resolved: {exc}",
|
||||
behavior=policy.unresolved_target,
|
||||
)
|
||||
issues.append(issue)
|
||||
status = _apply_behavior(status, policy.unresolved_target)
|
||||
continue
|
||||
if entry_ref is None:
|
||||
issue = _issue(
|
||||
code="postbox_target_unresolved",
|
||||
message=f"Postbox target {target.id!r} does not exist.",
|
||||
behavior=policy.unresolved_target,
|
||||
)
|
||||
issues.append(issue)
|
||||
status = _apply_behavior(status, policy.unresolved_target)
|
||||
continue
|
||||
if entry_ref.id in seen_postbox_ids:
|
||||
issue = _issue(
|
||||
code="postbox_target_duplicate",
|
||||
message=(
|
||||
f"Postbox {entry_ref.address!r} is selected more than once; "
|
||||
"it will receive one message."
|
||||
),
|
||||
behavior=policy.duplicate_target,
|
||||
)
|
||||
issues.append(issue)
|
||||
status = _apply_behavior(status, policy.duplicate_target)
|
||||
continue
|
||||
seen_postbox_ids.add(entry_ref.id)
|
||||
resolved.append(
|
||||
_resolved_target_payload(
|
||||
target,
|
||||
entry_ref,
|
||||
position=position,
|
||||
)
|
||||
)
|
||||
if entry_ref.vacant:
|
||||
issue = _issue(
|
||||
code="postbox_target_vacant",
|
||||
message=(
|
||||
f"Postbox {entry_ref.address!r} currently has no function "
|
||||
"holder."
|
||||
),
|
||||
behavior=policy.vacant_target,
|
||||
)
|
||||
issues.append(issue)
|
||||
status = _apply_behavior(status, policy.vacant_target)
|
||||
return resolved, issues, status
|
||||
|
||||
|
||||
def delivery_catalog_payload(catalog: PostboxDeliveryCatalogRef) -> dict[str, Any]:
|
||||
return asdict(catalog)
|
||||
@@ -2,15 +2,31 @@ from __future__ import annotations
|
||||
|
||||
import csv
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from .addressing import effective_address_lists
|
||||
from .field_values import ignored_entry_field_overrides
|
||||
from .models import AttachmentConfig, CampaignConfig, EntryConfig, FieldType, SourceType, ZipArchiveConfig, ZipPasswordMode, ZipPasswordScope, ZipRuleMode
|
||||
from .models import (
|
||||
AttachmentConfig,
|
||||
CampaignConfig,
|
||||
DeliveryChannelPolicy,
|
||||
EntryConfig,
|
||||
FieldType,
|
||||
PostboxTargetConfig,
|
||||
SourceType,
|
||||
ZipArchiveConfig,
|
||||
ZipPasswordMode,
|
||||
ZipPasswordScope,
|
||||
ZipRuleMode,
|
||||
effective_delivery_channel_policy,
|
||||
effective_postbox_targets,
|
||||
)
|
||||
from ..attachments.resolver import resolve_campaign_attachments
|
||||
|
||||
|
||||
@@ -90,6 +106,8 @@ def _mapping_target_known(target: str, field_names: set[str]) -> bool:
|
||||
"merge_reply_to",
|
||||
"merge_bounce_to",
|
||||
"merge_disposition_notification_to",
|
||||
"merge_postbox_targets",
|
||||
"channel_policy",
|
||||
"combine_to",
|
||||
"combine_cc",
|
||||
"combine_bcc",
|
||||
@@ -337,10 +355,150 @@ def _global_value_issues(config: CampaignConfig, declared_names: set[str]) -> li
|
||||
]
|
||||
|
||||
|
||||
def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||
def _active_delivery_entries(config: CampaignConfig) -> list[EntryConfig]:
|
||||
if config.entries.is_inline:
|
||||
return [
|
||||
entry
|
||||
for entry in (config.entries.inline or [])
|
||||
if entry.active
|
||||
]
|
||||
return [config.entries.defaults or EntryConfig()]
|
||||
|
||||
|
||||
def _delivery_policies(config: CampaignConfig) -> set[DeliveryChannelPolicy]:
|
||||
return {
|
||||
effective_delivery_channel_policy(config, entry)
|
||||
for entry in _active_delivery_entries(config)
|
||||
}
|
||||
|
||||
|
||||
def _postbox_target_field_issues(
|
||||
config: CampaignConfig,
|
||||
target: PostboxTargetConfig,
|
||||
path: str,
|
||||
) -> list[SemanticIssue]:
|
||||
definitions = {field.name: field for field in config.fields}
|
||||
checks = (
|
||||
(
|
||||
target.organization_unit_field,
|
||||
FieldType.ORGANIZATION_UNIT,
|
||||
"organization unit",
|
||||
"organization_unit_field",
|
||||
),
|
||||
(
|
||||
target.function_field,
|
||||
FieldType.ORGANIZATION_FUNCTION,
|
||||
"organization function",
|
||||
"function_field",
|
||||
),
|
||||
(target.context_field, None, "context", "context_field"),
|
||||
)
|
||||
issues: list[SemanticIssue] = []
|
||||
for field_name, expected_type, label, key in checks:
|
||||
if not field_name:
|
||||
continue
|
||||
definition = definitions.get(field_name)
|
||||
if definition is None:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"postbox_target_field_missing",
|
||||
f"Postbox {label} field {field_name!r} is not declared.",
|
||||
f"{path}/{key}",
|
||||
)
|
||||
)
|
||||
elif expected_type is not None and definition.type != expected_type:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.WARNING,
|
||||
"postbox_target_field_type",
|
||||
(
|
||||
f"Postbox {label} field {field_name!r} should use "
|
||||
f"field type {expected_type.value!r}."
|
||||
),
|
||||
f"{path}/{key}",
|
||||
)
|
||||
)
|
||||
return issues
|
||||
|
||||
|
||||
def _postbox_delivery_issues(
|
||||
config: CampaignConfig,
|
||||
*,
|
||||
postbox_available: bool,
|
||||
) -> list[SemanticIssue]:
|
||||
issues: list[SemanticIssue] = []
|
||||
policies = _delivery_policies(config)
|
||||
if not any(policy.uses_postbox for policy in policies):
|
||||
return issues
|
||||
if not postbox_available:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"postbox_unavailable",
|
||||
(
|
||||
"This campaign uses Postbox delivery, but the Postbox "
|
||||
"module and its delivery directory are not active."
|
||||
),
|
||||
"/delivery/channel_policy",
|
||||
)
|
||||
)
|
||||
for entry_index, entry in enumerate(_active_delivery_entries(config)):
|
||||
policy = effective_delivery_channel_policy(config, entry)
|
||||
if not policy.uses_postbox:
|
||||
continue
|
||||
targets = effective_postbox_targets(config, entry)
|
||||
if not targets:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"postbox_target_missing",
|
||||
"Postbox delivery requires at least one target.",
|
||||
f"/entries/inline/{entry_index}/postbox_targets",
|
||||
)
|
||||
)
|
||||
continue
|
||||
seen_ids: set[str] = set()
|
||||
for target_index, target in enumerate(targets):
|
||||
target_path = (
|
||||
f"/entries/inline/{entry_index}/postbox_targets/"
|
||||
f"{target_index}"
|
||||
)
|
||||
if target.id in seen_ids:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.WARNING,
|
||||
"postbox_target_id_duplicate",
|
||||
f"Postbox target id {target.id!r} is repeated.",
|
||||
f"{target_path}/id",
|
||||
)
|
||||
)
|
||||
seen_ids.add(target.id)
|
||||
issues.extend(
|
||||
_postbox_target_field_issues(config, target, target_path)
|
||||
)
|
||||
return issues
|
||||
|
||||
|
||||
def _delivery_issues(
|
||||
config: CampaignConfig,
|
||||
*,
|
||||
postbox_available: bool,
|
||||
templates_available: bool,
|
||||
calendar_available: bool,
|
||||
) -> list[SemanticIssue]:
|
||||
issues: list[SemanticIssue] = []
|
||||
policies = _delivery_policies(config)
|
||||
uses_mail = any(policy.uses_mail for policy in policies)
|
||||
profile_id = (config.server.mail_profile_id or "").strip()
|
||||
if (config.campaign.mode == "send" or config.delivery.imap_append_sent.enabled) and not profile_id:
|
||||
if (
|
||||
(
|
||||
config.campaign.mode == "send"
|
||||
and uses_mail
|
||||
or config.delivery.imap_append_sent.enabled
|
||||
)
|
||||
and not profile_id
|
||||
):
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
@@ -350,7 +508,12 @@ def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||
)
|
||||
)
|
||||
capabilities = config.server.profile_capabilities
|
||||
if config.campaign.mode == "send" and profile_id and not capabilities.smtp_available:
|
||||
if (
|
||||
config.campaign.mode == "send"
|
||||
and uses_mail
|
||||
and profile_id
|
||||
and not capabilities.smtp_available
|
||||
):
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
@@ -368,13 +531,163 @@ def _delivery_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||
"/server/mail_profile_id",
|
||||
)
|
||||
)
|
||||
issues.extend(
|
||||
_postbox_delivery_issues(
|
||||
config,
|
||||
postbox_available=postbox_available,
|
||||
)
|
||||
)
|
||||
if any(policy.uses_print for policy in policies):
|
||||
if not templates_available:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"templates_unavailable",
|
||||
"Printable Campaign delivery requires the optional Templates renderer.",
|
||||
"/delivery/print/template_id",
|
||||
)
|
||||
)
|
||||
if not config.delivery.print.template_id:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"print_template_missing",
|
||||
"Select a published compatible template for printable Campaign output.",
|
||||
"/delivery/print/template_id",
|
||||
)
|
||||
)
|
||||
elif config.delivery.print.template_revision is None:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"print_template_revision_missing",
|
||||
"Printable delivery must pin one published template revision.",
|
||||
"/delivery/print/template_revision",
|
||||
)
|
||||
)
|
||||
for entry_index, entry in enumerate(_active_delivery_entries(config)):
|
||||
if not effective_delivery_channel_policy(config, entry).uses_print:
|
||||
continue
|
||||
if entry.print_target is None:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"print_target_missing",
|
||||
"Printable delivery requires an explicit postal or internal-mail target.",
|
||||
f"/entries/inline/{entry_index}/print_target",
|
||||
)
|
||||
)
|
||||
invitation = config.delivery.calendar_invitation
|
||||
if invitation.enabled:
|
||||
if not calendar_available:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_unavailable",
|
||||
"Calendar invitations require the optional Calendar module.",
|
||||
"/delivery/calendar_invitation/enabled",
|
||||
)
|
||||
)
|
||||
if not policies or any(not policy.uses_mail for policy in policies):
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_requires_mail",
|
||||
"Calendar invitations require Mail delivery for every active recipient.",
|
||||
"/delivery/channel_policy",
|
||||
)
|
||||
)
|
||||
if not invitation.calendar_id:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_calendar_missing",
|
||||
"Select a writable calendar for campaign invitation tracking.",
|
||||
"/delivery/calendar_invitation/calendar_id",
|
||||
)
|
||||
)
|
||||
if not invitation.start_at_template:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_start_missing",
|
||||
"Calendar invitations require a start date and time or a field template.",
|
||||
"/delivery/calendar_invitation/start_at_template",
|
||||
)
|
||||
)
|
||||
if invitation.timezone:
|
||||
try:
|
||||
ZoneInfo(invitation.timezone)
|
||||
except ZoneInfoNotFoundError:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_timezone_invalid",
|
||||
f"Unknown calendar invitation timezone: {invitation.timezone}",
|
||||
"/delivery/calendar_invitation/timezone",
|
||||
)
|
||||
)
|
||||
valid_start, fixed_start = _fixed_invitation_datetime(
|
||||
invitation.start_at_template
|
||||
)
|
||||
valid_end, fixed_end = _fixed_invitation_datetime(invitation.end_at_template)
|
||||
if invitation.start_at_template and not valid_start:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_start_invalid",
|
||||
"The fixed invitation start must be ISO 8601; recipient field templates are also supported.",
|
||||
"/delivery/calendar_invitation/start_at_template",
|
||||
)
|
||||
)
|
||||
if invitation.end_at_template and not valid_end:
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_end_invalid",
|
||||
"The fixed invitation end must be ISO 8601; recipient field templates are also supported.",
|
||||
"/delivery/calendar_invitation/end_at_template",
|
||||
)
|
||||
)
|
||||
if fixed_start and fixed_end and _invitation_range_invalid(
|
||||
fixed_start,
|
||||
fixed_end,
|
||||
):
|
||||
issues.append(
|
||||
_issue(
|
||||
Severity.ERROR,
|
||||
"calendar_invitation_range_invalid",
|
||||
"Calendar invitation end must be after its start.",
|
||||
"/delivery/calendar_invitation/end_at_template",
|
||||
)
|
||||
)
|
||||
return issues
|
||||
|
||||
|
||||
def _fixed_invitation_datetime(value: str | None) -> tuple[bool, datetime | None]:
|
||||
if not value:
|
||||
return True, None
|
||||
if "${" in value or "{{" in value:
|
||||
return True, None
|
||||
try:
|
||||
return True, datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
return False, None
|
||||
|
||||
|
||||
def _invitation_range_invalid(start_at: datetime, end_at: datetime) -> bool:
|
||||
if (start_at.tzinfo is None) != (end_at.tzinfo is None):
|
||||
return True
|
||||
return end_at <= start_at
|
||||
|
||||
|
||||
def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||
"""Require Campaign-owned sender data before a send-mode build."""
|
||||
|
||||
if config.campaign.mode != "send":
|
||||
if (
|
||||
config.campaign.mode != "send"
|
||||
or not any(policy.uses_mail for policy in _delivery_policies(config))
|
||||
):
|
||||
return []
|
||||
if config.entries.is_inline:
|
||||
return [
|
||||
@@ -385,7 +698,11 @@ def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
|
||||
f"/entries/inline/{index}/from",
|
||||
)
|
||||
for index, entry in enumerate(config.entries.inline or [])
|
||||
if entry.active and not effective_address_lists(config, entry)["from"]
|
||||
if (
|
||||
entry.active
|
||||
and effective_delivery_channel_policy(config, entry).uses_mail
|
||||
and not effective_address_lists(config, entry)["from"]
|
||||
)
|
||||
]
|
||||
if config.recipients.from_:
|
||||
return []
|
||||
@@ -611,6 +928,9 @@ def validate_campaign_config(
|
||||
*,
|
||||
campaign_file: str | Path | None = None,
|
||||
check_files: bool = False,
|
||||
postbox_available: bool = False,
|
||||
templates_available: bool = False,
|
||||
calendar_available: bool = False,
|
||||
) -> SemanticReport:
|
||||
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
|
||||
issues: list[SemanticIssue] = []
|
||||
@@ -622,7 +942,14 @@ def validate_campaign_config(
|
||||
issues.extend(_global_value_issues(config, declared_names))
|
||||
issues.extend(_attachment_path_issues(config))
|
||||
issues.extend(_zip_configuration_issues(config))
|
||||
issues.extend(_delivery_issues(config))
|
||||
issues.extend(
|
||||
_delivery_issues(
|
||||
config,
|
||||
postbox_available=postbox_available,
|
||||
templates_available=templates_available,
|
||||
calendar_available=calendar_available,
|
||||
)
|
||||
)
|
||||
issues.extend(_sender_issues(config))
|
||||
|
||||
entries = _entries_validation(
|
||||
|
||||
@@ -2,6 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable, Mapping
|
||||
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy import and_, or_
|
||||
|
||||
from govoplan_core.core.access import AccessDecisionProvenance, PrincipalRef
|
||||
@@ -14,12 +15,61 @@ from govoplan_core.core.campaigns import (
|
||||
CampaignPolicyContextProvider,
|
||||
CampaignRetentionProvider,
|
||||
)
|
||||
from govoplan_core.core.ownership import (
|
||||
OwnershipActionDecision,
|
||||
OwnershipSubjectRef,
|
||||
OwnershipTransferError,
|
||||
)
|
||||
from govoplan_core.security.module_permissions import scopes_grant_compatible
|
||||
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignShare
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignJob,
|
||||
CampaignShare,
|
||||
CampaignVersion,
|
||||
)
|
||||
|
||||
|
||||
READ_ACTIONS = {"campaigns:campaign:read", "campaign:read"}
|
||||
CAMPAIGN_RESOURCE_TYPES = {
|
||||
"campaign",
|
||||
"campaign_object",
|
||||
"campaigns:campaign",
|
||||
}
|
||||
CAMPAIGN_VERSION_RESOURCE_TYPES = {
|
||||
"campaign_version",
|
||||
"campaigns:version",
|
||||
}
|
||||
CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES = {
|
||||
"campaign_delivery_job",
|
||||
"campaign_job",
|
||||
"campaigns:delivery_job",
|
||||
}
|
||||
CAMPAIGN_REPORT_RESOURCE_TYPES = {
|
||||
"campaign_report",
|
||||
"campaigns:report",
|
||||
}
|
||||
|
||||
|
||||
def campaign_report_resource_id(
|
||||
*,
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
report_kind: str,
|
||||
) -> str:
|
||||
kind = report_kind.strip().lower()
|
||||
if not campaign_id or not version_id or not kind or ":" in kind:
|
||||
raise ValueError("Campaign report references require campaign, version, and a bounded kind")
|
||||
return f"{campaign_id}:{version_id}:{kind}"
|
||||
|
||||
|
||||
def _campaign_report_reference(
|
||||
resource_id: str,
|
||||
) -> tuple[str, str, str] | None:
|
||||
parts = resource_id.split(":", 2)
|
||||
if len(parts) != 3 or not all(part.strip() for part in parts):
|
||||
return None
|
||||
return parts[0], parts[1], parts[2].strip().lower()
|
||||
|
||||
|
||||
class CampaignMailPolicyContextService(CampaignMailPolicyContextProvider):
|
||||
@@ -120,20 +170,112 @@ class CampaignAccessService(CampaignAccessProvider):
|
||||
action: str,
|
||||
) -> tuple[AccessDecisionProvenance, ...]:
|
||||
normalized_type = resource_type.lower().strip()
|
||||
if normalized_type not in {"campaign", "campaign_object", "campaigns:campaign"}:
|
||||
return ()
|
||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id):
|
||||
return (
|
||||
AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=resource_id,
|
||||
tenant_id=principal.tenant_id,
|
||||
source="campaigns.not_found",
|
||||
details={"resource_type": "campaign", "found": False},
|
||||
),
|
||||
child_item: AccessDecisionProvenance | None = None
|
||||
campaign: Campaign | None
|
||||
if normalized_type in CAMPAIGN_RESOURCE_TYPES:
|
||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||
elif normalized_type in CAMPAIGN_VERSION_RESOURCE_TYPES:
|
||||
version = session.get(CampaignVersion, resource_id) # type: ignore[attr-defined]
|
||||
if version is None:
|
||||
return _missing_resource_provenance(
|
||||
principal,
|
||||
resource_type="campaign_version",
|
||||
resource_id=resource_id,
|
||||
)
|
||||
campaign = session.get(Campaign, version.campaign_id) # type: ignore[attr-defined]
|
||||
child_item = AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=version.id,
|
||||
label=f"Version {version.version_number}",
|
||||
tenant_id=campaign.tenant_id if campaign else principal.tenant_id,
|
||||
source="campaigns.version",
|
||||
details={
|
||||
"resource_type": "campaign_version",
|
||||
"campaign_id": version.campaign_id,
|
||||
"version_number": version.version_number,
|
||||
"workflow_state": version.workflow_state,
|
||||
"authorization_inherited_from": {
|
||||
"resource_type": "campaign",
|
||||
"resource_id": version.campaign_id,
|
||||
},
|
||||
},
|
||||
)
|
||||
items = [
|
||||
elif normalized_type in CAMPAIGN_DELIVERY_JOB_RESOURCE_TYPES:
|
||||
job = session.get(CampaignJob, resource_id) # type: ignore[attr-defined]
|
||||
if job is None:
|
||||
return _missing_resource_provenance(
|
||||
principal,
|
||||
resource_type="campaign_delivery_job",
|
||||
resource_id=resource_id,
|
||||
)
|
||||
campaign = session.get(Campaign, job.campaign_id) # type: ignore[attr-defined]
|
||||
child_item = AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=job.id,
|
||||
label=job.recipient_email or f"Recipient {job.entry_index}",
|
||||
tenant_id=job.tenant_id,
|
||||
source="campaigns.delivery_job",
|
||||
details={
|
||||
"resource_type": "campaign_delivery_job",
|
||||
"campaign_id": job.campaign_id,
|
||||
"campaign_version_id": job.campaign_version_id,
|
||||
"queue_status": job.queue_status,
|
||||
"send_status": job.send_status,
|
||||
"authorization_inherited_from": {
|
||||
"resource_type": "campaign",
|
||||
"resource_id": job.campaign_id,
|
||||
},
|
||||
},
|
||||
)
|
||||
elif normalized_type in CAMPAIGN_REPORT_RESOURCE_TYPES:
|
||||
reference = _campaign_report_reference(resource_id)
|
||||
if reference is None:
|
||||
return _missing_resource_provenance(
|
||||
principal,
|
||||
resource_type="campaign_report",
|
||||
resource_id=resource_id,
|
||||
reason="invalid_reference",
|
||||
)
|
||||
campaign_id, version_id, report_kind = reference
|
||||
campaign = session.get(Campaign, campaign_id) # type: ignore[attr-defined]
|
||||
version = session.get(CampaignVersion, version_id) # type: ignore[attr-defined]
|
||||
if (
|
||||
campaign is None
|
||||
or version is None
|
||||
or version.campaign_id != campaign.id
|
||||
):
|
||||
return _missing_resource_provenance(
|
||||
principal,
|
||||
resource_type="campaign_report",
|
||||
resource_id=resource_id,
|
||||
)
|
||||
child_item = AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=resource_id,
|
||||
label=report_kind,
|
||||
tenant_id=campaign.tenant_id,
|
||||
source="campaigns.report",
|
||||
details={
|
||||
"resource_type": "campaign_report",
|
||||
"campaign_id": campaign.id,
|
||||
"campaign_version_id": version.id,
|
||||
"report_kind": report_kind,
|
||||
"persisted": False,
|
||||
"authorization_inherited_from": {
|
||||
"resource_type": "campaign",
|
||||
"resource_id": campaign.id,
|
||||
},
|
||||
},
|
||||
)
|
||||
else:
|
||||
return ()
|
||||
if campaign is None or (principal.tenant_id and campaign.tenant_id != principal.tenant_id):
|
||||
return _missing_resource_provenance(
|
||||
principal,
|
||||
resource_type=normalized_type,
|
||||
resource_id=resource_id,
|
||||
)
|
||||
items = ([child_item] if child_item is not None else []) + [
|
||||
AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=campaign.id,
|
||||
@@ -149,7 +291,11 @@ class CampaignAccessService(CampaignAccessProvider):
|
||||
]
|
||||
items.extend(_owner_provenance(campaign, principal))
|
||||
items.extend(_tenant_admin_provenance(principal))
|
||||
permission_values = {"read", "write"} if action in READ_ACTIONS else {"write"}
|
||||
permission_values = (
|
||||
{"read", "write"}
|
||||
if action in READ_ACTIONS or action.strip().lower().endswith(":read")
|
||||
else {"write"}
|
||||
)
|
||||
shares = (
|
||||
session.query(CampaignShare) # type: ignore[attr-defined]
|
||||
.filter(
|
||||
@@ -187,6 +333,283 @@ def access_capability(context: object) -> CampaignAccessService:
|
||||
return CampaignAccessService()
|
||||
|
||||
|
||||
class CampaignOwnershipService:
|
||||
def current_owner(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
resource_id: str,
|
||||
) -> OwnershipSubjectRef | None:
|
||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||
if campaign is None or campaign.tenant_id != tenant_id:
|
||||
return None
|
||||
if campaign.owner_user_id:
|
||||
return OwnershipSubjectRef(type="user", id=campaign.owner_user_id)
|
||||
if campaign.owner_group_id:
|
||||
return OwnershipSubjectRef(type="group", id=campaign.owner_group_id)
|
||||
return None
|
||||
|
||||
def authorize_ownership_action(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
resource_id: str,
|
||||
action: str,
|
||||
actor: OwnershipSubjectRef,
|
||||
current_owner: OwnershipSubjectRef,
|
||||
target_owner: OwnershipSubjectRef,
|
||||
) -> OwnershipActionDecision:
|
||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||
if campaign is None or campaign.tenant_id != tenant_id:
|
||||
return OwnershipActionDecision(False, "Campaign was not found")
|
||||
if target_owner.type not in {"user", "group"}:
|
||||
return OwnershipActionDecision(
|
||||
False,
|
||||
"Campaign ownership can be assigned only to a user or group",
|
||||
)
|
||||
|
||||
actor_is_current_owner = _actor_controls_owner(actor, current_owner)
|
||||
actor_is_target = _actor_controls_owner(actor, target_owner)
|
||||
actor_can_share = scopes_grant_compatible(
|
||||
actor.scopes,
|
||||
"campaigns:campaign:share",
|
||||
)
|
||||
actor_can_accept_for_group = scopes_grant_compatible(
|
||||
actor.scopes,
|
||||
"campaigns:ownership:accept_group",
|
||||
)
|
||||
actor_can_recover = scopes_grant_compatible(
|
||||
actor.scopes,
|
||||
"campaigns:ownership:recover",
|
||||
)
|
||||
actor_can_read = (
|
||||
scopes_grant_compatible(
|
||||
actor.scopes,
|
||||
"campaigns:campaign:read",
|
||||
)
|
||||
and CampaignAccessService().can_read_campaign(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
campaign_id=resource_id,
|
||||
user_id=actor.id,
|
||||
group_ids=actor.group_ids,
|
||||
tenant_admin=scopes_grant_compatible(
|
||||
actor.scopes,
|
||||
"tenant:*",
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
if action in {
|
||||
"propose_transfer",
|
||||
"request_ownership",
|
||||
"request_recovery",
|
||||
}:
|
||||
target_decision = _valid_campaign_owner_target(
|
||||
tenant_id=tenant_id,
|
||||
target_owner=target_owner,
|
||||
)
|
||||
if target_decision is not None:
|
||||
return target_decision
|
||||
|
||||
if action == "view_transfer":
|
||||
allowed = (
|
||||
_actor_is_owner_member(actor, current_owner)
|
||||
or _actor_is_owner_member(actor, target_owner)
|
||||
or actor_can_recover
|
||||
)
|
||||
elif action in {
|
||||
"propose_transfer",
|
||||
"approve_requested_transfer",
|
||||
}:
|
||||
allowed = actor_is_current_owner and actor_can_share
|
||||
elif action == "cancel_transfer":
|
||||
allowed = (
|
||||
(actor_is_current_owner and actor_can_share)
|
||||
or actor_is_target
|
||||
)
|
||||
elif action == "request_ownership":
|
||||
allowed = actor_can_read and actor_is_target and (
|
||||
target_owner.type != "group" or actor_can_accept_for_group
|
||||
)
|
||||
elif action == "accept_transfer":
|
||||
allowed = target_owner.type == "user" and actor_is_target
|
||||
elif action == "accept_group_transfer":
|
||||
allowed = (
|
||||
target_owner.type == "group"
|
||||
and actor_is_target
|
||||
and actor_can_accept_for_group
|
||||
)
|
||||
elif action == "decline_transfer":
|
||||
allowed = (
|
||||
actor_is_target
|
||||
and (
|
||||
target_owner.type != "group"
|
||||
or actor_can_accept_for_group
|
||||
)
|
||||
) or (
|
||||
actor_is_current_owner
|
||||
and actor_can_share
|
||||
)
|
||||
elif action in {
|
||||
"request_recovery",
|
||||
"approve_recovery",
|
||||
"execute_recovery",
|
||||
}:
|
||||
allowed = actor_can_recover
|
||||
else:
|
||||
allowed = False
|
||||
|
||||
return OwnershipActionDecision(
|
||||
allowed=allowed,
|
||||
reason=None if allowed else f"Campaign ownership action is not allowed: {action}",
|
||||
)
|
||||
|
||||
def apply_owner(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
resource_id: str,
|
||||
expected_owner: OwnershipSubjectRef,
|
||||
target_owner: OwnershipSubjectRef,
|
||||
actor: OwnershipSubjectRef,
|
||||
reason: str | None,
|
||||
) -> None:
|
||||
del actor, reason
|
||||
campaign = session.get(Campaign, resource_id) # type: ignore[attr-defined]
|
||||
if campaign is None or campaign.tenant_id != tenant_id:
|
||||
raise OwnershipTransferError("Campaign was not found")
|
||||
if target_owner.type not in {"user", "group"}:
|
||||
raise OwnershipTransferError(
|
||||
"Campaign ownership can be assigned only to a user or group"
|
||||
)
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_clear_current_version_mail_profile_for_owner_transfer,
|
||||
)
|
||||
|
||||
try:
|
||||
_clear_current_version_mail_profile_for_owner_transfer(
|
||||
session, # type: ignore[arg-type]
|
||||
campaign,
|
||||
)
|
||||
except HTTPException as exc:
|
||||
raise OwnershipTransferError(
|
||||
str(
|
||||
exc.detail
|
||||
or "Campaign owner cannot be changed in its current state"
|
||||
)
|
||||
) from exc
|
||||
expected_filter = (
|
||||
(
|
||||
Campaign.owner_user_id == expected_owner.id,
|
||||
Campaign.owner_group_id.is_(None),
|
||||
)
|
||||
if expected_owner.type == "user"
|
||||
else (
|
||||
Campaign.owner_user_id.is_(None),
|
||||
Campaign.owner_group_id == expected_owner.id,
|
||||
)
|
||||
)
|
||||
updated = (
|
||||
session.query(Campaign) # type: ignore[attr-defined]
|
||||
.filter(
|
||||
Campaign.id == resource_id,
|
||||
Campaign.tenant_id == tenant_id,
|
||||
*expected_filter,
|
||||
)
|
||||
.update(
|
||||
{
|
||||
Campaign.owner_user_id: (
|
||||
target_owner.id
|
||||
if target_owner.type == "user"
|
||||
else None
|
||||
),
|
||||
Campaign.owner_group_id: (
|
||||
target_owner.id
|
||||
if target_owner.type == "group"
|
||||
else None
|
||||
),
|
||||
},
|
||||
synchronize_session=False,
|
||||
)
|
||||
)
|
||||
if updated != 1:
|
||||
raise OwnershipTransferError(
|
||||
"Campaign owner changed while the transfer was pending"
|
||||
)
|
||||
session.expire(campaign) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def _valid_campaign_owner_target(
|
||||
*,
|
||||
tenant_id: str,
|
||||
target_owner: OwnershipSubjectRef,
|
||||
) -> OwnershipActionDecision | None:
|
||||
from govoplan_campaign.backend.route_support import _access_directory
|
||||
|
||||
directory = _access_directory()
|
||||
if target_owner.type == "user":
|
||||
item = directory.get_user(target_owner.id)
|
||||
elif target_owner.type == "group":
|
||||
item = directory.get_group(target_owner.id)
|
||||
else:
|
||||
return OwnershipActionDecision(
|
||||
False,
|
||||
"Campaign ownership can be assigned only to a user or group",
|
||||
)
|
||||
if (
|
||||
item is None
|
||||
or item.tenant_id != tenant_id
|
||||
or item.status != "active"
|
||||
):
|
||||
return OwnershipActionDecision(
|
||||
False,
|
||||
f"Target {target_owner.type} is not active in this tenant",
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _actor_is_owner_member(
|
||||
actor: OwnershipSubjectRef,
|
||||
owner: OwnershipSubjectRef,
|
||||
) -> bool:
|
||||
if owner.type == "group":
|
||||
return owner.id in actor.group_ids
|
||||
return actor.type == owner.type and actor.id == owner.id
|
||||
|
||||
|
||||
def _actor_controls_owner(
|
||||
actor: OwnershipSubjectRef,
|
||||
owner: OwnershipSubjectRef,
|
||||
) -> bool:
|
||||
return _actor_is_owner_member(actor, owner)
|
||||
|
||||
|
||||
def _missing_resource_provenance(
|
||||
principal: PrincipalRef,
|
||||
*,
|
||||
resource_type: str,
|
||||
resource_id: str,
|
||||
reason: str = "not_found",
|
||||
) -> tuple[AccessDecisionProvenance, ...]:
|
||||
return (
|
||||
AccessDecisionProvenance(
|
||||
kind="resource",
|
||||
id=resource_id,
|
||||
tenant_id=principal.tenant_id,
|
||||
source="campaigns.not_found",
|
||||
details={
|
||||
"resource_type": resource_type,
|
||||
"found": False,
|
||||
"reason": reason,
|
||||
},
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _owner_provenance(campaign: Campaign, principal: PrincipalRef) -> tuple[AccessDecisionProvenance, ...]:
|
||||
if campaign.owner_user_id and campaign.owner_user_id == principal.membership_id:
|
||||
return (
|
||||
@@ -279,6 +702,12 @@ class CampaignDeliveryTaskService(CampaignDeliveryTaskProvider):
|
||||
|
||||
|
||||
def delivery_tasks_capability(context: object) -> CampaignDeliveryTaskService:
|
||||
from govoplan_campaign.backend.runtime import configure_runtime
|
||||
|
||||
configure_runtime(
|
||||
registry=getattr(context, "registry", None),
|
||||
settings=getattr(context, "settings", None),
|
||||
)
|
||||
return CampaignDeliveryTaskService()
|
||||
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ from govoplan_campaign.backend.db.models import (
|
||||
CampaignShare,
|
||||
CampaignVersion,
|
||||
ImapAppendAttempt,
|
||||
PostboxDeliveryAttempt,
|
||||
PrintOutputAttempt,
|
||||
SendAttempt,
|
||||
new_uuid,
|
||||
)
|
||||
@@ -55,7 +57,10 @@ def _record_campaign_changes(session: OrmSession, _flush_context: object, _insta
|
||||
_record_job_change(session, obj)
|
||||
elif isinstance(obj, CampaignIssue):
|
||||
_record_issue_change(session, obj)
|
||||
elif isinstance(obj, (SendAttempt, ImapAppendAttempt)):
|
||||
elif isinstance(
|
||||
obj,
|
||||
(SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt, PrintOutputAttempt),
|
||||
):
|
||||
_record_attempt_change(session, obj)
|
||||
|
||||
|
||||
@@ -182,8 +187,13 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
||||
"validation_status",
|
||||
"queue_status",
|
||||
"send_status",
|
||||
"delivery_channel_policy",
|
||||
"postbox_status",
|
||||
"print_status",
|
||||
"imap_status",
|
||||
"attempt_count",
|
||||
"postbox_attempt_count",
|
||||
"print_attempt_count",
|
||||
"last_error",
|
||||
"queued_at",
|
||||
"claimed_at",
|
||||
@@ -191,6 +201,9 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
||||
"outcome_unknown_at",
|
||||
"sent_at",
|
||||
"resolved_recipients",
|
||||
"delivery_provenance",
|
||||
"resolved_postbox_targets",
|
||||
"resolved_print_output",
|
||||
"resolved_attachments",
|
||||
"issues_snapshot",
|
||||
),
|
||||
@@ -217,6 +230,9 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
|
||||
"validation_status": job.validation_status,
|
||||
"queue_status": job.queue_status,
|
||||
"send_status": job.send_status,
|
||||
"delivery_channel_policy": job.delivery_channel_policy,
|
||||
"postbox_status": job.postbox_status,
|
||||
"print_status": job.print_status,
|
||||
"imap_status": job.imap_status,
|
||||
},
|
||||
)
|
||||
@@ -247,10 +263,27 @@ def _record_issue_change(session: OrmSession, issue: CampaignIssue) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppendAttempt) -> None:
|
||||
def _record_attempt_change(
|
||||
session: OrmSession,
|
||||
attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt | PrintOutputAttempt,
|
||||
) -> None:
|
||||
operation = _operation_for_object(
|
||||
attempt,
|
||||
changed_attrs=("status", "claim_token", "smtp_status_code", "smtp_response", "error_type", "error_message", "folder"),
|
||||
changed_attrs=(
|
||||
"status",
|
||||
"claim_token",
|
||||
"smtp_status_code",
|
||||
"smtp_response",
|
||||
"error_type",
|
||||
"error_message",
|
||||
"folder",
|
||||
"provider_delivery_id",
|
||||
"provider_message_id",
|
||||
"postbox_id",
|
||||
"render_id",
|
||||
"artifact_sha256",
|
||||
"evidence",
|
||||
),
|
||||
)
|
||||
if operation is None:
|
||||
return
|
||||
@@ -270,7 +303,15 @@ def _record_attempt_change(session: OrmSession, attempt: SendAttempt | ImapAppen
|
||||
payload={
|
||||
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
|
||||
"attempt_id": attempt_id,
|
||||
"attempt_kind": "imap" if isinstance(attempt, ImapAppendAttempt) else "smtp",
|
||||
"attempt_kind": (
|
||||
"postbox"
|
||||
if isinstance(attempt, PostboxDeliveryAttempt)
|
||||
else "print"
|
||||
if isinstance(attempt, PrintOutputAttempt)
|
||||
else "imap"
|
||||
if isinstance(attempt, ImapAppendAttempt)
|
||||
else "smtp"
|
||||
),
|
||||
"job_id": job.id,
|
||||
"version_id": job.campaign_version_id,
|
||||
},
|
||||
|
||||
@@ -8,6 +8,7 @@ from typing import Any
|
||||
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from govoplan_core.core.concurrency import strong_resource_etag
|
||||
from govoplan_core.db.base import Base, TimestampMixin
|
||||
|
||||
|
||||
@@ -83,6 +84,10 @@ class JobSendStatus(StrEnum):
|
||||
CLAIMED = "claimed"
|
||||
SENDING = "sending"
|
||||
SMTP_ACCEPTED = "smtp_accepted"
|
||||
POSTBOX_ACCEPTED = "postbox_accepted"
|
||||
PRINT_ACCEPTED = "print_accepted"
|
||||
DELIVERED = "delivered"
|
||||
PARTIALLY_ACCEPTED = "partially_accepted"
|
||||
SENT = "sent" # legacy value retained for existing databases/reports
|
||||
OUTCOME_UNKNOWN = "outcome_unknown"
|
||||
FAILED_TEMPORARY = "failed_temporary"
|
||||
@@ -90,6 +95,28 @@ class JobSendStatus(StrEnum):
|
||||
CANCELLED = "cancelled"
|
||||
|
||||
|
||||
class JobPostboxStatus(StrEnum):
|
||||
NOT_REQUESTED = "not_requested"
|
||||
PENDING = "pending"
|
||||
DELIVERING = "delivering"
|
||||
ACCEPTED = "accepted"
|
||||
ACCEPTED_VACANT = "accepted_vacant"
|
||||
PARTIALLY_ACCEPTED = "partially_accepted"
|
||||
REJECTED_TEMPORARY = "rejected_temporary"
|
||||
REJECTED_PERMANENT = "rejected_permanent"
|
||||
OUTCOME_UNKNOWN = "outcome_unknown"
|
||||
SKIPPED = "skipped"
|
||||
|
||||
|
||||
class JobPrintStatus(StrEnum):
|
||||
NOT_REQUESTED = "not_requested"
|
||||
READY = "ready"
|
||||
ACCEPTING = "accepting"
|
||||
ACCEPTED = "accepted"
|
||||
FAILED = "failed"
|
||||
SKIPPED = "skipped"
|
||||
|
||||
|
||||
class JobImapStatus(StrEnum):
|
||||
NOT_REQUESTED = "not_requested"
|
||||
PENDING = "pending"
|
||||
@@ -172,6 +199,11 @@ class CampaignVersion(Base, TimestampMixin):
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||
version_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
edit_revision: Mapped[int] = mapped_column(
|
||||
Integer,
|
||||
default=1,
|
||||
nullable=False,
|
||||
)
|
||||
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
|
||||
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
|
||||
source_filename: Mapped[str | None] = mapped_column(String(500))
|
||||
@@ -218,6 +250,18 @@ class CampaignVersion(Base, TimestampMixin):
|
||||
|
||||
campaign: Mapped[Campaign] = relationship(back_populates="versions")
|
||||
|
||||
__mapper_args__ = {
|
||||
"version_id_col": edit_revision,
|
||||
}
|
||||
|
||||
@property
|
||||
def strong_etag(self) -> str:
|
||||
return strong_resource_etag(
|
||||
"campaign_version",
|
||||
self.id,
|
||||
self.edit_revision,
|
||||
)
|
||||
|
||||
@property
|
||||
def mail_profile_migration_required(self) -> bool:
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_boundary_violations
|
||||
@@ -249,9 +293,37 @@ class CampaignJob(Base, TimestampMixin):
|
||||
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
|
||||
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
|
||||
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
|
||||
delivery_channel_policy: Mapped[str] = mapped_column(
|
||||
String(30),
|
||||
default="mail",
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
postbox_status: Mapped[str] = mapped_column(
|
||||
String(50),
|
||||
default=JobPostboxStatus.NOT_REQUESTED.value,
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
print_status: Mapped[str] = mapped_column(
|
||||
String(50),
|
||||
default=JobPrintStatus.NOT_REQUESTED.value,
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
|
||||
|
||||
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
postbox_attempt_count: Mapped[int] = mapped_column(
|
||||
Integer,
|
||||
default=0,
|
||||
nullable=False,
|
||||
)
|
||||
print_attempt_count: Mapped[int] = mapped_column(
|
||||
Integer,
|
||||
default=0,
|
||||
nullable=False,
|
||||
)
|
||||
last_error: Mapped[str | None] = mapped_column(Text)
|
||||
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
@@ -263,6 +335,20 @@ class CampaignJob(Base, TimestampMixin):
|
||||
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||
delivery_provenance: Mapped[dict[str, Any]] = mapped_column(
|
||||
JSON,
|
||||
default=dict,
|
||||
nullable=False,
|
||||
)
|
||||
resolved_postbox_targets: Mapped[list[dict[str, Any]]] = mapped_column(
|
||||
JSON,
|
||||
default=list,
|
||||
nullable=False,
|
||||
)
|
||||
resolved_print_output: Mapped[dict[str, Any] | None] = mapped_column(
|
||||
JSON,
|
||||
nullable=True,
|
||||
)
|
||||
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||
|
||||
@@ -327,6 +413,139 @@ class SendAttempt(Base, TimestampMixin):
|
||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class CampaignMessageAction(Base, TimestampMixin):
|
||||
__tablename__ = "campaign_message_actions"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"tenant_id",
|
||||
"idempotency_key",
|
||||
name="uq_campaign_message_actions_idempotency",
|
||||
),
|
||||
Index(
|
||||
"ix_campaign_message_actions_job_created",
|
||||
"job_id",
|
||||
"created_at",
|
||||
),
|
||||
Index(
|
||||
"ix_campaign_message_actions_campaign_kind",
|
||||
"campaign_id",
|
||||
"kind",
|
||||
"status",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||
campaign_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaigns.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
campaign_version_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaign_versions.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
job_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
|
||||
idempotency_key: Mapped[str] = mapped_column(String(200), nullable=False)
|
||||
canonical_request_hash: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
context: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
|
||||
actor_user_id: Mapped[str | None] = mapped_column(
|
||||
ForeignKey("access_users.id", ondelete="SET NULL"),
|
||||
nullable=True,
|
||||
index=True,
|
||||
)
|
||||
actor_api_key_id: Mapped[str | None] = mapped_column(String(36), nullable=True)
|
||||
message_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
message_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
recipient_manifest_sha256: Mapped[str] = mapped_column(
|
||||
String(64),
|
||||
nullable=False,
|
||||
)
|
||||
recipient_count: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
prior_send_status: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||
prior_attempt_count: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
final_send_status: Mapped[str | None] = mapped_column(String(50), nullable=True)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(50),
|
||||
default="initiated",
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
refusal_summary: Mapped[dict[str, Any]] = mapped_column(
|
||||
JSON,
|
||||
default=dict,
|
||||
nullable=False,
|
||||
)
|
||||
error_type: Mapped[str | None] = mapped_column(String(120), nullable=True)
|
||||
error_message: Mapped[str | None] = mapped_column(String(500), nullable=True)
|
||||
linked_send_attempt_id: Mapped[str | None] = mapped_column(
|
||||
ForeignKey("send_attempts.id", ondelete="SET NULL"),
|
||||
nullable=True,
|
||||
index=True,
|
||||
)
|
||||
effect_started_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=True,
|
||||
)
|
||||
completed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=True,
|
||||
)
|
||||
|
||||
|
||||
class CampaignMessageActionAttempt(Base, TimestampMixin):
|
||||
__tablename__ = "campaign_message_action_attempts"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"action_id",
|
||||
"attempt_number",
|
||||
name="uq_campaign_message_action_attempt_number",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
action_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaign_message_actions.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(50),
|
||||
default="initiated",
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
started_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=False,
|
||||
)
|
||||
effect_started_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=True,
|
||||
)
|
||||
completed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=True,
|
||||
)
|
||||
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
outcome_code: Mapped[str | None] = mapped_column(String(80), nullable=True)
|
||||
diagnostic_summary: Mapped[str | None] = mapped_column(
|
||||
String(500),
|
||||
nullable=True,
|
||||
)
|
||||
|
||||
|
||||
class ImapAppendAttempt(Base, TimestampMixin):
|
||||
__tablename__ = "imap_append_attempts"
|
||||
__table_args__ = (
|
||||
@@ -342,6 +561,112 @@ class ImapAppendAttempt(Base, TimestampMixin):
|
||||
error_message: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
|
||||
class PostboxDeliveryAttempt(Base, TimestampMixin):
|
||||
__tablename__ = "campaign_postbox_delivery_attempts"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"job_id",
|
||||
"target_key",
|
||||
"attempt_number",
|
||||
name="uq_campaign_postbox_attempt_target_number",
|
||||
),
|
||||
Index(
|
||||
"ix_campaign_postbox_attempt_idempotency",
|
||||
"tenant_id",
|
||||
"idempotency_key",
|
||||
),
|
||||
Index(
|
||||
"ix_campaign_postbox_attempt_job_status",
|
||||
"job_id",
|
||||
"status",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(
|
||||
String(36),
|
||||
primary_key=True,
|
||||
default=new_uuid,
|
||||
)
|
||||
tenant_id: Mapped[str] = mapped_column(
|
||||
String(36),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
job_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
target_key: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
target_index: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(50),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
target_snapshot: Mapped[dict[str, Any]] = mapped_column(
|
||||
JSON,
|
||||
default=dict,
|
||||
nullable=False,
|
||||
)
|
||||
provider_delivery_id: Mapped[str | None] = mapped_column(String(36))
|
||||
provider_message_id: Mapped[str | None] = mapped_column(String(36))
|
||||
postbox_id: Mapped[str | None] = mapped_column(String(36), index=True)
|
||||
address: Mapped[str | None] = mapped_column(String(500))
|
||||
holder_count: Mapped[int | None] = mapped_column(Integer)
|
||||
vacant: Mapped[bool | None] = mapped_column(Boolean)
|
||||
duplicate: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||
evidence: Mapped[dict[str, Any]] = mapped_column(
|
||||
JSON,
|
||||
default=dict,
|
||||
nullable=False,
|
||||
)
|
||||
error_type: Mapped[str | None] = mapped_column(String(255))
|
||||
error_code: Mapped[str | None] = mapped_column(String(100))
|
||||
error_message: Mapped[str | None] = mapped_column(Text)
|
||||
started_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
)
|
||||
finished_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
)
|
||||
|
||||
|
||||
class PrintOutputAttempt(Base, TimestampMixin):
|
||||
__tablename__ = "campaign_print_output_attempts"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"job_id",
|
||||
"attempt_number",
|
||||
name="uq_campaign_print_attempt_job_number",
|
||||
),
|
||||
UniqueConstraint(
|
||||
"tenant_id",
|
||||
"idempotency_key",
|
||||
name="uq_campaign_print_attempt_idempotency",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||
job_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
index=True,
|
||||
)
|
||||
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
status: Mapped[str] = mapped_column(String(50), nullable=False, index=True)
|
||||
render_id: Mapped[str | None] = mapped_column(String(36), index=True)
|
||||
artifact_sha256: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
evidence: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
|
||||
error_message: Mapped[str | None] = mapped_column(Text)
|
||||
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
|
||||
|
||||
__all__ = [
|
||||
@@ -359,8 +684,12 @@ __all__ = [
|
||||
"IssueSeverity",
|
||||
"JobBuildStatus",
|
||||
"JobImapStatus",
|
||||
"JobPostboxStatus",
|
||||
"JobPrintStatus",
|
||||
"JobQueueStatus",
|
||||
"JobSendStatus",
|
||||
"JobValidationStatus",
|
||||
"SendAttempt",
|
||||
"PostboxDeliveryAttempt",
|
||||
"PrintOutputAttempt",
|
||||
]
|
||||
|
||||
@@ -43,6 +43,11 @@ _FILES_INTEGRATION = "files.campaign_attachments"
|
||||
_MAIL_INTEGRATION = "mail.campaign_delivery"
|
||||
_ADDRESSES_LOOKUP_INTEGRATION = "addresses.lookup"
|
||||
_ADDRESSES_SOURCE_INTEGRATION = "addresses.recipient_source"
|
||||
_DISTRIBUTION_LIST_SOURCE_INTEGRATION = "dist_lists.source"
|
||||
_DISTRIBUTION_LIST_EXPAND_INTEGRATION = "dist_lists.expand"
|
||||
_TEMPLATE_CATALOG_INTEGRATION = "templates.catalog"
|
||||
_TEMPLATE_RENDERER_INTEGRATION = "templates.renderer"
|
||||
_CALENDAR_INVITATION_INTEGRATION = "calendar.invitations"
|
||||
_NOTIFICATIONS_INTEGRATION = "notifications.dispatch"
|
||||
|
||||
|
||||
@@ -219,6 +224,84 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
related_topic_ids=("campaigns.workflow.import-recipients", "campaigns.workflow.prepare-validate-and-build"),
|
||||
related_modules=("addresses",),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.import-distribution-list",
|
||||
title="Freeze a Distribution List into a campaign",
|
||||
summary="Resolve a reusable audience, inspect its channel and policy decisions, and copy an immutable snapshot into the current campaign version.",
|
||||
body="A Distribution List stays live and versioned in its owning module. Campaign freezes one exact expansion; later list or provider changes only produce a drift warning and never rewrite the saved Campaign recipients.",
|
||||
order=33,
|
||||
audience=("campaign_manager", "campaign_author"),
|
||||
required_modules=("campaigns", "dist_lists"),
|
||||
required_capabilities=(
|
||||
_DISTRIBUTION_LIST_SOURCE_INTEGRATION,
|
||||
_DISTRIBUTION_LIST_EXPAND_INTEGRATION,
|
||||
),
|
||||
required_scopes=(
|
||||
"campaigns:campaign:read",
|
||||
"campaigns:campaign:update",
|
||||
"campaigns:recipient:read",
|
||||
"campaigns:recipient:write",
|
||||
"campaigns:recipient:import",
|
||||
),
|
||||
route="/campaigns/{campaign_id}/recipients",
|
||||
screen="Recipient data",
|
||||
help_contexts=("campaign.recipients", "campaign.recipient-data"),
|
||||
prerequisites=(
|
||||
"A visible Distribution List can be expanded for campaign delivery.",
|
||||
"The current Campaign version is editable.",
|
||||
),
|
||||
steps=(
|
||||
"Open Recipient data and select Import Distribution List.",
|
||||
"Choose the list, requested channels, and any declared parameters, then preview the expansion.",
|
||||
"Review included and excluded recipients, stale provider evidence, diagnostics, and every visible primary and optional fallback route.",
|
||||
"Choose append or replace, freeze and import the expansion, inspect the copied rows, and save the Campaign version.",
|
||||
"Use the drift warning for a deliberate refresh when the reusable list changes later.",
|
||||
),
|
||||
outcome="A Campaign-local recipient snapshot with immutable audience, provider, policy, and channel-decision evidence.",
|
||||
verification="The saved recipient rows retain the list revision and snapshot reference, and later list changes do not alter them automatically.",
|
||||
related_topic_ids=("campaigns.workflow.import-recipients", "campaigns.workflow.prepare-validate-and-build"),
|
||||
related_modules=("dist_lists", "templates", "postbox"),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.prepare-printable-delivery",
|
||||
title="Prepare governed printable delivery",
|
||||
summary="Select a published output template, build one deterministic artifact, and review its route and hash evidence before postal or internal-mail distribution.",
|
||||
body="Printable delivery is optional and provider-neutral. Campaign freezes recipient route decisions while Templates owns compatibility and rendering; Files may own the resulting managed artifact. Ordered fallback is used only after a confirmed rejection before acceptance and never after an accepted or outcome-unknown digital effect.",
|
||||
order=34,
|
||||
audience=("campaign_manager", "campaign_author", "campaign_reviewer"),
|
||||
required_modules=("campaigns", "templates"),
|
||||
required_capabilities=(
|
||||
_TEMPLATE_CATALOG_INTEGRATION,
|
||||
_TEMPLATE_RENDERER_INTEGRATION,
|
||||
),
|
||||
required_scopes=(
|
||||
"campaigns:campaign:read",
|
||||
"campaigns:campaign:update",
|
||||
"campaigns:campaign:validate",
|
||||
"campaigns:campaign:build",
|
||||
"campaigns:recipient:read",
|
||||
"templates:template:read",
|
||||
"templates:template:render",
|
||||
),
|
||||
route="/campaigns/{campaign_id}/template",
|
||||
screen="Template",
|
||||
help_contexts=("campaign.template", "campaign.review-send"),
|
||||
prerequisites=(
|
||||
"At least one included recipient has an explicit postal or internal-mail route.",
|
||||
"A compatible Templates definition is published and visible to you.",
|
||||
),
|
||||
steps=(
|
||||
"Open Template and select the published printable template, output format, and storage choice.",
|
||||
"Save, validate, and resolve every missing-field or compatibility error.",
|
||||
"Build the Campaign to generate one deterministic artifact for the frozen printable recipients.",
|
||||
"In Review and send, download and inspect the artifact and compare its template, input, and output hashes.",
|
||||
"Complete review and execute delivery; use reports to verify per-recipient print acceptance and route provenance.",
|
||||
),
|
||||
outcome="A reviewed printable artifact and idempotent per-recipient distribution evidence.",
|
||||
verification="The build summary exposes the artifact and hashes, and the Campaign report records print status and one acceptance attempt per routed recipient.",
|
||||
related_topic_ids=("campaigns.workflow.import-distribution-list", "campaigns.workflow.prepare-validate-and-build"),
|
||||
related_modules=("templates", "files", "dist_lists"),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.use-managed-attachments",
|
||||
title="Use managed files as campaign attachments",
|
||||
@@ -258,6 +341,43 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
),
|
||||
related_modules=("files",),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.send-calendar-invitations",
|
||||
title="Send individualized calendar invitations",
|
||||
summary="Freeze one iCalendar request per recipient, deliver it through Mail, and review live attendee answers from Calendar.",
|
||||
body="Campaign owns the recipient expansion, exact invitation request, message delivery evidence, and report. Calendar owns the mirrored VEVENT and attendee answer state. The mirror is created only after a delivery channel accepts the message; a Calendar failure never rewrites accepted Mail evidence. Mail can forward METHOD:REPLY parts from a configured IMAP delivery-status source.",
|
||||
order=35,
|
||||
audience=("campaign_manager", "campaign_sender"),
|
||||
required_modules=("campaigns", "mail", "calendar"),
|
||||
required_capabilities=(_MAIL_INTEGRATION, _CALENDAR_INVITATION_INTEGRATION),
|
||||
required_scopes=(
|
||||
"campaigns:campaign:read",
|
||||
"campaigns:campaign:update",
|
||||
"campaigns:campaign:build",
|
||||
"campaigns:campaign:send",
|
||||
"campaigns:recipient:read",
|
||||
"calendar:calendar:read",
|
||||
),
|
||||
route="/campaigns/{campaign_id}/global-settings",
|
||||
screen="Campaign settings",
|
||||
help_contexts=("campaign.global-settings", "campaign.report"),
|
||||
prerequisites=(
|
||||
"Mail and Calendar are active and you can select a writable calendar.",
|
||||
"Every queueable recipient has a Mail To address and the invitation start template resolves to ISO 8601.",
|
||||
),
|
||||
steps=(
|
||||
"Open Campaign settings, enable individualized Calendar invitations, and select the tracking calendar.",
|
||||
"Enter summary, start, optional end, timezone, location, description, and category templates.",
|
||||
"Validate and build; inspect the frozen METHOD:REQUEST attachment for each recipient before delivery.",
|
||||
"Deliver the reviewed build and use Report to compare Mail delivery with the live RSVP state.",
|
||||
"Configure a Mail IMAP delivery-status source when inbound METHOD:REPLY reconciliation is required.",
|
||||
),
|
||||
outcome="Individually delivered invitations with correlated Calendar events and recipient-level RSVP reporting.",
|
||||
verification="The Campaign job shows accepted delivery, a mirrored Calendar event ID, and the current attendee status; repeated mailbox ingestion does not duplicate the response effect.",
|
||||
related_topic_ids=("campaigns.workflow.prepare-validate-and-build", "campaigns.workflow.view-delivery-report"),
|
||||
related_modules=("mail", "calendar"),
|
||||
limitations=("Recurring Campaign invitation series require a separate series workflow; this slice creates individual VEVENT requests."),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.queue-delivery",
|
||||
title="Queue a campaign for controlled delivery",
|
||||
@@ -292,7 +412,7 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
related_topic_ids=("campaigns.workflow.complete-review", "campaigns.workflow.retry-and-reconcile"),
|
||||
links=(
|
||||
DocumentationLink(label="Campaigns", href="/campaigns", kind="runtime"),
|
||||
DocumentationLink(label="Campaign operator queue", href="/operator", kind="runtime"),
|
||||
DocumentationLink(label="Campaign operator queue", href="/campaigns/queue", kind="runtime"),
|
||||
),
|
||||
related_modules=("mail", "notifications"),
|
||||
),
|
||||
@@ -338,7 +458,7 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
order=37,
|
||||
audience=("campaign_aggregate_reader", "campaign_reader", "campaign_manager"),
|
||||
required_scopes=("campaigns:report:read",),
|
||||
route="/reports",
|
||||
route="/campaigns/reports",
|
||||
screen="Reports",
|
||||
help_contexts=("campaign.report",),
|
||||
prerequisites=("The campaign is owned by or explicitly shared with you, or you hold tenant-wide authority.",),
|
||||
@@ -351,7 +471,7 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
outcome="A business-level Campaign outcome view with small-group and recipient privacy preserved.",
|
||||
verification="No row, address, message, attachment, diagnostic, filter, drill-down, or export action is available from the aggregate view.",
|
||||
related_topic_ids=("campaigns.workflow.view-delivery-report",),
|
||||
links=(DocumentationLink(label="Aggregate Campaign reports", href="/reports", kind="runtime"),),
|
||||
links=(DocumentationLink(label="Aggregate Campaign reports", href="/campaigns/reports", kind="runtime"),),
|
||||
),
|
||||
_workflow_topic(
|
||||
topic_id="campaigns.workflow.view-delivery-report",
|
||||
@@ -449,14 +569,13 @@ CAMPAIGN_USER_DOCUMENTATION = (
|
||||
steps=(
|
||||
"Open Report and resolve every active or outcome-unknown delivery state.",
|
||||
"Confirm that the campaign should leave active work while its evidence remains retained.",
|
||||
"Invoke the authorized Archive action from a supporting client.",
|
||||
"Open the campaign overview, choose Archive campaign, and confirm the retained-evidence consequence.",
|
||||
"Reopen or query the campaign and confirm its state is Archived.",
|
||||
),
|
||||
outcome="An archived campaign whose versions, reports, and audit evidence remain preserved.",
|
||||
verification="The campaign state is Archived and its report remains available. Ask an authorized audit reader to verify the platform audit event.",
|
||||
related_topic_ids=("campaigns.workflow.view-delivery-report", "campaigns.workflow.delete-untouched-draft"),
|
||||
limitations=(
|
||||
"The current Campaign Web UI does not yet expose the archive action; use an authorized supporting client or API.",
|
||||
"The Campaign-local Audit page is not integrated yet; audit verification uses the platform audit surface or API.",
|
||||
),
|
||||
),
|
||||
@@ -669,6 +788,23 @@ def _integration_summary(registry: object, principal: object) -> tuple[tuple[str
|
||||
else:
|
||||
limitations.append("Automatic in-app Campaign status notifications are not configured.")
|
||||
|
||||
calendar_available = _integration_available(
|
||||
registry,
|
||||
_CALENDAR_INVITATION_INTEGRATION,
|
||||
)
|
||||
if calendar_available and mail_available:
|
||||
configured.append(
|
||||
"Installed composition: Campaign can freeze individualized iCalendar requests, mirror accepted deliveries into Calendar, and report live attendee answers."
|
||||
)
|
||||
elif calendar_available:
|
||||
limitations.append(
|
||||
"Calendar invitation tracking is active, but Mail is not available to deliver Campaign invitation messages."
|
||||
)
|
||||
else:
|
||||
limitations.append(
|
||||
"Calendar-backed invitation and RSVP tracking is not available in this composition."
|
||||
)
|
||||
|
||||
return tuple(configured), tuple(limitations)
|
||||
|
||||
|
||||
|
||||
@@ -7,11 +7,57 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterator
|
||||
|
||||
from govoplan_core.core.approvals import (
|
||||
ApprovalCheck,
|
||||
ApprovalRequestCreateCommand,
|
||||
ApprovalRequestProvider,
|
||||
ApprovalRequestRef,
|
||||
CAPABILITY_APPROVAL_REQUESTS,
|
||||
)
|
||||
from govoplan_core.core.calendar import (
|
||||
CAPABILITY_CALENDAR_INVITATIONS,
|
||||
CalendarInvitationAttendeeRequest,
|
||||
CalendarInvitationCalendarRef,
|
||||
CalendarInvitationProvider,
|
||||
CalendarInvitationRef,
|
||||
CalendarInvitationRequest,
|
||||
)
|
||||
from govoplan_core.core.postbox import (
|
||||
CAPABILITY_POSTBOX_DIRECTORY,
|
||||
CAPABILITY_POSTBOX_DELIVERY,
|
||||
CAPABILITY_POSTBOX_EVIDENCE,
|
||||
PostboxDeliveryCatalogRef,
|
||||
PostboxDeliveryProvider,
|
||||
PostboxDeliveryReceiptSummaryRef,
|
||||
PostboxDeliveryRequest,
|
||||
PostboxDeliveryResult,
|
||||
PostboxDirectoryEntryRef,
|
||||
PostboxDirectoryProvider,
|
||||
PostboxEvidenceProvider,
|
||||
PostboxTargetRef,
|
||||
)
|
||||
from govoplan_core.core.templates import (
|
||||
CAPABILITY_TEMPLATE_CATALOG,
|
||||
CAPABILITY_TEMPLATE_RENDERER,
|
||||
TemplateCatalogProvider,
|
||||
TemplateCompatibility,
|
||||
TemplateRef,
|
||||
TemplateRenderRequest,
|
||||
TemplateRenderResult,
|
||||
TemplateRendererProvider,
|
||||
)
|
||||
from govoplan_campaign.backend.runtime import capability
|
||||
|
||||
|
||||
FILES_CAPABILITY = "files.campaign_attachments"
|
||||
MAIL_CAPABILITY = "mail.campaign_delivery"
|
||||
POSTBOX_CAPABILITY = CAPABILITY_POSTBOX_DELIVERY
|
||||
POSTBOX_DIRECTORY_CAPABILITY = CAPABILITY_POSTBOX_DIRECTORY
|
||||
POSTBOX_EVIDENCE_CAPABILITY = CAPABILITY_POSTBOX_EVIDENCE
|
||||
APPROVALS_CAPABILITY = CAPABILITY_APPROVAL_REQUESTS
|
||||
TEMPLATE_CATALOG_CAPABILITY = CAPABILITY_TEMPLATE_CATALOG
|
||||
TEMPLATE_RENDERER_CAPABILITY = CAPABILITY_TEMPLATE_RENDERER
|
||||
CALENDAR_INVITATIONS_CAPABILITY = CAPABILITY_CALENDAR_INVITATIONS
|
||||
|
||||
|
||||
class OptionalModuleUnavailable(RuntimeError):
|
||||
@@ -23,7 +69,9 @@ class SmtpConfigurationError(RuntimeError):
|
||||
|
||||
|
||||
class SmtpSendError(RuntimeError):
|
||||
def __init__(self, message: str, *, temporary: bool = False, outcome_unknown: bool = False) -> None:
|
||||
def __init__(
|
||||
self, message: str, *, temporary: bool = False, outcome_unknown: bool = False
|
||||
) -> None:
|
||||
super().__init__(message)
|
||||
self.temporary = temporary
|
||||
self.outcome_unknown = outcome_unknown
|
||||
@@ -50,6 +98,26 @@ class MailProfileError(OptionalModuleUnavailable):
|
||||
pass
|
||||
|
||||
|
||||
class MailDeliveryCommandError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class PostboxDeliveryUnavailable(OptionalModuleUnavailable):
|
||||
pass
|
||||
|
||||
|
||||
class ApprovalGateUnavailable(OptionalModuleUnavailable):
|
||||
pass
|
||||
|
||||
|
||||
class TemplateOutputUnavailable(OptionalModuleUnavailable):
|
||||
pass
|
||||
|
||||
|
||||
class CalendarInvitationUnavailable(OptionalModuleUnavailable):
|
||||
pass
|
||||
|
||||
|
||||
class _PreparedCampaignSnapshot:
|
||||
def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None:
|
||||
self._directory = directory
|
||||
@@ -78,20 +146,30 @@ class FilesCampaignIntegration:
|
||||
yield prepared
|
||||
return
|
||||
|
||||
raw_json = kwargs.get("raw_json") if isinstance(kwargs.get("raw_json"), dict) else {}
|
||||
raw_json = (
|
||||
kwargs.get("raw_json") if isinstance(kwargs.get("raw_json"), dict) else {}
|
||||
)
|
||||
prefix = str(kwargs.get("prefix") or "govoplan-campaign-")
|
||||
directory = Path(tempfile.mkdtemp(prefix=prefix))
|
||||
snapshot = _PreparedCampaignSnapshot(directory, directory / "campaign.json", raw_json)
|
||||
snapshot.path.write_text(json.dumps(raw_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
snapshot = _PreparedCampaignSnapshot(
|
||||
directory, directory / "campaign.json", raw_json
|
||||
)
|
||||
snapshot.path.write_text(
|
||||
json.dumps(raw_json, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
try:
|
||||
yield snapshot
|
||||
finally:
|
||||
snapshot.cleanup()
|
||||
|
||||
def managed_match_payloads(self, matches: Any, managed_files_by_local_path: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
def managed_match_payloads(
|
||||
self, matches: Any, managed_files_by_local_path: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
if self._delegate is None:
|
||||
return []
|
||||
return self._delegate.managed_match_payloads(matches, managed_files_by_local_path)
|
||||
return self._delegate.managed_match_payloads(
|
||||
matches, managed_files_by_local_path
|
||||
)
|
||||
|
||||
def public_attachment_summary_payload(self, attachment: Any) -> dict[str, Any]:
|
||||
if self._delegate is not None:
|
||||
@@ -102,20 +180,30 @@ class FilesCampaignIntegration:
|
||||
return dict(attachment)
|
||||
return {"path": str(attachment)}
|
||||
|
||||
def annotate_built_messages_with_managed_files(self, built_messages: Any, managed_files_by_local_path: dict[str, Any]) -> None:
|
||||
def annotate_built_messages_with_managed_files(
|
||||
self, built_messages: Any, managed_files_by_local_path: dict[str, Any]
|
||||
) -> None:
|
||||
if self._delegate is not None:
|
||||
self._delegate.annotate_built_messages_with_managed_files(built_messages, managed_files_by_local_path)
|
||||
self._delegate.annotate_built_messages_with_managed_files(
|
||||
built_messages, managed_files_by_local_path
|
||||
)
|
||||
|
||||
def record_campaign_attachment_uses_for_jobs(self, session: Any, jobs: Any, *, stage: str) -> None:
|
||||
def record_campaign_attachment_uses_for_jobs(
|
||||
self, session: Any, jobs: Any, *, stage: str
|
||||
) -> None:
|
||||
if self._delegate is not None:
|
||||
self._delegate.record_campaign_attachment_uses_for_jobs(session, jobs, stage=stage)
|
||||
self._delegate.record_campaign_attachment_uses_for_jobs(
|
||||
session, jobs, stage=stage
|
||||
)
|
||||
|
||||
def current_version_and_blob(self, session: Any, asset: Any) -> tuple[Any, Any]:
|
||||
if self._delegate is None:
|
||||
raise OptionalModuleUnavailable("Files module is not available")
|
||||
return self._delegate.current_version_and_blob(session, asset)
|
||||
|
||||
def share_assets_with_campaign(self, session: Any, **kwargs: Any) -> list[dict[str, Any]]:
|
||||
def share_assets_with_campaign(
|
||||
self, session: Any, **kwargs: Any
|
||||
) -> list[dict[str, Any]]:
|
||||
if self._delegate is None:
|
||||
raise OptionalModuleUnavailable("Files module is not available")
|
||||
return self._delegate.share_assets_with_campaign(session, **kwargs)
|
||||
@@ -129,7 +217,9 @@ class MailCampaignIntegration:
|
||||
def __init__(self, delegate: Any | None = None) -> None:
|
||||
self._delegate = delegate
|
||||
if delegate is not None:
|
||||
self.MailProfileError = getattr(delegate, "MailProfileError", MailProfileError)
|
||||
self.MailProfileError = getattr(
|
||||
delegate, "MailProfileError", MailProfileError
|
||||
)
|
||||
|
||||
MailProfileError = MailProfileError
|
||||
SmtpConfigurationError = SmtpConfigurationError
|
||||
@@ -141,31 +231,49 @@ class MailCampaignIntegration:
|
||||
def available(self) -> bool:
|
||||
return self._delegate is not None
|
||||
|
||||
@property
|
||||
def durable_delivery_available(self) -> bool:
|
||||
return self._delegate is not None and callable(
|
||||
getattr(self._delegate, "submit_delivery_command", None)
|
||||
)
|
||||
|
||||
def _require(self) -> Any:
|
||||
if self._delegate is None:
|
||||
raise MailProfileError("Mail module is not available")
|
||||
return self._delegate
|
||||
|
||||
def assert_campaign_mail_policy_allows_json(self, session: Any, **kwargs: Any) -> None:
|
||||
def assert_campaign_mail_policy_allows_json(
|
||||
self, session: Any, **kwargs: Any
|
||||
) -> None:
|
||||
if self._delegate is None:
|
||||
raw_json = kwargs.get("raw_json")
|
||||
profile_id = self.mail_profile_id_from_campaign_json(raw_json if isinstance(raw_json, dict) else {})
|
||||
profile_id = self.mail_profile_id_from_campaign_json(
|
||||
raw_json if isinstance(raw_json, dict) else {}
|
||||
)
|
||||
if profile_id:
|
||||
raise MailProfileError("Campaign mail-server profiles require the mail module")
|
||||
raise MailProfileError(
|
||||
"Campaign mail-server profiles require the mail module"
|
||||
)
|
||||
return None
|
||||
try:
|
||||
return self._delegate.assert_campaign_mail_policy_allows_json(session, **kwargs)
|
||||
return self._delegate.assert_campaign_mail_policy_allows_json(
|
||||
session, **kwargs
|
||||
)
|
||||
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
|
||||
raise MailProfileError(str(exc)) from exc
|
||||
|
||||
def mail_profile_id_from_campaign_json(self, raw_json: dict[str, Any]) -> str | None:
|
||||
def mail_profile_id_from_campaign_json(
|
||||
self, raw_json: dict[str, Any]
|
||||
) -> str | None:
|
||||
if self._delegate is not None:
|
||||
return self._delegate.mail_profile_id_from_campaign_json(raw_json)
|
||||
server = raw_json.get("server") if isinstance(raw_json, dict) else None
|
||||
profile_id = server.get("mail_profile_id") if isinstance(server, dict) else None
|
||||
return str(profile_id).strip() if profile_id else None
|
||||
|
||||
def campaign_profile_delivery_summary(self, session: Any, **kwargs: Any) -> dict[str, Any]:
|
||||
def campaign_profile_delivery_summary(
|
||||
self, session: Any, **kwargs: Any
|
||||
) -> dict[str, Any]:
|
||||
delegate = self._require()
|
||||
try:
|
||||
return delegate.campaign_profile_delivery_summary(session, **kwargs)
|
||||
@@ -182,8 +290,14 @@ class MailCampaignIntegration:
|
||||
try:
|
||||
return delegate.send_campaign_email_bytes(*args, **kwargs)
|
||||
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
|
||||
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
|
||||
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
|
||||
raise SmtpSendError(
|
||||
str(exc),
|
||||
temporary=bool(getattr(exc, "temporary", False)),
|
||||
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
|
||||
) from exc
|
||||
except getattr(
|
||||
delegate, "SmtpConfigurationError", SmtpConfigurationError
|
||||
) as exc:
|
||||
raise SmtpConfigurationError(str(exc)) from exc
|
||||
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
||||
raise MailProfileError(str(exc)) from exc
|
||||
@@ -198,20 +312,495 @@ class MailCampaignIntegration:
|
||||
temporary=getattr(exc, "temporary", None),
|
||||
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
|
||||
) from exc
|
||||
except getattr(delegate, "ImapConfigurationError", ImapConfigurationError) as exc:
|
||||
except getattr(
|
||||
delegate, "ImapConfigurationError", ImapConfigurationError
|
||||
) as exc:
|
||||
raise ImapConfigurationError(str(exc)) from exc
|
||||
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
|
||||
raise MailProfileError(str(exc)) from exc
|
||||
|
||||
def submit_delivery_command(self, session: Any, **kwargs: Any) -> dict[str, Any]:
|
||||
delegate = self._require()
|
||||
method = getattr(delegate, "submit_delivery_command", None)
|
||||
if not callable(method):
|
||||
raise MailDeliveryCommandError(
|
||||
"The installed Mail module does not provide durable delivery commands"
|
||||
)
|
||||
try:
|
||||
return dict(method(session, **kwargs))
|
||||
except Exception as exc:
|
||||
raise MailDeliveryCommandError(str(exc)) from exc
|
||||
|
||||
def delivery_command_summary(
|
||||
self,
|
||||
session: Any,
|
||||
*,
|
||||
tenant_id: str,
|
||||
command_id: str,
|
||||
) -> dict[str, Any]:
|
||||
delegate = self._require()
|
||||
method = getattr(delegate, "delivery_command_summary", None)
|
||||
if not callable(method):
|
||||
raise MailDeliveryCommandError(
|
||||
"The installed Mail module does not provide durable delivery status"
|
||||
)
|
||||
try:
|
||||
return dict(
|
||||
method(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
command_id=command_id,
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
raise MailDeliveryCommandError(str(exc)) from exc
|
||||
|
||||
def mock_mailbox(self) -> Any | None:
|
||||
if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"):
|
||||
return None
|
||||
return self._delegate.mock_mailbox()
|
||||
|
||||
|
||||
class PostboxCampaignIntegration:
|
||||
def __init__(
|
||||
self,
|
||||
delivery_delegate: object | None = None,
|
||||
directory_delegate: object | None = None,
|
||||
evidence_delegate: object | None = None,
|
||||
) -> None:
|
||||
self._delivery_delegate = (
|
||||
delivery_delegate
|
||||
if isinstance(delivery_delegate, PostboxDeliveryProvider)
|
||||
else None
|
||||
)
|
||||
self._directory_delegate = (
|
||||
directory_delegate
|
||||
if isinstance(directory_delegate, PostboxDirectoryProvider)
|
||||
else None
|
||||
)
|
||||
self._evidence_delegate = (
|
||||
evidence_delegate
|
||||
if isinstance(evidence_delegate, PostboxEvidenceProvider)
|
||||
else None
|
||||
)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return (
|
||||
self._delivery_delegate is not None and self._directory_delegate is not None
|
||||
)
|
||||
|
||||
@property
|
||||
def receipt_evidence_available(self) -> bool:
|
||||
return self._evidence_delegate is not None
|
||||
|
||||
def delivery_catalog(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
) -> PostboxDeliveryCatalogRef:
|
||||
if self._directory_delegate is None:
|
||||
raise PostboxDeliveryUnavailable(
|
||||
"Postbox targets are unavailable because the Postbox module "
|
||||
"is not active."
|
||||
)
|
||||
return self._directory_delegate.delivery_catalog(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
)
|
||||
|
||||
def resolve_postbox(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
target: PostboxTargetRef,
|
||||
materialize: bool = False,
|
||||
) -> PostboxDirectoryEntryRef | None:
|
||||
if self._directory_delegate is None:
|
||||
raise PostboxDeliveryUnavailable(
|
||||
"Postbox targets are unavailable because the Postbox module "
|
||||
"is not active."
|
||||
)
|
||||
return self._directory_delegate.resolve_postbox(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
target=target,
|
||||
materialize=materialize,
|
||||
)
|
||||
|
||||
def deliver(
|
||||
self,
|
||||
session: object,
|
||||
request: PostboxDeliveryRequest,
|
||||
) -> PostboxDeliveryResult:
|
||||
if self._delivery_delegate is None:
|
||||
raise PostboxDeliveryUnavailable(
|
||||
"Postbox delivery is unavailable because the Postbox module "
|
||||
"is not active."
|
||||
)
|
||||
return self._delivery_delegate.deliver(session, request)
|
||||
|
||||
def delivery_receipt_summaries(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
delivery_ids: list[str] | tuple[str, ...],
|
||||
) -> dict[str, PostboxDeliveryReceiptSummaryRef]:
|
||||
if self._evidence_delegate is None:
|
||||
return {}
|
||||
unique_ids = tuple(dict.fromkeys(delivery_ids))
|
||||
summaries: dict[str, PostboxDeliveryReceiptSummaryRef] = {}
|
||||
for offset in range(0, len(unique_ids), 500):
|
||||
summaries.update(
|
||||
self._evidence_delegate.delivery_receipt_summaries(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
producer_module="campaigns",
|
||||
delivery_ids=unique_ids[offset : offset + 500],
|
||||
)
|
||||
)
|
||||
return summaries
|
||||
|
||||
|
||||
class ApprovalCampaignIntegration:
|
||||
def __init__(self, delegate: object | None = None) -> None:
|
||||
self._delegate = (
|
||||
delegate if isinstance(delegate, ApprovalRequestProvider) else None
|
||||
)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self._delegate is not None
|
||||
|
||||
def create_request(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
command: ApprovalRequestCreateCommand,
|
||||
idempotency_key: str,
|
||||
) -> ApprovalRequestRef:
|
||||
if self._delegate is None:
|
||||
raise ApprovalGateUnavailable(
|
||||
"Campaign approval gates require the Approvals module."
|
||||
)
|
||||
return self._delegate.create_request(
|
||||
session,
|
||||
principal,
|
||||
command=command,
|
||||
idempotency_key=idempotency_key,
|
||||
)
|
||||
|
||||
def check_approved(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
request_id: str,
|
||||
subject_module: str,
|
||||
subject_type: str,
|
||||
subject_id: str,
|
||||
subject_version: str | None,
|
||||
subject_digest: str,
|
||||
) -> ApprovalCheck:
|
||||
if self._delegate is None:
|
||||
raise ApprovalGateUnavailable(
|
||||
"Campaign delivery is approval-gated, but the Approvals module is unavailable."
|
||||
)
|
||||
return self._delegate.check_approved(
|
||||
session,
|
||||
principal,
|
||||
request_id=request_id,
|
||||
subject_module=subject_module,
|
||||
subject_type=subject_type,
|
||||
subject_id=subject_id,
|
||||
subject_version=subject_version,
|
||||
subject_digest=subject_digest,
|
||||
)
|
||||
|
||||
|
||||
class TemplatesCampaignIntegration:
|
||||
def __init__(
|
||||
self,
|
||||
catalog_delegate: object | None = None,
|
||||
renderer_delegate: object | None = None,
|
||||
) -> None:
|
||||
self._catalog = (
|
||||
catalog_delegate
|
||||
if isinstance(catalog_delegate, TemplateCatalogProvider)
|
||||
else None
|
||||
)
|
||||
self._renderer = (
|
||||
renderer_delegate
|
||||
if isinstance(renderer_delegate, TemplateRendererProvider)
|
||||
else None
|
||||
)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self._catalog is not None and self._renderer is not None
|
||||
|
||||
def list_templates(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
query: str = "",
|
||||
limit: int = 100,
|
||||
) -> tuple[TemplateRef, ...]:
|
||||
if self._catalog is None:
|
||||
return ()
|
||||
return tuple(
|
||||
self._catalog.list_templates(
|
||||
session,
|
||||
principal,
|
||||
query=query,
|
||||
usage="campaign_print",
|
||||
limit=limit,
|
||||
)
|
||||
)
|
||||
|
||||
def check_compatibility(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
template_id: str,
|
||||
revision: int | None,
|
||||
output_format: str,
|
||||
available_fields: dict[str, str] | tuple[str, ...],
|
||||
) -> TemplateCompatibility:
|
||||
if self._catalog is None:
|
||||
raise TemplateOutputUnavailable(
|
||||
"Printable output is unavailable because Templates is not active."
|
||||
)
|
||||
return self._catalog.check_compatibility(
|
||||
session,
|
||||
principal,
|
||||
template_id=template_id,
|
||||
revision=revision,
|
||||
usage="campaign_print",
|
||||
output_format=output_format,
|
||||
available_fields=available_fields,
|
||||
)
|
||||
|
||||
def get_template(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
template_id: str,
|
||||
revision: int,
|
||||
) -> TemplateRef | None:
|
||||
if self._catalog is None:
|
||||
return None
|
||||
return self._catalog.get_template(
|
||||
session,
|
||||
principal,
|
||||
template_id=template_id,
|
||||
revision=revision,
|
||||
)
|
||||
|
||||
def render(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
request: TemplateRenderRequest,
|
||||
) -> TemplateRenderResult:
|
||||
if self._renderer is None:
|
||||
raise TemplateOutputUnavailable(
|
||||
"Printable output is unavailable because Templates is not active."
|
||||
)
|
||||
return self._renderer.render(session, principal, request=request)
|
||||
|
||||
|
||||
class CalendarCampaignIntegration:
|
||||
def __init__(self, delegate: object | None = None) -> None:
|
||||
self._delegate = (
|
||||
delegate if isinstance(delegate, CalendarInvitationProvider) else None
|
||||
)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self._delegate is not None
|
||||
|
||||
def list_calendars(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str | None,
|
||||
group_ids: tuple[str, ...] = (),
|
||||
can_admin: bool = False,
|
||||
) -> tuple[CalendarInvitationCalendarRef, ...]:
|
||||
if self._delegate is None:
|
||||
return ()
|
||||
return tuple(
|
||||
self._delegate.list_calendars(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=group_ids,
|
||||
can_admin=can_admin,
|
||||
)
|
||||
)
|
||||
|
||||
def render_invitation(self, request: CalendarInvitationRequest) -> str:
|
||||
if self._delegate is None:
|
||||
raise CalendarInvitationUnavailable(
|
||||
"Calendar invitations require the optional Calendar module."
|
||||
)
|
||||
return self._delegate.render_invitation(request)
|
||||
|
||||
def upsert_invitation(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str | None,
|
||||
request: CalendarInvitationRequest,
|
||||
) -> CalendarInvitationRef:
|
||||
if self._delegate is None:
|
||||
raise CalendarInvitationUnavailable(
|
||||
"Calendar invitations require the optional Calendar module."
|
||||
)
|
||||
return self._delegate.upsert_invitation(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
request=request,
|
||||
)
|
||||
|
||||
def get_invitations(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
correlation_ids: tuple[str, ...],
|
||||
) -> dict[str, CalendarInvitationRef]:
|
||||
if self._delegate is None or not correlation_ids:
|
||||
return {}
|
||||
result: dict[str, CalendarInvitationRef] = {}
|
||||
unique_ids = tuple(dict.fromkeys(correlation_ids))
|
||||
for offset in range(0, len(unique_ids), 500):
|
||||
result.update(
|
||||
self._delegate.get_invitations(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
correlation_ids=unique_ids[offset : offset + 500],
|
||||
)
|
||||
)
|
||||
return result
|
||||
|
||||
def summarize_invitations(
|
||||
self,
|
||||
session: object,
|
||||
*,
|
||||
tenant_id: str,
|
||||
source_resource_id: str | None,
|
||||
) -> dict[str, object]:
|
||||
if self._delegate is None:
|
||||
return {
|
||||
"available": False,
|
||||
"reason": "The Calendar invitation capability is not active.",
|
||||
}
|
||||
return dict(
|
||||
self._delegate.summarize_invitations(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
source_module="campaigns",
|
||||
source_resource_type="campaign_version",
|
||||
source_resource_id=source_resource_id,
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def request_from_payload(payload: dict[str, Any]) -> CalendarInvitationRequest:
|
||||
from datetime import datetime
|
||||
|
||||
attendees = tuple(
|
||||
CalendarInvitationAttendeeRequest(
|
||||
address=str(item.get("address") or ""),
|
||||
name=str(item["name"]) if item.get("name") else None,
|
||||
role=str(item.get("role") or "REQ-PARTICIPANT"),
|
||||
participation_status=str(
|
||||
item.get("participation_status") or "NEEDS-ACTION"
|
||||
),
|
||||
rsvp=bool(item.get("rsvp", True)),
|
||||
)
|
||||
for item in payload.get("attendees") or []
|
||||
if isinstance(item, dict)
|
||||
)
|
||||
return CalendarInvitationRequest(
|
||||
correlation_id=str(payload.get("correlation_id") or ""),
|
||||
source_module="campaigns",
|
||||
source_resource_type="campaign_version",
|
||||
source_resource_id=(
|
||||
str(payload["source_resource_id"])
|
||||
if payload.get("source_resource_id")
|
||||
else None
|
||||
),
|
||||
calendar_id=(
|
||||
str(payload["calendar_id"]) if payload.get("calendar_id") else None
|
||||
),
|
||||
summary=str(payload.get("summary") or ""),
|
||||
description=(
|
||||
str(payload["description"]) if payload.get("description") else None
|
||||
),
|
||||
location=str(payload["location"]) if payload.get("location") else None,
|
||||
start_at=datetime.fromisoformat(str(payload.get("start_at") or "")),
|
||||
end_at=(
|
||||
datetime.fromisoformat(str(payload["end_at"]))
|
||||
if payload.get("end_at")
|
||||
else None
|
||||
),
|
||||
timezone=str(payload["timezone"]) if payload.get("timezone") else None,
|
||||
organizer=(
|
||||
dict(payload["organizer"])
|
||||
if isinstance(payload.get("organizer"), dict)
|
||||
else None
|
||||
),
|
||||
attendees=attendees,
|
||||
classification=str(payload.get("classification") or "PUBLIC"),
|
||||
categories=tuple(str(value) for value in payload.get("categories") or []),
|
||||
metadata=(
|
||||
dict(payload["metadata"])
|
||||
if isinstance(payload.get("metadata"), dict)
|
||||
else {}
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def files_integration() -> FilesCampaignIntegration:
|
||||
return FilesCampaignIntegration(capability(FILES_CAPABILITY))
|
||||
|
||||
|
||||
def mail_integration() -> MailCampaignIntegration:
|
||||
return MailCampaignIntegration(capability(MAIL_CAPABILITY))
|
||||
|
||||
|
||||
def postbox_integration() -> PostboxCampaignIntegration:
|
||||
return PostboxCampaignIntegration(
|
||||
capability(POSTBOX_CAPABILITY),
|
||||
capability(POSTBOX_DIRECTORY_CAPABILITY),
|
||||
capability(POSTBOX_EVIDENCE_CAPABILITY),
|
||||
)
|
||||
|
||||
|
||||
def approvals_integration() -> ApprovalCampaignIntegration:
|
||||
return ApprovalCampaignIntegration(capability(APPROVALS_CAPABILITY))
|
||||
|
||||
|
||||
def templates_integration() -> TemplatesCampaignIntegration:
|
||||
return TemplatesCampaignIntegration(
|
||||
capability(TEMPLATE_CATALOG_CAPABILITY),
|
||||
capability(TEMPLATE_RENDERER_CAPABILITY),
|
||||
)
|
||||
|
||||
|
||||
def calendar_integration() -> CalendarCampaignIntegration:
|
||||
return CalendarCampaignIntegration(capability(CALENDAR_INVITATIONS_CAPABILITY))
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -16,6 +16,7 @@ from govoplan_campaign.backend.attachments.resolver import (
|
||||
EntryAttachmentResolution,
|
||||
MessageAttachmentStatus,
|
||||
ResolvedAttachment,
|
||||
effective_send_without_attachments_behavior,
|
||||
resolve_entry_attachments,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.addressing import effective_address_lists, formatted_recipient
|
||||
@@ -33,6 +34,7 @@ from govoplan_campaign.backend.campaign.models import (
|
||||
ZipArchiveConfig,
|
||||
ZipPasswordMode,
|
||||
ZipPasswordScope,
|
||||
effective_delivery_channel_policy,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.template_values import build_template_values
|
||||
from govoplan_campaign.backend.services.zip_service import create_zip_archive
|
||||
@@ -171,6 +173,11 @@ def _attachment_summaries(resolution: EntryAttachmentResolution) -> list[Message
|
||||
label=attachment.label,
|
||||
status=attachment.status.value,
|
||||
behavior=attachment.behavior.value if attachment.behavior else None,
|
||||
missing_policy=(
|
||||
attachment.missing_policy.model_dump(mode="json")
|
||||
if attachment.missing_policy
|
||||
else None
|
||||
),
|
||||
required=attachment.required,
|
||||
allow_multiple=attachment.allow_multiple,
|
||||
zip_enabled=attachment.zip_enabled,
|
||||
@@ -199,6 +206,7 @@ def _message_issues_from_attachment_resolution(resolution: EntryAttachmentResolu
|
||||
message=issue.message,
|
||||
behavior=issue.behavior.value if issue.behavior else None,
|
||||
source="attachments",
|
||||
details=issue.details,
|
||||
)
|
||||
for issue in resolution.issues
|
||||
]
|
||||
@@ -218,12 +226,6 @@ def _append_no_attachment_coverage_issue(issues: list[MessageIssue]) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
|
||||
return config.attachments.send_without_attachments_behavior or (
|
||||
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
|
||||
)
|
||||
|
||||
|
||||
def _safe_filename(value: str | None, fallback: str) -> str:
|
||||
raw = value or fallback
|
||||
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._")
|
||||
@@ -425,8 +427,14 @@ def _attach_files(
|
||||
|
||||
return attached_count
|
||||
|
||||
def _imap_initial_status(config: CampaignConfig) -> ImapStatus:
|
||||
if config.delivery.imap_append_sent.enabled:
|
||||
def _imap_initial_status(
|
||||
config: CampaignConfig,
|
||||
entry: EntryConfig,
|
||||
) -> ImapStatus:
|
||||
if (
|
||||
effective_delivery_channel_policy(config, entry).uses_mail
|
||||
and config.delivery.imap_append_sent.enabled
|
||||
):
|
||||
return ImapStatus.PENDING
|
||||
return ImapStatus.NOT_REQUESTED
|
||||
|
||||
@@ -520,7 +528,11 @@ def _message_draft(
|
||||
send_status = SendStatus.SKIPPED
|
||||
imap_status = ImapStatus.SKIPPED
|
||||
if imap_status is None:
|
||||
imap_status = _imap_initial_status(config) if build_status == BuildStatus.BUILT else ImapStatus.SKIPPED
|
||||
imap_status = (
|
||||
_imap_initial_status(config, entry)
|
||||
if build_status == BuildStatus.BUILT
|
||||
else ImapStatus.SKIPPED
|
||||
)
|
||||
return MessageDraft(
|
||||
entry_index=entry_index,
|
||||
entry_id=entry.id,
|
||||
@@ -529,6 +541,10 @@ def _message_draft(
|
||||
validation_status=validation_status,
|
||||
send_status=send_status,
|
||||
imap_status=imap_status,
|
||||
delivery_channel_policy=effective_delivery_channel_policy(
|
||||
config,
|
||||
entry,
|
||||
).value,
|
||||
subject=subject,
|
||||
from_=_message_address(context.sender),
|
||||
from_all=_message_addresses(context.senders),
|
||||
@@ -746,7 +762,11 @@ def _build_mime_message(
|
||||
values=rendered.values,
|
||||
work_dir=work_dir,
|
||||
)
|
||||
if attachment_count == 0 and context.resolution.attachments and _send_without_attachments_behavior(config) == Behavior.BLOCK:
|
||||
if (
|
||||
attachment_count == 0
|
||||
and context.resolution.attachments
|
||||
and effective_send_without_attachments_behavior(config) == Behavior.BLOCK
|
||||
):
|
||||
_append_no_attachment_coverage_issue(context.issues)
|
||||
return _MimeBuildResult(
|
||||
message=None,
|
||||
@@ -770,6 +790,16 @@ def _build_mime_message(
|
||||
source="attachments",
|
||||
)
|
||||
)
|
||||
except OSError as exc:
|
||||
context.issues.append(
|
||||
MessageIssue(
|
||||
severity="error",
|
||||
code="attachment_unreadable",
|
||||
message="A resolved attachment could not be read while building the message.",
|
||||
behavior="block",
|
||||
source=f"attachments:{type(exc).__name__}",
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
context.issues.append(
|
||||
MessageIssue(
|
||||
@@ -809,17 +839,19 @@ def build_entry_message(
|
||||
if not entry.active:
|
||||
return _inactive_entry_message(config=config, entry=entry, entry_index=entry_index, context=context)
|
||||
|
||||
context.validation_status = _validate_required_sender(
|
||||
context.senders,
|
||||
context.issues,
|
||||
context.validation_status,
|
||||
)
|
||||
context.validation_status = _validate_required_recipients(
|
||||
config,
|
||||
context.recipients,
|
||||
context.issues,
|
||||
context.validation_status,
|
||||
)
|
||||
channel_policy = effective_delivery_channel_policy(config, entry)
|
||||
if channel_policy.uses_mail:
|
||||
context.validation_status = _validate_required_sender(
|
||||
context.senders,
|
||||
context.issues,
|
||||
context.validation_status,
|
||||
)
|
||||
context.validation_status = _validate_required_recipients(
|
||||
config,
|
||||
context.recipients,
|
||||
context.issues,
|
||||
context.validation_status,
|
||||
)
|
||||
rendered = _render_message_template(config, campaign_file, entry)
|
||||
context.validation_status = _validate_rendered_template(
|
||||
config,
|
||||
|
||||
@@ -33,6 +33,7 @@ class MessageIssue(BaseModel):
|
||||
message: str
|
||||
behavior: str | None = None
|
||||
source: str | None = None
|
||||
details: dict[str, object] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class MessageAddress(BaseModel):
|
||||
@@ -49,6 +50,7 @@ class MessageAttachmentSummary(BaseModel):
|
||||
label: str | None = None
|
||||
status: str
|
||||
behavior: str | None = None
|
||||
missing_policy: dict[str, object] | None = None
|
||||
required: bool
|
||||
allow_multiple: bool
|
||||
zip_enabled: bool
|
||||
@@ -78,6 +80,7 @@ class MessageDraft(BaseModel):
|
||||
validation_status: MessageValidationStatus
|
||||
send_status: SendStatus
|
||||
imap_status: ImapStatus
|
||||
delivery_channel_policy: str = "mail"
|
||||
|
||||
subject: str | None = None
|
||||
from_: MessageAddress | None = Field(default=None, alias="from")
|
||||
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
"""Development wrapper for the canonical printable-delivery migration."""
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
_migration = import_module(
|
||||
"govoplan_campaign.backend.migrations.versions."
|
||||
"b7c8d9e0f1a2_campaign_print_delivery"
|
||||
)
|
||||
|
||||
revision = _migration.revision
|
||||
down_revision = _migration.down_revision
|
||||
branch_labels = _migration.branch_labels
|
||||
depends_on = _migration.depends_on
|
||||
upgrade = _migration.upgrade
|
||||
downgrade = _migration.downgrade
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
"""add audit-proof single-message action ledger
|
||||
|
||||
Revision ID: c1a69e4f2b70
|
||||
Revises: f0a1b2c3d4e5
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
message_actions = import_module(
|
||||
"govoplan_campaign.backend.migrations.versions.c1a69e4f2b70_campaign_message_actions"
|
||||
)
|
||||
|
||||
|
||||
revision = message_actions.revision
|
||||
down_revision = message_actions.down_revision
|
||||
branch_labels = message_actions.branch_labels
|
||||
depends_on = message_actions.depends_on
|
||||
upgrade = message_actions.upgrade
|
||||
downgrade = message_actions.downgrade
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
"""add monotonic campaign version edit revision
|
||||
|
||||
Revision ID: d2b7af503c81
|
||||
Revises: c1a69e4f2b70
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
edit_revision = import_module(
|
||||
"govoplan_campaign.backend.migrations.versions.d2b7af503c81_campaign_version_edit_revision"
|
||||
)
|
||||
|
||||
|
||||
revision = edit_revision.revision
|
||||
down_revision = edit_revision.down_revision
|
||||
branch_labels = edit_revision.branch_labels
|
||||
depends_on = edit_revision.depends_on
|
||||
upgrade = edit_revision.upgrade
|
||||
downgrade = edit_revision.downgrade
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
"""repair a missing IMAP append attempt claim token
|
||||
|
||||
Revision ID: e9f0a1b2c3d4
|
||||
Revises: d8b3e2c1f4a5
|
||||
Create Date: 2026-07-28 23:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "e9f0a1b2c3d4"
|
||||
down_revision = "d8b3e2c1f4a5"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
|
||||
if "claim_token" not in columns:
|
||||
op.add_column(
|
||||
"imap_append_attempts",
|
||||
sa.Column("claim_token", sa.String(length=36), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# The column belongs to revision 3c4d5e6f8192. This repair revision only
|
||||
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
|
||||
pass
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
"""add governed campaign Postbox delivery
|
||||
|
||||
Revision ID: f0a1b2c3d4e5
|
||||
Revises: e9f0a1b2c3d4
|
||||
Create Date: 2026-07-29 02:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
_migration = import_module(
|
||||
"govoplan_campaign.backend.migrations.versions."
|
||||
"f0a1b2c3d4e5_v0115_postbox_delivery"
|
||||
)
|
||||
|
||||
revision = _migration.revision
|
||||
down_revision = _migration.down_revision
|
||||
branch_labels = _migration.branch_labels
|
||||
depends_on = _migration.depends_on
|
||||
upgrade = _migration.upgrade
|
||||
downgrade = _migration.downgrade
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
"""add governed campaign printable delivery
|
||||
|
||||
Revision ID: b7c8d9e0f1a2
|
||||
Revises: f0a1b2c3d4e5
|
||||
Create Date: 2026-08-02 12:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "b7c8d9e0f1a2"
|
||||
down_revision = "f0a1b2c3d4e5"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"print_status",
|
||||
sa.String(length=50),
|
||||
nullable=False,
|
||||
server_default="not_requested",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"print_attempt_count",
|
||||
sa.Integer(),
|
||||
nullable=False,
|
||||
server_default="0",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column("resolved_print_output", sa.JSON(), nullable=True),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"delivery_provenance",
|
||||
sa.JSON(),
|
||||
nullable=False,
|
||||
server_default="{}",
|
||||
),
|
||||
)
|
||||
op.create_index(
|
||||
op.f("ix_campaign_jobs_print_status"),
|
||||
"campaign_jobs",
|
||||
["print_status"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_table(
|
||||
"campaign_print_output_attempts",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("job_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("attempt_number", sa.Integer(), nullable=False),
|
||||
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
|
||||
sa.Column("status", sa.String(length=50), nullable=False),
|
||||
sa.Column("render_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("artifact_sha256", sa.String(length=64), nullable=True),
|
||||
sa.Column("evidence", sa.JSON(), nullable=False),
|
||||
sa.Column("error_message", sa.Text(), nullable=True),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["job_id"],
|
||||
["campaign_jobs.id"],
|
||||
name=op.f("fk_campaign_print_output_attempts_job_id_campaign_jobs"),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.PrimaryKeyConstraint(
|
||||
"id",
|
||||
name=op.f("pk_campaign_print_output_attempts"),
|
||||
),
|
||||
sa.UniqueConstraint(
|
||||
"job_id",
|
||||
"attempt_number",
|
||||
name="uq_campaign_print_attempt_job_number",
|
||||
),
|
||||
sa.UniqueConstraint(
|
||||
"tenant_id",
|
||||
"idempotency_key",
|
||||
name="uq_campaign_print_attempt_idempotency",
|
||||
),
|
||||
)
|
||||
for column in ("tenant_id", "job_id", "status", "render_id", "artifact_sha256"):
|
||||
op.create_index(
|
||||
op.f(f"ix_campaign_print_output_attempts_{column}"),
|
||||
"campaign_print_output_attempts",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("campaign_print_output_attempts")
|
||||
op.drop_index(
|
||||
op.f("ix_campaign_jobs_print_status"),
|
||||
table_name="campaign_jobs",
|
||||
)
|
||||
op.drop_column("campaign_jobs", "resolved_print_output")
|
||||
op.drop_column("campaign_jobs", "delivery_provenance")
|
||||
op.drop_column("campaign_jobs", "print_attempt_count")
|
||||
op.drop_column("campaign_jobs", "print_status")
|
||||
+186
@@ -0,0 +1,186 @@
|
||||
"""add audit-proof single-message action ledger
|
||||
|
||||
Revision ID: c1a69e4f2b70
|
||||
Revises: f0a1b2c3d4e5
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "c1a69e4f2b70"
|
||||
down_revision = "f0a1b2c3d4e5"
|
||||
branch_labels = None
|
||||
depends_on = "c91f0a72be34"
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||
if "campaign_message_actions" not in tables:
|
||||
op.create_table(
|
||||
"campaign_message_actions",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("campaign_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("campaign_version_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("job_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("kind", sa.String(length=30), nullable=False),
|
||||
sa.Column("idempotency_key", sa.String(length=200), nullable=False),
|
||||
sa.Column("canonical_request_hash", sa.String(length=64), nullable=False),
|
||||
sa.Column("reason", sa.Text(), nullable=True),
|
||||
sa.Column("context", sa.JSON(), nullable=False),
|
||||
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("actor_api_key_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("message_sha256", sa.String(length=64), nullable=False),
|
||||
sa.Column("message_size_bytes", sa.Integer(), nullable=True),
|
||||
sa.Column(
|
||||
"recipient_manifest_sha256",
|
||||
sa.String(length=64),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("recipient_count", sa.Integer(), nullable=False),
|
||||
sa.Column("prior_send_status", sa.String(length=50), nullable=False),
|
||||
sa.Column("prior_attempt_count", sa.Integer(), nullable=False),
|
||||
sa.Column("final_send_status", sa.String(length=50), nullable=True),
|
||||
sa.Column("status", sa.String(length=50), nullable=False),
|
||||
sa.Column("accepted_count", sa.Integer(), nullable=False),
|
||||
sa.Column("refused_count", sa.Integer(), nullable=False),
|
||||
sa.Column("refusal_summary", sa.JSON(), nullable=False),
|
||||
sa.Column("error_type", sa.String(length=120), nullable=True),
|
||||
sa.Column("error_message", sa.String(length=500), nullable=True),
|
||||
sa.Column("linked_send_attempt_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["campaign_id"],
|
||||
["campaigns.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_actions_campaign_id_campaigns"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["campaign_version_id"],
|
||||
["campaign_versions.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_actions_campaign_version_id_campaign_versions"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["job_id"],
|
||||
["campaign_jobs.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_actions_job_id_campaign_jobs"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["actor_user_id"],
|
||||
["access_users.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_actions_actor_user_id_access_users"
|
||||
),
|
||||
ondelete="SET NULL",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["linked_send_attempt_id"],
|
||||
["send_attempts.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_actions_linked_send_attempt_id_send_attempts"
|
||||
),
|
||||
ondelete="SET NULL",
|
||||
),
|
||||
sa.PrimaryKeyConstraint(
|
||||
"id",
|
||||
name=op.f("pk_campaign_message_actions"),
|
||||
),
|
||||
sa.UniqueConstraint(
|
||||
"tenant_id",
|
||||
"idempotency_key",
|
||||
name="uq_campaign_message_actions_idempotency",
|
||||
),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_campaign_message_actions_job_created",
|
||||
"campaign_message_actions",
|
||||
["job_id", "created_at"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_index(
|
||||
"ix_campaign_message_actions_campaign_kind",
|
||||
"campaign_message_actions",
|
||||
["campaign_id", "kind", "status"],
|
||||
unique=False,
|
||||
)
|
||||
for column in (
|
||||
"tenant_id",
|
||||
"campaign_id",
|
||||
"campaign_version_id",
|
||||
"job_id",
|
||||
"kind",
|
||||
"actor_user_id",
|
||||
"status",
|
||||
"linked_send_attempt_id",
|
||||
):
|
||||
op.create_index(
|
||||
op.f(f"ix_campaign_message_actions_{column}"),
|
||||
"campaign_message_actions",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||
if "campaign_message_action_attempts" not in tables:
|
||||
op.create_table(
|
||||
"campaign_message_action_attempts",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("action_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("attempt_number", sa.Integer(), nullable=False),
|
||||
sa.Column("status", sa.String(length=50), nullable=False),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("accepted_count", sa.Integer(), nullable=False),
|
||||
sa.Column("refused_count", sa.Integer(), nullable=False),
|
||||
sa.Column("outcome_code", sa.String(length=80), nullable=True),
|
||||
sa.Column("diagnostic_summary", sa.String(length=500), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["action_id"],
|
||||
["campaign_message_actions.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_message_action_attempts_action_id_campaign_message_actions"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.PrimaryKeyConstraint(
|
||||
"id",
|
||||
name=op.f("pk_campaign_message_action_attempts"),
|
||||
),
|
||||
sa.UniqueConstraint(
|
||||
"action_id",
|
||||
"attempt_number",
|
||||
name="uq_campaign_message_action_attempt_number",
|
||||
),
|
||||
)
|
||||
for column in ("action_id", "status"):
|
||||
op.create_index(
|
||||
op.f(f"ix_campaign_message_action_attempts_{column}"),
|
||||
"campaign_message_action_attempts",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
tables = set(sa.inspect(op.get_bind()).get_table_names())
|
||||
if "campaign_message_action_attempts" in tables:
|
||||
op.drop_table("campaign_message_action_attempts")
|
||||
if "campaign_message_actions" in tables:
|
||||
op.drop_table("campaign_message_actions")
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
"""add monotonic campaign version edit revision
|
||||
|
||||
Revision ID: d2b7af503c81
|
||||
Revises: c1a69e4f2b70
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "d2b7af503c81"
|
||||
down_revision = "c1a69e4f2b70"
|
||||
branch_labels = None
|
||||
depends_on = "c91f0a72be34"
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
inspector = sa.inspect(op.get_bind())
|
||||
columns = {
|
||||
column["name"]
|
||||
for column in inspector.get_columns("campaign_versions")
|
||||
}
|
||||
if "edit_revision" not in columns:
|
||||
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column(
|
||||
"edit_revision",
|
||||
sa.Integer(),
|
||||
server_default="1",
|
||||
nullable=False,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
inspector = sa.inspect(op.get_bind())
|
||||
columns = {
|
||||
column["name"]
|
||||
for column in inspector.get_columns("campaign_versions")
|
||||
}
|
||||
if "edit_revision" in columns:
|
||||
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||
batch_op.drop_column("edit_revision")
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
"""repair a missing IMAP append attempt claim token
|
||||
|
||||
Revision ID: e9f0a1b2c3d4
|
||||
Revises: d8b3e2c1f4a5
|
||||
Create Date: 2026-07-28 23:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "e9f0a1b2c3d4"
|
||||
down_revision = "d8b3e2c1f4a5"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
|
||||
if "claim_token" not in columns:
|
||||
op.add_column(
|
||||
"imap_append_attempts",
|
||||
sa.Column("claim_token", sa.String(length=36), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# The column belongs to revision 3c4d5e6f8192. This repair revision only
|
||||
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
|
||||
pass
|
||||
+153
@@ -0,0 +1,153 @@
|
||||
"""add governed campaign Postbox delivery
|
||||
|
||||
Revision ID: f0a1b2c3d4e5
|
||||
Revises: e9f0a1b2c3d4
|
||||
Create Date: 2026-07-29 02:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "f0a1b2c3d4e5"
|
||||
down_revision = "e9f0a1b2c3d4"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"delivery_channel_policy",
|
||||
sa.String(length=30),
|
||||
nullable=False,
|
||||
server_default="mail",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"postbox_status",
|
||||
sa.String(length=50),
|
||||
nullable=False,
|
||||
server_default="not_requested",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"postbox_attempt_count",
|
||||
sa.Integer(),
|
||||
nullable=False,
|
||||
server_default="0",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"campaign_jobs",
|
||||
sa.Column(
|
||||
"resolved_postbox_targets",
|
||||
sa.JSON(),
|
||||
nullable=False,
|
||||
server_default="[]",
|
||||
),
|
||||
)
|
||||
op.create_index(
|
||||
op.f("ix_campaign_jobs_delivery_channel_policy"),
|
||||
"campaign_jobs",
|
||||
["delivery_channel_policy"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_index(
|
||||
op.f("ix_campaign_jobs_postbox_status"),
|
||||
"campaign_jobs",
|
||||
["postbox_status"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"campaign_postbox_delivery_attempts",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("job_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("target_key", sa.String(length=64), nullable=False),
|
||||
sa.Column("target_index", sa.Integer(), nullable=False),
|
||||
sa.Column("attempt_number", sa.Integer(), nullable=False),
|
||||
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
|
||||
sa.Column("status", sa.String(length=50), nullable=False),
|
||||
sa.Column("target_snapshot", sa.JSON(), nullable=False),
|
||||
sa.Column("provider_delivery_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("provider_message_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("postbox_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("address", sa.String(length=500), nullable=True),
|
||||
sa.Column("holder_count", sa.Integer(), nullable=True),
|
||||
sa.Column("vacant", sa.Boolean(), nullable=True),
|
||||
sa.Column(
|
||||
"duplicate",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default=sa.false(),
|
||||
),
|
||||
sa.Column("evidence", sa.JSON(), nullable=False),
|
||||
sa.Column("error_type", sa.String(length=255), nullable=True),
|
||||
sa.Column("error_code", sa.String(length=100), nullable=True),
|
||||
sa.Column("error_message", sa.Text(), nullable=True),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["job_id"],
|
||||
["campaign_jobs.id"],
|
||||
name=op.f(
|
||||
"fk_campaign_postbox_delivery_attempts_job_id_campaign_jobs"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.PrimaryKeyConstraint(
|
||||
"id",
|
||||
name=op.f("pk_campaign_postbox_delivery_attempts"),
|
||||
),
|
||||
sa.UniqueConstraint(
|
||||
"job_id",
|
||||
"target_key",
|
||||
"attempt_number",
|
||||
name="uq_campaign_postbox_attempt_target_number",
|
||||
),
|
||||
)
|
||||
for column in ("tenant_id", "job_id", "status", "postbox_id"):
|
||||
op.create_index(
|
||||
op.f(f"ix_campaign_postbox_delivery_attempts_{column}"),
|
||||
"campaign_postbox_delivery_attempts",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
op.create_index(
|
||||
"ix_campaign_postbox_attempt_job_status",
|
||||
"campaign_postbox_delivery_attempts",
|
||||
["job_id", "status"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_index(
|
||||
"ix_campaign_postbox_attempt_idempotency",
|
||||
"campaign_postbox_delivery_attempts",
|
||||
["tenant_id", "idempotency_key"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("campaign_postbox_delivery_attempts")
|
||||
op.drop_index(
|
||||
op.f("ix_campaign_jobs_postbox_status"),
|
||||
table_name="campaign_jobs",
|
||||
)
|
||||
op.drop_index(
|
||||
op.f("ix_campaign_jobs_delivery_channel_policy"),
|
||||
table_name="campaign_jobs",
|
||||
)
|
||||
op.drop_column("campaign_jobs", "resolved_postbox_targets")
|
||||
op.drop_column("campaign_jobs", "postbox_attempt_count")
|
||||
op.drop_column("campaign_jobs", "postbox_status")
|
||||
op.drop_column("campaign_jobs", "delivery_channel_policy")
|
||||
@@ -0,0 +1,56 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
|
||||
from govoplan_core.core.object_storage import (
|
||||
StorageBackendError,
|
||||
configured_storage_backend,
|
||||
)
|
||||
from govoplan_core.core.operations import OperationalCheck
|
||||
from govoplan_core.settings import settings as core_settings
|
||||
from govoplan_campaign.backend.runtime import get_settings
|
||||
|
||||
|
||||
def generated_eml_storage_check() -> OperationalCheck:
|
||||
"""Verify Campaign evidence against the deployment object-store boundary."""
|
||||
|
||||
storage = configured_storage_backend(get_settings() or core_settings)
|
||||
probe = f"campaign-artifacts/.health/{secrets.token_hex(16)}.probe"
|
||||
payload = secrets.token_bytes(64)
|
||||
try:
|
||||
storage.put_bytes(probe, payload, content_type="application/octet-stream")
|
||||
if storage.get_bytes(probe) != payload:
|
||||
raise OSError("generated EML persistence returned different bytes")
|
||||
except (OSError, StorageBackendError) as exc:
|
||||
return OperationalCheck(
|
||||
id="campaign.generated_eml_storage",
|
||||
label="Generated Campaign EML evidence",
|
||||
state="error",
|
||||
detail=(
|
||||
"The generated EML object store failed a bounded write/read probe "
|
||||
f"({type(exc).__name__})."
|
||||
),
|
||||
readiness_critical=True,
|
||||
metrics={"backend": storage.name},
|
||||
)
|
||||
finally:
|
||||
try:
|
||||
storage.delete(probe)
|
||||
except StorageBackendError:
|
||||
pass
|
||||
|
||||
node_local = storage.name == "local"
|
||||
return OperationalCheck(
|
||||
id="campaign.generated_eml_storage",
|
||||
label="Generated Campaign EML evidence",
|
||||
state="warning" if node_local else "ok",
|
||||
detail=(
|
||||
"Generated EML passed the object-store write/read/delete probe. "
|
||||
+ (
|
||||
"The configured backend is node-local and is suitable only for a single-node profile."
|
||||
if node_local
|
||||
else "The configured backend is shared across application and worker nodes."
|
||||
)
|
||||
),
|
||||
metrics={"backend": storage.name},
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -58,6 +58,10 @@ class AggregateOutcomeCounts(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
smtp_accepted: AggregateCount
|
||||
postbox_accepted: AggregateCount
|
||||
print_accepted: AggregateCount
|
||||
delivered: AggregateCount
|
||||
partially_accepted: AggregateCount
|
||||
failed: AggregateCount
|
||||
outcome_unknown: AggregateCount
|
||||
queued_or_active: AggregateCount
|
||||
@@ -105,6 +109,10 @@ class AggregateCampaignReport(BaseModel):
|
||||
|
||||
_OUTCOME_KEYS = (
|
||||
"smtp_accepted",
|
||||
"postbox_accepted",
|
||||
"print_accepted",
|
||||
"delivered",
|
||||
"partially_accepted",
|
||||
"failed",
|
||||
"outcome_unknown",
|
||||
"queued_or_active",
|
||||
@@ -175,7 +183,19 @@ def _query_aggregate_facts(
|
||||
if version is None:
|
||||
return _AggregateFacts.empty()
|
||||
|
||||
accepted = CampaignJob.send_status.in_({"smtp_accepted", "sent"})
|
||||
smtp_accepted = CampaignJob.send_status.in_(
|
||||
{"smtp_accepted", "sent"}
|
||||
)
|
||||
accepted = CampaignJob.send_status.in_(
|
||||
{
|
||||
"smtp_accepted",
|
||||
"postbox_accepted",
|
||||
"print_accepted",
|
||||
"delivered",
|
||||
"partially_accepted",
|
||||
"sent",
|
||||
}
|
||||
)
|
||||
failed = CampaignJob.send_status.in_({"failed_temporary", "failed_permanent"})
|
||||
unknown = CampaignJob.send_status == "outcome_unknown"
|
||||
active = CampaignJob.send_status.in_({"queued", "claimed", "sending"})
|
||||
@@ -188,7 +208,39 @@ def _query_aggregate_facts(
|
||||
row = (
|
||||
session.query(
|
||||
func.count(CampaignJob.id).label("denominator"),
|
||||
func.sum(case((accepted, 1), else_=0)).label("smtp_accepted"),
|
||||
func.sum(case((smtp_accepted, 1), else_=0)).label(
|
||||
"smtp_accepted"
|
||||
),
|
||||
func.sum(
|
||||
case(
|
||||
(
|
||||
CampaignJob.send_status == "postbox_accepted",
|
||||
1,
|
||||
),
|
||||
else_=0,
|
||||
)
|
||||
).label("postbox_accepted"),
|
||||
func.sum(
|
||||
case(
|
||||
(CampaignJob.send_status == "print_accepted", 1),
|
||||
else_=0,
|
||||
)
|
||||
).label("print_accepted"),
|
||||
func.sum(
|
||||
case(
|
||||
(CampaignJob.send_status == "delivered", 1),
|
||||
else_=0,
|
||||
)
|
||||
).label("delivered"),
|
||||
func.sum(
|
||||
case(
|
||||
(
|
||||
CampaignJob.send_status == "partially_accepted",
|
||||
1,
|
||||
),
|
||||
else_=0,
|
||||
)
|
||||
).label("partially_accepted"),
|
||||
func.sum(case((failed, 1), else_=0)).label("failed"),
|
||||
func.sum(case((unknown, 1), else_=0)).label("outcome_unknown"),
|
||||
func.sum(case((active, 1), else_=0)).label("queued_or_active"),
|
||||
@@ -353,6 +405,14 @@ def _outcome_counts(jobs: list[CampaignJob]) -> dict[str, int]:
|
||||
status = job.send_status
|
||||
if status in {"smtp_accepted", "sent"}:
|
||||
counts["smtp_accepted"] += 1
|
||||
elif status == "postbox_accepted":
|
||||
counts["postbox_accepted"] += 1
|
||||
elif status == "print_accepted":
|
||||
counts["print_accepted"] += 1
|
||||
elif status == "delivered":
|
||||
counts["delivered"] += 1
|
||||
elif status == "partially_accepted":
|
||||
counts["partially_accepted"] += 1
|
||||
elif status in {"failed_temporary", "failed_permanent"}:
|
||||
counts["failed"] += 1
|
||||
elif status == "outcome_unknown":
|
||||
@@ -460,10 +520,16 @@ def _completion_state(
|
||||
return "outcome_unknown"
|
||||
if counts["queued_or_active"]:
|
||||
return "in_progress"
|
||||
accepted = counts["smtp_accepted"]
|
||||
if accepted == total:
|
||||
fully_accepted = (
|
||||
counts["smtp_accepted"]
|
||||
+ counts["postbox_accepted"]
|
||||
+ counts["print_accepted"]
|
||||
+ counts["delivered"]
|
||||
)
|
||||
partially_accepted = counts["partially_accepted"]
|
||||
if fully_accepted == total:
|
||||
return "completed"
|
||||
if accepted:
|
||||
if fully_accepted or partially_accepted:
|
||||
return "partially_completed"
|
||||
return "incomplete"
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -13,7 +13,11 @@ from govoplan_campaign.backend.campaign.models import CampaignConfig
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_id
|
||||
from govoplan_campaign.backend.persistence.campaigns import load_version_config
|
||||
from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
||||
from govoplan_campaign.backend.integrations import SmtpConfigurationError
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
MailDeliveryCommandError,
|
||||
SmtpConfigurationError,
|
||||
mail_integration,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, ensure_execution_snapshot
|
||||
|
||||
|
||||
@@ -32,6 +36,9 @@ class CampaignReportEmailResult:
|
||||
attached_jobs_csv: bool
|
||||
attached_report_json: bool
|
||||
accepted_count: int | None = None
|
||||
command_id: str | None = None
|
||||
delivery_status: str | None = None
|
||||
duplicate: bool = False
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
@@ -44,6 +51,22 @@ class CampaignReportEmailResult:
|
||||
"attached_jobs_csv": self.attached_jobs_csv,
|
||||
"attached_report_json": self.attached_report_json,
|
||||
"accepted_count": self.accepted_count,
|
||||
"command_id": self.command_id,
|
||||
"delivery_status": self.delivery_status,
|
||||
"duplicate": self.duplicate,
|
||||
}
|
||||
|
||||
def audit_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"campaign_id": self.campaign_id,
|
||||
"version_id": self.version_id,
|
||||
"recipient_count": len(self.to),
|
||||
"dry_run": self.dry_run,
|
||||
"attached_jobs_csv": self.attached_jobs_csv,
|
||||
"attached_report_json": self.attached_report_json,
|
||||
"command_id": self.command_id,
|
||||
"delivery_status": self.delivery_status,
|
||||
"duplicate": self.duplicate,
|
||||
}
|
||||
|
||||
|
||||
@@ -149,6 +172,8 @@ def send_campaign_report_email(
|
||||
attach_jobs_csv: bool = False,
|
||||
attach_report_json: bool = False,
|
||||
dry_run: bool = False,
|
||||
idempotency_key: str | None = None,
|
||||
created_by_user_id: str | None = None,
|
||||
) -> CampaignReportEmailResult:
|
||||
campaign = session.get(Campaign, campaign_id)
|
||||
if not campaign or campaign.tenant_id != tenant_id:
|
||||
@@ -175,11 +200,16 @@ def send_campaign_report_email(
|
||||
) from exc
|
||||
if not snapshot.smtp_transport_revision:
|
||||
raise CampaignReportEmailError("Campaign build evidence has no SMTP transport revision")
|
||||
mail = mail_integration()
|
||||
if not dry_run:
|
||||
raise CampaignReportEmailError(
|
||||
"Report email delivery is disabled until it uses a durable, idempotent Mail-owned outbox with unknown-outcome reconciliation."
|
||||
)
|
||||
|
||||
if not mail.durable_delivery_available:
|
||||
raise CampaignReportEmailError(
|
||||
"Report email delivery requires the durable, idempotent Mail-owned outbox."
|
||||
)
|
||||
if not str(idempotency_key or "").strip():
|
||||
raise CampaignReportEmailError(
|
||||
"Live report email delivery requires an idempotency key"
|
||||
)
|
||||
report = generate_campaign_report(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
@@ -202,6 +232,48 @@ def send_campaign_report_email(
|
||||
jobs_csv=jobs_csv,
|
||||
report_json=report_json,
|
||||
)
|
||||
if not dry_run:
|
||||
clean_idempotency_key = str(idempotency_key or "").strip()
|
||||
from_email, _from_name = _effective_from(config)
|
||||
if not snapshot.mail_profile_id:
|
||||
raise CampaignReportEmailError(
|
||||
"Campaign build evidence has no Mail profile reference"
|
||||
)
|
||||
try:
|
||||
command = mail.submit_delivery_command(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
command_type="campaign_report",
|
||||
source_module="campaigns",
|
||||
source_resource_type="campaign",
|
||||
source_resource_id=campaign.id,
|
||||
source_version_id=version.id,
|
||||
idempotency_key=clean_idempotency_key,
|
||||
profile_id=snapshot.mail_profile_id,
|
||||
message_bytes=message.as_bytes(),
|
||||
envelope_from=from_email,
|
||||
envelope_recipients=to,
|
||||
from_header=str(message["From"]),
|
||||
expected_smtp_transport_revision=snapshot.smtp_transport_revision,
|
||||
smtp_server_id=snapshot.smtp_server_id,
|
||||
smtp_credential_id=snapshot.smtp_credential_id,
|
||||
created_by_user_id=created_by_user_id,
|
||||
)
|
||||
except MailDeliveryCommandError as exc:
|
||||
raise CampaignReportEmailError(str(exc)) from exc
|
||||
return CampaignReportEmailResult(
|
||||
campaign_id=campaign.id,
|
||||
version_id=version.id,
|
||||
to=to,
|
||||
subject=str(message["Subject"]),
|
||||
dry_run=False,
|
||||
sent=False,
|
||||
attached_jobs_csv=jobs_csv is not None,
|
||||
attached_report_json=report_json is not None,
|
||||
command_id=str(command["id"]),
|
||||
delivery_status=str(command["status"]),
|
||||
duplicate=bool(command.get("duplicate")),
|
||||
)
|
||||
return CampaignReportEmailResult(
|
||||
campaign_id=campaign.id,
|
||||
version_id=version.id,
|
||||
|
||||
@@ -0,0 +1,308 @@
|
||||
"""Cross-module provider for Campaign's recipient-free aggregate report."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.db.models import Campaign
|
||||
from govoplan_campaign.backend.report_privacy_policy import (
|
||||
effective_campaign_report_privacy_policy,
|
||||
)
|
||||
from govoplan_campaign.backend.reports.aggregate import (
|
||||
generate_aggregate_campaign_report,
|
||||
)
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_campaign_query_for_principal,
|
||||
_get_campaign_for_principal,
|
||||
)
|
||||
from govoplan_core.auth import has_scope
|
||||
from govoplan_core.core.reporting import (
|
||||
REPORT_PROVIDER_CONTRACT_VERSION,
|
||||
ReportDescriptor,
|
||||
ReportParameterDescriptor,
|
||||
ReportParameterOption,
|
||||
ReportPrivacyTransform,
|
||||
ReportProviderRequest,
|
||||
ReportProviderResult,
|
||||
ReportResultField,
|
||||
)
|
||||
|
||||
|
||||
CAMPAIGN_AGGREGATE_REPORT_ID = "delivery-outcomes"
|
||||
CAMPAIGN_REPORT_PRIVACY_TRANSFORMS = (
|
||||
"server_side_aggregation",
|
||||
"small_cell_suppression",
|
||||
"complementary_suppression",
|
||||
"explicit_denominator",
|
||||
"recipient_payload_exclusion",
|
||||
)
|
||||
|
||||
|
||||
class CampaignAggregateReportProvider:
|
||||
provider_id = "campaigns"
|
||||
contract_version = REPORT_PROVIDER_CONTRACT_VERSION
|
||||
|
||||
def list_reports(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
) -> tuple[ReportDescriptor, ...]:
|
||||
del session
|
||||
if not has_scope(principal, "campaigns:report:read"):
|
||||
return ()
|
||||
return (_descriptor(),)
|
||||
|
||||
def parameter_options(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
report_id: str,
|
||||
parameter_key: str,
|
||||
query: str,
|
||||
limit: int,
|
||||
) -> tuple[ReportParameterOption, ...]:
|
||||
if report_id != CAMPAIGN_AGGREGATE_REPORT_ID or parameter_key != "campaign_id":
|
||||
return ()
|
||||
if not has_scope(principal, "campaigns:report:read"):
|
||||
return ()
|
||||
sql_session = _session(session)
|
||||
rows = _campaign_query_for_principal(sql_session, principal)
|
||||
clean_query = query.strip().casefold()
|
||||
campaigns = (
|
||||
rows.order_by(Campaign.updated_at.desc(), Campaign.id.asc())
|
||||
.limit(max(1, min(limit, 200)) if not clean_query else 500)
|
||||
.all()
|
||||
)
|
||||
if clean_query:
|
||||
campaigns = [
|
||||
campaign
|
||||
for campaign in campaigns
|
||||
if clean_query in campaign.name.casefold()
|
||||
][: max(1, min(limit, 200))]
|
||||
return tuple(
|
||||
ReportParameterOption(
|
||||
value=campaign.id,
|
||||
label=campaign.name,
|
||||
description=campaign.status,
|
||||
)
|
||||
for campaign in campaigns
|
||||
)
|
||||
|
||||
def execute_report(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
request: ReportProviderRequest,
|
||||
) -> ReportProviderResult:
|
||||
if request.report_id != CAMPAIGN_AGGREGATE_REPORT_ID:
|
||||
raise LookupError("Campaign report provider does not know this report")
|
||||
if not has_scope(principal, "campaigns:report:read"):
|
||||
raise PermissionError("Missing scope: campaigns:report:read")
|
||||
campaign_id = str(request.parameters.get("campaign_id") or "").strip()
|
||||
if not campaign_id:
|
||||
raise ValueError("campaign_id is required")
|
||||
version_id = str(request.parameters.get("version_id") or "").strip() or None
|
||||
sql_session = _session(session)
|
||||
campaign = _get_campaign_for_principal(sql_session, campaign_id, principal)
|
||||
report = generate_aggregate_campaign_report(
|
||||
sql_session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign.id,
|
||||
version_id=version_id,
|
||||
)
|
||||
policy = effective_campaign_report_privacy_policy(
|
||||
sql_session,
|
||||
tenant_id=principal.tenant_id,
|
||||
)
|
||||
selected_version_id = version_id or campaign.current_version_id
|
||||
return ReportProviderResult(
|
||||
report_id=CAMPAIGN_AGGREGATE_REPORT_ID,
|
||||
generated_at=report.generated_at,
|
||||
payload=report.model_dump(mode="json"),
|
||||
source_revisions=(
|
||||
{
|
||||
"module_id": "campaigns",
|
||||
"resource_type": "campaign",
|
||||
"resource_id": campaign.id,
|
||||
"revision_type": "campaign_version",
|
||||
"revision_id": selected_version_id,
|
||||
"version_number": report.version_number,
|
||||
"updated_at": campaign.updated_at.isoformat(),
|
||||
},
|
||||
),
|
||||
effective_scope={
|
||||
"tenant_id": principal.tenant_id,
|
||||
"campaign_id": campaign.id,
|
||||
"campaign_version_id": selected_version_id,
|
||||
"audience": dict(request.audience_scope),
|
||||
},
|
||||
applied_privacy_transforms=CAMPAIGN_REPORT_PRIVACY_TRANSFORMS,
|
||||
provenance={
|
||||
"provider": "campaigns",
|
||||
"projection": "recipient-free-delivery-outcomes-v1",
|
||||
"privacy_policy": policy.as_dict(),
|
||||
"purpose": request.purpose,
|
||||
},
|
||||
)
|
||||
|
||||
def authorize_result(
|
||||
self,
|
||||
session: object,
|
||||
principal: object,
|
||||
*,
|
||||
report_id: str,
|
||||
source_revisions: tuple[Mapping[str, object], ...],
|
||||
effective_scope: Mapping[str, object],
|
||||
) -> bool:
|
||||
del source_revisions
|
||||
if report_id != CAMPAIGN_AGGREGATE_REPORT_ID or not has_scope(
|
||||
principal, "campaigns:report:read"
|
||||
):
|
||||
return False
|
||||
campaign_id = str(effective_scope.get("campaign_id") or "").strip()
|
||||
tenant_id = str(effective_scope.get("tenant_id") or "").strip()
|
||||
if not campaign_id or tenant_id != str(getattr(principal, "tenant_id", "")):
|
||||
return False
|
||||
return (
|
||||
_campaign_query_for_principal(_session(session), principal)
|
||||
.filter(Campaign.id == campaign_id)
|
||||
.first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _descriptor() -> ReportDescriptor:
|
||||
fields = (
|
||||
("generated_at", "Generated", "datetime", "Campaign"),
|
||||
("campaign.id", "Campaign ID", "string", "Campaign"),
|
||||
("campaign.name", "Campaign", "string", "Campaign"),
|
||||
("campaign.status", "Status", "string", "Campaign"),
|
||||
("version_number", "Version", "integer", "Campaign"),
|
||||
("completion_state", "Completion", "string", "Campaign"),
|
||||
(
|
||||
"population.denominator",
|
||||
"Report denominator",
|
||||
"suppressed_count",
|
||||
"Population",
|
||||
),
|
||||
(
|
||||
"population.denominator_definition",
|
||||
"Denominator definition",
|
||||
"string",
|
||||
"Population",
|
||||
),
|
||||
(
|
||||
"population.inactive_source_entries",
|
||||
"Inactive source entries",
|
||||
"suppressed_count",
|
||||
"Population",
|
||||
),
|
||||
(
|
||||
"population.excluded_or_blocked_jobs",
|
||||
"Excluded or blocked jobs",
|
||||
"suppressed_count",
|
||||
"Population",
|
||||
),
|
||||
("outcomes.smtp_accepted", "SMTP accepted", "suppressed_count", "Outcomes"),
|
||||
(
|
||||
"outcomes.postbox_accepted",
|
||||
"Postbox accepted",
|
||||
"suppressed_count",
|
||||
"Outcomes",
|
||||
),
|
||||
(
|
||||
"outcomes.print_accepted",
|
||||
"Printable output accepted",
|
||||
"suppressed_count",
|
||||
"Outcomes",
|
||||
),
|
||||
(
|
||||
"outcomes.delivered",
|
||||
"Both channels accepted",
|
||||
"suppressed_count",
|
||||
"Outcomes",
|
||||
),
|
||||
(
|
||||
"outcomes.partially_accepted",
|
||||
"Partially accepted",
|
||||
"suppressed_count",
|
||||
"Outcomes",
|
||||
),
|
||||
("outcomes.failed", "Failed", "suppressed_count", "Outcomes"),
|
||||
("outcomes.outcome_unknown", "Outcome unknown", "suppressed_count", "Outcomes"),
|
||||
(
|
||||
"outcomes.queued_or_active",
|
||||
"Queued or active",
|
||||
"suppressed_count",
|
||||
"Outcomes",
|
||||
),
|
||||
("outcomes.not_attempted", "Not attempted", "suppressed_count", "Outcomes"),
|
||||
("outcomes.cancelled", "Cancelled", "suppressed_count", "Outcomes"),
|
||||
("outcomes.excluded", "Excluded", "suppressed_count", "Outcomes"),
|
||||
("time_range.first_activity_at", "First activity", "datetime", "Activity"),
|
||||
("time_range.last_activity_at", "Last activity", "datetime", "Activity"),
|
||||
("time_range.suppressed", "Activity range suppressed", "boolean", "Activity"),
|
||||
("privacy.small_cell_threshold", "Small-cell threshold", "integer", "Privacy"),
|
||||
("privacy.suppression_applied", "Suppression applied", "boolean", "Privacy"),
|
||||
("privacy.rule", "Privacy rule", "string", "Privacy"),
|
||||
)
|
||||
return ReportDescriptor(
|
||||
provider_id="campaigns",
|
||||
report_id=CAMPAIGN_AGGREGATE_REPORT_ID,
|
||||
revision="campaign.aggregate.v1",
|
||||
title="Campaign delivery outcomes",
|
||||
summary=(
|
||||
"Privacy-protected delivery outcomes without recipient-level records."
|
||||
),
|
||||
parameters=(
|
||||
ReportParameterDescriptor(
|
||||
key="campaign_id",
|
||||
label="Campaign",
|
||||
type="reference",
|
||||
required=True,
|
||||
options_from_provider=True,
|
||||
),
|
||||
ReportParameterDescriptor(
|
||||
key="version_id",
|
||||
label="Campaign version",
|
||||
type="string",
|
||||
required=False,
|
||||
description="Leave empty to use the current campaign version.",
|
||||
),
|
||||
),
|
||||
result_schema=tuple(
|
||||
ReportResultField(
|
||||
path=path,
|
||||
label=label,
|
||||
type=field_type, # type: ignore[arg-type]
|
||||
group=group,
|
||||
nullable=path.startswith("time_range.") or path == "version_number",
|
||||
)
|
||||
for path, label, field_type, group in fields
|
||||
),
|
||||
privacy_transforms=tuple(
|
||||
ReportPrivacyTransform(id=item, label=item.replace("_", " ").title())
|
||||
for item in CAMPAIGN_REPORT_PRIVACY_TRANSFORMS
|
||||
),
|
||||
retention_class="stored_report_detail",
|
||||
export_formats=("json",),
|
||||
reidentification_risk="low",
|
||||
presentation={"kind": "metric_summary"},
|
||||
)
|
||||
|
||||
|
||||
def _session(value: object) -> Session:
|
||||
if not isinstance(value, Session):
|
||||
raise TypeError("Campaign report provider requires a SQLAlchemy Session")
|
||||
return value
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CAMPAIGN_AGGREGATE_REPORT_ID",
|
||||
"CAMPAIGN_REPORT_PRIVACY_TRANSFORMS",
|
||||
"CampaignAggregateReportProvider",
|
||||
]
|
||||
@@ -110,6 +110,7 @@ def public_delivery_result_message(
|
||||
last_error: Any,
|
||||
send_status: Any,
|
||||
imap_status: Any,
|
||||
postbox_status: Any = None,
|
||||
) -> str | None:
|
||||
"""Map persisted provider text to a stable business-safe explanation."""
|
||||
|
||||
@@ -117,10 +118,22 @@ def public_delivery_result_message(
|
||||
return None
|
||||
clean_send_status = str(send_status or "")
|
||||
clean_imap_status = str(imap_status or "")
|
||||
clean_postbox_status = str(postbox_status or "")
|
||||
if clean_postbox_status == "outcome_unknown":
|
||||
return "Postbox delivery outcome requires operator reconciliation."
|
||||
if clean_send_status == "outcome_unknown":
|
||||
return "SMTP delivery outcome requires operator reconciliation."
|
||||
return "Delivery outcome requires operator reconciliation."
|
||||
if clean_postbox_status in {
|
||||
"rejected_temporary",
|
||||
"rejected_permanent",
|
||||
}:
|
||||
return "Postbox delivery was rejected; an operator can inspect restricted diagnostics."
|
||||
if clean_postbox_status == "partially_accepted":
|
||||
return "Some Postbox targets accepted the message and others rejected it."
|
||||
if clean_send_status in {"failed_temporary", "failed_permanent"}:
|
||||
return "SMTP delivery failed; an operator can inspect restricted diagnostics."
|
||||
if clean_postbox_status in {"", "not_requested"}:
|
||||
return "SMTP delivery failed; an operator can inspect restricted diagnostics."
|
||||
return "Delivery failed; an operator can inspect restricted diagnostics."
|
||||
if clean_imap_status in {"outcome_unknown", "appending"}:
|
||||
return "Sent-folder append outcome requires operator reconciliation."
|
||||
if clean_imap_status in {"failed", "skipped"}:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
from dataclasses import dataclass
|
||||
import hashlib
|
||||
import json
|
||||
from datetime import datetime, timedelta, timezone
|
||||
@@ -9,7 +10,29 @@ from typing import Any, Callable
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.recovery import (
|
||||
RecoveryGuaranteeError,
|
||||
RecoveryMode,
|
||||
RecoveryPlan,
|
||||
RecoveryStatus,
|
||||
)
|
||||
from govoplan_core.core.recovery_runtime import (
|
||||
DurableRecoveryOperation,
|
||||
RecoveryOperationBusy,
|
||||
RecoveryOperationStateConflict,
|
||||
begin_durable_recovery_operation,
|
||||
)
|
||||
from govoplan_core.core.object_storage import (
|
||||
StorageBackend,
|
||||
StorageBackendError,
|
||||
StorageObjectMissing,
|
||||
configured_storage_backend,
|
||||
)
|
||||
from govoplan_core.core.runtime_coordination import process_runtime_identity
|
||||
from govoplan_core.db.session import get_database
|
||||
from govoplan_core.settings import settings as core_settings
|
||||
from govoplan_campaign.backend.db.models import CampaignJob, CampaignVersion, JobImapStatus, JobQueueStatus
|
||||
from govoplan_campaign.backend.runtime import get_settings
|
||||
|
||||
FINAL_VERSION_STATES = {
|
||||
"completed",
|
||||
@@ -28,6 +51,15 @@ FINAL_EML_SEND_STATUSES = {
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _GeneratedArtifactRecovery:
|
||||
operation: DurableRecoveryOperation
|
||||
job_id: str
|
||||
storage_key: str | None
|
||||
local_path: str | None
|
||||
storage: StorageBackend | None
|
||||
|
||||
|
||||
def _cutoff(days: int | None, *, now: datetime) -> datetime | None:
|
||||
if days is None:
|
||||
return None
|
||||
@@ -105,14 +137,221 @@ def _apply_raw_json_retention(
|
||||
return result
|
||||
|
||||
|
||||
def _artifact_locator_sha256(
|
||||
*,
|
||||
storage_key: str | None,
|
||||
local_path: str | None,
|
||||
) -> str:
|
||||
return _json_sha256(
|
||||
{
|
||||
"storage_key": storage_key,
|
||||
"local_path": local_path,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _begin_generated_artifact_recovery(
|
||||
*,
|
||||
job: CampaignJob,
|
||||
storage: StorageBackend | None,
|
||||
) -> _GeneratedArtifactRecovery | None:
|
||||
storage_key = str(job.eml_storage_key) if job.eml_storage_key else None
|
||||
local_path = str(job.eml_local_path) if job.eml_local_path else None
|
||||
locator_sha256 = _artifact_locator_sha256(
|
||||
storage_key=storage_key,
|
||||
local_path=local_path,
|
||||
)
|
||||
try:
|
||||
started = begin_durable_recovery_operation(
|
||||
get_database().SessionLocal,
|
||||
identity=process_runtime_identity(),
|
||||
module_id="campaigns",
|
||||
operation_type="generated-artifact-retention",
|
||||
idempotency_key=(
|
||||
f"campaign-retention:{job.id}:{locator_sha256[:32]}"
|
||||
),
|
||||
request={
|
||||
"tenant_id": job.tenant_id,
|
||||
"campaign_id": job.campaign_id,
|
||||
"version_id": job.campaign_version_id,
|
||||
"job_id": job.id,
|
||||
"message_sha256": job.eml_sha256,
|
||||
"artifact_locator_sha256": locator_sha256,
|
||||
},
|
||||
recovery_plan=RecoveryPlan(
|
||||
mode=RecoveryMode.FORWARD_RECOVERY,
|
||||
preconditions=(
|
||||
"the Campaign job is terminal and outside its retention window",
|
||||
"no IMAP append or delivery outcome remains unresolved",
|
||||
),
|
||||
forward_recovery_steps=(
|
||||
"verify whether each recorded artifact still exists",
|
||||
"clear the database locator only after absence is established",
|
||||
),
|
||||
verification_steps=(
|
||||
"reload the Campaign job through an independent session",
|
||||
"probe every original object or local-development path",
|
||||
),
|
||||
),
|
||||
precondition_evidence={
|
||||
"job_id": job.id,
|
||||
"queue_status": job.queue_status,
|
||||
"send_status": job.send_status,
|
||||
"imap_status": job.imap_status,
|
||||
"message_sha256": job.eml_sha256,
|
||||
"artifact_locator_sha256": locator_sha256,
|
||||
},
|
||||
lease_resource_key=f"campaign:retention:{job.tenant_id}:{job.id}",
|
||||
lease_ttl_seconds=15 * 60,
|
||||
resource_type="campaign_job",
|
||||
resource_id=job.id,
|
||||
metadata={
|
||||
"resources": [
|
||||
"postgresql",
|
||||
"object-storage" if storage_key else "local-development-storage",
|
||||
],
|
||||
},
|
||||
)
|
||||
except (RecoveryOperationBusy, RecoveryOperationStateConflict):
|
||||
return None
|
||||
if started.replayed or started.operation is None:
|
||||
return None
|
||||
return _GeneratedArtifactRecovery(
|
||||
operation=started.operation,
|
||||
job_id=job.id,
|
||||
storage_key=storage_key,
|
||||
local_path=local_path,
|
||||
storage=storage,
|
||||
)
|
||||
|
||||
|
||||
def _generated_artifact_recovery_evidence(
|
||||
recovery: _GeneratedArtifactRecovery,
|
||||
) -> tuple[str, dict[str, Any]]:
|
||||
probes: dict[str, bool | None] = {}
|
||||
if recovery.storage_key:
|
||||
try:
|
||||
if recovery.storage is None:
|
||||
raise StorageBackendError("Artifact storage is unavailable")
|
||||
probes["object_missing"] = not recovery.storage.exists(
|
||||
recovery.storage_key
|
||||
)
|
||||
except (StorageBackendError, OSError):
|
||||
probes["object_missing"] = None
|
||||
if recovery.local_path:
|
||||
try:
|
||||
probes["local_path_missing"] = not Path(recovery.local_path).exists()
|
||||
except OSError:
|
||||
probes["local_path_missing"] = None
|
||||
|
||||
with get_database().SessionLocal() as evidence_session:
|
||||
job = evidence_session.get(CampaignJob, recovery.job_id)
|
||||
job_present = job is not None
|
||||
metadata_cleared = bool(
|
||||
job is None
|
||||
or (
|
||||
(
|
||||
not recovery.storage_key
|
||||
or job.eml_storage_key != recovery.storage_key
|
||||
)
|
||||
and (
|
||||
not recovery.local_path
|
||||
or job.eml_local_path != recovery.local_path
|
||||
)
|
||||
)
|
||||
)
|
||||
metadata_intact = bool(
|
||||
job is not None
|
||||
and job.eml_storage_key == recovery.storage_key
|
||||
and job.eml_local_path == recovery.local_path
|
||||
)
|
||||
|
||||
probe_values = tuple(probes.values())
|
||||
probe_verified = bool(probe_values) and all(
|
||||
value is not None for value in probe_values
|
||||
)
|
||||
artifacts_absent = probe_verified and all(value is True for value in probe_values)
|
||||
artifacts_intact = probe_verified and all(value is False for value in probe_values)
|
||||
evidence = {
|
||||
"verified": probe_verified,
|
||||
"checks": {
|
||||
"job_state_reloaded": True,
|
||||
"artifact_locations_probed": probe_verified,
|
||||
},
|
||||
"job_present": job_present,
|
||||
"metadata_cleared": metadata_cleared,
|
||||
"metadata_intact": metadata_intact,
|
||||
"artifact_probes": probes,
|
||||
}
|
||||
if not probe_verified:
|
||||
return "outcome_unknown", evidence
|
||||
if artifacts_absent and metadata_cleared:
|
||||
return "succeeded", evidence
|
||||
if artifacts_intact and metadata_intact:
|
||||
return "failed", evidence
|
||||
return "recovery_required", evidence
|
||||
|
||||
|
||||
def _finish_generated_artifact_recovery(
|
||||
recovery: _GeneratedArtifactRecovery,
|
||||
) -> None:
|
||||
outcome, evidence = _generated_artifact_recovery_evidence(recovery)
|
||||
if outcome == "succeeded":
|
||||
recovery.operation.succeed(evidence=evidence)
|
||||
elif outcome == "failed":
|
||||
recovery.operation.reject(
|
||||
summary="Generated Campaign artifacts were not deleted",
|
||||
evidence=evidence,
|
||||
)
|
||||
elif outcome == "outcome_unknown":
|
||||
recovery.operation.unresolved(
|
||||
status=RecoveryStatus.OUTCOME_UNKNOWN,
|
||||
summary="Generated artifact deletion could not be verified",
|
||||
evidence=evidence,
|
||||
failure_summary="Artifact storage availability prevented verification",
|
||||
)
|
||||
else:
|
||||
recovery.operation.unresolved(
|
||||
status=RecoveryStatus.RECOVERY_REQUIRED,
|
||||
summary="Generated artifact retention is only partially complete",
|
||||
evidence=evidence,
|
||||
failure_summary="Artifact and Campaign metadata state require reconciliation",
|
||||
)
|
||||
|
||||
|
||||
def _finish_generated_artifact_recoveries(
|
||||
recoveries: list[_GeneratedArtifactRecovery],
|
||||
) -> None:
|
||||
failures: list[Exception] = []
|
||||
for recovery in recoveries:
|
||||
try:
|
||||
_finish_generated_artifact_recovery(recovery)
|
||||
except Exception as exc: # preserve every operation's chance to close
|
||||
failures.append(exc)
|
||||
if failures:
|
||||
raise RecoveryGuaranteeError(
|
||||
f"{len(failures)} Campaign retention recovery operation(s) could not be finalized"
|
||||
) from failures[0]
|
||||
|
||||
|
||||
def _apply_eml_retention(
|
||||
session: Session,
|
||||
*,
|
||||
dry_run: bool,
|
||||
now: datetime,
|
||||
policy_for_campaign_id: Callable[[str | None], object],
|
||||
storage: StorageBackend | None = None,
|
||||
recovery_operations: list[_GeneratedArtifactRecovery] | None = None,
|
||||
) -> dict[str, int]:
|
||||
result = {"eligible": 0, "metadata_cleared": 0, "files_deleted": 0, "files_missing": 0, "skipped_not_final": 0}
|
||||
result = {
|
||||
"eligible": 0,
|
||||
"metadata_cleared": 0,
|
||||
"files_deleted": 0,
|
||||
"files_missing": 0,
|
||||
"delete_failed": 0,
|
||||
"recovery_blocked": 0,
|
||||
"skipped_not_final": 0,
|
||||
}
|
||||
jobs = (
|
||||
session.query(CampaignJob)
|
||||
.filter((CampaignJob.eml_local_path.is_not(None)) | (CampaignJob.eml_storage_key.is_not(None)))
|
||||
@@ -137,6 +376,33 @@ def _apply_eml_retention(
|
||||
result["eligible"] += 1
|
||||
if dry_run:
|
||||
continue
|
||||
active_storage = storage
|
||||
if job.eml_storage_key and active_storage is None:
|
||||
active_storage = configured_storage_backend(
|
||||
get_settings() or core_settings
|
||||
)
|
||||
if recovery_operations is not None:
|
||||
recovery = _begin_generated_artifact_recovery(
|
||||
job=job,
|
||||
storage=active_storage,
|
||||
)
|
||||
if recovery is None:
|
||||
result["recovery_blocked"] += 1
|
||||
continue
|
||||
recovery_operations.append(recovery)
|
||||
if job.eml_storage_key:
|
||||
assert active_storage is not None
|
||||
try:
|
||||
if active_storage.exists(job.eml_storage_key):
|
||||
active_storage.delete(job.eml_storage_key)
|
||||
result["files_deleted"] += 1
|
||||
else:
|
||||
result["files_missing"] += 1
|
||||
except StorageObjectMissing:
|
||||
result["files_missing"] += 1
|
||||
except StorageBackendError:
|
||||
result["delete_failed"] += 1
|
||||
continue
|
||||
if job.eml_local_path:
|
||||
path = Path(job.eml_local_path)
|
||||
if path.exists():
|
||||
@@ -188,8 +454,42 @@ def apply_campaign_retention(
|
||||
now: datetime,
|
||||
policy_for_campaign_id: Callable[[str | None], object],
|
||||
) -> dict[str, dict[str, int]]:
|
||||
return {
|
||||
"raw_campaign_json": _apply_raw_json_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
|
||||
"generated_eml": _apply_eml_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
|
||||
"stored_report_detail": _apply_report_detail_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
|
||||
}
|
||||
recoveries: list[_GeneratedArtifactRecovery] = []
|
||||
try:
|
||||
# Start external-effect fences before queries for database-only
|
||||
# redaction can autoflush unrelated changes in the caller session.
|
||||
generated_eml = _apply_eml_retention(
|
||||
session,
|
||||
dry_run=dry_run,
|
||||
now=now,
|
||||
policy_for_campaign_id=policy_for_campaign_id,
|
||||
recovery_operations=None if dry_run else recoveries,
|
||||
)
|
||||
result = {
|
||||
"raw_campaign_json": _apply_raw_json_retention(
|
||||
session,
|
||||
dry_run=dry_run,
|
||||
now=now,
|
||||
policy_for_campaign_id=policy_for_campaign_id,
|
||||
),
|
||||
"generated_eml": generated_eml,
|
||||
"stored_report_detail": _apply_report_detail_retention(
|
||||
session,
|
||||
dry_run=dry_run,
|
||||
now=now,
|
||||
policy_for_campaign_id=policy_for_campaign_id,
|
||||
),
|
||||
}
|
||||
if not dry_run:
|
||||
# External artifact deletion and its locator update form one
|
||||
# module-owned recovery boundary. The outer Policy audit commits
|
||||
# separately after Campaign has verified this boundary.
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
if recoveries:
|
||||
_finish_generated_artifact_recoveries(recoveries)
|
||||
raise
|
||||
if recoveries:
|
||||
_finish_generated_artifact_recoveries(recoveries)
|
||||
return result
|
||||
|
||||
@@ -0,0 +1,666 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import dataclasses
|
||||
from collections.abc import Callable
|
||||
from typing import Any
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy import and_, exists, or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||
CAMPAIGN_MAIL_SERVER_KEYS,
|
||||
campaign_mail_profile_id,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignIssue,
|
||||
CampaignJob,
|
||||
CampaignShare,
|
||||
CampaignStatus,
|
||||
CampaignVersion,
|
||||
CampaignVersionWorkflowState,
|
||||
RecipientImportMappingProfile,
|
||||
)
|
||||
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
|
||||
from govoplan_campaign.backend.persistence.campaigns import CampaignPersistenceError
|
||||
from govoplan_campaign.backend.persistence.versions import (
|
||||
LockedCampaignVersionError,
|
||||
is_user_locked_version,
|
||||
is_version_final_locked,
|
||||
is_version_locked,
|
||||
update_campaign_version,
|
||||
)
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
CampaignVersionDetailResponse,
|
||||
CampaignVersionUpdateRequest,
|
||||
RecipientImportMappingProfilePayload,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.execution import (
|
||||
clear_execution_snapshot,
|
||||
)
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_core.auth import ApiPrincipal, has_scope
|
||||
from govoplan_core.core.access import CAPABILITY_ACCESS_DIRECTORY, AccessDirectory
|
||||
from govoplan_core.core.concurrency import (
|
||||
ConcurrencyError,
|
||||
MissingPreconditionError,
|
||||
RevisionConflictError,
|
||||
assert_revision_precondition,
|
||||
)
|
||||
from govoplan_core.core.runtime import get_registry
|
||||
|
||||
|
||||
def _capability_payload(value: object) -> dict[str, Any]:
|
||||
if dataclasses.is_dataclass(value):
|
||||
return dataclasses.asdict(value)
|
||||
if isinstance(value, dict):
|
||||
return dict(value)
|
||||
payload: dict[str, Any] = {}
|
||||
for key in (
|
||||
"contact_id",
|
||||
"address_book_id",
|
||||
"display_name",
|
||||
"email",
|
||||
"email_label",
|
||||
"organization",
|
||||
"role_title",
|
||||
"tags",
|
||||
"source_kind",
|
||||
"source_ref",
|
||||
"source_revision",
|
||||
"source_id",
|
||||
"source_label",
|
||||
"recipient_count",
|
||||
"generated_at",
|
||||
"recipients",
|
||||
"fields",
|
||||
"provenance",
|
||||
):
|
||||
if hasattr(value, key):
|
||||
payload[key] = getattr(value, key)
|
||||
return payload
|
||||
|
||||
|
||||
def _registry_capability(name: str) -> object | None:
|
||||
registry = get_registry()
|
||||
if (
|
||||
registry is None
|
||||
or not hasattr(registry, "has_capability")
|
||||
or not registry.has_capability(name)
|
||||
):
|
||||
return None
|
||||
return registry.capability(name)
|
||||
|
||||
|
||||
def _access_directory() -> AccessDirectory:
|
||||
registry = get_registry()
|
||||
if (
|
||||
registry is None
|
||||
or not hasattr(registry, "has_capability")
|
||||
or not registry.has_capability(CAPABILITY_ACCESS_DIRECTORY)
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="Access directory capability is not configured",
|
||||
)
|
||||
capability = registry.require_capability(CAPABILITY_ACCESS_DIRECTORY)
|
||||
if not isinstance(capability, AccessDirectory):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="Access directory capability is invalid",
|
||||
)
|
||||
return capability
|
||||
|
||||
|
||||
def _get_campaign_for_tenant(
|
||||
session: Session, campaign_id: str, tenant_id: str
|
||||
) -> Campaign:
|
||||
campaign = session.get(Campaign, campaign_id)
|
||||
if not campaign or campaign.tenant_id != tenant_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign not found"
|
||||
)
|
||||
return campaign
|
||||
|
||||
|
||||
def _get_version_for_tenant(
|
||||
session: Session, version_id: str, tenant_id: str
|
||||
) -> CampaignVersion:
|
||||
version = session.get(CampaignVersion, version_id)
|
||||
if not version:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
campaign = session.get(Campaign, version.campaign_id)
|
||||
if not campaign or campaign.tenant_id != tenant_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
return version
|
||||
|
||||
|
||||
def _principal_group_ids(session: Session, principal: ApiPrincipal) -> set[str]:
|
||||
del session
|
||||
return {
|
||||
group.id
|
||||
for group in _access_directory().groups_for_user(
|
||||
principal.user.id, tenant_id=principal.tenant_id
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
def _campaign_acl_filter(session: Session, principal: ApiPrincipal):
|
||||
if has_scope(principal, "tenant:*"):
|
||||
return None
|
||||
group_ids = _principal_group_ids(session, principal)
|
||||
clauses = [Campaign.owner_user_id == principal.user.id]
|
||||
if group_ids:
|
||||
clauses.append(Campaign.owner_group_id.in_(group_ids))
|
||||
share_clauses = [
|
||||
and_(
|
||||
CampaignShare.tenant_id == Campaign.tenant_id,
|
||||
CampaignShare.campaign_id == Campaign.id,
|
||||
CampaignShare.revoked_at.is_(None),
|
||||
CampaignShare.target_type == "user",
|
||||
CampaignShare.target_id == principal.user.id,
|
||||
)
|
||||
]
|
||||
if group_ids:
|
||||
share_clauses.append(
|
||||
and_(
|
||||
CampaignShare.tenant_id == Campaign.tenant_id,
|
||||
CampaignShare.campaign_id == Campaign.id,
|
||||
CampaignShare.revoked_at.is_(None),
|
||||
CampaignShare.target_type == "group",
|
||||
CampaignShare.target_id.in_(group_ids),
|
||||
)
|
||||
)
|
||||
clauses.append(exists().where(or_(*share_clauses)))
|
||||
return or_(*clauses)
|
||||
|
||||
|
||||
def _campaign_acl_allows(
|
||||
session: Session,
|
||||
campaign: Campaign,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
write: bool = False,
|
||||
) -> bool:
|
||||
if has_scope(principal, "tenant:*"):
|
||||
return True
|
||||
if campaign.owner_user_id == principal.user.id:
|
||||
return True
|
||||
group_ids = _principal_group_ids(session, principal)
|
||||
if campaign.owner_group_id and campaign.owner_group_id in group_ids:
|
||||
return True
|
||||
target_ids = [principal.user.id, *group_ids]
|
||||
if not target_ids:
|
||||
return False
|
||||
query = session.query(CampaignShare).filter(
|
||||
CampaignShare.tenant_id == campaign.tenant_id,
|
||||
CampaignShare.campaign_id == campaign.id,
|
||||
CampaignShare.revoked_at.is_(None),
|
||||
or_(
|
||||
CampaignShare.target_type == "user",
|
||||
CampaignShare.target_type == "group",
|
||||
),
|
||||
CampaignShare.target_id.in_(target_ids),
|
||||
)
|
||||
shares = query.all()
|
||||
if not shares:
|
||||
return False
|
||||
if not write:
|
||||
return True
|
||||
return any(item.permission == "write" for item in shares)
|
||||
|
||||
|
||||
def _require_campaign_acl(
|
||||
session: Session,
|
||||
campaign: Campaign,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
write: bool = False,
|
||||
) -> None:
|
||||
if not _campaign_acl_allows(session, campaign, principal, write=write):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="Campaign is not shared with this principal",
|
||||
)
|
||||
|
||||
|
||||
def _get_campaign_for_principal(
|
||||
session: Session, campaign_id: str, principal: ApiPrincipal, *, write: bool = False
|
||||
) -> Campaign:
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
_require_campaign_acl(session, campaign, principal, write=write)
|
||||
return campaign
|
||||
|
||||
|
||||
def _require_permission(principal: ApiPrincipal, scope: str) -> None:
|
||||
if not has_scope(principal, scope):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {scope}"
|
||||
)
|
||||
|
||||
|
||||
def _campaign_query_for_principal(session: Session, principal: ApiPrincipal):
|
||||
query = session.query(Campaign).filter(
|
||||
Campaign.tenant_id == principal.tenant_id, Campaign.status != "deleted"
|
||||
)
|
||||
acl_filter = _campaign_acl_filter(session, principal)
|
||||
if acl_filter is not None:
|
||||
query = query.filter(acl_filter)
|
||||
return query
|
||||
|
||||
|
||||
def _get_recipient_import_profile_for_principal(
|
||||
session: Session, profile_id: str, principal: ApiPrincipal
|
||||
) -> RecipientImportMappingProfile:
|
||||
profile = session.get(RecipientImportMappingProfile, profile_id)
|
||||
if (
|
||||
not profile
|
||||
or profile.tenant_id != principal.tenant_id
|
||||
or profile.owner_user_id != principal.user.id
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Recipient import mapping profile not found",
|
||||
)
|
||||
return profile
|
||||
|
||||
|
||||
def _apply_recipient_import_profile_payload(
|
||||
profile: RecipientImportMappingProfile,
|
||||
payload: RecipientImportMappingProfilePayload,
|
||||
) -> None:
|
||||
profile.name = payload.name.strip()
|
||||
profile.column_count = payload.column_count
|
||||
profile.headers = list(payload.headers)
|
||||
profile.normalized_headers = list(payload.normalized_headers)
|
||||
profile.ordered_header_fingerprint = payload.ordered_header_fingerprint
|
||||
profile.unordered_header_fingerprint = payload.unordered_header_fingerprint
|
||||
profile.delimiter = payload.delimiter
|
||||
profile.header_rows = payload.header_rows
|
||||
profile.quoted = payload.quoted
|
||||
profile.value_separators = payload.value_separators
|
||||
profile.mappings = [mapping.model_dump(mode="json") for mapping in payload.mappings]
|
||||
|
||||
|
||||
def _recipient_sections_changed(
|
||||
current: dict[str, object] | None, proposed: dict[str, object] | None
|
||||
) -> bool:
|
||||
if proposed is None:
|
||||
return False
|
||||
current = current or {}
|
||||
return any(
|
||||
current.get(key) != proposed.get(key) for key in ("recipients", "entries")
|
||||
)
|
||||
|
||||
|
||||
def _campaign_mail_profile_id(raw_json: dict[str, object] | None) -> str | None:
|
||||
return campaign_mail_profile_id(raw_json)
|
||||
|
||||
|
||||
def _require_mail_profile_use_if_needed(
|
||||
principal: ApiPrincipal, raw_json: dict[str, object] | None
|
||||
) -> None:
|
||||
if _campaign_mail_profile_id(raw_json) and not has_scope(
|
||||
principal, "mail:profile:use"
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="Missing scope: mail:profile:use",
|
||||
)
|
||||
|
||||
|
||||
def _campaign_response_context(principal: ApiPrincipal) -> dict[str, bool]:
|
||||
return {"include_diagnostics": has_scope(principal, "campaigns:diagnostic:read")}
|
||||
|
||||
|
||||
def _campaign_version_detail_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
campaign_id: str,
|
||||
mutation: Callable[[], CampaignVersion],
|
||||
*,
|
||||
audit_action: str,
|
||||
details: dict[str, Any] | Callable[[CampaignVersion], dict[str, Any]] | None = None,
|
||||
validation_error_status: int | None = None,
|
||||
) -> CampaignVersionDetailResponse:
|
||||
try:
|
||||
version = mutation()
|
||||
audit_details = (
|
||||
details(version)
|
||||
if callable(details)
|
||||
else dict(details or {"campaign_id": campaign_id})
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action=audit_action,
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details=audit_details,
|
||||
commit=True,
|
||||
)
|
||||
_write_current_version_snapshot_if_available(version)
|
||||
return CampaignVersionDetailResponse.model_validate(
|
||||
version,
|
||||
context=_campaign_response_context(principal),
|
||||
)
|
||||
except LockedCampaignVersionError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||
) from exc
|
||||
except CampaignPathSecurityError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except CampaignPersistenceError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
except RevisionConflictError:
|
||||
session.rollback()
|
||||
raise
|
||||
except Exception as exc:
|
||||
session.rollback()
|
||||
if validation_error_status is None:
|
||||
raise
|
||||
raise HTTPException(
|
||||
status_code=validation_error_status, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
def _update_campaign_version_detail_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignVersionUpdateRequest,
|
||||
*,
|
||||
if_match: str | None,
|
||||
autosave: bool,
|
||||
audit_action: str,
|
||||
) -> CampaignVersionDetailResponse:
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
current_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
if payload.base_revision is None:
|
||||
error = MissingPreconditionError(
|
||||
resource_type="campaign_version",
|
||||
resource_id=version_id,
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
|
||||
detail=error.as_dict(),
|
||||
)
|
||||
try:
|
||||
assert_revision_precondition(
|
||||
if_match,
|
||||
resource_type="campaign_version",
|
||||
resource_id=version_id,
|
||||
submitted_base_revision=payload.base_revision,
|
||||
)
|
||||
except MissingPreconditionError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
|
||||
detail=exc.as_dict(),
|
||||
) from exc
|
||||
except ConcurrencyError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail={
|
||||
"code": "invalid_precondition",
|
||||
"message": str(exc),
|
||||
},
|
||||
) from exc
|
||||
if _recipient_sections_changed(current_version.raw_json, payload.campaign_json):
|
||||
_require_permission(principal, "campaigns:recipient:write")
|
||||
_require_mail_profile_use_if_needed(principal, payload.campaign_json)
|
||||
try:
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: update_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
raw_json=payload.campaign_json,
|
||||
current_flow=payload.current_flow,
|
||||
current_step=payload.current_step,
|
||||
workflow_state=payload.workflow_state,
|
||||
is_complete=payload.is_complete,
|
||||
editor_state=payload.editor_state,
|
||||
source_filename=payload.source_filename,
|
||||
source_base_path=payload.source_base_path,
|
||||
autosave=autosave,
|
||||
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
|
||||
expected_revision=payload.base_revision,
|
||||
commit=False,
|
||||
),
|
||||
audit_action=audit_action,
|
||||
details=lambda version: {
|
||||
"campaign_id": campaign_id,
|
||||
"current_flow": version.current_flow,
|
||||
"current_step": version.current_step,
|
||||
"base_revision": payload.base_revision,
|
||||
"result_revision": version.edit_revision,
|
||||
"reconciliation_kind": payload.reconciliation_kind,
|
||||
"resolved_conflict_path_count": len(payload.resolved_conflict_paths),
|
||||
"resolved_conflict_sections": sorted(
|
||||
{
|
||||
path.strip("/").split("/", 1)[0][:80]
|
||||
for path in payload.resolved_conflict_paths[:100]
|
||||
if path.strip("/")
|
||||
}
|
||||
),
|
||||
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
|
||||
},
|
||||
validation_error_status=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
)
|
||||
except RevisionConflictError as exc:
|
||||
session.rollback()
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.version_conflict_detected",
|
||||
object_type="campaign_version",
|
||||
object_id=version_id,
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"submitted_base_revision": exc.submitted_base_revision,
|
||||
"current_revision": exc.current_revision,
|
||||
"retryable": True,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_412_PRECONDITION_FAILED,
|
||||
detail=exc.as_dict(),
|
||||
) from exc
|
||||
|
||||
|
||||
def _require_campaign_profile_use_if_needed(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
campaign_id: str,
|
||||
version_id: str | None = None,
|
||||
) -> None:
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
target_version_id = version_id or campaign.current_version_id
|
||||
if not target_version_id:
|
||||
return
|
||||
version = _get_version_for_tenant(session, target_version_id, principal.tenant_id)
|
||||
if version.campaign_id != campaign.id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
|
||||
|
||||
def _require_campaign_versions_profile_use(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
campaign_id: str,
|
||||
version_ids: set[str],
|
||||
) -> None:
|
||||
"""Authorize every historical version affected by a campaign-wide action."""
|
||||
|
||||
for version_id in sorted(version_ids):
|
||||
_require_campaign_profile_use_if_needed(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
version_id,
|
||||
)
|
||||
|
||||
|
||||
def _get_version_for_principal(
|
||||
session: Session,
|
||||
version_id: str,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
write: bool = False,
|
||||
) -> CampaignVersion:
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
campaign = _get_campaign_for_tenant(
|
||||
session, version.campaign_id, principal.tenant_id
|
||||
)
|
||||
_require_campaign_acl(session, campaign, principal, write=write)
|
||||
return version
|
||||
|
||||
|
||||
def _sync_campaign_metadata_to_current_version(
|
||||
session: Session, campaign: Campaign
|
||||
) -> None:
|
||||
"""Keep editable version JSON aligned with version-independent campaign metadata.
|
||||
|
||||
Campaign metadata can be edited from the overview while individual campaign
|
||||
sections save the current version JSON later. Without this sync, a later
|
||||
version save can re-apply stale `campaign.name` / `campaign.id` values from
|
||||
raw_json and make the old overview metadata appear to come back. Audit-safe
|
||||
or validation-locked versions are left untouched.
|
||||
"""
|
||||
|
||||
if not campaign.current_version_id:
|
||||
return
|
||||
|
||||
version = session.get(CampaignVersion, campaign.current_version_id)
|
||||
if not version or version.campaign_id != campaign.id or is_version_locked(version):
|
||||
return
|
||||
|
||||
raw_json = copy.deepcopy(
|
||||
version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
campaign_section = (
|
||||
raw_json.get("campaign") if isinstance(raw_json.get("campaign"), dict) else {}
|
||||
)
|
||||
raw_json["campaign"] = {
|
||||
**campaign_section,
|
||||
"id": campaign.external_id,
|
||||
"name": campaign.name,
|
||||
"description": campaign.description or "",
|
||||
}
|
||||
version.raw_json = raw_json
|
||||
session.add(version)
|
||||
|
||||
|
||||
def _clear_current_version_mail_profile_for_owner_transfer(
|
||||
session: Session, campaign: Campaign
|
||||
) -> bool:
|
||||
"""Force explicit profile reselection after campaign ownership changes.
|
||||
|
||||
User/group-scoped reusable mail profiles are evaluated against the current
|
||||
owner. Instead of trying to keep a stale selection across an ownership
|
||||
transfer, clear the profile from the editable current version and invalidate
|
||||
validation/build state so the operator has to reselect and revalidate.
|
||||
"""
|
||||
|
||||
if not campaign.current_version_id:
|
||||
return False
|
||||
|
||||
version = session.get(CampaignVersion, campaign.current_version_id)
|
||||
if not version or version.campaign_id != campaign.id:
|
||||
return False
|
||||
|
||||
raw_json = copy.deepcopy(
|
||||
version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
server = (
|
||||
raw_json.get("server") if isinstance(raw_json.get("server"), dict) else None
|
||||
)
|
||||
if not isinstance(server, dict):
|
||||
return False
|
||||
|
||||
profile_id = _campaign_mail_profile_id(raw_json)
|
||||
if not profile_id:
|
||||
return False
|
||||
|
||||
if is_version_final_locked(version) or is_user_locked_version(version):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Change owner only after creating an editable campaign version; the current version has a selected mail profile and is locked.",
|
||||
)
|
||||
|
||||
next_server = dict(server)
|
||||
for key in CAMPAIGN_MAIL_SERVER_KEYS:
|
||||
next_server.pop(key, None)
|
||||
next_server.pop("profile_id", None)
|
||||
raw_json["server"] = next_server
|
||||
|
||||
version.raw_json = raw_json
|
||||
version.validation_summary = None
|
||||
version.build_summary = None
|
||||
clear_execution_snapshot(version)
|
||||
version.locked_at = None
|
||||
version.locked_by_user_id = None
|
||||
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
|
||||
version.is_complete = False
|
||||
|
||||
editor_state = copy.deepcopy(version.editor_state or {})
|
||||
editor_state.pop("review_send", None)
|
||||
editor_state.pop("approval_gate", None)
|
||||
version.editor_state = editor_state
|
||||
|
||||
session.query(CampaignIssue).filter(
|
||||
CampaignIssue.campaign_version_id == version.id
|
||||
).delete(synchronize_session=False)
|
||||
session.query(CampaignJob).filter(
|
||||
CampaignJob.campaign_version_id == version.id
|
||||
).delete(synchronize_session=False)
|
||||
campaign.status = CampaignStatus.DRAFT.value
|
||||
session.add(version)
|
||||
_write_current_version_snapshot_if_available(version)
|
||||
return True
|
||||
|
||||
|
||||
def _write_current_version_snapshot_if_available(version: CampaignVersion) -> None:
|
||||
# Kept as a compatibility no-op for callers outside this package. Campaign
|
||||
# JSON is database-authoritative and no longer mirrored onto an API node.
|
||||
del version
|
||||
|
||||
|
||||
def bounded_query_rows(query, *, limit: int, label: str):
|
||||
rows = query.limit(limit + 1).all()
|
||||
if len(rows) > limit:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
|
||||
detail=(
|
||||
f"{label} exceeds the maximum response size of {limit} rows. "
|
||||
"Narrow the request or use a paginated/delta endpoint."
|
||||
),
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def job_attempt_rows(query, *, label: str):
|
||||
return bounded_query_rows(query, limit=1000, label=label)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1 @@
|
||||
"""Focused HTTP route modules for the campaign API."""
|
||||
@@ -0,0 +1,398 @@
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_campaign.backend.persistence.campaigns import (
|
||||
load_campaign_config_from_json,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
files_integration,
|
||||
)
|
||||
from govoplan_campaign.backend.path_security import (
|
||||
CampaignPathSecurityError,
|
||||
assert_server_safe_campaign_paths,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.loader import load_campaign_json
|
||||
from govoplan_campaign.backend.attachments.resolver import resolve_campaign_attachments
|
||||
from govoplan_campaign.backend.persistence.versions import (
|
||||
is_version_final_locked,
|
||||
is_user_locked_version,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_get_campaign_for_principal,
|
||||
_get_campaign_for_tenant,
|
||||
_get_version_for_tenant,
|
||||
_require_mail_profile_use_if_needed,
|
||||
_require_permission,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
|
||||
|
||||
class CampaignAttachmentPreviewRequest(BaseModel):
|
||||
include_unmatched: bool = True
|
||||
include_unlinked_candidates: bool = False
|
||||
campaign_json: dict[str, object] | None = None
|
||||
|
||||
|
||||
class CampaignAttachmentPreviewResponse(BaseModel):
|
||||
campaign_id: str
|
||||
version_id: str
|
||||
shared_file_count: int
|
||||
candidate_file_count: int = 0
|
||||
matched_file_count: int = 0
|
||||
linked_file_count: int = 0
|
||||
unlinked_file_count: int = 0
|
||||
rules: list[dict[str, object]] = Field(default_factory=list)
|
||||
linkable_files: list[dict[str, object]] = Field(default_factory=list)
|
||||
unused_shared_files: list[dict[str, object]] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CampaignAttachmentLinkMatchesRequest(BaseModel):
|
||||
campaign_json: dict[str, object] | None = None
|
||||
dry_run: bool = False
|
||||
|
||||
|
||||
class CampaignAttachmentLinkMatchesResponse(BaseModel):
|
||||
campaign_id: str
|
||||
version_id: str
|
||||
matched_file_count: int
|
||||
already_linked_file_count: int
|
||||
linked_file_count: int
|
||||
dry_run: bool = False
|
||||
linked_files: list[dict[str, object]] = Field(default_factory=list)
|
||||
linkable_files: list[dict[str, object]] = Field(default_factory=list)
|
||||
|
||||
|
||||
def _file_preview(session: Session, asset) -> dict[str, object]:
|
||||
version, blob = files_integration().current_version_and_blob(session, asset)
|
||||
return {
|
||||
"id": asset.id,
|
||||
"version_id": version.id,
|
||||
"blob_id": blob.id,
|
||||
"display_path": asset.display_path,
|
||||
"filename": asset.filename,
|
||||
"owner_type": asset.owner_type,
|
||||
"owner_id": asset.owner_user_id
|
||||
if asset.owner_type == "user"
|
||||
else asset.owner_group_id,
|
||||
"checksum_sha256": blob.checksum_sha256,
|
||||
"size_bytes": blob.size_bytes,
|
||||
"content_type": blob.content_type,
|
||||
"linked_to_campaign": True,
|
||||
}
|
||||
|
||||
|
||||
def _managed_preview_file(item: dict[str, object]) -> dict[str, object]:
|
||||
return {
|
||||
"id": item["asset_id"],
|
||||
"version_id": item["version_id"],
|
||||
"blob_id": item["blob_id"],
|
||||
"display_path": item["display_path"],
|
||||
"filename": item["filename"],
|
||||
"owner_type": item["owner_type"],
|
||||
"owner_id": item["owner_id"],
|
||||
"checksum_sha256": item["checksum_sha256"],
|
||||
"size_bytes": item["size_bytes"],
|
||||
"content_type": item["content_type"],
|
||||
"linked_to_campaign": bool(item.get("linked_to_campaign", True)),
|
||||
}
|
||||
|
||||
|
||||
def _attachment_preview_for_version(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
campaign: Campaign,
|
||||
version: CampaignVersion,
|
||||
raw: dict[str, object],
|
||||
include_unmatched: bool,
|
||||
include_unlinked_candidates: bool,
|
||||
) -> CampaignAttachmentPreviewResponse:
|
||||
files = files_integration()
|
||||
assert_server_safe_campaign_paths(raw, managed_files_available=files.available)
|
||||
with files.prepared_campaign_snapshot(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign.id,
|
||||
raw_json=raw,
|
||||
include_bytes=False,
|
||||
prefix="govoplan-managed-preview-",
|
||||
include_unlinked_candidates=include_unlinked_candidates,
|
||||
user_id=principal.user.id,
|
||||
is_admin=has_scope(principal, "files:file:admin"),
|
||||
) as prepared:
|
||||
prepared_raw = load_campaign_json(prepared.path)
|
||||
config = load_campaign_config_from_json(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
raw_json=prepared_raw,
|
||||
campaign_id=campaign.id,
|
||||
)
|
||||
report = resolve_campaign_attachments(config, campaign_file=prepared.path)
|
||||
rules: list[dict[str, object]] = []
|
||||
matched_asset_ids: set[str] = set()
|
||||
linked_asset_ids: set[str] = set()
|
||||
linkable_by_id: dict[str, dict[str, object]] = {}
|
||||
|
||||
for entry in report.entries:
|
||||
for attachment in entry.attachments:
|
||||
managed_matches = files.managed_match_payloads(
|
||||
attachment.matches, prepared.managed_files_by_local_path
|
||||
)
|
||||
matches: list[dict[str, object]] = []
|
||||
for item in managed_matches:
|
||||
asset_id = str(item["asset_id"])
|
||||
matched_asset_ids.add(asset_id)
|
||||
if bool(item.get("linked_to_campaign", True)):
|
||||
linked_asset_ids.add(asset_id)
|
||||
preview = _managed_preview_file(item)
|
||||
matches.append(preview)
|
||||
if not preview["linked_to_campaign"]:
|
||||
linkable_by_id.setdefault(asset_id, preview)
|
||||
if not matches:
|
||||
matches = [
|
||||
{
|
||||
"id": "",
|
||||
"display_path": match,
|
||||
"filename": match.rsplit("/", 1)[-1].rsplit("\\", 1)[-1],
|
||||
"owner_type": "legacy",
|
||||
"owner_id": "",
|
||||
"linked_to_campaign": True,
|
||||
}
|
||||
for match in attachment.matches
|
||||
]
|
||||
rules.append(
|
||||
{
|
||||
"source": attachment.scope.value,
|
||||
"entry_index": entry.entry_index,
|
||||
"entry_id": entry.entry_id,
|
||||
"index": attachment.index,
|
||||
"attachment_id": attachment.attachment_id,
|
||||
"label": attachment.label,
|
||||
"required": attachment.required,
|
||||
"pattern": attachment.file_filter,
|
||||
"base_path_name": attachment.base_path_name,
|
||||
"base_path": attachment.base_path,
|
||||
"status": attachment.status.value,
|
||||
"behavior": attachment.behavior.value
|
||||
if attachment.behavior
|
||||
else None,
|
||||
"zip_included": attachment.zip_enabled,
|
||||
"zip_mode": attachment.zip_mode.value,
|
||||
"zip_archive_id": attachment.zip_archive_id,
|
||||
"zip_filename": attachment.zip_filename,
|
||||
"matches": matches,
|
||||
"match_count": len(matches),
|
||||
"linked_match_count": sum(
|
||||
1
|
||||
for match in matches
|
||||
if bool(match.get("linked_to_campaign", True))
|
||||
),
|
||||
"unlinked_match_count": sum(
|
||||
1
|
||||
for match in matches
|
||||
if not bool(match.get("linked_to_campaign", True))
|
||||
),
|
||||
"issues": [
|
||||
issue.model_dump(mode="json") for issue in attachment.issues
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
unused = [
|
||||
asset
|
||||
for asset in prepared.shared_assets
|
||||
if asset.id not in matched_asset_ids
|
||||
]
|
||||
return CampaignAttachmentPreviewResponse(
|
||||
campaign_id=campaign.id,
|
||||
version_id=version.id,
|
||||
shared_file_count=len(prepared.shared_assets),
|
||||
candidate_file_count=len(
|
||||
getattr(prepared, "candidate_assets", prepared.shared_assets)
|
||||
),
|
||||
matched_file_count=len(matched_asset_ids),
|
||||
linked_file_count=len(linked_asset_ids),
|
||||
unlinked_file_count=len(linkable_by_id),
|
||||
rules=rules,
|
||||
linkable_files=list(linkable_by_id.values()),
|
||||
unused_shared_files=[_file_preview(session, asset) for asset in unused]
|
||||
if include_unmatched
|
||||
else [],
|
||||
)
|
||||
|
||||
|
||||
def _link_campaign_attachment_matches(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
campaign: Campaign,
|
||||
version: CampaignVersion,
|
||||
raw: dict[str, object],
|
||||
dry_run: bool = False,
|
||||
) -> CampaignAttachmentLinkMatchesResponse:
|
||||
preview = _attachment_preview_for_version(
|
||||
session,
|
||||
principal,
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
raw=raw,
|
||||
include_unmatched=False,
|
||||
include_unlinked_candidates=True,
|
||||
)
|
||||
file_ids = [
|
||||
str(item.get("id") or "") for item in preview.linkable_files if item.get("id")
|
||||
]
|
||||
linked_files: list[dict[str, object]] = []
|
||||
if file_ids and not dry_run:
|
||||
files = files_integration()
|
||||
shares = files.share_assets_with_campaign(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign.id,
|
||||
file_ids=file_ids,
|
||||
user_id=principal.user.id,
|
||||
is_admin=has_scope(principal, "files:file:admin"),
|
||||
)
|
||||
share_by_asset_id = {
|
||||
str(item.get("file_asset_id") or ""): item for item in shares
|
||||
}
|
||||
linked_files = [
|
||||
{**item, "share": share_by_asset_id.get(str(item.get("id") or ""))}
|
||||
for item in preview.linkable_files
|
||||
]
|
||||
return CampaignAttachmentLinkMatchesResponse(
|
||||
campaign_id=campaign.id,
|
||||
version_id=version.id,
|
||||
matched_file_count=preview.matched_file_count,
|
||||
already_linked_file_count=preview.linked_file_count,
|
||||
linked_file_count=0 if dry_run else len(file_ids),
|
||||
dry_run=dry_run,
|
||||
linked_files=linked_files,
|
||||
linkable_files=preview.linkable_files,
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/attachments/preview",
|
||||
response_model=CampaignAttachmentPreviewResponse,
|
||||
)
|
||||
def preview_campaign_attachments(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignAttachmentPreviewRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
if version.campaign_id != campaign.id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
|
||||
payload = payload or CampaignAttachmentPreviewRequest()
|
||||
raw = (
|
||||
payload.campaign_json
|
||||
if isinstance(payload.campaign_json, dict)
|
||||
else version.raw_json
|
||||
)
|
||||
raw = raw if isinstance(raw, dict) else {}
|
||||
_require_mail_profile_use_if_needed(principal, raw)
|
||||
try:
|
||||
return _attachment_preview_for_version(
|
||||
session,
|
||||
principal,
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
raw=raw,
|
||||
include_unmatched=payload.include_unmatched,
|
||||
include_unlinked_candidates=payload.include_unlinked_candidates,
|
||||
)
|
||||
except CampaignPathSecurityError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/attachments/link-matches",
|
||||
response_model=CampaignAttachmentLinkMatchesResponse,
|
||||
)
|
||||
def link_campaign_attachment_matches(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignAttachmentLinkMatchesRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||
):
|
||||
_require_permission(principal, "files:file:share")
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
if version.campaign_id != campaign.id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
if is_user_locked_version(version) or is_version_final_locked(version):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="Locked campaign versions cannot link new attachment files",
|
||||
)
|
||||
payload = payload or CampaignAttachmentLinkMatchesRequest()
|
||||
raw = (
|
||||
payload.campaign_json
|
||||
if isinstance(payload.campaign_json, dict)
|
||||
else version.raw_json
|
||||
)
|
||||
raw = raw if isinstance(raw, dict) else {}
|
||||
_require_mail_profile_use_if_needed(principal, raw)
|
||||
try:
|
||||
result = _link_campaign_attachment_matches(
|
||||
session,
|
||||
principal,
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
raw=raw,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.attachment_matches_linked"
|
||||
if not payload.dry_run
|
||||
else "campaign.attachment_matches_link_previewed",
|
||||
object_type="campaign_version",
|
||||
object_id=version_id,
|
||||
details={
|
||||
"matched_file_count": result.matched_file_count,
|
||||
"already_linked_file_count": result.already_linked_file_count,
|
||||
"linked_file_count": result.linked_file_count,
|
||||
"dry_run": result.dry_run,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return result
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,749 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
AppendSentRequest,
|
||||
CampaignActionResponse,
|
||||
CampaignRetryJobsRequest,
|
||||
CampaignSendJobRequest,
|
||||
CampaignSendUnattemptedRequest,
|
||||
CampaignResolveOutcomeRequest,
|
||||
CampaignDeliveryOptionsResponse,
|
||||
MockCampaignSendRequest,
|
||||
MockCampaignSendResponse,
|
||||
QueueCampaignRequest,
|
||||
QueueCampaignResponse,
|
||||
SendCampaignNowRequest,
|
||||
SendCampaignNowResponse,
|
||||
)
|
||||
from govoplan_campaign.backend.approval_gate import (
|
||||
CampaignApprovalGateError,
|
||||
campaign_approval_status,
|
||||
request_campaign_approval,
|
||||
)
|
||||
from govoplan_campaign.backend.approval_schemas import CampaignApprovalRequestInput
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignJob,
|
||||
JobImapStatus,
|
||||
JobQueueStatus,
|
||||
JobSendStatus,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
postbox_integration,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_campaign.backend.response_security import (
|
||||
public_send_campaign_now_result,
|
||||
send_campaign_now_audit_details,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.campaigns import (
|
||||
CampaignPersistenceError,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.versions import (
|
||||
is_user_locked_version,
|
||||
)
|
||||
|
||||
from govoplan_campaign.backend.dev.mock_campaign import (
|
||||
MockCampaignSendError,
|
||||
run_mock_campaign_send,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError
|
||||
from govoplan_campaign.backend.sending.jobs import (
|
||||
QueueingError,
|
||||
SynchronousSendRejected,
|
||||
cancel_campaign_jobs,
|
||||
enqueue_pending_imap_appends,
|
||||
pause_campaign_jobs,
|
||||
queue_campaign_jobs,
|
||||
queue_failed_jobs_for_retry,
|
||||
queue_unattempted_jobs,
|
||||
reconcile_job_outcome,
|
||||
resume_campaign_jobs,
|
||||
send_campaign_now,
|
||||
send_single_campaign_job,
|
||||
synchronous_send_options,
|
||||
)
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_get_campaign_for_principal,
|
||||
_get_campaign_for_tenant,
|
||||
_get_version_for_tenant,
|
||||
_require_campaign_profile_use_if_needed,
|
||||
_require_campaign_versions_profile_use,
|
||||
_require_mail_profile_use_if_needed,
|
||||
_require_permission,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{campaign_id}/delivery-options", response_model=CampaignDeliveryOptionsResponse
|
||||
)
|
||||
def campaign_delivery_options(
|
||||
campaign_id: str,
|
||||
version_id: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope("campaigns:campaign:send", "campaigns:campaign:queue")
|
||||
),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
try:
|
||||
options = synchronous_send_options(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
)
|
||||
version = _get_version_for_tenant(
|
||||
session, str(options["version_id"]), principal.tenant_id
|
||||
)
|
||||
return CampaignDeliveryOptionsResponse(
|
||||
**options,
|
||||
postbox_available=postbox_integration().available,
|
||||
approval_gate=campaign_approval_status(
|
||||
session, tenant_id=principal.tenant_id, version=version
|
||||
),
|
||||
)
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/approval-request",
|
||||
response_model=dict[str, object],
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_campaign_approval_request(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignApprovalRequestInput,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:review")),
|
||||
) -> dict[str, object]:
|
||||
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
if campaign.current_version_id != version_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="Approval can only be requested for the current Campaign version.",
|
||||
)
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
if not version.build_summary:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Build Campaign messages before requesting approval.",
|
||||
)
|
||||
try:
|
||||
request = request_campaign_approval(
|
||||
session,
|
||||
principal,
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
title=payload.title,
|
||||
description=payload.description,
|
||||
steps=tuple(step.to_definition() for step in payload.steps),
|
||||
idempotency_key=payload.idempotency_key,
|
||||
template_id=payload.template_id,
|
||||
template_revision=payload.template_revision,
|
||||
unique_actors_across_steps=payload.unique_actors_across_steps,
|
||||
expires_at=payload.expires_at,
|
||||
policy_refs=tuple(payload.policy_refs),
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.approval_requested",
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details={
|
||||
"campaign_id": campaign.id,
|
||||
"approval_request_id": request.id,
|
||||
"approval_request_revision": request.revision,
|
||||
"execution_snapshot_hash": version.execution_snapshot_hash,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return {
|
||||
"request_id": request.id,
|
||||
"request_revision": request.revision,
|
||||
"request_state": request.state,
|
||||
"approval_gate": campaign_approval_status(
|
||||
session, tenant_id=principal.tenant_id, version=version
|
||||
),
|
||||
}
|
||||
except (CampaignApprovalGateError, ExecutionSnapshotError) as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/queue", response_model=QueueCampaignResponse)
|
||||
def queue_campaign(
|
||||
campaign_id: str,
|
||||
payload: QueueCampaignRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
payload = payload or QueueCampaignRequest()
|
||||
_require_campaign_profile_use_if_needed(
|
||||
session, principal, campaign_id, payload.version_id
|
||||
)
|
||||
try:
|
||||
result = queue_campaign_jobs(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=payload.version_id,
|
||||
include_warnings=payload.include_warnings,
|
||||
enqueue_celery=payload.enqueue_celery,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.queued"
|
||||
if not payload.dry_run
|
||||
else "campaign.queue_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result.as_dict(),
|
||||
commit=True,
|
||||
)
|
||||
return QueueCampaignResponse(**result.as_dict())
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/jobs/retry", response_model=CampaignActionResponse)
|
||||
def retry_campaign_jobs(
|
||||
campaign_id: str,
|
||||
payload: CampaignRetryJobsRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:retry")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
payload = payload or CampaignRetryJobsRequest()
|
||||
_require_campaign_profile_use_if_needed(
|
||||
session, principal, campaign_id, payload.version_id
|
||||
)
|
||||
try:
|
||||
result = queue_failed_jobs_for_retry(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=payload.version_id,
|
||||
job_ids=payload.job_ids or None,
|
||||
include_permanent=payload.include_permanent,
|
||||
force_max_attempts=payload.force_max_attempts,
|
||||
enqueue_celery=payload.enqueue_celery,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.jobs_retry_queued"
|
||||
if not payload.dry_run
|
||||
else "campaign.jobs_retry_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/jobs/send-unattempted", response_model=CampaignActionResponse
|
||||
)
|
||||
def send_unattempted_campaign_jobs(
|
||||
campaign_id: str,
|
||||
payload: CampaignSendUnattemptedRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:queue")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
payload = payload or CampaignSendUnattemptedRequest()
|
||||
_require_campaign_profile_use_if_needed(
|
||||
session, principal, campaign_id, payload.version_id
|
||||
)
|
||||
try:
|
||||
result = queue_unattempted_jobs(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=payload.version_id,
|
||||
job_ids=payload.job_ids or None,
|
||||
enqueue_celery=payload.enqueue_celery,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.unattempted_jobs_queued"
|
||||
if not payload.dry_run
|
||||
else "campaign.unattempted_jobs_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/jobs/{job_id}/send", response_model=CampaignActionResponse)
|
||||
def send_single_campaign_job_endpoint(
|
||||
campaign_id: str,
|
||||
job_id: str,
|
||||
payload: CampaignSendJobRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"campaigns:campaign:send",
|
||||
"campaigns:campaign:send_test",
|
||||
)
|
||||
),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
_require_permission(
|
||||
principal,
|
||||
(
|
||||
"campaigns:campaign:send_test"
|
||||
if payload.kind == "test"
|
||||
else "campaigns:campaign:send"
|
||||
),
|
||||
)
|
||||
_require_campaign_profile_use_if_needed(session, principal, campaign_id, None)
|
||||
try:
|
||||
result = send_single_campaign_job(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
job_id=job_id,
|
||||
kind=payload.kind,
|
||||
idempotency_key=payload.idempotency_key,
|
||||
actor_user_id=principal.user.id,
|
||||
actor_api_key_id=getattr(principal, "api_key_id", None),
|
||||
reason=payload.reason,
|
||||
action_context=payload.context,
|
||||
include_warnings=payload.include_warnings,
|
||||
use_rate_limit=payload.use_rate_limit,
|
||||
enqueue_imap_task=payload.enqueue_imap_task,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action=f"campaign.message_{payload.kind}",
|
||||
object_type="campaign_job",
|
||||
object_id=job_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
logger.exception(
|
||||
"Unexpected single-message campaign action failure",
|
||||
extra={
|
||||
"campaign_id": campaign_id,
|
||||
"job_id": job_id,
|
||||
"action_kind": payload.kind,
|
||||
},
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="The message action failed because of an internal error.",
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/jobs/{job_id}/resolve-outcome",
|
||||
response_model=CampaignActionResponse,
|
||||
)
|
||||
def resolve_campaign_job_outcome(
|
||||
campaign_id: str,
|
||||
job_id: str,
|
||||
payload: CampaignResolveOutcomeRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:reconcile")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
try:
|
||||
result = reconcile_job_outcome(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
job_id=job_id,
|
||||
decision=payload.decision,
|
||||
note=payload.note,
|
||||
attempt_id=payload.attempt_id,
|
||||
commit=False,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.job_outcome_reconciled",
|
||||
object_type="campaign_job",
|
||||
object_id=job_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except (QueueingError, ExecutionSnapshotError) as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/mock-send", response_model=MockCampaignSendResponse)
|
||||
def mock_send_campaign(
|
||||
campaign_id: str,
|
||||
payload: MockCampaignSendRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send_test")),
|
||||
):
|
||||
"""Run a fully visible mock delivery flow without mutating campaign state.
|
||||
|
||||
The route validates and builds the selected version, then optionally records
|
||||
mock SMTP deliveries and mock IMAP appends. It never talks to the configured
|
||||
real SMTP/IMAP servers and it does not mark the version sent/final.
|
||||
"""
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
|
||||
payload = payload or MockCampaignSendRequest()
|
||||
_require_campaign_profile_use_if_needed(
|
||||
session, principal, campaign_id, payload.version_id
|
||||
)
|
||||
try:
|
||||
result = run_mock_campaign_send(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=payload.version_id,
|
||||
send=payload.send,
|
||||
include_warnings=payload.include_warnings,
|
||||
include_needs_review=payload.include_needs_review,
|
||||
append_sent=payload.append_sent,
|
||||
clear_mailbox=payload.clear_mailbox,
|
||||
check_files=payload.check_files,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.mock_send"
|
||||
if payload.send
|
||||
else "campaign.mock_send_review",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details={
|
||||
"version_id": result.get("version_id"),
|
||||
"send_requested": payload.send,
|
||||
"sent_count": result.get("send", {}).get("sent_count"),
|
||||
"failed_count": result.get("send", {}).get("failed_count"),
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return MockCampaignSendResponse(result=result)
|
||||
except MockCampaignSendError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/send-now", response_model=SendCampaignNowResponse)
|
||||
def send_campaign_now_endpoint(
|
||||
campaign_id: str,
|
||||
payload: SendCampaignNowRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
|
||||
):
|
||||
"""Preflight and synchronously send a policy-bounded built execution."""
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
|
||||
payload = payload or SendCampaignNowRequest()
|
||||
try:
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
version_id = payload.version_id or campaign.current_version_id
|
||||
if not version_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Campaign has no current version",
|
||||
)
|
||||
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
validation_result: dict[str, object] | None = (
|
||||
version.validation_summary
|
||||
if isinstance(version.validation_summary, dict)
|
||||
else None
|
||||
)
|
||||
build_result: dict[str, object] | None = (
|
||||
version.build_summary if isinstance(version.build_summary, dict) else None
|
||||
)
|
||||
if is_user_locked_version(version):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="User-locked audit-safe versions cannot be dry-run or sent. Create an editable copy and validate it instead.",
|
||||
)
|
||||
if (
|
||||
not version.locked_at
|
||||
or not validation_result
|
||||
or validation_result.get("ok") is not True
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Campaign version must be validated and locked before dry-run or sending.",
|
||||
)
|
||||
if not build_result:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Campaign version must be built before dry-run or sending.",
|
||||
)
|
||||
|
||||
delivery_result = send_campaign_now(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
include_warnings=payload.include_warnings,
|
||||
dry_run=payload.dry_run,
|
||||
use_rate_limit=payload.use_rate_limit,
|
||||
enqueue_imap_task=payload.enqueue_imap_task,
|
||||
).as_dict()
|
||||
response_result = public_send_campaign_now_result(
|
||||
delivery_result,
|
||||
validation_summary=validation_result,
|
||||
build_summary=build_result,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.sent_now"
|
||||
if not payload.dry_run
|
||||
else "campaign.send_now_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=send_campaign_now_audit_details(delivery_result),
|
||||
commit=True,
|
||||
)
|
||||
return SendCampaignNowResponse(result=response_result)
|
||||
except SynchronousSendRejected as exc:
|
||||
# A synchronous request stages queue state before the all-message
|
||||
# preflight can run. Rejecting that preflight must not leave work
|
||||
# eligible for a background worker when no provider effect occurred.
|
||||
session.rollback()
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.send_now_rejected",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details={
|
||||
**exc.audit_details(),
|
||||
"version_id": payload.version_id,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except HTTPException:
|
||||
raise
|
||||
except (CampaignPersistenceError, QueueingError) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/pause", response_model=CampaignActionResponse)
|
||||
def pause_campaign(
|
||||
campaign_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
try:
|
||||
result = pause_campaign_jobs(
|
||||
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.paused",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/resume", response_model=CampaignActionResponse)
|
||||
def resume_campaign(
|
||||
campaign_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
version_ids = {
|
||||
row[0]
|
||||
for row in session.query(CampaignJob.campaign_version_id)
|
||||
.filter(
|
||||
CampaignJob.tenant_id == principal.tenant_id,
|
||||
CampaignJob.campaign_id == campaign_id,
|
||||
CampaignJob.queue_status == JobQueueStatus.PAUSED.value,
|
||||
)
|
||||
.distinct()
|
||||
.all()
|
||||
}
|
||||
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
|
||||
try:
|
||||
result = resume_campaign_jobs(
|
||||
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.resumed",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/cancel", response_model=CampaignActionResponse)
|
||||
def cancel_campaign(
|
||||
campaign_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:control")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
try:
|
||||
result = cancel_campaign_jobs(
|
||||
session, tenant_id=principal.tenant_id, campaign_id=campaign_id
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.cancelled",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/append-sent", response_model=CampaignActionResponse)
|
||||
def append_sent(
|
||||
campaign_id: str,
|
||||
payload: AppendSentRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:send")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
payload = payload or AppendSentRequest()
|
||||
version_ids = {
|
||||
row[0]
|
||||
for row in session.query(CampaignJob.campaign_version_id)
|
||||
.filter(
|
||||
CampaignJob.tenant_id == principal.tenant_id,
|
||||
CampaignJob.campaign_id == campaign_id,
|
||||
CampaignJob.send_status.in_(
|
||||
[JobSendStatus.SMTP_ACCEPTED.value, JobSendStatus.SENT.value]
|
||||
),
|
||||
CampaignJob.imap_status.in_(
|
||||
[JobImapStatus.PENDING.value, JobImapStatus.FAILED.value]
|
||||
),
|
||||
)
|
||||
.distinct()
|
||||
.all()
|
||||
}
|
||||
_require_campaign_versions_profile_use(session, principal, campaign_id, version_ids)
|
||||
try:
|
||||
result = enqueue_pending_imap_appends(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
enqueue_celery=payload.enqueue_celery,
|
||||
run_inline=payload.run_inline,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.append_sent_enqueued"
|
||||
if not payload.dry_run
|
||||
else "campaign.append_sent_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result,
|
||||
commit=True,
|
||||
)
|
||||
return CampaignActionResponse(result=result)
|
||||
except QueueingError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
@@ -0,0 +1,530 @@
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
CampaignJobsResponse,
|
||||
CampaignJobsDeltaResponse,
|
||||
CampaignJobDetailResponse,
|
||||
CampaignJobDiagnosticsResponse,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_core.core.change_sequence import (
|
||||
decode_sequence_watermark,
|
||||
encode_sequence_watermark,
|
||||
sequence_entries_since,
|
||||
sequence_watermark_is_expired,
|
||||
)
|
||||
from govoplan_campaign.backend.change_tracking import (
|
||||
CAMPAIGNS_MODULE_ID,
|
||||
CAMPAIGN_JOBS_COLLECTION,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignJob,
|
||||
CampaignMessageAction,
|
||||
CampaignMessageActionAttempt,
|
||||
ImapAppendAttempt,
|
||||
PostboxDeliveryAttempt,
|
||||
PrintOutputAttempt,
|
||||
SendAttempt,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import postbox_integration
|
||||
from govoplan_core.db.session import get_session
|
||||
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_get_campaign_for_principal,
|
||||
_get_campaign_for_tenant,
|
||||
_require_permission,
|
||||
job_attempt_rows as _job_attempt_rows,
|
||||
)
|
||||
from govoplan_campaign.backend.services.job_queries import (
|
||||
CampaignJobsQuery,
|
||||
_campaign_jobs_delta_watermark,
|
||||
_campaign_jobs_page_response,
|
||||
_campaign_jobs_query_context,
|
||||
_job_attempts_payload,
|
||||
_calendar_invitations_for_jobs,
|
||||
_job_detail_payload,
|
||||
_job_diagnostics_payload,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
|
||||
|
||||
def _postbox_receipts_for_attempts(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
attempts: list[PostboxDeliveryAttempt],
|
||||
):
|
||||
integration = postbox_integration()
|
||||
if not integration.receipt_evidence_available:
|
||||
return None
|
||||
delivery_ids = [
|
||||
attempt.provider_delivery_id
|
||||
for attempt in attempts
|
||||
if attempt.provider_delivery_id
|
||||
]
|
||||
return integration.delivery_receipt_summaries(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
delivery_ids=delivery_ids,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/jobs", response_model=CampaignJobsResponse)
|
||||
def list_jobs(
|
||||
campaign_id: str,
|
||||
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
"""Return a lightweight, paginated job list with server-side filters.
|
||||
|
||||
Complete recipients, attachment metadata, issues and attempt history are
|
||||
available from the separate job-detail endpoint.
|
||||
"""
|
||||
|
||||
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||
_campaign_jobs_query_context(
|
||||
session,
|
||||
principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
)
|
||||
)
|
||||
return _campaign_jobs_page_response(
|
||||
session,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
base_filters=base_filters,
|
||||
filtered=filtered,
|
||||
reviewed_keys=reviewed_keys,
|
||||
review_metadata=review_metadata,
|
||||
page=filters.page,
|
||||
page_size=filters.page_size,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
cursor=filters.cursor,
|
||||
)
|
||||
|
||||
|
||||
def _campaign_jobs_full_delta_response(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
campaign_id: str,
|
||||
version_id: str | None,
|
||||
page: int,
|
||||
page_size: int,
|
||||
send_status: list[str] | None,
|
||||
validation_status: list[str] | None,
|
||||
imap_status: list[str] | None,
|
||||
query_text: str | None,
|
||||
grid_filters: dict[str, str] | None,
|
||||
sort_by: str,
|
||||
sort_direction: str,
|
||||
cursor: str | None = None,
|
||||
) -> CampaignJobsDeltaResponse:
|
||||
_campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||
_campaign_jobs_query_context(
|
||||
session,
|
||||
principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
send_status=send_status,
|
||||
validation_status=validation_status,
|
||||
imap_status=imap_status,
|
||||
query_text=query_text,
|
||||
grid_filters=grid_filters,
|
||||
)
|
||||
)
|
||||
payload = _campaign_jobs_page_response(
|
||||
session,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
base_filters=base_filters,
|
||||
filtered=filtered,
|
||||
reviewed_keys=reviewed_keys,
|
||||
review_metadata=review_metadata,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
send_status=send_status,
|
||||
validation_status=validation_status,
|
||||
imap_status=imap_status,
|
||||
query_text=query_text,
|
||||
grid_filters=grid_filters,
|
||||
sort_by=sort_by,
|
||||
sort_direction=sort_direction,
|
||||
cursor=cursor,
|
||||
)
|
||||
return CampaignJobsDeltaResponse(
|
||||
**payload.model_dump(),
|
||||
deleted=[],
|
||||
watermark=_campaign_jobs_delta_watermark(session, principal.tenant_id),
|
||||
has_more=False,
|
||||
full=True,
|
||||
)
|
||||
|
||||
|
||||
def _job_filter_membership_can_shift(
|
||||
*,
|
||||
send_status: list[str] | None,
|
||||
validation_status: list[str] | None,
|
||||
imap_status: list[str] | None,
|
||||
query_text: str | None,
|
||||
grid_filters: dict[str, str] | None,
|
||||
sort_by: str,
|
||||
sort_direction: str,
|
||||
) -> bool:
|
||||
return bool(
|
||||
send_status
|
||||
or validation_status
|
||||
or imap_status
|
||||
or (query_text and query_text.strip())
|
||||
or grid_filters
|
||||
or sort_by != "number"
|
||||
or sort_direction != "asc"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/jobs/delta", response_model=CampaignJobsDeltaResponse)
|
||||
def list_jobs_delta(
|
||||
campaign_id: str,
|
||||
filters: CampaignJobsQuery = Depends(CampaignJobsQuery),
|
||||
since: str | None = None,
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
if since is None:
|
||||
return _campaign_jobs_full_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
page=filters.page,
|
||||
page_size=filters.page_size,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
cursor=filters.cursor,
|
||||
)
|
||||
|
||||
campaign, base_filters, filtered, review_metadata, reviewed_keys = (
|
||||
_campaign_jobs_query_context(
|
||||
session,
|
||||
principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
)
|
||||
)
|
||||
try:
|
||||
since_sequence = decode_sequence_watermark(since)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
|
||||
) from exc
|
||||
if sequence_watermark_is_expired(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=CAMPAIGNS_MODULE_ID,
|
||||
collections=(CAMPAIGN_JOBS_COLLECTION,),
|
||||
):
|
||||
return _campaign_jobs_full_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
page=filters.page,
|
||||
page_size=filters.page_size,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
cursor=filters.cursor,
|
||||
)
|
||||
|
||||
entries_plus_one = sequence_entries_since(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=CAMPAIGNS_MODULE_ID,
|
||||
collections=(CAMPAIGN_JOBS_COLLECTION,),
|
||||
limit=limit + 1,
|
||||
)
|
||||
has_more = len(entries_plus_one) > limit
|
||||
entries = entries_plus_one[:limit]
|
||||
relevant_entries = [
|
||||
entry
|
||||
for entry in entries
|
||||
if (entry.payload or {}).get("campaign_id") == campaign.id
|
||||
and (
|
||||
not filters.version_id
|
||||
or (entry.payload or {}).get("version_id") == filters.version_id
|
||||
)
|
||||
]
|
||||
|
||||
if relevant_entries and (
|
||||
_job_filter_membership_can_shift(
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
)
|
||||
or any(entry.operation in {"created", "deleted"} for entry in relevant_entries)
|
||||
):
|
||||
return _campaign_jobs_full_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
page=filters.page,
|
||||
page_size=filters.page_size,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
cursor=filters.cursor,
|
||||
)
|
||||
|
||||
changed_job_ids = {
|
||||
entry.resource_id
|
||||
for entry in relevant_entries
|
||||
if entry.resource_type == "campaign_job" and entry.operation != "deleted"
|
||||
}
|
||||
payload = _campaign_jobs_page_response(
|
||||
session,
|
||||
campaign_id=campaign_id,
|
||||
version_id=filters.version_id,
|
||||
base_filters=base_filters,
|
||||
filtered=filtered,
|
||||
reviewed_keys=reviewed_keys,
|
||||
review_metadata=review_metadata,
|
||||
page=filters.page,
|
||||
page_size=filters.page_size,
|
||||
send_status=filters.send_status,
|
||||
validation_status=filters.validation_status,
|
||||
imap_status=filters.imap_status,
|
||||
query_text=filters.query_text,
|
||||
grid_filters=filters.grid_filters,
|
||||
sort_by=filters.sort_by,
|
||||
sort_direction=filters.sort_direction,
|
||||
cursor=filters.cursor,
|
||||
changed_job_ids=changed_job_ids,
|
||||
)
|
||||
watermark = (
|
||||
encode_sequence_watermark(entries[-1].id)
|
||||
if has_more and entries
|
||||
else _campaign_jobs_delta_watermark(session, principal.tenant_id)
|
||||
)
|
||||
return CampaignJobsDeltaResponse(
|
||||
**payload.model_dump(),
|
||||
deleted=[],
|
||||
watermark=watermark,
|
||||
has_more=has_more,
|
||||
full=False,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/jobs/{job_id}", response_model=CampaignJobDetailResponse)
|
||||
def get_job_detail(
|
||||
campaign_id: str,
|
||||
job_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
job = session.get(CampaignJob, job_id)
|
||||
if (
|
||||
not job
|
||||
or job.campaign_id != campaign.id
|
||||
or job.tenant_id != principal.tenant_id
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
|
||||
)
|
||||
send_attempts = _job_attempt_rows(
|
||||
session.query(SendAttempt)
|
||||
.filter(SendAttempt.job_id == job.id)
|
||||
.order_by(SendAttempt.attempt_number.asc()),
|
||||
label="SMTP attempts for this campaign job",
|
||||
)
|
||||
imap_attempts = _job_attempt_rows(
|
||||
session.query(ImapAppendAttempt)
|
||||
.filter(ImapAppendAttempt.job_id == job.id)
|
||||
.order_by(ImapAppendAttempt.attempt_number.asc()),
|
||||
label="IMAP attempts for this campaign job",
|
||||
)
|
||||
postbox_attempts = _job_attempt_rows(
|
||||
session.query(PostboxDeliveryAttempt)
|
||||
.filter(PostboxDeliveryAttempt.job_id == job.id)
|
||||
.order_by(
|
||||
PostboxDeliveryAttempt.target_index.asc(),
|
||||
PostboxDeliveryAttempt.attempt_number.asc(),
|
||||
),
|
||||
label="Postbox attempts for this campaign job",
|
||||
)
|
||||
print_attempts = _job_attempt_rows(
|
||||
session.query(PrintOutputAttempt)
|
||||
.filter(PrintOutputAttempt.job_id == job.id)
|
||||
.order_by(PrintOutputAttempt.attempt_number.asc()),
|
||||
label="Printable output attempts for this campaign job",
|
||||
)
|
||||
message_actions = _job_attempt_rows(
|
||||
session.query(CampaignMessageAction)
|
||||
.filter(CampaignMessageAction.job_id == job.id)
|
||||
.order_by(CampaignMessageAction.created_at.asc()),
|
||||
label="Single-message actions for this campaign job",
|
||||
)
|
||||
action_ids = [action.id for action in message_actions]
|
||||
message_action_attempts = (
|
||||
_job_attempt_rows(
|
||||
session.query(CampaignMessageActionAttempt)
|
||||
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
|
||||
.order_by(CampaignMessageActionAttempt.started_at.asc()),
|
||||
label="Single-message action attempts for this campaign job",
|
||||
)
|
||||
if action_ids
|
||||
else []
|
||||
)
|
||||
return CampaignJobDetailResponse(
|
||||
job=_job_detail_payload(
|
||||
job,
|
||||
calendar_invitation=_calendar_invitations_for_jobs(
|
||||
session,
|
||||
[job],
|
||||
).get(job.id),
|
||||
),
|
||||
attempts=_job_attempts_payload(
|
||||
send_attempts,
|
||||
imap_attempts,
|
||||
postbox_attempts=postbox_attempts,
|
||||
print_attempts=print_attempts,
|
||||
message_actions=message_actions,
|
||||
message_action_attempts=message_action_attempts,
|
||||
postbox_receipts=_postbox_receipts_for_attempts(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
attempts=postbox_attempts,
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{campaign_id}/jobs/{job_id}/diagnostics",
|
||||
response_model=CampaignJobDiagnosticsResponse,
|
||||
)
|
||||
def get_job_diagnostics(
|
||||
campaign_id: str,
|
||||
job_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:diagnostic:read")),
|
||||
):
|
||||
"""Return infrastructure details only to campaign operators/admins."""
|
||||
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
job = session.get(CampaignJob, job_id)
|
||||
if (
|
||||
not job
|
||||
or job.campaign_id != campaign.id
|
||||
or job.tenant_id != principal.tenant_id
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign job not found"
|
||||
)
|
||||
send_attempts = _job_attempt_rows(
|
||||
session.query(SendAttempt)
|
||||
.filter(SendAttempt.job_id == job.id)
|
||||
.order_by(SendAttempt.attempt_number.asc()),
|
||||
label="SMTP diagnostics for this campaign job",
|
||||
)
|
||||
imap_attempts = _job_attempt_rows(
|
||||
session.query(ImapAppendAttempt)
|
||||
.filter(ImapAppendAttempt.job_id == job.id)
|
||||
.order_by(ImapAppendAttempt.attempt_number.asc()),
|
||||
label="IMAP diagnostics for this campaign job",
|
||||
)
|
||||
postbox_attempts = _job_attempt_rows(
|
||||
session.query(PostboxDeliveryAttempt)
|
||||
.filter(PostboxDeliveryAttempt.job_id == job.id)
|
||||
.order_by(
|
||||
PostboxDeliveryAttempt.target_index.asc(),
|
||||
PostboxDeliveryAttempt.attempt_number.asc(),
|
||||
),
|
||||
label="Postbox diagnostics for this campaign job",
|
||||
)
|
||||
print_attempts = _job_attempt_rows(
|
||||
session.query(PrintOutputAttempt)
|
||||
.filter(PrintOutputAttempt.job_id == job.id)
|
||||
.order_by(PrintOutputAttempt.attempt_number.asc()),
|
||||
label="Printable output diagnostics for this campaign job",
|
||||
)
|
||||
message_actions = _job_attempt_rows(
|
||||
session.query(CampaignMessageAction)
|
||||
.filter(CampaignMessageAction.job_id == job.id)
|
||||
.order_by(CampaignMessageAction.created_at.asc()),
|
||||
label="Single-message action diagnostics for this campaign job",
|
||||
)
|
||||
action_ids = [action.id for action in message_actions]
|
||||
message_action_attempts = (
|
||||
_job_attempt_rows(
|
||||
session.query(CampaignMessageActionAttempt)
|
||||
.filter(CampaignMessageActionAttempt.action_id.in_(action_ids))
|
||||
.order_by(CampaignMessageActionAttempt.started_at.asc()),
|
||||
label="Single-message action-attempt diagnostics for this campaign job",
|
||||
)
|
||||
if action_ids
|
||||
else []
|
||||
)
|
||||
return _job_diagnostics_payload(
|
||||
job,
|
||||
send_attempts,
|
||||
imap_attempts,
|
||||
postbox_attempts=postbox_attempts,
|
||||
print_attempts=print_attempts,
|
||||
message_actions=message_actions,
|
||||
message_action_attempts=message_action_attempts,
|
||||
postbox_receipts=_postbox_receipts_for_attempts(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
attempts=postbox_attempts,
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,102 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, status
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.artifact_reconciliation import (
|
||||
CampaignArtifactReconciliationError,
|
||||
reconcile_campaign_artifacts,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.campaigns import _object_storage
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
CampaignArtifactReconcileRequest,
|
||||
CampaignArtifactReconcileResponse,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_core.core.object_storage import StorageBackendError
|
||||
from govoplan_core.core.recovery import RecoveryGuaranteeError
|
||||
from govoplan_core.core.recovery_runtime import (
|
||||
RecoveryOperationBusy,
|
||||
RecoveryOperationStateConflict,
|
||||
)
|
||||
from govoplan_core.db.session import get_database, get_session
|
||||
from govoplan_core.server.runtime_agent import application_runtime_identity
|
||||
|
||||
|
||||
router = APIRouter(prefix="/campaigns/operations", tags=["campaigns"])
|
||||
|
||||
|
||||
@router.post(
|
||||
"/artifacts/reconcile",
|
||||
response_model=CampaignArtifactReconcileResponse,
|
||||
)
|
||||
def reconcile_artifacts(
|
||||
payload: CampaignArtifactReconcileRequest,
|
||||
request: Request,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("system:settings:write")),
|
||||
) -> CampaignArtifactReconcileResponse:
|
||||
"""Inventory or remove old, unreferenced Campaign-owned build objects."""
|
||||
|
||||
try:
|
||||
result = reconcile_campaign_artifacts(
|
||||
get_database().SessionLocal,
|
||||
storage=_object_storage(),
|
||||
identity=application_runtime_identity(request.app),
|
||||
tenant_id=principal.tenant_id,
|
||||
apply=payload.apply,
|
||||
idempotency_key=payload.idempotency_key,
|
||||
grace_period_hours=payload.grace_period_hours,
|
||||
cursor=payload.cursor,
|
||||
page_size=payload.page_size,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except (RecoveryOperationBusy, RecoveryOperationStateConflict) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except (
|
||||
CampaignArtifactReconciliationError,
|
||||
RecoveryGuaranteeError,
|
||||
StorageBackendError,
|
||||
SQLAlchemyError,
|
||||
OSError,
|
||||
) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=(
|
||||
"Campaign artifact reconciliation is temporarily unavailable "
|
||||
f"({type(exc).__name__})."
|
||||
),
|
||||
) from exc
|
||||
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action=(
|
||||
"campaign.artifact_orphans_reconciled"
|
||||
if payload.apply
|
||||
else "campaign.artifact_inventory_scanned"
|
||||
),
|
||||
object_type="campaign_artifact_namespace",
|
||||
object_id=principal.tenant_id,
|
||||
details={
|
||||
"apply": payload.apply,
|
||||
"status": result["status"],
|
||||
"scanned_count": result["scanned_count"],
|
||||
"candidate_count": result["candidate_count"],
|
||||
"deleted_count": result["deleted_count"],
|
||||
"failure_count": result["failure_count"],
|
||||
"manifest_sha256": result["manifest_sha256"],
|
||||
"recovery_operation_id": result["recovery_operation_id"],
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return CampaignArtifactReconcileResponse.model_validate(result)
|
||||
@@ -0,0 +1,257 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Response, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
ReportEmailRequest,
|
||||
ReportEmailResponse,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_campaign.backend.reports.campaigns import (
|
||||
CampaignReportError,
|
||||
generate_campaign_report,
|
||||
generate_jobs_csv,
|
||||
)
|
||||
from govoplan_campaign.backend.reports.emailing import (
|
||||
CampaignReportEmailError,
|
||||
send_campaign_report_email,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
MailDeliveryCommandError,
|
||||
MailProfileError,
|
||||
SmtpConfigurationError,
|
||||
SmtpSendError,
|
||||
mail_integration,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_get_campaign_for_principal,
|
||||
_require_mail_profile_use_if_needed,
|
||||
_require_permission,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _enqueue_mail_command() -> None:
|
||||
try:
|
||||
from govoplan_core.celery_app import celery
|
||||
from govoplan_core.settings import settings
|
||||
|
||||
if settings.celery_enabled:
|
||||
celery.send_task(
|
||||
"govoplan.mail.dispatch_outbox",
|
||||
args=[None, 25],
|
||||
queue="mail",
|
||||
)
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"Mail delivery command is durable but immediate worker wake-up failed",
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/summary")
|
||||
def campaign_summary(
|
||||
campaign_id: str,
|
||||
version_id: str | None = None,
|
||||
include_jobs: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
"""Return dashboard-friendly campaign status counters and summaries."""
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
if include_jobs:
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
|
||||
try:
|
||||
return generate_campaign_report(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
include_jobs=include_jobs,
|
||||
include_recent_failures=include_jobs,
|
||||
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||
)
|
||||
except CampaignReportError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/report")
|
||||
def campaign_report(
|
||||
campaign_id: str,
|
||||
version_id: str | None = None,
|
||||
include_jobs: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
|
||||
):
|
||||
"""Return the recipient-level JSON report for one campaign."""
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
|
||||
try:
|
||||
return generate_campaign_report(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
include_jobs=include_jobs,
|
||||
include_recent_failures=include_jobs,
|
||||
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||
)
|
||||
except CampaignReportError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/report/jobs.csv")
|
||||
def campaign_jobs_csv(
|
||||
campaign_id: str,
|
||||
version_id: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:report:export")),
|
||||
):
|
||||
"""Export per-job campaign status as CSV."""
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:export")
|
||||
|
||||
try:
|
||||
csv_text = generate_jobs_csv(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||
)
|
||||
except CampaignReportError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
return Response(
|
||||
content=csv_text,
|
||||
media_type="text/csv; charset=utf-8",
|
||||
headers={
|
||||
"Content-Disposition": f'attachment; filename="campaign-{campaign_id}-jobs.csv"'
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{campaign_id}/report/email", response_model=ReportEmailResponse)
|
||||
def email_campaign_report(
|
||||
campaign_id: str,
|
||||
payload: ReportEmailRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:report:send")),
|
||||
):
|
||||
"""Generate a campaign report and send it to one or more email addresses."""
|
||||
campaign = _get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:export")
|
||||
selected_version_id = payload.version_id or campaign.current_version_id
|
||||
selected_version = (
|
||||
session.get(CampaignVersion, selected_version_id)
|
||||
if selected_version_id
|
||||
else None
|
||||
)
|
||||
if selected_version is not None and selected_version.campaign_id == campaign.id:
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal,
|
||||
selected_version.raw_json
|
||||
if isinstance(selected_version.raw_json, dict)
|
||||
else {},
|
||||
)
|
||||
try:
|
||||
result = send_campaign_report_email(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=payload.version_id,
|
||||
to=payload.to,
|
||||
include_jobs=payload.include_jobs,
|
||||
attach_jobs_csv=payload.attach_jobs_csv,
|
||||
attach_report_json=payload.attach_report_json,
|
||||
dry_run=payload.dry_run,
|
||||
idempotency_key=payload.idempotency_key,
|
||||
created_by_user_id=principal.user.id,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="report.email_requested"
|
||||
if not payload.dry_run
|
||||
else "report.email_dry_run",
|
||||
object_type="campaign",
|
||||
object_id=campaign_id,
|
||||
details=result.audit_dict(),
|
||||
commit=True,
|
||||
)
|
||||
if not payload.dry_run:
|
||||
_enqueue_mail_command()
|
||||
return ReportEmailResponse(result=result.as_dict())
|
||||
except CampaignReportError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
except (
|
||||
CampaignReportEmailError,
|
||||
MailProfileError,
|
||||
MailDeliveryCommandError,
|
||||
SmtpConfigurationError,
|
||||
SmtpSendError,
|
||||
) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
logger.error("Campaign report email failed with an unexpected internal error")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="Campaign report email could not be completed.",
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{campaign_id}/report/email/{command_id}",
|
||||
response_model=ReportEmailResponse,
|
||||
)
|
||||
def campaign_report_email_status(
|
||||
campaign_id: str,
|
||||
command_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:report:read")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
try:
|
||||
result = mail_integration().delivery_command_summary(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
command_id=command_id,
|
||||
)
|
||||
except MailDeliveryCommandError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
if (
|
||||
result.get("source_module") != "campaigns"
|
||||
or result.get("source_resource_type") != "campaign"
|
||||
or result.get("source_resource_id") != campaign_id
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Campaign report delivery not found",
|
||||
)
|
||||
return ReportEmailResponse(result=result)
|
||||
@@ -0,0 +1,288 @@
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.api.v1.schemas import (
|
||||
ReferenceOptionListResponse,
|
||||
ReferenceOptionResponse,
|
||||
)
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
CampaignShareItem,
|
||||
CampaignShareListResponse,
|
||||
CampaignShareTargetItem,
|
||||
CampaignShareTargetsResponse,
|
||||
CampaignShareUpsertRequest,
|
||||
CampaignOwnerUpdateRequest,
|
||||
CampaignResponse,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignShare,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_core.core.references import (
|
||||
access_scope_reference_page,
|
||||
access_scope_reference_provider_available,
|
||||
)
|
||||
from govoplan_core.core.runtime import get_registry
|
||||
from govoplan_core.security.time import utc_now
|
||||
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_access_directory,
|
||||
_get_campaign_for_principal,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{campaign_id}/share-target-options",
|
||||
response_model=ReferenceOptionListResponse,
|
||||
)
|
||||
def search_campaign_share_targets(
|
||||
campaign_id: str,
|
||||
target_type: Literal["user", "group"],
|
||||
q: str = "",
|
||||
selected: list[str] = Query(default=[]),
|
||||
limit: int = Query(default=50, ge=1, le=200),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
) -> ReferenceOptionListResponse:
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
registry = get_registry()
|
||||
try:
|
||||
page = access_scope_reference_page(
|
||||
registry,
|
||||
principal,
|
||||
scope_type=target_type,
|
||||
reference_kind="membership" if target_type == "user" else "group",
|
||||
query=q,
|
||||
selected_values=selected,
|
||||
limit=limit,
|
||||
cursor=cursor,
|
||||
administrative=True,
|
||||
session=session,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
return ReferenceOptionListResponse(
|
||||
options=[
|
||||
ReferenceOptionResponse(**option.to_dict())
|
||||
for option in page.options
|
||||
],
|
||||
provider_available=access_scope_reference_provider_available(registry),
|
||||
next_cursor=page.next_cursor,
|
||||
has_more=page.has_more,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/share-targets", response_model=CampaignShareTargetsResponse)
|
||||
def list_campaign_share_targets(
|
||||
campaign_id: str,
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
directory = _access_directory()
|
||||
users = [
|
||||
user
|
||||
for user in directory.users_for_tenant(principal.tenant_id)
|
||||
if user.status == "active"
|
||||
]
|
||||
groups = [
|
||||
group
|
||||
for group in directory.groups_for_tenant(principal.tenant_id)
|
||||
if group.status == "active"
|
||||
]
|
||||
if len(users) > limit or len(groups) > limit:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
|
||||
detail=(
|
||||
f"Campaign share targets exceed the maximum response size of {limit} "
|
||||
"users or groups. Use a searchable directory selector."
|
||||
),
|
||||
)
|
||||
return CampaignShareTargetsResponse(
|
||||
users=[
|
||||
CampaignShareTargetItem(
|
||||
id=item.id, name=item.display_name or item.email, secondary=item.email
|
||||
)
|
||||
for item in users
|
||||
],
|
||||
groups=[
|
||||
CampaignShareTargetItem(id=item.id, name=item.name, secondary=None)
|
||||
for item in groups
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/shares", response_model=CampaignShareListResponse)
|
||||
def list_campaign_shares(
|
||||
campaign_id: str,
|
||||
page: int = Query(default=1, ge=1),
|
||||
page_size: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
):
|
||||
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
query = session.query(CampaignShare).filter(
|
||||
CampaignShare.tenant_id == principal.tenant_id,
|
||||
CampaignShare.campaign_id == campaign.id,
|
||||
CampaignShare.revoked_at.is_(None),
|
||||
)
|
||||
total = query.order_by(None).count()
|
||||
pages = max(1, (total + page_size - 1) // page_size)
|
||||
shares = (
|
||||
query.order_by(
|
||||
CampaignShare.target_type.asc(),
|
||||
CampaignShare.target_id.asc(),
|
||||
CampaignShare.id.asc(),
|
||||
)
|
||||
.offset((page - 1) * page_size)
|
||||
.limit(page_size)
|
||||
.all()
|
||||
)
|
||||
return CampaignShareListResponse(
|
||||
shares=[CampaignShareItem.model_validate(item) for item in shares],
|
||||
total=total,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
pages=pages,
|
||||
)
|
||||
|
||||
|
||||
@router.put("/{campaign_id}/owner", response_model=CampaignResponse)
|
||||
def update_campaign_owner(
|
||||
campaign_id: str,
|
||||
payload: CampaignOwnerUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
):
|
||||
del payload
|
||||
_get_campaign_for_principal(
|
||||
session,
|
||||
campaign_id,
|
||||
principal,
|
||||
write=True,
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=(
|
||||
"Direct campaign owner mutation has been retired. Use the "
|
||||
"ownership transfer workflow so the target can accept."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/shares",
|
||||
response_model=CampaignShareItem,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def upsert_campaign_share(
|
||||
campaign_id: str,
|
||||
payload: CampaignShareUpsertRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
):
|
||||
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
directory = _access_directory()
|
||||
if payload.target_type == "user":
|
||||
target = directory.get_user(payload.target_id)
|
||||
if target is not None and (
|
||||
target.tenant_id != principal.tenant_id or target.status != "active"
|
||||
):
|
||||
target = None
|
||||
else:
|
||||
target = directory.get_group(payload.target_id)
|
||||
if target is not None and (
|
||||
target.tenant_id != principal.tenant_id or target.status != "active"
|
||||
):
|
||||
target = None
|
||||
if target is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Share target not found"
|
||||
)
|
||||
share = (
|
||||
session.query(CampaignShare)
|
||||
.filter(
|
||||
CampaignShare.campaign_id == campaign.id,
|
||||
CampaignShare.target_type == payload.target_type,
|
||||
CampaignShare.target_id == payload.target_id,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if share is None:
|
||||
share = CampaignShare(
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign.id,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
permission=payload.permission,
|
||||
created_by_user_id=principal.user.id,
|
||||
)
|
||||
else:
|
||||
share.permission = payload.permission
|
||||
share.revoked_at = None
|
||||
session.add(share)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.share_upserted",
|
||||
object_type="campaign",
|
||||
object_id=campaign.id,
|
||||
details=payload.model_dump(),
|
||||
commit=True,
|
||||
)
|
||||
return CampaignShareItem.model_validate(share)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/{campaign_id}/shares/{share_id}", status_code=status.HTTP_204_NO_CONTENT
|
||||
)
|
||||
def revoke_campaign_share(
|
||||
campaign_id: str,
|
||||
share_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:share")),
|
||||
):
|
||||
campaign = _get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
share = (
|
||||
session.query(CampaignShare)
|
||||
.filter(
|
||||
CampaignShare.id == share_id,
|
||||
CampaignShare.campaign_id == campaign.id,
|
||||
CampaignShare.tenant_id == principal.tenant_id,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if share is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign share not found"
|
||||
)
|
||||
share.revoked_at = utc_now()
|
||||
session.add(share)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.share_revoked",
|
||||
object_type="campaign",
|
||||
object_id=campaign.id,
|
||||
details={"share_id": share_id},
|
||||
commit=True,
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
# Queue / delivery control -------------------------------------------------
|
||||
@@ -0,0 +1,908 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from urllib.parse import quote
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, Response, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.schemas import (
|
||||
BuildCampaignRequest,
|
||||
CampaignCreateResponse,
|
||||
CampaignResponse,
|
||||
CampaignVersionDetailResponse,
|
||||
CampaignVersionResponse,
|
||||
CampaignVersionSetStepRequest,
|
||||
CampaignReviewStateRequest,
|
||||
CampaignVersionUpdateRequest,
|
||||
CampaignPartialValidationRequest,
|
||||
CampaignPartialValidationResponse,
|
||||
ValidateCampaignRequest,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_core.core.object_storage import StorageBackendError
|
||||
from govoplan_core.core.recovery import (
|
||||
RecoveryGuaranteeError,
|
||||
RecoveryMode,
|
||||
RecoveryPlan,
|
||||
)
|
||||
from govoplan_core.core.recovery_runtime import (
|
||||
RecoveryOperationBusy,
|
||||
RecoveryOperationStateConflict,
|
||||
begin_durable_recovery_operation,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_core.db.session import get_database, get_session
|
||||
from govoplan_core.server.runtime_agent import application_runtime_identity
|
||||
from govoplan_campaign.backend.response_security import (
|
||||
public_campaign_payload,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.campaigns import (
|
||||
CampaignPersistenceError,
|
||||
_object_storage,
|
||||
build_campaign_version,
|
||||
validate_campaign_version,
|
||||
)
|
||||
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
|
||||
from govoplan_campaign.backend.persistence.versions import (
|
||||
LockedCampaignVersionError,
|
||||
fork_campaign_version_for_edit,
|
||||
is_version_final_locked,
|
||||
is_user_locked_version,
|
||||
get_campaign_version_for_tenant,
|
||||
lock_campaign_version_temporarily,
|
||||
permanently_lock_campaign_version,
|
||||
publish_campaign_version,
|
||||
unlock_user_locked_campaign_version,
|
||||
unlock_validated_campaign_version,
|
||||
update_campaign_version,
|
||||
update_campaign_review_state,
|
||||
validate_campaign_partial,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_campaign.backend.route_support import (
|
||||
_campaign_response_context,
|
||||
_campaign_version_detail_response,
|
||||
_get_campaign_for_principal,
|
||||
_get_campaign_for_tenant,
|
||||
_get_version_for_principal,
|
||||
_get_version_for_tenant,
|
||||
_require_mail_profile_use_if_needed,
|
||||
_require_permission,
|
||||
_update_campaign_version_detail_response,
|
||||
_write_current_version_snapshot_if_available,
|
||||
bounded_query_rows as _bounded_query_rows,
|
||||
)
|
||||
from govoplan_campaign.backend.routes.attachments import (
|
||||
CampaignAttachmentLinkMatchesResponse,
|
||||
_link_campaign_attachment_matches,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||
|
||||
|
||||
def _canonical_sha256(value: object) -> str:
|
||||
return hashlib.sha256(
|
||||
json.dumps(
|
||||
value,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
).hexdigest()
|
||||
|
||||
|
||||
def _campaign_build_recovery_plan(raw_json: dict[str, object]) -> RecoveryPlan:
|
||||
delivery = raw_json.get("delivery")
|
||||
print_config = delivery.get("print") if isinstance(delivery, dict) else None
|
||||
persists_managed_output = bool(
|
||||
isinstance(print_config, dict)
|
||||
and print_config.get("persist_to_files", True)
|
||||
)
|
||||
if persists_managed_output:
|
||||
return RecoveryPlan(
|
||||
mode=RecoveryMode.FORWARD_RECOVERY,
|
||||
preconditions=("validated Campaign version is locked",),
|
||||
forward_recovery_steps=(
|
||||
"reuse the Templates render idempotency key",
|
||||
"reconcile the managed output and Campaign build manifests",
|
||||
),
|
||||
verification_steps=(
|
||||
"compare committed Campaign jobs with generated object evidence",
|
||||
),
|
||||
)
|
||||
return RecoveryPlan(
|
||||
mode=RecoveryMode.COMPENSATION,
|
||||
preconditions=("validated Campaign version is locked",),
|
||||
compensation_steps=("delete every object under the reserved build prefix",),
|
||||
verification_steps=(
|
||||
"compare committed Campaign jobs with generated object evidence",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/versions/{version_id}/print-output/download")
|
||||
def download_print_output(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
) -> Response:
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
try:
|
||||
version = get_campaign_version_for_tenant(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
)
|
||||
except CampaignPersistenceError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
build_summary = (
|
||||
version.build_summary if isinstance(version.build_summary, dict) else {}
|
||||
)
|
||||
print_output = build_summary.get("print_output")
|
||||
artifact = (
|
||||
print_output.get("artifact")
|
||||
if isinstance(print_output, dict)
|
||||
and isinstance(print_output.get("artifact"), dict)
|
||||
else {}
|
||||
)
|
||||
storage_key = artifact.get("storage_key")
|
||||
if artifact.get("kind") != "bounded_download" or not storage_key:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="This printable output is not stored as a Campaign download.",
|
||||
)
|
||||
try:
|
||||
payload = _object_storage().get_bytes(str(storage_key))
|
||||
except StorageBackendError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="The printable output is temporarily unavailable.",
|
||||
) from exc
|
||||
expected_sha256 = str(print_output.get("output_sha256") or "")
|
||||
actual_sha256 = hashlib.sha256(payload).hexdigest()
|
||||
if not expected_sha256 or actual_sha256 != expected_sha256:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="The printable output failed its integrity check.",
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.print_output_downloaded",
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"output_sha256": actual_sha256,
|
||||
"template_id": print_output.get("template_id"),
|
||||
"template_revision_id": print_output.get("template_revision_id"),
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
filename = quote(
|
||||
str(artifact.get("filename") or "campaign-print-output.html"),
|
||||
safe="._-",
|
||||
)
|
||||
return Response(
|
||||
content=payload,
|
||||
media_type=str(artifact.get("content_type") or "application/octet-stream"),
|
||||
headers={
|
||||
"Content-Disposition": f"attachment; filename*=UTF-8''{filename}",
|
||||
"X-Content-SHA256": actual_sha256,
|
||||
"X-Content-Type-Options": "nosniff",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{campaign_id}/versions", response_model=list[CampaignVersionResponse])
|
||||
def list_versions(
|
||||
campaign_id: str,
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
versions = _bounded_query_rows(
|
||||
session.query(CampaignVersion)
|
||||
.filter(CampaignVersion.campaign_id == campaign.id)
|
||||
.order_by(CampaignVersion.version_number.desc()),
|
||||
limit=limit,
|
||||
label="Campaign version history",
|
||||
)
|
||||
return [
|
||||
CampaignVersionResponse.model_validate(
|
||||
item,
|
||||
context=_campaign_response_context(principal),
|
||||
)
|
||||
for item in versions
|
||||
]
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
|
||||
)
|
||||
def get_version_detail(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
response: Response,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
try:
|
||||
version = get_campaign_version_for_tenant(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
)
|
||||
result = CampaignVersionDetailResponse.model_validate(
|
||||
version,
|
||||
context=_campaign_response_context(principal),
|
||||
)
|
||||
response.headers["ETag"] = version.strong_etag
|
||||
return result
|
||||
except CampaignPersistenceError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/fork", response_model=CampaignCreateResponse
|
||||
)
|
||||
def fork_version_for_edit(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignVersionUpdateRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:copy")),
|
||||
):
|
||||
"""Create the campaign's next and only editable working version.
|
||||
|
||||
A new working copy may be created only after the current version is
|
||||
permanently user-locked or delivery-final. Validation and temporary user
|
||||
locks must be removed in place instead of creating parallel drafts.
|
||||
"""
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
|
||||
payload = payload or CampaignVersionUpdateRequest()
|
||||
source_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
if source_version.campaign_id != campaign_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
|
||||
)
|
||||
effective_json = (
|
||||
payload.campaign_json
|
||||
if isinstance(payload.campaign_json, dict)
|
||||
else source_version.raw_json
|
||||
)
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal,
|
||||
effective_json if isinstance(effective_json, dict) else {},
|
||||
)
|
||||
try:
|
||||
version = fork_campaign_version_for_edit(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
raw_json=payload.campaign_json,
|
||||
current_flow=payload.current_flow or "manual",
|
||||
current_step=payload.current_step,
|
||||
editor_state=payload.editor_state,
|
||||
source_filename=payload.source_filename,
|
||||
source_base_path=payload.source_base_path,
|
||||
autosave=True,
|
||||
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
|
||||
commit=False,
|
||||
)
|
||||
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.version_forked_for_edit",
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"source_version_id": version_id,
|
||||
"version_number": version.version_number,
|
||||
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
_write_current_version_snapshot_if_available(version)
|
||||
return CampaignCreateResponse(
|
||||
campaign=CampaignResponse.model_validate(campaign),
|
||||
version=CampaignVersionResponse.model_validate(
|
||||
version,
|
||||
context=_campaign_response_context(principal),
|
||||
),
|
||||
)
|
||||
except LockedCampaignVersionError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||
) from exc
|
||||
except CampaignPathSecurityError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except CampaignPersistenceError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/unlock-validation",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def unlock_version_validation(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
"""Unlock a successfully validated version before delivery starts.
|
||||
|
||||
Unlocking invalidates validation/build state and removes generated jobs for
|
||||
that version. Sent/final versions cannot be unlocked and must be copied.
|
||||
"""
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: unlock_validated_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_validation_unlocked",
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/lock-temporarily",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def lock_version_temporarily(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: lock_campaign_version_temporarily(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
user_id=principal.user.id,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_user_locked_temporarily",
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/unlock-user-lock",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def unlock_version_user_lock(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: unlock_user_locked_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_user_lock_removed",
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/lock-permanently",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def lock_version_permanently(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: permanently_lock_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
user_id=principal.user.id,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_user_locked_permanently",
|
||||
)
|
||||
|
||||
|
||||
@router.put(
|
||||
"/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse
|
||||
)
|
||||
def update_version_detail(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignVersionUpdateRequest,
|
||||
response: Response,
|
||||
if_match: str | None = Header(default=None, alias="If-Match"),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
result = _update_campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
version_id,
|
||||
payload,
|
||||
if_match=if_match,
|
||||
autosave=False,
|
||||
audit_action="campaign.version_updated",
|
||||
)
|
||||
response.headers["ETag"] = result.strong_etag
|
||||
return result
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/autosave",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def autosave_version(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignVersionUpdateRequest,
|
||||
response: Response,
|
||||
if_match: str | None = Header(default=None, alias="If-Match"),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
result = _update_campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
version_id,
|
||||
payload,
|
||||
if_match=if_match,
|
||||
autosave=True,
|
||||
audit_action="campaign.version_autosaved",
|
||||
)
|
||||
response.headers["ETag"] = result.strong_etag
|
||||
return result
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/set-step",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def set_version_step(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignVersionSetStepRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: update_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
current_flow=payload.current_flow,
|
||||
current_step=payload.current_step,
|
||||
autosave=True,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_step_updated",
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"current_flow": payload.current_flow,
|
||||
"current_step": payload.current_step,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/review-state",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def set_version_review_state(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignReviewStateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:review")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
try:
|
||||
version = update_campaign_review_state(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
inspection_complete=payload.inspection_complete,
|
||||
reviewed_message_keys=payload.reviewed_message_keys,
|
||||
issue_decisions=[
|
||||
item.model_dump()
|
||||
for item in payload.issue_decisions
|
||||
],
|
||||
user_id=principal.user.id,
|
||||
commit=False,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.message_review_updated",
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"inspection_complete": payload.inspection_complete,
|
||||
"reviewed_message_count": len(payload.reviewed_message_keys),
|
||||
"issue_decision_count": len(payload.issue_decisions),
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return CampaignVersionDetailResponse.model_validate(
|
||||
version,
|
||||
context=_campaign_response_context(principal),
|
||||
)
|
||||
except LockedCampaignVersionError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
|
||||
) from exc
|
||||
except CampaignPersistenceError as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/validate-partial",
|
||||
response_model=CampaignPartialValidationResponse,
|
||||
)
|
||||
def validate_version_partial(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
payload: CampaignPartialValidationRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
try:
|
||||
version = get_campaign_version_for_tenant(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
)
|
||||
campaign_json = (
|
||||
payload.campaign_json
|
||||
if payload and payload.campaign_json is not None
|
||||
else version.raw_json
|
||||
)
|
||||
result = validate_campaign_partial(
|
||||
campaign_json, section=payload.section if payload else None
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.version_partially_validated",
|
||||
object_type="campaign_version",
|
||||
object_id=version.id,
|
||||
details={
|
||||
"campaign_id": campaign_id,
|
||||
"section": result.get("section"),
|
||||
"ok": result.get("ok"),
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return CampaignPartialValidationResponse(**result)
|
||||
except CampaignPersistenceError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{campaign_id}/versions/{version_id}/publish",
|
||||
response_model=CampaignVersionDetailResponse,
|
||||
)
|
||||
def publish_version(
|
||||
campaign_id: str,
|
||||
version_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:update")),
|
||||
):
|
||||
_get_campaign_for_principal(session, campaign_id, principal, write=True)
|
||||
return _campaign_version_detail_response(
|
||||
session,
|
||||
principal,
|
||||
campaign_id,
|
||||
lambda: publish_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
version_id=version_id,
|
||||
user_id=principal.user.id,
|
||||
commit=False,
|
||||
),
|
||||
audit_action="campaign.version_user_locked_permanently",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/versions/{version_id}/validate")
|
||||
def validate_version(
|
||||
version_id: str,
|
||||
payload: ValidateCampaignRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:validate")),
|
||||
):
|
||||
_get_version_for_principal(session, version_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
payload = payload or ValidateCampaignRequest()
|
||||
try:
|
||||
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
if is_user_locked_version(version) or is_version_final_locked(version):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail="This version has a user lock or final delivery lock and cannot be validated. Remove a temporary lock or create an editable copy.",
|
||||
)
|
||||
link_result: CampaignAttachmentLinkMatchesResponse | None = None
|
||||
if payload.check_files and payload.link_unshared_matches:
|
||||
_require_permission(principal, "files:file:share")
|
||||
campaign = _get_campaign_for_tenant(
|
||||
session, version.campaign_id, principal.tenant_id
|
||||
)
|
||||
link_result = _link_campaign_attachment_matches(
|
||||
session,
|
||||
principal,
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
raw=version.raw_json if isinstance(version.raw_json, dict) else {},
|
||||
dry_run=False,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.attachment_matches_linked",
|
||||
object_type="campaign_version",
|
||||
object_id=version_id,
|
||||
details={
|
||||
"matched_file_count": link_result.matched_file_count,
|
||||
"already_linked_file_count": link_result.already_linked_file_count,
|
||||
"linked_file_count": link_result.linked_file_count,
|
||||
"during_validation": True,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
result = validate_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
version_id=version_id,
|
||||
check_files=payload.check_files,
|
||||
user_id=principal.user.id,
|
||||
principal=principal,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.validated",
|
||||
object_type="campaign_version",
|
||||
object_id=version_id,
|
||||
details={
|
||||
"check_files": payload.check_files,
|
||||
"link_unshared_matches": payload.link_unshared_matches,
|
||||
"linked_file_count": link_result.linked_file_count
|
||||
if link_result
|
||||
else 0,
|
||||
"ok": result.get("ok"),
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return public_campaign_payload(
|
||||
result,
|
||||
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except CampaignPersistenceError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
@router.post("/versions/{version_id}/build")
|
||||
def build_version(
|
||||
version_id: str,
|
||||
request: Request,
|
||||
payload: BuildCampaignRequest | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:build")),
|
||||
):
|
||||
version = _get_version_for_principal(session, version_id, principal, write=True)
|
||||
_require_permission(principal, "campaigns:recipient:read")
|
||||
_require_mail_profile_use_if_needed(
|
||||
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
)
|
||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
write_eml = payload.write_eml if payload else True
|
||||
source_sha256 = _canonical_sha256(raw_json)
|
||||
validation_sha256 = _canonical_sha256(
|
||||
version.validation_summary
|
||||
if isinstance(version.validation_summary, dict)
|
||||
else {}
|
||||
)
|
||||
idempotency_key = (
|
||||
payload.idempotency_key
|
||||
if payload and payload.idempotency_key
|
||||
else (
|
||||
f"campaign-build:{version.id}:{source_sha256}:"
|
||||
f"{validation_sha256}:{int(write_eml)}"
|
||||
)
|
||||
)
|
||||
try:
|
||||
identity = application_runtime_identity(request.app)
|
||||
recovery_start = begin_durable_recovery_operation(
|
||||
get_database().SessionLocal,
|
||||
identity=identity,
|
||||
module_id="campaigns",
|
||||
operation_type="build-artifacts",
|
||||
idempotency_key=idempotency_key,
|
||||
request={
|
||||
"tenant_id": principal.tenant_id,
|
||||
"campaign_id": version.campaign_id,
|
||||
"version_id": version.id,
|
||||
"source_sha256": source_sha256,
|
||||
"validation_sha256": validation_sha256,
|
||||
"write_eml": write_eml,
|
||||
},
|
||||
recovery_plan=_campaign_build_recovery_plan(raw_json),
|
||||
precondition_evidence={
|
||||
"campaign_version_id": version.id,
|
||||
"source_sha256": source_sha256,
|
||||
"validation_sha256": validation_sha256,
|
||||
"locked_at": version.locked_at.isoformat()
|
||||
if version.locked_at
|
||||
else None,
|
||||
"workflow_state": version.workflow_state,
|
||||
},
|
||||
lease_resource_key=(
|
||||
f"campaign:build:{principal.tenant_id}:{version.id}"
|
||||
),
|
||||
lease_ttl_seconds=30 * 60,
|
||||
resource_type="campaign_version",
|
||||
resource_id=version.id,
|
||||
metadata={"actor_account_id": principal.account_id},
|
||||
)
|
||||
if recovery_start.replayed:
|
||||
session.refresh(version)
|
||||
if not isinstance(version.build_summary, dict):
|
||||
raise RecoveryGuaranteeError(
|
||||
"A completed build operation has no committed Campaign summary"
|
||||
)
|
||||
return public_campaign_payload(
|
||||
version.build_summary,
|
||||
include_diagnostics=has_scope(
|
||||
principal, "campaigns:diagnostic:read"
|
||||
),
|
||||
)
|
||||
recovery_operation = recovery_start.operation
|
||||
if recovery_operation is None: # pragma: no cover - guarded by replay branch
|
||||
raise RecoveryGuaranteeError("Campaign build authority was not created")
|
||||
result = build_campaign_version(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
version_id=version_id,
|
||||
write_eml=write_eml,
|
||||
user_id=principal.user.id,
|
||||
principal=principal,
|
||||
recovery_operation=recovery_operation,
|
||||
build_id=recovery_start.operation_id,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="campaign.messages_built",
|
||||
object_type="campaign_version",
|
||||
object_id=version_id,
|
||||
details={
|
||||
"write_eml": write_eml,
|
||||
"built_count": result.get("built_count"),
|
||||
"recovery_operation_id": recovery_start.operation_id,
|
||||
},
|
||||
commit=True,
|
||||
)
|
||||
return public_campaign_payload(
|
||||
result,
|
||||
include_diagnostics=has_scope(principal, "campaigns:diagnostic:read"),
|
||||
)
|
||||
except CampaignPersistenceError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
|
||||
) from exc
|
||||
except (RecoveryOperationBusy, RecoveryOperationStateConflict) as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except RecoveryGuaranteeError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="Campaign build coordination is unavailable",
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
|
||||
) from exc
|
||||
@@ -60,7 +60,9 @@
|
||||
"integer",
|
||||
"double",
|
||||
"date",
|
||||
"password"
|
||||
"password",
|
||||
"organization_unit",
|
||||
"organization_function"
|
||||
],
|
||||
"default": "string"
|
||||
},
|
||||
@@ -92,7 +94,27 @@
|
||||
"mail_profile_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"description": "Stable reference to an authorized profile owned by the Mail module. Campaign JSON never stores SMTP/IMAP settings or credentials."
|
||||
"description": "Stable reference to an authorized server envelope owned by the Mail module. Campaign JSON never stores SMTP/IMAP settings or credentials."
|
||||
},
|
||||
"smtp_server_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"description": "Optional explicit Mail-owned SMTP server selection."
|
||||
},
|
||||
"smtp_credential_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"description": "Optional explicit core credential envelope bound to the selected SMTP server."
|
||||
},
|
||||
"imap_server_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"description": "Optional explicit Mail-owned IMAP server selection."
|
||||
},
|
||||
"imap_credential_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"description": "Optional explicit core credential envelope bound to the selected IMAP server."
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
@@ -337,7 +359,7 @@
|
||||
"continue",
|
||||
"warn"
|
||||
],
|
||||
"default": "ask"
|
||||
"default": "warn"
|
||||
},
|
||||
"ambiguous_behavior": {
|
||||
"type": "string",
|
||||
@@ -433,7 +455,7 @@
|
||||
"continue",
|
||||
"warn"
|
||||
],
|
||||
"default": "ask"
|
||||
"default": "block"
|
||||
},
|
||||
"missing_optional_attachment": {
|
||||
"type": "string",
|
||||
@@ -509,6 +531,15 @@
|
||||
"delivery": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"channel_policy": {
|
||||
"$ref": "#/$defs/delivery_channel_policy"
|
||||
},
|
||||
"postbox": {
|
||||
"$ref": "#/$defs/postbox_delivery"
|
||||
},
|
||||
"print": {
|
||||
"$ref": "#/$defs/print_delivery"
|
||||
},
|
||||
"rate_limit": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -626,6 +657,249 @@
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"delivery_channel_policy": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"mail",
|
||||
"postbox",
|
||||
"print",
|
||||
"mail_and_postbox",
|
||||
"mail_then_postbox",
|
||||
"postbox_then_mail",
|
||||
"mail_then_print",
|
||||
"postbox_then_print"
|
||||
],
|
||||
"default": "mail"
|
||||
},
|
||||
"postbox_target": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"mode"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 120
|
||||
},
|
||||
"mode": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"direct",
|
||||
"derived"
|
||||
]
|
||||
},
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 500
|
||||
},
|
||||
"postbox_id": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 36
|
||||
},
|
||||
"address_key": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 500
|
||||
},
|
||||
"template_id": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 36
|
||||
},
|
||||
"organization_unit_id": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 36
|
||||
},
|
||||
"organization_unit_field": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 255
|
||||
},
|
||||
"organization_unit_match": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"id",
|
||||
"slug"
|
||||
],
|
||||
"default": "id"
|
||||
},
|
||||
"function_id": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 36
|
||||
},
|
||||
"function_field": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 255
|
||||
},
|
||||
"function_match": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"id",
|
||||
"slug"
|
||||
],
|
||||
"default": "id"
|
||||
},
|
||||
"context_key": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 255
|
||||
},
|
||||
"context_field": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"maxLength": 255
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"postbox_delivery": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"targets": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/postbox_target"
|
||||
},
|
||||
"maxItems": 50,
|
||||
"default": []
|
||||
},
|
||||
"classification": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 50,
|
||||
"default": "internal"
|
||||
},
|
||||
"unresolved_target": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"block",
|
||||
"ask",
|
||||
"drop",
|
||||
"continue",
|
||||
"warn"
|
||||
],
|
||||
"default": "block"
|
||||
},
|
||||
"vacant_target": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"block",
|
||||
"ask",
|
||||
"drop",
|
||||
"continue",
|
||||
"warn"
|
||||
],
|
||||
"default": "warn"
|
||||
},
|
||||
"duplicate_target": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"block",
|
||||
"ask",
|
||||
"drop",
|
||||
"continue",
|
||||
"warn"
|
||||
],
|
||||
"default": "warn"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"default": {}
|
||||
},
|
||||
"print_target": {
|
||||
"type": "object",
|
||||
"required": ["channel", "target", "target_key"],
|
||||
"properties": {
|
||||
"channel": {
|
||||
"type": "string",
|
||||
"enum": ["postal", "internal_mail"]
|
||||
},
|
||||
"target": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 4000
|
||||
},
|
||||
"target_key": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 500
|
||||
},
|
||||
"contact_point_id": {
|
||||
"type": ["string", "null"],
|
||||
"maxLength": 36
|
||||
},
|
||||
"locale": {
|
||||
"type": ["string", "null"],
|
||||
"maxLength": 35
|
||||
},
|
||||
"decision_provenance": {
|
||||
"type": "object",
|
||||
"default": {}
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"print_delivery": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"template_id": {
|
||||
"type": ["string", "null"],
|
||||
"maxLength": 36
|
||||
},
|
||||
"template_revision": {
|
||||
"type": ["integer", "null"],
|
||||
"minimum": 1
|
||||
},
|
||||
"usage": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 100,
|
||||
"default": "campaign_print"
|
||||
},
|
||||
"output_format": {
|
||||
"type": "string",
|
||||
"enum": ["html", "text"],
|
||||
"default": "html"
|
||||
},
|
||||
"profile_id": {
|
||||
"type": ["string", "null"],
|
||||
"maxLength": 120
|
||||
},
|
||||
"persist_to_files": {
|
||||
"type": "boolean",
|
||||
"default": true
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"default": {}
|
||||
},
|
||||
"attachment_config": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -869,6 +1143,36 @@
|
||||
"default": true,
|
||||
"description": "Deprecated compatibility alias for merge_disposition_notification_to. New campaign JSON should use merge_*."
|
||||
},
|
||||
"channel_policy": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/delivery_channel_policy"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
"postbox_targets": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/postbox_target"
|
||||
},
|
||||
"maxItems": 50,
|
||||
"default": []
|
||||
},
|
||||
"merge_postbox_targets": {
|
||||
"type": "boolean",
|
||||
"default": true
|
||||
},
|
||||
"print_target": {
|
||||
"oneOf": [
|
||||
{ "$ref": "#/$defs/print_target" },
|
||||
{ "type": "null" }
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
"attachments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -885,6 +1189,11 @@
|
||||
"additionalProperties": true,
|
||||
"default": {}
|
||||
},
|
||||
"distribution_source": {
|
||||
"type": "object",
|
||||
"description": "Immutable Distribution List recipient, route-decision, and source evidence captured for this Campaign version.",
|
||||
"additionalProperties": true
|
||||
},
|
||||
"last_sent": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
@@ -971,7 +1280,8 @@
|
||||
"csv",
|
||||
"xlsx",
|
||||
"text",
|
||||
"addresses"
|
||||
"addresses",
|
||||
"distribution_list"
|
||||
]
|
||||
},
|
||||
"source_id": {
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationInfo, field_validator, model_validator
|
||||
from pydantic import (
|
||||
BaseModel,
|
||||
BeforeValidator,
|
||||
ConfigDict,
|
||||
Field,
|
||||
ValidationInfo,
|
||||
field_validator,
|
||||
model_validator,
|
||||
)
|
||||
|
||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||
@@ -25,8 +33,6 @@ class CampaignCreateRequest(BaseModel):
|
||||
source_base_path: str | None = None
|
||||
|
||||
|
||||
|
||||
|
||||
class CampaignUpdateRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
@@ -58,10 +64,15 @@ class CampaignVersionUpdateRequest(BaseModel):
|
||||
source_filename: str | None = None
|
||||
source_base_path: str | None = None
|
||||
migrate_legacy_mail_settings: bool = False
|
||||
base_revision: int | None = Field(default=None, ge=1)
|
||||
reconciliation_kind: Literal["none", "auto_merge", "manual"] = "none"
|
||||
resolved_conflict_paths: list[str] = Field(default_factory=list, max_length=100)
|
||||
|
||||
@field_validator("editor_state")
|
||||
@classmethod
|
||||
def validate_editor_state(cls, value: dict[str, Any] | None) -> dict[str, Any] | None:
|
||||
def validate_editor_state(
|
||||
cls, value: dict[str, Any] | None
|
||||
) -> dict[str, Any] | None:
|
||||
return validate_campaign_editor_state(value) if value is not None else None
|
||||
|
||||
|
||||
@@ -72,11 +83,23 @@ class CampaignVersionSetStepRequest(BaseModel):
|
||||
current_step: str
|
||||
|
||||
|
||||
class CampaignReviewDecisionRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
job_id: str = Field(min_length=1, max_length=36)
|
||||
decision: Literal["accept"] = "accept"
|
||||
reason: str | None = Field(default=None, max_length=4_000)
|
||||
|
||||
|
||||
class CampaignReviewStateRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
inspection_complete: bool = False
|
||||
reviewed_message_keys: list[str] = Field(default_factory=list)
|
||||
issue_decisions: list[CampaignReviewDecisionRequest] = Field(
|
||||
default_factory=list,
|
||||
max_length=100_000,
|
||||
)
|
||||
|
||||
|
||||
class CampaignPartialValidationRequest(BaseModel):
|
||||
@@ -92,6 +115,8 @@ class CampaignVersionResponse(BaseModel):
|
||||
id: str
|
||||
campaign_id: str
|
||||
version_number: int
|
||||
edit_revision: int = 1
|
||||
strong_etag: str = ""
|
||||
schema_version: str
|
||||
source_filename: str | None = None
|
||||
workflow_state: str = "editing"
|
||||
@@ -112,12 +137,16 @@ class CampaignVersionResponse(BaseModel):
|
||||
build_summary: dict[str, Any] | None = None
|
||||
execution_snapshot_hash: str | None = None
|
||||
execution_snapshot_at: datetime | None = None
|
||||
delivery_mode: Literal["synchronous", "worker_queue", "database_queue"] | None = None
|
||||
delivery_mode: Literal["synchronous", "worker_queue", "database_queue"] | None = (
|
||||
None
|
||||
)
|
||||
delivery_mode_selected_at: datetime | None = None
|
||||
|
||||
@field_validator("editor_state", mode="before")
|
||||
@classmethod
|
||||
def remove_unsupported_editor_state(cls, value: Any, info: ValidationInfo) -> dict[str, Any]:
|
||||
def remove_unsupported_editor_state(
|
||||
cls, value: Any, info: ValidationInfo
|
||||
) -> dict[str, Any]:
|
||||
return public_campaign_editor_state(
|
||||
value,
|
||||
include_diagnostics=bool((info.context or {}).get("include_diagnostics")),
|
||||
@@ -194,6 +223,10 @@ class CampaignDeltaResponse(BaseModel):
|
||||
watermark: str | None = None
|
||||
has_more: bool = False
|
||||
full: bool = False
|
||||
total: int = 0
|
||||
page: int = 1
|
||||
page_size: int = 500
|
||||
pages: int = 1
|
||||
|
||||
|
||||
class CampaignWorkspaceDeltaResponse(CampaignWorkspaceResponse):
|
||||
@@ -216,6 +249,10 @@ class CampaignShareItem(BaseModel):
|
||||
|
||||
class CampaignShareListResponse(BaseModel):
|
||||
shares: list[CampaignShareItem]
|
||||
total: int = 0
|
||||
page: int = 1
|
||||
page_size: int = 500
|
||||
pages: int = 1
|
||||
|
||||
|
||||
class CampaignShareTargetItem(BaseModel):
|
||||
@@ -266,7 +303,9 @@ class RecipientImportColumnMappingPayload(BaseModel):
|
||||
column_index: int = Field(ge=0, alias="columnIndex")
|
||||
kind: RecipientImportColumnKind
|
||||
field_name: str | None = Field(default=None, max_length=255, alias="fieldName")
|
||||
new_field_name: str | None = Field(default=None, max_length=255, alias="newFieldName")
|
||||
new_field_name: str | None = Field(
|
||||
default=None, max_length=255, alias="newFieldName"
|
||||
)
|
||||
|
||||
|
||||
class RecipientImportMappingProfilePayload(BaseModel):
|
||||
@@ -275,14 +314,22 @@ class RecipientImportMappingProfilePayload(BaseModel):
|
||||
name: str = Field(min_length=1, max_length=255)
|
||||
column_count: int = Field(ge=0, le=500, alias="columnCount")
|
||||
headers: list[str] = Field(default_factory=list, max_length=500)
|
||||
normalized_headers: list[str] = Field(default_factory=list, max_length=500, alias="normalizedHeaders")
|
||||
ordered_header_fingerprint: str = Field(min_length=1, max_length=64, alias="orderedHeaderFingerprint")
|
||||
unordered_header_fingerprint: str = Field(min_length=1, max_length=64, alias="unorderedHeaderFingerprint")
|
||||
normalized_headers: list[str] = Field(
|
||||
default_factory=list, max_length=500, alias="normalizedHeaders"
|
||||
)
|
||||
ordered_header_fingerprint: str = Field(
|
||||
min_length=1, max_length=64, alias="orderedHeaderFingerprint"
|
||||
)
|
||||
unordered_header_fingerprint: str = Field(
|
||||
min_length=1, max_length=64, alias="unorderedHeaderFingerprint"
|
||||
)
|
||||
delimiter: Literal[",", ";", "\t"]
|
||||
header_rows: int = Field(ge=0, le=10, alias="headerRows")
|
||||
quoted: bool = True
|
||||
value_separators: str = Field(default=",;|", max_length=50, alias="valueSeparators")
|
||||
mappings: list[RecipientImportColumnMappingPayload] = Field(default_factory=list, max_length=500)
|
||||
mappings: list[RecipientImportColumnMappingPayload] = Field(
|
||||
default_factory=list, max_length=500
|
||||
)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_column_shape(self) -> "RecipientImportMappingProfilePayload":
|
||||
@@ -342,10 +389,24 @@ class CampaignRecipientAddressSourcesResponse(BaseModel):
|
||||
sources: list[CampaignRecipientAddressSource] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CampaignPostboxCatalogResponse(BaseModel):
|
||||
available: bool = False
|
||||
postboxes: list[dict[str, Any]] = Field(default_factory=list)
|
||||
templates: list[dict[str, Any]] = Field(default_factory=list)
|
||||
organization_units: list[dict[str, Any]] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CampaignCalendarCatalogResponse(BaseModel):
|
||||
available: bool = False
|
||||
reason: str | None = None
|
||||
calendars: list[dict[str, Any]] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CampaignRecipientAddressSourceSnapshotRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
source_id: str = Field(min_length=1)
|
||||
purpose: str = Field(default="campaign_delivery", min_length=1, max_length=120)
|
||||
|
||||
|
||||
class CampaignRecipientSnapshotItem(BaseModel):
|
||||
@@ -357,6 +418,18 @@ class CampaignRecipientSnapshotItem(BaseModel):
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignRecipientSnapshotExcludedItem(BaseModel):
|
||||
contact_id: str
|
||||
display_name: str
|
||||
channel: str
|
||||
target: str
|
||||
contact_point_id: str | None = None
|
||||
status: str
|
||||
reason_code: str | None = None
|
||||
explanation: str | None = None
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignRecipientAddressSourceSnapshotResponse(BaseModel):
|
||||
source_id: str
|
||||
source_label: str
|
||||
@@ -364,9 +437,139 @@ class CampaignRecipientAddressSourceSnapshotResponse(BaseModel):
|
||||
source_revision: str
|
||||
generated_at: str
|
||||
recipients: list[CampaignRecipientSnapshotItem] = Field(default_factory=list)
|
||||
excluded: list[CampaignRecipientSnapshotExcludedItem] = Field(default_factory=list)
|
||||
included_count: int = 0
|
||||
excluded_count: int = 0
|
||||
purpose: str = "campaign_delivery"
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionListParameter(BaseModel):
|
||||
key: str
|
||||
value_type: str
|
||||
label: str | None = None
|
||||
required: bool = False
|
||||
default: Any = None
|
||||
allowed_values: list[Any] = Field(default_factory=list)
|
||||
minimum: float | None = None
|
||||
maximum: float | None = None
|
||||
pattern: str | None = None
|
||||
description: str | None = None
|
||||
|
||||
|
||||
class CampaignDistributionListSource(BaseModel):
|
||||
id: str
|
||||
tenant_id: str
|
||||
name: str
|
||||
revision_id: str
|
||||
revision: int
|
||||
definition_hash: str
|
||||
definition_kind: str = "static"
|
||||
description: str | None = None
|
||||
status: str = "active"
|
||||
entry_count: int = 0
|
||||
read_only: bool = False
|
||||
stale: bool = False
|
||||
parameters: list[CampaignDistributionListParameter] = Field(default_factory=list)
|
||||
updated_at: datetime | None = None
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionListSourcesResponse(BaseModel):
|
||||
available: bool = False
|
||||
expand_available: bool = False
|
||||
sources: list[CampaignDistributionListSource] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CampaignDistributionListExpansionRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
list_id: str = Field(min_length=1, max_length=36)
|
||||
revision: int | None = Field(default=None, ge=1)
|
||||
effective_at: datetime | None = None
|
||||
purpose: str = Field(default="campaign_delivery", min_length=1, max_length=120)
|
||||
requested_channels: list[Literal["email", "postal", "internal_mail", "portal"]] = Field(
|
||||
default_factory=list,
|
||||
max_length=4,
|
||||
)
|
||||
parameters: dict[str, Any] = Field(default_factory=dict)
|
||||
idempotency_key: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
|
||||
|
||||
class CampaignDistributionSourceReference(BaseModel):
|
||||
provider: str
|
||||
resource_type: str
|
||||
resource_id: str
|
||||
revision: str | None = None
|
||||
fingerprint: str | None = None
|
||||
label: str | None = None
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionExplanation(BaseModel):
|
||||
code: str
|
||||
message: str
|
||||
severity: str
|
||||
provider: str | None = None
|
||||
source: CampaignDistributionSourceReference | None = None
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionChannelCandidate(BaseModel):
|
||||
channel: str
|
||||
target: str
|
||||
target_key: str
|
||||
status: str
|
||||
contact_point_id: str | None = None
|
||||
locale: str | None = None
|
||||
preferred: bool = False
|
||||
reason_code: str | None = None
|
||||
explanation: str | None = None
|
||||
source: CampaignDistributionSourceReference | None = None
|
||||
decision_provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionRecipient(BaseModel):
|
||||
recipient_key: str
|
||||
display_name: str
|
||||
status: str
|
||||
channels: list[CampaignDistributionChannelCandidate] = Field(default_factory=list)
|
||||
identity_id: str | None = None
|
||||
account_id: str | None = None
|
||||
contact_id: str | None = None
|
||||
organization_unit_id: str | None = None
|
||||
function_id: str | None = None
|
||||
source_entry_ids: list[str] = Field(default_factory=list)
|
||||
explanations: list[CampaignDistributionExplanation] = Field(default_factory=list)
|
||||
attributes: dict[str, Any] = Field(default_factory=dict)
|
||||
provenance: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionProviderEvidence(BaseModel):
|
||||
provider: str
|
||||
source: CampaignDistributionSourceReference
|
||||
actual_revision: str | None = None
|
||||
actual_fingerprint: str | None = None
|
||||
stale: bool = False
|
||||
generated_at: datetime | None = None
|
||||
details: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CampaignDistributionListExpansionResponse(BaseModel):
|
||||
source: CampaignDistributionListSource
|
||||
request: dict[str, Any] = Field(default_factory=dict)
|
||||
recipients: list[CampaignDistributionRecipient] = Field(default_factory=list)
|
||||
excluded: list[CampaignDistributionRecipient] = Field(default_factory=list)
|
||||
diagnostics: list[CampaignDistributionExplanation] = Field(default_factory=list)
|
||||
provider_evidence: list[CampaignDistributionProviderEvidence] = Field(default_factory=list)
|
||||
expansion_hash: str
|
||||
generated_at: datetime | None = None
|
||||
snapshot_id: str | None = None
|
||||
stale: bool = False
|
||||
truncated: bool = False
|
||||
|
||||
|
||||
class CampaignJobsResponse(BaseModel):
|
||||
jobs: list[dict[str, Any]]
|
||||
page: int = 1
|
||||
@@ -425,11 +628,21 @@ class CampaignSendUnattemptedRequest(BaseModel):
|
||||
class CampaignSendJobRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
kind: Literal["test", "single_send", "single_resend"]
|
||||
idempotency_key: str = Field(min_length=1, max_length=200)
|
||||
reason: str | None = Field(default=None, max_length=2000)
|
||||
context: dict[str, Any] = Field(default_factory=dict)
|
||||
include_warnings: bool = True
|
||||
dry_run: bool = False
|
||||
use_rate_limit: bool = True
|
||||
enqueue_imap_task: bool = False
|
||||
|
||||
@model_validator(mode="after")
|
||||
def require_resend_reason(self):
|
||||
self.reason = (self.reason or "").strip() or None
|
||||
if self.kind == "single_resend" and not self.reason:
|
||||
raise ValueError("single_resend requires a reason")
|
||||
return self
|
||||
|
||||
|
||||
class CampaignResolveOutcomeRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
@@ -439,8 +652,11 @@ class CampaignResolveOutcomeRequest(BaseModel):
|
||||
"not_sent",
|
||||
"imap_appended",
|
||||
"imap_not_appended",
|
||||
"postbox_accepted",
|
||||
"postbox_not_accepted",
|
||||
]
|
||||
note: str | None = Field(default=None, max_length=2000)
|
||||
attempt_id: str | None = Field(default=None, max_length=36)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def require_reconciliation_evidence(self) -> "CampaignResolveOutcomeRequest":
|
||||
@@ -461,6 +677,56 @@ class BuildCampaignRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
write_eml: bool = True
|
||||
idempotency_key: str | None = Field(default=None, min_length=1, max_length=200)
|
||||
|
||||
|
||||
class CampaignArtifactReconcileRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
apply: bool = False
|
||||
idempotency_key: str | None = Field(default=None, min_length=1, max_length=200)
|
||||
grace_period_hours: int = Field(default=24, ge=24, le=24 * 90)
|
||||
cursor: str | None = Field(default=None, min_length=1, max_length=1000)
|
||||
page_size: int = Field(default=250, ge=1, le=1000)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def require_apply_idempotency_key(self) -> "CampaignArtifactReconcileRequest":
|
||||
if self.apply and not self.idempotency_key:
|
||||
raise ValueError("Applied artifact cleanup requires an idempotency key")
|
||||
return self
|
||||
|
||||
|
||||
class CampaignArtifactCandidateResponse(BaseModel):
|
||||
key: str
|
||||
size_bytes: int
|
||||
modified_at: datetime
|
||||
age_seconds: int
|
||||
reason: str
|
||||
disposition: str
|
||||
failure_type: str | None = None
|
||||
|
||||
|
||||
class CampaignArtifactReconcileResponse(BaseModel):
|
||||
apply: bool
|
||||
status: str
|
||||
recovery_operation_id: str | None = None
|
||||
tenant_prefix: str
|
||||
cursor: str | None = None
|
||||
next_cursor: str | None = None
|
||||
scanned_count: int
|
||||
scanned_bytes: int
|
||||
referenced_count: int
|
||||
active_build_count: int
|
||||
young_count: int
|
||||
unknown_age_count: int
|
||||
invalid_shape_count: int
|
||||
candidate_count: int
|
||||
candidate_bytes: int
|
||||
deleted_count: int
|
||||
deleted_bytes: int
|
||||
failure_count: int
|
||||
manifest_sha256: str | None = None
|
||||
candidates: list[CampaignArtifactCandidateResponse] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ApiKeyCreateRequest(BaseModel):
|
||||
@@ -520,6 +786,8 @@ class CampaignDeliveryOptionsResponse(BaseModel):
|
||||
campaign_id: str
|
||||
version_id: str
|
||||
worker_queue_available: bool
|
||||
postbox_available: bool = False
|
||||
approval_gate: dict[str, Any] = Field(default_factory=dict)
|
||||
synchronous_send: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
@@ -551,62 +819,71 @@ class CampaignActionResponse(BaseModel):
|
||||
result: dict[str, Any]
|
||||
|
||||
|
||||
def _valid_report_email_domain(domain: str) -> bool:
|
||||
if not domain or domain.startswith(".") or domain.endswith(".") or ".." in domain:
|
||||
return False
|
||||
return all(
|
||||
label
|
||||
and not label.startswith("-")
|
||||
and not label.endswith("-")
|
||||
and all(character.isalnum() or character == "-" for character in label)
|
||||
for label in domain.split(".")
|
||||
)
|
||||
|
||||
|
||||
def _normalize_report_recipient(value: Any) -> str:
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("report recipients must be email-address strings")
|
||||
recipient = value.strip()
|
||||
if len(recipient) > 320:
|
||||
raise ValueError("report recipient addresses must be at most 320 characters")
|
||||
if any(ord(character) < 32 or ord(character) == 127 for character in recipient):
|
||||
raise ValueError(
|
||||
"report recipient addresses must not contain control characters"
|
||||
)
|
||||
if recipient.count("@") != 1:
|
||||
raise ValueError("report recipients must be email addresses")
|
||||
local, domain = recipient.split("@", 1)
|
||||
invalid_local = (
|
||||
not local or local.startswith(".") or local.endswith(".") or ".." in local
|
||||
)
|
||||
invalid_address = any(character.isspace() for character in recipient) or any(
|
||||
character in ',;:<>[]()\\"' for character in recipient
|
||||
)
|
||||
if invalid_local or invalid_address or not _valid_report_email_domain(domain):
|
||||
raise ValueError("report recipients must be email addresses")
|
||||
return recipient
|
||||
|
||||
|
||||
ReportEmailAddress = Annotated[str, BeforeValidator(_normalize_report_recipient)]
|
||||
|
||||
|
||||
def _deduplicate_report_recipients(value: list[str]) -> list[str]:
|
||||
recipients: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for recipient in value:
|
||||
key = recipient.casefold()
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
recipients.append(recipient)
|
||||
return recipients
|
||||
|
||||
|
||||
class ReportEmailRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
to: list[str] = Field(min_length=1, max_length=50)
|
||||
to: list[ReportEmailAddress] = Field(min_length=1, max_length=50)
|
||||
version_id: str | None = None
|
||||
include_jobs: bool = False
|
||||
attach_jobs_csv: bool = False
|
||||
attach_report_json: bool = False
|
||||
dry_run: bool = False
|
||||
idempotency_key: str | None = Field(default=None, min_length=1, max_length=200)
|
||||
|
||||
@field_validator("to", mode="before")
|
||||
@field_validator("to")
|
||||
@classmethod
|
||||
def normalize_and_validate_recipients(cls, value: Any) -> Any:
|
||||
if not isinstance(value, list):
|
||||
return value
|
||||
if not 1 <= len(value) <= 50:
|
||||
raise ValueError("report email requires between 1 and 50 recipients")
|
||||
recipients: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for item in value:
|
||||
if not isinstance(item, str):
|
||||
raise ValueError("report recipients must be email-address strings")
|
||||
recipient = item.strip()
|
||||
if len(recipient) > 320:
|
||||
raise ValueError("report recipient addresses must be at most 320 characters")
|
||||
if any(ord(character) < 32 or ord(character) == 127 for character in recipient):
|
||||
raise ValueError("report recipient addresses must not contain control characters")
|
||||
if recipient.count("@") != 1:
|
||||
raise ValueError("report recipients must be email addresses")
|
||||
local, domain = recipient.split("@", 1)
|
||||
if (
|
||||
not local
|
||||
or not domain
|
||||
or any(character.isspace() for character in recipient)
|
||||
or any(character in ',;:<>[]()\\"' for character in recipient)
|
||||
or local.startswith(".")
|
||||
or local.endswith(".")
|
||||
or ".." in local
|
||||
or domain.startswith(".")
|
||||
or domain.endswith(".")
|
||||
or ".." in domain
|
||||
or any(
|
||||
not label
|
||||
or label.startswith("-")
|
||||
or label.endswith("-")
|
||||
or not all(character.isalnum() or character == "-" for character in label)
|
||||
for label in domain.split(".")
|
||||
)
|
||||
):
|
||||
raise ValueError("report recipients must be email addresses")
|
||||
key = recipient.casefold()
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
recipients.append(recipient)
|
||||
return recipients
|
||||
def normalize_and_validate_recipients(cls, value: list[str]) -> list[str]:
|
||||
return _deduplicate_report_recipients(value)
|
||||
|
||||
|
||||
class ReportEmailResponse(BaseModel):
|
||||
|
||||
@@ -9,16 +9,21 @@ from pydantic import BaseModel, ConfigDict
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion, JobValidationStatus
|
||||
from govoplan_campaign.backend.campaign.models import DeliveryConfig
|
||||
from govoplan_campaign.backend.campaign.models import (
|
||||
DeliveryChannelPolicy,
|
||||
DeliveryConfig,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||
CampaignMailProfileBoundaryError,
|
||||
assert_campaign_uses_mail_profile_reference,
|
||||
campaign_mail_profile_id,
|
||||
campaign_mail_resource_ids,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import MailProfileError, files_integration, mail_integration
|
||||
from govoplan_campaign.backend.path_security import CampaignPathSecurityError, assert_server_safe_campaign_paths
|
||||
|
||||
SNAPSHOT_VERSION = "5"
|
||||
SNAPSHOT_VERSION = "8"
|
||||
SUPPORTED_SNAPSHOT_VERSIONS = {"6", "7", SNAPSHOT_VERSION}
|
||||
|
||||
|
||||
class ExecutionSnapshotError(RuntimeError):
|
||||
@@ -40,7 +45,11 @@ class ExecutionSnapshot(BaseModel):
|
||||
snapshot_version: str = SNAPSHOT_VERSION
|
||||
campaign_version_id: str
|
||||
campaign_json_sha256: str
|
||||
mail_profile_id: str
|
||||
mail_profile_id: str | None = None
|
||||
smtp_server_id: str | None = None
|
||||
smtp_credential_id: str | None = None
|
||||
imap_server_id: str | None = None
|
||||
imap_credential_id: str | None = None
|
||||
created_at: str
|
||||
build_token: str | None = None
|
||||
built_at: str | None = None
|
||||
@@ -50,6 +59,9 @@ class ExecutionSnapshot(BaseModel):
|
||||
effective_policy_sha256: str | None = None
|
||||
smtp_transport_revision: str | None = None
|
||||
imap_transport_revision: str | None = None
|
||||
uses_mail: bool = True
|
||||
uses_postbox: bool = False
|
||||
uses_print: bool = False
|
||||
delivery: DeliveryConfig
|
||||
|
||||
|
||||
@@ -67,7 +79,7 @@ def snapshot_hash(payload: dict[str, Any]) -> str:
|
||||
|
||||
def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict[str, Any]:
|
||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
_assert_version_mail_profile_boundary(raw_json)
|
||||
_assert_version_mail_profile_boundary(raw_json, require_profile=True)
|
||||
mail = mail_integration()
|
||||
profile_id = campaign_mail_profile_id(raw_json)
|
||||
if profile_id is None: # Kept explicit for static typing; the assertion above requires it.
|
||||
@@ -75,12 +87,17 @@ def profile_delivery_summary(session: Session, version: CampaignVersion) -> dict
|
||||
campaign = session.get(Campaign, version.campaign_id)
|
||||
if campaign is None:
|
||||
raise ExecutionSnapshotError("Campaign not found for mail-server profile resolution")
|
||||
references = campaign_mail_resource_ids(raw_json)
|
||||
try:
|
||||
return mail.campaign_profile_delivery_summary(
|
||||
session,
|
||||
tenant_id=campaign.tenant_id,
|
||||
campaign_id=campaign.id,
|
||||
profile_id=profile_id,
|
||||
smtp_server_id=references["smtp_server_id"],
|
||||
smtp_credential_id=references["smtp_credential_id"],
|
||||
imap_server_id=references["imap_server_id"],
|
||||
imap_credential_id=references["imap_credential_id"],
|
||||
)
|
||||
except MailProfileError as exc:
|
||||
raise ExecutionSnapshotError(str(exc)) from exc
|
||||
@@ -96,27 +113,63 @@ def profile_transport_revisions(session: Session, version: CampaignVersion) -> d
|
||||
|
||||
def _assert_snapshot_profile_matches_version(version: CampaignVersion, snapshot: ExecutionSnapshot) -> None:
|
||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
_assert_version_mail_profile_boundary(raw_json)
|
||||
_assert_version_mail_profile_boundary(
|
||||
raw_json,
|
||||
require_profile=snapshot.uses_mail,
|
||||
)
|
||||
if not snapshot.uses_mail:
|
||||
return
|
||||
if campaign_mail_profile_id(raw_json) != snapshot.mail_profile_id:
|
||||
raise ExecutionSnapshotError(
|
||||
"The campaign's Mail profile reference differs from the built execution snapshot. "
|
||||
"Revalidate and rebuild the campaign before delivery."
|
||||
)
|
||||
references = campaign_mail_resource_ids(raw_json)
|
||||
for key in (
|
||||
"smtp_server_id",
|
||||
"smtp_credential_id",
|
||||
"imap_server_id",
|
||||
"imap_credential_id",
|
||||
):
|
||||
configured = references[key]
|
||||
if configured and configured != getattr(snapshot, key):
|
||||
raise ExecutionSnapshotError(
|
||||
"The campaign's Mail server or credential selection differs from the built "
|
||||
"execution snapshot. Revalidate and rebuild before delivery."
|
||||
)
|
||||
|
||||
|
||||
def _assert_version_mail_profile_boundary(raw_json: dict[str, Any]) -> None:
|
||||
def _assert_version_mail_profile_boundary(
|
||||
raw_json: dict[str, Any],
|
||||
*,
|
||||
require_profile: bool,
|
||||
) -> None:
|
||||
try:
|
||||
assert_campaign_uses_mail_profile_reference(raw_json, require_profile=True)
|
||||
assert_campaign_uses_mail_profile_reference(
|
||||
raw_json,
|
||||
require_profile=require_profile,
|
||||
)
|
||||
except CampaignMailProfileBoundaryError as exc:
|
||||
raise ExecutionSnapshotError(str(exc)) from exc
|
||||
|
||||
|
||||
def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> str:
|
||||
def _policy_fingerprint(
|
||||
raw_json: dict[str, Any],
|
||||
delivery: DeliveryConfig,
|
||||
*,
|
||||
snapshot_version: str = SNAPSHOT_VERSION,
|
||||
) -> str:
|
||||
delivery_payload = delivery.model_dump(mode="json")
|
||||
if snapshot_version == "6":
|
||||
delivery_payload.pop("channel_policy", None)
|
||||
delivery_payload.pop("postbox", None)
|
||||
if snapshot_version in {"6", "7"}:
|
||||
delivery_payload.pop("print", None)
|
||||
return _sha256(
|
||||
{
|
||||
"validation_policy": raw_json.get("validation_policy"),
|
||||
"policy": raw_json.get("policy"),
|
||||
"delivery": delivery.model_dump(mode="json"),
|
||||
"delivery": delivery_payload,
|
||||
"attachment_defaults": (raw_json.get("attachments") or {}).get("defaults")
|
||||
if isinstance(raw_json.get("attachments"), dict)
|
||||
else None,
|
||||
@@ -124,8 +177,12 @@ def _policy_fingerprint(raw_json: dict[str, Any], delivery: DeliveryConfig) -> s
|
||||
)
|
||||
|
||||
|
||||
def _job_execution_input_payload(job: CampaignJob) -> dict[str, Any]:
|
||||
return {
|
||||
def _job_execution_input_payload(
|
||||
job: CampaignJob,
|
||||
*,
|
||||
snapshot_version: str = SNAPSHOT_VERSION,
|
||||
) -> dict[str, Any]:
|
||||
payload = {
|
||||
"job_id": job.id,
|
||||
"entry_index": job.entry_index,
|
||||
"entry_id": job.entry_id,
|
||||
@@ -140,17 +197,54 @@ def _job_execution_input_payload(job: CampaignJob) -> dict[str, Any]:
|
||||
"resolved_attachments_sha256": _sha256(job.resolved_attachments or []),
|
||||
"issues_sha256": _sha256(job.issues_snapshot or []),
|
||||
}
|
||||
if snapshot_version != "6":
|
||||
payload.update(
|
||||
{
|
||||
"delivery_channel_policy": getattr(
|
||||
job,
|
||||
"delivery_channel_policy",
|
||||
DeliveryChannelPolicy.MAIL.value,
|
||||
),
|
||||
"resolved_postbox_targets_sha256": _sha256(
|
||||
getattr(job, "resolved_postbox_targets", None) or []
|
||||
),
|
||||
}
|
||||
)
|
||||
if snapshot_version not in {"6", "7"}:
|
||||
payload["delivery_provenance_sha256"] = _sha256(
|
||||
getattr(job, "delivery_provenance", None) or {}
|
||||
)
|
||||
payload["resolved_print_output_sha256"] = _sha256(
|
||||
getattr(job, "resolved_print_output", None) or {}
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
def job_execution_input_hash(job: CampaignJob) -> str:
|
||||
return _sha256(_job_execution_input_payload(job))
|
||||
def job_execution_input_hash(
|
||||
job: CampaignJob,
|
||||
*,
|
||||
snapshot_version: str = SNAPSHOT_VERSION,
|
||||
) -> str:
|
||||
return _sha256(
|
||||
_job_execution_input_payload(
|
||||
job,
|
||||
snapshot_version=snapshot_version,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def job_manifest_hash(jobs: Iterable[CampaignJob]) -> str:
|
||||
def job_manifest_hash(
|
||||
jobs: Iterable[CampaignJob],
|
||||
*,
|
||||
snapshot_version: str = SNAPSHOT_VERSION,
|
||||
) -> str:
|
||||
"""Hash the immutable per-message execution records in stable order."""
|
||||
|
||||
payload = [
|
||||
_job_execution_input_payload(job)
|
||||
_job_execution_input_payload(
|
||||
job,
|
||||
snapshot_version=snapshot_version,
|
||||
)
|
||||
for job in sorted(jobs, key=lambda item: (item.entry_index, item.id))
|
||||
]
|
||||
return _sha256(payload)
|
||||
@@ -159,31 +253,69 @@ def job_manifest_hash(jobs: Iterable[CampaignJob]) -> str:
|
||||
def create_execution_snapshot(
|
||||
version: CampaignVersion,
|
||||
*,
|
||||
mail_profile_id: str,
|
||||
smtp_transport_revision: str,
|
||||
mail_profile_id: str | None,
|
||||
smtp_transport_revision: str | None,
|
||||
imap_transport_revision: str | None,
|
||||
delivery: DeliveryConfig,
|
||||
smtp_server_id: str | None = None,
|
||||
smtp_credential_id: str | None = None,
|
||||
imap_server_id: str | None = None,
|
||||
imap_credential_id: str | None = None,
|
||||
jobs: Iterable[CampaignJob] = (),
|
||||
build_summary: dict[str, Any] | None = None,
|
||||
) -> tuple[dict[str, Any], str]:
|
||||
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
|
||||
job_list = list(jobs)
|
||||
channel_policies = {
|
||||
DeliveryChannelPolicy(
|
||||
getattr(
|
||||
job,
|
||||
"delivery_channel_policy",
|
||||
DeliveryChannelPolicy.MAIL.value,
|
||||
)
|
||||
)
|
||||
for job in job_list
|
||||
}
|
||||
uses_mail = any(policy.uses_mail for policy in channel_policies)
|
||||
uses_postbox = any(policy.uses_postbox for policy in channel_policies)
|
||||
uses_print = any(policy.uses_print for policy in channel_policies)
|
||||
for job in job_list:
|
||||
job.execution_input_sha256 = job_execution_input_hash(job)
|
||||
job.execution_input_sha256 = job_execution_input_hash(
|
||||
job,
|
||||
snapshot_version=SNAPSHOT_VERSION,
|
||||
)
|
||||
summary = build_summary if isinstance(build_summary, dict) else {}
|
||||
queueable_statuses = {JobValidationStatus.READY.value, JobValidationStatus.WARNING.value}
|
||||
payload = ExecutionSnapshot(
|
||||
campaign_version_id=version.id,
|
||||
campaign_json_sha256=_sha256(raw_json),
|
||||
mail_profile_id=mail_profile_id,
|
||||
smtp_server_id=smtp_server_id,
|
||||
smtp_credential_id=smtp_credential_id,
|
||||
imap_server_id=imap_server_id,
|
||||
imap_credential_id=imap_credential_id,
|
||||
build_token=str(summary.get("build_token") or "") or None,
|
||||
built_at=str(summary.get("built_at") or "") or None,
|
||||
job_count=len(job_list),
|
||||
queueable_job_count=sum(1 for job in job_list if job.validation_status in queueable_statuses),
|
||||
job_manifest_sha256=job_manifest_hash(job_list) if job_list else None,
|
||||
effective_policy_sha256=_policy_fingerprint(raw_json, delivery),
|
||||
job_manifest_sha256=(
|
||||
job_manifest_hash(
|
||||
job_list,
|
||||
snapshot_version=SNAPSHOT_VERSION,
|
||||
)
|
||||
if job_list
|
||||
else None
|
||||
),
|
||||
effective_policy_sha256=_policy_fingerprint(
|
||||
raw_json,
|
||||
delivery,
|
||||
snapshot_version=SNAPSHOT_VERSION,
|
||||
),
|
||||
smtp_transport_revision=smtp_transport_revision,
|
||||
imap_transport_revision=imap_transport_revision,
|
||||
uses_mail=uses_mail,
|
||||
uses_postbox=uses_postbox,
|
||||
uses_print=uses_print,
|
||||
created_at=datetime.now(timezone.utc).isoformat(),
|
||||
delivery=delivery,
|
||||
).model_dump(mode="json")
|
||||
@@ -207,13 +339,17 @@ def _assert_snapshot_matches_persisted_inputs(
|
||||
"Campaign inputs changed after this execution snapshot was built. "
|
||||
"Revalidate and rebuild the campaign before delivery."
|
||||
)
|
||||
if not snapshot.smtp_transport_revision:
|
||||
if snapshot.uses_mail and not snapshot.smtp_transport_revision:
|
||||
raise ExecutionSnapshotError("Execution snapshot has no SMTP transport revision")
|
||||
if not snapshot.job_manifest_sha256:
|
||||
raise ExecutionSnapshotError("Execution snapshot has no built-job manifest checksum")
|
||||
if not snapshot.effective_policy_sha256:
|
||||
raise ExecutionSnapshotError("Execution snapshot has no effective-policy checksum")
|
||||
if snapshot.effective_policy_sha256 != _policy_fingerprint(raw_json, snapshot.delivery):
|
||||
if snapshot.effective_policy_sha256 != _policy_fingerprint(
|
||||
raw_json,
|
||||
snapshot.delivery,
|
||||
snapshot_version=snapshot.snapshot_version,
|
||||
):
|
||||
raise ExecutionSnapshotError(
|
||||
"Campaign delivery policy changed after the execution snapshot was created. "
|
||||
"Revalidate and rebuild the campaign before delivery."
|
||||
@@ -224,7 +360,10 @@ def _assert_snapshot_matches_persisted_inputs(
|
||||
raise ExecutionSnapshotError("Campaign job does not belong to the snapshotted version")
|
||||
if not getattr(effect_job, "execution_input_sha256", None):
|
||||
raise ExecutionSnapshotError("Campaign job has no execution-input checksum; rebuild before delivery")
|
||||
if effect_job.execution_input_sha256 != job_execution_input_hash(effect_job):
|
||||
if effect_job.execution_input_sha256 != job_execution_input_hash(
|
||||
effect_job,
|
||||
snapshot_version=snapshot.snapshot_version,
|
||||
):
|
||||
raise ExecutionSnapshotError(
|
||||
"Built campaign job inputs changed after the execution snapshot was created. "
|
||||
"Revalidate and rebuild the campaign before delivery."
|
||||
@@ -246,8 +385,19 @@ def _assert_snapshot_matches_persisted_inputs(
|
||||
queueable_count = sum(1 for job in jobs if job.validation_status in queueable_statuses)
|
||||
if (
|
||||
snapshot.queueable_job_count != queueable_count
|
||||
or snapshot.job_manifest_sha256 != job_manifest_hash(jobs)
|
||||
or any(getattr(job, "execution_input_sha256", None) != job_execution_input_hash(job) for job in jobs)
|
||||
or snapshot.job_manifest_sha256
|
||||
!= job_manifest_hash(
|
||||
jobs,
|
||||
snapshot_version=snapshot.snapshot_version,
|
||||
)
|
||||
or any(
|
||||
getattr(job, "execution_input_sha256", None)
|
||||
!= job_execution_input_hash(
|
||||
job,
|
||||
snapshot_version=snapshot.snapshot_version,
|
||||
)
|
||||
for job in jobs
|
||||
)
|
||||
):
|
||||
raise ExecutionSnapshotError(
|
||||
"Built campaign job inputs changed after the execution snapshot was created. "
|
||||
@@ -276,17 +426,20 @@ def ensure_execution_snapshot(
|
||||
)
|
||||
except CampaignPathSecurityError as exc:
|
||||
raise ExecutionSnapshotError(str(exc)) from exc
|
||||
_assert_version_mail_profile_boundary(raw_json)
|
||||
_assert_version_mail_profile_boundary(raw_json, require_profile=False)
|
||||
|
||||
if isinstance(version.execution_snapshot, dict):
|
||||
if str(version.execution_snapshot.get("snapshot_version") or "") != SNAPSHOT_VERSION:
|
||||
stored_version = str(
|
||||
version.execution_snapshot.get("snapshot_version") or ""
|
||||
)
|
||||
if stored_version not in SUPPORTED_SNAPSHOT_VERSIONS:
|
||||
raise ExecutionSnapshotError(
|
||||
"This campaign has a legacy execution snapshot that may contain campaign-owned transport data. "
|
||||
"It is preserved for audit only and cannot be delivered; select a Mail profile, then revalidate "
|
||||
"and rebuild a new campaign version."
|
||||
)
|
||||
snapshot = ExecutionSnapshot.model_validate(version.execution_snapshot)
|
||||
expected = snapshot_hash(snapshot.model_dump(mode="json"))
|
||||
expected = snapshot_hash(version.execution_snapshot)
|
||||
if not version.execution_snapshot_hash:
|
||||
raise ExecutionSnapshotError("Execution snapshot checksum is missing")
|
||||
if version.execution_snapshot_hash != expected:
|
||||
@@ -303,11 +456,6 @@ def ensure_execution_snapshot(
|
||||
from govoplan_campaign.backend.persistence.campaigns import load_version_config
|
||||
|
||||
_, _, config = load_version_config(session, version.id)
|
||||
profile_id = campaign_mail_profile_id(raw_json)
|
||||
if not config.server.profile_capabilities.smtp_available:
|
||||
raise ExecutionSnapshotError("The selected Mail profile has no SMTP configuration")
|
||||
if profile_id is None:
|
||||
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
|
||||
jobs = (
|
||||
session.query(CampaignJob)
|
||||
.filter(CampaignJob.campaign_version_id == version.id)
|
||||
@@ -316,14 +464,33 @@ def ensure_execution_snapshot(
|
||||
)
|
||||
if not jobs:
|
||||
raise ExecutionSnapshotError("Campaign version has no built jobs; rebuild it before delivery")
|
||||
revisions = profile_transport_revisions(session, version)
|
||||
if not revisions["smtp"]:
|
||||
raise ExecutionSnapshotError("The selected Mail profile has no SMTP transport revision")
|
||||
uses_mail = any(
|
||||
DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail
|
||||
for job in jobs
|
||||
)
|
||||
profile_id = campaign_mail_profile_id(raw_json)
|
||||
summary: dict[str, Any] = {}
|
||||
if uses_mail:
|
||||
if not config.server.profile_capabilities.smtp_available:
|
||||
raise ExecutionSnapshotError(
|
||||
"The selected Mail profile has no SMTP configuration"
|
||||
)
|
||||
if profile_id is None:
|
||||
raise ExecutionSnapshotError("Campaign has no Mail profile reference")
|
||||
summary = profile_delivery_summary(session, version)
|
||||
if not summary.get("smtp_transport_revision"):
|
||||
raise ExecutionSnapshotError(
|
||||
"The selected Mail profile has no SMTP transport revision"
|
||||
)
|
||||
payload, digest = create_execution_snapshot(
|
||||
version,
|
||||
mail_profile_id=profile_id,
|
||||
smtp_transport_revision=revisions["smtp"],
|
||||
imap_transport_revision=revisions["imap"],
|
||||
smtp_server_id=summary.get("smtp_server_id"),
|
||||
smtp_credential_id=summary.get("smtp_credential_id"),
|
||||
imap_server_id=summary.get("imap_server_id"),
|
||||
imap_credential_id=summary.get("imap_credential_id"),
|
||||
smtp_transport_revision=summary.get("smtp_transport_revision"),
|
||||
imap_transport_revision=summary.get("imap_transport_revision"),
|
||||
delivery=config.delivery,
|
||||
jobs=jobs,
|
||||
build_summary=version.build_summary if isinstance(version.build_summary, dict) else {},
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,506 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from email import policy
|
||||
from email.parser import BytesParser
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import func
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.postbox import (
|
||||
PostboxAttachmentRef,
|
||||
PostboxDeliveryOutcomeUnknown,
|
||||
PostboxDeliveryRejected,
|
||||
PostboxDeliveryRequest,
|
||||
PostboxParticipantRef,
|
||||
PostboxTargetRef,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignJob,
|
||||
JobPostboxStatus,
|
||||
PostboxDeliveryAttempt,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
PostboxDeliveryUnavailable,
|
||||
postbox_integration,
|
||||
)
|
||||
from govoplan_core.security.time import utc_now
|
||||
|
||||
|
||||
ACCEPTED_POSTBOX_ATTEMPT_STATUSES = {
|
||||
JobPostboxStatus.ACCEPTED.value,
|
||||
JobPostboxStatus.ACCEPTED_VACANT.value,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class PostboxChannelOutcome:
|
||||
accepted: int = 0
|
||||
accepted_vacant: int = 0
|
||||
rejected_temporary: int = 0
|
||||
rejected_permanent: int = 0
|
||||
outcome_unknown: int = 0
|
||||
messages: list[str] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def accepted_count(self) -> int:
|
||||
return self.accepted + self.accepted_vacant
|
||||
|
||||
@property
|
||||
def rejected_count(self) -> int:
|
||||
return self.rejected_temporary + self.rejected_permanent
|
||||
|
||||
@property
|
||||
def all_rejected_before_acceptance(self) -> bool:
|
||||
return (
|
||||
self.accepted_count == 0
|
||||
and self.outcome_unknown == 0
|
||||
and self.rejected_count > 0
|
||||
)
|
||||
|
||||
@property
|
||||
def status(self) -> str:
|
||||
if self.outcome_unknown:
|
||||
return JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||
if self.accepted_count and self.rejected_count:
|
||||
return JobPostboxStatus.PARTIALLY_ACCEPTED.value
|
||||
if self.accepted_vacant and not self.accepted:
|
||||
return JobPostboxStatus.ACCEPTED_VACANT.value
|
||||
if self.accepted_count:
|
||||
return JobPostboxStatus.ACCEPTED.value
|
||||
if self.rejected_temporary:
|
||||
return JobPostboxStatus.REJECTED_TEMPORARY.value
|
||||
return JobPostboxStatus.REJECTED_PERMANENT.value
|
||||
|
||||
|
||||
def _target_key(target: dict[str, Any]) -> str:
|
||||
payload = json.dumps(
|
||||
target,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
return hashlib.sha256(payload).hexdigest()
|
||||
|
||||
|
||||
def _attempt_number(
|
||||
session: Session,
|
||||
*,
|
||||
job_id: str,
|
||||
target_key: str,
|
||||
) -> int:
|
||||
current = (
|
||||
session.query(func.max(PostboxDeliveryAttempt.attempt_number))
|
||||
.filter(
|
||||
PostboxDeliveryAttempt.job_id == job_id,
|
||||
PostboxDeliveryAttempt.target_key == target_key,
|
||||
)
|
||||
.scalar()
|
||||
)
|
||||
return int(current or 0) + 1
|
||||
|
||||
|
||||
def _accepted_attempt(
|
||||
session: Session,
|
||||
*,
|
||||
job_id: str,
|
||||
target_key: str,
|
||||
) -> PostboxDeliveryAttempt | None:
|
||||
return (
|
||||
session.query(PostboxDeliveryAttempt)
|
||||
.filter(
|
||||
PostboxDeliveryAttempt.job_id == job_id,
|
||||
PostboxDeliveryAttempt.target_key == target_key,
|
||||
PostboxDeliveryAttempt.status.in_(
|
||||
list(ACCEPTED_POSTBOX_ATTEMPT_STATUSES)
|
||||
),
|
||||
)
|
||||
.order_by(PostboxDeliveryAttempt.attempt_number.desc())
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def _message_body(message_bytes: bytes) -> str | None:
|
||||
message = BytesParser(policy=policy.default).parsebytes(message_bytes)
|
||||
body = message.get_body(preferencelist=("plain", "html"))
|
||||
if body is None:
|
||||
payload = message.get_payload(decode=True)
|
||||
if not isinstance(payload, bytes):
|
||||
return None
|
||||
return payload.decode(message.get_content_charset() or "utf-8", "replace")
|
||||
try:
|
||||
content = body.get_content()
|
||||
except (LookupError, UnicodeError):
|
||||
payload = body.get_payload(decode=True)
|
||||
if not isinstance(payload, bytes):
|
||||
return None
|
||||
return payload.decode(body.get_content_charset() or "utf-8", "replace")
|
||||
return str(content)
|
||||
|
||||
|
||||
def _participants(job: CampaignJob) -> tuple[PostboxParticipantRef, ...]:
|
||||
recipients = job.resolved_recipients or {}
|
||||
values: list[PostboxParticipantRef] = []
|
||||
for key in (
|
||||
"from_all",
|
||||
"to",
|
||||
"cc",
|
||||
"bcc",
|
||||
"reply_to",
|
||||
"bounce_to",
|
||||
"disposition_notification_to",
|
||||
):
|
||||
for item in recipients.get(key) or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
address = str(item.get("email") or "").strip() or None
|
||||
label = str(item.get("name") or "").strip() or None
|
||||
if address is None and label is None:
|
||||
continue
|
||||
values.append(
|
||||
PostboxParticipantRef(
|
||||
kind="sender" if key == "from_all" else key,
|
||||
reference_type="mail_address",
|
||||
label=label,
|
||||
address=address,
|
||||
)
|
||||
)
|
||||
return tuple(values)
|
||||
|
||||
|
||||
def _attachments(job: CampaignJob) -> tuple[PostboxAttachmentRef, ...]:
|
||||
values = [_eml_attachment(job)]
|
||||
for rule_index, rule in enumerate(job.resolved_attachments or []):
|
||||
if not isinstance(rule, dict):
|
||||
continue
|
||||
managed_matches = rule.get("managed_matches")
|
||||
if isinstance(managed_matches, list) and managed_matches:
|
||||
values.extend(_managed_attachments(managed_matches))
|
||||
continue
|
||||
matches = rule.get("matches")
|
||||
if isinstance(matches, list):
|
||||
values.extend(_campaign_attachments(job, rule_index=rule_index, matches=matches))
|
||||
return tuple(values)
|
||||
|
||||
|
||||
def _eml_attachment(job: CampaignJob) -> PostboxAttachmentRef:
|
||||
return PostboxAttachmentRef(
|
||||
reference_type="campaign_eml",
|
||||
reference_id=job.id,
|
||||
name=f"{job.entry_id or job.entry_index}.eml",
|
||||
media_type="message/rfc822",
|
||||
size_bytes=job.eml_size_bytes,
|
||||
digest=job.eml_sha256,
|
||||
metadata={"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id},
|
||||
)
|
||||
|
||||
|
||||
MANAGED_ATTACHMENT_METADATA_KEYS = {
|
||||
"asset_id",
|
||||
"version_id",
|
||||
"blob_id",
|
||||
"display_path",
|
||||
"relative_path",
|
||||
"owner_type",
|
||||
"owner_id",
|
||||
"source_revision",
|
||||
}
|
||||
|
||||
|
||||
def _managed_attachment(match: dict[str, Any]) -> PostboxAttachmentRef | None:
|
||||
reference_id = str(match.get("version_id") or match.get("asset_id") or match.get("blob_id") or "").strip()
|
||||
if not reference_id:
|
||||
return None
|
||||
return PostboxAttachmentRef(
|
||||
reference_type="file_version" if match.get("version_id") else "file_asset",
|
||||
reference_id=reference_id,
|
||||
name=str(match.get("filename") or "").strip() or None,
|
||||
media_type=str(match.get("content_type")) if match.get("content_type") else None,
|
||||
size_bytes=int(match["size_bytes"]) if match.get("size_bytes") is not None else None,
|
||||
digest=str(match.get("checksum_sha256")) if match.get("checksum_sha256") else None,
|
||||
metadata={key: value for key, value in match.items() if key in MANAGED_ATTACHMENT_METADATA_KEYS},
|
||||
)
|
||||
|
||||
|
||||
def _managed_attachments(matches: list[Any]) -> list[PostboxAttachmentRef]:
|
||||
attachments = (_managed_attachment(match) for match in matches if isinstance(match, dict))
|
||||
return [attachment for attachment in attachments if attachment is not None]
|
||||
|
||||
|
||||
def _campaign_attachments(
|
||||
job: CampaignJob,
|
||||
*,
|
||||
rule_index: int,
|
||||
matches: list[Any],
|
||||
) -> list[PostboxAttachmentRef]:
|
||||
metadata = {"campaign_id": job.campaign_id, "campaign_version_id": job.campaign_version_id, "job_id": job.id}
|
||||
return [
|
||||
PostboxAttachmentRef(
|
||||
reference_type="campaign_attachment",
|
||||
reference_id=f"{job.id}:{rule_index}:{match_index}",
|
||||
name=str(match).rsplit("/", 1)[-1] or None,
|
||||
metadata=metadata,
|
||||
)
|
||||
for match_index, match in enumerate(matches)
|
||||
]
|
||||
|
||||
|
||||
def _sender_label(job: CampaignJob) -> str | None:
|
||||
recipients = job.resolved_recipients or {}
|
||||
sender = recipients.get("from")
|
||||
if not isinstance(sender, dict):
|
||||
return None
|
||||
name = str(sender.get("name") or "").strip()
|
||||
address = str(sender.get("email") or "").strip()
|
||||
if name and address:
|
||||
return f"{name} <{address}>"
|
||||
return address or name or None
|
||||
|
||||
|
||||
def _request(
|
||||
job: CampaignJob,
|
||||
target: dict[str, Any],
|
||||
*,
|
||||
target_key: str,
|
||||
body_text: str | None,
|
||||
classification: str,
|
||||
) -> PostboxDeliveryRequest:
|
||||
return PostboxDeliveryRequest(
|
||||
tenant_id=job.tenant_id,
|
||||
target=PostboxTargetRef(postbox_id=str(target["postbox_id"])),
|
||||
producer_module="campaigns",
|
||||
producer_resource_type="campaign_job",
|
||||
producer_resource_id=job.id,
|
||||
idempotency_key=(
|
||||
f"campaign:{job.campaign_version_id}:{job.id}:postbox:"
|
||||
f"{target_key}"
|
||||
),
|
||||
subject=(job.subject or "").strip() or "(No subject)",
|
||||
body_text=body_text,
|
||||
sender_label=_sender_label(job),
|
||||
classification=classification,
|
||||
participants=_participants(job),
|
||||
attachments=_attachments(job),
|
||||
metadata={
|
||||
"campaign_id": job.campaign_id,
|
||||
"campaign_version_id": job.campaign_version_id,
|
||||
"campaign_job_id": job.id,
|
||||
"entry_id": job.entry_id,
|
||||
"entry_index": job.entry_index,
|
||||
"delivery_channel_policy": job.delivery_channel_policy,
|
||||
"target_snapshot": target,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _record_rejection(
|
||||
session: Session,
|
||||
*,
|
||||
job_id: str,
|
||||
attempt_id: str,
|
||||
exc: Exception,
|
||||
temporary: bool,
|
||||
) -> None:
|
||||
session.rollback()
|
||||
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||
job = session.get(CampaignJob, job_id)
|
||||
if attempt is None or job is None:
|
||||
raise RuntimeError(
|
||||
"Postbox rejection could not be written to Campaign evidence."
|
||||
) from exc
|
||||
attempt.status = (
|
||||
JobPostboxStatus.REJECTED_TEMPORARY.value
|
||||
if temporary
|
||||
else JobPostboxStatus.REJECTED_PERMANENT.value
|
||||
)
|
||||
attempt.error_type = exc.__class__.__name__
|
||||
attempt.error_code = str(getattr(exc, "code", "") or "") or None
|
||||
attempt.error_message = str(exc)
|
||||
attempt.finished_at = utc_now()
|
||||
session.add(attempt)
|
||||
session.add(job)
|
||||
session.commit()
|
||||
|
||||
|
||||
def _record_unknown(
|
||||
session: Session,
|
||||
*,
|
||||
job_id: str,
|
||||
attempt_id: str,
|
||||
exc: Exception,
|
||||
) -> None:
|
||||
session.rollback()
|
||||
attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||
job = session.get(CampaignJob, job_id)
|
||||
if attempt is None or job is None:
|
||||
raise RuntimeError(
|
||||
"Unknown Postbox outcome could not be written to Campaign evidence."
|
||||
) from exc
|
||||
attempt.status = JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||
attempt.error_type = exc.__class__.__name__
|
||||
attempt.error_code = str(getattr(exc, "code", "") or "") or None
|
||||
attempt.error_message = str(exc)
|
||||
attempt.finished_at = utc_now()
|
||||
job.postbox_status = JobPostboxStatus.OUTCOME_UNKNOWN.value
|
||||
session.add(attempt)
|
||||
session.add(job)
|
||||
session.commit()
|
||||
|
||||
|
||||
def deliver_campaign_job_to_postboxes(
|
||||
session: Session,
|
||||
*,
|
||||
job: CampaignJob,
|
||||
message_bytes: bytes,
|
||||
classification: str,
|
||||
) -> PostboxChannelOutcome:
|
||||
outcome = PostboxChannelOutcome()
|
||||
targets = [
|
||||
target
|
||||
for target in (job.resolved_postbox_targets or [])
|
||||
if isinstance(target, dict) and target.get("postbox_id")
|
||||
]
|
||||
if not targets:
|
||||
outcome.rejected_permanent = 1
|
||||
outcome.messages.append("No frozen Postbox target is available.")
|
||||
job.postbox_status = JobPostboxStatus.REJECTED_PERMANENT.value
|
||||
session.add(job)
|
||||
session.commit()
|
||||
return outcome
|
||||
|
||||
body_text = _message_body(message_bytes)
|
||||
for target_index, target in enumerate(targets):
|
||||
key = _target_key(target)
|
||||
accepted = _accepted_attempt(
|
||||
session,
|
||||
job_id=job.id,
|
||||
target_key=key,
|
||||
)
|
||||
if accepted is not None:
|
||||
if accepted.vacant:
|
||||
outcome.accepted_vacant += 1
|
||||
else:
|
||||
outcome.accepted += 1
|
||||
continue
|
||||
|
||||
attempt_number = _attempt_number(
|
||||
session,
|
||||
job_id=job.id,
|
||||
target_key=key,
|
||||
)
|
||||
request = _request(
|
||||
job,
|
||||
target,
|
||||
target_key=key,
|
||||
body_text=body_text,
|
||||
classification=classification,
|
||||
)
|
||||
attempt = PostboxDeliveryAttempt(
|
||||
tenant_id=job.tenant_id,
|
||||
job_id=job.id,
|
||||
target_key=key,
|
||||
target_index=target_index,
|
||||
attempt_number=attempt_number,
|
||||
idempotency_key=request.idempotency_key,
|
||||
status=JobPostboxStatus.DELIVERING.value,
|
||||
target_snapshot=target,
|
||||
evidence={},
|
||||
started_at=utc_now(),
|
||||
)
|
||||
job.postbox_attempt_count += 1
|
||||
job.postbox_status = JobPostboxStatus.DELIVERING.value
|
||||
session.add(attempt)
|
||||
session.add(job)
|
||||
session.commit()
|
||||
attempt_id = attempt.id
|
||||
|
||||
try:
|
||||
result = postbox_integration().deliver(session, request)
|
||||
current_attempt = session.get(PostboxDeliveryAttempt, attempt_id)
|
||||
current_job = session.get(CampaignJob, job.id)
|
||||
if current_attempt is None or current_job is None:
|
||||
raise RuntimeError(
|
||||
"Campaign Postbox attempt disappeared before acceptance."
|
||||
)
|
||||
current_attempt.status = (
|
||||
JobPostboxStatus.ACCEPTED_VACANT.value
|
||||
if result.vacant
|
||||
else JobPostboxStatus.ACCEPTED.value
|
||||
)
|
||||
current_attempt.provider_delivery_id = result.delivery_id
|
||||
current_attempt.provider_message_id = result.message_id
|
||||
current_attempt.postbox_id = result.postbox_id
|
||||
current_attempt.address = result.address
|
||||
current_attempt.holder_count = result.holder_count
|
||||
current_attempt.vacant = result.vacant
|
||||
current_attempt.duplicate = result.duplicate
|
||||
current_attempt.evidence = dict(result.evidence)
|
||||
current_attempt.finished_at = utc_now()
|
||||
session.add(current_attempt)
|
||||
session.add(current_job)
|
||||
session.commit()
|
||||
if result.vacant:
|
||||
outcome.accepted_vacant += 1
|
||||
else:
|
||||
outcome.accepted += 1
|
||||
except PostboxDeliveryOutcomeUnknown as exc:
|
||||
_record_unknown(
|
||||
session,
|
||||
job_id=job.id,
|
||||
attempt_id=attempt_id,
|
||||
exc=exc,
|
||||
)
|
||||
outcome.outcome_unknown += 1
|
||||
outcome.messages.append(str(exc))
|
||||
except PostboxDeliveryRejected as exc:
|
||||
if exc.code == "idempotency_conflict":
|
||||
_record_unknown(
|
||||
session,
|
||||
job_id=job.id,
|
||||
attempt_id=attempt_id,
|
||||
exc=exc,
|
||||
)
|
||||
outcome.outcome_unknown += 1
|
||||
else:
|
||||
_record_rejection(
|
||||
session,
|
||||
job_id=job.id,
|
||||
attempt_id=attempt_id,
|
||||
exc=exc,
|
||||
temporary=exc.temporary,
|
||||
)
|
||||
if exc.temporary:
|
||||
outcome.rejected_temporary += 1
|
||||
else:
|
||||
outcome.rejected_permanent += 1
|
||||
outcome.messages.append(str(exc))
|
||||
except PostboxDeliveryUnavailable as exc:
|
||||
_record_rejection(
|
||||
session,
|
||||
job_id=job.id,
|
||||
attempt_id=attempt_id,
|
||||
exc=exc,
|
||||
temporary=True,
|
||||
)
|
||||
outcome.rejected_temporary += 1
|
||||
outcome.messages.append(str(exc))
|
||||
except Exception as exc:
|
||||
_record_unknown(
|
||||
session,
|
||||
job_id=job.id,
|
||||
attempt_id=attempt_id,
|
||||
exc=exc,
|
||||
)
|
||||
outcome.outcome_unknown += 1
|
||||
outcome.messages.append(str(exc))
|
||||
|
||||
current_job = session.get(CampaignJob, job.id)
|
||||
if current_job is not None:
|
||||
current_job.postbox_status = outcome.status
|
||||
session.add(current_job)
|
||||
session.commit()
|
||||
return outcome
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_core.core import runtime_coordination
|
||||
from govoplan_core.core.runtime_coordination import (
|
||||
RuntimeIdentity,
|
||||
bind_process_runtime_identity,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def _bind_test_runtime_identity():
|
||||
previous = runtime_coordination._process_runtime_identity
|
||||
bind_process_runtime_identity(
|
||||
RuntimeIdentity(
|
||||
installation_id="campaign-tests",
|
||||
node_id="campaign-test-process",
|
||||
incarnation="campaign-test-incarnation",
|
||||
role="test",
|
||||
software_version="test",
|
||||
composition_hash="a" * 64,
|
||||
)
|
||||
)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
bind_process_runtime_identity(previous)
|
||||
@@ -1,14 +1,26 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_campaign.backend.capabilities import CampaignAccessService
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignShare
|
||||
from govoplan_campaign.backend.capabilities import (
|
||||
CampaignAccessService,
|
||||
CampaignOwnershipService,
|
||||
campaign_report_resource_id,
|
||||
)
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignJob,
|
||||
CampaignShare,
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_core.core.access import PrincipalRef
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.core.ownership import OwnershipSubjectRef, OwnershipTransferError
|
||||
from govoplan_core.db.base import Base
|
||||
|
||||
|
||||
@@ -69,10 +81,326 @@ class CampaignAccessProviderTests(unittest.TestCase):
|
||||
self.assertTrue(any(item.kind == "share" and item.id == "share-write" for item in items))
|
||||
self.assertFalse(any(item.kind == "share" and item.id == "share-read" for item in items))
|
||||
|
||||
def test_campaign_children_explain_their_parent_campaign_access(self) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
campaign = Campaign(
|
||||
id="campaign-child",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_user_id=OTHER_USER_ID,
|
||||
external_id="child",
|
||||
name="Child resources",
|
||||
)
|
||||
version = CampaignVersion(
|
||||
id="version-child",
|
||||
campaign_id=campaign.id,
|
||||
version_number=1,
|
||||
raw_json={},
|
||||
)
|
||||
job = CampaignJob(
|
||||
id="job-child",
|
||||
tenant_id=TENANT_ID,
|
||||
campaign_id=campaign.id,
|
||||
campaign_version_id=version.id,
|
||||
entry_index=1,
|
||||
recipient_email="recipient@example.test",
|
||||
)
|
||||
session.add_all(
|
||||
[
|
||||
campaign,
|
||||
version,
|
||||
job,
|
||||
CampaignShare(
|
||||
id="share-child",
|
||||
tenant_id=TENANT_ID,
|
||||
campaign_id=campaign.id,
|
||||
target_type="group",
|
||||
target_id=GROUP_ID,
|
||||
permission="read",
|
||||
),
|
||||
]
|
||||
)
|
||||
session.commit()
|
||||
|
||||
service = CampaignAccessService()
|
||||
principal = _principal(group_ids={GROUP_ID})
|
||||
version_items = service.explain_resource_provenance(
|
||||
session,
|
||||
principal,
|
||||
resource_type="campaign_version",
|
||||
resource_id=version.id,
|
||||
action="campaigns:version:read",
|
||||
)
|
||||
job_items = service.explain_resource_provenance(
|
||||
session,
|
||||
principal,
|
||||
resource_type="campaign_delivery_job",
|
||||
resource_id=job.id,
|
||||
action="campaigns:delivery_job:read",
|
||||
)
|
||||
report_id = campaign_report_resource_id(
|
||||
campaign_id=campaign.id,
|
||||
version_id=version.id,
|
||||
report_kind="delivery",
|
||||
)
|
||||
report_items = service.explain_resource_provenance(
|
||||
session,
|
||||
principal,
|
||||
resource_type="campaign_report",
|
||||
resource_id=report_id,
|
||||
action="campaigns:report:read",
|
||||
)
|
||||
|
||||
for items, source in (
|
||||
(version_items, "campaigns.version"),
|
||||
(job_items, "campaigns.delivery_job"),
|
||||
(report_items, "campaigns.report"),
|
||||
):
|
||||
child = next(item for item in items if item.source == source)
|
||||
self.assertEqual(
|
||||
child.details["authorization_inherited_from"],
|
||||
{
|
||||
"resource_type": "campaign",
|
||||
"resource_id": campaign.id,
|
||||
},
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
item.source == "campaigns.campaign"
|
||||
and item.id == campaign.id
|
||||
for item in items
|
||||
)
|
||||
)
|
||||
self.assertTrue(
|
||||
any(item.kind == "share" and item.id == "share-child" for item in items)
|
||||
)
|
||||
|
||||
report = next(item for item in report_items if item.source == "campaigns.report")
|
||||
self.assertEqual(report.details["campaign_version_id"], version.id)
|
||||
self.assertEqual(report.details["report_kind"], "delivery")
|
||||
self.assertFalse(report.details["persisted"])
|
||||
|
||||
def test_campaign_ownership_provider_requires_group_acceptance_authority(self) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
campaign = Campaign(
|
||||
id="campaign-ownership",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_user_id=USER_ID,
|
||||
external_id="ownership",
|
||||
name="Ownership",
|
||||
)
|
||||
session.add(campaign)
|
||||
session.commit()
|
||||
|
||||
service = CampaignOwnershipService()
|
||||
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||
target_group = OwnershipSubjectRef(type="group", id=GROUP_ID)
|
||||
ordinary_member = OwnershipSubjectRef(
|
||||
type="user",
|
||||
id=OTHER_USER_ID,
|
||||
group_ids=frozenset({GROUP_ID}),
|
||||
)
|
||||
group_manager = OwnershipSubjectRef(
|
||||
type="user",
|
||||
id=OTHER_USER_ID,
|
||||
group_ids=frozenset({GROUP_ID}),
|
||||
scopes=frozenset({"campaigns:ownership:accept_group"}),
|
||||
)
|
||||
|
||||
denied = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="accept_group_transfer",
|
||||
actor=ordinary_member,
|
||||
current_owner=current_owner,
|
||||
target_owner=target_group,
|
||||
)
|
||||
allowed = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="accept_group_transfer",
|
||||
actor=group_manager,
|
||||
current_owner=current_owner,
|
||||
target_owner=target_group,
|
||||
)
|
||||
|
||||
self.assertFalse(denied.allowed)
|
||||
self.assertTrue(allowed.allowed)
|
||||
|
||||
@patch(
|
||||
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
|
||||
return_value=None,
|
||||
)
|
||||
def test_campaign_ownership_request_requires_existing_read_access(
|
||||
self,
|
||||
_target_validation,
|
||||
) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
campaign = Campaign(
|
||||
id="campaign-request",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_user_id=USER_ID,
|
||||
external_id="request",
|
||||
name="Request",
|
||||
)
|
||||
session.add(campaign)
|
||||
session.commit()
|
||||
service = CampaignOwnershipService()
|
||||
requester = OwnershipSubjectRef(
|
||||
type="user",
|
||||
id=OTHER_USER_ID,
|
||||
scopes=frozenset({"campaigns:campaign:read"}),
|
||||
)
|
||||
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||
|
||||
denied = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="request_ownership",
|
||||
actor=requester,
|
||||
current_owner=current_owner,
|
||||
target_owner=requester,
|
||||
)
|
||||
session.add(
|
||||
CampaignShare(
|
||||
id="share-requester",
|
||||
tenant_id=TENANT_ID,
|
||||
campaign_id=campaign.id,
|
||||
target_type="user",
|
||||
target_id=OTHER_USER_ID,
|
||||
permission="read",
|
||||
)
|
||||
)
|
||||
session.commit()
|
||||
allowed = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="request_ownership",
|
||||
actor=requester,
|
||||
current_owner=current_owner,
|
||||
target_owner=requester,
|
||||
)
|
||||
|
||||
self.assertFalse(denied.allowed)
|
||||
self.assertTrue(allowed.allowed)
|
||||
|
||||
@patch(
|
||||
"govoplan_campaign.backend.capabilities._valid_campaign_owner_target",
|
||||
return_value=None,
|
||||
)
|
||||
def test_campaign_owner_proposal_requires_share_authority(
|
||||
self,
|
||||
_target_validation,
|
||||
) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
campaign = Campaign(
|
||||
id="campaign-proposal",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_user_id=USER_ID,
|
||||
external_id="proposal",
|
||||
name="Proposal",
|
||||
)
|
||||
session.add(campaign)
|
||||
session.commit()
|
||||
service = CampaignOwnershipService()
|
||||
current_owner = OwnershipSubjectRef(type="user", id=USER_ID)
|
||||
target_owner = OwnershipSubjectRef(type="user", id=OTHER_USER_ID)
|
||||
|
||||
denied = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="propose_transfer",
|
||||
actor=current_owner,
|
||||
current_owner=current_owner,
|
||||
target_owner=target_owner,
|
||||
)
|
||||
allowed = service.authorize_ownership_action(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
action="propose_transfer",
|
||||
actor=OwnershipSubjectRef(
|
||||
type="user",
|
||||
id=USER_ID,
|
||||
scopes=frozenset({"campaigns:campaign:share"}),
|
||||
),
|
||||
current_owner=current_owner,
|
||||
target_owner=target_owner,
|
||||
)
|
||||
|
||||
self.assertFalse(denied.allowed)
|
||||
self.assertTrue(allowed.allowed)
|
||||
|
||||
def test_campaign_ownership_provider_applies_only_against_expected_owner(self) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
campaign = Campaign(
|
||||
id="campaign-owner-apply",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_user_id=USER_ID,
|
||||
external_id="owner-apply",
|
||||
name="Owner apply",
|
||||
)
|
||||
session.add(campaign)
|
||||
session.commit()
|
||||
|
||||
service = CampaignOwnershipService()
|
||||
service.apply_owner(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||
target_owner=OwnershipSubjectRef(type="group", id=GROUP_ID),
|
||||
actor=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
|
||||
reason=None,
|
||||
)
|
||||
session.flush()
|
||||
self.assertIsNone(campaign.owner_user_id)
|
||||
self.assertEqual(campaign.owner_group_id, GROUP_ID)
|
||||
|
||||
with self.assertRaisesRegex(
|
||||
OwnershipTransferError,
|
||||
"changed while the transfer was pending",
|
||||
):
|
||||
service.apply_owner(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
resource_id=campaign.id,
|
||||
expected_owner=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||
target_owner=OwnershipSubjectRef(type="user", id=OTHER_USER_ID),
|
||||
actor=OwnershipSubjectRef(type="user", id=USER_ID),
|
||||
reason=None,
|
||||
)
|
||||
|
||||
|
||||
def _session():
|
||||
engine = create_engine("sqlite:///:memory:", future=True)
|
||||
Base.metadata.create_all(bind=engine, tables=[Account.__table__, User.__table__, Group.__table__, Campaign.__table__, CampaignShare.__table__])
|
||||
Base.metadata.create_all(
|
||||
bind=engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
Campaign.__table__,
|
||||
CampaignShare.__table__,
|
||||
CampaignVersion.__table__,
|
||||
CampaignJob.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
],
|
||||
)
|
||||
return sessionmaker(bind=engine, future=True)()
|
||||
|
||||
|
||||
|
||||
@@ -9,12 +9,20 @@ from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_campaign.backend import router
|
||||
from govoplan_campaign.backend import router as campaign_api
|
||||
from govoplan_campaign.backend.routes import campaigns as campaign_routes
|
||||
from govoplan_campaign.backend.routes import jobs as job_routes
|
||||
from govoplan_campaign.backend.routes import reports as report_routes
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
|
||||
from govoplan_campaign.backend.reports.aggregate import (
|
||||
AggregateCampaignReportError,
|
||||
generate_aggregate_campaign_report,
|
||||
)
|
||||
from govoplan_campaign.backend.reports.provider import (
|
||||
CAMPAIGN_REPORT_PRIVACY_TRANSFORMS,
|
||||
CampaignAggregateReportProvider,
|
||||
)
|
||||
from govoplan_core.core.reporting import ReportProviderRequest
|
||||
from govoplan_campaign.backend.schemas import ReportEmailRequest
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
@@ -37,10 +45,12 @@ def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
|
||||
campaign = SimpleNamespace(id="campaign-1", tenant_id="tenant-1")
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(
|
||||
report_routes, "_get_campaign_for_principal", return_value=campaign
|
||||
),
|
||||
pytest.raises(HTTPException) as full_report_denied,
|
||||
):
|
||||
router.campaign_report(
|
||||
report_routes.campaign_report(
|
||||
"campaign-1",
|
||||
session=session,
|
||||
principal=principal, # type: ignore[arg-type]
|
||||
@@ -49,10 +59,10 @@ def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
|
||||
assert "campaigns:recipient:read" in full_report_denied.value.detail
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||
pytest.raises(HTTPException) as job_detail_denied,
|
||||
):
|
||||
router.get_job_detail(
|
||||
job_routes.get_job_detail(
|
||||
"campaign-1",
|
||||
"job-1",
|
||||
session=session,
|
||||
@@ -61,10 +71,12 @@ def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
|
||||
assert job_detail_denied.value.status_code == 403
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(
|
||||
report_routes, "_get_campaign_for_principal", return_value=campaign
|
||||
),
|
||||
pytest.raises(HTTPException) as report_email_denied,
|
||||
):
|
||||
router.email_campaign_report(
|
||||
report_routes.email_campaign_report(
|
||||
"campaign-1",
|
||||
ReportEmailRequest(to=["auditor@example.test"]),
|
||||
session=session,
|
||||
@@ -74,10 +86,10 @@ def test_full_report_and_job_detail_reject_aggregate_only_principal() -> None:
|
||||
assert "campaigns:recipient:export" in report_email_denied.value.detail
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(job_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||
pytest.raises(HTTPException) as diagnostics_denied,
|
||||
):
|
||||
router.get_job_diagnostics(
|
||||
job_routes.get_job_diagnostics(
|
||||
"campaign-1",
|
||||
"job-1",
|
||||
session=session,
|
||||
@@ -93,14 +105,14 @@ def test_aggregate_route_uses_only_the_safe_projection() -> None:
|
||||
safe_projection = Mock()
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal") as acl,
|
||||
patch.object(campaign_routes, "_get_campaign_for_principal") as acl,
|
||||
patch.object(
|
||||
router,
|
||||
campaign_routes,
|
||||
"generate_aggregate_campaign_report",
|
||||
return_value=safe_projection,
|
||||
) as generate,
|
||||
):
|
||||
result = router.aggregate_campaign_report(
|
||||
result = campaign_routes.aggregate_campaign_report(
|
||||
"campaign-1",
|
||||
session=session,
|
||||
principal=principal, # type: ignore[arg-type]
|
||||
@@ -116,10 +128,15 @@ def test_aggregate_route_uses_only_the_safe_projection() -> None:
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", ["/campaigns/aggregate-reports", "/campaigns/aggregate-reports/{campaign_id}"])
|
||||
@pytest.mark.parametrize(
|
||||
"path",
|
||||
["/campaigns/aggregate-reports", "/campaigns/aggregate-reports/{campaign_id}"],
|
||||
)
|
||||
def test_aggregate_routes_require_report_read_permission(path: str) -> None:
|
||||
route = next(item for item in router.router.routes if item.path == path)
|
||||
dependency = next(item for item in route.dependant.dependencies if item.name == "principal")
|
||||
route = next(item for item in campaign_api.router.routes if item.path == path)
|
||||
dependency = next(
|
||||
item for item in route.dependant.dependencies if item.name == "principal"
|
||||
)
|
||||
|
||||
with pytest.raises(HTTPException) as denied:
|
||||
dependency.call(_Principal())
|
||||
@@ -145,7 +162,9 @@ def test_aggregate_projection_is_tenant_isolated_and_needs_no_optional_module()
|
||||
],
|
||||
)
|
||||
with Session(engine) as session:
|
||||
session.add(Tenant(id="tenant-1", slug="tenant-1", name="Tenant 1", settings={}))
|
||||
session.add(
|
||||
Tenant(id="tenant-1", slug="tenant-1", name="Tenant 1", settings={})
|
||||
)
|
||||
campaign = Campaign(
|
||||
id="campaign-1",
|
||||
tenant_id="tenant-1",
|
||||
@@ -165,20 +184,22 @@ def test_aggregate_projection_is_tenant_isolated_and_needs_no_optional_module()
|
||||
campaign.current_version_id = version.id
|
||||
session.add_all([campaign, version])
|
||||
for index in range(5):
|
||||
session.add(CampaignJob(
|
||||
id=f"job-{index}",
|
||||
tenant_id="tenant-1",
|
||||
campaign_id=campaign.id,
|
||||
campaign_version_id=version.id,
|
||||
entry_index=index,
|
||||
recipient_email=f"private-{index}@example.test",
|
||||
subject="Private",
|
||||
build_status="built",
|
||||
validation_status="ready",
|
||||
queue_status="queued",
|
||||
send_status="smtp_accepted",
|
||||
imap_status="not_requested",
|
||||
))
|
||||
session.add(
|
||||
CampaignJob(
|
||||
id=f"job-{index}",
|
||||
tenant_id="tenant-1",
|
||||
campaign_id=campaign.id,
|
||||
campaign_version_id=version.id,
|
||||
entry_index=index,
|
||||
recipient_email=f"private-{index}@example.test",
|
||||
subject="Private",
|
||||
build_status="built",
|
||||
validation_status="ready",
|
||||
queue_status="queued",
|
||||
send_status="smtp_accepted",
|
||||
imap_status="not_requested",
|
||||
)
|
||||
)
|
||||
session.commit()
|
||||
|
||||
report = generate_aggregate_campaign_report(
|
||||
@@ -189,6 +210,30 @@ def test_aggregate_projection_is_tenant_isolated_and_needs_no_optional_module()
|
||||
assert report.population.denominator.value == 5
|
||||
assert report.outcomes.smtp_accepted.value == 5
|
||||
|
||||
provider = CampaignAggregateReportProvider()
|
||||
provider_principal = _Principal(
|
||||
"campaigns:report:read",
|
||||
"tenant:*",
|
||||
)
|
||||
descriptors = provider.list_reports(session, provider_principal)
|
||||
assert descriptors[0].report_id == "delivery-outcomes"
|
||||
assert descriptors[0].reidentification_risk == "low"
|
||||
provided = provider.execute_report(
|
||||
session,
|
||||
provider_principal,
|
||||
request=ReportProviderRequest(
|
||||
report_id="delivery-outcomes",
|
||||
parameters={"campaign_id": campaign.id},
|
||||
purpose="Tenant delivery overview",
|
||||
audience_scope={"scope_type": "tenant", "scope_id": "tenant-1"},
|
||||
),
|
||||
)
|
||||
assert set(provided.applied_privacy_transforms) == set(
|
||||
CAMPAIGN_REPORT_PRIVACY_TRANSFORMS
|
||||
)
|
||||
assert "recipient_email" not in repr(provided.payload)
|
||||
assert provided.source_revisions[0]["revision_id"] == version.id
|
||||
|
||||
with pytest.raises(AggregateCampaignReportError):
|
||||
generate_aggregate_campaign_report(
|
||||
session,
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_core.core.approvals import (
|
||||
ApprovalActorSelector,
|
||||
ApprovalCheck,
|
||||
ApprovalRequestRef,
|
||||
ApprovalStepDefinition,
|
||||
)
|
||||
from govoplan_campaign.backend.approval_gate import (
|
||||
CampaignApprovalGateError,
|
||||
assert_campaign_approval,
|
||||
request_campaign_approval,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Principal:
|
||||
tenant_id: str = "tenant-1"
|
||||
account_id: str = "requester"
|
||||
|
||||
|
||||
class Session:
|
||||
def add(self, _value: object) -> None:
|
||||
return None
|
||||
|
||||
def flush(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class ApprovalStub:
|
||||
available = True
|
||||
|
||||
def __init__(self, *, state: str = "pending") -> None:
|
||||
self.state = state
|
||||
self.command = None
|
||||
|
||||
def create_request(self, _session, _principal, *, command, idempotency_key):
|
||||
assert idempotency_key == "request-1"
|
||||
self.command = command
|
||||
return ApprovalRequestRef("approval-1", 1, "pending", "release")
|
||||
|
||||
def check_approved(self, _session, _principal, **kwargs):
|
||||
return ApprovalCheck(
|
||||
request_id=str(kwargs["request_id"]),
|
||||
revision=2,
|
||||
state=self.state,
|
||||
approved=self.state == "approved",
|
||||
subject_module=str(kwargs["subject_module"]),
|
||||
subject_type=str(kwargs["subject_type"]),
|
||||
subject_id=str(kwargs["subject_id"]),
|
||||
subject_version=kwargs["subject_version"],
|
||||
subject_digest=str(kwargs["subject_digest"]),
|
||||
)
|
||||
|
||||
|
||||
def _objects():
|
||||
campaign = SimpleNamespace(
|
||||
id="campaign-1",
|
||||
tenant_id="tenant-1",
|
||||
created_by_user_id="author",
|
||||
owner_user_id="owner",
|
||||
)
|
||||
version = SimpleNamespace(
|
||||
id="version-1",
|
||||
campaign_id="campaign-1",
|
||||
version_number=7,
|
||||
execution_snapshot_hash="a" * 64,
|
||||
editor_state={"review_send": {"updated_by_user_id": "reviewer"}},
|
||||
validation_summary={"validated_by_user_id": "validator"},
|
||||
build_summary={"built_by_user_id": "builder"},
|
||||
locked_by_user_id="validator",
|
||||
)
|
||||
return campaign, version
|
||||
|
||||
|
||||
def test_request_binds_snapshot_and_action_evidence() -> None:
|
||||
campaign, version = _objects()
|
||||
provider = ApprovalStub()
|
||||
snapshot = SimpleNamespace(build_token="build-7", snapshot_version="7")
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.approval_gate.ensure_execution_snapshot",
|
||||
return_value=snapshot,
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.approval_gate.approvals_integration",
|
||||
return_value=provider,
|
||||
),
|
||||
):
|
||||
request_campaign_approval(
|
||||
Session(),
|
||||
Principal(),
|
||||
campaign=campaign,
|
||||
version=version,
|
||||
title="Release",
|
||||
description=None,
|
||||
steps=(
|
||||
ApprovalStepDefinition(
|
||||
"release",
|
||||
"Release",
|
||||
(ApprovalActorSelector("role", "sender"),),
|
||||
forbidden_evidence_roles=("builder", "reviewer"),
|
||||
),
|
||||
),
|
||||
idempotency_key="request-1",
|
||||
)
|
||||
assert provider.command.subject_digest == "a" * 64
|
||||
assert provider.command.evidence_actors["builder"] == ("builder",)
|
||||
assert provider.command.evidence_actors["reviewer"] == ("reviewer",)
|
||||
assert version.editor_state["approval_gate"]["request_id"] == "approval-1"
|
||||
|
||||
|
||||
def test_gate_fails_closed_until_exact_request_is_approved() -> None:
|
||||
campaign, version = _objects()
|
||||
version.editor_state["approval_gate"] = {
|
||||
"request_id": "approval-1",
|
||||
"request_revision": 1,
|
||||
"subject_version": "build-7",
|
||||
"subject_digest": "a" * 64,
|
||||
"requested_at": "2026-08-01T00:00:00+00:00",
|
||||
"requested_by_user_id": "requester",
|
||||
}
|
||||
snapshot = SimpleNamespace(build_token="build-7", snapshot_version="7")
|
||||
provider = ApprovalStub(state="pending")
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.approval_gate.ensure_execution_snapshot",
|
||||
return_value=snapshot,
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.approval_gate.approvals_integration",
|
||||
return_value=provider,
|
||||
),
|
||||
):
|
||||
with pytest.raises(CampaignApprovalGateError, match="pending"):
|
||||
assert_campaign_approval(Session(), tenant_id="tenant-1", version=version)
|
||||
provider.state = "approved"
|
||||
assert_campaign_approval(Session(), tenant_id="tenant-1", version=version)
|
||||
version.execution_snapshot_hash = "b" * 64
|
||||
with pytest.raises(CampaignApprovalGateError, match="changed"):
|
||||
assert_campaign_approval(Session(), tenant_id="tenant-1", version=version)
|
||||
@@ -0,0 +1,337 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from sqlalchemy.pool import StaticPool
|
||||
|
||||
from govoplan_campaign.backend.artifact_reconciliation import (
|
||||
campaign_artifact_inventory,
|
||||
reconcile_campaign_artifacts,
|
||||
)
|
||||
from govoplan_core.core.object_storage import (
|
||||
StorageBackendError,
|
||||
StorageObjectInfo,
|
||||
StorageObjectMissing,
|
||||
StorageObjectPage,
|
||||
)
|
||||
from govoplan_core.core.recovery import RecoveryCheckpoint, RecoveryOperation
|
||||
from govoplan_core.core.recovery_runtime import RecoveryOperationBusy
|
||||
from govoplan_core.core.runtime_coordination import (
|
||||
DistributedLease,
|
||||
RuntimeIdentity,
|
||||
acquire_lease,
|
||||
)
|
||||
|
||||
|
||||
NOW = datetime(2026, 8, 3, 12, 0, tzinfo=timezone.utc)
|
||||
PREFIX = "campaign-artifacts/tenant-1/campaign-1/version-1/"
|
||||
|
||||
|
||||
class _MemoryStorage:
|
||||
name = "memory"
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.objects: dict[str, bytes] = {}
|
||||
self.modified_at: dict[str, datetime | None] = {}
|
||||
self.delete_failures: set[str] = set()
|
||||
self.exists_failures: set[str] = set()
|
||||
|
||||
def add(
|
||||
self,
|
||||
key: str,
|
||||
*,
|
||||
payload: bytes = b"artifact",
|
||||
modified_at: datetime | None = NOW - timedelta(days=2),
|
||||
) -> None:
|
||||
self.objects[key] = payload
|
||||
self.modified_at[key] = modified_at
|
||||
|
||||
def put_bytes(self, key: str, data: bytes, **_kwargs) -> None:
|
||||
self.add(key, payload=data, modified_at=NOW)
|
||||
|
||||
def get_bytes(self, key: str) -> bytes:
|
||||
try:
|
||||
return self.objects[key]
|
||||
except KeyError as exc:
|
||||
raise StorageObjectMissing("missing") from exc
|
||||
|
||||
def iter_bytes(self, key: str, *, chunk_size: int = 1024 * 1024):
|
||||
del chunk_size
|
||||
yield self.get_bytes(key)
|
||||
|
||||
def delete(self, key: str) -> None:
|
||||
if key in self.delete_failures:
|
||||
raise StorageBackendError("delete unavailable")
|
||||
self.objects.pop(key, None)
|
||||
self.modified_at.pop(key, None)
|
||||
|
||||
def exists(self, key: str) -> bool:
|
||||
if key in self.exists_failures:
|
||||
raise StorageBackendError("probe unavailable")
|
||||
return key in self.objects
|
||||
|
||||
def stat(self, key: str) -> StorageObjectInfo:
|
||||
if key not in self.objects:
|
||||
raise StorageObjectMissing("missing")
|
||||
return StorageObjectInfo(
|
||||
key=key,
|
||||
size_bytes=len(self.objects[key]),
|
||||
modified_at=self.modified_at[key],
|
||||
)
|
||||
|
||||
def list_objects(
|
||||
self,
|
||||
*,
|
||||
prefix: str,
|
||||
after: str | None = None,
|
||||
limit: int = 500,
|
||||
) -> StorageObjectPage:
|
||||
keys = [
|
||||
key
|
||||
for key in sorted(self.objects)
|
||||
if key.startswith(prefix) and (after is None or key > after)
|
||||
]
|
||||
selected = keys[:limit]
|
||||
return StorageObjectPage(
|
||||
objects=tuple(self.stat(key) for key in selected),
|
||||
next_cursor=(selected[-1] if len(keys) > len(selected) else None),
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def recovery_session_factory():
|
||||
engine = create_engine(
|
||||
"sqlite://",
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=StaticPool,
|
||||
)
|
||||
DistributedLease.__table__.create(engine)
|
||||
RecoveryOperation.__table__.create(engine)
|
||||
RecoveryCheckpoint.__table__.create(engine)
|
||||
factory = sessionmaker(bind=engine, expire_on_commit=False)
|
||||
try:
|
||||
yield factory
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def _identity(*, node_id: str = "node-1", incarnation: str = "run-1"):
|
||||
return RuntimeIdentity(
|
||||
installation_id="campaign-artifact-tests",
|
||||
node_id=node_id,
|
||||
incarnation=incarnation,
|
||||
role="worker",
|
||||
software_version="test",
|
||||
composition_hash="a" * 64,
|
||||
)
|
||||
|
||||
|
||||
def _without_domain_references():
|
||||
return (
|
||||
patch(
|
||||
"govoplan_campaign.backend.artifact_reconciliation._referenced_artifact_keys",
|
||||
return_value=set(),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.artifact_reconciliation._active_build_ids",
|
||||
return_value=set(),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_inventory_classifies_reference_grace_active_build_and_unknown_age() -> None:
|
||||
storage = _MemoryStorage()
|
||||
orphan = f"{PREFIX}build-orphan/message.eml"
|
||||
referenced = f"{PREFIX}build-referenced/message.eml"
|
||||
active = f"{PREFIX}build-active/message.eml"
|
||||
young = f"{PREFIX}build-young/message.eml"
|
||||
unknown_age = f"{PREFIX}build-unknown/message.eml"
|
||||
malformed = "campaign-artifacts/tenant-1/not-a-build-object"
|
||||
storage.add(orphan)
|
||||
storage.add(referenced)
|
||||
storage.add(active)
|
||||
storage.add(young, modified_at=NOW - timedelta(hours=1))
|
||||
storage.add(unknown_age, modified_at=None)
|
||||
storage.add(malformed)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.artifact_reconciliation._referenced_artifact_keys",
|
||||
return_value={referenced},
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.artifact_reconciliation._active_build_ids",
|
||||
return_value={"build-active"},
|
||||
),
|
||||
):
|
||||
inventory = campaign_artifact_inventory(
|
||||
object(), # type: ignore[arg-type]
|
||||
storage=storage,
|
||||
tenant_id="tenant-1",
|
||||
grace_period=timedelta(hours=24),
|
||||
page_size=20,
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
assert [candidate.key for candidate in inventory.candidates] == [orphan]
|
||||
assert inventory.referenced_count == 1
|
||||
assert inventory.active_build_count == 1
|
||||
assert inventory.young_count == 1
|
||||
assert inventory.unknown_age_count == 1
|
||||
assert inventory.invalid_shape_count == 1
|
||||
|
||||
|
||||
def test_process_loss_orphan_is_deleted_once_and_same_request_replays(
|
||||
recovery_session_factory,
|
||||
) -> None:
|
||||
storage = _MemoryStorage()
|
||||
orphan = f"{PREFIX}lost-build/message.eml"
|
||||
storage.add(orphan)
|
||||
reference_patch, active_patch = _without_domain_references()
|
||||
with reference_patch, active_patch:
|
||||
dry_run = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
now=NOW,
|
||||
)
|
||||
applied = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="cleanup-lost-build",
|
||||
now=NOW,
|
||||
)
|
||||
replayed = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="cleanup-lost-build",
|
||||
now=NOW,
|
||||
)
|
||||
repeated = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="cleanup-empty-page",
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
assert dry_run["candidate_count"] == 1
|
||||
assert dry_run["deleted_count"] == 0
|
||||
assert applied["status"] == "applied"
|
||||
assert applied["deleted_count"] == 1
|
||||
assert orphan not in storage.objects
|
||||
assert replayed["status"] == "already_completed"
|
||||
assert repeated["candidate_count"] == 0
|
||||
|
||||
|
||||
def test_competing_node_cannot_acquire_cleanup_authority(
|
||||
recovery_session_factory,
|
||||
) -> None:
|
||||
with recovery_session_factory() as session:
|
||||
claim = acquire_lease(
|
||||
session,
|
||||
installation_id="campaign-artifact-tests",
|
||||
resource_key="campaign:artifact-reconcile:tenant-1",
|
||||
holder_node_id="node-other",
|
||||
holder_incarnation="run-other",
|
||||
ttl_seconds=900,
|
||||
now=NOW,
|
||||
)
|
||||
assert claim is not None
|
||||
session.commit()
|
||||
|
||||
storage = _MemoryStorage()
|
||||
reference_patch, active_patch = _without_domain_references()
|
||||
with reference_patch, active_patch, pytest.raises(RecoveryOperationBusy):
|
||||
reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="competing-cleanup",
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
|
||||
def test_partial_storage_outage_remains_visible_and_retryable(
|
||||
recovery_session_factory,
|
||||
) -> None:
|
||||
storage = _MemoryStorage()
|
||||
removed = f"{PREFIX}build-a/message.eml"
|
||||
retained = f"{PREFIX}build-b/message.eml"
|
||||
storage.add(removed)
|
||||
storage.add(retained)
|
||||
storage.delete_failures.add(retained)
|
||||
reference_patch, active_patch = _without_domain_references()
|
||||
with reference_patch, active_patch:
|
||||
partial = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="partial-cleanup",
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
storage.delete_failures.clear()
|
||||
retry = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="partial-cleanup-retry",
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
assert partial["status"] == "recovery_required"
|
||||
assert partial["deleted_count"] == 1
|
||||
assert partial["failure_count"] == 1
|
||||
assert removed not in storage.objects
|
||||
assert retry["status"] == "applied"
|
||||
assert retry["deleted_count"] == 1
|
||||
assert storage.objects == {}
|
||||
with recovery_session_factory() as session:
|
||||
states = session.execute(
|
||||
select(RecoveryOperation.status).order_by(RecoveryOperation.created_at)
|
||||
).scalars().all()
|
||||
assert "recovery_required" in states
|
||||
assert states[-1] == "succeeded"
|
||||
|
||||
|
||||
def test_unverifiable_delete_is_outcome_unknown(
|
||||
recovery_session_factory,
|
||||
) -> None:
|
||||
storage = _MemoryStorage()
|
||||
orphan = f"{PREFIX}build-unknown/message.eml"
|
||||
storage.add(orphan)
|
||||
storage.exists_failures.add(orphan)
|
||||
reference_patch, active_patch = _without_domain_references()
|
||||
with reference_patch, active_patch:
|
||||
result = reconcile_campaign_artifacts(
|
||||
recovery_session_factory,
|
||||
storage=storage,
|
||||
identity=_identity(),
|
||||
tenant_id="tenant-1",
|
||||
apply=True,
|
||||
idempotency_key="unknown-cleanup",
|
||||
now=NOW,
|
||||
)
|
||||
|
||||
assert result["status"] == "outcome_unknown"
|
||||
assert result["failure_count"] == 1
|
||||
@@ -100,9 +100,9 @@ class CampaignAttachmentBuildTests(unittest.TestCase):
|
||||
cases = {
|
||||
"block": ("build_failed", "blocked", 0, "block", False),
|
||||
"ask": ("built", "needs_review", 0, "ask", True),
|
||||
"drop": ("built", "excluded", 0, "drop", True),
|
||||
"drop": ("built", "needs_review", 0, "ask", True),
|
||||
"warn": ("built", "warning", 1, "warn", True),
|
||||
"continue": ("built", "ready", 1, None, True),
|
||||
"continue": ("built", "warning", 1, None, True),
|
||||
}
|
||||
for behavior, (build_status, validation_status, queueable_count, issue_behavior, has_mime) in cases.items():
|
||||
with self.subTest(behavior=behavior):
|
||||
@@ -133,6 +133,71 @@ class CampaignAttachmentBuildTests(unittest.TestCase):
|
||||
self.assertEqual(coverage_issues[0].behavior, issue_behavior)
|
||||
self.assertEqual(result.built_messages[0].mime is not None, has_mime)
|
||||
|
||||
def test_required_missing_policy_cannot_be_loosened_by_rule(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
campaign_file = root / "campaign.json"
|
||||
campaign_file.write_text("{}", encoding="utf-8")
|
||||
config = self._no_attachment_config(
|
||||
behavior="continue",
|
||||
configure_missing_rule=True,
|
||||
)
|
||||
rule = config.attachments.global_[0]
|
||||
rule.required = True
|
||||
rule.missing_behavior = "continue"
|
||||
config.attachments.missing_behavior = "continue"
|
||||
|
||||
result = build_campaign_messages(
|
||||
config,
|
||||
campaign_file=campaign_file,
|
||||
output_dir=root / "out",
|
||||
write_eml=True,
|
||||
)
|
||||
|
||||
message = result.report.messages[0]
|
||||
self.assertEqual(message.validation_status.value, "blocked")
|
||||
issue = next(
|
||||
item
|
||||
for item in message.issues
|
||||
if item.code == "missing_required_attachment"
|
||||
)
|
||||
self.assertEqual(issue.behavior, "block")
|
||||
self.assertEqual(
|
||||
issue.details["effective_policy"]["requirement_policy"],
|
||||
"block",
|
||||
)
|
||||
self.assertEqual(
|
||||
message.attachments[0].missing_policy["effective_behavior"],
|
||||
"block",
|
||||
)
|
||||
|
||||
def test_optional_missing_policy_warns_by_default(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
campaign_file = root / "campaign.json"
|
||||
campaign_file.write_text("{}", encoding="utf-8")
|
||||
config = self._no_attachment_config(
|
||||
behavior="continue",
|
||||
configure_missing_rule=True,
|
||||
)
|
||||
config.attachments.global_[0].missing_behavior = None
|
||||
|
||||
result = build_campaign_messages(
|
||||
config,
|
||||
campaign_file=campaign_file,
|
||||
output_dir=root / "out",
|
||||
write_eml=True,
|
||||
)
|
||||
|
||||
message = result.report.messages[0]
|
||||
self.assertEqual(message.validation_status.value, "warning")
|
||||
issue = next(
|
||||
item
|
||||
for item in message.issues
|
||||
if item.code == "missing_optional_attachment"
|
||||
)
|
||||
self.assertEqual(issue.behavior, "warn")
|
||||
|
||||
def test_missing_pattern_does_not_create_zip_member_or_count_as_attachment(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from email import policy
|
||||
from email.message import EmailMessage
|
||||
from email.parser import BytesParser
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from govoplan_campaign.backend.campaign.models import (
|
||||
BuildStatus,
|
||||
CampaignConfig,
|
||||
SendStatus,
|
||||
)
|
||||
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
||||
from govoplan_campaign.backend.messages.builder import BuiltMessage
|
||||
from govoplan_campaign.backend.messages.models import (
|
||||
ImapStatus,
|
||||
MessageAddress,
|
||||
MessageDraft,
|
||||
MessageValidationStatus,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.campaigns import (
|
||||
_prepare_built_calendar_invitations,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.jobs import _mark_accepted_job_artifacts
|
||||
from govoplan_core.core.calendar import CalendarInvitationRef
|
||||
|
||||
|
||||
def _config() -> CampaignConfig:
|
||||
return CampaignConfig.model_validate(
|
||||
{
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "campaign-1",
|
||||
"name": "Invitation campaign",
|
||||
"mode": "send",
|
||||
},
|
||||
"template": {"subject": "Planning", "text": "Please reply."},
|
||||
"entries": {
|
||||
"inline": [
|
||||
{
|
||||
"id": "recipient-1",
|
||||
"name": "Ada",
|
||||
"to": [{"email": "ada@example.test", "name": "Ada"}],
|
||||
"fields": {"appointment_start": "2026-08-05T09:00:00"},
|
||||
}
|
||||
]
|
||||
},
|
||||
"delivery": {
|
||||
"channel_policy": "mail",
|
||||
"calendar_invitation": {
|
||||
"enabled": True,
|
||||
"calendar_id": "calendar-1",
|
||||
"summary_template": "Planning with {{name}}",
|
||||
"start_at_template": "{{appointment_start}}",
|
||||
"end_at_template": "2026-08-05T10:00:00",
|
||||
"timezone": "Europe/Berlin",
|
||||
},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class _CalendarIntegration:
|
||||
available = True
|
||||
|
||||
def request_from_payload(self, payload):
|
||||
return payload
|
||||
|
||||
def render_invitation(self, request):
|
||||
assert request["correlation_id"].startswith("campaign:version-1:")
|
||||
return "\r\n".join(
|
||||
(
|
||||
"BEGIN:VCALENDAR",
|
||||
"VERSION:2.0",
|
||||
"METHOD:REQUEST",
|
||||
"BEGIN:VEVENT",
|
||||
"UID:invitation-1@govoplan.local",
|
||||
"SUMMARY:Planning with Ada",
|
||||
"END:VEVENT",
|
||||
"END:VCALENDAR",
|
||||
"",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
class _MirroringIntegration:
|
||||
available = True
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.upsert_count = 0
|
||||
|
||||
def request_from_payload(self, payload):
|
||||
return SimpleNamespace(
|
||||
metadata=payload.get("metadata") or {},
|
||||
payload=payload,
|
||||
)
|
||||
|
||||
def upsert_invitation(self, _session, **_kwargs):
|
||||
self.upsert_count += 1
|
||||
return CalendarInvitationRef(
|
||||
event_id="event-1",
|
||||
calendar_id="calendar-1",
|
||||
uid="invitation-1@govoplan.local",
|
||||
correlation_id="campaign:version-1:entry-1",
|
||||
source_module="campaigns",
|
||||
source_resource_type="campaign_version",
|
||||
source_resource_id="version-1",
|
||||
)
|
||||
|
||||
|
||||
class _Session:
|
||||
def __init__(self) -> None:
|
||||
self.added: list[object] = []
|
||||
|
||||
def add(self, value) -> None:
|
||||
self.added.append(value)
|
||||
|
||||
|
||||
class CampaignCalendarInvitationTests(unittest.TestCase):
|
||||
def test_validation_requires_calendar_capability(self) -> None:
|
||||
report = validate_campaign_config(_config(), calendar_available=False)
|
||||
|
||||
self.assertIn(
|
||||
"calendar_invitation_unavailable",
|
||||
{issue.code for issue in report.issues},
|
||||
)
|
||||
|
||||
def test_build_freezes_individual_request_and_ics_attachment(self) -> None:
|
||||
config = _config()
|
||||
entry = config.entries.inline[0] # type: ignore[index]
|
||||
message = EmailMessage()
|
||||
message["From"] = "Organizer <organizer@example.test>"
|
||||
message["To"] = "Ada <ada@example.test>"
|
||||
message["Subject"] = "Planning"
|
||||
message.set_content("Please reply.")
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
eml_path = Path(temp_dir) / "message.eml"
|
||||
eml_path.write_bytes(bytes(message))
|
||||
draft = MessageDraft(
|
||||
entry_index=1,
|
||||
entry_id="recipient-1",
|
||||
active=True,
|
||||
build_status=BuildStatus.BUILT,
|
||||
validation_status=MessageValidationStatus.READY,
|
||||
send_status=SendStatus.DRAFT,
|
||||
imap_status=ImapStatus.NOT_REQUESTED,
|
||||
subject="Planning",
|
||||
**{
|
||||
"from": MessageAddress(
|
||||
email="organizer@example.test",
|
||||
name="Organizer",
|
||||
)
|
||||
},
|
||||
to=[MessageAddress(email="ada@example.test", name="Ada")],
|
||||
eml_path=str(eml_path),
|
||||
)
|
||||
provenance: dict[int, dict[str, object]] = {1: {}}
|
||||
|
||||
with patch(
|
||||
"govoplan_campaign.backend.persistence.campaigns.calendar_integration",
|
||||
return_value=_CalendarIntegration(),
|
||||
):
|
||||
_prepare_built_calendar_invitations(
|
||||
version=SimpleNamespace(
|
||||
id="version-1",
|
||||
campaign_id="campaign-1",
|
||||
),
|
||||
config=config,
|
||||
built_messages=[BuiltMessage(draft=draft, mime=message)],
|
||||
entries_by_index={1: entry},
|
||||
delivery_provenance_by_index=provenance,
|
||||
user_id="user-1",
|
||||
)
|
||||
|
||||
parsed = BytesParser(policy=policy.default).parsebytes(
|
||||
eml_path.read_bytes()
|
||||
)
|
||||
attachments = list(parsed.iter_attachments())
|
||||
self.assertEqual("text/calendar", attachments[0].get_content_type())
|
||||
self.assertEqual("REQUEST", attachments[0].get_param("method"))
|
||||
invitation = provenance[1]["calendar_invitation"]
|
||||
self.assertEqual("prepared", invitation["state"])
|
||||
self.assertEqual(
|
||||
"2026-08-05T09:00:00+02:00",
|
||||
invitation["request"]["start_at"],
|
||||
)
|
||||
|
||||
def test_delivery_acceptance_mirrors_once_without_reopening_delivery(self) -> None:
|
||||
integration = _MirroringIntegration()
|
||||
job = SimpleNamespace(
|
||||
tenant_id="tenant-1",
|
||||
delivery_provenance={
|
||||
"calendar_invitation": {
|
||||
"state": "prepared",
|
||||
"request": {
|
||||
"metadata": {"prepared_by_user_id": "user-1"},
|
||||
},
|
||||
}
|
||||
},
|
||||
)
|
||||
session = _Session()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.calendar_integration",
|
||||
return_value=integration,
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.files_integration"
|
||||
),
|
||||
):
|
||||
_mark_accepted_job_artifacts(session, job)
|
||||
_mark_accepted_job_artifacts(session, job)
|
||||
|
||||
invitation = job.delivery_provenance["calendar_invitation"]
|
||||
self.assertEqual("mirrored", invitation["state"])
|
||||
self.assertEqual("event-1", invitation["event_id"])
|
||||
self.assertEqual(1, integration.upsert_count)
|
||||
self.assertEqual([job], session.added)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -6,8 +6,11 @@ from unittest.mock import MagicMock, patch
|
||||
import pytest
|
||||
from fastapi import HTTPException
|
||||
|
||||
from govoplan_campaign.backend import router
|
||||
from govoplan_campaign.backend import route_support
|
||||
from govoplan_campaign.backend.routes import campaigns as campaign_routes
|
||||
from govoplan_campaign.backend.routes import versions as version_routes
|
||||
from govoplan_campaign.backend.schemas import CampaignUpdateRequest, CampaignVersionUpdateRequest
|
||||
from govoplan_core.core.concurrency import strong_resource_etag
|
||||
|
||||
|
||||
def _principal() -> SimpleNamespace:
|
||||
@@ -26,6 +29,7 @@ def test_version_update_rolls_back_when_its_audit_record_cannot_be_written() ->
|
||||
raw_json={},
|
||||
current_flow="manual",
|
||||
current_step="recipients",
|
||||
edit_revision=1,
|
||||
)
|
||||
|
||||
def mutate(*_args, **kwargs):
|
||||
@@ -34,18 +38,26 @@ def test_version_update_rolls_back_when_its_audit_record_cannot_be_written() ->
|
||||
return version
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal"),
|
||||
patch.object(router, "_get_version_for_tenant", return_value=version),
|
||||
patch.object(router, "update_campaign_version", side_effect=mutate),
|
||||
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
patch.object(route_support, "_get_campaign_for_principal"),
|
||||
patch.object(route_support, "_get_version_for_tenant", return_value=version),
|
||||
patch.object(route_support, "update_campaign_version", side_effect=mutate),
|
||||
patch.object(route_support, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
):
|
||||
with pytest.raises(HTTPException, match="audit unavailable") as captured:
|
||||
router._update_campaign_version_detail_response( # noqa: SLF001 - transaction regression test
|
||||
route_support._update_campaign_version_detail_response( # noqa: SLF001 - transaction regression test
|
||||
session,
|
||||
principal, # type: ignore[arg-type]
|
||||
"campaign-1",
|
||||
"version-1",
|
||||
CampaignVersionUpdateRequest(current_step="recipients"),
|
||||
CampaignVersionUpdateRequest(
|
||||
current_step="recipients",
|
||||
base_revision=1,
|
||||
),
|
||||
if_match=strong_resource_etag(
|
||||
"campaign_version",
|
||||
"version-1",
|
||||
1,
|
||||
),
|
||||
autosave=True,
|
||||
audit_action="campaign.version_autosaved",
|
||||
)
|
||||
@@ -68,15 +80,15 @@ def test_version_fork_rolls_back_when_its_audit_record_cannot_be_written() -> No
|
||||
return forked
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(router, "_require_permission"),
|
||||
patch.object(router, "_get_version_for_tenant", return_value=source),
|
||||
patch.object(router, "_get_campaign_for_tenant", return_value=campaign),
|
||||
patch.object(router, "fork_campaign_version_for_edit", side_effect=mutate),
|
||||
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
patch.object(version_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(version_routes, "_require_permission"),
|
||||
patch.object(version_routes, "_get_version_for_tenant", return_value=source),
|
||||
patch.object(version_routes, "_get_campaign_for_tenant", return_value=campaign),
|
||||
patch.object(version_routes, "fork_campaign_version_for_edit", side_effect=mutate),
|
||||
patch.object(version_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="audit unavailable"):
|
||||
router.fork_version_for_edit(
|
||||
version_routes.fork_version_for_edit(
|
||||
"campaign-1",
|
||||
"version-1",
|
||||
CampaignVersionUpdateRequest(),
|
||||
@@ -102,12 +114,12 @@ def test_metadata_update_rolls_back_when_its_audit_record_cannot_be_written() ->
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(router, "_sync_campaign_metadata_to_current_version"),
|
||||
patch.object(router, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
patch.object(campaign_routes, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(campaign_routes, "_sync_campaign_metadata_to_current_version"),
|
||||
patch.object(campaign_routes, "audit_from_principal", side_effect=RuntimeError("audit unavailable")),
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="audit unavailable"):
|
||||
router.update_campaign_metadata_endpoint(
|
||||
campaign_routes.update_campaign_metadata_endpoint(
|
||||
"campaign-1",
|
||||
CampaignUpdateRequest(name="New name"),
|
||||
session=session,
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_core.core.object_storage import LocalFilesystemStorageBackend
|
||||
from govoplan_core.core.recovery import RecoveryMode
|
||||
from govoplan_campaign.backend.persistence.campaigns import (
|
||||
CampaignPersistenceError,
|
||||
_StoredEmlArtifact,
|
||||
_build_storage_expectations,
|
||||
_delete_storage_keys,
|
||||
_verify_build_storage_manifest,
|
||||
_verify_storage_keys_absent,
|
||||
)
|
||||
from govoplan_campaign.backend.routes.versions import _campaign_build_recovery_plan
|
||||
|
||||
|
||||
def test_object_only_and_managed_output_builds_use_distinct_recovery_modes() -> None:
|
||||
assert _campaign_build_recovery_plan({}).mode == RecoveryMode.COMPENSATION
|
||||
assert (
|
||||
_campaign_build_recovery_plan(
|
||||
{"delivery": {"print": {"persist_to_files": True}}}
|
||||
).mode
|
||||
== RecoveryMode.FORWARD_RECOVERY
|
||||
)
|
||||
assert (
|
||||
_campaign_build_recovery_plan(
|
||||
{"delivery": {"print": {"persist_to_files": False}}}
|
||||
).mode
|
||||
== RecoveryMode.COMPENSATION
|
||||
)
|
||||
|
||||
|
||||
def test_generated_object_manifest_verifies_exact_bytes(tmp_path: Path) -> None:
|
||||
storage = LocalFilesystemStorageBackend(tmp_path)
|
||||
payload = b"Message-ID: <build@example.test>\r\n\r\nbody"
|
||||
key = "campaign-artifacts/tenant/campaign/version/build/00000001.eml"
|
||||
storage.put_bytes(key, payload)
|
||||
artifact = _StoredEmlArtifact(
|
||||
storage_key=key,
|
||||
size_bytes=len(payload),
|
||||
sha256=hashlib.sha256(payload).hexdigest(),
|
||||
message_id_header="<build@example.test>",
|
||||
)
|
||||
|
||||
evidence = _verify_build_storage_manifest(
|
||||
storage,
|
||||
_build_storage_expectations(
|
||||
stored_eml_by_index={1: artifact},
|
||||
print_outputs_by_index={},
|
||||
),
|
||||
)
|
||||
|
||||
assert evidence["object_count"] == 1
|
||||
assert evidence["total_bytes"] == len(payload)
|
||||
storage.put_bytes(key, b"tampered")
|
||||
with pytest.raises(CampaignPersistenceError, match="does not match"):
|
||||
_verify_build_storage_manifest(
|
||||
storage,
|
||||
_build_storage_expectations(
|
||||
stored_eml_by_index={1: artifact},
|
||||
print_outputs_by_index={},
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_compensation_requires_verified_object_absence(tmp_path: Path) -> None:
|
||||
storage = LocalFilesystemStorageBackend(tmp_path)
|
||||
key = "campaign-artifacts/tenant/campaign/version/build/object"
|
||||
storage.put_bytes(key, b"payload")
|
||||
assert _delete_storage_keys(storage, [key]) == []
|
||||
assert _verify_storage_keys_absent(storage, [key]) == (True, 0)
|
||||
|
||||
|
||||
class _UnremovableStorage:
|
||||
name = "unremovable"
|
||||
|
||||
def delete(self, _key: str) -> None:
|
||||
from govoplan_core.core.object_storage import StorageBackendError
|
||||
|
||||
raise StorageBackendError("unavailable")
|
||||
|
||||
def exists(self, _key: str) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def test_failed_compensation_remains_observable() -> None:
|
||||
storage = _UnremovableStorage()
|
||||
key = "campaign-artifacts/tenant/campaign/version/build/object"
|
||||
assert _delete_storage_keys(storage, [key]) == [key] # type: ignore[arg-type]
|
||||
assert _verify_storage_keys_absent(storage, [key]) == (False, 1) # type: ignore[arg-type]
|
||||
@@ -0,0 +1,119 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_core.core.recovery import RecoveryStatus
|
||||
from govoplan_campaign.backend.sending import jobs
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("outcome", "expected_method", "expected_status"),
|
||||
[
|
||||
("succeeded", "succeed", None),
|
||||
("failed", "reject", None),
|
||||
("outcome_unknown", "unresolved", RecoveryStatus.OUTCOME_UNKNOWN),
|
||||
("recovery_required", "unresolved", RecoveryStatus.RECOVERY_REQUIRED),
|
||||
],
|
||||
)
|
||||
def test_delivery_recovery_maps_verified_job_state(
|
||||
monkeypatch,
|
||||
outcome: str,
|
||||
expected_method: str,
|
||||
expected_status: RecoveryStatus | None,
|
||||
) -> None:
|
||||
evidence = {
|
||||
"verified": outcome != "recovery_required",
|
||||
"checks": {"job_state_reloaded": True},
|
||||
"send_status": outcome,
|
||||
}
|
||||
monkeypatch.setattr(
|
||||
jobs,
|
||||
"_job_delivery_recovery_evidence",
|
||||
lambda _job_id: (outcome, evidence),
|
||||
)
|
||||
operation = Mock()
|
||||
|
||||
jobs._finish_job_delivery_recovery(operation, job_id="job-1")
|
||||
|
||||
method = getattr(operation, expected_method)
|
||||
method.assert_called_once()
|
||||
if expected_status is not None:
|
||||
assert method.call_args.kwargs["status"] == expected_status
|
||||
for other in {"succeed", "reject", "unresolved"} - {expected_method}:
|
||||
getattr(operation, other).assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("outcome", "expected_method", "expected_status"),
|
||||
[
|
||||
("succeeded", "succeed", None),
|
||||
("failed", "reject", None),
|
||||
("outcome_unknown", "unresolved", RecoveryStatus.OUTCOME_UNKNOWN),
|
||||
("recovery_required", "unresolved", RecoveryStatus.RECOVERY_REQUIRED),
|
||||
],
|
||||
)
|
||||
def test_imap_recovery_maps_verified_append_state(
|
||||
monkeypatch,
|
||||
outcome: str,
|
||||
expected_method: str,
|
||||
expected_status: RecoveryStatus | None,
|
||||
) -> None:
|
||||
evidence = {
|
||||
"verified": outcome != "recovery_required",
|
||||
"checks": {"job_state_reloaded": True},
|
||||
"imap_status": outcome,
|
||||
}
|
||||
monkeypatch.setattr(
|
||||
jobs,
|
||||
"_imap_recovery_evidence",
|
||||
lambda _job_id: (outcome, evidence),
|
||||
)
|
||||
operation = Mock()
|
||||
|
||||
jobs._finish_imap_append_recovery(operation, job_id="job-1")
|
||||
|
||||
method = getattr(operation, expected_method)
|
||||
method.assert_called_once()
|
||||
if expected_status is not None:
|
||||
assert method.call_args.kwargs["status"] == expected_status
|
||||
for other in {"succeed", "reject", "unresolved"} - {expected_method}:
|
||||
getattr(operation, other).assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("outcome", "expected_method", "expected_status"),
|
||||
[
|
||||
("succeeded", "succeed", None),
|
||||
("failed", "reject", None),
|
||||
("outcome_unknown", "unresolved", RecoveryStatus.OUTCOME_UNKNOWN),
|
||||
("recovery_required", "unresolved", RecoveryStatus.RECOVERY_REQUIRED),
|
||||
],
|
||||
)
|
||||
def test_single_action_recovery_maps_verified_action_state(
|
||||
monkeypatch,
|
||||
outcome: str,
|
||||
expected_method: str,
|
||||
expected_status: RecoveryStatus | None,
|
||||
) -> None:
|
||||
evidence = {
|
||||
"verified": outcome != "recovery_required",
|
||||
"checks": {"action_state_reloaded": True},
|
||||
"action_status": outcome,
|
||||
}
|
||||
monkeypatch.setattr(
|
||||
jobs,
|
||||
"_single_action_recovery_evidence",
|
||||
lambda _action_id: (outcome, evidence),
|
||||
)
|
||||
operation = Mock()
|
||||
|
||||
jobs._finish_single_action_delivery_recovery(operation, action_id="action-1")
|
||||
|
||||
method = getattr(operation, expected_method)
|
||||
method.assert_called_once()
|
||||
if expected_status is not None:
|
||||
assert method.call_args.kwargs["status"] == expected_status
|
||||
for other in {"succeed", "reject", "unresolved"} - {expected_method}:
|
||||
getattr(operation, other).assert_not_called()
|
||||
@@ -0,0 +1,165 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from sqlalchemy import Column, String, Table, create_engine
|
||||
from sqlalchemy.orm import Session, sessionmaker
|
||||
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
Campaign,
|
||||
CampaignIssue,
|
||||
CampaignVersion,
|
||||
)
|
||||
from govoplan_campaign.backend.persistence.versions import update_campaign_version
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.core.concurrency import RevisionConflictError
|
||||
from govoplan_core.db.base import Base
|
||||
|
||||
|
||||
class CampaignOptimisticConcurrencyTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temp_dir = tempfile.TemporaryDirectory()
|
||||
database_path = Path(self.temp_dir.name) / "campaign.db"
|
||||
self.engine = create_engine(f"sqlite+pysqlite:///{database_path}")
|
||||
access_users = Base.metadata.tables.get("access_users")
|
||||
if access_users is None:
|
||||
access_users = Table(
|
||||
"access_users",
|
||||
Base.metadata,
|
||||
Column("id", String(36), primary_key=True),
|
||||
)
|
||||
access_groups = Base.metadata.tables.get("access_groups")
|
||||
if access_groups is None:
|
||||
access_groups = Table(
|
||||
"access_groups",
|
||||
Base.metadata,
|
||||
Column("id", String(36), primary_key=True),
|
||||
)
|
||||
Base.metadata.create_all(
|
||||
self.engine,
|
||||
tables=[
|
||||
access_users,
|
||||
access_groups,
|
||||
ChangeSequenceEntry.__table__,
|
||||
Campaign.__table__,
|
||||
CampaignVersion.__table__,
|
||||
CampaignIssue.__table__,
|
||||
],
|
||||
)
|
||||
self.SessionLocal = sessionmaker(
|
||||
bind=self.engine,
|
||||
class_=Session,
|
||||
expire_on_commit=False,
|
||||
)
|
||||
with self.SessionLocal() as session:
|
||||
campaign = Campaign(
|
||||
id="campaign-1",
|
||||
tenant_id="tenant-1",
|
||||
external_id="C-1",
|
||||
name="Campaign",
|
||||
current_version_id="version-1",
|
||||
)
|
||||
version = CampaignVersion(
|
||||
id="version-1",
|
||||
campaign_id=campaign.id,
|
||||
version_number=1,
|
||||
raw_json={
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "C-1",
|
||||
"name": "Campaign",
|
||||
"description": "Base",
|
||||
},
|
||||
},
|
||||
)
|
||||
session.add_all((campaign, version))
|
||||
session.commit()
|
||||
self.addCleanup(self.engine.dispose)
|
||||
self.addCleanup(self.temp_dir.cleanup)
|
||||
|
||||
@staticmethod
|
||||
def _update(
|
||||
session: Session,
|
||||
*,
|
||||
expected_revision: int,
|
||||
name: str,
|
||||
) -> CampaignVersion:
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.persistence.versions._updated_runtime_json",
|
||||
side_effect=lambda _session, **kwargs: kwargs["raw_json"],
|
||||
),
|
||||
):
|
||||
return update_campaign_version(
|
||||
session,
|
||||
tenant_id="tenant-1",
|
||||
campaign_id="campaign-1",
|
||||
version_id="version-1",
|
||||
raw_json={
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "C-1",
|
||||
"name": name,
|
||||
"description": "Base",
|
||||
},
|
||||
},
|
||||
expected_revision=expected_revision,
|
||||
)
|
||||
|
||||
def test_only_one_of_two_writers_can_commit_the_same_revision(self) -> None:
|
||||
first = self.SessionLocal()
|
||||
second = self.SessionLocal()
|
||||
self.addCleanup(first.close)
|
||||
self.addCleanup(second.close)
|
||||
first.get(CampaignVersion, "version-1")
|
||||
second.get(CampaignVersion, "version-1")
|
||||
|
||||
saved = self._update(
|
||||
first,
|
||||
expected_revision=1,
|
||||
name="First writer",
|
||||
)
|
||||
self.assertEqual(saved.edit_revision, 2)
|
||||
|
||||
with self.assertRaises(RevisionConflictError) as raised:
|
||||
self._update(
|
||||
second,
|
||||
expected_revision=1,
|
||||
name="Second writer",
|
||||
)
|
||||
self.assertEqual(raised.exception.current_revision, 2)
|
||||
self.assertEqual(raised.exception.submitted_base_revision, 1)
|
||||
|
||||
with self.SessionLocal() as verification:
|
||||
current = verification.get(CampaignVersion, "version-1")
|
||||
assert current is not None
|
||||
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
|
||||
self.assertEqual(current.edit_revision, 2)
|
||||
|
||||
def test_stale_revision_is_rejected_before_mutation(self) -> None:
|
||||
with self.SessionLocal() as first:
|
||||
self._update(
|
||||
first,
|
||||
expected_revision=1,
|
||||
name="First writer",
|
||||
)
|
||||
with self.SessionLocal() as stale:
|
||||
with self.assertRaises(RevisionConflictError):
|
||||
self._update(
|
||||
stale,
|
||||
expected_revision=1,
|
||||
name="Stale writer",
|
||||
)
|
||||
stale.rollback()
|
||||
|
||||
with self.SessionLocal() as verification:
|
||||
current = verification.get(CampaignVersion, "version-1")
|
||||
assert current is not None
|
||||
self.assertEqual(current.raw_json["campaign"]["name"], "First writer")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -5,10 +5,17 @@ from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from govoplan_core.core.postbox import PostboxDeliveryReceiptSummaryRef
|
||||
from govoplan_core.privacy.schemas import PrivacyRetentionPolicyItem
|
||||
from govoplan_campaign.backend.reports.campaigns import (
|
||||
_JobReportAggregate,
|
||||
_campaign_retention_projection,
|
||||
_campaign_postbox_receipt_summary,
|
||||
_job_evidence_row,
|
||||
_latest_by_job_id,
|
||||
_load_delivery_info,
|
||||
_review_decision_summary,
|
||||
_review_decisions_by_job,
|
||||
generate_campaign_report,
|
||||
)
|
||||
|
||||
@@ -82,7 +89,21 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
|
||||
updated_at=_dt(),
|
||||
)
|
||||
|
||||
row = _job_evidence_row(job, latest_smtp=smtp, latest_imap=imap)
|
||||
row = _job_evidence_row(
|
||||
job,
|
||||
latest_smtp=smtp,
|
||||
latest_imap=imap,
|
||||
review_decision={
|
||||
"decision": "accept",
|
||||
"reason": "Recipient confirmed no attachment was expected.",
|
||||
"actor_user_id": "reviewer-1",
|
||||
"decided_at": "2026-07-08T12:30:00+00:00",
|
||||
"review_key": "recipient-1",
|
||||
"message_sha256": "abc123",
|
||||
"issue_fingerprint": "def456",
|
||||
"issue_codes": ["missing_optional_attachment"],
|
||||
},
|
||||
)
|
||||
|
||||
assert row["campaign_id"] == "campaign-1"
|
||||
assert row["campaign_version_id"] == "version-1"
|
||||
@@ -101,6 +122,38 @@ def test_job_evidence_row_contains_transport_and_message_evidence() -> None:
|
||||
assert "latest_imap_error_message" not in row
|
||||
assert "eml_storage_key" not in row
|
||||
assert "eml_local_path" not in row
|
||||
assert row["review_decision"] == "accept"
|
||||
assert row["review_actor_user_id"] == "reviewer-1"
|
||||
assert row["review_issue_codes"] == "missing_optional_attachment"
|
||||
assert "review_message_sha256" not in row
|
||||
|
||||
|
||||
def test_report_uses_only_review_decisions_for_the_current_build() -> None:
|
||||
decision = {
|
||||
"job_id": "job-1",
|
||||
"decision": "accept",
|
||||
"issue_codes": ["missing_optional_attachment"],
|
||||
}
|
||||
version = SimpleNamespace(
|
||||
build_summary={"build_token": "build-2"},
|
||||
editor_state={
|
||||
"review_send": {
|
||||
"build_token": "build-2",
|
||||
"inspection_complete": True,
|
||||
"issue_decisions": [decision],
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
decisions = _review_decisions_by_job(version)
|
||||
|
||||
assert decisions == {"job-1": decision}
|
||||
assert _review_decision_summary(decisions) == {
|
||||
"exception_decision_count": 1,
|
||||
"by_issue_code": {"missing_optional_attachment": 1},
|
||||
}
|
||||
version.editor_state["review_send"]["build_token"] = "stale-build"
|
||||
assert _review_decisions_by_job(version) == {}
|
||||
|
||||
|
||||
def test_latest_by_job_id_keeps_highest_attempt_number() -> None:
|
||||
@@ -161,6 +214,171 @@ def test_aggregate_report_omits_recipient_level_failures_by_default() -> None:
|
||||
assert payload.call_args.kwargs["include_recent_failures"] is False
|
||||
|
||||
|
||||
def test_postbox_receipt_report_aggregates_provider_state_without_identities() -> None:
|
||||
class _Query:
|
||||
def join(self, *args, **kwargs):
|
||||
return self
|
||||
|
||||
def filter(self, *args, **kwargs):
|
||||
return self
|
||||
|
||||
def all(self):
|
||||
return [("delivery-1",), ("delivery-2",), ("delivery-1",)]
|
||||
|
||||
integration = SimpleNamespace(
|
||||
receipt_evidence_available=True,
|
||||
delivery_receipt_summaries=lambda session, **kwargs: {
|
||||
"delivery-1": PostboxDeliveryReceiptSummaryRef(
|
||||
delivery_id="delivery-1",
|
||||
message_id="message-1",
|
||||
postbox_id="postbox-1",
|
||||
delivery_status="accepted",
|
||||
accepted_at=_dt(),
|
||||
currently_readable=True,
|
||||
read_receipt_count=2,
|
||||
acknowledged_receipt_count=1,
|
||||
routed_message_count=1,
|
||||
),
|
||||
"delivery-2": PostboxDeliveryReceiptSummaryRef(
|
||||
delivery_id="delivery-2",
|
||||
message_id="message-2",
|
||||
postbox_id="postbox-2",
|
||||
delivery_status="accepted_vacant",
|
||||
accepted_at=_dt(),
|
||||
currently_readable=False,
|
||||
expired_message_count=1,
|
||||
),
|
||||
},
|
||||
)
|
||||
session = SimpleNamespace(query=lambda *args: _Query())
|
||||
with patch(
|
||||
"govoplan_campaign.backend.reports.campaigns.postbox_integration",
|
||||
return_value=integration,
|
||||
):
|
||||
report = _campaign_postbox_receipt_summary(
|
||||
session, # type: ignore[arg-type]
|
||||
tenant_id="tenant-1",
|
||||
campaign_id="campaign-1",
|
||||
version=SimpleNamespace(id="version-1"),
|
||||
)
|
||||
|
||||
assert report["delivery_count"] == 2
|
||||
assert report["currently_readable_delivery_count"] == 1
|
||||
assert report["read_delivery_count"] == 1
|
||||
assert report["acknowledged_delivery_count"] == 1
|
||||
assert report["routed_message_count"] == 1
|
||||
assert report["expired_message_count"] == 1
|
||||
assert "account_id" not in repr(report)
|
||||
assert "identity_id" not in repr(report)
|
||||
|
||||
|
||||
def test_retention_projection_explains_defaults_and_redacted_evidence() -> None:
|
||||
aggregate = _JobReportAggregate(
|
||||
eml_retained=2,
|
||||
eml_expired=1,
|
||||
eml_not_generated=3,
|
||||
)
|
||||
version = SimpleNamespace(
|
||||
raw_json={
|
||||
"_retention": {
|
||||
"raw_json_redacted": True,
|
||||
"redacted_at": "2026-07-08T12:30:00+00:00",
|
||||
}
|
||||
},
|
||||
validation_summary={
|
||||
"_retention": {
|
||||
"report_detail_redacted": True,
|
||||
"redacted_at": "2026-07-08T12:30:00+00:00",
|
||||
}
|
||||
},
|
||||
build_summary={"ok": True},
|
||||
)
|
||||
|
||||
with patch(
|
||||
"govoplan_campaign.backend.reports.campaigns.capability",
|
||||
return_value=None,
|
||||
):
|
||||
projection = _campaign_retention_projection(
|
||||
object(), # type: ignore[arg-type]
|
||||
campaign=SimpleNamespace(id="campaign-1"),
|
||||
version=version, # type: ignore[arg-type]
|
||||
aggregate=aggregate,
|
||||
postbox_receipts={
|
||||
"status": "available",
|
||||
"currently_readable_delivery_count": 2,
|
||||
"expired_message_count": 1,
|
||||
},
|
||||
)
|
||||
|
||||
assert projection["policy_status"] == "defaults"
|
||||
assert projection["effective_policy"]["store_raw_campaign_json"] is True
|
||||
assert projection["evidence"]["raw_campaign_json"]["state"] == "redacted"
|
||||
assert projection["evidence"]["stored_report_detail"]["state"] == "partially_redacted"
|
||||
assert projection["evidence"]["generated_eml"] == {
|
||||
"state": "partially_expired",
|
||||
"retained_count": 2,
|
||||
"expired_count": 1,
|
||||
"not_generated_count": 3,
|
||||
}
|
||||
assert projection["evidence"]["postbox_copies"]["state"] == "partially_expired"
|
||||
assert "generated_eml" in projection["privacy_impact"]["minimized_categories"]
|
||||
|
||||
|
||||
def test_retention_projection_uses_only_public_policy_contract_metadata() -> None:
|
||||
class _RetentionService:
|
||||
privacy_policy_from_settings = lambda self, *args, **kwargs: None
|
||||
privacy_policy_from_session = lambda self, *args, **kwargs: None
|
||||
set_privacy_policy = lambda self, *args, **kwargs: None
|
||||
parent_privacy_policy = lambda self, *args, **kwargs: None
|
||||
parent_privacy_policy_sources = lambda self, *args, **kwargs: []
|
||||
get_privacy_policy_for_scope = lambda self, *args, **kwargs: {}
|
||||
set_privacy_policy_for_scope = lambda self, *args, **kwargs: {}
|
||||
sanitize_audit_details_for_policy = lambda self, *args, **kwargs: {}
|
||||
apply_retention_policy = lambda self, *args, **kwargs: {}
|
||||
|
||||
def effective_privacy_policy(self, *args, **kwargs):
|
||||
return PrivacyRetentionPolicyItem(
|
||||
generated_eml_retention_days=30,
|
||||
stored_report_detail_retention_days=90,
|
||||
)
|
||||
|
||||
def effective_privacy_policy_sources(self, *args, **kwargs):
|
||||
return [{
|
||||
"scope_type": "tenant",
|
||||
"path": "tenant:tenant-1",
|
||||
"label": "Tenant",
|
||||
"applied_fields": ["generated_eml_retention_days"],
|
||||
"policy": {"generated_eml_retention_days": 30},
|
||||
"provider_secret": "must-not-leak",
|
||||
}]
|
||||
|
||||
with patch(
|
||||
"govoplan_campaign.backend.reports.campaigns.capability",
|
||||
return_value=_RetentionService(),
|
||||
):
|
||||
projection = _campaign_retention_projection(
|
||||
object(), # type: ignore[arg-type]
|
||||
campaign=SimpleNamespace(id="campaign-1"),
|
||||
version=SimpleNamespace(
|
||||
raw_json={},
|
||||
validation_summary={},
|
||||
build_summary={},
|
||||
), # type: ignore[arg-type]
|
||||
aggregate=_JobReportAggregate(),
|
||||
postbox_receipts={"status": "not_applicable"},
|
||||
)
|
||||
|
||||
assert projection["policy_status"] == "configured"
|
||||
assert projection["effective_policy"]["generated_eml_retention_days"] == 30
|
||||
assert projection["sources"] == [{
|
||||
"scope_type": "tenant",
|
||||
"path": "tenant:tenant-1",
|
||||
"label": "Tenant",
|
||||
"applied_fields": ["generated_eml_retention_days"],
|
||||
}]
|
||||
assert "must-not-leak" not in repr(projection)
|
||||
|
||||
|
||||
class CampaignReportProjectionTests(unittest.TestCase):
|
||||
def test_evidence_projection(self) -> None:
|
||||
test_job_evidence_row_contains_transport_and_message_evidence()
|
||||
|
||||
@@ -0,0 +1,314 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
import sys
|
||||
import tempfile
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||
RUNNER_PATH = (
|
||||
REPOSITORY_ROOT
|
||||
/ "dev"
|
||||
/ "mail-testbed"
|
||||
/ "run_celery_redelivery_acceptance.py"
|
||||
)
|
||||
COMPOSE_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "docker-compose.yml"
|
||||
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
|
||||
TASK_ID = "12345678-1234-4234-8234-123456789abc"
|
||||
|
||||
|
||||
def _load_runner():
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"govoplan_campaign_celery_redelivery_acceptance",
|
||||
RUNNER_PATH,
|
||||
)
|
||||
assert spec is not None and spec.loader is not None
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
runner = _load_runner()
|
||||
|
||||
|
||||
class _Response:
|
||||
def __init__(self, status_code: int, payload: dict) -> None:
|
||||
self.status_code = status_code
|
||||
self._payload = payload
|
||||
|
||||
def json(self) -> dict:
|
||||
return self._payload
|
||||
|
||||
|
||||
class _Client:
|
||||
def post(self, path: str, **_kwargs) -> _Response:
|
||||
assert path.endswith("/queue")
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"queued_count": 1,
|
||||
"skipped_count": 0,
|
||||
"blocked_count": 0,
|
||||
"enqueued_count": 1,
|
||||
"delivery_mode": "worker_queue",
|
||||
"worker_queue_available": True,
|
||||
"dry_run": False,
|
||||
},
|
||||
)
|
||||
|
||||
def get(self, path: str, **_kwargs) -> _Response:
|
||||
assert path.endswith("/report")
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"cards": {
|
||||
"jobs_total": 1,
|
||||
"outcome_unknown": 1,
|
||||
"needs_attention": 1,
|
||||
},
|
||||
"status_counts": {
|
||||
"send": {"outcome_unknown": 1},
|
||||
"imap": {"pending": 1},
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
class _Endpoint:
|
||||
host = "127.0.0.1"
|
||||
port = 4025
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.release_count = 0
|
||||
|
||||
def wait_for_data(self, _timeout_seconds: int) -> bool:
|
||||
return True
|
||||
|
||||
def release_held_connection(self) -> None:
|
||||
self.release_count += 1
|
||||
|
||||
def evidence(self) -> dict[str, int]:
|
||||
return {
|
||||
"connection_count": 1,
|
||||
"accepted_rcpt_commands": 1,
|
||||
"refused_rcpt_commands": 0,
|
||||
"data_transactions": 1,
|
||||
}
|
||||
|
||||
|
||||
def _settings():
|
||||
return runner.TestbedSettings(
|
||||
smtp_host="127.0.0.1",
|
||||
smtp_port=3025,
|
||||
imap_host="127.0.0.1",
|
||||
imap_port=3143,
|
||||
username="campaign-test@govoplan.test",
|
||||
password="local-test-password",
|
||||
sender="campaign-test@govoplan.test",
|
||||
recipient="campaign-test@govoplan.test",
|
||||
sent_folder="Sent",
|
||||
provider_timeout_seconds=5,
|
||||
)
|
||||
|
||||
|
||||
def test_compose_redis_is_isolated_durable_and_health_checked() -> None:
|
||||
compose = COMPOSE_PATH.read_text(encoding="utf-8")
|
||||
|
||||
assert "redis:7-alpine" in compose
|
||||
assert '"--appendonly", "yes"' in compose
|
||||
assert "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379" in compose
|
||||
assert 'test: ["CMD", "redis-cli", "ping"]' in compose
|
||||
assert "campaign-redis-data:/data" in compose
|
||||
|
||||
|
||||
def test_runbook_keeps_local_redelivery_distinct_from_production_supervision() -> None:
|
||||
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
|
||||
assert "run_celery_redelivery_acceptance.py" in testbed
|
||||
assert "same Celery task identity must be redelivered" in testbed
|
||||
assert "production daemon supervision" in testbed
|
||||
assert "empty broker queue/unacked set" in runbook
|
||||
assert "production process manager" in runbook
|
||||
|
||||
|
||||
def test_compose_lifecycle_targets_only_isolated_redis_service() -> None:
|
||||
up = runner._compose_command(
|
||||
compose_file=COMPOSE_PATH,
|
||||
project_name="govoplan-campaign-redelivery-test",
|
||||
operation="up",
|
||||
)
|
||||
down = runner._compose_command(
|
||||
compose_file=COMPOSE_PATH,
|
||||
project_name="govoplan-campaign-redelivery-test",
|
||||
operation="down",
|
||||
)
|
||||
|
||||
assert up[-3:] == ["up", "--detach", "redis"]
|
||||
assert down[-3:] == ["down", "--volumes", "--remove-orphans"]
|
||||
assert "greenmail" not in up
|
||||
assert "--project-name" in up
|
||||
|
||||
|
||||
def test_worker_bootstrap_uses_real_late_ack_solo_celery_worker() -> None:
|
||||
source = runner.WORKER_BOOTSTRAP
|
||||
|
||||
assert "celery.worker_main" in source
|
||||
assert '"--pool=solo"' in source
|
||||
assert '"--queues=send_email"' in source
|
||||
assert '"visibility_timeout"' in source
|
||||
assert '"polling_interval"' in source
|
||||
assert "send_email.run" not in source
|
||||
|
||||
|
||||
def test_runtime_root_uses_platform_temp_selection() -> None:
|
||||
with mock.patch(
|
||||
"govoplan_campaign_celery_redelivery_acceptance.tempfile.mkdtemp",
|
||||
return_value="/selected-temp/govoplan-campaign-celery-redelivery-test",
|
||||
) as mkdtemp:
|
||||
runtime_root = runner._create_runtime_root()
|
||||
|
||||
assert runtime_root == Path(
|
||||
"/selected-temp/govoplan-campaign-celery-redelivery-test"
|
||||
)
|
||||
mkdtemp.assert_called_once_with(prefix="govoplan-campaign-celery-redelivery-")
|
||||
|
||||
|
||||
def test_worker_log_projection_matches_redelivered_task_without_retaining_id() -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary_directory:
|
||||
log_path = Path(temporary_directory) / "worker.log"
|
||||
log_path.write_text(
|
||||
"\n".join(
|
||||
[
|
||||
f"Task govoplan.campaigns.send_email[{TASK_ID}] received",
|
||||
f"Task govoplan.campaigns.send_email[{TASK_ID}] succeeded in 0.1s",
|
||||
]
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
with log_path.open("ab") as handle:
|
||||
worker = runner.WorkerProcess(
|
||||
process=SimpleNamespace(),
|
||||
log_path=log_path,
|
||||
log_handle=handle,
|
||||
)
|
||||
|
||||
assert worker.received_task_ids() == (TASK_ID,)
|
||||
assert worker.succeeded_task_ids() == (TASK_ID,)
|
||||
|
||||
|
||||
def test_queue_projection_fails_closed_if_no_task_was_published() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="one Celery task"):
|
||||
runner._queue_evidence(
|
||||
{
|
||||
"queued_count": 1,
|
||||
"skipped_count": 0,
|
||||
"blocked_count": 0,
|
||||
"enqueued_count": 0,
|
||||
"delivery_mode": "database_queue",
|
||||
"worker_queue_available": False,
|
||||
"dry_run": False,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_redelivery_orchestration_requires_same_task_and_no_second_smtp_effect(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
first_worker = mock.Mock()
|
||||
first_worker.received_task_ids.return_value = (TASK_ID,)
|
||||
replacement_worker = mock.Mock()
|
||||
replacement_worker.received_task_ids.return_value = (TASK_ID,)
|
||||
workers = iter([first_worker, replacement_worker])
|
||||
endpoint = _Endpoint()
|
||||
durable_states = iter(
|
||||
[
|
||||
{
|
||||
"job_count": 1,
|
||||
"send_status_counts": {"sending": 1},
|
||||
"attempt_status_counts": {"smtp_in_progress": 1},
|
||||
"unfinished_attempt_count": 1,
|
||||
},
|
||||
{
|
||||
"job_count": 1,
|
||||
"send_status_counts": {"outcome_unknown": 1},
|
||||
"attempt_status_counts": {"outcome_unknown": 1},
|
||||
"unfinished_attempt_count": 0,
|
||||
},
|
||||
]
|
||||
)
|
||||
prepared = SimpleNamespace(
|
||||
campaign_id="campaign-internal",
|
||||
version_id="version-internal",
|
||||
public_evidence=lambda: {
|
||||
"validation": {"ok": True},
|
||||
"build": {"built_count": 1},
|
||||
"campaign_mail_boundary": {
|
||||
"profile_reference_only": True,
|
||||
"smtp_revision_frozen": True,
|
||||
"imap_revision_frozen": True,
|
||||
"resolved_transport_material_present": False,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
monkeypatch.setattr(runner, "create_mail_profile", lambda *args, **kwargs: "profile-internal")
|
||||
monkeypatch.setattr(runner, "prepare_campaign_scenario", lambda *args, **kwargs: prepared)
|
||||
monkeypatch.setattr(runner, "_start_worker", lambda *args, **kwargs: next(workers))
|
||||
monkeypatch.setattr(runner, "_wait_for_worker_ready", lambda *args, **kwargs: None)
|
||||
received = iter([TASK_ID, TASK_ID])
|
||||
monkeypatch.setattr(runner, "_wait_for_received_task", lambda *args, **kwargs: next(received))
|
||||
monkeypatch.setattr(runner, "_wait_for_task_success", lambda *args, **kwargs: None)
|
||||
monkeypatch.setattr(runner, "_kill_worker", lambda *args, **kwargs: -9)
|
||||
monkeypatch.setattr(runner, "_stop_worker", lambda *args, **kwargs: None)
|
||||
monkeypatch.setattr(
|
||||
runner,
|
||||
"_wait_for_broker_drained",
|
||||
lambda *args, **kwargs: runner.RedisBrokerState(0, 0, 0),
|
||||
)
|
||||
|
||||
evidence = runner.execute_redelivery_scenario(
|
||||
_Client(),
|
||||
{"Authorization": "not-retained"},
|
||||
fixture_path=FIXTURE_PATH,
|
||||
settings=_settings(),
|
||||
endpoint=endpoint,
|
||||
redis_url="redis://127.0.0.1:36379/0",
|
||||
runtime_root=Path("/not-used"),
|
||||
snapshot_probe=lambda _version_id: ({}, {}),
|
||||
audit_probe=lambda _campaign_id, _version_id: {
|
||||
"campaign.created": 1,
|
||||
"campaign.validated": 1,
|
||||
"campaign.messages_built": 1,
|
||||
"campaign.queued": 1,
|
||||
},
|
||||
delivery_probe=lambda _campaign_id, _version_id: next(durable_states),
|
||||
)
|
||||
|
||||
assert evidence["broker"] == {
|
||||
"transport": "redis",
|
||||
"same_task_identity_redelivered": True,
|
||||
"first_worker_received_count": 1,
|
||||
"replacement_worker_received_count": 1,
|
||||
"queue_depth": 0,
|
||||
"unacked_hash_count": 0,
|
||||
"unacked_index_count": 0,
|
||||
}
|
||||
assert evidence["protocol"]["connection_count"] == 1
|
||||
assert evidence["protocol"]["data_transactions"] == 1
|
||||
assert evidence["recovered_durable_state"]["send_status_counts"] == {
|
||||
"outcome_unknown": 1
|
||||
}
|
||||
assert TASK_ID not in json.dumps(evidence, sort_keys=True)
|
||||
assert endpoint.release_count >= 1
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -53,6 +54,30 @@ def test_campaign_runtime_documentation_provider_is_registered() -> None:
|
||||
assert documentation_topics in get_manifest().documentation_providers
|
||||
|
||||
|
||||
def test_complete_review_workflow_documents_each_attention_class() -> None:
|
||||
from govoplan_campaign.backend.manifest import get_manifest
|
||||
|
||||
topic = next(
|
||||
item
|
||||
for item in get_manifest().documentation
|
||||
if item.id == "campaigns.workflow.complete-review"
|
||||
)
|
||||
|
||||
rendered = "\n".join(
|
||||
(
|
||||
topic.summary,
|
||||
topic.body,
|
||||
*topic.metadata["steps"],
|
||||
topic.metadata["verification"],
|
||||
)
|
||||
)
|
||||
assert "Critical blockers" in rendered
|
||||
assert "Individual review" in rendered
|
||||
assert "Group review" in rendered
|
||||
assert "Remaining is zero" in rendered
|
||||
assert topic.metadata["help_contexts"] == ["campaign.review-send"]
|
||||
|
||||
|
||||
def test_runtime_documentation_is_user_only_and_requires_a_campaign_task() -> None:
|
||||
assert _topics({"docs:documentation:read"}) == ()
|
||||
assert _topics({"campaigns:campaign:read"}, documentation_type="admin") == ()
|
||||
@@ -288,7 +313,7 @@ def test_sender_sees_queue_and_send_only_with_the_mail_contract_and_profile_auth
|
||||
topic for topic in CAMPAIGN_USER_DOCUMENTATION
|
||||
if topic.id == "campaigns.workflow.queue-delivery"
|
||||
)
|
||||
assert any(link.href == "/operator" for link in queue_topic.links)
|
||||
assert any(link.href == "/campaigns/queue" for link in queue_topic.links)
|
||||
|
||||
|
||||
def test_connected_authoring_tasks_require_their_declared_contracts_and_permissions() -> None:
|
||||
@@ -360,10 +385,22 @@ def test_aggregate_report_task_never_implies_recipient_detail_or_export_authorit
|
||||
item for item in CAMPAIGN_USER_DOCUMENTATION
|
||||
if item.id == "campaigns.workflow.view-aggregate-delivery-report"
|
||||
)
|
||||
assert topic.metadata["route"] == "/reports"
|
||||
assert topic.metadata["route"] == "/campaigns/reports"
|
||||
assert "export" in topic.metadata["verification"].lower()
|
||||
|
||||
|
||||
def test_handbook_distinguishes_shipped_aggregate_reports_from_detailed_report_gaps() -> None:
|
||||
handbook = " ".join(
|
||||
(
|
||||
Path(__file__).resolve().parents[1] / "docs" / "CAMPAIGN_HANDBOOK.md"
|
||||
).read_text(encoding="utf-8").lower().split()
|
||||
)
|
||||
|
||||
assert "aggregate-only reader ui remains open" not in handbook
|
||||
assert "separate aggregate **reports** surface" in handbook
|
||||
assert "permission-aware action visibility on that detailed surface remains open work" in handbook
|
||||
|
||||
|
||||
def test_static_campaign_handbook_has_unique_ids_help_contexts_and_no_planned_resend_claim() -> None:
|
||||
from govoplan_campaign.backend.manifest import get_manifest
|
||||
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURE_ROOT = REPOSITORY_ROOT / "examples" / "simple-announcement"
|
||||
|
||||
|
||||
_ISOLATED_ACCEPTANCE_PROGRAM = r"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import importlib.abc
|
||||
import json
|
||||
import socket
|
||||
import sys
|
||||
from email import policy
|
||||
from email.parser import BytesParser
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
source_root = Path(sys.argv[1]).resolve()
|
||||
fixture_root = Path(sys.argv[2]).resolve()
|
||||
sys.path.insert(0, str(source_root))
|
||||
|
||||
|
||||
class AbsentOptionalModuleFinder(importlib.abc.MetaPathFinder):
|
||||
absent_roots = {"govoplan_files", "govoplan_mail"}
|
||||
|
||||
def find_spec(self, fullname, path=None, target=None):
|
||||
if fullname.partition(".")[0] in self.absent_roots:
|
||||
raise ModuleNotFoundError(
|
||||
f"{fullname} is intentionally absent in the Campaign fixture check",
|
||||
name=fullname,
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
sys.meta_path.insert(0, AbsentOptionalModuleFinder())
|
||||
|
||||
|
||||
def deny_network(*args, **kwargs):
|
||||
raise AssertionError("the Campaign validate/build fixture must not open a network connection")
|
||||
|
||||
|
||||
class NoNetworkSocket(socket.socket):
|
||||
def connect(self, address):
|
||||
deny_network(address)
|
||||
|
||||
def connect_ex(self, address):
|
||||
deny_network(address)
|
||||
|
||||
|
||||
socket.create_connection = deny_network
|
||||
socket.socket = NoNetworkSocket
|
||||
|
||||
from govoplan_campaign.backend.campaign import ( # noqa: E402
|
||||
load_campaign_config,
|
||||
load_campaign_json,
|
||||
validate_campaign_config,
|
||||
)
|
||||
from govoplan_campaign.backend.messages import build_campaign_messages # noqa: E402
|
||||
|
||||
|
||||
metadata = json.loads((fixture_root / "fixture.json").read_text(encoding="utf-8"))
|
||||
assert metadata["required_modules"] == ["core", "access", "campaigns"]
|
||||
assert metadata["absent_optional_modules"] == ["files", "mail"]
|
||||
assert metadata["external_effects"] == "forbidden"
|
||||
|
||||
campaign_file = fixture_root / metadata["campaign_file"]
|
||||
raw_campaign = load_campaign_json(campaign_file)
|
||||
|
||||
|
||||
def iter_keys(value):
|
||||
if isinstance(value, dict):
|
||||
for key, nested in value.items():
|
||||
yield key.casefold()
|
||||
yield from iter_keys(nested)
|
||||
elif isinstance(value, list):
|
||||
for nested in value:
|
||||
yield from iter_keys(nested)
|
||||
|
||||
|
||||
forbidden_key_fragments = ("credential", "imap", "mail_profile", "password", "secret", "smtp")
|
||||
assert not [
|
||||
key
|
||||
for key in iter_keys(raw_campaign)
|
||||
if any(fragment in key for fragment in forbidden_key_fragments)
|
||||
]
|
||||
|
||||
config = load_campaign_config(campaign_file)
|
||||
assert config.server.mail_profile_id is None
|
||||
assert not config.attachments.global_
|
||||
|
||||
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||
assert validation.ok
|
||||
assert validation.error_count == 0
|
||||
assert validation.warning_count == 0
|
||||
assert validation.entries_count == metadata["expected"]["entries_count"]
|
||||
|
||||
|
||||
def build(output_name):
|
||||
return build_campaign_messages(
|
||||
config,
|
||||
campaign_file=campaign_file,
|
||||
output_dir=fixture_root.parent / output_name,
|
||||
write_eml=True,
|
||||
)
|
||||
|
||||
|
||||
first = build("build-first")
|
||||
second = build("build-second")
|
||||
expected = metadata["expected"]
|
||||
for result in (first, second):
|
||||
assert result.report.campaign_id == expected["campaign_id"]
|
||||
assert result.report.entries_count == expected["entries_count"]
|
||||
assert result.report.built_count == expected["built_count"]
|
||||
assert result.report.build_failed_count == 0
|
||||
assert result.report.queueable_count == expected["queueable_count"]
|
||||
assert len(result.built_messages) == 1
|
||||
|
||||
built = result.built_messages[0]
|
||||
assert built.mime is not None
|
||||
assert built.draft.subject == expected["subject"]
|
||||
assert built.draft.validation_status.value == "ready"
|
||||
assert built.draft.send_status.value == "draft"
|
||||
assert built.draft.imap_status.value == "not_requested"
|
||||
assert built.draft.attachment_count == expected["attachment_count"]
|
||||
assert not built.draft.attachments
|
||||
assert not built.draft.issues
|
||||
assert built.draft.from_ is not None
|
||||
assert built.draft.from_.email == "announcements@example.test"
|
||||
assert [address.email for address in built.draft.to] == ["recipient@example.test"]
|
||||
assert built.mime["Subject"] == expected["subject"]
|
||||
assert "Hello Example Recipient" in built.mime.get_content()
|
||||
assert list(built.mime.iter_attachments()) == []
|
||||
|
||||
|
||||
def normalized_eml(path_value):
|
||||
message = BytesParser(policy=policy.default).parsebytes(Path(path_value).read_bytes())
|
||||
del message["Date"]
|
||||
del message["Message-ID"]
|
||||
return message.as_bytes(policy=policy.default)
|
||||
|
||||
|
||||
first_eml = normalized_eml(first.report.messages[0].eml_path)
|
||||
second_eml = normalized_eml(second.report.messages[0].eml_path)
|
||||
assert first_eml == second_eml
|
||||
assert not any(
|
||||
name == root or name.startswith(root + ".")
|
||||
for name in sys.modules
|
||||
for root in ("govoplan_files", "govoplan_mail")
|
||||
)
|
||||
|
||||
print(json.dumps({
|
||||
"campaign_id": first.report.campaign_id,
|
||||
"built_count": first.report.built_count,
|
||||
"queueable_count": first.report.queueable_count,
|
||||
"normalized_eml_sha256": hashlib.sha256(first_eml).hexdigest(),
|
||||
}, sort_keys=True))
|
||||
"""
|
||||
|
||||
|
||||
class CampaignExampleAcceptanceTests(unittest.TestCase):
|
||||
def test_simple_announcement_validates_and_builds_without_mail_or_files(self) -> None:
|
||||
self.assertTrue((FIXTURE_ROOT / "campaign.json").is_file())
|
||||
self.assertTrue((FIXTURE_ROOT / "fixture.json").is_file())
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-campaign-acceptance-", dir="/tmp") as temp_dir:
|
||||
workspace = Path(temp_dir).resolve()
|
||||
self.assertNotIn(REPOSITORY_ROOT, workspace.parents)
|
||||
isolated_fixture = workspace / "simple-announcement"
|
||||
shutil.copytree(FIXTURE_ROOT, isolated_fixture)
|
||||
|
||||
completed = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-I",
|
||||
"-c",
|
||||
_ISOLATED_ACCEPTANCE_PROGRAM,
|
||||
str(REPOSITORY_ROOT / "src"),
|
||||
str(isolated_fixture),
|
||||
],
|
||||
cwd=workspace,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
self.assertEqual(completed.returncode, 0, completed.stderr or completed.stdout)
|
||||
evidence = json.loads(completed.stdout)
|
||||
self.assertEqual(evidence["campaign_id"], "simple-announcement")
|
||||
self.assertEqual(evidence["built_count"], 1)
|
||||
self.assertEqual(evidence["queueable_count"], 1)
|
||||
self.assertRegex(evidence["normalized_eml_sha256"], r"^[0-9a-f]{64}$")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -94,7 +94,7 @@ def _ensure(session: _Session, version) -> None:
|
||||
ensure_execution_snapshot(session, version) # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_v5_snapshot_requires_its_persisted_checksum() -> None:
|
||||
def test_current_snapshot_requires_its_persisted_checksum() -> None:
|
||||
job = _job()
|
||||
version = _snapshotted_version(job)
|
||||
version.execution_snapshot_hash = None
|
||||
|
||||
@@ -7,9 +7,11 @@ from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine, text
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
from sqlalchemy import create_engine, inspect, text
|
||||
|
||||
from govoplan_campaign.backend import router
|
||||
from govoplan_campaign.backend.routes import delivery as router
|
||||
from govoplan_campaign.backend.db.models import (
|
||||
CampaignJob,
|
||||
ImapAppendAttempt,
|
||||
@@ -128,6 +130,10 @@ def test_post_provider_persistence_failure_freezes_imap_retry() -> None:
|
||||
mail_profile_id="profile-1",
|
||||
smtp_transport_revision="smtp-revision",
|
||||
imap_transport_revision="imap-revision",
|
||||
smtp_server_id=None,
|
||||
smtp_credential_id=None,
|
||||
imap_server_id=None,
|
||||
imap_credential_id=None,
|
||||
delivery=SimpleNamespace(
|
||||
imap_append_sent=SimpleNamespace(enabled=True, folder="Sent"),
|
||||
),
|
||||
@@ -153,6 +159,11 @@ def test_post_provider_persistence_failure_freezes_imap_retry() -> None:
|
||||
patch("govoplan_campaign.backend.sending.jobs._load_eml_bytes_for_job", return_value=b"message"),
|
||||
patch("govoplan_campaign.backend.sending.jobs._claim_job_for_imap_append", return_value="claim-1"),
|
||||
patch("govoplan_campaign.backend.sending.jobs._record_imap_attempt_start", return_value=attempt),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._begin_imap_append_recovery",
|
||||
return_value=SimpleNamespace(replayed=False, operation=MagicMock()),
|
||||
),
|
||||
patch("govoplan_campaign.backend.sending.jobs._finish_imap_append_recovery"),
|
||||
patch("govoplan_campaign.backend.sending.jobs.mail_integration", return_value=Mail()),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._record_imap_append_success",
|
||||
@@ -218,6 +229,31 @@ def test_imap_claim_migration_preserves_and_renumbers_duplicate_attempts() -> No
|
||||
]
|
||||
|
||||
|
||||
def test_imap_attempt_claim_repair_adds_only_the_missing_column() -> None:
|
||||
migration = importlib.import_module(
|
||||
"govoplan_campaign.backend.migrations.versions.e9f0a1b2c3d4_v0114_repair_imap_attempt_claim"
|
||||
)
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
with engine.begin() as connection:
|
||||
connection.execute(
|
||||
text(
|
||||
"CREATE TABLE imap_append_attempts ("
|
||||
"id VARCHAR(36) PRIMARY KEY, job_id VARCHAR(36) NOT NULL)"
|
||||
)
|
||||
)
|
||||
context = MigrationContext.configure(connection)
|
||||
with patch.object(migration, "op", Operations(context)):
|
||||
migration.upgrade()
|
||||
migration.upgrade()
|
||||
|
||||
columns = {
|
||||
column["name"]
|
||||
for column in inspect(connection).get_columns("imap_append_attempts")
|
||||
}
|
||||
|
||||
assert columns == {"id", "job_id", "claim_token"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("decision", ["smtp_accepted", "not_sent", "imap_appended", "imap_not_appended"])
|
||||
def test_reconciliation_requires_an_evidence_note(decision: str) -> None:
|
||||
with pytest.raises(ValueError, match="evidence note"):
|
||||
@@ -258,6 +294,7 @@ def test_imap_reconciliation_preserves_attempt_and_only_not_appended_is_retryabl
|
||||
imap_claimed_at=datetime.now(timezone.utc),
|
||||
imap_claim_token="claim-1",
|
||||
last_error="unknown",
|
||||
delivery_provenance={},
|
||||
)
|
||||
attempt = SimpleNamespace(
|
||||
id="attempt-1",
|
||||
@@ -379,14 +416,14 @@ def test_reconciliation_rolls_back_state_when_audit_fails(
|
||||
return {"decision": decision, "job_id": "job-1"}
|
||||
|
||||
with (
|
||||
patch("govoplan_campaign.backend.router._get_campaign_for_principal"),
|
||||
patch("govoplan_campaign.backend.router._require_permission"),
|
||||
patch("govoplan_campaign.backend.routes.delivery._get_campaign_for_principal"),
|
||||
patch("govoplan_campaign.backend.routes.delivery._require_permission"),
|
||||
patch(
|
||||
"govoplan_campaign.backend.router.reconcile_job_outcome",
|
||||
"govoplan_campaign.backend.routes.delivery.reconcile_job_outcome",
|
||||
side_effect=mutate_without_commit,
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.router.audit_from_principal",
|
||||
"govoplan_campaign.backend.routes.delivery.audit_from_principal",
|
||||
side_effect=RuntimeError("audit unavailable"),
|
||||
),
|
||||
):
|
||||
|
||||
@@ -8,7 +8,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
|
||||
from govoplan_campaign.backend.router import (
|
||||
from govoplan_campaign.backend.services.job_queries import (
|
||||
_campaign_jobs_grid_filter_expressions,
|
||||
_campaign_jobs_ordering,
|
||||
_campaign_jobs_page_response,
|
||||
|
||||
@@ -6,7 +6,8 @@ from unittest.mock import call, patch
|
||||
import pytest
|
||||
from fastapi import HTTPException
|
||||
|
||||
from govoplan_campaign.backend import router
|
||||
from govoplan_campaign.backend import route_support
|
||||
from govoplan_campaign.backend.routes import versions as router
|
||||
from govoplan_campaign.backend.campaign.loader import CampaignSchemaError, validate_against_schema
|
||||
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
|
||||
CampaignMailProfileBoundaryError,
|
||||
@@ -73,7 +74,7 @@ def test_mail_profile_documentation_is_classified_for_adaptive_views() -> None:
|
||||
def test_campaign_mail_contract_rejects_every_legacy_server_field(legacy_key: str) -> None:
|
||||
raw = _campaign_json({"mail_profile_id": "profile-1", legacy_key: {}})
|
||||
|
||||
with pytest.raises(CampaignMailProfileBoundaryError, match="select an authorized Mail profile"):
|
||||
with pytest.raises(CampaignMailProfileBoundaryError, match="select authorized Mail resources"):
|
||||
assert_campaign_uses_mail_profile_reference(raw)
|
||||
|
||||
|
||||
@@ -138,7 +139,7 @@ def test_new_execution_snapshot_stores_reference_and_evidence_not_transport_mate
|
||||
delivery=DeliveryConfig(),
|
||||
)
|
||||
|
||||
assert payload["snapshot_version"] == "5"
|
||||
assert payload["snapshot_version"] == "8"
|
||||
assert payload["mail_profile_id"] == "profile-1"
|
||||
assert "smtp" not in payload
|
||||
assert "imap" not in payload
|
||||
@@ -214,7 +215,7 @@ def test_fork_inherited_profile_requires_mail_profile_use_scope() -> None:
|
||||
patch.object(router, "_get_campaign_for_principal", return_value=campaign),
|
||||
patch.object(router, "_require_permission"),
|
||||
patch.object(router, "_get_version_for_tenant", return_value=source),
|
||||
patch.object(router, "has_scope", return_value=False),
|
||||
patch.object(route_support, "has_scope", return_value=False),
|
||||
patch.object(router, "fork_campaign_version_for_edit") as fork,
|
||||
):
|
||||
with pytest.raises(HTTPException) as captured:
|
||||
@@ -234,8 +235,8 @@ def test_campaign_wide_effect_authorizes_every_affected_version() -> None:
|
||||
session = object()
|
||||
principal = SimpleNamespace(tenant_id="tenant-1")
|
||||
|
||||
with patch.object(router, "_require_campaign_profile_use_if_needed") as require_profile:
|
||||
router._require_campaign_versions_profile_use( # noqa: SLF001 - security boundary regression test
|
||||
with patch.object(route_support, "_require_campaign_profile_use_if_needed") as require_profile:
|
||||
route_support._require_campaign_versions_profile_use( # noqa: SLF001 - security boundary regression test
|
||||
session, # type: ignore[arg-type]
|
||||
principal, # type: ignore[arg-type]
|
||||
"campaign-1",
|
||||
|
||||
@@ -0,0 +1,480 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import smtplib
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_campaign.backend.campaign import load_campaign_config
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||
RUNNER_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "run_campaign_acceptance.py"
|
||||
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
|
||||
|
||||
|
||||
def _load_runner():
|
||||
spec = importlib.util.spec_from_file_location("govoplan_campaign_greenmail_acceptance", RUNNER_PATH)
|
||||
assert spec is not None and spec.loader is not None
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
runner = _load_runner()
|
||||
|
||||
|
||||
class _Response:
|
||||
def __init__(self, status_code: int, payload: dict[str, Any]) -> None:
|
||||
self.status_code = status_code
|
||||
self._payload = payload
|
||||
|
||||
def json(self) -> dict[str, Any]:
|
||||
return self._payload
|
||||
|
||||
|
||||
class _AcceptanceClient:
|
||||
def __init__(self) -> None:
|
||||
self.campaign_json: dict[str, Any] | None = None
|
||||
self.send_calls = 0
|
||||
|
||||
def post(self, path: str, **kwargs: Any) -> _Response:
|
||||
if path == "/api/v1/campaigns":
|
||||
self.campaign_json = kwargs["json"]["config"]
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"campaign": {"id": "campaign-internal"},
|
||||
"version": {"id": "version-internal"},
|
||||
},
|
||||
)
|
||||
if path.endswith("/validate"):
|
||||
return _Response(200, {"ok": True, "error_count": 0, "warning_count": 0})
|
||||
if path.endswith("/build"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"built_count": 1,
|
||||
"build_failed_count": 0,
|
||||
"queueable_count": 1,
|
||||
},
|
||||
)
|
||||
if path.endswith("/send-now"):
|
||||
self.send_calls += 1
|
||||
if self.send_calls == 2:
|
||||
return _Response(422, {"detail": "Already accepted"})
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"result": {
|
||||
"attempted_count": 1,
|
||||
"sent_count": 1,
|
||||
"failed_count": 0,
|
||||
"outcome_unknown_count": 0,
|
||||
"skipped_count": 0,
|
||||
"delivery_mode": "synchronous",
|
||||
"results": [{"job_id": "not-retained", "status": "smtp_accepted"}],
|
||||
}
|
||||
},
|
||||
)
|
||||
if path.endswith("/append-sent"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"result": {
|
||||
"pending_count": 1,
|
||||
"processed_count": 1,
|
||||
"appended_count": 1,
|
||||
"failed_count": 0,
|
||||
"skipped_count": 0,
|
||||
"results": [{"job_id": "not-retained", "status": "appended"}],
|
||||
}
|
||||
},
|
||||
)
|
||||
raise AssertionError(f"unexpected POST {path}")
|
||||
|
||||
def get(self, path: str, **kwargs: Any) -> _Response:
|
||||
if path.endswith("/report"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"cards": {
|
||||
"jobs_total": 1,
|
||||
"sent": 1,
|
||||
"smtp_accepted": 1,
|
||||
"failed": 0,
|
||||
"outcome_unknown": 0,
|
||||
"retryable": 0,
|
||||
"needs_attention": 0,
|
||||
"imap_appended": 1,
|
||||
"imap_failed": 0,
|
||||
},
|
||||
"status_counts": {
|
||||
"send": {"smtp_accepted": 1},
|
||||
"imap": {"appended": 1},
|
||||
},
|
||||
},
|
||||
)
|
||||
raise AssertionError(f"unexpected GET {path}")
|
||||
|
||||
|
||||
def _settings():
|
||||
return runner.TestbedSettings(
|
||||
smtp_host="127.0.0.1",
|
||||
smtp_port=3025,
|
||||
imap_host="127.0.0.1",
|
||||
imap_port=3143,
|
||||
username="campaign-test@govoplan.test",
|
||||
password="local-test-password",
|
||||
sender="campaign-test@govoplan.test",
|
||||
recipient="campaign-test@govoplan.test",
|
||||
sent_folder="Sent",
|
||||
provider_timeout_seconds=5,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["localhost", "mail.test", "192.168.1.20", "8.8.8.8"])
|
||||
def test_testbed_rejects_hostnames_and_non_loopback_addresses(host: str) -> None:
|
||||
settings = _settings()
|
||||
rejected = runner.TestbedSettings(
|
||||
smtp_host=host,
|
||||
smtp_port=settings.smtp_port,
|
||||
imap_host=settings.imap_host,
|
||||
imap_port=settings.imap_port,
|
||||
username=settings.username,
|
||||
password=settings.password,
|
||||
sender=settings.sender,
|
||||
recipient=settings.recipient,
|
||||
sent_folder=settings.sent_folder,
|
||||
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="literal loopback|restricted to the loopback"):
|
||||
rejected.assert_local_testbed()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["127.0.0.1", "127.8.9.10", "::1"])
|
||||
def test_testbed_accepts_literal_loopback_and_preserves_it_in_profile(host: str) -> None:
|
||||
settings = _settings()
|
||||
accepted = runner.TestbedSettings(
|
||||
smtp_host=host,
|
||||
smtp_port=settings.smtp_port,
|
||||
imap_host=host,
|
||||
imap_port=settings.imap_port,
|
||||
username=settings.username,
|
||||
password=settings.password,
|
||||
sender=settings.sender,
|
||||
recipient=settings.recipient,
|
||||
sent_folder=settings.sent_folder,
|
||||
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
|
||||
accepted.assert_local_testbed()
|
||||
profile = runner._profile_payload(accepted, name="Literal loopback")
|
||||
assert profile["smtp"]["host"] == host
|
||||
assert profile["imap"]["host"] == host
|
||||
|
||||
|
||||
def test_campaign_acceptance_orchestration_retains_only_profile_reference_and_safe_evidence() -> None:
|
||||
client = _AcceptanceClient()
|
||||
|
||||
def snapshot_probe(_version_id: str):
|
||||
assert client.campaign_json is not None
|
||||
return client.campaign_json, {
|
||||
"mail_profile_id": "profile-1",
|
||||
"smtp_transport_revision": "opaque-smtp-revision",
|
||||
"imap_transport_revision": "opaque-imap-revision",
|
||||
"delivery": {"imap_append_sent": {"enabled": True, "folder": "Sent"}},
|
||||
}
|
||||
|
||||
audit = {
|
||||
"campaign.created": 1,
|
||||
"campaign.validated": 1,
|
||||
"campaign.messages_built": 1,
|
||||
"campaign.sent_now": 1,
|
||||
"campaign.send_now_rejected": 1,
|
||||
"campaign.append_sent_enqueued": 1,
|
||||
}
|
||||
evidence, subject = runner.execute_campaign_scenario(
|
||||
client,
|
||||
{"Authorization": "not-retained"},
|
||||
fixture_path=FIXTURE_PATH,
|
||||
profile_id="profile-1",
|
||||
settings=_settings(),
|
||||
scenario="success",
|
||||
snapshot_probe=snapshot_probe,
|
||||
audit_probe=lambda _campaign_id, _version_id: audit,
|
||||
append_sent=True,
|
||||
repeat_send=True,
|
||||
)
|
||||
evidence["provider_verification"] = {
|
||||
"inbox_increment": 1,
|
||||
"sent_increment": 1,
|
||||
"unique_subject_matches_in_inbox": 1,
|
||||
"unique_subject_matches_in_sent": 1,
|
||||
}
|
||||
|
||||
runner._assert_success_evidence(evidence)
|
||||
runner._assert_evidence_safe(
|
||||
{
|
||||
"schema_version": runner.EVIDENCE_SCHEMA,
|
||||
"coverage": {
|
||||
"smtp_acceptance": True,
|
||||
"partial_envelope_refusal": False,
|
||||
"post_data_connection_loss_outcome_unknown": False,
|
||||
"source_artifact_provenance": False,
|
||||
"worker_restart_interruption": False,
|
||||
},
|
||||
"success": evidence,
|
||||
},
|
||||
settings=_settings(),
|
||||
)
|
||||
assert subject.startswith("[GovOPlaN acceptance ")
|
||||
assert client.send_calls == 2
|
||||
assert client.campaign_json is not None
|
||||
assert client.campaign_json["server"] == {"mail_profile_id": "profile-1"}
|
||||
assert "credentials" not in json.dumps(client.campaign_json).casefold()
|
||||
serialized = json.dumps(evidence, sort_keys=True)
|
||||
assert "not-retained" not in serialized
|
||||
assert "local-test-password" not in serialized
|
||||
|
||||
|
||||
def test_campaign_boundary_rejects_resolved_transport_material() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="forbidden transport material"):
|
||||
runner.assert_campaign_boundary(
|
||||
{"server": {"mail_profile_id": "profile-1"}},
|
||||
{
|
||||
"mail_profile_id": "profile-1",
|
||||
"smtp_transport_revision": "smtp-revision",
|
||||
"imap_transport_revision": "imap-revision",
|
||||
"smtp": {"host": "should-not-be-here"},
|
||||
},
|
||||
profile_id="profile-1",
|
||||
)
|
||||
|
||||
|
||||
def test_success_projection_fails_closed_on_inconsistent_campaign_report() -> None:
|
||||
evidence = {
|
||||
"send": {
|
||||
"attempted_count": 1,
|
||||
"sent_count": 1,
|
||||
"failed_count": 0,
|
||||
"outcome_unknown_count": 0,
|
||||
"skipped_count": 0,
|
||||
"delivery_mode": "synchronous",
|
||||
"statuses": {"smtp_accepted": 1},
|
||||
},
|
||||
"append_sent": {
|
||||
"pending_count": 1,
|
||||
"processed_count": 1,
|
||||
"appended_count": 1,
|
||||
"failed_count": 0,
|
||||
"skipped_count": 0,
|
||||
"statuses": {"appended": 1},
|
||||
},
|
||||
"report": {
|
||||
"cards": {
|
||||
"jobs_total": 1,
|
||||
"sent": 0,
|
||||
"smtp_accepted": 0,
|
||||
"failed": 0,
|
||||
"outcome_unknown": 0,
|
||||
"needs_attention": 0,
|
||||
"imap_appended": 0,
|
||||
"imap_failed": 0,
|
||||
},
|
||||
"send_status_counts": {},
|
||||
"imap_status_counts": {},
|
||||
},
|
||||
"provider_verification": {
|
||||
"inbox_increment": 1,
|
||||
"sent_increment": 1,
|
||||
"unique_subject_matches_in_inbox": 1,
|
||||
"unique_subject_matches_in_sent": 1,
|
||||
},
|
||||
"campaign_mail_boundary": {
|
||||
"profile_reference_only": True,
|
||||
"smtp_revision_frozen": True,
|
||||
"imap_revision_frozen": True,
|
||||
"resolved_transport_material_present": False,
|
||||
},
|
||||
}
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="report does not agree"):
|
||||
runner._assert_success_evidence(evidence)
|
||||
|
||||
|
||||
def test_evidence_projection_rejects_unknown_status_keys() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="unsupported status"):
|
||||
runner._send_evidence(
|
||||
{
|
||||
"delivery_mode": "synchronous",
|
||||
"results": [{"status": "provider diagnostic: recipient@example.test"}],
|
||||
}
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="durable attempt status"):
|
||||
runner._durable_state_evidence(
|
||||
{
|
||||
"job_count": 1,
|
||||
"send_status_counts": {"sending": 1},
|
||||
"attempt_status_counts": {"provider-secret": 1},
|
||||
"unfinished_attempt_count": 1,
|
||||
}
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="synchronous delivery mode"):
|
||||
runner._send_evidence(
|
||||
{
|
||||
"delivery_mode": "provider diagnostic: recipient@example.test",
|
||||
"results": [],
|
||||
}
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="unsupported"):
|
||||
runner._report_evidence(
|
||||
{
|
||||
"cards": {},
|
||||
"status_counts": {
|
||||
"send": {"smtp_accepted": 1, "provider-secret": 1},
|
||||
"imap": {},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _open_fault_smtp(endpoint):
|
||||
client = smtplib.SMTP(endpoint.host, endpoint.port, timeout=5)
|
||||
client.ehlo()
|
||||
client.login("acceptance-user", "acceptance-password")
|
||||
return client
|
||||
|
||||
|
||||
def test_explicit_temporary_smtp_response_occurs_after_complete_data() -> None:
|
||||
with runner.smtp_fault_endpoint("temporary_data_response") as endpoint:
|
||||
client = _open_fault_smtp(endpoint)
|
||||
try:
|
||||
with pytest.raises(smtplib.SMTPDataError) as captured:
|
||||
client.sendmail(
|
||||
"sender@example.test",
|
||||
["recipient@example.test"],
|
||||
b"Subject: temporary\r\n\r\nmessage",
|
||||
)
|
||||
finally:
|
||||
client.close()
|
||||
assert captured.value.smtp_code == 451
|
||||
assert endpoint.evidence() == {
|
||||
"connection_count": 1,
|
||||
"accepted_rcpt_commands": 1,
|
||||
"refused_rcpt_commands": 0,
|
||||
"data_transactions": 1,
|
||||
}
|
||||
|
||||
|
||||
def test_partial_recipient_refusal_retains_one_accepted_envelope() -> None:
|
||||
with runner.smtp_fault_endpoint("partial_recipient_refusal") as endpoint:
|
||||
client = _open_fault_smtp(endpoint)
|
||||
try:
|
||||
refused = client.sendmail(
|
||||
"sender@example.test",
|
||||
["accepted@example.test", "refused@example.test"],
|
||||
b"Subject: partial\r\n\r\nmessage",
|
||||
)
|
||||
finally:
|
||||
client.quit()
|
||||
assert set(refused) == {"refused@example.test"}
|
||||
assert endpoint.evidence() == {
|
||||
"connection_count": 1,
|
||||
"accepted_rcpt_commands": 1,
|
||||
"refused_rcpt_commands": 1,
|
||||
"data_transactions": 1,
|
||||
}
|
||||
|
||||
|
||||
def test_post_data_disconnect_is_a_real_ambiguous_protocol_boundary() -> None:
|
||||
with runner.smtp_fault_endpoint("post_data_disconnect") as endpoint:
|
||||
client = _open_fault_smtp(endpoint)
|
||||
try:
|
||||
with pytest.raises(smtplib.SMTPServerDisconnected):
|
||||
client.sendmail(
|
||||
"sender@example.test",
|
||||
["recipient@example.test"],
|
||||
b"Subject: ambiguous\r\n\r\nmessage",
|
||||
)
|
||||
finally:
|
||||
client.close()
|
||||
assert endpoint.wait_for_data(1)
|
||||
assert endpoint.evidence()["data_transactions"] == 1
|
||||
|
||||
|
||||
def test_partial_refusal_fixture_adds_a_second_distinct_recipient() -> None:
|
||||
raw, _subject = runner.materialize_campaign_fixture(
|
||||
FIXTURE_PATH,
|
||||
profile_id="profile-1",
|
||||
settings=_settings(),
|
||||
scenario="partial_envelope_refusal",
|
||||
run_token="0123456789ab",
|
||||
additional_envelope_recipient=True,
|
||||
)
|
||||
|
||||
recipients = raw["entries"]["inline"][0]["to"]
|
||||
assert len(recipients) == 2
|
||||
assert recipients[0]["email"] != recipients[1]["email"]
|
||||
assert recipients[1]["email"].endswith("@govoplan.test")
|
||||
|
||||
def test_fixture_contains_no_transport_credentials() -> None:
|
||||
raw = json.loads(FIXTURE_PATH.read_text(encoding="utf-8"))
|
||||
runner._assert_no_forbidden_campaign_keys(raw)
|
||||
assert raw["server"] == {"mail_profile_id": "00000000-0000-4000-8000-000000000001"}
|
||||
config = load_campaign_config(FIXTURE_PATH)
|
||||
assert config.server.mail_profile_id == "00000000-0000-4000-8000-000000000001"
|
||||
|
||||
|
||||
def test_fixture_composition_versions_are_complete_and_exact() -> None:
|
||||
versions = {
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"audit": "0.1.8",
|
||||
"campaigns": "0.1.11",
|
||||
"mail": "0.1.10",
|
||||
"files": "0.1.9",
|
||||
}
|
||||
|
||||
assert runner.required_composition_versions(FIXTURE_PATH, versions) == {
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"audit": "0.1.8",
|
||||
"campaigns": "0.1.11",
|
||||
"mail": "0.1.10",
|
||||
}
|
||||
|
||||
|
||||
def test_fixture_composition_fails_closed_when_a_required_version_is_missing() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="versions are unavailable"):
|
||||
runner.required_composition_versions(
|
||||
FIXTURE_PATH,
|
||||
{
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"campaigns": "0.1.11",
|
||||
"mail": "0.1.10",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_testbed_documentation_distinguishes_proven_and_open_failure_drills() -> None:
|
||||
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(encoding="utf-8")
|
||||
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(encoding="utf-8")
|
||||
|
||||
assert "second ordinary send must be rejected before another provider effect" in testbed
|
||||
assert "connection loss after complete DATA is frozen" in testbed
|
||||
assert "dedicated OS process" in testbed
|
||||
assert "Redis/Celery delivery" in runbook
|
||||
assert "broker redelivery" in runbook
|
||||
assert "celery_broker_redelivery" in testbed
|
||||
assert "raw provider diagnostics" in runbook
|
||||
@@ -0,0 +1,64 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from govoplan_campaign.backend.manifest import (
|
||||
CAMPAIGN_MODULE_REQUIRED_ANY,
|
||||
OPERATOR_QUEUE_REQUIRED_ANY,
|
||||
OPERATOR_QUEUE_SURFACE_ID,
|
||||
REPORTS_REQUIRED_ANY,
|
||||
REPORTS_SURFACE_ID,
|
||||
get_manifest,
|
||||
)
|
||||
from govoplan_core.core.registry import manifest_view_surfaces
|
||||
|
||||
|
||||
def test_operator_queue_is_an_integrated_campaign_view() -> None:
|
||||
manifest = get_manifest()
|
||||
assert manifest.frontend is not None
|
||||
|
||||
assert "/operator" not in {item.path for item in manifest.nav_items}
|
||||
assert "/operator" not in {item.path for item in manifest.frontend.nav_items}
|
||||
|
||||
routes = {route.path: route for route in manifest.frontend.routes}
|
||||
assert "/operator" not in routes
|
||||
queue = routes["/campaigns/queue"]
|
||||
assert queue.component == "OperatorQueuePage"
|
||||
assert queue.required_all == ("campaigns:campaign:read",)
|
||||
assert queue.required_any == OPERATOR_QUEUE_REQUIRED_ANY
|
||||
assert OPERATOR_QUEUE_SURFACE_ID == "campaigns.route.operator"
|
||||
assert queue.surface_id == OPERATOR_QUEUE_SURFACE_ID
|
||||
|
||||
queue_surfaces = [
|
||||
surface
|
||||
for surface in manifest_view_surfaces(manifest)
|
||||
if surface.id == OPERATOR_QUEUE_SURFACE_ID
|
||||
]
|
||||
assert len(queue_surfaces) == 1
|
||||
assert queue_surfaces[0].description == "/campaigns/queue"
|
||||
|
||||
|
||||
def test_aggregate_reports_are_an_integrated_campaign_view() -> None:
|
||||
manifest = get_manifest()
|
||||
assert manifest.frontend is not None
|
||||
|
||||
backend_nav = {item.path: item for item in manifest.nav_items}
|
||||
frontend_nav = {item.path: item for item in manifest.frontend.nav_items}
|
||||
assert "/reports" not in backend_nav
|
||||
assert "/reports" not in frontend_nav
|
||||
assert backend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
|
||||
assert frontend_nav["/campaigns"].required_any == CAMPAIGN_MODULE_REQUIRED_ANY
|
||||
|
||||
routes = {route.path: route for route in manifest.frontend.routes}
|
||||
assert "/reports" not in routes
|
||||
report = routes["/campaigns/reports"]
|
||||
assert report.component == "AggregateReportsPage"
|
||||
assert report.required_any == REPORTS_REQUIRED_ANY
|
||||
assert REPORTS_SURFACE_ID == "campaigns.route.reports"
|
||||
assert report.surface_id == REPORTS_SURFACE_ID
|
||||
|
||||
report_surfaces = [
|
||||
surface
|
||||
for surface in manifest_view_surfaces(manifest)
|
||||
if surface.id == REPORTS_SURFACE_ID
|
||||
]
|
||||
assert len(report_surfaces) == 1
|
||||
assert report_surfaces[0].description == "/campaigns/reports"
|
||||
@@ -0,0 +1,18 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from govoplan_core.core.object_storage import LocalFilesystemStorageBackend
|
||||
from govoplan_campaign.backend.operational_checks import generated_eml_storage_check
|
||||
|
||||
|
||||
def test_generated_eml_probe_reports_node_local_boundary(tmp_path) -> None:
|
||||
with patch(
|
||||
"govoplan_campaign.backend.operational_checks.configured_storage_backend",
|
||||
return_value=LocalFilesystemStorageBackend(tmp_path),
|
||||
):
|
||||
result = generated_eml_storage_check()
|
||||
|
||||
assert result.state == "warning"
|
||||
assert "node-local" in result.detail
|
||||
assert list(tmp_path.rglob("*.probe")) == []
|
||||
@@ -5,7 +5,7 @@ import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from govoplan_campaign.backend import router
|
||||
from govoplan_campaign.backend.routes import campaigns as router
|
||||
from govoplan_campaign.backend.campaign.models import CampaignConfig
|
||||
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
|
||||
from govoplan_campaign.backend.persistence.versions import validate_campaign_partial
|
||||
@@ -78,6 +78,177 @@ class CampaignPartialValidationTests(unittest.TestCase):
|
||||
self.assertTrue(response.available)
|
||||
self.assertEqual(response.candidates[0].email, "ada@example.local")
|
||||
|
||||
def test_address_source_snapshot_preserves_governance_exclusions(self) -> None:
|
||||
class RecipientSourceCapability:
|
||||
def snapshot(self, _session, _principal, **kwargs):
|
||||
self.kwargs = kwargs
|
||||
return {
|
||||
"source_id": "addresses:address_book:book-1",
|
||||
"source_label": "Residents",
|
||||
"source_kind": "local",
|
||||
"source_revision": "revision-2",
|
||||
"generated_at": "2026-07-31T10:00:00+00:00",
|
||||
"purpose": "campaign_delivery",
|
||||
"recipients": [
|
||||
{
|
||||
"contact_id": "contact-1",
|
||||
"display_name": "Ada Lovelace",
|
||||
"email": "ada@example.local",
|
||||
"fields": {},
|
||||
"provenance": {"channel_decision": {"governance_state": "opted_in"}},
|
||||
}
|
||||
],
|
||||
"excluded": [
|
||||
{
|
||||
"contact_id": "contact-2",
|
||||
"display_name": "Grace Hopper",
|
||||
"channel": "email",
|
||||
"target": "grace@example.local",
|
||||
"status": "suppressed",
|
||||
"reason_code": "addresses.channel.opted_out",
|
||||
"provenance": {"channel_decision": {"governance_state": "opted_out"}},
|
||||
}
|
||||
],
|
||||
"provenance": {"governance_applied": True},
|
||||
}
|
||||
|
||||
capability = RecipientSourceCapability()
|
||||
payload = router.CampaignRecipientAddressSourceSnapshotRequest(
|
||||
source_id="addresses:address_book:book-1"
|
||||
)
|
||||
with patch.object(router, "_get_campaign_for_principal", return_value=None), patch.object(
|
||||
router, "_registry_capability", return_value=capability
|
||||
):
|
||||
response = router.snapshot_campaign_recipient_address_source(
|
||||
"campaign-1", payload, session=object(), principal=object()
|
||||
)
|
||||
|
||||
self.assertEqual(capability.kwargs["purpose"], "campaign_delivery")
|
||||
self.assertEqual(capability.kwargs["requested_channels"], ("email",))
|
||||
self.assertEqual(response.included_count, 1)
|
||||
self.assertEqual(response.excluded_count, 1)
|
||||
self.assertEqual(response.excluded[0].reason_code, "addresses.channel.opted_out")
|
||||
self.assertTrue(response.provenance["governance_applied"])
|
||||
|
||||
def test_distribution_list_preview_preserves_provider_and_route_evidence(self) -> None:
|
||||
class ExpansionCapability:
|
||||
def expand(self, _session, _principal, *, request):
|
||||
self.request = request
|
||||
return _distribution_expansion_payload()
|
||||
|
||||
capability = ExpansionCapability()
|
||||
payload = router.CampaignDistributionListExpansionRequest(
|
||||
list_id="11111111-1111-1111-1111-111111111111",
|
||||
revision=3,
|
||||
requested_channels=["email", "postal"],
|
||||
parameters={"district": "north"},
|
||||
)
|
||||
with patch.object(router, "_get_campaign_for_principal", return_value=None), patch.object(
|
||||
router, "_registry_capability", return_value=capability
|
||||
):
|
||||
response = router.preview_campaign_recipient_distribution_list(
|
||||
"campaign-1", payload, session=object(), principal=object()
|
||||
)
|
||||
|
||||
self.assertTrue(capability.request.preview)
|
||||
self.assertFalse(capability.request.freeze)
|
||||
self.assertEqual(capability.request.parameters, {"district": "north"})
|
||||
self.assertEqual(response.source.revision, 3)
|
||||
self.assertEqual(response.recipients[0].source_entry_ids, ["entry-1"])
|
||||
self.assertEqual(response.recipients[0].channels[0].decision_provenance["policy"], "allow")
|
||||
self.assertEqual(response.provider_evidence[0].actual_revision, "provider-r7")
|
||||
|
||||
def test_distribution_list_snapshot_requires_and_forwards_idempotency(self) -> None:
|
||||
class ExpansionCapability:
|
||||
def expand(self, _session, _principal, *, request):
|
||||
self.request = request
|
||||
payload = _distribution_expansion_payload()
|
||||
payload["snapshot_id"] = "snapshot-1"
|
||||
return payload
|
||||
|
||||
capability = ExpansionCapability()
|
||||
payload = router.CampaignDistributionListExpansionRequest(
|
||||
list_id="11111111-1111-1111-1111-111111111111",
|
||||
requested_channels=["email"],
|
||||
idempotency_key="campaign:one:list:one",
|
||||
)
|
||||
with patch.object(router, "_get_campaign_for_principal", return_value=None), patch.object(
|
||||
router, "_registry_capability", return_value=capability
|
||||
), patch.object(router, "audit_from_principal") as audit:
|
||||
response = router.snapshot_campaign_recipient_distribution_list(
|
||||
"campaign-1", payload, session=object(), principal=object()
|
||||
)
|
||||
|
||||
self.assertTrue(capability.request.freeze)
|
||||
self.assertFalse(capability.request.preview)
|
||||
self.assertEqual(capability.request.idempotency_key, "campaign:one:list:one")
|
||||
self.assertEqual(response.snapshot_id, "snapshot-1")
|
||||
self.assertTrue(audit.call_args.kwargs["commit"])
|
||||
|
||||
|
||||
def _distribution_expansion_payload() -> dict[str, object]:
|
||||
source = {
|
||||
"id": "11111111-1111-1111-1111-111111111111",
|
||||
"tenant_id": "tenant-1",
|
||||
"name": "Residents",
|
||||
"revision_id": "revision-id-3",
|
||||
"revision": 3,
|
||||
"definition_hash": "definition-hash",
|
||||
"definition_kind": "parameterized",
|
||||
"entry_count": 1,
|
||||
"parameters": [],
|
||||
"provenance": {},
|
||||
"metadata": {},
|
||||
}
|
||||
reference = {
|
||||
"provider": "addresses",
|
||||
"resource_type": "address_list",
|
||||
"resource_id": "list-1",
|
||||
"revision": "provider-r7",
|
||||
"metadata": {},
|
||||
}
|
||||
recipient = {
|
||||
"recipient_key": "contact:1",
|
||||
"display_name": "Ada Lovelace",
|
||||
"status": "usable",
|
||||
"channels": [
|
||||
{
|
||||
"channel": "email",
|
||||
"target": "ada@example.local",
|
||||
"target_key": "email:ada@example.local",
|
||||
"status": "usable",
|
||||
"preferred": True,
|
||||
"source": reference,
|
||||
"decision_provenance": {"policy": "allow"},
|
||||
}
|
||||
],
|
||||
"contact_id": "contact-1",
|
||||
"source_entry_ids": ["entry-1"],
|
||||
"explanations": [],
|
||||
"attributes": {"district": "north"},
|
||||
"provenance": {"provider": "addresses"},
|
||||
}
|
||||
return {
|
||||
"source": source,
|
||||
"request": {"parameters": {"district": "north"}},
|
||||
"recipients": [recipient],
|
||||
"excluded": [],
|
||||
"diagnostics": [],
|
||||
"provider_evidence": [
|
||||
{
|
||||
"provider": "addresses",
|
||||
"source": reference,
|
||||
"actual_revision": "provider-r7",
|
||||
"stale": False,
|
||||
"details": {},
|
||||
}
|
||||
],
|
||||
"expansion_hash": "expansion-hash",
|
||||
"generated_at": "2026-08-02T10:00:00+00:00",
|
||||
"stale": False,
|
||||
"truncated": False,
|
||||
}
|
||||
|
||||
|
||||
class CampaignSemanticValidationTests(unittest.TestCase):
|
||||
def test_send_mode_requires_campaign_owned_sender_for_each_inline_entry(self) -> None:
|
||||
|
||||
@@ -0,0 +1,478 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy
|
||||
from govoplan_campaign.backend.db.models import JobSendStatus
|
||||
from govoplan_campaign.backend.sending.jobs import (
|
||||
SendJobResult,
|
||||
_MailChannelOutcome,
|
||||
_PrintChannelOutcome,
|
||||
_final_multichannel_status,
|
||||
_send_claimed_multichannel_job,
|
||||
)
|
||||
from govoplan_campaign.backend.sending.postbox_delivery import (
|
||||
PostboxChannelOutcome,
|
||||
)
|
||||
|
||||
|
||||
def _context():
|
||||
return SimpleNamespace(
|
||||
message_bytes=b"message",
|
||||
snapshot=SimpleNamespace(
|
||||
delivery=SimpleNamespace(
|
||||
postbox=SimpleNamespace(classification="internal")
|
||||
)
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _job():
|
||||
return SimpleNamespace(id="job-1")
|
||||
|
||||
|
||||
class _Session:
|
||||
def __init__(self, job) -> None:
|
||||
self.job = job
|
||||
|
||||
def get(self, _model, _id):
|
||||
return self.job
|
||||
|
||||
def add(self, _value) -> None:
|
||||
return None
|
||||
|
||||
def commit(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class PostboxFallbackOrchestrationTests(unittest.TestCase):
|
||||
def test_mail_unknown_never_starts_print_fallback(self) -> None:
|
||||
job = SimpleNamespace(id="job-1", print_status="ready")
|
||||
expected = SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||
attempt_number=1,
|
||||
)
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||
return_value=_MailChannelOutcome(outcome_unknown=True),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_print_channel"
|
||||
) as deliver_print,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=expected,
|
||||
),
|
||||
):
|
||||
result = _send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_PRINT,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
self.assertIs(result, expected)
|
||||
deliver_print.assert_not_called()
|
||||
|
||||
def test_mail_preacceptance_rejection_starts_print_fallback(self) -> None:
|
||||
job = SimpleNamespace(id="job-1", print_status="ready")
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||
return_value=_MailChannelOutcome(rejected_permanent=True),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_print_channel",
|
||||
return_value=_PrintChannelOutcome(accepted=True),
|
||||
) as deliver_print,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.PRINT_ACCEPTED.value,
|
||||
attempt_number=1,
|
||||
),
|
||||
),
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_PRINT,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_print.assert_called_once()
|
||||
|
||||
def test_mail_unknown_never_starts_postbox_fallback(self) -> None:
|
||||
job = _job()
|
||||
expected = SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||
attempt_number=1,
|
||||
)
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||
return_value=_MailChannelOutcome(outcome_unknown=True),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||
return_value=PostboxChannelOutcome(),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
|
||||
) as deliver_postbox,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=expected,
|
||||
),
|
||||
):
|
||||
result = _send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
self.assertIs(result, expected)
|
||||
deliver_postbox.assert_not_called()
|
||||
|
||||
def test_mail_preacceptance_rejection_starts_postbox_fallback(self) -> None:
|
||||
job = _job()
|
||||
postbox_outcome = PostboxChannelOutcome(accepted=1)
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||
return_value=_MailChannelOutcome(rejected_permanent=True),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||
return_value=PostboxChannelOutcome(),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||
return_value=postbox_outcome,
|
||||
) as deliver_postbox,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||
attempt_number=1,
|
||||
),
|
||||
),
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_postbox.assert_called_once()
|
||||
|
||||
def test_accepted_postbox_fallback_prevents_later_mail_retry(self) -> None:
|
||||
job = _job()
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||
return_value=PostboxChannelOutcome(
|
||||
accepted=1,
|
||||
rejected_temporary=1,
|
||||
),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||
) as deliver_mail,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||
return_value=PostboxChannelOutcome(accepted=2),
|
||||
) as deliver_postbox,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||
attempt_number=3,
|
||||
),
|
||||
),
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_mail.assert_not_called()
|
||||
deliver_postbox.assert_called_once()
|
||||
|
||||
def test_unknown_postbox_fallback_prevents_every_later_effect(self) -> None:
|
||||
job = _job()
|
||||
prior = PostboxChannelOutcome(outcome_unknown=1)
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._postbox_outcome_from_attempts",
|
||||
return_value=prior,
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||
) as deliver_mail,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes"
|
||||
) as deliver_postbox,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||
attempt_number=1,
|
||||
),
|
||||
) as finalize,
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_mail.assert_not_called()
|
||||
deliver_postbox.assert_not_called()
|
||||
self.assertIs(finalize.call_args.kwargs["postbox"], prior)
|
||||
|
||||
def test_postbox_acceptance_stops_mail_fallback(self) -> None:
|
||||
job = _job()
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||
return_value=PostboxChannelOutcome(
|
||||
accepted=1,
|
||||
rejected_permanent=1,
|
||||
),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel"
|
||||
) as deliver_mail,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
attempt_number=2,
|
||||
),
|
||||
),
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_mail.assert_not_called()
|
||||
|
||||
def test_all_postbox_rejections_start_mail_fallback(self) -> None:
|
||||
job = _job()
|
||||
with (
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs.deliver_campaign_job_to_postboxes",
|
||||
return_value=PostboxChannelOutcome(
|
||||
rejected_temporary=1,
|
||||
rejected_permanent=1,
|
||||
),
|
||||
),
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._deliver_mail_channel",
|
||||
return_value=_MailChannelOutcome(accepted=True),
|
||||
) as deliver_mail,
|
||||
patch(
|
||||
"govoplan_campaign.backend.sending.jobs._finalize_multichannel_job",
|
||||
return_value=SendJobResult(
|
||||
job_id=job.id,
|
||||
status=JobSendStatus.SMTP_ACCEPTED.value,
|
||||
attempt_number=1,
|
||||
),
|
||||
),
|
||||
):
|
||||
_send_claimed_multichannel_job(
|
||||
_Session(job), # type: ignore[arg-type]
|
||||
job=job, # type: ignore[arg-type]
|
||||
claim_token="claim-1",
|
||||
context=_context(), # type: ignore[arg-type]
|
||||
channel_policy=DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||
use_rate_limit=False,
|
||||
enqueue_imap_task=False,
|
||||
)
|
||||
|
||||
deliver_mail.assert_called_once()
|
||||
|
||||
def test_explicit_dual_delivery_reports_partial_and_unknown(self) -> None:
|
||||
self.assertEqual(
|
||||
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
_final_multichannel_status(
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
mail=_MailChannelOutcome(accepted=True),
|
||||
postbox=PostboxChannelOutcome(rejected_permanent=1),
|
||||
),
|
||||
)
|
||||
self.assertEqual(
|
||||
JobSendStatus.OUTCOME_UNKNOWN.value,
|
||||
_final_multichannel_status(
|
||||
channel_policy=DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
mail=_MailChannelOutcome(accepted=True),
|
||||
postbox=PostboxChannelOutcome(outcome_unknown=1),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
|
||||
@pytest.mark.parametrize(
|
||||
("mail", "postbox"),
|
||||
[
|
||||
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome()),
|
||||
(_MailChannelOutcome(accepted=True), PostboxChannelOutcome(outcome_unknown=1)),
|
||||
(_MailChannelOutcome(outcome_unknown=True), PostboxChannelOutcome(accepted=1)),
|
||||
],
|
||||
)
|
||||
def test_outcome_unknown_has_precedence_for_every_delivery_policy(
|
||||
policy: DeliveryChannelPolicy,
|
||||
mail: _MailChannelOutcome,
|
||||
postbox: PostboxChannelOutcome,
|
||||
) -> None:
|
||||
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == JobSendStatus.OUTCOME_UNKNOWN.value
|
||||
|
||||
|
||||
@pytest.mark.parametrize("policy", list(DeliveryChannelPolicy))
|
||||
@pytest.mark.parametrize(
|
||||
("mail", "postbox", "expected"),
|
||||
[
|
||||
(
|
||||
_MailChannelOutcome(rejected_permanent=True),
|
||||
PostboxChannelOutcome(rejected_permanent=1),
|
||||
JobSendStatus.FAILED_PERMANENT.value,
|
||||
),
|
||||
(
|
||||
_MailChannelOutcome(rejected_temporary=True),
|
||||
PostboxChannelOutcome(rejected_permanent=1),
|
||||
JobSendStatus.FAILED_TEMPORARY.value,
|
||||
),
|
||||
(
|
||||
_MailChannelOutcome(rejected_permanent=True),
|
||||
PostboxChannelOutcome(rejected_temporary=1),
|
||||
JobSendStatus.FAILED_TEMPORARY.value,
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_rejection_precedence_is_exhaustive_for_every_delivery_policy(
|
||||
policy: DeliveryChannelPolicy,
|
||||
mail: _MailChannelOutcome,
|
||||
postbox: PostboxChannelOutcome,
|
||||
expected: str,
|
||||
) -> None:
|
||||
assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("policy", "mail", "postbox", "expected"),
|
||||
[
|
||||
(
|
||||
DeliveryChannelPolicy.MAIL,
|
||||
_MailChannelOutcome(accepted=True),
|
||||
PostboxChannelOutcome(),
|
||||
JobSendStatus.SMTP_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.POSTBOX,
|
||||
None,
|
||||
PostboxChannelOutcome(accepted=1),
|
||||
JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.POSTBOX,
|
||||
None,
|
||||
PostboxChannelOutcome(accepted=1, rejected_permanent=1),
|
||||
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
_MailChannelOutcome(accepted=True),
|
||||
PostboxChannelOutcome(accepted=1),
|
||||
JobSendStatus.DELIVERED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
_MailChannelOutcome(accepted=True),
|
||||
PostboxChannelOutcome(rejected_permanent=1),
|
||||
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
|
||||
_MailChannelOutcome(rejected_permanent=True),
|
||||
PostboxChannelOutcome(accepted=1),
|
||||
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
|
||||
_MailChannelOutcome(rejected_permanent=True),
|
||||
PostboxChannelOutcome(accepted=1),
|
||||
JobSendStatus.POSTBOX_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
|
||||
_MailChannelOutcome(accepted=True),
|
||||
PostboxChannelOutcome(rejected_permanent=1),
|
||||
JobSendStatus.PARTIALLY_ACCEPTED.value,
|
||||
),
|
||||
(
|
||||
DeliveryChannelPolicy.PRINT,
|
||||
None,
|
||||
PostboxChannelOutcome(),
|
||||
JobSendStatus.PRINT_ACCEPTED.value,
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_accepted_delivery_decision_table(
|
||||
policy: DeliveryChannelPolicy,
|
||||
mail: _MailChannelOutcome | None,
|
||||
postbox: PostboxChannelOutcome,
|
||||
expected: str,
|
||||
) -> None:
|
||||
print_output = _PrintChannelOutcome(accepted=True) if policy == DeliveryChannelPolicy.PRINT else None
|
||||
assert _final_multichannel_status(
|
||||
channel_policy=policy,
|
||||
mail=mail,
|
||||
postbox=postbox,
|
||||
print_output=print_output,
|
||||
) == expected
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,145 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from govoplan_core.core.postbox import (
|
||||
PostboxDeliveryCatalogRef,
|
||||
PostboxDirectoryEntryRef,
|
||||
PostboxDeliveryRequest,
|
||||
PostboxDeliveryReceiptSummaryRef,
|
||||
PostboxDeliveryResult,
|
||||
PostboxTargetRef,
|
||||
)
|
||||
from govoplan_campaign.backend.integrations import (
|
||||
PostboxCampaignIntegration,
|
||||
PostboxDeliveryUnavailable,
|
||||
)
|
||||
|
||||
|
||||
class _PostboxDelivery:
|
||||
def __init__(self) -> None:
|
||||
self.requests = []
|
||||
|
||||
def deliver(self, session, request):
|
||||
self.requests.append((session, request))
|
||||
return PostboxDeliveryResult(
|
||||
delivery_id="delivery-1",
|
||||
postbox_id="postbox-1",
|
||||
message_id="message-1",
|
||||
address="intake@postbox",
|
||||
status="accepted",
|
||||
vacant=False,
|
||||
holder_count=1,
|
||||
)
|
||||
|
||||
def delivery_catalog(self, session, *, tenant_id):
|
||||
return PostboxDeliveryCatalogRef()
|
||||
|
||||
def list_visible_postboxes(self, session, *, tenant_id, actor):
|
||||
return ()
|
||||
|
||||
def resolve_postbox(
|
||||
self,
|
||||
session,
|
||||
*,
|
||||
tenant_id,
|
||||
target,
|
||||
materialize=False,
|
||||
):
|
||||
return PostboxDirectoryEntryRef(
|
||||
id=target.postbox_id or "postbox-1",
|
||||
tenant_id=tenant_id,
|
||||
address="intake@postbox",
|
||||
address_key="intake",
|
||||
name="Intake",
|
||||
status="active",
|
||||
classification="internal",
|
||||
)
|
||||
|
||||
def link_evidence(self, session, *, tenant_id, message_id, attachment):
|
||||
del session, tenant_id, message_id, attachment
|
||||
return None
|
||||
|
||||
def delivery_receipt_summaries(
|
||||
self,
|
||||
session,
|
||||
*,
|
||||
tenant_id,
|
||||
producer_module,
|
||||
delivery_ids,
|
||||
):
|
||||
del session
|
||||
assert tenant_id == "tenant-1"
|
||||
assert producer_module == "campaigns"
|
||||
return {
|
||||
delivery_id: PostboxDeliveryReceiptSummaryRef(
|
||||
delivery_id=delivery_id,
|
||||
message_id="message-1",
|
||||
postbox_id="postbox-1",
|
||||
delivery_status="accepted",
|
||||
accepted_at=datetime(2026, 7, 30, tzinfo=UTC),
|
||||
currently_readable=True,
|
||||
read_receipt_count=1,
|
||||
)
|
||||
for delivery_id in delivery_ids
|
||||
}
|
||||
|
||||
|
||||
class PostboxCampaignIntegrationTests(unittest.TestCase):
|
||||
def test_optional_delivery_boundary_is_typed_and_explicit(self) -> None:
|
||||
delegate = _PostboxDelivery()
|
||||
integration = PostboxCampaignIntegration(delegate, delegate, delegate)
|
||||
request = PostboxDeliveryRequest(
|
||||
tenant_id="tenant-1",
|
||||
target=PostboxTargetRef(postbox_id="postbox-1"),
|
||||
producer_module="campaigns",
|
||||
producer_resource_type="campaign_job",
|
||||
producer_resource_id="job-1",
|
||||
idempotency_key="campaign-1:job-1:postbox-1",
|
||||
subject="Decision",
|
||||
)
|
||||
|
||||
result = integration.deliver(object(), request)
|
||||
|
||||
self.assertTrue(integration.available)
|
||||
self.assertTrue(integration.receipt_evidence_available)
|
||||
self.assertEqual("delivery-1", result.delivery_id)
|
||||
self.assertEqual(request, delegate.requests[0][1])
|
||||
summaries = integration.delivery_receipt_summaries(
|
||||
object(),
|
||||
tenant_id="tenant-1",
|
||||
delivery_ids=["delivery-1"],
|
||||
)
|
||||
self.assertEqual(1, summaries["delivery-1"].read_receipt_count)
|
||||
|
||||
def test_missing_postbox_is_reported_without_importing_module_code(
|
||||
self,
|
||||
) -> None:
|
||||
integration = PostboxCampaignIntegration()
|
||||
request = PostboxDeliveryRequest(
|
||||
tenant_id="tenant-1",
|
||||
target=PostboxTargetRef(postbox_id="postbox-1"),
|
||||
producer_module="campaigns",
|
||||
producer_resource_type="campaign_job",
|
||||
producer_resource_id="job-1",
|
||||
idempotency_key="campaign-1:job-1:postbox-1",
|
||||
subject="Decision",
|
||||
)
|
||||
|
||||
self.assertFalse(integration.available)
|
||||
self.assertFalse(integration.receipt_evidence_available)
|
||||
self.assertEqual(
|
||||
{},
|
||||
integration.delivery_receipt_summaries(
|
||||
object(),
|
||||
tenant_id="tenant-1",
|
||||
delivery_ids=["delivery-1"],
|
||||
),
|
||||
)
|
||||
with self.assertRaises(PostboxDeliveryUnavailable):
|
||||
integration.deliver(object(), request)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user