GovOPlaN/govoplan-campaign
1
0
Fork 0
Code Issues 46 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(campaign): bound synchronous delivery

Browse Source
This commit is contained in:
Albrecht Degering
2026-07-22 08:21:42 +02:00
parent a8c0750dd7
commit 7e1660344d
10 changed files with 689 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@@ -93,6 +93,7 @@ Platform RBAC and governance rules are documented in `govoplan-core/docs/`.
- [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives.
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
- Immediate delivery is bounded to 25 exact eligible recipient jobs by default. Deployments may set `GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0–500), and tenants may narrow that ceiling through `campaign_delivery_policy.synchronous_send_max_recipients` in tenant settings.
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/add-ideas/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
- [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path.
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.

19
docs/CAMPAIGN_DELIVERY_RUNBOOK.md
View File

@@ -5,10 +5,12 @@ been validated, built, reviewed, and locked.
## Operating Modes
- Local direct send: `CELERY_ENABLED=false`. Queueing stores jobs in the DB, and
small development runs can be processed with "Send queued now".
- Local direct send: `CELERY_ENABLED=false`. **Send now** is available only for
exact built runs within the effective synchronous limit. It preflights the
complete batch before the first SMTP effect.
- Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing
publishes delivery tasks, and the Review & Send page polls summary counters.
publishes durable delivery tasks, and Review and send polls their persisted
summary counters even after the initiating request has returned.
- Mock send: use only for development review. It does not prove real SMTP/IMAP
credentials or server policy.
@@ -26,6 +28,8 @@ been validated, built, reviewed, and locked.
ZIP before using production recipients.
- Keep the report page open during tests; it is the operational source of truth
for attempts, outcomes, and reconciliation.
- Confirm the effective Send now recipient-job limit. The safe default is 25;
use Queue for workers for ordinary batches or any run above that limit.
## Deliverability Preflight
@@ -49,10 +53,15 @@ Before the first live send for a sender domain or mail-server profile:
1. Validate the version with file checks enabled.
2. Build the version and inspect all blocking review items.
3. Queue only after the selected version is the intended immutable execution
version.
4. In local mode, use "Send queued now" for small test runs.
version. Select **Queue for workers**, then verify the committed and
published counts.
4. Use **Send now** only if the exact eligible count is non-zero and at or below
the effective deployment/tenant limit shown on the page.
5. In worker mode, verify queue counters move from queued/claimed/sending to a
terminal SMTP state.
6. If a synchronous request is used, keep Review and send open: it polls the
durable counters while the request runs. A rejection occurs before SMTP and
directs oversized runs to workers.
## Outcome Handling

20
docs/CAMPAIGN_HANDBOOK.md
View File

@@ -156,11 +156,13 @@ operator sequence.
At a minimum:
1. Queue only a validated, locked, built version. The current Web UI does not
expose Queue; use an authorized supporting client or API.
2. Use worker delivery for ordinary batches. Synchronous Send now has no
server-enforced job-count bound and must be used only after an operator has
deliberately confirmed a small controlled run.
1. Queue only a validated, locked, built version. Use **Queue for workers** for
ordinary batches; the durable progress remains visible after leaving and
returning to Review and send.
2. Use **Send now** only when the exact persisted eligible build is within the
effective synchronous limit shown by the UI. The default deployment limit
is 25 recipient jobs. The backend repeats the count and preflights every
message and the Mail profile revision before contacting SMTP.
3. Treat `smtp_accepted` as protected from ordinary retry.
4. Retry `failed_temporary` explicitly after inspecting the cause.
5. Include `failed_permanent` only after correcting the cause and making a
@@ -179,6 +181,14 @@ Pause stops new eligible work but cannot undo a provider effect already in
progress. Cancel marks work that has not yet produced a protected SMTP outcome;
it cannot recall accepted mail.
The deployment ceiling is configured with
`GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0 disables Send now; the
accepted range is 0–500). A tenant may only narrow that ceiling with
`tenant.settings.campaign_delivery_policy.synchronous_send_max_recipients`.
The effective value and source are returned by the protected delivery-options
API, recorded for successful/rejected synchronous commands, and stated in the
configured handbook topic.
### Test and one-message actions
The current baseline includes mock send, queue dry-run, synchronous immediate

107
src/govoplan_campaign/backend/delivery_policy.py Normal file
View File

@@ -0,0 +1,107 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.tenancy.scope import Tenant
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 25
ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 500
SYNCHRONOUS_SEND_MAX_ENV = "GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS"
CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY = "campaign_delivery_policy"
SYNCHRONOUS_SEND_MAX_SETTINGS_KEY = "synchronous_send_max_recipients"
class CampaignDeliveryPolicyError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class SynchronousSendPolicy:
max_recipient_jobs: int
source: str
deployment_max_recipient_jobs: int
tenant_max_recipient_jobs: int | None = None
def as_dict(self) -> dict[str, Any]:
return {
"max_recipient_jobs": self.max_recipient_jobs,
"source": self.source,
"deployment_max_recipient_jobs": self.deployment_max_recipient_jobs,
"tenant_max_recipient_jobs": self.tenant_max_recipient_jobs,
"deployment_setting": SYNCHRONOUS_SEND_MAX_ENV,
"tenant_setting": (
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
}
def effective_synchronous_send_policy(
session: Session,
*,
tenant_id: str,
environ: Mapping[str, str] | None = None,
) -> SynchronousSendPolicy:
env = os.environ if environ is None else environ
deployment_value = _configured_limit(
env.get(SYNCHRONOUS_SEND_MAX_ENV),
source=SYNCHRONOUS_SEND_MAX_ENV,
default=DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
)
tenant = session.get(Tenant, tenant_id)
tenant_raw = _tenant_limit_value(tenant.settings if tenant is not None else None)
if tenant_raw is None:
return SynchronousSendPolicy(
max_recipient_jobs=deployment_value,
source=("deployment" if env.get(SYNCHRONOUS_SEND_MAX_ENV) not in (None, "") else "deployment_default"),
deployment_max_recipient_jobs=deployment_value,
)
tenant_value = _configured_limit(
tenant_raw,
source=(
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
)
effective_value = min(deployment_value, tenant_value)
return SynchronousSendPolicy(
max_recipient_jobs=effective_value,
source="tenant" if tenant_value <= deployment_value else "deployment_ceiling",
deployment_max_recipient_jobs=deployment_value,
tenant_max_recipient_jobs=tenant_value,
)
def _tenant_limit_value(settings: Mapping[str, Any] | None) -> object | None:
if not isinstance(settings, Mapping):
return None
policy = settings.get(CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY)
if not isinstance(policy, Mapping):
return None
return policy.get(SYNCHRONOUS_SEND_MAX_SETTINGS_KEY)
def _configured_limit(value: object, *, source: str, default: int | None = None) -> int:
if value is None or (isinstance(value, str) and not value.strip()):
if default is not None:
return default
raise CampaignDeliveryPolicyError(f"{source} must be configured as an integer")
if isinstance(value, bool):
raise CampaignDeliveryPolicyError(f"{source} must be an integer, not a boolean")
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise CampaignDeliveryPolicyError(f"{source} must be an integer") from exc
if str(parsed) != str(value).strip() and not isinstance(value, int):
raise CampaignDeliveryPolicyError(f"{source} must be an integer")
if parsed < 0 or parsed > ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS:
raise CampaignDeliveryPolicyError(
f"{source} must be between 0 and {ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS}"
)
return parsed

40
src/govoplan_campaign/backend/documentation.py
View File

@@ -2,6 +2,11 @@ from __future__ import annotations
from govoplan_core.core.modules import DocumentationCondition, DocumentationContext, DocumentationLink, DocumentationTopic
from govoplan_campaign.backend.delivery_policy import (
CampaignDeliveryPolicyError,
effective_synchronous_send_policy,
)
_CAMPAIGN_USER_SCOPES = (
"campaigns:campaign:read",
@@ -278,21 +283,20 @@ CAMPAIGN_USER_DOCUMENTATION = (
),
steps=(
"Open Review and send and confirm the selected version, build, recipients, warnings, and review completion.",
"Use an authorized supporting client to run the queue dry run and resolve every blocker.",
"Invoke the Queue action through that client so background workers can process the build.",
"Open the campaign Report and monitor queue, SMTP, IMAP, failure, and uncertain-outcome counts.",
"Select Queue for workers and confirm the exact durable execution that will be committed.",
"Remain on Review and send or leave and return later; both views read the same durable job states.",
"Use the delivery controls to pause, resume, cancel unsent work, or retry eligible failures without requeueing accepted or uncertain outcomes.",
),
outcome="Eligible jobs queued with an auditable execution snapshot and protected delivery state.",
verification="The Report shows the selected version's jobs as queued or progressing, without changing any accepted job back to retryable.",
related_topic_ids=("campaigns.workflow.complete-review", "campaigns.workflow.retry-and-reconcile"),
related_modules=("mail", "notifications"),
limitations=("The current Campaign Web UI does not expose the Queue action; use an authorized supporting client or API.",),
),
_workflow_topic(
topic_id="campaigns.workflow.send-small-controlled-run",
title="Send a campaign immediately",
summary="Run the eligible jobs synchronously only after deliberately confirming that the reviewed campaign is small enough for an interactive request.",
body="The current synchronous action does not enforce a server-side job-count limit. It must not replace the worker queue for ordinary batches, but it does preserve the same protected SMTP, IMAP, retry, and uncertain-outcome rules.",
body="Send now is protected by an effective deployment/tenant recipient-job maximum. The server counts the exact persisted eligible build, rejects an oversized or empty run before SMTP, and preflights every message and the Mail profile revision before the first provider effect.",
order=36,
audience=("campaign_sender", "campaign_operator"),
required_modules=("campaigns", "mail"),
@@ -308,12 +312,12 @@ CAMPAIGN_USER_DOCUMENTATION = (
help_contexts=("campaign.review-send",),
prerequisites=(
"The selected version is validated, locked, built, and reviewed.",
"An operator has deliberately confirmed that this is a small controlled run; the server does not currently enforce that bound.",
"The exact eligible recipient-job count is within the effective limit shown on Review and send.",
"Its Mail profile remains available and authorized for the current campaign context.",
),
steps=(
"Open Review and send and inspect the exact build and delivery readiness checks.",
"Confirm that this is an intentionally small controlled run; use Queue for an ordinary batch.",
"Review the effective synchronous limit and exact eligible count; use Queue for workers when Send now is unavailable or for an ordinary batch.",
"Select Send now and confirm the protected delivery action.",
"Open Report and inspect each resulting delivery state before attempting any recovery action.",
),
@@ -321,7 +325,6 @@ CAMPAIGN_USER_DOCUMENTATION = (
verification="The Report distinguishes accepted, failed, unattempted, cancelled, and outcome-unknown jobs and retains their attempt history.",
related_topic_ids=("campaigns.workflow.queue-delivery", "campaigns.workflow.retry-and-reconcile"),
related_modules=("mail",),
limitations=("No server-side synchronous job-count limit is currently enforced; use Queue and workers for ordinary batches.",),
),
_workflow_topic(
topic_id="campaigns.workflow.view-delivery-report",
@@ -481,10 +484,13 @@ def documentation_topics(context: DocumentationContext) -> tuple[DocumentationTo
current_configuration = list(_actor_capabilities(principal, mail_available=mail_available))
integration_configuration, integration_limitations = _integration_summary(context.registry, principal)
current_configuration.extend(integration_configuration)
delivery_configuration, delivery_limitations = _delivery_policy_summary(context)
current_configuration.extend(delivery_configuration)
limitations = [
"Access to a particular campaign still depends on its owner or sharing rules and on the campaign's current state.",
"Profile, file, and address pickers re-evaluate the actual resources visible in the selected campaign; this overview does not claim that an eligible item currently exists.",
*integration_limitations,
*delivery_limitations,
]
return (
@@ -640,6 +646,24 @@ def _integration_summary(registry: object, principal: object) -> tuple[tuple[str
return tuple(configured), tuple(limitations)
def _delivery_policy_summary(context: DocumentationContext) -> tuple[tuple[str, ...], tuple[str, ...]]:
session = context.session
tenant_id = str(getattr(context.principal, "tenant_id", "") or "").strip()
if session is None or not tenant_id:
return (), ("The effective synchronous Campaign limit could not be resolved for this documentation request.",)
try:
policy = effective_synchronous_send_policy(session, tenant_id=tenant_id) # type: ignore[arg-type]
except CampaignDeliveryPolicyError as exc:
return (), (f"Synchronous Campaign delivery is disabled until its policy configuration is corrected: {exc}",)
return (
(
"Delivery policy: Send now is limited to "
f"{policy.max_recipient_jobs} eligible recipient job(s) for this tenant. "
"The exact built count and Queue for workers alternative are shown before confirmation."
),
), ()
def _append_if(
target: list[str],
principal: object,

50
src/govoplan_campaign/backend/router.py
View File

@@ -44,6 +44,7 @@ from govoplan_campaign.backend.schemas import (
CampaignSendJobRequest,
CampaignSendUnattemptedRequest,
CampaignResolveOutcomeRequest,
CampaignDeliveryOptionsResponse,
CampaignListResponse,
CampaignResponse,
CampaignVersionDetailResponse,
@@ -65,7 +66,7 @@ from govoplan_campaign.backend.schemas import (
SendCampaignNowRequest,
SendCampaignNowResponse,
)
from govoplan_core.auth import ApiPrincipal, has_scope, require_scope
from govoplan_core.auth import ApiPrincipal, has_scope, require_any_scope, require_scope
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.core.access import CAPABILITY_ACCESS_DIRECTORY, AccessDirectory
from govoplan_core.core.change_sequence import (
@@ -148,6 +149,7 @@ from govoplan_campaign.backend.dev.mock_campaign import MockCampaignSendError, r
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, clear_execution_snapshot
from govoplan_campaign.backend.sending.jobs import (
QueueingError,
SynchronousSendRejected,
cancel_campaign_jobs,
enqueue_pending_imap_appends,
pause_campaign_jobs,
@@ -158,6 +160,7 @@ from govoplan_campaign.backend.sending.jobs import (
resume_campaign_jobs,
send_campaign_now,
send_single_campaign_job,
synchronous_send_options,
)
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
@@ -3019,6 +3022,30 @@ def revoke_campaign_share(
# Queue / delivery control -------------------------------------------------
@router.get("/{campaign_id}/delivery-options", response_model=CampaignDeliveryOptionsResponse)
def campaign_delivery_options(
campaign_id: str,
version_id: str | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(
require_any_scope("campaigns:campaign:send", "campaigns:campaign:queue")
),
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
try:
return CampaignDeliveryOptionsResponse(
**synchronous_send_options(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
)
)
except QueueingError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)) from exc
@router.post("/{campaign_id}/queue", response_model=QueueCampaignResponse)
def queue_campaign(
campaign_id: str,
@@ -3264,12 +3291,7 @@ def send_campaign_now_endpoint(
):
_get_campaign_for_principal(session, campaign_id, principal, write=True)
_require_permission(principal, "campaigns:recipient:read")
"""Validate/build/queue and synchronously send every eligible job.
No server-side job-count bound is currently enforced. Ordinary batches
must use the queue/Celery flow; this endpoint is for deliberately small
interactive runs.
"""
"""Preflight and synchronously send a policy-bounded built execution."""
payload = payload or SendCampaignNowRequest()
try:
@@ -3320,6 +3342,20 @@ def send_campaign_now_endpoint(
commit=True,
)
return SendCampaignNowResponse(result=result)
except SynchronousSendRejected as exc:
audit_from_principal(
session,
principal,
action="campaign.send_now_rejected",
object_type="campaign",
object_id=campaign_id,
details={
**exc.audit_details(),
"version_id": payload.version_id,
},
commit=True,
)
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)) from exc
except HTTPException:
raise
except (CampaignPersistenceError, QueueingError) as exc:

7
src/govoplan_campaign/backend/schemas.py
View File

@@ -512,6 +512,13 @@ class SendCampaignNowResponse(BaseModel):
result: dict[str, Any]
class CampaignDeliveryOptionsResponse(BaseModel):
campaign_id: str
version_id: str
worker_queue_available: bool
synchronous_send: dict[str, Any] = Field(default_factory=dict)
class MockCampaignSendRequest(BaseModel):
model_config = ConfigDict(extra="forbid")

283
src/govoplan_campaign/backend/sending/jobs.py
View File

@@ -28,7 +28,17 @@ from govoplan_campaign.backend.db.models import (
ImapAppendAttempt,
SendAttempt,
)
from govoplan_campaign.backend.sending.execution import ExecutionSnapshot, ExecutionSnapshotError, ensure_execution_snapshot
from govoplan_campaign.backend.delivery_policy import (
CampaignDeliveryPolicyError,
SynchronousSendPolicy,
effective_synchronous_send_policy,
)
from govoplan_campaign.backend.sending.execution import (
ExecutionSnapshot,
ExecutionSnapshotError,
ensure_execution_snapshot,
profile_delivery_summary,
)
from govoplan_campaign.backend.runtime import get_registry
from govoplan_campaign.backend.integrations import (
ImapAppendError,
@@ -49,6 +59,29 @@ class SendJobError(RuntimeError):
pass
class SynchronousSendRejected(QueueingError):
def __init__(
self,
message: str,
*,
reason: str,
eligible_count: int | None = None,
policy: SynchronousSendPolicy | None = None,
) -> None:
super().__init__(message)
self.reason = reason
self.eligible_count = eligible_count
self.policy = policy
def audit_details(self) -> dict[str, Any]:
return {
"delivery_mode": "synchronous",
"rejection_reason": self.reason,
"eligible_recipient_job_count": self.eligible_count,
"synchronous_send_policy": self.policy.as_dict() if self.policy is not None else None,
}
@dataclass(frozen=True, slots=True)
class QueueCampaignResult:
campaign_id: str
@@ -80,6 +113,8 @@ class SendCampaignNowResult:
failed_count: int
outcome_unknown_count: int
skipped_count: int
preflight_count: int = 0
synchronous_send_policy: dict[str, Any] | None = None
dry_run: bool = False
results: list[dict[str, Any]] | None = None
@@ -92,6 +127,9 @@ class SendCampaignNowResult:
"failed_count": self.failed_count,
"outcome_unknown_count": self.outcome_unknown_count,
"skipped_count": self.skipped_count,
"preflight_count": self.preflight_count,
"delivery_mode": "synchronous",
"synchronous_send_policy": self.synchronous_send_policy or {},
"dry_run": self.dry_run,
"results": self.results or [],
}
@@ -444,6 +482,115 @@ def _select_campaign_jobs_for_queue(
return queued, skipped_count, blocked_count
def synchronous_send_candidate_jobs(
version: CampaignVersion,
jobs: list[CampaignJob],
*,
include_warnings: bool = True,
) -> list[CampaignJob]:
"""Return the exact persisted job set an immediate send may attempt.
This projection is deliberately side-effect free so the delivery-options
API and adaptive documentation can explain the current built execution
without changing queue state. The final command repeats the projection
after queueing and before any SMTP effect.
"""
allowed_validation = _queue_validation_statuses(include_warnings=include_warnings)
reviewed_needs_review_keys = _reviewed_needs_review_keys(version)
candidates: list[CampaignJob] = []
for job in jobs:
if (
job.queue_status == JobQueueStatus.QUEUED.value
and job.send_status == JobSendStatus.QUEUED.value
):
candidates.append(job)
continue
if job.send_status in INITIAL_QUEUE_SKIPPED_SEND_STATUSES:
continue
if job.queue_status in INITIAL_QUEUE_SKIPPED_QUEUE_STATUSES:
continue
validation_allowed = job.validation_status in allowed_validation or (
job.validation_status == JobValidationStatus.NEEDS_REVIEW.value
and _job_review_key(job) in reviewed_needs_review_keys
)
if job.build_status != JobBuildStatus.BUILT.value or not validation_allowed:
continue
if not job.eml_local_path and not job.eml_storage_key:
continue
candidates.append(job)
return candidates
def synchronous_send_options(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str | None = None,
include_warnings: bool = True,
) -> dict[str, Any]:
"""Describe the actor-independent effective delivery-mode constraints."""
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
version = _get_version_for_campaign(session, campaign, version_id=version_id)
jobs = _campaign_jobs_for_version(session, tenant_id=tenant_id, version_id=version.id)
candidates = synchronous_send_candidate_jobs(version, jobs, include_warnings=include_warnings)
try:
policy = effective_synchronous_send_policy(session, tenant_id=tenant_id)
except CampaignDeliveryPolicyError as exc:
return {
"campaign_id": campaign.id,
"version_id": version.id,
"worker_queue_available": bool(_celery_enabled()),
"synchronous_send": {
"allowed": False,
"reason": "policy_configuration_invalid",
"message": str(exc),
"eligible_recipient_job_count": len(candidates),
"policy": {},
},
}
ready = _version_is_validated_and_locked(version) and bool(version.build_summary)
eligible_count = len(candidates)
if not ready:
reason = "version_not_ready"
elif eligible_count == 0:
reason = "no_eligible_recipient_jobs"
elif eligible_count > policy.max_recipient_jobs:
reason = "recipient_limit_exceeded"
else:
reason = None
return {
"campaign_id": campaign.id,
"version_id": version.id,
"worker_queue_available": bool(_celery_enabled()),
"synchronous_send": {
"allowed": reason is None,
"reason": reason,
"eligible_recipient_job_count": eligible_count,
"policy": policy.as_dict(),
},
}
def _campaign_jobs_for_version(
session: Session,
*,
tenant_id: str,
version_id: str,
) -> list[CampaignJob]:
return (
session.query(CampaignJob)
.filter(
CampaignJob.tenant_id == tenant_id,
CampaignJob.campaign_version_id == version_id,
)
.order_by(CampaignJob.entry_index.asc())
.all()
)
def _persist_campaign_queue(
session: Session,
*,
@@ -546,13 +693,36 @@ def send_campaign_now(
) -> SendCampaignNowResult:
"""Queue and send all eligible jobs for one campaign synchronously.
The service currently has no server-side job-count bound. Callers must use
the worker queue for ordinary batches and reserve this path for deliberately
small interactive runs.
The effective deployment/tenant limit is checked against the immutable,
persisted job set before queue state changes. After queueing, every message
and transport revision is preflighted before the first SMTP effect starts.
"""
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
version = _get_current_version(session, campaign, version_id=version_id)
_ensure_version_validated_and_locked(version)
_ensure_campaign_execution_snapshot(session, version)
try:
synchronous_policy = effective_synchronous_send_policy(session, tenant_id=tenant_id)
except CampaignDeliveryPolicyError as exc:
raise SynchronousSendRejected(
f"Synchronous Campaign delivery is disabled by an invalid policy configuration: {exc}",
reason="policy_configuration_invalid",
) from exc
initial_jobs = _campaign_jobs_for_queue(
session,
tenant_id=tenant_id,
version_id=version.id,
)
initial_candidates = synchronous_send_candidate_jobs(
version,
initial_jobs,
include_warnings=include_warnings,
)
_ensure_synchronous_send_count_allowed(
len(initial_candidates),
policy=synchronous_policy,
)
queue_result = queue_campaign_jobs(
session,
@@ -572,20 +742,31 @@ def send_campaign_now(
failed_count=0,
outcome_unknown_count=0,
skipped_count=queue_result.skipped_count + queue_result.blocked_count,
preflight_count=len(initial_candidates),
synchronous_send_policy=synchronous_policy.as_dict(),
dry_run=True,
results=[queue_result.as_dict()],
)
jobs = (
session.query(CampaignJob)
.filter(
CampaignJob.tenant_id == tenant_id,
CampaignJob.campaign_version_id == version.id,
CampaignJob.queue_status == JobQueueStatus.QUEUED.value,
CampaignJob.send_status.in_([JobSendStatus.QUEUED.value, JobSendStatus.FAILED_TEMPORARY.value]),
jobs = [
job
for job in _campaign_jobs_for_version(
session,
tenant_id=tenant_id,
version_id=version.id,
)
.order_by(CampaignJob.entry_index.asc())
.all()
if job.queue_status == JobQueueStatus.QUEUED.value
and job.send_status == JobSendStatus.QUEUED.value
]
# Repeat the hard bound against the post-queue set. This closes the window
# where a concurrent queue operation could otherwise enlarge an immediate
# run between the initial decision and the first provider effect.
_ensure_synchronous_send_count_allowed(len(jobs), policy=synchronous_policy)
delivery_contexts = _preflight_synchronous_send_batch(
session,
version=version,
jobs=jobs,
policy=synchronous_policy,
)
results: list[dict[str, Any]] = []
@@ -595,10 +776,16 @@ def send_campaign_now(
skipped_after_queue = 0
for job in jobs:
try:
result = send_campaign_job(
claimed = _claimed_campaign_job_for_delivery(session, job)
if isinstance(claimed, SendJobResult):
result = claimed
else:
claimed_job, claim_token = claimed
result = _send_claimed_campaign_job(
session,
job_id=job.id,
dry_run=False,
job=claimed_job,
claim_token=claim_token,
context=delivery_contexts[job.id],
use_rate_limit=use_rate_limit,
enqueue_imap_task=enqueue_imap_task,
)
@@ -622,11 +809,75 @@ def send_campaign_now(
failed_count=failed_count,
outcome_unknown_count=outcome_unknown_count,
skipped_count=queue_result.skipped_count + queue_result.blocked_count + skipped_after_queue,
preflight_count=len(delivery_contexts),
synchronous_send_policy=synchronous_policy.as_dict(),
dry_run=False,
results=results,
)
def _ensure_synchronous_send_count_allowed(
eligible_count: int,
*,
policy: SynchronousSendPolicy,
) -> None:
if eligible_count <= 0:
raise SynchronousSendRejected(
"No eligible built recipient messages are available for synchronous delivery.",
reason="no_eligible_recipient_jobs",
eligible_count=eligible_count,
policy=policy,
)
if eligible_count > policy.max_recipient_jobs:
raise SynchronousSendRejected(
(
f"This built run contains {eligible_count} eligible recipient jobs, above the effective "
f"synchronous limit of {policy.max_recipient_jobs}. Queue it for background workers instead."
),
reason="recipient_limit_exceeded",
eligible_count=eligible_count,
policy=policy,
)
def _preflight_synchronous_send_batch(
session: Session,
*,
version: CampaignVersion,
jobs: list[CampaignJob],
policy: SynchronousSendPolicy,
) -> dict[str, _SendJobDeliveryContext]:
"""Freeze and validate every local input before the first SMTP effect."""
contexts: dict[str, _SendJobDeliveryContext] = {}
try:
for job in jobs:
state_result = _preflight_send_campaign_job(session, job, dry_run=True)
if state_result is not None:
raise SendJobError(
f"Job {job.id} changed to {state_result.status} during synchronous preflight"
)
contexts[job.id] = _send_job_delivery_context(session, job)
profile_summary = profile_delivery_summary(session, version)
expected_revision = next(iter(contexts.values())).snapshot.smtp_transport_revision
if profile_summary.get("smtp_transport_revision") != expected_revision:
raise SendJobError(
"The selected Mail profile changed after this execution was built. Revalidate and rebuild before delivery."
)
except (ExecutionSnapshotError, MailProfileError, SmtpConfigurationError, SendJobError, OSError) as exc:
raise SynchronousSendRejected(
(
"Synchronous preflight stopped before contacting SMTP because one or more frozen message "
"inputs or the selected Mail profile no longer match the reviewed build. Rebuild and review "
"the version before trying again."
),
reason="batch_preflight_failed",
eligible_count=len(jobs),
policy=policy,
) from exc
return contexts
def enqueue_existing_queued_jobs(session: Session, *, tenant_id: str, campaign_id: str) -> int:
if not _celery_enabled():
return 0

28
tests/test_documentation.py
View File

@@ -2,6 +2,7 @@ from __future__ import annotations
from dataclasses import dataclass
from types import SimpleNamespace
from unittest.mock import patch
import pytest
@@ -12,6 +13,7 @@ from govoplan_core.core.modules import DocumentationContext
@dataclass(frozen=True)
class _Principal:
scopes: frozenset[str]
tenant_id: str = "tenant-1"
def has(self, scope: str) -> bool:
namespace, resource, _action = scope.split(":", 2)
@@ -181,6 +183,32 @@ def test_runtime_documentation_full_composition_uses_only_user_facing_names() ->
assert "secret" not in rendered.lower()
def test_runtime_documentation_states_the_effective_synchronous_limit() -> None:
session = SimpleNamespace(
get=lambda _model, _id: SimpleNamespace(
settings={
"campaign_delivery_policy": {
"synchronous_send_max_recipients": 12,
}
}
)
)
with patch.dict("os.environ", {"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "25"}):
topic = documentation_topics(
DocumentationContext(
registry=_Registry({"mail.campaign_delivery"}),
principal=_Principal(frozenset({"campaigns:campaign:read", "campaigns:campaign:send"})),
session=session,
documentation_type="user",
)
)[0]
assert any(
"Send now is limited to 12 eligible recipient job(s)" in item
for item in topic.metadata["current_configuration"]
)
def _visible_static_topics(
scopes: set[str],
*,

171
tests/test_synchronous_delivery_policy.py Normal file
View File

@@ -0,0 +1,171 @@
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import Mock, patch
import pytest
from govoplan_campaign.backend.delivery_policy import (
CampaignDeliveryPolicyError,
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
effective_synchronous_send_policy,
)
from govoplan_campaign.backend.db.models import (
JobBuildStatus,
JobQueueStatus,
JobSendStatus,
JobValidationStatus,
)
from govoplan_campaign.backend.sending.jobs import (
SynchronousSendRejected,
_ensure_synchronous_send_count_allowed,
_preflight_synchronous_send_batch,
synchronous_send_candidate_jobs,
)
class _PolicySession:
def __init__(self, settings: dict[str, object] | None = None) -> None:
self.tenant = SimpleNamespace(settings=settings or {})
def get(self, _model, _id):
return self.tenant
def _version() -> SimpleNamespace:
return SimpleNamespace(
id="version-1",
build_summary={"build_token": "build-1"},
editor_state={
"review_send": {
"build_token": "build-1",
"inspection_complete": True,
"reviewed_message_keys": ["reviewed"],
}
},
)
def _job(job_id: str, **overrides: object) -> SimpleNamespace:
values: dict[str, object] = {
"id": job_id,
"entry_id": job_id,
"entry_index": 1,
"build_status": JobBuildStatus.BUILT.value,
"validation_status": JobValidationStatus.READY.value,
"queue_status": JobQueueStatus.DRAFT.value,
"send_status": JobSendStatus.NOT_QUEUED.value,
"eml_local_path": f"{job_id}.eml",
"eml_storage_key": None,
}
values.update(overrides)
return SimpleNamespace(**values)
def test_synchronous_policy_defaults_to_25_and_tenant_can_only_narrow() -> None:
default = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={},
)
assert default.max_recipient_jobs == DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS == 25
assert default.source == "deployment_default"
narrowed = effective_synchronous_send_policy(
_PolicySession(
{"campaign_delivery_policy": {"synchronous_send_max_recipients": 10}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "40"},
)
assert narrowed.max_recipient_jobs == 10
assert narrowed.source == "tenant"
ceiling = effective_synchronous_send_policy(
_PolicySession(
{"campaign_delivery_policy": {"synchronous_send_max_recipients": 100}}
), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "40"},
)
assert ceiling.max_recipient_jobs == 40
assert ceiling.source == "deployment_ceiling"
@pytest.mark.parametrize("value", [True, -1, 501, "2.5", "unbounded"])
def test_invalid_synchronous_policy_fails_closed(value: object) -> None:
with pytest.raises(CampaignDeliveryPolicyError):
effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": value}, # type: ignore[dict-item]
)
def test_candidate_count_uses_exact_built_and_reviewed_job_states() -> None:
jobs = [
_job("ready"),
_job("queued", queue_status=JobQueueStatus.QUEUED.value, send_status=JobSendStatus.QUEUED.value),
_job("reviewed", validation_status=JobValidationStatus.NEEDS_REVIEW.value),
_job("blocked", validation_status=JobValidationStatus.BLOCKED.value),
_job("failed", send_status=JobSendStatus.FAILED_TEMPORARY.value),
_job("missing-eml", eml_local_path=None),
]
candidates = synchronous_send_candidate_jobs(_version(), jobs) # type: ignore[arg-type]
assert [job.id for job in candidates] == ["ready", "queued", "reviewed"]
def test_limit_and_zero_count_reject_before_delivery() -> None:
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={"GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS": "2"},
)
with pytest.raises(SynchronousSendRejected, match="No eligible") as empty:
_ensure_synchronous_send_count_allowed(0, policy=policy)
assert empty.value.reason == "no_eligible_recipient_jobs"
with pytest.raises(SynchronousSendRejected, match="Queue it") as oversized:
_ensure_synchronous_send_count_allowed(3, policy=policy)
assert oversized.value.audit_details()["eligible_recipient_job_count"] == 3
assert oversized.value.audit_details()["synchronous_send_policy"]["max_recipient_jobs"] == 2
def test_batch_preflight_checks_every_message_before_provider_effects() -> None:
jobs = [_job("one"), _job("two")]
contexts = {
"one": SimpleNamespace(snapshot=SimpleNamespace(smtp_transport_revision="revision-1")),
"two": SimpleNamespace(snapshot=SimpleNamespace(smtp_transport_revision="revision-1")),
}
policy = effective_synchronous_send_policy(
_PolicySession(), # type: ignore[arg-type]
tenant_id="tenant-1",
environ={},
)
provider = Mock()
with (
patch("govoplan_campaign.backend.sending.jobs._preflight_send_campaign_job", return_value=None) as state_preflight,
patch(
"govoplan_campaign.backend.sending.jobs._send_job_delivery_context",
side_effect=lambda _session, job: contexts[job.id],
) as input_preflight,
patch(
"govoplan_campaign.backend.sending.jobs.profile_delivery_summary",
return_value={"smtp_transport_revision": "revision-1"},
),
patch("govoplan_campaign.backend.sending.jobs.mail_integration", return_value=provider),
):
result = _preflight_synchronous_send_batch(
object(), # type: ignore[arg-type]
version=_version(), # type: ignore[arg-type]
jobs=jobs, # type: ignore[arg-type]
policy=policy,
)
assert list(result) == ["one", "two"]
assert state_preflight.call_count == 2
assert input_preflight.call_count == 2
provider.send_campaign_email_bytes.assert_not_called()