diff --git a/README.md b/README.md index 304fb99..1feabe8 100644 --- a/README.md +++ b/README.md @@ -28,12 +28,14 @@ provider contract. Reporting owns the global `/reports` route. Campaign keeps its module-local `/campaigns/reports` view and does not claim the global route when Reporting is absent. -Generated EML is durable execution material, not a node-local runtime cache. -Campaign stores it through Core's shared object-storage contract under opaque -Campaign-owned keys; database job rows retain the expected size, digest, and -Message-ID. Workers resolve and verify the object before delivery. Build -failure compensates objects written before database commit, and retention keeps -the database reference when object deletion fails so cleanup can be retried. +Generated EML and printable artifacts are durable execution material, not a +node-local runtime cache. Campaign stores EML through Core's shared +object-storage contract under opaque Campaign-owned keys. Templates returns a +bounded artifact or a Files-managed artifact for printable output. Database job +rows retain the expected hashes and provenance. Workers resolve and verify the +frozen evidence before delivery. Build failure compensates objects written +before database commit, and retention keeps database references when object +deletion fails so cleanup can be retried. ## Dependencies @@ -41,10 +43,15 @@ The module has one required runtime dependency: - `govoplan-core` for platform services, auth, RBAC, DB/session lifecycle, migrations, and WebUI shell integration -Files and mail are optional module integrations declared in the campaign manifest: +Files, Mail, Distribution Lists, Templates, and Postbox are optional module integrations declared in the campaign manifest: - `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Server/API campaigns require this integration for attachments and never resolve caller-supplied local filesystem paths. Legacy file-oriented loading remains available only to explicitly trusted operator/library workflows. - `govoplan-mail` owns reusable profiles, encrypted SMTP/IMAP credentials, delivery policy checks, connection tests, and transport execution. Campaign JSON stores only `server.mail_profile_id`; inline transport settings and credentials are rejected. Without Mail, campaigns can still be authored, but profile validation and real delivery are unavailable. +- `govoplan-dist-lists` expands reusable governed audiences. Campaign freezes the exact list revision, provider evidence, candidates, and explicit per-recipient primary/fallback route into its own version. +- `govoplan-templates` validates and renders published label, envelope, letter, and list-layout templates for postal or internal-mail delivery. Generated output is hash-bound to its template, inputs, actor, route decisions, and Campaign version. +- `govoplan-postbox` resolves exact or organization-derived Postbox targets and records provider acceptance and receipt evidence. It remains optional; Mail-only and print-only campaigns do not require it. + +Hybrid delivery never treats an opt-in as an implicit duplicate-send instruction. The Campaign author selects one primary route per recipient and may select a supported fallback. A fallback runs only after the first channel rejects before acceptance; accepted or outcome-unknown effects stop cross-channel retry. Printable output is generated once during build, optionally persisted through Files, reviewed with the exact Campaign version, and accepted idempotently per recipient job during delivery. Public campaign, version, job, and report responses expose business data and delivery evidence, but never process-local paths, storage-backend keys, or diff --git a/docs/CAMPAIGN_HANDBOOK.md b/docs/CAMPAIGN_HANDBOOK.md index 26f9801..b8a2cb6 100644 --- a/docs/CAMPAIGN_HANDBOOK.md +++ b/docs/CAMPAIGN_HANDBOOK.md @@ -30,12 +30,13 @@ The shorter task documents remain useful companions: ## What Campaign is for -Campaign turns governed source data into individually built messages and then -controls their review, delivery, and evidence. It is intentionally a -composition module: it demonstrates how one user journey can use optional Mail, -Files, Addresses, and Notifications capabilities alongside Core access/audit -infrastructure without copying ownership from those modules. Policy may consume -Campaign context through a narrow capability; Campaign does not import Policy. +Campaign turns governed source data into individually built messages or +printable output and then controls their review, delivery, and evidence. It is +intentionally a composition module: it demonstrates how one user journey can +use optional Mail, Files, Addresses, Distribution Lists, Templates, Postbox, +and Notifications capabilities alongside Core access/audit infrastructure +without copying ownership from those modules. Policy may consume Campaign +context through a narrow capability; Campaign does not import Policy. Campaign owns: @@ -66,9 +67,9 @@ The supported process is a controlled progression, not a single "send" call: create/edit -> validate and resolve policy/integrations -> review warnings and blockers - -> build exact recipient messages + -> build exact recipient messages and/or printable artifacts -> complete review and queue - -> SMTP attempt per job + -> selected Mail, Postbox, or print effect per job -> optional IMAP append per accepted job -> report, retry, reconcile, or correct -> archive when no active/uncertain delivery remains @@ -413,6 +414,7 @@ Current principal contracts include: | `addresses.lookup` 0.1.x | Addresses -> Campaign | Optional address suggestions | | `addresses.recipient_source` 0.1.x | Addresses -> Campaign | Optional versioned recipient-source snapshots | | `dist_lists.source` / `dist_lists.expand` 0.1.x | Distribution Lists -> Campaign | Discover, preview, and freeze reusable audiences without importing module internals | +| `templates.catalog` / `templates.renderer` 0.1.x | Templates -> Campaign | Select compatible published printable templates and produce deterministic, evidence-bearing artifacts | | `campaigns.access` 0.1.x | Campaign -> platform | Explain campaign access/existence without exporting ORM objects | | `campaigns.mail_policy_context` 0.1.x | Campaign -> Mail | Resolve campaign tenant/owner context for Mail policy | | `campaigns.delivery_tasks` 0.1.x | Campaign -> workers | Execute narrow queued send/append tasks | @@ -433,16 +435,46 @@ the resulting rows into the editable Campaign version. Each copied row retains the list and revision IDs, definition and expansion hashes, snapshot ID, source entry IDs, provider references, channel candidates, -the one selected route where it is unambiguous, fallback candidates, and the -decision explanation. Campaign-only fields, attachment rules, review state, -and outcomes remain local to Campaign and never mutate the reusable list. +the explicitly selected primary route, optional fallback, and the decision +explanation. Campaign-only fields, attachment rules, review state, and outcomes +remain local to Campaign and never mutate the reusable list. A later list revision only raises a drift warning. Refresh is deliberate and uses append or replace; saving that changed Campaign version clears prior validation, build, review, and execution state through the normal content -invalidation path. Postal-only or otherwise unsupported routes remain present -in the frozen evidence but inactive until a compatible Campaign output path is -configured. +invalidation path. Preferred or single usable candidates are preselected +visibly; ambiguous rows must be decided before freezing. Postal and +internal-mail routes remain active when a compatible published Templates output +is selected. + +### Governed hybrid and printable delivery + +Campaign supports Mail, Postbox, printable output, and bounded ordered +fallbacks without making any of those provider modules mandatory. Opt-in and +channel-preference data are inputs to the visible routing decision; they never +silently cause duplicate delivery. + +For a printable route, select a published label, envelope, serial-letter, +form-letter, list-layout, or generic template on the Template page. Validation +checks the selected revision, output format, and required fields. Build sends +one deterministic item collection to `templates.renderer`, records template, +input, output, actor, route, and artifact hashes, and stores the resulting +artifact through Files when configured. The review stage exposes that exact +artifact and its hashes before execution. + +Each recipient job records an idempotent print acceptance attempt for its item +in the frozen artifact. `mail_then_print` and `postbox_then_print` invoke print +only after a confirmed rejection before acceptance. An accepted or +outcome-unknown digital effect never falls through to print because that could +produce duplicate delivery. Reports and CSV exports include route provenance, +print state, attempts, artifact reference, and hashes. + +Without Templates, Campaign still loads and Mail/Postbox authoring remains +available; validation explains why a configured print route cannot proceed. +Without Files, Templates may return a bounded artifact instead of a managed +file. Campaign copies that payload into shared object storage and exposes it +through the Campaign ACL plus `campaigns:recipient:read`; it never redistributes +the broader Templates URL. A print-only Campaign does not require Mail or Postbox. ### External API expectations diff --git a/src/govoplan_campaign/backend/campaign/models.py b/src/govoplan_campaign/backend/campaign/models.py index cb7d1f2..eddb862 100644 --- a/src/govoplan_campaign/backend/campaign/models.py +++ b/src/govoplan_campaign/backend/campaign/models.py @@ -97,9 +97,12 @@ class SendStatus(StrEnum): class DeliveryChannelPolicy(StrEnum): MAIL = "mail" POSTBOX = "postbox" + PRINT = "print" MAIL_AND_POSTBOX = "mail_and_postbox" MAIL_THEN_POSTBOX = "mail_then_postbox" POSTBOX_THEN_MAIL = "postbox_then_mail" + MAIL_THEN_PRINT = "mail_then_print" + POSTBOX_THEN_PRINT = "postbox_then_print" @property def uses_mail(self) -> bool: @@ -108,11 +111,26 @@ class DeliveryChannelPolicy(StrEnum): DeliveryChannelPolicy.MAIL_AND_POSTBOX, DeliveryChannelPolicy.MAIL_THEN_POSTBOX, DeliveryChannelPolicy.POSTBOX_THEN_MAIL, + DeliveryChannelPolicy.MAIL_THEN_PRINT, } @property def uses_postbox(self) -> bool: - return self != DeliveryChannelPolicy.MAIL + return self in { + DeliveryChannelPolicy.POSTBOX, + DeliveryChannelPolicy.MAIL_AND_POSTBOX, + DeliveryChannelPolicy.MAIL_THEN_POSTBOX, + DeliveryChannelPolicy.POSTBOX_THEN_MAIL, + DeliveryChannelPolicy.POSTBOX_THEN_PRINT, + } + + @property + def uses_print(self) -> bool: + return self in { + DeliveryChannelPolicy.PRINT, + DeliveryChannelPolicy.MAIL_THEN_PRINT, + DeliveryChannelPolicy.POSTBOX_THEN_PRINT, + } class PostboxTargetMode(StrEnum): @@ -192,6 +210,15 @@ class PostboxTargetConfig(StrictModel): return self +class PrintTargetConfig(StrictModel): + channel: Literal["postal", "internal_mail"] + target: str = Field(min_length=1, max_length=4000) + target_key: str = Field(min_length=1, max_length=500) + contact_point_id: str | None = Field(default=None, max_length=36) + locale: str | None = Field(default=None, max_length=35) + decision_provenance: dict[str, Any] = Field(default_factory=dict) + + class CampaignMeta(StrictModel): id: str name: str @@ -556,11 +583,16 @@ class EntryConfig(StrictModel): max_length=50, ) merge_postbox_targets: bool = True + print_target: PrintTargetConfig | None = None attachments: list[AttachmentConfig] = Field(default_factory=list) combine_attachments: bool = True fields: dict[str, Any] = Field(default_factory=dict) + # Frozen channel candidates, source revisions and the explicit route + # decision imported from Distribution Lists. Campaign owns this snapshot; + # it never re-resolves the audience during build or delivery. + distribution_source: dict[str, Any] = Field(default_factory=dict) last_sent: str | None = None @@ -576,7 +608,7 @@ class ImportProvenance(StrictModel): id: str imported_at: str mode: Literal["append", "replace"] - source_type: Literal["csv", "xlsx", "text", "addresses"] + source_type: Literal["csv", "xlsx", "text", "addresses", "distribution_list"] source_id: str | None = None source_label: str | None = None source_revision: str | None = None @@ -678,9 +710,19 @@ class PostboxDeliveryConfig(StrictModel): duplicate_target: Behavior = Behavior.WARN +class PrintDeliveryConfig(StrictModel): + template_id: str | None = Field(default=None, max_length=36) + template_revision: int | None = Field(default=None, ge=1) + usage: str = Field(default="campaign_print", min_length=1, max_length=100) + output_format: Literal["html", "text"] = "html" + profile_id: str | None = Field(default=None, max_length=120) + persist_to_files: bool = True + + class DeliveryConfig(StrictModel): channel_policy: DeliveryChannelPolicy = DeliveryChannelPolicy.MAIL postbox: PostboxDeliveryConfig = Field(default_factory=PostboxDeliveryConfig) + print: PrintDeliveryConfig = Field(default_factory=PrintDeliveryConfig) rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig) imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig) retry: RetryConfig = Field(default_factory=RetryConfig) diff --git a/src/govoplan_campaign/backend/campaign/validation.py b/src/govoplan_campaign/backend/campaign/validation.py index 19853cc..75920da 100644 --- a/src/govoplan_campaign/backend/campaign/validation.py +++ b/src/govoplan_campaign/backend/campaign/validation.py @@ -482,6 +482,7 @@ def _delivery_issues( config: CampaignConfig, *, postbox_available: bool, + templates_available: bool, ) -> list[SemanticIssue]: issues: list[SemanticIssue] = [] policies = _delivery_policies(config) @@ -533,6 +534,46 @@ def _delivery_issues( postbox_available=postbox_available, ) ) + if any(policy.uses_print for policy in policies): + if not templates_available: + issues.append( + _issue( + Severity.ERROR, + "templates_unavailable", + "Printable Campaign delivery requires the optional Templates renderer.", + "/delivery/print/template_id", + ) + ) + if not config.delivery.print.template_id: + issues.append( + _issue( + Severity.ERROR, + "print_template_missing", + "Select a published compatible template for printable Campaign output.", + "/delivery/print/template_id", + ) + ) + elif config.delivery.print.template_revision is None: + issues.append( + _issue( + Severity.ERROR, + "print_template_revision_missing", + "Printable delivery must pin one published template revision.", + "/delivery/print/template_revision", + ) + ) + for entry_index, entry in enumerate(_active_delivery_entries(config)): + if not effective_delivery_channel_policy(config, entry).uses_print: + continue + if entry.print_target is None: + issues.append( + _issue( + Severity.ERROR, + "print_target_missing", + "Printable delivery requires an explicit postal or internal-mail target.", + f"/entries/inline/{entry_index}/print_target", + ) + ) return issues @@ -784,6 +825,7 @@ def validate_campaign_config( campaign_file: str | Path | None = None, check_files: bool = False, postbox_available: bool = False, + templates_available: bool = False, ) -> SemanticReport: campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json" issues: list[SemanticIssue] = [] @@ -799,6 +841,7 @@ def validate_campaign_config( _delivery_issues( config, postbox_available=postbox_available, + templates_available=templates_available, ) ) issues.extend(_sender_issues(config)) diff --git a/src/govoplan_campaign/backend/change_tracking.py b/src/govoplan_campaign/backend/change_tracking.py index 5e5c643..66463ac 100644 --- a/src/govoplan_campaign/backend/change_tracking.py +++ b/src/govoplan_campaign/backend/change_tracking.py @@ -22,6 +22,7 @@ from govoplan_campaign.backend.db.models import ( CampaignVersion, ImapAppendAttempt, PostboxDeliveryAttempt, + PrintOutputAttempt, SendAttempt, new_uuid, ) @@ -58,7 +59,7 @@ def _record_campaign_changes(session: OrmSession, _flush_context: object, _insta _record_issue_change(session, obj) elif isinstance( obj, - (SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt), + (SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt, PrintOutputAttempt), ): _record_attempt_change(session, obj) @@ -188,9 +189,11 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None: "send_status", "delivery_channel_policy", "postbox_status", + "print_status", "imap_status", "attempt_count", "postbox_attempt_count", + "print_attempt_count", "last_error", "queued_at", "claimed_at", @@ -198,7 +201,9 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None: "outcome_unknown_at", "sent_at", "resolved_recipients", + "delivery_provenance", "resolved_postbox_targets", + "resolved_print_output", "resolved_attachments", "issues_snapshot", ), @@ -227,6 +232,7 @@ def _record_job_change(session: OrmSession, job: CampaignJob) -> None: "send_status": job.send_status, "delivery_channel_policy": job.delivery_channel_policy, "postbox_status": job.postbox_status, + "print_status": job.print_status, "imap_status": job.imap_status, }, ) @@ -259,7 +265,7 @@ def _record_issue_change(session: OrmSession, issue: CampaignIssue) -> None: def _record_attempt_change( session: OrmSession, - attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt, + attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt | PrintOutputAttempt, ) -> None: operation = _operation_for_object( attempt, @@ -274,6 +280,8 @@ def _record_attempt_change( "provider_delivery_id", "provider_message_id", "postbox_id", + "render_id", + "artifact_sha256", "evidence", ), ) @@ -298,6 +306,8 @@ def _record_attempt_change( "attempt_kind": ( "postbox" if isinstance(attempt, PostboxDeliveryAttempt) + else "print" + if isinstance(attempt, PrintOutputAttempt) else "imap" if isinstance(attempt, ImapAppendAttempt) else "smtp" diff --git a/src/govoplan_campaign/backend/db/models.py b/src/govoplan_campaign/backend/db/models.py index e3b0771..4c05801 100644 --- a/src/govoplan_campaign/backend/db/models.py +++ b/src/govoplan_campaign/backend/db/models.py @@ -85,6 +85,7 @@ class JobSendStatus(StrEnum): SENDING = "sending" SMTP_ACCEPTED = "smtp_accepted" POSTBOX_ACCEPTED = "postbox_accepted" + PRINT_ACCEPTED = "print_accepted" DELIVERED = "delivered" PARTIALLY_ACCEPTED = "partially_accepted" SENT = "sent" # legacy value retained for existing databases/reports @@ -107,6 +108,15 @@ class JobPostboxStatus(StrEnum): SKIPPED = "skipped" +class JobPrintStatus(StrEnum): + NOT_REQUESTED = "not_requested" + READY = "ready" + ACCEPTING = "accepting" + ACCEPTED = "accepted" + FAILED = "failed" + SKIPPED = "skipped" + + class JobImapStatus(StrEnum): NOT_REQUESTED = "not_requested" PENDING = "pending" @@ -295,6 +305,12 @@ class CampaignJob(Base, TimestampMixin): nullable=False, index=True, ) + print_status: Mapped[str] = mapped_column( + String(50), + default=JobPrintStatus.NOT_REQUESTED.value, + nullable=False, + index=True, + ) imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True) attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False) @@ -303,6 +319,11 @@ class CampaignJob(Base, TimestampMixin): default=0, nullable=False, ) + print_attempt_count: Mapped[int] = mapped_column( + Integer, + default=0, + nullable=False, + ) last_error: Mapped[str | None] = mapped_column(Text) queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) @@ -314,11 +335,20 @@ class CampaignJob(Base, TimestampMixin): sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True) + delivery_provenance: Mapped[dict[str, Any]] = mapped_column( + JSON, + default=dict, + nullable=False, + ) resolved_postbox_targets: Mapped[list[dict[str, Any]]] = mapped_column( JSON, default=list, nullable=False, ) + resolved_print_output: Mapped[dict[str, Any] | None] = mapped_column( + JSON, + nullable=True, + ) resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) @@ -604,6 +634,39 @@ class PostboxDeliveryAttempt(Base, TimestampMixin): ) +class PrintOutputAttempt(Base, TimestampMixin): + __tablename__ = "campaign_print_output_attempts" + __table_args__ = ( + UniqueConstraint( + "job_id", + "attempt_number", + name="uq_campaign_print_attempt_job_number", + ), + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_campaign_print_attempt_idempotency", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid) + tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True) + job_id: Mapped[str] = mapped_column( + ForeignKey("campaign_jobs.id", ondelete="CASCADE"), + nullable=False, + index=True, + ) + attempt_number: Mapped[int] = mapped_column(Integer, nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False) + status: Mapped[str] = mapped_column(String(50), nullable=False, index=True) + render_id: Mapped[str | None] = mapped_column(String(36), index=True) + artifact_sha256: Mapped[str | None] = mapped_column(String(64), index=True) + evidence: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False) + error_message: Mapped[str | None] = mapped_column(Text) + started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + + __all__ = [ @@ -622,9 +685,11 @@ __all__ = [ "JobBuildStatus", "JobImapStatus", "JobPostboxStatus", + "JobPrintStatus", "JobQueueStatus", "JobSendStatus", "JobValidationStatus", "SendAttempt", "PostboxDeliveryAttempt", + "PrintOutputAttempt", ] diff --git a/src/govoplan_campaign/backend/documentation.py b/src/govoplan_campaign/backend/documentation.py index 03849aa..396d813 100644 --- a/src/govoplan_campaign/backend/documentation.py +++ b/src/govoplan_campaign/backend/documentation.py @@ -45,6 +45,8 @@ _ADDRESSES_LOOKUP_INTEGRATION = "addresses.lookup" _ADDRESSES_SOURCE_INTEGRATION = "addresses.recipient_source" _DISTRIBUTION_LIST_SOURCE_INTEGRATION = "dist_lists.source" _DISTRIBUTION_LIST_EXPAND_INTEGRATION = "dist_lists.expand" +_TEMPLATE_CATALOG_INTEGRATION = "templates.catalog" +_TEMPLATE_RENDERER_INTEGRATION = "templates.renderer" _NOTIFICATIONS_INTEGRATION = "notifications.dispatch" @@ -250,15 +252,54 @@ CAMPAIGN_USER_DOCUMENTATION = ( steps=( "Open Recipient data and select Import Distribution List.", "Choose the list, requested channels, and any declared parameters, then preview the expansion.", - "Review included and excluded recipients, stale provider evidence, diagnostics, and unresolved route choices.", + "Review included and excluded recipients, stale provider evidence, diagnostics, and every visible primary and optional fallback route.", "Choose append or replace, freeze and import the expansion, inspect the copied rows, and save the Campaign version.", "Use the drift warning for a deliberate refresh when the reusable list changes later.", ), outcome="A Campaign-local recipient snapshot with immutable audience, provider, policy, and channel-decision evidence.", verification="The saved recipient rows retain the list revision and snapshot reference, and later list changes do not alter them automatically.", related_topic_ids=("campaigns.workflow.import-recipients", "campaigns.workflow.prepare-validate-and-build"), - related_modules=("dist_lists",), - limitations=("Unsupported non-email output routes remain inactive until a compatible Campaign output integration is configured.",), + related_modules=("dist_lists", "templates", "postbox"), + ), + _workflow_topic( + topic_id="campaigns.workflow.prepare-printable-delivery", + title="Prepare governed printable delivery", + summary="Select a published output template, build one deterministic artifact, and review its route and hash evidence before postal or internal-mail distribution.", + body="Printable delivery is optional and provider-neutral. Campaign freezes recipient route decisions while Templates owns compatibility and rendering; Files may own the resulting managed artifact. Ordered fallback is used only after a confirmed rejection before acceptance and never after an accepted or outcome-unknown digital effect.", + order=34, + audience=("campaign_manager", "campaign_author", "campaign_reviewer"), + required_modules=("campaigns", "templates"), + required_capabilities=( + _TEMPLATE_CATALOG_INTEGRATION, + _TEMPLATE_RENDERER_INTEGRATION, + ), + required_scopes=( + "campaigns:campaign:read", + "campaigns:campaign:update", + "campaigns:campaign:validate", + "campaigns:campaign:build", + "campaigns:recipient:read", + "templates:template:read", + "templates:template:render", + ), + route="/campaigns/{campaign_id}/template", + screen="Template", + help_contexts=("campaign.template", "campaign.review-send"), + prerequisites=( + "At least one included recipient has an explicit postal or internal-mail route.", + "A compatible Templates definition is published and visible to you.", + ), + steps=( + "Open Template and select the published printable template, output format, and storage choice.", + "Save, validate, and resolve every missing-field or compatibility error.", + "Build the Campaign to generate one deterministic artifact for the frozen printable recipients.", + "In Review and send, download and inspect the artifact and compare its template, input, and output hashes.", + "Complete review and execute delivery; use reports to verify per-recipient print acceptance and route provenance.", + ), + outcome="A reviewed printable artifact and idempotent per-recipient distribution evidence.", + verification="The build summary exposes the artifact and hashes, and the Campaign report records print status and one acceptance attempt per routed recipient.", + related_topic_ids=("campaigns.workflow.import-distribution-list", "campaigns.workflow.prepare-validate-and-build"), + related_modules=("templates", "files", "dist_lists"), ), _workflow_topic( topic_id="campaigns.workflow.use-managed-attachments", diff --git a/src/govoplan_campaign/backend/integrations.py b/src/govoplan_campaign/backend/integrations.py index fdcbb4b..532bdfd 100644 --- a/src/govoplan_campaign/backend/integrations.py +++ b/src/govoplan_campaign/backend/integrations.py @@ -28,6 +28,16 @@ from govoplan_core.core.postbox import ( PostboxEvidenceProvider, PostboxTargetRef, ) +from govoplan_core.core.templates import ( + CAPABILITY_TEMPLATE_CATALOG, + CAPABILITY_TEMPLATE_RENDERER, + TemplateCatalogProvider, + TemplateCompatibility, + TemplateRef, + TemplateRenderRequest, + TemplateRenderResult, + TemplateRendererProvider, +) from govoplan_campaign.backend.runtime import capability @@ -37,6 +47,8 @@ POSTBOX_CAPABILITY = CAPABILITY_POSTBOX_DELIVERY POSTBOX_DIRECTORY_CAPABILITY = CAPABILITY_POSTBOX_DIRECTORY POSTBOX_EVIDENCE_CAPABILITY = CAPABILITY_POSTBOX_EVIDENCE APPROVALS_CAPABILITY = CAPABILITY_APPROVAL_REQUESTS +TEMPLATE_CATALOG_CAPABILITY = CAPABILITY_TEMPLATE_CATALOG +TEMPLATE_RENDERER_CAPABILITY = CAPABILITY_TEMPLATE_RENDERER class OptionalModuleUnavailable(RuntimeError): @@ -89,6 +101,10 @@ class ApprovalGateUnavailable(OptionalModuleUnavailable): pass +class TemplateOutputUnavailable(OptionalModuleUnavailable): + pass + + class _PreparedCampaignSnapshot: def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None: self._directory = directory @@ -493,6 +509,102 @@ class ApprovalCampaignIntegration: ) +class TemplatesCampaignIntegration: + def __init__( + self, + catalog_delegate: object | None = None, + renderer_delegate: object | None = None, + ) -> None: + self._catalog = ( + catalog_delegate + if isinstance(catalog_delegate, TemplateCatalogProvider) + else None + ) + self._renderer = ( + renderer_delegate + if isinstance(renderer_delegate, TemplateRendererProvider) + else None + ) + + @property + def available(self) -> bool: + return self._catalog is not None and self._renderer is not None + + def list_templates( + self, + session: object, + principal: object, + *, + query: str = "", + limit: int = 100, + ) -> tuple[TemplateRef, ...]: + if self._catalog is None: + return () + return tuple( + self._catalog.list_templates( + session, + principal, + query=query, + usage="campaign_print", + limit=limit, + ) + ) + + def check_compatibility( + self, + session: object, + principal: object, + *, + template_id: str, + revision: int | None, + output_format: str, + available_fields: dict[str, str] | tuple[str, ...], + ) -> TemplateCompatibility: + if self._catalog is None: + raise TemplateOutputUnavailable( + "Printable output is unavailable because Templates is not active." + ) + return self._catalog.check_compatibility( + session, + principal, + template_id=template_id, + revision=revision, + usage="campaign_print", + output_format=output_format, + available_fields=available_fields, + ) + + def get_template( + self, + session: object, + principal: object, + *, + template_id: str, + revision: int, + ) -> TemplateRef | None: + if self._catalog is None: + return None + return self._catalog.get_template( + session, + principal, + template_id=template_id, + revision=revision, + ) + + def render( + self, + session: object, + principal: object, + *, + request: TemplateRenderRequest, + ) -> TemplateRenderResult: + if self._renderer is None: + raise TemplateOutputUnavailable( + "Printable output is unavailable because Templates is not active." + ) + return self._renderer.render(session, principal, request=request) + + def files_integration() -> FilesCampaignIntegration: return FilesCampaignIntegration(capability(FILES_CAPABILITY)) @@ -511,3 +623,10 @@ def postbox_integration() -> PostboxCampaignIntegration: def approvals_integration() -> ApprovalCampaignIntegration: return ApprovalCampaignIntegration(capability(APPROVALS_CAPABILITY)) + + +def templates_integration() -> TemplatesCampaignIntegration: + return TemplatesCampaignIntegration( + capability(TEMPLATE_CATALOG_CAPABILITY), + capability(TEMPLATE_RENDERER_CAPABILITY), + ) diff --git a/src/govoplan_campaign/backend/manifest.py b/src/govoplan_campaign/backend/manifest.py index 85632fe..88692cb 100644 --- a/src/govoplan_campaign/backend/manifest.py +++ b/src/govoplan_campaign/backend/manifest.py @@ -40,6 +40,10 @@ from govoplan_core.core.distribution_lists import ( CAPABILITY_DISTRIBUTION_LIST_EXPAND, CAPABILITY_DISTRIBUTION_LIST_SOURCE, ) +from govoplan_core.core.templates import ( + CAPABILITY_TEMPLATE_CATALOG, + CAPABILITY_TEMPLATE_RENDERER, +) from govoplan_core.core.operations import OperationalCheckProviderRegistration from govoplan_core.core.provider_governance import declared_module_architecture from govoplan_core.core.views import ViewSurface @@ -170,7 +174,7 @@ PERMISSIONS = ( _permission( "campaigns:campaign:send", "Send campaigns", - "Start real Mail or Postbox delivery.", + "Start real Mail, Postbox, or printable delivery.", "Campaigns", ), _permission( @@ -182,7 +186,7 @@ PERMISSIONS = ( _permission( "campaigns:campaign:reconcile", "Reconcile delivery", - "Resolve outcome-unknown Mail, Postbox, or IMAP attempts after inspection.", + "Resolve outcome-unknown Mail, Postbox, printable, or IMAP attempts after inspection.", "Campaigns", ), _permission( @@ -368,6 +372,7 @@ manifest = ModuleManifest( "notifications", "addresses", "dist_lists", + "templates", "postbox", "approvals", "reporting", @@ -432,6 +437,18 @@ manifest = ModuleManifest( version_max_exclusive="0.2.0", optional=True, ), + ModuleInterfaceRequirement( + name=CAPABILITY_TEMPLATE_CATALOG, + version_min="0.1.0", + version_max_exclusive="0.2.0", + optional=True, + ), + ModuleInterfaceRequirement( + name=CAPABILITY_TEMPLATE_RENDERER, + version_min="0.1.0", + version_max_exclusive="0.2.0", + optional=True, + ), ModuleInterfaceRequirement( name=CAPABILITY_POSTBOX_DELIVERY, version_min="0.1.1", @@ -545,6 +562,7 @@ manifest = ModuleManifest( campaign_models.CampaignMessageActionAttempt, campaign_models.ImapAppendAttempt, campaign_models.PostboxDeliveryAttempt, + campaign_models.PrintOutputAttempt, label="Campaigns", ), retirement_notes="Destructive retirement drops campaign-owned database tables after the installer captures a database snapshot.", @@ -564,6 +582,7 @@ manifest = ModuleManifest( campaign_models.CampaignMessageActionAttempt, campaign_models.ImapAppendAttempt, campaign_models.PostboxDeliveryAttempt, + campaign_models.PrintOutputAttempt, label="Campaigns", ), ), @@ -1139,6 +1158,9 @@ manifest = ModuleManifest( non_owned_concepts=( "mail transport", "postbox", + "distribution list", + "template definition and rendering", + "print artifact storage", "durable address directory", "file storage", ), diff --git a/src/govoplan_campaign/backend/migrations/dev_versions/b7c8d9e0f1a2_campaign_print_delivery.py b/src/govoplan_campaign/backend/migrations/dev_versions/b7c8d9e0f1a2_campaign_print_delivery.py new file mode 100644 index 0000000..cc28f4b --- /dev/null +++ b/src/govoplan_campaign/backend/migrations/dev_versions/b7c8d9e0f1a2_campaign_print_delivery.py @@ -0,0 +1,17 @@ +"""Development wrapper for the canonical printable-delivery migration.""" +from __future__ import annotations + +from importlib import import_module + + +_migration = import_module( + "govoplan_campaign.backend.migrations.versions." + "b7c8d9e0f1a2_campaign_print_delivery" +) + +revision = _migration.revision +down_revision = _migration.down_revision +branch_labels = _migration.branch_labels +depends_on = _migration.depends_on +upgrade = _migration.upgrade +downgrade = _migration.downgrade diff --git a/src/govoplan_campaign/backend/migrations/versions/b7c8d9e0f1a2_campaign_print_delivery.py b/src/govoplan_campaign/backend/migrations/versions/b7c8d9e0f1a2_campaign_print_delivery.py new file mode 100644 index 0000000..efe3c0e --- /dev/null +++ b/src/govoplan_campaign/backend/migrations/versions/b7c8d9e0f1a2_campaign_print_delivery.py @@ -0,0 +1,112 @@ +"""add governed campaign printable delivery + +Revision ID: b7c8d9e0f1a2 +Revises: f0a1b2c3d4e5 +Create Date: 2026-08-02 12:00:00.000000 +""" +from __future__ import annotations + +from alembic import op +import sqlalchemy as sa + + +revision = "b7c8d9e0f1a2" +down_revision = "f0a1b2c3d4e5" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column( + "campaign_jobs", + sa.Column( + "print_status", + sa.String(length=50), + nullable=False, + server_default="not_requested", + ), + ) + op.add_column( + "campaign_jobs", + sa.Column( + "print_attempt_count", + sa.Integer(), + nullable=False, + server_default="0", + ), + ) + op.add_column( + "campaign_jobs", + sa.Column("resolved_print_output", sa.JSON(), nullable=True), + ) + op.add_column( + "campaign_jobs", + sa.Column( + "delivery_provenance", + sa.JSON(), + nullable=False, + server_default="{}", + ), + ) + op.create_index( + op.f("ix_campaign_jobs_print_status"), + "campaign_jobs", + ["print_status"], + unique=False, + ) + op.create_table( + "campaign_print_output_attempts", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=36), nullable=False), + sa.Column("job_id", sa.String(length=36), nullable=False), + sa.Column("attempt_number", sa.Integer(), nullable=False), + sa.Column("idempotency_key", sa.String(length=255), nullable=False), + sa.Column("status", sa.String(length=50), nullable=False), + sa.Column("render_id", sa.String(length=36), nullable=True), + sa.Column("artifact_sha256", sa.String(length=64), nullable=True), + sa.Column("evidence", sa.JSON(), nullable=False), + sa.Column("error_message", sa.Text(), nullable=True), + sa.Column("started_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False), + sa.ForeignKeyConstraint( + ["job_id"], + ["campaign_jobs.id"], + name=op.f("fk_campaign_print_output_attempts_job_id_campaign_jobs"), + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint( + "id", + name=op.f("pk_campaign_print_output_attempts"), + ), + sa.UniqueConstraint( + "job_id", + "attempt_number", + name="uq_campaign_print_attempt_job_number", + ), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_campaign_print_attempt_idempotency", + ), + ) + for column in ("tenant_id", "job_id", "status", "render_id", "artifact_sha256"): + op.create_index( + op.f(f"ix_campaign_print_output_attempts_{column}"), + "campaign_print_output_attempts", + [column], + unique=False, + ) + + +def downgrade() -> None: + op.drop_table("campaign_print_output_attempts") + op.drop_index( + op.f("ix_campaign_jobs_print_status"), + table_name="campaign_jobs", + ) + op.drop_column("campaign_jobs", "resolved_print_output") + op.drop_column("campaign_jobs", "delivery_provenance") + op.drop_column("campaign_jobs", "print_attempt_count") + op.drop_column("campaign_jobs", "print_status") diff --git a/src/govoplan_campaign/backend/persistence/campaigns.py b/src/govoplan_campaign/backend/persistence/campaigns.py index c866fd1..05256f1 100644 --- a/src/govoplan_campaign/backend/persistence/campaigns.py +++ b/src/govoplan_campaign/backend/persistence/campaigns.py @@ -2,7 +2,8 @@ from __future__ import annotations import copy import hashlib -from dataclasses import dataclass +import json +from dataclasses import asdict, dataclass from datetime import UTC, datetime from email import policy from email.parser import BytesParser @@ -14,11 +15,13 @@ from uuid import uuid4 from sqlalchemy import func from sqlalchemy.orm import Session +from govoplan_core.auth import ApiPrincipal from govoplan_core.core.object_storage import ( StorageBackend, StorageBackendError, configured_storage_backend, ) +from govoplan_core.core.templates import TemplateRenderRequest from govoplan_core.settings import settings as core_settings from govoplan_campaign.backend.db.models import ( Campaign, @@ -29,6 +32,7 @@ from govoplan_campaign.backend.db.models import ( CampaignVersionWorkflowState, JobImapStatus, JobPostboxStatus, + JobPrintStatus, JobQueueStatus, JobSendStatus, JobValidationStatus, @@ -42,13 +46,23 @@ from govoplan_campaign.backend.campaign.mail_profile_boundary import ( campaign_mail_profile_id, campaign_mail_resource_ids, ) -from govoplan_campaign.backend.campaign.validation import validate_campaign_config +from govoplan_campaign.backend.campaign.validation import ( + SemanticIssue, + Severity, + validate_campaign_config, +) from govoplan_campaign.backend.campaign.entries import load_campaign_entries from govoplan_campaign.backend.campaign.postbox_targets import ( resolve_entry_postbox_targets, ) +from govoplan_campaign.backend.campaign.field_values import ( + effective_entry_field_values, +) from govoplan_campaign.backend.messages.builder import build_campaign_messages -from govoplan_campaign.backend.messages.models import MessageDraft +from govoplan_campaign.backend.messages.models import ( + MessageDraft, + MessageValidationStatus, +) from govoplan_campaign.backend.sending.execution import ( create_execution_snapshot, profile_delivery_summary, @@ -62,6 +76,7 @@ from govoplan_campaign.backend.integrations import ( files_integration, mail_integration, postbox_integration, + templates_integration, ) from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths from govoplan_campaign.backend.runtime import get_settings @@ -426,6 +441,129 @@ def load_version_config(session: Session, version_id: str): ) +def _campaign_uses_print(config: CampaignConfig) -> bool: + entries = ( + config.entries.inline + if config.entries.is_inline + else [config.entries.defaults] + ) + return any( + entry is not None + and entry.active + and (entry.channel_policy or config.delivery.channel_policy).uses_print + for entry in entries or [] + ) + + +def _print_template_available_fields(config: CampaignConfig) -> dict[str, str]: + type_map = { + "double": "number", + "organization_unit": "string", + "organization_function": "string", + "password": "string", + } + fields: dict[str, str] = { + "campaign.id": "string", + "campaign.name": "string", + "recipient_key": "string", + "display_name": "string", + "print_target.channel": "string", + "print_target.target": "string", + "print_target.target_key": "string", + "distribution.list_id": "string", + "distribution.list_revision": "integer", + "distribution.expansion_hash": "string", + } + for field in config.fields: + value_type = type_map.get(field.type.value, field.type.value) + fields[field.name] = value_type + fields[f"fields.{field.name}"] = value_type + return fields + + +def _print_template_compatibility_issues( + session: Session, + *, + principal: ApiPrincipal | None, + config: CampaignConfig, +) -> list[SemanticIssue]: + if not _campaign_uses_print(config) or not config.delivery.print.template_id: + return [] + integration = templates_integration() + if not integration.available: + return [] # The semantic availability check already explains this. + if principal is None: + return [ + SemanticIssue( + severity=Severity.ERROR, + code="print_template_principal_missing", + message="Printable output must be validated by an authenticated Campaign actor.", + path="/delivery/print/template_id", + ) + ] + if not ( + principal.has("templates:template:render") + or principal.has("templates:template:admin") + ): + return [ + SemanticIssue( + severity=Severity.ERROR, + code="print_template_render_forbidden", + message="You may select this template but are not permitted to render printable output.", + path="/delivery/print/template_id", + ) + ] + if config.delivery.print.template_revision is None: + return [] # The semantic validation report already requires an exact revision. + try: + template = integration.get_template( + session, + principal, + template_id=config.delivery.print.template_id, + revision=config.delivery.print.template_revision, + ) + if template is None or template.revision is None: + raise ValueError("Template revision not found.") + if template.revision.published_at is None: + raise ValueError("The selected template revision is not published.") + result = integration.check_compatibility( + session, + principal, + template_id=config.delivery.print.template_id, + revision=config.delivery.print.template_revision, + output_format=config.delivery.print.output_format, + available_fields=_print_template_available_fields(config), + ) + except (PermissionError, RuntimeError, ValueError) as exc: + return [ + SemanticIssue( + severity=Severity.ERROR, + code="print_template_unavailable", + message=f"The selected printable template cannot be used: {exc}", + path="/delivery/print/template_id", + ) + ] + if result.compatible: + return [] + details = [*result.missing_fields, *result.incompatible_fields] + suffix = ( + f" Missing or incompatible fields: {', '.join(details)}." + if details + else "" + ) + return [ + SemanticIssue( + severity=Severity.ERROR, + code="print_template_incompatible", + message=( + "The selected printable template is not compatible with this Campaign." + f"{suffix}" + ), + path="/delivery/print/template_id", + ) + ] + + def validate_campaign_version( session: Session, *, @@ -433,6 +571,7 @@ def validate_campaign_version( version_id: str, check_files: bool = False, user_id: str | None = None, + principal: ApiPrincipal | None = None, lock_on_success: bool = True, ) -> dict[str, Any]: version, snapshot_path, config = load_version_config(session, version_id) @@ -483,6 +622,7 @@ def validate_campaign_version( campaign_file=prepared.path, check_files=True, postbox_available=postbox_integration().available, + templates_available=templates_integration().available, ) else: report = validate_campaign_config( @@ -490,7 +630,15 @@ def validate_campaign_version( campaign_file=snapshot_path, check_files=False, postbox_available=postbox_integration().available, + templates_available=templates_integration().available, ) + report.issues.extend( + _print_template_compatibility_issues( + session, + principal=principal, + config=config, + ) + ) report_json = report.model_dump(mode="json") report_json.update( { @@ -569,6 +717,8 @@ def _job_from_message( version_id: str, message: MessageDraft, resolved_postbox_targets: list[dict[str, Any]] | None = None, + resolved_print_output: dict[str, Any] | None = None, + delivery_provenance: dict[str, Any] | None = None, stored_eml: _StoredEmlArtifact | None = None, ) -> CampaignJob: recipient_email = message.to[0].email if message.to else None @@ -576,6 +726,7 @@ def _job_from_message( if stored_eml is not None: eml_sha256 = stored_eml.sha256 message_id_header = stored_eml.message_id_header + channel_policy = DeliveryChannelPolicy(message.delivery_channel_policy) return CampaignJob( tenant_id=tenant_id, campaign_id=campaign_id, @@ -602,9 +753,16 @@ def _job_from_message( delivery_channel_policy=message.delivery_channel_policy, postbox_status=( JobPostboxStatus.PENDING.value - if DeliveryChannelPolicy(message.delivery_channel_policy).uses_postbox + if channel_policy.uses_postbox else JobPostboxStatus.NOT_REQUESTED.value ), + print_status=( + JobPrintStatus.READY.value + if channel_policy.uses_print and resolved_print_output + else JobPrintStatus.FAILED.value + if channel_policy.uses_print + else JobPrintStatus.NOT_REQUESTED.value + ), imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value, @@ -621,7 +779,9 @@ def _job_from_message( for item in message.disposition_notification_to ], }, + delivery_provenance=delivery_provenance or {}, resolved_postbox_targets=resolved_postbox_targets or [], + resolved_print_output=resolved_print_output, resolved_attachments=[ files_integration().public_attachment_summary_payload(item) for item in message.attachments @@ -665,6 +825,192 @@ def _resolve_built_postbox_targets( return resolved_by_index +def _print_render_item( + config: CampaignConfig, + entry: Any, +) -> dict[str, Any]: + fields = effective_entry_field_values(config, entry) + distribution = dict(entry.distribution_source or {}) + target = entry.print_target.model_dump(mode="json") if entry.print_target else {} + return { + **fields, + "fields": fields, + "campaign": { + "id": config.campaign.id, + "name": config.campaign.name, + "description": config.campaign.description, + }, + "recipient_key": str(distribution.get("recipient_key") or entry.id or ""), + "display_name": entry.name or "", + "print_target": target, + "distribution": distribution, + } + + +def _canonical_sha256(value: object) -> str: + payload = json.dumps( + value, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ).encode("utf-8") + return hashlib.sha256(payload).hexdigest() + + +def _resolve_built_print_outputs( + session: Session, + *, + storage: StorageBackend, + tenant_id: str, + build_id: str, + version: CampaignVersion, + principal: ApiPrincipal | None, + config: CampaignConfig, + built_messages: list[Any], + entries_by_index: dict[int, Any], +) -> dict[int, dict[str, Any]]: + printable: list[tuple[Any, Any, dict[str, Any]]] = [] + for built in built_messages: + policy = DeliveryChannelPolicy(built.draft.delivery_channel_policy) + if not policy.uses_print: + continue + entry = entries_by_index.get(built.draft.entry_index) + if entry is None: + raise CampaignPersistenceError( + "A printable recipient row is missing from the Campaign input." + ) + if built.draft.validation_status not in { + MessageValidationStatus.READY, + MessageValidationStatus.WARNING, + }: + continue + printable.append((built, entry, _print_render_item(config, entry))) + if not printable: + return {} + if principal is None: + raise CampaignPersistenceError( + "Printable output requires the authenticated actor that validated the Campaign." + ) + print_config = config.delivery.print + if not print_config.template_id: + raise CampaignPersistenceError( + "Printable output requires a selected template." + ) + items = tuple(item for _built, _entry, item in printable) + routes = [ + { + "entry_index": built.draft.entry_index, + "entry_id": built.draft.entry_id, + "channel_policy": built.draft.delivery_channel_policy, + "print_target": item["print_target"], + "distribution": item["distribution"], + } + for built, _entry, item in printable + ] + input_snapshot = { + "producer_module": "campaigns", + "campaign_id": version.campaign_id, + "campaign_version_id": version.id, + "campaign_version_number": version.version_number, + "actor_account_id": principal.account_id, + "routes": routes, + } + render_key = _canonical_sha256( + { + "template_id": print_config.template_id, + "template_revision": print_config.template_revision, + "usage": print_config.usage, + "output_format": print_config.output_format, + "profile_id": print_config.profile_id, + "items": items, + "input_snapshot": input_snapshot, + } + ) + result = templates_integration().render( + session, + principal, + request=TemplateRenderRequest( + template_id=print_config.template_id, + revision=print_config.template_revision, + usage=print_config.usage, + output_format=print_config.output_format, + profile_id=print_config.profile_id, + items=items, + input_snapshot=input_snapshot, + mode="final", + idempotency_key=f"campaign:{version.id}:print:{render_key}", + persist_to_files=print_config.persist_to_files, + ), + ) + artifact = asdict(result.artifact) if result.artifact is not None else None + if artifact and artifact.get("kind") == "bounded_download": + if result.payload is None: + raise CampaignPersistenceError( + "Templates returned a bounded print artifact without its payload." + ) + storage_key = ( + f"campaign-artifacts/{tenant_id}/{version.campaign_id}/{version.id}/" + f"{build_id}/print-{result.output_sha256}" + ) + try: + storage.put_bytes( + storage_key, + result.payload, + content_type=result.content_type, + ) + except StorageBackendError as exc: + raise CampaignPersistenceError( + f"Printable Campaign output could not be persisted: {exc}" + ) from exc + artifact["storage_key"] = storage_key + artifact["download_path"] = ( + f"/api/v1/campaigns/{version.campaign_id}/versions/{version.id}/" + "print-output/download" + ) + artifact["provenance"] = { + **dict(artifact.get("provenance") or {}), + "module": "campaigns", + "source_module": "templates", + "source_render_id": result.render_id, + "bounded": True, + } + common = { + "status": "ready", + "render_id": result.render_id, + "template_id": result.template_id, + "template_revision_id": result.revision_id, + "template_revision": result.revision, + "template_hash": result.template_hash, + "input_hash": result.input_hash, + "renderer_version": result.renderer_version, + "output_format": result.output_format, + "output_sha256": result.output_sha256, + "output_size_bytes": result.output_size_bytes, + "item_count": result.item_count, + "page_count": result.page_count, + "artifact": artifact, + "diagnostics": [dict(item) for item in result.diagnostics], + "actor_account_id": principal.account_id, + "render_idempotency_key": f"campaign:{version.id}:print:{render_key}", + } + resolved: dict[int, dict[str, Any]] = {} + for item_index, (built, entry, _item) in enumerate(printable): + resolved[built.draft.entry_index] = { + **common, + "item_index": item_index, + "recipient_key": str( + entry.distribution_source.get("recipient_key") + or entry.id + or built.draft.entry_index + ), + "route": entry.print_target.model_dump(mode="json") + if entry.print_target + else None, + } + return resolved + + def _campaign_build_report(result: Any, files: Any) -> dict[str, Any]: report_json = result.report.model_dump(mode="json", by_alias=True) for message_payload, message in zip( @@ -701,18 +1047,26 @@ def _replace_version_jobs( version_id: str, built_messages: list[Any], postbox_targets_by_index: dict[int, list[dict[str, Any]]], + print_outputs_by_index: dict[int, dict[str, Any]], + delivery_provenance_by_index: dict[int, dict[str, Any]], stored_eml_by_index: dict[int, _StoredEmlArtifact], ) -> tuple[list[tuple[CampaignJob, MessageDraft]], list[str]]: - old_storage_keys = [ - str(key) - for (key,) in session.query(CampaignJob.eml_storage_key) - .filter( - CampaignJob.campaign_version_id == version_id, - CampaignJob.eml_storage_key.is_not(None), + old_storage_keys: list[str] = [] + old_job_artifacts = ( + session.query( + CampaignJob.eml_storage_key, + CampaignJob.resolved_print_output, ) + .filter(CampaignJob.campaign_version_id == version_id) .all() - if key - ] + ) + for eml_storage_key, print_output in old_job_artifacts: + if eml_storage_key: + old_storage_keys.append(str(eml_storage_key)) + if isinstance(print_output, dict): + artifact = print_output.get("artifact") + if isinstance(artifact, dict) and artifact.get("storage_key"): + old_storage_keys.append(str(artifact["storage_key"])) session.query(CampaignIssue).filter( CampaignIssue.campaign_version_id == version_id, CampaignIssue.job_id.is_not(None), @@ -732,6 +1086,13 @@ def _replace_version_jobs( resolved_postbox_targets=postbox_targets_by_index.get( built.draft.entry_index, [] ), + resolved_print_output=print_outputs_by_index.get( + built.draft.entry_index + ), + delivery_provenance=delivery_provenance_by_index.get( + built.draft.entry_index, + {}, + ), stored_eml=stored_eml_by_index.get(built.draft.entry_index), ) session.add(job) @@ -851,6 +1212,7 @@ def build_campaign_version( version_id: str, write_eml: bool = True, user_id: str | None = None, + principal: ApiPrincipal | None = None, ) -> dict[str, Any]: version, snapshot_path, config = load_version_config(session, version_id) campaign = session.get(Campaign, version.campaign_id) @@ -911,6 +1273,10 @@ def build_campaign_version( start=1, ) } + delivery_provenance_by_index = { + index: dict(entry.distribution_source or {}) + for index, entry in entries_by_index.items() + } resolved_postbox_targets_by_index = _resolve_built_postbox_targets( session, tenant_id=tenant_id, @@ -918,18 +1284,67 @@ def build_campaign_version( built_messages=result.built_messages, entries_by_index=entries_by_index, ) - stored_eml_by_index = _persist_built_eml_artifacts( - storage=storage, - tenant_id=tenant_id, - campaign_id=campaign.id, - version_id=version.id, - build_id=build_id, - built_messages=result.built_messages, + resolved_print_outputs_by_index = _resolve_built_print_outputs( + session, + storage=storage, + tenant_id=tenant_id, + build_id=build_id, + version=version, + principal=principal, + config=managed_config, + built_messages=result.built_messages, + entries_by_index=entries_by_index, + ) + new_print_storage_keys = sorted( + { + str(artifact["storage_key"]) + for output in resolved_print_outputs_by_index.values() + if isinstance(output, dict) + for artifact in [output.get("artifact")] + if isinstance(artifact, dict) and artifact.get("storage_key") + } ) - new_storage_keys = [item.storage_key for item in stored_eml_by_index.values()] + try: + stored_eml_by_index = _persist_built_eml_artifacts( + storage=storage, + tenant_id=tenant_id, + campaign_id=campaign.id, + version_id=version.id, + build_id=build_id, + built_messages=result.built_messages, + ) + except Exception: + _delete_storage_keys(storage, new_print_storage_keys) + raise + new_storage_keys = [ + *new_print_storage_keys, + *(item.storage_key for item in stored_eml_by_index.values()), + ] try: report_json = _campaign_build_report(result, files) report_json["built_by_user_id"] = user_id + if resolved_print_outputs_by_index: + first_output = next(iter(resolved_print_outputs_by_index.values())) + report_json["print_output"] = { + key: first_output.get(key) + for key in ( + "render_id", + "template_id", + "template_revision_id", + "template_revision", + "template_hash", + "input_hash", + "renderer_version", + "output_format", + "output_sha256", + "output_size_bytes", + "item_count", + "page_count", + "artifact", + "diagnostics", + "actor_account_id", + ) + } version.build_summary = report_json editor_state = copy.deepcopy(version.editor_state or {}) editor_state.pop("review_send", None) @@ -943,6 +1358,8 @@ def build_campaign_version( version_id=version.id, built_messages=result.built_messages, postbox_targets_by_index=resolved_postbox_targets_by_index, + print_outputs_by_index=resolved_print_outputs_by_index, + delivery_provenance_by_index=delivery_provenance_by_index, stored_eml_by_index=stored_eml_by_index, ) jobs = [job for job, _message in job_build_pairs] diff --git a/src/govoplan_campaign/backend/reports/aggregate.py b/src/govoplan_campaign/backend/reports/aggregate.py index 8bf5e77..cd4ade9 100644 --- a/src/govoplan_campaign/backend/reports/aggregate.py +++ b/src/govoplan_campaign/backend/reports/aggregate.py @@ -59,6 +59,7 @@ class AggregateOutcomeCounts(BaseModel): smtp_accepted: AggregateCount postbox_accepted: AggregateCount + print_accepted: AggregateCount delivered: AggregateCount partially_accepted: AggregateCount failed: AggregateCount @@ -109,6 +110,7 @@ class AggregateCampaignReport(BaseModel): _OUTCOME_KEYS = ( "smtp_accepted", "postbox_accepted", + "print_accepted", "delivered", "partially_accepted", "failed", @@ -188,6 +190,7 @@ def _query_aggregate_facts( { "smtp_accepted", "postbox_accepted", + "print_accepted", "delivered", "partially_accepted", "sent", @@ -217,6 +220,12 @@ def _query_aggregate_facts( else_=0, ) ).label("postbox_accepted"), + func.sum( + case( + (CampaignJob.send_status == "print_accepted", 1), + else_=0, + ) + ).label("print_accepted"), func.sum( case( (CampaignJob.send_status == "delivered", 1), @@ -398,6 +407,8 @@ def _outcome_counts(jobs: list[CampaignJob]) -> dict[str, int]: counts["smtp_accepted"] += 1 elif status == "postbox_accepted": counts["postbox_accepted"] += 1 + elif status == "print_accepted": + counts["print_accepted"] += 1 elif status == "delivered": counts["delivered"] += 1 elif status == "partially_accepted": @@ -512,6 +523,7 @@ def _completion_state( fully_accepted = ( counts["smtp_accepted"] + counts["postbox_accepted"] + + counts["print_accepted"] + counts["delivered"] ) partially_accepted = counts["partially_accepted"] diff --git a/src/govoplan_campaign/backend/reports/campaigns.py b/src/govoplan_campaign/backend/reports/campaigns.py index e09bc06..55e5e45 100644 --- a/src/govoplan_campaign/backend/reports/campaigns.py +++ b/src/govoplan_campaign/backend/reports/campaigns.py @@ -295,6 +295,7 @@ class _JobReportAggregate: queue: Counter[str] = field(default_factory=Counter) send: Counter[str] = field(default_factory=Counter) postbox: Counter[str] = field(default_factory=Counter) + print: Counter[str] = field(default_factory=Counter) imap: Counter[str] = field(default_factory=Counter) issue_total: int = 0 issue_severity: Counter[str] = field(default_factory=Counter) @@ -333,6 +334,7 @@ class _JobReportAggregate: self.queue[job.queue_status or "unknown"] += 1 self.send[job.send_status or "unknown"] += 1 self.postbox[job.postbox_status or "unknown"] += 1 + self.print[getattr(job, "print_status", None) or "unknown"] += 1 self.imap[job.imap_status or "unknown"] += 1 def _add_delivery_counts(self, job: CampaignJob, *, retry_max_attempts: int | None) -> None: @@ -398,6 +400,7 @@ NON_CANCELLABLE_SEND_STATUSES = { "skipped", "smtp_accepted", "postbox_accepted", + "print_accepted", "delivered", "partially_accepted", "sent", @@ -416,6 +419,7 @@ def _job_is_queueable_unattempted(job: CampaignJob) -> bool: return ( job.attempt_count == 0 and job.postbox_attempt_count == 0 + and getattr(job, "print_attempt_count", 0) == 0 and job.send_status in {"not_queued", "cancelled"} and _job_is_queueable(job) ) @@ -449,6 +453,7 @@ def _job_needs_attention(job: CampaignJob) -> bool: "rejected_permanent", "outcome_unknown", } + or getattr(job, "print_status", "not_requested") == "failed" or job.imap_status == "failed" ) @@ -465,6 +470,7 @@ def _job_is_recent_failure(job: CampaignJob) -> bool: "rejected_permanent", "outcome_unknown", } + or getattr(job, "print_status", "not_requested") == "failed" or job.imap_status == "failed" ) @@ -601,12 +607,20 @@ def _job_row( "postbox_status", "not_requested", ), + "print_status": getattr(job, "print_status", "not_requested"), "imap_status": job.imap_status, "attempt_count": job.attempt_count, "postbox_attempt_count": getattr(job, "postbox_attempt_count", 0), + "print_attempt_count": getattr(job, "print_attempt_count", 0), "postbox_target_count": len( getattr(job, "resolved_postbox_targets", None) or [] ), + "print_output": public_campaign_payload( + getattr(job, "resolved_print_output", None) + ), + "delivery_provenance": public_campaign_payload( + getattr(job, "delivery_provenance", None) or {} + ), "queued_at": job.queued_at.isoformat() if job.queued_at else None, "outcome_unknown_at": job.outcome_unknown_at.isoformat() if job.outcome_unknown_at else None, "sent_at": job.sent_at.isoformat() if job.sent_at else None, @@ -713,6 +727,7 @@ def _job_evidence_row( "message_id_header": job.message_id_header, **_job_evidence_addresses(recipients), "postbox_targets": _job_postbox_target_summary(job), + **_job_print_and_route_evidence(job), "attachment_names": _attachment_names(job.resolved_attachments), **_job_attempt_evidence(latest_smtp=latest_smtp, latest_imap=latest_imap), "latest_message_action_kind": ( @@ -766,6 +781,45 @@ def _job_postbox_target_summary(job: CampaignJob) -> str: ) +def _job_print_and_route_evidence(job: CampaignJob) -> dict[str, Any]: + output = ( + job.resolved_print_output + if isinstance(getattr(job, "resolved_print_output", None), dict) + else {} + ) + artifact = output.get("artifact") if isinstance(output.get("artifact"), dict) else {} + provenance = ( + job.delivery_provenance + if isinstance(getattr(job, "delivery_provenance", None), dict) + else {} + ) + selected_route = ( + provenance.get("selected_route") + if isinstance(provenance.get("selected_route"), dict) + else {} + ) + return { + "distribution_list_id": provenance.get("list_id"), + "distribution_list_revision": provenance.get("list_revision"), + "distribution_snapshot_id": provenance.get("snapshot_id"), + "distribution_expansion_hash": provenance.get("expansion_hash"), + "distribution_recipient_key": provenance.get("recipient_key"), + "selected_route_channel": selected_route.get("channel"), + "selected_route_target_key": selected_route.get("target_key"), + "route_reason": provenance.get("route_reason"), + "print_render_id": output.get("render_id"), + "print_template_id": output.get("template_id"), + "print_template_revision_id": output.get("template_revision_id"), + "print_template_hash": output.get("template_hash"), + "print_input_hash": output.get("input_hash"), + "print_output_sha256": output.get("output_sha256"), + "print_artifact_kind": artifact.get("kind"), + "print_artifact_file_id": artifact.get("file_asset_id"), + "print_artifact_download_path": artifact.get("download_path"), + "print_actor_account_id": output.get("actor_account_id"), + } + + def _iso_timestamp(value: datetime | None) -> str | None: return value.isoformat() if value else None @@ -1421,6 +1475,7 @@ def _campaign_report_status_counts_from_aggregate( "queue": dict(aggregate.queue), "send": dict(aggregate.send), "postbox": dict(aggregate.postbox), + "print": dict(aggregate.print), "imap": dict(aggregate.imap), } @@ -1435,6 +1490,7 @@ def _campaign_report_cards_from_aggregate( send_counts.get("sent", 0) + send_counts.get("smtp_accepted", 0) + send_counts.get("postbox_accepted", 0) + + send_counts.get("print_accepted", 0) + send_counts.get("delivered", 0) + send_counts.get("partially_accepted", 0) ) @@ -1460,6 +1516,7 @@ def _campaign_report_cards_from_aggregate( + send_counts.get("smtp_accepted", 0) ), "postbox_accepted": send_counts.get("postbox_accepted", 0), + "print_accepted": send_counts.get("print_accepted", 0), "delivered": send_counts.get("delivered", 0), "partially_accepted": send_counts.get("partially_accepted", 0), "failed": failed, @@ -1594,11 +1651,31 @@ def generate_jobs_csv( "send_status", "delivery_channel_policy", "postbox_status", + "print_status", "imap_status", "attempt_count", "postbox_attempt_count", + "print_attempt_count", "postbox_target_count", "postbox_targets", + "distribution_list_id", + "distribution_list_revision", + "distribution_snapshot_id", + "distribution_expansion_hash", + "distribution_recipient_key", + "selected_route_channel", + "selected_route_target_key", + "route_reason", + "print_render_id", + "print_template_id", + "print_template_revision_id", + "print_template_hash", + "print_input_hash", + "print_output_sha256", + "print_artifact_kind", + "print_artifact_file_id", + "print_artifact_download_path", + "print_actor_account_id", "queued_at", "outcome_unknown_at", "sent_at", diff --git a/src/govoplan_campaign/backend/reports/provider.py b/src/govoplan_campaign/backend/reports/provider.py index a7ac28e..1c1e589 100644 --- a/src/govoplan_campaign/backend/reports/provider.py +++ b/src/govoplan_campaign/backend/reports/provider.py @@ -214,6 +214,12 @@ def _descriptor() -> ReportDescriptor: "suppressed_count", "Outcomes", ), + ( + "outcomes.print_accepted", + "Printable output accepted", + "suppressed_count", + "Outcomes", + ), ( "outcomes.delivered", "Both channels accepted", diff --git a/src/govoplan_campaign/backend/routes/campaigns.py b/src/govoplan_campaign/backend/routes/campaigns.py index a26044b..59ff80c 100644 --- a/src/govoplan_campaign/backend/routes/campaigns.py +++ b/src/govoplan_campaign/backend/routes/campaigns.py @@ -65,6 +65,7 @@ from govoplan_campaign.backend.campaign.postbox_targets import ( from govoplan_campaign.backend.integrations import ( PostboxDeliveryUnavailable, postbox_integration, + templates_integration, ) from govoplan_core.db.session import get_session from govoplan_core.core.distribution_lists import ( @@ -755,6 +756,29 @@ def campaign_postbox_catalog( ) +@router.get("/{campaign_id}/print-templates") +def campaign_print_templates( + campaign_id: str, + query: str = Query(default="", min_length=0, max_length=200), + session: Session = Depends(get_session), + principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")), +): + _get_campaign_for_principal(session, campaign_id, principal) + integration = templates_integration() + if not integration.available: + return {"available": False, "templates": []} + templates = integration.list_templates( + session, + principal, + query=query, + limit=250, + ) + return { + "available": True, + "templates": [dataclasses.asdict(template) for template in templates], + } + + @router.post( "/{campaign_id}/recipient-address-sources/snapshot", response_model=CampaignRecipientAddressSourceSnapshotResponse, diff --git a/src/govoplan_campaign/backend/routes/jobs.py b/src/govoplan_campaign/backend/routes/jobs.py index 85f7bee..a660245 100644 --- a/src/govoplan_campaign/backend/routes/jobs.py +++ b/src/govoplan_campaign/backend/routes/jobs.py @@ -27,6 +27,7 @@ from govoplan_campaign.backend.db.models import ( CampaignMessageActionAttempt, ImapAppendAttempt, PostboxDeliveryAttempt, + PrintOutputAttempt, SendAttempt, ) from govoplan_campaign.backend.integrations import postbox_integration @@ -397,6 +398,12 @@ def get_job_detail( ), label="Postbox attempts for this campaign job", ) + print_attempts = _job_attempt_rows( + session.query(PrintOutputAttempt) + .filter(PrintOutputAttempt.job_id == job.id) + .order_by(PrintOutputAttempt.attempt_number.asc()), + label="Printable output attempts for this campaign job", + ) message_actions = _job_attempt_rows( session.query(CampaignMessageAction) .filter(CampaignMessageAction.job_id == job.id) @@ -419,9 +426,10 @@ def get_job_detail( attempts=_job_attempts_payload( send_attempts, imap_attempts, - postbox_attempts, - message_actions, - message_action_attempts, + postbox_attempts=postbox_attempts, + print_attempts=print_attempts, + message_actions=message_actions, + message_action_attempts=message_action_attempts, postbox_receipts=_postbox_receipts_for_attempts( session, tenant_id=principal.tenant_id, @@ -476,6 +484,12 @@ def get_job_diagnostics( ), label="Postbox diagnostics for this campaign job", ) + print_attempts = _job_attempt_rows( + session.query(PrintOutputAttempt) + .filter(PrintOutputAttempt.job_id == job.id) + .order_by(PrintOutputAttempt.attempt_number.asc()), + label="Printable output diagnostics for this campaign job", + ) message_actions = _job_attempt_rows( session.query(CampaignMessageAction) .filter(CampaignMessageAction.job_id == job.id) @@ -497,9 +511,10 @@ def get_job_diagnostics( job, send_attempts, imap_attempts, - postbox_attempts, - message_actions, - message_action_attempts, + postbox_attempts=postbox_attempts, + print_attempts=print_attempts, + message_actions=message_actions, + message_action_attempts=message_action_attempts, postbox_receipts=_postbox_receipts_for_attempts( session, tenant_id=principal.tenant_id, diff --git a/src/govoplan_campaign/backend/routes/versions.py b/src/govoplan_campaign/backend/routes/versions.py index 1005440..ac9304c 100644 --- a/src/govoplan_campaign/backend/routes/versions.py +++ b/src/govoplan_campaign/backend/routes/versions.py @@ -1,5 +1,7 @@ from __future__ import annotations +import hashlib +from urllib.parse import quote from fastapi import APIRouter, Depends, Header, HTTPException, Query, Response, status from sqlalchemy.orm import Session @@ -19,6 +21,7 @@ from govoplan_campaign.backend.schemas import ( ) from govoplan_core.auth import ApiPrincipal, has_scope, require_scope from govoplan_core.audit.logging import audit_from_principal +from govoplan_core.core.object_storage import StorageBackendError from govoplan_campaign.backend.db.models import ( CampaignVersion, ) @@ -28,6 +31,7 @@ from govoplan_campaign.backend.response_security import ( ) from govoplan_campaign.backend.persistence.campaigns import ( CampaignPersistenceError, + _object_storage, build_campaign_version, validate_campaign_version, ) @@ -70,6 +74,86 @@ from govoplan_campaign.backend.routes.attachments import ( router = APIRouter(prefix="/campaigns", tags=["campaigns"]) +@router.get("/{campaign_id}/versions/{version_id}/print-output/download") +def download_print_output( + campaign_id: str, + version_id: str, + session: Session = Depends(get_session), + principal: ApiPrincipal = Depends(require_scope("campaigns:campaign:read")), +) -> Response: + _get_campaign_for_principal(session, campaign_id, principal) + _require_permission(principal, "campaigns:recipient:read") + try: + version = get_campaign_version_for_tenant( + session, + tenant_id=principal.tenant_id, + campaign_id=campaign_id, + version_id=version_id, + ) + except CampaignPersistenceError as exc: + raise HTTPException( + status_code=status.HTTP_404_NOT_FOUND, + detail=str(exc), + ) from exc + build_summary = ( + version.build_summary if isinstance(version.build_summary, dict) else {} + ) + print_output = build_summary.get("print_output") + artifact = ( + print_output.get("artifact") + if isinstance(print_output, dict) + and isinstance(print_output.get("artifact"), dict) + else {} + ) + storage_key = artifact.get("storage_key") + if artifact.get("kind") != "bounded_download" or not storage_key: + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="This printable output is not stored as a Campaign download.", + ) + try: + payload = _object_storage().get_bytes(str(storage_key)) + except StorageBackendError as exc: + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, + detail="The printable output is temporarily unavailable.", + ) from exc + expected_sha256 = str(print_output.get("output_sha256") or "") + actual_sha256 = hashlib.sha256(payload).hexdigest() + if not expected_sha256 or actual_sha256 != expected_sha256: + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="The printable output failed its integrity check.", + ) + audit_from_principal( + session, + principal, + action="campaign.print_output_downloaded", + object_type="campaign_version", + object_id=version.id, + details={ + "campaign_id": campaign_id, + "output_sha256": actual_sha256, + "template_id": print_output.get("template_id"), + "template_revision_id": print_output.get("template_revision_id"), + }, + commit=True, + ) + filename = quote( + str(artifact.get("filename") or "campaign-print-output.html"), + safe="._-", + ) + return Response( + content=payload, + media_type=str(artifact.get("content_type") or "application/octet-stream"), + headers={ + "Content-Disposition": f"attachment; filename*=UTF-8''{filename}", + "X-Content-SHA256": actual_sha256, + "X-Content-Type-Options": "nosniff", + }, + ) + + @router.get("/{campaign_id}/versions", response_model=list[CampaignVersionResponse]) def list_versions( campaign_id: str, @@ -607,6 +691,7 @@ def validate_version( version_id=version_id, check_files=payload.check_files, user_id=principal.user.id, + principal=principal, ) audit_from_principal( session, @@ -659,6 +744,7 @@ def build_version( version_id=version_id, write_eml=payload.write_eml if payload else True, user_id=principal.user.id, + principal=principal, ) audit_from_principal( session, diff --git a/src/govoplan_campaign/backend/schema/campaign.schema.json b/src/govoplan_campaign/backend/schema/campaign.schema.json index d9bd3a5..802dff1 100644 --- a/src/govoplan_campaign/backend/schema/campaign.schema.json +++ b/src/govoplan_campaign/backend/schema/campaign.schema.json @@ -537,6 +537,9 @@ "postbox": { "$ref": "#/$defs/postbox_delivery" }, + "print": { + "$ref": "#/$defs/print_delivery" + }, "rate_limit": { "type": "object", "properties": { @@ -659,9 +662,12 @@ "enum": [ "mail", "postbox", + "print", "mail_and_postbox", "mail_then_postbox", - "postbox_then_mail" + "postbox_then_mail", + "mail_then_print", + "postbox_then_print" ], "default": "mail" }, @@ -827,6 +833,73 @@ "additionalProperties": false, "default": {} }, + "print_target": { + "type": "object", + "required": ["channel", "target", "target_key"], + "properties": { + "channel": { + "type": "string", + "enum": ["postal", "internal_mail"] + }, + "target": { + "type": "string", + "minLength": 1, + "maxLength": 4000 + }, + "target_key": { + "type": "string", + "minLength": 1, + "maxLength": 500 + }, + "contact_point_id": { + "type": ["string", "null"], + "maxLength": 36 + }, + "locale": { + "type": ["string", "null"], + "maxLength": 35 + }, + "decision_provenance": { + "type": "object", + "default": {} + } + }, + "additionalProperties": false + }, + "print_delivery": { + "type": "object", + "properties": { + "template_id": { + "type": ["string", "null"], + "maxLength": 36 + }, + "template_revision": { + "type": ["integer", "null"], + "minimum": 1 + }, + "usage": { + "type": "string", + "minLength": 1, + "maxLength": 100, + "default": "campaign_print" + }, + "output_format": { + "type": "string", + "enum": ["html", "text"], + "default": "html" + }, + "profile_id": { + "type": ["string", "null"], + "maxLength": 120 + }, + "persist_to_files": { + "type": "boolean", + "default": true + } + }, + "additionalProperties": false, + "default": {} + }, "attachment_config": { "type": "object", "required": [ @@ -1093,6 +1166,13 @@ "type": "boolean", "default": true }, + "print_target": { + "oneOf": [ + { "$ref": "#/$defs/print_target" }, + { "type": "null" } + ], + "default": null + }, "attachments": { "type": "array", "items": { diff --git a/src/govoplan_campaign/backend/sending/execution.py b/src/govoplan_campaign/backend/sending/execution.py index 69be903..dd7d8cc 100644 --- a/src/govoplan_campaign/backend/sending/execution.py +++ b/src/govoplan_campaign/backend/sending/execution.py @@ -22,8 +22,8 @@ from govoplan_campaign.backend.campaign.mail_profile_boundary import ( from govoplan_campaign.backend.integrations import MailProfileError, files_integration, mail_integration from govoplan_campaign.backend.path_security import CampaignPathSecurityError, assert_server_safe_campaign_paths -SNAPSHOT_VERSION = "7" -SUPPORTED_SNAPSHOT_VERSIONS = {"6", SNAPSHOT_VERSION} +SNAPSHOT_VERSION = "8" +SUPPORTED_SNAPSHOT_VERSIONS = {"6", "7", SNAPSHOT_VERSION} class ExecutionSnapshotError(RuntimeError): @@ -61,6 +61,7 @@ class ExecutionSnapshot(BaseModel): imap_transport_revision: str | None = None uses_mail: bool = True uses_postbox: bool = False + uses_print: bool = False delivery: DeliveryConfig @@ -162,6 +163,8 @@ def _policy_fingerprint( if snapshot_version == "6": delivery_payload.pop("channel_policy", None) delivery_payload.pop("postbox", None) + if snapshot_version in {"6", "7"}: + delivery_payload.pop("print", None) return _sha256( { "validation_policy": raw_json.get("validation_policy"), @@ -207,6 +210,13 @@ def _job_execution_input_payload( ), } ) + if snapshot_version not in {"6", "7"}: + payload["delivery_provenance_sha256"] = _sha256( + getattr(job, "delivery_provenance", None) or {} + ) + payload["resolved_print_output_sha256"] = _sha256( + getattr(job, "resolved_print_output", None) or {} + ) return payload @@ -268,6 +278,7 @@ def create_execution_snapshot( } uses_mail = any(policy.uses_mail for policy in channel_policies) uses_postbox = any(policy.uses_postbox for policy in channel_policies) + uses_print = any(policy.uses_print for policy in channel_policies) for job in job_list: job.execution_input_sha256 = job_execution_input_hash( job, @@ -304,6 +315,7 @@ def create_execution_snapshot( imap_transport_revision=imap_transport_revision, uses_mail=uses_mail, uses_postbox=uses_postbox, + uses_print=uses_print, created_at=datetime.now(timezone.utc).isoformat(), delivery=delivery, ).model_dump(mode="json") diff --git a/src/govoplan_campaign/backend/sending/jobs.py b/src/govoplan_campaign/backend/sending/jobs.py index d809575..e7370ee 100644 --- a/src/govoplan_campaign/backend/sending/jobs.py +++ b/src/govoplan_campaign/backend/sending/jobs.py @@ -37,11 +37,13 @@ from govoplan_campaign.backend.db.models import ( JobBuildStatus, JobImapStatus, JobPostboxStatus, + JobPrintStatus, JobQueueStatus, JobSendStatus, JobValidationStatus, ImapAppendAttempt, PostboxDeliveryAttempt, + PrintOutputAttempt, SendAttempt, ) from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy @@ -241,17 +243,26 @@ class _MailChannelOutcome: ) +@dataclass(frozen=True, slots=True) +class _PrintChannelOutcome: + accepted: bool = False + rejected_permanent: bool = False + message: str | None = None + + @dataclass(frozen=True, slots=True) class _DeliveryOutcomeSummary: mail_accepted: bool postbox_accepted: int postbox_rejected: int + print_accepted: bool + print_rejected: bool outcome_unknown: bool temporary_rejection: bool @property def accepted_count(self) -> int: - return int(self.mail_accepted) + self.postbox_accepted + return int(self.mail_accepted) + self.postbox_accepted + int(self.print_accepted) @dataclass(frozen=True, slots=True) @@ -275,11 +286,13 @@ QUEUEABLE_VALIDATION_STATUSES = { SMTP_ACCEPTED_STATUSES = {JobSendStatus.SMTP_ACCEPTED.value, JobSendStatus.SENT.value} DELIVERY_ACCEPTED_STATUSES = SMTP_ACCEPTED_STATUSES | { JobSendStatus.POSTBOX_ACCEPTED.value, + JobSendStatus.PRINT_ACCEPTED.value, JobSendStatus.DELIVERED.value, JobSendStatus.PARTIALLY_ACCEPTED.value, } FULLY_ACCEPTED_STATUSES = SMTP_ACCEPTED_STATUSES | { JobSendStatus.POSTBOX_ACCEPTED.value, + JobSendStatus.PRINT_ACCEPTED.value, JobSendStatus.DELIVERED.value, } DELIVERY_MODE_SYNCHRONOUS = "synchronous" @@ -2890,6 +2903,11 @@ def send_campaign_job( descriptions.append( f"Postbox to {len(job.resolved_postbox_targets or [])} target(s)" ) + if policy.uses_print: + output = job.resolved_print_output or {} + descriptions.append( + f"Printable output item {int(output.get('item_index') or 0) + 1}" + ) return SendJobResult( job_id=job.id, status="dry_run", @@ -2985,10 +3003,14 @@ def _send_job_delivery_context( except ExecutionSnapshotError as exc: raise SendJobError(str(exc)) from exc - message_bytes = _load_eml_bytes_for_job(job) channel_policy = DeliveryChannelPolicy( getattr(job, "delivery_channel_policy", DeliveryChannelPolicy.MAIL.value) ) + message_bytes = ( + _load_eml_bytes_for_job(job) + if channel_policy.uses_mail or channel_policy.uses_postbox + else b"" + ) envelope_from: str | None = None envelope_recipients: list[str] = [] if channel_policy.uses_mail: @@ -3058,6 +3080,16 @@ def _send_claimed_campaign_job( use_rate_limit=use_rate_limit, enqueue_imap_task=enqueue_imap_task, ) + if channel_policy == DeliveryChannelPolicy.PRINT: + print_outcome = _deliver_print_channel(session, job=job) + return _finalize_multichannel_job( + session, + job_id=job.id, + channel_policy=channel_policy, + mail=None, + postbox=None, + print_output=print_outcome, + ) return _send_claimed_multichannel_job( session, job=job, @@ -3262,13 +3294,115 @@ def _empty_postbox_outcome() -> PostboxChannelOutcome: return PostboxChannelOutcome() +def _deliver_print_channel( + session: Session, + *, + job: CampaignJob, +) -> _PrintChannelOutcome: + current = session.get(CampaignJob, job.id) + if current is None: + raise SendJobError("Campaign job disappeared before printable output acceptance.") + if current.print_status == JobPrintStatus.ACCEPTED.value: + return _PrintChannelOutcome(accepted=True, message="Output was already accepted.") + output = ( + current.resolved_print_output + if isinstance(current.resolved_print_output, dict) + else {} + ) + artifact = output.get("artifact") if isinstance(output.get("artifact"), dict) else {} + output_sha256 = str(output.get("output_sha256") or artifact.get("sha256") or "") + render_id = str(output.get("render_id") or "") + if ( + current.print_status != JobPrintStatus.READY.value + or not render_id + or not output_sha256 + ): + current.print_status = JobPrintStatus.FAILED.value + current.last_error = "The frozen printable output artifact is missing or incomplete." + session.add(current) + session.commit() + return _PrintChannelOutcome( + rejected_permanent=True, + message=current.last_error, + ) + attempt_number = current.print_attempt_count + 1 + idempotency_key = ( + f"campaign:{current.campaign_version_id}:{current.id}:print:" + f"{render_id}:{int(output.get('item_index') or 0)}" + ) + existing = ( + session.query(PrintOutputAttempt) + .filter( + PrintOutputAttempt.tenant_id == current.tenant_id, + PrintOutputAttempt.idempotency_key == idempotency_key, + ) + .one_or_none() + ) + if existing is not None and existing.status == JobPrintStatus.ACCEPTED.value: + current.print_status = JobPrintStatus.ACCEPTED.value + current.print_attempt_count = max( + current.print_attempt_count, + existing.attempt_number, + ) + session.add(current) + session.commit() + return _PrintChannelOutcome(accepted=True, message="Output was already accepted.") + now = _utcnow() + attempt = PrintOutputAttempt( + tenant_id=current.tenant_id, + job_id=current.id, + attempt_number=attempt_number, + idempotency_key=idempotency_key, + status=JobPrintStatus.ACCEPTED.value, + render_id=render_id, + artifact_sha256=output_sha256, + evidence={ + "template_id": output.get("template_id"), + "template_revision_id": output.get("template_revision_id"), + "template_hash": output.get("template_hash"), + "input_hash": output.get("input_hash"), + "output_sha256": output_sha256, + "artifact": artifact, + "route": output.get("route"), + "recipient_key": output.get("recipient_key"), + "item_index": output.get("item_index"), + "actor_account_id": output.get("actor_account_id"), + }, + started_at=now, + finished_at=now, + ) + current.print_status = JobPrintStatus.ACCEPTED.value + current.print_attempt_count = attempt_number + session.add(attempt) + session.add(current) + session.commit() + return _PrintChannelOutcome( + accepted=True, + message="Frozen printable output was accepted for distribution.", + ) + + +def _skip_print_channel(session: Session, job_id: str) -> None: + current = session.get(CampaignJob, job_id) + if current is None or current.print_status == JobPrintStatus.ACCEPTED.value: + return + current.print_status = JobPrintStatus.SKIPPED.value + session.add(current) + session.commit() + + def _final_multichannel_status( *, channel_policy: DeliveryChannelPolicy, mail: _MailChannelOutcome | None, postbox: PostboxChannelOutcome | None, + print_output: _PrintChannelOutcome | None = None, ) -> str: - outcome = _delivery_outcome_summary(mail=mail, postbox=postbox) + outcome = _delivery_outcome_summary( + mail=mail, + postbox=postbox, + print_output=print_output, + ) if outcome.outcome_unknown: return JobSendStatus.OUTCOME_UNKNOWN.value if not outcome.accepted_count: @@ -3287,11 +3421,14 @@ def _delivery_outcome_summary( *, mail: _MailChannelOutcome | None, postbox: PostboxChannelOutcome | None, + print_output: _PrintChannelOutcome | None = None, ) -> _DeliveryOutcomeSummary: return _DeliveryOutcomeSummary( mail_accepted=bool(mail and mail.accepted), postbox_accepted=int(postbox.accepted_count if postbox else 0), postbox_rejected=int(postbox.rejected_count if postbox else 0), + print_accepted=bool(print_output and print_output.accepted), + print_rejected=bool(print_output and print_output.rejected_permanent), outcome_unknown=bool( (mail and mail.outcome_unknown) or (postbox and postbox.outcome_unknown) ), @@ -3310,6 +3447,10 @@ def _classify_postbox_delivery(outcome: _DeliveryOutcomeSummary) -> str: ) +def _classify_print_delivery(_outcome: _DeliveryOutcomeSummary) -> str: + return JobSendStatus.PRINT_ACCEPTED.value + + def _classify_dual_delivery(outcome: _DeliveryOutcomeSummary) -> str: fully_delivered = ( outcome.mail_accepted @@ -3326,15 +3467,16 @@ def _classify_dual_delivery(outcome: _DeliveryOutcomeSummary) -> str: def _classify_fallback_delivery(outcome: _DeliveryOutcomeSummary) -> str: if outcome.postbox_rejected: return JobSendStatus.PARTIALLY_ACCEPTED.value - return ( - JobSendStatus.SMTP_ACCEPTED.value - if outcome.mail_accepted - else JobSendStatus.POSTBOX_ACCEPTED.value - ) + if outcome.mail_accepted: + return JobSendStatus.SMTP_ACCEPTED.value + if outcome.postbox_accepted: + return JobSendStatus.POSTBOX_ACCEPTED.value + return JobSendStatus.PRINT_ACCEPTED.value _ACCEPTED_DELIVERY_CLASSIFIERS = { DeliveryChannelPolicy.POSTBOX: _classify_postbox_delivery, + DeliveryChannelPolicy.PRINT: _classify_print_delivery, DeliveryChannelPolicy.MAIL_AND_POSTBOX: _classify_dual_delivery, } @@ -3342,12 +3484,15 @@ _ACCEPTED_DELIVERY_CLASSIFIERS = { def _multichannel_messages( mail: _MailChannelOutcome | None, postbox: PostboxChannelOutcome | None, + print_output: _PrintChannelOutcome | None = None, ) -> list[str]: values: list[str] = [] if mail and mail.message: values.append(f"Mail: {mail.message}") if postbox: values.extend(f"Postbox: {message}" for message in postbox.messages if message) + if print_output and print_output.message: + values.append(f"Print: {print_output.message}") return values @@ -3358,6 +3503,7 @@ def _finalize_multichannel_job( channel_policy: DeliveryChannelPolicy, mail: _MailChannelOutcome | None, postbox: PostboxChannelOutcome | None, + print_output: _PrintChannelOutcome | None = None, commit: bool = True, ) -> SendJobResult: job = session.get(CampaignJob, job_id) @@ -3367,10 +3513,11 @@ def _finalize_multichannel_job( channel_policy=channel_policy, mail=mail, postbox=postbox, + print_output=print_output, ) accepted = status in DELIVERY_ACCEPTED_STATUSES unknown = status == JobSendStatus.OUTCOME_UNKNOWN.value - messages = _multichannel_messages(mail, postbox) + messages = _multichannel_messages(mail, postbox, print_output) job.queue_status = JobQueueStatus.DRAFT.value job.send_status = status job.claim_token = None @@ -3378,7 +3525,11 @@ def _finalize_multichannel_job( job.outcome_unknown_at = _utcnow() if unknown else None if accepted or ( unknown - and bool((mail and mail.accepted) or (postbox and postbox.accepted_count)) + and bool( + (mail and mail.accepted) + or (postbox and postbox.accepted_count) + or (print_output and print_output.accepted) + ) ): job.sent_at = job.sent_at or _utcnow() files_integration().mark_job_attachment_uses_sent(session, job) @@ -3397,7 +3548,11 @@ def _finalize_multichannel_job( return SendJobResult( job_id=job.id, status=status, - attempt_number=job.attempt_count + job.postbox_attempt_count, + attempt_number=( + job.attempt_count + + job.postbox_attempt_count + + getattr(job, "print_attempt_count", 0) + ), message=job.last_error, ) @@ -3414,8 +3569,55 @@ def _send_claimed_multichannel_job( ) -> SendJobResult: mail_outcome: _MailChannelOutcome | None = None postbox_outcome: PostboxChannelOutcome | None = None + print_outcome: _PrintChannelOutcome | None = None - if channel_policy == DeliveryChannelPolicy.MAIL_THEN_POSTBOX: + if channel_policy == DeliveryChannelPolicy.MAIL_THEN_PRINT: + if job.print_status == JobPrintStatus.ACCEPTED.value: + print_outcome = _PrintChannelOutcome( + accepted=True, + message="Output was already accepted.", + ) + else: + mail_outcome = _deliver_mail_channel( + session, + job=job, + claim_token=claim_token, + context=context, + use_rate_limit=use_rate_limit, + enqueue_imap_task=enqueue_imap_task, + ) + if mail_outcome.rejected_before_acceptance: + current = session.get(CampaignJob, job.id) + if current is None: + raise SendJobError( + "Campaign job disappeared before printable fallback." + ) + print_outcome = _deliver_print_channel(session, job=current) + elif mail_outcome.accepted: + _skip_print_channel(session, job.id) + elif channel_policy == DeliveryChannelPolicy.POSTBOX_THEN_PRINT: + if job.print_status == JobPrintStatus.ACCEPTED.value: + print_outcome = _PrintChannelOutcome( + accepted=True, + message="Output was already accepted.", + ) + else: + postbox_outcome = deliver_campaign_job_to_postboxes( + session, + job=job, + message_bytes=context.message_bytes, + classification=context.snapshot.delivery.postbox.classification, + ) + if postbox_outcome.all_rejected_before_acceptance: + current = session.get(CampaignJob, job.id) + if current is None: + raise SendJobError( + "Campaign job disappeared before printable fallback." + ) + print_outcome = _deliver_print_channel(session, job=current) + elif postbox_outcome.accepted_count: + _skip_print_channel(session, job.id) + elif channel_policy == DeliveryChannelPolicy.MAIL_THEN_POSTBOX: prior_postbox_outcome = _postbox_outcome_from_attempts(session, job) if prior_postbox_outcome.outcome_unknown: postbox_outcome = prior_postbox_outcome @@ -3486,6 +3688,7 @@ def _send_claimed_multichannel_job( channel_policy=channel_policy, mail=mail_outcome, postbox=postbox_outcome, + print_output=print_outcome, ) diff --git a/src/govoplan_campaign/backend/services/job_queries.py b/src/govoplan_campaign/backend/services/job_queries.py index 909baae..a71f857 100644 --- a/src/govoplan_campaign/backend/services/job_queries.py +++ b/src/govoplan_campaign/backend/services/job_queries.py @@ -37,10 +37,12 @@ from govoplan_campaign.backend.db.models import ( ImapAppendAttempt, JobImapStatus, JobPostboxStatus, + JobPrintStatus, JobQueueStatus, JobSendStatus, JobValidationStatus, PostboxDeliveryAttempt, + PrintOutputAttempt, SendAttempt, ) from govoplan_campaign.backend.response_security import ( @@ -84,11 +86,13 @@ def _job_summary_payload( "send_status": job.send_status, "delivery_channel_policy": getattr(job, "delivery_channel_policy", "mail"), "postbox_status": getattr(job, "postbox_status", "not_requested"), + "print_status": getattr(job, "print_status", "not_requested"), "imap_status": job.imap_status, "eml_size_bytes": job.eml_size_bytes, "eml_sha256": job.eml_sha256, "attempt_count": job.attempt_count, "postbox_attempt_count": getattr(job, "postbox_attempt_count", 0), + "print_attempt_count": getattr(job, "print_attempt_count", 0), "postbox_target_count": len( getattr(job, "resolved_postbox_targets", None) or [] ), @@ -122,8 +126,12 @@ def _job_detail_payload(job: CampaignJob) -> dict[str, object]: "issues": job.issues_snapshot or [], "attachments": public_campaign_payload(job.resolved_attachments or []), "resolved_recipients": job.resolved_recipients or {}, + "delivery_provenance": getattr(job, "delivery_provenance", None) or {}, "resolved_postbox_targets": getattr(job, "resolved_postbox_targets", None) or [], + "resolved_print_output": public_campaign_payload( + getattr(job, "resolved_print_output", None) + ), } @@ -131,6 +139,7 @@ def _job_attempts_payload( send_attempts: list[SendAttempt], imap_attempts: list[ImapAppendAttempt], postbox_attempts: Sequence[PostboxDeliveryAttempt] = (), + print_attempts: Sequence[PrintOutputAttempt] = (), message_actions: Sequence[CampaignMessageAction] = (), message_action_attempts: Sequence[CampaignMessageActionAttempt] = (), *, @@ -212,6 +221,22 @@ def _job_attempts_payload( receipt_summary ) postbox_payloads.append(payload) + print_payloads: list[dict[str, object]] = [] + for attempt in print_attempts: + payload = { + "id": attempt.id, + "attempt_number": attempt.attempt_number, + "status": attempt.status, + "render_id": attempt.render_id, + "artifact_sha256": attempt.artifact_sha256, + "started_at": attempt.started_at, + "finished_at": attempt.finished_at, + } + if include_diagnostics: + payload["idempotency_key"] = attempt.idempotency_key + payload["evidence"] = attempt.evidence or {} + payload["error_message"] = attempt.error_message + print_payloads.append(payload) action_attempts_by_action = { attempt.action_id: attempt for attempt in message_action_attempts @@ -270,6 +295,7 @@ def _job_attempts_payload( "smtp": smtp_payloads, "imap": imap_payloads, "postbox": postbox_payloads, + "print": print_payloads, "message_actions": message_action_payloads, } @@ -305,6 +331,7 @@ def _job_diagnostics_payload( send_attempts: list[SendAttempt], imap_attempts: list[ImapAppendAttempt], postbox_attempts: Sequence[PostboxDeliveryAttempt] = (), + print_attempts: Sequence[PrintOutputAttempt] = (), message_actions: Sequence[CampaignMessageAction] = (), message_action_attempts: Sequence[CampaignMessageActionAttempt] = (), *, @@ -333,6 +360,7 @@ def _job_diagnostics_payload( send_attempts, imap_attempts, postbox_attempts, + print_attempts, message_actions, message_action_attempts, postbox_receipts=postbox_receipts, @@ -452,6 +480,7 @@ def _status_counts( "queue_status", "send_status", "postbox_status", + "print_status", "imap_status", ): column = getattr(CampaignJob, field_name) @@ -475,6 +504,7 @@ CAMPAIGN_JOB_GRID_SORT_COLUMNS = { "queue": CampaignJob.queue_status, "send": CampaignJob.send_status, "postbox": CampaignJob.postbox_status, + "print": CampaignJob.print_status, "imap": CampaignJob.imap_status, "attempts": CampaignJob.attempt_count, "updated": CampaignJob.updated_at, @@ -490,6 +520,10 @@ CAMPAIGN_JOB_GRID_LIST_FILTERS = { CampaignJob.postbox_status, {item.value for item in JobPostboxStatus}, ), + "print": ( + CampaignJob.print_status, + {item.value for item in JobPrintStatus}, + ), "imap": (CampaignJob.imap_status, {item.value for item in JobImapStatus}), } @@ -957,6 +991,7 @@ class CampaignJobsQuery: "queue", "send", "postbox", + "print", "imap", "attempts", "updated", @@ -968,6 +1003,7 @@ class CampaignJobsQuery: filter_queue: str | None = Query(default=None, max_length=1000), filter_send: str | None = Query(default=None, max_length=1000), filter_postbox: str | None = Query(default=None, max_length=1000), + filter_print: str | None = Query(default=None, max_length=1000), filter_imap: str | None = Query(default=None, max_length=1000), filter_attempts: str | None = Query(default=None, max_length=100), filter_evidence: str | None = Query(default=None, max_length=500), @@ -991,6 +1027,7 @@ class CampaignJobsQuery: "queue": filter_queue, "send": filter_send, "postbox": filter_postbox, + "print": filter_print, "imap": filter_imap, "attempts": filter_attempts, "evidence": filter_evidence, diff --git a/tests/test_mail_profile_boundary.py b/tests/test_mail_profile_boundary.py index d7df714..e42b99a 100644 --- a/tests/test_mail_profile_boundary.py +++ b/tests/test_mail_profile_boundary.py @@ -139,7 +139,7 @@ def test_new_execution_snapshot_stores_reference_and_evidence_not_transport_mate delivery=DeliveryConfig(), ) - assert payload["snapshot_version"] == "7" + assert payload["snapshot_version"] == "8" assert payload["mail_profile_id"] == "profile-1" assert "smtp" not in payload assert "imap" not in payload diff --git a/tests/test_postbox_delivery_orchestration.py b/tests/test_postbox_delivery_orchestration.py index 955a0a8..0aae8a8 100644 --- a/tests/test_postbox_delivery_orchestration.py +++ b/tests/test_postbox_delivery_orchestration.py @@ -11,6 +11,7 @@ from govoplan_campaign.backend.db.models import JobSendStatus from govoplan_campaign.backend.sending.jobs import ( SendJobResult, _MailChannelOutcome, + _PrintChannelOutcome, _final_multichannel_status, _send_claimed_multichannel_job, ) @@ -49,6 +50,71 @@ class _Session: class PostboxFallbackOrchestrationTests(unittest.TestCase): + def test_mail_unknown_never_starts_print_fallback(self) -> None: + job = SimpleNamespace(id="job-1", print_status="ready") + expected = SendJobResult( + job_id=job.id, + status=JobSendStatus.OUTCOME_UNKNOWN.value, + attempt_number=1, + ) + with ( + patch( + "govoplan_campaign.backend.sending.jobs._deliver_mail_channel", + return_value=_MailChannelOutcome(outcome_unknown=True), + ), + patch( + "govoplan_campaign.backend.sending.jobs._deliver_print_channel" + ) as deliver_print, + patch( + "govoplan_campaign.backend.sending.jobs._finalize_multichannel_job", + return_value=expected, + ), + ): + result = _send_claimed_multichannel_job( + _Session(job), # type: ignore[arg-type] + job=job, # type: ignore[arg-type] + claim_token="claim-1", + context=_context(), # type: ignore[arg-type] + channel_policy=DeliveryChannelPolicy.MAIL_THEN_PRINT, + use_rate_limit=False, + enqueue_imap_task=False, + ) + + self.assertIs(result, expected) + deliver_print.assert_not_called() + + def test_mail_preacceptance_rejection_starts_print_fallback(self) -> None: + job = SimpleNamespace(id="job-1", print_status="ready") + with ( + patch( + "govoplan_campaign.backend.sending.jobs._deliver_mail_channel", + return_value=_MailChannelOutcome(rejected_permanent=True), + ), + patch( + "govoplan_campaign.backend.sending.jobs._deliver_print_channel", + return_value=_PrintChannelOutcome(accepted=True), + ) as deliver_print, + patch( + "govoplan_campaign.backend.sending.jobs._finalize_multichannel_job", + return_value=SendJobResult( + job_id=job.id, + status=JobSendStatus.PRINT_ACCEPTED.value, + attempt_number=1, + ), + ), + ): + _send_claimed_multichannel_job( + _Session(job), # type: ignore[arg-type] + job=job, # type: ignore[arg-type] + claim_token="claim-1", + context=_context(), # type: ignore[arg-type] + channel_policy=DeliveryChannelPolicy.MAIL_THEN_PRINT, + use_rate_limit=False, + enqueue_imap_task=False, + ) + + deliver_print.assert_called_once() + def test_mail_unknown_never_starts_postbox_fallback(self) -> None: job = _job() expected = SendJobResult( @@ -385,6 +451,12 @@ def test_rejection_precedence_is_exhaustive_for_every_delivery_policy( PostboxChannelOutcome(rejected_permanent=1), JobSendStatus.PARTIALLY_ACCEPTED.value, ), + ( + DeliveryChannelPolicy.PRINT, + None, + PostboxChannelOutcome(), + JobSendStatus.PRINT_ACCEPTED.value, + ), ], ) def test_accepted_delivery_decision_table( @@ -393,7 +465,13 @@ def test_accepted_delivery_decision_table( postbox: PostboxChannelOutcome, expected: str, ) -> None: - assert _final_multichannel_status(channel_policy=policy, mail=mail, postbox=postbox) == expected + print_output = _PrintChannelOutcome(accepted=True) if policy == DeliveryChannelPolicy.PRINT else None + assert _final_multichannel_status( + channel_policy=policy, + mail=mail, + postbox=postbox, + print_output=print_output, + ) == expected if __name__ == "__main__": diff --git a/tests/test_postbox_target_resolution.py b/tests/test_postbox_target_resolution.py index 793073b..b76437d 100644 --- a/tests/test_postbox_target_resolution.py +++ b/tests/test_postbox_target_resolution.py @@ -180,6 +180,42 @@ def test_postbox_only_campaign_does_not_require_mail() -> None: } +def test_print_only_campaign_requires_templates_and_an_explicit_target_not_mail() -> None: + config = CampaignConfig.model_validate( + { + "version": "1.0", + "campaign": {"id": "campaign-print", "name": "Printed notice", "mode": "send"}, + "template": {"subject": "Notice", "text": "Printed body", "body_mode": "text"}, + "entries": { + "inline": [ + { + "id": "recipient-1", + "channel_policy": "print", + "print_target": { + "channel": "postal", + "target": "Example Street 1", + "target_key": "postal:example-street-1", + }, + } + ] + }, + "delivery": { + "channel_policy": "print", + "print": {"template_id": "template-1", "template_revision": 2}, + }, + } + ) + + available = validate_campaign_config(config, templates_available=True) + unavailable = validate_campaign_config(config, templates_available=False) + + assert "missing_mail_profile" not in {issue.code for issue in available.issues} + assert "missing_sender" not in {issue.code for issue in available.issues} + assert "print_template_missing" not in {issue.code for issue in available.issues} + assert "print_target_missing" not in {issue.code for issue in available.issues} + assert "templates_unavailable" in {issue.code for issue in unavailable.issues} + + def test_row_resolves_multiple_direct_and_field_derived_postboxes() -> None: config = _config() integration = _PostboxIntegration() diff --git a/tests/test_print_delivery_idempotency.py b/tests/test_print_delivery_idempotency.py new file mode 100644 index 0000000..3a29bbf --- /dev/null +++ b/tests/test_print_delivery_idempotency.py @@ -0,0 +1,282 @@ +from __future__ import annotations + +import hashlib +from types import SimpleNamespace +from unittest.mock import Mock, patch + +import pytest +from fastapi import HTTPException +from sqlalchemy import create_engine +from sqlalchemy.orm import Session + +from govoplan_access.backend.db.models import Account, Group, User +from govoplan_campaign.backend.db.models import ( + Campaign, + CampaignJob, + CampaignVersion, + JobPrintStatus, + PrintOutputAttempt, +) +from govoplan_campaign.backend.campaign.models import CampaignConfig +from govoplan_campaign.backend.messages.models import MessageValidationStatus +from govoplan_campaign.backend.persistence.campaigns import _resolve_built_print_outputs +from govoplan_campaign.backend.routes import versions as version_routes +from govoplan_campaign.backend.sending.jobs import _deliver_print_channel +from govoplan_core.core.change_sequence import ChangeSequenceEntry +from govoplan_core.core.templates import TemplateArtifactRef, TemplateRenderResult +from govoplan_core.db.base import Base + + +def test_print_acceptance_is_idempotent_per_frozen_artifact() -> None: + engine = create_engine("sqlite+pysqlite:///:memory:") + Base.metadata.create_all( + engine, + tables=[ + Account.__table__, + User.__table__, + Group.__table__, + Campaign.__table__, + CampaignVersion.__table__, + CampaignJob.__table__, + PrintOutputAttempt.__table__, + ChangeSequenceEntry.__table__, + ], + ) + with Session(engine) as session: + job = CampaignJob( + id="job-print-1", + tenant_id="tenant-1", + campaign_id="campaign-1", + campaign_version_id="version-1", + entry_index=1, + entry_id="entry-1", + recipient_email=None, + subject="Printable notice", + build_status="built", + validation_status="ready", + queue_status="draft", + send_status="not_queued", + print_status=JobPrintStatus.READY.value, + delivery_channel_policy="print", + resolved_attachments=[], + issues_snapshot=[], + resolved_print_output={ + "render_id": "render-1", + "output_sha256": "a" * 64, + "template_id": "template-1", + "template_revision_id": "revision-1", + "template_hash": "b" * 64, + "input_hash": "c" * 64, + "recipient_key": "recipient-1", + "item_index": 0, + "route": {"channel": "postal", "target_key": "postal:1"}, + }, + ) + session.add(job) + session.commit() + + first = _deliver_print_channel(session, job=job) + second = _deliver_print_channel(session, job=job) + + assert first.accepted is True + assert second.accepted is True + assert session.get(CampaignJob, job.id).print_attempt_count == 1 + attempts = session.query(PrintOutputAttempt).all() + assert len(attempts) == 1 + assert attempts[0].artifact_sha256 == "a" * 64 + + engine.dispose() + + +def test_print_build_uses_one_deterministic_template_render_request() -> None: + config = CampaignConfig.model_validate( + { + "version": "1.0", + "campaign": {"id": "campaign-1", "name": "Printed notice", "mode": "send"}, + "fields": [{"name": "case_number", "type": "string"}], + "template": {"subject": "Notice", "text": "Body", "body_mode": "text"}, + "entries": { + "inline": [ + { + "id": "entry-1", + "name": "Ada", + "channel_policy": "print", + "fields": {"case_number": "C-1"}, + "print_target": { + "channel": "postal", + "target": "Example Street 1", + "target_key": "postal:example-street-1", + }, + "distribution_source": { + "list_id": "list-1", + "list_revision": 3, + "expansion_hash": "expansion-1", + "recipient_key": "recipient-1", + }, + } + ] + }, + "delivery": { + "channel_policy": "print", + "print": { + "template_id": "template-1", + "template_revision": 4, + "output_format": "html", + "persist_to_files": True, + }, + }, + } + ) + draft = SimpleNamespace( + entry_index=1, + entry_id="entry-1", + delivery_channel_policy="print", + validation_status=MessageValidationStatus.READY, + ) + built = [SimpleNamespace(draft=draft)] + requests = [] + stored = {} + + class Storage: + def put_bytes(self, key, data, **_kwargs): + stored[key] = data + + class Templates: + def render(self, _session, _principal, *, request): + requests.append(request) + return TemplateRenderResult( + render_id="render-1", + template_id="template-1", + revision_id="revision-4", + revision=4, + template_hash="b" * 64, + input_hash="c" * 64, + renderer_version="templates-1", + output_format="html", + content_type="text/html", + filename="printed-notice.html", + item_count=1, + page_count=1, + output_sha256=hashlib.sha256(b"

Printed notice

").hexdigest(), + output_size_bytes=len(b"

Printed notice

"), + artifact=TemplateArtifactRef( + kind="bounded_download", + filename="printed-notice.html", + content_type="text/html", + size_bytes=len(b"

Printed notice

"), + sha256=hashlib.sha256(b"

Printed notice

").hexdigest(), + download_path="/api/v1/templates/renders/render-1/download", + ), + payload=b"

Printed notice

", + ) + + version = SimpleNamespace( + id="version-1", + campaign_id="campaign-1", + version_number=2, + ) + principal = SimpleNamespace(account_id="account-1") + storage = Storage() + with patch( + "govoplan_campaign.backend.persistence.campaigns.templates_integration", + return_value=Templates(), + ): + first = _resolve_built_print_outputs( + object(), # type: ignore[arg-type] + storage=storage, # type: ignore[arg-type] + tenant_id="tenant-1", + build_id="build-1", + version=version, # type: ignore[arg-type] + principal=principal, # type: ignore[arg-type] + config=config, + built_messages=built, + entries_by_index={1: config.entries.inline[0]}, + ) + second = _resolve_built_print_outputs( + object(), # type: ignore[arg-type] + storage=storage, # type: ignore[arg-type] + tenant_id="tenant-1", + build_id="build-1", + version=version, # type: ignore[arg-type] + principal=principal, # type: ignore[arg-type] + config=config, + built_messages=built, + entries_by_index={1: config.entries.inline[0]}, + ) + + assert requests[0].idempotency_key == requests[1].idempotency_key + assert requests[0].items[0]["case_number"] == "C-1" + assert requests[0].persist_to_files is True + assert first == second + assert first[1]["artifact"]["storage_key"] in stored + assert first[1]["artifact"]["download_path"] == ( + "/api/v1/campaigns/campaign-1/versions/version-1/print-output/download" + ) + assert first[1]["route"]["target_key"] == "postal:example-street-1" + + +class _RoutePrincipal: + tenant_id = "tenant-1" + account_id = "account-1" + user = SimpleNamespace(id="user-1") + + def has(self, scope: str) -> bool: + return scope in {"campaigns:campaign:read", "campaigns:recipient:read"} + + +def test_print_download_is_authorized_by_campaign_and_hash_checked() -> None: + payload = b"

Printable output

" + digest = hashlib.sha256(payload).hexdigest() + version = SimpleNamespace( + id="version-1", + build_summary={ + "print_output": { + "output_sha256": digest, + "template_id": "template-1", + "template_revision_id": "revision-1", + "artifact": { + "kind": "bounded_download", + "filename": "letters.html", + "content_type": "text/html", + "storage_key": "campaign-artifacts/tenant-1/letters.html", + }, + } + }, + ) + storage = SimpleNamespace(get_bytes=lambda _key: payload) + with ( + patch.object(version_routes, "_get_campaign_for_principal"), + patch.object( + version_routes, + "get_campaign_version_for_tenant", + return_value=version, + ), + patch.object(version_routes, "_object_storage", return_value=storage), + patch.object(version_routes, "audit_from_principal") as audit, + ): + response = version_routes.download_print_output( + "campaign-1", + "version-1", + session=Mock(), + principal=_RoutePrincipal(), # type: ignore[arg-type] + ) + + assert response.body == payload + assert response.headers["x-content-sha256"] == digest + audit.assert_called_once() + + +def test_print_download_requires_recipient_read_authority() -> None: + principal = _RoutePrincipal() + principal.has = lambda scope: scope == "campaigns:campaign:read" # type: ignore[method-assign] + with ( + patch.object(version_routes, "_get_campaign_for_principal"), + pytest.raises(HTTPException) as denied, + ): + version_routes.download_print_output( + "campaign-1", + "version-1", + session=Mock(), + principal=principal, # type: ignore[arg-type] + ) + assert denied.value.status_code == 403 diff --git a/tests/test_response_security.py b/tests/test_response_security.py index 929c54f..b0ee376 100644 --- a/tests/test_response_security.py +++ b/tests/test_response_security.py @@ -57,6 +57,14 @@ def _job() -> SimpleNamespace: } ], resolved_recipients={"to": [{"email": "person@example.test"}]}, + resolved_print_output={ + "output_sha256": "print-sha256", + "artifact": { + "filename": "letters.html", + "storage_key": "campaign/private/letters.html", + "download_path": "/api/v1/campaigns/campaign-1/versions/version-1/print-output/download", + }, + }, ) @@ -238,6 +246,13 @@ def test_ordinary_job_detail_and_attempts_do_not_expose_diagnostics() -> None: assert "claimed_at" not in job_payload assert "smtp_started_at" not in job_payload assert job_payload["attachments"] == [{"filename": "public.pdf"}] + assert job_payload["resolved_print_output"] == { + "output_sha256": "print-sha256", + "artifact": { + "filename": "letters.html", + "download_path": "/api/v1/campaigns/campaign-1/versions/version-1/print-output/download", + }, + } assert "claim_token" not in attempts["smtp"][0] assert "claim_token" not in attempts["imap"][0] assert "smtp_response" not in attempts["smtp"][0] diff --git a/tests/test_route_registration.py b/tests/test_route_registration.py index 4e8c3c7..a0e06d8 100644 --- a/tests/test_route_registration.py +++ b/tests/test_route_registration.py @@ -38,7 +38,7 @@ def test_campaign_router_composes_every_workflow_operation_once() -> None: actual = _operation_keys(router) assert actual == expected - assert len(actual) == 65 + assert len(actual) == 70 assert not [operation for operation, count in Counter(actual).items() if count > 1] diff --git a/webui/src/api/campaigns.ts b/webui/src/api/campaigns.ts index c73fb08..372314a 100644 --- a/webui/src/api/campaigns.ts +++ b/webui/src/api/campaigns.ts @@ -368,6 +368,39 @@ export type CampaignPostboxCatalog = { organization_units: CampaignPostboxOrganizationUnit[]; }; +export type CampaignPrintTemplateOutputProfile = { + id: string; + label: string; + output_format: "html" | "text"; + media_type: string; + channel: string; + capabilities: string[]; + page: Record; +}; + +export type CampaignPrintTemplate = { + id: string; + name: string; + slug: string; + template_type: string; + status: string; + current_revision: number; + current_revision_id: string; + published_revision_id?: string | null; + description?: string | null; + read_only: boolean; + revision?: { + revision: number; + output_profiles: CampaignPrintTemplateOutputProfile[]; + required_fields: Array<{path: string;label?: string | null;required: boolean;}>; + } | null; +}; + +export type CampaignPrintTemplatesResponse = { + available: boolean; + templates: CampaignPrintTemplate[]; +}; + export type CampaignRecipientSnapshotItem = { contact_id: string; display_name: string; @@ -494,6 +527,8 @@ export type CampaignSummary = { needs_attention?: number; sent?: number; smtp_accepted?: number; + postbox_accepted?: number; + print_accepted?: number; failed?: number; outcome_unknown?: number; not_attempted?: number; @@ -729,6 +764,7 @@ export type CampaignJobDetailResponse = { smtp?: Record[]; imap?: Record[]; postbox?: Record[]; + print?: Record[]; }; }; @@ -771,6 +807,7 @@ export type AggregateCampaignReport = { outcomes: { smtp_accepted: AggregateReportCount; postbox_accepted: AggregateReportCount; + print_accepted: AggregateReportCount; delivered: AggregateReportCount; partially_accepted: AggregateReportCount; failed: AggregateReportCount; @@ -908,6 +945,15 @@ campaignId: string) return apiFetch(settings, `/api/v1/campaigns/${campaignId}/postbox-catalog`); } +export async function listCampaignPrintTemplates( +settings: ApiSettings, +campaignId: string, +query = "") +: Promise { + const suffix = query.trim() ? `?query=${encodeURIComponent(query.trim())}` : ""; + return apiFetch(settings, `/api/v1/campaigns/${campaignId}/print-templates${suffix}`); +} + export async function snapshotCampaignRecipientAddressSource( settings: ApiSettings, campaignId: string, @@ -1242,6 +1288,20 @@ versionId?: string) ); } +export async function downloadCampaignPrintArtifact( +settings: ApiSettings, +downloadPath: string, +filename: string) +: Promise { + if (![ + "/api/v1/campaigns/", + "/api/v1/files/" + ].some((prefix) => downloadPath.startsWith(prefix))) { + throw new Error("The printable output download path is invalid."); + } + await apiDownload(settings, downloadPath, filename || "campaign-print-output.html"); +} + export async function emailCampaignReport( settings: ApiSettings, campaignId: string, diff --git a/webui/src/features/campaigns/CampaignReportPage.tsx b/webui/src/features/campaigns/CampaignReportPage.tsx index 5164d36..cddc885 100644 --- a/webui/src/features/campaigns/CampaignReportPage.tsx +++ b/webui/src/features/campaigns/CampaignReportPage.tsx @@ -45,6 +45,7 @@ const SEND_STATUS_OPTIONS: DataGridListOption[] = [ "sending", "smtp_accepted", "postbox_accepted", +"print_accepted", "delivered", "partially_accepted", "sent", @@ -54,6 +55,14 @@ const SEND_STATUS_OPTIONS: DataGridListOption[] = [ "cancelled"]. map((value) => ({ value, label: deliveryStatusLabel(value) ?? humanize(value) })); +const PRINT_STATUS_OPTIONS: DataGridListOption[] = [ +"not_requested", +"ready", +"accepted", +"failed", +"skipped"]. +map((value) => ({ value, label: deliveryStatusLabel(value) ?? humanize(value) })); + const POSTBOX_STATUS_OPTIONS: DataGridListOption[] = [ "not_requested", "pending", @@ -162,6 +171,8 @@ export default function CampaignReportPage({ settings, campaignId }: {settings: const deliveryOutcomeShortcuts: { label: string; value: string | number; shortcutId: ReportGridShortcutId }[] = [ { label: "i18n:govoplan-campaign.jobs_total.98da65bc", value: cards?.jobs_total ?? "—", shortcutId: "all" }, { label: "i18n:govoplan-campaign.smtp_accepted.e3aa7603", value: cards?.smtp_accepted ?? cards?.sent ?? 0, shortcutId: "smtp_accepted" }, + { label: "Postbox accepted", value: cards?.postbox_accepted ?? 0, shortcutId: "postbox_accepted" }, + { label: "Print accepted", value: cards?.print_accepted ?? 0, shortcutId: "print_accepted" }, { label: "i18n:govoplan-campaign.failed.09fef5d8", value: cards?.failed ?? 0, shortcutId: "failed" }, { label: "i18n:govoplan-campaign.outcome_unknown.6e929fca", value: cards?.outcome_unknown ?? 0, shortcutId: "outcome_unknown" }, { label: "i18n:govoplan-campaign.not_attempted.e1be3c69", value: cards?.not_attempted ?? 0, shortcutId: "not_attempted" }, @@ -274,7 +285,7 @@ export default function CampaignReportPage({ settings, campaignId }: {settings: }); const sendResult = asRecord(asRecord(sendResponse.result ?? sendResponse).result); const status = String(sendResult.status ?? "submitted"); - if (["smtp_accepted", "postbox_accepted", "delivered", "partially_accepted", "already_accepted"].includes(status)) accepted += 1; + if (["smtp_accepted", "postbox_accepted", "print_accepted", "delivered", "partially_accepted", "already_accepted"].includes(status)) accepted += 1; else failures.push(`${shortJobId(jobId)}: ${humanize(status)}`); } catch (err) { failures.push(`${shortJobId(jobId)}: ${err instanceof Error ? err.message : String(err)}`); @@ -377,6 +388,7 @@ export default function CampaignReportPage({ settings, campaignId }: {settings: { id: "queue", header: "i18n:govoplan-campaign.queue.d325fcd9", width: 130, sortable: true, filterable: true, columnType: "from-list", list: { options: QUEUE_STATUS_OPTIONS, display: "pill" }, render: (row) => , value: (row) => String(row.queue_status ?? "unknown") }, { id: "send", header: "Delivery", width: 160, sortable: true, filterable: true, columnType: "from-list", list: { options: SEND_STATUS_OPTIONS, display: "pill" }, render: (row) => , value: (row) => String(row.send_status ?? "unknown") }, { id: "postbox", header: "Postbox", width: 155, sortable: true, filterable: true, columnType: "from-list", list: { options: POSTBOX_STATUS_OPTIONS, display: "pill" }, render: (row) => , value: (row) => String(row.postbox_status ?? "unknown") }, + { id: "print", header: "Print", width: 135, sortable: true, filterable: true, columnType: "from-list", list: { options: PRINT_STATUS_OPTIONS, display: "pill" }, render: (row) => , value: (row) => String(row.print_status ?? "unknown") }, { id: "imap", header: "i18n:govoplan-campaign.imap.271f9ef2", width: 130, sortable: true, filterable: true, columnType: "from-list", list: { options: IMAP_STATUS_OPTIONS, display: "pill" }, render: (row) => , value: (row) => String(row.imap_status ?? "unknown") }, { id: "attempts", header: "i18n:govoplan-campaign.attempts.5a29585e", width: 105, align: "right", sortable: true, filterType: "integer", value: (row) => Number(row.attempt_count ?? 0), render: (row) => String(Number(row.attempt_count ?? 0) + Number(row.postbox_attempt_count ?? 0)) }, { @@ -591,6 +603,7 @@ export default function CampaignReportPage({ settings, campaignId }: {settings:
i18n:govoplan-campaign.smtp_state.ff372566
Postbox state
Postbox targets
{String(detail.job.postbox_target_count ?? 0)}
+
Print state
i18n:govoplan-campaign.imap_state.03b83be0
i18n:govoplan-campaign.attachments.6771ade6
{String(detail.job.matched_file_count ?? detail.job.attachment_count ?? 0)}
Message SHA-256
{String(detail.job.eml_sha256 ?? "—")}
@@ -605,6 +618,7 @@ export default function CampaignReportPage({ settings, campaignId }: {settings: + } @@ -918,12 +932,14 @@ function shortEvidenceId(value: string): string { return value.length > 16 ? `${value.slice(0, 16)}...` : value; } -function AttemptHistoryTable({ kind, rows }: {kind: "smtp" | "imap" | "postbox";rows: Record[];}) { +function AttemptHistoryTable({ kind, rows }: {kind: "smtp" | "imap" | "postbox" | "print";rows: Record[];}) { const title = kind === "smtp" ? "i18n:govoplan-campaign.smtp_attempts.eb0a9ca6" : kind === "postbox" ? "Postbox delivery attempts" - : "i18n:govoplan-campaign.imap_append_attempts.b30e980a"; + : kind === "print" + ? "Printable output attempts" + : "i18n:govoplan-campaign.imap_append_attempts.b30e980a"; if (rows.length === 0) { return (
@@ -938,6 +954,8 @@ function AttemptHistoryTable({ kind, rows }: {kind: "smtp" | "imap" | "postbox"; { id: "status", header: "i18n:govoplan-campaign.status.bae7d5be", width: 150, sortable: true, filterable: true, value: (row) => String(row.status ?? "unknown"), render: (row) => }, kind === "imap" ? { id: "folder", header: "i18n:govoplan-campaign.folder.30baa249", width: 180, sortable: true, filterable: true, value: (row) => String(row.folder ?? "—"), render: (row) => String(row.folder ?? "—") } : + kind === "print" ? + { id: "render", header: "Render", width: 220, sortable: true, filterable: true, value: (row) => String(row.render_id ?? "—"), render: (row) => String(row.render_id ?? "—") } : kind === "postbox" ? { id: "target", header: "Postbox", width: 220, sortable: true, filterable: true, value: (row) => String(row.address ?? asRecord(row.target).address ?? row.postbox_id ?? "—"), render: (row) => String(row.address ?? asRecord(row.target).address ?? row.postbox_id ?? "—") } : { id: "code", header: "i18n:govoplan-campaign.code.adac6937", width: 110, sortable: true, value: (row) => String(row.smtp_status_code ?? "—"), render: (row) => String(row.smtp_status_code ?? "—") }, @@ -1005,10 +1023,12 @@ function initialReportGridFilters(): Record { const send = statusParameters(params, "send_status", SEND_STATUS_OPTIONS); const imap = statusParameters(params, "imap_status", IMAP_STATUS_OPTIONS); const postbox = statusParameters(params, "postbox_status", POSTBOX_STATUS_OPTIONS); + const print = statusParameters(params, "print_status", PRINT_STATUS_OPTIONS); const validation = statusParameters(params, "validation_status", VALIDATION_STATUS_OPTIONS); if (send.length > 0) result.send = send; if (imap.length > 0) result.imap = imap; if (postbox.length > 0) result.postbox = postbox; + if (print.length > 0) result.print = print; if (validation.length > 0) result.validation = validation; return result; } @@ -1040,7 +1060,7 @@ function serializeInitialGridFilters(filters: Record) } function campaignJobSortColumn(value?: string): CampaignJobSortColumn { - if (value === "recipient" || value === "subject" || value === "validation" || value === "queue" || value === "send" || value === "postbox" || value === "imap" || value === "attempts" || value === "updated") { + if (value === "recipient" || value === "subject" || value === "validation" || value === "queue" || value === "send" || value === "postbox" || value === "print" || value === "imap" || value === "attempts" || value === "updated") { return value; } return "number"; diff --git a/webui/src/features/campaigns/RecipientDataPage.tsx b/webui/src/features/campaigns/RecipientDataPage.tsx index 99b783f..4185218 100644 --- a/webui/src/features/campaigns/RecipientDataPage.tsx +++ b/webui/src/features/campaigns/RecipientDataPage.tsx @@ -44,7 +44,8 @@ import AddressSourceImportDialog from "./recipients/AddressSourceImportDialog"; import DistributionListImportDialog from "./recipients/DistributionListImportDialog"; import { distributionListDrift, - materializeDistributionListExpansion + materializeDistributionListExpansion, + type DistributionRouteSelections } from "./utils/distributionListImport"; import { AddressHeaderControl, @@ -72,6 +73,7 @@ export default function RecipientDataPage({ settings, campaignId }: {settings: A const { translateText } = usePlatformLanguage(); const filesModuleInstalled = usePlatformModuleInstalled("files"); const postboxModuleInstalled = usePlatformModuleInstalled("postbox"); + const templatesModuleInstalled = usePlatformModuleInstalled("templates"); const { data, loading, error, reload, setError } = useCampaignWorkspaceData(settings, campaignId); const [importOpen, setImportOpen] = useState(false); const [addressSourceImportOpen, setAddressSourceImportOpen] = useState(false); @@ -359,9 +361,13 @@ export default function RecipientDataPage({ settings, campaignId }: {settings: A setAddressSourceImportOpen(false); } - function applyDistributionListImport(snapshot: CampaignDistributionListExpansion, mode: RecipientImportMode) { + function applyDistributionListImport( + snapshot: CampaignDistributionListExpansion, + mode: RecipientImportMode, + routeSelections: DistributionRouteSelections + ) { if (locked || !draft) return; - setDraft(materializeDistributionListExpansion(draft, snapshot, mode)); + setDraft(materializeDistributionListExpansion(draft, snapshot, mode, routeSelections)); markDirty(); setDistributionListImportOpen(false); } @@ -541,6 +547,7 @@ export default function RecipientDataPage({ settings, campaignId }: {settings: A locked, filesModuleInstalled, postboxModuleInstalled, + templatesModuleInstalled, postboxCatalog, entries: inlineEntries, fieldDefinitions, diff --git a/webui/src/features/campaigns/ReviewSendPage.tsx b/webui/src/features/campaigns/ReviewSendPage.tsx index 6a3877b..0a2f91b 100644 --- a/webui/src/features/campaigns/ReviewSendPage.tsx +++ b/webui/src/features/campaigns/ReviewSendPage.tsx @@ -12,6 +12,7 @@ import { appendSent, buildVersion, cancelCampaign, + downloadCampaignPrintArtifact, getCampaignDeliveryOptions, getCampaignJobs, getCampaignJobsDelta, @@ -151,6 +152,8 @@ export default function ReviewSendPage({ ); const validation = asRecord(version?.validation_summary); const build = asRecord(version?.build_summary); + const printOutput = asRecord(build.print_output); + const printArtifact = asRecord(printOutput.artifact); const summary = liveSummary ?? data.summary; const cards = summary?.cards; const attachmentSummary = asRecord(summary?.attachments); @@ -1449,6 +1452,34 @@ export default function ReviewSendPage({

i18n:govoplan-campaign.building_freezes_the_current_recipients_rendered.273a8170

+ {getText(printOutput, "render_id") && ( +
+
+
+

Printable output

+

+ Template revision {String(printOutput.template_revision ?? "—")} · {String(printOutput.item_count ?? 0)} recipient item(s) · {String(printOutput.page_count ?? 0)} page(s) · {String(printOutput.output_size_bytes ?? 0)} B +

+
+ {getText(printArtifact, "download_path") && ( + + )} +
+
+
Template hash
{getText(printOutput, "template_hash") || "—"}
+
Input hash
{getText(printOutput, "input_hash") || "—"}
+
Output hash
{getText(printOutput, "output_sha256") || "—"}
+
+
+ )}
+ {postboxModuleInstalled && ( + + )} + {printTarget.target && ( + + {printTarget.channel === "internal_mail" ? "Internal mail" : "Postal"}: {String(printTarget.target)} + + )}
); }, - value: (entry) => `${String(entry.channel_policy ?? "default")} ${normalizePostboxTargets(entry.postbox_targets).map((target) => target.label ?? target.postbox_id ?? target.template_id ?? "").join(" ")}` + value: (entry) => `${String(entry.channel_policy ?? "default")} ${normalizePostboxTargets(entry.postbox_targets).map((target) => target.label ?? target.postbox_id ?? target.template_id ?? "").join(" ")} ${String(asRecord(entry.print_target).target ?? "")}` } as DataGridColumn>] : []), ...(individualAttachmentBasePaths.length > 0 ? [{ id: "attachments", diff --git a/webui/src/features/campaigns/utils/distributionListImport.ts b/webui/src/features/campaigns/utils/distributionListImport.ts index 56b0edc..3c51944 100644 --- a/webui/src/features/campaigns/utils/distributionListImport.ts +++ b/webui/src/features/campaigns/utils/distributionListImport.ts @@ -61,17 +61,31 @@ export type DistributionListExpansionSnapshot = { truncated: boolean; }; +export type DistributionRouteSelection = { + primaryTargetKey: string; + fallbackTargetKey?: string; +}; + +export type DistributionRouteSelections = Record; + export function materializeDistributionListExpansion( draft: JsonRecord, expansion: DistributionListExpansionSnapshot, - mode: RecipientImportMode + mode: RecipientImportMode, + routeSelections?: DistributionRouteSelections ): JsonRecord { const currentEntries = asRecord(draft.entries); const existingEntries = asArray(currentEntries.inline).map(asRecord); const usedIds = new Set(existingEntries.map((entry) => text(entry.id)).filter(Boolean)); const fieldNames = new Set(); const importedEntries = expansion.recipients.map((recipient) => { - const entry = recipientEntry(expansion, recipient, usedIds); + const entry = recipientEntry( + expansion, + recipient, + usedIds, + routeSelections?.[recipient.recipient_key], + routeSelections !== undefined + ); Object.keys(asRecord(entry.fields)).forEach((name) => fieldNames.add(name)); return entry; }); @@ -145,26 +159,42 @@ export function distributionListDrift( function recipientEntry( expansion: DistributionListExpansionSnapshot, recipient: DistributionRecipientSnapshot, - usedIds: Set + usedIds: Set, + routeSelection?: DistributionRouteSelection, + explicitRouting = false ): JsonRecord { const usableChannels = recipient.channels.filter((candidate) => candidate.status === "usable"); - const preferredChannels = usableChannels.filter((candidate) => candidate.preferred); - const selectedRoute = preferredChannels.length === 1 - ? preferredChannels[0] - : usableChannels.length === 1 - ? usableChannels[0] - : null; - const email = selectedRoute?.channel === "email" ? selectedRoute.target.trim() : ""; + const selectedRoute = routeSelection + ? usableChannels.find((candidate) => candidate.target_key === routeSelection.primaryTargetKey) ?? null + : explicitRouting + ? null + : inferredRoute(usableChannels); + const fallbackRoute = routeSelection?.fallbackTargetKey + ? usableChannels.find((candidate) => candidate.target_key === routeSelection.fallbackTargetKey) ?? null + : null; + const emailRoute = [selectedRoute, fallbackRoute].find((candidate) => candidate?.channel === "email") ?? null; + const postboxRoute = explicitRouting + ? [selectedRoute, fallbackRoute].find((candidate) => candidate?.channel === "portal") ?? null + : null; + const printRoute = explicitRouting + ? [selectedRoute, fallbackRoute].find((candidate) => isPrintChannel(candidate?.channel)) ?? null + : null; + const email = emailRoute?.target.trim() ?? ""; const fields = stringFields(recipient.attributes); const routeReason = selectedRoute - ? (selectedRoute.preferred ? "preferred_channel" : "single_usable_channel") + ? (routeSelection ? "explicit_user_selection" : selectedRoute.preferred ? "preferred_channel" : "single_usable_channel") : usableChannels.length > 1 ? "explicit_route_required" : "no_usable_channel"; + const channelPolicy = explicitRouting + ? routePolicy(selectedRoute, fallbackRoute) + : selectedRoute?.channel === "email" + ? "mail" + : null; return { id: uniqueRecipientId(recipient.recipient_key, usedIds), - active: recipient.status === "usable" && Boolean(email), + active: recipient.status === "usable" && Boolean(selectedRoute && channelPolicy), name: recipient.display_name, email, from: [], @@ -176,6 +206,10 @@ function recipientEntry( merge_cc: true, merge_bcc: true, merge_reply_to: true, + channel_policy: channelPolicy, + postbox_targets: postboxRoute ? [postboxTarget(postboxRoute, recipient.display_name)] : [], + merge_postbox_targets: false, + print_target: printRoute ? printTarget(printRoute) : null, fields, attachments: [], combine_attachments: true, @@ -196,9 +230,7 @@ function recipientEntry( function_id: recipient.function_id ?? null, channels: recipient.channels, selected_route: selectedRoute, - fallback_routes: selectedRoute - ? usableChannels.filter((candidate) => candidate.target_key !== selectedRoute.target_key) - : [], + fallback_routes: fallbackRoute ? [fallbackRoute] : [], route_reason: routeReason, explanations: recipient.explanations, attributes: recipient.attributes, @@ -207,6 +239,54 @@ function recipientEntry( }; } +function inferredRoute(usableChannels: DistributionChannelCandidateSnapshot[]): DistributionChannelCandidateSnapshot | null { + const preferredChannels = usableChannels.filter((candidate) => candidate.preferred); + if (preferredChannels.length === 1) return preferredChannels[0]; + return usableChannels.length === 1 ? usableChannels[0] : null; +} + +function routePolicy( + primary: DistributionChannelCandidateSnapshot | null, + fallback: DistributionChannelCandidateSnapshot | null +): string | null { + if (!primary) return null; + if (primary.channel === "email") { + if (fallback?.channel === "portal") return "mail_then_postbox"; + if (isPrintChannel(fallback?.channel)) return "mail_then_print"; + return fallback ? null : "mail"; + } + if (primary.channel === "portal") { + if (fallback?.channel === "email") return "postbox_then_mail"; + if (isPrintChannel(fallback?.channel)) return "postbox_then_print"; + return fallback ? null : "postbox"; + } + return isPrintChannel(primary.channel) && !fallback ? "print" : null; +} + +function isPrintChannel(channel: string | undefined): channel is "postal" | "internal_mail" { + return channel === "postal" || channel === "internal_mail"; +} + +function postboxTarget(candidate: DistributionChannelCandidateSnapshot, label: string): JsonRecord { + return { + id: `distribution-${safeId(candidate.target_key).slice(0, 95)}`, + mode: "direct", + label: label || candidate.target, + address_key: candidate.target + }; +} + +function printTarget(candidate: DistributionChannelCandidateSnapshot): JsonRecord { + return { + channel: candidate.channel, + target: candidate.target, + target_key: candidate.target_key, + contact_point_id: candidate.contact_point_id ?? null, + locale: candidate.locale ?? null, + decision_provenance: candidate.decision_provenance ?? {} + }; +} + function distributionListImportProvenance( expansion: DistributionListExpansionSnapshot, mode: RecipientImportMode, diff --git a/webui/src/features/campaigns/utils/jobListQuery.ts b/webui/src/features/campaigns/utils/jobListQuery.ts index 4210d13..67fa956 100644 --- a/webui/src/features/campaigns/utils/jobListQuery.ts +++ b/webui/src/features/campaigns/utils/jobListQuery.ts @@ -6,6 +6,7 @@ export type CampaignJobSortColumn = | "queue" | "send" | "postbox" + | "print" | "imap" | "attempts" | "updated"; @@ -33,6 +34,7 @@ const FILTER_PARAMETERS: Record = { queue: "filter_queue", send: "filter_send", postbox: "filter_postbox", + print: "filter_print", imap: "filter_imap", attempts: "filter_attempts", evidence: "filter_evidence" diff --git a/webui/src/features/campaigns/utils/reportGridShortcuts.ts b/webui/src/features/campaigns/utils/reportGridShortcuts.ts index cad2889..ca955bf 100644 --- a/webui/src/features/campaigns/utils/reportGridShortcuts.ts +++ b/webui/src/features/campaigns/utils/reportGridShortcuts.ts @@ -8,6 +8,8 @@ export const DEFAULT_REPORT_GRID_SORT = { columnId: "number", direction: "asc" a export type ReportGridShortcutId = | "all" | "smtp_accepted" + | "postbox_accepted" + | "print_accepted" | "failed" | "outcome_unknown" | "not_attempted" @@ -20,6 +22,8 @@ export type ReportGridShortcutId = const REPORT_GRID_SHORTCUT_FILTERS: Record> = { all: {}, smtp_accepted: { send: listFilter(["smtp_accepted", "sent"]) }, + postbox_accepted: { send: listFilter(["postbox_accepted"]) }, + print_accepted: { send: listFilter(["print_accepted"]) }, failed: { send: listFilter(["failed_temporary", "failed_permanent"]) }, outcome_unknown: { send: listFilter(["outcome_unknown"]) }, not_attempted: { send: listFilter(["not_queued"]) }, diff --git a/webui/src/features/reports/AggregateReportsPage.tsx b/webui/src/features/reports/AggregateReportsPage.tsx index f6380c6..3418807 100644 --- a/webui/src/features/reports/AggregateReportsPage.tsx +++ b/webui/src/features/reports/AggregateReportsPage.tsx @@ -216,6 +216,7 @@ export default function AggregateReportsPage({ settings }: {settings: ApiSetting
+ diff --git a/webui/tests/import-utils.test.ts b/webui/tests/import-utils.test.ts index f79fc0d..3767ff6 100644 --- a/webui/tests/import-utils.test.ts +++ b/webui/tests/import-utils.test.ts @@ -237,6 +237,24 @@ assert(distributionImports[0].source_type === "distribution_list", "Campaign sto assert((asRecord(distributionImports[0].source_provenance).exclusions as unknown[]).length === 1, "excluded recipients remain in immutable import evidence"); assert(distributionListDrift(distributionEntries.imports, [{ id: "list-1", revision: 4, revision_id: "revision-4", definition_hash: "definition-hash-4" }]).length === 1, "list revision drift is detected without changing frozen recipients"); +const explicitlyRoutedDraft = materializeDistributionListExpansion( + draft, + distributionExpansion, + "replace", + { + "contact:ada": { + primaryTargetKey: "email:ada@example.org", + fallbackTargetKey: "postal:ada" + }, + "contact:postal": { primaryTargetKey: "postal:only" } + } +); +const explicitlyRoutedEntries = asRecord(explicitlyRoutedDraft.entries).inline as Record[]; +assert(explicitlyRoutedEntries[0].channel_policy === "mail_then_print", "an explicit postal fallback is frozen into the delivery policy"); +assert(asRecord(explicitlyRoutedEntries[0].print_target).target_key === "postal:ada", "the exact printable fallback target is retained"); +assert(explicitlyRoutedEntries[1].active === true && explicitlyRoutedEntries[1].channel_policy === "print", "print-only recipients remain active without a Mail address"); +assert(asRecord(explicitlyRoutedEntries[1].distribution_source).route_reason === "explicit_user_selection", "route provenance records the user decision"); + void runXlsxImportAssertions(); async function runXlsxImportAssertions(): Promise {