Compare commits
4 Commits
v0.1.10
...
01ef541917
| Author | SHA1 | Date | |
|---|---|---|---|
| 01ef541917 | |||
| d567257311 | |||
| 3075ef7f5b | |||
| 1ab4e91ffd |
@@ -44,6 +44,57 @@ If the smoke is started immediately after `docker compose up -d`, GreenMail may
|
||||
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
|
||||
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
|
||||
|
||||
## Campaign Acceptance
|
||||
|
||||
The transport smoke proves the Mail adapters. The Campaign acceptance runner
|
||||
proves the public composition: it creates an isolated temporary Core database,
|
||||
creates a Mail-owned encrypted profile through the API, materializes the
|
||||
credential-free [`greenmail-delivery`](../../examples/greenmail-delivery/campaign.json)
|
||||
fixture with only that profile reference, validates/builds it, sends the exact
|
||||
generated EML through Campaign once, appends it once, and cross-checks Campaign
|
||||
report/audit state with one unique-subject message in the GreenMail INBOX and
|
||||
Sent folders. Provider mailbox verification is not a byte-for-byte comparison
|
||||
after provider-side header or storage transformations.
|
||||
|
||||
```bash
|
||||
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
|
||||
set -a
|
||||
. ./.env
|
||||
set +a
|
||||
/mnt/DATA/git/govoplan/.venv/bin/python run_campaign_acceptance.py \
|
||||
--evidence /tmp/govoplan-campaign-greenmail-evidence.json
|
||||
```
|
||||
|
||||
The default run also uses controlled loopback protocol endpoints to prove that
|
||||
an SMTP connection loss before transmission is temporary, an explicit SMTP
|
||||
authentication rejection is permanent, and an IMAP authentication rejection
|
||||
after SMTP acceptance leaves the send accepted while the append fails. A
|
||||
second ordinary send must be rejected before another provider effect.
|
||||
|
||||
The bounded JSON contains no endpoint, account, address, credential, profile,
|
||||
campaign, version, or job identifiers. It records module versions, the fixture
|
||||
hash, normalized classifications/counts, the Mail-profile boundary, required
|
||||
audit actions, provider mailbox increments, and coverage flags. Runtime version
|
||||
declarations identify the exercised composition; they do not claim that the
|
||||
sources are clean, tagged, signed, or release-provenanced. The evidence names
|
||||
them `declared_module_versions` and keeps `source_artifact_provenance` false;
|
||||
exact commit/artifact provenance belongs to the package and release gate.
|
||||
|
||||
This runner is restricted to literal loopback IP addresses and the synchronous
|
||||
Campaign delivery mode. Hostnames such as `localhost` and every non-loopback
|
||||
address fail before profile creation, avoiding a DNS change between validation
|
||||
and connection. It is local target-like evidence, not approval of an
|
||||
institution's SMTP/IMAP service. A post-DATA connection loss
|
||||
with ambiguous SMTP outcome, an explicit temporary SMTP response, partial
|
||||
recipient refusal, and a real worker restart remain separate drills and are
|
||||
explicitly `false` in the coverage projection. Use `--success-only` only when
|
||||
testing the success journey without the local failure endpoints.
|
||||
|
||||
Starting the maintained test bed requires a working Docker CLI, Compose plugin,
|
||||
daemon/socket access, and permission to pull `greenmail/standalone:2.1.9`. The
|
||||
Campaign runner needs only the already-running loopback endpoints; it neither
|
||||
starts Docker nor claims that it did.
|
||||
|
||||
## Use With A Campaign
|
||||
|
||||
Use the same settings in a campaign mail profile:
|
||||
|
||||
1312
dev/mail-testbed/run_campaign_acceptance.py
Normal file
1312
dev/mail-testbed/run_campaign_acceptance.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -104,6 +104,19 @@ bed where possible:
|
||||
- IMAP append failure after SMTP acceptance.
|
||||
- Worker restart with queued, claimed, and sending jobs.
|
||||
|
||||
For the maintained loopback baseline, run
|
||||
`dev/mail-testbed/run_campaign_acceptance.py`. It proves the public Campaign
|
||||
path for SMTP acceptance, IMAP append, repeat-send blocking, an SMTP connection
|
||||
failure before transmission, an explicit SMTP authentication rejection, and an
|
||||
IMAP authentication rejection after SMTP acceptance. Its evidence is an
|
||||
allowlisted classification/count projection; raw provider diagnostics and
|
||||
transport/account identifiers are deliberately excluded.
|
||||
|
||||
The runner does not simulate a connection loss after SMTP DATA and does not
|
||||
restart a real worker. Keep those two checklist items open until a controlled
|
||||
fault proxy/target provider and supervised worker environment can produce the
|
||||
corresponding `outcome_unknown`, restart, retry, and reconciliation evidence.
|
||||
|
||||
## Reporting Checks
|
||||
|
||||
- Partial delivery must show accepted, failed, and unknown counts separately.
|
||||
|
||||
@@ -144,10 +144,14 @@ paths, storage keys, worker claim tokens, and raw provider diagnostics require
|
||||
the dedicated diagnostic permission and must not leak through ordinary campaign,
|
||||
version, job, or report responses.
|
||||
|
||||
The current Campaign Report Web UI additionally requires recipient-read access
|
||||
and does not yet hide every action control that the actor lacks. The server
|
||||
still authorizes each action, but an aggregate-only reader UI remains open
|
||||
work; do not promise that experience from `campaigns:report:read` alone.
|
||||
Campaign now provides a separate aggregate **Reports** surface for readers with
|
||||
`campaigns:report:read` and access to the campaign. It loads only the safe
|
||||
aggregate projections, applies small-cell suppression, and offers no recipient
|
||||
rows, drill-down, filtering, export, or delivery actions. The recipient-aware
|
||||
**Campaign Report** still requires recipient-read access and does not yet hide
|
||||
every action control that the actor lacks. The server authorizes each action,
|
||||
but permission-aware action visibility on that detailed surface remains open
|
||||
work; do not confuse it with the aggregate reader experience.
|
||||
|
||||
### Deliver and resolve outcomes
|
||||
|
||||
|
||||
@@ -10,7 +10,11 @@ scenario catalogue lives in `examples/README.md`; committed fixture files should
|
||||
be added under `examples/` only when they validate against the current campaign
|
||||
schema and are safe to run in non-production environments.
|
||||
|
||||
- simple announcement with one active recipient and no attachments
|
||||
- [`simple-announcement`](../examples/simple-announcement/campaign.json), a
|
||||
credential-free campaign with one active recipient and no attachments; its
|
||||
automated acceptance check physically blocks Mail and Files imports, denies
|
||||
network connections, and validates/builds from an unrelated temporary
|
||||
workspace
|
||||
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
|
||||
notification fields
|
||||
- campaign with global attachments and recipient-specific attachment rules
|
||||
@@ -22,7 +26,9 @@ schema and are safe to run in non-production environments.
|
||||
- campaign with blocked recipients or attachment errors that must not be sent
|
||||
- mock delivery campaign that captures SMTP and IMAP append messages in the mail
|
||||
development mailbox
|
||||
- real non-production delivery campaign against the GreenMail test bed
|
||||
- [`greenmail-delivery`](../examples/greenmail-delivery/campaign.json), a
|
||||
credential-free real-delivery Campaign materialized with a temporary
|
||||
Mail-owned profile by the loopback acceptance runner
|
||||
|
||||
## Fixture Rules
|
||||
|
||||
@@ -41,12 +47,21 @@ Before tagging a campaign release:
|
||||
|
||||
- Review `examples/README.md` and update the scenario catalogue when a release
|
||||
adds or removes delivery behavior.
|
||||
- Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'` and
|
||||
retain its isolated validate/build result as release evidence.
|
||||
- Run core module permutation tests with campaign installed both with and
|
||||
without files/mail.
|
||||
- Validate and build each maintained example campaign.
|
||||
- Run the mock delivery example when the mail development mailbox capability is
|
||||
enabled.
|
||||
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
|
||||
- Run `dev/mail-testbed/run_campaign_acceptance.py` and retain its bounded JSON
|
||||
projection. It must show one SMTP acceptance, one IMAP append, no duplicate
|
||||
effect from a repeated ordinary send, matching Campaign report/audit state,
|
||||
and no resolved transport material in Campaign JSON or its execution
|
||||
snapshot.
|
||||
- Do not interpret the local failure drills as post-DATA ambiguity or worker
|
||||
restart proof; the evidence coverage flags must keep those gaps explicit.
|
||||
- Confirm reusable mail profile selection is revalidated after campaign owner
|
||||
transfer.
|
||||
- Confirm every inline SMTP/IMAP field is rejected on import/write, omitted
|
||||
|
||||
@@ -9,7 +9,7 @@ campaign schema and do not require production data.
|
||||
|
||||
| Scenario | Required Modules | Release Check |
|
||||
| --- | --- | --- |
|
||||
| `simple-announcement` | core, access, campaigns | Validate and build one active recipient without attachments. |
|
||||
| [`simple-announcement`](simple-announcement/campaign.json) | core, access, campaigns | Validate and build one active recipient without attachments while Mail and Files are absent. |
|
||||
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
|
||||
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
|
||||
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
|
||||
@@ -19,7 +19,7 @@ campaign schema and do not require production data.
|
||||
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
|
||||
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
|
||||
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
|
||||
| `greenmail-delivery` | core, access, campaigns, mail | Send no-attachment, normal attachment, and ZIP attachment variants through `dev/mail-testbed`. |
|
||||
| [`greenmail-delivery`](greenmail-delivery/campaign.json) | core, access, audit, campaigns, mail | Run a credential-free Campaign through a Mail-owned profile, GreenMail SMTP/IMAP, report/audit checks, repeat-send protection, and bounded failure drills. |
|
||||
|
||||
## Fixture Rules
|
||||
|
||||
@@ -37,9 +37,14 @@ campaign schema and do not require production data.
|
||||
Before a release tag:
|
||||
|
||||
1. Run module permutation startup checks from core.
|
||||
2. Validate every committed example fixture against the current campaign schema.
|
||||
3. Build exact messages for each fixture.
|
||||
4. Run the mock-delivery example when the dev mailbox capability is enabled.
|
||||
5. Run `dev/mail-testbed/run_transport_smoke.py`.
|
||||
6. Execute the delivery checklist in
|
||||
2. Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'`
|
||||
from this repository. The acceptance test copies each maintained fixture to
|
||||
an unrelated temporary workspace before using Campaign's public loader,
|
||||
validator, and message builder.
|
||||
3. Validate every committed example fixture against the current campaign schema.
|
||||
4. Build exact messages for each fixture.
|
||||
5. Run the mock-delivery example when the dev mailbox capability is enabled.
|
||||
6. Run `dev/mail-testbed/run_transport_smoke.py` for low-level transport and attachment variants.
|
||||
7. Run `dev/mail-testbed/run_campaign_acceptance.py` for the Campaign journey and bounded evidence.
|
||||
8. Execute the delivery checklist in
|
||||
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.
|
||||
|
||||
88
examples/greenmail-delivery/campaign.json
Normal file
88
examples/greenmail-delivery/campaign.json
Normal file
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "greenmail-delivery",
|
||||
"name": "GreenMail delivery acceptance",
|
||||
"description": "Credential-free Campaign fixture for the local SMTP/IMAP acceptance test bed.",
|
||||
"mode": "test"
|
||||
},
|
||||
"fields": [
|
||||
{
|
||||
"name": "display_name",
|
||||
"type": "string",
|
||||
"label": "Display name",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"name": "acceptance_run",
|
||||
"type": "string",
|
||||
"label": "Acceptance run",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"server": {
|
||||
"mail_profile_id": "00000000-0000-4000-8000-000000000001"
|
||||
},
|
||||
"recipients": {
|
||||
"from": [
|
||||
{
|
||||
"email": "campaign-test@govoplan.test",
|
||||
"name": "GovOPlaN acceptance",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"allow_individual_to": true
|
||||
},
|
||||
"template": {
|
||||
"subject": "[GovOPlaN acceptance ${acceptance_run}] Campaign delivery",
|
||||
"text": "Hello ${display_name},\n\nThis is an isolated GovOPlaN Campaign SMTP/IMAP acceptance message.\n",
|
||||
"body_mode": "text"
|
||||
},
|
||||
"attachments": {
|
||||
"base_path": ".",
|
||||
"send_without_attachments_behavior": "continue",
|
||||
"global": []
|
||||
},
|
||||
"entries": {
|
||||
"inline": [
|
||||
{
|
||||
"id": "greenmail-recipient",
|
||||
"to": [
|
||||
{
|
||||
"email": "campaign-test@govoplan.test",
|
||||
"name": "GreenMail recipient",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"fields": {
|
||||
"display_name": "GreenMail recipient",
|
||||
"acceptance_run": "fixture"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"validation_policy": {
|
||||
"missing_email": "block",
|
||||
"template_error": "block"
|
||||
},
|
||||
"delivery": {
|
||||
"rate_limit": {
|
||||
"messages_per_minute": 60
|
||||
},
|
||||
"retry": {
|
||||
"max_attempts": 3,
|
||||
"backoff_seconds": [
|
||||
1,
|
||||
5,
|
||||
30
|
||||
]
|
||||
},
|
||||
"imap_append_sent": {
|
||||
"enabled": true,
|
||||
"folder": "Sent"
|
||||
}
|
||||
},
|
||||
"status_tracking": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
22
examples/greenmail-delivery/fixture.json
Normal file
22
examples/greenmail-delivery/fixture.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"scenario": "greenmail-delivery",
|
||||
"campaign_file": "campaign.json",
|
||||
"required_modules": [
|
||||
"core",
|
||||
"access",
|
||||
"audit",
|
||||
"campaigns",
|
||||
"mail"
|
||||
],
|
||||
"required_capabilities": [
|
||||
"mail.campaign_delivery"
|
||||
],
|
||||
"transport": "local GreenMail SMTP/IMAP test bed",
|
||||
"credentials": "local environment only; never copied into Campaign JSON or evidence",
|
||||
"expected": {
|
||||
"entries_count": 1,
|
||||
"built_count": 1,
|
||||
"smtp_accepted_count": 1,
|
||||
"imap_appended_count": 1
|
||||
}
|
||||
}
|
||||
54
examples/simple-announcement/campaign.json
Normal file
54
examples/simple-announcement/campaign.json
Normal file
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"campaign": {
|
||||
"id": "simple-announcement",
|
||||
"name": "Simple announcement",
|
||||
"description": "Credential-free release fixture for Campaign validation and message building.",
|
||||
"mode": "test"
|
||||
},
|
||||
"fields": [
|
||||
{
|
||||
"name": "display_name",
|
||||
"type": "string",
|
||||
"label": "Display name",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"recipients": {
|
||||
"from": [
|
||||
{
|
||||
"email": "announcements@example.test",
|
||||
"name": "GovOPlaN Example",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"allow_individual_to": true
|
||||
},
|
||||
"template": {
|
||||
"subject": "Planned service maintenance for ${display_name}",
|
||||
"text": "Hello ${display_name},\n\nThe example service will be unavailable during the announced maintenance window.\n\nThis message was built locally and was not sent.\n",
|
||||
"body_mode": "text"
|
||||
},
|
||||
"attachments": {
|
||||
"base_path": ".",
|
||||
"send_without_attachments_behavior": "continue",
|
||||
"global": []
|
||||
},
|
||||
"entries": {
|
||||
"inline": [
|
||||
{
|
||||
"id": "example-recipient",
|
||||
"to": [
|
||||
{
|
||||
"email": "recipient@example.test",
|
||||
"name": "Example Recipient",
|
||||
"type": "to"
|
||||
}
|
||||
],
|
||||
"fields": {
|
||||
"display_name": "Example Recipient"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
23
examples/simple-announcement/fixture.json
Normal file
23
examples/simple-announcement/fixture.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"id": "simple-announcement",
|
||||
"campaign_file": "campaign.json",
|
||||
"required_modules": [
|
||||
"core",
|
||||
"access",
|
||||
"campaigns"
|
||||
],
|
||||
"absent_optional_modules": [
|
||||
"files",
|
||||
"mail"
|
||||
],
|
||||
"external_effects": "forbidden",
|
||||
"expected": {
|
||||
"campaign_id": "simple-announcement",
|
||||
"entries_count": 1,
|
||||
"built_count": 1,
|
||||
"queueable_count": 1,
|
||||
"attachment_count": 0,
|
||||
"subject": "Planned service maintenance for Example Recipient"
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -364,6 +365,18 @@ def test_aggregate_report_task_never_implies_recipient_detail_or_export_authorit
|
||||
assert "export" in topic.metadata["verification"].lower()
|
||||
|
||||
|
||||
def test_handbook_distinguishes_shipped_aggregate_reports_from_detailed_report_gaps() -> None:
|
||||
handbook = " ".join(
|
||||
(
|
||||
Path(__file__).resolve().parents[1] / "docs" / "CAMPAIGN_HANDBOOK.md"
|
||||
).read_text(encoding="utf-8").lower().split()
|
||||
)
|
||||
|
||||
assert "aggregate-only reader ui remains open" not in handbook
|
||||
assert "separate aggregate **reports** surface" in handbook
|
||||
assert "permission-aware action visibility on that detailed surface remains open work" in handbook
|
||||
|
||||
|
||||
def test_static_campaign_handbook_has_unique_ids_help_contexts_and_no_planned_resend_claim() -> None:
|
||||
from govoplan_campaign.backend.manifest import get_manifest
|
||||
|
||||
|
||||
207
tests/test_example_campaigns.py
Normal file
207
tests/test_example_campaigns.py
Normal file
@@ -0,0 +1,207 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURE_ROOT = REPOSITORY_ROOT / "examples" / "simple-announcement"
|
||||
|
||||
|
||||
_ISOLATED_ACCEPTANCE_PROGRAM = r"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import importlib.abc
|
||||
import json
|
||||
import socket
|
||||
import sys
|
||||
from email import policy
|
||||
from email.parser import BytesParser
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
source_root = Path(sys.argv[1]).resolve()
|
||||
fixture_root = Path(sys.argv[2]).resolve()
|
||||
sys.path.insert(0, str(source_root))
|
||||
|
||||
|
||||
class AbsentOptionalModuleFinder(importlib.abc.MetaPathFinder):
|
||||
absent_roots = {"govoplan_files", "govoplan_mail"}
|
||||
|
||||
def find_spec(self, fullname, path=None, target=None):
|
||||
if fullname.partition(".")[0] in self.absent_roots:
|
||||
raise ModuleNotFoundError(
|
||||
f"{fullname} is intentionally absent in the Campaign fixture check",
|
||||
name=fullname,
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
sys.meta_path.insert(0, AbsentOptionalModuleFinder())
|
||||
|
||||
|
||||
def deny_network(*args, **kwargs):
|
||||
raise AssertionError("the Campaign validate/build fixture must not open a network connection")
|
||||
|
||||
|
||||
class NoNetworkSocket(socket.socket):
|
||||
def connect(self, address):
|
||||
deny_network(address)
|
||||
|
||||
def connect_ex(self, address):
|
||||
deny_network(address)
|
||||
|
||||
|
||||
socket.create_connection = deny_network
|
||||
socket.socket = NoNetworkSocket
|
||||
|
||||
from govoplan_campaign.backend.campaign import ( # noqa: E402
|
||||
load_campaign_config,
|
||||
load_campaign_json,
|
||||
validate_campaign_config,
|
||||
)
|
||||
from govoplan_campaign.backend.messages import build_campaign_messages # noqa: E402
|
||||
|
||||
|
||||
metadata = json.loads((fixture_root / "fixture.json").read_text(encoding="utf-8"))
|
||||
assert metadata["required_modules"] == ["core", "access", "campaigns"]
|
||||
assert metadata["absent_optional_modules"] == ["files", "mail"]
|
||||
assert metadata["external_effects"] == "forbidden"
|
||||
|
||||
campaign_file = fixture_root / metadata["campaign_file"]
|
||||
raw_campaign = load_campaign_json(campaign_file)
|
||||
|
||||
|
||||
def iter_keys(value):
|
||||
if isinstance(value, dict):
|
||||
for key, nested in value.items():
|
||||
yield key.casefold()
|
||||
yield from iter_keys(nested)
|
||||
elif isinstance(value, list):
|
||||
for nested in value:
|
||||
yield from iter_keys(nested)
|
||||
|
||||
|
||||
forbidden_key_fragments = ("credential", "imap", "mail_profile", "password", "secret", "smtp")
|
||||
assert not [
|
||||
key
|
||||
for key in iter_keys(raw_campaign)
|
||||
if any(fragment in key for fragment in forbidden_key_fragments)
|
||||
]
|
||||
|
||||
config = load_campaign_config(campaign_file)
|
||||
assert config.server.mail_profile_id is None
|
||||
assert not config.attachments.global_
|
||||
|
||||
validation = validate_campaign_config(config, campaign_file=campaign_file, check_files=True)
|
||||
assert validation.ok
|
||||
assert validation.error_count == 0
|
||||
assert validation.warning_count == 0
|
||||
assert validation.entries_count == metadata["expected"]["entries_count"]
|
||||
|
||||
|
||||
def build(output_name):
|
||||
return build_campaign_messages(
|
||||
config,
|
||||
campaign_file=campaign_file,
|
||||
output_dir=fixture_root.parent / output_name,
|
||||
write_eml=True,
|
||||
)
|
||||
|
||||
|
||||
first = build("build-first")
|
||||
second = build("build-second")
|
||||
expected = metadata["expected"]
|
||||
for result in (first, second):
|
||||
assert result.report.campaign_id == expected["campaign_id"]
|
||||
assert result.report.entries_count == expected["entries_count"]
|
||||
assert result.report.built_count == expected["built_count"]
|
||||
assert result.report.build_failed_count == 0
|
||||
assert result.report.queueable_count == expected["queueable_count"]
|
||||
assert len(result.built_messages) == 1
|
||||
|
||||
built = result.built_messages[0]
|
||||
assert built.mime is not None
|
||||
assert built.draft.subject == expected["subject"]
|
||||
assert built.draft.validation_status.value == "ready"
|
||||
assert built.draft.send_status.value == "draft"
|
||||
assert built.draft.imap_status.value == "not_requested"
|
||||
assert built.draft.attachment_count == expected["attachment_count"]
|
||||
assert not built.draft.attachments
|
||||
assert not built.draft.issues
|
||||
assert built.draft.from_ is not None
|
||||
assert built.draft.from_.email == "announcements@example.test"
|
||||
assert [address.email for address in built.draft.to] == ["recipient@example.test"]
|
||||
assert built.mime["Subject"] == expected["subject"]
|
||||
assert "Hello Example Recipient" in built.mime.get_content()
|
||||
assert list(built.mime.iter_attachments()) == []
|
||||
|
||||
|
||||
def normalized_eml(path_value):
|
||||
message = BytesParser(policy=policy.default).parsebytes(Path(path_value).read_bytes())
|
||||
del message["Date"]
|
||||
del message["Message-ID"]
|
||||
return message.as_bytes(policy=policy.default)
|
||||
|
||||
|
||||
first_eml = normalized_eml(first.report.messages[0].eml_path)
|
||||
second_eml = normalized_eml(second.report.messages[0].eml_path)
|
||||
assert first_eml == second_eml
|
||||
assert not any(
|
||||
name == root or name.startswith(root + ".")
|
||||
for name in sys.modules
|
||||
for root in ("govoplan_files", "govoplan_mail")
|
||||
)
|
||||
|
||||
print(json.dumps({
|
||||
"campaign_id": first.report.campaign_id,
|
||||
"built_count": first.report.built_count,
|
||||
"queueable_count": first.report.queueable_count,
|
||||
"normalized_eml_sha256": hashlib.sha256(first_eml).hexdigest(),
|
||||
}, sort_keys=True))
|
||||
"""
|
||||
|
||||
|
||||
class CampaignExampleAcceptanceTests(unittest.TestCase):
|
||||
def test_simple_announcement_validates_and_builds_without_mail_or_files(self) -> None:
|
||||
self.assertTrue((FIXTURE_ROOT / "campaign.json").is_file())
|
||||
self.assertTrue((FIXTURE_ROOT / "fixture.json").is_file())
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-campaign-acceptance-", dir="/tmp") as temp_dir:
|
||||
workspace = Path(temp_dir).resolve()
|
||||
self.assertNotIn(REPOSITORY_ROOT, workspace.parents)
|
||||
isolated_fixture = workspace / "simple-announcement"
|
||||
shutil.copytree(FIXTURE_ROOT, isolated_fixture)
|
||||
|
||||
completed = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-I",
|
||||
"-c",
|
||||
_ISOLATED_ACCEPTANCE_PROGRAM,
|
||||
str(REPOSITORY_ROOT / "src"),
|
||||
str(isolated_fixture),
|
||||
],
|
||||
cwd=workspace,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
self.assertEqual(completed.returncode, 0, completed.stderr or completed.stdout)
|
||||
evidence = json.loads(completed.stdout)
|
||||
self.assertEqual(evidence["campaign_id"], "simple-announcement")
|
||||
self.assertEqual(evidence["built_count"], 1)
|
||||
self.assertEqual(evidence["queueable_count"], 1)
|
||||
self.assertRegex(evidence["normalized_eml_sha256"], r"^[0-9a-f]{64}$")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
388
tests/test_mail_testbed_acceptance.py
Normal file
388
tests/test_mail_testbed_acceptance.py
Normal file
@@ -0,0 +1,388 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from govoplan_campaign.backend.campaign import load_campaign_config
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
|
||||
RUNNER_PATH = REPOSITORY_ROOT / "dev" / "mail-testbed" / "run_campaign_acceptance.py"
|
||||
FIXTURE_PATH = REPOSITORY_ROOT / "examples" / "greenmail-delivery" / "campaign.json"
|
||||
|
||||
|
||||
def _load_runner():
|
||||
spec = importlib.util.spec_from_file_location("govoplan_campaign_greenmail_acceptance", RUNNER_PATH)
|
||||
assert spec is not None and spec.loader is not None
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
runner = _load_runner()
|
||||
|
||||
|
||||
class _Response:
|
||||
def __init__(self, status_code: int, payload: dict[str, Any]) -> None:
|
||||
self.status_code = status_code
|
||||
self._payload = payload
|
||||
|
||||
def json(self) -> dict[str, Any]:
|
||||
return self._payload
|
||||
|
||||
|
||||
class _AcceptanceClient:
|
||||
def __init__(self) -> None:
|
||||
self.campaign_json: dict[str, Any] | None = None
|
||||
self.send_calls = 0
|
||||
|
||||
def post(self, path: str, **kwargs: Any) -> _Response:
|
||||
if path == "/api/v1/campaigns":
|
||||
self.campaign_json = kwargs["json"]["config"]
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"campaign": {"id": "campaign-internal"},
|
||||
"version": {"id": "version-internal"},
|
||||
},
|
||||
)
|
||||
if path.endswith("/validate"):
|
||||
return _Response(200, {"ok": True, "error_count": 0, "warning_count": 0})
|
||||
if path.endswith("/build"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"built_count": 1,
|
||||
"build_failed_count": 0,
|
||||
"queueable_count": 1,
|
||||
},
|
||||
)
|
||||
if path.endswith("/send-now"):
|
||||
self.send_calls += 1
|
||||
if self.send_calls == 2:
|
||||
return _Response(422, {"detail": "Already accepted"})
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"result": {
|
||||
"attempted_count": 1,
|
||||
"sent_count": 1,
|
||||
"failed_count": 0,
|
||||
"outcome_unknown_count": 0,
|
||||
"skipped_count": 0,
|
||||
"delivery_mode": "synchronous",
|
||||
"results": [{"job_id": "not-retained", "status": "smtp_accepted"}],
|
||||
}
|
||||
},
|
||||
)
|
||||
if path.endswith("/append-sent"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"result": {
|
||||
"pending_count": 1,
|
||||
"processed_count": 1,
|
||||
"appended_count": 1,
|
||||
"failed_count": 0,
|
||||
"skipped_count": 0,
|
||||
"results": [{"job_id": "not-retained", "status": "appended"}],
|
||||
}
|
||||
},
|
||||
)
|
||||
raise AssertionError(f"unexpected POST {path}")
|
||||
|
||||
def get(self, path: str, **kwargs: Any) -> _Response:
|
||||
if path.endswith("/report"):
|
||||
return _Response(
|
||||
200,
|
||||
{
|
||||
"cards": {
|
||||
"jobs_total": 1,
|
||||
"sent": 1,
|
||||
"smtp_accepted": 1,
|
||||
"failed": 0,
|
||||
"outcome_unknown": 0,
|
||||
"retryable": 0,
|
||||
"needs_attention": 0,
|
||||
"imap_appended": 1,
|
||||
"imap_failed": 0,
|
||||
},
|
||||
"status_counts": {
|
||||
"send": {"smtp_accepted": 1},
|
||||
"imap": {"appended": 1},
|
||||
},
|
||||
},
|
||||
)
|
||||
raise AssertionError(f"unexpected GET {path}")
|
||||
|
||||
|
||||
def _settings():
|
||||
return runner.TestbedSettings(
|
||||
smtp_host="127.0.0.1",
|
||||
smtp_port=3025,
|
||||
imap_host="127.0.0.1",
|
||||
imap_port=3143,
|
||||
username="campaign-test@govoplan.test",
|
||||
password="local-test-password",
|
||||
sender="campaign-test@govoplan.test",
|
||||
recipient="campaign-test@govoplan.test",
|
||||
sent_folder="Sent",
|
||||
provider_timeout_seconds=5,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["localhost", "mail.test", "192.168.1.20", "8.8.8.8"])
|
||||
def test_testbed_rejects_hostnames_and_non_loopback_addresses(host: str) -> None:
|
||||
settings = _settings()
|
||||
rejected = runner.TestbedSettings(
|
||||
smtp_host=host,
|
||||
smtp_port=settings.smtp_port,
|
||||
imap_host=settings.imap_host,
|
||||
imap_port=settings.imap_port,
|
||||
username=settings.username,
|
||||
password=settings.password,
|
||||
sender=settings.sender,
|
||||
recipient=settings.recipient,
|
||||
sent_folder=settings.sent_folder,
|
||||
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="literal loopback|restricted to the loopback"):
|
||||
rejected.assert_local_testbed()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["127.0.0.1", "127.8.9.10", "::1"])
|
||||
def test_testbed_accepts_literal_loopback_and_preserves_it_in_profile(host: str) -> None:
|
||||
settings = _settings()
|
||||
accepted = runner.TestbedSettings(
|
||||
smtp_host=host,
|
||||
smtp_port=settings.smtp_port,
|
||||
imap_host=host,
|
||||
imap_port=settings.imap_port,
|
||||
username=settings.username,
|
||||
password=settings.password,
|
||||
sender=settings.sender,
|
||||
recipient=settings.recipient,
|
||||
sent_folder=settings.sent_folder,
|
||||
provider_timeout_seconds=settings.provider_timeout_seconds,
|
||||
)
|
||||
|
||||
accepted.assert_local_testbed()
|
||||
profile = runner._profile_payload(accepted, name="Literal loopback")
|
||||
assert profile["smtp"]["host"] == host
|
||||
assert profile["imap"]["host"] == host
|
||||
|
||||
|
||||
def test_campaign_acceptance_orchestration_retains_only_profile_reference_and_safe_evidence() -> None:
|
||||
client = _AcceptanceClient()
|
||||
|
||||
def snapshot_probe(_version_id: str):
|
||||
assert client.campaign_json is not None
|
||||
return client.campaign_json, {
|
||||
"mail_profile_id": "profile-1",
|
||||
"smtp_transport_revision": "opaque-smtp-revision",
|
||||
"imap_transport_revision": "opaque-imap-revision",
|
||||
"delivery": {"imap_append_sent": {"enabled": True, "folder": "Sent"}},
|
||||
}
|
||||
|
||||
audit = {
|
||||
"campaign.created": 1,
|
||||
"campaign.validated": 1,
|
||||
"campaign.messages_built": 1,
|
||||
"campaign.sent_now": 1,
|
||||
"campaign.send_now_rejected": 1,
|
||||
"campaign.append_sent_enqueued": 1,
|
||||
}
|
||||
evidence, subject = runner.execute_campaign_scenario(
|
||||
client,
|
||||
{"Authorization": "not-retained"},
|
||||
fixture_path=FIXTURE_PATH,
|
||||
profile_id="profile-1",
|
||||
settings=_settings(),
|
||||
scenario="success",
|
||||
snapshot_probe=snapshot_probe,
|
||||
audit_probe=lambda _campaign_id, _version_id: audit,
|
||||
append_sent=True,
|
||||
repeat_send=True,
|
||||
)
|
||||
evidence["provider_verification"] = {
|
||||
"inbox_increment": 1,
|
||||
"sent_increment": 1,
|
||||
"unique_subject_matches_in_inbox": 1,
|
||||
"unique_subject_matches_in_sent": 1,
|
||||
}
|
||||
|
||||
runner._assert_success_evidence(evidence)
|
||||
runner._assert_evidence_safe(
|
||||
{
|
||||
"schema_version": runner.EVIDENCE_SCHEMA,
|
||||
"coverage": {
|
||||
"smtp_acceptance": True,
|
||||
"partial_envelope_refusal": False,
|
||||
"post_data_connection_loss_outcome_unknown": False,
|
||||
"source_artifact_provenance": False,
|
||||
"worker_restart_interruption": False,
|
||||
},
|
||||
"success": evidence,
|
||||
},
|
||||
settings=_settings(),
|
||||
)
|
||||
assert subject.startswith("[GovOPlaN acceptance ")
|
||||
assert client.send_calls == 2
|
||||
assert client.campaign_json is not None
|
||||
assert client.campaign_json["server"] == {"mail_profile_id": "profile-1"}
|
||||
assert "credentials" not in json.dumps(client.campaign_json).casefold()
|
||||
serialized = json.dumps(evidence, sort_keys=True)
|
||||
assert "not-retained" not in serialized
|
||||
assert "local-test-password" not in serialized
|
||||
|
||||
|
||||
def test_campaign_boundary_rejects_resolved_transport_material() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="forbidden transport material"):
|
||||
runner.assert_campaign_boundary(
|
||||
{"server": {"mail_profile_id": "profile-1"}},
|
||||
{
|
||||
"mail_profile_id": "profile-1",
|
||||
"smtp_transport_revision": "smtp-revision",
|
||||
"imap_transport_revision": "imap-revision",
|
||||
"smtp": {"host": "should-not-be-here"},
|
||||
},
|
||||
profile_id="profile-1",
|
||||
)
|
||||
|
||||
|
||||
def test_success_projection_fails_closed_on_inconsistent_campaign_report() -> None:
|
||||
evidence = {
|
||||
"send": {
|
||||
"attempted_count": 1,
|
||||
"sent_count": 1,
|
||||
"failed_count": 0,
|
||||
"outcome_unknown_count": 0,
|
||||
"skipped_count": 0,
|
||||
"delivery_mode": "synchronous",
|
||||
"statuses": {"smtp_accepted": 1},
|
||||
},
|
||||
"append_sent": {
|
||||
"pending_count": 1,
|
||||
"processed_count": 1,
|
||||
"appended_count": 1,
|
||||
"failed_count": 0,
|
||||
"skipped_count": 0,
|
||||
"statuses": {"appended": 1},
|
||||
},
|
||||
"report": {
|
||||
"cards": {
|
||||
"jobs_total": 1,
|
||||
"sent": 0,
|
||||
"smtp_accepted": 0,
|
||||
"failed": 0,
|
||||
"outcome_unknown": 0,
|
||||
"needs_attention": 0,
|
||||
"imap_appended": 0,
|
||||
"imap_failed": 0,
|
||||
},
|
||||
"send_status_counts": {},
|
||||
"imap_status_counts": {},
|
||||
},
|
||||
"provider_verification": {
|
||||
"inbox_increment": 1,
|
||||
"sent_increment": 1,
|
||||
"unique_subject_matches_in_inbox": 1,
|
||||
"unique_subject_matches_in_sent": 1,
|
||||
},
|
||||
"campaign_mail_boundary": {
|
||||
"profile_reference_only": True,
|
||||
"smtp_revision_frozen": True,
|
||||
"imap_revision_frozen": True,
|
||||
"resolved_transport_material_present": False,
|
||||
},
|
||||
}
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="report does not agree"):
|
||||
runner._assert_success_evidence(evidence)
|
||||
|
||||
|
||||
def test_evidence_projection_rejects_unknown_status_keys() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="unsupported status"):
|
||||
runner._send_evidence(
|
||||
{
|
||||
"delivery_mode": "synchronous",
|
||||
"results": [{"status": "provider diagnostic: recipient@example.test"}],
|
||||
}
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="synchronous delivery mode"):
|
||||
runner._send_evidence(
|
||||
{
|
||||
"delivery_mode": "provider diagnostic: recipient@example.test",
|
||||
"results": [],
|
||||
}
|
||||
)
|
||||
|
||||
with pytest.raises(runner.AcceptanceError, match="unsupported"):
|
||||
runner._report_evidence(
|
||||
{
|
||||
"cards": {},
|
||||
"status_counts": {
|
||||
"send": {"smtp_accepted": 1, "provider-secret": 1},
|
||||
"imap": {},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_fixture_contains_no_transport_credentials() -> None:
|
||||
raw = json.loads(FIXTURE_PATH.read_text(encoding="utf-8"))
|
||||
runner._assert_no_forbidden_campaign_keys(raw)
|
||||
assert raw["server"] == {"mail_profile_id": "00000000-0000-4000-8000-000000000001"}
|
||||
config = load_campaign_config(FIXTURE_PATH)
|
||||
assert config.server.mail_profile_id == "00000000-0000-4000-8000-000000000001"
|
||||
|
||||
|
||||
def test_fixture_composition_versions_are_complete_and_exact() -> None:
|
||||
versions = {
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"audit": "0.1.8",
|
||||
"campaigns": "0.1.10",
|
||||
"mail": "0.1.10",
|
||||
"files": "0.1.9",
|
||||
}
|
||||
|
||||
assert runner.required_composition_versions(FIXTURE_PATH, versions) == {
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"audit": "0.1.8",
|
||||
"campaigns": "0.1.10",
|
||||
"mail": "0.1.10",
|
||||
}
|
||||
|
||||
|
||||
def test_fixture_composition_fails_closed_when_a_required_version_is_missing() -> None:
|
||||
with pytest.raises(runner.AcceptanceError, match="versions are unavailable"):
|
||||
runner.required_composition_versions(
|
||||
FIXTURE_PATH,
|
||||
{
|
||||
"core": "0.1.13",
|
||||
"access": "0.1.11",
|
||||
"campaigns": "0.1.10",
|
||||
"mail": "0.1.10",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_testbed_documentation_distinguishes_proven_and_open_failure_drills() -> None:
|
||||
testbed = (REPOSITORY_ROOT / "dev" / "mail-testbed" / "README.md").read_text(encoding="utf-8")
|
||||
runbook = (REPOSITORY_ROOT / "docs" / "CAMPAIGN_DELIVERY_RUNBOOK.md").read_text(encoding="utf-8")
|
||||
|
||||
assert "second ordinary send must be rejected before another provider effect" in testbed
|
||||
assert "post-DATA connection loss" in testbed
|
||||
assert "real worker restart" in testbed
|
||||
assert "does not simulate a connection loss after SMTP DATA" in runbook
|
||||
assert "raw provider diagnostics" in runbook
|
||||
Reference in New Issue
Block a user