inital commit
This commit is contained in:
68
.env.example
Normal file
68
.env.example
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
# Copy to .env and adjust.
|
||||||
|
COMPOSE_PROJECT_NAME=multimailer
|
||||||
|
APP_ENV=dev
|
||||||
|
|
||||||
|
# API
|
||||||
|
APP_HOST=0.0.0.0
|
||||||
|
APP_PORT=8000
|
||||||
|
API_PUBLISHED_PORT=8000
|
||||||
|
|
||||||
|
# Local Traefik
|
||||||
|
TRAEFIK_WEB_PORT=8080
|
||||||
|
TRAEFIK_DASHBOARD_PORT=8081
|
||||||
|
TRAEFIK_DASHBOARD_INSECURE=true
|
||||||
|
TRAEFIK_ROUTER_NAME=multimailer
|
||||||
|
TRAEFIK_SERVICE_NAME=multimailer
|
||||||
|
TRAEFIK_RULE=PathPrefix(`/`)
|
||||||
|
|
||||||
|
# Database
|
||||||
|
DATABASE_URL=postgresql+psycopg://multimailer:multimailer@postgres:5432/multimailer
|
||||||
|
POSTGRES_DB=multimailer
|
||||||
|
POSTGRES_USER=multimailer
|
||||||
|
POSTGRES_PASSWORD=multimailer
|
||||||
|
|
||||||
|
# Redis
|
||||||
|
REDIS_URL=redis://redis:6379/0
|
||||||
|
|
||||||
|
# Garage / S3-compatible object storage
|
||||||
|
S3_ENDPOINT_URL=http://garage:3900
|
||||||
|
S3_REGION=garage
|
||||||
|
S3_BUCKET=attachments
|
||||||
|
S3_ACCESS_KEY_ID=GKmultimailerdev0000000000000000
|
||||||
|
S3_SECRET_ACCESS_KEY=multimailer-dev-secret-change-me
|
||||||
|
GARAGE_S3_PORT=3900
|
||||||
|
GARAGE_ADMIN_PORT=3903
|
||||||
|
|
||||||
|
# Crypto: required before storing real SMTP/IMAP credentials.
|
||||||
|
# Generate:
|
||||||
|
# python -c "import os,base64; print(base64.b64encode(os.urandom(32)).decode())"
|
||||||
|
MASTER_KEY_B64=
|
||||||
|
|
||||||
|
# Limits
|
||||||
|
MAX_UPLOAD_MB=50
|
||||||
|
MAX_ATTACHMENTS_PER_JOB=50
|
||||||
|
DEFAULT_SEND_RATE_PER_MIN=30
|
||||||
|
DEFAULT_CONCURRENCY=2
|
||||||
|
|
||||||
|
# Worker tuning
|
||||||
|
CELERY_QUEUES=send_email,append_sent,default
|
||||||
|
CELERY_CONCURRENCY=4
|
||||||
|
CELERY_PREFETCH_MULTIPLIER=1
|
||||||
|
CELERY_MAX_TASKS_PER_CHILD=200
|
||||||
|
CELERY_LOGLEVEL=INFO
|
||||||
|
|
||||||
|
# Existing Traefik/proxy network example
|
||||||
|
EXTERNAL_PROXY_NETWORK=proxy
|
||||||
|
TRAEFIK_ENTRYPOINT=websecure
|
||||||
|
|
||||||
|
# Web UI
|
||||||
|
WEBUI_PUBLISHED_PORT=5173
|
||||||
|
VITE_API_BASE_URL=/api/v1
|
||||||
|
# For local Vite development outside Docker:
|
||||||
|
# VITE_DEV_API_PROXY_TARGET=http://127.0.0.1:8000
|
||||||
|
CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173,http://localhost:8080
|
||||||
|
MULTIMAILER_HOST=multimailer.localhost
|
||||||
|
TRAEFIK_API_ROUTER_NAME=multimailer-api
|
||||||
|
TRAEFIK_API_SERVICE_NAME=multimailer-api
|
||||||
|
TRAEFIK_WEBUI_ROUTER_NAME=multimailer-webui
|
||||||
|
TRAEFIK_WEBUI_SERVICE_NAME=multimailer-webui
|
||||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -174,3 +174,5 @@ cython_debug/
|
|||||||
# PyPI configuration file
|
# PyPI configuration file
|
||||||
.pypirc
|
.pypirc
|
||||||
|
|
||||||
|
.vscode
|
||||||
|
.vscode/**
|
||||||
57
JAVA_PORT_NOTES.md
Normal file
57
JAVA_PORT_NOTES.md
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
# Java MultiMailer port notes
|
||||||
|
|
||||||
|
This skeleton now contains a first Python port of the important Java domain logic:
|
||||||
|
|
||||||
|
- `Field`, `FieldDescription`, `FieldConfiguration`, `FieldContents`
|
||||||
|
- `MailTemplate` with `${global::field}` / `${local::field}` replacement
|
||||||
|
- `Recipient`, `RecipientList`
|
||||||
|
- `MailServerSettings`
|
||||||
|
- `MailAttachmentConfig`
|
||||||
|
- `MailCampaign`, `MailEntry`, `MailQueue`
|
||||||
|
- attachment glob matching
|
||||||
|
- encrypted ZIP creation with `pyzipper`
|
||||||
|
- MIME assembly using Python's `email.message.EmailMessage`
|
||||||
|
- basic SMTP queue sending using `smtplib`
|
||||||
|
|
||||||
|
The Java Swing UI was not ported, because the planned UI is React.
|
||||||
|
The SimpleJavaMail dependency was replaced by stdlib `email` / `smtplib` equivalents.
|
||||||
|
|
||||||
|
Example port of the provided `MultiMailerSettings.java.example`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.examples.rechnungslegung_2026_05
|
||||||
|
```
|
||||||
|
|
||||||
|
Current limitations:
|
||||||
|
|
||||||
|
- IMAP append-to-Sent is not implemented yet.
|
||||||
|
- Server-side persisted campaign/job models are not implemented yet.
|
||||||
|
- Attachment storage still uses local filesystem paths in the Java-compatible domain layer; Garage object storage integration is the next layer.
|
||||||
|
- ZIP password encryption uses AES via `pyzipper`, not the Java zip4j implementation.
|
||||||
|
- The Java `getNotSentFiles` behavior is approximated and should be revisited once we define the new campaign/job data model.
|
||||||
|
|
||||||
|
## Added after persistence foundation
|
||||||
|
|
||||||
|
- Queue transition for built jobs.
|
||||||
|
- Campaign-level queue/pause/resume/cancel API endpoints.
|
||||||
|
- Direct CLI queued-job processor for development without Celery.
|
||||||
|
- Celery `multimailer.send_email` task now sends DB-backed jobs.
|
||||||
|
- Per-campaign Redis-backed rate limiting.
|
||||||
|
- Send attempts persisted in `send_attempts`.
|
||||||
|
- IMAP status kept independent from SMTP status; actual IMAP APPEND remains next.
|
||||||
|
|
||||||
|
## IMAP append implementation
|
||||||
|
|
||||||
|
The Python port now implements the Java-era "save to sent folder" concept as a separate post-send worker action:
|
||||||
|
|
||||||
|
- SMTP success updates `send_status=sent`.
|
||||||
|
- If configured, the job receives `imap_status=pending`.
|
||||||
|
- A separate `append_sent` task or CLI command appends the exact generated EML to the IMAP Sent folder.
|
||||||
|
- IMAP failures do not undo SMTP success.
|
||||||
|
|
||||||
|
This preserves the important distinction between delivery and mailbox archival.
|
||||||
|
|
||||||
|
## Reporting/dashboard milestone
|
||||||
|
|
||||||
|
Added campaign reporting helpers and API endpoints. This is the backend bridge toward the first web UI: the report contains card-friendly counts, grouped statuses, attachment and issue summaries, delivery/rate-limit estimates, recent failures, and optional per-job rows. No new DB schema is required; the report is derived from persisted campaigns/jobs/attempts.
|
||||||
924
README.md
924
README.md
@@ -1,2 +1,924 @@
|
|||||||
# multimailer
|
# MultiMailer skeleton
|
||||||
|
|
||||||
|
Local-first bulk mailer backend skeleton:
|
||||||
|
|
||||||
|
- FastAPI API
|
||||||
|
- Celery worker(s)
|
||||||
|
- Redis queue
|
||||||
|
- Postgres database
|
||||||
|
- Garage S3-compatible object storage
|
||||||
|
- Optional local Traefik
|
||||||
|
- Modular Compose layout so you can reuse existing infrastructure
|
||||||
|
|
||||||
|
## Quick start: full local development stack
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env
|
||||||
|
./scripts/up-local.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Equivalent explicit command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.yml -f compose.local.yml up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Health check:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/health
|
||||||
|
# or direct API port:
|
||||||
|
curl http://localhost:8000/health
|
||||||
|
```
|
||||||
|
|
||||||
|
Garage S3 local endpoint:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://localhost:3900
|
||||||
|
```
|
||||||
|
|
||||||
|
## App-only mode with existing Postgres, Redis, Garage, Traefik
|
||||||
|
|
||||||
|
1. Copy `.env.example` to `.env`
|
||||||
|
2. Point these values to your existing services:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
DATABASE_URL=postgresql+psycopg://user:password@existing-postgres:5432/multimailer
|
||||||
|
REDIS_URL=redis://existing-redis:6379/0
|
||||||
|
S3_ENDPOINT_URL=https://garage.example.org
|
||||||
|
S3_REGION=garage
|
||||||
|
S3_BUCKET=attachments
|
||||||
|
S3_ACCESS_KEY_ID=...
|
||||||
|
S3_SECRET_ACCESS_KEY=...
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Start app services only:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.yml up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Or use the example external Traefik override:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.yml -f compose.external.example.yml up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scale within one worker container
|
||||||
|
|
||||||
|
For small PDFs zipped per recipient, start with long-lived Celery worker processes:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
CELERY_CONCURRENCY=4
|
||||||
|
CELERY_PREFETCH_MULTIPLIER=1
|
||||||
|
CELERY_MAX_TASKS_PER_CHILD=200
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scale with more containers
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f compose.yml -f compose.local.yml up --build --scale worker=3 --scale api=2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Set `MASTER_KEY_B64` before storing real SMTP/IMAP credentials.
|
||||||
|
- Garage 2.3+ can create the local default bucket and key from environment variables when started with `--single-node --default-bucket`.
|
||||||
|
- The local Garage config uses `replication_factor = 1`; this is fine for development, not a production storage strategy.
|
||||||
|
|
||||||
|
## Campaign JSON
|
||||||
|
|
||||||
|
The project now includes a first schema and example campaign file:
|
||||||
|
|
||||||
|
```text
|
||||||
|
server/app/mailer/schemas/campaign.schema.json
|
||||||
|
server/app/mailer/examples/campaign.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The schema preserves the Java object model concepts: global recipients, individual recipients, combine flags, campaign-level `attachments.base_path`, global and entry-specific attachment configs, validation behavior for missing/ambiguous attachments, rate limits, and optional IMAP append-to-Sent handling.
|
||||||
|
|
||||||
|
## Attachment resolver dry run
|
||||||
|
|
||||||
|
The attachment resolver loads the campaign JSON, normalizes inline or external entries, renders `${global::...}` and `${local::...}` placeholders in `base_dir` and `file_filter`, and reports missing or ambiguous attachment matches without sending mail.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.resolve_attachments \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
Machine-readable output for the future web UI:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.resolve_attachments \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build message drafts without sending
|
||||||
|
|
||||||
|
After validating the campaign and resolving attachments, build reviewable message drafts:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.build_messages \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--output-dir ./build/messages \
|
||||||
|
--verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
This writes `.eml` files for review and prints per-entry status:
|
||||||
|
|
||||||
|
- `ready`: queueable
|
||||||
|
- `warning`: queueable, but with non-blocking issues
|
||||||
|
- `needs_review`: user decision required before queueing
|
||||||
|
- `blocked`: must not be queued
|
||||||
|
- `excluded`: excluded by policy, but still reported
|
||||||
|
- `inactive`: inactive entry
|
||||||
|
|
||||||
|
Machine-readable report:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.build_messages \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
## SMTP test send
|
||||||
|
|
||||||
|
After building messages, you can send one generated message to a test recipient without changing campaign/job status:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.send_test_message \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--to your.test.address@example.org \
|
||||||
|
--dry-run \
|
||||||
|
--write-eml ./build/test-message.eml
|
||||||
|
```
|
||||||
|
|
||||||
|
When ready to actually connect to SMTP, remove `--dry-run`. SMTP settings are read from `campaign.json`, but can be overridden without editing the file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
MULTIMAILER_SMTP_PASSWORD='...' \
|
||||||
|
python -m app.mailer.commands.send_test_message \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--to your.test.address@example.org \
|
||||||
|
--smtp-host smtp.example.org \
|
||||||
|
--smtp-port 587 \
|
||||||
|
--smtp-security starttls \
|
||||||
|
--smtp-username user@example.org
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported override environment variables:
|
||||||
|
|
||||||
|
```text
|
||||||
|
MULTIMAILER_SMTP_HOST
|
||||||
|
MULTIMAILER_SMTP_PORT
|
||||||
|
MULTIMAILER_SMTP_SECURITY # plain | tls | starttls
|
||||||
|
MULTIMAILER_SMTP_USERNAME
|
||||||
|
MULTIMAILER_SMTP_PASSWORD
|
||||||
|
```
|
||||||
|
|
||||||
|
The test-send command always replaces the real To/Cc/Bcc recipients with the test recipient before sending.
|
||||||
|
|
||||||
|
## Persistence/API-key milestone
|
||||||
|
|
||||||
|
This version adds the first persistent backend layer:
|
||||||
|
|
||||||
|
- SQLAlchemy models for tenants, users, roles, API keys, campaigns, campaign versions, jobs, issues, attachments, send attempts, IMAP append attempts and audit log entries.
|
||||||
|
- Alembic setup under `server/alembic`.
|
||||||
|
- Development database bootstrap command.
|
||||||
|
- API-key authentication for campaign API endpoints.
|
||||||
|
- Campaign JSON import into versioned DB snapshots.
|
||||||
|
- Campaign validation/build results persisted as campaign jobs.
|
||||||
|
|
||||||
|
### Local DB quick start without Docker
|
||||||
|
|
||||||
|
From `server/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# SQLite is the default when DATABASE_URL is not set.
|
||||||
|
python -m app.mailer.commands.init_db --with-dev-data
|
||||||
|
|
||||||
|
python -m app.mailer.commands.import_campaign \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--validate \
|
||||||
|
--build
|
||||||
|
|
||||||
|
python -m app.mailer.commands.list_db_campaigns \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
### API quick smoke test
|
||||||
|
|
||||||
|
Start the API:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvicorn app.main:app --reload --host 127.0.0.1 --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
The dev bootstrap runs automatically in `APP_ENV=dev` and creates this API key unless disabled:
|
||||||
|
|
||||||
|
```text
|
||||||
|
dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
List campaigns:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -H 'X-API-Key: dev-multimailer-api-key' \
|
||||||
|
http://127.0.0.1:8000/api/v1/campaigns
|
||||||
|
```
|
||||||
|
|
||||||
|
Create a campaign from JSON:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python - <<'PY'
|
||||||
|
import json, requests
|
||||||
|
config = json.load(open('app/mailer/examples/campaign.json'))
|
||||||
|
response = requests.post(
|
||||||
|
'http://127.0.0.1:8000/api/v1/campaigns',
|
||||||
|
headers={'X-API-Key': 'dev-multimailer-api-key'},
|
||||||
|
json={'config': config, 'source_filename': 'app/mailer/examples/campaign.json'},
|
||||||
|
)
|
||||||
|
print(response.status_code)
|
||||||
|
print(response.text)
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
|
||||||
|
### API endpoints added
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /api/v1/campaigns
|
||||||
|
POST /api/v1/campaigns
|
||||||
|
GET /api/v1/campaigns/{campaign_id}
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/versions
|
||||||
|
POST /api/v1/campaigns/versions/{version_id}/validate
|
||||||
|
POST /api/v1/campaigns/versions/{version_id}/build
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/jobs
|
||||||
|
POST /api/v1/admin/api-keys
|
||||||
|
```
|
||||||
|
|
||||||
|
All campaign/admin endpoints require an API key via either:
|
||||||
|
|
||||||
|
```text
|
||||||
|
X-API-Key: ...
|
||||||
|
Authorization: Bearer ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Note about local file paths
|
||||||
|
|
||||||
|
When importing a file-based campaign JSON, pass `source_filename` or use the CLI importer. Relative paths inside the campaign are normalized to absolute paths at import time so the DB snapshot can still find CSV files, templates and attachment folders later.
|
||||||
|
|
||||||
|
## Queueing and sending step
|
||||||
|
|
||||||
|
This version adds the first real bulk-delivery transition:
|
||||||
|
|
||||||
|
```text
|
||||||
|
built campaign jobs -> queued jobs -> SMTP worker/direct sender -> send attempts
|
||||||
|
```
|
||||||
|
|
||||||
|
### Queue jobs without starting Celery
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.queue_campaign \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--no-celery
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `--dry-run` to see what would be queued without modifying the database.
|
||||||
|
|
||||||
|
### Process queued jobs directly from CLI
|
||||||
|
|
||||||
|
This is useful before the web UI and before running a Celery worker:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.send_queued_jobs \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Remove `--dry-run` only after SMTP credentials in the campaign JSON point to a real server.
|
||||||
|
|
||||||
|
### Queue jobs and send through Celery
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.queue_campaign \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
|
||||||
|
celery -A app.celery_app.celery worker \
|
||||||
|
--loglevel=INFO \
|
||||||
|
--queues=send_email,append_sent,default \
|
||||||
|
--concurrency=1 \
|
||||||
|
--prefetch-multiplier=1
|
||||||
|
```
|
||||||
|
|
||||||
|
### API endpoints added
|
||||||
|
|
||||||
|
```text
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/queue
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/pause
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/resume
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/cancel
|
||||||
|
```
|
||||||
|
|
||||||
|
Queueing still respects the review gate: only `ready` and, by default, `warning` jobs are queued. `needs_review`, `blocked`, `excluded`, and `inactive` jobs are not queued.
|
||||||
|
|
||||||
|
### Rate limiting
|
||||||
|
|
||||||
|
SMTP sending now applies campaign-level rate limiting from the campaign JSON:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The worker uses Redis for distributed throttling when available. If Redis is unavailable in local development, the sender falls back gracefully, but production should use Redis.
|
||||||
|
|
||||||
|
### IMAP status
|
||||||
|
|
||||||
|
SMTP success now keeps IMAP state separate:
|
||||||
|
|
||||||
|
```text
|
||||||
|
send_status = sent
|
||||||
|
imap_status = pending | not_requested
|
||||||
|
```
|
||||||
|
|
||||||
|
Actual IMAP APPEND is still the next implementation step.
|
||||||
|
|
||||||
|
## IMAP append-to-Sent
|
||||||
|
|
||||||
|
SMTP sending and IMAP Sent-folder append are intentionally separate states. A job can be `send_status=sent` while `imap_status=failed`; the email was still sent, but saving the copy to Sent needs attention.
|
||||||
|
|
||||||
|
When Celery is used, successful SMTP delivery automatically enqueues an `append_sent` task if the campaign has:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"delivery": {
|
||||||
|
"imap_append_sent": { "enabled": true, "folder": "auto" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
and `server.imap.enabled=true`.
|
||||||
|
|
||||||
|
Direct CLI sending can append immediately in the same process:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.send_queued_jobs \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--append-sent
|
||||||
|
```
|
||||||
|
|
||||||
|
Or process pending/failed appends separately:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.append_pending_sent \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
Dry-run IMAP append check:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.append_pending_sent \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
API endpoint:
|
||||||
|
|
||||||
|
```text
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/append-sent
|
||||||
|
```
|
||||||
|
|
||||||
|
with body:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"enqueue_celery": true,
|
||||||
|
"dry_run": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Folder resolution order:
|
||||||
|
|
||||||
|
1. `delivery.imap_append_sent.folder` if not `auto`
|
||||||
|
2. `server.imap.sent_folder` if not `auto`
|
||||||
|
3. IMAP `LIST` discovery using `\\Sent` special-use flags
|
||||||
|
4. common fallback names such as `Sent`, `Sent Items`, `Gesendet`
|
||||||
|
|
||||||
|
## Campaign reporting / dashboard data
|
||||||
|
|
||||||
|
This version adds dashboard/report payloads that are intended to feed the first web interface.
|
||||||
|
They summarize persisted jobs, validation issues, attachment resolution, sending attempts, IMAP append state, and rate-limit estimates.
|
||||||
|
|
||||||
|
CLI report:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
Machine-readable report:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--json \
|
||||||
|
--include-jobs
|
||||||
|
```
|
||||||
|
|
||||||
|
Per-job CSV export:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--jobs-csv ./build/campaign-jobs.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
API endpoints:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/summary
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/report
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/report/jobs.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
These endpoints are API-key protected with the `campaign:read` scope and return data shaped for a web dashboard: top-level cards, status counters, attachment summary, issue summary, send/IMAP attempts, recent failures, and optional per-job rows.
|
||||||
|
|
||||||
|
## Audit log and report emailing
|
||||||
|
|
||||||
|
This version adds a tenant-scoped audit log and a first report-emailing flow.
|
||||||
|
|
||||||
|
Audit log entries are written for API key creation, campaign import/create, validation, message building, queue actions, IMAP append enqueueing and report email sending/dry runs.
|
||||||
|
|
||||||
|
List audit entries from the CLI:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd server
|
||||||
|
python -m app.mailer.commands.audit_log \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--limit 50
|
||||||
|
```
|
||||||
|
|
||||||
|
Machine-readable audit output:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.audit_log \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
API endpoint:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /api/v1/audit
|
||||||
|
```
|
||||||
|
|
||||||
|
Optional filters:
|
||||||
|
|
||||||
|
```text
|
||||||
|
?action=campaign.queued&object_type=campaign&object_id=<campaign-db-uuid>&limit=100&offset=0
|
||||||
|
```
|
||||||
|
|
||||||
|
Email a campaign report:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.email_campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--to recipient@example.org \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Actual sending uses the campaign's `server.smtp` configuration and the configured campaign sender. Remove `--dry-run` only once SMTP is configured.
|
||||||
|
|
||||||
|
Attach the full JSON report as well as the default per-job CSV:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.email_campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--to recipient@example.org \
|
||||||
|
--attach-report-json
|
||||||
|
```
|
||||||
|
|
||||||
|
API endpoint:
|
||||||
|
|
||||||
|
```text
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/report/email
|
||||||
|
```
|
||||||
|
|
||||||
|
Body:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"to": ["recipient@example.org"],
|
||||||
|
"include_jobs": false,
|
||||||
|
"attach_jobs_csv": true,
|
||||||
|
"attach_report_json": false,
|
||||||
|
"dry_run": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Required API scopes:
|
||||||
|
|
||||||
|
```text
|
||||||
|
audit:read
|
||||||
|
reports:send
|
||||||
|
```
|
||||||
|
|
||||||
|
## Supported CLI commands
|
||||||
|
|
||||||
|
All commands are run from `server/`.
|
||||||
|
|
||||||
|
### Database and API-key setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.init_db --with-dev-data
|
||||||
|
```
|
||||||
|
|
||||||
|
Creates local tables and development data, including the default development API key if enabled.
|
||||||
|
|
||||||
|
### Campaign JSON validation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.validate_campaign \
|
||||||
|
--campaign app/mailer/examples/campaign.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `--check-files` to verify referenced CSV/template/attachment paths.
|
||||||
|
|
||||||
|
### Attachment resolution dry run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.resolve_attachments \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `--json` for machine-readable output.
|
||||||
|
|
||||||
|
### Build message drafts from a campaign JSON file
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.build_messages \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--output-dir ./build/messages \
|
||||||
|
--verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
Builds reviewable `.eml` files without sending. Use `--json` for machine-readable output.
|
||||||
|
|
||||||
|
### SMTP test-send one generated message
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.send_test_message \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--to your.test.address@example.org \
|
||||||
|
--dry-run \
|
||||||
|
--write-eml ./build/test-message.eml
|
||||||
|
```
|
||||||
|
|
||||||
|
Supports SMTP overrides via CLI flags or these environment variables:
|
||||||
|
|
||||||
|
```text
|
||||||
|
MULTIMAILER_SMTP_HOST
|
||||||
|
MULTIMAILER_SMTP_PORT
|
||||||
|
MULTIMAILER_SMTP_SECURITY
|
||||||
|
MULTIMAILER_SMTP_USERNAME
|
||||||
|
MULTIMAILER_SMTP_PASSWORD
|
||||||
|
```
|
||||||
|
|
||||||
|
### Import campaign JSON into the database
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.import_campaign \
|
||||||
|
--campaign app/mailer/examples/campaign.json \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--validate \
|
||||||
|
--build
|
||||||
|
```
|
||||||
|
|
||||||
|
Creates a versioned DB campaign snapshot, optionally validates and builds DB-backed jobs.
|
||||||
|
|
||||||
|
### List database campaigns
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.list_db_campaigns \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
### Queue built campaign jobs
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.queue_campaign \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--no-celery
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--dry-run
|
||||||
|
--exclude-warnings
|
||||||
|
--version-id <campaign-version-uuid>
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Process queued jobs directly without Celery
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.send_queued_jobs \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--limit <n>
|
||||||
|
--no-rate-limit
|
||||||
|
--append-sent
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Append already-sent messages to IMAP Sent
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.append_pending_sent \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--limit <n>
|
||||||
|
--include-failed
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Generate campaign report
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--json
|
||||||
|
--include-jobs
|
||||||
|
--jobs-csv ./build/campaign-jobs.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
### Email campaign report
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.email_campaign_report \
|
||||||
|
--campaign-id <campaign-db-uuid> \
|
||||||
|
--api-key dev-multimailer-api-key \
|
||||||
|
--to recipient@example.org \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful flags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--to <email> # repeat for multiple recipients
|
||||||
|
--include-jobs
|
||||||
|
--no-jobs-csv
|
||||||
|
--attach-report-json
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
### List audit log
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.mailer.commands.audit_log \
|
||||||
|
--api-key dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful filters:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--action campaign.queued
|
||||||
|
--object-type campaign
|
||||||
|
--object-id <campaign-db-uuid>
|
||||||
|
--limit 100
|
||||||
|
--offset 0
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Web UI MVP
|
||||||
|
|
||||||
|
The project now includes a first React/Vite WebUI in `webui/`. It is intentionally API-first: the browser talks to the same FastAPI endpoints that the CLI uses.
|
||||||
|
|
||||||
|
### Run API locally and WebUI in VSCodium/dev mode
|
||||||
|
|
||||||
|
Terminal 1, from `server/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m uvicorn app.main:app --reload --host 127.0.0.1 --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
Terminal 2, from `webui/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Open:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://127.0.0.1:5173
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the development API key:
|
||||||
|
|
||||||
|
```text
|
||||||
|
dev-multimailer-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
The Vite dev server proxies `/api` to `http://127.0.0.1:8000` by default. Override this with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
VITE_DEV_API_PROXY_TARGET=http://127.0.0.1:8000 npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
### Run full local stack with Docker Compose
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env
|
||||||
|
./scripts/up-local.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Open:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://localhost:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Traefik routes:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/api/* and /health -> FastAPI
|
||||||
|
/* -> WebUI
|
||||||
|
```
|
||||||
|
|
||||||
|
Direct ports are also published by default:
|
||||||
|
|
||||||
|
```text
|
||||||
|
API: http://localhost:8000
|
||||||
|
WebUI: http://localhost:5173
|
||||||
|
```
|
||||||
|
|
||||||
|
### WebUI features included in this MVP
|
||||||
|
|
||||||
|
- API-key based connection screen
|
||||||
|
- Campaign list and selection
|
||||||
|
- Campaign JSON import from file or textarea
|
||||||
|
- Version listing
|
||||||
|
- Campaign validation
|
||||||
|
- Message building
|
||||||
|
- Queue dry-run and queue action
|
||||||
|
- Pause/resume/cancel actions
|
||||||
|
- Report dashboard cards
|
||||||
|
- Job table with build/validation/queue/send/IMAP states
|
||||||
|
- Jobs CSV download
|
||||||
|
- IMAP append dry-run/action
|
||||||
|
- Campaign report email dry-run/action
|
||||||
|
- Audit log view
|
||||||
|
- Local WebUI settings
|
||||||
|
|
||||||
|
### Important security note
|
||||||
|
|
||||||
|
For this MVP, the WebUI stores the API base URL and API key in browser `localStorage`. This is acceptable for local development, but production should move to proper login/session handling and short-lived browser tokens. External tools and desktop clients can continue using explicit scoped API keys.
|
||||||
|
|
||||||
|
## WebUI source layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
webui/
|
||||||
|
├─ Dockerfile
|
||||||
|
├─ nginx.conf
|
||||||
|
├─ package.json
|
||||||
|
├─ vite.config.ts
|
||||||
|
├─ index.html
|
||||||
|
└─ src/
|
||||||
|
├─ api.ts
|
||||||
|
├─ main.tsx
|
||||||
|
├─ styles.css
|
||||||
|
└─ types.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
## WebUI next steps
|
||||||
|
|
||||||
|
Recommended next UI iterations:
|
||||||
|
|
||||||
|
1. Add a dedicated campaign configuration editor instead of JSON-only import.
|
||||||
|
2. Add a review screen for `needs_review`, `blocked`, missing attachments and ambiguous attachments.
|
||||||
|
3. Add mail-account administration with encrypted credential storage.
|
||||||
|
4. Add users/groups/RBAC/API-key administration.
|
||||||
|
5. Add attachment upload and Garage browser.
|
||||||
|
6. Add housekeeping/retention/quota screens.
|
||||||
|
7. Add backup/export screens.
|
||||||
|
8. Add scheduled campaign runs.
|
||||||
|
|
||||||
|
## Editable campaign versions / WebUI editor API
|
||||||
|
|
||||||
|
Campaigns now use normal `Campaign` + `CampaignVersion` records for draft/edit/autosave workflows. There is no separate `CampaignDraft` table: an editable campaign is simply a campaign version with `workflow_state=editing` and editor metadata.
|
||||||
|
|
||||||
|
New version/editor fields on `campaign_versions`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
source_base_path
|
||||||
|
workflow_state # editing, under_review, approved, built, queued, sending, completed, cancelled, archived
|
||||||
|
current_flow # create, review, send, manual, json
|
||||||
|
current_step # basics, sender, fields, recipients, template, attachments, review, send, json, ...
|
||||||
|
is_complete
|
||||||
|
editor_state
|
||||||
|
autosaved_at
|
||||||
|
published_at
|
||||||
|
locked_at
|
||||||
|
locked_by_user_id
|
||||||
|
```
|
||||||
|
|
||||||
|
New API endpoints:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /api/v1/schemas/campaign
|
||||||
|
POST /api/v1/campaigns/new
|
||||||
|
GET /api/v1/campaigns/{campaign_id}/versions/{version_id}
|
||||||
|
PUT /api/v1/campaigns/{campaign_id}/versions/{version_id}
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/versions/{version_id}/autosave
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/versions/{version_id}/set-step
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/versions/{version_id}/validate-partial
|
||||||
|
POST /api/v1/campaigns/{campaign_id}/versions/{version_id}/publish
|
||||||
|
```
|
||||||
|
|
||||||
|
Minimal campaign creation example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'http://127.0.0.1:8000/api/v1/campaigns/new' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'X-API-Key: dev-multimailer-api-key' \
|
||||||
|
-d '{
|
||||||
|
"external_id": "my-new-campaign",
|
||||||
|
"name": "My New Campaign",
|
||||||
|
"description": "Created from the WebUI wizard",
|
||||||
|
"current_flow": "create",
|
||||||
|
"current_step": "basics"
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Autosave example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'http://127.0.0.1:8000/api/v1/campaigns/<campaign-id>/versions/<version-id>/autosave' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'X-API-Key: dev-multimailer-api-key' \
|
||||||
|
-d '{
|
||||||
|
"campaign_json": {"version": "1.0", "campaign": {"id": "my-new-campaign", "name": "My New Campaign", "mode": "draft"}},
|
||||||
|
"current_flow": "create",
|
||||||
|
"current_step": "recipients",
|
||||||
|
"is_complete": false
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Partial validation example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'http://127.0.0.1:8000/api/v1/campaigns/<campaign-id>/versions/<version-id>/validate-partial' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'X-API-Key: dev-multimailer-api-key' \
|
||||||
|
-d '{"section": "recipients"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Strict validation/build/send endpoints are unchanged. The WebUI should use partial validation while editing, and only call strict validation/build when the user reaches Review/Send.
|
||||||
|
|||||||
42
compose.external.example.yml
Normal file
42
compose.external.example.yml
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
# Optional override example when Postgres, Redis, Garage and Traefik already exist.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# docker compose -f compose.yml -f compose.external.example.yml up --build
|
||||||
|
#
|
||||||
|
# Adjust EXTERNAL_PROXY_NETWORK and the .env service URLs.
|
||||||
|
|
||||||
|
services:
|
||||||
|
api:
|
||||||
|
ports: []
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
- proxy
|
||||||
|
labels:
|
||||||
|
- traefik.enable=true
|
||||||
|
- traefik.docker.network=${EXTERNAL_PROXY_NETWORK:-proxy}
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.rule=Host(`${MULTIMAILER_HOST:-multimailer.localhost}`) && (PathPrefix(`/api`) || Path(`/health`))
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.entrypoints=${TRAEFIK_ENTRYPOINT:-websecure}
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.priority=100
|
||||||
|
- traefik.http.services.${TRAEFIK_API_SERVICE_NAME:-multimailer-api}.loadbalancer.server.port=8000
|
||||||
|
|
||||||
|
webui:
|
||||||
|
ports: []
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
- proxy
|
||||||
|
labels:
|
||||||
|
- traefik.enable=true
|
||||||
|
- traefik.docker.network=${EXTERNAL_PROXY_NETWORK:-proxy}
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.rule=Host(`${MULTIMAILER_HOST:-multimailer.localhost}`)
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.entrypoints=${TRAEFIK_ENTRYPOINT:-websecure}
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.priority=1
|
||||||
|
- traefik.http.services.${TRAEFIK_WEBUI_SERVICE_NAME:-multimailer-webui}.loadbalancer.server.port=80
|
||||||
|
|
||||||
|
worker:
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
|
||||||
|
networks:
|
||||||
|
proxy:
|
||||||
|
external: true
|
||||||
|
name: ${EXTERNAL_PROXY_NETWORK:-proxy}
|
||||||
115
compose.local.yml
Normal file
115
compose.local.yml
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
services:
|
||||||
|
api:
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
garage:
|
||||||
|
condition: service_started
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
- edge
|
||||||
|
labels:
|
||||||
|
- traefik.enable=true
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.rule=PathPrefix(`/api`) || Path(`/health`)
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.entrypoints=web
|
||||||
|
- traefik.http.routers.${TRAEFIK_API_ROUTER_NAME:-multimailer-api}.priority=100
|
||||||
|
- traefik.http.services.${TRAEFIK_API_SERVICE_NAME:-multimailer-api}.loadbalancer.server.port=8000
|
||||||
|
|
||||||
|
worker:
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
garage:
|
||||||
|
condition: service_started
|
||||||
|
|
||||||
|
|
||||||
|
webui:
|
||||||
|
depends_on:
|
||||||
|
api:
|
||||||
|
condition: service_started
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
- edge
|
||||||
|
labels:
|
||||||
|
- traefik.enable=true
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.rule=PathPrefix(`/`)
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.entrypoints=web
|
||||||
|
- traefik.http.routers.${TRAEFIK_WEBUI_ROUTER_NAME:-multimailer-webui}.priority=1
|
||||||
|
- traefik.http.services.${TRAEFIK_WEBUI_SERVICE_NAME:-multimailer-webui}.loadbalancer.server.port=80
|
||||||
|
|
||||||
|
traefik:
|
||||||
|
image: traefik:v3.2
|
||||||
|
command:
|
||||||
|
- --api.dashboard=true
|
||||||
|
- --api.insecure=${TRAEFIK_DASHBOARD_INSECURE:-true}
|
||||||
|
- --providers.docker=true
|
||||||
|
- --providers.docker.exposedbydefault=false
|
||||||
|
- --entrypoints.web.address=:${TRAEFIK_WEB_PORT:-8080}
|
||||||
|
ports:
|
||||||
|
- "${TRAEFIK_WEB_PORT:-8080}:${TRAEFIK_WEB_PORT:-8080}"
|
||||||
|
- "${TRAEFIK_DASHBOARD_PORT:-8081}:8080"
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
networks:
|
||||||
|
- edge
|
||||||
|
- app
|
||||||
|
|
||||||
|
postgres:
|
||||||
|
image: postgres:16
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: ${POSTGRES_DB:-multimailer}
|
||||||
|
POSTGRES_USER: ${POSTGRES_USER:-multimailer}
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-multimailer}
|
||||||
|
volumes:
|
||||||
|
- postgres_data:/var/lib/postgresql/data
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-multimailer} -d ${POSTGRES_DB:-multimailer}"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
command: ["redis-server", "--appendonly", "yes"]
|
||||||
|
volumes:
|
||||||
|
- redis_data:/data
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
garage:
|
||||||
|
image: dxflrs/garage:v2.3.0
|
||||||
|
command: ["garage", "-c", "/etc/garage.toml", "server", "--single-node", "--default-bucket"]
|
||||||
|
environment:
|
||||||
|
GARAGE_DEFAULT_ACCESS_KEY: ${S3_ACCESS_KEY_ID:-GKmultimailerdev0000000000000000}
|
||||||
|
GARAGE_DEFAULT_SECRET_KEY: ${S3_SECRET_ACCESS_KEY:-multimailer-dev-secret-change-me}
|
||||||
|
GARAGE_DEFAULT_BUCKET: ${S3_BUCKET:-attachments}
|
||||||
|
ports:
|
||||||
|
- "${GARAGE_S3_PORT:-3900}:3900"
|
||||||
|
- "${GARAGE_ADMIN_PORT:-3903}:3903"
|
||||||
|
volumes:
|
||||||
|
- ./infra/garage/garage.toml:/etc/garage.toml:ro
|
||||||
|
- garage_meta:/var/lib/garage/meta
|
||||||
|
- garage_data:/var/lib/garage/data
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
|
||||||
|
networks:
|
||||||
|
edge:
|
||||||
|
name: ${COMPOSE_PROJECT_NAME:-multimailer}_edge
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
postgres_data:
|
||||||
|
redis_data:
|
||||||
|
garage_meta:
|
||||||
|
garage_data:
|
||||||
78
compose.yml
Normal file
78
compose.yml
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
services:
|
||||||
|
api:
|
||||||
|
build:
|
||||||
|
context: ./server
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
environment:
|
||||||
|
APP_ROLE: api
|
||||||
|
APP_ENV: ${APP_ENV:-dev}
|
||||||
|
APP_HOST: ${APP_HOST:-0.0.0.0}
|
||||||
|
APP_PORT: ${APP_PORT:-8000}
|
||||||
|
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:5173,http://127.0.0.1:5173,http://localhost:8080}
|
||||||
|
|
||||||
|
DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://multimailer:multimailer@postgres:5432/multimailer}
|
||||||
|
REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
|
||||||
|
|
||||||
|
S3_ENDPOINT_URL: ${S3_ENDPOINT_URL:-http://garage:3900}
|
||||||
|
S3_REGION: ${S3_REGION:-garage}
|
||||||
|
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-GKmultimailerdev0000000000000000}
|
||||||
|
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-multimailer-dev-secret-change-me}
|
||||||
|
S3_BUCKET: ${S3_BUCKET:-attachments}
|
||||||
|
|
||||||
|
MASTER_KEY_B64: ${MASTER_KEY_B64:-}
|
||||||
|
|
||||||
|
MAX_UPLOAD_MB: ${MAX_UPLOAD_MB:-50}
|
||||||
|
MAX_ATTACHMENTS_PER_JOB: ${MAX_ATTACHMENTS_PER_JOB:-50}
|
||||||
|
DEFAULT_SEND_RATE_PER_MIN: ${DEFAULT_SEND_RATE_PER_MIN:-30}
|
||||||
|
DEFAULT_CONCURRENCY: ${DEFAULT_CONCURRENCY:-2}
|
||||||
|
ports:
|
||||||
|
- "${API_PUBLISHED_PORT:-8000}:8000"
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health').read()"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
worker:
|
||||||
|
build:
|
||||||
|
context: ./server
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
environment:
|
||||||
|
APP_ROLE: worker
|
||||||
|
APP_ENV: ${APP_ENV:-dev}
|
||||||
|
|
||||||
|
DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://multimailer:multimailer@postgres:5432/multimailer}
|
||||||
|
REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
|
||||||
|
|
||||||
|
S3_ENDPOINT_URL: ${S3_ENDPOINT_URL:-http://garage:3900}
|
||||||
|
S3_REGION: ${S3_REGION:-garage}
|
||||||
|
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-GKmultimailerdev0000000000000000}
|
||||||
|
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-multimailer-dev-secret-change-me}
|
||||||
|
S3_BUCKET: ${S3_BUCKET:-attachments}
|
||||||
|
|
||||||
|
MASTER_KEY_B64: ${MASTER_KEY_B64:-}
|
||||||
|
|
||||||
|
CELERY_QUEUES: ${CELERY_QUEUES:-send_email,append_sent,default}
|
||||||
|
CELERY_CONCURRENCY: ${CELERY_CONCURRENCY:-4}
|
||||||
|
CELERY_PREFETCH_MULTIPLIER: ${CELERY_PREFETCH_MULTIPLIER:-1}
|
||||||
|
CELERY_MAX_TASKS_PER_CHILD: ${CELERY_MAX_TASKS_PER_CHILD:-200}
|
||||||
|
CELERY_LOGLEVEL: ${CELERY_LOGLEVEL:-INFO}
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
|
||||||
|
webui:
|
||||||
|
build:
|
||||||
|
context: ./webui
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
args:
|
||||||
|
VITE_API_BASE_URL: ${VITE_API_BASE_URL:-/api/v1}
|
||||||
|
ports:
|
||||||
|
- "${WEBUI_PUBLISHED_PORT:-5173}:80"
|
||||||
|
networks:
|
||||||
|
- app
|
||||||
|
|
||||||
|
networks:
|
||||||
|
app:
|
||||||
|
name: ${COMPOSE_PROJECT_NAME:-multimailer}_app
|
||||||
18
infra/garage/garage.toml
Normal file
18
infra/garage/garage.toml
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
metadata_dir = "/var/lib/garage/meta"
|
||||||
|
data_dir = "/var/lib/garage/data"
|
||||||
|
|
||||||
|
db_engine = "sqlite"
|
||||||
|
replication_factor = 1
|
||||||
|
compression_level = 1
|
||||||
|
|
||||||
|
rpc_bind_addr = "[::]:3901"
|
||||||
|
rpc_secret = "change-this-rpc-secret-minimum-32-chars"
|
||||||
|
|
||||||
|
[s3_api]
|
||||||
|
s3_region = "garage"
|
||||||
|
api_bind_addr = "[::]:3900"
|
||||||
|
root_domain = "localhost"
|
||||||
|
|
||||||
|
[admin]
|
||||||
|
api_bind_addr = "[::]:3903"
|
||||||
|
admin_token = "change-this-admin-token"
|
||||||
BIN
multimailer_python_java_port_with_editable_campaign_versions.zip
Normal file
BIN
multimailer_python_java_port_with_editable_campaign_versions.zip
Normal file
Binary file not shown.
3
scripts/down-local.sh
Normal file
3
scripts/down-local.sh
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
set -e
|
||||||
|
docker compose -f compose.yml -f compose.local.yml down
|
||||||
2
scripts/generate-master-key.sh
Normal file
2
scripts/generate-master-key.sh
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
python -c "import os,base64; print(base64.b64encode(os.urandom(32)).decode())"
|
||||||
3
scripts/up-local.sh
Normal file
3
scripts/up-local.sh
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
set -e
|
||||||
|
docker compose -f compose.yml -f compose.local.yml up --build
|
||||||
22
server/Dockerfile
Normal file
22
server/Dockerfile
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
curl \
|
||||||
|
build-essential \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY requirements.txt /app/requirements.txt
|
||||||
|
RUN pip install --no-cache-dir -r /app/requirements.txt
|
||||||
|
|
||||||
|
COPY entrypoint.sh /app/entrypoint.sh
|
||||||
|
RUN chmod +x /app/entrypoint.sh
|
||||||
|
|
||||||
|
COPY app /app/app
|
||||||
|
|
||||||
|
EXPOSE 8000
|
||||||
|
ENTRYPOINT ["/app/entrypoint.sh"]
|
||||||
38
server/alembic.ini
Normal file
38
server/alembic.ini
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
[alembic]
|
||||||
|
script_location = alembic
|
||||||
|
prepend_sys_path = .
|
||||||
|
sqlalchemy.url = sqlite:///./multimailer-dev.db
|
||||||
|
|
||||||
|
[loggers]
|
||||||
|
keys = root,sqlalchemy,alembic
|
||||||
|
|
||||||
|
[handlers]
|
||||||
|
keys = console
|
||||||
|
|
||||||
|
[formatters]
|
||||||
|
keys = generic
|
||||||
|
|
||||||
|
[logger_root]
|
||||||
|
level = WARN
|
||||||
|
handlers = console
|
||||||
|
qualname =
|
||||||
|
|
||||||
|
[logger_sqlalchemy]
|
||||||
|
level = WARN
|
||||||
|
handlers =
|
||||||
|
qualname = sqlalchemy.engine
|
||||||
|
|
||||||
|
[logger_alembic]
|
||||||
|
level = INFO
|
||||||
|
handlers =
|
||||||
|
qualname = alembic
|
||||||
|
|
||||||
|
[handler_console]
|
||||||
|
class = StreamHandler
|
||||||
|
args = (sys.stderr,)
|
||||||
|
level = NOTSET
|
||||||
|
formatter = generic
|
||||||
|
|
||||||
|
[formatter_generic]
|
||||||
|
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||||
|
datefmt = %H:%M:%S
|
||||||
45
server/alembic/env.py
Normal file
45
server/alembic/env.py
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from logging.config import fileConfig
|
||||||
|
|
||||||
|
from alembic import context
|
||||||
|
from sqlalchemy import engine_from_config, pool
|
||||||
|
|
||||||
|
from app.db.base import Base
|
||||||
|
from app.db import models # noqa: F401 - ensure models are imported
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
config = context.config
|
||||||
|
config.set_main_option("sqlalchemy.url", settings.database_url)
|
||||||
|
|
||||||
|
if config.config_file_name is not None:
|
||||||
|
fileConfig(config.config_file_name)
|
||||||
|
|
||||||
|
target_metadata = Base.metadata
|
||||||
|
|
||||||
|
|
||||||
|
def run_migrations_offline() -> None:
|
||||||
|
url = config.get_main_option("sqlalchemy.url")
|
||||||
|
context.configure(url=url, target_metadata=target_metadata, literal_binds=True, dialect_opts={"paramstyle": "named"})
|
||||||
|
|
||||||
|
with context.begin_transaction():
|
||||||
|
context.run_migrations()
|
||||||
|
|
||||||
|
|
||||||
|
def run_migrations_online() -> None:
|
||||||
|
connectable = engine_from_config(
|
||||||
|
config.get_section(config.config_ini_section, {}),
|
||||||
|
prefix="sqlalchemy.",
|
||||||
|
poolclass=pool.NullPool,
|
||||||
|
)
|
||||||
|
|
||||||
|
with connectable.connect() as connection:
|
||||||
|
context.configure(connection=connection, target_metadata=target_metadata)
|
||||||
|
with context.begin_transaction():
|
||||||
|
context.run_migrations()
|
||||||
|
|
||||||
|
|
||||||
|
if context.is_offline_mode():
|
||||||
|
run_migrations_offline()
|
||||||
|
else:
|
||||||
|
run_migrations_online()
|
||||||
24
server/alembic/script.py.mako
Normal file
24
server/alembic/script.py.mako
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
"""${message}
|
||||||
|
|
||||||
|
Revision ID: ${up_revision}
|
||||||
|
Revises: ${down_revision | comma,n}
|
||||||
|
Create Date: ${create_date}
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
${imports if imports else ""}
|
||||||
|
|
||||||
|
revision = ${repr(up_revision)}
|
||||||
|
down_revision = ${repr(down_revision)}
|
||||||
|
branch_labels = ${repr(branch_labels)}
|
||||||
|
depends_on = ${repr(depends_on)}
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
${upgrades if upgrades else "pass"}
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
${downgrades if downgrades else "pass"}
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
"""editable campaign versions
|
||||||
|
|
||||||
|
Revision ID: 1f8d4c2a0b7e
|
||||||
|
Revises: b57c5b216bce
|
||||||
|
Create Date: 2026-06-08 08:00:00.000000
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "1f8d4c2a0b7e"
|
||||||
|
down_revision = "b57c5b216bce"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||||
|
batch_op.add_column(sa.Column("source_base_path", sa.String(length=1000), nullable=True))
|
||||||
|
batch_op.add_column(sa.Column("workflow_state", sa.String(length=50), nullable=False, server_default="editing"))
|
||||||
|
batch_op.add_column(sa.Column("current_flow", sa.String(length=50), nullable=False, server_default="manual"))
|
||||||
|
batch_op.add_column(sa.Column("current_step", sa.String(length=100), nullable=True))
|
||||||
|
batch_op.add_column(sa.Column("is_complete", sa.Boolean(), nullable=False, server_default=sa.false()))
|
||||||
|
batch_op.add_column(sa.Column("editor_state", sa.JSON(), nullable=False, server_default=sa.text("'{}'")))
|
||||||
|
batch_op.add_column(sa.Column("autosaved_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch_op.add_column(sa.Column("published_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch_op.add_column(sa.Column("locked_at", sa.DateTime(timezone=True), nullable=True))
|
||||||
|
batch_op.add_column(sa.Column("locked_by_user_id", sa.String(length=36), nullable=True))
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
op.f("fk_campaign_versions_locked_by_user_id_users"),
|
||||||
|
"users",
|
||||||
|
["locked_by_user_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="SET NULL",
|
||||||
|
)
|
||||||
|
batch_op.create_index(op.f("ix_campaign_versions_workflow_state"), ["workflow_state"], unique=False)
|
||||||
|
batch_op.create_index(op.f("ix_campaign_versions_current_flow"), ["current_flow"], unique=False)
|
||||||
|
batch_op.create_index(op.f("ix_campaign_versions_locked_by_user_id"), ["locked_by_user_id"], unique=False)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("campaign_versions") as batch_op:
|
||||||
|
batch_op.drop_index(op.f("ix_campaign_versions_locked_by_user_id"))
|
||||||
|
batch_op.drop_index(op.f("ix_campaign_versions_current_flow"))
|
||||||
|
batch_op.drop_index(op.f("ix_campaign_versions_workflow_state"))
|
||||||
|
batch_op.drop_constraint(op.f("fk_campaign_versions_locked_by_user_id_users"), type_="foreignkey")
|
||||||
|
batch_op.drop_column("locked_by_user_id")
|
||||||
|
batch_op.drop_column("locked_at")
|
||||||
|
batch_op.drop_column("published_at")
|
||||||
|
batch_op.drop_column("autosaved_at")
|
||||||
|
batch_op.drop_column("editor_state")
|
||||||
|
batch_op.drop_column("is_complete")
|
||||||
|
batch_op.drop_column("current_step")
|
||||||
|
batch_op.drop_column("current_flow")
|
||||||
|
batch_op.drop_column("workflow_state")
|
||||||
|
batch_op.drop_column("source_base_path")
|
||||||
@@ -0,0 +1,346 @@
|
|||||||
|
"""initial persistence models
|
||||||
|
|
||||||
|
Revision ID: b57c5b216bce
|
||||||
|
Revises:
|
||||||
|
Create Date: 2026-06-07 19:29:10.222504
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = 'b57c5b216bce'
|
||||||
|
down_revision = None
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.create_table('tenants',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('slug', sa.String(length=100), nullable=False),
|
||||||
|
sa.Column('name', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('is_active', sa.Boolean(), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_tenants'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_tenants_slug'), 'tenants', ['slug'], unique=True)
|
||||||
|
op.create_table('attachment_blobs',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('sha256', sa.String(length=64), nullable=False),
|
||||||
|
sa.Column('size_bytes', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('mime_type', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('storage_bucket', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('storage_key', sa.String(length=1000), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_attachment_blobs_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_blobs')),
|
||||||
|
sa.UniqueConstraint('tenant_id', 'sha256', name='uq_attachment_blobs_tenant_sha256')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_attachment_blobs_sha256'), 'attachment_blobs', ['sha256'], unique=False)
|
||||||
|
op.create_index(op.f('ix_attachment_blobs_tenant_id'), 'attachment_blobs', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('groups',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('slug', sa.String(length=100), nullable=False),
|
||||||
|
sa.Column('name', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_groups_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_groups')),
|
||||||
|
sa.UniqueConstraint('tenant_id', 'slug', name='uq_groups_tenant_slug')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_groups_tenant_id'), 'groups', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('roles',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('slug', sa.String(length=100), nullable=False),
|
||||||
|
sa.Column('name', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('permissions', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_roles_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_roles')),
|
||||||
|
sa.UniqueConstraint('tenant_id', 'slug', name='uq_roles_tenant_slug')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_roles_tenant_id'), 'roles', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('users',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('email', sa.String(length=320), nullable=False),
|
||||||
|
sa.Column('display_name', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('is_active', sa.Boolean(), nullable=False),
|
||||||
|
sa.Column('is_tenant_admin', sa.Boolean(), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_users_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_users')),
|
||||||
|
sa.UniqueConstraint('tenant_id', 'email', name='uq_users_tenant_email')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_users_email'), 'users', ['email'], unique=False)
|
||||||
|
op.create_index(op.f('ix_users_tenant_id'), 'users', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('api_keys',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('user_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('name', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('prefix', sa.String(length=16), nullable=False),
|
||||||
|
sa.Column('key_hash', sa.String(length=128), nullable=False),
|
||||||
|
sa.Column('scopes', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('last_used_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_api_keys_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['user_id'], ['users.id'], name=op.f('fk_api_keys_user_id_users'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_api_keys'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_api_keys_prefix'), 'api_keys', ['prefix'], unique=False)
|
||||||
|
op.create_index(op.f('ix_api_keys_tenant_id'), 'api_keys', ['tenant_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_api_keys_user_id'), 'api_keys', ['user_id'], unique=False)
|
||||||
|
op.create_table('campaigns',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('external_id', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('name', sa.String(length=255), nullable=False),
|
||||||
|
sa.Column('description', sa.Text(), nullable=True),
|
||||||
|
sa.Column('status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('current_version_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['created_by_user_id'], ['users.id'], name=op.f('fk_campaigns_created_by_user_id_users'), ondelete='SET NULL'),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaigns_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaigns')),
|
||||||
|
sa.UniqueConstraint('tenant_id', 'external_id', name='uq_campaigns_tenant_external_id')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_campaigns_created_by_user_id'), 'campaigns', ['created_by_user_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaigns_external_id'), 'campaigns', ['external_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaigns_status'), 'campaigns', ['status'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaigns_tenant_id'), 'campaigns', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('attachment_instances',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('owner_user_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('campaign_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('blob_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('logical_name', sa.String(length=500), nullable=True),
|
||||||
|
sa.Column('filename', sa.String(length=500), nullable=False),
|
||||||
|
sa.Column('tags', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('metadata', sa.JSON(), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['blob_id'], ['attachment_blobs.id'], name=op.f('fk_attachment_instances_blob_id_attachment_blobs'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_attachment_instances_campaign_id_campaigns'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['owner_user_id'], ['users.id'], name=op.f('fk_attachment_instances_owner_user_id_users'), ondelete='SET NULL'),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_attachment_instances_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_instances'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_attachment_instances_blob_id'), 'attachment_instances', ['blob_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_attachment_instances_campaign_id'), 'attachment_instances', ['campaign_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_attachment_instances_owner_user_id'), 'attachment_instances', ['owner_user_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_attachment_instances_tenant_id'), 'attachment_instances', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('audit_log',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('user_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('api_key_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('action', sa.String(length=100), nullable=False),
|
||||||
|
sa.Column('object_type', sa.String(length=100), nullable=True),
|
||||||
|
sa.Column('object_id', sa.String(length=100), nullable=True),
|
||||||
|
sa.Column('details', sa.JSON(), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['api_key_id'], ['api_keys.id'], name=op.f('fk_audit_log_api_key_id_api_keys'), ondelete='SET NULL'),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_audit_log_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['user_id'], ['users.id'], name=op.f('fk_audit_log_user_id_users'), ondelete='SET NULL'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_audit_log'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_audit_log_action'), 'audit_log', ['action'], unique=False)
|
||||||
|
op.create_index(op.f('ix_audit_log_api_key_id'), 'audit_log', ['api_key_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_audit_log_object_id'), 'audit_log', ['object_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_audit_log_object_type'), 'audit_log', ['object_type'], unique=False)
|
||||||
|
op.create_index(op.f('ix_audit_log_tenant_id'), 'audit_log', ['tenant_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_audit_log_user_id'), 'audit_log', ['user_id'], unique=False)
|
||||||
|
op.create_table('campaign_versions',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('campaign_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('version_number', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('raw_json', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('schema_version', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('source_filename', sa.String(length=500), nullable=True),
|
||||||
|
sa.Column('validation_summary', sa.JSON(), nullable=True),
|
||||||
|
sa.Column('build_summary', sa.JSON(), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_versions_campaign_id_campaigns'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_versions')),
|
||||||
|
sa.UniqueConstraint('campaign_id', 'version_number', name='uq_campaign_versions_campaign_number')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_campaign_versions_campaign_id'), 'campaign_versions', ['campaign_id'], unique=False)
|
||||||
|
op.create_table('campaign_jobs',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('campaign_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('campaign_version_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('entry_index', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('entry_id', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('recipient_email', sa.String(length=320), nullable=True),
|
||||||
|
sa.Column('subject', sa.String(length=998), nullable=True),
|
||||||
|
sa.Column('message_id_header', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('eml_storage_key', sa.String(length=1000), nullable=True),
|
||||||
|
sa.Column('eml_local_path', sa.String(length=1000), nullable=True),
|
||||||
|
sa.Column('eml_size_bytes', sa.Integer(), nullable=True),
|
||||||
|
sa.Column('build_status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('validation_status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('queue_status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('send_status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('imap_status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('attempt_count', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('last_error', sa.Text(), nullable=True),
|
||||||
|
sa.Column('queued_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('sent_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('resolved_recipients', sa.JSON(), nullable=True),
|
||||||
|
sa.Column('resolved_attachments', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('issues_snapshot', sa.JSON(), nullable=False),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_jobs_campaign_id_campaigns'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_jobs_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaign_jobs_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_jobs')),
|
||||||
|
sa.UniqueConstraint('campaign_version_id', 'entry_index', name='uq_campaign_jobs_version_entry')
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_build_status'), 'campaign_jobs', ['build_status'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_campaign_id'), 'campaign_jobs', ['campaign_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_campaign_version_id'), 'campaign_jobs', ['campaign_version_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_entry_id'), 'campaign_jobs', ['entry_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_imap_status'), 'campaign_jobs', ['imap_status'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_queue_status'), 'campaign_jobs', ['queue_status'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_recipient_email'), 'campaign_jobs', ['recipient_email'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_send_status'), 'campaign_jobs', ['send_status'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_tenant_id'), 'campaign_jobs', ['tenant_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_jobs_validation_status'), 'campaign_jobs', ['validation_status'], unique=False)
|
||||||
|
op.create_table('campaign_issues',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('tenant_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('campaign_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('campaign_version_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('job_id', sa.String(length=36), nullable=True),
|
||||||
|
sa.Column('severity', sa.String(length=20), nullable=False),
|
||||||
|
sa.Column('code', sa.String(length=100), nullable=False),
|
||||||
|
sa.Column('message', sa.Text(), nullable=False),
|
||||||
|
sa.Column('source', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('behavior', sa.String(length=50), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_issues_campaign_id_campaigns'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_issues_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_campaign_issues_job_id_campaign_jobs'), ondelete='CASCADE'),
|
||||||
|
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaign_issues_tenant_id_tenants'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_issues'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_campaign_id'), 'campaign_issues', ['campaign_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_campaign_version_id'), 'campaign_issues', ['campaign_version_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_code'), 'campaign_issues', ['code'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_job_id'), 'campaign_issues', ['job_id'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_severity'), 'campaign_issues', ['severity'], unique=False)
|
||||||
|
op.create_index(op.f('ix_campaign_issues_tenant_id'), 'campaign_issues', ['tenant_id'], unique=False)
|
||||||
|
op.create_table('imap_append_attempts',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('job_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('attempt_number', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('folder', sa.String(length=500), nullable=True),
|
||||||
|
sa.Column('status', sa.String(length=50), nullable=False),
|
||||||
|
sa.Column('error_message', sa.Text(), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_imap_append_attempts_job_id_campaign_jobs'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_imap_append_attempts'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_imap_append_attempts_job_id'), 'imap_append_attempts', ['job_id'], unique=False)
|
||||||
|
op.create_table('send_attempts',
|
||||||
|
sa.Column('id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('job_id', sa.String(length=36), nullable=False),
|
||||||
|
sa.Column('attempt_number', sa.Integer(), nullable=False),
|
||||||
|
sa.Column('smtp_status_code', sa.Integer(), nullable=True),
|
||||||
|
sa.Column('smtp_response', sa.Text(), nullable=True),
|
||||||
|
sa.Column('error_type', sa.String(length=255), nullable=True),
|
||||||
|
sa.Column('error_message', sa.Text(), nullable=True),
|
||||||
|
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_send_attempts_job_id_campaign_jobs'), ondelete='CASCADE'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_send_attempts'))
|
||||||
|
)
|
||||||
|
op.create_index(op.f('ix_send_attempts_job_id'), 'send_attempts', ['job_id'], unique=False)
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.drop_index(op.f('ix_send_attempts_job_id'), table_name='send_attempts')
|
||||||
|
op.drop_table('send_attempts')
|
||||||
|
op.drop_index(op.f('ix_imap_append_attempts_job_id'), table_name='imap_append_attempts')
|
||||||
|
op.drop_table('imap_append_attempts')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_tenant_id'), table_name='campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_severity'), table_name='campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_job_id'), table_name='campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_code'), table_name='campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_campaign_version_id'), table_name='campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_issues_campaign_id'), table_name='campaign_issues')
|
||||||
|
op.drop_table('campaign_issues')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_validation_status'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_tenant_id'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_send_status'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_recipient_email'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_queue_status'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_imap_status'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_entry_id'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_campaign_version_id'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_campaign_id'), table_name='campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_jobs_build_status'), table_name='campaign_jobs')
|
||||||
|
op.drop_table('campaign_jobs')
|
||||||
|
op.drop_index(op.f('ix_campaign_versions_campaign_id'), table_name='campaign_versions')
|
||||||
|
op.drop_table('campaign_versions')
|
||||||
|
op.drop_index(op.f('ix_audit_log_user_id'), table_name='audit_log')
|
||||||
|
op.drop_index(op.f('ix_audit_log_tenant_id'), table_name='audit_log')
|
||||||
|
op.drop_index(op.f('ix_audit_log_object_type'), table_name='audit_log')
|
||||||
|
op.drop_index(op.f('ix_audit_log_object_id'), table_name='audit_log')
|
||||||
|
op.drop_index(op.f('ix_audit_log_api_key_id'), table_name='audit_log')
|
||||||
|
op.drop_index(op.f('ix_audit_log_action'), table_name='audit_log')
|
||||||
|
op.drop_table('audit_log')
|
||||||
|
op.drop_index(op.f('ix_attachment_instances_tenant_id'), table_name='attachment_instances')
|
||||||
|
op.drop_index(op.f('ix_attachment_instances_owner_user_id'), table_name='attachment_instances')
|
||||||
|
op.drop_index(op.f('ix_attachment_instances_campaign_id'), table_name='attachment_instances')
|
||||||
|
op.drop_index(op.f('ix_attachment_instances_blob_id'), table_name='attachment_instances')
|
||||||
|
op.drop_table('attachment_instances')
|
||||||
|
op.drop_index(op.f('ix_campaigns_tenant_id'), table_name='campaigns')
|
||||||
|
op.drop_index(op.f('ix_campaigns_status'), table_name='campaigns')
|
||||||
|
op.drop_index(op.f('ix_campaigns_external_id'), table_name='campaigns')
|
||||||
|
op.drop_index(op.f('ix_campaigns_created_by_user_id'), table_name='campaigns')
|
||||||
|
op.drop_table('campaigns')
|
||||||
|
op.drop_index(op.f('ix_api_keys_user_id'), table_name='api_keys')
|
||||||
|
op.drop_index(op.f('ix_api_keys_tenant_id'), table_name='api_keys')
|
||||||
|
op.drop_index(op.f('ix_api_keys_prefix'), table_name='api_keys')
|
||||||
|
op.drop_table('api_keys')
|
||||||
|
op.drop_index(op.f('ix_users_tenant_id'), table_name='users')
|
||||||
|
op.drop_index(op.f('ix_users_email'), table_name='users')
|
||||||
|
op.drop_table('users')
|
||||||
|
op.drop_index(op.f('ix_roles_tenant_id'), table_name='roles')
|
||||||
|
op.drop_table('roles')
|
||||||
|
op.drop_index(op.f('ix_groups_tenant_id'), table_name='groups')
|
||||||
|
op.drop_table('groups')
|
||||||
|
op.drop_index(op.f('ix_attachment_blobs_tenant_id'), table_name='attachment_blobs')
|
||||||
|
op.drop_index(op.f('ix_attachment_blobs_sha256'), table_name='attachment_blobs')
|
||||||
|
op.drop_table('attachment_blobs')
|
||||||
|
op.drop_index(op.f('ix_tenants_slug'), table_name='tenants')
|
||||||
|
op.drop_table('tenants')
|
||||||
|
# ### end Alembic commands ###
|
||||||
0
server/app/__init__.py
Normal file
0
server/app/__init__.py
Normal file
0
server/app/api/__init__.py
Normal file
0
server/app/api/__init__.py
Normal file
12
server/app/api/v1/__init__.py
Normal file
12
server/app/api/v1/__init__.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from .admin import router as admin_router
|
||||||
|
from .campaigns import router as campaigns_router
|
||||||
|
from .audit import router as audit_router
|
||||||
|
from .system import router as system_router
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1")
|
||||||
|
router.include_router(campaigns_router)
|
||||||
|
router.include_router(admin_router)
|
||||||
|
router.include_router(audit_router)
|
||||||
|
router.include_router(system_router)
|
||||||
37
server/app/api/v1/admin.py
Normal file
37
server/app/api/v1/admin.py
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api.v1.schemas import ApiKeyCreateRequest, ApiKeyCreateResponse
|
||||||
|
from app.auth.dependencies import ApiPrincipal, require_scope
|
||||||
|
from app.audit.logging import audit_from_principal
|
||||||
|
from app.db.session import get_session
|
||||||
|
from app.security.api_keys import create_api_key
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/admin", tags=["admin"])
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api-keys", response_model=ApiKeyCreateResponse)
|
||||||
|
def create_personal_api_key(
|
||||||
|
payload: ApiKeyCreateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("admin:settings")),
|
||||||
|
):
|
||||||
|
created = create_api_key(session, user=principal.user, name=payload.name, scopes=payload.scopes or ["campaign:read"])
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="api_key.created",
|
||||||
|
object_type="api_key",
|
||||||
|
object_id=created.model.id,
|
||||||
|
details={"name": created.model.name, "prefix": created.model.prefix, "scopes": created.model.scopes},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return ApiKeyCreateResponse(
|
||||||
|
id=created.model.id,
|
||||||
|
name=created.model.name,
|
||||||
|
prefix=created.model.prefix,
|
||||||
|
scopes=created.model.scopes,
|
||||||
|
secret=created.secret,
|
||||||
|
)
|
||||||
32
server/app/api/v1/audit.py
Normal file
32
server/app/api/v1/audit.py
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api.v1.schemas import AuditLogItemResponse, AuditLogListResponse
|
||||||
|
from app.auth.dependencies import ApiPrincipal, require_scope
|
||||||
|
from app.db.models import AuditLog
|
||||||
|
from app.db.session import get_session
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/audit", tags=["audit"])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=AuditLogListResponse)
|
||||||
|
def list_audit_log(
|
||||||
|
limit: int = Query(default=100, ge=1, le=500),
|
||||||
|
offset: int = Query(default=0, ge=0),
|
||||||
|
action: str | None = None,
|
||||||
|
object_type: str | None = None,
|
||||||
|
object_id: str | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("audit:read")),
|
||||||
|
):
|
||||||
|
query = session.query(AuditLog).filter(AuditLog.tenant_id == principal.tenant_id)
|
||||||
|
if action:
|
||||||
|
query = query.filter(AuditLog.action == action)
|
||||||
|
if object_type:
|
||||||
|
query = query.filter(AuditLog.object_type == object_type)
|
||||||
|
if object_id:
|
||||||
|
query = query.filter(AuditLog.object_id == object_id)
|
||||||
|
items = query.order_by(AuditLog.created_at.desc()).offset(offset).limit(limit).all()
|
||||||
|
return AuditLogListResponse(items=[AuditLogItemResponse.model_validate(item) for item in items])
|
||||||
651
server/app/api/v1/campaigns.py
Normal file
651
server/app/api/v1/campaigns.py
Normal file
@@ -0,0 +1,651 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Response, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api.v1.schemas import (
|
||||||
|
BuildCampaignRequest,
|
||||||
|
CampaignCreateRequest,
|
||||||
|
CampaignCreateResponse,
|
||||||
|
CampaignCreateMinimalRequest,
|
||||||
|
CampaignJobsResponse,
|
||||||
|
CampaignListResponse,
|
||||||
|
CampaignResponse,
|
||||||
|
CampaignVersionDetailResponse,
|
||||||
|
CampaignVersionResponse,
|
||||||
|
CampaignVersionSetStepRequest,
|
||||||
|
CampaignVersionUpdateRequest,
|
||||||
|
CampaignPartialValidationRequest,
|
||||||
|
CampaignPartialValidationResponse,
|
||||||
|
ValidateCampaignRequest,
|
||||||
|
ReportEmailRequest,
|
||||||
|
ReportEmailResponse,
|
||||||
|
)
|
||||||
|
from app.auth.dependencies import ApiPrincipal, require_scope
|
||||||
|
from app.audit.logging import audit_from_principal
|
||||||
|
from app.db.models import Campaign, CampaignJob, CampaignVersion
|
||||||
|
from app.db.session import get_session
|
||||||
|
from app.mailer.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
||||||
|
from app.mailer.reports.emailing import CampaignReportEmailError, send_campaign_report_email
|
||||||
|
from app.mailer.persistence.campaigns import (
|
||||||
|
CampaignPersistenceError,
|
||||||
|
build_campaign_version,
|
||||||
|
create_campaign_version_from_json,
|
||||||
|
validate_campaign_version,
|
||||||
|
)
|
||||||
|
from app.mailer.persistence.versions import (
|
||||||
|
create_minimal_campaign,
|
||||||
|
get_campaign_version_for_tenant,
|
||||||
|
publish_campaign_version,
|
||||||
|
update_campaign_version,
|
||||||
|
validate_campaign_partial,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/campaigns", tags=["campaigns"])
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign_for_tenant(session: Session, campaign_id: str, tenant_id: str) -> Campaign:
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Campaign not found")
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _get_version_for_tenant(session: Session, version_id: str, tenant_id: str) -> CampaignVersion:
|
||||||
|
version = session.get(CampaignVersion, version_id)
|
||||||
|
if not version:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found")
|
||||||
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found")
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("", response_model=CampaignCreateResponse)
|
||||||
|
def create_campaign(
|
||||||
|
payload: CampaignCreateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
campaign, version = create_campaign_version_from_json(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
raw_json=payload.config,
|
||||||
|
source_filename=payload.source_filename,
|
||||||
|
source_base_path=payload.source_base_path,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.created",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign.id,
|
||||||
|
details={"version_id": version.id, "external_id": campaign.external_id},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
return CampaignCreateResponse(campaign=CampaignResponse.model_validate(campaign), version=CampaignVersionResponse.model_validate(version))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/new", response_model=CampaignCreateResponse)
|
||||||
|
def create_minimal_campaign_endpoint(
|
||||||
|
payload: CampaignCreateMinimalRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
"""Create a minimal editable campaign/version for the WebUI wizard.
|
||||||
|
|
||||||
|
This is intentionally different from importing a complete campaign JSON. It
|
||||||
|
returns a normal Campaign + CampaignVersion whose version is a working copy
|
||||||
|
and can be autosaved while incomplete.
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
campaign, version = create_minimal_campaign(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
external_id=payload.external_id,
|
||||||
|
name=payload.name,
|
||||||
|
description=payload.description,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.created_minimal",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign.id,
|
||||||
|
details={"version_id": version.id, "external_id": campaign.external_id},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignCreateResponse(campaign=CampaignResponse.model_validate(campaign), version=CampaignVersionResponse.model_validate(version))
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=CampaignListResponse)
|
||||||
|
def list_campaigns(
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
campaigns = (
|
||||||
|
session.query(Campaign)
|
||||||
|
.filter(Campaign.tenant_id == principal.tenant_id)
|
||||||
|
.order_by(Campaign.updated_at.desc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return CampaignListResponse(campaigns=[CampaignResponse.model_validate(item) for item in campaigns])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}", response_model=CampaignResponse)
|
||||||
|
def get_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
return CampaignResponse.model_validate(_get_campaign_for_tenant(session, campaign_id, principal.tenant_id))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/versions", response_model=list[CampaignVersionResponse])
|
||||||
|
def list_versions(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
versions = (
|
||||||
|
session.query(CampaignVersion)
|
||||||
|
.filter(CampaignVersion.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignVersion.version_number.desc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return [CampaignVersionResponse.model_validate(item) for item in versions]
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse)
|
||||||
|
def get_version_detail(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = get_campaign_version_for_tenant(
|
||||||
|
session, tenant_id=principal.tenant_id, campaign_id=campaign_id, version_id=version_id
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(version)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/{campaign_id}/versions/{version_id}", response_model=CampaignVersionDetailResponse)
|
||||||
|
def update_version_detail(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
raw_json=payload.campaign_json,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
workflow_state=payload.workflow_state,
|
||||||
|
is_complete=payload.is_complete,
|
||||||
|
editor_state=payload.editor_state,
|
||||||
|
source_filename=payload.source_filename,
|
||||||
|
source_base_path=payload.source_base_path,
|
||||||
|
autosave=False,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_updated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={"campaign_id": campaign_id, "current_flow": version.current_flow, "current_step": version.current_step},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(version)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/versions/{version_id}/autosave", response_model=CampaignVersionDetailResponse)
|
||||||
|
def autosave_version(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionUpdateRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
raw_json=payload.campaign_json,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
workflow_state=payload.workflow_state,
|
||||||
|
is_complete=payload.is_complete,
|
||||||
|
editor_state=payload.editor_state,
|
||||||
|
source_filename=payload.source_filename,
|
||||||
|
source_base_path=payload.source_base_path,
|
||||||
|
autosave=True,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_autosaved",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={"campaign_id": campaign_id, "current_flow": version.current_flow, "current_step": version.current_step},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(version)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/versions/{version_id}/set-step", response_model=CampaignVersionDetailResponse)
|
||||||
|
def set_version_step(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignVersionSetStepRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = update_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=version_id,
|
||||||
|
current_flow=payload.current_flow,
|
||||||
|
current_step=payload.current_step,
|
||||||
|
autosave=True,
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(version)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/versions/{version_id}/validate-partial", response_model=CampaignPartialValidationResponse)
|
||||||
|
def validate_version_partial(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
payload: CampaignPartialValidationRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:validate")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = get_campaign_version_for_tenant(
|
||||||
|
session, tenant_id=principal.tenant_id, campaign_id=campaign_id, version_id=version_id
|
||||||
|
)
|
||||||
|
campaign_json = payload.campaign_json if payload and payload.campaign_json is not None else version.raw_json
|
||||||
|
result = validate_campaign_partial(campaign_json, section=payload.section if payload else None)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_partially_validated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={"campaign_id": campaign_id, "section": result.get("section"), "ok": result.get("ok")},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignPartialValidationResponse(**result)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/versions/{version_id}/publish", response_model=CampaignVersionDetailResponse)
|
||||||
|
def publish_version(
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:write")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
version = publish_campaign_version(session, tenant_id=principal.tenant_id, campaign_id=campaign_id, version_id=version_id)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.version_published",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version.id,
|
||||||
|
details={"campaign_id": campaign_id},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignVersionDetailResponse.model_validate(version)
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/versions/{version_id}/validate")
|
||||||
|
def validate_version(
|
||||||
|
version_id: str,
|
||||||
|
payload: ValidateCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:validate")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = validate_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
version_id=version_id,
|
||||||
|
check_files=payload.check_files if payload else False,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.validated",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={"check_files": payload.check_files if payload else False, "ok": result.get("ok")},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/versions/{version_id}/build")
|
||||||
|
def build_version(
|
||||||
|
version_id: str,
|
||||||
|
payload: BuildCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:build")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = build_campaign_version(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
version_id=version_id,
|
||||||
|
write_eml=payload.write_eml if payload else True,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.messages_built",
|
||||||
|
object_type="campaign_version",
|
||||||
|
object_id=version_id,
|
||||||
|
details={"write_eml": payload.write_eml if payload else True, "built_count": result.get("built_count")},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
except CampaignPersistenceError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/jobs", response_model=CampaignJobsResponse)
|
||||||
|
def list_jobs(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(CampaignJob.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return CampaignJobsResponse(
|
||||||
|
jobs=[
|
||||||
|
{
|
||||||
|
"id": job.id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"recipient_email": job.recipient_email,
|
||||||
|
"subject": job.subject,
|
||||||
|
"build_status": job.build_status,
|
||||||
|
"validation_status": job.validation_status,
|
||||||
|
"queue_status": job.queue_status,
|
||||||
|
"send_status": job.send_status,
|
||||||
|
"imap_status": job.imap_status,
|
||||||
|
"eml_local_path": job.eml_local_path,
|
||||||
|
"eml_size_bytes": job.eml_size_bytes,
|
||||||
|
"attempt_count": job.attempt_count,
|
||||||
|
"last_error": job.last_error,
|
||||||
|
"queued_at": job.queued_at,
|
||||||
|
"sent_at": job.sent_at,
|
||||||
|
"issues": job.issues_snapshot,
|
||||||
|
"attachments": job.resolved_attachments,
|
||||||
|
}
|
||||||
|
for job in jobs
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/summary")
|
||||||
|
def campaign_summary(
|
||||||
|
campaign_id: str,
|
||||||
|
include_jobs: bool = False,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
"""Return dashboard-friendly campaign status counters and summaries."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
return generate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
include_jobs=include_jobs,
|
||||||
|
)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/report")
|
||||||
|
def campaign_report(
|
||||||
|
campaign_id: str,
|
||||||
|
include_jobs: bool = True,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
"""Return the full JSON report for one campaign."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
return generate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
include_jobs=include_jobs,
|
||||||
|
)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{campaign_id}/report/jobs.csv")
|
||||||
|
def campaign_jobs_csv(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
):
|
||||||
|
"""Export per-job campaign status as CSV."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
csv_text = generate_jobs_csv(session, tenant_id=principal.tenant_id, campaign_id=campaign_id)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
return Response(
|
||||||
|
content=csv_text,
|
||||||
|
media_type="text/csv; charset=utf-8",
|
||||||
|
headers={"Content-Disposition": f'attachment; filename="campaign-{campaign_id}-jobs.csv"'},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/report/email", response_model=ReportEmailResponse)
|
||||||
|
def email_campaign_report(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: ReportEmailRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("reports:send")),
|
||||||
|
):
|
||||||
|
"""Generate a campaign report and send it to one or more email addresses."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = send_campaign_report_email(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
to=payload.to,
|
||||||
|
include_jobs=payload.include_jobs,
|
||||||
|
attach_jobs_csv=payload.attach_jobs_csv,
|
||||||
|
attach_report_json=payload.attach_report_json,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="report.email_sent" if not payload.dry_run else "report.email_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result.as_dict(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return ReportEmailResponse(result=result.as_dict())
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
|
||||||
|
except (CampaignReportEmailError, Exception) as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
# Queue / delivery control -------------------------------------------------
|
||||||
|
from app.api.v1.schemas import AppendSentRequest, CampaignActionResponse, QueueCampaignRequest, QueueCampaignResponse
|
||||||
|
from app.mailer.sending.jobs import (
|
||||||
|
QueueingError,
|
||||||
|
cancel_campaign_jobs,
|
||||||
|
enqueue_pending_imap_appends,
|
||||||
|
pause_campaign_jobs,
|
||||||
|
queue_campaign_jobs,
|
||||||
|
resume_campaign_jobs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/queue", response_model=QueueCampaignResponse)
|
||||||
|
def queue_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: QueueCampaignRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:queue")),
|
||||||
|
):
|
||||||
|
payload = payload or QueueCampaignRequest()
|
||||||
|
try:
|
||||||
|
result = queue_campaign_jobs(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
version_id=payload.version_id,
|
||||||
|
include_warnings=payload.include_warnings,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.queued" if not payload.dry_run else "campaign.queue_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result.as_dict(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return QueueCampaignResponse(**result.as_dict())
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/pause", response_model=CampaignActionResponse)
|
||||||
|
def pause_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:queue")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = pause_campaign_jobs(session, tenant_id=principal.tenant_id, campaign_id=campaign_id)
|
||||||
|
audit_from_principal(session, principal, action="campaign.paused", object_type="campaign", object_id=campaign_id, details=result, commit=True)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/resume", response_model=CampaignActionResponse)
|
||||||
|
def resume_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:queue")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = resume_campaign_jobs(session, tenant_id=principal.tenant_id, campaign_id=campaign_id)
|
||||||
|
audit_from_principal(session, principal, action="campaign.resumed", object_type="campaign", object_id=campaign_id, details=result, commit=True)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/cancel", response_model=CampaignActionResponse)
|
||||||
|
def cancel_campaign(
|
||||||
|
campaign_id: str,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:queue")),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = cancel_campaign_jobs(session, tenant_id=principal.tenant_id, campaign_id=campaign_id)
|
||||||
|
audit_from_principal(session, principal, action="campaign.cancelled", object_type="campaign", object_id=campaign_id, details=result, commit=True)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{campaign_id}/append-sent", response_model=CampaignActionResponse)
|
||||||
|
def append_sent(
|
||||||
|
campaign_id: str,
|
||||||
|
payload: AppendSentRequest | None = None,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:send")),
|
||||||
|
):
|
||||||
|
payload = payload or AppendSentRequest()
|
||||||
|
try:
|
||||||
|
result = enqueue_pending_imap_appends(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
enqueue_celery=payload.enqueue_celery,
|
||||||
|
dry_run=payload.dry_run,
|
||||||
|
)
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="campaign.append_sent_enqueued" if not payload.dry_run else "campaign.append_sent_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign_id,
|
||||||
|
details=result,
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
return CampaignActionResponse(result=result)
|
||||||
|
except QueueingError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
|
||||||
202
server/app/api/v1/schemas.py
Normal file
202
server/app/api/v1/schemas.py
Normal file
@@ -0,0 +1,202 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignCreateRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
config: dict[str, Any]
|
||||||
|
source_filename: str | None = None
|
||||||
|
source_base_path: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignCreateMinimalRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
external_id: str
|
||||||
|
name: str
|
||||||
|
description: str | None = None
|
||||||
|
current_flow: str = "create"
|
||||||
|
current_step: str = "basics"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionUpdateRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaign_json: dict[str, Any] | None = None
|
||||||
|
current_flow: str | None = None
|
||||||
|
current_step: str | None = None
|
||||||
|
workflow_state: str | None = None
|
||||||
|
is_complete: bool | None = None
|
||||||
|
editor_state: dict[str, Any] | None = None
|
||||||
|
source_filename: str | None = None
|
||||||
|
source_base_path: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionSetStepRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
current_flow: str | None = None
|
||||||
|
current_step: str
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignPartialValidationRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaign_json: dict[str, Any] | None = None
|
||||||
|
section: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionResponse(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
id: str
|
||||||
|
campaign_id: str
|
||||||
|
version_number: int
|
||||||
|
schema_version: str
|
||||||
|
source_filename: str | None = None
|
||||||
|
source_base_path: str | None = None
|
||||||
|
workflow_state: str = "editing"
|
||||||
|
current_flow: str = "manual"
|
||||||
|
current_step: str | None = None
|
||||||
|
is_complete: bool = False
|
||||||
|
editor_state: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
autosaved_at: datetime | None = None
|
||||||
|
published_at: datetime | None = None
|
||||||
|
locked_at: datetime | None = None
|
||||||
|
locked_by_user_id: str | None = None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
validation_summary: dict[str, Any] | None = None
|
||||||
|
build_summary: dict[str, Any] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionDetailResponse(CampaignVersionResponse):
|
||||||
|
raw_json: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignPartialValidationResponse(BaseModel):
|
||||||
|
ok: bool
|
||||||
|
section: str | None = None
|
||||||
|
error_count: int
|
||||||
|
warning_count: int
|
||||||
|
info_count: int
|
||||||
|
issues: list[dict[str, Any]]
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignResponse(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
id: str
|
||||||
|
external_id: str
|
||||||
|
name: str
|
||||||
|
description: str | None = None
|
||||||
|
status: str
|
||||||
|
current_version_id: str | None = None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignCreateResponse(BaseModel):
|
||||||
|
campaign: CampaignResponse
|
||||||
|
version: CampaignVersionResponse
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignListResponse(BaseModel):
|
||||||
|
campaigns: list[CampaignResponse]
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignJobsResponse(BaseModel):
|
||||||
|
jobs: list[dict[str, Any]]
|
||||||
|
|
||||||
|
|
||||||
|
class ValidateCampaignRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
check_files: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class BuildCampaignRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
write_eml: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class ApiKeyCreateRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
name: str
|
||||||
|
scopes: list[str] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class ApiKeyCreateResponse(BaseModel):
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
prefix: str
|
||||||
|
scopes: list[str]
|
||||||
|
secret: str
|
||||||
|
|
||||||
|
|
||||||
|
class QueueCampaignRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
version_id: str | None = None
|
||||||
|
include_warnings: bool = True
|
||||||
|
enqueue_celery: bool = True
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class QueueCampaignResponse(BaseModel):
|
||||||
|
campaign_id: str
|
||||||
|
version_id: str
|
||||||
|
queued_count: int
|
||||||
|
skipped_count: int
|
||||||
|
blocked_count: int
|
||||||
|
enqueued_count: int
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class AppendSentRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
enqueue_celery: bool = True
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignActionResponse(BaseModel):
|
||||||
|
result: dict[str, Any]
|
||||||
|
|
||||||
|
class ReportEmailRequest(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
to: list[str]
|
||||||
|
include_jobs: bool = False
|
||||||
|
attach_jobs_csv: bool = True
|
||||||
|
attach_report_json: bool = False
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class ReportEmailResponse(BaseModel):
|
||||||
|
result: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
class AuditLogItemResponse(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
id: str
|
||||||
|
tenant_id: str | None = None
|
||||||
|
user_id: str | None = None
|
||||||
|
api_key_id: str | None = None
|
||||||
|
action: str
|
||||||
|
object_type: str | None = None
|
||||||
|
object_id: str | None = None
|
||||||
|
details: dict[str, Any] | None = None
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class AuditLogListResponse(BaseModel):
|
||||||
|
items: list[AuditLogItemResponse]
|
||||||
25
server/app/api/v1/system.py
Normal file
25
server/app/api/v1/system.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
|
||||||
|
from app.auth.dependencies import ApiPrincipal, require_scope
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/schemas", tags=["schemas"])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/campaign")
|
||||||
|
def get_campaign_schema(
|
||||||
|
principal: ApiPrincipal = Depends(require_scope("campaign:read")),
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Return the authoritative campaign JSON Schema used by the backend.
|
||||||
|
|
||||||
|
The WebUI can fetch this instead of carrying a stale copy. A future UI
|
||||||
|
schema can be served alongside this endpoint.
|
||||||
|
"""
|
||||||
|
|
||||||
|
schema_path = Path(__file__).resolve().parents[2] / "mailer" / "schemas" / "campaign.schema.json"
|
||||||
|
return json.loads(schema_path.read_text(encoding="utf-8"))
|
||||||
1
server/app/audit/__init__.py
Normal file
1
server/app/audit/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""Audit logging helpers."""
|
||||||
91
server/app/audit/logging.py
Normal file
91
server/app/audit/logging.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.auth.dependencies import ApiPrincipal
|
||||||
|
from app.db.models import AuditLog
|
||||||
|
|
||||||
|
|
||||||
|
SENSITIVE_DETAIL_KEYS = {
|
||||||
|
"password",
|
||||||
|
"smtp_password",
|
||||||
|
"imap_password",
|
||||||
|
"secret",
|
||||||
|
"api_key",
|
||||||
|
"token",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_details(value: Any) -> Any:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
sanitized: dict[str, Any] = {}
|
||||||
|
for key, item in value.items():
|
||||||
|
if str(key).lower() in SENSITIVE_DETAIL_KEYS:
|
||||||
|
sanitized[key] = "<redacted>"
|
||||||
|
else:
|
||||||
|
sanitized[key] = _sanitize_details(item)
|
||||||
|
return sanitized
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [_sanitize_details(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def audit_event(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str | None,
|
||||||
|
action: str,
|
||||||
|
user_id: str | None = None,
|
||||||
|
api_key_id: str | None = None,
|
||||||
|
object_type: str | None = None,
|
||||||
|
object_id: str | None = None,
|
||||||
|
details: dict[str, Any] | None = None,
|
||||||
|
commit: bool = False,
|
||||||
|
) -> AuditLog:
|
||||||
|
"""Persist one audit event.
|
||||||
|
|
||||||
|
The function deliberately accepts primitive IDs so it can be used from API
|
||||||
|
handlers, CLI commands and worker code without coupling the lower-level
|
||||||
|
services to FastAPI request objects.
|
||||||
|
"""
|
||||||
|
|
||||||
|
item = AuditLog(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
user_id=user_id,
|
||||||
|
api_key_id=api_key_id,
|
||||||
|
action=action,
|
||||||
|
object_type=object_type,
|
||||||
|
object_id=object_id,
|
||||||
|
details=_sanitize_details(details or {}),
|
||||||
|
)
|
||||||
|
session.add(item)
|
||||||
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
else:
|
||||||
|
session.flush()
|
||||||
|
return item
|
||||||
|
|
||||||
|
|
||||||
|
def audit_from_principal(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
object_type: str | None = None,
|
||||||
|
object_id: str | None = None,
|
||||||
|
details: dict[str, Any] | None = None,
|
||||||
|
commit: bool = False,
|
||||||
|
) -> AuditLog:
|
||||||
|
return audit_event(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
api_key_id=principal.api_key.id,
|
||||||
|
action=action,
|
||||||
|
object_type=object_type,
|
||||||
|
object_id=object_id,
|
||||||
|
details=details,
|
||||||
|
commit=commit,
|
||||||
|
)
|
||||||
0
server/app/auth/__init__.py
Normal file
0
server/app/auth/__init__.py
Normal file
52
server/app/auth/dependencies.py
Normal file
52
server/app/auth/dependencies.py
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from fastapi import Depends, Header, HTTPException, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import ApiKey, User
|
||||||
|
from app.db.session import get_session
|
||||||
|
from app.security.api_keys import authenticate_api_key, has_scope
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class ApiPrincipal:
|
||||||
|
api_key: ApiKey
|
||||||
|
user: User
|
||||||
|
tenant_id: str
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_api_key(authorization: str | None, x_api_key: str | None) -> str | None:
|
||||||
|
if x_api_key:
|
||||||
|
return x_api_key.strip()
|
||||||
|
if authorization and authorization.lower().startswith("bearer "):
|
||||||
|
return authorization[7:].strip()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def get_api_principal(
|
||||||
|
session: Session = Depends(get_session),
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
x_api_key: str | None = Header(default=None, alias="X-API-Key"),
|
||||||
|
) -> ApiPrincipal:
|
||||||
|
secret = _extract_api_key(authorization, x_api_key)
|
||||||
|
if not secret:
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing API key")
|
||||||
|
api_key = authenticate_api_key(session, secret)
|
||||||
|
if not api_key:
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid API key")
|
||||||
|
user = session.get(User, api_key.user_id)
|
||||||
|
if not user or not user.is_active:
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Inactive user")
|
||||||
|
session.commit()
|
||||||
|
return ApiPrincipal(api_key=api_key, user=user, tenant_id=api_key.tenant_id)
|
||||||
|
|
||||||
|
|
||||||
|
def require_scope(required_scope: str):
|
||||||
|
def dependency(principal: ApiPrincipal = Depends(get_api_principal)) -> ApiPrincipal:
|
||||||
|
if not has_scope(principal.api_key, required_scope):
|
||||||
|
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {required_scope}")
|
||||||
|
return principal
|
||||||
|
|
||||||
|
return dependency
|
||||||
81
server/app/celery_app.py
Normal file
81
server/app/celery_app.py
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from celery import Celery
|
||||||
|
|
||||||
|
from .settings import settings
|
||||||
|
|
||||||
|
celery = Celery(
|
||||||
|
"multimailer",
|
||||||
|
broker=settings.redis_url,
|
||||||
|
backend=settings.redis_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
celery.conf.update(
|
||||||
|
task_default_queue="default",
|
||||||
|
task_routes={
|
||||||
|
"multimailer.send_email": {"queue": "send_email"},
|
||||||
|
"multimailer.append_sent": {"queue": "append_sent"},
|
||||||
|
},
|
||||||
|
worker_prefetch_multiplier=1,
|
||||||
|
task_acks_late=True,
|
||||||
|
task_reject_on_worker_lost=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@celery.task(name="multimailer.ping")
|
||||||
|
def ping():
|
||||||
|
return "pong"
|
||||||
|
|
||||||
|
|
||||||
|
@celery.task(name="multimailer.send_email", bind=True, max_retries=None)
|
||||||
|
def send_email(self, job_id: str):
|
||||||
|
"""Send one queued campaign job.
|
||||||
|
|
||||||
|
The task records all state changes in the database. Temporary SMTP/network
|
||||||
|
failures are retried with the campaign's configured backoff.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from app.db.models import CampaignVersion, JobSendStatus
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.persistence.campaigns import load_version_config
|
||||||
|
from app.mailer.sending.jobs import SendJobError, next_retry_delay, send_campaign_job
|
||||||
|
from app.mailer.sending.smtp import SmtpSendError
|
||||||
|
|
||||||
|
with SessionLocal() as session:
|
||||||
|
try:
|
||||||
|
return send_campaign_job(session, job_id=job_id, enqueue_imap_task=True).as_dict()
|
||||||
|
except SmtpSendError as exc:
|
||||||
|
# send_campaign_job has already updated the job attempt/status.
|
||||||
|
from app.db.models import CampaignJob
|
||||||
|
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if job and job.send_status == JobSendStatus.FAILED_TEMPORARY.value:
|
||||||
|
version = session.get(CampaignVersion, job.campaign_version_id)
|
||||||
|
delay = 60
|
||||||
|
if version:
|
||||||
|
try:
|
||||||
|
_, _, config = load_version_config(session, version.id)
|
||||||
|
delay = next_retry_delay(config, job.attempt_count)
|
||||||
|
except Exception:
|
||||||
|
delay = 60
|
||||||
|
raise self.retry(exc=exc, countdown=delay)
|
||||||
|
raise
|
||||||
|
except SendJobError:
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@celery.task(name="multimailer.append_sent", bind=True, max_retries=None)
|
||||||
|
def append_sent(self, job_id: str):
|
||||||
|
"""Append the exact sent MIME to the configured IMAP Sent folder."""
|
||||||
|
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.sending.imap import ImapAppendError
|
||||||
|
from app.mailer.sending.jobs import append_sent_for_job
|
||||||
|
|
||||||
|
with SessionLocal() as session:
|
||||||
|
try:
|
||||||
|
return append_sent_for_job(session, job_id=job_id).as_dict()
|
||||||
|
except ImapAppendError as exc:
|
||||||
|
if getattr(exc, "temporary", None) is True:
|
||||||
|
raise self.retry(exc=exc, countdown=300)
|
||||||
|
raise
|
||||||
0
server/app/core/__init__.py
Normal file
0
server/app/core/__init__.py
Normal file
0
server/app/db/__init__.py
Normal file
0
server/app/db/__init__.py
Normal file
36
server/app/db/base.py
Normal file
36
server/app/db/base.py
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import MetaData
|
||||||
|
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
|
||||||
|
from sqlalchemy.types import DateTime
|
||||||
|
|
||||||
|
|
||||||
|
NAMING_CONVENTION = {
|
||||||
|
"ix": "ix_%(column_0_label)s",
|
||||||
|
"uq": "uq_%(table_name)s_%(column_0_name)s",
|
||||||
|
"ck": "ck_%(table_name)s_%(constraint_name)s",
|
||||||
|
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
|
||||||
|
"pk": "pk_%(table_name)s",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def utcnow() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
class Base(DeclarativeBase):
|
||||||
|
metadata = MetaData(naming_convention=NAMING_CONVENTION)
|
||||||
|
|
||||||
|
type_annotation_map = {
|
||||||
|
dict[str, Any]: __import__("sqlalchemy").JSON,
|
||||||
|
list[dict[str, Any]]: __import__("sqlalchemy").JSON,
|
||||||
|
list[str]: __import__("sqlalchemy").JSON,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class TimestampMixin:
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, nullable=False)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow, nullable=False)
|
||||||
96
server/app/db/bootstrap.py
Normal file
96
server/app/db/bootstrap.py
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.base import Base
|
||||||
|
from app.db.models import Role, Tenant, User
|
||||||
|
from app.db.session import engine
|
||||||
|
from app.security.api_keys import CreatedApiKey, create_api_key
|
||||||
|
|
||||||
|
DEFAULT_SCOPES = [
|
||||||
|
"campaign:read",
|
||||||
|
"campaign:write",
|
||||||
|
"campaign:validate",
|
||||||
|
"campaign:build",
|
||||||
|
"campaign:queue",
|
||||||
|
"campaign:send_test",
|
||||||
|
"campaign:send",
|
||||||
|
"attachments:read",
|
||||||
|
"attachments:write",
|
||||||
|
"reports:read",
|
||||||
|
"reports:send",
|
||||||
|
"audit:read",
|
||||||
|
"admin:users",
|
||||||
|
"admin:settings",
|
||||||
|
]
|
||||||
|
|
||||||
|
DEFAULT_ROLES = {
|
||||||
|
"owner": ["*"],
|
||||||
|
"admin": [
|
||||||
|
"campaign:read", "campaign:write", "campaign:validate", "campaign:build",
|
||||||
|
"campaign:queue", "campaign:send_test", "campaign:send",
|
||||||
|
"attachments:read", "attachments:write", "reports:read", "reports:send", "audit:read",
|
||||||
|
"admin:users", "admin:settings",
|
||||||
|
],
|
||||||
|
"campaign_manager": ["campaign:read", "campaign:write", "campaign:validate", "campaign:build", "reports:read"],
|
||||||
|
"sender": ["campaign:read", "campaign:queue", "campaign:send_test", "campaign:send", "reports:read", "reports:send"],
|
||||||
|
"reviewer": ["campaign:read", "campaign:validate", "reports:read"],
|
||||||
|
"viewer": ["campaign:read", "reports:read"],
|
||||||
|
"auditor": ["campaign:read", "reports:read", "audit:read"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class BootstrapResult:
|
||||||
|
tenant: Tenant
|
||||||
|
user: User
|
||||||
|
created_api_key: CreatedApiKey | None
|
||||||
|
|
||||||
|
|
||||||
|
def create_all_tables() -> None:
|
||||||
|
# Import models so SQLAlchemy sees all tables.
|
||||||
|
from app.db import models # noqa: F401
|
||||||
|
|
||||||
|
Base.metadata.create_all(bind=engine)
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap_dev_data(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
api_key_secret: str | None = None,
|
||||||
|
tenant_slug: str = "default",
|
||||||
|
user_email: str = "admin@example.local",
|
||||||
|
) -> BootstrapResult:
|
||||||
|
tenant = session.query(Tenant).filter(Tenant.slug == tenant_slug).one_or_none()
|
||||||
|
if tenant is None:
|
||||||
|
tenant = Tenant(slug=tenant_slug, name="Default Tenant")
|
||||||
|
session.add(tenant)
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
for slug, permissions in DEFAULT_ROLES.items():
|
||||||
|
role = session.query(Role).filter(Role.tenant_id == tenant.id, Role.slug == slug).one_or_none()
|
||||||
|
if role is None:
|
||||||
|
session.add(Role(tenant_id=tenant.id, slug=slug, name=slug.replace("_", " ").title(), permissions=permissions))
|
||||||
|
|
||||||
|
user = session.query(User).filter(User.tenant_id == tenant.id, User.email == user_email).one_or_none()
|
||||||
|
if user is None:
|
||||||
|
user = User(tenant_id=tenant.id, email=user_email, display_name="Development Admin", is_tenant_admin=True)
|
||||||
|
session.add(user)
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
created_api_key = None
|
||||||
|
if api_key_secret:
|
||||||
|
existing = [key for key in user.api_keys if key.name == "Development API key" and key.revoked_at is None]
|
||||||
|
if not existing:
|
||||||
|
created_api_key = create_api_key(
|
||||||
|
session,
|
||||||
|
user=user,
|
||||||
|
name="Development API key",
|
||||||
|
scopes=["*"],
|
||||||
|
secret=api_key_secret,
|
||||||
|
)
|
||||||
|
|
||||||
|
session.commit()
|
||||||
|
return BootstrapResult(tenant=tenant, user=user, created_api_key=created_api_key)
|
||||||
335
server/app/db/models.py
Normal file
335
server/app/db/models.py
Normal file
@@ -0,0 +1,335 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime
|
||||||
|
from enum import StrEnum
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, JSON
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
|
from .base import Base, TimestampMixin
|
||||||
|
|
||||||
|
|
||||||
|
def new_uuid() -> str:
|
||||||
|
return str(uuid.uuid4())
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignStatus(StrEnum):
|
||||||
|
DRAFT = "draft"
|
||||||
|
VALIDATED = "validated"
|
||||||
|
NEEDS_REVIEW = "needs_review"
|
||||||
|
READY_TO_QUEUE = "ready_to_queue"
|
||||||
|
QUEUED = "queued"
|
||||||
|
SENDING = "sending"
|
||||||
|
SENT = "sent"
|
||||||
|
FAILED = "failed"
|
||||||
|
CANCELLED = "cancelled"
|
||||||
|
ARCHIVED = "archived"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionWorkflowState(StrEnum):
|
||||||
|
EDITING = "editing"
|
||||||
|
UNDER_REVIEW = "under_review"
|
||||||
|
APPROVED = "approved"
|
||||||
|
BUILT = "built"
|
||||||
|
QUEUED = "queued"
|
||||||
|
SENDING = "sending"
|
||||||
|
COMPLETED = "completed"
|
||||||
|
CANCELLED = "cancelled"
|
||||||
|
ARCHIVED = "archived"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersionFlow(StrEnum):
|
||||||
|
CREATE = "create"
|
||||||
|
REVIEW = "review"
|
||||||
|
SEND = "send"
|
||||||
|
MANUAL = "manual"
|
||||||
|
JSON = "json"
|
||||||
|
|
||||||
|
|
||||||
|
class JobBuildStatus(StrEnum):
|
||||||
|
PENDING = "pending"
|
||||||
|
BUILT = "built"
|
||||||
|
BUILD_FAILED = "build_failed"
|
||||||
|
|
||||||
|
|
||||||
|
class JobValidationStatus(StrEnum):
|
||||||
|
READY = "ready"
|
||||||
|
WARNING = "warning"
|
||||||
|
NEEDS_REVIEW = "needs_review"
|
||||||
|
BLOCKED = "blocked"
|
||||||
|
EXCLUDED = "excluded"
|
||||||
|
INACTIVE = "inactive"
|
||||||
|
|
||||||
|
|
||||||
|
class JobQueueStatus(StrEnum):
|
||||||
|
DRAFT = "draft"
|
||||||
|
QUEUED = "queued"
|
||||||
|
SENDING = "sending"
|
||||||
|
PAUSED = "paused"
|
||||||
|
CANCELLED = "cancelled"
|
||||||
|
|
||||||
|
|
||||||
|
class JobSendStatus(StrEnum):
|
||||||
|
NOT_QUEUED = "not_queued"
|
||||||
|
QUEUED = "queued"
|
||||||
|
SENDING = "sending"
|
||||||
|
SENT = "sent"
|
||||||
|
FAILED_TEMPORARY = "failed_temporary"
|
||||||
|
FAILED_PERMANENT = "failed_permanent"
|
||||||
|
CANCELLED = "cancelled"
|
||||||
|
|
||||||
|
|
||||||
|
class JobImapStatus(StrEnum):
|
||||||
|
NOT_REQUESTED = "not_requested"
|
||||||
|
PENDING = "pending"
|
||||||
|
APPENDED = "appended"
|
||||||
|
FAILED = "failed"
|
||||||
|
SKIPPED = "skipped"
|
||||||
|
|
||||||
|
|
||||||
|
class IssueSeverity(StrEnum):
|
||||||
|
INFO = "info"
|
||||||
|
WARNING = "warning"
|
||||||
|
ERROR = "error"
|
||||||
|
|
||||||
|
|
||||||
|
class Tenant(Base, TimestampMixin):
|
||||||
|
__tablename__ = "tenants"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
slug: Mapped[str] = mapped_column(String(100), unique=True, nullable=False, index=True)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
|
||||||
|
|
||||||
|
users: Mapped[list[User]] = relationship(back_populates="tenant", cascade="all, delete-orphan")
|
||||||
|
campaigns: Mapped[list[Campaign]] = relationship(back_populates="tenant", cascade="all, delete-orphan")
|
||||||
|
|
||||||
|
|
||||||
|
class User(Base, TimestampMixin):
|
||||||
|
__tablename__ = "users"
|
||||||
|
__table_args__ = (UniqueConstraint("tenant_id", "email", name="uq_users_tenant_email"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
email: Mapped[str] = mapped_column(String(320), nullable=False, index=True)
|
||||||
|
display_name: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
|
||||||
|
is_tenant_admin: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||||
|
|
||||||
|
tenant: Mapped[Tenant] = relationship(back_populates="users")
|
||||||
|
api_keys: Mapped[list[ApiKey]] = relationship(back_populates="user", cascade="all, delete-orphan")
|
||||||
|
|
||||||
|
|
||||||
|
class Group(Base, TimestampMixin):
|
||||||
|
__tablename__ = "groups"
|
||||||
|
__table_args__ = (UniqueConstraint("tenant_id", "slug", name="uq_groups_tenant_slug"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
slug: Mapped[str] = mapped_column(String(100), nullable=False)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class Role(Base, TimestampMixin):
|
||||||
|
__tablename__ = "roles"
|
||||||
|
__table_args__ = (UniqueConstraint("tenant_id", "slug", name="uq_roles_tenant_slug"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str | None] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=True, index=True)
|
||||||
|
slug: Mapped[str] = mapped_column(String(100), nullable=False)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
permissions: Mapped[list[str]] = mapped_column(JSON, default=list)
|
||||||
|
|
||||||
|
|
||||||
|
class ApiKey(Base, TimestampMixin):
|
||||||
|
__tablename__ = "api_keys"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
user_id: Mapped[str] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
prefix: Mapped[str] = mapped_column(String(16), nullable=False, index=True)
|
||||||
|
key_hash: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
scopes: Mapped[list[str]] = mapped_column(JSON, default=list)
|
||||||
|
expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
|
user: Mapped[User] = relationship(back_populates="api_keys")
|
||||||
|
|
||||||
|
|
||||||
|
class Campaign(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaigns"
|
||||||
|
__table_args__ = (UniqueConstraint("tenant_id", "external_id", name="uq_campaigns_tenant_external_id"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||||
|
external_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
description: Mapped[str | None] = mapped_column(Text)
|
||||||
|
status: Mapped[str] = mapped_column(String(50), default=CampaignStatus.DRAFT.value, nullable=False, index=True)
|
||||||
|
current_version_id: Mapped[str | None] = mapped_column(String(36), nullable=True)
|
||||||
|
|
||||||
|
tenant: Mapped[Tenant] = relationship(back_populates="campaigns")
|
||||||
|
versions: Mapped[list[CampaignVersion]] = relationship(back_populates="campaign", cascade="all, delete-orphan")
|
||||||
|
jobs: Mapped[list[CampaignJob]] = relationship(back_populates="campaign", cascade="all, delete-orphan")
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignVersion(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_versions"
|
||||||
|
__table_args__ = (UniqueConstraint("campaign_id", "version_number", name="uq_campaign_versions_campaign_number"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
version_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
|
||||||
|
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
|
||||||
|
source_filename: Mapped[str | None] = mapped_column(String(500))
|
||||||
|
source_base_path: Mapped[str | None] = mapped_column(String(1000))
|
||||||
|
|
||||||
|
# Editor/workflow metadata used by the WebUI and future desktop clients.
|
||||||
|
# A campaign version can be the autosaved working copy of a new or existing
|
||||||
|
# campaign, so no separate CampaignDraft entity is needed.
|
||||||
|
workflow_state: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
default=CampaignVersionWorkflowState.EDITING.value,
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
current_flow: Mapped[str] = mapped_column(
|
||||||
|
String(50),
|
||||||
|
default=CampaignVersionFlow.MANUAL.value,
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
current_step: Mapped[str | None] = mapped_column(String(100))
|
||||||
|
is_complete: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||||
|
editor_state: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
|
||||||
|
autosaved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
locked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
locked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||||
|
|
||||||
|
validation_summary: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
|
build_summary: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
|
|
||||||
|
campaign: Mapped[Campaign] = relationship(back_populates="versions")
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignJob(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_jobs"
|
||||||
|
__table_args__ = (UniqueConstraint("campaign_version_id", "entry_index", name="uq_campaign_jobs_version_entry"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
campaign_version_id: Mapped[str] = mapped_column(ForeignKey("campaign_versions.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
entry_index: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
entry_id: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||||
|
|
||||||
|
recipient_email: Mapped[str | None] = mapped_column(String(320), index=True)
|
||||||
|
subject: Mapped[str | None] = mapped_column(String(998))
|
||||||
|
message_id_header: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
eml_storage_key: Mapped[str | None] = mapped_column(String(1000))
|
||||||
|
eml_local_path: Mapped[str | None] = mapped_column(String(1000))
|
||||||
|
eml_size_bytes: Mapped[int | None] = mapped_column(Integer)
|
||||||
|
|
||||||
|
build_status: Mapped[str] = mapped_column(String(50), default=JobBuildStatus.PENDING.value, nullable=False, index=True)
|
||||||
|
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
|
||||||
|
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
|
||||||
|
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
|
||||||
|
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
|
||||||
|
|
||||||
|
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||||
|
last_error: Mapped[str | None] = mapped_column(Text)
|
||||||
|
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
|
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
|
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||||
|
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
|
||||||
|
|
||||||
|
campaign: Mapped[Campaign] = relationship(back_populates="jobs")
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignIssue(Base, TimestampMixin):
|
||||||
|
__tablename__ = "campaign_issues"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
campaign_version_id: Mapped[str | None] = mapped_column(ForeignKey("campaign_versions.id", ondelete="CASCADE"), nullable=True, index=True)
|
||||||
|
job_id: Mapped[str | None] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=True, index=True)
|
||||||
|
severity: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
|
||||||
|
code: Mapped[str] = mapped_column(String(100), nullable=False, index=True)
|
||||||
|
message: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
source: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
behavior: Mapped[str | None] = mapped_column(String(50))
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentBlob(Base, TimestampMixin):
|
||||||
|
__tablename__ = "attachment_blobs"
|
||||||
|
__table_args__ = (UniqueConstraint("tenant_id", "sha256", name="uq_attachment_blobs_tenant_sha256"),)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
sha256: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||||
|
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
mime_type: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
storage_bucket: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
storage_key: Mapped[str] = mapped_column(String(1000), nullable=False)
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentInstance(Base, TimestampMixin):
|
||||||
|
__tablename__ = "attachment_instances"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
owner_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||||
|
campaign_id: Mapped[str | None] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=True, index=True)
|
||||||
|
blob_id: Mapped[str] = mapped_column(ForeignKey("attachment_blobs.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
logical_name: Mapped[str | None] = mapped_column(String(500))
|
||||||
|
filename: Mapped[str] = mapped_column(String(500), nullable=False)
|
||||||
|
tags: Mapped[list[str]] = mapped_column(JSON, default=list)
|
||||||
|
metadata_: Mapped[dict[str, Any] | None] = mapped_column("metadata", JSON, nullable=True)
|
||||||
|
|
||||||
|
|
||||||
|
class SendAttempt(Base, TimestampMixin):
|
||||||
|
__tablename__ = "send_attempts"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
job_id: Mapped[str] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
smtp_status_code: Mapped[int | None] = mapped_column(Integer)
|
||||||
|
smtp_response: Mapped[str | None] = mapped_column(Text)
|
||||||
|
error_type: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
error_message: Mapped[str | None] = mapped_column(Text)
|
||||||
|
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
|
|
||||||
|
class ImapAppendAttempt(Base, TimestampMixin):
|
||||||
|
__tablename__ = "imap_append_attempts"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
job_id: Mapped[str] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||||
|
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
folder: Mapped[str | None] = mapped_column(String(500))
|
||||||
|
status: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||||
|
error_message: Mapped[str | None] = mapped_column(Text)
|
||||||
|
|
||||||
|
|
||||||
|
class AuditLog(Base, TimestampMixin):
|
||||||
|
__tablename__ = "audit_log"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||||
|
tenant_id: Mapped[str | None] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=True, index=True)
|
||||||
|
user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||||
|
api_key_id: Mapped[str | None] = mapped_column(ForeignKey("api_keys.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||||
|
action: Mapped[str] = mapped_column(String(100), nullable=False, index=True)
|
||||||
|
object_type: Mapped[str | None] = mapped_column(String(100), index=True)
|
||||||
|
object_id: Mapped[str | None] = mapped_column(String(100), index=True)
|
||||||
|
details: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
|
||||||
17
server/app/db/session.py
Normal file
17
server/app/db/session.py
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Generator
|
||||||
|
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import Session, sessionmaker
|
||||||
|
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
connect_args = {"check_same_thread": False} if settings.database_url.startswith("sqlite") else {}
|
||||||
|
engine = create_engine(settings.database_url, pool_pre_ping=True, connect_args=connect_args)
|
||||||
|
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, expire_on_commit=False)
|
||||||
|
|
||||||
|
|
||||||
|
def get_session() -> Generator[Session, None, None]:
|
||||||
|
with SessionLocal() as session:
|
||||||
|
yield session
|
||||||
0
server/app/mailer/__init__.py
Normal file
0
server/app/mailer/__init__.py
Normal file
0
server/app/mailer/attachments/__init__.py
Normal file
0
server/app/mailer/attachments/__init__.py
Normal file
318
server/app/mailer/attachments/resolver.py
Normal file
318
server/app/mailer/attachments/resolver.py
Normal file
@@ -0,0 +1,318 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import fnmatch
|
||||||
|
from enum import StrEnum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from app.mailer.campaign.entries import load_campaign_entries
|
||||||
|
from app.mailer.campaign.models import AttachmentConfig, Behavior, CampaignConfig, EntryConfig
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentScope(StrEnum):
|
||||||
|
GLOBAL = "global"
|
||||||
|
ENTRY = "entry"
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentMatchStatus(StrEnum):
|
||||||
|
OK = "ok"
|
||||||
|
MISSING = "missing"
|
||||||
|
AMBIGUOUS = "ambiguous"
|
||||||
|
|
||||||
|
|
||||||
|
class MessageAttachmentStatus(StrEnum):
|
||||||
|
READY = "ready"
|
||||||
|
WARNING = "warning"
|
||||||
|
NEEDS_REVIEW = "needs_review"
|
||||||
|
BLOCKED = "blocked"
|
||||||
|
EXCLUDED = "excluded"
|
||||||
|
INACTIVE = "inactive"
|
||||||
|
|
||||||
|
|
||||||
|
class ResolutionSeverity(StrEnum):
|
||||||
|
INFO = "info"
|
||||||
|
WARNING = "warning"
|
||||||
|
ERROR = "error"
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentIssue(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
severity: ResolutionSeverity
|
||||||
|
code: str
|
||||||
|
message: str
|
||||||
|
behavior: Behavior | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class ResolvedAttachment(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
scope: AttachmentScope
|
||||||
|
index: int
|
||||||
|
attachment_id: str | None = None
|
||||||
|
label: str | None = None
|
||||||
|
base_dir_template: str
|
||||||
|
file_filter_template: str
|
||||||
|
base_dir: str
|
||||||
|
file_filter: str
|
||||||
|
directory: str
|
||||||
|
include_subdirs: bool
|
||||||
|
required: bool
|
||||||
|
allow_multiple: bool
|
||||||
|
zip_enabled: bool
|
||||||
|
status: AttachmentMatchStatus
|
||||||
|
behavior: Behavior | None = None
|
||||||
|
matches: list[str] = Field(default_factory=list)
|
||||||
|
issues: list[AttachmentIssue] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class EntryAttachmentResolution(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
entry_index: int
|
||||||
|
entry_id: str | None = None
|
||||||
|
active: bool
|
||||||
|
status: MessageAttachmentStatus
|
||||||
|
attachments: list[ResolvedAttachment] = Field(default_factory=list)
|
||||||
|
issues: list[AttachmentIssue] = Field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def match_count(self) -> int:
|
||||||
|
return sum(len(item.matches) for item in self.attachments)
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentResolutionReport(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaign_id: str
|
||||||
|
campaign_name: str
|
||||||
|
campaign_file: str
|
||||||
|
attachments_base_path: str
|
||||||
|
entries_count: int
|
||||||
|
entries: list[EntryAttachmentResolution] = Field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ready_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.READY)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def warning_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.WARNING)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def needs_review_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.NEEDS_REVIEW)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def blocked_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.BLOCKED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def excluded_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.EXCLUDED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def inactive_count(self) -> int:
|
||||||
|
return sum(1 for entry in self.entries if entry.status == MessageAttachmentStatus.INACTIVE)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_path(campaign_file: str | Path, raw_path: str) -> Path:
|
||||||
|
campaign_path = Path(campaign_file).resolve()
|
||||||
|
path = Path(raw_path).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return path
|
||||||
|
return (campaign_path.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
|
def _render_template(template: str, values: dict[str, Any]) -> str:
|
||||||
|
rendered = template
|
||||||
|
for key, value in values.items():
|
||||||
|
rendered = rendered.replace("${" + key + "}", "" if value is None else str(value))
|
||||||
|
return rendered
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_values(entry: EntryConfig) -> dict[str, str]:
|
||||||
|
values: dict[str, str] = {}
|
||||||
|
for list_name in ["to", "cc", "bcc", "reply_to", "bounce_to", "disposition_notification_to"]:
|
||||||
|
recipients = getattr(entry, list_name)
|
||||||
|
for index, recipient in enumerate(recipients):
|
||||||
|
prefix = f"{list_name}.{index}"
|
||||||
|
values[f"local::{prefix}.email"] = recipient.email
|
||||||
|
values[f"local::{prefix}.name"] = recipient.name or ""
|
||||||
|
values[f"local::{prefix}.type"] = recipient.recipient_type.value
|
||||||
|
if entry.from_:
|
||||||
|
values["local::from.email"] = entry.from_.email
|
||||||
|
values["local::from.name"] = entry.from_.name or ""
|
||||||
|
values["local::from.type"] = entry.from_.recipient_type.value
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def _template_values(config: CampaignConfig, entry: EntryConfig) -> dict[str, Any]:
|
||||||
|
values: dict[str, Any] = {}
|
||||||
|
for key, value in config.global_values.items():
|
||||||
|
values[f"global::{key}"] = value
|
||||||
|
for key, value in entry.fields.items():
|
||||||
|
values[f"local::{key}"] = value
|
||||||
|
if entry.id:
|
||||||
|
values["local::id"] = entry.id
|
||||||
|
values["local::active"] = entry.active
|
||||||
|
values.update(_recipient_values(entry))
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_effective_attachment_configs(config: CampaignConfig, entry: EntryConfig) -> Iterable[tuple[AttachmentScope, int, AttachmentConfig]]:
|
||||||
|
if entry.combine_attachments:
|
||||||
|
for index, attachment_config in enumerate(config.attachments.global_):
|
||||||
|
yield AttachmentScope.GLOBAL, index, attachment_config
|
||||||
|
if config.attachments.allow_individual:
|
||||||
|
for index, attachment_config in enumerate(entry.attachments):
|
||||||
|
yield AttachmentScope.ENTRY, index, attachment_config
|
||||||
|
|
||||||
|
|
||||||
|
def _match_files(directory: Path, file_filter: str, include_subdirs: bool) -> list[Path]:
|
||||||
|
if not directory.exists() or not directory.is_dir():
|
||||||
|
return []
|
||||||
|
if include_subdirs:
|
||||||
|
# pathlib.rglob accepts glob patterns, but fnmatch keeps behavior predictable
|
||||||
|
# when file_filter is supplied as the Java-style filter portion only.
|
||||||
|
return sorted(path for path in directory.rglob("*") if path.is_file() and fnmatch.fnmatch(path.name, file_filter))
|
||||||
|
return sorted(path for path in directory.glob(file_filter) if path.is_file())
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_for_missing(config: AttachmentConfig, behavior: Behavior) -> AttachmentIssue:
|
||||||
|
code = "missing_required_attachment" if config.required else "missing_optional_attachment"
|
||||||
|
severity = ResolutionSeverity.ERROR if config.required and behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
||||||
|
return AttachmentIssue(
|
||||||
|
severity=severity,
|
||||||
|
code=code,
|
||||||
|
message=f"No file matched attachment filter {config.file_filter!r}",
|
||||||
|
behavior=behavior,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_for_ambiguous(config: AttachmentConfig, behavior: Behavior, match_count: int) -> AttachmentIssue:
|
||||||
|
severity = ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
|
||||||
|
return AttachmentIssue(
|
||||||
|
severity=severity,
|
||||||
|
code="ambiguous_attachment_match",
|
||||||
|
message=f"Attachment filter {config.file_filter!r} matched {match_count} files, but allow_multiple is false",
|
||||||
|
behavior=behavior,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_one_config(
|
||||||
|
*,
|
||||||
|
campaign_file: str | Path,
|
||||||
|
attachments_base_path: Path,
|
||||||
|
values: dict[str, Any],
|
||||||
|
scope: AttachmentScope,
|
||||||
|
index: int,
|
||||||
|
config: AttachmentConfig,
|
||||||
|
) -> ResolvedAttachment:
|
||||||
|
rendered_base_dir = _render_template(config.base_dir, values)
|
||||||
|
rendered_file_filter = _render_template(config.file_filter, values)
|
||||||
|
directory = (attachments_base_path / rendered_base_dir).resolve()
|
||||||
|
matches = _match_files(directory, rendered_file_filter, config.include_subdirs)
|
||||||
|
|
||||||
|
issues: list[AttachmentIssue] = []
|
||||||
|
behavior: Behavior | None = None
|
||||||
|
|
||||||
|
if not matches:
|
||||||
|
status = AttachmentMatchStatus.MISSING
|
||||||
|
behavior = config.missing_behavior
|
||||||
|
issues.append(_issue_for_missing(config, behavior))
|
||||||
|
elif len(matches) > 1 and not config.allow_multiple:
|
||||||
|
status = AttachmentMatchStatus.AMBIGUOUS
|
||||||
|
behavior = config.ambiguous_behavior
|
||||||
|
issues.append(_issue_for_ambiguous(config, behavior, len(matches)))
|
||||||
|
else:
|
||||||
|
status = AttachmentMatchStatus.OK
|
||||||
|
|
||||||
|
return ResolvedAttachment(
|
||||||
|
scope=scope,
|
||||||
|
index=index,
|
||||||
|
attachment_id=config.id,
|
||||||
|
label=config.label,
|
||||||
|
base_dir_template=config.base_dir,
|
||||||
|
file_filter_template=config.file_filter,
|
||||||
|
base_dir=rendered_base_dir,
|
||||||
|
file_filter=rendered_file_filter,
|
||||||
|
directory=str(directory),
|
||||||
|
include_subdirs=config.include_subdirs,
|
||||||
|
required=config.required,
|
||||||
|
allow_multiple=config.allow_multiple,
|
||||||
|
zip_enabled=config.zip.enabled,
|
||||||
|
status=status,
|
||||||
|
behavior=behavior,
|
||||||
|
matches=[str(path) for path in matches],
|
||||||
|
issues=issues,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _status_from_issues(active: bool, issues: list[AttachmentIssue]) -> MessageAttachmentStatus:
|
||||||
|
if not active:
|
||||||
|
return MessageAttachmentStatus.INACTIVE
|
||||||
|
behaviors = {issue.behavior for issue in issues if issue.behavior is not None}
|
||||||
|
if Behavior.BLOCK in behaviors:
|
||||||
|
return MessageAttachmentStatus.BLOCKED
|
||||||
|
if Behavior.DROP in behaviors:
|
||||||
|
return MessageAttachmentStatus.EXCLUDED
|
||||||
|
if Behavior.ASK in behaviors:
|
||||||
|
return MessageAttachmentStatus.NEEDS_REVIEW
|
||||||
|
if Behavior.WARN in behaviors:
|
||||||
|
return MessageAttachmentStatus.WARNING
|
||||||
|
return MessageAttachmentStatus.READY
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_entry_attachments(
|
||||||
|
*,
|
||||||
|
config: CampaignConfig,
|
||||||
|
campaign_file: str | Path,
|
||||||
|
entry: EntryConfig,
|
||||||
|
entry_index: int,
|
||||||
|
) -> EntryAttachmentResolution:
|
||||||
|
attachments_base_path = _resolve_path(campaign_file, config.attachments.base_path)
|
||||||
|
values = _template_values(config, entry)
|
||||||
|
resolved: list[ResolvedAttachment] = []
|
||||||
|
|
||||||
|
if entry.active:
|
||||||
|
for scope, index, attachment_config in _iter_effective_attachment_configs(config, entry):
|
||||||
|
resolved.append(
|
||||||
|
_resolve_one_config(
|
||||||
|
campaign_file=campaign_file,
|
||||||
|
attachments_base_path=attachments_base_path,
|
||||||
|
values=values,
|
||||||
|
scope=scope,
|
||||||
|
index=index,
|
||||||
|
config=attachment_config,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
issues = [issue for item in resolved for issue in item.issues]
|
||||||
|
return EntryAttachmentResolution(
|
||||||
|
entry_index=entry_index,
|
||||||
|
entry_id=entry.id,
|
||||||
|
active=entry.active,
|
||||||
|
status=_status_from_issues(entry.active, issues),
|
||||||
|
attachments=resolved,
|
||||||
|
issues=issues,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_campaign_attachments(config: CampaignConfig, *, campaign_file: str | Path) -> AttachmentResolutionReport:
|
||||||
|
entries = load_campaign_entries(config, campaign_file=campaign_file)
|
||||||
|
base_path = _resolve_path(campaign_file, config.attachments.base_path)
|
||||||
|
resolved_entries = [
|
||||||
|
resolve_entry_attachments(config=config, campaign_file=campaign_file, entry=entry, entry_index=index)
|
||||||
|
for index, entry in enumerate(entries, start=1)
|
||||||
|
]
|
||||||
|
return AttachmentResolutionReport(
|
||||||
|
campaign_id=config.campaign.id,
|
||||||
|
campaign_name=config.campaign.name,
|
||||||
|
campaign_file=str(Path(campaign_file).resolve()),
|
||||||
|
attachments_base_path=str(base_path),
|
||||||
|
entries_count=len(entries),
|
||||||
|
entries=resolved_entries,
|
||||||
|
)
|
||||||
14
server/app/mailer/campaign/__init__.py
Normal file
14
server/app/mailer/campaign/__init__.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
"""Campaign JSON model, loading and validation helpers."""
|
||||||
|
|
||||||
|
from .models import CampaignConfig
|
||||||
|
from .loader import load_campaign_config, load_campaign_json
|
||||||
|
from .validation import validate_campaign_config, SemanticIssue, SemanticReport
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"CampaignConfig",
|
||||||
|
"load_campaign_config",
|
||||||
|
"load_campaign_json",
|
||||||
|
"validate_campaign_config",
|
||||||
|
"SemanticIssue",
|
||||||
|
"SemanticReport",
|
||||||
|
]
|
||||||
215
server/app/mailer/campaign/entries.py
Normal file
215
server/app/mailer/campaign/entries.py
Normal file
@@ -0,0 +1,215 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import csv
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
from .models import AttachmentConfig, CampaignConfig, EntryConfig, RecipientConfig, SourceType
|
||||||
|
|
||||||
|
|
||||||
|
class EntryLoadError(ValueError):
|
||||||
|
"""Raised when campaign entries cannot be loaded from inline or external sources."""
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve(campaign_file: str | Path, raw_path: str) -> Path:
|
||||||
|
campaign_path = Path(campaign_file).resolve()
|
||||||
|
path = Path(raw_path).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return path
|
||||||
|
return (campaign_path.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_bool(value: Any) -> bool:
|
||||||
|
if isinstance(value, bool):
|
||||||
|
return value
|
||||||
|
if value is None:
|
||||||
|
return False
|
||||||
|
text = str(value).strip().lower()
|
||||||
|
if text in {"1", "true", "yes", "y", "ja", "j", "x", "active", "aktiv"}:
|
||||||
|
return True
|
||||||
|
if text in {"0", "false", "no", "n", "nein", "", "inactive", "inaktiv"}:
|
||||||
|
return False
|
||||||
|
raise EntryLoadError(f"cannot parse boolean value: {value!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_scalar_for_target(target: str, value: Any) -> Any:
|
||||||
|
bool_targets = {
|
||||||
|
"active",
|
||||||
|
"combine_to",
|
||||||
|
"combine_cc",
|
||||||
|
"combine_bcc",
|
||||||
|
"combine_reply_to",
|
||||||
|
"combine_bounce_to",
|
||||||
|
"combine_disposition_notification_to",
|
||||||
|
"combine_attachments",
|
||||||
|
}
|
||||||
|
if target in bool_targets:
|
||||||
|
return _parse_bool(value)
|
||||||
|
if target.endswith(".include_subdirs") or target.endswith(".required") or target.endswith(".allow_multiple"):
|
||||||
|
return _parse_bool(value)
|
||||||
|
if target.endswith(".zip.enabled"):
|
||||||
|
return _parse_bool(value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_list_length(values: list[Any], index: int, factory: Any) -> None:
|
||||||
|
while len(values) <= index:
|
||||||
|
values.append(factory())
|
||||||
|
|
||||||
|
|
||||||
|
def _set_recipient_value(entry_data: dict[str, Any], target: str, value: Any) -> bool:
|
||||||
|
# Examples: from.email, to.0.email, cc.0.name
|
||||||
|
if target.startswith("from."):
|
||||||
|
entry_data.setdefault("from", {})
|
||||||
|
_, field = target.split(".", 1)
|
||||||
|
if field == "type":
|
||||||
|
field = "type"
|
||||||
|
entry_data["from"][field] = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
for recipient_list_name in ["to", "cc", "bcc", "reply_to", "bounce_to", "disposition_notification_to"]:
|
||||||
|
prefix = recipient_list_name + "."
|
||||||
|
if not target.startswith(prefix):
|
||||||
|
continue
|
||||||
|
parts = target.split(".")
|
||||||
|
if len(parts) != 3 or not parts[1].isdigit():
|
||||||
|
raise EntryLoadError(f"invalid recipient mapping target: {target}")
|
||||||
|
index = int(parts[1])
|
||||||
|
field = parts[2]
|
||||||
|
recipients = entry_data.setdefault(recipient_list_name, [])
|
||||||
|
_ensure_list_length(recipients, index, dict)
|
||||||
|
recipients[index][field] = value
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _set_attachment_value(entry_data: dict[str, Any], target: str, value: Any) -> bool:
|
||||||
|
if not target.startswith("attachments."):
|
||||||
|
return False
|
||||||
|
parts = target.split(".")
|
||||||
|
if len(parts) < 3 or not parts[1].isdigit():
|
||||||
|
raise EntryLoadError(f"invalid attachment mapping target: {target}")
|
||||||
|
|
||||||
|
index = int(parts[1])
|
||||||
|
attachments = entry_data.setdefault("attachments", [])
|
||||||
|
_ensure_list_length(attachments, index, dict)
|
||||||
|
attachment = attachments[index]
|
||||||
|
|
||||||
|
if parts[2] == "zip":
|
||||||
|
if len(parts) != 4:
|
||||||
|
raise EntryLoadError(f"invalid zip attachment mapping target: {target}")
|
||||||
|
attachment.setdefault("zip", {})[parts[3]] = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
if len(parts) != 3:
|
||||||
|
raise EntryLoadError(f"invalid attachment mapping target: {target}")
|
||||||
|
attachment[parts[2]] = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _set_entry_value(entry_data: dict[str, Any], target: str, value: Any) -> None:
|
||||||
|
value = _parse_scalar_for_target(target, value)
|
||||||
|
if value is None:
|
||||||
|
return
|
||||||
|
if isinstance(value, str) and value == "":
|
||||||
|
return
|
||||||
|
|
||||||
|
if target.startswith("fields."):
|
||||||
|
_, field_name = target.split(".", 1)
|
||||||
|
entry_data.setdefault("fields", {})[field_name] = value
|
||||||
|
return
|
||||||
|
|
||||||
|
if _set_recipient_value(entry_data, target, value):
|
||||||
|
return
|
||||||
|
if _set_attachment_value(entry_data, target, value):
|
||||||
|
return
|
||||||
|
|
||||||
|
entry_data[target] = value
|
||||||
|
|
||||||
|
|
||||||
|
def _entry_defaults_data(config: CampaignConfig) -> dict[str, Any]:
|
||||||
|
if config.entries.defaults is None:
|
||||||
|
return {}
|
||||||
|
return config.entries.defaults.model_dump(mode="json", by_alias=True, exclude_none=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_csv_rows(path: Path, *, delimiter: str, encoding: str) -> list[dict[str, Any]]:
|
||||||
|
try:
|
||||||
|
with path.open("r", encoding=encoding, newline="") as handle:
|
||||||
|
reader = csv.DictReader(handle, delimiter=delimiter)
|
||||||
|
return [dict(row) for row in reader]
|
||||||
|
except OSError as exc:
|
||||||
|
raise EntryLoadError(f"could not read CSV entries source {path}: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _load_json_rows(path: Path, *, encoding: str) -> list[dict[str, Any]]:
|
||||||
|
try:
|
||||||
|
with path.open("r", encoding=encoding) as handle:
|
||||||
|
data = json.load(handle)
|
||||||
|
except OSError as exc:
|
||||||
|
raise EntryLoadError(f"could not read JSON entries source {path}: {exc}") from exc
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise EntryLoadError(f"invalid JSON entries source {path}: {exc}") from exc
|
||||||
|
|
||||||
|
if isinstance(data, list):
|
||||||
|
rows = data
|
||||||
|
elif isinstance(data, dict) and isinstance(data.get("entries"), list):
|
||||||
|
rows = data["entries"]
|
||||||
|
else:
|
||||||
|
raise EntryLoadError("JSON entries source must be a list or an object with an 'entries' list")
|
||||||
|
|
||||||
|
if not all(isinstance(row, dict) for row in rows):
|
||||||
|
raise EntryLoadError("JSON entries source rows must be objects")
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
|
||||||
|
def _row_to_entry(defaults_data: dict[str, Any], mapping: dict[str, str], row: dict[str, Any], row_number: int) -> EntryConfig:
|
||||||
|
entry_data = copy.deepcopy(defaults_data)
|
||||||
|
for target, source_name in mapping.items():
|
||||||
|
if source_name not in row:
|
||||||
|
# Detailed missing-column validation is handled in semantic validation.
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
_set_entry_value(entry_data, target, row[source_name])
|
||||||
|
except EntryLoadError:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
raise EntryLoadError(f"row {row_number}: could not map {source_name!r} to {target!r}: {exc}") from exc
|
||||||
|
try:
|
||||||
|
return EntryConfig.model_validate(entry_data)
|
||||||
|
except Exception as exc:
|
||||||
|
raise EntryLoadError(f"row {row_number}: mapped entry is invalid: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def load_campaign_entries(config: CampaignConfig, *, campaign_file: str | Path) -> list[EntryConfig]:
|
||||||
|
"""Load and normalize campaign entries from inline data or external CSV/JSON source.
|
||||||
|
|
||||||
|
The normalized output is always a list of EntryConfig instances. This is intentionally
|
||||||
|
UI/API friendly: a future web interface can generate the same JSON structure and use the
|
||||||
|
same resolver without code changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if config.entries.inline is not None:
|
||||||
|
return list(config.entries.inline)
|
||||||
|
|
||||||
|
if config.entries.source is None or config.entries.mapping is None:
|
||||||
|
raise EntryLoadError("external entries require source and mapping")
|
||||||
|
|
||||||
|
source = config.entries.source
|
||||||
|
path = _resolve(campaign_file, source.path)
|
||||||
|
if not path.exists():
|
||||||
|
raise EntryLoadError(f"entries source file does not exist: {path}")
|
||||||
|
|
||||||
|
if source.type == SourceType.CSV:
|
||||||
|
if not source.has_header:
|
||||||
|
raise EntryLoadError("CSV entries currently require has_header=true")
|
||||||
|
rows = _load_csv_rows(path, delimiter=source.delimiter, encoding=source.encoding)
|
||||||
|
elif source.type == SourceType.JSON:
|
||||||
|
rows = _load_json_rows(path, encoding=source.encoding)
|
||||||
|
else: # pragma: no cover - defensive; Pydantic constrains this already.
|
||||||
|
raise EntryLoadError(f"unsupported entries source type: {source.type}")
|
||||||
|
|
||||||
|
defaults_data = _entry_defaults_data(config)
|
||||||
|
return [_row_to_entry(defaults_data, config.entries.mapping, row, index + 2) for index, row in enumerate(rows)]
|
||||||
79
server/app/mailer/campaign/loader.py
Normal file
79
server/app/mailer/campaign/loader.py
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from jsonschema import Draft202012Validator, FormatChecker
|
||||||
|
|
||||||
|
from .models import CampaignConfig
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignLoadError(ValueError):
|
||||||
|
"""Raised when the campaign JSON cannot be loaded or parsed."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SchemaValidationError:
|
||||||
|
path: str
|
||||||
|
message: str
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignSchemaError(CampaignLoadError):
|
||||||
|
def __init__(self, errors: list[SchemaValidationError]) -> None:
|
||||||
|
self.errors = errors
|
||||||
|
details = "; ".join(f"{error.path}: {error.message}" for error in errors[:5])
|
||||||
|
if len(errors) > 5:
|
||||||
|
details += f"; ... and {len(errors) - 5} more"
|
||||||
|
super().__init__(f"campaign schema validation failed: {details}")
|
||||||
|
|
||||||
|
|
||||||
|
def load_campaign_json(path: str | Path) -> dict[str, Any]:
|
||||||
|
campaign_path = Path(path)
|
||||||
|
try:
|
||||||
|
with campaign_path.open("r", encoding="utf-8") as handle:
|
||||||
|
data = json.load(handle)
|
||||||
|
except OSError as exc:
|
||||||
|
raise CampaignLoadError(f"could not read campaign JSON {campaign_path}: {exc}") from exc
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise CampaignLoadError(f"invalid campaign JSON {campaign_path}: {exc}") from exc
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise CampaignLoadError("campaign JSON root must be an object")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _default_schema_path() -> Path:
|
||||||
|
return Path(__file__).resolve().parents[1] / "schema" / "campaign.schema.json"
|
||||||
|
|
||||||
|
|
||||||
|
def load_campaign_schema(schema_path: str | Path | None = None) -> dict[str, Any]:
|
||||||
|
path = Path(schema_path) if schema_path else _default_schema_path()
|
||||||
|
return load_campaign_json(path)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_against_schema(data: dict[str, Any], schema_path: str | Path | None = None) -> None:
|
||||||
|
schema = load_campaign_schema(schema_path)
|
||||||
|
validator = Draft202012Validator(schema, format_checker=FormatChecker())
|
||||||
|
errors = sorted(validator.iter_errors(data), key=lambda error: list(error.path))
|
||||||
|
if errors:
|
||||||
|
normalized = [
|
||||||
|
SchemaValidationError(
|
||||||
|
path="/" + "/".join(str(part) for part in error.absolute_path),
|
||||||
|
message=error.message,
|
||||||
|
)
|
||||||
|
for error in errors
|
||||||
|
]
|
||||||
|
raise CampaignSchemaError(normalized)
|
||||||
|
|
||||||
|
|
||||||
|
def load_campaign_config(
|
||||||
|
path: str | Path,
|
||||||
|
*,
|
||||||
|
validate_schema: bool = True,
|
||||||
|
schema_path: str | Path | None = None,
|
||||||
|
) -> CampaignConfig:
|
||||||
|
data = load_campaign_json(path)
|
||||||
|
if validate_schema:
|
||||||
|
validate_against_schema(data, schema_path=schema_path)
|
||||||
|
return CampaignConfig.model_validate(data)
|
||||||
363
server/app/mailer/campaign/models.py
Normal file
363
server/app/mailer/campaign/models.py
Normal file
@@ -0,0 +1,363 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from enum import StrEnum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||||
|
|
||||||
|
|
||||||
|
class StrictModel(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid", populate_by_name=True)
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignMode(StrEnum):
|
||||||
|
DRAFT = "draft"
|
||||||
|
TEST = "test"
|
||||||
|
SEND = "send"
|
||||||
|
|
||||||
|
|
||||||
|
class FieldType(StrEnum):
|
||||||
|
STRING = "string"
|
||||||
|
INTEGER = "integer"
|
||||||
|
DOUBLE = "double"
|
||||||
|
DATE = "date"
|
||||||
|
PASSWORD = "password"
|
||||||
|
|
||||||
|
|
||||||
|
class TransportSecurity(StrEnum):
|
||||||
|
PLAIN = "plain"
|
||||||
|
TLS = "tls"
|
||||||
|
STARTTLS = "starttls"
|
||||||
|
|
||||||
|
|
||||||
|
class RecipientType(StrEnum):
|
||||||
|
TO = "to"
|
||||||
|
CC = "cc"
|
||||||
|
BCC = "bcc"
|
||||||
|
REPLY_TO = "reply_to"
|
||||||
|
BOUNCE_TO = "bounce_to"
|
||||||
|
DISPOSITION_NOTIFICATION_TO = "disposition_notification_to"
|
||||||
|
|
||||||
|
|
||||||
|
class Behavior(StrEnum):
|
||||||
|
BLOCK = "block"
|
||||||
|
ASK = "ask"
|
||||||
|
DROP = "drop"
|
||||||
|
CONTINUE = "continue"
|
||||||
|
WARN = "warn"
|
||||||
|
|
||||||
|
|
||||||
|
class MissingAddressBehavior(StrEnum):
|
||||||
|
BLOCK = "block"
|
||||||
|
DROP = "drop"
|
||||||
|
|
||||||
|
|
||||||
|
class InactiveEntryBehavior(StrEnum):
|
||||||
|
DROP = "drop"
|
||||||
|
BLOCK = "block"
|
||||||
|
WARN = "warn"
|
||||||
|
|
||||||
|
|
||||||
|
class SourceType(StrEnum):
|
||||||
|
CSV = "csv"
|
||||||
|
JSON = "json"
|
||||||
|
|
||||||
|
|
||||||
|
class ZipMethod(StrEnum):
|
||||||
|
ZIP_STANDARD = "zip_standard"
|
||||||
|
AES = "aes"
|
||||||
|
|
||||||
|
|
||||||
|
class BuildStatus(StrEnum):
|
||||||
|
BUILT = "built"
|
||||||
|
BUILD_FAILED = "build_failed"
|
||||||
|
|
||||||
|
|
||||||
|
class SendStatus(StrEnum):
|
||||||
|
DRAFT = "draft"
|
||||||
|
QUEUED = "queued"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignMeta(StrictModel):
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
description: str | None = None
|
||||||
|
mode: CampaignMode = CampaignMode.DRAFT
|
||||||
|
|
||||||
|
|
||||||
|
class FieldDefinition(StrictModel):
|
||||||
|
name: str
|
||||||
|
type: FieldType = FieldType.STRING
|
||||||
|
label: str | None = None
|
||||||
|
required: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class SmtpConfig(StrictModel):
|
||||||
|
host: str | None = None
|
||||||
|
port: int | None = Field(default=None, ge=1, le=65535)
|
||||||
|
username: str | None = None
|
||||||
|
password: str | None = None
|
||||||
|
security: TransportSecurity = TransportSecurity.STARTTLS
|
||||||
|
timeout_seconds: int = Field(default=30, ge=1)
|
||||||
|
|
||||||
|
|
||||||
|
class ImapConfig(StrictModel):
|
||||||
|
enabled: bool = False
|
||||||
|
host: str | None = None
|
||||||
|
port: int | None = Field(default=None, ge=1, le=65535)
|
||||||
|
username: str | None = None
|
||||||
|
password: str | None = None
|
||||||
|
security: TransportSecurity = TransportSecurity.TLS
|
||||||
|
sent_folder: str = "auto"
|
||||||
|
timeout_seconds: int = Field(default=30, ge=1)
|
||||||
|
|
||||||
|
|
||||||
|
class ServerConfig(StrictModel):
|
||||||
|
smtp: SmtpConfig | None = None
|
||||||
|
imap: ImapConfig | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class RecipientConfig(StrictModel):
|
||||||
|
email: str
|
||||||
|
name: str | None = None
|
||||||
|
recipient_type: RecipientType = Field(default=RecipientType.TO, alias="type")
|
||||||
|
|
||||||
|
@field_validator("email")
|
||||||
|
@classmethod
|
||||||
|
def email_should_look_like_address(cls, value: str) -> str:
|
||||||
|
# JSON Schema's format=email remains the stricter validation layer.
|
||||||
|
# Keep this deliberately lightweight to avoid an extra email-validator dependency.
|
||||||
|
if "@" not in value:
|
||||||
|
raise ValueError("email must contain '@'")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class RecipientsConfig(StrictModel):
|
||||||
|
from_: RecipientConfig | None = Field(default=None, alias="from")
|
||||||
|
allow_individual_from: bool = False
|
||||||
|
|
||||||
|
to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_to: bool = False
|
||||||
|
|
||||||
|
cc: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_cc: bool = False
|
||||||
|
|
||||||
|
bcc: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_bcc: bool = False
|
||||||
|
|
||||||
|
reply_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_reply_to: bool = False
|
||||||
|
|
||||||
|
bounce_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_bounce_to: bool = False
|
||||||
|
|
||||||
|
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
allow_individual_disposition_notification_to: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class TemplateSourceConfig(StrictModel):
|
||||||
|
type: Literal["files"] = "files"
|
||||||
|
subject_path: str | None = None
|
||||||
|
text_path: str | None = None
|
||||||
|
html_path: str | None = None
|
||||||
|
encoding: str = "utf-8"
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def at_least_one_path(self) -> "TemplateSourceConfig":
|
||||||
|
if not any([self.subject_path, self.text_path, self.html_path]):
|
||||||
|
raise ValueError("template.source must define subject_path, text_path or html_path")
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
class TemplateConfig(StrictModel):
|
||||||
|
subject: str | None = None
|
||||||
|
text: str | None = None
|
||||||
|
html: str | None = None
|
||||||
|
source: TemplateSourceConfig | None = None
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def inline_or_source(self) -> "TemplateConfig":
|
||||||
|
inline_values = any(value is not None for value in [self.subject, self.text, self.html])
|
||||||
|
if self.source and inline_values:
|
||||||
|
raise ValueError("template must be either inline or source-based, not both")
|
||||||
|
if self.source:
|
||||||
|
return self
|
||||||
|
if not self.subject:
|
||||||
|
raise ValueError("inline template requires subject")
|
||||||
|
return self
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_external(self) -> bool:
|
||||||
|
return self.source is not None
|
||||||
|
|
||||||
|
|
||||||
|
class ZipConfig(StrictModel):
|
||||||
|
enabled: bool = False
|
||||||
|
filename_template: str | None = None
|
||||||
|
password_template: str | None = None
|
||||||
|
method: ZipMethod = ZipMethod.AES
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentConfig(StrictModel):
|
||||||
|
id: str | None = None
|
||||||
|
label: str | None = None
|
||||||
|
base_dir: str
|
||||||
|
file_filter: str
|
||||||
|
include_subdirs: bool = False
|
||||||
|
required: bool = True
|
||||||
|
allow_multiple: bool = False
|
||||||
|
missing_behavior: Behavior = Behavior.ASK
|
||||||
|
ambiguous_behavior: Behavior = Behavior.ASK
|
||||||
|
zip: ZipConfig = Field(default_factory=ZipConfig)
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentsConfig(StrictModel):
|
||||||
|
base_path: str = "."
|
||||||
|
allow_individual: bool = False
|
||||||
|
send_without_attachments: bool = True
|
||||||
|
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
|
||||||
|
missing_behavior: Behavior = Behavior.ASK
|
||||||
|
ambiguous_behavior: Behavior = Behavior.ASK
|
||||||
|
|
||||||
|
|
||||||
|
class EntryConfig(StrictModel):
|
||||||
|
id: str | None = None
|
||||||
|
active: bool = True
|
||||||
|
|
||||||
|
from_: RecipientConfig | None = Field(default=None, alias="from")
|
||||||
|
|
||||||
|
to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_to: bool = True
|
||||||
|
|
||||||
|
cc: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_cc: bool = True
|
||||||
|
|
||||||
|
bcc: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_bcc: bool = True
|
||||||
|
|
||||||
|
reply_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_reply_to: bool = True
|
||||||
|
|
||||||
|
bounce_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_bounce_to: bool = True
|
||||||
|
|
||||||
|
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
|
||||||
|
combine_disposition_notification_to: bool = True
|
||||||
|
|
||||||
|
attachments: list[AttachmentConfig] = Field(default_factory=list)
|
||||||
|
combine_attachments: bool = True
|
||||||
|
|
||||||
|
fields: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
last_sent: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class SourceConfig(StrictModel):
|
||||||
|
type: SourceType
|
||||||
|
path: str
|
||||||
|
delimiter: str = ";"
|
||||||
|
encoding: str = "utf-8"
|
||||||
|
has_header: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class EntriesConfig(StrictModel):
|
||||||
|
inline: list[EntryConfig] | None = None
|
||||||
|
source: SourceConfig | None = None
|
||||||
|
mapping: dict[str, str] | None = None
|
||||||
|
defaults: EntryConfig | None = None
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def inline_or_external(self) -> "EntriesConfig":
|
||||||
|
has_inline = self.inline is not None
|
||||||
|
has_external = self.source is not None or self.mapping is not None or self.defaults is not None
|
||||||
|
if has_inline and has_external:
|
||||||
|
raise ValueError("entries must be either inline or source-based, not both")
|
||||||
|
if has_inline:
|
||||||
|
return self
|
||||||
|
if self.source is None or self.mapping is None:
|
||||||
|
raise ValueError("external entries require source and mapping")
|
||||||
|
return self
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_inline(self) -> bool:
|
||||||
|
return self.inline is not None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_external(self) -> bool:
|
||||||
|
return self.source is not None
|
||||||
|
|
||||||
|
|
||||||
|
class ValidationPolicy(StrictModel):
|
||||||
|
missing_required_attachment: Behavior = Behavior.ASK
|
||||||
|
missing_optional_attachment: Behavior = Behavior.WARN
|
||||||
|
ambiguous_attachment_match: Behavior = Behavior.ASK
|
||||||
|
missing_email: MissingAddressBehavior = MissingAddressBehavior.BLOCK
|
||||||
|
template_error: MissingAddressBehavior = MissingAddressBehavior.BLOCK
|
||||||
|
inactive_entry: InactiveEntryBehavior = InactiveEntryBehavior.DROP
|
||||||
|
|
||||||
|
|
||||||
|
class RateLimitConfig(StrictModel):
|
||||||
|
messages_per_minute: int = Field(default=5, ge=1)
|
||||||
|
concurrency: int = Field(default=1, ge=1)
|
||||||
|
|
||||||
|
|
||||||
|
class ImapAppendSentConfig(StrictModel):
|
||||||
|
enabled: bool = False
|
||||||
|
folder: str = "auto"
|
||||||
|
|
||||||
|
|
||||||
|
class RetryConfig(StrictModel):
|
||||||
|
max_attempts: int = Field(default=3, ge=1)
|
||||||
|
backoff_seconds: list[int] = Field(default_factory=lambda: [60, 300, 900])
|
||||||
|
|
||||||
|
@field_validator("backoff_seconds")
|
||||||
|
@classmethod
|
||||||
|
def backoff_values_must_be_positive(cls, values: list[int]) -> list[int]:
|
||||||
|
if any(value < 1 for value in values):
|
||||||
|
raise ValueError("backoff_seconds values must be >= 1")
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
class DeliveryConfig(StrictModel):
|
||||||
|
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
|
||||||
|
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
|
||||||
|
retry: RetryConfig = Field(default_factory=RetryConfig)
|
||||||
|
|
||||||
|
|
||||||
|
class StatusTrackingConfig(StrictModel):
|
||||||
|
enabled: bool = True
|
||||||
|
initial_build_status: BuildStatus = BuildStatus.BUILT
|
||||||
|
initial_send_status: SendStatus = SendStatus.DRAFT
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignConfig(StrictModel):
|
||||||
|
version: Literal["1.0"]
|
||||||
|
campaign: CampaignMeta
|
||||||
|
fields: list[FieldDefinition] = Field(default_factory=list)
|
||||||
|
global_values: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
server: ServerConfig = Field(default_factory=ServerConfig)
|
||||||
|
recipients: RecipientsConfig = Field(default_factory=RecipientsConfig)
|
||||||
|
template: TemplateConfig
|
||||||
|
attachments: AttachmentsConfig = Field(default_factory=AttachmentsConfig)
|
||||||
|
entries: EntriesConfig
|
||||||
|
validation_policy: ValidationPolicy = Field(default_factory=ValidationPolicy)
|
||||||
|
delivery: DeliveryConfig = Field(default_factory=DeliveryConfig)
|
||||||
|
status_tracking: StatusTrackingConfig = Field(default_factory=StatusTrackingConfig)
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def field_names_must_be_unique(self) -> "CampaignConfig":
|
||||||
|
names = [field.name for field in self.fields]
|
||||||
|
duplicates = sorted({name for name in names if names.count(name) > 1})
|
||||||
|
if duplicates:
|
||||||
|
raise ValueError(f"duplicate field definitions: {', '.join(duplicates)}")
|
||||||
|
return self
|
||||||
|
|
||||||
|
@property
|
||||||
|
def field_names(self) -> set[str]:
|
||||||
|
return {field.name for field in self.fields}
|
||||||
|
|
||||||
|
def resolve_relative_path(self, campaign_file: Path, raw_path: str) -> Path:
|
||||||
|
path = Path(raw_path).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return path
|
||||||
|
return (campaign_file.parent / path).resolve()
|
||||||
261
server/app/mailer/campaign/validation.py
Normal file
261
server/app/mailer/campaign/validation.py
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
from enum import StrEnum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from .models import CampaignConfig, SourceType
|
||||||
|
|
||||||
|
|
||||||
|
class Severity(StrEnum):
|
||||||
|
INFO = "info"
|
||||||
|
WARNING = "warning"
|
||||||
|
ERROR = "error"
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticIssue(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
severity: Severity
|
||||||
|
code: str
|
||||||
|
message: str
|
||||||
|
path: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticReport(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaign_id: str
|
||||||
|
campaign_name: str
|
||||||
|
issues: list[SemanticIssue] = Field(default_factory=list)
|
||||||
|
entries_mode: str
|
||||||
|
entries_count: int | None = None
|
||||||
|
attachments_base_path: str
|
||||||
|
rate_limit: str
|
||||||
|
imap_append_enabled: bool
|
||||||
|
|
||||||
|
@property
|
||||||
|
def error_count(self) -> int:
|
||||||
|
return sum(1 for issue in self.issues if issue.severity == Severity.ERROR)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def warning_count(self) -> int:
|
||||||
|
return sum(1 for issue in self.issues if issue.severity == Severity.WARNING)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ok(self) -> bool:
|
||||||
|
return self.error_count == 0
|
||||||
|
|
||||||
|
|
||||||
|
def _issue(severity: Severity, code: str, message: str, path: str | None = None) -> SemanticIssue:
|
||||||
|
return SemanticIssue(severity=severity, code=code, message=message, path=path)
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve(campaign_file: Path, raw_path: str) -> Path:
|
||||||
|
path = Path(raw_path).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return path
|
||||||
|
return (campaign_file.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
|
def _mapping_target_known(target: str, field_names: set[str]) -> bool:
|
||||||
|
direct_targets = {
|
||||||
|
"id",
|
||||||
|
"active",
|
||||||
|
"last_sent",
|
||||||
|
"combine_to",
|
||||||
|
"combine_cc",
|
||||||
|
"combine_bcc",
|
||||||
|
"combine_reply_to",
|
||||||
|
"combine_bounce_to",
|
||||||
|
"combine_disposition_notification_to",
|
||||||
|
"combine_attachments",
|
||||||
|
}
|
||||||
|
if target in direct_targets:
|
||||||
|
return True
|
||||||
|
if target.startswith("fields."):
|
||||||
|
name = target.split(".", 1)[1]
|
||||||
|
return not field_names or name in field_names
|
||||||
|
if target.startswith("from."):
|
||||||
|
return target in {"from.email", "from.name", "from.type"}
|
||||||
|
for prefix in ["to", "cc", "bcc", "reply_to", "bounce_to", "disposition_notification_to"]:
|
||||||
|
if target.startswith(prefix + "."):
|
||||||
|
parts = target.split(".")
|
||||||
|
return len(parts) == 3 and parts[1].isdigit() and parts[2] in {"email", "name", "type"}
|
||||||
|
if target.startswith("attachments."):
|
||||||
|
parts = target.split(".")
|
||||||
|
# attachments.0.zip.filename_template etc.
|
||||||
|
if len(parts) >= 3 and parts[1].isdigit():
|
||||||
|
if parts[2] in {
|
||||||
|
"id",
|
||||||
|
"label",
|
||||||
|
"base_dir",
|
||||||
|
"file_filter",
|
||||||
|
"include_subdirs",
|
||||||
|
"required",
|
||||||
|
"allow_multiple",
|
||||||
|
"missing_behavior",
|
||||||
|
"ambiguous_behavior",
|
||||||
|
}:
|
||||||
|
return len(parts) == 3
|
||||||
|
if parts[2] == "zip" and len(parts) == 4:
|
||||||
|
return parts[3] in {"enabled", "filename_template", "password_template", "method"}
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _csv_header(path: Path, delimiter: str, encoding: str) -> list[str] | None:
|
||||||
|
with path.open("r", encoding=encoding, newline="") as handle:
|
||||||
|
reader = csv.reader(handle, delimiter=delimiter)
|
||||||
|
try:
|
||||||
|
return next(reader)
|
||||||
|
except StopIteration:
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_template_source_paths(config: CampaignConfig) -> Iterable[tuple[str, str]]:
|
||||||
|
if not config.template.source:
|
||||||
|
return []
|
||||||
|
source = config.template.source
|
||||||
|
paths: list[tuple[str, str]] = []
|
||||||
|
if source.subject_path:
|
||||||
|
paths.append(("/template/source/subject_path", source.subject_path))
|
||||||
|
if source.text_path:
|
||||||
|
paths.append(("/template/source/text_path", source.text_path))
|
||||||
|
if source.html_path:
|
||||||
|
paths.append(("/template/source/html_path", source.html_path))
|
||||||
|
return paths
|
||||||
|
|
||||||
|
|
||||||
|
def validate_campaign_config(
|
||||||
|
config: CampaignConfig,
|
||||||
|
*,
|
||||||
|
campaign_file: str | Path | None = None,
|
||||||
|
check_files: bool = False,
|
||||||
|
) -> SemanticReport:
|
||||||
|
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
|
||||||
|
issues: list[SemanticIssue] = []
|
||||||
|
|
||||||
|
field_names = config.field_names
|
||||||
|
declared_names = {field.name for field in config.fields}
|
||||||
|
|
||||||
|
for key in config.global_values:
|
||||||
|
if declared_names and key not in declared_names:
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"unknown_global_value",
|
||||||
|
f"global_values contains {key!r}, but it is not declared in fields",
|
||||||
|
f"/global_values/{key}",
|
||||||
|
))
|
||||||
|
|
||||||
|
if config.server.imap and config.server.imap.enabled:
|
||||||
|
missing = [name for name in ["host", "port", "username", "password"] if getattr(config.server.imap, name) in (None, "")]
|
||||||
|
if missing:
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"incomplete_imap_config",
|
||||||
|
"IMAP append is enabled, but these IMAP settings are missing: " + ", ".join(missing),
|
||||||
|
"/server/imap",
|
||||||
|
))
|
||||||
|
|
||||||
|
if config.delivery.imap_append_sent.enabled and not (config.server.imap and config.server.imap.enabled):
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"delivery_imap_enabled_without_server_imap",
|
||||||
|
"delivery.imap_append_sent is enabled, but server.imap.enabled is not true",
|
||||||
|
"/delivery/imap_append_sent/enabled",
|
||||||
|
))
|
||||||
|
|
||||||
|
if config.campaign.mode == "send" and not config.server.smtp:
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"missing_smtp_config",
|
||||||
|
"campaign mode is 'send', but no server.smtp configuration is present",
|
||||||
|
"/server/smtp",
|
||||||
|
))
|
||||||
|
|
||||||
|
if config.server.smtp:
|
||||||
|
missing = [name for name in ["host", "port"] if getattr(config.server.smtp, name) in (None, "")]
|
||||||
|
if missing:
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"incomplete_smtp_config",
|
||||||
|
"SMTP settings are present, but these settings are missing: " + ", ".join(missing),
|
||||||
|
"/server/smtp",
|
||||||
|
))
|
||||||
|
|
||||||
|
if config.entries.is_inline:
|
||||||
|
entries_count = len(config.entries.inline or [])
|
||||||
|
entries_mode = "inline"
|
||||||
|
if entries_count == 0:
|
||||||
|
issues.append(_issue(Severity.WARNING, "no_inline_entries", "entries.inline is empty", "/entries/inline"))
|
||||||
|
else:
|
||||||
|
entries_count = None
|
||||||
|
entries_mode = f"external:{config.entries.source.type.value if config.entries.source else 'unknown'}"
|
||||||
|
mapping = config.entries.mapping or {}
|
||||||
|
if not mapping:
|
||||||
|
issues.append(_issue(Severity.ERROR, "empty_mapping", "external entries require a non-empty mapping", "/entries/mapping"))
|
||||||
|
for target in mapping:
|
||||||
|
if not _mapping_target_known(target, field_names):
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"unknown_mapping_target",
|
||||||
|
f"mapping target {target!r} is not recognized by the current campaign model",
|
||||||
|
f"/entries/mapping/{target}",
|
||||||
|
))
|
||||||
|
if check_files and config.entries.source:
|
||||||
|
source_path = _resolve(campaign_path, config.entries.source.path)
|
||||||
|
if not source_path.exists():
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"entries_source_not_found",
|
||||||
|
f"entries source file does not exist: {source_path}",
|
||||||
|
"/entries/source/path",
|
||||||
|
))
|
||||||
|
elif config.entries.source.type == SourceType.CSV and config.entries.source.has_header:
|
||||||
|
try:
|
||||||
|
header = _csv_header(source_path, config.entries.source.delimiter, config.entries.source.encoding)
|
||||||
|
header_set = set(header or [])
|
||||||
|
missing_columns = sorted({source_name for source_name in mapping.values() if source_name not in header_set})
|
||||||
|
if missing_columns:
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"mapping_columns_missing",
|
||||||
|
"CSV mapping refers to missing columns: " + ", ".join(missing_columns),
|
||||||
|
"/entries/mapping",
|
||||||
|
))
|
||||||
|
except OSError as exc:
|
||||||
|
issues.append(_issue(Severity.ERROR, "entries_source_read_error", str(exc), "/entries/source/path"))
|
||||||
|
|
||||||
|
if check_files:
|
||||||
|
attachments_base_path = _resolve(campaign_path, config.attachments.base_path)
|
||||||
|
if not attachments_base_path.exists():
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.WARNING,
|
||||||
|
"attachments_base_path_not_found",
|
||||||
|
f"attachments.base_path does not exist: {attachments_base_path}",
|
||||||
|
"/attachments/base_path",
|
||||||
|
))
|
||||||
|
for schema_path, raw_path in _iter_template_source_paths(config):
|
||||||
|
path = _resolve(campaign_path, raw_path)
|
||||||
|
if not path.exists():
|
||||||
|
issues.append(_issue(
|
||||||
|
Severity.ERROR,
|
||||||
|
"template_source_not_found",
|
||||||
|
f"template source file does not exist: {path}",
|
||||||
|
schema_path,
|
||||||
|
))
|
||||||
|
|
||||||
|
report = SemanticReport(
|
||||||
|
campaign_id=config.campaign.id,
|
||||||
|
campaign_name=config.campaign.name,
|
||||||
|
issues=issues,
|
||||||
|
entries_mode=entries_mode,
|
||||||
|
entries_count=entries_count,
|
||||||
|
attachments_base_path=config.attachments.base_path,
|
||||||
|
rate_limit=f"{config.delivery.rate_limit.messages_per_minute}/min, concurrency {config.delivery.rate_limit.concurrency}",
|
||||||
|
imap_append_enabled=config.delivery.imap_append_sent.enabled,
|
||||||
|
)
|
||||||
|
return report
|
||||||
1
server/app/mailer/commands/__init__.py
Normal file
1
server/app/mailer/commands/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""CLI commands for MultiMailer development workflows."""
|
||||||
69
server/app/mailer/commands/append_pending_sent.py
Normal file
69
server/app/mailer/commands/append_pending_sent.py
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from time import sleep
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.models import CampaignJob, JobImapStatus, JobSendStatus
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.sending.jobs import append_sent_for_job
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Append sent campaign messages to the configured IMAP Sent folder.")
|
||||||
|
parser.add_argument("--campaign-id", required=True, help="Database campaign UUID")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--limit", type=int, default=0, help="Maximum jobs to process; 0 means all pending/failed IMAP appends")
|
||||||
|
parser.add_argument("--dry-run", action="store_true")
|
||||||
|
parser.add_argument("--include-failed", action="store_true", help="Also retry jobs with imap_status=failed")
|
||||||
|
parser.add_argument("--json", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
results = []
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
|
||||||
|
statuses = [JobImapStatus.PENDING.value]
|
||||||
|
if args.include_failed:
|
||||||
|
statuses.append(JobImapStatus.FAILED.value)
|
||||||
|
|
||||||
|
query = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == api_key.tenant_id,
|
||||||
|
CampaignJob.campaign_id == args.campaign_id,
|
||||||
|
CampaignJob.send_status == JobSendStatus.SENT.value,
|
||||||
|
CampaignJob.imap_status.in_(statuses),
|
||||||
|
)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
)
|
||||||
|
if args.limit > 0:
|
||||||
|
query = query.limit(args.limit)
|
||||||
|
|
||||||
|
jobs = query.all()
|
||||||
|
for job in jobs:
|
||||||
|
try:
|
||||||
|
result = append_sent_for_job(session, job_id=job.id, dry_run=args.dry_run)
|
||||||
|
results.append(result.as_dict())
|
||||||
|
if not args.json:
|
||||||
|
print(f"{job.entry_index}: {result.status} ({job.recipient_email}) folder={result.folder or '-'}")
|
||||||
|
except Exception as exc:
|
||||||
|
results.append({"job_id": job.id, "status": "error", "error": str(exc)})
|
||||||
|
if not args.json:
|
||||||
|
print(f"{job.entry_index}: ERROR {exc} ({job.recipient_email})")
|
||||||
|
sleep(0.1)
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({"processed": len(results), "results": results}, indent=2))
|
||||||
|
elif not jobs:
|
||||||
|
print("No pending IMAP append jobs found")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
65
server/app/mailer/commands/audit_log.py
Normal file
65
server/app/mailer/commands/audit_log.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.models import AuditLog
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def _row(item: AuditLog) -> dict:
|
||||||
|
return {
|
||||||
|
"id": item.id,
|
||||||
|
"created_at": item.created_at.isoformat() if item.created_at else None,
|
||||||
|
"tenant_id": item.tenant_id,
|
||||||
|
"user_id": item.user_id,
|
||||||
|
"api_key_id": item.api_key_id,
|
||||||
|
"action": item.action,
|
||||||
|
"object_type": item.object_type,
|
||||||
|
"object_id": item.object_id,
|
||||||
|
"details": item.details,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="List audit log entries.")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--limit", type=int, default=50)
|
||||||
|
parser.add_argument("--offset", type=int, default=0)
|
||||||
|
parser.add_argument("--action")
|
||||||
|
parser.add_argument("--object-type")
|
||||||
|
parser.add_argument("--object-id")
|
||||||
|
parser.add_argument("--json", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
query = session.query(AuditLog).filter(AuditLog.tenant_id == api_key.tenant_id)
|
||||||
|
if args.action:
|
||||||
|
query = query.filter(AuditLog.action == args.action)
|
||||||
|
if args.object_type:
|
||||||
|
query = query.filter(AuditLog.object_type == args.object_type)
|
||||||
|
if args.object_id:
|
||||||
|
query = query.filter(AuditLog.object_id == args.object_id)
|
||||||
|
items = query.order_by(AuditLog.created_at.desc()).offset(args.offset).limit(args.limit).all()
|
||||||
|
rows = [_row(item) for item in items]
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({"items": rows}, indent=2, ensure_ascii=False, default=str))
|
||||||
|
return
|
||||||
|
|
||||||
|
for row in rows:
|
||||||
|
print(
|
||||||
|
f"{row['created_at']} | {row['action']} | "
|
||||||
|
f"{row['object_type'] or '-'}:{row['object_id'] or '-'} | {row['details'] or {}}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
99
server/app/mailer/commands/build_messages.py
Normal file
99
server/app/mailer/commands/build_messages.py
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.campaign.entries import EntryLoadError
|
||||||
|
from app.mailer.campaign.loader import CampaignLoadError, load_campaign_config
|
||||||
|
from app.mailer.messages.builder import build_campaign_messages
|
||||||
|
from app.mailer.messages.models import CampaignBuildReport
|
||||||
|
|
||||||
|
|
||||||
|
def _print_report(report: CampaignBuildReport, *, verbose: bool = False) -> None:
|
||||||
|
print(f"Campaign: {report.campaign_name} ({report.campaign_id})")
|
||||||
|
print(f"Campaign file: {report.campaign_file}")
|
||||||
|
print(f"Entries: {report.entries_count}")
|
||||||
|
print(
|
||||||
|
"Build: "
|
||||||
|
f"built={report.built_count}, "
|
||||||
|
f"failed={report.build_failed_count}, "
|
||||||
|
f"queueable={report.queueable_count}"
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
"Validation: "
|
||||||
|
f"ready={report.ready_count}, "
|
||||||
|
f"warning={report.warning_count}, "
|
||||||
|
f"needs_review={report.needs_review_count}, "
|
||||||
|
f"blocked={report.blocked_count}, "
|
||||||
|
f"excluded={report.excluded_count}, "
|
||||||
|
f"inactive={report.inactive_count}"
|
||||||
|
)
|
||||||
|
|
||||||
|
for message in report.messages:
|
||||||
|
print("---")
|
||||||
|
label = message.entry_id or f"#{message.entry_index}"
|
||||||
|
eml = f", eml={message.eml_path}" if message.eml_path else ""
|
||||||
|
print(
|
||||||
|
f"Entry {label}: "
|
||||||
|
f"build={message.build_status.value}, "
|
||||||
|
f"validation={message.validation_status.value}, "
|
||||||
|
f"send={message.send_status.value}, "
|
||||||
|
f"imap={message.imap_status.value}, "
|
||||||
|
f"attachments={message.attachment_count}{eml}"
|
||||||
|
)
|
||||||
|
if message.subject:
|
||||||
|
print(f" Subject: {message.subject}")
|
||||||
|
if message.to:
|
||||||
|
print(" To: " + ", ".join(item.email for item in message.to))
|
||||||
|
for issue in message.issues:
|
||||||
|
behavior = f", behavior={issue.behavior}" if issue.behavior else ""
|
||||||
|
source = f", source={issue.source}" if issue.source else ""
|
||||||
|
print(f" [{issue.severity}] {issue.code}{behavior}{source}: {issue.message}")
|
||||||
|
if verbose:
|
||||||
|
for attachment in message.attachments:
|
||||||
|
print(
|
||||||
|
f" - attachment {attachment.attachment_id or ''}: "
|
||||||
|
f"{attachment.status}, matches={len(attachment.matches)}, "
|
||||||
|
f"zip={attachment.zip_enabled}, filter={attachment.directory}/{attachment.file_filter}"
|
||||||
|
)
|
||||||
|
for match in attachment.matches:
|
||||||
|
print(f" {match}")
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Build campaign message drafts and review statuses without sending.")
|
||||||
|
parser.add_argument("--campaign", required=True, help="Path to campaign.json")
|
||||||
|
parser.add_argument("--output-dir", default=None, help="Optional directory for generated .eml files")
|
||||||
|
parser.add_argument("--write-eml", action="store_true", help="Write generated messages as .eml files")
|
||||||
|
parser.add_argument("--json", action="store_true", help="Output machine-readable JSON report")
|
||||||
|
parser.add_argument("--verbose", "-v", action="store_true", help="Print attachment details")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
campaign_path = Path(args.campaign).resolve()
|
||||||
|
output_dir = Path(args.output_dir).resolve() if args.output_dir else None
|
||||||
|
write_eml = args.write_eml or output_dir is not None
|
||||||
|
|
||||||
|
try:
|
||||||
|
config = load_campaign_config(campaign_path)
|
||||||
|
result = build_campaign_messages(
|
||||||
|
config,
|
||||||
|
campaign_file=campaign_path,
|
||||||
|
output_dir=output_dir,
|
||||||
|
write_eml=write_eml,
|
||||||
|
)
|
||||||
|
except (CampaignLoadError, EntryLoadError, ValueError, OSError) as exc:
|
||||||
|
print(f"Error: {exc}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps(result.report.model_dump(mode="json", by_alias=True), ensure_ascii=False, indent=2))
|
||||||
|
else:
|
||||||
|
_print_report(result.report, verbose=args.verbose)
|
||||||
|
|
||||||
|
return 0 if result.report.build_failed_count == 0 else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
78
server/app/mailer/commands/campaign_report.py
Normal file
78
server/app/mailer/commands/campaign_report.py
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def _print_text_report(report: dict) -> None:
|
||||||
|
campaign = report["campaign"]
|
||||||
|
cards = report["cards"]
|
||||||
|
delivery = report["delivery"]
|
||||||
|
print(f"Campaign: {campaign['name']} ({campaign['id']})")
|
||||||
|
print(f"Status: {campaign['status']}")
|
||||||
|
print(f"Jobs: {cards['jobs_total']} total | {cards['queueable']} queueable | {cards['needs_attention']} need attention")
|
||||||
|
print(f"Sending: {cards['sent']} sent | {cards['failed']} failed")
|
||||||
|
print(f"IMAP: {cards['imap_appended']} appended | {cards['imap_failed']} failed")
|
||||||
|
if delivery.get("rate_limit", {}).get("messages_per_minute"):
|
||||||
|
print(
|
||||||
|
"Rate: "
|
||||||
|
f"{delivery['rate_limit']['messages_per_minute']}/min, concurrency {delivery['rate_limit']['concurrency']}"
|
||||||
|
)
|
||||||
|
if delivery.get("estimated_remaining_send_human"):
|
||||||
|
print(f"ETA: {delivery['estimated_remaining_send_human']}")
|
||||||
|
print("Validation counts:", report["status_counts"]["validation"])
|
||||||
|
print("Send counts: ", report["status_counts"]["send"])
|
||||||
|
print("Issue codes: ", report["issues"]["by_code"])
|
||||||
|
print("Attachments: ", report["attachments"])
|
||||||
|
failures = report.get("recent_failures") or []
|
||||||
|
if failures:
|
||||||
|
print("Recent failures:")
|
||||||
|
for failure in failures[:10]:
|
||||||
|
print(
|
||||||
|
f" - entry={failure['entry_index']} recipient={failure['recipient_email']} "
|
||||||
|
f"send={failure['send_status']} imap={failure['imap_status']} error={failure['last_error']}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Generate a campaign status/report payload.")
|
||||||
|
parser.add_argument("--campaign-id", required=True, help="Database campaign UUID")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--json", action="store_true", help="Print machine-readable JSON")
|
||||||
|
parser.add_argument("--include-jobs", action="store_true", help="Include per-job rows in JSON output")
|
||||||
|
parser.add_argument("--jobs-csv", help="Write per-job report CSV to this path")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
try:
|
||||||
|
report = generate_campaign_report(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
campaign_id=args.campaign_id,
|
||||||
|
include_jobs=args.include_jobs,
|
||||||
|
)
|
||||||
|
if args.jobs_csv:
|
||||||
|
csv_text = generate_jobs_csv(session, tenant_id=api_key.tenant_id, campaign_id=args.campaign_id)
|
||||||
|
Path(args.jobs_csv).write_text(csv_text, encoding="utf-8")
|
||||||
|
print(f"Wrote {args.jobs_csv}")
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps(report, indent=2, ensure_ascii=False, default=str))
|
||||||
|
else:
|
||||||
|
_print_text_report(report)
|
||||||
|
except CampaignReportError as exc:
|
||||||
|
raise SystemExit(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
74
server/app/mailer/commands/email_campaign_report.py
Normal file
74
server/app/mailer/commands/email_campaign_report.py
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.audit.logging import audit_event
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.reports.emailing import CampaignReportEmailError, send_campaign_report_email
|
||||||
|
from app.mailer.reports.campaigns import CampaignReportError
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Email a campaign report to one or more recipients.")
|
||||||
|
parser.add_argument("--campaign-id", required=True, help="Database campaign UUID")
|
||||||
|
parser.add_argument("--to", action="append", required=True, help="Report recipient. Repeat for multiple recipients.")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--include-jobs", action="store_true", help="Include per-job rows in the JSON report payload before rendering")
|
||||||
|
parser.add_argument("--no-jobs-csv", action="store_true", help="Do not attach the per-job CSV report")
|
||||||
|
parser.add_argument("--attach-report-json", action="store_true", help="Attach the complete report JSON")
|
||||||
|
parser.add_argument("--dry-run", action="store_true", help="Build and validate the report email without SMTP sending")
|
||||||
|
parser.add_argument("--json", action="store_true", help="Print machine-readable JSON")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
try:
|
||||||
|
result = send_campaign_report_email(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
campaign_id=args.campaign_id,
|
||||||
|
to=args.to,
|
||||||
|
include_jobs=args.include_jobs,
|
||||||
|
attach_jobs_csv=not args.no_jobs_csv,
|
||||||
|
attach_report_json=args.attach_report_json,
|
||||||
|
dry_run=args.dry_run,
|
||||||
|
)
|
||||||
|
audit_event(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
user_id=api_key.user_id,
|
||||||
|
api_key_id=api_key.id,
|
||||||
|
action="report.email_sent" if not args.dry_run else "report.email_dry_run",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=args.campaign_id,
|
||||||
|
details=result.as_dict(),
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
except (CampaignReportError, CampaignReportEmailError) as exc:
|
||||||
|
raise SystemExit(str(exc)) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise SystemExit(f"Could not email campaign report: {exc}") from exc
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps(result.as_dict(), indent=2, ensure_ascii=False))
|
||||||
|
else:
|
||||||
|
print(f"Campaign: {result.campaign_id}")
|
||||||
|
print(f"To: {', '.join(result.to)}")
|
||||||
|
print(f"Subject: {result.subject}")
|
||||||
|
print(f"Dry run: {result.dry_run}")
|
||||||
|
print(f"Sent: {result.sent}")
|
||||||
|
print(f"CSV: {result.attached_jobs_csv}")
|
||||||
|
print(f"JSON: {result.attached_report_json}")
|
||||||
|
if result.smtp_host:
|
||||||
|
print(f"SMTP: {result.smtp_host}:{result.smtp_port}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
67
server/app/mailer/commands/import_campaign.py
Normal file
67
server/app/mailer/commands/import_campaign.py
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.audit.logging import audit_event
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.db.models import User
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
from app.mailer.persistence.campaigns import create_campaign_version_from_json, validate_campaign_version, build_campaign_version
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Import a campaign JSON into the database and optionally validate/build it.")
|
||||||
|
parser.add_argument("--campaign", required=True, help="Path to campaign.json")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key, help="API key used as the importing principal")
|
||||||
|
parser.add_argument("--validate", action="store_true", help="Run semantic validation after import")
|
||||||
|
parser.add_argument("--build", action="store_true", help="Build message jobs after import")
|
||||||
|
parser.add_argument("--no-eml", action="store_true", help="Do not write generated .eml files during build")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
campaign_path = Path(args.campaign).resolve()
|
||||||
|
raw_json = json.loads(campaign_path.read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key. Run init_db --with-dev-data first or pass --api-key.")
|
||||||
|
user = session.get(User, api_key.user_id)
|
||||||
|
campaign, version = create_campaign_version_from_json(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
user_id=user.id if user else None,
|
||||||
|
raw_json=raw_json,
|
||||||
|
source_filename=str(campaign_path),
|
||||||
|
)
|
||||||
|
audit_event(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
user_id=api_key.user_id,
|
||||||
|
api_key_id=api_key.id,
|
||||||
|
action="campaign.imported",
|
||||||
|
object_type="campaign",
|
||||||
|
object_id=campaign.id,
|
||||||
|
details={"version_id": version.id, "source_filename": str(campaign_path)},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
print(f"Campaign: {campaign.name} ({campaign.id})")
|
||||||
|
print(f"Version: {version.version_number} ({version.id})")
|
||||||
|
|
||||||
|
if args.validate:
|
||||||
|
report = validate_campaign_version(session, tenant_id=api_key.tenant_id, version_id=version.id)
|
||||||
|
audit_event(session, tenant_id=api_key.tenant_id, user_id=api_key.user_id, api_key_id=api_key.id, action="campaign.validated", object_type="campaign_version", object_id=version.id, details={"ok": report.get("ok")}, commit=True)
|
||||||
|
print(f"Validation: ok={report['ok']}, errors={report['error_count']}, warnings={report['warning_count']}")
|
||||||
|
|
||||||
|
if args.build:
|
||||||
|
report = build_campaign_version(session, tenant_id=api_key.tenant_id, version_id=version.id, write_eml=not args.no_eml)
|
||||||
|
audit_event(session, tenant_id=api_key.tenant_id, user_id=api_key.user_id, api_key_id=api_key.id, action="campaign.messages_built", object_type="campaign_version", object_id=version.id, details={"built_count": report.get("built_count"), "write_eml": not args.no_eml}, commit=True)
|
||||||
|
print(f"Build: built={report['built_count']}, queueable={report['queueable_count']}, needs_review={report['needs_review_count']}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
32
server/app/mailer/commands/init_db.py
Normal file
32
server/app/mailer/commands/init_db.py
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
|
||||||
|
from app.db.bootstrap import bootstrap_dev_data, create_all_tables
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Initialize the MultiMailer database")
|
||||||
|
parser.add_argument("--with-dev-data", action="store_true", help="Create default tenant/user/roles and a development API key")
|
||||||
|
parser.add_argument("--dev-api-key", default=settings.dev_bootstrap_api_key, help="Development API key secret to create")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
print("Database tables ensured.")
|
||||||
|
|
||||||
|
if args.with_dev_data:
|
||||||
|
with SessionLocal() as session:
|
||||||
|
result = bootstrap_dev_data(session, api_key_secret=args.dev_api_key)
|
||||||
|
print(f"Tenant: {result.tenant.slug} ({result.tenant.id})")
|
||||||
|
print(f"User: {result.user.email} ({result.user.id})")
|
||||||
|
if result.created_api_key:
|
||||||
|
print("Development API key created:")
|
||||||
|
print(result.created_api_key.secret)
|
||||||
|
else:
|
||||||
|
print("Development API key already exists or was not requested.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
29
server/app/mailer/commands/list_db_campaigns.py
Normal file
29
server/app/mailer/commands/list_db_campaigns.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.models import Campaign, CampaignJob
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="List persisted campaigns and job counts.")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
campaigns = session.query(Campaign).filter(Campaign.tenant_id == api_key.tenant_id).order_by(Campaign.updated_at.desc()).all()
|
||||||
|
for campaign in campaigns:
|
||||||
|
jobs = session.query(CampaignJob).filter(CampaignJob.campaign_id == campaign.id).count()
|
||||||
|
print(f"{campaign.id} | {campaign.status:15s} | jobs={jobs:4d} | {campaign.name}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
54
server/app/mailer/commands/queue_campaign.py
Normal file
54
server/app/mailer/commands/queue_campaign.py
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.models import Campaign
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.sending.jobs import queue_campaign_jobs
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Queue built campaign jobs for sending.")
|
||||||
|
parser.add_argument("--campaign-id", required=True, help="Database campaign UUID, not external campaign id")
|
||||||
|
parser.add_argument("--version-id", default=None, help="Optional campaign version UUID; defaults to current version")
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--no-celery", action="store_true", help="Only mark jobs as queued; do not enqueue Celery tasks")
|
||||||
|
parser.add_argument("--exclude-warnings", action="store_true", help="Queue only validation_status=ready, not warnings")
|
||||||
|
parser.add_argument("--dry-run", action="store_true")
|
||||||
|
parser.add_argument("--json", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
result = queue_campaign_jobs(
|
||||||
|
session,
|
||||||
|
tenant_id=api_key.tenant_id,
|
||||||
|
campaign_id=args.campaign_id,
|
||||||
|
version_id=args.version_id,
|
||||||
|
enqueue_celery=not args.no_celery,
|
||||||
|
include_warnings=not args.exclude_warnings,
|
||||||
|
dry_run=args.dry_run,
|
||||||
|
)
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps(result.as_dict(), indent=2))
|
||||||
|
return
|
||||||
|
campaign = session.get(Campaign, args.campaign_id)
|
||||||
|
print(f"Campaign: {campaign.name if campaign else args.campaign_id}")
|
||||||
|
print(f"Version: {result.version_id}")
|
||||||
|
print(f"Queued: {result.queued_count}")
|
||||||
|
print(f"Skipped: {result.skipped_count}")
|
||||||
|
print(f"Blocked: {result.blocked_count}")
|
||||||
|
print(f"Enqueued Celery tasks: {result.enqueued_count}")
|
||||||
|
if result.dry_run:
|
||||||
|
print("Dry run: no database changes were committed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
69
server/app/mailer/commands/resolve_attachments.py
Normal file
69
server/app/mailer/commands/resolve_attachments.py
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.attachments.resolver import AttachmentResolutionReport, resolve_campaign_attachments
|
||||||
|
from app.mailer.campaign.loader import CampaignLoadError, load_campaign_config
|
||||||
|
from app.mailer.campaign.entries import EntryLoadError
|
||||||
|
|
||||||
|
|
||||||
|
def _print_report(report: AttachmentResolutionReport, *, verbose: bool = False) -> None:
|
||||||
|
print(f"Campaign: {report.campaign_name} ({report.campaign_id})")
|
||||||
|
print(f"Campaign file: {report.campaign_file}")
|
||||||
|
print(f"Attachments base path: {report.attachments_base_path}")
|
||||||
|
print(f"Entries: {report.entries_count}")
|
||||||
|
print(
|
||||||
|
"Status: "
|
||||||
|
f"ready={report.ready_count}, "
|
||||||
|
f"warning={report.warning_count}, "
|
||||||
|
f"needs_review={report.needs_review_count}, "
|
||||||
|
f"blocked={report.blocked_count}, "
|
||||||
|
f"excluded={report.excluded_count}, "
|
||||||
|
f"inactive={report.inactive_count}"
|
||||||
|
)
|
||||||
|
|
||||||
|
for entry in report.entries:
|
||||||
|
print("---")
|
||||||
|
label = entry.entry_id or f"#{entry.entry_index}"
|
||||||
|
print(f"Entry {label}: {entry.status.value}, matches={entry.match_count}")
|
||||||
|
for issue in entry.issues:
|
||||||
|
behavior = f", behavior={issue.behavior.value}" if issue.behavior else ""
|
||||||
|
print(f" [{issue.severity.value}] {issue.code}{behavior}: {issue.message}")
|
||||||
|
if verbose:
|
||||||
|
for attachment in entry.attachments:
|
||||||
|
print(
|
||||||
|
f" - {attachment.scope.value}[{attachment.index}] "
|
||||||
|
f"{attachment.attachment_id or ''} "
|
||||||
|
f"{attachment.status.value}: {attachment.directory}/{attachment.file_filter}"
|
||||||
|
)
|
||||||
|
for match in attachment.matches:
|
||||||
|
print(f" {match}")
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Resolve campaign attachment patterns and report missing/ambiguous matches.")
|
||||||
|
parser.add_argument("--campaign", required=True, help="Path to campaign.json")
|
||||||
|
parser.add_argument("--json", action="store_true", help="Output machine-readable JSON")
|
||||||
|
parser.add_argument("--verbose", "-v", action="store_true", help="Print resolved configs and matched files")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
campaign_path = Path(args.campaign)
|
||||||
|
try:
|
||||||
|
config = load_campaign_config(campaign_path)
|
||||||
|
report = resolve_campaign_attachments(config, campaign_file=campaign_path)
|
||||||
|
except (CampaignLoadError, EntryLoadError, ValueError) as exc:
|
||||||
|
print(f"Error: {exc}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps(report.model_dump(mode="json"), ensure_ascii=False, indent=2))
|
||||||
|
else:
|
||||||
|
_print_report(report, verbose=args.verbose)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
71
server/app/mailer/commands/send_queued_jobs.py
Normal file
71
server/app/mailer/commands/send_queued_jobs.py
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from time import sleep
|
||||||
|
|
||||||
|
from app.db.bootstrap import create_all_tables
|
||||||
|
from app.db.models import CampaignJob, JobQueueStatus, JobSendStatus
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.mailer.sending.jobs import append_sent_for_job, send_campaign_job
|
||||||
|
from app.security.api_keys import authenticate_api_key
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Process queued campaign jobs directly, without a Celery worker.")
|
||||||
|
parser.add_argument("--campaign-id", required=True)
|
||||||
|
parser.add_argument("--api-key", default=settings.dev_bootstrap_api_key)
|
||||||
|
parser.add_argument("--limit", type=int, default=0, help="Maximum jobs to process; 0 means all queued jobs")
|
||||||
|
parser.add_argument("--dry-run", action="store_true", help="Validate/send path without SMTP delivery or status mutation to SENT")
|
||||||
|
parser.add_argument("--no-rate-limit", action="store_true")
|
||||||
|
parser.add_argument("--append-sent", action="store_true", help="After successful SMTP delivery, immediately run IMAP append-to-Sent in this CLI process")
|
||||||
|
parser.add_argument("--json", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
create_all_tables()
|
||||||
|
results = []
|
||||||
|
with SessionLocal() as session:
|
||||||
|
api_key = authenticate_api_key(session, args.api_key)
|
||||||
|
if not api_key:
|
||||||
|
raise SystemExit("Invalid API key")
|
||||||
|
query = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == api_key.tenant_id,
|
||||||
|
CampaignJob.campaign_id == args.campaign_id,
|
||||||
|
CampaignJob.queue_status == JobQueueStatus.QUEUED.value,
|
||||||
|
CampaignJob.send_status.in_([JobSendStatus.QUEUED.value, JobSendStatus.FAILED_TEMPORARY.value]),
|
||||||
|
)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
)
|
||||||
|
if args.limit > 0:
|
||||||
|
query = query.limit(args.limit)
|
||||||
|
jobs = query.all()
|
||||||
|
for job in jobs:
|
||||||
|
try:
|
||||||
|
result = send_campaign_job(session, job_id=job.id, dry_run=args.dry_run, use_rate_limit=not args.no_rate_limit)
|
||||||
|
result_dict = result.as_dict()
|
||||||
|
if args.append_sent and result.status == "sent":
|
||||||
|
append_result = append_sent_for_job(session, job_id=job.id, dry_run=args.dry_run)
|
||||||
|
result_dict["imap_append"] = append_result.as_dict()
|
||||||
|
results.append(result_dict)
|
||||||
|
if not args.json:
|
||||||
|
line = f"{job.entry_index}: {result.status} ({job.recipient_email})"
|
||||||
|
if "imap_append" in result_dict:
|
||||||
|
line += f"; IMAP: {result_dict['imap_append']['status']}"
|
||||||
|
print(line)
|
||||||
|
except Exception as exc:
|
||||||
|
results.append({"job_id": job.id, "status": "error", "error": str(exc)})
|
||||||
|
if not args.json:
|
||||||
|
print(f"{job.entry_index}: ERROR {exc} ({job.recipient_email})")
|
||||||
|
# Continue with the next job; individual attempts/statuses are recorded.
|
||||||
|
sleep(0.1)
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({"processed": len(results), "results": results}, indent=2))
|
||||||
|
elif not jobs:
|
||||||
|
print("No queued jobs found")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
226
server/app/mailer/commands/send_test_message.py
Normal file
226
server/app/mailer/commands/send_test_message.py
Normal file
@@ -0,0 +1,226 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import getpass
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.campaign.entries import EntryLoadError
|
||||||
|
from app.mailer.campaign.loader import CampaignLoadError, load_campaign_config
|
||||||
|
from app.mailer.campaign.models import SmtpConfig, TransportSecurity
|
||||||
|
from app.mailer.messages.builder import BuiltMessage, build_campaign_messages
|
||||||
|
from app.mailer.messages.models import MessageDraft
|
||||||
|
from app.mailer.sending.smtp import (
|
||||||
|
SmtpConfigurationError,
|
||||||
|
SmtpSendError,
|
||||||
|
prepare_test_message,
|
||||||
|
send_email_message,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _env(name: str) -> str | None:
|
||||||
|
value = os.getenv(name)
|
||||||
|
return value if value not in {None, ""} else None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_security(value: str | None, fallback: TransportSecurity) -> TransportSecurity:
|
||||||
|
if not value:
|
||||||
|
return fallback
|
||||||
|
try:
|
||||||
|
return TransportSecurity(value.lower())
|
||||||
|
except ValueError as exc:
|
||||||
|
allowed = ", ".join(item.value for item in TransportSecurity)
|
||||||
|
raise ValueError(f"invalid SMTP security '{value}', expected one of: {allowed}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_port(value: str | None, fallback: int | None) -> int | None:
|
||||||
|
if not value:
|
||||||
|
return fallback
|
||||||
|
try:
|
||||||
|
return int(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ValueError(f"invalid SMTP port '{value}'") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _smtp_config_with_overrides(args: argparse.Namespace, base: SmtpConfig | None) -> SmtpConfig:
|
||||||
|
config = base or SmtpConfig()
|
||||||
|
|
||||||
|
password = args.smtp_password
|
||||||
|
if args.smtp_password_env:
|
||||||
|
password = _env(args.smtp_password_env)
|
||||||
|
if password is None:
|
||||||
|
raise ValueError(f"environment variable {args.smtp_password_env} is empty or not set")
|
||||||
|
if args.ask_password:
|
||||||
|
password = getpass.getpass("SMTP password: ")
|
||||||
|
|
||||||
|
return SmtpConfig(
|
||||||
|
host=args.smtp_host or _env("MULTIMAILER_SMTP_HOST") or config.host,
|
||||||
|
port=_parse_port(args.smtp_port or _env("MULTIMAILER_SMTP_PORT"), config.port),
|
||||||
|
username=args.smtp_username or _env("MULTIMAILER_SMTP_USERNAME") or config.username,
|
||||||
|
password=password or _env("MULTIMAILER_SMTP_PASSWORD") or config.password,
|
||||||
|
security=_parse_security(args.smtp_security or _env("MULTIMAILER_SMTP_SECURITY"), config.security),
|
||||||
|
timeout_seconds=args.smtp_timeout or config.timeout_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _select_message(
|
||||||
|
messages: list[BuiltMessage],
|
||||||
|
*,
|
||||||
|
entry_id: str | None,
|
||||||
|
entry_index: int | None,
|
||||||
|
allow_non_queueable: bool,
|
||||||
|
) -> BuiltMessage:
|
||||||
|
candidates = messages
|
||||||
|
if entry_id is not None:
|
||||||
|
candidates = [item for item in candidates if item.draft.entry_id == entry_id]
|
||||||
|
if not candidates:
|
||||||
|
raise ValueError(f"no generated message found for entry id '{entry_id}'")
|
||||||
|
if entry_index is not None:
|
||||||
|
candidates = [item for item in candidates if item.draft.entry_index == entry_index]
|
||||||
|
if not candidates:
|
||||||
|
raise ValueError(f"no generated message found for entry index {entry_index}")
|
||||||
|
|
||||||
|
if not allow_non_queueable:
|
||||||
|
queueable = [item for item in candidates if item.draft.is_queueable and item.mime is not None]
|
||||||
|
if queueable:
|
||||||
|
return queueable[0]
|
||||||
|
raise ValueError(
|
||||||
|
"no queueable built message found. Fix validation issues or pass --allow-non-queueable for a deliberate test send."
|
||||||
|
)
|
||||||
|
|
||||||
|
built = [item for item in candidates if item.mime is not None]
|
||||||
|
if not built:
|
||||||
|
raise ValueError("no built MIME message found")
|
||||||
|
return built[0]
|
||||||
|
|
||||||
|
|
||||||
|
def _envelope_from(draft: MessageDraft) -> str:
|
||||||
|
if draft.bounce_to:
|
||||||
|
return draft.bounce_to[0].email
|
||||||
|
if draft.from_:
|
||||||
|
return draft.from_.email
|
||||||
|
raise SmtpConfigurationError("message has no sender; cannot determine SMTP envelope sender")
|
||||||
|
|
||||||
|
|
||||||
|
def _write_test_eml(path: Path, message) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_bytes(bytes(message))
|
||||||
|
|
||||||
|
|
||||||
|
def _print_summary(*, draft: MessageDraft, test_to: str, smtp_config: SmtpConfig, envelope_from: str) -> None:
|
||||||
|
print(f"Entry: {draft.entry_id or '#' + str(draft.entry_index)}")
|
||||||
|
print(f"Subject: {draft.subject or ''}")
|
||||||
|
print(f"Original validation: {draft.validation_status.value}")
|
||||||
|
print(f"Original send status: {draft.send_status.value}")
|
||||||
|
print(f"Test recipient: {test_to}")
|
||||||
|
print(f"Envelope sender: {envelope_from}")
|
||||||
|
print(f"SMTP: {smtp_config.host}:{smtp_config.port} ({smtp_config.security.value})")
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Send one generated campaign message to a test recipient. Does not mutate campaign/job status."
|
||||||
|
)
|
||||||
|
parser.add_argument("--campaign", required=True, help="Path to campaign.json")
|
||||||
|
parser.add_argument("--to", required=True, help="Test recipient email address. Real campaign recipients are never used.")
|
||||||
|
parser.add_argument("--to-name", default=None, help="Optional display name for the test recipient")
|
||||||
|
parser.add_argument("--entry-id", default=None, help="Select a specific entry by id")
|
||||||
|
parser.add_argument("--entry-index", type=int, default=None, help="Select a specific entry by 1-based index")
|
||||||
|
parser.add_argument(
|
||||||
|
"--allow-non-queueable",
|
||||||
|
action="store_true",
|
||||||
|
help="Allow test-send of a built message whose validation status is needs_review/warning/blocked/excluded",
|
||||||
|
)
|
||||||
|
parser.add_argument("--write-eml", default=None, help="Write the prepared test .eml to this path")
|
||||||
|
parser.add_argument("--dry-run", action="store_true", help="Build and prepare the test message but do not connect to SMTP")
|
||||||
|
parser.add_argument("--json", action="store_true", help="Output machine-readable JSON")
|
||||||
|
|
||||||
|
parser.add_argument("--smtp-host", default=None, help="Override SMTP host")
|
||||||
|
parser.add_argument("--smtp-port", default=None, help="Override SMTP port")
|
||||||
|
parser.add_argument("--smtp-security", default=None, choices=[item.value for item in TransportSecurity], help="Override SMTP security")
|
||||||
|
parser.add_argument("--smtp-username", default=None, help="Override SMTP username")
|
||||||
|
parser.add_argument("--smtp-password", default=None, help="Override SMTP password (prefer --smtp-password-env or --ask-password)")
|
||||||
|
parser.add_argument("--smtp-password-env", default=None, help="Read SMTP password from this environment variable")
|
||||||
|
parser.add_argument("--ask-password", action="store_true", help="Prompt for SMTP password")
|
||||||
|
parser.add_argument("--smtp-timeout", type=int, default=None, help="Override SMTP timeout in seconds")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
campaign_path = Path(args.campaign).resolve()
|
||||||
|
|
||||||
|
try:
|
||||||
|
config = load_campaign_config(campaign_path)
|
||||||
|
smtp_config = _smtp_config_with_overrides(args, config.server.smtp)
|
||||||
|
result = build_campaign_messages(config, campaign_file=campaign_path)
|
||||||
|
selected = _select_message(
|
||||||
|
result.built_messages,
|
||||||
|
entry_id=args.entry_id,
|
||||||
|
entry_index=args.entry_index,
|
||||||
|
allow_non_queueable=args.allow_non_queueable,
|
||||||
|
)
|
||||||
|
assert selected.mime is not None
|
||||||
|
|
||||||
|
envelope_from = _envelope_from(selected.draft)
|
||||||
|
test_message = prepare_test_message(selected.mime, test_recipient=args.to, test_recipient_name=args.to_name)
|
||||||
|
|
||||||
|
if args.write_eml:
|
||||||
|
_write_test_eml(Path(args.write_eml).resolve(), test_message)
|
||||||
|
|
||||||
|
if not args.json:
|
||||||
|
_print_summary(draft=selected.draft, test_to=args.to, smtp_config=smtp_config, envelope_from=envelope_from)
|
||||||
|
|
||||||
|
send_result = None
|
||||||
|
if not args.dry_run:
|
||||||
|
send_result = send_email_message(
|
||||||
|
test_message,
|
||||||
|
smtp_config=smtp_config,
|
||||||
|
envelope_from=envelope_from,
|
||||||
|
envelope_recipients=[args.to],
|
||||||
|
)
|
||||||
|
|
||||||
|
except (CampaignLoadError, EntryLoadError, ValueError, SmtpConfigurationError, SmtpSendError, OSError) as exc:
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({"ok": False, "error": str(exc)}, ensure_ascii=False, indent=2))
|
||||||
|
else:
|
||||||
|
print(f"Error: {exc}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
payload = {
|
||||||
|
"ok": True,
|
||||||
|
"dry_run": args.dry_run,
|
||||||
|
"campaign_id": config.campaign.id,
|
||||||
|
"entry_id": selected.draft.entry_id,
|
||||||
|
"entry_index": selected.draft.entry_index,
|
||||||
|
"test_recipient": args.to,
|
||||||
|
"validation_status": selected.draft.validation_status.value,
|
||||||
|
"smtp": {
|
||||||
|
"host": smtp_config.host,
|
||||||
|
"port": smtp_config.port,
|
||||||
|
"security": smtp_config.security.value,
|
||||||
|
},
|
||||||
|
"send_result": None
|
||||||
|
if send_result is None
|
||||||
|
else {
|
||||||
|
"accepted_count": send_result.accepted_count,
|
||||||
|
"refused_recipients": send_result.refused_recipients,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
||||||
|
else:
|
||||||
|
if args.dry_run:
|
||||||
|
print("Dry run only; no SMTP connection attempted.")
|
||||||
|
else:
|
||||||
|
assert send_result is not None
|
||||||
|
print(f"SMTP accepted recipients: {send_result.accepted_count}/{len(send_result.envelope_recipients)}")
|
||||||
|
if send_result.refused_recipients:
|
||||||
|
print(f"SMTP refused recipients: {send_result.refused_recipients}")
|
||||||
|
else:
|
||||||
|
print("Test message sent.")
|
||||||
|
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
71
server/app/mailer/commands/validate_campaign.py
Normal file
71
server/app/mailer/commands/validate_campaign.py
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.campaign.loader import CampaignLoadError, CampaignSchemaError, load_campaign_config
|
||||||
|
from app.mailer.campaign.validation import Severity, validate_campaign_config
|
||||||
|
|
||||||
|
|
||||||
|
def _default_campaign_path() -> Path:
|
||||||
|
return Path(__file__).resolve().parents[1] / "examples" / "campaign.json"
|
||||||
|
|
||||||
|
|
||||||
|
def _print_text_report(report) -> None:
|
||||||
|
print(f"Campaign: {report.campaign_name} ({report.campaign_id})")
|
||||||
|
print(f"Entries: {report.entries_mode}" + (f", {report.entries_count} item(s)" if report.entries_count is not None else ""))
|
||||||
|
print(f"Attachments base path: {report.attachments_base_path}")
|
||||||
|
print(f"Rate limit: {report.rate_limit}")
|
||||||
|
print(f"IMAP append: {'enabled' if report.imap_append_enabled else 'disabled'}")
|
||||||
|
print(f"Issues: {report.error_count} error(s), {report.warning_count} warning(s)")
|
||||||
|
if report.issues:
|
||||||
|
print()
|
||||||
|
for issue in report.issues:
|
||||||
|
location = f" [{issue.path}]" if issue.path else ""
|
||||||
|
print(f"- {issue.severity.upper()} {issue.code}{location}: {issue.message}")
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Validate a MultiMailer campaign JSON file.")
|
||||||
|
parser.add_argument("--campaign", default=str(_default_campaign_path()), help="Path to campaign JSON file")
|
||||||
|
parser.add_argument("--schema", default=None, help="Optional path to campaign.schema.json")
|
||||||
|
parser.add_argument("--no-schema", action="store_true", help="Skip JSON Schema validation")
|
||||||
|
parser.add_argument("--check-files", action="store_true", help="Check referenced local files and CSV headers")
|
||||||
|
parser.add_argument("--json", action="store_true", help="Print machine-readable validation report")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
campaign_path = Path(args.campaign).resolve()
|
||||||
|
try:
|
||||||
|
config = load_campaign_config(
|
||||||
|
campaign_path,
|
||||||
|
validate_schema=not args.no_schema,
|
||||||
|
schema_path=args.schema,
|
||||||
|
)
|
||||||
|
report = validate_campaign_config(config, campaign_file=campaign_path, check_files=args.check_files)
|
||||||
|
except CampaignSchemaError as exc:
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({"ok": False, "schema_errors": [error.__dict__ for error in exc.errors]}, indent=2), file=sys.stdout)
|
||||||
|
else:
|
||||||
|
print(str(exc), file=sys.stderr)
|
||||||
|
for error in exc.errors:
|
||||||
|
print(f"- {error.path}: {error.message}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
except CampaignLoadError as exc:
|
||||||
|
print(str(exc), file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"campaign validation failed: {exc}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(report.model_dump_json(indent=2))
|
||||||
|
else:
|
||||||
|
_print_text_report(report)
|
||||||
|
|
||||||
|
return 0 if report.ok else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
0
server/app/mailer/domain/__init__.py
Normal file
0
server/app/mailer/domain/__init__.py
Normal file
210
server/app/mailer/domain/campaign.py
Normal file
210
server/app/mailer/domain/campaign.py
Normal file
@@ -0,0 +1,210 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import StrEnum
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .fields import Field, FieldConfiguration, FieldContents
|
||||||
|
from .recipients import Recipient, RecipientList, RecipientType
|
||||||
|
from .template import MailTemplate
|
||||||
|
|
||||||
|
|
||||||
|
class TransportSecurity(StrEnum):
|
||||||
|
PLAIN = "plain"
|
||||||
|
TLS = "tls"
|
||||||
|
STARTTLS = "starttls"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def standard_port(self) -> int:
|
||||||
|
return 465 if self == TransportSecurity.TLS else 587
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MailServerSettings:
|
||||||
|
server: str = ""
|
||||||
|
port: int | None = None
|
||||||
|
username: str = ""
|
||||||
|
password: str = ""
|
||||||
|
transport_security: TransportSecurity = TransportSecurity.PLAIN
|
||||||
|
|
||||||
|
def use_plain(self) -> "MailServerSettings":
|
||||||
|
self.transport_security = TransportSecurity.PLAIN
|
||||||
|
self.port = self.port or self.transport_security.standard_port
|
||||||
|
return self
|
||||||
|
|
||||||
|
def use_tls(self) -> "MailServerSettings":
|
||||||
|
self.transport_security = TransportSecurity.TLS
|
||||||
|
self.port = self.port or self.transport_security.standard_port
|
||||||
|
return self
|
||||||
|
|
||||||
|
def use_starttls(self) -> "MailServerSettings":
|
||||||
|
self.transport_security = TransportSecurity.STARTTLS
|
||||||
|
self.port = self.port or self.transport_security.standard_port
|
||||||
|
return self
|
||||||
|
|
||||||
|
def resolved_port(self) -> int:
|
||||||
|
return self.port or self.transport_security.standard_port
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class MailAttachmentConfig:
|
||||||
|
base_dir: Path
|
||||||
|
file_filter: str = "*"
|
||||||
|
include_subdirs: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MailEntry:
|
||||||
|
field_config: FieldConfiguration
|
||||||
|
is_active: bool = True
|
||||||
|
from_recipient: Recipient | None = None
|
||||||
|
to: RecipientList = field(default_factory=RecipientList)
|
||||||
|
cc: RecipientList = field(default_factory=RecipientList)
|
||||||
|
bcc: RecipientList = field(default_factory=RecipientList)
|
||||||
|
combine_to: bool = True
|
||||||
|
combine_cc: bool = True
|
||||||
|
combine_bcc: bool = True
|
||||||
|
attachment_configs: list[MailAttachmentConfig] = field(default_factory=list)
|
||||||
|
combine_attachments: bool = True
|
||||||
|
field_contents: FieldContents = field(init=False)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
self.field_contents = FieldContents(self.field_config)
|
||||||
|
|
||||||
|
def add_to(self, recipient: Recipient) -> "MailEntry":
|
||||||
|
self.to.add_recipient(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_cc(self, recipient: Recipient) -> "MailEntry":
|
||||||
|
self.cc.add_recipient(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_bcc(self, recipient: Recipient) -> "MailEntry":
|
||||||
|
self.bcc.add_recipient(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def no_combine_to(self) -> "MailEntry":
|
||||||
|
self.combine_to = False
|
||||||
|
return self
|
||||||
|
|
||||||
|
def combine_to_recipients(self) -> "MailEntry":
|
||||||
|
self.combine_to = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def no_combine_attachments(self) -> "MailEntry":
|
||||||
|
self.combine_attachments = False
|
||||||
|
return self
|
||||||
|
|
||||||
|
def combine_attachments_with_global(self) -> "MailEntry":
|
||||||
|
self.combine_attachments = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_mail_attachment_config(self, config: MailAttachmentConfig) -> "MailEntry":
|
||||||
|
self.attachment_configs.append(config)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def set_field_content_for_name(self, name: str, value: Field | object) -> "MailEntry":
|
||||||
|
if not self.field_contents.set_field_content_for_name(name, value):
|
||||||
|
raise KeyError(f"unknown field: {name}")
|
||||||
|
return self
|
||||||
|
|
||||||
|
def get_field_content_from_name(self, name: str) -> Field:
|
||||||
|
return self.field_contents.get_field_content_from_name(name)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MailCampaign:
|
||||||
|
mail_server_settings: MailServerSettings | None = None
|
||||||
|
global_from: Recipient | None = None
|
||||||
|
global_to: RecipientList = field(default_factory=RecipientList)
|
||||||
|
global_cc: RecipientList = field(default_factory=RecipientList)
|
||||||
|
global_bcc: RecipientList = field(default_factory=RecipientList)
|
||||||
|
individual_from: bool = False
|
||||||
|
individual_to: bool = False
|
||||||
|
individual_cc: bool = False
|
||||||
|
individual_bcc: bool = False
|
||||||
|
base_attachment_path: Path = Path(".")
|
||||||
|
global_attachment_configs: list[MailAttachmentConfig] = field(default_factory=list)
|
||||||
|
individual_attachments: bool = False
|
||||||
|
send_without_attachments: bool = True
|
||||||
|
field_config: FieldConfiguration = field(default_factory=FieldConfiguration)
|
||||||
|
field_contents: FieldContents = field(init=False)
|
||||||
|
subject_template: MailTemplate = field(default_factory=MailTemplate)
|
||||||
|
mail_template: MailTemplate = field(default_factory=MailTemplate)
|
||||||
|
mail_entries: list[MailEntry] = field(default_factory=list)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
self.field_contents = FieldContents(self.field_config)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def with_server_settings(cls, settings: MailServerSettings) -> "MailCampaign":
|
||||||
|
return cls(mail_server_settings=settings)
|
||||||
|
|
||||||
|
def add_field(self, name: str, field_type) -> "MailCampaign":
|
||||||
|
from .fields import FieldDescription
|
||||||
|
self.field_config.add_field_at_end(FieldDescription(name, field_type))
|
||||||
|
self.field_contents.ensure_field(self.field_config.get_field_description(name)) # type: ignore[arg-type]
|
||||||
|
for entry in self.mail_entries:
|
||||||
|
entry.field_contents.ensure_field(self.field_config.get_field_description(name)) # type: ignore[arg-type]
|
||||||
|
return self
|
||||||
|
|
||||||
|
def set_from(self, recipient: Recipient) -> "MailCampaign":
|
||||||
|
self.global_from = recipient
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_to(self, recipient: Recipient) -> "MailCampaign":
|
||||||
|
self.global_to.add_recipient(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def allow_individual_to(self) -> "MailCampaign":
|
||||||
|
self.individual_to = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def disallow_individual_to(self) -> "MailCampaign":
|
||||||
|
self.individual_to = False
|
||||||
|
return self
|
||||||
|
|
||||||
|
def allow_individual_attachments(self) -> "MailCampaign":
|
||||||
|
self.individual_attachments = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def disallow_individual_attachments(self) -> "MailCampaign":
|
||||||
|
self.individual_attachments = False
|
||||||
|
return self
|
||||||
|
|
||||||
|
def dont_send_without_attachments(self) -> "MailCampaign":
|
||||||
|
self.send_without_attachments = False
|
||||||
|
return self
|
||||||
|
|
||||||
|
def send_without_attachments_allowed(self) -> "MailCampaign":
|
||||||
|
self.send_without_attachments = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_new_mail_entry(self) -> MailEntry:
|
||||||
|
entry = MailEntry(self.field_config)
|
||||||
|
self.mail_entries.append(entry)
|
||||||
|
return entry
|
||||||
|
|
||||||
|
def set_field_content_for_name(self, name: str, value: Field | object) -> "MailCampaign":
|
||||||
|
if not self.field_contents.set_field_content_for_name(name, value):
|
||||||
|
raise KeyError(f"unknown field: {name}")
|
||||||
|
return self
|
||||||
|
|
||||||
|
def get_field_content_from_name(self, name: str) -> Field:
|
||||||
|
return self.field_contents.get_field_content_from_name(name)
|
||||||
|
|
||||||
|
def all_recipients_for(self, entry: MailEntry) -> list[Recipient]:
|
||||||
|
recipients: list[Recipient] = []
|
||||||
|
if not self.individual_to or entry.combine_to:
|
||||||
|
recipients.extend(self.global_to.recipients)
|
||||||
|
if not self.individual_cc or entry.combine_cc:
|
||||||
|
recipients.extend(self.global_cc.recipients)
|
||||||
|
if not self.individual_bcc or entry.combine_bcc:
|
||||||
|
recipients.extend(self.global_bcc.recipients)
|
||||||
|
if self.individual_to:
|
||||||
|
recipients.extend(entry.to.recipients)
|
||||||
|
if self.individual_cc:
|
||||||
|
recipients.extend(entry.cc.recipients)
|
||||||
|
if self.individual_bcc:
|
||||||
|
recipients.extend(entry.bcc.recipients)
|
||||||
|
return recipients
|
||||||
125
server/app/mailer/domain/fields.py
Normal file
125
server/app/mailer/domain/fields.py
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import date, datetime
|
||||||
|
from enum import StrEnum
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class FieldType(StrEnum):
|
||||||
|
STRING = "string"
|
||||||
|
INTEGER = "integer"
|
||||||
|
DOUBLE = "double"
|
||||||
|
DATE = "date"
|
||||||
|
PASSWORD = "password"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class FieldDescription:
|
||||||
|
name: str
|
||||||
|
type: FieldType = FieldType.STRING
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class Field:
|
||||||
|
content: Any
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def with_content(cls, content: Any) -> "Field":
|
||||||
|
if content is None:
|
||||||
|
raise ValueError("content must not be None")
|
||||||
|
return cls(content=content)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def type(self) -> FieldType:
|
||||||
|
if isinstance(self.content, bool):
|
||||||
|
return FieldType.STRING
|
||||||
|
if isinstance(self.content, int):
|
||||||
|
return FieldType.INTEGER
|
||||||
|
if isinstance(self.content, float):
|
||||||
|
return FieldType.DOUBLE
|
||||||
|
if isinstance(self.content, (date, datetime)):
|
||||||
|
return FieldType.DATE
|
||||||
|
if isinstance(self.content, (bytes, bytearray)):
|
||||||
|
return FieldType.PASSWORD
|
||||||
|
return FieldType.STRING
|
||||||
|
|
||||||
|
def as_string(self) -> str:
|
||||||
|
if isinstance(self.content, (bytes, bytearray)):
|
||||||
|
return self.content.decode("utf-8")
|
||||||
|
if isinstance(self.content, (date, datetime)):
|
||||||
|
return self.content.isoformat()
|
||||||
|
return str(self.content)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FieldConfiguration:
|
||||||
|
fields: list[FieldDescription] = field(default_factory=list)
|
||||||
|
|
||||||
|
def add_field_at_end(self, field_description: FieldDescription) -> "FieldConfiguration":
|
||||||
|
return self.add_field_at_position(len(self.fields), field_description)
|
||||||
|
|
||||||
|
def add_field_at_start(self, field_description: FieldDescription) -> "FieldConfiguration":
|
||||||
|
return self.add_field_at_position(0, field_description)
|
||||||
|
|
||||||
|
def add_field_at_position(self, position: int, field_description: FieldDescription) -> "FieldConfiguration":
|
||||||
|
if self.has_field(field_description.name):
|
||||||
|
raise ValueError(f"field already exists: {field_description.name}")
|
||||||
|
position = max(0, min(position, len(self.fields)))
|
||||||
|
self.fields.insert(position, field_description)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def has_field(self, name: str) -> bool:
|
||||||
|
return any(f.name == name for f in self.fields)
|
||||||
|
|
||||||
|
def get_field_description(self, name: str) -> FieldDescription | None:
|
||||||
|
return next((f for f in self.fields if f.name == name), None)
|
||||||
|
|
||||||
|
def get_field_names(self) -> list[str]:
|
||||||
|
return [f.name for f in self.fields]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FieldContents:
|
||||||
|
field_config: FieldConfiguration
|
||||||
|
field_map: dict[str, Field] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for field_description in self.field_config.fields:
|
||||||
|
self.ensure_field(field_description)
|
||||||
|
|
||||||
|
def ensure_field(self, field_description: FieldDescription) -> None:
|
||||||
|
if field_description.name in self.field_map:
|
||||||
|
return
|
||||||
|
match field_description.type:
|
||||||
|
case FieldType.INTEGER:
|
||||||
|
value = 0
|
||||||
|
case FieldType.DOUBLE:
|
||||||
|
value = 0.0
|
||||||
|
case FieldType.DATE:
|
||||||
|
value = date.today()
|
||||||
|
case FieldType.PASSWORD:
|
||||||
|
value = b""
|
||||||
|
case _:
|
||||||
|
value = ""
|
||||||
|
self.field_map[field_description.name] = Field.with_content(value)
|
||||||
|
|
||||||
|
def get_field_content_from_name(self, name: str) -> Field:
|
||||||
|
try:
|
||||||
|
return self.field_map[name]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise KeyError(f"unknown field: {name}") from exc
|
||||||
|
|
||||||
|
def set_field_content_for_name(self, name: str, value: Field | Any) -> bool:
|
||||||
|
if name not in self.field_map:
|
||||||
|
return False
|
||||||
|
if not isinstance(value, Field):
|
||||||
|
value = Field.with_content(value)
|
||||||
|
expected = self.field_map[name].type
|
||||||
|
if expected != value.type and expected != FieldType.PASSWORD:
|
||||||
|
raise TypeError(f"field {name!r} expects {expected}, got {value.type}")
|
||||||
|
self.field_map[name] = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
def as_value_map(self, prefix: str) -> dict[str, str]:
|
||||||
|
return {f"{prefix}::{name}": field.as_string() for name, field in self.field_map.items()}
|
||||||
29
server/app/mailer/domain/queue.py
Normal file
29
server/app/mailer/domain/queue.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from typing import Iterator
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MailQueue:
|
||||||
|
messages: list[EmailMessage] = field(default_factory=list)
|
||||||
|
|
||||||
|
def add_mail(self, message: EmailMessage) -> None:
|
||||||
|
self.messages.append(message)
|
||||||
|
|
||||||
|
def remove_mail(self, message: EmailMessage) -> bool:
|
||||||
|
if message in self.messages:
|
||||||
|
self.messages.remove(message)
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def mail_count(self) -> int:
|
||||||
|
return len(self.messages)
|
||||||
|
|
||||||
|
def is_empty(self) -> bool:
|
||||||
|
return not self.messages
|
||||||
|
|
||||||
|
def __iter__(self) -> Iterator[EmailMessage]:
|
||||||
|
return iter(self.messages)
|
||||||
43
server/app/mailer/domain/recipients.py
Normal file
43
server/app/mailer/domain/recipients.py
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from email.utils import formataddr
|
||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
|
|
||||||
|
class RecipientType(StrEnum):
|
||||||
|
TO = "to"
|
||||||
|
CC = "cc"
|
||||||
|
BCC = "bcc"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Recipient:
|
||||||
|
address: str
|
||||||
|
name: str | None = None
|
||||||
|
type: RecipientType = RecipientType.TO
|
||||||
|
|
||||||
|
def formatted(self) -> str:
|
||||||
|
return formataddr((self.name or self.address, self.address))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RecipientList:
|
||||||
|
recipients: list[Recipient] = field(default_factory=list)
|
||||||
|
|
||||||
|
def add_recipient(self, recipient: Recipient) -> "RecipientList":
|
||||||
|
if recipient not in self.recipients:
|
||||||
|
self.recipients.append(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_multiple_recipients(self, recipients: list[Recipient] | tuple[Recipient, ...]) -> "RecipientList":
|
||||||
|
for recipient in recipients:
|
||||||
|
self.add_recipient(recipient)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def clear_all_recipients(self) -> "RecipientList":
|
||||||
|
self.recipients.clear()
|
||||||
|
return self
|
||||||
|
|
||||||
|
def by_type(self, recipient_type: RecipientType) -> list[Recipient]:
|
||||||
|
return [r for r in self.recipients if r.type == recipient_type]
|
||||||
28
server/app/mailer/domain/template.py
Normal file
28
server/app/mailer/domain/template.py
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MailTemplate:
|
||||||
|
template_string: str = ""
|
||||||
|
|
||||||
|
def set_template_string(self, template: str) -> "MailTemplate":
|
||||||
|
self.template_string = template
|
||||||
|
return self
|
||||||
|
|
||||||
|
def get_used_fields(self) -> set[str]:
|
||||||
|
return set(_FIELD_PATTERN.findall(self.template_string))
|
||||||
|
|
||||||
|
def apply_values(self, values: dict[str, str], *, keep_missing: bool = True) -> str:
|
||||||
|
def replace(match: re.Match[str]) -> str:
|
||||||
|
key = match.group(1)
|
||||||
|
if key in values:
|
||||||
|
return values[key]
|
||||||
|
return match.group(0) if keep_missing else ""
|
||||||
|
|
||||||
|
rendered = _FIELD_PATTERN.sub(replace, self.template_string)
|
||||||
|
return rendered.replace(r"\${", "${").replace(r"\}", "}")
|
||||||
0
server/app/mailer/examples/__init__.py
Normal file
0
server/app/mailer/examples/__init__.py
Normal file
180
server/app/mailer/examples/campaign.json
Normal file
180
server/app/mailer/examples/campaign.json
Normal file
@@ -0,0 +1,180 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "rechnungslegung-2026-05",
|
||||||
|
"name": "Rechnungslegung 2026-05",
|
||||||
|
"mode": "draft",
|
||||||
|
"description": "Example campaign migrated from the Java object model."
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "monthyear",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Month/year",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "number",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Dienststelle",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "anrede",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Salutation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "zip_password",
|
||||||
|
"type": "password",
|
||||||
|
"label": "ZIP password",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"global_values": {
|
||||||
|
"monthyear": "05 / 2026"
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "smtp.example.org",
|
||||||
|
"port": 587,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "starttls"
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": true,
|
||||||
|
"host": "imap.example.org",
|
||||||
|
"port": 993,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "to"
|
||||||
|
},
|
||||||
|
"allow_individual_from": false,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": true,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": false,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": false,
|
||||||
|
"reply_to": [
|
||||||
|
{
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "reply_to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_reply_to": false,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": false,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": false
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}",
|
||||||
|
"text": "${local::anrede},\n\nbeigefügt erhalten Sie die Rechnungslegungslisten für ${global::monthyear}.\n\nMit freundlichen Grüßen"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": "./data/attachments",
|
||||||
|
"allow_individual": true,
|
||||||
|
"send_without_attachments": false,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask"
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"source": {
|
||||||
|
"type": "csv",
|
||||||
|
"path": "./data/recipients.csv",
|
||||||
|
"delimiter": ";",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"has_header": true
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"id": "ID",
|
||||||
|
"active": "Aktiv",
|
||||||
|
"to.0.email": "E-Mail",
|
||||||
|
"to.0.name": "Name",
|
||||||
|
"fields.number": "Dienststelle",
|
||||||
|
"fields.anrede": "Anrede",
|
||||||
|
"fields.zip_password": "ZIP-Passwort",
|
||||||
|
"attachments.0.base_dir": "Unterordner",
|
||||||
|
"attachments.0.file_filter": "Dateimuster"
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"active": true,
|
||||||
|
"to": [],
|
||||||
|
"combine_to": false,
|
||||||
|
"cc": [],
|
||||||
|
"combine_cc": true,
|
||||||
|
"bcc": [],
|
||||||
|
"combine_bcc": true,
|
||||||
|
"reply_to": [],
|
||||||
|
"combine_reply_to": true,
|
||||||
|
"bounce_to": [],
|
||||||
|
"combine_bounce_to": true,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"combine_disposition_notification_to": true,
|
||||||
|
"attachments": [
|
||||||
|
{
|
||||||
|
"id": "individual-documents",
|
||||||
|
"label": "Personalized PDF bundle",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "${local::number}_*.pdf",
|
||||||
|
"include_subdirs": false,
|
||||||
|
"required": true,
|
||||||
|
"allow_multiple": true,
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "continue",
|
||||||
|
"zip": {
|
||||||
|
"enabled": true,
|
||||||
|
"filename_template": "Rechnungslegung_${local::number}.zip",
|
||||||
|
"password_template": "${local::zip_password}",
|
||||||
|
"method": "aes"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"combine_attachments": true,
|
||||||
|
"fields": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"inactive_entry": "drop"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "auto"
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true,
|
||||||
|
"initial_build_status": "built",
|
||||||
|
"initial_send_status": "draft"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
dummy example attachment for resolver smoke tests
|
||||||
2
server/app/mailer/examples/data/recipients.csv
Normal file
2
server/app/mailer/examples/data/recipients.csv
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
ID;Aktiv;E-Mail;Name;Dienststelle;Anrede;ZIP-Passwort;Unterordner;Dateimuster
|
||||||
|
entry-001;true;mail@example.com;Example Recipient;ab0000;Sehr geehrte Damen und Herren;secret-demo;xls;ab????-123456-*.XLSX
|
||||||
|
73
server/app/mailer/examples/rechnungslegung_2026_05.py
Normal file
73
server/app/mailer/examples/rechnungslegung_2026_05.py
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
"""Python port of the provided Java MultiMailerSettings example.
|
||||||
|
|
||||||
|
This is intentionally safe: credentials and real recipients are placeholders.
|
||||||
|
Run from server/ with:
|
||||||
|
python -m app.mailer.examples.rechnungslegung_2026_05
|
||||||
|
"""
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.domain.campaign import MailAttachmentConfig, MailCampaign, MailServerSettings
|
||||||
|
from app.mailer.domain.fields import FieldType
|
||||||
|
from app.mailer.domain.recipients import Recipient
|
||||||
|
from app.mailer.services.campaign_executor import build_mail_queue
|
||||||
|
|
||||||
|
|
||||||
|
def build_campaign() -> MailCampaign:
|
||||||
|
mail_settings = MailServerSettings(
|
||||||
|
server="smtp.example.org",
|
||||||
|
username="user@example.org",
|
||||||
|
password="change-me",
|
||||||
|
).use_starttls()
|
||||||
|
|
||||||
|
campaign = MailCampaign.with_server_settings(mail_settings)
|
||||||
|
campaign.set_from(Recipient(address="d5-rechnungslegung@example.org", name="Rechnungslegung D5"))
|
||||||
|
campaign.allow_individual_to()
|
||||||
|
campaign.allow_individual_attachments()
|
||||||
|
campaign.dont_send_without_attachments()
|
||||||
|
campaign.base_attachment_path = Path("/mnt/FLASH/rele/202606")
|
||||||
|
|
||||||
|
campaign.add_field("monthyear", FieldType.STRING)
|
||||||
|
campaign.add_field("number", FieldType.STRING)
|
||||||
|
campaign.add_field("password", FieldType.PASSWORD)
|
||||||
|
campaign.add_field("anrede", FieldType.STRING)
|
||||||
|
|
||||||
|
campaign.set_field_content_for_name("monthyear", "05 / 2026")
|
||||||
|
campaign.subject_template.set_template_string(
|
||||||
|
"Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}"
|
||||||
|
)
|
||||||
|
campaign.mail_template.set_template_string(
|
||||||
|
"${local::anrede},\r\n\r\n"
|
||||||
|
"in der Anlage erhalten Sie die Rechnungslegungslisten für die Dienststelle "
|
||||||
|
"${local::number} für den Abrechnungsmonat ${global::monthyear} im Excel-Format. "
|
||||||
|
"Bitte verwenden Sie zum öffnen das dauerhafte Passwort, das Ihnen bereits in der Vergangenheit zugeschickt wurde.\r\n"
|
||||||
|
"Die Rechnungslegungslisten liefern den Nachweis (inkl. Brutto-/Netto-Darstellung) "
|
||||||
|
"der auf Ihren dezentral bewirtschafteten Fonds gebuchten Personalkosten. Sie dienen der "
|
||||||
|
"Überwachung und Kontrolle und ggf. als Nachweis gegenüber Drittmittelgebern.\r\n"
|
||||||
|
"Die Listen erhalten vertrauliche personenbezogene Daten, daher sind diese nur berechtigten "
|
||||||
|
"Personen zugänglich zu machen und nur für einen unbedingt notwendigen Zeitraum aufzubewahren.\r\n"
|
||||||
|
"Falls Sie Rechnungslegungslisten erhalten haben sollten, die nicht zu Ihrer Einrichtung gehören, "
|
||||||
|
"bitten wir Sie um entsprechende Rückmeldung.\r\n\r\n"
|
||||||
|
"Mit freundlichen Grüßen\r\n\r\n"
|
||||||
|
"Rechnungslegungsteam Dezernat 5"
|
||||||
|
)
|
||||||
|
|
||||||
|
campaign.add_new_mail_entry() \
|
||||||
|
.add_to(Recipient(address="mail@example.com", name="mail@example.com")) \
|
||||||
|
.no_combine_to() \
|
||||||
|
.combine_attachments_with_global() \
|
||||||
|
.add_mail_attachment_config(MailAttachmentConfig(Path("xls/"), "ab????-123456-*.XLSX", False)) \
|
||||||
|
.set_field_content_for_name("number", "ab0000") \
|
||||||
|
.set_field_content_for_name("password", b"..........") \
|
||||||
|
.set_field_content_for_name("anrede", "Sehr geehrte Damen und Herren")
|
||||||
|
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
mc = build_campaign()
|
||||||
|
queue = build_mail_queue(mc, zip_attachments=False)
|
||||||
|
print(f"Built {queue.mail_count} message(s).")
|
||||||
|
for message in queue:
|
||||||
|
print("---")
|
||||||
|
print("To:", message.get("To"))
|
||||||
|
print("Subject:", message.get("Subject"))
|
||||||
12
server/app/mailer/messages/__init__.py
Normal file
12
server/app/mailer/messages/__init__.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
"""Message building and review helpers."""
|
||||||
|
|
||||||
|
from .builder import build_campaign_messages
|
||||||
|
from .models import CampaignBuildReport, MessageDraft, MessageIssue, MessageValidationStatus
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"build_campaign_messages",
|
||||||
|
"CampaignBuildReport",
|
||||||
|
"MessageDraft",
|
||||||
|
"MessageIssue",
|
||||||
|
"MessageValidationStatus",
|
||||||
|
]
|
||||||
547
server/app/mailer/messages/builder.py
Normal file
547
server/app/mailer/messages/builder.py
Normal file
@@ -0,0 +1,547 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import mimetypes
|
||||||
|
import re
|
||||||
|
import tempfile
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from email.utils import formataddr, make_msgid, formatdate
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
from app.mailer.attachments.resolver import (
|
||||||
|
AttachmentMatchStatus,
|
||||||
|
EntryAttachmentResolution,
|
||||||
|
MessageAttachmentStatus,
|
||||||
|
ResolvedAttachment,
|
||||||
|
resolve_entry_attachments,
|
||||||
|
)
|
||||||
|
from app.mailer.campaign.entries import load_campaign_entries
|
||||||
|
from app.mailer.campaign.models import (
|
||||||
|
Behavior,
|
||||||
|
BuildStatus,
|
||||||
|
CampaignConfig,
|
||||||
|
EntryConfig,
|
||||||
|
MissingAddressBehavior,
|
||||||
|
RecipientConfig,
|
||||||
|
SendStatus,
|
||||||
|
)
|
||||||
|
from app.mailer.services.zip_service import create_encrypted_zip
|
||||||
|
|
||||||
|
from .models import (
|
||||||
|
CampaignBuildReport,
|
||||||
|
ImapStatus,
|
||||||
|
MessageAddress,
|
||||||
|
MessageAttachmentSummary,
|
||||||
|
MessageDraft,
|
||||||
|
MessageIssue,
|
||||||
|
MessageValidationStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class BuiltMessage:
|
||||||
|
draft: MessageDraft
|
||||||
|
mime: EmailMessage | None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class CampaignBuildResult:
|
||||||
|
report: CampaignBuildReport
|
||||||
|
built_messages: list[BuiltMessage]
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve(campaign_file: str | Path, raw_path: str) -> Path:
|
||||||
|
campaign_path = Path(campaign_file).resolve()
|
||||||
|
path = Path(raw_path).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return path
|
||||||
|
return (campaign_path.parent / path).resolve()
|
||||||
|
|
||||||
|
|
||||||
|
def _read_text(campaign_file: str | Path, raw_path: str | None, encoding: str = "utf-8") -> str | None:
|
||||||
|
if not raw_path:
|
||||||
|
return None
|
||||||
|
path = _resolve(campaign_file, raw_path)
|
||||||
|
return path.read_text(encoding=encoding)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_template(template: str, values: dict[str, Any], *, keep_missing: bool = True) -> str:
|
||||||
|
def replace(match: re.Match[str]) -> str:
|
||||||
|
key = match.group(1)
|
||||||
|
if key in values:
|
||||||
|
value = values[key]
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
return match.group(0) if keep_missing else ""
|
||||||
|
|
||||||
|
rendered = _FIELD_PATTERN.sub(replace, template)
|
||||||
|
return rendered.replace(r"\${", "${").replace(r"\}", "}")
|
||||||
|
|
||||||
|
|
||||||
|
def _find_unresolved_placeholders(text: str | None) -> set[str]:
|
||||||
|
if not text:
|
||||||
|
return set()
|
||||||
|
return set(_FIELD_PATTERN.findall(text))
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_values(entry: EntryConfig) -> dict[str, str]:
|
||||||
|
values: dict[str, str] = {}
|
||||||
|
for list_name in ["to", "cc", "bcc", "reply_to", "bounce_to", "disposition_notification_to"]:
|
||||||
|
recipients = getattr(entry, list_name)
|
||||||
|
for index, recipient in enumerate(recipients):
|
||||||
|
prefix = f"{list_name}.{index}"
|
||||||
|
values[f"local::{prefix}.email"] = recipient.email
|
||||||
|
values[f"local::{prefix}.name"] = recipient.name or ""
|
||||||
|
values[f"local::{prefix}.type"] = recipient.recipient_type.value
|
||||||
|
if entry.from_:
|
||||||
|
values["local::from.email"] = entry.from_.email
|
||||||
|
values["local::from.name"] = entry.from_.name or ""
|
||||||
|
values["local::from.type"] = entry.from_.recipient_type.value
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def _template_values(config: CampaignConfig, entry: EntryConfig) -> dict[str, Any]:
|
||||||
|
values: dict[str, Any] = {}
|
||||||
|
for key, value in config.global_values.items():
|
||||||
|
values[f"global::{key}"] = value
|
||||||
|
for key, value in entry.fields.items():
|
||||||
|
values[f"local::{key}"] = value
|
||||||
|
if entry.id:
|
||||||
|
values["local::id"] = entry.id
|
||||||
|
values["local::active"] = entry.active
|
||||||
|
values.update(_recipient_values(entry))
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def _message_address(recipient: RecipientConfig | None) -> MessageAddress | None:
|
||||||
|
if recipient is None:
|
||||||
|
return None
|
||||||
|
return MessageAddress(email=recipient.email, name=recipient.name)
|
||||||
|
|
||||||
|
|
||||||
|
def _message_addresses(recipients: Iterable[RecipientConfig]) -> list[MessageAddress]:
|
||||||
|
return [MessageAddress(email=recipient.email, name=recipient.name) for recipient in recipients]
|
||||||
|
|
||||||
|
|
||||||
|
def _format_recipient(recipient: RecipientConfig) -> str:
|
||||||
|
return formataddr((recipient.name or recipient.email, recipient.email))
|
||||||
|
|
||||||
|
|
||||||
|
def _format_recipient_header(recipients: Iterable[RecipientConfig]) -> str:
|
||||||
|
return ", ".join(_format_recipient(recipient) for recipient in recipients)
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_sender(config: CampaignConfig, entry: EntryConfig) -> RecipientConfig | None:
|
||||||
|
if config.recipients.allow_individual_from and entry.from_:
|
||||||
|
return entry.from_
|
||||||
|
return config.recipients.from_
|
||||||
|
|
||||||
|
|
||||||
|
def _combine_recipients(
|
||||||
|
*,
|
||||||
|
allow_individual: bool,
|
||||||
|
combine: bool,
|
||||||
|
global_recipients: list[RecipientConfig],
|
||||||
|
entry_recipients: list[RecipientConfig],
|
||||||
|
) -> list[RecipientConfig]:
|
||||||
|
recipients: list[RecipientConfig] = []
|
||||||
|
if not allow_individual or combine:
|
||||||
|
recipients.extend(global_recipients)
|
||||||
|
if allow_individual:
|
||||||
|
recipients.extend(entry_recipients)
|
||||||
|
# keep order while avoiding exact duplicate email/type pairs
|
||||||
|
seen: set[tuple[str, str]] = set()
|
||||||
|
unique: list[RecipientConfig] = []
|
||||||
|
for recipient in recipients:
|
||||||
|
key = (recipient.email.lower(), recipient.recipient_type.value)
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
unique.append(recipient)
|
||||||
|
return unique
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_recipients(config: CampaignConfig, entry: EntryConfig) -> dict[str, list[RecipientConfig]]:
|
||||||
|
return {
|
||||||
|
"to": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_to,
|
||||||
|
combine=entry.combine_to,
|
||||||
|
global_recipients=config.recipients.to,
|
||||||
|
entry_recipients=entry.to,
|
||||||
|
),
|
||||||
|
"cc": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_cc,
|
||||||
|
combine=entry.combine_cc,
|
||||||
|
global_recipients=config.recipients.cc,
|
||||||
|
entry_recipients=entry.cc,
|
||||||
|
),
|
||||||
|
"bcc": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_bcc,
|
||||||
|
combine=entry.combine_bcc,
|
||||||
|
global_recipients=config.recipients.bcc,
|
||||||
|
entry_recipients=entry.bcc,
|
||||||
|
),
|
||||||
|
"reply_to": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_reply_to,
|
||||||
|
combine=entry.combine_reply_to,
|
||||||
|
global_recipients=config.recipients.reply_to,
|
||||||
|
entry_recipients=entry.reply_to,
|
||||||
|
),
|
||||||
|
"bounce_to": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_bounce_to,
|
||||||
|
combine=entry.combine_bounce_to,
|
||||||
|
global_recipients=config.recipients.bounce_to,
|
||||||
|
entry_recipients=entry.bounce_to,
|
||||||
|
),
|
||||||
|
"disposition_notification_to": _combine_recipients(
|
||||||
|
allow_individual=config.recipients.allow_individual_disposition_notification_to,
|
||||||
|
combine=entry.combine_disposition_notification_to,
|
||||||
|
global_recipients=config.recipients.disposition_notification_to,
|
||||||
|
entry_recipients=entry.disposition_notification_to,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_template_parts(config: CampaignConfig, campaign_file: str | Path) -> tuple[str, str | None, str | None]:
|
||||||
|
template = config.template
|
||||||
|
if template.source:
|
||||||
|
subject = _read_text(campaign_file, template.source.subject_path, template.source.encoding)
|
||||||
|
text = _read_text(campaign_file, template.source.text_path, template.source.encoding)
|
||||||
|
html = _read_text(campaign_file, template.source.html_path, template.source.encoding)
|
||||||
|
return subject or "", text, html
|
||||||
|
return template.subject or "", template.text, template.html
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_from_behavior(*, code: str, message: str, behavior: str, source: str) -> MessageIssue:
|
||||||
|
severity = "error" if behavior == "block" else "warning"
|
||||||
|
return MessageIssue(severity=severity, code=code, message=message, behavior=behavior, source=source)
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_behavior(current: MessageValidationStatus, behavior: str) -> MessageValidationStatus:
|
||||||
|
if behavior == Behavior.BLOCK.value:
|
||||||
|
return MessageValidationStatus.BLOCKED
|
||||||
|
if behavior == Behavior.DROP.value:
|
||||||
|
return MessageValidationStatus.EXCLUDED
|
||||||
|
if behavior == Behavior.ASK.value:
|
||||||
|
if current not in {MessageValidationStatus.BLOCKED, MessageValidationStatus.EXCLUDED}:
|
||||||
|
return MessageValidationStatus.NEEDS_REVIEW
|
||||||
|
if behavior == Behavior.WARN.value:
|
||||||
|
if current == MessageValidationStatus.READY:
|
||||||
|
return MessageValidationStatus.WARNING
|
||||||
|
# continue leaves status as-is
|
||||||
|
return current
|
||||||
|
|
||||||
|
|
||||||
|
def _validation_status_from_attachment_status(status: MessageAttachmentStatus) -> MessageValidationStatus:
|
||||||
|
return MessageValidationStatus(status.value)
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_summaries(resolution: EntryAttachmentResolution) -> list[MessageAttachmentSummary]:
|
||||||
|
return [
|
||||||
|
MessageAttachmentSummary(
|
||||||
|
attachment_id=attachment.attachment_id,
|
||||||
|
label=attachment.label,
|
||||||
|
status=attachment.status.value,
|
||||||
|
behavior=attachment.behavior.value if attachment.behavior else None,
|
||||||
|
required=attachment.required,
|
||||||
|
allow_multiple=attachment.allow_multiple,
|
||||||
|
zip_enabled=attachment.zip_enabled,
|
||||||
|
file_filter=attachment.file_filter,
|
||||||
|
directory=attachment.directory,
|
||||||
|
matches=attachment.matches,
|
||||||
|
)
|
||||||
|
for attachment in resolution.attachments
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _message_issues_from_attachment_resolution(resolution: EntryAttachmentResolution) -> list[MessageIssue]:
|
||||||
|
return [
|
||||||
|
MessageIssue(
|
||||||
|
severity=issue.severity.value,
|
||||||
|
code=issue.code,
|
||||||
|
message=issue.message,
|
||||||
|
behavior=issue.behavior.value if issue.behavior else None,
|
||||||
|
source="attachments",
|
||||||
|
)
|
||||||
|
for issue in resolution.issues
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_filename(value: str | None, fallback: str) -> str:
|
||||||
|
raw = value or fallback
|
||||||
|
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", raw).strip("._")
|
||||||
|
return safe or fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_bytes(path: Path) -> tuple[bytes, str, str]:
|
||||||
|
data = path.read_bytes()
|
||||||
|
mime_type, _ = mimetypes.guess_type(str(path))
|
||||||
|
if not mime_type:
|
||||||
|
return data, "application", "octet-stream"
|
||||||
|
maintype, subtype = mime_type.split("/", 1)
|
||||||
|
return data, maintype, subtype
|
||||||
|
|
||||||
|
|
||||||
|
def _render_zip_filename(
|
||||||
|
*,
|
||||||
|
attachment: ResolvedAttachment,
|
||||||
|
values: dict[str, Any],
|
||||||
|
entry: EntryConfig,
|
||||||
|
default_index: int,
|
||||||
|
) -> str:
|
||||||
|
template = attachment.attachment_id or attachment.label or f"attachments-{default_index}"
|
||||||
|
# The resolver summary does not carry the full ZipConfig, so the build step receives
|
||||||
|
# filename/password through the resolved attachment's original config by re-resolving
|
||||||
|
# via a private companion in _zip_config_for_attachment.
|
||||||
|
rendered = _render_template(template, values, keep_missing=False)
|
||||||
|
if not rendered.lower().endswith(".zip"):
|
||||||
|
rendered += ".zip"
|
||||||
|
return _safe_filename(rendered, f"entry-{entry.id or default_index}.zip")
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_attachment_configs_for_resolution(config: CampaignConfig, entry: EntryConfig):
|
||||||
|
if entry.combine_attachments:
|
||||||
|
for index, attachment_config in enumerate(config.attachments.global_):
|
||||||
|
yield "global", index, attachment_config
|
||||||
|
if config.attachments.allow_individual:
|
||||||
|
for index, attachment_config in enumerate(entry.attachments):
|
||||||
|
yield "entry", index, attachment_config
|
||||||
|
|
||||||
|
|
||||||
|
def _zip_config_for_attachment(config: CampaignConfig, entry: EntryConfig, resolved: ResolvedAttachment):
|
||||||
|
for scope, index, attachment_config in _iter_attachment_configs_for_resolution(config, entry):
|
||||||
|
if scope == resolved.scope.value and index == resolved.index:
|
||||||
|
return attachment_config.zip
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _attach_files(
|
||||||
|
*,
|
||||||
|
message: EmailMessage,
|
||||||
|
config: CampaignConfig,
|
||||||
|
entry: EntryConfig,
|
||||||
|
resolution: EntryAttachmentResolution,
|
||||||
|
values: dict[str, Any],
|
||||||
|
work_dir: Path,
|
||||||
|
) -> int:
|
||||||
|
attached_count = 0
|
||||||
|
zip_dir = work_dir / "_zip"
|
||||||
|
zip_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
for index, attachment in enumerate(resolution.attachments, start=1):
|
||||||
|
# Missing/ambiguous configs still keep the message draft. They simply do not add files.
|
||||||
|
if attachment.status != AttachmentMatchStatus.OK:
|
||||||
|
continue
|
||||||
|
match_paths = [Path(match) for match in attachment.matches]
|
||||||
|
if not match_paths:
|
||||||
|
continue
|
||||||
|
|
||||||
|
zip_config = _zip_config_for_attachment(config, entry, attachment)
|
||||||
|
if attachment.zip_enabled:
|
||||||
|
filename_template = zip_config.filename_template if zip_config else None
|
||||||
|
if filename_template:
|
||||||
|
filename = _safe_filename(_render_template(filename_template, values, keep_missing=False), f"entry-{entry.entry_id if hasattr(entry, 'entry_id') else index}.zip")
|
||||||
|
if not filename.lower().endswith(".zip"):
|
||||||
|
filename += ".zip"
|
||||||
|
else:
|
||||||
|
filename = _render_zip_filename(attachment=attachment, values=values, entry=entry, default_index=index)
|
||||||
|
password = _render_template(zip_config.password_template or "", values, keep_missing=False) if zip_config else ""
|
||||||
|
zip_path = create_encrypted_zip(zip_dir / filename, match_paths, password)
|
||||||
|
files_to_attach = [zip_path]
|
||||||
|
else:
|
||||||
|
files_to_attach = match_paths
|
||||||
|
|
||||||
|
for path in files_to_attach:
|
||||||
|
data, maintype, subtype = _attachment_bytes(path)
|
||||||
|
message.add_attachment(data, maintype=maintype, subtype=subtype, filename=path.name)
|
||||||
|
attached_count += 1
|
||||||
|
return attached_count
|
||||||
|
|
||||||
|
|
||||||
|
def _imap_initial_status(config: CampaignConfig) -> ImapStatus:
|
||||||
|
if config.delivery.imap_append_sent.enabled:
|
||||||
|
return ImapStatus.PENDING
|
||||||
|
return ImapStatus.NOT_REQUESTED
|
||||||
|
|
||||||
|
|
||||||
|
def _write_eml(message: EmailMessage, output_dir: Path, entry: EntryConfig, entry_index: int) -> tuple[str, int]:
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
filename = _safe_filename(entry.id, f"entry-{entry_index:04d}") + ".eml"
|
||||||
|
path = output_dir / filename
|
||||||
|
path.write_bytes(bytes(message))
|
||||||
|
return str(path), path.stat().st_size
|
||||||
|
|
||||||
|
|
||||||
|
def build_entry_message(
|
||||||
|
*,
|
||||||
|
config: CampaignConfig,
|
||||||
|
campaign_file: str | Path,
|
||||||
|
entry: EntryConfig,
|
||||||
|
entry_index: int,
|
||||||
|
output_dir: Path | None = None,
|
||||||
|
write_eml: bool = False,
|
||||||
|
work_dir: Path | None = None,
|
||||||
|
) -> BuiltMessage:
|
||||||
|
resolution = resolve_entry_attachments(config=config, campaign_file=campaign_file, entry=entry, entry_index=entry_index)
|
||||||
|
recipients = _effective_recipients(config, entry)
|
||||||
|
sender = _effective_sender(config, entry)
|
||||||
|
issues = _message_issues_from_attachment_resolution(resolution)
|
||||||
|
validation_status = _validation_status_from_attachment_status(resolution.status)
|
||||||
|
|
||||||
|
if not entry.active:
|
||||||
|
draft = MessageDraft(
|
||||||
|
entry_index=entry_index,
|
||||||
|
entry_id=entry.id,
|
||||||
|
active=False,
|
||||||
|
build_status=BuildStatus.BUILD_FAILED,
|
||||||
|
validation_status=MessageValidationStatus.INACTIVE,
|
||||||
|
send_status=SendStatus.DRAFT,
|
||||||
|
imap_status=ImapStatus.SKIPPED,
|
||||||
|
from_=_message_address(sender),
|
||||||
|
to=_message_addresses(recipients["to"]),
|
||||||
|
cc=_message_addresses(recipients["cc"]),
|
||||||
|
bcc=_message_addresses(recipients["bcc"]),
|
||||||
|
reply_to=_message_addresses(recipients["reply_to"]),
|
||||||
|
bounce_to=_message_addresses(recipients["bounce_to"]),
|
||||||
|
disposition_notification_to=_message_addresses(recipients["disposition_notification_to"]),
|
||||||
|
attachments=_attachment_summaries(resolution),
|
||||||
|
issues=[MessageIssue(severity="info", code="inactive_entry", message="Entry is inactive", behavior=config.validation_policy.inactive_entry.value, source="entry")],
|
||||||
|
)
|
||||||
|
return BuiltMessage(draft=draft, mime=None)
|
||||||
|
|
||||||
|
if not recipients["to"]:
|
||||||
|
behavior = config.validation_policy.missing_email.value
|
||||||
|
issues.append(_issue_from_behavior(code="missing_email", message="No effective To recipient is configured", behavior=behavior, source="recipients"))
|
||||||
|
validation_status = MessageValidationStatus.BLOCKED if behavior == MissingAddressBehavior.BLOCK.value else MessageValidationStatus.EXCLUDED
|
||||||
|
|
||||||
|
subject_template, text_template, html_template = _load_template_parts(config, campaign_file)
|
||||||
|
values = _template_values(config, entry)
|
||||||
|
subject = _render_template(subject_template, values)
|
||||||
|
text_body = _render_template(text_template or "", values) if text_template is not None else None
|
||||||
|
html_body = _render_template(html_template or "", values) if html_template is not None else None
|
||||||
|
|
||||||
|
unresolved = sorted(
|
||||||
|
_find_unresolved_placeholders(subject)
|
||||||
|
| _find_unresolved_placeholders(text_body)
|
||||||
|
| _find_unresolved_placeholders(html_body)
|
||||||
|
)
|
||||||
|
if unresolved:
|
||||||
|
behavior = config.validation_policy.template_error.value
|
||||||
|
issues.append(
|
||||||
|
_issue_from_behavior(
|
||||||
|
code="template_error",
|
||||||
|
message="Unresolved template placeholder(s): " + ", ".join(unresolved),
|
||||||
|
behavior=behavior,
|
||||||
|
source="template",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
validation_status = MessageValidationStatus.BLOCKED if behavior == MissingAddressBehavior.BLOCK.value else MessageValidationStatus.EXCLUDED
|
||||||
|
|
||||||
|
message = EmailMessage()
|
||||||
|
try:
|
||||||
|
message["Date"] = formatdate(localtime=True)
|
||||||
|
message["Message-ID"] = make_msgid()
|
||||||
|
if sender:
|
||||||
|
message["From"] = _format_recipient(sender)
|
||||||
|
if recipients["to"]:
|
||||||
|
message["To"] = _format_recipient_header(recipients["to"])
|
||||||
|
if recipients["cc"]:
|
||||||
|
message["Cc"] = _format_recipient_header(recipients["cc"])
|
||||||
|
# Bcc deliberately remains envelope-only and is tracked in MessageDraft.
|
||||||
|
if recipients["reply_to"]:
|
||||||
|
message["Reply-To"] = _format_recipient_header(recipients["reply_to"])
|
||||||
|
if recipients["disposition_notification_to"]:
|
||||||
|
message["Disposition-Notification-To"] = _format_recipient_header(recipients["disposition_notification_to"])
|
||||||
|
# bounce_to is tracked but not emitted as Return-Path. That should be the SMTP envelope sender.
|
||||||
|
message["Subject"] = subject
|
||||||
|
|
||||||
|
if html_body is not None:
|
||||||
|
message.set_content(text_body or "")
|
||||||
|
message.add_alternative(html_body, subtype="html")
|
||||||
|
else:
|
||||||
|
message.set_content(text_body or "")
|
||||||
|
|
||||||
|
if work_dir is None:
|
||||||
|
work_dir = output_dir or Path(tempfile.mkdtemp(prefix="multimailer-build-"))
|
||||||
|
attachment_count = _attach_files(
|
||||||
|
message=message,
|
||||||
|
config=config,
|
||||||
|
entry=entry,
|
||||||
|
resolution=resolution,
|
||||||
|
values=values,
|
||||||
|
work_dir=work_dir,
|
||||||
|
)
|
||||||
|
build_status = BuildStatus.BUILT
|
||||||
|
except Exception as exc:
|
||||||
|
issues.append(MessageIssue(severity="error", code="build_failed", message=str(exc), behavior="block", source="builder"))
|
||||||
|
validation_status = MessageValidationStatus.BLOCKED
|
||||||
|
build_status = BuildStatus.BUILD_FAILED
|
||||||
|
attachment_count = 0
|
||||||
|
message = None # type: ignore[assignment]
|
||||||
|
|
||||||
|
eml_path: str | None = None
|
||||||
|
eml_size: int | None = None
|
||||||
|
if write_eml and output_dir is not None and message is not None:
|
||||||
|
eml_path, eml_size = _write_eml(message, output_dir, entry, entry_index)
|
||||||
|
|
||||||
|
draft = MessageDraft(
|
||||||
|
entry_index=entry_index,
|
||||||
|
entry_id=entry.id,
|
||||||
|
active=entry.active,
|
||||||
|
build_status=build_status,
|
||||||
|
validation_status=validation_status,
|
||||||
|
send_status=SendStatus.DRAFT,
|
||||||
|
imap_status=_imap_initial_status(config) if build_status == BuildStatus.BUILT else ImapStatus.SKIPPED,
|
||||||
|
subject=subject,
|
||||||
|
from_=_message_address(sender),
|
||||||
|
to=_message_addresses(recipients["to"]),
|
||||||
|
cc=_message_addresses(recipients["cc"]),
|
||||||
|
bcc=_message_addresses(recipients["bcc"]),
|
||||||
|
reply_to=_message_addresses(recipients["reply_to"]),
|
||||||
|
bounce_to=_message_addresses(recipients["bounce_to"]),
|
||||||
|
disposition_notification_to=_message_addresses(recipients["disposition_notification_to"]),
|
||||||
|
attachment_count=attachment_count,
|
||||||
|
attachments=_attachment_summaries(resolution),
|
||||||
|
issues=issues,
|
||||||
|
eml_path=eml_path,
|
||||||
|
eml_size_bytes=eml_size,
|
||||||
|
)
|
||||||
|
return BuiltMessage(draft=draft, mime=message)
|
||||||
|
|
||||||
|
|
||||||
|
def build_campaign_messages(
|
||||||
|
config: CampaignConfig,
|
||||||
|
*,
|
||||||
|
campaign_file: str | Path,
|
||||||
|
output_dir: str | Path | None = None,
|
||||||
|
write_eml: bool = False,
|
||||||
|
) -> CampaignBuildResult:
|
||||||
|
campaign_path = Path(campaign_file).resolve()
|
||||||
|
entries = load_campaign_entries(config, campaign_file=campaign_path)
|
||||||
|
output_path = Path(output_dir).resolve() if output_dir is not None else None
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory(prefix="multimailer-build-") as tmp:
|
||||||
|
work_dir = output_path or Path(tmp)
|
||||||
|
built_messages = [
|
||||||
|
build_entry_message(
|
||||||
|
config=config,
|
||||||
|
campaign_file=campaign_path,
|
||||||
|
entry=entry,
|
||||||
|
entry_index=index,
|
||||||
|
output_dir=output_path,
|
||||||
|
write_eml=write_eml,
|
||||||
|
work_dir=work_dir,
|
||||||
|
)
|
||||||
|
for index, entry in enumerate(entries, start=1)
|
||||||
|
]
|
||||||
|
|
||||||
|
report = CampaignBuildReport(
|
||||||
|
campaign_id=config.campaign.id,
|
||||||
|
campaign_name=config.campaign.name,
|
||||||
|
campaign_file=str(campaign_path),
|
||||||
|
entries_count=len(entries),
|
||||||
|
messages=[built.draft for built in built_messages],
|
||||||
|
)
|
||||||
|
return CampaignBuildResult(report=report, built_messages=built_messages)
|
||||||
139
server/app/mailer/messages/models.py
Normal file
139
server/app/mailer/messages/models.py
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from enum import StrEnum
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from app.mailer.campaign.models import BuildStatus, SendStatus
|
||||||
|
|
||||||
|
|
||||||
|
class MessageValidationStatus(StrEnum):
|
||||||
|
READY = "ready"
|
||||||
|
WARNING = "warning"
|
||||||
|
NEEDS_REVIEW = "needs_review"
|
||||||
|
BLOCKED = "blocked"
|
||||||
|
EXCLUDED = "excluded"
|
||||||
|
INACTIVE = "inactive"
|
||||||
|
|
||||||
|
|
||||||
|
class ImapStatus(StrEnum):
|
||||||
|
NOT_REQUESTED = "not_requested"
|
||||||
|
PENDING = "pending"
|
||||||
|
APPENDED = "appended"
|
||||||
|
FAILED = "failed"
|
||||||
|
SKIPPED = "skipped"
|
||||||
|
|
||||||
|
|
||||||
|
class MessageIssue(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
severity: Literal["info", "warning", "error"]
|
||||||
|
code: str
|
||||||
|
message: str
|
||||||
|
behavior: str | None = None
|
||||||
|
source: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class MessageAddress(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
email: str
|
||||||
|
name: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class MessageAttachmentSummary(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
attachment_id: str | None = None
|
||||||
|
label: str | None = None
|
||||||
|
status: str
|
||||||
|
behavior: str | None = None
|
||||||
|
required: bool
|
||||||
|
allow_multiple: bool
|
||||||
|
zip_enabled: bool
|
||||||
|
file_filter: str
|
||||||
|
directory: str
|
||||||
|
matches: list[str] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class MessageDraft(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid", populate_by_name=True)
|
||||||
|
|
||||||
|
entry_index: int
|
||||||
|
entry_id: str | None = None
|
||||||
|
active: bool
|
||||||
|
|
||||||
|
build_status: BuildStatus
|
||||||
|
validation_status: MessageValidationStatus
|
||||||
|
send_status: SendStatus
|
||||||
|
imap_status: ImapStatus
|
||||||
|
|
||||||
|
subject: str | None = None
|
||||||
|
from_: MessageAddress | None = Field(default=None, alias="from")
|
||||||
|
to: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
cc: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
bcc: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
reply_to: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
bounce_to: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
disposition_notification_to: list[MessageAddress] = Field(default_factory=list)
|
||||||
|
|
||||||
|
attachment_count: int = 0
|
||||||
|
attachments: list[MessageAttachmentSummary] = Field(default_factory=list)
|
||||||
|
issues: list[MessageIssue] = Field(default_factory=list)
|
||||||
|
|
||||||
|
eml_path: str | None = None
|
||||||
|
eml_size_bytes: int | None = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_queueable(self) -> bool:
|
||||||
|
return self.active and self.build_status == BuildStatus.BUILT and self.validation_status in {
|
||||||
|
MessageValidationStatus.READY,
|
||||||
|
MessageValidationStatus.WARNING,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignBuildReport(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
campaign_id: str
|
||||||
|
campaign_name: str
|
||||||
|
campaign_file: str
|
||||||
|
entries_count: int
|
||||||
|
messages: list[MessageDraft] = Field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def built_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.build_status == BuildStatus.BUILT)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def build_failed_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.build_status == BuildStatus.BUILD_FAILED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ready_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.READY)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def warning_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.WARNING)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def needs_review_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.NEEDS_REVIEW)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def blocked_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.BLOCKED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def excluded_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.EXCLUDED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def inactive_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.validation_status == MessageValidationStatus.INACTIVE)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def queueable_count(self) -> int:
|
||||||
|
return sum(1 for message in self.messages if message.is_queueable)
|
||||||
0
server/app/mailer/persistence/__init__.py
Normal file
0
server/app/mailer/persistence/__init__.py
Normal file
309
server/app/mailer/persistence/campaigns.py
Normal file
309
server/app/mailer/persistence/campaigns.py
Normal file
@@ -0,0 +1,309 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
import copy
|
||||||
|
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignIssue,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignStatus,
|
||||||
|
CampaignVersion,
|
||||||
|
JobBuildStatus,
|
||||||
|
JobImapStatus,
|
||||||
|
JobQueueStatus,
|
||||||
|
JobSendStatus,
|
||||||
|
JobValidationStatus,
|
||||||
|
)
|
||||||
|
from app.mailer.campaign.loader import load_campaign_config
|
||||||
|
from app.mailer.campaign.validation import Severity, validate_campaign_config
|
||||||
|
from app.mailer.messages.builder import build_campaign_messages
|
||||||
|
from app.mailer.messages.models import MessageDraft
|
||||||
|
|
||||||
|
RUNTIME_DIR = Path(__file__).resolve().parents[3] / "runtime"
|
||||||
|
CAMPAIGN_SNAPSHOT_DIR = RUNTIME_DIR / "campaign_snapshots"
|
||||||
|
BUILD_OUTPUT_DIR = RUNTIME_DIR / "generated_eml"
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignPersistenceError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_dirs() -> None:
|
||||||
|
CAMPAIGN_SNAPSHOT_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
BUILD_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_campaign_snapshot(version: CampaignVersion) -> Path:
|
||||||
|
_ensure_dirs()
|
||||||
|
path = CAMPAIGN_SNAPSHOT_DIR / f"{version.id}.json"
|
||||||
|
path.write_text(json.dumps(version.raw_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def _next_version_number(session: Session, campaign_id: str) -> int:
|
||||||
|
current = session.query(func.max(CampaignVersion.version_number)).filter(CampaignVersion.campaign_id == campaign_id).scalar()
|
||||||
|
return int(current or 0) + 1
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_runtime_path(base_path: Path | None, value: str | None) -> str | None:
|
||||||
|
if not value or base_path is None:
|
||||||
|
return value
|
||||||
|
path = Path(value).expanduser()
|
||||||
|
if path.is_absolute():
|
||||||
|
return str(path)
|
||||||
|
return str((base_path / path).resolve())
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_campaign_paths(raw_json: dict[str, Any], source_base_path: str | Path | None) -> dict[str, Any]:
|
||||||
|
"""Return a DB/runtime-safe campaign JSON snapshot.
|
||||||
|
|
||||||
|
The CLI naturally resolves relative paths against the campaign.json file.
|
||||||
|
Once the campaign is stored in the database, the JSON snapshot lives in
|
||||||
|
app/mailer/runtime/campaign_snapshots. To keep existing file-based
|
||||||
|
campaigns working, relative file paths are normalized to absolute paths at
|
||||||
|
import time when a source_base_path is known.
|
||||||
|
"""
|
||||||
|
base = Path(source_base_path).expanduser().resolve() if source_base_path else None
|
||||||
|
data = copy.deepcopy(raw_json)
|
||||||
|
|
||||||
|
template_source = data.get("template", {}).get("source") if isinstance(data.get("template"), dict) else None
|
||||||
|
if isinstance(template_source, dict):
|
||||||
|
for key in ("subject_path", "text_path", "html_path"):
|
||||||
|
template_source[key] = _resolve_runtime_path(base, template_source.get(key))
|
||||||
|
|
||||||
|
entries_source = data.get("entries", {}).get("source") if isinstance(data.get("entries"), dict) else None
|
||||||
|
if isinstance(entries_source, dict):
|
||||||
|
entries_source["path"] = _resolve_runtime_path(base, entries_source.get("path"))
|
||||||
|
|
||||||
|
attachments = data.get("attachments")
|
||||||
|
if isinstance(attachments, dict):
|
||||||
|
attachments["base_path"] = _resolve_runtime_path(base, attachments.get("base_path")) or "."
|
||||||
|
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def create_campaign_version_from_json(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
user_id: str | None,
|
||||||
|
raw_json: dict[str, Any],
|
||||||
|
source_filename: str | None = None,
|
||||||
|
source_base_path: str | None = None,
|
||||||
|
) -> tuple[Campaign, CampaignVersion]:
|
||||||
|
if source_base_path is None and source_filename:
|
||||||
|
source_path = Path(source_filename).expanduser()
|
||||||
|
source_base_path = str(source_path.parent if source_path.suffix else source_path)
|
||||||
|
|
||||||
|
runtime_json = normalize_campaign_paths(raw_json, source_base_path)
|
||||||
|
|
||||||
|
# load_campaign_config is file-oriented. Use a temporary snapshot for schema/Pydantic validation.
|
||||||
|
_ensure_dirs()
|
||||||
|
tmp_path = CAMPAIGN_SNAPSHOT_DIR / "_incoming_campaign.json"
|
||||||
|
tmp_path.write_text(json.dumps(runtime_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||||
|
config = load_campaign_config(tmp_path)
|
||||||
|
|
||||||
|
campaign = (
|
||||||
|
session.query(Campaign)
|
||||||
|
.filter(Campaign.tenant_id == tenant_id, Campaign.external_id == config.campaign.id)
|
||||||
|
.one_or_none()
|
||||||
|
)
|
||||||
|
if campaign is None:
|
||||||
|
campaign = Campaign(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
created_by_user_id=user_id,
|
||||||
|
external_id=config.campaign.id,
|
||||||
|
name=config.campaign.name,
|
||||||
|
description=config.campaign.description,
|
||||||
|
status=CampaignStatus.DRAFT.value,
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.flush()
|
||||||
|
else:
|
||||||
|
campaign.name = config.campaign.name
|
||||||
|
campaign.description = config.campaign.description
|
||||||
|
|
||||||
|
version = CampaignVersion(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_number=_next_version_number(session, campaign.id),
|
||||||
|
raw_json=runtime_json,
|
||||||
|
schema_version=raw_json.get("version", "1.0"),
|
||||||
|
source_filename=source_filename,
|
||||||
|
source_base_path=source_base_path,
|
||||||
|
)
|
||||||
|
session.add(version)
|
||||||
|
session.flush()
|
||||||
|
campaign.current_version_id = version.id
|
||||||
|
session.add(campaign)
|
||||||
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
return campaign, version
|
||||||
|
|
||||||
|
|
||||||
|
def load_version_config(session: Session, version_id: str):
|
||||||
|
version = session.get(CampaignVersion, version_id)
|
||||||
|
if not version:
|
||||||
|
raise CampaignPersistenceError(f"Campaign version not found: {version_id}")
|
||||||
|
path = _write_campaign_snapshot(version)
|
||||||
|
return version, path, load_campaign_config(path)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_campaign_version(session: Session, *, tenant_id: str, version_id: str, check_files: bool = False) -> dict[str, Any]:
|
||||||
|
version, snapshot_path, config = load_version_config(session, version_id)
|
||||||
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise CampaignPersistenceError("Campaign version is not accessible for this tenant")
|
||||||
|
|
||||||
|
report = validate_campaign_config(config, campaign_file=snapshot_path, check_files=check_files)
|
||||||
|
report_json = report.model_dump(mode="json")
|
||||||
|
report_json.update({"ok": report.ok, "error_count": report.error_count, "warning_count": report.warning_count})
|
||||||
|
version.validation_summary = report_json
|
||||||
|
|
||||||
|
# Replace version-level semantic issues from previous validations.
|
||||||
|
(
|
||||||
|
session.query(CampaignIssue)
|
||||||
|
.filter(CampaignIssue.campaign_version_id == version.id, CampaignIssue.job_id.is_(None))
|
||||||
|
.delete(synchronize_session=False)
|
||||||
|
)
|
||||||
|
for issue in report.issues:
|
||||||
|
session.add(
|
||||||
|
CampaignIssue(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
campaign_version_id=version.id,
|
||||||
|
severity=issue.severity.value,
|
||||||
|
code=issue.code,
|
||||||
|
message=issue.message,
|
||||||
|
source=issue.path,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
campaign.status = CampaignStatus.VALIDATED.value if report.ok else CampaignStatus.NEEDS_REVIEW.value
|
||||||
|
if report.ok:
|
||||||
|
version.workflow_state = "under_review"
|
||||||
|
version.is_complete = True
|
||||||
|
session.add(version)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
return report_json
|
||||||
|
|
||||||
|
|
||||||
|
def _job_validation_status(value: str) -> str:
|
||||||
|
allowed = {item.value for item in JobValidationStatus}
|
||||||
|
return value if value in allowed else JobValidationStatus.NEEDS_REVIEW.value
|
||||||
|
|
||||||
|
|
||||||
|
def _job_from_message(
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
message: MessageDraft,
|
||||||
|
) -> CampaignJob:
|
||||||
|
recipient_email = message.to[0].email if message.to else None
|
||||||
|
return CampaignJob(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
campaign_version_id=version_id,
|
||||||
|
entry_index=message.entry_index,
|
||||||
|
entry_id=message.entry_id,
|
||||||
|
recipient_email=recipient_email,
|
||||||
|
subject=message.subject,
|
||||||
|
eml_local_path=message.eml_path,
|
||||||
|
eml_size_bytes=message.eml_size_bytes,
|
||||||
|
build_status=message.build_status.value if hasattr(message.build_status, "value") else str(message.build_status),
|
||||||
|
validation_status=_job_validation_status(message.validation_status.value),
|
||||||
|
queue_status=JobQueueStatus.DRAFT.value,
|
||||||
|
send_status=JobSendStatus.NOT_QUEUED.value,
|
||||||
|
imap_status=message.imap_status.value if hasattr(message.imap_status, "value") else JobImapStatus.NOT_REQUESTED.value,
|
||||||
|
resolved_recipients={
|
||||||
|
"from": message.from_.model_dump(mode="json") if message.from_ else None,
|
||||||
|
"to": [item.model_dump(mode="json") for item in message.to],
|
||||||
|
"cc": [item.model_dump(mode="json") for item in message.cc],
|
||||||
|
"bcc": [item.model_dump(mode="json") for item in message.bcc],
|
||||||
|
"reply_to": [item.model_dump(mode="json") for item in message.reply_to],
|
||||||
|
"bounce_to": [item.model_dump(mode="json") for item in message.bounce_to],
|
||||||
|
"disposition_notification_to": [item.model_dump(mode="json") for item in message.disposition_notification_to],
|
||||||
|
},
|
||||||
|
resolved_attachments=[item.model_dump(mode="json") for item in message.attachments],
|
||||||
|
issues_snapshot=[item.model_dump(mode="json") for item in message.issues],
|
||||||
|
last_error="; ".join(issue.message for issue in message.issues if issue.severity == "error") or None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_campaign_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
version_id: str,
|
||||||
|
write_eml: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
version, snapshot_path, config = load_version_config(session, version_id)
|
||||||
|
campaign = session.get(Campaign, version.campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise CampaignPersistenceError("Campaign version is not accessible for this tenant")
|
||||||
|
|
||||||
|
output_dir = BUILD_OUTPUT_DIR / campaign.id / version.id
|
||||||
|
result = build_campaign_messages(config, campaign_file=snapshot_path, output_dir=output_dir, write_eml=write_eml)
|
||||||
|
report_json = result.report.model_dump(mode="json", by_alias=True)
|
||||||
|
report_json.update({
|
||||||
|
"built_count": result.report.built_count,
|
||||||
|
"build_failed_count": result.report.build_failed_count,
|
||||||
|
"ready_count": result.report.ready_count,
|
||||||
|
"warning_count": result.report.warning_count,
|
||||||
|
"needs_review_count": result.report.needs_review_count,
|
||||||
|
"blocked_count": result.report.blocked_count,
|
||||||
|
"excluded_count": result.report.excluded_count,
|
||||||
|
"inactive_count": result.report.inactive_count,
|
||||||
|
"queueable_count": result.report.queueable_count,
|
||||||
|
})
|
||||||
|
version.build_summary = report_json
|
||||||
|
|
||||||
|
# Rebuild jobs for the current version. Later, protect sent jobs from destructive rebuilds.
|
||||||
|
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id, CampaignIssue.job_id.is_not(None)).delete(synchronize_session=False)
|
||||||
|
session.query(CampaignJob).filter(CampaignJob.campaign_version_id == version.id).delete(synchronize_session=False)
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
for built in result.built_messages:
|
||||||
|
job = _job_from_message(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
message=built.draft,
|
||||||
|
)
|
||||||
|
session.add(job)
|
||||||
|
session.flush()
|
||||||
|
for issue in built.draft.issues:
|
||||||
|
session.add(
|
||||||
|
CampaignIssue(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
campaign_version_id=version.id,
|
||||||
|
job_id=job.id,
|
||||||
|
severity=issue.severity,
|
||||||
|
code=issue.code,
|
||||||
|
message=issue.message,
|
||||||
|
source=issue.source,
|
||||||
|
behavior=issue.behavior,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.report.needs_review_count or result.report.blocked_count:
|
||||||
|
campaign.status = CampaignStatus.NEEDS_REVIEW.value
|
||||||
|
version.workflow_state = "under_review"
|
||||||
|
elif result.report.queueable_count > 0:
|
||||||
|
campaign.status = CampaignStatus.READY_TO_QUEUE.value
|
||||||
|
version.workflow_state = "built"
|
||||||
|
else:
|
||||||
|
campaign.status = CampaignStatus.VALIDATED.value
|
||||||
|
|
||||||
|
session.add(version)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
return report_json
|
||||||
346
server/app/mailer/persistence/versions.py
Normal file
346
server/app/mailer/persistence/versions.py
Normal file
@@ -0,0 +1,346 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignIssue,
|
||||||
|
CampaignStatus,
|
||||||
|
CampaignVersion,
|
||||||
|
CampaignVersionFlow,
|
||||||
|
CampaignVersionWorkflowState,
|
||||||
|
)
|
||||||
|
from app.mailer.campaign.loader import load_campaign_config
|
||||||
|
from app.mailer.persistence.campaigns import (
|
||||||
|
CAMPAIGN_SNAPSHOT_DIR,
|
||||||
|
CampaignPersistenceError,
|
||||||
|
_ensure_dirs,
|
||||||
|
_next_version_number,
|
||||||
|
_write_campaign_snapshot,
|
||||||
|
normalize_campaign_paths,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def minimal_campaign_json(*, external_id: str, name: str, description: str | None = None) -> dict[str, Any]:
|
||||||
|
"""Return a WebUI-friendly starter campaign JSON.
|
||||||
|
|
||||||
|
It is intentionally usable as an editable working copy. It contains the
|
||||||
|
main sections the UI expects, but it may still be incomplete from the
|
||||||
|
strict send/build perspective until the user configures recipients,
|
||||||
|
template and sender details.
|
||||||
|
"""
|
||||||
|
|
||||||
|
return {
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": external_id,
|
||||||
|
"name": name,
|
||||||
|
"description": description or "",
|
||||||
|
"mode": "draft",
|
||||||
|
},
|
||||||
|
"fields": [],
|
||||||
|
"global_values": {},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "",
|
||||||
|
"port": 587,
|
||||||
|
"username": "",
|
||||||
|
"password": "",
|
||||||
|
"security": "starttls",
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": False,
|
||||||
|
"host": "",
|
||||||
|
"port": 993,
|
||||||
|
"username": "",
|
||||||
|
"password": "",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {"name": "", "email": ""},
|
||||||
|
"allow_individual_from": False,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": True,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": False,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": False,
|
||||||
|
"reply_to": [],
|
||||||
|
"allow_individual_reply_to": False,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": False,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": False,
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "",
|
||||||
|
"text": "",
|
||||||
|
"html": None,
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": ".",
|
||||||
|
"allow_individual": True,
|
||||||
|
"send_without_attachments": False,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask",
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"inline": [],
|
||||||
|
"defaults": {
|
||||||
|
"active": True,
|
||||||
|
"combine_to": False,
|
||||||
|
"combine_cc": True,
|
||||||
|
"combine_bcc": True,
|
||||||
|
"combine_reply_to": True,
|
||||||
|
"combine_bounce_to": True,
|
||||||
|
"combine_disposition_notification_to": True,
|
||||||
|
"combine_attachments": True,
|
||||||
|
"attachments": [],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1,
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": False,
|
||||||
|
"folder": "auto",
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [60, 300, 900],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": True,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def create_minimal_campaign(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
user_id: str | None,
|
||||||
|
external_id: str,
|
||||||
|
name: str,
|
||||||
|
description: str | None = None,
|
||||||
|
current_flow: str = CampaignVersionFlow.CREATE.value,
|
||||||
|
current_step: str = "basics",
|
||||||
|
) -> tuple[Campaign, CampaignVersion]:
|
||||||
|
existing = session.query(Campaign).filter(Campaign.tenant_id == tenant_id, Campaign.external_id == external_id).one_or_none()
|
||||||
|
if existing:
|
||||||
|
raise CampaignPersistenceError(f"Campaign with id '{external_id}' already exists for this tenant")
|
||||||
|
|
||||||
|
campaign = Campaign(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
created_by_user_id=user_id,
|
||||||
|
external_id=external_id,
|
||||||
|
name=name,
|
||||||
|
description=description,
|
||||||
|
status=CampaignStatus.DRAFT.value,
|
||||||
|
)
|
||||||
|
session.add(campaign)
|
||||||
|
session.flush()
|
||||||
|
|
||||||
|
version = CampaignVersion(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_number=1,
|
||||||
|
raw_json=minimal_campaign_json(external_id=external_id, name=name, description=description),
|
||||||
|
schema_version="1.0",
|
||||||
|
workflow_state=CampaignVersionWorkflowState.EDITING.value,
|
||||||
|
current_flow=current_flow,
|
||||||
|
current_step=current_step,
|
||||||
|
is_complete=False,
|
||||||
|
editor_state={"created_from": "minimal_campaign"},
|
||||||
|
autosaved_at=datetime.now(UTC),
|
||||||
|
)
|
||||||
|
session.add(version)
|
||||||
|
session.flush()
|
||||||
|
campaign.current_version_id = version.id
|
||||||
|
session.add(campaign)
|
||||||
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
return campaign, version
|
||||||
|
|
||||||
|
|
||||||
|
def get_campaign_version_for_tenant(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
version = session.get(CampaignVersion, version_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id or not version or version.campaign_id != campaign.id:
|
||||||
|
raise CampaignPersistenceError("Campaign version not found")
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def update_campaign_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
raw_json: dict[str, Any] | None = None,
|
||||||
|
current_flow: str | None = None,
|
||||||
|
current_step: str | None = None,
|
||||||
|
workflow_state: str | None = None,
|
||||||
|
is_complete: bool | None = None,
|
||||||
|
editor_state: dict[str, Any] | None = None,
|
||||||
|
source_filename: str | None = None,
|
||||||
|
source_base_path: str | None = None,
|
||||||
|
autosave: bool = False,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
assert campaign is not None
|
||||||
|
|
||||||
|
if raw_json is not None:
|
||||||
|
runtime_json = normalize_campaign_paths(raw_json, source_base_path) if source_base_path else copy.deepcopy(raw_json)
|
||||||
|
version.raw_json = runtime_json
|
||||||
|
version.schema_version = str(runtime_json.get("version", version.schema_version or "1.0"))
|
||||||
|
campaign_meta = runtime_json.get("campaign") if isinstance(runtime_json.get("campaign"), dict) else {}
|
||||||
|
if campaign_meta:
|
||||||
|
campaign.name = campaign_meta.get("name") or campaign.name
|
||||||
|
campaign.description = campaign_meta.get("description", campaign.description)
|
||||||
|
campaign.external_id = campaign_meta.get("id") or campaign.external_id
|
||||||
|
|
||||||
|
if current_flow is not None:
|
||||||
|
version.current_flow = current_flow
|
||||||
|
if current_step is not None:
|
||||||
|
version.current_step = current_step
|
||||||
|
if workflow_state is not None:
|
||||||
|
version.workflow_state = workflow_state
|
||||||
|
if is_complete is not None:
|
||||||
|
version.is_complete = is_complete
|
||||||
|
if editor_state is not None:
|
||||||
|
version.editor_state = editor_state
|
||||||
|
if source_filename is not None:
|
||||||
|
version.source_filename = source_filename
|
||||||
|
if source_base_path is not None:
|
||||||
|
version.source_base_path = source_base_path
|
||||||
|
if autosave:
|
||||||
|
version.autosaved_at = datetime.now(UTC)
|
||||||
|
|
||||||
|
# Changes invalidate previous build and validation summaries.
|
||||||
|
if raw_json is not None:
|
||||||
|
version.validation_summary = None
|
||||||
|
version.build_summary = None
|
||||||
|
session.query(CampaignIssue).filter(CampaignIssue.campaign_version_id == version.id).delete(synchronize_session=False)
|
||||||
|
|
||||||
|
session.add(version)
|
||||||
|
session.add(campaign)
|
||||||
|
session.flush()
|
||||||
|
_write_campaign_snapshot(version)
|
||||||
|
session.commit()
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def publish_campaign_version(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str,
|
||||||
|
) -> CampaignVersion:
|
||||||
|
version = get_campaign_version_for_tenant(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version_id)
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
assert campaign is not None
|
||||||
|
version.workflow_state = CampaignVersionWorkflowState.APPROVED.value
|
||||||
|
version.published_at = datetime.now(UTC)
|
||||||
|
campaign.current_version_id = version.id
|
||||||
|
campaign.status = CampaignStatus.VALIDATED.value
|
||||||
|
session.add(version)
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def validate_campaign_partial(raw_json: dict[str, Any], *, section: str | None = None) -> dict[str, Any]:
|
||||||
|
"""Lightweight UI-facing validation for incomplete campaign working copies.
|
||||||
|
|
||||||
|
This is intentionally less strict than campaign.schema.json validation. It
|
||||||
|
lets the WebUI autosave and validate one wizard step at a time.
|
||||||
|
"""
|
||||||
|
|
||||||
|
issues: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def issue(severity: str, sec: str, field: str, code: str, message: str) -> None:
|
||||||
|
if section is None or section == sec:
|
||||||
|
issues.append({
|
||||||
|
"severity": severity,
|
||||||
|
"section": sec,
|
||||||
|
"field": field,
|
||||||
|
"code": code,
|
||||||
|
"message": message,
|
||||||
|
})
|
||||||
|
|
||||||
|
campaign = raw_json.get("campaign") if isinstance(raw_json.get("campaign"), dict) else {}
|
||||||
|
if not campaign.get("id"):
|
||||||
|
issue("error", "basics", "campaign.id", "missing_campaign_id", "Campaign id is required.")
|
||||||
|
if not campaign.get("name"):
|
||||||
|
issue("error", "basics", "campaign.name", "missing_campaign_name", "Campaign name is required.")
|
||||||
|
|
||||||
|
recipients = raw_json.get("recipients") if isinstance(raw_json.get("recipients"), dict) else {}
|
||||||
|
sender = recipients.get("from") if isinstance(recipients.get("from"), dict) else {}
|
||||||
|
if not sender.get("email"):
|
||||||
|
issue("warning", "sender", "recipients.from.email", "missing_sender_email", "Sender email is not configured yet.")
|
||||||
|
|
||||||
|
entries = raw_json.get("entries") if isinstance(raw_json.get("entries"), dict) else {}
|
||||||
|
has_inline = bool(entries.get("inline"))
|
||||||
|
has_source = isinstance(entries.get("source"), dict)
|
||||||
|
if not has_inline and not has_source:
|
||||||
|
issue("warning", "recipients", "entries", "missing_recipients", "No inline recipients or external recipient source configured yet.")
|
||||||
|
if has_source:
|
||||||
|
mapping = entries.get("mapping") if isinstance(entries.get("mapping"), dict) else {}
|
||||||
|
if not any(key in mapping for key in ("to.0.email", "to.email", "email")):
|
||||||
|
issue("warning", "recipients", "entries.mapping", "missing_email_mapping", "No email field mapping is configured.")
|
||||||
|
|
||||||
|
template = raw_json.get("template") if isinstance(raw_json.get("template"), dict) else {}
|
||||||
|
if not template.get("subject") and not (isinstance(template.get("source"), dict) and template["source"].get("subject_path")):
|
||||||
|
issue("warning", "template", "template.subject", "missing_subject", "Template subject is empty.")
|
||||||
|
if not template.get("text") and not template.get("html") and not isinstance(template.get("source"), dict):
|
||||||
|
issue("warning", "template", "template", "missing_template_body", "No text, HTML or file-based template body configured yet.")
|
||||||
|
|
||||||
|
attachments = raw_json.get("attachments") if isinstance(raw_json.get("attachments"), dict) else {}
|
||||||
|
if not attachments.get("base_path"):
|
||||||
|
issue("info", "attachments", "attachments.base_path", "missing_attachment_base_path", "Attachment base path is not configured yet.")
|
||||||
|
|
||||||
|
delivery = raw_json.get("delivery") if isinstance(raw_json.get("delivery"), dict) else {}
|
||||||
|
rate_limit = delivery.get("rate_limit") if isinstance(delivery.get("rate_limit"), dict) else {}
|
||||||
|
messages_per_minute = rate_limit.get("messages_per_minute")
|
||||||
|
if messages_per_minute is not None:
|
||||||
|
try:
|
||||||
|
if int(messages_per_minute) < 1:
|
||||||
|
issue("error", "send", "delivery.rate_limit.messages_per_minute", "invalid_rate_limit", "Messages per minute must be at least 1.")
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
issue("error", "send", "delivery.rate_limit.messages_per_minute", "invalid_rate_limit", "Messages per minute must be a number.")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"ok": not any(item["severity"] == "error" for item in issues),
|
||||||
|
"section": section,
|
||||||
|
"error_count": sum(1 for item in issues if item["severity"] == "error"),
|
||||||
|
"warning_count": sum(1 for item in issues if item["severity"] == "warning"),
|
||||||
|
"info_count": sum(1 for item in issues if item["severity"] == "info"),
|
||||||
|
"issues": issues,
|
||||||
|
}
|
||||||
1
server/app/mailer/reports/__init__.py
Normal file
1
server/app/mailer/reports/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""Reporting helpers for campaigns and jobs."""
|
||||||
351
server/app/mailer/reports/campaigns.py
Normal file
351
server/app/mailer/reports/campaigns.py
Normal file
@@ -0,0 +1,351 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import io
|
||||||
|
import math
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignIssue,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignVersion,
|
||||||
|
ImapAppendAttempt,
|
||||||
|
SendAttempt,
|
||||||
|
)
|
||||||
|
from app.mailer.campaign.loader import load_campaign_config
|
||||||
|
from app.mailer.persistence.campaigns import _write_campaign_snapshot
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignReportError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _utcnow_iso() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def _counter(values: list[str | None]) -> dict[str, int]:
|
||||||
|
return dict(Counter(value or "unknown" for value in values))
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign(session: Session, *, tenant_id: str, campaign_id: str) -> Campaign:
|
||||||
|
campaign = session.query(Campaign).filter(Campaign.tenant_id == tenant_id, Campaign.id == campaign_id).one_or_none()
|
||||||
|
if not campaign:
|
||||||
|
raise CampaignReportError(f"Campaign not found or not accessible: {campaign_id}")
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _current_version(session: Session, campaign: Campaign) -> CampaignVersion | None:
|
||||||
|
if not campaign.current_version_id:
|
||||||
|
return None
|
||||||
|
version = session.get(CampaignVersion, campaign.current_version_id)
|
||||||
|
if version and version.campaign_id == campaign.id:
|
||||||
|
return version
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _version_info(version: CampaignVersion | None) -> dict[str, Any] | None:
|
||||||
|
if not version:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"id": version.id,
|
||||||
|
"version_number": version.version_number,
|
||||||
|
"schema_version": version.schema_version,
|
||||||
|
"source_filename": version.source_filename,
|
||||||
|
"created_at": version.created_at.isoformat() if version.created_at else None,
|
||||||
|
"validation_summary": version.validation_summary,
|
||||||
|
"build_summary": version.build_summary,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_delivery_info(version: CampaignVersion | None, jobs: list[CampaignJob]) -> dict[str, Any]:
|
||||||
|
"""Extract rate-limit and IMAP settings from the version JSON where possible.
|
||||||
|
|
||||||
|
This stays best-effort so reports still work if the schema evolves or a
|
||||||
|
partial/invalid campaign snapshot exists.
|
||||||
|
"""
|
||||||
|
|
||||||
|
default = {
|
||||||
|
"rate_limit": {"messages_per_minute": None, "concurrency": None},
|
||||||
|
"imap_append_sent": {"enabled": None, "folder": None},
|
||||||
|
"retry": {"max_attempts": None, "backoff_seconds": []},
|
||||||
|
"estimated_remaining_send_seconds": None,
|
||||||
|
"estimated_remaining_send_human": None,
|
||||||
|
}
|
||||||
|
if not version:
|
||||||
|
return default
|
||||||
|
try:
|
||||||
|
snapshot_path = _write_campaign_snapshot(version)
|
||||||
|
config = load_campaign_config(snapshot_path)
|
||||||
|
except Exception as exc: # pragma: no cover - reporting should not fail hard here
|
||||||
|
default["load_error"] = str(exc)
|
||||||
|
return default
|
||||||
|
|
||||||
|
messages_per_minute = config.delivery.rate_limit.messages_per_minute
|
||||||
|
pending = [job for job in jobs if job.send_status in {"queued", "failed_temporary", "sending"}]
|
||||||
|
estimated_seconds = None
|
||||||
|
if messages_per_minute and pending:
|
||||||
|
estimated_seconds = int(math.ceil((len(pending) / messages_per_minute) * 60))
|
||||||
|
|
||||||
|
return {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": messages_per_minute,
|
||||||
|
"concurrency": config.delivery.rate_limit.concurrency,
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": config.delivery.imap_append_sent.enabled,
|
||||||
|
"folder": config.delivery.imap_append_sent.folder,
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": config.delivery.retry.max_attempts,
|
||||||
|
"backoff_seconds": config.delivery.retry.backoff_seconds,
|
||||||
|
},
|
||||||
|
"estimated_remaining_send_seconds": estimated_seconds,
|
||||||
|
"estimated_remaining_send_human": _human_duration(estimated_seconds),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _human_duration(seconds: int | None) -> str | None:
|
||||||
|
if seconds is None:
|
||||||
|
return None
|
||||||
|
if seconds < 60:
|
||||||
|
return f"{seconds}s"
|
||||||
|
minutes, sec = divmod(seconds, 60)
|
||||||
|
if minutes < 60:
|
||||||
|
return f"{minutes}m {sec}s" if sec else f"{minutes}m"
|
||||||
|
hours, minute = divmod(minutes, 60)
|
||||||
|
return f"{hours}h {minute}m" if minute else f"{hours}h"
|
||||||
|
|
||||||
|
|
||||||
|
def _issue_summary_from_jobs(jobs: list[CampaignJob]) -> dict[str, Any]:
|
||||||
|
severity_counter: Counter[str] = Counter()
|
||||||
|
code_counter: Counter[str] = Counter()
|
||||||
|
behavior_counter: Counter[str] = Counter()
|
||||||
|
total = 0
|
||||||
|
for job in jobs:
|
||||||
|
for issue in job.issues_snapshot or []:
|
||||||
|
if not isinstance(issue, dict):
|
||||||
|
continue
|
||||||
|
total += 1
|
||||||
|
severity_counter[issue.get("severity") or "unknown"] += 1
|
||||||
|
code_counter[issue.get("code") or "unknown"] += 1
|
||||||
|
if issue.get("behavior"):
|
||||||
|
behavior_counter[issue["behavior"]] += 1
|
||||||
|
return {
|
||||||
|
"total": total,
|
||||||
|
"by_severity": dict(severity_counter),
|
||||||
|
"by_code": dict(code_counter),
|
||||||
|
"by_behavior": dict(behavior_counter),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_summary(jobs: list[CampaignJob]) -> dict[str, Any]:
|
||||||
|
status_counter: Counter[str] = Counter()
|
||||||
|
behavior_counter: Counter[str] = Counter()
|
||||||
|
total_configs = 0
|
||||||
|
total_matched_files = 0
|
||||||
|
zip_enabled = 0
|
||||||
|
missing = 0
|
||||||
|
ambiguous = 0
|
||||||
|
for job in jobs:
|
||||||
|
for attachment in job.resolved_attachments or []:
|
||||||
|
if not isinstance(attachment, dict):
|
||||||
|
continue
|
||||||
|
total_configs += 1
|
||||||
|
status = attachment.get("status") or "unknown"
|
||||||
|
status_counter[status] += 1
|
||||||
|
if attachment.get("behavior"):
|
||||||
|
behavior_counter[attachment["behavior"]] += 1
|
||||||
|
matches = attachment.get("matches") or []
|
||||||
|
if isinstance(matches, list):
|
||||||
|
total_matched_files += len(matches)
|
||||||
|
if attachment.get("zip_enabled"):
|
||||||
|
zip_enabled += 1
|
||||||
|
if status == "missing":
|
||||||
|
missing += 1
|
||||||
|
if status == "ambiguous":
|
||||||
|
ambiguous += 1
|
||||||
|
return {
|
||||||
|
"total_attachment_configs": total_configs,
|
||||||
|
"total_matched_files": total_matched_files,
|
||||||
|
"zip_enabled_configs": zip_enabled,
|
||||||
|
"missing_configs": missing,
|
||||||
|
"ambiguous_configs": ambiguous,
|
||||||
|
"by_status": dict(status_counter),
|
||||||
|
"by_behavior": dict(behavior_counter),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _recent_failures(jobs: list[CampaignJob], *, limit: int = 20) -> list[dict[str, Any]]:
|
||||||
|
failed = [job for job in jobs if job.last_error or str(job.send_status).startswith("failed") or job.imap_status == "failed"]
|
||||||
|
failed.sort(key=lambda job: job.updated_at or job.created_at, reverse=True)
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"job_id": job.id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"recipient_email": job.recipient_email,
|
||||||
|
"validation_status": job.validation_status,
|
||||||
|
"send_status": job.send_status,
|
||||||
|
"imap_status": job.imap_status,
|
||||||
|
"attempt_count": job.attempt_count,
|
||||||
|
"last_error": job.last_error,
|
||||||
|
"updated_at": job.updated_at.isoformat() if job.updated_at else None,
|
||||||
|
}
|
||||||
|
for job in failed[:limit]
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _job_row(job: CampaignJob) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"job_id": job.id,
|
||||||
|
"entry_index": job.entry_index,
|
||||||
|
"entry_id": job.entry_id,
|
||||||
|
"recipient_email": job.recipient_email,
|
||||||
|
"subject": job.subject,
|
||||||
|
"build_status": job.build_status,
|
||||||
|
"validation_status": job.validation_status,
|
||||||
|
"queue_status": job.queue_status,
|
||||||
|
"send_status": job.send_status,
|
||||||
|
"imap_status": job.imap_status,
|
||||||
|
"attempt_count": job.attempt_count,
|
||||||
|
"queued_at": job.queued_at.isoformat() if job.queued_at else None,
|
||||||
|
"sent_at": job.sent_at.isoformat() if job.sent_at else None,
|
||||||
|
"last_error": job.last_error,
|
||||||
|
"eml_size_bytes": job.eml_size_bytes,
|
||||||
|
"issues_count": len(job.issues_snapshot or []),
|
||||||
|
"attachment_config_count": len(job.resolved_attachments or []),
|
||||||
|
"matched_file_count": sum(len(item.get("matches") or []) for item in (job.resolved_attachments or []) if isinstance(item, dict)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def generate_campaign_report(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
include_jobs: bool = False,
|
||||||
|
include_recent_failures: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Generate a dashboard/report payload for one campaign.
|
||||||
|
|
||||||
|
The shape is intentionally web-UI friendly: status counters for cards,
|
||||||
|
issue/attachment summaries for review panels, and optional job rows for
|
||||||
|
tables/export.
|
||||||
|
"""
|
||||||
|
|
||||||
|
campaign = _get_campaign(session, tenant_id=tenant_id, campaign_id=campaign_id)
|
||||||
|
version = _current_version(session, campaign)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(CampaignJob.tenant_id == tenant_id, CampaignJob.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
job_ids = [job.id for job in jobs]
|
||||||
|
send_attempts = session.query(SendAttempt).filter(SendAttempt.job_id.in_(job_ids)).count() if job_ids else 0
|
||||||
|
imap_attempts = session.query(ImapAppendAttempt).filter(ImapAppendAttempt.job_id.in_(job_ids)).count() if job_ids else 0
|
||||||
|
persisted_issues = session.query(CampaignIssue).filter(CampaignIssue.tenant_id == tenant_id, CampaignIssue.campaign_id == campaign.id).count()
|
||||||
|
|
||||||
|
validation_counts = _counter([job.validation_status for job in jobs])
|
||||||
|
queue_counts = _counter([job.queue_status for job in jobs])
|
||||||
|
send_counts = _counter([job.send_status for job in jobs])
|
||||||
|
imap_counts = _counter([job.imap_status for job in jobs])
|
||||||
|
build_counts = _counter([job.build_status for job in jobs])
|
||||||
|
|
||||||
|
queueable = sum(1 for job in jobs if job.validation_status in {"ready", "warning"} and job.build_status == "built")
|
||||||
|
needs_attention = sum(
|
||||||
|
1
|
||||||
|
for job in jobs
|
||||||
|
if job.validation_status in {"needs_review", "blocked"}
|
||||||
|
or job.send_status in {"failed_temporary", "failed_permanent"}
|
||||||
|
or job.imap_status == "failed"
|
||||||
|
)
|
||||||
|
sent = send_counts.get("sent", 0)
|
||||||
|
failed = send_counts.get("failed_temporary", 0) + send_counts.get("failed_permanent", 0)
|
||||||
|
|
||||||
|
report: dict[str, Any] = {
|
||||||
|
"generated_at": _utcnow_iso(),
|
||||||
|
"campaign": {
|
||||||
|
"id": campaign.id,
|
||||||
|
"external_id": campaign.external_id,
|
||||||
|
"name": campaign.name,
|
||||||
|
"description": campaign.description,
|
||||||
|
"status": campaign.status,
|
||||||
|
"created_at": campaign.created_at.isoformat() if campaign.created_at else None,
|
||||||
|
"updated_at": campaign.updated_at.isoformat() if campaign.updated_at else None,
|
||||||
|
},
|
||||||
|
"current_version": _version_info(version),
|
||||||
|
"cards": {
|
||||||
|
"jobs_total": len(jobs),
|
||||||
|
"queueable": queueable,
|
||||||
|
"needs_attention": needs_attention,
|
||||||
|
"sent": sent,
|
||||||
|
"failed": failed,
|
||||||
|
"imap_appended": imap_counts.get("appended", 0),
|
||||||
|
"imap_failed": imap_counts.get("failed", 0),
|
||||||
|
},
|
||||||
|
"status_counts": {
|
||||||
|
"build": build_counts,
|
||||||
|
"validation": validation_counts,
|
||||||
|
"queue": queue_counts,
|
||||||
|
"send": send_counts,
|
||||||
|
"imap": imap_counts,
|
||||||
|
},
|
||||||
|
"issues": {
|
||||||
|
**_issue_summary_from_jobs(jobs),
|
||||||
|
"persisted_campaign_issue_count": persisted_issues,
|
||||||
|
},
|
||||||
|
"attachments": _attachment_summary(jobs),
|
||||||
|
"attempts": {
|
||||||
|
"send_attempts": int(send_attempts),
|
||||||
|
"imap_append_attempts": int(imap_attempts),
|
||||||
|
},
|
||||||
|
"delivery": _load_delivery_info(version, jobs),
|
||||||
|
}
|
||||||
|
if include_recent_failures:
|
||||||
|
report["recent_failures"] = _recent_failures(jobs)
|
||||||
|
if include_jobs:
|
||||||
|
report["jobs"] = [_job_row(job) for job in jobs]
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
def generate_jobs_csv(session: Session, *, tenant_id: str, campaign_id: str) -> str:
|
||||||
|
campaign = _get_campaign(session, tenant_id=tenant_id, campaign_id=campaign_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(CampaignJob.tenant_id == tenant_id, CampaignJob.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
rows = [_job_row(job) for job in jobs]
|
||||||
|
fieldnames = [
|
||||||
|
"job_id",
|
||||||
|
"entry_index",
|
||||||
|
"entry_id",
|
||||||
|
"recipient_email",
|
||||||
|
"subject",
|
||||||
|
"build_status",
|
||||||
|
"validation_status",
|
||||||
|
"queue_status",
|
||||||
|
"send_status",
|
||||||
|
"imap_status",
|
||||||
|
"attempt_count",
|
||||||
|
"queued_at",
|
||||||
|
"sent_at",
|
||||||
|
"last_error",
|
||||||
|
"eml_size_bytes",
|
||||||
|
"issues_count",
|
||||||
|
"attachment_config_count",
|
||||||
|
"matched_file_count",
|
||||||
|
]
|
||||||
|
buffer = io.StringIO()
|
||||||
|
writer = csv.DictWriter(buffer, fieldnames=fieldnames)
|
||||||
|
writer.writeheader()
|
||||||
|
writer.writerows(rows)
|
||||||
|
return buffer.getvalue()
|
||||||
210
server/app/mailer/reports/emailing.py
Normal file
210
server/app/mailer/reports/emailing.py
Normal file
@@ -0,0 +1,210 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from email.utils import formataddr
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import Campaign, CampaignVersion
|
||||||
|
from app.mailer.campaign.loader import load_campaign_config
|
||||||
|
from app.mailer.campaign.models import CampaignConfig, SmtpConfig
|
||||||
|
from app.mailer.persistence.campaigns import _write_campaign_snapshot
|
||||||
|
from app.mailer.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
|
||||||
|
from app.mailer.sending.smtp import SmtpConfigurationError, SmtpSendResult, send_email_message
|
||||||
|
|
||||||
|
|
||||||
|
class CampaignReportEmailError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class CampaignReportEmailResult:
|
||||||
|
campaign_id: str
|
||||||
|
to: list[str]
|
||||||
|
subject: str
|
||||||
|
dry_run: bool
|
||||||
|
sent: bool
|
||||||
|
attached_jobs_csv: bool
|
||||||
|
attached_report_json: bool
|
||||||
|
smtp_host: str | None = None
|
||||||
|
smtp_port: int | None = None
|
||||||
|
accepted_count: int | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"campaign_id": self.campaign_id,
|
||||||
|
"to": self.to,
|
||||||
|
"subject": self.subject,
|
||||||
|
"dry_run": self.dry_run,
|
||||||
|
"sent": self.sent,
|
||||||
|
"attached_jobs_csv": self.attached_jobs_csv,
|
||||||
|
"attached_report_json": self.attached_report_json,
|
||||||
|
"smtp_host": self.smtp_host,
|
||||||
|
"smtp_port": self.smtp_port,
|
||||||
|
"accepted_count": self.accepted_count,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _current_version(session: Session, campaign: Campaign) -> CampaignVersion:
|
||||||
|
version = session.get(CampaignVersion, campaign.current_version_id) if campaign.current_version_id else None
|
||||||
|
if version is None:
|
||||||
|
version = (
|
||||||
|
session.query(CampaignVersion)
|
||||||
|
.filter(CampaignVersion.campaign_id == campaign.id)
|
||||||
|
.order_by(CampaignVersion.version_number.desc())
|
||||||
|
.first()
|
||||||
|
)
|
||||||
|
if version is None:
|
||||||
|
raise CampaignReportEmailError("Campaign has no version")
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _load_config(version: CampaignVersion) -> CampaignConfig:
|
||||||
|
snapshot_path = _write_campaign_snapshot(version)
|
||||||
|
return load_campaign_config(snapshot_path)
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_from(config: CampaignConfig) -> tuple[str, str | None]:
|
||||||
|
if config.recipients.from_:
|
||||||
|
return config.recipients.from_.email, config.recipients.from_.name
|
||||||
|
if config.server.smtp and config.server.smtp.username and "@" in config.server.smtp.username:
|
||||||
|
return config.server.smtp.username, None
|
||||||
|
raise SmtpConfigurationError("Report email requires recipients.from.email or an SMTP username that is an email address")
|
||||||
|
|
||||||
|
|
||||||
|
def _text_summary(report: dict[str, Any]) -> str:
|
||||||
|
campaign = report["campaign"]
|
||||||
|
cards = report["cards"]
|
||||||
|
status = report["status_counts"]
|
||||||
|
delivery = report.get("delivery", {})
|
||||||
|
lines = [
|
||||||
|
f"Campaign report: {campaign['name']}",
|
||||||
|
"",
|
||||||
|
f"Campaign ID: {campaign['id']}",
|
||||||
|
f"External ID: {campaign['external_id']}",
|
||||||
|
f"Status: {campaign['status']}",
|
||||||
|
"",
|
||||||
|
"Overview",
|
||||||
|
f"- Jobs total: {cards['jobs_total']}",
|
||||||
|
f"- Queueable: {cards['queueable']}",
|
||||||
|
f"- Needs attention: {cards['needs_attention']}",
|
||||||
|
f"- Sent: {cards['sent']}",
|
||||||
|
f"- Failed: {cards['failed']}",
|
||||||
|
f"- IMAP appended: {cards['imap_appended']}",
|
||||||
|
f"- IMAP failed: {cards['imap_failed']}",
|
||||||
|
"",
|
||||||
|
f"Build status: {status.get('build', {})}",
|
||||||
|
f"Validation status: {status.get('validation', {})}",
|
||||||
|
f"Queue status: {status.get('queue', {})}",
|
||||||
|
f"Send status: {status.get('send', {})}",
|
||||||
|
f"IMAP status: {status.get('imap', {})}",
|
||||||
|
]
|
||||||
|
if delivery.get("estimated_remaining_send_human"):
|
||||||
|
lines.extend(["", f"Estimated remaining send time: {delivery['estimated_remaining_send_human']}"])
|
||||||
|
lines.extend(["", "This report was generated by MultiMailer."])
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def build_report_message(
|
||||||
|
*,
|
||||||
|
campaign: Campaign,
|
||||||
|
config: CampaignConfig,
|
||||||
|
report: dict[str, Any],
|
||||||
|
to: list[str],
|
||||||
|
jobs_csv: str | None = None,
|
||||||
|
report_json: dict[str, Any] | None = None,
|
||||||
|
) -> EmailMessage:
|
||||||
|
from_email, from_name = _effective_from(config)
|
||||||
|
subject = f"MultiMailer report: {campaign.name}"
|
||||||
|
|
||||||
|
msg = EmailMessage()
|
||||||
|
msg["Subject"] = subject
|
||||||
|
msg["From"] = formataddr((from_name or from_email, from_email))
|
||||||
|
msg["To"] = ", ".join(to)
|
||||||
|
msg["X-MultiMailer-Report"] = "campaign"
|
||||||
|
msg.set_content(_text_summary(report))
|
||||||
|
|
||||||
|
if jobs_csv is not None:
|
||||||
|
filename = f"multimailer-{campaign.external_id}-jobs.csv"
|
||||||
|
msg.add_attachment(jobs_csv.encode("utf-8"), maintype="text", subtype="csv", filename=filename)
|
||||||
|
if report_json is not None:
|
||||||
|
filename = f"multimailer-{campaign.external_id}-report.json"
|
||||||
|
msg.add_attachment(
|
||||||
|
json.dumps(report_json, indent=2, ensure_ascii=False, default=str).encode("utf-8"),
|
||||||
|
maintype="application",
|
||||||
|
subtype="json",
|
||||||
|
filename=filename,
|
||||||
|
)
|
||||||
|
return msg
|
||||||
|
|
||||||
|
|
||||||
|
def send_campaign_report_email(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
to: list[str],
|
||||||
|
include_jobs: bool = False,
|
||||||
|
attach_jobs_csv: bool = True,
|
||||||
|
attach_report_json: bool = False,
|
||||||
|
dry_run: bool = False,
|
||||||
|
) -> CampaignReportEmailResult:
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
if not campaign or campaign.tenant_id != tenant_id:
|
||||||
|
raise CampaignReportError("Campaign not found")
|
||||||
|
if not to:
|
||||||
|
raise CampaignReportEmailError("At least one report recipient is required")
|
||||||
|
|
||||||
|
version = _current_version(session, campaign)
|
||||||
|
config = _load_config(version)
|
||||||
|
smtp_config: SmtpConfig | None = config.server.smtp
|
||||||
|
if smtp_config is None:
|
||||||
|
raise SmtpConfigurationError("Campaign has no SMTP configuration")
|
||||||
|
|
||||||
|
report = generate_campaign_report(session, tenant_id=tenant_id, campaign_id=campaign_id, include_jobs=include_jobs)
|
||||||
|
jobs_csv = generate_jobs_csv(session, tenant_id=tenant_id, campaign_id=campaign_id) if attach_jobs_csv else None
|
||||||
|
report_json = report if attach_report_json else None
|
||||||
|
message = build_report_message(
|
||||||
|
campaign=campaign,
|
||||||
|
config=config,
|
||||||
|
report=report,
|
||||||
|
to=to,
|
||||||
|
jobs_csv=jobs_csv,
|
||||||
|
report_json=report_json,
|
||||||
|
)
|
||||||
|
envelope_from, _ = _effective_from(config)
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
return CampaignReportEmailResult(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
to=to,
|
||||||
|
subject=str(message["Subject"]),
|
||||||
|
dry_run=True,
|
||||||
|
sent=False,
|
||||||
|
attached_jobs_csv=jobs_csv is not None,
|
||||||
|
attached_report_json=report_json is not None,
|
||||||
|
smtp_host=smtp_config.host,
|
||||||
|
smtp_port=smtp_config.port,
|
||||||
|
)
|
||||||
|
|
||||||
|
result: SmtpSendResult = send_email_message(
|
||||||
|
message,
|
||||||
|
smtp_config=smtp_config,
|
||||||
|
envelope_from=envelope_from,
|
||||||
|
envelope_recipients=to,
|
||||||
|
)
|
||||||
|
return CampaignReportEmailResult(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
to=to,
|
||||||
|
subject=str(message["Subject"]),
|
||||||
|
dry_run=False,
|
||||||
|
sent=True,
|
||||||
|
attached_jobs_csv=jobs_csv is not None,
|
||||||
|
attached_report_json=report_json is not None,
|
||||||
|
smtp_host=result.host,
|
||||||
|
smtp_port=result.port,
|
||||||
|
accepted_count=result.accepted_count,
|
||||||
|
)
|
||||||
804
server/app/mailer/schema/campaign.schema.json
Normal file
804
server/app/mailer/schema/campaign.schema.json
Normal file
@@ -0,0 +1,804 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "https://multimailer.local/schema/campaign.schema.json",
|
||||||
|
"title": "MultiMailer Campaign",
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"version",
|
||||||
|
"campaign",
|
||||||
|
"template",
|
||||||
|
"entries"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"version": {
|
||||||
|
"type": "string",
|
||||||
|
"const": "1.0"
|
||||||
|
},
|
||||||
|
"campaign": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"id",
|
||||||
|
"name"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"id": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"description": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"mode": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"draft",
|
||||||
|
"test",
|
||||||
|
"send"
|
||||||
|
],
|
||||||
|
"default": "draft"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"fields": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"name"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"type": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"string",
|
||||||
|
"integer",
|
||||||
|
"double",
|
||||||
|
"date",
|
||||||
|
"password"
|
||||||
|
],
|
||||||
|
"default": "string"
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"required": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"global_values": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": true,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"smtp": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"host": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"port": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 65535
|
||||||
|
},
|
||||||
|
"username": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"password": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"security": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"plain",
|
||||||
|
"tls",
|
||||||
|
"starttls"
|
||||||
|
],
|
||||||
|
"default": "starttls"
|
||||||
|
},
|
||||||
|
"timeout_seconds": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"default": 30
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"enabled": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"host": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"port": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 65535
|
||||||
|
},
|
||||||
|
"username": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"password": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"security": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"plain",
|
||||||
|
"tls",
|
||||||
|
"starttls"
|
||||||
|
],
|
||||||
|
"default": "tls"
|
||||||
|
},
|
||||||
|
"sent_folder": {
|
||||||
|
"type": "string",
|
||||||
|
"default": "auto"
|
||||||
|
},
|
||||||
|
"timeout_seconds": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"default": 30
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"from": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"allow_individual_from": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"cc": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_cc": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"bcc": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_bcc": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"reply_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_reply_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"bounce_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_bounce_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"disposition_notification_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"allow_individual_disposition_notification_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"oneOf": [
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"subject"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"subject": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"text": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"html": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"source"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"source": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"type"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"type": {
|
||||||
|
"type": "string",
|
||||||
|
"const": "files"
|
||||||
|
},
|
||||||
|
"subject_path": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"text_path": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"html_path": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"encoding": {
|
||||||
|
"type": "string",
|
||||||
|
"default": "utf-8"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"base_path": {
|
||||||
|
"type": "string",
|
||||||
|
"default": ".",
|
||||||
|
"description": "Campaign-level base path. Global and entry attachment base_dir values are resolved relative to this path unless absolute."
|
||||||
|
},
|
||||||
|
"allow_individual": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"send_without_attachments": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true,
|
||||||
|
"description": "Legacy compatibility flag. Prefer validation_policy and per-config missing_behavior for new campaigns."
|
||||||
|
},
|
||||||
|
"global": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/attachment_config"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"missing_behavior": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
},
|
||||||
|
"ambiguous_behavior": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {
|
||||||
|
"base_path": ".",
|
||||||
|
"global": []
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"oneOf": [
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"inline"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"inline": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/entry"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"source",
|
||||||
|
"mapping"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"source": {
|
||||||
|
"$ref": "#/$defs/source"
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "Internal campaign path -> source column/key. Examples: to.0.email, fields.number, attachments.0.file_filter.",
|
||||||
|
"additionalProperties": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"$ref": "#/$defs/entry"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"missing_required_attachment": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
},
|
||||||
|
"missing_optional_attachment": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "warn"
|
||||||
|
},
|
||||||
|
"ambiguous_attachment_match": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
},
|
||||||
|
"missing_email": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"drop"
|
||||||
|
],
|
||||||
|
"default": "block"
|
||||||
|
},
|
||||||
|
"template_error": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"drop"
|
||||||
|
],
|
||||||
|
"default": "block"
|
||||||
|
},
|
||||||
|
"inactive_entry": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"drop",
|
||||||
|
"block",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "drop"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"rate_limit": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"messages_per_minute": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"default": 5
|
||||||
|
},
|
||||||
|
"concurrency": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"default": 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"enabled": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"folder": {
|
||||||
|
"type": "string",
|
||||||
|
"default": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"max_attempts": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"default": 3
|
||||||
|
},
|
||||||
|
"backoff_seconds": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1
|
||||||
|
},
|
||||||
|
"default": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"enabled": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"initial_build_status": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"built",
|
||||||
|
"build_failed"
|
||||||
|
],
|
||||||
|
"default": "built"
|
||||||
|
},
|
||||||
|
"initial_send_status": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"draft",
|
||||||
|
"queued"
|
||||||
|
],
|
||||||
|
"default": "draft"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {
|
||||||
|
"enabled": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"$defs": {
|
||||||
|
"recipient": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"email"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"email": {
|
||||||
|
"type": "string",
|
||||||
|
"format": "email"
|
||||||
|
},
|
||||||
|
"name": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"type": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"to",
|
||||||
|
"cc",
|
||||||
|
"bcc",
|
||||||
|
"reply_to",
|
||||||
|
"bounce_to",
|
||||||
|
"disposition_notification_to"
|
||||||
|
],
|
||||||
|
"default": "to"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"attachment_config": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"base_dir",
|
||||||
|
"file_filter"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"id": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional stable ID for UI/status references."
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"base_dir": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Directory relative to attachments.base_path unless absolute."
|
||||||
|
},
|
||||||
|
"file_filter": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Glob/filter expression, rendered with global/local fields before matching."
|
||||||
|
},
|
||||||
|
"include_subdirs": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"required": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"allow_multiple": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"missing_behavior": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
},
|
||||||
|
"ambiguous_behavior": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"block",
|
||||||
|
"ask",
|
||||||
|
"drop",
|
||||||
|
"continue",
|
||||||
|
"warn"
|
||||||
|
],
|
||||||
|
"default": "ask"
|
||||||
|
},
|
||||||
|
"zip": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"enabled": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": false
|
||||||
|
},
|
||||||
|
"filename_template": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"password_template": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"method": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"zip_standard",
|
||||||
|
"aes"
|
||||||
|
],
|
||||||
|
"default": "aes"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false,
|
||||||
|
"default": {
|
||||||
|
"enabled": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"entry": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"id": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"active": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"from": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"cc": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_cc": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"bcc": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_bcc": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"reply_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_reply_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"bounce_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_bounce_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"disposition_notification_to": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/recipient"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_disposition_notification_to": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/$defs/attachment_config"
|
||||||
|
},
|
||||||
|
"default": []
|
||||||
|
},
|
||||||
|
"combine_attachments": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"fields": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": true,
|
||||||
|
"default": {}
|
||||||
|
},
|
||||||
|
"last_sent": {
|
||||||
|
"type": "string",
|
||||||
|
"format": "date-time"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"source": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"type",
|
||||||
|
"path"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"type": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"csv",
|
||||||
|
"json"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"delimiter": {
|
||||||
|
"type": "string",
|
||||||
|
"default": ";"
|
||||||
|
},
|
||||||
|
"encoding": {
|
||||||
|
"type": "string",
|
||||||
|
"default": "utf-8"
|
||||||
|
},
|
||||||
|
"has_header": {
|
||||||
|
"type": "boolean",
|
||||||
|
"default": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
1
server/app/mailer/sending/__init__.py
Normal file
1
server/app/mailer/sending/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
"""Sending helpers for MultiMailer."""
|
||||||
195
server/app/mailer/sending/imap.py
Normal file
195
server/app/mailer/sending/imap.py
Normal file
@@ -0,0 +1,195 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import imaplib
|
||||||
|
import re
|
||||||
|
import socket
|
||||||
|
import ssl
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from app.mailer.campaign.models import ImapConfig, TransportSecurity
|
||||||
|
|
||||||
|
|
||||||
|
class ImapConfigurationError(ValueError):
|
||||||
|
"""Raised when IMAP settings are incomplete or inconsistent."""
|
||||||
|
|
||||||
|
|
||||||
|
class ImapAppendError(RuntimeError):
|
||||||
|
"""Raised when APPENDing to Sent fails.
|
||||||
|
|
||||||
|
temporary=True means retrying later may help. temporary=False means the
|
||||||
|
configuration or mailbox choice probably needs user/admin attention.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, message: str, *, temporary: bool | None = None):
|
||||||
|
super().__init__(message)
|
||||||
|
self.temporary = temporary
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ImapAppendResult:
|
||||||
|
host: str
|
||||||
|
port: int
|
||||||
|
security: str
|
||||||
|
folder: str
|
||||||
|
bytes_appended: int
|
||||||
|
response: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _require_imap_config(config: ImapConfig) -> tuple[str, int]:
|
||||||
|
if not config.enabled:
|
||||||
|
raise ImapConfigurationError("IMAP is disabled")
|
||||||
|
if not config.host:
|
||||||
|
raise ImapConfigurationError("IMAP host is required")
|
||||||
|
if not config.port:
|
||||||
|
raise ImapConfigurationError("IMAP port is required")
|
||||||
|
if bool(config.username) != bool(config.password):
|
||||||
|
raise ImapConfigurationError("IMAP username and password must be provided together, or both omitted")
|
||||||
|
return config.host, config.port
|
||||||
|
|
||||||
|
|
||||||
|
def _open_imap(config: ImapConfig) -> imaplib.IMAP4:
|
||||||
|
host, port = _require_imap_config(config)
|
||||||
|
context = ssl.create_default_context()
|
||||||
|
|
||||||
|
try:
|
||||||
|
if config.security == TransportSecurity.TLS:
|
||||||
|
client: imaplib.IMAP4 = imaplib.IMAP4_SSL(host=host, port=port, timeout=config.timeout_seconds, ssl_context=context)
|
||||||
|
else:
|
||||||
|
client = imaplib.IMAP4(host=host, port=port, timeout=config.timeout_seconds)
|
||||||
|
if config.security == TransportSecurity.STARTTLS:
|
||||||
|
typ, data = client.starttls(ssl_context=context)
|
||||||
|
if typ != "OK":
|
||||||
|
raise ImapAppendError(f"IMAP STARTTLS failed: {data!r}", temporary=True)
|
||||||
|
|
||||||
|
if config.username and config.password:
|
||||||
|
typ, data = client.login(config.username, config.password)
|
||||||
|
if typ != "OK":
|
||||||
|
raise ImapAppendError(f"IMAP login failed: {data!r}", temporary=False)
|
||||||
|
return client
|
||||||
|
except Exception:
|
||||||
|
try:
|
||||||
|
client.logout() # type: ignore[possibly-undefined]
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_item(item: bytes | str | None) -> str:
|
||||||
|
if item is None:
|
||||||
|
return ""
|
||||||
|
if isinstance(item, bytes):
|
||||||
|
return item.decode("utf-8", errors="replace")
|
||||||
|
return item
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_mailbox_name(list_response_line: bytes | str) -> tuple[str, set[str]] | None:
|
||||||
|
"""Best-effort parser for IMAP LIST response lines.
|
||||||
|
|
||||||
|
Example lines:
|
||||||
|
(\\HasNoChildren \\Sent) "/" "Sent"
|
||||||
|
(\\HasNoChildren) "/" "Sent Items"
|
||||||
|
"""
|
||||||
|
|
||||||
|
line = _decode_item(list_response_line).strip()
|
||||||
|
flags_match = re.match(r"^\((?P<flags>[^)]*)\)\s+", line)
|
||||||
|
flags = set()
|
||||||
|
if flags_match:
|
||||||
|
flags = {part.lower() for part in flags_match.group("flags").split()}
|
||||||
|
|
||||||
|
quoted = re.findall(r'"((?:[^"\\]|\\.)*)"', line)
|
||||||
|
if quoted:
|
||||||
|
# Usually: delimiter, mailbox. Take the last quoted token.
|
||||||
|
return quoted[-1].replace(r'\"', '"'), flags
|
||||||
|
|
||||||
|
# Fallback for unquoted final atom.
|
||||||
|
parts = line.split()
|
||||||
|
if parts:
|
||||||
|
return parts[-1], flags
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def discover_sent_folder(client: imaplib.IMAP4) -> str | None:
|
||||||
|
typ, data = client.list()
|
||||||
|
if typ != "OK" or not data:
|
||||||
|
return None
|
||||||
|
|
||||||
|
parsed: list[tuple[str, set[str]]] = []
|
||||||
|
for item in data:
|
||||||
|
extracted = _extract_mailbox_name(item)
|
||||||
|
if extracted:
|
||||||
|
parsed.append(extracted)
|
||||||
|
|
||||||
|
for name, flags in parsed:
|
||||||
|
if "\\sent" in flags or "\\sentmail" in flags:
|
||||||
|
return name
|
||||||
|
|
||||||
|
common_names = [
|
||||||
|
"Sent",
|
||||||
|
"Sent Items",
|
||||||
|
"Sent Messages",
|
||||||
|
"Gesendet",
|
||||||
|
"Gesendete Elemente",
|
||||||
|
"INBOX.Sent",
|
||||||
|
"INBOX/Sent",
|
||||||
|
]
|
||||||
|
names = {name.lower(): name for name, _ in parsed}
|
||||||
|
for candidate in common_names:
|
||||||
|
if candidate.lower() in names:
|
||||||
|
return names[candidate.lower()]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_sent_folder(*, config: ImapConfig, requested_folder: str | None, client: imaplib.IMAP4) -> str:
|
||||||
|
if requested_folder and requested_folder != "auto":
|
||||||
|
return requested_folder
|
||||||
|
if config.sent_folder and config.sent_folder != "auto":
|
||||||
|
return config.sent_folder
|
||||||
|
discovered = discover_sent_folder(client)
|
||||||
|
if discovered:
|
||||||
|
return discovered
|
||||||
|
raise ImapConfigurationError("Could not discover Sent folder; configure delivery.imap_append_sent.folder or server.imap.sent_folder")
|
||||||
|
|
||||||
|
|
||||||
|
def append_message_to_sent(
|
||||||
|
message_bytes: bytes,
|
||||||
|
*,
|
||||||
|
imap_config: ImapConfig,
|
||||||
|
folder: str | None = None,
|
||||||
|
) -> ImapAppendResult:
|
||||||
|
"""Append a sent MIME message to the configured IMAP Sent folder.
|
||||||
|
|
||||||
|
The SMTP send remains authoritative. APPEND is a separate best-effort step
|
||||||
|
and should not be used to decide whether an email was sent.
|
||||||
|
"""
|
||||||
|
|
||||||
|
host, port = _require_imap_config(imap_config)
|
||||||
|
client: imaplib.IMAP4 | None = None
|
||||||
|
try:
|
||||||
|
client = _open_imap(imap_config)
|
||||||
|
target_folder = _effective_sent_folder(config=imap_config, requested_folder=folder, client=client)
|
||||||
|
internal_date = imaplib.Time2Internaldate(time.time())
|
||||||
|
typ, data = client.append(target_folder, "\\Seen", internal_date, message_bytes)
|
||||||
|
if typ != "OK":
|
||||||
|
raise ImapAppendError(f"IMAP APPEND failed for folder {target_folder!r}: {data!r}", temporary=False)
|
||||||
|
response = "; ".join(_decode_item(item) for item in (data or [])) or None
|
||||||
|
return ImapAppendResult(
|
||||||
|
host=host,
|
||||||
|
port=port,
|
||||||
|
security=imap_config.security.value,
|
||||||
|
folder=target_folder,
|
||||||
|
bytes_appended=len(message_bytes),
|
||||||
|
response=response,
|
||||||
|
)
|
||||||
|
except ImapAppendError:
|
||||||
|
raise
|
||||||
|
except (OSError, socket.timeout, imaplib.IMAP4.abort) as exc:
|
||||||
|
raise ImapAppendError(f"IMAP append failed: {exc}", temporary=True) from exc
|
||||||
|
except imaplib.IMAP4.error as exc:
|
||||||
|
raise ImapAppendError(f"IMAP append failed: {exc}", temporary=False) from exc
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
client.logout()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
606
server/app/mailer/sending/jobs.py
Normal file
606
server/app/mailer/sending/jobs.py
Normal file
@@ -0,0 +1,606 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from email import policy
|
||||||
|
from email.parser import BytesParser
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import (
|
||||||
|
Campaign,
|
||||||
|
CampaignJob,
|
||||||
|
CampaignStatus,
|
||||||
|
CampaignVersion,
|
||||||
|
JobBuildStatus,
|
||||||
|
JobImapStatus,
|
||||||
|
JobQueueStatus,
|
||||||
|
JobSendStatus,
|
||||||
|
JobValidationStatus,
|
||||||
|
ImapAppendAttempt,
|
||||||
|
SendAttempt,
|
||||||
|
)
|
||||||
|
from app.mailer.campaign.loader import load_campaign_config
|
||||||
|
from app.mailer.campaign.models import CampaignConfig
|
||||||
|
from app.mailer.persistence.campaigns import _write_campaign_snapshot
|
||||||
|
from app.mailer.sending.rate_limit import wait_for_rate_limit
|
||||||
|
from app.mailer.sending.smtp import SmtpConfigurationError, SmtpSendError, send_email_message
|
||||||
|
from app.mailer.sending.imap import ImapAppendError, ImapConfigurationError, append_message_to_sent
|
||||||
|
|
||||||
|
|
||||||
|
class QueueingError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class SendJobError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class QueueCampaignResult:
|
||||||
|
campaign_id: str
|
||||||
|
version_id: str
|
||||||
|
queued_count: int
|
||||||
|
skipped_count: int
|
||||||
|
blocked_count: int
|
||||||
|
enqueued_count: int
|
||||||
|
dry_run: bool = False
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"campaign_id": self.campaign_id,
|
||||||
|
"version_id": self.version_id,
|
||||||
|
"queued_count": self.queued_count,
|
||||||
|
"skipped_count": self.skipped_count,
|
||||||
|
"blocked_count": self.blocked_count,
|
||||||
|
"enqueued_count": self.enqueued_count,
|
||||||
|
"dry_run": self.dry_run,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class SendJobResult:
|
||||||
|
job_id: str
|
||||||
|
status: str
|
||||||
|
attempt_number: int
|
||||||
|
dry_run: bool = False
|
||||||
|
message: str | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"job_id": self.job_id,
|
||||||
|
"status": self.status,
|
||||||
|
"attempt_number": self.attempt_number,
|
||||||
|
"dry_run": self.dry_run,
|
||||||
|
"message": self.message,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class AppendSentResult:
|
||||||
|
job_id: str
|
||||||
|
status: str
|
||||||
|
attempt_number: int
|
||||||
|
dry_run: bool = False
|
||||||
|
folder: str | None = None
|
||||||
|
message: str | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"job_id": self.job_id,
|
||||||
|
"status": self.status,
|
||||||
|
"attempt_number": self.attempt_number,
|
||||||
|
"dry_run": self.dry_run,
|
||||||
|
"folder": self.folder,
|
||||||
|
"message": self.message,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
QUEUEABLE_VALIDATION_STATUSES = {
|
||||||
|
JobValidationStatus.READY.value,
|
||||||
|
JobValidationStatus.WARNING.value,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _utcnow() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_campaign_for_tenant(session: Session, *, campaign_id: str, tenant_id: str) -> Campaign:
|
||||||
|
campaign = session.query(Campaign).filter(Campaign.id == campaign_id, Campaign.tenant_id == tenant_id).one_or_none()
|
||||||
|
if not campaign:
|
||||||
|
raise QueueingError(f"Campaign not found or not accessible: {campaign_id}")
|
||||||
|
return campaign
|
||||||
|
|
||||||
|
|
||||||
|
def _get_current_version(session: Session, campaign: Campaign, version_id: str | None = None) -> CampaignVersion:
|
||||||
|
wanted = version_id or campaign.current_version_id
|
||||||
|
if not wanted:
|
||||||
|
raise QueueingError("Campaign has no current version")
|
||||||
|
version = session.get(CampaignVersion, wanted)
|
||||||
|
if not version or version.campaign_id != campaign.id:
|
||||||
|
raise QueueingError(f"Campaign version not found or not part of campaign: {wanted}")
|
||||||
|
return version
|
||||||
|
|
||||||
|
|
||||||
|
def _load_version_campaign_config(version: CampaignVersion) -> tuple[Path, CampaignConfig]:
|
||||||
|
snapshot_path = _write_campaign_snapshot(version)
|
||||||
|
return snapshot_path, load_campaign_config(snapshot_path)
|
||||||
|
|
||||||
|
|
||||||
|
def _celery_enqueue_send_job(job_id: str) -> None:
|
||||||
|
from app.celery_app import celery
|
||||||
|
|
||||||
|
celery.send_task("multimailer.send_email", args=[job_id], queue="send_email")
|
||||||
|
|
||||||
|
|
||||||
|
def _celery_enqueue_append_sent_job(job_id: str) -> None:
|
||||||
|
from app.celery_app import celery
|
||||||
|
|
||||||
|
celery.send_task("multimailer.append_sent", args=[job_id], queue="append_sent")
|
||||||
|
|
||||||
|
|
||||||
|
def queue_campaign_jobs(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
version_id: str | None = None,
|
||||||
|
enqueue_celery: bool = True,
|
||||||
|
include_warnings: bool = True,
|
||||||
|
dry_run: bool = False,
|
||||||
|
) -> QueueCampaignResult:
|
||||||
|
"""Move queueable DB jobs to QUEUED and optionally enqueue Celery tasks."""
|
||||||
|
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
version = _get_current_version(session, campaign, version_id=version_id)
|
||||||
|
|
||||||
|
allowed_validation = {JobValidationStatus.READY.value}
|
||||||
|
if include_warnings:
|
||||||
|
allowed_validation.add(JobValidationStatus.WARNING.value)
|
||||||
|
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(CampaignJob.tenant_id == tenant_id, CampaignJob.campaign_version_id == version.id)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
if not jobs:
|
||||||
|
raise QueueingError("Campaign version has no jobs. Build messages before queueing.")
|
||||||
|
|
||||||
|
queued: list[CampaignJob] = []
|
||||||
|
skipped_count = 0
|
||||||
|
blocked_count = 0
|
||||||
|
for job in jobs:
|
||||||
|
if job.queue_status in {JobQueueStatus.CANCELLED.value, JobQueueStatus.SENDING.value} or job.send_status == JobSendStatus.SENT.value:
|
||||||
|
skipped_count += 1
|
||||||
|
continue
|
||||||
|
if job.build_status != JobBuildStatus.BUILT.value or job.validation_status not in allowed_validation:
|
||||||
|
blocked_count += 1
|
||||||
|
continue
|
||||||
|
if not job.eml_local_path and not job.eml_storage_key:
|
||||||
|
job.last_error = "Job has no generated EML path/storage key. Rebuild with write_eml enabled before queueing."
|
||||||
|
blocked_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
queued.append(job)
|
||||||
|
if not dry_run:
|
||||||
|
job.queue_status = JobQueueStatus.QUEUED.value
|
||||||
|
job.send_status = JobSendStatus.QUEUED.value
|
||||||
|
job.queued_at = _utcnow()
|
||||||
|
job.last_error = None
|
||||||
|
session.add(job)
|
||||||
|
|
||||||
|
if not dry_run:
|
||||||
|
if queued:
|
||||||
|
campaign.status = CampaignStatus.QUEUED.value
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
enqueued_count = 0
|
||||||
|
if enqueue_celery and not dry_run:
|
||||||
|
for job in queued:
|
||||||
|
_celery_enqueue_send_job(job.id)
|
||||||
|
enqueued_count += 1
|
||||||
|
|
||||||
|
return QueueCampaignResult(
|
||||||
|
campaign_id=campaign.id,
|
||||||
|
version_id=version.id,
|
||||||
|
queued_count=len(queued),
|
||||||
|
skipped_count=skipped_count,
|
||||||
|
blocked_count=blocked_count,
|
||||||
|
enqueued_count=enqueued_count,
|
||||||
|
dry_run=dry_run,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def enqueue_existing_queued_jobs(session: Session, *, tenant_id: str, campaign_id: str) -> int:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.queue_status == JobQueueStatus.QUEUED.value,
|
||||||
|
CampaignJob.send_status.in_([JobSendStatus.QUEUED.value, JobSendStatus.FAILED_TEMPORARY.value]),
|
||||||
|
)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
for job in jobs:
|
||||||
|
_celery_enqueue_send_job(job.id)
|
||||||
|
return len(jobs)
|
||||||
|
|
||||||
|
|
||||||
|
def pause_campaign_jobs(session: Session, *, tenant_id: str, campaign_id: str) -> dict[str, Any]:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
changed = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.queue_status == JobQueueStatus.QUEUED.value,
|
||||||
|
)
|
||||||
|
.update({CampaignJob.queue_status: JobQueueStatus.PAUSED.value}, synchronize_session=False)
|
||||||
|
)
|
||||||
|
if changed:
|
||||||
|
campaign.status = CampaignStatus.READY_TO_QUEUE.value
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
return {"campaign_id": campaign.id, "paused_count": int(changed)}
|
||||||
|
|
||||||
|
|
||||||
|
def resume_campaign_jobs(session: Session, *, tenant_id: str, campaign_id: str, enqueue_celery: bool = True) -> dict[str, Any]:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.queue_status == JobQueueStatus.PAUSED.value,
|
||||||
|
)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
for job in jobs:
|
||||||
|
job.queue_status = JobQueueStatus.QUEUED.value
|
||||||
|
job.send_status = JobSendStatus.QUEUED.value
|
||||||
|
session.add(job)
|
||||||
|
if jobs:
|
||||||
|
campaign.status = CampaignStatus.QUEUED.value
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
enqueued_count = 0
|
||||||
|
if enqueue_celery:
|
||||||
|
for job in jobs:
|
||||||
|
_celery_enqueue_send_job(job.id)
|
||||||
|
enqueued_count += 1
|
||||||
|
return {"campaign_id": campaign.id, "resumed_count": len(jobs), "enqueued_count": enqueued_count}
|
||||||
|
|
||||||
|
|
||||||
|
def cancel_campaign_jobs(session: Session, *, tenant_id: str, campaign_id: str) -> dict[str, Any]:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.send_status.notin_([JobSendStatus.SENT.value]),
|
||||||
|
)
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
for job in jobs:
|
||||||
|
if job.queue_status != JobQueueStatus.SENDING.value:
|
||||||
|
job.queue_status = JobQueueStatus.CANCELLED.value
|
||||||
|
job.send_status = JobSendStatus.CANCELLED.value
|
||||||
|
session.add(job)
|
||||||
|
campaign.status = CampaignStatus.CANCELLED.value
|
||||||
|
session.add(campaign)
|
||||||
|
session.commit()
|
||||||
|
return {"campaign_id": campaign.id, "cancelled_count": len(jobs)}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_eml_bytes_for_job(job: CampaignJob) -> bytes:
|
||||||
|
if job.eml_local_path:
|
||||||
|
path = Path(job.eml_local_path)
|
||||||
|
if not path.exists():
|
||||||
|
raise SendJobError(f"Generated EML file does not exist: {path}")
|
||||||
|
return path.read_bytes()
|
||||||
|
raise SendJobError("Only local EML paths are supported for sending in this implementation step")
|
||||||
|
|
||||||
|
|
||||||
|
def _load_eml_for_job(job: CampaignJob):
|
||||||
|
return BytesParser(policy=policy.default).parsebytes(_load_eml_bytes_for_job(job))
|
||||||
|
|
||||||
|
|
||||||
|
def _addresses_from_job(job: CampaignJob, field: str) -> list[str]:
|
||||||
|
data = job.resolved_recipients or {}
|
||||||
|
values = data.get(field) or []
|
||||||
|
return [item.get("email") for item in values if isinstance(item, dict) and item.get("email")]
|
||||||
|
|
||||||
|
|
||||||
|
def _sender_from_job(job: CampaignJob, config: CampaignConfig) -> str:
|
||||||
|
data = job.resolved_recipients or {}
|
||||||
|
bounce_to = _addresses_from_job(job, "bounce_to")
|
||||||
|
if bounce_to:
|
||||||
|
return bounce_to[0]
|
||||||
|
from_data = data.get("from") if isinstance(data, dict) else None
|
||||||
|
if isinstance(from_data, dict) and from_data.get("email"):
|
||||||
|
return from_data["email"]
|
||||||
|
if config.server.smtp and config.server.smtp.username:
|
||||||
|
return config.server.smtp.username
|
||||||
|
raise SmtpConfigurationError("No envelope sender could be determined")
|
||||||
|
|
||||||
|
|
||||||
|
def _recipients_from_job(job: CampaignJob) -> list[str]:
|
||||||
|
recipients: list[str] = []
|
||||||
|
for field in ["to", "cc", "bcc"]:
|
||||||
|
recipients.extend(_addresses_from_job(job, field))
|
||||||
|
# Preserve order while de-duplicating.
|
||||||
|
return list(dict.fromkeys(recipients))
|
||||||
|
|
||||||
|
|
||||||
|
def _record_attempt_start(session: Session, job: CampaignJob) -> SendAttempt:
|
||||||
|
attempt = SendAttempt(
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_number=job.attempt_count + 1,
|
||||||
|
started_at=_utcnow(),
|
||||||
|
)
|
||||||
|
job.attempt_count += 1
|
||||||
|
job.queue_status = JobQueueStatus.SENDING.value
|
||||||
|
job.send_status = JobSendStatus.SENDING.value
|
||||||
|
job.last_error = None
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return attempt
|
||||||
|
|
||||||
|
|
||||||
|
def _update_campaign_after_job(session: Session, campaign_id: str) -> None:
|
||||||
|
session.flush()
|
||||||
|
campaign = session.get(Campaign, campaign_id)
|
||||||
|
if not campaign:
|
||||||
|
return
|
||||||
|
remaining = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.queue_status.in_([JobQueueStatus.QUEUED.value, JobQueueStatus.SENDING.value, JobQueueStatus.PAUSED.value]),
|
||||||
|
)
|
||||||
|
.count()
|
||||||
|
)
|
||||||
|
failed = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.campaign_id == campaign_id,
|
||||||
|
CampaignJob.send_status.in_([JobSendStatus.FAILED_TEMPORARY.value, JobSendStatus.FAILED_PERMANENT.value]),
|
||||||
|
)
|
||||||
|
.count()
|
||||||
|
)
|
||||||
|
sent = session.query(CampaignJob).filter(CampaignJob.campaign_id == campaign_id, CampaignJob.send_status == JobSendStatus.SENT.value).count()
|
||||||
|
if remaining:
|
||||||
|
campaign.status = CampaignStatus.QUEUED.value
|
||||||
|
elif failed:
|
||||||
|
campaign.status = CampaignStatus.FAILED.value if not sent else CampaignStatus.NEEDS_REVIEW.value
|
||||||
|
elif sent:
|
||||||
|
campaign.status = CampaignStatus.SENT.value
|
||||||
|
session.add(campaign)
|
||||||
|
|
||||||
|
|
||||||
|
def send_campaign_job(session: Session, *, job_id: str, dry_run: bool = False, use_rate_limit: bool = True, enqueue_imap_task: bool = False) -> SendJobResult:
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if not job:
|
||||||
|
raise SendJobError(f"Job not found: {job_id}")
|
||||||
|
if job.queue_status == JobQueueStatus.CANCELLED.value or job.send_status == JobSendStatus.CANCELLED.value:
|
||||||
|
return SendJobResult(job_id=job_id, status="cancelled", attempt_number=job.attempt_count, dry_run=dry_run)
|
||||||
|
if job.queue_status == JobQueueStatus.PAUSED.value:
|
||||||
|
return SendJobResult(job_id=job_id, status="paused", attempt_number=job.attempt_count, dry_run=dry_run)
|
||||||
|
if job.send_status == JobSendStatus.SENT.value:
|
||||||
|
return SendJobResult(job_id=job_id, status="already_sent", attempt_number=job.attempt_count, dry_run=dry_run)
|
||||||
|
if job.queue_status not in {JobQueueStatus.QUEUED.value, JobQueueStatus.SENDING.value}:
|
||||||
|
raise SendJobError(f"Job is not queued: {job.queue_status}")
|
||||||
|
|
||||||
|
version = session.get(CampaignVersion, job.campaign_version_id)
|
||||||
|
if not version:
|
||||||
|
raise SendJobError("Campaign version not found")
|
||||||
|
_, config = _load_version_campaign_config(version)
|
||||||
|
if not config.server.smtp:
|
||||||
|
raise SmtpConfigurationError("Campaign has no SMTP configuration")
|
||||||
|
|
||||||
|
message = _load_eml_for_job(job)
|
||||||
|
envelope_from = _sender_from_job(job, config)
|
||||||
|
envelope_recipients = _recipients_from_job(job)
|
||||||
|
if not envelope_recipients:
|
||||||
|
raise SmtpConfigurationError("No envelope recipients could be determined")
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
return SendJobResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status="dry_run",
|
||||||
|
attempt_number=job.attempt_count,
|
||||||
|
dry_run=True,
|
||||||
|
message=f"Would send to {len(envelope_recipients)} recipient(s) from {envelope_from}",
|
||||||
|
)
|
||||||
|
|
||||||
|
attempt = _record_attempt_start(session, job)
|
||||||
|
try:
|
||||||
|
wait_for_rate_limit(
|
||||||
|
key=f"tenant:{job.tenant_id}:campaign:{job.campaign_id}",
|
||||||
|
messages_per_minute=config.delivery.rate_limit.messages_per_minute,
|
||||||
|
enabled=use_rate_limit,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = send_email_message(
|
||||||
|
message,
|
||||||
|
smtp_config=config.server.smtp,
|
||||||
|
envelope_from=envelope_from,
|
||||||
|
envelope_recipients=envelope_recipients,
|
||||||
|
)
|
||||||
|
attempt.finished_at = _utcnow()
|
||||||
|
attempt.smtp_response = json.dumps(asdict(result), default=str)
|
||||||
|
job.queue_status = JobQueueStatus.DRAFT.value
|
||||||
|
job.send_status = JobSendStatus.SENT.value
|
||||||
|
job.sent_at = _utcnow()
|
||||||
|
if config.delivery.imap_append_sent.enabled:
|
||||||
|
job.imap_status = JobImapStatus.PENDING.value
|
||||||
|
else:
|
||||||
|
job.imap_status = JobImapStatus.NOT_REQUESTED.value
|
||||||
|
job.last_error = None
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
_update_campaign_after_job(session, job.campaign_id)
|
||||||
|
session.commit()
|
||||||
|
if enqueue_imap_task and job.imap_status == JobImapStatus.PENDING.value:
|
||||||
|
_celery_enqueue_append_sent_job(job.id)
|
||||||
|
return SendJobResult(job_id=job.id, status="sent", attempt_number=attempt.attempt_number)
|
||||||
|
|
||||||
|
except (SmtpConfigurationError, SmtpSendError, SendJobError, OSError) as exc:
|
||||||
|
attempt.finished_at = _utcnow()
|
||||||
|
attempt.error_type = exc.__class__.__name__
|
||||||
|
attempt.error_message = str(exc)
|
||||||
|
job.last_error = str(exc)
|
||||||
|
retry = getattr(exc, "temporary", None) is True
|
||||||
|
max_attempts = config.delivery.retry.max_attempts
|
||||||
|
if retry and job.attempt_count < max_attempts:
|
||||||
|
job.queue_status = JobQueueStatus.QUEUED.value
|
||||||
|
job.send_status = JobSendStatus.FAILED_TEMPORARY.value
|
||||||
|
else:
|
||||||
|
job.queue_status = JobQueueStatus.DRAFT.value
|
||||||
|
job.send_status = JobSendStatus.FAILED_PERMANENT.value
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
_update_campaign_after_job(session, job.campaign_id)
|
||||||
|
session.commit()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def _record_imap_attempt_start(session: Session, job: CampaignJob) -> ImapAppendAttempt:
|
||||||
|
existing_count = session.query(ImapAppendAttempt).filter(ImapAppendAttempt.job_id == job.id).count()
|
||||||
|
attempt = ImapAppendAttempt(
|
||||||
|
job_id=job.id,
|
||||||
|
attempt_number=existing_count + 1,
|
||||||
|
status="running",
|
||||||
|
)
|
||||||
|
job.imap_status = JobImapStatus.PENDING.value
|
||||||
|
job.last_error = None
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return attempt
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_imap_folder(config: CampaignConfig) -> str:
|
||||||
|
delivery_folder = config.delivery.imap_append_sent.folder
|
||||||
|
if delivery_folder and delivery_folder != "auto":
|
||||||
|
return delivery_folder
|
||||||
|
if config.server.imap and config.server.imap.sent_folder and config.server.imap.sent_folder != "auto":
|
||||||
|
return config.server.imap.sent_folder
|
||||||
|
return "auto"
|
||||||
|
|
||||||
|
|
||||||
|
def append_sent_for_job(session: Session, *, job_id: str, dry_run: bool = False) -> AppendSentResult:
|
||||||
|
"""Append one successfully sent job's exact EML to the configured IMAP Sent folder."""
|
||||||
|
|
||||||
|
job = session.get(CampaignJob, job_id)
|
||||||
|
if not job:
|
||||||
|
raise SendJobError(f"Job not found: {job_id}")
|
||||||
|
if job.send_status != JobSendStatus.SENT.value:
|
||||||
|
return AppendSentResult(job_id=job_id, status="not_sent", attempt_number=0, dry_run=dry_run, message="Job has not been sent")
|
||||||
|
if job.imap_status == JobImapStatus.NOT_REQUESTED.value:
|
||||||
|
return AppendSentResult(job_id=job_id, status="not_requested", attempt_number=0, dry_run=dry_run)
|
||||||
|
if job.imap_status == JobImapStatus.APPENDED.value:
|
||||||
|
attempts = session.query(ImapAppendAttempt).filter(ImapAppendAttempt.job_id == job.id).count()
|
||||||
|
return AppendSentResult(job_id=job_id, status="already_appended", attempt_number=attempts, dry_run=dry_run)
|
||||||
|
if job.imap_status == JobImapStatus.SKIPPED.value:
|
||||||
|
attempts = session.query(ImapAppendAttempt).filter(ImapAppendAttempt.job_id == job.id).count()
|
||||||
|
return AppendSentResult(job_id=job_id, status="skipped", attempt_number=attempts, dry_run=dry_run)
|
||||||
|
|
||||||
|
version = session.get(CampaignVersion, job.campaign_version_id)
|
||||||
|
if not version:
|
||||||
|
raise SendJobError("Campaign version not found")
|
||||||
|
_, config = _load_version_campaign_config(version)
|
||||||
|
if not config.delivery.imap_append_sent.enabled:
|
||||||
|
job.imap_status = JobImapStatus.NOT_REQUESTED.value
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return AppendSentResult(job_id=job.id, status="not_requested", attempt_number=0, dry_run=dry_run)
|
||||||
|
if not config.server.imap or not config.server.imap.enabled:
|
||||||
|
job.imap_status = JobImapStatus.SKIPPED.value
|
||||||
|
job.last_error = "IMAP append requested, but server.imap is missing or disabled"
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return AppendSentResult(job_id=job.id, status="skipped", attempt_number=0, dry_run=dry_run, message=job.last_error)
|
||||||
|
|
||||||
|
message_bytes = _load_eml_bytes_for_job(job)
|
||||||
|
folder = _effective_imap_folder(config)
|
||||||
|
if dry_run:
|
||||||
|
attempts = session.query(ImapAppendAttempt).filter(ImapAppendAttempt.job_id == job.id).count()
|
||||||
|
return AppendSentResult(
|
||||||
|
job_id=job.id,
|
||||||
|
status="dry_run",
|
||||||
|
attempt_number=attempts,
|
||||||
|
dry_run=True,
|
||||||
|
folder=folder,
|
||||||
|
message=f"Would append {len(message_bytes)} bytes to IMAP folder {folder!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
attempt = _record_imap_attempt_start(session, job)
|
||||||
|
try:
|
||||||
|
result = append_message_to_sent(
|
||||||
|
message_bytes,
|
||||||
|
imap_config=config.server.imap,
|
||||||
|
folder=None if folder == "auto" else folder,
|
||||||
|
)
|
||||||
|
attempt.status = "appended"
|
||||||
|
attempt.folder = result.folder
|
||||||
|
job.imap_status = JobImapStatus.APPENDED.value
|
||||||
|
job.last_error = None
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
return AppendSentResult(job_id=job.id, status="appended", attempt_number=attempt.attempt_number, folder=result.folder)
|
||||||
|
except (ImapConfigurationError, ImapAppendError, SendJobError, OSError) as exc:
|
||||||
|
attempt.status = "failed"
|
||||||
|
attempt.folder = None if folder == "auto" else folder
|
||||||
|
attempt.error_message = str(exc)
|
||||||
|
job.imap_status = JobImapStatus.FAILED.value
|
||||||
|
job.last_error = str(exc)
|
||||||
|
session.add(attempt)
|
||||||
|
session.add(job)
|
||||||
|
session.commit()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def enqueue_pending_imap_appends(session: Session, *, tenant_id: str, campaign_id: str, enqueue_celery: bool = True, dry_run: bool = False) -> dict[str, Any]:
|
||||||
|
campaign = _get_campaign_for_tenant(session, campaign_id=campaign_id, tenant_id=tenant_id)
|
||||||
|
jobs = (
|
||||||
|
session.query(CampaignJob)
|
||||||
|
.filter(
|
||||||
|
CampaignJob.tenant_id == tenant_id,
|
||||||
|
CampaignJob.campaign_id == campaign.id,
|
||||||
|
CampaignJob.send_status == JobSendStatus.SENT.value,
|
||||||
|
CampaignJob.imap_status.in_([JobImapStatus.PENDING.value, JobImapStatus.FAILED.value]),
|
||||||
|
)
|
||||||
|
.order_by(CampaignJob.entry_index.asc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
if not dry_run and enqueue_celery:
|
||||||
|
for job in jobs:
|
||||||
|
_celery_enqueue_append_sent_job(job.id)
|
||||||
|
return {
|
||||||
|
"campaign_id": campaign.id,
|
||||||
|
"pending_count": len(jobs),
|
||||||
|
"enqueued_count": 0 if dry_run or not enqueue_celery else len(jobs),
|
||||||
|
"dry_run": dry_run,
|
||||||
|
}
|
||||||
|
|
||||||
|
def next_retry_delay(config: CampaignConfig, attempt_count: int) -> int:
|
||||||
|
delays = config.delivery.retry.backoff_seconds or [60]
|
||||||
|
index = max(0, min(attempt_count - 1, len(delays) - 1))
|
||||||
|
return int(delays[index])
|
||||||
57
server/app/mailer/sending/rate_limit.py
Normal file
57
server/app/mailer/sending/rate_limit.py
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from redis import Redis
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class RateLimitDecision:
|
||||||
|
key: str
|
||||||
|
messages_per_minute: int
|
||||||
|
gap_seconds: float
|
||||||
|
waited_seconds: float
|
||||||
|
|
||||||
|
|
||||||
|
def _redis_client() -> Redis:
|
||||||
|
return Redis.from_url(settings.redis_url, decode_responses=True)
|
||||||
|
|
||||||
|
|
||||||
|
def wait_for_rate_limit(*, key: str, messages_per_minute: int, enabled: bool = True) -> RateLimitDecision:
|
||||||
|
"""Throttle sends across worker processes using Redis when available.
|
||||||
|
|
||||||
|
The implementation stores the next allowed send timestamp per key. A Redis
|
||||||
|
lock keeps multiple Celery processes from reading/updating the timestamp at
|
||||||
|
the same time. If Redis is unavailable, it falls back to no distributed wait;
|
||||||
|
the per-container Celery concurrency still protects local development.
|
||||||
|
"""
|
||||||
|
|
||||||
|
messages_per_minute = max(1, int(messages_per_minute or 1))
|
||||||
|
gap = 60.0 / messages_per_minute
|
||||||
|
if not enabled:
|
||||||
|
return RateLimitDecision(key=key, messages_per_minute=messages_per_minute, gap_seconds=gap, waited_seconds=0.0)
|
||||||
|
|
||||||
|
redis_key = f"multimailer:ratelimit:{key}:next_allowed"
|
||||||
|
lock_key = f"multimailer:ratelimit:{key}:lock"
|
||||||
|
waited = 0.0
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = _redis_client()
|
||||||
|
with client.lock(lock_key, timeout=30, blocking_timeout=30):
|
||||||
|
now = time.time()
|
||||||
|
raw_next = client.get(redis_key)
|
||||||
|
next_allowed = float(raw_next) if raw_next else now
|
||||||
|
if next_allowed > now:
|
||||||
|
waited = next_allowed - now
|
||||||
|
time.sleep(waited)
|
||||||
|
now = time.time()
|
||||||
|
client.set(redis_key, now + gap, ex=max(60, int(gap * 10)))
|
||||||
|
except (RedisError, TimeoutError, ValueError):
|
||||||
|
# Development fallback: do not fail sending because Redis is absent.
|
||||||
|
waited = 0.0
|
||||||
|
|
||||||
|
return RateLimitDecision(key=key, messages_per_minute=messages_per_minute, gap_seconds=gap, waited_seconds=waited)
|
||||||
157
server/app/mailer/sending/smtp.py
Normal file
157
server/app/mailer/sending/smtp.py
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import smtplib
|
||||||
|
import ssl
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from email.utils import formataddr
|
||||||
|
|
||||||
|
from app.mailer.campaign.models import SmtpConfig, TransportSecurity
|
||||||
|
|
||||||
|
|
||||||
|
class SmtpConfigurationError(ValueError):
|
||||||
|
"""Raised when SMTP settings are incomplete or inconsistent."""
|
||||||
|
|
||||||
|
|
||||||
|
class SmtpSendError(RuntimeError):
|
||||||
|
"""Raised when an SMTP send attempt fails."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class SmtpSendResult:
|
||||||
|
host: str
|
||||||
|
port: int
|
||||||
|
security: str
|
||||||
|
envelope_from: str
|
||||||
|
envelope_recipients: list[str]
|
||||||
|
refused_recipients: dict[str, tuple[int, bytes | str]]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def accepted_count(self) -> int:
|
||||||
|
return len(self.envelope_recipients) - len(self.refused_recipients)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_smtp_config(config: SmtpConfig) -> tuple[str, int]:
|
||||||
|
if not config.host:
|
||||||
|
raise SmtpConfigurationError("SMTP host is required")
|
||||||
|
if not config.port:
|
||||||
|
raise SmtpConfigurationError("SMTP port is required")
|
||||||
|
if bool(config.username) != bool(config.password):
|
||||||
|
raise SmtpConfigurationError("SMTP username and password must be provided together, or both omitted")
|
||||||
|
return config.host, config.port
|
||||||
|
|
||||||
|
|
||||||
|
def _open_smtp(config: SmtpConfig) -> smtplib.SMTP:
|
||||||
|
host, port = _require_smtp_config(config)
|
||||||
|
context = ssl.create_default_context()
|
||||||
|
|
||||||
|
try:
|
||||||
|
if config.security == TransportSecurity.TLS:
|
||||||
|
smtp: smtplib.SMTP = smtplib.SMTP_SSL(host=host, port=port, timeout=config.timeout_seconds, context=context)
|
||||||
|
smtp.ehlo()
|
||||||
|
else:
|
||||||
|
smtp = smtplib.SMTP(host=host, port=port, timeout=config.timeout_seconds)
|
||||||
|
smtp.ehlo()
|
||||||
|
if config.security == TransportSecurity.STARTTLS:
|
||||||
|
smtp.starttls(context=context)
|
||||||
|
smtp.ehlo()
|
||||||
|
|
||||||
|
if config.username and config.password:
|
||||||
|
smtp.login(config.username, config.password)
|
||||||
|
return smtp
|
||||||
|
except Exception:
|
||||||
|
# If construction/login fails after a socket was created, smtplib usually closes
|
||||||
|
# on GC, but explicit cleanup is safer when the variable exists.
|
||||||
|
try:
|
||||||
|
smtp.quit() # type: ignore[possibly-undefined]
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_refused(refused: dict[str, tuple[int, bytes]]) -> dict[str, tuple[int, bytes | str]]:
|
||||||
|
normalized: dict[str, tuple[int, bytes | str]] = {}
|
||||||
|
for recipient, (code, response) in refused.items():
|
||||||
|
try:
|
||||||
|
normalized[recipient] = (code, response.decode("utf-8", errors="replace"))
|
||||||
|
except AttributeError:
|
||||||
|
normalized[recipient] = (code, response)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_test_message(
|
||||||
|
message: EmailMessage,
|
||||||
|
*,
|
||||||
|
test_recipient: str,
|
||||||
|
test_recipient_name: str | None = None,
|
||||||
|
) -> EmailMessage:
|
||||||
|
"""Return a safe copy of a generated campaign message for test delivery.
|
||||||
|
|
||||||
|
The original recipient headers are removed so a test send cannot accidentally
|
||||||
|
leak the real To/Cc list or deliver to the real recipients. The envelope
|
||||||
|
recipient must also be supplied separately to send_email_message().
|
||||||
|
"""
|
||||||
|
|
||||||
|
test_message = copy.deepcopy(message)
|
||||||
|
|
||||||
|
for header in ["To", "Cc", "Bcc"]:
|
||||||
|
if header in test_message:
|
||||||
|
del test_message[header]
|
||||||
|
|
||||||
|
# Replace potential previous marker headers if the user test-sends an EML twice.
|
||||||
|
for header in ["X-MultiMailer-Test-Send"]:
|
||||||
|
if header in test_message:
|
||||||
|
del test_message[header]
|
||||||
|
|
||||||
|
test_message["To"] = formataddr((test_recipient_name or test_recipient, test_recipient))
|
||||||
|
test_message["X-MultiMailer-Test-Send"] = "true"
|
||||||
|
return test_message
|
||||||
|
|
||||||
|
|
||||||
|
def send_email_message(
|
||||||
|
message: EmailMessage,
|
||||||
|
*,
|
||||||
|
smtp_config: SmtpConfig,
|
||||||
|
envelope_from: str,
|
||||||
|
envelope_recipients: list[str],
|
||||||
|
) -> SmtpSendResult:
|
||||||
|
"""Send an EmailMessage through SMTP.
|
||||||
|
|
||||||
|
This low-level function deliberately receives explicit envelope sender and
|
||||||
|
recipients. Headers and SMTP envelope are related but not identical; Bcc and
|
||||||
|
future bounce-address handling depend on keeping them separate.
|
||||||
|
"""
|
||||||
|
|
||||||
|
host, port = _require_smtp_config(smtp_config)
|
||||||
|
if not envelope_from:
|
||||||
|
raise SmtpConfigurationError("SMTP envelope sender is required")
|
||||||
|
if not envelope_recipients:
|
||||||
|
raise SmtpConfigurationError("at least one SMTP envelope recipient is required")
|
||||||
|
|
||||||
|
try:
|
||||||
|
with _open_smtp(smtp_config) as smtp:
|
||||||
|
refused = smtp.send_message(
|
||||||
|
message,
|
||||||
|
from_addr=envelope_from,
|
||||||
|
to_addrs=envelope_recipients,
|
||||||
|
)
|
||||||
|
except smtplib.SMTPAuthenticationError as exc:
|
||||||
|
raise SmtpSendError(f"SMTP authentication failed: {exc.smtp_code} {exc.smtp_error!r}") from exc
|
||||||
|
except smtplib.SMTPRecipientsRefused as exc:
|
||||||
|
raise SmtpSendError(f"all SMTP recipients were refused: {_decode_refused(exc.recipients)}") from exc
|
||||||
|
except smtplib.SMTPSenderRefused as exc:
|
||||||
|
raise SmtpSendError(f"SMTP sender was refused: {exc.smtp_code} {exc.smtp_error!r}") from exc
|
||||||
|
except smtplib.SMTPResponseException as exc:
|
||||||
|
raise SmtpSendError(f"SMTP error: {exc.smtp_code} {exc.smtp_error!r}") from exc
|
||||||
|
except (OSError, smtplib.SMTPException) as exc:
|
||||||
|
raise SmtpSendError(f"SMTP send failed: {exc}") from exc
|
||||||
|
|
||||||
|
return SmtpSendResult(
|
||||||
|
host=host,
|
||||||
|
port=port,
|
||||||
|
security=smtp_config.security.value,
|
||||||
|
envelope_from=envelope_from,
|
||||||
|
envelope_recipients=list(envelope_recipients),
|
||||||
|
refused_recipients=_decode_refused(refused),
|
||||||
|
)
|
||||||
0
server/app/mailer/services/__init__.py
Normal file
0
server/app/mailer/services/__init__.py
Normal file
13
server/app/mailer/services/attachment_matching.py
Normal file
13
server/app/mailer/services/attachment_matching.py
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.mailer.domain.campaign import MailAttachmentConfig
|
||||||
|
|
||||||
|
|
||||||
|
def match_files(base_path: Path, config: MailAttachmentConfig) -> list[Path]:
|
||||||
|
directory = base_path / config.base_dir
|
||||||
|
if not directory.exists():
|
||||||
|
return []
|
||||||
|
iterator = directory.rglob(config.file_filter) if config.include_subdirs else directory.glob(config.file_filter)
|
||||||
|
return sorted(path for path in iterator if path.is_file() and path.stat().st_size >= 0)
|
||||||
135
server/app/mailer/services/campaign_executor.py
Normal file
135
server/app/mailer/services/campaign_executor.py
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import smtplib
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
from app.mailer.domain.campaign import MailCampaign, MailEntry, MailServerSettings, TransportSecurity
|
||||||
|
from app.mailer.domain.queue import MailQueue
|
||||||
|
from app.mailer.domain.recipients import Recipient, RecipientType
|
||||||
|
from app.mailer.services.attachment_matching import match_files
|
||||||
|
from app.mailer.services.zip_service import create_encrypted_zip
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_header(recipients: Iterable[Recipient], recipient_type: RecipientType) -> str:
|
||||||
|
return ", ".join(r.formatted() for r in recipients if r.type == recipient_type)
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_values(recipients: list[Recipient]) -> dict[str, str]:
|
||||||
|
def rows(recipient_type: RecipientType) -> list[Recipient]:
|
||||||
|
return [r for r in recipients if r.type == recipient_type]
|
||||||
|
|
||||||
|
def joined(recipient_type: RecipientType, mode: str) -> str:
|
||||||
|
selected = rows(recipient_type)
|
||||||
|
if mode == "address":
|
||||||
|
return ", ".join(r.address for r in selected)
|
||||||
|
if mode == "name":
|
||||||
|
return ", ".join(r.name or r.address for r in selected)
|
||||||
|
return ", ".join(r.formatted() for r in selected)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"mm_recipients": joined(RecipientType.TO, "formatted"),
|
||||||
|
"mm_recipients_address": joined(RecipientType.TO, "address"),
|
||||||
|
"mm_recipients_name": joined(RecipientType.TO, "name"),
|
||||||
|
"mm_cc": joined(RecipientType.CC, "formatted"),
|
||||||
|
"mm_cc_address": joined(RecipientType.CC, "address"),
|
||||||
|
"mm_cc_name": joined(RecipientType.CC, "name"),
|
||||||
|
"mm_bcc": joined(RecipientType.BCC, "formatted"),
|
||||||
|
"mm_bcc_address": joined(RecipientType.BCC, "address"),
|
||||||
|
"mm_bcc_name": joined(RecipientType.BCC, "name"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _message_attachment_paths(campaign: MailCampaign, entry: MailEntry) -> list[Path]:
|
||||||
|
paths: list[Path] = []
|
||||||
|
if entry.combine_attachments:
|
||||||
|
for config in campaign.global_attachment_configs:
|
||||||
|
paths.extend(match_files(campaign.base_attachment_path, config))
|
||||||
|
if campaign.individual_attachments:
|
||||||
|
for config in entry.attachment_configs:
|
||||||
|
paths.extend(match_files(campaign.base_attachment_path, config))
|
||||||
|
return paths
|
||||||
|
|
||||||
|
|
||||||
|
def get_not_sent_files(campaign: MailCampaign) -> set[Path]:
|
||||||
|
"""Return files matching campaign attachment configs that are not referenced by any built entry."""
|
||||||
|
all_files: set[Path] = set()
|
||||||
|
used_files: set[Path] = set()
|
||||||
|
for config in campaign.global_attachment_configs:
|
||||||
|
all_files.update(match_files(campaign.base_attachment_path, config))
|
||||||
|
for entry in campaign.mail_entries:
|
||||||
|
files = set(_message_attachment_paths(campaign, entry))
|
||||||
|
used_files.update(files)
|
||||||
|
all_files.update(files)
|
||||||
|
return all_files - used_files
|
||||||
|
|
||||||
|
|
||||||
|
def build_message(campaign: MailCampaign, entry: MailEntry, *, zip_attachments: bool = True) -> EmailMessage | None:
|
||||||
|
recipients = campaign.all_recipients_for(entry)
|
||||||
|
attachments = _message_attachment_paths(campaign, entry)
|
||||||
|
if not attachments and not campaign.send_without_attachments:
|
||||||
|
return None
|
||||||
|
|
||||||
|
values = {}
|
||||||
|
values.update(_recipient_values(recipients))
|
||||||
|
values.update(campaign.field_contents.as_value_map("global"))
|
||||||
|
values.update(entry.field_contents.as_value_map("local"))
|
||||||
|
|
||||||
|
message = EmailMessage()
|
||||||
|
sender = entry.from_recipient if campaign.individual_from and entry.from_recipient else campaign.global_from
|
||||||
|
if sender:
|
||||||
|
message["From"] = sender.formatted()
|
||||||
|
to_header = _recipient_header(recipients, RecipientType.TO)
|
||||||
|
cc_header = _recipient_header(recipients, RecipientType.CC)
|
||||||
|
if to_header:
|
||||||
|
message["To"] = to_header
|
||||||
|
if cc_header:
|
||||||
|
message["Cc"] = cc_header
|
||||||
|
message["Subject"] = campaign.subject_template.apply_values(values)
|
||||||
|
message.set_content(campaign.mail_template.apply_values(values))
|
||||||
|
|
||||||
|
attachment_paths = attachments
|
||||||
|
|
||||||
|
if attachments and zip_attachments:
|
||||||
|
number = entry.get_field_content_from_name("number").as_string() if "number" in entry.field_contents.field_map else "attachments"
|
||||||
|
password = entry.get_field_content_from_name("password").as_string() if "password" in entry.field_contents.field_map else ""
|
||||||
|
zip_path = campaign.base_attachment_path / f"{number}.zip"
|
||||||
|
attachment_paths = [create_encrypted_zip(zip_path, attachments, password)]
|
||||||
|
|
||||||
|
for attachment_path in attachment_paths:
|
||||||
|
data = attachment_path.read_bytes()
|
||||||
|
message.add_attachment(data, maintype="application", subtype="octet-stream", filename=attachment_path.name)
|
||||||
|
return message
|
||||||
|
|
||||||
|
|
||||||
|
def build_mail_queue(campaign: MailCampaign, *, zip_attachments: bool = True) -> MailQueue:
|
||||||
|
queue = MailQueue()
|
||||||
|
for entry in campaign.mail_entries:
|
||||||
|
if not entry.is_active:
|
||||||
|
continue
|
||||||
|
message = build_message(campaign, entry, zip_attachments=zip_attachments)
|
||||||
|
if message is not None:
|
||||||
|
queue.add_mail(message)
|
||||||
|
return queue
|
||||||
|
|
||||||
|
|
||||||
|
def send_mail_queue(settings: MailServerSettings, queue: MailQueue) -> MailQueue:
|
||||||
|
retry_queue = MailQueue()
|
||||||
|
if settings.transport_security == TransportSecurity.TLS:
|
||||||
|
smtp = smtplib.SMTP_SSL(settings.server, settings.resolved_port(), timeout=60)
|
||||||
|
else:
|
||||||
|
smtp = smtplib.SMTP(settings.server, settings.resolved_port(), timeout=60)
|
||||||
|
try:
|
||||||
|
if settings.transport_security == TransportSecurity.STARTTLS:
|
||||||
|
smtp.starttls()
|
||||||
|
if settings.username:
|
||||||
|
smtp.login(settings.username, settings.password)
|
||||||
|
for message in queue:
|
||||||
|
try:
|
||||||
|
smtp.send_message(message)
|
||||||
|
except Exception:
|
||||||
|
retry_queue.add_mail(message)
|
||||||
|
finally:
|
||||||
|
smtp.quit()
|
||||||
|
return retry_queue
|
||||||
25
server/app/mailer/services/zip_service.py
Normal file
25
server/app/mailer/services/zip_service.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
try:
|
||||||
|
import pyzipper
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
pyzipper = None
|
||||||
|
|
||||||
|
|
||||||
|
def create_encrypted_zip(output_path: Path, files: list[Path], password: str) -> Path:
|
||||||
|
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
if pyzipper is None:
|
||||||
|
raise RuntimeError("pyzipper is required for writing encrypted ZIP files")
|
||||||
|
with pyzipper.AESZipFile(
|
||||||
|
output_path,
|
||||||
|
"w",
|
||||||
|
compression=pyzipper.ZIP_DEFLATED,
|
||||||
|
encryption=pyzipper.WZ_AES,
|
||||||
|
) as zip_file:
|
||||||
|
if password:
|
||||||
|
zip_file.setpassword(password.encode("utf-8"))
|
||||||
|
for file_path in files:
|
||||||
|
zip_file.write(file_path, arcname=file_path.name)
|
||||||
|
return output_path
|
||||||
47
server/app/main.py
Normal file
47
server/app/main.py
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
|
||||||
|
from app.api.v1 import router as api_v1_router
|
||||||
|
from app.db.bootstrap import bootstrap_dev_data, create_all_tables
|
||||||
|
from app.db.session import SessionLocal
|
||||||
|
from app.settings import settings
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
if settings.app_env == "dev" and settings.dev_bootstrap_enabled:
|
||||||
|
create_all_tables()
|
||||||
|
with SessionLocal() as session:
|
||||||
|
bootstrap_dev_data(session, api_key_secret=settings.dev_bootstrap_api_key)
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(title="MultiMailer Server", version="0.2.0", lifespan=lifespan)
|
||||||
|
|
||||||
|
origins = [item.strip() for item in settings.cors_origins.split(",") if item.strip()]
|
||||||
|
if origins:
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"] if "*" in origins else origins,
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
app.include_router(api_v1_router)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
def health():
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"env": settings.app_env,
|
||||||
|
"api": {"version": "v1", "auth": "api-key"},
|
||||||
|
"storage": {
|
||||||
|
"endpoint": settings.s3_endpoint_url,
|
||||||
|
"bucket": settings.s3_bucket,
|
||||||
|
"region": settings.s3_region,
|
||||||
|
},
|
||||||
|
}
|
||||||
0
server/app/security/__init__.py
Normal file
0
server/app/security/__init__.py
Normal file
80
server/app/security/api_keys.py
Normal file
80
server/app/security/api_keys.py
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import secrets
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db.models import ApiKey, User
|
||||||
|
|
||||||
|
API_KEY_PREFIX_LENGTH = 12
|
||||||
|
API_KEY_RANDOM_BYTES = 32
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class CreatedApiKey:
|
||||||
|
model: ApiKey
|
||||||
|
secret: str
|
||||||
|
|
||||||
|
|
||||||
|
def hash_api_key(secret: str) -> str:
|
||||||
|
return hashlib.sha256(secret.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def verify_api_key(secret: str, expected_hash: str) -> bool:
|
||||||
|
return hmac.compare_digest(hash_api_key(secret), expected_hash)
|
||||||
|
|
||||||
|
|
||||||
|
def generate_api_key_secret() -> str:
|
||||||
|
return "mm_" + secrets.token_urlsafe(API_KEY_RANDOM_BYTES)
|
||||||
|
|
||||||
|
|
||||||
|
def api_key_prefix(secret: str) -> str:
|
||||||
|
# Prefix is only a lookup helper and must not be enough to authenticate.
|
||||||
|
return secret[:API_KEY_PREFIX_LENGTH]
|
||||||
|
|
||||||
|
|
||||||
|
def create_api_key(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
user: User,
|
||||||
|
name: str,
|
||||||
|
scopes: list[str],
|
||||||
|
secret: str | None = None,
|
||||||
|
expires_at: datetime | None = None,
|
||||||
|
) -> CreatedApiKey:
|
||||||
|
secret = secret or generate_api_key_secret()
|
||||||
|
model = ApiKey(
|
||||||
|
tenant_id=user.tenant_id,
|
||||||
|
user_id=user.id,
|
||||||
|
name=name,
|
||||||
|
prefix=api_key_prefix(secret),
|
||||||
|
key_hash=hash_api_key(secret),
|
||||||
|
scopes=scopes,
|
||||||
|
expires_at=expires_at,
|
||||||
|
)
|
||||||
|
session.add(model)
|
||||||
|
session.flush()
|
||||||
|
return CreatedApiKey(model=model, secret=secret)
|
||||||
|
|
||||||
|
|
||||||
|
def authenticate_api_key(session: Session, secret: str) -> ApiKey | None:
|
||||||
|
prefix = api_key_prefix(secret)
|
||||||
|
candidates = session.query(ApiKey).filter(ApiKey.prefix == prefix, ApiKey.revoked_at.is_(None)).all()
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
for candidate in candidates:
|
||||||
|
if candidate.expires_at and candidate.expires_at < now:
|
||||||
|
continue
|
||||||
|
if verify_api_key(secret, candidate.key_hash):
|
||||||
|
candidate.last_used_at = now
|
||||||
|
session.add(candidate)
|
||||||
|
return candidate
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def has_scope(api_key: ApiKey, required_scope: str) -> bool:
|
||||||
|
scopes = set(api_key.scopes or [])
|
||||||
|
return "*" in scopes or required_scope in scopes
|
||||||
33
server/app/settings.py
Normal file
33
server/app/settings.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
from pydantic import Field
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(env_file=None, extra="ignore")
|
||||||
|
|
||||||
|
app_env: str = Field(default="dev", alias="APP_ENV")
|
||||||
|
|
||||||
|
database_url: str = Field(
|
||||||
|
default="sqlite:///./multimailer-dev.db",
|
||||||
|
alias="DATABASE_URL",
|
||||||
|
)
|
||||||
|
redis_url: str = Field(default="redis://redis:6379/0", alias="REDIS_URL")
|
||||||
|
|
||||||
|
s3_endpoint_url: str = Field(default="http://garage:3900", alias="S3_ENDPOINT_URL")
|
||||||
|
s3_region: str = Field(default="garage", alias="S3_REGION")
|
||||||
|
s3_access_key_id: str = Field(default="GKmultimailerdev0000000000000000", alias="S3_ACCESS_KEY_ID")
|
||||||
|
s3_secret_access_key: str = Field(default="multimailer-dev-secret-change-me", alias="S3_SECRET_ACCESS_KEY")
|
||||||
|
s3_bucket: str = Field(default="attachments", alias="S3_BUCKET")
|
||||||
|
|
||||||
|
master_key_b64: str | None = Field(default=None, alias="MASTER_KEY_B64")
|
||||||
|
celery_queues: str = Field(default="send_email,append_sent,default", alias="CELERY_QUEUES")
|
||||||
|
|
||||||
|
# Development bootstrap only. Do not use this in production.
|
||||||
|
dev_bootstrap_api_key: str | None = Field(default="dev-multimailer-api-key", alias="DEV_BOOTSTRAP_API_KEY")
|
||||||
|
dev_bootstrap_enabled: bool = Field(default=True, alias="DEV_BOOTSTRAP_ENABLED")
|
||||||
|
|
||||||
|
# Comma-separated list. Use * only for local development.
|
||||||
|
cors_origins: str = Field(default="http://localhost:5173,http://127.0.0.1:5173,http://localhost:8080", alias="CORS_ORIGINS")
|
||||||
|
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
21
server/entrypoint.sh
Normal file
21
server/entrypoint.sh
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
set -e
|
||||||
|
|
||||||
|
ROLE="${APP_ROLE:-api}"
|
||||||
|
|
||||||
|
if [ "$ROLE" = "api" ]; then
|
||||||
|
exec uvicorn app.main:app \
|
||||||
|
--host "${APP_HOST:-0.0.0.0}" \
|
||||||
|
--port "${APP_PORT:-8000}" \
|
||||||
|
--proxy-headers
|
||||||
|
elif [ "$ROLE" = "worker" ]; then
|
||||||
|
exec celery -A app.celery_app.celery worker \
|
||||||
|
--loglevel="${CELERY_LOGLEVEL:-INFO}" \
|
||||||
|
--queues="${CELERY_QUEUES:-send_email,append_sent,default}" \
|
||||||
|
--concurrency="${CELERY_CONCURRENCY:-4}" \
|
||||||
|
--prefetch-multiplier="${CELERY_PREFETCH_MULTIPLIER:-1}" \
|
||||||
|
--max-tasks-per-child="${CELERY_MAX_TASKS_PER_CHILD:-200}"
|
||||||
|
else
|
||||||
|
echo "Unknown APP_ROLE=$ROLE (expected api|worker)"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
BIN
server/multimailer-dev.db
Normal file
BIN
server/multimailer-dev.db
Normal file
Binary file not shown.
20
server/requirements.txt
Normal file
20
server/requirements.txt
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
fastapi==0.115.6
|
||||||
|
uvicorn[standard]==0.32.1
|
||||||
|
|
||||||
|
celery==5.4.0
|
||||||
|
redis==5.2.1
|
||||||
|
|
||||||
|
pydantic==2.10.3
|
||||||
|
pydantic-settings==2.6.1
|
||||||
|
|
||||||
|
SQLAlchemy==2.0.36
|
||||||
|
psycopg[binary]==3.2.3
|
||||||
|
alembic==1.14.0
|
||||||
|
|
||||||
|
boto3==1.35.68
|
||||||
|
python-multipart==0.0.17
|
||||||
|
|
||||||
|
cryptography==44.0.0
|
||||||
|
pyzipper==0.3.6
|
||||||
|
|
||||||
|
jsonschema==4.23.0
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "rechnungslegung-2026-05",
|
||||||
|
"name": "Rechnungslegung 2026-05",
|
||||||
|
"mode": "draft",
|
||||||
|
"description": "Example campaign migrated from the Java object model."
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "monthyear",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Month/year",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "number",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Dienststelle",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "anrede",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Salutation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "zip_password",
|
||||||
|
"type": "password",
|
||||||
|
"label": "ZIP password",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"global_values": {
|
||||||
|
"monthyear": "05 / 2026"
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "smtp.example.org",
|
||||||
|
"port": 587,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "starttls"
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": true,
|
||||||
|
"host": "imap.example.org",
|
||||||
|
"port": 993,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "to"
|
||||||
|
},
|
||||||
|
"allow_individual_from": false,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": true,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": false,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": false,
|
||||||
|
"reply_to": [
|
||||||
|
{
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "reply_to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_reply_to": false,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": false,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": false
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}",
|
||||||
|
"text": "${local::anrede},\n\nbeigefügt erhalten Sie die Rechnungslegungslisten für ${global::monthyear}.\n\nMit freundlichen Grüßen"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": "/mnt/data/mm_next/server/app/mailer/examples/data/attachments",
|
||||||
|
"allow_individual": true,
|
||||||
|
"send_without_attachments": false,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask"
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"source": {
|
||||||
|
"type": "csv",
|
||||||
|
"path": "/mnt/data/mm_next/server/app/mailer/examples/data/recipients.csv",
|
||||||
|
"delimiter": ";",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"has_header": true
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"id": "ID",
|
||||||
|
"active": "Aktiv",
|
||||||
|
"to.0.email": "E-Mail",
|
||||||
|
"to.0.name": "Name",
|
||||||
|
"fields.number": "Dienststelle",
|
||||||
|
"fields.anrede": "Anrede",
|
||||||
|
"fields.zip_password": "ZIP-Passwort",
|
||||||
|
"attachments.0.base_dir": "Unterordner",
|
||||||
|
"attachments.0.file_filter": "Dateimuster"
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"active": true,
|
||||||
|
"to": [],
|
||||||
|
"combine_to": false,
|
||||||
|
"cc": [],
|
||||||
|
"combine_cc": true,
|
||||||
|
"bcc": [],
|
||||||
|
"combine_bcc": true,
|
||||||
|
"reply_to": [],
|
||||||
|
"combine_reply_to": true,
|
||||||
|
"bounce_to": [],
|
||||||
|
"combine_bounce_to": true,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"combine_disposition_notification_to": true,
|
||||||
|
"attachments": [
|
||||||
|
{
|
||||||
|
"id": "individual-documents",
|
||||||
|
"label": "Personalized PDF bundle",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "${local::number}_*.pdf",
|
||||||
|
"include_subdirs": false,
|
||||||
|
"required": true,
|
||||||
|
"allow_multiple": true,
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "continue",
|
||||||
|
"zip": {
|
||||||
|
"enabled": true,
|
||||||
|
"filename_template": "Rechnungslegung_${local::number}.zip",
|
||||||
|
"password_template": "${local::zip_password}",
|
||||||
|
"method": "aes"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"combine_attachments": true,
|
||||||
|
"fields": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"inactive_entry": "drop"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "auto"
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true,
|
||||||
|
"initial_build_status": "built",
|
||||||
|
"initial_send_status": "draft"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "rechnungslegung-2026-05",
|
||||||
|
"name": "Rechnungslegung 2026-05",
|
||||||
|
"mode": "draft",
|
||||||
|
"description": "Example campaign migrated from the Java object model."
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "monthyear",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Month/year",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "number",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Dienststelle",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "anrede",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Salutation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "zip_password",
|
||||||
|
"type": "password",
|
||||||
|
"label": "ZIP password",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"global_values": {
|
||||||
|
"monthyear": "05 / 2026"
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "smtp.example.org",
|
||||||
|
"port": 587,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "starttls"
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": true,
|
||||||
|
"host": "imap.example.org",
|
||||||
|
"port": 993,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "to"
|
||||||
|
},
|
||||||
|
"allow_individual_from": false,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": true,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": false,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": false,
|
||||||
|
"reply_to": [
|
||||||
|
{
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "reply_to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_reply_to": false,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": false,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": false
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}",
|
||||||
|
"text": "${local::anrede},\n\nbeigefügt erhalten Sie die Rechnungslegungslisten für ${global::monthyear}.\n\nMit freundlichen Grüßen"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": "/mnt/data/mm_next/server/app/mailer/examples/data/attachments",
|
||||||
|
"allow_individual": true,
|
||||||
|
"send_without_attachments": false,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask"
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"source": {
|
||||||
|
"type": "csv",
|
||||||
|
"path": "/mnt/data/mm_next/server/app/mailer/examples/data/recipients.csv",
|
||||||
|
"delimiter": ";",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"has_header": true
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"id": "ID",
|
||||||
|
"active": "Aktiv",
|
||||||
|
"to.0.email": "E-Mail",
|
||||||
|
"to.0.name": "Name",
|
||||||
|
"fields.number": "Dienststelle",
|
||||||
|
"fields.anrede": "Anrede",
|
||||||
|
"fields.zip_password": "ZIP-Passwort",
|
||||||
|
"attachments.0.base_dir": "Unterordner",
|
||||||
|
"attachments.0.file_filter": "Dateimuster"
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"active": true,
|
||||||
|
"to": [],
|
||||||
|
"combine_to": false,
|
||||||
|
"cc": [],
|
||||||
|
"combine_cc": true,
|
||||||
|
"bcc": [],
|
||||||
|
"combine_bcc": true,
|
||||||
|
"reply_to": [],
|
||||||
|
"combine_reply_to": true,
|
||||||
|
"bounce_to": [],
|
||||||
|
"combine_bounce_to": true,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"combine_disposition_notification_to": true,
|
||||||
|
"attachments": [
|
||||||
|
{
|
||||||
|
"id": "individual-documents",
|
||||||
|
"label": "Personalized PDF bundle",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "${local::number}_*.pdf",
|
||||||
|
"include_subdirs": false,
|
||||||
|
"required": true,
|
||||||
|
"allow_multiple": true,
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "continue",
|
||||||
|
"zip": {
|
||||||
|
"enabled": true,
|
||||||
|
"filename_template": "Rechnungslegung_${local::number}.zip",
|
||||||
|
"password_template": "${local::zip_password}",
|
||||||
|
"method": "aes"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"combine_attachments": true,
|
||||||
|
"fields": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"inactive_entry": "drop"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "auto"
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true,
|
||||||
|
"initial_build_status": "built",
|
||||||
|
"initial_send_status": "draft"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "rechnungslegung-2026-05",
|
||||||
|
"name": "Rechnungslegung 2026-05",
|
||||||
|
"mode": "draft",
|
||||||
|
"description": "Example campaign migrated from the Java object model."
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "monthyear",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Month/year",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "number",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Dienststelle",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "anrede",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Salutation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "zip_password",
|
||||||
|
"type": "password",
|
||||||
|
"label": "ZIP password",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"global_values": {
|
||||||
|
"monthyear": "05 / 2026"
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "smtp.example.org",
|
||||||
|
"port": 587,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "starttls"
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": true,
|
||||||
|
"host": "imap.example.org",
|
||||||
|
"port": 993,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "to"
|
||||||
|
},
|
||||||
|
"allow_individual_from": false,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": true,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": false,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": false,
|
||||||
|
"reply_to": [
|
||||||
|
{
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "reply_to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_reply_to": false,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": false,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": false
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}",
|
||||||
|
"text": "${local::anrede},\n\nbeigefügt erhalten Sie die Rechnungslegungslisten für ${global::monthyear}.\n\nMit freundlichen Grüßen"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": "/mnt/DATA/git/multi-seal-mail/server/app/mailer/examples/data/attachments",
|
||||||
|
"allow_individual": true,
|
||||||
|
"send_without_attachments": false,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask"
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"source": {
|
||||||
|
"type": "csv",
|
||||||
|
"path": "/mnt/DATA/git/multi-seal-mail/server/app/mailer/examples/data/recipients.csv",
|
||||||
|
"delimiter": ";",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"has_header": true
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"id": "ID",
|
||||||
|
"active": "Aktiv",
|
||||||
|
"to.0.email": "E-Mail",
|
||||||
|
"to.0.name": "Name",
|
||||||
|
"fields.number": "Dienststelle",
|
||||||
|
"fields.anrede": "Anrede",
|
||||||
|
"fields.zip_password": "ZIP-Passwort",
|
||||||
|
"attachments.0.base_dir": "Unterordner",
|
||||||
|
"attachments.0.file_filter": "Dateimuster"
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"active": true,
|
||||||
|
"to": [],
|
||||||
|
"combine_to": false,
|
||||||
|
"cc": [],
|
||||||
|
"combine_cc": true,
|
||||||
|
"bcc": [],
|
||||||
|
"combine_bcc": true,
|
||||||
|
"reply_to": [],
|
||||||
|
"combine_reply_to": true,
|
||||||
|
"bounce_to": [],
|
||||||
|
"combine_bounce_to": true,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"combine_disposition_notification_to": true,
|
||||||
|
"attachments": [
|
||||||
|
{
|
||||||
|
"id": "individual-documents",
|
||||||
|
"label": "Personalized PDF bundle",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "${local::number}_*.pdf",
|
||||||
|
"include_subdirs": false,
|
||||||
|
"required": true,
|
||||||
|
"allow_multiple": true,
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "continue",
|
||||||
|
"zip": {
|
||||||
|
"enabled": true,
|
||||||
|
"filename_template": "Rechnungslegung_${local::number}.zip",
|
||||||
|
"password_template": "${local::zip_password}",
|
||||||
|
"method": "aes"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"combine_attachments": true,
|
||||||
|
"fields": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"inactive_entry": "drop"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "auto"
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true,
|
||||||
|
"initial_build_status": "built",
|
||||||
|
"initial_send_status": "draft"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"campaign": {
|
||||||
|
"id": "rechnungslegung-2026-05",
|
||||||
|
"name": "Rechnungslegung 2026-05",
|
||||||
|
"mode": "draft",
|
||||||
|
"description": "Example campaign migrated from the Java object model."
|
||||||
|
},
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "monthyear",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Month/year",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "number",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Dienststelle",
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "anrede",
|
||||||
|
"type": "string",
|
||||||
|
"label": "Salutation"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "zip_password",
|
||||||
|
"type": "password",
|
||||||
|
"label": "ZIP password",
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"global_values": {
|
||||||
|
"monthyear": "05 / 2026"
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"smtp": {
|
||||||
|
"host": "smtp.example.org",
|
||||||
|
"port": 587,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "starttls"
|
||||||
|
},
|
||||||
|
"imap": {
|
||||||
|
"enabled": true,
|
||||||
|
"host": "imap.example.org",
|
||||||
|
"port": 993,
|
||||||
|
"username": "user@example.org",
|
||||||
|
"password": "CHANGE_ME_OR_REFERENCE_SECRET",
|
||||||
|
"security": "tls",
|
||||||
|
"sent_folder": "auto"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"recipients": {
|
||||||
|
"from": {
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "to"
|
||||||
|
},
|
||||||
|
"allow_individual_from": false,
|
||||||
|
"to": [],
|
||||||
|
"allow_individual_to": true,
|
||||||
|
"cc": [],
|
||||||
|
"allow_individual_cc": false,
|
||||||
|
"bcc": [],
|
||||||
|
"allow_individual_bcc": false,
|
||||||
|
"reply_to": [
|
||||||
|
{
|
||||||
|
"name": "Rechnungslegung D5",
|
||||||
|
"email": "d5-rechnungslegung@example.org",
|
||||||
|
"type": "reply_to"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"allow_individual_reply_to": false,
|
||||||
|
"bounce_to": [],
|
||||||
|
"allow_individual_bounce_to": false,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"allow_individual_disposition_notification_to": false
|
||||||
|
},
|
||||||
|
"template": {
|
||||||
|
"subject": "Rechnungslegungslisten für ${global::monthyear} und Dienststelle ${local::number}",
|
||||||
|
"text": "${local::anrede},\n\nbeigefügt erhalten Sie die Rechnungslegungslisten für ${global::monthyear}.\n\nMit freundlichen Grüßen"
|
||||||
|
},
|
||||||
|
"attachments": {
|
||||||
|
"base_path": "./data/attachments",
|
||||||
|
"allow_individual": true,
|
||||||
|
"send_without_attachments": false,
|
||||||
|
"global": [],
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "ask"
|
||||||
|
},
|
||||||
|
"entries": {
|
||||||
|
"source": {
|
||||||
|
"type": "csv",
|
||||||
|
"path": "./data/recipients.csv",
|
||||||
|
"delimiter": ";",
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"has_header": true
|
||||||
|
},
|
||||||
|
"mapping": {
|
||||||
|
"id": "ID",
|
||||||
|
"active": "Aktiv",
|
||||||
|
"to.0.email": "E-Mail",
|
||||||
|
"to.0.name": "Name",
|
||||||
|
"fields.number": "Dienststelle",
|
||||||
|
"fields.anrede": "Anrede",
|
||||||
|
"fields.zip_password": "ZIP-Passwort",
|
||||||
|
"attachments.0.base_dir": "Unterordner",
|
||||||
|
"attachments.0.file_filter": "Dateimuster"
|
||||||
|
},
|
||||||
|
"defaults": {
|
||||||
|
"active": true,
|
||||||
|
"to": [],
|
||||||
|
"combine_to": false,
|
||||||
|
"cc": [],
|
||||||
|
"combine_cc": true,
|
||||||
|
"bcc": [],
|
||||||
|
"combine_bcc": true,
|
||||||
|
"reply_to": [],
|
||||||
|
"combine_reply_to": true,
|
||||||
|
"bounce_to": [],
|
||||||
|
"combine_bounce_to": true,
|
||||||
|
"disposition_notification_to": [],
|
||||||
|
"combine_disposition_notification_to": true,
|
||||||
|
"attachments": [
|
||||||
|
{
|
||||||
|
"id": "individual-documents",
|
||||||
|
"label": "Personalized PDF bundle",
|
||||||
|
"base_dir": ".",
|
||||||
|
"file_filter": "${local::number}_*.pdf",
|
||||||
|
"include_subdirs": false,
|
||||||
|
"required": true,
|
||||||
|
"allow_multiple": true,
|
||||||
|
"missing_behavior": "ask",
|
||||||
|
"ambiguous_behavior": "continue",
|
||||||
|
"zip": {
|
||||||
|
"enabled": true,
|
||||||
|
"filename_template": "Rechnungslegung_${local::number}.zip",
|
||||||
|
"password_template": "${local::zip_password}",
|
||||||
|
"method": "aes"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"combine_attachments": true,
|
||||||
|
"fields": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"validation_policy": {
|
||||||
|
"missing_required_attachment": "ask",
|
||||||
|
"missing_optional_attachment": "warn",
|
||||||
|
"ambiguous_attachment_match": "ask",
|
||||||
|
"missing_email": "block",
|
||||||
|
"template_error": "block",
|
||||||
|
"inactive_entry": "drop"
|
||||||
|
},
|
||||||
|
"delivery": {
|
||||||
|
"rate_limit": {
|
||||||
|
"messages_per_minute": 5,
|
||||||
|
"concurrency": 1
|
||||||
|
},
|
||||||
|
"imap_append_sent": {
|
||||||
|
"enabled": true,
|
||||||
|
"folder": "auto"
|
||||||
|
},
|
||||||
|
"retry": {
|
||||||
|
"max_attempts": 3,
|
||||||
|
"backoff_seconds": [
|
||||||
|
60,
|
||||||
|
300,
|
||||||
|
900
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"status_tracking": {
|
||||||
|
"enabled": true,
|
||||||
|
"initial_build_status": "built",
|
||||||
|
"initial_send_status": "draft"
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user