feat: run dataflows through durable workers

README.md

@@ -91,11 +91,20 @@ manual-refresh preview modes.
 
 Saved revisions can also be started through the versioned
 `dataflow.runLifecycle` capability or the Run dialog. The first runner is
-synchronous and bounded. It records an idempotent terminal run, pins the
-pipeline revision, and can atomically publish a complete result through
-`datasources.publication`. Publication is rejected when a source or result was
-truncated; larger asynchronous and artifact-backed execution belongs to a
-subsequent runner slice.
+durable and worker-backed: API requests only enqueue idempotent work, while
+the `dataflow.runWorker` capability claims leased runs on the dedicated Celery
+queue. Access rechecks the persisted least-privilege authorization grant before
+each attempt. Runs expose progress, cancellation, bounded exponential retries,
+tenant queue/concurrency quotas, notifications, operational metrics, and
+evidence retention. Expired retention payloads are purged while hashes, row
+counts, outcomes, and publication references remain as audit evidence.
+
+Development runs may use the bounded reference backend. Staging and production
+runs require the short-lived isolated DuckDB process. A revision must be
+promoted from development to staging and then from staging to production before
+it can run in those environments. Complete results can be published atomically
+through `datasources.publication`; publication is rejected when a source or
+result was truncated.
 
 ## Governed Definitions And Automation
 
@@ -108,10 +117,11 @@ execution, reuse, inheritance, or automation permissions.
 
 Complete active flows support explicit user/API starts, administrative
 backfills, one-time schedules, interval schedules, and exact-match platform
-events. Trigger deliveries are durable and idempotent. They pin the pipeline
-revision and a least-privilege scope grant, then ask Access to rebuild the
-owner's current principal before every run. Revoked memberships or reduced
-permissions block the delivery before source access or output publication.
+events. Trigger deliveries are durable and idempotent. They enqueue the same
+worker-backed run contract, pin the pipeline revision and a least-privilege
+scope grant, then ask Access to rebuild the owner's current principal before
+both delivery and execution. Revoked memberships or reduced permissions block
+the run before source access or output publication.
 Confidential and restricted events are not accepted through the direct
 ingress; those require Core's transactional event bridge.