feat: extract headless workflow engine
This commit is contained in:
@@ -0,0 +1,84 @@
|
||||
# Native BPMN Graph
|
||||
|
||||
GovOPlaN uses BPMN 2.0 as Workflow's canonical graph language while keeping
|
||||
notation support distinct from executable runtime support.
|
||||
|
||||
## Current Contract
|
||||
|
||||
- The native Workflow graph stores BPMN element and flow types, process
|
||||
membership, containment, geometry, properties, and preserved extension
|
||||
content. There is one editor and one graph representation.
|
||||
- BPMN XML import maps standard elements and BPMN DI into the native graph.
|
||||
Export deterministically renders XML and DI from the current graph. The
|
||||
normalized XML artifact, its hash, and the native profile version are pinned
|
||||
with every immutable revision.
|
||||
- No browser-side BPMN modeler is required. The WebUI uses the same graph
|
||||
surface and shared controls as the rest of GovOPlaN.
|
||||
- `GET /api/v1/workflow/bpmn/profile` publishes all installed, versioned
|
||||
conformance profiles.
|
||||
- `POST /api/v1/workflow/bpmn/inspect` safely parses bounded BPMN 2.0 XML,
|
||||
inventories every BPMN model element, detects duplicate IDs and selected
|
||||
dangling references, and classifies elements as interchange-only, natively
|
||||
mappable, or natively executable.
|
||||
- `POST /api/v1/workflow/bpmn/compile` imports a bounded BPMN document into the
|
||||
canonical native graph.
|
||||
- `POST /api/v1/workflow/bpmn/render` exports a native graph as normalized BPMN
|
||||
XML with BPMN DI geometry.
|
||||
- `GET /api/v1/workflow/definitions/{id}/revisions/{revision}/bpmn` returns
|
||||
the exact pinned document and its current availability/conformance
|
||||
assessment.
|
||||
- XML entities, DTD-based expansion, oversized documents, and malformed roots
|
||||
are rejected.
|
||||
|
||||
Inspection is not XML Schema validation and does not claim that every editable
|
||||
BPMN construct can be executed. Notation and interchange remain available when
|
||||
the native runtime cannot activate the document.
|
||||
|
||||
## Built-In Profiles
|
||||
|
||||
- `govoplan.native.bpmn@1.0.0` is the canonical graph and interchange profile.
|
||||
It maps the supported BPMN vocabulary into native nodes and edges. Activation
|
||||
separately validates whether every execution semantic is implemented.
|
||||
- `govoplan.native.linear@1.0.0` and `bpmn.interchange@1.0.0` remain registered
|
||||
for historical revision compatibility; new editor revisions use the native
|
||||
BPMN profile.
|
||||
|
||||
Gateways, subprocesses, event definitions, transactions, compensation,
|
||||
collaboration, and choreography remain editable and exportable even when their
|
||||
token or lifecycle semantics are not yet implemented.
|
||||
|
||||
## Execution Boundary
|
||||
|
||||
Adding a BPMN shape is not equivalent to implementing its token semantics,
|
||||
event subscriptions, compensation, transactions, choreography, or conformance
|
||||
behavior. Each executable mapping therefore needs:
|
||||
|
||||
1. an explicit native semantic mapping;
|
||||
2. validation rules and lifecycle behavior;
|
||||
3. resumability and idempotency tests;
|
||||
4. migration and round-trip fixtures;
|
||||
5. a declared fallback when the installed runtime cannot execute it.
|
||||
|
||||
Unsupported execution constructs remain visible in the native graph, but
|
||||
activation remains blocked until an execution adapter declares support.
|
||||
|
||||
## Adapter Boundary
|
||||
|
||||
Adapter packages register through the
|
||||
`govoplan.workflow.bpmn_adapters` Python entry-point group. Workflow discovers
|
||||
them without importing a concrete module. An adapter publishes a stable ID,
|
||||
version, runtime kind, conformance statement, supported elements and event
|
||||
definitions, operational requirements, validation, and canonical graph
|
||||
materialization.
|
||||
|
||||
Revisions pin the exact adapter version. If that version is unavailable after
|
||||
an installation change, the document remains readable and exportable but
|
||||
cannot activate. External-engine adapters must still materialize lifecycle,
|
||||
handoff, retry, cancellation, and audit evidence through the canonical
|
||||
Workflow instance contract; a remote engine's private state is not the
|
||||
platform record.
|
||||
|
||||
The conformance fixtures under `tests/fixtures/bpmn` cover processes,
|
||||
collaboration, choreography, events, transactions, compensation, and data
|
||||
elements. Every fixture must import and export through the native graph without
|
||||
losing modeled nodes or flows; activation has its own narrower test matrix.
|
||||
+201
@@ -0,0 +1,201 @@
|
||||
# govoplan-workflow Concept
|
||||
|
||||
## Purpose
|
||||
|
||||
`govoplan-workflow-engine` is the headless process orchestration module.
|
||||
`govoplan-workflow` is its optional authoring and inspection surface. See
|
||||
`ENGINE_EDITOR_SPLIT.md`.
|
||||
|
||||
Workflow does not own business records. A case, task, file, appointment,
|
||||
template, payment, or postbox message remains owned by its domain module.
|
||||
Workflow coordinates those modules through stable contracts.
|
||||
|
||||
Workflow is also the first home for GovOPlaN's action/effect automation layer.
|
||||
That layer must keep automated actions governed, previewable, idempotent,
|
||||
auditable, and recoverable. If action catalogues, schedules, rule execution, or
|
||||
cross-module automation grow beyond workflow ownership, the runner can later be
|
||||
split into a dedicated `govoplan-automation` module without changing the
|
||||
action/effect contracts.
|
||||
|
||||
## Ownership
|
||||
|
||||
Workflow Engine owns:
|
||||
|
||||
- workflow definitions and versions
|
||||
- workflow instances and current state
|
||||
- transitions, guards, and transition history
|
||||
- timers, deadlines, and wait states that belong to process execution
|
||||
- command plans and command execution records
|
||||
- retry/manual-intervention state for failed command handoffs
|
||||
- action/effect execution records for workflow-triggered automation
|
||||
- workflow audit/event emission
|
||||
- workflow diagram metadata
|
||||
|
||||
Workflow owns visual authoring, catalogue, comparison, activation, inspection,
|
||||
override, and reset surfaces. It owns no process tables or transition runtime.
|
||||
|
||||
The module does not own:
|
||||
|
||||
- case records and case evidence
|
||||
- task queues and task completion semantics
|
||||
- form schemas or submissions
|
||||
- file storage, documents, mail, notifications, appointments, payments, ledgers,
|
||||
or records
|
||||
- external protocol adapters
|
||||
|
||||
## Workflow Model
|
||||
|
||||
A workflow definition should contain:
|
||||
|
||||
- definition id, version, tenant scope, status
|
||||
- states with labels, categories, and terminal markers
|
||||
- transitions with from/to states, required scopes, guards, and commands
|
||||
- input/output data schema references
|
||||
- timers and escalation rules
|
||||
- extension metadata for diagrams and operator UI
|
||||
|
||||
Instances should contain:
|
||||
|
||||
- instance id, tenant id, definition id/version
|
||||
- subject references such as `case_id` or `submission_id`
|
||||
- current state and previous state
|
||||
- process variables with strict redaction rules
|
||||
- transition history
|
||||
- pending commands and manual actions
|
||||
|
||||
## Core Contracts
|
||||
|
||||
The module should integrate through:
|
||||
|
||||
- module manifest metadata, route factories, permissions, and migrations
|
||||
- events such as `workflow.instance_started`, `workflow.transitioned`,
|
||||
`workflow.command_requested`, `workflow.command_failed`, and
|
||||
`workflow.instance_completed`
|
||||
- commands such as `workflow.start`, `workflow.transition`,
|
||||
`workflow.retry_command`, and `workflow.cancel`
|
||||
- capability lookups for domain commands, for example cases, tasks, templates,
|
||||
appointments, and payments
|
||||
- configuration-package fragments that install workflow definitions
|
||||
|
||||
Command handoff must be explicit. A transition should record which module
|
||||
capability was requested, with input payload, result summary, and failure reason.
|
||||
|
||||
The shared action/effect doctrine lives in
|
||||
`govoplan-core/docs/ACTION_EFFECT_AUTOMATION_LAYER.md`.
|
||||
|
||||
## Reference Journey
|
||||
|
||||
Permit-to-payment MVP:
|
||||
|
||||
1. A form submission starts a workflow instance.
|
||||
2. Workflow commands cases to create a case.
|
||||
3. Workflow commands tasks to create an intake task.
|
||||
4. Completion of the task transitions the instance to appointment proposal.
|
||||
5. Appointment acceptance transitions to review/decision.
|
||||
6. Workflow commands templates to generate a permit or decision.
|
||||
7. Workflow commands payments/ledger handoff.
|
||||
8. Workflow closes the case and emits evidence events.
|
||||
|
||||
## MVP Slice
|
||||
|
||||
The first executable slice now provides:
|
||||
|
||||
- a versioned, workflow-specific graphical node library
|
||||
- shared Core graph DTO and validation primitives also consumed by Dataflow
|
||||
- workflow constraints that permit loops while requiring one trigger and one or
|
||||
more outcomes
|
||||
- trigger, activity, review, decision, wait, module-action, Dataflow, and outcome
|
||||
nodes
|
||||
- API discovery and validation endpoints
|
||||
- revision-pinned, idempotent Workflow instances
|
||||
- persisted steps and append-only transition evidence
|
||||
- durable Dataflow handoff, progress reconciliation, output references,
|
||||
retries, cancellation, and warning/review paths
|
||||
- manual activity, review, and wait handoffs with comments and evidence
|
||||
- a worker capability with current-authorization rechecks
|
||||
- an operator dialog for starting, inspecting, and advancing instances
|
||||
|
||||
The next execution slices should provide:
|
||||
|
||||
- static workflow definition registration from configuration packages
|
||||
- event, API, schedule, and parent-workflow start dispatchers
|
||||
- guard hooks implemented through capability calls
|
||||
- registry-driven generic module-action execution records
|
||||
- action/effect previews for transitions that call other modules
|
||||
- explicit blocked, retryable, quarantined, manual-required, and
|
||||
compensation-required states
|
||||
- dashboard summary provider
|
||||
- event emission and audit integration
|
||||
|
||||
## Permissions
|
||||
|
||||
Candidate scopes:
|
||||
|
||||
- `workflow:definition:read`
|
||||
- `workflow:definition:write`
|
||||
- `workflow:instance:read`
|
||||
- `workflow:instance:start`
|
||||
- `workflow:instance:transition`
|
||||
- `workflow:instance:admin`
|
||||
|
||||
State transitions may require both workflow scopes and domain-module permission
|
||||
checks for the command being executed.
|
||||
|
||||
Workflow guard evaluation should consume access semantics through kernel
|
||||
capabilities instead of importing access internals:
|
||||
|
||||
- `access.semanticDirectory` resolves identity, account, organization unit,
|
||||
function assignment, delegation, and role facts.
|
||||
- `access.explanation` records why a transition was allowed or denied in terms
|
||||
of identity/account/function/role/right provenance.
|
||||
|
||||
Transitions that allow a person to act in place of another function holder must
|
||||
require an explicit acting context and must record both the real actor account
|
||||
and the represented account/function assignment in transition history and audit
|
||||
details.
|
||||
|
||||
## Data Model Sketch
|
||||
|
||||
Current tables:
|
||||
|
||||
- `workflow_definitions`
|
||||
- `workflow_definition_revisions`
|
||||
- `workflow_instances`
|
||||
- `workflow_instance_steps`
|
||||
- `workflow_instance_events`
|
||||
|
||||
Future generic action execution and timers may add:
|
||||
|
||||
- `workflow_command_records`
|
||||
- `workflow_timers`
|
||||
|
||||
Definitions should be immutable by version after activation. Instances should
|
||||
reference the exact version used at start.
|
||||
|
||||
## WebUI
|
||||
|
||||
Current route contribution:
|
||||
|
||||
- `/workflow`
|
||||
|
||||
The route combines the definition editor and a fixed run dialog showing current
|
||||
state, available transitions, failed handoffs, comments/evidence, immutable
|
||||
event history, and linked Dataflow results. It does not import Dataflow or other
|
||||
domain UI components.
|
||||
|
||||
## Tests
|
||||
|
||||
Minimum tests:
|
||||
|
||||
- core starts with workflow installed but cases/tasks/templates absent
|
||||
- workflow definition versioning is immutable after activation
|
||||
- transition guard denial is recorded and visible
|
||||
- command failure is retryable and does not partially advance state
|
||||
- events are emitted for start/transition/completion
|
||||
- configuration package can install a simple workflow definition
|
||||
|
||||
## Open Decisions
|
||||
|
||||
- Whether long-running timers use Celery beat, a module scheduler, or an ops
|
||||
scheduler abstraction.
|
||||
- How workflow variables are redacted and retained.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Workflow Engine And Workflow Editor Split
|
||||
|
||||
## Decision
|
||||
|
||||
Split the current module into two installable modules:
|
||||
|
||||
- `govoplan-workflow-engine` (runtime module ID `workflow_engine`) is the
|
||||
headless definition and execution platform available to all modules.
|
||||
- `govoplan-workflow` remains the optional authoring, inspection, catalogue,
|
||||
diff, override, and reset WebUI.
|
||||
|
||||
Other modules depend on Workflow Engine capabilities, never on the Workflow
|
||||
editor package. Workflow depends on Workflow Engine.
|
||||
|
||||
This is a packaging and ownership split, not a second workflow model. BPMN 2.0
|
||||
and the existing native graph remain the canonical language and use the same
|
||||
versioned contracts.
|
||||
|
||||
## Existing Versioning
|
||||
|
||||
Workflow definitions are already versioned:
|
||||
|
||||
- `workflow_definitions.current_revision` identifies the latest graph revision.
|
||||
- `workflow_definitions.active_revision` selects the revision used for new
|
||||
instances.
|
||||
- `workflow_definition_revisions` stores immutable graph content, hashes, node
|
||||
library versions, execution mode, pinned View revision, BPMN XML/hash, and
|
||||
execution-adapter profile/version.
|
||||
- Graph changes create a new revision and return an active definition to draft;
|
||||
metadata-only changes do not create graph revisions.
|
||||
- Updates use `expected_revision` optimistic concurrency.
|
||||
- Instances pin `definition_revision_id`; later edits or module upgrades cannot
|
||||
mutate running or historical instances.
|
||||
- Derived definitions record source definition, source revision/hash, actor,
|
||||
Policy decision, scope, and effective ancestor limits.
|
||||
|
||||
The missing model is a durable module-owned baseline and local override/reset
|
||||
relationship.
|
||||
|
||||
## Workflow Engine Ownership
|
||||
|
||||
Workflow Engine owns:
|
||||
|
||||
- definition, immutable revision, instance, step, event, command, timer, and
|
||||
execution-record persistence and migrations
|
||||
- definition CRUD/activation/derivation APIs and headless read APIs
|
||||
- Workflow graph/BPMN schemas, validation, import/export, and conformance
|
||||
- node-library and execution-adapter registries
|
||||
- instance start/transition/cancel/retry/reconciliation services
|
||||
- runtime worker, event/API/schedule/parent dispatch, idempotency, and recovery
|
||||
- definition governance capability integration and audit/event emission
|
||||
- configuration-package workflow fragments
|
||||
- module-contributed standard definition discovery and reconciliation
|
||||
|
||||
It contributes no primary navigation or full editor. A module can install and
|
||||
run its workflows when the editor is absent.
|
||||
|
||||
## Workflow Editor Ownership
|
||||
|
||||
Workflow owns:
|
||||
|
||||
- the Workflow workspace and visual BPMN/graph editor
|
||||
- definition/revision catalogue, preview, diff, validation, and activation UI
|
||||
- instance inspection and operator controls built on Engine APIs
|
||||
- derivation and governed override UX
|
||||
- module-standard update comparison and reset-to-standard UX
|
||||
- reusable embedded editor/inspector components for other module surfaces
|
||||
|
||||
The editor never owns workflow tables or executes transitions directly.
|
||||
|
||||
## Module-Contributed Definitions
|
||||
|
||||
Modules announce standard workflows through a versioned Engine contribution
|
||||
contract or a module-owned configuration-package fragment. A contribution has:
|
||||
|
||||
- origin module ID/version, stable definition key, contribution schema version,
|
||||
and content hash
|
||||
- graph plus BPMN representation, node-library/profile requirements, and
|
||||
execution mode
|
||||
- default scope, start/reuse/automation ceilings, required capabilities, and
|
||||
Policy metadata
|
||||
- upgrade compatibility and optional migration diagnostics
|
||||
|
||||
Engine reconciles contributions idempotently after module discovery. The
|
||||
module-provided baseline is immutable. A module update may add a new baseline
|
||||
revision, but it never rewrites a running instance or silently replaces a local
|
||||
override.
|
||||
|
||||
## Override And Reset
|
||||
|
||||
Editing a system/module standard creates a local override derived from a pinned
|
||||
baseline revision. The UI may present this as editing the effective definition,
|
||||
but the canonical baseline remains available.
|
||||
|
||||
- View is always possible when the caller can read the definition.
|
||||
- Edit/derive is controlled by Policy and scope ceilings.
|
||||
- An upstream baseline update is shown as an available update with a three-way
|
||||
diff; it is not merged silently.
|
||||
- Reset archives the local override and selects the latest permitted baseline.
|
||||
- Historical overrides, baselines, and instances remain addressable for audit.
|
||||
- A tenant/group/user override cannot loosen inherited restrictions.
|
||||
|
||||
## Compatibility And Extraction Order
|
||||
|
||||
1. Define Engine-owned capability and contribution DTOs in Core-neutral
|
||||
contracts while preserving existing `workflow.*` interface names.
|
||||
2. Create `govoplan-workflow-engine` and move backend code, tests, migrations,
|
||||
and runtime workers without changing table names or API paths.
|
||||
3. Transfer migration ownership without replaying the existing chain. Test both
|
||||
upgrades and fresh installs with Engine alone.
|
||||
4. Keep a compatibility facade in `govoplan-workflow` for one release line;
|
||||
make it depend on `workflow_engine` and retain only WebUI/editor code.
|
||||
5. Update Core workers and consuming modules to resolve Engine capabilities.
|
||||
6. Add module contributions, immutable baselines, override/update/reset, and
|
||||
configuration-package support.
|
||||
7. Remove compatibility imports only under the platform compatibility policy.
|
||||
|
||||
The extraction must preserve existing definition IDs, revision IDs, active
|
||||
revision selection, instance foreign keys, idempotency keys, API routes, and
|
||||
audit references.
|
||||
|
||||
## Implemented Boundary
|
||||
|
||||
The split is implemented in the `govoplan-workflow-engine` repository. Engine
|
||||
owns the unchanged migration chain and `/api/v1/workflow` API, retains the
|
||||
existing `workflow:*` permission namespace through an explicit manifest
|
||||
compatibility field, and exposes headless runtime and contribution
|
||||
capabilities. `govoplan-workflow` now has a hard dependency on runtime module
|
||||
ID `workflow_engine`, contributes only its WebUI/navigation/editor contract,
|
||||
and keeps one release line of `govoplan_workflow.backend` import facades.
|
||||
|
||||
Module manifests can announce versioned workflow baselines. Reconciliation is
|
||||
idempotent, records module/schema/hash provenance, keeps a newly supplied
|
||||
baseline revision inactive when an older revision is active, and fails closed
|
||||
when required capabilities or interfaces are absent. Baselines are immutable;
|
||||
editing derives a pinned local override, and reset archives that override
|
||||
without removing revision or instance history.
|
||||
Reference in New Issue
Block a user