User story: guided monthly data workflow from incoming files to campaign send #8

New Issue

zemion · 2026-07-14T14:01:26+02:00

zemion commented

2026-07-14 14:01:26 +02:00

User Story

As an operator responsible for a recurring monthly data process, I want GovOPlaN to guide me through receiving structured files, importing and comparing data, correcting and rerunning processing steps, and finally producing and sending campaign output, so that the process becomes traceable, resumable, configurable, and less error-prone.

Real Process Context

Monthly files arrive with structured information. The current process has three major branches:

Part of the structured information must be imported into an existing database.
Another part must be processed and compared against data from a second database. The comparison needs an interactive UI for review/decisions and an Excel export of the comparison result.
Another part must be processed and linked with existing database records. This is currently a run -> verify -> correct -> rerun loop. The final result is a large bulk load of files that is sent through the campaign module.

The preferred path is direct database access from GovOPlaN. A fallback path should support static exports/imports when direct access is not possible.

Product Shape

The target experience is an interactive workflow that can be configured and then executed by users. It should guide operators through the case-specific process, preserve progress, allow leaving and resuming, surface open workflow tasks in a reminder/widget, and hand off to other modules when needed.

For example, a workflow step may create or prepare the necessary campaign configuration, route the user to the campaign module, frame the campaign work as part of the active workflow, wait for successful sending, and then continue the workflow.

Module Touchpoints

Likely modules involved:

module/workflow: owns workflow definition, execution state, resumability, task framing, and cross-module orchestration.
module/files: receives monthly files, stores source artifacts, generated artifacts, exports, and bulk output files.
module/connectors: direct database connections and external source connectors where feasible.
module/campaign: campaign setup, generated recipients/assets, send handoff, send status callback.
module/reporting: comparison views and exportable result output, especially Excel-style exports.
module/notifications: reminders for open workflow tasks and completed async steps.
module/tasks: optional personal/team task projection for workflow steps that require human action.

Potential missing or future module concepts:

datasource: database/static-export source definitions, credentials, schema mapping, import contracts.
dataflow: reusable transformation/comparison/linking pipelines and step outputs.

These may be separate modules later, or initially capabilities owned by workflow/connectors/reporting until boundaries become clearer.

Acceptance Criteria

A workflow can be configured as a sequence of typed steps, including file input, database/static-source input, processing, comparison, manual review, export, campaign preparation, campaign send handoff, verification, and completion.
A workflow execution has durable state and can be left and resumed later.
A user can see open workflow tasks and return to the active execution context from a widget or reminder surface.
A workflow step can call module capabilities without hard module-to-module imports, using manifest/capability contracts.
A comparison step can present interactive review decisions and export results to Excel-compatible output.
A processing/linking step can support a run -> verify -> correct -> rerun loop with source/version/provenance tracking.
A campaign handoff step can create or prepare campaign configuration, guide the user into campaign, and wait for a successful send/result before continuing.
Direct database querying is supported when configured; static exports remain a fallback path.
Every imported/generated artifact is traceable back to input files, source queries/exports, workflow execution, and user decisions.

Open Decisions

Whether datasource/dataflow should become first-class modules immediately or start as workflow/connectors/reporting capabilities.
How much of the workflow step UI is generic versus owned by each participating module.
How power-user configuration should be represented: declarative workflow definitions, low-code UI, code-backed plugins, or a hybrid.
Which database types and static export formats should be supported first.
How to model provenance, reruns, corrections, and auditability across files, workflow, and campaign.

First Vertical Slice Candidate

A pragmatic first slice could be:

Upload or select a monthly source file from files.
Configure a small structured import/mapping step.
Run a comparison against a static second source.
Show an interactive comparison table with accept/reject/correct decisions.
Export the comparison result to Excel.
Persist workflow state and show the open workflow in a reminder/task surface.

Campaign handoff can be the second slice once the state model and step UI are proven.

## User Story As an operator responsible for a recurring monthly data process, I want GovOPlaN to guide me through receiving structured files, importing and comparing data, correcting and rerunning processing steps, and finally producing and sending campaign output, so that the process becomes traceable, resumable, configurable, and less error-prone. ## Real Process Context Monthly files arrive with structured information. The current process has three major branches: 1. Part of the structured information must be imported into an existing database. 2. Another part must be processed and compared against data from a second database. The comparison needs an interactive UI for review/decisions and an Excel export of the comparison result. 3. Another part must be processed and linked with existing database records. This is currently a run -> verify -> correct -> rerun loop. The final result is a large bulk load of files that is sent through the campaign module. The preferred path is direct database access from GovOPlaN. A fallback path should support static exports/imports when direct access is not possible. ## Product Shape The target experience is an interactive workflow that can be configured and then executed by users. It should guide operators through the case-specific process, preserve progress, allow leaving and resuming, surface open workflow tasks in a reminder/widget, and hand off to other modules when needed. For example, a workflow step may create or prepare the necessary campaign configuration, route the user to the campaign module, frame the campaign work as part of the active workflow, wait for successful sending, and then continue the workflow. ## Module Touchpoints Likely modules involved: - `module/workflow`: owns workflow definition, execution state, resumability, task framing, and cross-module orchestration. - `module/files`: receives monthly files, stores source artifacts, generated artifacts, exports, and bulk output files. - `module/connectors`: direct database connections and external source connectors where feasible. - `module/campaign`: campaign setup, generated recipients/assets, send handoff, send status callback. - `module/reporting`: comparison views and exportable result output, especially Excel-style exports. - `module/notifications`: reminders for open workflow tasks and completed async steps. - `module/tasks`: optional personal/team task projection for workflow steps that require human action. Potential missing or future module concepts: - datasource: database/static-export source definitions, credentials, schema mapping, import contracts. - dataflow: reusable transformation/comparison/linking pipelines and step outputs. These may be separate modules later, or initially capabilities owned by workflow/connectors/reporting until boundaries become clearer. ## Acceptance Criteria - A workflow can be configured as a sequence of typed steps, including file input, database/static-source input, processing, comparison, manual review, export, campaign preparation, campaign send handoff, verification, and completion. - A workflow execution has durable state and can be left and resumed later. - A user can see open workflow tasks and return to the active execution context from a widget or reminder surface. - A workflow step can call module capabilities without hard module-to-module imports, using manifest/capability contracts. - A comparison step can present interactive review decisions and export results to Excel-compatible output. - A processing/linking step can support a run -> verify -> correct -> rerun loop with source/version/provenance tracking. - A campaign handoff step can create or prepare campaign configuration, guide the user into campaign, and wait for a successful send/result before continuing. - Direct database querying is supported when configured; static exports remain a fallback path. - Every imported/generated artifact is traceable back to input files, source queries/exports, workflow execution, and user decisions. ## Open Decisions - Whether datasource/dataflow should become first-class modules immediately or start as workflow/connectors/reporting capabilities. - How much of the workflow step UI is generic versus owned by each participating module. - How power-user configuration should be represented: declarative workflow definitions, low-code UI, code-backed plugins, or a hybrid. - Which database types and static export formats should be supported first. - How to model provenance, reruns, corrections, and auditability across files, workflow, and campaign. ## First Vertical Slice Candidate A pragmatic first slice could be: 1. Upload or select a monthly source file from files. 2. Configure a small structured import/mapping step. 3. Run a comparison against a static second source. 4. Show an interactive comparison table with accept/reject/correct decisions. 5. Export the comparison result to Excel. 6. Persist workflow state and show the open workflow in a reminder/task surface. Campaign handoff can be the second slice once the state model and step UI are proven.

zemion added the