GovOPlaN/govoplan-reporting
2
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

[Feature] Add provider-neutral BI semantic model #4

New Issue
Closed
opened 2026-07-30 18:17:54 +02:00 by zemion · 3 comments
zemion commented 2026-07-30 18:17:54 +02:00
Owner
Copy Link
Copy Source

Context

Implement the semantic core defined by #3 on top of reproducible analytical datasets from #2. PostgreSQL is the first execution target, but definitions must remain provider-neutral.

Acceptance criteria

  • Define versioned analytical dataset, dimension, level/hierarchy, measure, calculated measure, format, total, parameter, and report-definition schemas.
  • Consume datasets through capabilities and opaque references; do not import Dataflow or Datasources internals.
  • Persist source revision/fingerprint, schema, freshness, lineage, quality state, effective Policy decision, and retention provenance.
  • Support system/tenant/group/user scope inheritance without broadening ancestor constraints.
  • Provide a PostgreSQL-backed query planner with bounded filters, grouping, measures, ordering, and result limits.
  • Add permission and row-level-policy hooks before planning and again before result delivery.
  • Add migration, API, and module-permutation tests.

Depends on #2. Architecture reference: docs/SUPERX_CAPABILITY_ASSESSMENT.md.

Architecture reconciliation (2026-07-31)

Parent direction: GovOPlaN/govoplan#29.

Add to acceptance:

  • Each material report/indicator retains its institutional question, goal or obligation, owner, datasource/materialization and Dataflow revisions, freshness/quality, calculation version, purpose/visibility/retention, publication, and consuming decision/action references.
  • Reporting owns semantic report planning, policy-aware result shaping, and presentation; Dataflow owns generic validation, expression evaluation, SQL compilation, execution, and schema propagation, while Datasources and referenced institutional modules retain source and domain ownership.
  • Catalogue metadata and aggregate visibility must not grant access to protected detail rows.
## Context Implement the semantic core defined by #3 on top of reproducible analytical datasets from #2. PostgreSQL is the first execution target, but definitions must remain provider-neutral. ## Acceptance criteria - [ ] Define versioned analytical dataset, dimension, level/hierarchy, measure, calculated measure, format, total, parameter, and report-definition schemas. - [ ] Consume datasets through capabilities and opaque references; do not import Dataflow or Datasources internals. - [ ] Persist source revision/fingerprint, schema, freshness, lineage, quality state, effective Policy decision, and retention provenance. - [ ] Support system/tenant/group/user scope inheritance without broadening ancestor constraints. - [ ] Provide a PostgreSQL-backed query planner with bounded filters, grouping, measures, ordering, and result limits. - [ ] Add permission and row-level-policy hooks before planning and again before result delivery. - [ ] Add migration, API, and module-permutation tests. Depends on #2. Architecture reference: `docs/SUPERX_CAPABILITY_ASSESSMENT.md`. ## Architecture reconciliation (2026-07-31) Parent direction: [GovOPlaN/govoplan#29](https://git.add-ideas.de/GovOPlaN/govoplan/issues/29). Add to acceptance: - Each material report/indicator retains its institutional question, goal or obligation, owner, datasource/materialization and Dataflow revisions, freshness/quality, calculation version, purpose/visibility/retention, publication, and consuming decision/action references. - Reporting owns semantic report planning, policy-aware result shaping, and presentation; Dataflow owns generic validation, expression evaluation, SQL compilation, execution, and schema propagation, while Datasources and referenced institutional modules retain source and domain ownership. - Catalogue metadata and aggregate visibility must not grant access to protected detail rows. <!-- codex-institutional-governance-reporting-2026-07-31 -->
zemion commented 2026-08-01 14:55:20 +02:00
Author
Owner
Copy Link
Copy Source

Codex State: progress

Summary

  • The provider-neutral immutable dataset/semantic/report schemas, typed dimensions/hierarchies/measures/calculations, provenance, access and row-policy hooks, bounded execution, migrations, APIs, WebUI and permutation-safe contracts are implemented locally.

Verification

  • 7 Reporting tests passed; Ruff passed; Core WebUI production build passed.

Next / Blocked

  • Retain this issue for the explicit PostgreSQL push-down planner and the full transitive system/tenant/group/user inheritance restriction matrix; the current safe engine evaluates bounded provider rows in the Reporting runtime.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - The provider-neutral immutable dataset/semantic/report schemas, typed dimensions/hierarchies/measures/calculations, provenance, access and row-policy hooks, bounded execution, migrations, APIs, WebUI and permutation-safe contracts are implemented locally. ### Verification - `7 Reporting tests passed; Ruff passed; Core WebUI production build passed.` ### Next / Blocked - Retain this issue for the explicit PostgreSQL push-down planner and the full transitive system/tenant/group/user inheritance restriction matrix; the current safe engine evaluates bounded provider rows in the Reporting runtime. Suggested status label: `status/in-progress`
zemion added
status
in-progress
 and removed
status
ready
 labels 2026-08-01 14:56:06 +02:00
zemion commented 2026-08-03 20:41:09 +02:00
Author
Owner
Copy Link
Copy Source

Closure audit (2026-08-03)

The versioned provider-neutral dataset, semantic model, dimensions/hierarchies/measures, provenance, capability boundary, row-policy hook, migrations, and bounded in-process execution are implemented. This issue is not yet closable: query execution currently materializes rows and filters/groups in Python; the required PostgreSQL pushdown planner is absent. The complete system/tenant/group/user inheritance matrix that proves ancestor restrictions cannot be broadened is also incomplete.

Closure requires the PostgreSQL planner plus the transitive scope-inheritance/policy matrix. Those are the correct next slices before #5 is finalized.

## Closure audit (2026-08-03) The versioned provider-neutral dataset, semantic model, dimensions/hierarchies/measures, provenance, capability boundary, row-policy hook, migrations, and bounded in-process execution are implemented. This issue is **not yet closable**: query execution currently materializes rows and filters/groups in Python; the required PostgreSQL pushdown planner is absent. The complete system/tenant/group/user inheritance matrix that proves ancestor restrictions cannot be broadened is also incomplete. Closure requires the PostgreSQL planner plus the transitive scope-inheritance/policy matrix. Those are the correct next slices before #5 is finalized.
zemion commented 2026-08-04 02:24:19 +02:00
Author
Owner
Copy Link
Copy Source

Implemented and published in 8337aec. Reporting definitions now persist restrictive system/tenant/group/user governance provenance; parent limits are intersected and cannot be broadened. Execution performs pre-plan and pre-delivery authorization, and PostgreSQL installations receive parameterized bounded semantic plans for filters, grouping, base/calculated measures, ordering, offsets, and limits. Provider ownership remains capability-based. Verification: 11 Reporting tests, Ruff, 65/65 manifest registry, strict endpoint inventory, module-permutation probes, and the integrated Core WebUI production build.

Implemented and published in 8337aec. Reporting definitions now persist restrictive system/tenant/group/user governance provenance; parent limits are intersected and cannot be broadened. Execution performs pre-plan and pre-delivery authorization, and PostgreSQL installations receive parameterized bounded semantic plans for filters, grouping, base/calculated measures, ordering, offsets, and limits. Provider ownership remains capability-based. Verification: 11 Reporting tests, Ruff, 65/65 manifest registry, strict endpoint inventory, module-permutation probes, and the integrated Core WebUI production build.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/api
HTTP API contracts, routers, schemas, or API smoke behavior.
 area/auth
Authentication, sessions, access bootstrap, or login behavior.
 area/db
Database sessions, models, transactions, or persistence primitives.
 area/devex
Local developer workflow, scripts, tests, tooling, or release helpers.
 area/docs
Durable documentation and project guidance.
 area/governance
Governance policy, audit, privacy, retention, or compliance behavior.
 area/marketing
Public website, product messaging, publication copy, or legal page content.
 area/migrations
Alembic migrations, schema bootstrap, or persistence evolution.
 area/module-system
Module discovery, manifests, capabilities, routing, or optional integrations.
 area/rbac
Permissions, roles, delegation, or authorization policy.
 area/release
Versioning, release locks, tags, packaging, or dependency pins.
 area/security
Security posture, static analysis, supply-chain hardening, or vulnerability remediation.
 area/tenancy
Tenant boundaries, provisioning, or tenant-scoped data behavior.
 area/webui
Shared WebUI shell, frontend components, routing, or frontend tests.
 audit/complexity
Complexity finding from Radon, Xenon, or equivalent maintainability scans.
 audit/duplication
Duplicated-code finding from jscpd or equivalent similarity scans.
 audit/false-positive
Audit finding reviewed as a narrow false positive or acceptable risk.
 audit/needs-design
Audit finding that needs an architectural or product decision before implementation.
 audit/quick-fix
Audit finding that appears narrow and directly fixable.
 audit/structural
Audit finding that needs design, refactoring, or behavior review.
 codex/needs-human
Needs an explicit human decision before Codex should implement.
 codex/ready
Suitable for Codex to pick up with the existing issue context.
 module/access
GovOPlaN access, identity, authentication, RBAC, and administration behavior.
 module/addresses
GovOPlaN Addresses module behavior or integration.
 module/admin
GovOPlaN Admin module behavior or integration.
 module/appointments
GovOPlaN Appointments module behavior or integration.
 module/approvals
GovOPlaN Approvals module behavior or integration.
 module/audit
GovOPlaN Audit module behavior or integration.
 module/calendar
GovOPlaN Calendar module behavior or integration.
 module/campaign
GovOPlaN campaign module behavior or integration.
 module/cases
GovOPlaN Cases module behavior or integration.
 module/committee
GovOPlaN Committee module behavior or integration.
 module/connectors
GovOPlaN Connectors module behavior or integration.
 module/core
GovOPlaN core runner, shared primitives, shell, or extension points.
 module/dashboard
GovOPlaN Dashboard module behavior or integration.
 module/dataflow
GovOPlaN Dataflow module behavior or integration.
 module/datasources
GovOPlaN governed datasource contracts, catalogs, and integrations.
 module/decisions
GovOPlaN formal Decisions module behavior or integration.
 module/dist-lists
GovOPlaN Distribution Lists module behavior or integration.
 module/dms
GovOPlaN Dms module behavior or integration.
 module/docs
GovOPlaN Docs module behavior or integration.
 module/encryption
GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.
 module/erp
GovOPlaN Erp module behavior or integration.
 module/evaluation
GovOPlaN Evaluation module behavior or integration.
 module/files
GovOPlaN files module behavior or integration.
 module/fit-connect
GovOPlaN Fit Connect module behavior or integration.
 module/forms
GovOPlaN Forms module behavior or integration.
 module/forms-runtime
GovOPlaN Forms Runtime module behavior or integration.
 module/helpdesk
GovOPlaN Helpdesk module behavior or integration.
 module/identity
GovOPlaN Identity module behavior or integration.
 module/identity-trust
GovOPlaN Identity Trust module behavior or integration.
 module/idm
GovOPlaN Idm module behavior or integration.
 module/ledger
GovOPlaN Ledger module behavior or integration.
 module/mail
GovOPlaN mail module behavior or integration.
 module/mandates
GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.
 module/notifications
GovOPlaN Notifications module behavior or integration.
 module/ops
GovOPlaN Ops module behavior or integration.
 module/organizations
GovOPlaN Organizations module behavior or integration.
 module/parties
GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.
 module/payments
GovOPlaN Payments module behavior or integration.
 module/permits
GovOPlaN Permits module behavior or integration.
 module/policy
GovOPlaN Policy module behavior or integration.
 module/poll
GovOPlaN Poll module behavior or integration.
 module/portal
GovOPlaN Portal module behavior or integration.
 module/postbox
GovOPlaN Postbox module behavior or integration.
 module/projects
GovOPlaN Projects module behavior or integration.
 module/quick-access
GovOPlaN configurable task-local Quick Access behavior and integrations.
 module/records
GovOPlaN Records and eAkte lifecycle behavior or integration.
 module/reporting
GovOPlaN Reporting module behavior or integration.
 module/risk-compliance
GovOPlaN Risk Compliance module behavior or integration.
 module/scheduling
GovOPlaN Scheduling module behavior or integration.
 module/search
GovOPlaN Search module behavior or integration.
 module/services
GovOPlaN versioned institutional Services behavior or integration.
 module/tasks
GovOPlaN Tasks module behavior or integration.
 module/templates
GovOPlaN Templates module behavior or integration.
 module/tenancy
GovOPlaN Tenancy module behavior or integration.
 module/tickets
GovOPlaN Tickets module behavior or integration.
 module/views
GovOPlaN governed task views, interface projections, and workflow view integration.
 module/voting
GovOPlaN Voting module behavior or integration.
 module/wiki
GovOPlaN Wiki module behavior or integration.
 module/workflow
GovOPlaN Workflow module behavior or integration.
 module/workflow-engine
GovOPlaN Workflow Engine runtime, persistence, or integration.
 module/xoev
GovOPlaN Xoev module behavior or integration.
 module/xrechnung
GovOPlaN Xrechnung module behavior or integration.
 module/xta-osci
GovOPlaN Xta Osci module behavior or integration.
priority
p0
Immediate stop-the-line priority.
priority
p1
High priority for the next focused work window.
priority
p2
Normal planned priority.
priority
p3
Low priority or opportunistic cleanup.
source/backlog-import
Imported from markdown backlog, roadmap, plan, or TODO files.
 source/security-audit
Created from a structured security or code-quality audit report.
 source/todo-scan
Imported from inline TODO/FIXME/HACK markers by the Gitea TODO importer.
status
blocked
Cannot progress without a decision, dependency, credential, or external change.
status
in-progress
Currently being worked.
status
needs-info
Needs clarifying input before implementation can proceed safely.
status
ready
Ready for implementation.
status
triage
Needs review, ownership, priority, or acceptance criteria.
type
bug
A reproducible defect, regression, or incorrect behavior.
type
debt
Cleanup, refactoring, risk reduction, or deferred engineering work.
type
docs
Documentation, process, or developer workflow work.
type
feature
New user-visible behavior or platform capability.
type
task
Implementation, maintenance, migration, or operational work.
type
user-story
End-to-end user journey or real-world process story used to steer product slices.
No labels area/api codex/ready module/dataflow module/datasources module/policy module/reporting
priority
p1
status
in-progress
type
feature
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
govoplan-package-publisher zemion (Albrecht Degering)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-reporting#4