GovOPlaN/govoplan-search
2
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

[Epic] Deliver permission-aware global and targeted search #1

New Issue
Closed
opened 2026-07-29 14:27:57 +02:00 by zemion · 5 comments
zemion commented 2026-07-29 14:27:57 +02:00
Owner
Copy Link
Copy Source

Outcome

GovOPlaN provides one global search entry point and module-scoped search views across the modules installed in the current composition, without leaking the existence, count, title, snippet, or metadata of resources the principal cannot access.

Architecture

  • govoplan-search owns query orchestration, ranking/result DTOs, saved/search UI, indexing state, and backend adapters.
  • Core owns versioned provider/index-change contracts and shell contribution points, not module-specific search logic.
  • Each domain module owns extraction, resource references, ACL inputs, result labels, deep links, and reindex/delta behavior for its records.
  • PostgreSQL full-text/trigram search is the required self-hosted baseline. OpenSearch is an optional backend, not a mandatory deployment dependency.

Acceptance criteria

  • Global and targeted query APIs support text, module/type filters, stable cursors, bounded result windows, and explainable result provenance.
  • Search permissions are fail-closed and rechecked at query time where indexed ACL material is insufficient or stale.
  • Index updates consume durable versioned change events/backfills and reconcile deletes, moves, permission changes, and module disable/uninstall.
  • The shell exposes a keyboard-accessible search surface; modules can contribute targeted search without source imports.
  • Search degrades cleanly when optional providers or an optional external backend are absent.
  • Security tests prove no result/count/snippet/timing shortcut exposes denied resources across tenants.

Supersedes the one-line routing placeholder govoplan-core#133.

## Outcome GovOPlaN provides one global search entry point and module-scoped search views across the modules installed in the current composition, without leaking the existence, count, title, snippet, or metadata of resources the principal cannot access. ## Architecture - `govoplan-search` owns query orchestration, ranking/result DTOs, saved/search UI, indexing state, and backend adapters. - Core owns versioned provider/index-change contracts and shell contribution points, not module-specific search logic. - Each domain module owns extraction, resource references, ACL inputs, result labels, deep links, and reindex/delta behavior for its records. - PostgreSQL full-text/trigram search is the required self-hosted baseline. OpenSearch is an optional backend, not a mandatory deployment dependency. ## Acceptance criteria - Global and targeted query APIs support text, module/type filters, stable cursors, bounded result windows, and explainable result provenance. - Search permissions are fail-closed and rechecked at query time where indexed ACL material is insufficient or stale. - Index updates consume durable versioned change events/backfills and reconcile deletes, moves, permission changes, and module disable/uninstall. - The shell exposes a keyboard-accessible search surface; modules can contribute targeted search without source imports. - Search degrades cleanly when optional providers or an optional external backend are absent. - Security tests prove no result/count/snippet/timing shortcut exposes denied resources across tenants. Supersedes the one-line routing placeholder `govoplan-core#133`.
zemion commented 2026-07-29 15:52:07 +02:00
Author
Owner
Copy Link
Copy Source

Codex State: progress

Summary

  • The first self-hosted Search slice is pushed in govoplan-search@c60ca27 and core shell/contracts in govoplan-core@d9b5708.
  • Search remains optional, providers fail independently, inactive modules are excluded, and the titlebar uses context contributions when a module announces them.

Next / Blocked

  • Complete #2, add source-module indexing announcements, then implement the optional OpenSearch adapter in #3.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - The first self-hosted Search slice is pushed in govoplan-search@c60ca27 and core shell/contracts in govoplan-core@d9b5708. - Search remains optional, providers fail independently, inactive modules are excluded, and the titlebar uses context contributions when a module announces them. ### Next / Blocked - Complete #2, add source-module indexing announcements, then implement the optional OpenSearch adapter in #3. Suggested status label: `status/in-progress`
zemion added
status
in-progress
 and removed
status
ready
 labels 2026-07-29 15:54:01 +02:00
zemion commented 2026-07-29 18:09:56 +02:00
Author
Owner
Copy Link
Copy Source

The production baseline in #2 is complete (Core 920e3c9, Search a156e3d): global/context-aware shell UI, permission-safe query orchestration, cursor API, source contracts, durable indexing lifecycle, and diagnostics are in place. #3 remains intentionally optional and is not required to run Search.

Keeping this epic open while native domain modules begin registering concrete source providers and while a live PostgreSQL CI job is added. The contract and built-in backend are ready for those integrations; missing or failing sensitive providers deny results rather than leaking stale indexed data.

The production baseline in #2 is complete (Core 920e3c9, Search a156e3d): global/context-aware shell UI, permission-safe query orchestration, cursor API, source contracts, durable indexing lifecycle, and diagnostics are in place. #3 remains intentionally optional and is not required to run Search. Keeping this epic open while native domain modules begin registering concrete source providers and while a live PostgreSQL CI job is added. The contract and built-in backend are ready for those integrations; missing or failing sensitive providers deny results rather than leaking stale indexed data.
zemion commented 2026-07-29 20:38:03 +02:00
Author
Owner
Copy Link
Copy Source

Search rollout progressed with a first governed native source and a live-dialect gate:

  • Dataflow 5f3303f registers dataflow.pipelines, performs stable tenant-bounded backfills, and rechecks Dataflow scope/governance at query time. Search results deep-link to the selected pipeline.
  • Search 6fcd9e0 adds a PostgreSQL integration test proving full-text queries remain tenant- and ACL-bounded against migrated tables.
  • Meta 3f9567a starts PostgreSQL in the module-matrix job, applies the Search migrations, runs startup smoke, and executes the dialect-specific test before the normal matrix.

The local consolidated check is green. Keeping this epic open for native source and durable delta rollout across Files, Campaign, Calendar, Mail, IDM/Postbox, and the future Wiki/Projects/Tickets modules. OpenSearch remains optional in #3 and is not part of the baseline completion path.

Search rollout progressed with a first governed native source and a live-dialect gate: - Dataflow 5f3303f registers dataflow.pipelines, performs stable tenant-bounded backfills, and rechecks Dataflow scope/governance at query time. Search results deep-link to the selected pipeline. - Search 6fcd9e0 adds a PostgreSQL integration test proving full-text queries remain tenant- and ACL-bounded against migrated tables. - Meta 3f9567a starts PostgreSQL in the module-matrix job, applies the Search migrations, runs startup smoke, and executes the dialect-specific test before the normal matrix. The local consolidated check is green. Keeping this epic open for native source and durable delta rollout across Files, Campaign, Calendar, Mail, IDM/Postbox, and the future Wiki/Projects/Tickets modules. OpenSearch remains optional in #3 and is not part of the baseline completion path.
zemion commented 2026-08-03 20:41:10 +02:00
Author
Owner
Copy Link
Copy Source

Closure audit (2026-08-03)

The PostgreSQL baseline, fail-closed orchestration, stable cursor API, global/context overlay, F3 focus, durable index-change/rebuild contracts, diagnostics, module-disable behavior, and live PostgreSQL CI gate are implemented. This epic is not yet closable.

Current native search-source registrations exist for Dataflow, Projects, Reporting, and Risk Compliance. Files, Campaign, Calendar, Mail, IDM/Postbox, and later Wiki/Tickets still need permission-rechecked providers and durable create/update/delete/permission-change deltas. The admin diagnostics/rebuild/process/reconcile endpoints also need an operator UI.

Recommended closure order: admin diagnostics UI; Files and Campaign providers; Calendar/Mail; IDM/Postbox; remaining object modules; then the cross-tenant no-leak timing/count regression matrix. OpenSearch remains optional and is not a blocker.

## Closure audit (2026-08-03) The PostgreSQL baseline, fail-closed orchestration, stable cursor API, global/context overlay, F3 focus, durable index-change/rebuild contracts, diagnostics, module-disable behavior, and live PostgreSQL CI gate are implemented. This epic is **not yet closable**. Current native search-source registrations exist for Dataflow, Projects, Reporting, and Risk Compliance. Files, Campaign, Calendar, Mail, IDM/Postbox, and later Wiki/Tickets still need permission-rechecked providers and durable create/update/delete/permission-change deltas. The admin diagnostics/rebuild/process/reconcile endpoints also need an operator UI. Recommended closure order: admin diagnostics UI; Files and Campaign providers; Calendar/Mail; IDM/Postbox; remaining object modules; then the cross-tenant no-leak timing/count regression matrix. OpenSearch remains optional and is not a blocker.
zemion commented 2026-08-04 03:07:02 +02:00
Author
Owner
Copy Link
Copy Source

Implemented and verified on main.

Key commits:

  • Search operator UI, durable event ingestion, bounded rebuild/reconcile, and permission-aware query path: govoplan-search@1bb3381, govoplan-core@7117673
  • Native providers: Files 8ec31b1, Campaign 91890fd, Calendar 04438d9, Mail 72d6f84, IDM ad37b03, Postbox 889b5e6

The implementation provides PostgreSQL-backed global/targeted search, stable filters/cursors/provenance, fail-closed source authorization with query-time rechecks, durable change-event ingestion, bounded backfills, module activation reconciliation, keyboard-accessible overlay/full-page surfaces, and an operator index console. OpenSearch remains optional.

Verification included all Search backend tests, provider-specific source tests, Core durable-consumer/module-contract tests, Search overlay/interface structural tests, the integrated Core WebUI production build and bundle budgets, manifest-shape validation, Ruff, and diff checks. Cross-tenant/denied-resource cases and provider degradation are covered by focused tests.

Implemented and verified on `main`. Key commits: - Search operator UI, durable event ingestion, bounded rebuild/reconcile, and permission-aware query path: `govoplan-search@1bb3381`, `govoplan-core@7117673` - Native providers: Files `8ec31b1`, Campaign `91890fd`, Calendar `04438d9`, Mail `72d6f84`, IDM `ad37b03`, Postbox `889b5e6` The implementation provides PostgreSQL-backed global/targeted search, stable filters/cursors/provenance, fail-closed source authorization with query-time rechecks, durable change-event ingestion, bounded backfills, module activation reconciliation, keyboard-accessible overlay/full-page surfaces, and an operator index console. OpenSearch remains optional. Verification included all Search backend tests, provider-specific source tests, Core durable-consumer/module-contract tests, Search overlay/interface structural tests, the integrated Core WebUI production build and bundle budgets, manifest-shape validation, Ruff, and diff checks. Cross-tenant/denied-resource cases and provider degradation are covered by focused tests.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/api
HTTP API contracts, routers, schemas, or API smoke behavior.
 area/auth
Authentication, sessions, access bootstrap, or login behavior.
 area/db
Database sessions, models, transactions, or persistence primitives.
 area/devex
Local developer workflow, scripts, tests, tooling, or release helpers.
 area/docs
Durable documentation and project guidance.
 area/governance
Governance policy, audit, privacy, retention, or compliance behavior.
 area/marketing
Public website, product messaging, publication copy, or legal page content.
 area/migrations
Alembic migrations, schema bootstrap, or persistence evolution.
 area/module-system
Module discovery, manifests, capabilities, routing, or optional integrations.
 area/rbac
Permissions, roles, delegation, or authorization policy.
 area/release
Versioning, release locks, tags, packaging, or dependency pins.
 area/security
Security posture, static analysis, supply-chain hardening, or vulnerability remediation.
 area/tenancy
Tenant boundaries, provisioning, or tenant-scoped data behavior.
 area/webui
Shared WebUI shell, frontend components, routing, or frontend tests.
 audit/complexity
Complexity finding from Radon, Xenon, or equivalent maintainability scans.
 audit/duplication
Duplicated-code finding from jscpd or equivalent similarity scans.
 audit/false-positive
Audit finding reviewed as a narrow false positive or acceptable risk.
 audit/needs-design
Audit finding that needs an architectural or product decision before implementation.
 audit/quick-fix
Audit finding that appears narrow and directly fixable.
 audit/structural
Audit finding that needs design, refactoring, or behavior review.
 codex/needs-human
Needs an explicit human decision before Codex should implement.
 codex/ready
Suitable for Codex to pick up with the existing issue context.
 module/access
GovOPlaN access, identity, authentication, RBAC, and administration behavior.
 module/addresses
GovOPlaN Addresses module behavior or integration.
 module/admin
GovOPlaN Admin module behavior or integration.
 module/appointments
GovOPlaN Appointments module behavior or integration.
 module/approvals
GovOPlaN Approvals module behavior or integration.
 module/audit
GovOPlaN Audit module behavior or integration.
 module/calendar
GovOPlaN Calendar module behavior or integration.
 module/campaign
GovOPlaN campaign module behavior or integration.
 module/cases
GovOPlaN Cases module behavior or integration.
 module/committee
GovOPlaN Committee module behavior or integration.
 module/connectors
GovOPlaN Connectors module behavior or integration.
 module/core
GovOPlaN core runner, shared primitives, shell, or extension points.
 module/dashboard
GovOPlaN Dashboard module behavior or integration.
 module/dataflow
GovOPlaN Dataflow module behavior or integration.
 module/datasources
GovOPlaN governed datasource contracts, catalogs, and integrations.
 module/decisions
GovOPlaN formal Decisions module behavior or integration.
 module/dist-lists
GovOPlaN Distribution Lists module behavior or integration.
 module/dms
GovOPlaN Dms module behavior or integration.
 module/docs
GovOPlaN Docs module behavior or integration.
 module/encryption
GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.
 module/erp
GovOPlaN Erp module behavior or integration.
 module/evaluation
GovOPlaN Evaluation module behavior or integration.
 module/files
GovOPlaN files module behavior or integration.
 module/fit-connect
GovOPlaN Fit Connect module behavior or integration.
 module/forms
GovOPlaN Forms module behavior or integration.
 module/forms-runtime
GovOPlaN Forms Runtime module behavior or integration.
 module/helpdesk
GovOPlaN Helpdesk module behavior or integration.
 module/identity
GovOPlaN Identity module behavior or integration.
 module/identity-trust
GovOPlaN Identity Trust module behavior or integration.
 module/idm
GovOPlaN Idm module behavior or integration.
 module/ledger
GovOPlaN Ledger module behavior or integration.
 module/mail
GovOPlaN mail module behavior or integration.
 module/mandates
GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.
 module/notifications
GovOPlaN Notifications module behavior or integration.
 module/ops
GovOPlaN Ops module behavior or integration.
 module/organizations
GovOPlaN Organizations module behavior or integration.
 module/parties
GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.
 module/payments
GovOPlaN Payments module behavior or integration.
 module/permits
GovOPlaN Permits module behavior or integration.
 module/policy
GovOPlaN Policy module behavior or integration.
 module/poll
GovOPlaN Poll module behavior or integration.
 module/portal
GovOPlaN Portal module behavior or integration.
 module/postbox
GovOPlaN Postbox module behavior or integration.
 module/projects
GovOPlaN Projects module behavior or integration.
 module/quick-access
GovOPlaN configurable task-local Quick Access behavior and integrations.
 module/records
GovOPlaN Records and eAkte lifecycle behavior or integration.
 module/reporting
GovOPlaN Reporting module behavior or integration.
 module/risk-compliance
GovOPlaN Risk Compliance module behavior or integration.
 module/scheduling
GovOPlaN Scheduling module behavior or integration.
 module/search
GovOPlaN Search module behavior or integration.
 module/services
GovOPlaN versioned institutional Services behavior or integration.
 module/tasks
GovOPlaN Tasks module behavior or integration.
 module/templates
GovOPlaN Templates module behavior or integration.
 module/tenancy
GovOPlaN Tenancy module behavior or integration.
 module/tickets
GovOPlaN Tickets module behavior or integration.
 module/views
GovOPlaN governed task views, interface projections, and workflow view integration.
 module/voting
GovOPlaN Voting module behavior or integration.
 module/wiki
GovOPlaN Wiki module behavior or integration.
 module/workflow
GovOPlaN Workflow module behavior or integration.
 module/workflow-engine
GovOPlaN Workflow Engine runtime, persistence, or integration.
 module/xoev
GovOPlaN Xoev module behavior or integration.
 module/xrechnung
GovOPlaN Xrechnung module behavior or integration.
 module/xta-osci
GovOPlaN Xta Osci module behavior or integration.
priority
p0
Immediate stop-the-line priority.
priority
p1
High priority for the next focused work window.
priority
p2
Normal planned priority.
priority
p3
Low priority or opportunistic cleanup.
source/backlog-import
Imported from markdown backlog, roadmap, plan, or TODO files.
 source/security-audit
Created from a structured security or code-quality audit report.
 source/todo-scan
Imported from inline TODO/FIXME/HACK markers by the Gitea TODO importer.
status
blocked
Cannot progress without a decision, dependency, credential, or external change.
status
in-progress
Currently being worked.
status
needs-info
Needs clarifying input before implementation can proceed safely.
status
ready
Ready for implementation.
status
triage
Needs review, ownership, priority, or acceptance criteria.
type
bug
A reproducible defect, regression, or incorrect behavior.
type
debt
Cleanup, refactoring, risk reduction, or deferred engineering work.
type
docs
Documentation, process, or developer workflow work.
type
feature
New user-visible behavior or platform capability.
type
task
Implementation, maintenance, migration, or operational work.
type
user-story
End-to-end user journey or real-world process story used to steer product slices.
No labels area/api area/security area/webui codex/ready module/access module/core module/search
priority
p1
status
in-progress
type
feature
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
govoplan-package-publisher zemion (Albrecht Degering)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-search#1