add-ideas/govoplan-access
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

[Performance] Add pagination and batch loading to access admin lists #12

New Issue
Closed
opened 2026-07-11 11:39:42 +02:00 by zemion · 3 comments
zemion commented 2026-07-11 11:39:42 +02:00
Owner
Copy Link

Several access admin list endpoints fetch full tables and then call helper functions that run additional queries per row. This will become slow with larger tenants.

Code paths: src/govoplan_access/backend/api/v1/routes.py:1080, src/govoplan_access/backend/api/v1/routes.py:2037, src/govoplan_access/backend/api/v1/admin_common.py:85, src/govoplan_access/backend/api/v1/admin_common.py:111, src/govoplan_access/backend/api/v1/admin_common.py:141.

Acceptance: full-list endpoints use cursor/page-size parameters or route clients to delta/paginated APIs, summaries batch-load related data, tests assert bounded query counts, and small-tenant UI behavior remains intact.

Several access admin list endpoints fetch full tables and then call helper functions that run additional queries per row. This will become slow with larger tenants. Code paths: `src/govoplan_access/backend/api/v1/routes.py:1080`, `src/govoplan_access/backend/api/v1/routes.py:2037`, `src/govoplan_access/backend/api/v1/admin_common.py:85`, `src/govoplan_access/backend/api/v1/admin_common.py:111`, `src/govoplan_access/backend/api/v1/admin_common.py:141`. Acceptance: full-list endpoints use cursor/page-size parameters or route clients to delta/paginated APIs, summaries batch-load related data, tests assert bounded query counts, and small-tenant UI behavior remains intact. <!-- codex-audit-2026-07-11:access-nplus1 -->
zemion commented 2026-07-12 01:35:41 +02:00
Author
Owner
Copy Link

Codex State: progress

Summary

  • Added bounded pagination metadata and page/page_size parameters to legacy admin list endpoints for identities, users, groups, roles, system roles, system accounts, and API keys.
  • Left delta endpoints unchanged because their first-load/full-snapshot behavior needs frontend cursor continuation work before changing response shape.

Changed Files

  • src/govoplan_access/backend/api/v1/admin_schemas.py
  • src/govoplan_access/backend/api/v1/routes.py
  • tests/test_admin_pagination.py

Verification

  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination
  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests

Next / Blocked

  • Add bounded continuation semantics to delta/full-snapshot admin views if this issue is meant to cover the WebUI initial-load path too.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - Added bounded pagination metadata and page/page_size parameters to legacy admin list endpoints for identities, users, groups, roles, system roles, system accounts, and API keys. - Left delta endpoints unchanged because their first-load/full-snapshot behavior needs frontend cursor continuation work before changing response shape. ### Changed Files - `src/govoplan_access/backend/api/v1/admin_schemas.py` - `src/govoplan_access/backend/api/v1/routes.py` - `tests/test_admin_pagination.py` ### Verification - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination` - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests` ### Next / Blocked - Add bounded continuation semantics to delta/full-snapshot admin views if this issue is meant to cover the WebUI initial-load path too. Suggested status label: `status/in-progress`
zemion commented 2026-07-12 02:11:38 +02:00
Author
Owner
Copy Link

Codex State: progress

Summary

  • Extended the pagination slice with batch summary helpers for accounts, user groups, direct roles, group members, group roles, tenant role assignment counts, system role assignment counts, and API key owner accounts.
  • Wired the paginated users, groups, roles, system roles/system accounts shared-role list, and API key endpoints to the batch helpers while leaving delta/full-snapshot endpoints unchanged.

Changed Files

  • src/govoplan_access/backend/api/v1/admin_common.py
  • src/govoplan_access/backend/api/v1/routes.py
  • tests/test_admin_batch_helpers.py
  • tests/test_admin_pagination.py

Verification

  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination tests.test_admin_batch_helpers
  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests

Next / Blocked

  • Still open: decide and implement bounded continuation semantics for delta full snapshots, and add API-level query-count regression coverage before closing.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - Extended the pagination slice with batch summary helpers for accounts, user groups, direct roles, group members, group roles, tenant role assignment counts, system role assignment counts, and API key owner accounts. - Wired the paginated users, groups, roles, system roles/system accounts shared-role list, and API key endpoints to the batch helpers while leaving delta/full-snapshot endpoints unchanged. ### Changed Files - `src/govoplan_access/backend/api/v1/admin_common.py` - `src/govoplan_access/backend/api/v1/routes.py` - `tests/test_admin_batch_helpers.py` - `tests/test_admin_pagination.py` ### Verification - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination tests.test_admin_batch_helpers` - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests` ### Next / Blocked - Still open: decide and implement bounded continuation semantics for delta full snapshots, and add API-level query-count regression coverage before closing. Suggested status label: `status/in-progress`
zemion commented 2026-07-12 02:27:35 +02:00
Author
Owner
Copy Link

Codex State: done

Summary

  • Completed bounded access admin list loading: legacy list endpoints are paginated, identity/user/group/role/API-key summaries use batch helpers, and delta initial snapshots now page through full snapshots with full:* continuation cursors before returning a normal seq:* watermark.
  • Adjusted access WebUI delta merging so paged full snapshots accumulate instead of replacing earlier pages, including the custom system users panel loop.

Changed Files

  • src/govoplan_access/backend/api/v1/admin_common.py
  • src/govoplan_access/backend/api/v1/admin_schemas.py
  • src/govoplan_access/backend/api/v1/routes.py
  • webui/src/features/admin/utils/deltaRows.ts
  • webui/src/features/admin/SystemUsersPanel.tsx
  • tests/test_admin_batch_helpers.py
  • tests/test_admin_pagination.py

Verification

  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination tests.test_admin_batch_helpers tests.test_auth_dependencies
  • PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests
  • /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build in govoplan-core/webui
## Codex State: done ### Summary - Completed bounded access admin list loading: legacy list endpoints are paginated, identity/user/group/role/API-key summaries use batch helpers, and delta initial snapshots now page through full snapshots with full:* continuation cursors before returning a normal seq:* watermark. - Adjusted access WebUI delta merging so paged full snapshots accumulate instead of replacing earlier pages, including the custom system users panel loop. ### Changed Files - `src/govoplan_access/backend/api/v1/admin_common.py` - `src/govoplan_access/backend/api/v1/admin_schemas.py` - `src/govoplan_access/backend/api/v1/routes.py` - `webui/src/features/admin/utils/deltaRows.ts` - `webui/src/features/admin/SystemUsersPanel.tsx` - `tests/test_admin_batch_helpers.py` - `tests/test_admin_pagination.py` ### Verification - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_optional_tenancy_contract tests.test_admin_pagination tests.test_admin_batch_helpers tests.test_auth_dependencies` - `PYTHONPATH=/mnt/DATA/git/govoplan-core/src:/mnt/DATA/git/govoplan-access/src:/mnt/DATA/git/govoplan-tenancy/src /mnt/DATA/git/govoplan/.venv/bin/python -m compileall -q src tests` - `/home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build in govoplan-core/webui`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/api
HTTP API contracts, routers, schemas, or API smoke behavior.
area/auth
Authentication, sessions, access bootstrap, or login behavior.
area/db
Database sessions, models, transactions, or persistence primitives.
area/deployment
Deployment, release, operations, or runtime packaging.
area/devex
Local developer workflow, scripts, tests, tooling, or release helpers.
area/docs
Durable documentation and project guidance.
area/governance
Governance policy, audit, privacy, retention, or compliance behavior.
area/marketing
Public website, product messaging, publication copy, or legal page content.
area/migrations
Alembic migrations, schema bootstrap, or persistence evolution.
area/module-system
Module discovery, manifests, capabilities, routing, or optional integrations.
area/rbac
Permissions, roles, delegation, or authorization policy.
area/release
Versioning, release locks, tags, packaging, or dependency pins.
area/security
Security posture, static analysis, supply-chain hardening, or vulnerability remediation.
area/tenancy
Tenant boundaries, provisioning, or tenant-scoped data behavior.
area/webui
Shared WebUI shell, frontend components, routing, or frontend tests.
audit/complexity
Complexity finding from Radon, Xenon, or equivalent maintainability scans.
audit/duplication
Duplicated-code finding from jscpd or equivalent similarity scans.
audit/false-positive
Audit finding reviewed as a narrow false positive or acceptable risk.
audit/needs-design
Audit finding that needs an architectural or product decision before implementation.
audit/quick-fix
Audit finding that appears narrow and directly fixable.
audit/structural
Audit finding that needs design, refactoring, or behavior review.
codex/needs-human
Needs an explicit human decision before Codex should implement.
codex/ready
Suitable for Codex to pick up with the existing issue context.
module/access
GovOPlaN access, identity, authentication, RBAC, and administration behavior.
module/addresses
GovOPlaN Addresses module behavior or integration.
module/admin
GovOPlaN Admin module behavior or integration.
module/appointments
GovOPlaN Appointments module behavior or integration.
module/approvals
GovOPlaN Approvals module behavior or integration.
module/assets
GovOPlaN Assets module behavior or integration.
module/audit
GovOPlaN Audit module behavior or integration.
module/booking
GovOPlaN Booking module behavior or integration.
module/calendar
GovOPlaN Calendar module behavior or integration.
module/campaign
GovOPlaN campaign module behavior or integration.
module/cases
GovOPlaN Cases module behavior or integration.
module/certificates
GovOPlaN Certificates module behavior or integration.
module/committee
GovOPlaN Committee module behavior or integration.
module/connectors
GovOPlaN Connectors module behavior or integration.
module/consultation
GovOPlaN Consultation module behavior or integration.
module/contracts
GovOPlaN Contracts module behavior or integration.
module/core
GovOPlaN core runner, shared primitives, shell, or extension points.
module/dist-lists
GovOPlaN Distribution Lists module behavior or integration.
module/dms
GovOPlaN Dms module behavior or integration.
module/docs
GovOPlaN documentation layer behavior or integration.
module/erp
GovOPlaN Erp module behavior or integration.
module/evaluation
GovOPlaN Evaluation module behavior or integration.
module/facilities
GovOPlaN Facilities module behavior or integration.
module/files
GovOPlaN files module behavior or integration.
module/fit-connect
GovOPlaN Fit Connect module behavior or integration.
module/forms
GovOPlaN Forms module behavior or integration.
module/forms-runtime
GovOPlaN Forms Runtime module behavior or integration.
module/grants
GovOPlaN Grants module behavior or integration.
module/helpdesk
GovOPlaN Helpdesk module behavior or integration.
module/identity
GovOPlaN Identity module behavior or integration.
module/identity-trust
GovOPlaN Identity Trust module behavior or integration.
module/idm
GovOPlaN Idm module behavior or integration.
module/inspections
GovOPlaN Inspections module behavior or integration.
module/issue-reporting
GovOPlaN Issue Reporting module behavior or integration.
module/learning
GovOPlaN Learning module behavior or integration.
module/ledger
GovOPlaN Ledger module behavior or integration.
module/mail
GovOPlaN mail module behavior or integration.
module/notifications
GovOPlaN Notifications module behavior or integration.
module/ops
GovOPlaN Ops module behavior or integration.
module/organizations
GovOPlaN Organizations module behavior or integration.
module/payments
GovOPlaN Payments module behavior or integration.
module/permits
GovOPlaN Permits module behavior or integration.
module/policy
GovOPlaN Policy module behavior or integration.
module/poll
GovOPlaN Poll module behavior or integration.
module/portal
GovOPlaN Portal module behavior or integration.
module/postbox
GovOPlaN Postbox module behavior or integration.
module/procurement
GovOPlaN Procurement module behavior or integration.
module/records
GovOPlaN Records module behavior or integration.
module/reporting
GovOPlaN Reporting module behavior or integration.
module/resources
GovOPlaN Resources module behavior or integration.
module/risk-compliance
GovOPlaN Risk Compliance module behavior or integration.
module/scheduling
GovOPlaN Scheduling module behavior or integration.
module/search
GovOPlaN Search module behavior or integration.
module/tasks
GovOPlaN Tasks module behavior or integration.
module/templates
GovOPlaN Templates module behavior or integration.
module/tenancy
GovOPlaN Tenancy module behavior or integration.
module/transparency
GovOPlaN Transparency module behavior or integration.
module/web
GovOPlaN public website, product page, or publication content.
module/workflow
GovOPlaN Workflow module behavior or integration.
module/xoev
GovOPlaN Xoev module behavior or integration.
module/xrechnung
GovOPlaN Xrechnung module behavior or integration.
module/xta-osci
GovOPlaN Xta Osci module behavior or integration.
priority
p0
Immediate stop-the-line priority.
priority
p1
High priority for the next focused work window.
priority
p2
Normal planned priority.
priority
p3
Low priority or opportunistic cleanup.
source/backlog-import
Imported from markdown backlog, roadmap, plan, or TODO files.
source/module-roadmap
Created from GovOPlaN module and integration roadmap planning.
source/security-audit
Created from a structured security or code-quality audit report.
source/todo-scan
Imported from inline TODO/FIXME/HACK markers by the Gitea TODO importer.
status
blocked
Cannot progress without a decision, dependency, credential, or external change.
status
in-progress
Currently being worked.
status
needs-info
Needs clarifying input before implementation can proceed safely.
status
ready
Ready for implementation.
status
triage
Needs review, ownership, priority, or acceptance criteria.
type
bug
A reproducible defect, regression, or incorrect behavior.
type
debt
Cleanup, refactoring, risk reduction, or deferred engineering work.
type
docs
Documentation, process, or developer workflow work.
type
feature
New user-visible behavior or platform capability.
type
task
Implementation, maintenance, migration, or operational work.
type
user-story
End-to-end user journey or real-world process story used to steer product slices.
No Label
area/api
area/db
codex/ready
module/access
priority
p2
status
ready
type
debt
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
zemion
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: add-ideas/govoplan-access#12
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?