32 Commits
v0.1.4 ... main

Author SHA1 Message Date
e6f7c45f0a intermittent commit 2026-07-14 13:22:10 +02:00
8aa1943581 Register Scheduling WebUI package 2026-07-12 19:00:54 +02:00
b9badc9153 Harden module installation and imports 2026-07-11 18:49:35 +02:00
9a0c467d55 Release v0.1.8 2026-07-11 17:00:37 +02:00
a00ef54821 Release v0.1.7
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m35s
2026-07-11 03:11:06 +02:00
edb4687826 Cover access canonical directory migration
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m33s
2026-07-11 01:13:53 +02:00
fcfe0b69a3 Install cloned WebUI dependencies in release integration
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m38s
2026-07-11 01:00:09 +02:00
715bdcbebe Treat empty unittest discovery as release skip
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m28s
2026-07-11 00:43:14 +02:00
060f4da751 Add resource access explanation contract
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
2026-07-11 00:39:39 +02:00
c32951393a Skip empty cloned backend test suites
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m31s
2026-07-11 00:38:07 +02:00
7b85d6deae Install release WebUI modules as real packages
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m34s
2026-07-11 00:32:25 +02:00
2d2d9e7bc7 Temporarily limit heavy CI workflows to manual runs
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
2026-07-11 00:25:40 +02:00
dc1a250797 Install release WebUI modules sequentially in CI
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m36s
Release Integration / release-integration (push) Failing after 1m32s
Module Matrix / module-matrix (push) Successful in 3m54s
2026-07-11 00:23:44 +02:00
7b7cc8ada7 Fail CI after exhausted WebUI install retries
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m38s
Release Integration / release-integration (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-11 00:19:17 +02:00
722c9e5d1c Retry WebUI release dependency installs in CI
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m44s
Module Matrix / module-matrix (push) Successful in 4m9s
Release Integration / release-integration (push) Failing after 2m47s
2026-07-11 00:01:44 +02:00
63e54a67be Use checkout Alembic root in release integration
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 1m36s
Release Integration / release-integration (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 23:58:04 +02:00
12b623bec9 Include IDM in release integration install
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m47s
Module Matrix / module-matrix (push) Failing after 2m37s
Release Integration / release-integration (push) Failing after 1m30s
2026-07-10 23:36:52 +02:00
b788afcae1 Harden module update compatibility cleanup
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m46s
Module Matrix / module-matrix (push) Failing after 2m38s
Release Integration / release-integration (push) Failing after 1m41s
2026-07-10 23:27:48 +02:00
2b0cdf13f3 Add release integration workflow
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m44s
Module Matrix / module-matrix (push) Successful in 4m19s
Release Integration / release-integration (push) Failing after 1m40s
2026-07-10 23:19:09 +02:00
013e0b883d Refresh WebUI release install inputs in CI
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m45s
Module Matrix / module-matrix (push) Successful in 4m14s
2026-07-10 22:57:01 +02:00
50f60e4d43 Install test extras in module matrix workflow
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m41s
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 22:54:23 +02:00
ff4b514475 Align release WebUI dependency metadata
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 1m21s
Module Matrix / module-matrix (push) Failing after 1m30s
2026-07-10 22:47:38 +02:00
ca1206e8c9 Harden Gitea Actions release dependency SSH
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 49s
Module Matrix / module-matrix (push) Failing after 47s
2026-07-10 22:24:47 +02:00
94236a7d7e Prepare GovOPlaN self-hosted release workflow
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 50s
Module Matrix / module-matrix (push) Failing after 49s
2026-07-10 21:57:22 +02:00
8dd5123aab Cover identity and organization API cleanup
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 1m51s
Module Matrix / module-matrix (push) Failing after 13s
2026-07-10 18:57:40 +02:00
c61abe154c Add module event producer coverage
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 18:40:38 +02:00
83784ea19d Add retention policy option regression coverage
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 17:50:04 +02:00
81f0f649b5 Define governed platform event envelope
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 17:48:33 +02:00
0a130962d3 Mark DataGrid layout work deferred
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 17:43:28 +02:00
79af252e88 Decouple scopes from tenancy schema
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 17:33:43 +02:00
635d25c74c chore: consolidate platform split checks
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Module Matrix / module-matrix (push) Has been cancelled
2026-07-10 12:51:19 +02:00
150b720f12 Release v0.1.6 2026-07-07 16:00:38 +02:00
319 changed files with 43974 additions and 13775 deletions

28
.dockerignore Normal file
View File

@@ -0,0 +1,28 @@
.git
.venv
node_modules
webui/node_modules
audit-reports
runtime
dist
build
coverage
htmlcov
__pycache__
*.pyc
*.pyo
*.log
.mypy_cache
.pytest_cache
.ruff_cache
.cache
.component-test-build
.module-test-build
.policy-test-build
.template-preview-test-build
.import-test-build
webui/.component-test-build
webui/.module-test-build
webui/.policy-test-build
webui/.template-preview-test-build
webui/.import-test-build

View File

@@ -0,0 +1,35 @@
---
name: "Bug"
about: "Report a reproducible defect, regression, or incorrect behavior"
title: "[Bug] "
labels:
- type/bug
- status/triage
- module/core
---
## Scope
- Repository:
- Area/module:
- Affected version or commit:
## Behavior
Expected:
Actual:
## Reproduction
1.
2.
3.
## Evidence
Logs, screenshots, traces, or failing test output:
## Verification Target
Command or workflow that should pass when fixed:

View File

@@ -0,0 +1 @@
blank_issues_enabled: false

View File

@@ -0,0 +1,27 @@
---
name: "Docs / workflow"
about: "Request documentation, process, or developer workflow changes"
title: "[Docs] "
labels:
- type/docs
- status/triage
- module/core
- area/docs
---
## Scope
- Repository:
- Document or workflow:
## Current State
What is missing, unclear, duplicated, or stale?
## Desired State
What should the docs or workflow make clear?
## Verification Target
How should this be checked?

View File

@@ -0,0 +1,32 @@
---
name: "Feature"
about: "Propose new user-visible behavior or platform capability"
title: "[Feature] "
labels:
- type/feature
- status/triage
- module/core
---
## Problem
What user, operator, or developer problem should this solve?
## Proposed Capability
What should exist when this is done?
## Ownership
- Owning repository:
- Related module repositories:
- Extension point or integration boundary:
## Acceptance Criteria
- [ ]
- [ ]
## Verification Target
Command, scenario, or UI flow that should prove completion:

View File

@@ -0,0 +1,28 @@
---
name: "Task"
about: "Track implementation, maintenance, or migration work"
title: "[Task] "
labels:
- type/task
- status/triage
- module/core
---
## Objective
What needs to be completed?
## Scope
- Owning repository:
- In-scope:
- Out-of-scope:
## Checklist
- [ ]
- [ ]
## Verification Target
Command or manual check:

View File

@@ -0,0 +1,25 @@
---
name: "Tech debt"
about: "Track cleanup, refactoring, risk reduction, or deferred engineering work"
title: "[Debt] "
labels:
- type/debt
- status/triage
- module/core
---
## Current Cost
What does this make harder, riskier, slower, or more fragile?
## Desired Shape
What should the code, tests, or architecture look like afterwards?
## Constraints
What behavior, compatibility, or module boundary must be preserved?
## Verification Target
Focused checks that should pass:

View File

@@ -0,0 +1,15 @@
## Issue
Closes #
## Summary
-
## Verification
-
## Notes
Follow-up issues:

15
.gitignore vendored
View File

@@ -136,6 +136,21 @@ dist
.yarn/install-state.gz
.pnp.*
# Local WebUI test/build scratch directories
.component-test-build/
.module-test-build/
.policy-test-build/
.template-preview-test-build/
.import-test-build/
webui/.component-test-build/
webui/.module-test-build/
webui/.policy-test-build/
webui/.template-preview-test-build/
webui/.import-test-build/
# Security audit reports
audit-reports/
# ---> Python
# Byte-compiled / optimized / DLL files
__pycache__/

View File

@@ -6,6 +6,7 @@ This repository is the platform runner and shared core for GovOPlaN. It owns the
Sibling module repositories usually used with this repo:
- `/mnt/DATA/git/govoplan-access`
- `/mnt/DATA/git/govoplan-files`
- `/mnt/DATA/git/govoplan-mail`
- `/mnt/DATA/git/govoplan-campaign`
@@ -18,9 +19,9 @@ Use targeted commands first:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
./.venv/bin/python -m unittest tests.test_module_system
./.venv/bin/python -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
/mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_module_system
/mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
```
For WebUI checks:
@@ -35,8 +36,8 @@ PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versi
Run the consolidated focused check when a change touches module discovery, optional integrations, shared mail components, or mailbox listing:
```bash
cd /mnt/DATA/git/govoplan-core
./scripts/check-focused.sh
cd /mnt/DATA/git/govoplan
tools/checks/check-focused.sh
```
## Working Rules
@@ -44,5 +45,6 @@ cd /mnt/DATA/git/govoplan-core
- Prefer `rg`, `sed`, and targeted tests over broad recursive scans or full builds.
- Avoid DataGrid changes unless explicitly requested; it is intentionally brittle and has known deferred work.
- Do not add module-to-module imports for optional integrations. Use core registry/capability/module metadata paths.
- Treat Gitea issues as the canonical backlog and state log. Treat Gitea wiki pages as durable project context mirrored from repository and product docs. Use `docs/GITEA_ISSUES.md` for labels, templates, TODO import, wiki sync, and Codex issue updates.
- Do not keep generated WebUI test folders in git status; they should be ignored and removable.
- Do not start persistent dev servers unless the user asks.

View File

@@ -1,6 +1,17 @@
# govoplan-core
GovOPlaN core is the platform runner and shared foundation. It owns the server entry point, database/session primitives, tenant and RBAC infrastructure, governance policy, audit/auth helpers, module discovery, migration registration, and the shared WebUI shell. Feature code is supplied by installed modules.
<!-- govoplan-repository-type:start -->
**Repository type:** system (kernel).
<!-- govoplan-repository-type:end -->
[![Module Matrix](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/module-matrix.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=module-matrix.yml&actor=0&status=0)
[![Release Integration](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/release-integration.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=release-integration.yml&actor=0&status=0)
[![Dependency Audit](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/dependency-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=dependency-audit.yml&actor=0&status=0)
[![Security Audit](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/security-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=security-audit.yml&actor=0&status=0)
# govoplan-core
GovOPlaN core is the platform runner and shared foundation. It owns the server entry point, database/session primitives, module discovery, migration orchestration, capability contracts, install/uninstall orchestration, and the shared WebUI shell. Platform and feature behavior is supplied by installed modules.
## Repository ownership
@@ -9,37 +20,45 @@ Core owns:
- `govoplan_core.server.app:app`, the FastAPI entry point used by uvicorn
- `GovoplanServerConfig`, module discovery, registry validation, and route aggregation
- SQLAlchemy base/session helpers and module migration registration
- tenant/account/session/RBAC/governance/audit models and services
- core API routes for auth, admin, platform metadata, audit, and system health
- kernel APIs for platform metadata, module lifecycle, health, and development diagnostics
- `@govoplan/core-webui`, including login, CSRF/API helpers, shell layout, generic UI components, IconRail, DataGrid, access boundaries, and module route/nav contracts
Feature modules own their backend routers, models, migrations, permissions, frontend packages, nav items, and route contributions. Core should not import feature pages directly; it imports module manifests and renders their route contributions.
Platform and feature modules own their backend routers, models, migrations,
permissions, frontend packages, nav items, and route contributions. Access,
tenancy, policy, audit, and admin behavior live in their owning platform
modules. Core should not import feature pages directly; it imports module
manifests and renders their route contributions.
## Governance docs
Canonical policy documents live in `docs/`:
- [RBAC_MANIFEST.md](docs/RBAC_MANIFEST.md)
- [SYSTEM_GOVERNANCE_MANIFEST.md](docs/SYSTEM_GOVERNANCE_MANIFEST.md)
- [DOCUMENTATION_MAP.md](docs/DOCUMENTATION_MAP.md)
- [ACCESS_RBAC_MODEL.md](docs/ACCESS_RBAC_MODEL.md)
- [GOVERNANCE_MODEL.md](docs/GOVERNANCE_MODEL.md)
- [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md)
- [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md)
Modules may define module-specific permissions and policy behavior, but the platform-level permission model and governance hierarchy belong here.
Modules define module-specific permissions and policy behavior. Shared DTOs and
composition rules live in core only where they are stable kernel contracts.
## Backend development
Create or activate the core virtual environment, then install core and sibling modules from this repository:
For whole-product development, create the virtualenv from the meta repository:
```bash
cd /mnt/DATA/git/govoplan-core
cd /mnt/DATA/git/govoplan
python3 -m venv .venv
./.venv/bin/python -m pip install --upgrade pip
./.venv/bin/python -m pip install -r requirements-dev.txt
```
Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to `access,campaigns,files,mail`; set `ENABLED_MODULES` explicitly when testing a smaller module permutation.
Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to `tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops`; set `ENABLED_MODULES` explicitly when testing a smaller module permutation.
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.devserver \
/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver \
--host 127.0.0.1 \
--port 8000
```
@@ -48,29 +67,69 @@ For example, to test campaign without files or mail:
```bash
cd /mnt/DATA/git/govoplan-core
ENABLED_MODULES=access,campaigns ./.venv/bin/python -m govoplan_core.devserver \
ENABLED_MODULES=access,campaigns /mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver \
--host 127.0.0.1 \
--port 8000
```
The runner loads the same `GovoplanServerConfig` as `govoplan_core.server.app:app`, builds the platform registry, and passes core plus enabled module source roots to uvicorn as reload directories. After reinstalling the editable package, the same command is also available as `govoplan-devserver`.
The default development SQLite database lives at `runtime/multimailer-dev.db`, alongside other local runtime state.
The default development database is PostgreSQL at `postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev`. Store the password in `~/.pgpass`. To force the disposable SQLite fallback, run with `GOVOPLAN_DEV_DATABASE_BACKEND=sqlite`; that database lives below `runtime/`.
Local devserver runs do not require Redis. `CELERY_ENABLED` defaults to `false`, so campaign queue actions update database state without publishing Celery tasks. Use the synchronous send flow for local send tests, or set `CELERY_ENABLED=true` only when a Redis broker and worker are running.
If the configured local SQLite database is missing or empty, `govoplan_core.devserver` enables the development bootstrap before loading settings. This creates the schema and the default development login on startup. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead.
To run the production-like local profile with PostgreSQL, Redis, a Celery
worker, explicit module configuration, and persistent local file storage:
To verify the effective runtime paths and missing-SQLite bootstrap without starting uvicorn, run the smoke mode:
```bash
cd /mnt/DATA/git/govoplan
tools/launch/launch-production-like-dev.sh
```
See `/mnt/DATA/git/govoplan/dev/production-like/README.md` for ports,
environment overrides, and cleanup commands. Core keeps wrapper commands during
the migration, but whole-product profiles are owned by the meta repository.
## Security audit
The repository includes a containerized audit toolbox for SAST, secret scanning,
dependency checks, filesystem misconfiguration scans, duplication, and complexity
reports. See [SECURITY_AUDIT.md](docs/SECURITY_AUDIT.md) for the operating model.
```bash
cd /mnt/DATA/git/govoplan
tools/checks/security-audit/run.sh --mode ci --scope govoplan
tools/checks/security-audit/run.sh --mode full --scope govoplan
```
CI runs the `ci` profile in report-only mode and uploads `audit-reports/` as an
artifact. Once the baseline is clean, set `SECURITY_AUDIT_FAIL_ON_FINDINGS=1`
or pass `--strict` locally to turn findings into a failing gate.
`govoplan_core.devserver` enables the development bootstrap before loading settings. In dev, startup migrations create or upgrade the schema and the bootstrap creates the default development login if needed. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead.
To verify the effective runtime paths and bootstrap behavior without starting uvicorn, run the smoke mode:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
```
The smoke mode prints the effective config, runtime root, database URL, modules, reload state, and bootstrap decision, then creates the ASGI app and runs startup once.
`requirements-dev.txt` links local `govoplan-files`, `govoplan-mail`, and `govoplan-campaign` checkouts for development. `requirements-release.txt` installs those modules from tagged git refs for release builds. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md).
The meta repository owns whole-product `requirements-dev.txt`,
`requirements-release.txt`, and the root `.env.example` operator template. Core
keeps package metadata and runtime commands. See
[RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md).
For the install/runtime configuration contract and operator deployment flow, see [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md).
For self-hosted config bootstrap and validation:
```bash
cd /mnt/DATA/git/govoplan-core
/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.commands.config env-template --profile self-hosted --generate-secrets
/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted
```
## WebUI development

View File

@@ -1,7 +1,8 @@
[alembic]
script_location = alembic
path_separator = os
prepend_sys_path = .
sqlalchemy.url = sqlite:///./runtime/multimailer-dev.db
sqlalchemy.url = sqlite:///./runtime/govoplan-dev.db
[loggers]
keys = root,sqlalchemy,alembic

View File

@@ -0,0 +1,52 @@
"""add audit outbox events to detailed dev track
Revision ID: 0f1e2d3c4b5a
Revises: 4f2a9c8e7b6d
Create Date: 2026-07-11 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "0f1e2d3c4b5a"
down_revision = "4f2a9c8e7b6d"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"audit_outbox_events",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("event_id", sa.String(length=36), nullable=False),
sa.Column("event_type", sa.String(length=200), nullable=False),
sa.Column("module_id", sa.String(length=100), nullable=False),
sa.Column("correlation_id", sa.String(length=128), nullable=True),
sa.Column("causation_id", sa.String(length=128), nullable=True),
sa.Column("classification", sa.String(length=40), nullable=False),
sa.Column("payload", sa.JSON(), nullable=False),
sa.Column("status", sa.String(length=20), nullable=False),
sa.Column("attempts", sa.Integer(), nullable=False),
sa.Column("next_attempt_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("dispatched_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_audit_outbox_events")),
sa.UniqueConstraint("event_id", name="uq_audit_outbox_events_event_id"),
)
op.create_index("ix_audit_outbox_events_correlation_id", "audit_outbox_events", ["correlation_id"], unique=False)
op.create_index("ix_audit_outbox_events_event_type", "audit_outbox_events", ["event_type"], unique=False)
op.create_index(op.f("ix_audit_outbox_events_status"), "audit_outbox_events", ["status"], unique=False)
op.create_index(
"ix_audit_outbox_events_status_next_attempt_at",
"audit_outbox_events",
["status", "next_attempt_at"],
unique=False,
)
def downgrade() -> None:
op.drop_table("audit_outbox_events")

View File

@@ -94,10 +94,10 @@ def _scrub_policy_column(table_name: str) -> None:
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
tables = set(inspector.get_table_names())
for table_name in ("system_settings", "tenants"):
for table_name in ("core_system_settings", "tenancy_tenants"):
if table_name in tables and "settings" in {column["name"] for column in inspector.get_columns(table_name)}:
_scrub_settings_table(table_name)
for table_name in ("users", "groups", "campaigns"):
for table_name in ("access_users", "access_groups", "campaigns"):
if table_name in tables and "mail_profile_policy" in {column["name"] for column in inspector.get_columns(table_name)}:
_scrub_policy_column(table_name)

View File

@@ -30,7 +30,7 @@ def upgrade() -> None:
batch_op.add_column(sa.Column("locked_by_user_id", sa.String(length=36), nullable=True))
batch_op.create_foreign_key(
op.f("fk_campaign_versions_locked_by_user_id_users"),
"users",
"access_users",
["locked_by_user_id"],
["id"],
ondelete="SET NULL",

View File

@@ -0,0 +1,130 @@
"""auth sessions and RBAC assignments
Revision ID: 2c3d4e5f6a7b
Revises: 1f8d4c2a0b7e
Create Date: 2026-06-08 10:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "2c3d4e5f6a7b"
down_revision = "1f8d4c2a0b7e"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("access_users") as batch_op:
batch_op.add_column(sa.Column("auth_provider", sa.String(length=50), nullable=False, server_default="local"))
batch_op.add_column(sa.Column("password_hash", sa.String(length=500), nullable=True))
batch_op.add_column(sa.Column("last_login_at", sa.DateTime(timezone=True), nullable=True))
op.create_table(
"access_user_group_memberships",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("group_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["access_users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["group_id"], ["access_groups.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "user_id", "group_id", name="uq_user_group_memberships"),
)
op.create_index(op.f("ix_access_user_group_memberships_tenant_id"), "access_user_group_memberships", ["tenant_id"])
op.create_index(op.f("ix_access_user_group_memberships_user_id"), "access_user_group_memberships", ["user_id"])
op.create_index(op.f("ix_access_user_group_memberships_group_id"), "access_user_group_memberships", ["group_id"])
op.create_table(
"access_user_role_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("role_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["access_users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["access_roles.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "user_id", "role_id", name="uq_user_role_assignments"),
)
op.create_index(op.f("ix_access_user_role_assignments_tenant_id"), "access_user_role_assignments", ["tenant_id"])
op.create_index(op.f("ix_access_user_role_assignments_user_id"), "access_user_role_assignments", ["user_id"])
op.create_index(op.f("ix_access_user_role_assignments_role_id"), "access_user_role_assignments", ["role_id"])
op.create_table(
"access_group_role_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("group_id", sa.String(length=36), nullable=False),
sa.Column("role_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["group_id"], ["access_groups.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["access_roles.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "group_id", "role_id", name="uq_group_role_assignments"),
)
op.create_index(op.f("ix_access_group_role_assignments_tenant_id"), "access_group_role_assignments", ["tenant_id"])
op.create_index(op.f("ix_access_group_role_assignments_group_id"), "access_group_role_assignments", ["group_id"])
op.create_index(op.f("ix_access_group_role_assignments_role_id"), "access_group_role_assignments", ["role_id"])
op.create_table(
"access_auth_sessions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("token_hash", sa.String(length=128), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("user_agent", sa.String(length=500), nullable=True),
sa.Column("ip_address", sa.String(length=100), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["access_users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("token_hash"),
)
op.create_index(op.f("ix_access_auth_sessions_tenant_id"), "access_auth_sessions", ["tenant_id"])
op.create_index(op.f("ix_access_auth_sessions_user_id"), "access_auth_sessions", ["user_id"])
op.create_index(op.f("ix_access_auth_sessions_token_hash"), "access_auth_sessions", ["token_hash"])
op.create_index(op.f("ix_access_auth_sessions_expires_at"), "access_auth_sessions", ["expires_at"])
op.create_index(op.f("ix_access_auth_sessions_revoked_at"), "access_auth_sessions", ["revoked_at"])
def downgrade() -> None:
op.drop_index(op.f("ix_access_auth_sessions_revoked_at"), table_name="access_auth_sessions")
op.drop_index(op.f("ix_access_auth_sessions_expires_at"), table_name="access_auth_sessions")
op.drop_index(op.f("ix_access_auth_sessions_token_hash"), table_name="access_auth_sessions")
op.drop_index(op.f("ix_access_auth_sessions_user_id"), table_name="access_auth_sessions")
op.drop_index(op.f("ix_access_auth_sessions_tenant_id"), table_name="access_auth_sessions")
op.drop_table("access_auth_sessions")
op.drop_index(op.f("ix_access_group_role_assignments_role_id"), table_name="access_group_role_assignments")
op.drop_index(op.f("ix_access_group_role_assignments_group_id"), table_name="access_group_role_assignments")
op.drop_index(op.f("ix_access_group_role_assignments_tenant_id"), table_name="access_group_role_assignments")
op.drop_table("access_group_role_assignments")
op.drop_index(op.f("ix_access_user_role_assignments_role_id"), table_name="access_user_role_assignments")
op.drop_index(op.f("ix_access_user_role_assignments_user_id"), table_name="access_user_role_assignments")
op.drop_index(op.f("ix_access_user_role_assignments_tenant_id"), table_name="access_user_role_assignments")
op.drop_table("access_user_role_assignments")
op.drop_index(op.f("ix_access_user_group_memberships_group_id"), table_name="access_user_group_memberships")
op.drop_index(op.f("ix_access_user_group_memberships_user_id"), table_name="access_user_group_memberships")
op.drop_index(op.f("ix_access_user_group_memberships_tenant_id"), table_name="access_user_group_memberships")
op.drop_table("access_user_group_memberships")
with op.batch_alter_table("access_users") as batch_op:
batch_op.drop_column("last_login_at")
batch_op.drop_column("password_hash")
batch_op.drop_column("auth_provider")

View File

@@ -0,0 +1,82 @@
"""namespace platform-owned tables
Revision ID: 2e3f4a5b6c7d
Revises: 1b2c3d4e5f70
Create Date: 2026-07-09 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "2e3f4a5b6c7d"
down_revision = "1b2c3d4e5f70"
branch_labels = None
depends_on = None
_TABLE_RENAMES = (
("tenants", "tenancy_tenants"),
("accounts", "access_accounts"),
("users", "access_users"),
("groups", "access_groups"),
("roles", "access_roles"),
("system_role_assignments", "access_system_role_assignments"),
("user_group_memberships", "access_user_group_memberships"),
("user_role_assignments", "access_user_role_assignments"),
("group_role_assignments", "access_group_role_assignments"),
("api_keys", "access_api_keys"),
("auth_sessions", "access_auth_sessions"),
("system_settings", "core_system_settings"),
("governance_templates", "admin_governance_templates"),
("governance_template_assignments", "admin_governance_template_assignments"),
)
_KNOWN_TABLE_NAMES = {name for pair in _TABLE_RENAMES for name in pair}
def _row_count(bind: sa.Connection, table_name: str) -> int:
if table_name not in _KNOWN_TABLE_NAMES:
raise RuntimeError(f"Unexpected table name: {table_name}")
quoted = bind.dialect.identifier_preparer.quote(table_name)
return int(bind.execute(sa.text(f"SELECT COUNT(*) FROM {quoted}")).scalar_one()) # nosemgrep: python.sqlalchemy.security.audit.avoid-sqlalchemy-text.avoid-sqlalchemy-text
def _rename_tables(renames: tuple[tuple[str, str], ...]) -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
old_tables_to_drop: set[str] = set()
new_tables_to_drop: set[str] = set()
for old_name, new_name in renames:
if old_name not in tables or new_name not in tables:
continue
if _row_count(bind, old_name) == 0:
old_tables_to_drop.add(old_name)
elif _row_count(bind, new_name) == 0:
new_tables_to_drop.add(new_name)
else:
raise RuntimeError(f"Cannot rename non-empty {old_name} over non-empty {new_name}")
for old_name, _new_name in reversed(renames):
if old_name in old_tables_to_drop:
op.drop_table(old_name)
tables.remove(old_name)
for _old_name, new_name in reversed(renames):
if new_name in new_tables_to_drop:
op.drop_table(new_name)
tables.remove(new_name)
for old_name, new_name in renames:
if old_name not in tables:
continue
op.rename_table(old_name, new_name)
tables.remove(old_name)
tables.add(new_name)
def upgrade() -> None:
_rename_tables(_TABLE_RENAMES)
def downgrade() -> None:
_rename_tables(tuple((new_name, old_name) for old_name, new_name in reversed(_TABLE_RENAMES)))

View File

@@ -32,7 +32,7 @@ def upgrade() -> None:
sa.Column("retained_until", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "checksum_sha256", "size_bytes", name="uq_file_blobs_tenant_checksum_size"),
)
@@ -55,10 +55,10 @@ def upgrade() -> None:
sa.Column("metadata", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_group_id"], ["groups.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_group_id"], ["access_groups.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
for col in ["tenant_id", "owner_type", "owner_user_id", "owner_group_id", "current_version_id", "display_path", "filename", "created_by_user_id", "deleted_at"]:
@@ -80,9 +80,9 @@ def upgrade() -> None:
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["blob_id"], ["file_blobs.id"], ondelete="RESTRICT"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["file_asset_id"], ["file_assets.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("file_asset_id", "version_number", name="uq_file_versions_asset_number"),
)
@@ -101,9 +101,9 @@ def upgrade() -> None:
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["file_asset_id"], ["file_assets.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("file_asset_id", "target_type", "target_id", "revoked_at", name="uq_file_shares_active_target"),
)
@@ -136,7 +136,7 @@ def upgrade() -> None:
sa.ForeignKeyConstraint(["file_asset_id"], ["file_assets.id"], ondelete="RESTRICT"),
sa.ForeignKeyConstraint(["file_blob_id"], ["file_blobs.id"], ondelete="RESTRICT"),
sa.ForeignKeyConstraint(["file_version_id"], ["file_versions.id"], ondelete="RESTRICT"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("campaign_job_id", "file_version_id", "filename_used", "use_stage", name="uq_campaign_attachment_uses_job_file_stage"),
)

View File

@@ -0,0 +1,111 @@
"""add core change sequence
Revision ID: 3f4a5b6c7d8e
Revises: 2e3f4a5b6c7d
Create Date: 2026-07-09 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "3f4a5b6c7d8e"
down_revision = "2e3f4a5b6c7d"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if "core_change_sequence" not in inspector.get_table_names():
op.create_table(
"core_change_sequence",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), autoincrement=True, nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=True),
sa.Column("module_id", sa.String(length=100), nullable=False),
sa.Column("collection", sa.String(length=150), nullable=False),
sa.Column("resource_type", sa.String(length=100), nullable=False),
sa.Column("resource_id", sa.String(length=255), nullable=False),
sa.Column("operation", sa.String(length=30), nullable=False),
sa.Column("actor_type", sa.String(length=30), nullable=True),
sa.Column("actor_id", sa.String(length=255), nullable=True),
sa.Column("payload", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_core_change_sequence")),
)
inspector = sa.inspect(op.get_bind())
indexes = {item["name"] for item in inspector.get_indexes("core_change_sequence")}
for column in (
"tenant_id",
"module_id",
"collection",
"resource_type",
"resource_id",
"operation",
"actor_type",
"actor_id",
"created_at",
):
name = op.f(f"ix_core_change_sequence_{column}")
if name not in indexes:
op.create_index(name, "core_change_sequence", [column], unique=False)
for name, columns in (
("ix_core_change_sequence_tenant_module_id", ["tenant_id", "module_id", "id"]),
("ix_core_change_sequence_collection_id", ["collection", "id"]),
("ix_core_change_sequence_resource", ["module_id", "resource_type", "resource_id"]),
):
if name not in indexes:
op.create_index(name, "core_change_sequence", columns, unique=False)
inspector = sa.inspect(op.get_bind())
if "core_change_sequence_retention_floor" not in inspector.get_table_names():
op.create_table(
"core_change_sequence_retention_floor",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), autoincrement=True, nullable=False),
sa.Column("tenant_key", sa.String(length=36), nullable=False),
sa.Column("module_id", sa.String(length=100), nullable=False),
sa.Column("collection", sa.String(length=150), nullable=False),
sa.Column("min_valid_sequence", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_core_change_sequence_retention_floor")),
sa.UniqueConstraint("tenant_key", "module_id", "collection", name="uq_core_change_sequence_retention_scope"),
)
indexes = {item["name"] for item in inspector.get_indexes("core_change_sequence_retention_floor")}
if "ix_core_change_sequence_retention_scope" not in indexes:
op.create_index(
"ix_core_change_sequence_retention_scope",
"core_change_sequence_retention_floor",
["tenant_key", "module_id", "collection"],
unique=False,
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if "core_change_sequence_retention_floor" in inspector.get_table_names():
indexes = {item["name"] for item in inspector.get_indexes("core_change_sequence_retention_floor")}
if "ix_core_change_sequence_retention_scope" in indexes:
op.drop_index("ix_core_change_sequence_retention_scope", table_name="core_change_sequence_retention_floor")
op.drop_table("core_change_sequence_retention_floor")
if "core_change_sequence" not in inspector.get_table_names():
return
indexes = {item["name"] for item in inspector.get_indexes("core_change_sequence")}
for name in (
"ix_core_change_sequence_resource",
"ix_core_change_sequence_collection_id",
"ix_core_change_sequence_tenant_module_id",
op.f("ix_core_change_sequence_created_at"),
op.f("ix_core_change_sequence_actor_id"),
op.f("ix_core_change_sequence_actor_type"),
op.f("ix_core_change_sequence_operation"),
op.f("ix_core_change_sequence_resource_id"),
op.f("ix_core_change_sequence_resource_type"),
op.f("ix_core_change_sequence_collection"),
op.f("ix_core_change_sequence_module_id"),
op.f("ix_core_change_sequence_tenant_id"),
):
if name in indexes:
op.drop_index(name, table_name="core_change_sequence")
op.drop_table("core_change_sequence")

View File

@@ -31,10 +31,10 @@ def upgrade() -> None:
sa.Column("metadata", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_group_id"], ["groups.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_group_id"], ["access_groups.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["owner_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
for col in ["tenant_id", "owner_type", "owner_user_id", "owner_group_id", "path", "created_by_user_id", "deleted_at"]:

View File

@@ -0,0 +1,79 @@
"""rename tenancy scope table to core scopes
Revision ID: 4f2a9c8e7b6d
Revises: 3f4a5b6c7d8e
Create Date: 2026-07-10 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "4f2a9c8e7b6d"
down_revision = "3f4a5b6c7d8e"
branch_labels = None
depends_on = None
LEGACY_SCOPE_TABLE = "tenancy_tenants"
CORE_SCOPE_TABLE = "core_scopes"
LEGACY_SLUG_INDEX = "ix_tenancy_tenants_slug"
CORE_SLUG_INDEX = "ix_core_scopes_slug"
_KNOWN_SCOPE_TABLES = {LEGACY_SCOPE_TABLE, CORE_SCOPE_TABLE}
def _row_count(bind: sa.Connection, table_name: str) -> int:
if table_name not in _KNOWN_SCOPE_TABLES:
raise RuntimeError(f"Unexpected scope table name: {table_name}")
quoted = bind.dialect.identifier_preparer.quote(table_name)
return int(bind.execute(sa.text(f"SELECT COUNT(*) FROM {quoted}")).scalar_one()) # nosemgrep: python.sqlalchemy.security.audit.avoid-sqlalchemy-text.avoid-sqlalchemy-text
def _scope_tables(bind: sa.Connection) -> set[str]:
return set(sa.inspect(bind).get_table_names())
def _drop_table_if_empty(bind: sa.Connection, table_name: str) -> bool:
if _row_count(bind, table_name) != 0:
return False
op.drop_table(table_name)
return True
def _ensure_slug_index(bind: sa.Connection, table_name: str, index_name: str, old_index_name: str) -> None:
indexes = {index["name"] for index in sa.inspect(bind).get_indexes(table_name)}
if old_index_name in indexes:
op.drop_index(old_index_name, table_name=table_name)
indexes.remove(old_index_name)
if index_name not in indexes:
op.create_index(op.f(index_name), table_name, ["slug"], unique=True)
def _rename_scope_table(old_name: str, new_name: str) -> None:
bind = op.get_bind()
tables = _scope_tables(bind)
if old_name not in tables:
if new_name in tables:
_ensure_slug_index(bind, new_name, CORE_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else LEGACY_SLUG_INDEX, LEGACY_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else CORE_SLUG_INDEX)
return
if new_name in tables:
if _drop_table_if_empty(bind, new_name):
tables.remove(new_name)
elif _drop_table_if_empty(bind, old_name):
_ensure_slug_index(bind, new_name, CORE_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else LEGACY_SLUG_INDEX, LEGACY_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else CORE_SLUG_INDEX)
return
else:
raise RuntimeError(f"Cannot reconcile non-empty {old_name} over non-empty {new_name}")
if old_name in tables and new_name not in tables:
op.rename_table(old_name, new_name)
_ensure_slug_index(bind, new_name, CORE_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else LEGACY_SLUG_INDEX, LEGACY_SLUG_INDEX if new_name == CORE_SCOPE_TABLE else CORE_SLUG_INDEX)
def upgrade() -> None:
_rename_scope_table(LEGACY_SCOPE_TABLE, CORE_SCOPE_TABLE)
def downgrade() -> None:
_rename_scope_table(CORE_SCOPE_TABLE, LEGACY_SCOPE_TABLE)

View File

@@ -24,7 +24,7 @@ def upgrade() -> None:
batch_op.add_column(sa.Column("user_locked_by_user_id", sa.String(length=36), nullable=True))
batch_op.create_foreign_key(
"fk_campaign_versions_user_locked_by_user_id_users",
"users",
"access_users",
["user_locked_by_user_id"],
["id"],
ondelete="SET NULL",

View File

@@ -62,25 +62,25 @@ def upgrade() -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
user_columns = {column["name"] for column in inspector.get_columns("users")}
user_columns = {column["name"] for column in inspector.get_columns("access_users")}
# Base.metadata.create_all() from a newer application can create brand-new
# tables while leaving existing tables unaltered. Repair that known drift by
# removing only empty, unreferenced administration tables before applying the
# real migration. A non-empty table is never guessed at or discarded.
if "account_id" not in user_columns and "system_role_assignments" in tables:
count = bind.execute(sa.text("SELECT COUNT(*) FROM system_role_assignments")).scalar_one()
if "account_id" not in user_columns and "access_system_role_assignments" in tables:
count = bind.execute(sa.text("SELECT COUNT(*) FROM access_system_role_assignments")).scalar_one()
if count:
raise RuntimeError("Cannot reconcile non-empty create_all system_role_assignments table")
op.drop_table("system_role_assignments")
tables.remove("system_role_assignments")
if "account_id" not in user_columns and "accounts" in tables:
count = bind.execute(sa.text("SELECT COUNT(*) FROM accounts")).scalar_one()
op.drop_table("access_system_role_assignments")
tables.remove("access_system_role_assignments")
if "account_id" not in user_columns and "access_accounts" in tables:
count = bind.execute(sa.text("SELECT COUNT(*) FROM access_accounts")).scalar_one()
if count:
raise RuntimeError("Cannot reconcile non-empty create_all accounts table")
op.drop_table("accounts")
op.drop_table("access_accounts")
op.create_table(
"accounts",
"access_accounts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("email", sa.String(length=320), nullable=False),
sa.Column("normalized_email", sa.String(length=320), nullable=False),
@@ -95,29 +95,29 @@ def upgrade() -> None:
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("normalized_email", name="uq_accounts_normalized_email"),
)
op.create_index(op.f("ix_accounts_normalized_email"), "accounts", ["normalized_email"])
op.create_index(op.f("ix_access_accounts_normalized_email"), "access_accounts", ["normalized_email"])
with op.batch_alter_table("tenants") as batch_op:
with op.batch_alter_table("tenancy_tenants") as batch_op:
batch_op.add_column(sa.Column("description", sa.Text(), nullable=True))
batch_op.add_column(sa.Column("default_locale", sa.String(length=20), nullable=False, server_default="en"))
batch_op.add_column(sa.Column("settings", sa.JSON(), nullable=False, server_default="{}"))
with op.batch_alter_table("groups") as batch_op:
with op.batch_alter_table("access_groups") as batch_op:
batch_op.add_column(sa.Column("description", sa.Text(), nullable=True))
batch_op.add_column(sa.Column("is_active", sa.Boolean(), nullable=False, server_default=sa.true()))
with op.batch_alter_table("roles") as batch_op:
with op.batch_alter_table("access_roles") as batch_op:
batch_op.add_column(sa.Column("description", sa.Text(), nullable=True))
batch_op.add_column(sa.Column("is_builtin", sa.Boolean(), nullable=False, server_default=sa.false()))
batch_op.add_column(sa.Column("is_assignable", sa.Boolean(), nullable=False, server_default=sa.true()))
with op.batch_alter_table("users") as batch_op:
with op.batch_alter_table("access_users") as batch_op:
batch_op.add_column(sa.Column("account_id", sa.String(length=36), nullable=True))
with op.batch_alter_table("auth_sessions") as batch_op:
with op.batch_alter_table("access_auth_sessions") as batch_op:
batch_op.add_column(sa.Column("account_id", sa.String(length=36), nullable=True))
users = bind.execute(sa.text(
"SELECT id, tenant_id, email, display_name, is_active, is_tenant_admin, auth_provider, password_hash, last_login_at, created_at, updated_at FROM users ORDER BY created_at, id"
"SELECT id, tenant_id, email, display_name, is_active, is_tenant_admin, auth_provider, password_hash, last_login_at, created_at, updated_at FROM access_users ORDER BY created_at, id"
)).mappings().all()
accounts_by_email: dict[str, str] = {}
@@ -155,7 +155,7 @@ def upgrade() -> None:
first_system_owner_account_id = account_id
accounts_table = sa.table(
"accounts",
"access_accounts",
sa.column("id", sa.String), sa.column("email", sa.String), sa.column("normalized_email", sa.String),
sa.column("display_name", sa.String), sa.column("is_active", sa.Boolean), sa.column("auth_provider", sa.String),
sa.column("password_hash", sa.String), sa.column("password_reset_required", sa.Boolean),
@@ -166,54 +166,54 @@ def upgrade() -> None:
bind.execute(accounts_table.insert(), list(account_rows.values()))
for row in users:
bind.execute(
sa.text("UPDATE users SET account_id = :account_id WHERE id = :user_id"),
sa.text("UPDATE access_users SET account_id = :account_id WHERE id = :user_id"),
{"account_id": accounts_by_email[_normalize_email(row["email"])], "user_id": row["id"]},
)
bind.execute(sa.text(
"UPDATE auth_sessions SET account_id = (SELECT users.account_id FROM users WHERE users.id = auth_sessions.user_id)"
"UPDATE access_auth_sessions SET account_id = (SELECT access_users.account_id FROM access_users WHERE access_users.id = access_auth_sessions.user_id)"
))
with op.batch_alter_table("users") as batch_op:
with op.batch_alter_table("access_users") as batch_op:
batch_op.alter_column("account_id", existing_type=sa.String(length=36), nullable=False)
batch_op.create_foreign_key("fk_users_account_id_accounts", "accounts", ["account_id"], ["id"], ondelete="CASCADE")
batch_op.create_foreign_key("fk_users_account_id_accounts", "access_accounts", ["account_id"], ["id"], ondelete="CASCADE")
batch_op.create_unique_constraint("uq_users_tenant_account", ["tenant_id", "account_id"])
op.create_index(op.f("ix_users_account_id"), "users", ["account_id"])
op.create_index(op.f("ix_access_users_account_id"), "access_users", ["account_id"])
with op.batch_alter_table("auth_sessions") as batch_op:
with op.batch_alter_table("access_auth_sessions") as batch_op:
batch_op.alter_column("account_id", existing_type=sa.String(length=36), nullable=False)
batch_op.create_foreign_key("fk_auth_sessions_account_id_accounts", "accounts", ["account_id"], ["id"], ondelete="CASCADE")
op.create_index(op.f("ix_auth_sessions_account_id"), "auth_sessions", ["account_id"])
batch_op.create_foreign_key("fk_auth_sessions_account_id_accounts", "access_accounts", ["account_id"], ["id"], ondelete="CASCADE")
op.create_index(op.f("ix_access_auth_sessions_account_id"), "access_auth_sessions", ["account_id"])
op.create_table(
"system_role_assignments",
"access_system_role_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("account_id", sa.String(length=36), nullable=False),
sa.Column("role_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["account_id"], ["accounts.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["roles.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["account_id"], ["access_accounts.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["access_roles.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("account_id", "role_id", name="uq_system_role_assignments"),
)
op.create_index(op.f("ix_system_role_assignments_account_id"), "system_role_assignments", ["account_id"])
op.create_index(op.f("ix_system_role_assignments_role_id"), "system_role_assignments", ["role_id"])
op.create_index(op.f("ix_access_system_role_assignments_account_id"), "access_system_role_assignments", ["account_id"])
op.create_index(op.f("ix_access_system_role_assignments_role_id"), "access_system_role_assignments", ["role_id"])
roles_table = sa.table(
"roles",
"access_roles",
sa.column("id", sa.String), sa.column("tenant_id", sa.String), sa.column("slug", sa.String),
sa.column("name", sa.String), sa.column("description", sa.Text), sa.column("permissions", sa.JSON),
sa.column("is_builtin", sa.Boolean), sa.column("is_assignable", sa.Boolean),
sa.column("created_at", sa.DateTime(timezone=True)), sa.column("updated_at", sa.DateTime(timezone=True)),
)
now = _now()
tenant_ids = [row[0] for row in bind.execute(sa.text("SELECT id FROM tenants")).all()]
tenant_ids = [row[0] for row in bind.execute(sa.text("SELECT id FROM tenancy_tenants")).all()]
definitions = _role_definitions()
tenant_role_ids: dict[tuple[str, str], str] = {}
for tenant_id in tenant_ids:
for slug, definition in definitions.items():
existing = bind.execute(
sa.text("SELECT id FROM roles WHERE tenant_id = :tenant_id AND slug = :slug"),
sa.text("SELECT id FROM access_roles WHERE tenant_id = :tenant_id AND slug = :slug"),
{"tenant_id": tenant_id, "slug": slug},
).scalar_one_or_none()
if existing:
@@ -253,7 +253,7 @@ def upgrade() -> None:
op.create_index(
"uq_roles_system_slug",
"roles",
"access_roles",
["slug"],
unique=True,
sqlite_where=sa.text("tenant_id IS NULL"),
@@ -267,11 +267,11 @@ def upgrade() -> None:
continue
owner_role_id = tenant_role_ids[(row["tenant_id"], "owner")]
exists = bind.execute(sa.text(
"SELECT 1 FROM user_role_assignments WHERE tenant_id = :tenant_id AND user_id = :user_id AND role_id = :role_id"
"SELECT 1 FROM access_user_role_assignments WHERE tenant_id = :tenant_id AND user_id = :user_id AND role_id = :role_id"
), {"tenant_id": row["tenant_id"], "user_id": row["id"], "role_id": owner_role_id}).first()
if not exists:
bind.execute(sa.text(
"INSERT INTO user_role_assignments (id, tenant_id, user_id, role_id, created_at, updated_at) VALUES (:id, :tenant_id, :user_id, :role_id, :created_at, :updated_at)"
"INSERT INTO access_user_role_assignments (id, tenant_id, user_id, role_id, created_at, updated_at) VALUES (:id, :tenant_id, :user_id, :role_id, :created_at, :updated_at)"
), {"id": str(uuid.uuid4()), "tenant_id": row["tenant_id"], "user_id": row["id"], "role_id": owner_role_id, "created_at": now, "updated_at": now})
# Bootstrap rule for existing installations: the earliest active legacy
@@ -284,40 +284,40 @@ def upgrade() -> None:
), None)
if first_system_owner_account_id:
bind.execute(sa.text(
"INSERT INTO system_role_assignments (id, account_id, role_id, created_at, updated_at) VALUES (:id, :account_id, :role_id, :created_at, :updated_at)"
"INSERT INTO access_system_role_assignments (id, account_id, role_id, created_at, updated_at) VALUES (:id, :account_id, :role_id, :created_at, :updated_at)"
), {"id": str(uuid.uuid4()), "account_id": first_system_owner_account_id, "role_id": system_owner_role_id, "created_at": now, "updated_at": now})
def downgrade() -> None:
op.drop_index("uq_roles_system_slug", table_name="roles")
op.drop_index(op.f("ix_system_role_assignments_role_id"), table_name="system_role_assignments")
op.drop_index(op.f("ix_system_role_assignments_account_id"), table_name="system_role_assignments")
op.drop_table("system_role_assignments")
op.drop_index("uq_roles_system_slug", table_name="access_roles")
op.drop_index(op.f("ix_access_system_role_assignments_role_id"), table_name="access_system_role_assignments")
op.drop_index(op.f("ix_access_system_role_assignments_account_id"), table_name="access_system_role_assignments")
op.drop_table("access_system_role_assignments")
op.drop_index(op.f("ix_auth_sessions_account_id"), table_name="auth_sessions")
with op.batch_alter_table("auth_sessions") as batch_op:
op.drop_index(op.f("ix_access_auth_sessions_account_id"), table_name="access_auth_sessions")
with op.batch_alter_table("access_auth_sessions") as batch_op:
batch_op.drop_constraint("fk_auth_sessions_account_id_accounts", type_="foreignkey")
batch_op.drop_column("account_id")
op.drop_index(op.f("ix_users_account_id"), table_name="users")
with op.batch_alter_table("users") as batch_op:
op.drop_index(op.f("ix_access_users_account_id"), table_name="access_users")
with op.batch_alter_table("access_users") as batch_op:
batch_op.drop_constraint("uq_users_tenant_account", type_="unique")
batch_op.drop_constraint("fk_users_account_id_accounts", type_="foreignkey")
batch_op.drop_column("account_id")
with op.batch_alter_table("roles") as batch_op:
with op.batch_alter_table("access_roles") as batch_op:
batch_op.drop_column("is_assignable")
batch_op.drop_column("is_builtin")
batch_op.drop_column("description")
with op.batch_alter_table("groups") as batch_op:
with op.batch_alter_table("access_groups") as batch_op:
batch_op.drop_column("is_active")
batch_op.drop_column("description")
with op.batch_alter_table("tenants") as batch_op:
with op.batch_alter_table("tenancy_tenants") as batch_op:
batch_op.drop_column("settings")
batch_op.drop_column("default_locale")
batch_op.drop_column("description")
op.drop_index(op.f("ix_accounts_normalized_email"), table_name="accounts")
op.drop_table("accounts")
op.drop_index(op.f("ix_access_accounts_normalized_email"), table_name="access_accounts")
op.drop_table("access_accounts")

View File

@@ -16,6 +16,7 @@ revision = "9d0e1f2a3b4c"
down_revision = "8c9d0e1f2a3b"
branch_labels = None
depends_on = None
_RECONCILE_CREATE_ALL_TABLES = ("admin_governance_template_assignments", "admin_governance_templates", "core_system_settings")
def _now() -> datetime:
@@ -28,15 +29,16 @@ def upgrade() -> None:
tables = set(inspector.get_table_names())
# Reconcile only the empty create_all shape for the newly introduced tables.
for table_name in ("governance_template_assignments", "governance_templates", "system_settings"):
for table_name in _RECONCILE_CREATE_ALL_TABLES:
if table_name in tables:
count = bind.execute(sa.text(f"SELECT COUNT(*) FROM {table_name}")).scalar_one()
quoted = bind.dialect.identifier_preparer.quote(table_name)
count = bind.execute(sa.text(f"SELECT COUNT(*) FROM {quoted}")).scalar_one() # nosemgrep: python.sqlalchemy.security.audit.avoid-sqlalchemy-text.avoid-sqlalchemy-text
if count:
raise RuntimeError(f"Cannot reconcile non-empty create_all table {table_name}")
op.drop_table(table_name)
op.create_table(
"system_settings",
"core_system_settings",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("default_locale", sa.String(length=20), nullable=False, server_default="en"),
sa.Column("allow_tenant_custom_groups", sa.Boolean(), nullable=False, server_default=sa.true()),
@@ -50,16 +52,23 @@ def upgrade() -> None:
now = _now()
bind.execute(sa.text(
"""
INSERT INTO system_settings
INSERT INTO core_system_settings
(id, default_locale, allow_tenant_custom_groups, allow_tenant_custom_roles,
allow_tenant_api_keys, settings, created_at, updated_at)
VALUES
('global', 'en', 1, 1, 1, '{}', :created_at, :updated_at)
('global', 'en', :allow_tenant_custom_groups, :allow_tenant_custom_roles,
:allow_tenant_api_keys, '{}', :created_at, :updated_at)
"""
), {"created_at": now, "updated_at": now})
), {
"allow_tenant_custom_groups": True,
"allow_tenant_custom_roles": True,
"allow_tenant_api_keys": True,
"created_at": now,
"updated_at": now,
})
op.create_table(
"governance_templates",
"admin_governance_templates",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("kind", sa.String(length=20), nullable=False),
sa.Column("slug", sa.String(length=100), nullable=False),
@@ -72,51 +81,51 @@ def upgrade() -> None:
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("kind", "slug", name="uq_governance_templates_kind_slug"),
)
op.create_index(op.f("ix_governance_templates_kind"), "governance_templates", ["kind"])
op.create_index(op.f("ix_admin_governance_templates_kind"), "admin_governance_templates", ["kind"])
op.create_table(
"governance_template_assignments",
"admin_governance_template_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("template_id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("mode", sa.String(length=20), nullable=False, server_default="available"),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["template_id"], ["governance_templates.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["template_id"], ["admin_governance_templates.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("template_id", "tenant_id", name="uq_governance_template_tenant"),
)
op.create_index(op.f("ix_governance_template_assignments_template_id"), "governance_template_assignments", ["template_id"])
op.create_index(op.f("ix_governance_template_assignments_tenant_id"), "governance_template_assignments", ["tenant_id"])
op.create_index(op.f("ix_admin_governance_template_assignments_template_id"), "admin_governance_template_assignments", ["template_id"])
op.create_index(op.f("ix_admin_governance_template_assignments_tenant_id"), "admin_governance_template_assignments", ["tenant_id"])
with op.batch_alter_table("tenants") as batch_op:
with op.batch_alter_table("tenancy_tenants") as batch_op:
batch_op.add_column(sa.Column("allow_custom_groups", sa.Boolean(), nullable=True))
batch_op.add_column(sa.Column("allow_custom_roles", sa.Boolean(), nullable=True))
batch_op.add_column(sa.Column("allow_api_keys", sa.Boolean(), nullable=True))
with op.batch_alter_table("groups") as batch_op:
with op.batch_alter_table("access_groups") as batch_op:
batch_op.add_column(sa.Column("system_template_id", sa.String(length=36), nullable=True))
batch_op.add_column(sa.Column("system_required", sa.Boolean(), nullable=False, server_default=sa.false()))
batch_op.create_foreign_key(
"fk_groups_system_template_id_governance_templates",
"governance_templates", ["system_template_id"], ["id"], ondelete="SET NULL",
"admin_governance_templates", ["system_template_id"], ["id"], ondelete="SET NULL",
)
op.create_index(op.f("ix_groups_system_template_id"), "groups", ["system_template_id"])
op.create_index(op.f("ix_access_groups_system_template_id"), "access_groups", ["system_template_id"])
with op.batch_alter_table("roles") as batch_op:
with op.batch_alter_table("access_roles") as batch_op:
batch_op.add_column(sa.Column("system_template_id", sa.String(length=36), nullable=True))
batch_op.add_column(sa.Column("system_required", sa.Boolean(), nullable=False, server_default=sa.false()))
batch_op.create_foreign_key(
"fk_roles_system_template_id_governance_templates",
"governance_templates", ["system_template_id"], ["id"], ondelete="SET NULL",
"admin_governance_templates", ["system_template_id"], ["id"], ondelete="SET NULL",
)
op.create_index(op.f("ix_roles_system_template_id"), "roles", ["system_template_id"])
op.create_index(op.f("ix_access_roles_system_template_id"), "access_roles", ["system_template_id"])
# Existing system owners use system:* and need no data change. Extend the
# read-only built-in auditor role to the newly introduced read scopes.
auditor = bind.execute(sa.text(
"SELECT id, permissions FROM roles WHERE tenant_id IS NULL AND slug = 'system_auditor'"
"SELECT id, permissions FROM access_roles WHERE tenant_id IS NULL AND slug = 'system_auditor'"
)).mappings().first()
if auditor:
raw_permissions = auditor["permissions"] or []
@@ -125,7 +134,7 @@ def upgrade() -> None:
if scope not in permissions:
permissions.append(scope)
roles_table = sa.table(
"roles",
"access_roles",
sa.column("id", sa.String),
sa.column("permissions", sa.JSON),
sa.column("updated_at", sa.DateTime(timezone=True)),
@@ -138,26 +147,26 @@ def upgrade() -> None:
def downgrade() -> None:
op.drop_index(op.f("ix_roles_system_template_id"), table_name="roles")
with op.batch_alter_table("roles") as batch_op:
op.drop_index(op.f("ix_access_roles_system_template_id"), table_name="access_roles")
with op.batch_alter_table("access_roles") as batch_op:
batch_op.drop_constraint("fk_roles_system_template_id_governance_templates", type_="foreignkey")
batch_op.drop_column("system_required")
batch_op.drop_column("system_template_id")
op.drop_index(op.f("ix_groups_system_template_id"), table_name="groups")
with op.batch_alter_table("groups") as batch_op:
op.drop_index(op.f("ix_access_groups_system_template_id"), table_name="access_groups")
with op.batch_alter_table("access_groups") as batch_op:
batch_op.drop_constraint("fk_groups_system_template_id_governance_templates", type_="foreignkey")
batch_op.drop_column("system_required")
batch_op.drop_column("system_template_id")
with op.batch_alter_table("tenants") as batch_op:
with op.batch_alter_table("tenancy_tenants") as batch_op:
batch_op.drop_column("allow_api_keys")
batch_op.drop_column("allow_custom_roles")
batch_op.drop_column("allow_custom_groups")
op.drop_index(op.f("ix_governance_template_assignments_tenant_id"), table_name="governance_template_assignments")
op.drop_index(op.f("ix_governance_template_assignments_template_id"), table_name="governance_template_assignments")
op.drop_table("governance_template_assignments")
op.drop_index(op.f("ix_governance_templates_kind"), table_name="governance_templates")
op.drop_table("governance_templates")
op.drop_table("system_settings")
op.drop_index(op.f("ix_admin_governance_template_assignments_tenant_id"), table_name="admin_governance_template_assignments")
op.drop_index(op.f("ix_admin_governance_template_assignments_template_id"), table_name="admin_governance_template_assignments")
op.drop_table("admin_governance_template_assignments")
op.drop_index(op.f("ix_admin_governance_templates_kind"), table_name="admin_governance_templates")
op.drop_table("admin_governance_templates")
op.drop_table("core_system_settings")

View File

@@ -148,9 +148,9 @@ def upgrade() -> None:
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["campaign_id"], ["campaigns.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("campaign_id", "target_type", "target_id", name="uq_campaign_share_target"),
)
@@ -160,32 +160,32 @@ def upgrade() -> None:
op.create_index("ix_campaign_shares_target_id", "campaign_shares", ["target_id"])
op.create_index("ix_campaign_shares_created_by_user_id", "campaign_shares", ["created_by_user_id"])
op.create_index("ix_campaign_shares_revoked_at", "campaign_shares", ["revoked_at"])
if "roles" not in tables:
if "access_roles" not in tables:
return
rows = bind.execute(sa.text("SELECT id, permissions FROM roles")).mappings().all()
rows = bind.execute(sa.text("SELECT id, permissions FROM access_roles")).mappings().all()
for row in rows:
bind.execute(
sa.text("UPDATE roles SET permissions = :permissions WHERE id = :id"),
sa.text("UPDATE access_roles SET permissions = :permissions WHERE id = :id"),
{"id": row["id"], "permissions": _json(_expand_legacy(_decode(row["permissions"])))},
)
for slug, permissions in TENANT_ROLE_PERMISSIONS.items():
bind.execute(
sa.text("UPDATE roles SET permissions = :permissions WHERE tenant_id IS NOT NULL AND slug = :slug AND is_builtin = TRUE"),
sa.text("UPDATE access_roles SET permissions = :permissions WHERE tenant_id IS NOT NULL AND slug = :slug AND is_builtin = TRUE"),
{"slug": slug, "permissions": _json(permissions)},
)
now = datetime.now(timezone.utc)
for slug, permissions in SYSTEM_ROLE_PERMISSIONS.items():
existing = bind.execute(
sa.text("SELECT id FROM roles WHERE tenant_id IS NULL AND slug = :slug"), {"slug": slug}
sa.text("SELECT id FROM access_roles WHERE tenant_id IS NULL AND slug = :slug"), {"slug": slug}
).first()
is_protected = slug == "system_owner"
if existing:
bind.execute(
sa.text(
"UPDATE roles SET permissions = :permissions, is_builtin = :is_builtin, is_assignable = TRUE "
"UPDATE access_roles SET permissions = :permissions, is_builtin = :is_builtin, is_assignable = TRUE "
"WHERE tenant_id IS NULL AND slug = :slug"
),
{"slug": slug, "permissions": _json(permissions), "is_builtin": is_protected},
@@ -199,7 +199,7 @@ def upgrade() -> None:
name, description = names[slug]
bind.execute(
sa.text(
"INSERT INTO roles (id, tenant_id, slug, name, description, permissions, is_builtin, is_assignable, "
"INSERT INTO access_roles (id, tenant_id, slug, name, description, permissions, is_builtin, is_assignable, "
"system_template_id, system_required, created_at, updated_at) "
"VALUES (:id, NULL, :slug, :name, :description, :permissions, :is_builtin, TRUE, NULL, FALSE, :created_at, :updated_at)"
),

View File

@@ -49,7 +49,7 @@ def upgrade() -> None:
placeholders = ", ".join(f":action_{index}" for index, _ in enumerate(SYSTEM_ACTIONS))
params = {f"action_{index}": action for index, action in enumerate(SYSTEM_ACTIONS)}
bind.execute(
sa.text(f"UPDATE audit_log SET scope = 'system' WHERE action IN ({placeholders})"),
sa.text(f"UPDATE audit_log SET scope = 'system' WHERE action IN ({placeholders})"), # nosemgrep: python.sqlalchemy.security.audit.avoid-sqlalchemy-text.avoid-sqlalchemy-text
params,
)
bind.execute(sa.text("UPDATE audit_log SET scope = 'tenant' WHERE scope IS NULL OR scope NOT IN ('tenant', 'system')"))

View File

@@ -18,7 +18,7 @@ depends_on = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('tenants',
op.create_table('tenancy_tenants',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
@@ -27,7 +27,7 @@ def upgrade() -> None:
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_tenants'))
)
op.create_index(op.f('ix_tenants_slug'), 'tenants', ['slug'], unique=True)
op.create_index(op.f('ix_tenancy_tenants_slug'), 'tenancy_tenants', ['slug'], unique=True)
op.create_table('attachment_blobs',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
@@ -38,25 +38,25 @@ def upgrade() -> None:
sa.Column('storage_key', sa.String(length=1000), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_attachment_blobs_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_attachment_blobs_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_blobs')),
sa.UniqueConstraint('tenant_id', 'sha256', name='uq_attachment_blobs_tenant_sha256')
)
op.create_index(op.f('ix_attachment_blobs_sha256'), 'attachment_blobs', ['sha256'], unique=False)
op.create_index(op.f('ix_attachment_blobs_tenant_id'), 'attachment_blobs', ['tenant_id'], unique=False)
op.create_table('groups',
op.create_table('access_groups',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_groups_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_groups_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_groups')),
sa.UniqueConstraint('tenant_id', 'slug', name='uq_groups_tenant_slug')
)
op.create_index(op.f('ix_groups_tenant_id'), 'groups', ['tenant_id'], unique=False)
op.create_table('roles',
op.create_index(op.f('ix_access_groups_tenant_id'), 'access_groups', ['tenant_id'], unique=False)
op.create_table('access_roles',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=True),
sa.Column('slug', sa.String(length=100), nullable=False),
@@ -64,12 +64,12 @@ def upgrade() -> None:
sa.Column('permissions', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_roles_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_roles_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_roles')),
sa.UniqueConstraint('tenant_id', 'slug', name='uq_roles_tenant_slug')
)
op.create_index(op.f('ix_roles_tenant_id'), 'roles', ['tenant_id'], unique=False)
op.create_table('users',
op.create_index(op.f('ix_access_roles_tenant_id'), 'access_roles', ['tenant_id'], unique=False)
op.create_table('access_users',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('email', sa.String(length=320), nullable=False),
@@ -78,13 +78,13 @@ def upgrade() -> None:
sa.Column('is_tenant_admin', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_users_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_users_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_users')),
sa.UniqueConstraint('tenant_id', 'email', name='uq_users_tenant_email')
)
op.create_index(op.f('ix_users_email'), 'users', ['email'], unique=False)
op.create_index(op.f('ix_users_tenant_id'), 'users', ['tenant_id'], unique=False)
op.create_table('api_keys',
op.create_index(op.f('ix_access_users_email'), 'access_users', ['email'], unique=False)
op.create_index(op.f('ix_access_users_tenant_id'), 'access_users', ['tenant_id'], unique=False)
op.create_table('access_api_keys',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
@@ -97,13 +97,13 @@ def upgrade() -> None:
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_api_keys_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['users.id'], name=op.f('fk_api_keys_user_id_users'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_api_keys_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_api_keys_user_id_users'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_api_keys'))
)
op.create_index(op.f('ix_api_keys_prefix'), 'api_keys', ['prefix'], unique=False)
op.create_index(op.f('ix_api_keys_tenant_id'), 'api_keys', ['tenant_id'], unique=False)
op.create_index(op.f('ix_api_keys_user_id'), 'api_keys', ['user_id'], unique=False)
op.create_index(op.f('ix_access_api_keys_prefix'), 'access_api_keys', ['prefix'], unique=False)
op.create_index(op.f('ix_access_api_keys_tenant_id'), 'access_api_keys', ['tenant_id'], unique=False)
op.create_index(op.f('ix_access_api_keys_user_id'), 'access_api_keys', ['user_id'], unique=False)
op.create_table('campaigns',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
@@ -115,8 +115,8 @@ def upgrade() -> None:
sa.Column('current_version_id', sa.String(length=36), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['users.id'], name=op.f('fk_campaigns_created_by_user_id_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaigns_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_campaigns_created_by_user_id_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_campaigns_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaigns')),
sa.UniqueConstraint('tenant_id', 'external_id', name='uq_campaigns_tenant_external_id')
)
@@ -138,8 +138,8 @@ def upgrade() -> None:
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['blob_id'], ['attachment_blobs.id'], name=op.f('fk_attachment_instances_blob_id_attachment_blobs'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_attachment_instances_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['owner_user_id'], ['users.id'], name=op.f('fk_attachment_instances_owner_user_id_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_attachment_instances_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_attachment_instances_owner_user_id_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_attachment_instances_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_instances'))
)
op.create_index(op.f('ix_attachment_instances_blob_id'), 'attachment_instances', ['blob_id'], unique=False)
@@ -157,9 +157,9 @@ def upgrade() -> None:
sa.Column('details', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['api_key_id'], ['api_keys.id'], name=op.f('fk_audit_log_api_key_id_api_keys'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_audit_log_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['users.id'], name=op.f('fk_audit_log_user_id_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['api_key_id'], ['access_api_keys.id'], name=op.f('fk_audit_log_api_key_id_api_keys'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_audit_log_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_audit_log_user_id_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_audit_log'))
)
op.create_index(op.f('ix_audit_log_action'), 'audit_log', ['action'], unique=False)
@@ -213,7 +213,7 @@ def upgrade() -> None:
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_jobs_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_jobs_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaign_jobs_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_campaign_jobs_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_jobs')),
sa.UniqueConstraint('campaign_version_id', 'entry_index', name='uq_campaign_jobs_version_entry')
)
@@ -243,7 +243,7 @@ def upgrade() -> None:
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_issues_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_issues_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_campaign_issues_job_id_campaign_jobs'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], name=op.f('fk_campaign_issues_tenant_id_tenants'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['tenancy_tenants.id'], name=op.f('fk_campaign_issues_tenant_id_tenants'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_issues'))
)
op.create_index(op.f('ix_campaign_issues_campaign_id'), 'campaign_issues', ['campaign_id'], unique=False)
@@ -327,20 +327,20 @@ def downgrade() -> None:
op.drop_index(op.f('ix_campaigns_external_id'), table_name='campaigns')
op.drop_index(op.f('ix_campaigns_created_by_user_id'), table_name='campaigns')
op.drop_table('campaigns')
op.drop_index(op.f('ix_api_keys_user_id'), table_name='api_keys')
op.drop_index(op.f('ix_api_keys_tenant_id'), table_name='api_keys')
op.drop_index(op.f('ix_api_keys_prefix'), table_name='api_keys')
op.drop_table('api_keys')
op.drop_index(op.f('ix_users_tenant_id'), table_name='users')
op.drop_index(op.f('ix_users_email'), table_name='users')
op.drop_table('users')
op.drop_index(op.f('ix_roles_tenant_id'), table_name='roles')
op.drop_table('roles')
op.drop_index(op.f('ix_groups_tenant_id'), table_name='groups')
op.drop_table('groups')
op.drop_index(op.f('ix_access_api_keys_user_id'), table_name='access_api_keys')
op.drop_index(op.f('ix_access_api_keys_tenant_id'), table_name='access_api_keys')
op.drop_index(op.f('ix_access_api_keys_prefix'), table_name='access_api_keys')
op.drop_table('access_api_keys')
op.drop_index(op.f('ix_access_users_tenant_id'), table_name='access_users')
op.drop_index(op.f('ix_access_users_email'), table_name='access_users')
op.drop_table('access_users')
op.drop_index(op.f('ix_access_roles_tenant_id'), table_name='access_roles')
op.drop_table('access_roles')
op.drop_index(op.f('ix_access_groups_tenant_id'), table_name='access_groups')
op.drop_table('access_groups')
op.drop_index(op.f('ix_attachment_blobs_tenant_id'), table_name='attachment_blobs')
op.drop_index(op.f('ix_attachment_blobs_sha256'), table_name='attachment_blobs')
op.drop_table('attachment_blobs')
op.drop_index(op.f('ix_tenants_slug'), table_name='tenants')
op.drop_table('tenants')
op.drop_index(op.f('ix_tenancy_tenants_slug'), table_name='tenancy_tenants')
op.drop_table('tenancy_tenants')
# ### end Alembic commands ###

View File

@@ -19,10 +19,10 @@ def upgrade() -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
if "auth_sessions" in tables:
columns = {column["name"] for column in inspector.get_columns("auth_sessions")}
if "access_auth_sessions" in tables:
columns = {column["name"] for column in inspector.get_columns("access_auth_sessions")}
if "csrf_token_hash" not in columns:
op.add_column("auth_sessions", sa.Column("csrf_token_hash", sa.String(length=128), nullable=True))
op.add_column("access_auth_sessions", sa.Column("csrf_token_hash", sa.String(length=128), nullable=True))
if "mail_server_profiles" not in tables:
op.create_table(
"mail_server_profiles",
@@ -40,9 +40,9 @@ def upgrade() -> None:
sa.Column("updated_by_user_id", sa.String(length=36), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["updated_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenancy_tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["updated_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "slug", name="uq_mail_server_profiles_tenant_slug"),
)
@@ -67,7 +67,7 @@ def downgrade() -> None:
except Exception:
pass
op.drop_table("mail_server_profiles")
if "auth_sessions" in inspector.get_table_names():
columns = {column["name"] for column in inspector.get_columns("auth_sessions")}
if "access_auth_sessions" in inspector.get_table_names():
columns = {column["name"] for column in inspector.get_columns("access_auth_sessions")}
if "csrf_token_hash" in columns:
op.drop_column("auth_sessions", "csrf_token_hash")
op.drop_column("access_auth_sessions", "csrf_token_hash")

View File

@@ -31,7 +31,7 @@ def upgrade() -> None:
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
for table_name in ("users", "groups", "campaigns"):
for table_name in ("access_users", "access_groups", "campaigns"):
if table_name not in tables:
continue
columns = _columns(inspector, table_name)
@@ -83,6 +83,6 @@ def downgrade() -> None:
batch.drop_column("scope_type")
batch.alter_column("tenant_id", existing_type=sa.String(length=36), nullable=False)
for table_name in ("campaigns", "groups", "users"):
for table_name in ("campaigns", "access_groups", "access_users"):
if table_name in tables and "mail_profile_policy" in _columns(inspector, table_name):
op.drop_column(table_name, "mail_profile_policy")

View File

@@ -25,7 +25,7 @@ def upgrade() -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
for table_name in ("users", "groups", "campaigns"):
for table_name in ("access_users", "access_groups", "campaigns"):
if table_name not in tables:
continue
if "settings" not in _columns(inspector, table_name):
@@ -36,6 +36,6 @@ def downgrade() -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
tables = set(inspector.get_table_names())
for table_name in ("campaigns", "groups", "users"):
for table_name in ("campaigns", "access_groups", "access_users"):
if table_name in tables and "settings" in _columns(inspector, table_name):
op.drop_column(table_name, "settings")

View File

@@ -5,13 +5,15 @@ from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from govoplan_core.access.db import models as access_models # noqa: F401 - populate access metadata
from govoplan_access.backend.db import models as access_models # noqa: F401 - populate access metadata
from govoplan_core.admin import models as core_admin_models # noqa: F401 - populate core admin metadata
from govoplan_core.core import change_sequence as core_change_sequence_models # noqa: F401 - populate core metadata
from govoplan_core.core.migrations import migration_metadata_plan
from govoplan_core.db.base import Base
from govoplan_core.db import models # noqa: F401 - populate core metadata
from govoplan_core.server.default_config import get_server_config
from govoplan_core.server.registry import build_platform_registry
from govoplan_core.settings import settings
from govoplan_core.tenancy.scope import scope_registry
config = context.config
database_url = config.attributes.get("database_url") or settings.database_url
@@ -23,11 +25,13 @@ if config.config_file_name is not None:
def _target_metadata():
server_config = get_server_config()
enabled_modules = config.attributes.get("enabled_modules", server_config.enabled_modules)
manifest_factories = config.attributes.get("manifest_factories", server_config.manifest_factories)
registry = build_platform_registry(
server_config.enabled_modules,
manifest_factories=server_config.manifest_factories,
enabled_modules,
manifest_factories=manifest_factories,
)
plan = migration_metadata_plan(registry, extra_metadata=(Base.metadata,))
plan = migration_metadata_plan(registry, extra_metadata=(scope_registry.metadata, Base.metadata))
return tuple(dict.fromkeys(plan.metadata))

View File

@@ -1,130 +0,0 @@
"""auth sessions and RBAC assignments
Revision ID: 2c3d4e5f6a7b
Revises: 1f8d4c2a0b7e
Create Date: 2026-06-08 10:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "2c3d4e5f6a7b"
down_revision = "1f8d4c2a0b7e"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("users") as batch_op:
batch_op.add_column(sa.Column("auth_provider", sa.String(length=50), nullable=False, server_default="local"))
batch_op.add_column(sa.Column("password_hash", sa.String(length=500), nullable=True))
batch_op.add_column(sa.Column("last_login_at", sa.DateTime(timezone=True), nullable=True))
op.create_table(
"user_group_memberships",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("group_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["group_id"], ["groups.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "user_id", "group_id", name="uq_user_group_memberships"),
)
op.create_index(op.f("ix_user_group_memberships_tenant_id"), "user_group_memberships", ["tenant_id"])
op.create_index(op.f("ix_user_group_memberships_user_id"), "user_group_memberships", ["user_id"])
op.create_index(op.f("ix_user_group_memberships_group_id"), "user_group_memberships", ["group_id"])
op.create_table(
"user_role_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("role_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["roles.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "user_id", "role_id", name="uq_user_role_assignments"),
)
op.create_index(op.f("ix_user_role_assignments_tenant_id"), "user_role_assignments", ["tenant_id"])
op.create_index(op.f("ix_user_role_assignments_user_id"), "user_role_assignments", ["user_id"])
op.create_index(op.f("ix_user_role_assignments_role_id"), "user_role_assignments", ["role_id"])
op.create_table(
"group_role_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("group_id", sa.String(length=36), nullable=False),
sa.Column("role_id", sa.String(length=36), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["group_id"], ["groups.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["role_id"], ["roles.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("tenant_id", "group_id", "role_id", name="uq_group_role_assignments"),
)
op.create_index(op.f("ix_group_role_assignments_tenant_id"), "group_role_assignments", ["tenant_id"])
op.create_index(op.f("ix_group_role_assignments_group_id"), "group_role_assignments", ["group_id"])
op.create_index(op.f("ix_group_role_assignments_role_id"), "group_role_assignments", ["role_id"])
op.create_table(
"auth_sessions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("token_hash", sa.String(length=128), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("user_agent", sa.String(length=500), nullable=True),
sa.Column("ip_address", sa.String(length=100), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("token_hash"),
)
op.create_index(op.f("ix_auth_sessions_tenant_id"), "auth_sessions", ["tenant_id"])
op.create_index(op.f("ix_auth_sessions_user_id"), "auth_sessions", ["user_id"])
op.create_index(op.f("ix_auth_sessions_token_hash"), "auth_sessions", ["token_hash"])
op.create_index(op.f("ix_auth_sessions_expires_at"), "auth_sessions", ["expires_at"])
op.create_index(op.f("ix_auth_sessions_revoked_at"), "auth_sessions", ["revoked_at"])
def downgrade() -> None:
op.drop_index(op.f("ix_auth_sessions_revoked_at"), table_name="auth_sessions")
op.drop_index(op.f("ix_auth_sessions_expires_at"), table_name="auth_sessions")
op.drop_index(op.f("ix_auth_sessions_token_hash"), table_name="auth_sessions")
op.drop_index(op.f("ix_auth_sessions_user_id"), table_name="auth_sessions")
op.drop_index(op.f("ix_auth_sessions_tenant_id"), table_name="auth_sessions")
op.drop_table("auth_sessions")
op.drop_index(op.f("ix_group_role_assignments_role_id"), table_name="group_role_assignments")
op.drop_index(op.f("ix_group_role_assignments_group_id"), table_name="group_role_assignments")
op.drop_index(op.f("ix_group_role_assignments_tenant_id"), table_name="group_role_assignments")
op.drop_table("group_role_assignments")
op.drop_index(op.f("ix_user_role_assignments_role_id"), table_name="user_role_assignments")
op.drop_index(op.f("ix_user_role_assignments_user_id"), table_name="user_role_assignments")
op.drop_index(op.f("ix_user_role_assignments_tenant_id"), table_name="user_role_assignments")
op.drop_table("user_role_assignments")
op.drop_index(op.f("ix_user_group_memberships_group_id"), table_name="user_group_memberships")
op.drop_index(op.f("ix_user_group_memberships_user_id"), table_name="user_group_memberships")
op.drop_index(op.f("ix_user_group_memberships_tenant_id"), table_name="user_group_memberships")
op.drop_table("user_group_memberships")
with op.batch_alter_table("users") as batch_op:
batch_op.drop_column("last_login_at")
batch_op.drop_column("password_hash")
batch_op.drop_column("auth_provider")

View File

@@ -0,0 +1,892 @@
"""v0.1.7 core baseline
Revision ID: 4f2a9c8e7b6d
Revises: None
Create Date: 2026-07-11 00:00:00.000000
"""
from __future__ import annotations
from datetime import datetime, timezone
from uuid import uuid4
from alembic import op
import sqlalchemy as sa
revision = '4f2a9c8e7b6d'
down_revision = None
branch_labels = None
depends_on = None
def _now() -> datetime:
return datetime.now(timezone.utc)
def _seed_core_defaults() -> None:
bind = op.get_bind()
now = _now()
if not bind.execute(sa.text("SELECT 1 FROM core_system_settings WHERE id = 'global'")).first():
settings_table = sa.table(
"core_system_settings",
sa.column("id", sa.String),
sa.column("default_locale", sa.String),
sa.column("allow_tenant_custom_groups", sa.Boolean),
sa.column("allow_tenant_custom_roles", sa.Boolean),
sa.column("allow_tenant_api_keys", sa.Boolean),
sa.column("settings", sa.JSON),
sa.column("created_at", sa.DateTime(timezone=True)),
sa.column("updated_at", sa.DateTime(timezone=True)),
)
bind.execute(
settings_table.insert().values({
"id": "global",
"default_locale": "en",
"allow_tenant_custom_groups": True,
"allow_tenant_custom_roles": True,
"allow_tenant_api_keys": True,
"settings": {},
"created_at": now,
"updated_at": now,
}),
)
system_roles = {
"system_owner": {
"name": "System owner",
"description": "Protected full instance-wide administration.",
"permissions": ["system:*"],
"is_builtin": True,
},
"system_admin": {
"name": "System administrator",
"description": "Manage the instance without granting the protected System owner role.",
"permissions": [
"system:tenants:read", "system:tenants:create", "system:tenants:update", "system:tenants:suspend",
"system:accounts:read", "system:accounts:create", "system:accounts:update", "system:accounts:suspend",
"system:roles:read", "system:roles:write", "system:roles:assign", "system:access:read", "system:access:assign",
"system:audit:read", "system:settings:read", "system:settings:write", "system:governance:read", "system:governance:write",
],
"is_builtin": False,
},
"system_auditor": {
"name": "System auditor",
"description": "Read-only access to system administration and audit.",
"permissions": [
"system:tenants:read", "system:accounts:read", "system:roles:read", "system:access:read",
"system:audit:read", "system:settings:read", "system:governance:read",
],
"is_builtin": False,
},
}
roles_table = sa.table(
"access_roles",
sa.column("id", sa.String),
sa.column("tenant_id", sa.String),
sa.column("slug", sa.String),
sa.column("name", sa.String),
sa.column("description", sa.Text),
sa.column("permissions", sa.JSON),
sa.column("is_builtin", sa.Boolean),
sa.column("is_assignable", sa.Boolean),
sa.column("system_template_id", sa.String),
sa.column("system_required", sa.Boolean),
sa.column("created_at", sa.DateTime(timezone=True)),
sa.column("updated_at", sa.DateTime(timezone=True)),
)
for slug, role in system_roles.items():
if bind.execute(
sa.text("SELECT 1 FROM access_roles WHERE tenant_id IS NULL AND slug = :slug"),
{"slug": slug},
).first():
continue
bind.execute(
roles_table.insert().values(
id=str(uuid4()),
tenant_id=None,
slug=slug,
name=role["name"],
description=role["description"],
permissions=role["permissions"],
is_builtin=bool(role["is_builtin"]),
is_assignable=True,
system_template_id=None,
system_required=False,
created_at=now,
updated_at=now,
),
)
def upgrade() -> None:
op.create_table('core_scopes',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('default_locale', sa.String(length=20), nullable=False),
sa.Column('settings', sa.JSON(), nullable=False),
sa.Column('allow_custom_groups', sa.Boolean(), nullable=True),
sa.Column('allow_custom_roles', sa.Boolean(), nullable=True),
sa.Column('allow_api_keys', sa.Boolean(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_core_scopes'))
)
op.create_index(op.f('ix_core_scopes_slug'), 'core_scopes', ['slug'], unique=True)
op.create_table('access_accounts',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('email', sa.String(length=320), nullable=False),
sa.Column('normalized_email', sa.String(length=320), nullable=False),
sa.Column('display_name', sa.String(length=255), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('auth_provider', sa.String(length=50), nullable=False),
sa.Column('password_hash', sa.String(length=500), nullable=True),
sa.Column('password_reset_required', sa.Boolean(), nullable=False),
sa.Column('last_login_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_accounts'))
)
op.create_index(op.f('ix_access_accounts_normalized_email'), 'access_accounts', ['normalized_email'], unique=True)
op.create_table('access_groups',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('system_template_id', sa.String(length=36), nullable=True),
sa.Column('system_required', sa.Boolean(), nullable=False),
sa.Column('settings', sa.JSON(), nullable=False),
sa.Column('mail_profile_policy', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_groups')),
sa.UniqueConstraint('tenant_id', 'slug', name='uq_groups_tenant_slug')
)
op.create_index(op.f('ix_access_groups_system_template_id'), 'access_groups', ['system_template_id'], unique=False)
op.create_index(op.f('ix_access_groups_tenant_id'), 'access_groups', ['tenant_id'], unique=False)
op.create_table('access_roles',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=True),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('permissions', sa.JSON(), nullable=False),
sa.Column('is_builtin', sa.Boolean(), nullable=False),
sa.Column('is_assignable', sa.Boolean(), nullable=False),
sa.Column('system_template_id', sa.String(length=36), nullable=True),
sa.Column('system_required', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_roles')),
sa.UniqueConstraint('tenant_id', 'slug', name='uq_roles_tenant_slug')
)
op.create_index(op.f('ix_access_roles_system_template_id'), 'access_roles', ['system_template_id'], unique=False)
op.create_index(op.f('ix_access_roles_tenant_id'), 'access_roles', ['tenant_id'], unique=False)
op.create_index('uq_roles_system_slug', 'access_roles', ['slug'], unique=True, sqlite_where=sa.text('tenant_id IS NULL'), postgresql_where=sa.text('tenant_id IS NULL'))
op.create_table('admin_governance_templates',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('kind', sa.String(length=20), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('permissions', sa.JSON(), nullable=False),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_admin_governance_templates')),
sa.UniqueConstraint('kind', 'slug', name='uq_governance_templates_kind_slug')
)
op.create_index(op.f('ix_admin_governance_templates_kind'), 'admin_governance_templates', ['kind'], unique=False)
op.create_table('attachment_blobs',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('sha256', sa.String(length=64), nullable=False),
sa.Column('size_bytes', sa.Integer(), nullable=False),
sa.Column('mime_type', sa.String(length=255), nullable=True),
sa.Column('storage_bucket', sa.String(length=255), nullable=False),
sa.Column('storage_key', sa.String(length=1000), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_blobs')),
sa.UniqueConstraint('tenant_id', 'sha256', name='uq_attachment_blobs_tenant_sha256')
)
op.create_index(op.f('ix_attachment_blobs_sha256'), 'attachment_blobs', ['sha256'], unique=False)
op.create_index(op.f('ix_attachment_blobs_tenant_id'), 'attachment_blobs', ['tenant_id'], unique=False)
op.create_table('audit_outbox_events',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('event_id', sa.String(length=36), nullable=False),
sa.Column('event_type', sa.String(length=200), nullable=False),
sa.Column('module_id', sa.String(length=100), nullable=False),
sa.Column('correlation_id', sa.String(length=128), nullable=True),
sa.Column('causation_id', sa.String(length=128), nullable=True),
sa.Column('classification', sa.String(length=40), nullable=False),
sa.Column('payload', sa.JSON(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('attempts', sa.Integer(), nullable=False),
sa.Column('next_attempt_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('dispatched_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_audit_outbox_events')),
sa.UniqueConstraint('event_id', name='uq_audit_outbox_events_event_id')
)
op.create_index('ix_audit_outbox_events_correlation_id', 'audit_outbox_events', ['correlation_id'], unique=False)
op.create_index('ix_audit_outbox_events_event_type', 'audit_outbox_events', ['event_type'], unique=False)
op.create_index(op.f('ix_audit_outbox_events_status'), 'audit_outbox_events', ['status'], unique=False)
op.create_index('ix_audit_outbox_events_status_next_attempt_at', 'audit_outbox_events', ['status', 'next_attempt_at'], unique=False)
op.create_table('core_change_sequence',
sa.Column('id', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), autoincrement=True, nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=True),
sa.Column('module_id', sa.String(length=100), nullable=False),
sa.Column('collection', sa.String(length=150), nullable=False),
sa.Column('resource_type', sa.String(length=100), nullable=False),
sa.Column('resource_id', sa.String(length=255), nullable=False),
sa.Column('operation', sa.String(length=30), nullable=False),
sa.Column('actor_type', sa.String(length=30), nullable=True),
sa.Column('actor_id', sa.String(length=255), nullable=True),
sa.Column('payload', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_core_change_sequence'))
)
op.create_index(op.f('ix_core_change_sequence_actor_id'), 'core_change_sequence', ['actor_id'], unique=False)
op.create_index(op.f('ix_core_change_sequence_actor_type'), 'core_change_sequence', ['actor_type'], unique=False)
op.create_index(op.f('ix_core_change_sequence_collection'), 'core_change_sequence', ['collection'], unique=False)
op.create_index('ix_core_change_sequence_collection_id', 'core_change_sequence', ['collection', 'id'], unique=False)
op.create_index(op.f('ix_core_change_sequence_created_at'), 'core_change_sequence', ['created_at'], unique=False)
op.create_index(op.f('ix_core_change_sequence_module_id'), 'core_change_sequence', ['module_id'], unique=False)
op.create_index(op.f('ix_core_change_sequence_operation'), 'core_change_sequence', ['operation'], unique=False)
op.create_index('ix_core_change_sequence_resource', 'core_change_sequence', ['module_id', 'resource_type', 'resource_id'], unique=False)
op.create_index(op.f('ix_core_change_sequence_resource_id'), 'core_change_sequence', ['resource_id'], unique=False)
op.create_index(op.f('ix_core_change_sequence_resource_type'), 'core_change_sequence', ['resource_type'], unique=False)
op.create_index(op.f('ix_core_change_sequence_tenant_id'), 'core_change_sequence', ['tenant_id'], unique=False)
op.create_index('ix_core_change_sequence_tenant_module_id', 'core_change_sequence', ['tenant_id', 'module_id', 'id'], unique=False)
op.create_table('core_change_sequence_retention_floor',
sa.Column('id', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), autoincrement=True, nullable=False),
sa.Column('tenant_key', sa.String(length=36), nullable=False),
sa.Column('module_id', sa.String(length=100), nullable=False),
sa.Column('collection', sa.String(length=150), nullable=False),
sa.Column('min_valid_sequence', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_core_change_sequence_retention_floor')),
sa.UniqueConstraint('tenant_key', 'module_id', 'collection', name='uq_core_change_sequence_retention_scope')
)
op.create_index('ix_core_change_sequence_retention_scope', 'core_change_sequence_retention_floor', ['tenant_key', 'module_id', 'collection'], unique=False)
op.create_table('core_system_settings',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('default_locale', sa.String(length=20), nullable=False),
sa.Column('allow_tenant_custom_groups', sa.Boolean(), nullable=False),
sa.Column('allow_tenant_custom_roles', sa.Boolean(), nullable=False),
sa.Column('allow_tenant_api_keys', sa.Boolean(), nullable=False),
sa.Column('settings', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_core_system_settings'))
)
op.create_table('file_blobs',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('storage_backend', sa.String(length=50), nullable=False),
sa.Column('storage_bucket', sa.String(length=255), nullable=True),
sa.Column('storage_key', sa.String(length=1000), nullable=False),
sa.Column('checksum_sha256', sa.String(length=64), nullable=False),
sa.Column('size_bytes', sa.Integer(), nullable=False),
sa.Column('content_type', sa.String(length=255), nullable=True),
sa.Column('ref_count', sa.Integer(), nullable=False),
sa.Column('retained_until', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id', name=op.f('pk_file_blobs')),
sa.UniqueConstraint('tenant_id', 'checksum_sha256', 'size_bytes', name='uq_file_blobs_tenant_checksum_size')
)
op.create_index(op.f('ix_file_blobs_checksum_sha256'), 'file_blobs', ['checksum_sha256'], unique=False)
op.create_index(op.f('ix_file_blobs_tenant_id'), 'file_blobs', ['tenant_id'], unique=False)
op.create_table('access_group_role_assignments',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('group_id', sa.String(length=36), nullable=False),
sa.Column('role_id', sa.String(length=36), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['group_id'], ['access_groups.id'], name=op.f('fk_access_group_role_assignments_group_id_access_groups'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['role_id'], ['access_roles.id'], name=op.f('fk_access_group_role_assignments_role_id_access_roles'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_group_role_assignments')),
sa.UniqueConstraint('tenant_id', 'group_id', 'role_id', name='uq_group_role_assignments')
)
op.create_index(op.f('ix_access_group_role_assignments_group_id'), 'access_group_role_assignments', ['group_id'], unique=False)
op.create_index(op.f('ix_access_group_role_assignments_role_id'), 'access_group_role_assignments', ['role_id'], unique=False)
op.create_index(op.f('ix_access_group_role_assignments_tenant_id'), 'access_group_role_assignments', ['tenant_id'], unique=False)
op.create_table('access_system_role_assignments',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('account_id', sa.String(length=36), nullable=False),
sa.Column('role_id', sa.String(length=36), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['account_id'], ['access_accounts.id'], name=op.f('fk_access_system_role_assignments_account_id_access_accounts'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['role_id'], ['access_roles.id'], name=op.f('fk_access_system_role_assignments_role_id_access_roles'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_system_role_assignments')),
sa.UniqueConstraint('account_id', 'role_id', name='uq_system_role_assignments')
)
op.create_index(op.f('ix_access_system_role_assignments_account_id'), 'access_system_role_assignments', ['account_id'], unique=False)
op.create_index(op.f('ix_access_system_role_assignments_role_id'), 'access_system_role_assignments', ['role_id'], unique=False)
op.create_table('access_users',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('account_id', sa.String(length=36), nullable=False),
sa.Column('email', sa.String(length=320), nullable=False),
sa.Column('display_name', sa.String(length=255), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('is_tenant_admin', sa.Boolean(), nullable=False),
sa.Column('auth_provider', sa.String(length=50), nullable=False),
sa.Column('password_hash', sa.String(length=500), nullable=True),
sa.Column('last_login_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('settings', sa.JSON(), nullable=False),
sa.Column('mail_profile_policy', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['account_id'], ['access_accounts.id'], name=op.f('fk_access_users_account_id_access_accounts'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_users')),
sa.UniqueConstraint('tenant_id', 'account_id', name='uq_users_tenant_account'),
sa.UniqueConstraint('tenant_id', 'email', name='uq_users_tenant_email')
)
op.create_index(op.f('ix_access_users_account_id'), 'access_users', ['account_id'], unique=False)
op.create_index(op.f('ix_access_users_email'), 'access_users', ['email'], unique=False)
op.create_index(op.f('ix_access_users_tenant_id'), 'access_users', ['tenant_id'], unique=False)
op.create_table('admin_governance_template_assignments',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('template_id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('mode', sa.String(length=20), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['template_id'], ['admin_governance_templates.id'], name=op.f('fk_admin_governance_template_assignments_template_id_admin_governance_templates'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_admin_governance_template_assignments')),
sa.UniqueConstraint('template_id', 'tenant_id', name='uq_governance_template_tenant')
)
op.create_index(op.f('ix_admin_governance_template_assignments_template_id'), 'admin_governance_template_assignments', ['template_id'], unique=False)
op.create_index(op.f('ix_admin_governance_template_assignments_tenant_id'), 'admin_governance_template_assignments', ['tenant_id'], unique=False)
op.create_table('access_api_keys',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('prefix', sa.String(length=16), nullable=False),
sa.Column('key_hash', sa.String(length=128), nullable=False),
sa.Column('scopes', sa.JSON(), nullable=False),
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_used_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_access_api_keys_user_id_access_users'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_api_keys'))
)
op.create_index(op.f('ix_access_api_keys_prefix'), 'access_api_keys', ['prefix'], unique=False)
op.create_index(op.f('ix_access_api_keys_tenant_id'), 'access_api_keys', ['tenant_id'], unique=False)
op.create_index(op.f('ix_access_api_keys_user_id'), 'access_api_keys', ['user_id'], unique=False)
op.create_table('access_auth_sessions',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('account_id', sa.String(length=36), nullable=False),
sa.Column('token_hash', sa.String(length=128), nullable=False),
sa.Column('csrf_token_hash', sa.String(length=128), nullable=True),
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('last_seen_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('user_agent', sa.String(length=500), nullable=True),
sa.Column('ip_address', sa.String(length=100), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['account_id'], ['access_accounts.id'], name=op.f('fk_access_auth_sessions_account_id_access_accounts'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_access_auth_sessions_user_id_access_users'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_auth_sessions'))
)
op.create_index(op.f('ix_access_auth_sessions_account_id'), 'access_auth_sessions', ['account_id'], unique=False)
op.create_index(op.f('ix_access_auth_sessions_expires_at'), 'access_auth_sessions', ['expires_at'], unique=False)
op.create_index(op.f('ix_access_auth_sessions_revoked_at'), 'access_auth_sessions', ['revoked_at'], unique=False)
op.create_index(op.f('ix_access_auth_sessions_tenant_id'), 'access_auth_sessions', ['tenant_id'], unique=False)
op.create_index(op.f('ix_access_auth_sessions_token_hash'), 'access_auth_sessions', ['token_hash'], unique=True)
op.create_index(op.f('ix_access_auth_sessions_user_id'), 'access_auth_sessions', ['user_id'], unique=False)
op.create_table('access_user_group_memberships',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('group_id', sa.String(length=36), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['group_id'], ['access_groups.id'], name=op.f('fk_access_user_group_memberships_group_id_access_groups'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_access_user_group_memberships_user_id_access_users'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_user_group_memberships')),
sa.UniqueConstraint('tenant_id', 'user_id', 'group_id', name='uq_user_group_memberships')
)
op.create_index(op.f('ix_access_user_group_memberships_group_id'), 'access_user_group_memberships', ['group_id'], unique=False)
op.create_index(op.f('ix_access_user_group_memberships_tenant_id'), 'access_user_group_memberships', ['tenant_id'], unique=False)
op.create_index(op.f('ix_access_user_group_memberships_user_id'), 'access_user_group_memberships', ['user_id'], unique=False)
op.create_table('access_user_role_assignments',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('role_id', sa.String(length=36), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['role_id'], ['access_roles.id'], name=op.f('fk_access_user_role_assignments_role_id_access_roles'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_access_user_role_assignments_user_id_access_users'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_access_user_role_assignments')),
sa.UniqueConstraint('tenant_id', 'user_id', 'role_id', name='uq_user_role_assignments')
)
op.create_index(op.f('ix_access_user_role_assignments_role_id'), 'access_user_role_assignments', ['role_id'], unique=False)
op.create_index(op.f('ix_access_user_role_assignments_tenant_id'), 'access_user_role_assignments', ['tenant_id'], unique=False)
op.create_index(op.f('ix_access_user_role_assignments_user_id'), 'access_user_role_assignments', ['user_id'], unique=False)
op.create_table('campaigns',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('owner_user_id', sa.String(length=36), nullable=True),
sa.Column('owner_group_id', sa.String(length=36), nullable=True),
sa.Column('external_id', sa.String(length=255), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('status', sa.String(length=50), nullable=False),
sa.Column('current_version_id', sa.String(length=36), nullable=True),
sa.Column('settings', sa.JSON(), nullable=False),
sa.Column('mail_profile_policy', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_campaigns_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_group_id'], ['access_groups.id'], name=op.f('fk_campaigns_owner_group_id_access_groups'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_campaigns_owner_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaigns')),
sa.UniqueConstraint('tenant_id', 'external_id', name='uq_campaigns_tenant_external_id')
)
op.create_index(op.f('ix_campaigns_created_by_user_id'), 'campaigns', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_campaigns_external_id'), 'campaigns', ['external_id'], unique=False)
op.create_index(op.f('ix_campaigns_owner_group_id'), 'campaigns', ['owner_group_id'], unique=False)
op.create_index(op.f('ix_campaigns_owner_user_id'), 'campaigns', ['owner_user_id'], unique=False)
op.create_index(op.f('ix_campaigns_status'), 'campaigns', ['status'], unique=False)
op.create_index(op.f('ix_campaigns_tenant_id'), 'campaigns', ['tenant_id'], unique=False)
op.create_table('file_assets',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('owner_type', sa.String(length=20), nullable=False),
sa.Column('owner_user_id', sa.String(length=36), nullable=True),
sa.Column('owner_group_id', sa.String(length=36), nullable=True),
sa.Column('current_version_id', sa.String(length=36), nullable=True),
sa.Column('display_path', sa.String(length=1000), nullable=False),
sa.Column('filename', sa.String(length=500), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('deleted_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('metadata', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_file_assets_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_group_id'], ['access_groups.id'], name=op.f('fk_file_assets_owner_group_id_access_groups'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_file_assets_owner_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_file_assets'))
)
op.create_index(op.f('ix_file_assets_created_by_user_id'), 'file_assets', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_file_assets_current_version_id'), 'file_assets', ['current_version_id'], unique=False)
op.create_index(op.f('ix_file_assets_deleted_at'), 'file_assets', ['deleted_at'], unique=False)
op.create_index(op.f('ix_file_assets_display_path'), 'file_assets', ['display_path'], unique=False)
op.create_index(op.f('ix_file_assets_filename'), 'file_assets', ['filename'], unique=False)
op.create_index(op.f('ix_file_assets_owner_group_id'), 'file_assets', ['owner_group_id'], unique=False)
op.create_index(op.f('ix_file_assets_owner_type'), 'file_assets', ['owner_type'], unique=False)
op.create_index(op.f('ix_file_assets_owner_user_id'), 'file_assets', ['owner_user_id'], unique=False)
op.create_index(op.f('ix_file_assets_tenant_id'), 'file_assets', ['tenant_id'], unique=False)
op.create_table('file_folders',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('owner_type', sa.String(length=20), nullable=False),
sa.Column('owner_user_id', sa.String(length=36), nullable=True),
sa.Column('owner_group_id', sa.String(length=36), nullable=True),
sa.Column('path', sa.String(length=1000), nullable=False),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('deleted_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('metadata', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_file_folders_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_group_id'], ['access_groups.id'], name=op.f('fk_file_folders_owner_group_id_access_groups'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_file_folders_owner_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_file_folders'))
)
op.create_index(op.f('ix_file_folders_created_by_user_id'), 'file_folders', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_file_folders_deleted_at'), 'file_folders', ['deleted_at'], unique=False)
op.create_index(op.f('ix_file_folders_owner_group_id'), 'file_folders', ['owner_group_id'], unique=False)
op.create_index(op.f('ix_file_folders_owner_type'), 'file_folders', ['owner_type'], unique=False)
op.create_index(op.f('ix_file_folders_owner_user_id'), 'file_folders', ['owner_user_id'], unique=False)
op.create_index(op.f('ix_file_folders_path'), 'file_folders', ['path'], unique=False)
op.create_index(op.f('ix_file_folders_tenant_id'), 'file_folders', ['tenant_id'], unique=False)
op.create_index('uq_file_folders_active_group_path', 'file_folders', ['tenant_id', 'owner_group_id', 'path'], unique=True, sqlite_where=sa.text("owner_type = 'group' AND deleted_at IS NULL"), postgresql_where=sa.text("owner_type = 'group' AND deleted_at IS NULL"))
op.create_index('uq_file_folders_active_user_path', 'file_folders', ['tenant_id', 'owner_user_id', 'path'], unique=True, sqlite_where=sa.text("owner_type = 'user' AND deleted_at IS NULL"), postgresql_where=sa.text("owner_type = 'user' AND deleted_at IS NULL"))
op.create_table('mail_server_profiles',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=True),
sa.Column('scope_type', sa.String(length=20), nullable=False),
sa.Column('scope_id', sa.String(length=36), nullable=True),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('slug', sa.String(length=100), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('smtp_config', sa.JSON(), nullable=False),
sa.Column('smtp_username', sa.String(length=320), nullable=True),
sa.Column('smtp_password_encrypted', sa.Text(), nullable=True),
sa.Column('imap_config', sa.JSON(), nullable=True),
sa.Column('imap_username', sa.String(length=320), nullable=True),
sa.Column('imap_password_encrypted', sa.Text(), nullable=True),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('updated_by_user_id', sa.String(length=36), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_mail_server_profiles_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['updated_by_user_id'], ['access_users.id'], name=op.f('fk_mail_server_profiles_updated_by_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_mail_server_profiles')),
sa.UniqueConstraint('tenant_id', 'slug', name='uq_mail_server_profiles_tenant_slug')
)
op.create_index(op.f('ix_mail_server_profiles_created_by_user_id'), 'mail_server_profiles', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_mail_server_profiles_is_active'), 'mail_server_profiles', ['is_active'], unique=False)
op.create_index('ix_mail_server_profiles_scope', 'mail_server_profiles', ['scope_type', 'scope_id'], unique=False)
op.create_index(op.f('ix_mail_server_profiles_scope_id'), 'mail_server_profiles', ['scope_id'], unique=False)
op.create_index(op.f('ix_mail_server_profiles_scope_type'), 'mail_server_profiles', ['scope_type'], unique=False)
op.create_index(op.f('ix_mail_server_profiles_tenant_id'), 'mail_server_profiles', ['tenant_id'], unique=False)
op.create_index(op.f('ix_mail_server_profiles_updated_by_user_id'), 'mail_server_profiles', ['updated_by_user_id'], unique=False)
op.create_table('attachment_instances',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('owner_user_id', sa.String(length=36), nullable=True),
sa.Column('campaign_id', sa.String(length=36), nullable=True),
sa.Column('blob_id', sa.String(length=36), nullable=False),
sa.Column('logical_name', sa.String(length=500), nullable=True),
sa.Column('filename', sa.String(length=500), nullable=False),
sa.Column('tags', sa.JSON(), nullable=False),
sa.Column('metadata', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['blob_id'], ['attachment_blobs.id'], name=op.f('fk_attachment_instances_blob_id_attachment_blobs'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_attachment_instances_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_attachment_instances_owner_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_attachment_instances'))
)
op.create_index(op.f('ix_attachment_instances_blob_id'), 'attachment_instances', ['blob_id'], unique=False)
op.create_index(op.f('ix_attachment_instances_campaign_id'), 'attachment_instances', ['campaign_id'], unique=False)
op.create_index(op.f('ix_attachment_instances_owner_user_id'), 'attachment_instances', ['owner_user_id'], unique=False)
op.create_index(op.f('ix_attachment_instances_tenant_id'), 'attachment_instances', ['tenant_id'], unique=False)
op.create_table('audit_log',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('scope', sa.String(length=20), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=True),
sa.Column('user_id', sa.String(length=36), nullable=True),
sa.Column('api_key_id', sa.String(length=36), nullable=True),
sa.Column('action', sa.String(length=100), nullable=False),
sa.Column('object_type', sa.String(length=100), nullable=True),
sa.Column('object_id', sa.String(length=100), nullable=True),
sa.Column('details', sa.JSON(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['api_key_id'], ['access_api_keys.id'], name=op.f('fk_audit_log_api_key_id_access_api_keys'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['user_id'], ['access_users.id'], name=op.f('fk_audit_log_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_audit_log'))
)
op.create_index(op.f('ix_audit_log_action'), 'audit_log', ['action'], unique=False)
op.create_index(op.f('ix_audit_log_api_key_id'), 'audit_log', ['api_key_id'], unique=False)
op.create_index(op.f('ix_audit_log_object_id'), 'audit_log', ['object_id'], unique=False)
op.create_index(op.f('ix_audit_log_object_type'), 'audit_log', ['object_type'], unique=False)
op.create_index(op.f('ix_audit_log_scope'), 'audit_log', ['scope'], unique=False)
op.create_index('ix_audit_log_scope_created_at', 'audit_log', ['scope', 'created_at'], unique=False)
op.create_index(op.f('ix_audit_log_tenant_id'), 'audit_log', ['tenant_id'], unique=False)
op.create_index('ix_audit_log_tenant_scope_created_at', 'audit_log', ['tenant_id', 'scope', 'created_at'], unique=False)
op.create_index(op.f('ix_audit_log_user_id'), 'audit_log', ['user_id'], unique=False)
op.create_table('campaign_shares',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('campaign_id', sa.String(length=36), nullable=False),
sa.Column('target_type', sa.String(length=20), nullable=False),
sa.Column('target_id', sa.String(length=36), nullable=False),
sa.Column('permission', sa.String(length=20), nullable=False),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_shares_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_campaign_shares_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_shares')),
sa.UniqueConstraint('campaign_id', 'target_type', 'target_id', name='uq_campaign_share_target')
)
op.create_index(op.f('ix_campaign_shares_campaign_id'), 'campaign_shares', ['campaign_id'], unique=False)
op.create_index(op.f('ix_campaign_shares_created_by_user_id'), 'campaign_shares', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_campaign_shares_revoked_at'), 'campaign_shares', ['revoked_at'], unique=False)
op.create_index(op.f('ix_campaign_shares_target_id'), 'campaign_shares', ['target_id'], unique=False)
op.create_index(op.f('ix_campaign_shares_target_type'), 'campaign_shares', ['target_type'], unique=False)
op.create_index(op.f('ix_campaign_shares_tenant_id'), 'campaign_shares', ['tenant_id'], unique=False)
op.create_table('campaign_versions',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('campaign_id', sa.String(length=36), nullable=False),
sa.Column('version_number', sa.Integer(), nullable=False),
sa.Column('raw_json', sa.JSON(), nullable=False),
sa.Column('schema_version', sa.String(length=50), nullable=False),
sa.Column('source_filename', sa.String(length=500), nullable=True),
sa.Column('source_base_path', sa.String(length=1000), nullable=True),
sa.Column('workflow_state', sa.String(length=50), nullable=False),
sa.Column('current_flow', sa.String(length=50), nullable=False),
sa.Column('current_step', sa.String(length=100), nullable=True),
sa.Column('is_complete', sa.Boolean(), nullable=False),
sa.Column('editor_state', sa.JSON(), nullable=False),
sa.Column('autosaved_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('published_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('locked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('locked_by_user_id', sa.String(length=36), nullable=True),
sa.Column('user_lock_state', sa.String(length=20), nullable=True),
sa.Column('user_locked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('user_locked_by_user_id', sa.String(length=36), nullable=True),
sa.Column('validation_summary', sa.JSON(), nullable=True),
sa.Column('build_summary', sa.JSON(), nullable=True),
sa.Column('execution_snapshot', sa.JSON(), nullable=True),
sa.Column('execution_snapshot_hash', sa.String(length=64), nullable=True),
sa.Column('execution_snapshot_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_versions_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['locked_by_user_id'], ['access_users.id'], name=op.f('fk_campaign_versions_locked_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['user_locked_by_user_id'], ['access_users.id'], name=op.f('fk_campaign_versions_user_locked_by_user_id_access_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_versions')),
sa.UniqueConstraint('campaign_id', 'version_number', name='uq_campaign_versions_campaign_number')
)
op.create_index(op.f('ix_campaign_versions_campaign_id'), 'campaign_versions', ['campaign_id'], unique=False)
op.create_index(op.f('ix_campaign_versions_current_flow'), 'campaign_versions', ['current_flow'], unique=False)
op.create_index(op.f('ix_campaign_versions_execution_snapshot_hash'), 'campaign_versions', ['execution_snapshot_hash'], unique=False)
op.create_index(op.f('ix_campaign_versions_locked_by_user_id'), 'campaign_versions', ['locked_by_user_id'], unique=False)
op.create_index(op.f('ix_campaign_versions_user_lock_state'), 'campaign_versions', ['user_lock_state'], unique=False)
op.create_index(op.f('ix_campaign_versions_user_locked_by_user_id'), 'campaign_versions', ['user_locked_by_user_id'], unique=False)
op.create_index(op.f('ix_campaign_versions_workflow_state'), 'campaign_versions', ['workflow_state'], unique=False)
op.create_table('file_shares',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('file_asset_id', sa.String(length=36), nullable=False),
sa.Column('target_type', sa.String(length=20), nullable=False),
sa.Column('target_id', sa.String(length=36), nullable=False),
sa.Column('permission', sa.String(length=20), nullable=False),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_file_shares_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['file_asset_id'], ['file_assets.id'], name=op.f('fk_file_shares_file_asset_id_file_assets'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_file_shares')),
sa.UniqueConstraint('file_asset_id', 'target_type', 'target_id', 'revoked_at', name='uq_file_shares_active_target')
)
op.create_index(op.f('ix_file_shares_created_by_user_id'), 'file_shares', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_file_shares_file_asset_id'), 'file_shares', ['file_asset_id'], unique=False)
op.create_index(op.f('ix_file_shares_revoked_at'), 'file_shares', ['revoked_at'], unique=False)
op.create_index(op.f('ix_file_shares_target_id'), 'file_shares', ['target_id'], unique=False)
op.create_index(op.f('ix_file_shares_target_type'), 'file_shares', ['target_type'], unique=False)
op.create_index(op.f('ix_file_shares_tenant_id'), 'file_shares', ['tenant_id'], unique=False)
op.create_table('file_versions',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('file_asset_id', sa.String(length=36), nullable=False),
sa.Column('blob_id', sa.String(length=36), nullable=False),
sa.Column('version_number', sa.Integer(), nullable=False),
sa.Column('filename_at_upload', sa.String(length=500), nullable=False),
sa.Column('display_path_at_upload', sa.String(length=1000), nullable=False),
sa.Column('content_type', sa.String(length=255), nullable=True),
sa.Column('size_bytes', sa.Integer(), nullable=False),
sa.Column('checksum_sha256', sa.String(length=64), nullable=False),
sa.Column('created_by_user_id', sa.String(length=36), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['blob_id'], ['file_blobs.id'], name=op.f('fk_file_versions_blob_id_file_blobs'), ondelete='RESTRICT'),
sa.ForeignKeyConstraint(['created_by_user_id'], ['access_users.id'], name=op.f('fk_file_versions_created_by_user_id_access_users'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['file_asset_id'], ['file_assets.id'], name=op.f('fk_file_versions_file_asset_id_file_assets'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_file_versions')),
sa.UniqueConstraint('file_asset_id', 'version_number', name='uq_file_versions_asset_number')
)
op.create_index(op.f('ix_file_versions_blob_id'), 'file_versions', ['blob_id'], unique=False)
op.create_index(op.f('ix_file_versions_checksum_sha256'), 'file_versions', ['checksum_sha256'], unique=False)
op.create_index(op.f('ix_file_versions_created_by_user_id'), 'file_versions', ['created_by_user_id'], unique=False)
op.create_index(op.f('ix_file_versions_file_asset_id'), 'file_versions', ['file_asset_id'], unique=False)
op.create_index(op.f('ix_file_versions_tenant_id'), 'file_versions', ['tenant_id'], unique=False)
op.create_table('campaign_jobs',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('campaign_id', sa.String(length=36), nullable=False),
sa.Column('campaign_version_id', sa.String(length=36), nullable=False),
sa.Column('entry_index', sa.Integer(), nullable=False),
sa.Column('entry_id', sa.String(length=255), nullable=True),
sa.Column('recipient_email', sa.String(length=320), nullable=True),
sa.Column('subject', sa.String(length=998), nullable=True),
sa.Column('message_id_header', sa.String(length=255), nullable=True),
sa.Column('eml_storage_key', sa.String(length=1000), nullable=True),
sa.Column('eml_local_path', sa.String(length=1000), nullable=True),
sa.Column('eml_size_bytes', sa.Integer(), nullable=True),
sa.Column('eml_sha256', sa.String(length=64), nullable=True),
sa.Column('build_status', sa.String(length=50), nullable=False),
sa.Column('validation_status', sa.String(length=50), nullable=False),
sa.Column('queue_status', sa.String(length=50), nullable=False),
sa.Column('send_status', sa.String(length=50), nullable=False),
sa.Column('imap_status', sa.String(length=50), nullable=False),
sa.Column('attempt_count', sa.Integer(), nullable=False),
sa.Column('last_error', sa.Text(), nullable=True),
sa.Column('queued_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('claimed_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('claim_token', sa.String(length=36), nullable=True),
sa.Column('smtp_started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('outcome_unknown_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('sent_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('resolved_recipients', sa.JSON(), nullable=True),
sa.Column('resolved_attachments', sa.JSON(), nullable=False),
sa.Column('issues_snapshot', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_jobs_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_jobs_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_jobs')),
sa.UniqueConstraint('campaign_version_id', 'entry_index', name='uq_campaign_jobs_version_entry')
)
op.create_index(op.f('ix_campaign_jobs_build_status'), 'campaign_jobs', ['build_status'], unique=False)
op.create_index(op.f('ix_campaign_jobs_campaign_id'), 'campaign_jobs', ['campaign_id'], unique=False)
op.create_index(op.f('ix_campaign_jobs_campaign_version_id'), 'campaign_jobs', ['campaign_version_id'], unique=False)
op.create_index(op.f('ix_campaign_jobs_claim_token'), 'campaign_jobs', ['claim_token'], unique=False)
op.create_index(op.f('ix_campaign_jobs_eml_sha256'), 'campaign_jobs', ['eml_sha256'], unique=False)
op.create_index(op.f('ix_campaign_jobs_entry_id'), 'campaign_jobs', ['entry_id'], unique=False)
op.create_index(op.f('ix_campaign_jobs_imap_status'), 'campaign_jobs', ['imap_status'], unique=False)
op.create_index(op.f('ix_campaign_jobs_queue_status'), 'campaign_jobs', ['queue_status'], unique=False)
op.create_index(op.f('ix_campaign_jobs_recipient_email'), 'campaign_jobs', ['recipient_email'], unique=False)
op.create_index(op.f('ix_campaign_jobs_send_status'), 'campaign_jobs', ['send_status'], unique=False)
op.create_index(op.f('ix_campaign_jobs_tenant_id'), 'campaign_jobs', ['tenant_id'], unique=False)
op.create_index(op.f('ix_campaign_jobs_validation_status'), 'campaign_jobs', ['validation_status'], unique=False)
op.create_table('campaign_attachment_uses',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('campaign_id', sa.String(length=36), nullable=False),
sa.Column('campaign_version_id', sa.String(length=36), nullable=False),
sa.Column('campaign_job_id', sa.String(length=36), nullable=True),
sa.Column('entry_index', sa.Integer(), nullable=True),
sa.Column('entry_id', sa.String(length=255), nullable=True),
sa.Column('file_asset_id', sa.String(length=36), nullable=False),
sa.Column('file_version_id', sa.String(length=36), nullable=False),
sa.Column('file_blob_id', sa.String(length=36), nullable=False),
sa.Column('filename_used', sa.String(length=500), nullable=False),
sa.Column('checksum_sha256', sa.String(length=64), nullable=False),
sa.Column('size_bytes', sa.Integer(), nullable=False),
sa.Column('content_type', sa.String(length=255), nullable=True),
sa.Column('use_stage', sa.String(length=20), nullable=False),
sa.Column('used_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_attachment_uses_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_job_id'], ['campaign_jobs.id'], name=op.f('fk_campaign_attachment_uses_campaign_job_id_campaign_jobs'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_attachment_uses_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['file_asset_id'], ['file_assets.id'], name=op.f('fk_campaign_attachment_uses_file_asset_id_file_assets'), ondelete='RESTRICT'),
sa.ForeignKeyConstraint(['file_blob_id'], ['file_blobs.id'], name=op.f('fk_campaign_attachment_uses_file_blob_id_file_blobs'), ondelete='RESTRICT'),
sa.ForeignKeyConstraint(['file_version_id'], ['file_versions.id'], name=op.f('fk_campaign_attachment_uses_file_version_id_file_versions'), ondelete='RESTRICT'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_attachment_uses')),
sa.UniqueConstraint('campaign_job_id', 'file_version_id', 'filename_used', 'use_stage', name='uq_campaign_attachment_uses_job_file_stage')
)
op.create_index(op.f('ix_campaign_attachment_uses_campaign_id'), 'campaign_attachment_uses', ['campaign_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_campaign_job_id'), 'campaign_attachment_uses', ['campaign_job_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_campaign_version_id'), 'campaign_attachment_uses', ['campaign_version_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_entry_id'), 'campaign_attachment_uses', ['entry_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_file_asset_id'), 'campaign_attachment_uses', ['file_asset_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_file_blob_id'), 'campaign_attachment_uses', ['file_blob_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_file_version_id'), 'campaign_attachment_uses', ['file_version_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_tenant_id'), 'campaign_attachment_uses', ['tenant_id'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_use_stage'), 'campaign_attachment_uses', ['use_stage'], unique=False)
op.create_index(op.f('ix_campaign_attachment_uses_used_at'), 'campaign_attachment_uses', ['used_at'], unique=False)
op.create_table('campaign_issues',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('campaign_id', sa.String(length=36), nullable=False),
sa.Column('campaign_version_id', sa.String(length=36), nullable=True),
sa.Column('job_id', sa.String(length=36), nullable=True),
sa.Column('severity', sa.String(length=20), nullable=False),
sa.Column('code', sa.String(length=100), nullable=False),
sa.Column('message', sa.Text(), nullable=False),
sa.Column('source', sa.String(length=255), nullable=True),
sa.Column('behavior', sa.String(length=50), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['campaign_id'], ['campaigns.id'], name=op.f('fk_campaign_issues_campaign_id_campaigns'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['campaign_version_id'], ['campaign_versions.id'], name=op.f('fk_campaign_issues_campaign_version_id_campaign_versions'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_campaign_issues_job_id_campaign_jobs'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_issues'))
)
op.create_index(op.f('ix_campaign_issues_campaign_id'), 'campaign_issues', ['campaign_id'], unique=False)
op.create_index(op.f('ix_campaign_issues_campaign_version_id'), 'campaign_issues', ['campaign_version_id'], unique=False)
op.create_index(op.f('ix_campaign_issues_code'), 'campaign_issues', ['code'], unique=False)
op.create_index(op.f('ix_campaign_issues_job_id'), 'campaign_issues', ['job_id'], unique=False)
op.create_index(op.f('ix_campaign_issues_severity'), 'campaign_issues', ['severity'], unique=False)
op.create_index(op.f('ix_campaign_issues_tenant_id'), 'campaign_issues', ['tenant_id'], unique=False)
op.create_table('imap_append_attempts',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('job_id', sa.String(length=36), nullable=False),
sa.Column('attempt_number', sa.Integer(), nullable=False),
sa.Column('folder', sa.String(length=500), nullable=True),
sa.Column('status', sa.String(length=50), nullable=False),
sa.Column('error_message', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_imap_append_attempts_job_id_campaign_jobs'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_imap_append_attempts'))
)
op.create_index(op.f('ix_imap_append_attempts_job_id'), 'imap_append_attempts', ['job_id'], unique=False)
op.create_table('send_attempts',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('job_id', sa.String(length=36), nullable=False),
sa.Column('attempt_number', sa.Integer(), nullable=False),
sa.Column('status', sa.String(length=50), nullable=False),
sa.Column('claim_token', sa.String(length=36), nullable=True),
sa.Column('smtp_status_code', sa.Integer(), nullable=True),
sa.Column('smtp_response', sa.Text(), nullable=True),
sa.Column('error_type', sa.String(length=255), nullable=True),
sa.Column('error_message', sa.Text(), nullable=True),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['job_id'], ['campaign_jobs.id'], name=op.f('fk_send_attempts_job_id_campaign_jobs'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_send_attempts'))
)
op.create_index(op.f('ix_send_attempts_claim_token'), 'send_attempts', ['claim_token'], unique=False)
op.create_index(op.f('ix_send_attempts_job_id'), 'send_attempts', ['job_id'], unique=False)
op.create_index(op.f('ix_send_attempts_status'), 'send_attempts', ['status'], unique=False)
_seed_core_defaults()
def downgrade() -> None:
op.drop_table('send_attempts')
op.drop_table('imap_append_attempts')
op.drop_table('campaign_issues')
op.drop_table('campaign_attachment_uses')
op.drop_table('campaign_jobs')
op.drop_table('file_versions')
op.drop_table('file_shares')
op.drop_table('campaign_versions')
op.drop_table('campaign_shares')
op.drop_table('audit_log')
op.drop_table('attachment_instances')
op.drop_table('mail_server_profiles')
op.drop_table('file_folders')
op.drop_table('file_assets')
op.drop_table('campaigns')
op.drop_table('access_user_role_assignments')
op.drop_table('access_user_group_memberships')
op.drop_table('access_auth_sessions')
op.drop_table('access_api_keys')
op.drop_table('admin_governance_template_assignments')
op.drop_table('access_users')
op.drop_table('access_system_role_assignments')
op.drop_table('access_group_role_assignments')
op.drop_table('file_blobs')
op.drop_table('core_system_settings')
op.drop_table('core_change_sequence_retention_floor')
op.drop_table('core_change_sequence')
op.drop_table('audit_outbox_events')
op.drop_table('attachment_blobs')
op.drop_table('admin_governance_templates')
op.drop_table('access_roles')
op.drop_table('access_groups')
op.drop_table('access_accounts')
op.drop_table('core_scopes')

343
docs/ACCESS_RBAC_MODEL.md Normal file
View File

@@ -0,0 +1,343 @@
# GovOPlaN RBAC And Resource-Access Model
**Updated:** 2026-07-11
## Authorization Equation
An operation is permitted only when every applicable layer allows it:
```text
effective role/API-key capability
AND resource ownership/share access
AND workflow state
AND active governance/policy constraints
```
RBAC answers what an actor may do. ACLs answer which resource the actor may do
it to. Workflow state and policy decide whether the operation is currently
valid.
## Identity And Scope
```text
Account global login identity
+- User membership tenant-local identity
+- direct tenant roles
+- active group memberships
| +- inherited tenant roles
+- tenant-local API keys
Account
+- direct system-role assignments
```
A browser session has one active tenant membership. System privileges do not
silently grant tenant data access. API keys remain tenant-local and receive the
intersection of their configured scopes and their owner's live tenant scopes on
every request.
## Wildcards
```text
tenant:* every canonical tenant permission
system:* every canonical system permission
* legacy alias interpreted as tenant:* only
```
Tenant wildcards never grant system permissions.
## Canonical Tenant Permissions
Campaigns:
```text
campaign:read
campaign:create
campaign:update
campaign:copy
campaign:archive
campaign:delete
campaign:share
campaign:validate
campaign:build
campaign:review
campaign:send_test
campaign:queue
campaign:control
campaign:send
campaign:retry
campaign:reconcile
```
Recipients:
```text
recipients:read
recipients:write
recipients:import
recipients:export
```
Files:
```text
files:read
files:download
files:upload
files:organize
files:share
files:delete
files:admin
```
Reports and audit:
```text
reports:read
reports:export
reports:send
audit:read
```
Mail servers:
```text
mail_servers:read
mail_servers:use
mail_servers:test
mail_servers:write
mail_servers:manage_credentials
```
Tenant administration:
```text
admin:users:read
admin:users:create
admin:users:update
admin:users:suspend
admin:groups:read
admin:groups:write
admin:groups:manage_members
admin:roles:read
admin:roles:write
admin:roles:assign
admin:api_keys:read
admin:api_keys:create
admin:api_keys:revoke
admin:settings:read
admin:settings:write
admin:policies:read
admin:policies:write
```
## Canonical System Permissions
```text
system:tenants:read
system:tenants:create
system:tenants:update
system:tenants:suspend
system:accounts:read
system:accounts:create
system:accounts:update
system:accounts:suspend
system:roles:read
system:roles:write
system:roles:assign
system:access:read
system:access:assign
system:audit:read
system:settings:read
system:settings:write
system:governance:read
system:governance:write
```
`system:access:*` remains a read/assignment boundary for cross-tenant and
system access handling. It is not a separate primary UI area.
## Default Tenant Roles
- **Owner:** `tenant:*`. At least one active operational owner must remain.
- **Tenant administrator:** settings, policies, users, groups, roles, API keys,
and read access to campaigns/files/reports/audit. Real delivery remains
separately delegable.
- **Administrator:** all tenant permissions for upgraded installations.
- **Access administrator:** membership and assignment management within
delegation limits.
- **Campaign manager:** prepare, validate, and build campaigns; no review
approval or real delivery by default.
- **Reviewer:** inspect and approve prepared campaign messages.
- **Sender:** mock-test, queue, control, send, retry, and reconcile prepared
campaigns; can use/test approved mail profiles.
- **File manager:** managed file operations without campaign delivery rights.
- **Viewer:** read campaigns, recipients, files, and reports.
- **Auditor:** read campaigns, recipient evidence, reports, and audit records;
export detailed evidence.
## Default System Roles
- **System owner:** `system:*`, protected. At least one active account must
retain it.
- **System administrator:** all specific system permissions, editable and not
protected.
- **System auditor:** read-only system registry/settings/governance/audit role,
editable.
## Delegation Ceiling
For role definition, assignment, and API-key creation:
```text
requested scopes subset of actor delegateable scopes
```
Rules:
1. Tenant roles may contain tenant scopes only.
2. System roles may contain system scopes only.
3. Definition rights and assignment rights are separate.
4. Group definition and group membership management are separate.
5. API-key scopes are intersected with the owner's current effective scopes on
every request.
6. Suspended accounts, users, tenants, or groups stop contributing access
immediately.
7. Administrative updates are field-sensitive; a user with only status
authority cannot change role assignments.
## Campaign Ownership And ACLs
A campaign has exactly one owner:
```text
owner user OR owner group
```
Additional active shares may target users or groups with `read` or `write`.
Resolution:
- owner user: read and write;
- member of owner group: read and write;
- explicit read share: read;
- explicit write share: read and write;
- `tenant:*`: tenant-wide ACL bypass;
- ordinary campaign permission without ownership/share: no object access.
ACLs do not add capabilities. A write share still needs the specific permission
for update, validation, review, send, report, retry, or reconciliation.
## Files
The current file access model distinguishes tenant-level file capabilities from
space/folder/file ownership:
| Scope | Meaning |
| --- | --- |
| `files:read` | browse/read visible file spaces and metadata |
| `files:download` | download file content when ACL permits |
| `files:upload` | create files in writable spaces |
| `files:organize` | create folders, move files, and update metadata where ACL permits |
| `files:share` | share files/spaces according to owner and policy rules |
| `files:delete` | delete or retire files where ACL permits |
| `files:admin` | tenant-wide administration of user/group file spaces |
External file connections and spaces are additionally constrained by connector
policy and owner/group assignment in the files module.
## Resource Access Explanations
The access module exposes a diagnostic endpoint for explaining why a principal
can see or operate on a concrete resource:
```text
GET /api/v1/admin/access/resource-explanation
```
Required query values:
| Field | Meaning |
| --- | --- |
| `user_id` | Tenant membership to explain. The current module UIs pass the signed-in user. |
| `resource_type` | Module-owned type such as `file`, `folder`, or `campaign`. |
| `resource_id` | Stable module resource identifier. |
| `action` | Permission/action being explained, for example `files:file:read`. |
| `tenant_id` | Optional tenant override for system/admin contexts. |
The access module always contributes effective-scope provenance for the action.
Installed modules may add resource provenance by exposing a
`ResourceAccessExplanationProvider` through a capability consumed by access.
Providers should return only facts they own, using these provenance kinds:
| Kind | Meaning |
| --- | --- |
| `resource` | The concrete resource or an explicit not-found result. |
| `owner` | Matching user/group ownership. |
| `share` | Matching explicit user/group/tenant share. |
| `policy` | Administrative bypass or policy-derived grant. |
| `role` / `right` | Scope and role provenance from access itself. |
Files currently registers `files.access` and explains file assets plus folders.
Persisted folders use their database ID. Folder rows inferred from file paths use
a deterministic virtual ID:
```text
virtual-folder:v1:<tenant_id>:<owner_type>:<owner_id>:<base64url(normalized_path)>
```
The files provider validates virtual folder IDs before returning provenance: the
tenant and owner must match the resource ID, and at least one active file must
exist below the normalized folder path. This keeps virtual folder explanations
stable without forcing every inferred tree node to become a stored folder row.
Campaign currently registers `campaigns.access` and explains the campaign
ownership/sharing object itself. Finer-grained campaign sub-objects are tracked
separately in `govoplan-campaign#50` until their resource identifiers and access
rules are decided.
Cross-user resource explanation is a policy feature, not a module-local UI
detail. Until `govoplan-policy#6` is resolved, module UIs should default to
current-user explanation and avoid importing access-admin user-picking
components.
## Mail Servers
| Scope | Meaning |
| --- | --- |
| `mail_servers:read` | profile metadata and effective policy visibility |
| `mail_servers:use` | use an allowed profile for a campaign or message flow |
| `mail_servers:test` | run connectivity tests without revealing secrets |
| `mail_servers:write` | create/update profile metadata where policy allows |
| `mail_servers:manage_credentials` | create/replace SMTP/IMAP secrets or lower-level credentials where policy allows |
Reusable encrypted profiles exist. Effective usability is also constrained by
hierarchical mail-profile policy, ownership, allowed/forced profile sets,
credential inheritance mode, lower-level override switches, and allow/deny
patterns.
## Compatibility Aliases
Compatibility aliases may exist in backend code for upgraded installations, but
new UI and docs should use canonical scopes.
Current alias direction:
```text
* -> tenant:*
system:tenants:write -> create/update/suspend tenant scopes
system:access:write -> system access assignment/write scopes
```
A separate `retention:*` family is not currently canonical because retention is
managed through system settings and tenant policy scopes. Add it only if
retention operation duties need separation from general policy/settings
administration.

View File

@@ -0,0 +1,121 @@
# Action, Effect, And Automation Layer
GovOPlaN needs an automation layer because administrative processes will not be
only linear screen flows. Workflows, schedules, imports, connectors, policies,
and external events all need to request governed actions without bypassing the
same safety rules that apply to human users.
The first implementation should live in `govoplan-workflow` and core contracts.
Create a separate `govoplan-automation` module only if action planning,
schedulers, rule execution, or cross-module automation become too broad for
workflow ownership.
## Layer Purpose
The automation layer should provide:
- a typed action catalogue
- a typed effect catalogue
- consequence preview before execution
- policy and permission checks
- idempotent execution
- audit and provenance records
- retry, quarantine, and manual exception handling
- system-actor execution without hiding responsibility
Automation is not a shortcut around module boundaries. It is a governed caller
of module capabilities.
## Action Definition
An `ActionDefinition` describes something a human or system actor can request.
Recommended fields:
- `action_key`
- owning module
- input schema
- actor requirements and required scopes
- required capabilities
- policy checks
- risk level
- reversibility class: reversible, compensatable, corrective-only, or
irreversible
- expected effects
- idempotency key strategy
- audit event names
- preview provider
Examples:
- create a case from a form submission
- assign a task
- generate a document from a template
- send a postbox message
- send an email notification
- append evidence to records
- create a payment request
- call an external connector
## Effect Definition
An `EffectDefinition` describes the expected and observed result of an action.
Recommended fields:
- `effect_key`
- affected module and resource references
- external system references where applicable
- created, changed, deleted, sent, notified, locked, or retained markers
- visibility and privacy classification
- audit event references
- rollback or compensation hints
- operator-facing explanation
Effects should be recorded even when execution fails partially. This makes
manual recovery and audit review possible.
## Execution Model
The runner should execute an action plan as follows:
1. Resolve actor context: human, delegated actor, or system actor.
2. Validate input schema.
3. Resolve required module capabilities.
4. Run permission and policy checks.
5. Generate a consequence preview.
6. Reserve or verify the idempotency key.
7. Execute the owning module capability.
8. Record observed effects.
9. Emit events and audit records.
10. Mark the command complete, retryable, quarantined, or requiring manual
intervention.
The runner must never advance workflow state past a required side effect unless
the action definition explicitly allows asynchronous completion and the pending
state is visible.
## Failure States
Automation should use explicit failure states:
- `blocked`: policy, permission, missing capability, or invalid input prevents
execution.
- `retryable`: transient transport, timeout, rate-limit, or lock conflict.
- `quarantined`: unexpected response, schema mismatch, unsafe partial result,
or unknown external state.
- `manual_required`: human decision or correction is needed.
- `compensation_required`: a later action must correct an already observed
side effect.
These states should be visible in workflow, task, and admin diagnostics.
## Boundary
Core may own stable DTOs, registry contracts, and generic audit/event hooks.
`govoplan-workflow` should own the first runner because workflow is the first
module that coordinates cross-module process actions.
Domain modules own their own action providers. For example, templates own
document generation actions, postbox owns postbox message actions, and
connectors own external handoff actions.

View File

@@ -24,6 +24,15 @@ network_access = false
[projects."/mnt/DATA/git/govoplan-core"]
trust_level = "trusted"
[projects."/mnt/DATA/git/govoplan-access"]
trust_level = "trusted"
[projects."/mnt/DATA/git/govoplan-organizations"]
trust_level = "trusted"
[projects."/mnt/DATA/git/govoplan-identity"]
trust_level = "trusted"
[projects."/mnt/DATA/git/govoplan-mail"]
trust_level = "trusted"
@@ -42,23 +51,25 @@ Each active repository has an `AGENTS.md` file. These files define ownership, mo
Use `~/.codex/config.toml` for personal defaults, auth/runtime settings, writable roots, and trust decisions. Avoid checking in absolute-path writable roots or model preferences unless they are intentionally team-wide.
Use Gitea issues as the canonical backlog and state log. See `docs/GITEA_ISSUES.md` for label setup, TODO import, and Codex issue update commands. Durable docs should describe stable behavior; changing work state belongs on the issue.
## Focused Verification
Use the consolidated script after changes that touch module discovery, optional integrations, shared mail components, mailbox listing, or cross-module WebUI behavior:
```bash
cd /mnt/DATA/git/govoplan-core
./scripts/check-focused.sh
cd /mnt/DATA/git/govoplan
tools/checks/check-focused.sh
```
For smaller changes, prefer the narrow command named in the relevant `AGENTS.md` file. Examples:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m unittest tests.test_module_system
/mnt/DATA/git/govoplan/.venv/bin/python -m unittest tests.test_module_system
cd /mnt/DATA/git/govoplan-mail
/mnt/DATA/git/govoplan-core/.venv/bin/python -m unittest discover -s tests
/mnt/DATA/git/govoplan/.venv/bin/python -m unittest discover -s tests
cd /mnt/DATA/git/govoplan-core/webui
PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run test:module-permutations
@@ -70,4 +81,5 @@ PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versi
- Avoid broad recursive scans and full builds unless the change warrants them.
- Keep generated build/test folders ignored.
- Keep optional module behavior behind core registry/capability/module metadata boundaries.
- Create or update Gitea issues for TODOs, follow-ups, blockers, and feature requests instead of keeping local backlog files.
- Do not start persistent dev servers unless the user asks.

View File

@@ -0,0 +1,335 @@
# GovOPlaN Configuration Packages
Configuration packages are reusable, versioned preconfigurations for a working
GovOPlaN capability. They sit above modules: a module installs code,
migrations, routes, permissions, and capabilities; a configuration package wires
installed modules into a concrete operational setup.
Example: an application-handling package could configure a public portal form,
a case workflow, task creation, a mail template, payment processing, access
roles, audit evidence, and the interface bindings between those modules.
The guiding reference scenario is the government-operations permit journey in
`docs/GOVOPLAN_MASTER_ROADMAP.md`: a person applies through the portal,
uploads files, receives workflow-driven messages and appointment proposals, has
a case opened, gets a permit generated from a template, and completes payment.
Configuration packages are the mechanism that should make such processes
reusable and safely importable.
This document is durable architecture context and should be mirrored to the
Gitea wiki. Active implementation should be tracked in Gitea issues.
## Goals
- Import a preconfiguration from a trusted catalog or uploaded package.
- Detect which modules, module versions, and capabilities are required.
- Detect which configuration fragments must be created, updated, or bound.
- Explain import problems and offer actionable resolution paths.
- Ask the operator only for data needed to make the package work.
- Export an existing working configuration as a reusable package.
- Support signed catalogs so approved configurations can be shared, adapted,
re-exported, and provided to other installations.
## Vocabulary
| Term | Meaning |
| --- | --- |
| Module | Installable product code with manifest, migrations, routes, permissions, WebUI, events, and capabilities. |
| Configuration | Module-owned settings and resources that make behavior concrete: workflows, forms, templates, roles, policies, connector profiles, routing rules, and defaults. |
| Interface | A typed contract between modules: capabilities, commands, events, DTOs, data bindings, and UI extension points. |
| Data | Operator- or tenant-provided values needed for a working setup: legal text, sender addresses, account mappings, payment provider credentials, templates, defaults, and optionally reference data. |
The system should communicate these four layers explicitly:
```text
module = what can exist
configuration = what should exist here
interface = how configured parts connect
data = what the operator must provide for this deployment
```
## Package Model
A configuration package should be a signed, portable manifest plus module-owned
configuration fragments. The package is not a database dump. It should be
declarative, idempotent, tenant-aware, and explicit about secrets and external
systems.
Required package metadata:
- stable package id, name, version, description, publisher, and license
- supported GovOPlaN core/module compatibility bounds
- required modules with version constraints
- optional modules with conditional behavior
- required capabilities, commands, event subscriptions, and UI extension points
- configuration fragments grouped by owning module
- interface bindings between fragments
- data requirements to collect from the operator
- preflight checks and post-import health checks
- migration or transformation rules for older package versions
- provenance, export source metadata, and signature metadata
Configuration fragments are interpreted only by the module that owns them. For
example, workflow imports workflow definitions; forms imports form schemas;
mail imports mail templates and delivery defaults; payments imports payment
profiles; access imports groups, roles, and permission assignments.
## Module Provider Contract
Each module that wants to participate should expose a configuration-package
provider capability. The core orchestrator should not understand module internals
or write module-owned tables directly.
Baseline provider responsibilities:
- publish JSON Schema or equivalent typed schemas for importable/exportable
fragments
- publish data requirements that can be rendered by a generic wizard
- validate fragments without applying them
- report missing dependencies, missing permissions, conflicts, and unsafe
changes
- produce a dry-run plan with create, update, bind, skip, and blocked actions
- apply fragments idempotently inside module-owned boundaries
- export selected module-owned configuration into portable fragments
- redact secrets and mark secret placeholders during export
- provide post-apply health checks and operator-facing diagnostics
Provider operations should be versioned. The first stable contract can be small:
```text
describe() -> schemas, supported fragment types, exported scopes
preflight(package_fragment, context) -> diagnostics, required_data, plan
apply(package_fragment, supplied_data, context) -> result, created_refs
export(selection, context) -> fragment, data_placeholders, warnings
health(import_result, context) -> diagnostics
```
The initial core contract lives in
`govoplan_core.core.configuration_packages`. Modules register providers through
module-specific capability keys and implement the `ConfigurationProvider`
protocol. The generic `configuration.provider` key names the contract and can be
used in package requirements; concrete providers such as `access.configuration`
are resolved by the admin package wizard. The first DTO surface includes package
manifests, module/capability requirements, module-owned fragments, diagnostics,
required operator data, dry-run plan items, apply results, export selections,
and export results.
The initial implementation includes provider-neutral orchestration helpers:
- `dry_run_configuration_package(...)`
- `apply_configuration_package(...)`
- `export_configuration_package(...)`
The first concrete provider is `govoplan_access.backend.configuration_provider`.
It supports access-owned `roles`, `groups`, and `group_role_assignments`
fragments and applies them idempotently.
The admin wizard backend starts with these routes:
- `GET /api/v1/admin/configuration-packages/catalog`
- `POST /api/v1/admin/configuration-packages/dry-run`
- `POST /api/v1/admin/configuration-packages/apply`
- `POST /api/v1/admin/configuration-packages/export`
## Import Flow
1. Select a package from a trusted catalog or upload a package file.
2. Verify signature, channel, publisher trust, package compatibility, and schema.
3. Resolve required modules against installed modules and the module package
catalog.
4. Produce a dependency plan for modules that must be installed or upgraded.
5. Ask the operator for required data using a focused wizard.
6. Run dry-run preflight across all participating module providers.
7. Show problems grouped by severity, owner module, and resolution.
8. Apply module/package changes in a dependency-safe order.
9. Run post-import health checks and show the final working status.
10. Store import provenance, package version, supplied non-secret metadata, and
audit events.
The wizard should display everything necessary and nothing unnecessary. Generic
sections should cover package trust, dependency plan, required data, conflicts,
review, and result. Module-specific fields should appear only when the selected
package and installed modules require them.
## Problem Model
Import diagnostics should be structured and actionable, not free-text logs.
Every problem should include severity, owner, affected object, explanation, and
at least one suggested resolution when the system can infer it.
Common diagnostic types:
- missing module or incompatible module version
- missing capability or disabled optional integration
- missing operator-supplied data
- missing permission or insufficient administrator scope
- unresolved interface binding between modules
- conflicting existing configuration
- external service not reachable or credential test failed
- policy or tenant-governance violation
- unsafe overwrite, downgrade, or destructive change
- schema validation failure
- post-import health-check failure
Resolution actions can include install module, upgrade module, enable
integration, provide value, choose existing object, rename imported object,
skip optional fragment, replace existing configuration, or cancel import.
## Export Flow
Export should be possible from a working tenant or system configuration, but it
must be intentional about data boundaries.
1. Select export scope: system, tenant, module set, workflow, form, case type,
portal flow, or another domain object.
2. Ask participating modules to export owned fragments and data placeholders.
3. Classify exported material as configuration, reference data, sample data,
secrets, or deployment-local data.
4. Redact secrets by default and replace them with data requirements.
5. Let the operator choose whether to include reference/sample data.
6. Validate the assembled package by running the same preflight path against a
clean target context where possible.
7. Sign the package or produce an unsigned development package.
8. Optionally publish the package metadata to a configuration catalog.
Exported packages should record provenance: source GovOPlaN version, module
versions, exporter identity, timestamp, selected scope, redactions, and
validation status.
## Catalogs And Trust
Configuration catalogs should follow the existing module package catalog model:
a file-backed or remotely fetched JSON catalog with Ed25519 signatures, channel
gating, trusted key ids, and operator-controlled trust policy.
The initial catalog validator mirrors the module catalog environment model with
configuration-specific names:
```bash
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG=/srv/govoplan/configuration-catalogs/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_URL=https://govoplan.example/configuration-catalogs/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/configuration-catalog-cache/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_APPROVED_CHANNELS=stable,lts
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/configuration-catalog-keyring.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/configuration-catalog-sequences.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true
```
Catalog entries should include:
- package id, version, name, description, publisher, tags, and channel
- package artifact URL or repository ref
- signature metadata for the package artifact
- required modules and version ranges for quick compatibility display
- package category such as workflow, portal, case type, governance, connector,
report, or tenant baseline
- maturity flags such as official, verified, example, local, deprecated
The catalog verifies that a package is approved to view and import. The package
itself must still be signed and validated before use.
## Example Package
Application handling through a public portal:
Required modules:
- `portal` for the public user-facing entry point
- `forms` for the application form and validation
- `cases` for the case record and case type
- `workflow` for status transitions and automation
- `tasks` for internal task creation
- `templates` for mail/template rendering
- `mail` for delivery profile and outbound message sending
- `payments` for provider configuration and payment capture
- `access` for roles, groups, and permissions
- `audit` for import, case, mail, and payment evidence
Required configuration:
- portal route and access policy
- form schema, validation rules, confirmation texts, and attachments
- case type, fields, lifecycle states, and retention classification
- workflow steps, transitions, assignees, and completion triggers
- task template and queue/group assignment
- mail template and delivery rule
- payment product, amount rules, provider profile, and webhook binding
- access roles/groups for clerks, reviewers, supervisors, and operators
- audit event categories and evidence retention defaults
Required operator data:
- tenant or organizational unit
- service name and public contact details
- portal URL/path and legal imprint/privacy text
- mail sender profile and reply-to mailbox
- payment provider credentials or test-mode selection
- responsible groups or users for task assignment
- escalation contacts and deadlines
- template wording and localized text overrides
Potential problems:
- `payments` is missing or the provider capability is unavailable
- mail transport test fails for the selected sender profile
- imported role names conflict with existing tenant roles
- workflow references a task queue that does not exist
- public portal base URL is not configured
- required privacy/legal text is empty
- webhook endpoint cannot be validated from the payment provider
## Ownership
Core should own:
- package and catalog signature validation
- dependency resolution against installed modules and module catalogs
- orchestration of provider preflight/apply/export operations
- generic import/export APIs and audit envelope
- generic wizard shell and problem display components
Modules should own:
- schemas and semantics for their own fragments
- module-specific validation, apply, export, and health checks
- module-specific UI capabilities only when generic generated controls are not
sufficient
- redaction and classification of module-owned secrets or sensitive data
Access should own configuration fragments for users, groups, roles,
permissions, API keys, and principal mappings. It should not own workflow, mail,
payment, form, or portal semantics.
Admin should expose the operator-facing catalog, import, export, and history
screens through admin route contributions. The UI should keep the conceptual
layers visible: modules, configuration, interfaces, and data.
## Implementation Slices
1. Define package manifest and diagnostic schemas.
2. Add core configuration-package provider capability contracts.
3. Implement catalog validation and package signature verification.
4. Add dry-run orchestration against mock providers.
5. Add admin catalog/import wizard screens using provider data requirements.
6. Implement export/import providers for access-owned roles/groups first.
7. Add providers in workflow/forms/templates/mail/payments/portal/cases/tasks as
those modules mature.
8. Add round-trip tests: export a known setup, import into a clean tenant,
verify health checks.
9. Add documentation and field-level help for package authors and operators.
## Acceptance Criteria For Tracking Issue
- A Gitea tracking issue exists in `govoplan-core` for the cross-cutting
feature.
- The issue links module-specific follow-ups when implementation begins.
- The first implementation exposes typed contracts rather than direct
cross-module imports.
- A signed example catalog and unsigned development fixture exist.
- A dry-run import can identify required modules, missing data, and conflicts
before applying changes.
- An export can produce a package with secrets redacted into data requirements.
- Admin UI shows a focused wizard and actionable diagnostic list.
- Tests cover package validation, signature failure, dependency resolution,
provider preflight, export redaction, and import idempotency.

70
docs/DEPENDENCY_AUDITS.md Normal file
View File

@@ -0,0 +1,70 @@
# Dependency Audits
GovOPlaN keeps dependency vulnerability checks reproducible but separate from
the fast local smoke suite, because both Python and npm audits need network
metadata and can fail for newly disclosed advisories without a source change.
## Local Workflow
Install the whole-product development dependencies once from the meta
repository:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m pip install -r requirements-dev.txt
```
Run both backend and WebUI production audits:
```bash
cd /mnt/DATA/git/govoplan
bash tools/checks/check-dependency-audits.sh
```
The script runs:
- `tools/checks/check-dependency-hygiene.sh` for pip resolver consistency, stale
legacy editable package metadata, deprecated framework constants, and the
Starlette `TestClient` deprecation smoke when test dependencies are present
- `python -m pip_audit --progress-spinner off`
- `npm audit --omit=dev` in `webui`
For fast local checks without vulnerability metadata lookups, run:
```bash
cd /mnt/DATA/git/govoplan
CHECK_TESTCLIENT_DEPRECATIONS=1 \
bash tools/checks/check-dependency-hygiene.sh
```
This is also part of `govoplan/tools/checks/check-focused.sh`, so resolver drift and
deprecation regressions fail close to the code change that introduced them.
Override tool paths when testing from a disposable environment:
```bash
PYTHON=/tmp/govoplan-audit/bin/python \
NPM=/home/zemion/.nvm/versions/node/v22.22.3/bin/npm \
GOVOPLAN_CORE_ROOT=/mnt/DATA/git/govoplan-core \
bash /mnt/DATA/git/govoplan/tools/checks/check-dependency-audits.sh
```
## CI Workflow
`govoplan/.gitea/workflows/dependency-audit.yml` installs release dependencies from
tagged package refs, installs `pip-audit`, and runs the same script on pushes,
pull requests, and a weekly schedule.
The workflow intentionally uses release dependency refs instead of local
`file:` or editable sibling paths. Development lockfiles may keep local module
links, but release audit results should represent the installable product.
## Recording Results
When closing or triaging dependency-audit issues, add a short dated note under
`docs/audits/`. Record:
- the commands that were run
- whether Python and npm passed
- any advisories accepted as temporary risk
- follow-up issue links for required upgrades

View File

@@ -0,0 +1,426 @@
# GovOPlaN Deployment Operator Guide
This guide defines the current install/runtime configuration contract and the
operator flow for a production-realistic self-hosted deployment. Keep secrets in
the deployment environment or a secret manager; do not commit populated `.env`
files.
## Runtime Configuration Contract
Self-hosted installability follows the staged approach documented in
`SELF_HOSTED_INSTALLABILITY.md`: generate an explicit env template, validate it,
run production-like rehearsal with Compose-backed dependencies, then use the
installer CLI/daemon for package mutation under maintenance mode.
Generate a deployment-local template:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m govoplan_core.commands.config env-template \
--profile self-hosted \
--generate-secrets \
--output .env.self-hosted
```
Validate the active shell environment before migration or startup:
```bash
set -a
. .env.self-hosted
set +a
./.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted
```
### Required Runtime Identity
| Setting | Required outside dev | Purpose |
| --- | --- | --- |
| `APP_ENV` | yes | Runtime profile. Use `prod`, `staging`, or a deployment-specific value outside local development. |
| `MASTER_KEY_B64` | yes | Fernet key or base64 encoded 32-byte key used for encrypted module secrets. Rotate through an explicit operator plan. |
| `DATABASE_URL` | yes | SQLAlchemy database URL for core and installed modules. SQLite is supported for dev/small installs; PostgreSQL is the preferred production target. |
| `ENABLED_MODULES` | yes | Comma-separated startup module set. Keep `tenancy,access` enabled; keep `admin` enabled for operator UI. |
Generate a local key for a new non-production environment:
```bash
python - <<'PY'
from cryptography.fernet import Fernet
print(Fernet.generate_key().decode())
PY
```
### Database And Migrations
| Setting | Default | Notes |
| --- | --- | --- |
| `DATABASE_URL` | `postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev` | Local development and production-like profiles use PostgreSQL. Use `GOVOPLAN_DEV_DATABASE_BACKEND=sqlite` only for disposable SQLite runs. |
| `GOVOPLAN_MIGRATION_TRACK` | `release` | Use the release track for normal runtime and deployments. Use `dev` only for fresh/disposable databases that intentionally replay detailed development migrations. |
| `DEV_AUTO_MIGRATE_ENABLED` | `true` | Dev convenience only. Production should run migration commands explicitly during deployment. |
| `DEV_BOOTSTRAP_ENABLED` | `false` | Dev bootstrap only. `govoplan_core.devserver` and `govoplan/tools/launch/launch-dev.sh` default it to `true`; use controlled first-admin creation outside dev. |
Operator rule: take a database backup before applying migrations or destructive
module retirement. For non-SQLite databases, configure deployment-specific
backup/restore hooks for the module installer.
### PostgreSQL Production Target
PostgreSQL is the primary development and production target. SQLite remains
supported only for tiny disposable profiles and unit-test style smoke runs.
Production/staging deployments should use a managed PostgreSQL database and
explicit migration commands.
Install the full release profile from the meta repository so core, modules, and
the `psycopg` driver are available:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m pip install -r requirements-release.txt
```
Example runtime database URLs:
```bash
export DATABASE_URL='postgresql+psycopg://govoplan:change-me@db.example.internal:5432/govoplan'
export GOVOPLAN_DATABASE_URL_PGTOOLS='postgresql://govoplan:change-me@db.example.internal:5432/govoplan'
```
Use the SQLAlchemy URL for GovOPlaN. Use the pg-tools URL for `pg_dump`,
`pg_restore`, and `psql`; these tools do not understand the
`postgresql+psycopg://` driver marker.
Bootstrap or upgrade the schema explicitly during deployment:
```bash
export APP_ENV=prod
export ENABLED_MODULES=tenancy,access,admin,policy,audit,campaigns,files,mail,calendar,docs,ops
./.venv/bin/python -m govoplan_core.commands.init_db \
--database-url "$DATABASE_URL"
```
Backup and restore-check before migration-bearing package changes:
```bash
pg_dump --format=custom \
--file "$PWD/runtime/govoplan-$(date +%Y%m%d%H%M%S).dump" \
"$GOVOPLAN_DATABASE_URL_PGTOOLS"
pg_restore --list "$PWD/runtime/govoplan-YYYYMMDDHHMMSS.dump" >/dev/null
```
Restore a checked backup to the target database:
```bash
pg_restore --clean --if-exists \
--dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" \
"$PWD/runtime/govoplan-YYYYMMDDHHMMSS.dump"
```
For local development, create the host database described in
`/mnt/DATA/git/govoplan/dev/postgres/README.md`, then run:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m govoplan_core.commands.init_db \
--database-url postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev \
--with-dev-data
./.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
tools/launch/launch-dev.sh
```
For disposable local validation against a throwaway PostgreSQL instance, use
the bundled PostgreSQL testbed:
```bash
cd /mnt/DATA/git/govoplan/dev/postgres
cp .env.example .env
docker compose --env-file .env up -d
cd /mnt/DATA/git/govoplan
set -a
. /mnt/DATA/git/govoplan/dev/postgres/.env
set +a
tools/checks/postgres-integration-check.py \
--database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \
--reset-schema
```
The integration check runs migrations and startup smoke checks across the
standard module permutations. `--reset-schema` is destructive and belongs only
on throwaway databases.
### Broker And Workers
| Setting | Default | Notes |
| --- | --- | --- |
| `REDIS_URL` | `redis://redis:6379/0` | Celery broker/result backend when async workers are enabled. |
| `CELERY_ENABLED` | `false` | Local/dev can send synchronously. Production campaign delivery should run workers and set this to `true`. |
| `CELERY_QUEUES` | `send_email,append_sent,default` | Queue list expected by worker/process manager definitions. |
Worker command:
```bash
python -m celery -A govoplan_core.celery_app:celery worker \
--queues send_email,append_sent,default \
--loglevel INFO
```
### Storage
| Setting | Default | Notes |
| --- | --- | --- |
| `FILE_STORAGE_BACKEND` | `local` | Use `local` for dev/small deployments; use object storage when files must scale independently. |
| `FILE_STORAGE_LOCAL_ROOT` | `runtime/files` | Must live on durable storage and be backed up when `FILE_STORAGE_BACKEND=local`. |
| `FILE_STORAGE_LOCAL_FALLBACK_ROOTS` | empty | Read-only fallback roots for migrated local files. |
| `FILE_STORAGE_S3_ENDPOINT_URL` | empty | Object-store endpoint for the files module. |
| `FILE_STORAGE_S3_REGION` | empty | Object-store region. |
| `FILE_STORAGE_S3_ACCESS_KEY_ID` | empty | Secret; inject through deployment environment. |
| `FILE_STORAGE_S3_SECRET_ACCESS_KEY` | empty | Secret; inject through deployment environment. |
| `FILE_STORAGE_S3_BUCKET` | `files` | Managed-file object bucket. |
Legacy `S3_*` settings remain for older storage paths but new deployments should
prefer `FILE_STORAGE_*`.
### HTTP, Cookies, And Base URLs
| Setting | Default | Notes |
| --- | --- | --- |
| `CORS_ORIGINS` | local dev origins | Set to the exact WebUI origins in staging/production. |
| `AUTH_SESSION_COOKIE_NAME` | configured default | Change only through a controlled rollout because it logs users out. |
| `AUTH_CSRF_COOKIE_NAME` | configured default | Must match WebUI/API deployment. |
| `AUTH_COOKIE_SECURE` | `false` | Set `true` behind HTTPS. |
| `AUTH_COOKIE_SAMESITE` | `lax` | Use a stricter value only after testing login and CSRF flows. |
| `AUTH_COOKIE_DOMAIN` | empty | Set only when the API and WebUI intentionally share a parent domain. |
Public URLs are currently supplied by deployment/reverse-proxy configuration and
module settings. Do not hardcode them in core; configuration packages should ask
for portal, WebUI, postbox, and notification URLs when they become relevant.
### Module Catalogs, Licenses, And Trust Roots
| Setting | Purpose |
| --- | --- |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_URL` or `GOVOPLAN_MODULE_PACKAGE_CATALOG` | Module package catalog source. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE` | Preferred production keyring path. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL` | Approved catalog channel, for example `stable`. |
| `GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE` | Trusted license issuer keyring path. |
| `GOVOPLAN_LICENSE_ENFORCEMENT` | Enables license enforcement when set to `true`. |
Trust roots are deployment-managed and should not be editable through the
running WebUI.
### Mail Test Credentials
Dedicated SMTP/IMAP test credentials belong to the mail/campaign test-bed
configuration, not the core runtime contract. Store them in a local ignored
`.env` file for the test bed or in CI secrets. Required values are:
- SMTP host, port, TLS mode, username, password, and envelope/from address.
- IMAP host, port, TLS mode, username, password, and append folder.
- At least one recipient mailbox that is safe for automated send tests.
## First Deployment Flow
1. Create an environment file or secret set with the runtime contract above.
2. Install the tagged core and module packages from meta `requirements-release.txt`.
3. Build the WebUI from `webui/package.release.json` or deploy a prebuilt
artifact from the same release tag.
4. Run database migrations with the target `DATABASE_URL`.
5. Create the first tenant and system owner through the controlled bootstrap or
one-time admin command for the deployment.
6. Start the API service with `govoplan_core.server.app:app`.
7. Start workers when `CELERY_ENABLED=true`.
8. Start the WebUI/reverse proxy and verify CORS/cookie settings.
9. Open Admin > System > Modules, verify enabled modules, and save desired
module state if it differs from `ENABLED_MODULES`.
10. Run health checks:
```bash
curl -fsS http://127.0.0.1:8000/health
curl -fsS http://127.0.0.1:8000/api/v1/platform/modules
```
Authenticated health details require `system:settings:read`:
```bash
curl -fsS -H "X-API-Key: $GOVOPLAN_HEALTH_API_KEY" \
http://127.0.0.1:8000/health/details
```
## Production-Like Dev Profile
Use this profile to verify deployment behavior without publishing packages or
using real production credentials. The canonical launcher keeps API, worker, and
WebUI code in the editable repositories while Docker provides PostgreSQL and
Redis:
```bash
cd /mnt/DATA/git/govoplan
tools/launch/launch-production-like-dev.sh
```
The helper wrapper provides explicit lifecycle commands:
```bash
tools/launch/production-like-dev.sh validate-config
tools/launch/production-like-dev.sh seed
tools/launch/production-like-dev.sh start
tools/launch/production-like-dev.sh stop
tools/launch/production-like-dev.sh reset --yes
```
The launcher uses `govoplan/dev/production-like/.env` when present, otherwise
the checked in `.env.example`. It runs:
- PostgreSQL on `127.0.0.1:55433`
- Redis on `127.0.0.1:56379`
- explicit `ENABLED_MODULES`
- explicit migrations and `--with-dev-data` bootstrap
- API via the module-aware devserver
- a Celery worker for `send_email,append_sent,default`
- WebUI through the Vite dev server
- durable local files under `runtime/production-like/files`
This profile validates explicit migration execution, config loading, module
discovery, route aggregation, local storage paths, Redis broker connectivity,
worker heartbeats, and health/readiness startup without real production
credentials. It does not replace a managed PostgreSQL/Redis/WebUI/worker
deployment test.
To stop PostgreSQL and Redis when the launcher exits:
```bash
GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT=1 tools/launch/launch-production-like-dev.sh
```
## Module Install/Uninstall Operations
Use Admin > System > Modules for planning. The running API server validates and
queues install plans; it does not run pip/npm or restart itself from an HTTP
request. Package mutation belongs to the trusted installer CLI/daemon in an
operator shell while maintenance mode is active.
Preflight from the server shell:
```bash
govoplan-module-installer --format shell
```
Apply a prepared plan directly from a controlled shell:
```bash
govoplan-module-installer --apply --build-webui
```
For production-like runs, prefer supervised mode with migrations, database
backup/restore hooks, restart commands, and health checks:
```bash
govoplan-module-installer \
--supervise \
--migrate \
--restart-command 'systemctl restart govoplan-api' \
--restart-command 'systemctl restart govoplan-worker' \
--health-url http://127.0.0.1:8000/health \
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"'
```
To let the admin UI submit work without executing package managers inside the
API process, run the daemon in a separate operator shell:
```bash
govoplan-module-installer \
--daemon \
--migrate \
--build-webui \
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--health-url http://127.0.0.1:8000/health \
--restart-command '<restart govoplan server>'
```
The daemon claims one queued request at a time and writes request/run records
below `runtime/module-installer`. For process-manager one-shot usage or tests,
use `--daemon-once`. Check daemon status with:
```bash
govoplan-module-installer --daemon-status --format json
```
The installer uses a runtime lock, snapshots `pip freeze` plus WebUI package
files, writes a run record, and marks planned rows as applied only after all
commands succeed. With `--migrate`, SQLite databases are backed up through
SQLite's backup API; non-SQLite databases require
`--database-backup-command`, `--database-restore-check-command`, and
`--database-restore-command`.
Database hook commands receive:
- `GOVOPLAN_INSTALLER_RUN_DIR`
- `GOVOPLAN_DATABASE_URL`
- `GOVOPLAN_DATABASE_URL_PGTOOLS` for PostgreSQL tools
- `GOVOPLAN_DATABASE_BACKUP_PATH`
- `GOVOPLAN_DATABASE_BACKUP_METADATA`
Avoid embedding secrets directly in commands; prefer environment variables,
service credentials, or deployment-local secret injection.
Inspect installer history and lock state from the operator shell:
```bash
govoplan-module-installer --list-runs --format json
govoplan-module-installer --show-run <run-id> --format json
govoplan-module-installer --lock-status --format json
govoplan-module-installer --list-requests --format json
govoplan-module-installer --show-request <request-id> --format json
govoplan-module-installer --cancel-request <request-id> --format json
govoplan-module-installer --retry-request <request-id> --format json
```
Rollback uses the saved run snapshot:
```bash
govoplan-module-installer --rollback <run-id>
govoplan-module-installer --rollback <run-id> --database-restore-command '<override restore command>'
```
Uninstall is non-destructive by default. A planned uninstall row can set
`destroy_data: true` to request destructive module retirement. The module must
provide an automated retirement provider, and the installer snapshots the
database before dropping module-owned tables.
Run the rollback drill before relying on installer automation in a new
environment:
```bash
/mnt/DATA/git/govoplan/tools/checks/module-installer-rollback-drill.py --format json
```
The drill uses temporary SQLite databases and simulated package commands. It
does not install or uninstall real packages. It exercises:
- package command failure followed by supervised rollback;
- migration failure with a SQLite database snapshot;
- restart-command failure;
- health timeout after restart;
- destructive retirement executor failure with database rollback;
- PostgreSQL-style backup, restore-check, and restore hooks;
- daemon heartbeat, request queue claim/update, retry/cancel, and stale lock
detection/removal.
See `RELEASE_DEPENDENCIES.md` for release package refs, migration baseline
checks, catalog trust, signing, keyring, replay, and license operation.
## Operator Checklist
- Runtime secrets are injected outside git.
- `MASTER_KEY_B64` is set and backed up securely.
- Database backup and restore commands are tested.
- File/object storage is durable and backed up.
- `CORS_ORIGINS` and cookie settings match the deployed WebUI origin.
- Redis and workers are running before `CELERY_ENABLED=true`.
- Module catalog and license keyrings are pinned locally.
- Health endpoints are monitored.
- Test SMTP/IMAP credentials are non-production and isolated.
- Module installer rollback drill has passed in the deployment environment.

52
docs/DOCUMENTATION_MAP.md Normal file
View File

@@ -0,0 +1,52 @@
# GovOPlaN Documentation Map
This map defines the source-of-truth documents for the current repository docs.
Use it to avoid duplicating long procedures across architecture, release,
operator, and roadmap pages.
## Core Platform
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Module architecture and kernel contracts | `MODULE_ARCHITECTURE.md` | Stable module contracts, API efficiency contracts, durable boundary decisions, lifecycle, and WebUI contribution rules. |
| RBAC and resource access | `ACCESS_RBAC_MODEL.md` | Current permission, role, API-key, and resource-access model. |
| Governance hierarchy | `GOVERNANCE_MODEL.md` | System, tenant, user/group, campaign policy inheritance and admin UI structure. |
| Policy decision DTOs and provenance | `POLICY_CONTRACTS.md` | Shared explain/provenance shape; module-specific policy docs should link here. |
| Events and audit trace context | `EVENTS_AND_AUDIT.md` | Event dispatch semantics and audit payload conventions. |
| Action/effect automation layer | `ACTION_EFFECT_AUTOMATION_LAYER.md` | Action/effect contracts, consequence preview, runner semantics, and module boundary for automation. |
| Postbox E2EE target architecture | `POSTBOX_E2EE_ARCHITECTURE.md` | Strategic encrypted postbox/mailbox model, key ownership, role mailbox semantics, and retraction limits. |
## Release And Operations
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Runtime configuration and operator flow | `DEPLOYMENT_OPERATOR_GUIDE.md` | Production/staging configuration, migrations, backups, installer operation, and rollback drill. |
| Self-hosted installability | `SELF_HOSTED_INSTALLABILITY.md` | Packaging decision, generated env templates, config validation, production-like dev stack commands, and boundary gate. |
| Release dependencies and catalogs | `RELEASE_DEPENDENCIES.md` | Release package refs, migration baselines, release lockfiles, catalog trust/licensing, catalog publishing, and release checklist. |
| Dependency vulnerability audits | `DEPENDENCY_AUDITS.md` | Local and CI audit commands plus dated audit result notes. |
| Remote WebUI bundle design | `REMOTE_WEBUI_BUNDLES.md` | Experimental controlled-deployment design; normal releases still use package builds. |
## Product And Module Planning
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Product roadmap and module routing | `GOVOPLAN_MASTER_ROADMAP.md` | Product-level sequencing, implementation gates, issue routing, and missing-module decisions. |
| UI/UX decisions | `UI_UX_DECISION_LEDGER.md` | Binding guided-UI decisions, open decisions, impact index, and review checklist. |
| Interface ethics and design doctrine | `INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` | Product-level doctrine for context, decision, consequence, contestability, responsibility, and traceability. |
| Public-sector integration posture | `PUBLIC_SECTOR_INTEGRATION_STRATEGY.md` | Strategy index; executable target inventory lives in `govoplan-connectors`. |
| Configuration packages | `CONFIGURATION_PACKAGES.md` | Package model, provider contract, import/export flow, and tracking slices. |
## Workflow Docs
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Gitea issues and wiki sync | `GITEA_ISSUES.md` | Issue labels, imports, wiki mirroring, and Codex state updates. |
| Codex local workflow | `CODEX_WORKFLOW.md` | Local agent setup and focused verification commands. |
## Cross-Repo Rule
Core docs may keep strategy, kernel contracts, and routing decisions. Module
repositories should own executable module behavior, concrete API/UI contracts,
and operator notes for their own module. When content spans both, keep the
durable decision in core and link to the module document for implementation
details.

203
docs/EVENTS_AND_AUDIT.md Normal file
View File

@@ -0,0 +1,203 @@
# Events And Audit
GovOPlaN uses a small kernel event contract first, not a broad command bus.
Commands remain module-owned application-service methods or API endpoints until
there is a concrete need for durable asynchronous command orchestration. Events
are facts about completed work and are safe for audit, projections, optional
module reactions, and operator diagnostics.
## Production Transport Decision
The first production target is a **database outbox plus in-process immediate
dispatch**:
- Use `govoplan_core.core.events.PlatformEvent` for domain and platform events.
- Use `EventBus` as the in-process dispatch contract for same-process module
reactions that are safe to run inline.
- Use the shared `audit_event` / `audit_from_principal` helper for audited
module actions. The helper persists the audit row and immediately publishes a
governed `PlatformEvent` whose `type` is the audit action.
- Use `record_change` for module delta feeds. It persists the change-sequence
row and immediately publishes a generic module change event such as
`mail.profile.updated`.
- Persist durable integration/workflow events through a database outbox before
acknowledging the state change that produced them.
- Drain the outbox through a small dispatcher process. The dispatcher may call
in-process handlers in the same deployment first, but its storage contract is
database-backed.
- Treat Redis/Celery as worker/job infrastructure, not as the authoritative
first event transport. A Celery dispatcher can consume the outbox later.
- Keep the dispatch implementation pluggable behind the `PlatformEvent`
envelope so a future message broker can be added without changing event
producers.
- Keep commands out of the kernel until workflows need retryable, durable,
operator-visible command records.
This keeps the first contract small, PostgreSQL-friendly, auditable, and
recoverable after process crashes. It also avoids making Redis a correctness
dependency for deployments that only need synchronous mail/tests or light
background work.
## Dispatch Semantics
Event producers should write their domain state and outbox event in the same
database transaction wherever possible. Handlers must be idempotent because the
outbox dispatcher can retry after a crash or timeout.
Recommended first outbox columns:
- `event_id`, `event_type`, `module_id`
- `correlation_id`, `causation_id`
- `payload`, `occurred_at`
- `available_at`, `attempt_count`, `claimed_at`, `claim_token`
- `processed_at`, `last_error`
Inline `EventBus` handlers are allowed only for non-critical local reactions.
Anything that must survive process failure, restart, package update, or worker
redeployment belongs in the outbox.
## Trace IDs
Every `PlatformEvent` has:
- `event_id`: unique ID for that event.
- `correlation_id`: stable ID for the whole request, workflow, or job.
- `causation_id`: the event ID or external operation ID that caused this
event.
- `actor`: optional typed actor reference, for example user, API key, system
actor, delegated actor, or installer daemon.
- `tenant`: optional tenant reference when the event is tenant-scoped.
- `subject`: optional typed subject reference for the person, organization,
account, case, campaign, or other entity the event is about.
- `resource`: optional typed resource reference for the object changed or
observed by the event.
- `classification`: payload sensitivity, currently `public`, `internal`,
`confidential`, or `restricted`.
- `payload`: JSON-serializable module-owned event details.
The FastAPI app factory creates an event context for every request. It accepts
`X-Correlation-ID` or `X-Request-ID` when the value is a compact safe trace ID,
otherwise it generates a new ID. Responses include `X-Correlation-ID`.
Audit logging reads the current event context and stores trace IDs in
`details._trace`. Callers can also pass explicit `correlation_id` and
`causation_id` to `audit_event` or `audit_from_principal`. The same trace is
copied into the emitted `PlatformEvent`, so audit rows and event subscribers can
be correlated without route-specific glue code.
Admin and lifecycle code should use the compact operational detail shape
documented in `govoplan-audit/docs/AUDIT_TRACE_CONTEXT.md`. The core
`audit_operation_context` helper preserves `module_id`, `request_id`, `run_id`,
`outcome`, and `_trace` while applying the shared audit redaction pass to
additional detail values.
## Audit MVP Boundary
`govoplan-audit` owns:
- the `audit_log` table and audit API routes
- audit route contribution through its module manifest
- audit retention behavior in cooperation with policy/retention settings
- future audit sink/export capability implementations
`govoplan-core` owns:
- `AuditEvent` and `AuditSink` protocol contracts
- request and event trace context
- the compatibility `audit_event` helper while routes are still migrating
- retention orchestration that calls module capabilities
Feature modules should record audit facts through the shared helper or a future
`audit.sink` capability. They should not import audit storage internals. Module
state changes that only need delta-feed visibility should go through
`record_change`; route-level business actions should still record a semantic
audit action.
## Initial Domain Event Inventory
Access:
- `access.account.created`
- `access.account.updated`
- `access.membership.created`
- `access.membership.updated`
- `access.group.created`
- `access.group.updated`
- `access.role.created`
- `access.role.updated`
- `access.role.deleted`
- `access.api_key.created`
- `access.api_key.revoked`
- `access.session.created`
- `access.session.revoked`
Tenancy:
- `tenant.created`
- `tenant.updated`
- `tenant.suspended`
- `tenant.resumed`
- `tenant.deletion_requested`
- `tenant.erasure_completed`
Policy:
- `policy.system.updated`
- `policy.tenant.updated`
- `policy.user.updated`
- `policy.group.updated`
- `policy.campaign.updated`
- `policy.effective_policy.changed`
Files:
- `files.file.uploaded`
- `files.file.renamed`
- `files.file.deleted`
- `files.file.frozen`
- `files.share.created`
- `files.share.revoked`
- `files.connector.imported`
- `files.connector.access_denied`
Mail:
- `mail.profile.created`
- `mail.profile.updated`
- `mail.profile.credentials_rotated`
- `mail.profile.tested`
- `mail.message.sent`
- `mail.message.send_failed`
- `mail.imap.appended`
- `mail.imap.append_failed`
- `mail.mailbox.message_seen`
Campaign:
- `campaign.created`
- `campaign.version.created`
- `campaign.validated`
- `campaign.built`
- `campaign.reviewed`
- `campaign.queued`
- `campaign.send_started`
- `campaign.recipient_attempted`
- `campaign.recipient_delivered`
- `campaign.recipient_failed`
- `campaign.paused`
- `campaign.resumed`
- `campaign.cancelled`
- `campaign.report.exported`
## Event Payload Rules
- Payloads must be JSON-serializable.
- Use stable IDs, not ORM objects.
- Include tenant ID when tenant-scoped.
- Include actor/principal, subject, and resource references only as typed DTOs
or primitive IDs.
- Set `classification` to the highest sensitivity needed by the envelope or
payload, not the lowest sensitivity of any individual field.
- Do not include secrets, raw message bodies, full recipient lists, or file
content.
- Put large evidence in owning module storage and reference it by ID.

16
docs/GITEA_ISSUES.md Normal file
View File

@@ -0,0 +1,16 @@
# Gitea Issues And Wiki Workflow
The shared GovOPlaN Gitea issue, label, and wiki workflow tooling moved to the
meta repository.
Use:
```bash
cd /mnt/DATA/git/govoplan
tools/gitea/gitea-sync-labels.py --help
tools/gitea/gitea-sync-wiki.py --help
```
Canonical documentation:
- `/mnt/DATA/git/govoplan/docs/GITEA_ISSUES.md`

View File

@@ -1,11 +1,11 @@
# Multi Seal Mail - Current System and Tenant Governance Model
# GovOPlaN Governance Model
**Updated:** 2026-06-16
**Current migration head:** `f5a6b7c8d9e0`
**Updated:** 2026-07-09
## Governance Rule
System policy is authoritative for tenants and all lower levels. Each lower level may only narrow what it inherits:
System policy is authoritative for tenants and all lower levels. Each lower
level may only narrow what it inherits:
```text
system
@@ -14,52 +14,65 @@ system
-> campaign
```
Lower levels do not widen privileges, allowed profiles, retention durations or credential rights granted by a higher level.
Lower levels do not widen privileges, allowed profiles, retention durations, or
credential rights granted by a higher level.
## Administration Structure
GovOPlaN separates system administration from scoped configuration:
```text
SYSTEM
- Settings
- Retention
- Mail servers
ADMINISTRATION
- Modules
- Packages
- Maintenance
- Changes
GLOBAL
- Tenants
- Users
- Groups
- System roles
- Tenant roles
- Audit
- Roles
- Groups and users
- File connectors
- Mail servers
- API keys
- Retention
TENANT
- Settings boundary
- Users
- Groups
- Roles
- API keys
- Groups and users
- File connectors
- Mail servers
- API keys
- Retention
- Audit
USER
- User mail
- User retention
GROUP
- Group mail
- Group retention
- File connectors
- Mail servers
- API keys
- Retention
USER
- File connectors
- Mail servers
- API keys
- Retention
```
There is no separate System access page. Compatibility access scopes remain in the backend for assignment/read boundaries.
System access scopes remain in the backend for assignment/read boundaries, but
the UI should present the configuration hierarchy rather than a separate
"system access" concept.
## Tenant Governance
System settings define tenant defaults and whether tenants may narrow selected options. Tenant overrides can only restrict:
System settings define tenant defaults and whether tenants may narrow selected
options. Tenant overrides can only restrict:
- custom groups;
- custom roles;
- tenant API keys.
The backend enforces that tenant governance cannot widen system-denied privileges.
The backend enforces that tenant governance cannot widen system-denied
privileges.
## Mail-Profile Governance
@@ -73,7 +86,10 @@ group
campaign
```
Effective campaign profile availability follows campaign ownership. A campaign owned by a user resolves through system, tenant, that user and campaign policy. A group-owned campaign resolves through system, tenant, that group and campaign policy.
Effective campaign profile availability follows campaign ownership. A campaign
owned by a user resolves through system, tenant, that user, and campaign
policy. A group-owned campaign resolves through system, tenant, that group, and
campaign policy.
Policy semantics:
@@ -81,12 +97,18 @@ Policy semantics:
- lower levels can further restrict the set;
- forced profiles mean the lower level must choose from the forced set;
- a forced set with one profile effectively enforces that profile;
- campaign-level profile creation is allowed only if the effective policy permits it;
- SMTP/IMAP credentials use one inheritance decision per protocol: lower levels must inherit profile credentials, may inherit profile credentials, or must provide local credentials;
- the lower-level override switch for `smtp_credentials.inherit` and `imap_credentials.inherit` controls whether descendants may change that inheritance decision;
- campaign-level profile creation is allowed only if the effective policy
permits it;
- SMTP/IMAP credentials use one inheritance decision per protocol: lower levels
must inherit profile credentials, may inherit profile credentials, or must
provide local credentials;
- the lower-level override switch for `smtp_credentials.inherit` and
`imap_credentials.inherit` controls whether descendants may change that
inheritance decision;
- deny patterns always win over allow patterns;
- empty or `*` allowlist means allow all except denied;
- non-empty allowlist means at least one allow rule must match and no deny rule may match.
- non-empty allowlist means at least one allow rule must match and no deny rule
may match.
Pattern targets:
@@ -98,7 +120,23 @@ From header
recipient domains
```
Ownership transfer is intentionally deferred as a two-step workflow: original owner initiates, new owner accepts and reselects/repairs the mail profile if their effective policy requires it.
Ownership transfer is intentionally deferred as a two-step workflow: original
owner initiates, new owner accepts and reselects/repairs the mail profile if
their effective policy requires it.
## File-Connector Governance
File connector profiles and credentials are separated. Profiles describe
external endpoints; credentials bind authentication material and policy to a
scope. Concrete linked folders appear as file spaces in the files module.
Governance follows the same inheritance shape as mail:
- system and tenant policy can permit, require, or forbid lower-level
connections/credentials;
- user or group ownership controls which spaces appear to principals;
- spaces inherit endpoint and credential policy from their connector profile;
- connector health and credential tests must not reveal plaintext secrets.
## Retention Governance
@@ -120,20 +158,27 @@ Rules:
- system may set concrete defaults or unlimited retention;
- system exposes allow-limiting toggles per field;
- tenants, users/groups and campaigns may only shorten inherited retention where the parent allows limiting;
- tenants, users/groups, and campaigns may only shorten inherited retention
where the parent allows limiting;
- blank lower-level values inherit;
- mock mailbox retention is currently system-level because mock mailbox records do not yet carry tenant/campaign ownership metadata;
- dry-run/apply retention actions report affected classes before destructive cleanup.
- mock mailbox retention is currently system-level because mock mailbox records
do not yet carry tenant/campaign ownership metadata;
- dry-run/apply retention actions report affected classes before destructive
cleanup.
## Role Definitions and Assignments
## Role Definitions And Assignments
### System roles
### System Roles
System roles define instance-wide permissions. `system:*` is stored as one wildcard and displayed as granting the full system catalogue. System owner is protected.
System roles define instance-wide permissions. `system:*` is stored as one
wildcard and displayed as granting the full system catalogue. System owner is
protected.
### Tenant roles
### Tenant Roles
Tenant roles can be system-governed templates or tenant-local definitions, subject to system tenant-governance settings and actor delegation ceilings. Wildcard counts are expanded against the canonical tenant catalogue.
Tenant roles can be system-governed templates or tenant-local definitions,
subject to system tenant-governance settings and actor delegation ceilings.
Wildcard counts are expanded against the canonical tenant catalogue.
## Audit Access
@@ -144,15 +189,17 @@ system audit -> system:audit:read
tenant audit -> active tenant + audit:read
```
Audit pages use server pagination, filtering and bounded grids.
Audit pages use server pagination, filtering, and bounded grids.
## Tenant Switching
Tenant switching preserves the current URL when possible and falls back when a route/resource is not accessible in the new tenant context.
Tenant switching preserves the current URL when possible and falls back when a
route/resource is not accessible in the new tenant context.
The tenant selector is hidden for ordinary single-tenant accounts and visible for multi-tenant or system tenant-management contexts.
The tenant selector is hidden for ordinary single-tenant accounts and visible
for multi-tenant or system tenant-management contexts.
## DataGrid Contract in Administration
## Administration DataGrid Contract
Admin lists use bounded container grids:
@@ -164,14 +211,16 @@ Admin lists use bounded container grids:
- sticky headers where needed;
- server pagination for audit.
## Still Deferred
## Deferred Work
- DataGrid sizing and resize behavior remains explicitly deferred. The current
bounded-grid contract above is binding, but further layout changes should be
handled as a dedicated, isolated UI debt item because the component is shared
and brittle.
- real SMTP/IMAP test-bed verification and operator runbook;
- recipient import with column mapping;
- Seafile/external connector governance;
- system/tenant/group/user file-space hierarchy and external storage hierarchy;
- session/device revocation UI;
- backup/restore, monitoring and update procedures;
- backup/restore, monitoring, and update procedures;
- DSAR workflows and evidence bundle verifier;
- campaign ownership transfer workflow;
- policy impact analysis before delete/disable/unshare/change;

View File

@@ -0,0 +1,663 @@
# GovOPlaN Master Roadmap
This roadmap is the durable product north star and sequencing guide for
GovOPlaN as a modular platform for administrative operations. It keeps the
product moving without turning every possible public-sector need into an
immediate implementation track.
Use this document for product direction, sequencing, and module routing. Issues
are the active backlog; this document is durable planning context and should be
mirrored to the Gitea wiki.
## Product Thesis
GovOPlaN should become a configurable operations platform for public
institutions. It should help an institution model real administrative
procedures, connect existing systems, keep durable evidence, and explain the
configured system to users.
The goal is not to replace every existing system. GovOPlaN should connect
existing systems, provide better workflows where the current landscape is weak,
and make administrative processes configurable, auditable, and reusable.
The product should first provide a reliable administrative spine:
- identities, roles, tenants, policy, audit, and governance
- forms, files, cases, workflow, tasks, templates, and records
- postboxes, notifications, mail, portal, appointments, and booking
- identity trust, role-bound postboxes, and eventually encrypted administrative
communication
- configuration packages that assemble modules into repeatable procedures
- docs that explain the configured system, not the full theoretical product
Domain modules should come after the spine can run a reference procedure end to
end.
The platform should be a governance-capable runtime for modules, connectors,
configuration, and administrative decisions. The kernel must stay free of domain
semantics while still providing the contracts needed for modules to explain what
they do, what they require, which effects they create, and how operators can
verify or reverse those effects.
## Design Principles
- Modules must stay independently installable, enableable, and disableable.
- Cross-module behavior should use core-mediated capabilities, commands,
events, DTOs, and UI contribution points rather than direct imports.
- Configuration packages should turn installed modules into concrete, reusable
administrative processes.
- Operators should be able to configure the platform through the UI.
- Every powerful configuration path needs preflight, preview, audit, rollback,
RBAC, and policy checks.
- Context, decision, consequence, and traceability must be visible together for
consequential actions. The full doctrine lives in
`INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md`.
- Automation must use governed action/effect contracts, not hidden side
effects. The first automation layer is defined in
`ACTION_EFFECT_AUTOMATION_LAYER.md` and should start in `govoplan-workflow`
unless a separate automation module becomes justified.
- Encrypted postboxes are a strategic target. Early postbox, access, and
identity-trust contracts should stay compatible with the E2EE architecture in
`POSTBOX_E2EE_ARCHITECTURE.md`.
- Integration should be a first-class product path: connect to existing
systems, consume their data, and publish governed outputs back to them.
- GovOPlaN should scale from a small local installation to a larger deployment
with separately scalable web, API, worker, storage, and database components.
## User Experience Direction
GovOPlaN should expose the full power of the platform without forcing
non-technical users to face every field, flag, and internal representation at
once. The default experience should feel guided, explainable, and calm. Expert
depth should remain available, but it should be layered behind deliberate
interaction patterns.
Core UX rules:
- Use progressive disclosure. Common decisions stay visible; advanced,
hazardous, or rarely used options live in collapsed panels, secondary steps,
or explicit advanced areas.
- Do not use raw JSON as the primary configuration UI. Every configurable value
should have an appropriate control, validation, and plain-language help.
Import/export and diagnostics may show JSON as a secondary artifact.
- Prefer guided flows over option dumps. Connector setup, package import,
module installation, policy changes, and destructive operations should use
wizards that explain what is happening, why it matters, and what will happen
next.
- Discover values when the system can infer them. For example, a Nextcloud file
connection should start with the base URL, discover the WebDAV endpoint, and
fill technical fields for review instead of asking the user to know them
upfront.
- Make explanations always available without making every screen verbose.
Inline helper text should be short; richer explanations should be reachable
through expandable help, tooltips, side panels, or review steps.
- Explain blocked actions in actionable language. A disabled control or failed
step should say what is missing, who can fix it, and where to go, for example
"A system administrator must allow this provider" or "Configure the provider
in Settings > File Providers before linking a folder here."
- Reuse visual language and placements consistently. Similar configuration,
policy, connection, credential, review, and confirmation flows should share
components, button placement, modal behavior, problem lists, and empty/error
states.
- Use modals and step flows for focused creation/editing where they reduce page
clutter. Reserve large always-open pages for overview, comparison, and
repeated administration work.
- Treat diagnostics as product UX. Validation results, preflight blockers,
policy explanations, permission denials, and missing capabilities should be
understandable to a non-technical operator before exposing internal details.
This is a product quality gate. New admin/configuration surfaces should not be
considered complete if they expose all options at once, require JSON editing,
hide why an action is unavailable, or use a one-off layout where a shared
pattern exists.
## Focus Rules
1. Build one reference journey per wave.
2. Do not implement a module because the repository exists.
3. Do not add module-to-module imports for optional behavior.
4. Every new domain module must justify its own semantics beyond `cases`,
`workflow`, `tasks`, `forms`, and `files`.
5. Prefer connector first when an external specialist system is likely to remain
the system of record.
6. Keep active work in Gitea issues; keep durable context in docs and synced
wiki pages.
7. A module moves from scaffold to implementation only when it has an owner,
reference journey, boundary notes, capability contracts, and testable MVP.
## Capability Map
| Capability | Likely owner |
| --- | --- |
| Public application entry point | `govoplan-portal` |
| Structured forms and validation | `govoplan-forms` |
| Uploaded files and managed storage | `govoplan-files` |
| Case record and lifecycle | `govoplan-cases` |
| Workflow transitions and automation | `govoplan-workflow` |
| Action/effect catalogue and automation runner | first `govoplan-workflow`; possible future `govoplan-automation` if it outgrows workflow |
| Internal work queues and tasks | `govoplan-tasks` |
| Appointment proposals and booking | `govoplan-appointments`, `govoplan-calendar` |
| Postbox, email, and notifications | `govoplan-postbox`, `govoplan-mail`, `govoplan-notifications` |
| Identity trust, device keys, and encrypted postbox key contracts | `govoplan-identity-trust`, `govoplan-access`, `govoplan-postbox` |
| Service directory/catalog | `govoplan-portal` |
| Permit/document generation | `govoplan-templates`, `govoplan-dms` |
| Payment capture and accounting handoff | `govoplan-payments`, `govoplan-ledger` |
| Roles, permissions, tenants, policy, audit | `govoplan-access`, `govoplan-tenancy`, `govoplan-policy`, `govoplan-audit` |
| External software integration | `govoplan-connectors` |
| Recurring extraction and transformation | possible future `govoplan-datasources`, possible future `govoplan-dataflow` |
| Reports, BI, and management visibility | `govoplan-reporting` |
## Configuration And Safety Target
The long-term target is that operators configure the platform through the UI
instead of editing files for normal operation.
UI-managed configuration should include:
- module installation, enablement, lifecycle state, and health
- tenants, users, groups, roles, policies, and permissions
- connectors, credentials, secret references, and external service tests
- workflows, forms, templates, task queues, schedules, and notifications
- configuration package import/export and environment-specific data collection
- retention, audit, privacy, maintenance mode, and safety controls
- deployment-visible settings such as public URLs, mail senders, storage
profiles, queues, and worker capabilities
Safety controls should include dry-run plans, field-level validation, policy
explanations, two-person approval for destructive changes, versioned
configuration history, rollback paths, audit events, and maintenance-mode
guards.
The initial safety metadata contract lives in
`govoplan_core.core.configuration_safety`. It classifies known configuration
fields as UI-managed or deployment-managed, assigns risk levels, marks secret
handling as reference-only or env-only, and declares dry-run, policy
explanation, audit, approval, rollback-history, maintenance-mode, and RBAC
requirements. Admin UI editors should consume this metadata before exposing
powerful settings.
The initial executable guardrail path is `plan_configuration_change(...)`,
exposed through:
- `GET /api/v1/admin/configuration-safety`
- `POST /api/v1/admin/configuration-safety/plan`
The planner reports missing scopes, dry-run requirements, maintenance-mode
requirements, two-person approval status, secret-reference violations,
rollback-history requirements, policy explanations, and audit event names before
an editor applies a high-impact configuration change.
## Reference Journeys
The roadmap should be driven by three journeys.
### Journey 1: Permit To Payment
This is the primary public-administration journey.
1. A person applies for a permit through the public portal.
2. The applicant uploads required files and submits structured form data.
3. Submission creates a case, a workflow instance, and an internal task.
4. Completing the task creates a postbox message, a notification, and an email
notification with an appointment proposal.
5. The applicant accepts an appointment, which updates the calendar and the
workflow state.
6. During the appointment, the case is opened and the permit is generated from
a governed template.
7. The payment is processed and linked to the case and accounting handoff.
8. The permit, payment evidence, communication history, audit trail, retention
state, and records evidence remain available according to policy.
This journey proves the platform can coordinate modules without core knowing
module internals.
### Journey 2: Training To Certificate
This is the best university-administration and internal-administration journey.
1. Course or training offer is planned.
2. Room, trainer, resource, and capacity are booked.
3. Participants register or are assigned.
4. Attendance is tracked.
5. Certificate or participation confirmation is issued.
6. Evidence remains available through records, files, audit, and docs.
This journey keeps `booking`, `resources`, `learning`, and `certificates`
focused instead of becoming broad ERP replacements.
### Journey 3: Report To Resolution
This is the internal operations and municipal issue-reporting journey.
1. A person reports an issue.
2. The issue is triaged into helpdesk, facilities, assets, or a case.
3. Work is assigned, tracked, and escalated.
4. Evidence, communication, and status updates are preserved.
5. Reports show workload, SLA, recurring problems, and completion.
This journey prevents `helpdesk`, `issue-reporting`, `facilities`, and `assets`
from becoming disconnected ticket silos.
## Roadmap Waves
### Wave 0: Platform Spine
Goal: make the platform safe to configure and extend.
Refine:
- `govoplan-core`: module discovery, capabilities, events, migrations, release
catalog, configuration package runtime, WebUI shell.
- `govoplan-access`: identities, sessions, API keys, users, groups, roles,
memberships, function assignments, delegation, RBAC decisions.
- `govoplan-tenancy`: tenant and organizational-unit boundaries.
- `govoplan-identity-trust`: initial trust contracts for device keys, public key
directory, assurance, and later encrypted postbox key access.
- `govoplan-policy`: policy sources, policy decisions, retention inputs.
- `govoplan-audit`: audit sink, audit routes, trace context, retention
cooperation.
- `govoplan-admin`: UI-managed configuration with preflight, rollback, audit,
and approval controls.
- `govoplan-dashboard`: configurable user home assembled from module-provided
widgets, with core providing only a minimal fallback when the module is absent.
- `govoplan-docs`: configured-system documentation and evidence-aware help.
- `govoplan-ops`: deployment profiles, health checks, worker split, sizing
assumptions.
Do not expand domain scope in this wave. The output is a dependable platform
surface for later modules.
Exit criteria:
- module enablement and capability lookup are stable
- configuration package preflight works for at least one simple package
- audit and policy decisions are visible in admin flows
- access distinguishes identity, account, function, role, and right in durable
contracts
- action/effect automation contracts are specified before hidden side effects
spread across modules
- docs can show installed/enabled modules and configured routes
### Wave 1: Permit-To-Payment MVP
Goal: one complete public-administration process.
Create or refine in this order:
1. `govoplan-forms-runtime`: form submissions, drafts, validation, attachments,
and submission evidence.
2. `govoplan-portal`: public authenticated and unauthenticated entry points.
3. `govoplan-files`: upload, evidence links, file spaces, and permissioned
access.
4. `govoplan-cases`: case record, status, assignments, deadlines, and case
evidence.
5. `govoplan-workflow`: state machine, transitions, commands, and module
handoff.
6. `govoplan-tasks`: work queues, assignments, due dates, and follow-ups.
7. `govoplan-templates`: permit/decision document generation.
8. `govoplan-postbox`, `govoplan-mail`, `govoplan-notifications`: applicant and
internal communication, with the postbox model compatible with later E2EE
and role-bound access.
9. `govoplan-calendar`, `govoplan-appointments`, `govoplan-booking`: appointment
or booking handoff for the reference process.
10. `govoplan-payments`, `govoplan-ledger`, `govoplan-xrechnung`: payment
capture, payment evidence, and accounting/e-invoice handoff.
Exit criteria:
- a permit-to-payment package can be installed in a local demo
- every step has audit evidence
- every module integration uses capabilities, events, or DTOs
- user-facing documentation describes only the configured process
### Wave 2: Booking And Resource Operations
Goal: make time, capacity, and resource allocation a reusable product area.
Create or refine in this order:
1. `govoplan-booking`: booking rules, capacity, quotas, waitlists, cancellation,
attendance, no-shows, and approvals.
2. `govoplan-resources`: rooms, equipment, vehicles, counters, labs, capacity,
availability, and maintenance blocks.
3. `govoplan-calendar`: event and availability integration.
4. `govoplan-appointments`: appointment-specific booking surfaces.
5. `govoplan-facilities`: buildings, rooms, maintenance, access zones,
inspections, and defects.
6. `govoplan-assets`: inventory, assignment, lifecycle, maintenance, and handover
evidence.
Reference journey: reserve a room/resource for a training or appointment and
preserve all booking evidence.
Exit criteria:
- bookable resources are not hardcoded into appointments
- booking decisions are explainable and auditable
- resources can be blocked by maintenance or facility status
### Wave 3: Training And Certificates
Goal: support university-style and internal public-sector training without
building a full learning-management system first.
Create or refine in this order:
1. `govoplan-learning`: course planning, course booking, participant lists,
trainers, attendance, evaluations, and learning records.
2. `govoplan-certificates`: participation confirmations, certificates,
verification, revocation, and evidence links.
3. `govoplan-booking`, `govoplan-resources`, `govoplan-calendar`: reuse booking
and room/resource allocation.
4. `govoplan-templates`, `govoplan-files`, `govoplan-records`: certificate
generation and durable retention.
Reference journey: plan a course, book resources, register participants, track
attendance, and issue a certificate.
Exit criteria:
- attendance can produce certificate eligibility
- certificates can be verified later
- course operations do not depend on university-specific assumptions
### Wave 4: Report-To-Resolution Operations
Goal: cover internal support and public issue reporting.
Create or refine in this order:
1. `govoplan-issue-reporting`: public/internal reports, categories, intake,
location, evidence, and triage.
2. `govoplan-helpdesk`: service desk tickets, queues, SLAs, assignments,
escalation, and resolution evidence.
3. `govoplan-facilities` and `govoplan-assets`: issue handoff to maintenance or
asset lifecycle.
4. `govoplan-cases`, `govoplan-workflow`, `govoplan-tasks`: escalation into
formal administrative matters.
5. `govoplan-reporting`: workload, SLA, recurring defects, and status reports.
Reference journey: report a facility issue, triage it, assign work, resolve it,
and report recurring defects.
Exit criteria:
- issue reporting is not just a generic form inbox
- helpdesk tickets can remain lightweight unless formal case handling is needed
- operational metrics exist without custom SQL
### Wave 5: Records, DMS, Transparency
Goal: make evidence legally and organizationally durable.
Create or refine in this order:
1. `govoplan-dms`: document lifecycle, collaboration, versions, locks, approvals,
and DMS connectors.
2. `govoplan-records`: file plans, classification, retention schedules, disposal
holds, archive handoff, and records evidence.
3. `govoplan-policy` and `govoplan-audit`: retention policy, legal hold, audit
traceability.
4. `govoplan-search`: permissioned cross-module discovery.
5. `govoplan-transparency`: FOI/public-records requests, redaction, publication,
and disclosure evidence.
Reference journey: close a case, classify records, apply retention, and answer a
transparency request.
Exit criteria:
- files, documents, and records are not treated as the same concept
- transparency disclosure can redact and explain evidence
- retention and archive handoff are policy-governed
### Wave 6: Finance, Procurement, Contracts, Grants
Goal: cover the back-office procedures around spending, obligations, funding,
and revenue without replacing a finance system too early.
Create or refine in this order:
1. `govoplan-procurement`: purchase requests, approvals, vendor comparison,
tender references, goods receipt, and handoff.
2. `govoplan-contracts`: contract register, obligations, renewals, reminders,
and responsible units.
3. `govoplan-grants`: funding programs, applications, eligibility, awards,
milestones, claims, and reporting duties.
4. `govoplan-payments`, `govoplan-ledger`, `govoplan-xrechnung`: payment,
accounting, and e-invoice handoff.
5. `govoplan-erp`: integration with the actual system of record, not a full ERP
replacement unless later justified.
Reference journey: purchase or grant approval through obligation tracking and
financial handoff.
Exit criteria:
- ERP remains an integration point unless GovOPlaN must own semantics
- contracts and grants produce obligations, deadlines, and reporting tasks
- financial evidence links back to cases, files, and audit
### Wave 7: Institutional Governance And Participation
Goal: support public bodies, municipalities, ministries, and universities in
formal coordination.
Create or refine in this order:
1. `govoplan-committee`: committees, boards, councils, agendas, minutes,
decisions, voting, and follow-up tasks.
2. `govoplan-consultation`: hearings, public consultations, stakeholder
feedback, formal comments, response matrices, and publication workflows.
3. `govoplan-campaign`: communication campaigns, recipients, templates, review,
sending control, and reports.
4. `govoplan-addresses`: persons, organizations, contacts, distribution lists,
and recipient import/export.
5. `govoplan-portal` and `govoplan-transparency`: public participation and
publication surfaces.
Reference journey: prepare a committee decision, publish consultation material,
collect feedback, document the decision, and assign follow-up tasks.
Exit criteria:
- decisions create durable tasks and records
- consultations can publish evidence without exposing restricted material
- campaign and address behavior stays optional and capability-based
### Wave 8: Integration, Data, And Reporting
Goal: connect the platform to real institutional landscapes.
Refine:
- `govoplan-connectors`: integration catalog, connector metadata, adapter
lifecycle, test harnesses.
- `govoplan-idm`, `govoplan-identity-trust`: identity provider, directory, and
assurance integration.
- `govoplan-fit-connect`, `govoplan-xoev`, `govoplan-xta-osci`: public-sector
transport and standards integration.
- `govoplan-reporting`: operational reporting, scheduled outputs, exports, and
dashboard data.
- `govoplan-search`: permissioned cross-module discovery.
Create only when justified:
- `govoplan-datasources`: source catalog, connection profiles, schema discovery,
freshness, provenance.
- `govoplan-dataflow`: transformations, validation, lineage, scheduled runs,
publication outputs.
- `govoplan-projects`: only if OpenProject connectors and existing cases/tasks
cannot cover the required semantics.
Reference journey: monthly data extraction, transformation, validation, approval,
publication, and reporting.
Recurring extraction/transformation should start as a configuration package
across connectors, files, workflow, reporting, and templates. The package should
register sources, declare schemas, define mapping/validation versions, schedule
runs, produce previewable diffs, write governed outputs, and preserve lineage,
hashes, operator actions, and audit evidence. Create `govoplan-datasources` or
`govoplan-dataflow` only after this work exposes repeated contracts that do not
belong to existing modules.
Exit criteria:
- connector catalog exists before building many adapters
- dataflow is created only after recurring transformation becomes product
behavior
- reporting consumes governed sources with provenance
## Implementation Gates
Before a module receives implementation work, it needs:
- one reference journey step it owns
- ownership and boundary notes
- initial manifest and permission list
- capability contracts or API DTOs
- audit and policy expectations
- Gitea issues for MVP tasks
- docs page that can sync to the wiki
- focused tests that can run from the core environment
Before a module becomes release-included, it needs:
- installable Python package metadata where applicable
- WebUI package metadata where applicable
- module manifest entry point
- release catalog entry
- smoke test or permutation test
- no required imports from optional modules
## Priority Order Summary
1. Stabilize the platform spine.
2. Deliver permit-to-payment MVP.
3. Build booking and resource operations.
4. Add learning and certificates.
5. Add issue reporting and helpdesk.
6. Add records, DMS, search, and transparency.
7. Add procurement, contracts, grants, and finance handoff.
8. Add committee and consultation workflows.
9. Expand integration, dataflow, reporting, and operations.
## Deliberate Deferrals
Defer these until a reference journey proves the need:
- full ERP replacement
- native project management beyond connector support
- broad BI/dataflow platform
- every possible public-sector protocol adapter
- rich LMS behavior beyond training administration
- full qualified digital signing/trust services beyond the identity-trust and
encrypted-postbox contracts needed for early architecture safety
- mobile apps
- AI assistants embedded into workflows
These may become important, but they should not distract from the first complete
administrative journeys.
## Module And Integration Routing
This table maps current module and integration ideas to existing GovOPlaN
repositories or to explicit missing-module decisions.
| Idea | Owner | Tracking |
| --- | --- | --- |
| Government operations backbone reference model | `govoplan-core` | `add-ideas/govoplan-core#213` |
| Permit-to-payment configuration package | `govoplan-core` plus participating modules | `add-ideas/govoplan-core#214` |
| Fully UI-managed configuration with safety controls | `govoplan-admin`, `govoplan-core`, `govoplan-policy`, `govoplan-access`, `govoplan-audit` | `add-ideas/govoplan-core#218` |
| Access as a module | `govoplan-access` | `add-ideas/govoplan-access#7` |
| Interface ethics and decision-consequence doctrine | `govoplan-core` plus all UI-owning modules | `add-ideas/govoplan-core#227` |
| Action/effect automation layer | first `govoplan-workflow`; possible future `govoplan-automation` | `add-ideas/govoplan-workflow#1` |
| E2EE role/function postbox architecture | `govoplan-postbox`, `govoplan-identity-trust`, `govoplan-access`, `govoplan-policy`, `govoplan-audit` | `add-ideas/govoplan-postbox#15`, `add-ideas/govoplan-identity-trust#1` |
| Identity, account, function, role, right semantic model | `govoplan-access` | `add-ideas/govoplan-access#9` |
| Role-based service directory/catalog | `govoplan-portal` | `add-ideas/govoplan-portal#1` |
| Unified inbox across tasks, postbox, notifications, and portal | `govoplan-core` coordination plus owning modules | `add-ideas/govoplan-tasks#1`, `add-ideas/govoplan-notifications#1` |
| OpenProject API / project management connector | `govoplan-connectors` | `add-ideas/govoplan-connectors#1` |
| Native project-management module decision | connector-first through `govoplan-connectors`; no native project module yet | `add-ideas/govoplan-core#196`, `add-ideas/govoplan-connectors#1` |
| Datasources for databases, CSV, files, APIs | no repository yet; start with connectors/files/reporting and create `govoplan-datasources` only after the first package proves shared source-catalog ownership | `add-ideas/govoplan-core#197` |
| Dataflow for pipelines, BI, publication | no repository yet; start with workflow/reporting/connectors and create `govoplan-dataflow` only after repeated pipeline/lineage contracts emerge | `add-ideas/govoplan-core#198` |
| Monthly datasource and transformation workflows | first as configuration package across connectors, files, workflow, reporting, and templates | `add-ideas/govoplan-core#216` |
| Templates for letters, emails, forms, reports | `govoplan-templates`, separate from reporting | `add-ideas/govoplan-core#190`, `add-ideas/govoplan-templates#1` |
| Reporting and BI | `govoplan-reporting`, separate from templates | `add-ideas/govoplan-core#190`, `add-ideas/govoplan-reporting#1` |
| File connectors: Nextcloud, Seafile, SMB, NFS | `govoplan-files` | `add-ideas/govoplan-files#15` |
| Public-sector software integration catalogue | `govoplan-connectors` with core strategy index | `add-ideas/govoplan-core#191`, `add-ideas/govoplan-connectors#2` |
| Public-sector integration landscape catalogue | `govoplan-connectors` with core tracking | `add-ideas/govoplan-core#215` |
| Cases module concept | `govoplan-cases` | `add-ideas/govoplan-core#174` |
| Workflow module concept | `govoplan-workflow` | `add-ideas/govoplan-core#175` |
| Connectors module concept | `govoplan-connectors` | `add-ideas/govoplan-core#176` |
| Adrema-style address and distribution-list management | `govoplan-addresses` | `add-ideas/govoplan-addresses#1` |
| Consume sources and become a governed source | `govoplan-connectors` plus possible future `govoplan-dataflow` | `add-ideas/govoplan-connectors#3`, `add-ideas/govoplan-core#198` |
| Governed connector configuration, dry-run, and simulation runtime | `govoplan-connectors` | `add-ideas/govoplan-connectors#6` |
| Terminfindung and meeting scheduling polls | `govoplan-scheduling`; calendar primitives remain in calendar | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-scheduling#1` |
| Terminplaner and calendar primitives | `govoplan-calendar` | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-calendar#1` |
| Terminbuchung appointment booking | `govoplan-appointments` | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-appointments#1` |
| Collaborative documents | `govoplan-dms` | `add-ideas/govoplan-dms#1` |
| Forms | `govoplan-forms` for definitions and `govoplan-forms-runtime` for submissions/runtime behavior | `add-ideas/govoplan-core#194`, `add-ideas/govoplan-forms#1` |
| RSS consume and emit | `govoplan-connectors` | `add-ideas/govoplan-connectors#4` |
| LDAP, Active Directory, OpenDesk identity | `govoplan-idm` | `add-ideas/govoplan-idm#1` |
| OpenDesk stack integration map | integration profile across IDM/access, mail/calendar, files/DMS, and connectors; not a monolithic module | `add-ideas/govoplan-core#195`, `add-ideas/govoplan-connectors#5` |
| Open-Xchange mail/groupware | `govoplan-mail` | `add-ideas/govoplan-mail#5` |
| Open-Xchange calendar | `govoplan-calendar` | `add-ideas/govoplan-calendar#2` |
| Scalability profiles and autoscaling readiness | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#217` |
| Hardware sizing matrix and requirements calculator | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#219` |
| Collaboration suite integration strategy | `govoplan-connectors`, `govoplan-dms`, `govoplan-workflow`, `govoplan-tasks`, `govoplan-appointments`, `govoplan-calendar` | `add-ideas/govoplan-core#220` |
| Install/runtime configuration contract | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#19` |
| Installer/deployment operator flow | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#26` |
| Production-like deployment documentation | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#28` |
Boundary rationale lives in `MODULE_ARCHITECTURE.md`. Current decisions:
- templates and reporting are separate modules
- RSS/source consume-publish starts in connectors; datasources/dataflow are not
repositories yet
- calendar, scheduling, and appointments are three separate modules
- forms definitions and forms runtime are separate responsibilities
- OpenDesk is an integration profile across modules, not a monolithic module
- OpenProject is connector-first; no native projects module yet
- public-sector integration strategy stays in core; executable catalogue work
lives in connectors
- encrypted postbox and identity-trust are strategic contracts, not mail-module
behavior
- automation starts as workflow-owned action/effect execution and may split into
a dedicated module only after the runner becomes broader than workflow
The following modules are intentionally not created yet:
- `govoplan-datasources`
- `govoplan-dataflow`
- `govoplan-projects`
Create a repository only after a concrete implementation package proves that
existing connector, files, reporting, workflow, or task ownership is too narrow.
Core keeps the strategy index in
`PUBLIC_SECTOR_INTEGRATION_STRATEGY.md`: integration postures, default
ownership, and prioritization rules. `govoplan-connectors` owns the detailed
target inventory and connector entry shape in
`govoplan-connectors/docs/PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md`.
Release composition and tag-only repository handling are documented in
`RELEASE_DEPENDENCIES.md`.
## Next Practical Work
The next planning step should create or update Gitea issues for Wave 0 and Wave
1 only. Later waves should stay as roadmap context until the permit-to-payment
MVP is demonstrable.
Recommended immediate issue buckets:
- platform spine hardening
- configuration package preflight and rollback
- forms-runtime MVP
- portal submission MVP
- cases/workflow/tasks integration MVP
- template-generated decision document
- postbox/notification handoff
- appointment/booking handoff
- payment evidence handoff
- configured documentation for the reference process

View File

@@ -0,0 +1,102 @@
# GovOPlaN Interface Ethics And Design Doctrine
This document captures the product-level design doctrine for GovOPlaN. It is
more durable than an individual screen design and should guide admin,
configuration, workflow, policy, portal, and operational UI decisions.
GovOPlaN is meant to support administrative responsibility. The interface must
therefore make context, decision, consequence, and traceability visible enough
that users can act deliberately instead of being pushed through opaque
automation.
## Core Doctrine
1. Context, decision, and consequence belong together.
2. Decisions should not silently happen.
3. Transparency comes before convenience when rights, duties, records, money,
access, or legal effects are involved.
4. Explicit state is preferable to implicit state.
5. Navigation is not consent.
6. Responsibility cannot be delegated to the system.
7. Traceability is part of the action, not a later reporting feature.
8. Context loss is a product defect.
These rules do not mean every screen should become verbose. They mean the
interface must expose the right explanation at the moment of decision and keep
technical detail available without making it the default surface.
## Decision Surface Contract
Any action that changes records, rights, policies, retention, communication,
payments, external systems, or workflow state should answer these questions
before execution:
- What object, person, organization, or process is affected?
- Which authority or role allows the actor to do this?
- What will change immediately?
- What downstream effects may happen?
- Can the action be undone, superseded, or only corrected later?
- What evidence or audit entry will be created?
- Which policy, configuration, or missing capability blocks the action?
- Who can resolve a blocker?
The answer may be shown through inline labels, a review step, a side panel, a
problem list, or a confirmation dialog. The important point is that consequence
and responsibility are not hidden behind a generic submit button.
## Contestability
Administrative decisions are often contestable or reviewable. GovOPlaN should
therefore preserve the path from input to decision:
- source data and attachments
- workflow state and task assignment
- policy decisions and source path
- actor and delegation context
- generated document/template version
- external handoff result
- notification or postbox delivery evidence
- retention and record classification state
Where a user sees a decision, they should be able to reach the provenance that
explains how the system got there. This is especially important for denials,
locks, calculated defaults, generated documents, payment state, retention
state, and access decisions.
## Anti-Patterns
Avoid these patterns in GovOPlaN interfaces:
- Magical buttons that execute multiple side effects without preview.
- Process tunnels that hide where the user is in an administrative procedure.
- Silent automation that changes external systems without an audit-visible
command record.
- Friendly hiding that removes complexity at the cost of obscuring authority,
consequence, or accountability.
- Disabled controls without actionable explanation.
- Configuration screens that ask users to edit raw JSON as the normal path.
## Automation Rule
Automation must use the same governed action surface as a human actor. The
system may execute actions as a system actor, but it must still run through
policy checks, capability contracts, audit, idempotency, and failure handling.
When an automated decision is not clear, GovOPlaN should create a manual
exception, task, or review item instead of guessing silently.
## Relationship To UI Components
Shared components should make this doctrine easy to follow:
- preflight and problem-list components for blockers
- policy source path and effective decision displays
- action review panels for consequence preview
- audit/provenance links on decision outputs
- guided dialogs for risky configuration
- disabled-action explanations with actor and next step
- confirmation dialogs that distinguish reversible, corrective, and destructive
actions
The UI/UX decision ledger defines concrete implementation rules. This doctrine
defines why those rules exist.

File diff suppressed because it is too large Load Diff

129
docs/POLICY_CONTRACTS.md Normal file
View File

@@ -0,0 +1,129 @@
# GovOPlaN Policy Contracts
GovOPlaN has several policy families that are moving out of core into owning
modules. The shared kernel contract keeps their decision and provenance shape
consistent while each module still owns its domain rules.
## Current Policy Inventory
| Policy area | Current owner | Runtime surface | Notes |
| --- | --- | --- | --- |
| Privacy retention | `govoplan-policy` implementation and routes, with compatibility helpers in core | `/api/v1/admin/privacy-retention/policies/{scope}` and `/explain`; capability `policy.privacyRetention` | System, tenant, user, group, and campaign sources merge into the effective retention policy. Parent locks block lower-level widening. |
| Mail profile policy | `govoplan-mail` | `/api/v1/mail/policies/{scope}` | Uses the same source-step path format for system, tenant, owner, and campaign provenance. |
| RBAC/access policy | `govoplan-access` | access capabilities in `govoplan_core.core.access` | Permission decisions should use access capability contracts. Explain responses should adopt `PolicyDecision` when an API-level explanation is added. |
| Governance defaults | `govoplan-admin` plus `govoplan-access` materializer | admin settings, governance template routes, access materialization capability | System governance can block tenant-local groups, roles, and API keys. |
| Delegation and ownership policy | access/campaign/mail/files modules | capability checks and owner-scoped APIs | Source provenance should use this contract when policies become externally explainable. |
## Policy Decision
The shared DTO lives in `govoplan_core.core.policy.PolicyDecision`.
```json
{
"allowed": false,
"reason": "Parent retention policy locks lower-level changes.",
"source_path": [
{
"scope_type": "system",
"scope_id": null,
"path": "system",
"label": "System",
"applied_fields": ["allow_lower_level_limits"],
"policy": {}
}
],
"requirements": ["raw_campaign_json_retention_days"],
"details": {
"blocked_fields": ["raw_campaign_json_retention_days"]
}
}
```
`allowed` is the effective answer for the checked action. `reason` is a stable,
human-readable summary. `source_path` lists the policy sources that explain the
answer. `requirements` lists machine-readable blockers or prerequisites, and
`details` carries domain-specific structured context.
Every source step should be concrete enough for an operator to understand the
decision without knowing internal merge rules. Use real scope labels such as
`System`, `Tenant`, `Owner user`, `Group`, or a campaign/profile name. Include
the stable `path`, the fields applied by that step, and the local policy
fragment that caused them. This lets UIs render explanations like
`System: Allow > Tenant: Deny without override` without additional lookups.
If a policy family cannot expose the full local fragment for security reasons,
it must still include a redacted structured value that identifies the applied
field and the effective allow/deny or lock state.
## Source Path Format
Policy source paths are stable string identifiers for provenance steps:
- `system`
- `<scope_type>:<url-encoded-scope-id>`
Supported scope types are `system`, `tenant`, `user`, `group`, and `campaign`.
Examples:
- `tenant:4a45b4fe-1d86-43ce-9d10-6022333f4d4b`
- `campaign:campaign%2Fwith%20space`
Use `policy_source_path()` and `parse_policy_source_path()` instead of building
or splitting these strings manually.
## Retention Explain Endpoint
`GET /api/v1/admin/privacy-retention/policies/{scope_type}/explain` returns:
- `scope_type` and optional `scope_id`
- `decision`, using the shared `PolicyDecision` shape
- `effective_policy`
- optional `parent_policy`
- `effective_policy_sources`
- `parent_policy_sources`
- `blocked_fields`
The endpoint is read-only. Enforcement remains in the existing policy write
path. For lower-level scopes, `blocked_fields` is derived from the parent
policy's `allow_lower_level_limits`; clients can use it to disable local
controls before attempting a write.
The retention implementation lives in `govoplan-policy`
(`govoplan_policy.backend.retention`). Core keeps
`govoplan_core.privacy.retention` only as a compatibility facade for older
imports. Effective/scoped retention behavior dispatches through the
`policy.privacyRetention` capability; core does not import policy implementation
code as a hidden fallback when the module is disabled or no runtime is active.
New backend code should import policy-owned retention behavior from
`govoplan-policy` or request the capability, not add new implementation logic
to core.
The retention API DTOs live in `govoplan_core.privacy.schemas`.
`PrivacyRetentionPolicyItem`, `PrivacyRetentionPolicyPatchItem`,
`RETENTION_POLICY_FIELD_KEYS`, and `default_allow_lower_level_limits()` are
platform contracts because admin, access compatibility, and policy routes expose
the same stable retention payload shape. The policy engine's internal
`PrivacyRetentionPolicy` and `PrivacyRetentionPolicyPatch` models stay in
`govoplan-policy`, because they carry implementation validators and merge
behavior that are not generic API contracts.
Tenant administration DTOs remain owned by `govoplan-tenancy`; access keeps
matching compatibility DTOs only for its legacy admin surface. Admin overview
responses remain module-local because the same counters are exposed from
different menu contexts and are not yet a separately versioned platform API.
## Frontend Contract
Policy UIs must:
- render effective source provenance when `effective_policy_sources` is present
- display a field-level path when the source data is shown next to a specific
setting, using concrete source labels and stop at the first non-overridable
deny/lock
- disable local field controls when the parent policy sets that field's
lower-level limit to `false`
- avoid sending locked fields or re-enable attempts in save payloads
- show inherited values separately from local overrides
The core WebUI helper `privacyRetentionParentAllowsField()` centralizes the
field-lock decision used by the retention editor and its lightweight module
tests.

View File

@@ -0,0 +1,134 @@
# Postbox End-To-End Encryption Architecture
This document records the strategic encryption target for GovOPlaN postboxes.
It does not require the first postbox implementation to ship full E2EE, but it
defines the architecture so early data models and APIs do not make the stronger
model impossible.
The core principle is that a postbox can become a trusted administrative
communication channel without requiring the server to see plaintext content.
The server may route, store, authorize, audit, retain, and expire messages while
message bodies and attachments remain client-encrypted.
## Goals
- asynchronous encrypted delivery for internal and portal-facing postboxes
- personal, organizational, role-bound, and function-bound postboxes
- attachments encrypted with the message
- access based on current role/function membership when configured
- honest retraction and expiry semantics
- auditable key access, delivery, and fetch events
- support for external recipients without platform accounts
- replaceable identity and trust providers
## Envelope Model
The target model is envelope encryption:
- Generate one random data encryption key per message or attachment set.
- Encrypt content with an authenticated encryption algorithm.
- Wrap the data encryption key for each authorized recipient or role mailbox.
- Store only ciphertext, wrapped keys, signed manifests, and governed metadata
on the server.
Algorithm choices should remain replaceable behind a crypto profile. The first
profile should prefer standard, reviewed primitives such as HPKE for key
wrapping and AEAD encryption for content.
## Identity And Device Keys
The platform should distinguish:
- account identity
- tenant membership
- role/function assignment
- device key
- postbox binding
Identity providers and directories can authenticate users and provide membership
facts, but they must not see postbox private keys or message plaintext.
The trust layer should provide:
- public key directory
- account or identity signing keys
- per-device encryption keys
- device registration and revocation
- key rotation and epoch tracking
- recovery policy hooks
## Role And Function Postboxes
Role-bound access needs special handling. A postbox can be bound to an
organizational unit and a role or function. Current members can access current
messages according to policy; former members should lose access to not-yet
fetched material when revocation is still technically enforceable.
The target design should support role encryption keys or an equivalent
rewrapping service:
- sender encrypts the content key for the role/function postbox
- access service verifies current membership and required assurance
- trust service rewraps the content key to the actor's current device key
- audit records the key access decision and fetch event
Key epochs are required when role membership changes. Older messages may remain
readable according to policy, but new access must use the current epoch.
## External Recipients
External recipients may need one-time or time-limited access without a full
platform account. The target model should support capability links or invitation
tokens that are:
- scoped to specific message or attachment resources
- time-limited
- optionally one-time
- protected by an out-of-band secret, passphrase, or stronger external identity
proof
- revocable before key fetch
- fully audited
## Retraction Semantics
GovOPlaN should be honest about retraction.
Before a recipient fetches a key or decrypts content, the system can revoke
tokens, remove wrapped-key access, expire links, and delete ciphertext according
to retention policy.
After a recipient has decrypted or copied plaintext, the system cannot make the
recipient forget it. The platform can only record access, revoke future access,
notify parties, and apply legal or organizational controls.
The UI must explain this distinction whenever it offers expiry, retraction, or
message withdrawal.
## Server Responsibilities
The server remains important even when content is encrypted:
- store ciphertext and signed manifests
- store routing and policy metadata
- enforce access before key release or rewrapping
- provide public key directory access
- emit notifications without plaintext content
- record audit events
- enforce retention and expiry where possible
- expose diagnostics for delivery and key-access failures
## Module Ownership
- `govoplan-postbox` owns postbox bindings, postbox messages, message metadata,
and postbox UI.
- `govoplan-identity-trust` owns device keys, public key directory, key epochs,
and assurance integration.
- `govoplan-access` owns current identity, membership, function, delegation, and
permission decisions.
- `govoplan-policy` owns retention, retraction, and security policy decisions.
- `govoplan-audit` owns durable audit traces.
- `govoplan-files` owns managed file storage when encrypted postbox attachments
are backed by file objects.
No module should import another module's internals to decrypt content. All
interaction must use capabilities, DTOs, and audited service contracts.

View File

@@ -0,0 +1,276 @@
# Public-Sector Integration Strategy
GovOPlaN should integrate with the existing public-sector software landscape
before deciding to replace specialist workflows. This document is the core
strategy index. The executable connector catalogue lives in
`govoplan-connectors/docs/PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md`.
## Strategy Labels
Use one or more of these labels for every external system family:
- `integrate`: GovOPlaN talks to the system through a stable API/protocol.
- `link`: GovOPlaN stores external references and opens the external system for
source-of-truth work.
- `import`: GovOPlaN consumes data or files into governed module storage.
- `synchronize`: GovOPlaN keeps selected records aligned both ways or through a
source-of-truth rule.
- `replace selected workflow`: GovOPlaN may own a narrow workflow where the
external product is weak, but does not replace the whole product family.
- `no first-class support`: GovOPlaN only stores manual references unless a
deployment project creates a specific connector.
## Initial Classification
| System family | Examples | Default strategy | Likely owner |
| --- | --- | --- | --- |
| File providers | SMB/CIFS, WebDAV, Nextcloud, Seafile, SFTP, S3 | integrate, import, link | `govoplan-files`, connector inventory in `govoplan-connectors` |
| Project/task management | OpenProject, Jira, Redmine, Microsoft Planner | link, synchronize selected records, replace selected workflow only after proof | `govoplan-connectors`, later `govoplan-tasks` or workflow modules |
| Identity providers | LDAP, Active Directory, OIDC, SAML, OpenDesk IDM | integrate, synchronize | `govoplan-idm`, `govoplan-access` |
| Mail and groupware | IMAP/SMTP, Open-Xchange, Exchange/M365, CalDAV/CardDAV | integrate, link | `govoplan-mail`, `govoplan-calendar`, `govoplan-connectors` |
| DMS/e-file/archive | d.velop/d.3, enaio, ELO, Fabasoft, CMIS, VIS/eAkte | link, import, synchronize selected metadata | `govoplan-dms`, `govoplan-files`, `govoplan-records` |
| ERP/finance/procurement | SAP, MACH, Infoma, DATEV, procurement feeds | export, import, synchronize; do not replace by default | `govoplan-erp`, `govoplan-procurement`, `govoplan-ledger`, `govoplan-payments` |
| Public-sector transport | FIT-Connect, XTA/OSCI, Peppol access points | integrate, publish, receive | dedicated protocol modules plus `govoplan-connectors` inventory |
| Standards registries | XRepository, XOE/V catalogues | link, import metadata/cache | `govoplan-connectors`, `govoplan-xoev` |
| Publication/data exchange | RSS, open-data APIs, API feeds, CSV/Excel drops | consume, publish, transform | proposed `govoplan-datasources`, proposed `govoplan-dataflow`, `govoplan-reporting` |
| Collaboration suites | Matrix, Jitsi, BigBlueButton, Nextcloud Talk, Collabora/OnlyOffice | integrate, link; native behavior only for governed evidence | `govoplan-connectors`, `govoplan-dms`, `govoplan-workflow` |
| Specialist Fachverfahren | register-specific and domain-specific systems | link first; integrate/import when a real project supplies contracts | domain module or deployment-specific connector |
## Landscape Catalogue
This catalogue is intentionally implementation-oriented. Each entry records the
first API/auth/data assumptions needed to turn an inventory entry into a
connector or module issue.
### Citizen And Service Portals
- Strategy: integrate/link first; replace selected intake workflow only when a
GovOPlaN portal package owns the complete journey.
- Protocol/API surface: REST/JSON APIs, form submission webhooks, OIDC/SAML
login, eID interfaces where available, file-upload callbacks, case-status
callbacks.
- Auth model: OIDC/SAML service clients, signed webhook secrets, tenant-scoped
API keys, later eID/trust-provider handoff.
- Data shape: applicant identity reference, application form payload,
attachment references, consent declarations, status events, receipt IDs.
- Deployment assumptions: externally reachable HTTPS, reverse proxy, portal DMZ
separation, strict CSRF/origin settings, large upload path.
- Risks: personal data exposure, duplicate identity mapping, partial
submissions, upload malware, inconsistent portal status models.
- MVP test path: submit a test application with one file, create a form
submission/case/task stub, return a receipt and status reference.
- Owner/priority: `govoplan-portal`, `govoplan-forms-runtime`,
`govoplan-files`, Wave 1.
### DMS, E-File, Records, And Archives
- Strategy: link/import/synchronize selected metadata; do not replace the DMS by
default.
- Protocol/API surface: CMIS, WebDAV, vendor REST APIs, S3/object archive
staging, file-plan export/import, archive handoff APIs.
- Auth model: service accounts, OAuth/OIDC where supported, mTLS for regulated
archives, secret references for vendor tokens.
- Data shape: document ID, version, file-plan/classification code, retention
metadata, owner/case reference, external URL, checksum, lock/legal-hold state.
- Deployment assumptions: usually internal network or VPN, strict storage
quotas, existing retention policies, archive immutability requirements.
- Risks: record duplication, broken legal hold, permission mismatch, version
drift, destructive retention/export mistakes.
- MVP test path: create a connector inventory entry, test read-only metadata
lookup, link one GovOPlaN file/case evidence item to an external document.
- Owner/priority: `govoplan-dms`, `govoplan-records`, `govoplan-files`,
`govoplan-connectors`, Wave 2/5.
### ERP, Finance, Procurement, And Accounting
- Strategy: export/import/synchronize selected records; replacement only by
narrow domain decision.
- Protocol/API surface: vendor REST/SOAP APIs, CSV/XML batch exchange, SFTP,
XRechnung/Peppol, XBestellung/procurement feeds, payment reconciliation files.
- Auth model: service accounts, client certificates, mTLS, SFTP keys, token
references, environment-specific account separation.
- Data shape: debtor/creditor reference, payment request, invoice, order,
budget/cost-center code, booking status, receipt/evidence reference.
- Deployment assumptions: batch windows, finance-system approval workflows,
test tenants often separated from production by vendor process.
- Risks: financial posting errors, double export, tax/legal data retention,
inconsistent master data, irreversible accounting handoff.
- MVP test path: dry-run export of one payment/accounting handoff file with
checksum, validation report, and no remote posting.
- Owner/priority: `govoplan-payments`, `govoplan-ledger`,
`govoplan-xrechnung`, `govoplan-erp`, `govoplan-procurement`, Wave 1/6.
### Identity, IAM, And Directory Services
- Strategy: integrate/synchronize; access remains GovOPlaN's local
authorization boundary.
- Protocol/API surface: LDAP, Active Directory, SCIM, OIDC, SAML, OpenDesk IDM
APIs, group membership sync, account deactivation feeds.
- Auth model: bind accounts, service clients, OIDC/SAML metadata, SCIM tokens,
certificate-backed clients where required.
- Data shape: account, user, group, membership, role claim, tenant/org-unit
mapping, status, external directory ID.
- Deployment assumptions: directory is usually internal; identity provider may
be organization-wide and not GovOPlaN-owned.
- Risks: privilege escalation through group mapping, stale memberships, account
collision, deprovisioning latency, tenant-boundary mistakes.
- MVP test path: read-only directory profile test, map one external group to a
tenant group, show a dry-run membership diff.
- Owner/priority: `govoplan-idm`, `govoplan-access`, Wave 1.
### Groupware, Mail, Calendar, And Collaboration
- Strategy: integrate/link; native behavior only where GovOPlaN needs governed
evidence or process state.
- Protocol/API surface: IMAP/SMTP, CalDAV/CardDAV, Open-Xchange APIs, Microsoft
Graph/EWS, Matrix APIs, Jitsi/BigBlueButton APIs, Collabora/OnlyOffice
integration points.
- Auth model: service accounts, delegated OAuth/OIDC, app passwords, mailbox
credentials, groupware-specific tokens, secret references.
- Data shape: mailbox folder/message references, event/free-busy data, meeting
URL, chat room ID, participant list, document-editing session reference.
- Deployment assumptions: often internal/existing tenant infrastructure; mail
and calendar may be separate from identity even in OpenDesk-style stacks.
- Risks: mail credential exposure, calendar privacy, double invitations, room
booking conflicts, chat/document data escaping retention rules.
- MVP test path: profile test for mailbox/calendar reachability, read-only
folder/free-busy lookup, create a non-production event/message draft.
- Owner/priority: `govoplan-mail`, `govoplan-calendar`,
`govoplan-connectors`, Wave 1/2.
### Payment And Public Cashier Systems
- Strategy: integrate/export/import; keep the payment provider or cashier as
source of settlement truth.
- Protocol/API surface: payment provider APIs, redirect/callback flows,
reconciliation files, SEPA/export formats, cash-register/cashier interfaces.
- Auth model: provider API keys, signed webhooks, client certificates, mTLS,
tenant-specific merchant accounts.
- Data shape: payment intent, amount/currency, payer reference, provider
transaction ID, settlement status, receipt, refund/cancellation reference.
- Deployment assumptions: public callback URLs, strict environment separation,
PCI-sensitive providers, finance reconciliation cadence.
- Risks: duplicate charges, callback replay, amount mismatch, refund workflow
gaps, evidence-retention mistakes.
- MVP test path: sandbox payment intent, signed callback verification, receipt
evidence link, reconciliation dry-run.
- Owner/priority: `govoplan-payments`, `govoplan-ledger`, Wave 1/6.
### Reporting, BI, Open Data, And Publication
- Strategy: consume/publish/transform; native reporting owns GovOPlaN views, not
every external BI product.
- Protocol/API surface: SQL read replicas, CSV/Excel export/import, REST APIs,
RSS/Atom, open-data APIs, SFTP/WebDAV publication targets.
- Auth model: read-only DB users, API tokens, SFTP keys, OAuth clients, public
anonymous publication profiles where appropriate.
- Data shape: dataset metadata, schema/version, report parameters, generated
file references, publication URL, freshness/lineage, validation results.
- Deployment assumptions: publication can be public or internal; generated
datasets need retention and provenance.
- Risks: leaking restricted data, stale publications, schema drift, expensive
queries, untraceable manual transformations.
- MVP test path: publish one report/export as a governed file plus RSS/Atom
entry with checksum, timestamp, and permission check.
- Owner/priority: `govoplan-reporting`, `govoplan-connectors`, possible future
`govoplan-datasources`/`govoplan-dataflow`, Wave 2.
### Public-Sector Protocols And Registries
- Strategy: integrate/publish/receive; protocol modules own protocol semantics.
- Protocol/API surface: FIT-Connect, XTA/OSCI, XRepository, XOE/V, XRechnung,
XBestellung, Peppol, registry-specific Fachverfahren APIs.
- Auth model: certificates, mTLS, service accounts, destination credentials,
protocol-specific trust anchors and key rotation.
- Data shape: transport envelope, payload schema/version, destination IDs,
receipt/acknowledgement, message status, standard-specific metadata.
- Deployment assumptions: regulated trust chains, test/prod endpoint
separation, formal onboarding, strict logging and retention expectations.
- Risks: invalid schemas, failed delivery receipts, certificate expiry, wrong
destination routing, protocol version drift.
- MVP test path: validate a sample payload against a schema, test endpoint
reachability in sandbox, store receipt/evidence reference.
- Owner/priority: `govoplan-fit-connect`, `govoplan-xoev`,
`govoplan-xrechnung`, `govoplan-xta-osci`, `govoplan-connectors`, Wave 1/2.
### File Providers And Shared Storage
- Strategy: integrate/import/link; files module owns GovOPlaN file semantics.
- Protocol/API surface: SMB/CIFS, WebDAV, Nextcloud, Seafile, SFTP, S3,
local/object storage profiles.
- Auth model: service accounts, user credentials, app tokens, OAuth where
supported, secret references, environment variables for deployment-managed
credentials.
- Data shape: file ID/path, provider object ID, checksum, MIME type, size,
version/ETag, owner, permission snapshot, imported file reference.
- Deployment assumptions: internal networks, large files, existing shares,
variable permissions, provider-specific rate limits.
- Risks: permission mismatch, stale imports, overwrites, duplicate files, path
traversal, storage growth.
- MVP test path: profile test, list folder, import one file into governed
storage, keep provider reference and checksum.
- Owner/priority: `govoplan-files`, `govoplan-connectors`, Wave 0/1.
### Project, Task, And Case-Adjacent Systems
- Strategy: connector-first for OpenProject/Jira/Redmine; native module only
when GovOPlaN owns project semantics.
- Protocol/API surface: OpenProject API v3, webhooks, Jira/Redmine REST APIs,
Microsoft Graph for Planner/Project where applicable.
- Auth model: API tokens, OAuth/OIDC apps, webhook secrets, service accounts.
- Data shape: project ID, work package/task ID, status, assignee reference,
external URL, version/lock token, publish/sync trace.
- Deployment assumptions: external project tool remains source of truth for
broad project management; GovOPlaN links selected records.
- Risks: task duplication, bidirectional sync conflicts, permission mismatch,
over-mirroring comments/attachments.
- MVP test path: OpenProject profile test, project/work-package lookup,
external-reference round-trip.
- Owner/priority: `govoplan-connectors`, later `govoplan-tasks`/workflow/cases
consumers, Wave 0/2.
### Specialist Fachverfahren
- Strategy: link first; integrate/import only when a deployment project supplies
concrete contracts and a domain owner.
- Protocol/API surface: vendor APIs, CSV/XML batch imports, SFTP, database
views, message queues, protocol-specific transports.
- Auth model: usually service accounts, VPN, mTLS, SFTP keys, or vendor tokens.
- Data shape: domain-specific record IDs, status, applicant/person references,
file/evidence references, case/status events.
- Deployment assumptions: strongly local/vendor-specific, often no stable test
API, data model differs by jurisdiction.
- Risks: brittle vendor contracts, legal source-of-truth ambiguity, high
customization cost, migration expectations.
- MVP test path: inventory entry and manual external-reference link; require a
project-specific connector issue before automation.
- Owner/priority: domain module or deployment-specific connector, case by case.
## Prioritization Rules
1. Start with connectors that unblock Wave 0 or Wave 1 reference journeys.
2. Prefer open standards and self-hosted/open-source APIs where they are common
in public-sector deployments.
3. Treat inventory-only entries as useful because operators need a map of their
software landscape even before automation exists.
4. Keep connector code in the owning connector/protocol module. Domain modules
consume capabilities, DTOs, external references, and events through core.
5. Every executable connector needs health diagnostics, secret-reference
handling, lifecycle state, audit events, and retirement behavior.
## Connector Catalogue Handoff
`govoplan-connectors` owns the detailed catalogue entry shape:
- connector type key
- category and owner module
- supported directions and trigger modes
- credential and secret handling
- health check and diagnostics payload
- external-reference shape
- required capabilities and optional module combinations
- lifecycle support
Core should only keep strategy, routing, and cross-module architecture notes.
Connector implementation and public-sector target inventory belong in
`govoplan-connectors`.

View File

@@ -1,291 +0,0 @@
# Multi Seal Mail - Current RBAC and Resource-Access Model
**Updated:** 2026-06-16
**Current migration head:** `f5a6b7c8d9e0`
## Authorization Equation
An operation is permitted only when every applicable layer allows it:
```text
effective role/API-key capability
AND resource ownership/share access
AND workflow state
AND active governance/policy constraints
```
RBAC answers what an actor may do. ACLs answer which resource the actor may do it to. Workflow state and policy decide whether the operation is currently valid.
## Identity and Scope
```text
Account global login identity
+- User membership tenant-local identity
+- direct tenant roles
+- active group memberships
| +- inherited tenant roles
+- tenant-local API keys
Account
+- direct system-role assignments
```
A browser session has one active tenant membership. System privileges do not silently grant tenant data access. API keys remain tenant-local and receive the intersection of their configured scopes and their owner's live tenant scopes on every request.
## Wildcards
```text
tenant:* every canonical tenant permission
system:* every canonical system permission
* legacy alias interpreted as tenant:* only
```
Tenant wildcards never grant system permissions.
## Canonical Tenant Permissions - 53
### Campaigns
```text
campaign:read
campaign:create
campaign:update
campaign:copy
campaign:archive
campaign:delete
campaign:share
campaign:validate
campaign:build
campaign:review
campaign:send_test
campaign:queue
campaign:control
campaign:send
campaign:retry
campaign:reconcile
```
### Recipients
```text
recipients:read
recipients:write
recipients:import
recipients:export
```
### Files
```text
files:read
files:download
files:upload
files:organize
files:share
files:delete
files:admin
```
### Reports and Audit
```text
reports:read
reports:export
reports:send
audit:read
```
### Mail Servers
```text
mail_servers:read
mail_servers:use
mail_servers:test
mail_servers:write
mail_servers:manage_credentials
```
### Tenant Administration
```text
admin:users:read
admin:users:create
admin:users:update
admin:users:suspend
admin:groups:read
admin:groups:write
admin:groups:manage_members
admin:roles:read
admin:roles:write
admin:roles:assign
admin:api_keys:read
admin:api_keys:create
admin:api_keys:revoke
admin:settings:read
admin:settings:write
admin:policies:read
admin:policies:write
```
## Canonical System Permissions - 18
```text
system:tenants:read
system:tenants:create
system:tenants:update
system:tenants:suspend
system:accounts:read
system:accounts:create
system:accounts:update
system:accounts:suspend
system:roles:read
system:roles:write
system:roles:assign
system:access:read
system:access:assign
system:audit:read
system:settings:read
system:settings:write
system:governance:read
system:governance:write
```
`system:access:*` remains as a compatibility/read and assignment boundary for cross-tenant/system access handling. It is not a separate primary UI area.
## Default Tenant Roles
- **Owner:** `tenant:*`. At least one active operational owner must remain.
- **Tenant administrator:** settings, policies, users, groups, roles and API keys plus read access to campaigns/files/reports/audit. Real delivery remains separately delegable.
- **Administrator (legacy):** all tenant permissions for upgraded installations.
- **Access administrator:** membership and assignment management within delegation limits.
- **Campaign manager:** prepare, validate and build campaigns; no review approval or real delivery by default.
- **Reviewer:** inspect and approve prepared campaign messages.
- **Sender:** mock-test, queue, control, send, retry and reconcile prepared campaigns; can use/test approved mail profiles.
- **File manager:** managed file operations without campaign delivery rights.
- **Viewer:** read campaigns, recipients, files and reports.
- **Auditor:** read campaigns, recipient evidence, reports and audit records; export detailed evidence.
## Default System Roles
- **System owner:** `system:*`, protected. At least one active account must retain it.
- **System administrator:** all specific system permissions, editable and not protected.
- **System auditor:** read-only system registry/settings/governance/audit role, editable.
## Delegation Ceiling
For role definition, assignment and API-key creation:
```text
requested scopes subset of actor delegateable scopes
```
Rules:
1. Tenant roles may contain tenant scopes only.
2. System roles may contain system scopes only.
3. Definition rights and assignment rights are separate.
4. Group definition and group membership management are separate.
5. API-key scopes are intersected with the owner's current effective scopes on every request.
6. Suspended accounts, users, tenants or groups stop contributing access immediately.
7. Administrative updates are field-sensitive; a user with only status authority cannot change role assignments.
## Campaign Ownership and ACLs
A campaign has exactly one owner:
```text
owner user OR owner group
```
Additional active shares may target users or groups with `read` or `write`.
Resolution:
- owner user: read and write;
- member of owner group: read and write;
- explicit read share: read;
- explicit write share: read and write;
- `tenant:*`: tenant-wide ACL bypass;
- ordinary campaign permission without ownership/share: no object access.
ACLs do not add capabilities. A write share still needs the specific permission for update, validation, review, send, report, retry or reconciliation.
## Sensitive Recipient Boundary
Recipient-complete campaign JSON, message data and job detail require `recipients:read`. Recipient edits require `recipients:write`; exports require `recipients:export`; import is reserved for the dedicated recipient import/list workflow.
## Files
| Permission | Operations |
|---|---|
| `files:read` | list, search, inspect, resolve metadata |
| `files:download` | download file bytes and generated ZIP archives |
| `files:upload` | upload files and ZIP contents |
| `files:organize` | create folders, rename, move, copy and bulk rename |
| `files:share` | create/revoke file shares |
| `files:delete` | delete/hide files and folders subject to retention |
| `files:admin` | tenant-wide administration of user/group file spaces |
## Mail Servers
| Permission | Boundary |
|---|---|
| `mail_servers:read` | profile metadata and effective policy visibility |
| `mail_servers:use` | select an approved profile without reading secrets |
| `mail_servers:test` | run server-side connection tests |
| `mail_servers:write` | define/edit profiles in allowed scopes |
| `mail_servers:manage_credentials` | create/replace SMTP/IMAP secrets or campaign-level credentials where policy allows |
Reusable encrypted profiles now exist. Effective usability is also constrained by hierarchical mail-profile policy, ownership, allowed/forced profile sets, credential inheritance mode, the lower-level override switch for that mode, and allow/deny patterns.
## Sessions, API Keys and CSRF
- Browser login creates an HttpOnly session cookie and a separate readable CSRF cookie.
- Unsafe cookie-authenticated requests require matching CSRF cookie/header and stored CSRF hash.
- API keys remain supported for CLI/automation and do not use browser CSRF.
- Login responses still expose a compatibility session token in the response body; the WebUI does not persist it.
## Legacy Compatibility
Runtime aliases remain only for names that are no longer canonical, including:
```text
campaign:write
attachments:read
attachments:write
admin:users
admin:users:write
admin:api_keys:write
admin:settings
system:tenants:write
system:access:write
```
Canonical scopes are not widened by runtime alias expansion after migration.
## Deferred Permission Families
Add these only with their corresponding implemented features:
```text
templates:*
address_books:*
recipient_lists:*
connectors:*
dsar:*
system:monitoring:read
system:backups:run
system:backups:restore
system:updates:apply
system:updates:rollback
```
A separate `retention:*` family is not currently canonical because retention is managed through system settings and tenant policy scopes. Add it only if retention operation duties need separation from general policy/settings administration.

View File

@@ -1,71 +1,902 @@
# GovOPlaN Release Dependencies
Release installs must not depend on sibling checkout paths. Local development can keep editable installs and `file:` WebUI links, but release packaging should resolve modules from tagged git refs or from a package registry.
This document owns release package composition, signed package catalogs,
license checks, catalog publishing, migration baselines, and the final release
checklist.
## Backend
Operator runtime configuration and module install/uninstall execution live in
`DEPLOYMENT_OPERATOR_GUIDE.md`.
## Backend Packages
Release installs must not depend on sibling checkout paths. Local development
can keep editable installs and `file:` WebUI links, but release packaging must
resolve modules from tagged git refs or from a package registry.
Local development:
```bash
cd /mnt/DATA/git/govoplan-core
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m pip install -r requirements-dev.txt
```
Release install from a core checkout plus tagged module repositories:
Release install from the meta checkout plus tagged module repositories:
```bash
cd /mnt/DATA/git/govoplan-core
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m pip install -r requirements-release.txt
```
`.[server]` is resolved relative to the current working directory. If you create the virtualenv elsewhere, still run the install command from the core checkout:
`../govoplan-core[server]` is resolved relative to the meta requirements file.
If you create the virtualenv elsewhere, still run the install command from the
meta checkout:
```bash
cd /mnt/DATA/git/govoplan-core
cd /mnt/DATA/git/govoplan
/tmp/govoplan-release-test/bin/python -m pip install -r requirements-release.txt
```
`requirements-release.txt` pins the module repositories to the release tag. Update those refs when cutting a release:
`requirements-release.txt` pins the module repositories to the release tag.
Update those refs when cutting a release:
```text
govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.1
govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.1
govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.1
govoplan-access git@git.add-ideas.de:add-ideas/govoplan-access.git v0.1.6
govoplan-admin git@git.add-ideas.de:add-ideas/govoplan-admin.git v0.1.6
govoplan-tenancy git@git.add-ideas.de:add-ideas/govoplan-tenancy.git v0.1.6
govoplan-organizations git@git.add-ideas.de:add-ideas/govoplan-organizations.git v0.1.6
govoplan-identity git@git.add-ideas.de:add-ideas/govoplan-identity.git v0.1.6
govoplan-policy git@git.add-ideas.de:add-ideas/govoplan-policy.git v0.1.6
govoplan-audit git@git.add-ideas.de:add-ideas/govoplan-audit.git v0.1.6
govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.6
govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.6
govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.6
govoplan-calendar git@git.add-ideas.de:add-ideas/govoplan-calendar.git v0.1.6
```
## WebUI
## WebUI Packages
Local development uses `webui/package.json`, which may point at sibling module checkouts while active development is happening.
Local development uses `webui/package.json`, which may point at sibling module
checkouts while active development is happening.
Release WebUI installs should use `webui/package.release.json`. It points module dependencies at the same tagged git repositories. After the module tags referenced there exist, generate the committed release lockfile without touching the development package files:
Release WebUI installs should use `webui/package.release.json`. It points
module dependencies at the same tagged git repositories. After the module tags
referenced there exist, generate the committed release lockfile without
touching the development package files:
```bash
cd /mnt/DATA/git/govoplan-core
scripts/generate-release-lock.sh
cd webui
cd /mnt/DATA/git/govoplan
tools/release/generate-release-lock.sh
cd /mnt/DATA/git/govoplan-core/webui
PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build
```
The module repositories include root-level npm package manifests so git installs can resolve `@govoplan/files-webui`, `@govoplan/mail-webui`, and `@govoplan/campaign-webui` from repository roots even though their source lives below `webui/src`.
The module repositories include root-level npm package manifests so git
installs can resolve `@govoplan/access-webui`, `@govoplan/admin-webui`,
`@govoplan/files-webui`, `@govoplan/mail-webui`,
`@govoplan/campaign-webui`, and `@govoplan/calendar-webui` from repository
roots even though their source lives below `webui/src`.
The normal release path is automated by `scripts/push-release-tag.sh`: it bumps or accepts the target version, updates Python/WebUI/module manifest versions, commits/tags/pushes the module repositories first, regenerates `webui/package-lock.release.json`, and then commits/tags/pushes core. If the working tree has already been bumped, pass the current version explicitly:
### Release Lockfile Strategy
The supported release composition currently is the full GovOPlaN product: core
plus access, admin, tenancy, organizations, identity, policy, audit,
dashboard, files, mail, campaign, calendar, docs, and ops. Keep one committed
full-product release lockfile at
`webui/package-lock.release.json`, generated from
`webui/package.release.json` in a clean release workspace. Development
`package-lock.json` may continue to point at local `file:` dependencies.
Frontend module permutations are regression-tested through
`GOVOPLAN_WEBUI_MODULE_PACKAGES` and temporary build output, not through
committed lockfiles for every possible combination. If a smaller composition
becomes a separately shipped product, add an explicit release manifest and
lockfile pair for that product, for example
`package.release.files-mail.json` and `package-lock.release.files-mail.json`,
generated in a clean release workspace from tagged git dependencies.
## Release Tag Script
The normal release path is automated by `/mnt/DATA/git/govoplan/tools/release/push-release-tag.sh`: it bumps
or accepts the target version, updates Python/WebUI/module manifest versions,
commits/tags/pushes the module repositories first, regenerates
`webui/package-lock.release.json`, and then commits/tags/pushes core. If the
working tree has already been bumped, pass the current version explicitly:
```bash
cd /mnt/DATA/git/govoplan-core
scripts/push-release-tag.sh --version 0.1.2
cd /mnt/DATA/git/govoplan
tools/release/push-release-tag.sh --version 0.1.6
```
### Release lockfile strategy
`/mnt/DATA/git/govoplan/tools/release/generate-release-catalog.py` reads installed/discovered
`ModuleManifest` objects while writing catalog entries. When a manifest is
available, the catalog entry uses the manifest version, points package refs at
`v<manifest.version>`, and copies `provides_interfaces` /
`requires_interfaces` from the manifest. It also copies module migration order
and `migration_tasks` metadata when present. If a manifest cannot be
discovered, the entry falls back to the release version passed with `--version`
and omits interface and migration-task metadata. This keeps the catalog aligned
with independently versioned module packages instead of relying on a hardcoded
compatibility table.
The supported release composition currently is the full Multi Seal Mail product: core plus files, mail, and campaign. Keep one committed full-product release lockfile at `webui/package-lock.release.json`, generated from `webui/package.release.json` in a clean release workspace. Development `package-lock.json` may continue to point at local `file:` dependencies.
The script also includes GovOPlaN roadmap/scaffold module repositories that do
not yet have package metadata. Those repositories are committed, tagged, and
pushed with the same release tag, but they are tag-only until they contain
`pyproject.toml`, module manifests, or WebUI packages. Tag-only repositories
are not listed in `requirements-release.txt` or `webui/package.release.json`.
Frontend module permutations are regression-tested through `GOVOPLAN_WEBUI_MODULE_PACKAGES` and temporary build output, not through committed lockfiles for every possible combination. If a smaller composition becomes a separately shipped product, add an explicit release manifest and lockfile pair for that product, for example `package.release.files-mail.json` and `package-lock.release.files-mail.json`, generated in a clean release workspace from tagged git dependencies.
Current tag-only module repositories:
- `govoplan-appointments`
- `govoplan-cases`
- `govoplan-connectors`
- `govoplan-dms`
- `govoplan-erp`
- `govoplan-fit-connect`
- `govoplan-forms`
- `govoplan-identity-trust`
- `govoplan-idm`
- `govoplan-ledger`
- `govoplan-notifications`
- `govoplan-payments`
- `govoplan-portal`
- `govoplan-reporting`
- `govoplan-scheduling`
- `govoplan-search`
- `govoplan-tasks`
- `govoplan-templates`
- `govoplan-workflow`
- `govoplan-xoev`
- `govoplan-xrechnung`
- `govoplan-xta-osci`
## Catalog Trust And Licensing
GovOPlaN module install and uninstall must remain operator-controlled. The
running server may plan and validate package changes, but package mutation is
performed by the separate installer daemon or an operator shell during
maintenance mode.
`addideas-govoplan-website` is the public static distribution surface for
official catalog resources:
- signed module package catalogs, grouped by release channel
- public catalog keyrings
- public license verification keyrings
- examples and operator-facing download paths
`govoplan-core` is the verifier and orchestrator:
- fetches a local or remote module catalog
- verifies catalog signatures against configured trusted keys
- enforces approved release channels
- rejects expired or not-yet-valid catalogs
- records accepted catalog sequence numbers for replay protection
- checks catalog entry license feature requirements before planning installs
- writes installer plans and request records
Feature and platform modules own their package artifacts, manifests, migration
metadata, retirement providers, and optional lifecycle behavior.
Core accepts either a local catalog file or a remote URL:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG=/srv/govoplan/catalogs/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.example/catalogs/v1/channels/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json
```
If both file and URL are set, the URL wins. The cache is used when a remote
fetch fails, so an operator can still inspect the last known catalog. A cached
catalog must still pass signature, freshness, channel, and replay validation.
An official catalog is a JSON object with:
- `catalog_version`
- `channel`
- `sequence`
- `generated_at`
- `not_before` when delayed activation is needed
- `expires_at`
- `modules`
- `signatures`
Each module entry can declare:
- backend package name and pinned install reference
- WebUI package name and pinned install reference
- display metadata and tags
- `license_features`, the feature entitlements required to plan that install
- `dependencies` and `optional_dependencies`, the module ids expected in the
target module set
- `migration_safety`, one of `automatic`, `requires_review`, `forward_only`,
or `destructive`
- `migration_notes`, operator-facing data/migration guidance for review,
forward-only, or destructive changes
- `migration_after` and `migration_before`, explicit module ids used to order
module-owned migration heads when a release needs a live-data sequencing rule
- `migration_tasks`, constrained live-data tasks that run around Alembic
migration phases. Each task declares `task_id`, `phase`, `summary`,
`task_version`, `safety`, `idempotent`, and optionally `timeout_seconds`.
The allowed phases are `pre_migration_check`, `pre_migration_prepare`,
`post_migration_backfill`, and `post_migration_verify`.
- `current_version_min` and `current_version_max_exclusive`, the installed
version window from which this catalog target may be applied directly
- `bridge_release` and `bridge_notes`, marking a target as an intermediate
compatibility release in a staged update path
- `allow_downgrade` and `allow_same_version`, explicit opt-ins for reviewed
rollback or package-refresh plans
- `recovery_tested` and `recovery_notes`, documenting the rehearsal for
forward-only or destructive data changes
- `provides_interfaces`, named interface contracts exported by this module
- `requires_interfaces`, named interface contracts and version ranges required
by this module
The signature is Ed25519 over canonical JSON with both `signature` and
`signatures` removed. Core accepts the legacy single `signature` field and the
new `signatures` array.
When `APP_ENV` is `prod` or `production`, module package catalog signature
verification is required by default unless
`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=false` is set explicitly.
### Independent Module Versions And Interface Ranges
Modules do not need to ship on the same version number. Release catalogs should
pin each package to the exact backend/WebUI ref being installed and declare any
cross-module API compatibility through named interfaces.
Provider shape:
```json
"provides_interfaces": [
{ "name": "files.campaign_attachments", "version": "1.4.0" }
]
```
Requirement shape:
```json
"requires_interfaces": [
{
"name": "files.campaign_attachments",
"version_min": "1.0.0",
"version_max_exclusive": "2.0.0"
}
]
```
`version_min` is inclusive. `version_max_exclusive` is exclusive, so the range
above means `>= 1.0.0` and `< 2.0.0`. Use this for SemVer major-version
compatibility lines. Set `"optional": true` only when the module can operate
without that interface being present. If a provider is installed but its
version is outside the declared optional range, activation is still blocked.
Catalog validation normalizes these fields and warns when catalog entries do
not satisfy each other's ranges. Registry activation and installer preflight
perform the blocking checks against the discovered installed manifests before
the desired module set is activated.
The admin module-management UI shows catalog warnings in the package catalog
section and repeats them as warning-level installer preflight issues while a
package install is planned.
Install-plan items carry a `source` field. Manually entered items use
`source: "manual"`; entries planned from the package catalog use
`source: "catalog"`. Catalog-sourced items also carry a `catalog` metadata
object with the validation snapshot used when the item was planned: catalog
source/path, source type, cache path, channel, sequence, generated/validity
timestamps, signature state, trusted key id, and cache state where available.
Catalog provenance changes preflight severity:
- catalog-sourced installs and updates require a configured, valid package
catalog before activation
- invalid, untrusted, expired, not-yet-valid, replayed, or unapproved-channel
catalogs block catalog-sourced installs and updates
- the same catalog validation failures remain warnings for manual install
plans, so operators can still use offline or emergency package refs
- valid-catalog warnings, such as intentionally unsigned local catalogs when
signature enforcement is disabled, remain warnings
- selected catalog entries with unsatisfied non-optional named interface ranges
block activation before the installer runs
- selected catalog entries whose target dependencies are neither installed nor
planned block activation before the installer runs
- catalog update targets older than the installed module version block unless
the catalog entry declares `allow_downgrade: true`
- catalog update targets equal to the installed module version block unless the
catalog entry declares `allow_same_version: true`
- catalog update targets with a `current_version_min` /
`current_version_max_exclusive` window block when the installed version is
outside that window; publish and apply a bridge release instead
- catalog entries marked `forward_only` or `destructive` block activation until
the plan row has an explicit data-safety acknowledgement
- catalog entries marked `forward_only` or `destructive` also block unless the
catalog entry declares `recovery_tested: true` and either the catalog entry or
operator plan row contains recovery notes
- catalog entries marked `destructive` also require catalog migration notes or
operator notes describing the cleanup or retirement plan
### Update Paths
Package updates are target-state operations, not one-module-at-a-time runtime
toggles. The safe unit of planning is a desired module version set plus a
catalog validation snapshot. The installer may install multiple packages into
the environment before activation, then validate the discovered manifests and
activate the resulting set together.
Install-plan rows support explicit `install`, `update`, and `uninstall`
actions. Catalog planning writes `update` when the module is already installed.
Preflight resolves the target set from installed manifests plus the planned
catalog entries. Unplanned catalog entries are not treated as installed. When a
catalog entry would satisfy a missing dependency or named interface, preflight
blocks activation with a companion-update issue; the admin catalog planner adds
those companion rows automatically when it can resolve them from the current
catalog. The preflight response also includes a structured `target_plan` summary
with each planned module's action, current version, catalog target version,
package refs, migration-safety level, current-version update window, bridge
metadata, recovery metadata, and acknowledgement state.
Database migrations are planned against that same target module set. When the
installer is run with `--migrate`, it calls `govoplan_core.commands.init_db`
with the target enabled modules rather than the pre-update startup module list,
so newly installed module migration directories are discovered before
activation. Preflight also returns a structured migration plan. Its step order is
derived from:
- manifest and catalog `migration_after` / `migration_before` declarations
- module dependencies and optional dependencies when both modules are in the
target plan
- named interface provider/consumer relationships when both sides are in the
target plan
Preflight blocks cycles in that ordering graph. It also blocks non-idempotent
module migration tasks, forward-only/destructive tasks without operator
acknowledgement, and installed manifest tasks that declare no executor.
Catalog-only task executors are marked as pending because they can only be
confirmed after the target package is installed. The migrator runs pre-migration
tasks, upgrades the ordered module heads first, finishes with Alembic `heads`,
and then runs post-migration tasks, so Alembic's revision graph remains
authoritative while GovOPlaN still gives operators a module-aware live-data
order.
This avoids circular "upgrade A first / upgrade B first" traps: named interface
requirements are solved against the target set, not against each intermediate
package-install moment. If the target set cannot satisfy all non-optional
interfaces and module dependencies at once, the plan is invalid. Operators
should add the necessary module updates to the same plan instead of trying to
force an order.
Live data upgrades need an even stricter rule:
- migrations must be idempotent and ordered by module migration metadata
- destructive schema/data changes need an explicit retirement or cleanup plan,
not an automatic package update side effect
- cross-module data migrations must be compatible with both the old and target
provider interface until activation finishes
- rollback must restore the package set and database state together, or be
documented as forward-only with a tested recovery procedure
- if two modules require mutually incompatible live-data states, the catalog
must publish an intermediate compatibility release rather than a circular
update chain
The release catalog is the first safety gate for this. Generated catalogs mark
modules with registered migrations as `requires_review` by default. Release
authors should keep that value for ordinary reversible migrations, raise it to
`forward_only` when database rollback requires restoring a snapshot, and raise
it to `destructive` when the update removes or irreversibly rewrites persisted
data. Forward-only and destructive entries must include `recovery_tested: true`
and recovery notes after a verified restore or forward-recovery rehearsal. The
admin install-plan UI exposes the safety level and lets operators record an
explicit acknowledgement; preflight keeps acknowledged forward-only/destructive
changes visible as warnings.
In practice, circular dependencies are avoided by designing interfaces with
compatibility windows and by publishing bridge releases. A bridge release keeps
the old interface while introducing the new one, allowing dependent modules to
move first; a later release can retire the old interface after every dependent
module has a compatible target version. Use `current_version_min` and
`current_version_max_exclusive` to make those direct-update windows explicit in
the catalog, and set `bridge_release: true` on intermediate targets that exist
primarily to carry installations safely across a compatibility gap.
Trusted catalog keys are configured locally:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS='{"release-key-1":"<base64 public key>"}'
```
For development or tightly controlled deployments, a keyring can be read from a
URL and cached:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_URL=https://govoplan.example/catalogs/v1/keyring.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_CACHE=/srv/govoplan/runtime/catalog-cache/keyring.json
```
Production installations should pin the trusted keyring locally or ship it
through deployment configuration. Fetching trusted keys from the same public
origin as the catalog is convenient, but that origin must not become the only
trust root.
## Dependency Audits
Dependency vulnerability checks are documented in
[`DEPENDENCY_AUDITS.md`](DEPENDENCY_AUDITS.md). The local audit runner is:
```bash
cd /mnt/DATA/git/govoplan
bash tools/checks/check-dependency-audits.sh
```
The Gitea workflow in `govoplan/.gitea/workflows/dependency-audit.yml` runs the same
check against release dependency refs on pushes, pull requests, and a weekly
schedule.
Keyring entries support:
- `key_id`
- `public_key` or `public_key_base64`
- `status`: `active`, `next`, `retired`, `revoked`, or `disabled`
- `not_before`
- `not_after`
Rotation process:
1. Add the next public key to the local trusted keyring with status `next`.
2. Publish catalogs signed by both current and next keys.
3. Upgrade installations so the next key is locally trusted.
4. Promote the next key to `active`.
5. Retire the old key only after every supported installation trusts the new
key.
6. Mark a compromised key `revoked` and publish a higher sequence catalog
signed by an uncompromised key.
Use replay state in production:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true
```
Core records the accepted sequence per channel after a catalog entry is planned
from the admin interface. With strict sequence enforcement, a previously
accepted sequence is rejected; without strict enforcement, only older sequences
are rejected. Catalogs should always expire.
The sequence state file is operational state, not a trust root. Keep it on
persistent storage and include it in normal backups:
```json
{
"channels": {
"stable": {
"last_sequence": 42,
"accepted_at": "2026-07-07T12:00:00Z",
"key_id": "release-key-1",
"source": "https://govoplan.example/catalogs/v1/channels/stable.json"
}
}
}
```
If the file is lost, restore it from backup. If no backup exists, reconstruct
each channel from the highest sequence already accepted in installer run
records, release records, or the currently deployed module package set. Do not
lower `last_sequence` to make an older catalog pass; publish a new higher
sequence catalog when the accepted point is uncertain.
If the file is corrupted, copy it aside for incident review, validate the
current signed catalog with channel and freshness enforcement, then rewrite the
state with the known accepted sequence. Keep
`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true` and approved-channel
checks enabled during recovery. Temporarily disabling
`GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE` allows revalidating the same
sequence, but older sequences remain rejected once the reconstructed
`last_sequence` is in place.
Approved channels are deployment policy:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable,lts
GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true
```
The admin UI can display other catalog metadata, but core rejects catalogs from
unapproved channels when validation is configured.
Catalog entries can require license features:
```json
"license_features": ["module.mail", "support.standard"]
```
Core checks those requirements against an offline license file before allowing
the entry into the install plan.
```bash
GOVOPLAN_LICENSE_FILE=/srv/govoplan/license.json
GOVOPLAN_LICENSE_ENFORCEMENT=true
GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE=/srv/govoplan/trust/license-keyring.json
```
License files are JSON objects with:
- `license_id`
- `subject`
- `features`
- `valid_from`
- `valid_until`
- `signature`
Issue or renew a license from an operator/release shell that has the Ed25519
private key:
```bash
govoplan-module-installer \
--issue-license /srv/govoplan/license.json \
--license-id customer-2026-07 \
--license-subject "Example Municipality" \
--license-feature module.mail \
--license-feature support.standard \
--license-valid-until 2027-07-31T23:59:59Z \
--license-signing-key-id license-issuer-1 \
--license-signing-private-key /srv/govoplan/secrets/license-issuer-1.pem \
--format json
```
Validate an imported license without exposing secrets:
```bash
govoplan-module-installer \
--validate-license /srv/govoplan/license.json \
--license-trusted-key license-issuer-1="<base64 public key>" \
--require-trusted-license \
--license-required-feature module.mail \
--format json
```
The CLI and admin module catalog panel report the license id, subject,
validity window, signing key id, signed/trusted state, available features, and
missing entitlements for the configured package catalog. They do not expose
private signing material.
License enforcement can run in observe-only mode by leaving
`GOVOPLAN_LICENSE_ENFORCEMENT` unset. In that mode, missing or invalid license
data is surfaced as a warning but does not block planning.
Renewal is an ordinary re-issuance with a new `license_id`, extended
`valid_until`, and the full intended feature set. Import the renewed JSON to
`GOVOPLAN_LICENSE_FILE`, keep the previous file for audit, and validate it
before setting enforcement.
Revocation is handled through the trusted license keyring. Mark a compromised
or invalid issuer key as `revoked` or `disabled`, publish or deploy the updated
keyring, then reissue affected licenses with an active key. Installations that
run with `GOVOPLAN_LICENSE_ENFORCEMENT=true` reject licenses signed only by a
revoked key after the local keyring is updated.
Emergency fallback is deliberately explicit. Operators can temporarily unset
`GOVOPLAN_LICENSE_ENFORCEMENT` to keep package planning observable while a
license or keyring is recovered. Record the change in the operational incident
log, keep catalog signature and channel enforcement enabled, and restore
license enforcement after a trusted renewal validates successfully.
Licensing is intentionally separate from open-source code licensing. The
catalog/license mechanism can govern support channels, official release
eligibility, hosted update access, professional support, or commercial
entitlements without changing the source license of the repositories.
Production-grade distribution still needs remote registry/git artifact
resolution before package-manager apply, a hardened catalog publishing pipeline
that writes to `addideas-govoplan-website`, and automated key rotation and
emergency revocation drills.
## Release Catalog Publishing
GovOPlaN release catalogs are published to `addideas-govoplan-website` as
static JSON and verified by `govoplan-core` before installer plans are
accepted. Private signing keys must stay outside all git repositories. Public
keyrings are published with the website.
Create the first catalog signing key on the release machine:
```bash
cd /mnt/DATA/git/govoplan
KEY_DIR="$HOME/.config/govoplan/release-keys"
mkdir -p "$KEY_DIR"
./.venv/bin/python tools/release/generate-catalog-keypair.py \
--key-id release-key-1 \
--private-key "$KEY_DIR/release-key-1.pem" \
--public-key "$KEY_DIR/release-key-1.pub" \
--keyring "$KEY_DIR/catalog-keyring.json"
```
Keep `release-key-1.pem` private. The generated keyring contains only public
material.
Generate the signed catalog into `addideas-govoplan-website`:
```bash
cd /mnt/DATA/git/govoplan
KEY_DIR="$HOME/.config/govoplan/release-keys"
tools/release/publish-release-catalog.sh \
--version <x.y.z> \
--sequence 202607071340 \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \
--build-web
```
This writes:
- `/mnt/DATA/git/addideas-govoplan-website/public/catalogs/v1/channels/stable.json`
- `/mnt/DATA/git/addideas-govoplan-website/public/catalogs/v1/keyring.json`
The wrapper validates the catalog with core using the generated public keyring.
For normal module/core releases, first audit and record migration baselines,
then tag and push the module/core repos. Finally publish the website catalog:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python tools/release/release-migration-audit.py --strict
```
```bash
cd /mnt/DATA/git/govoplan
KEY_DIR="$HOME/.config/govoplan/release-keys"
tools/release/publish-release-catalog.sh \
--version <x.y.z> \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \
--build-web \
--commit \
--tag \
--push
```
The website tag is `catalog-v<x.y.z>`. The public URL is:
```text
https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json
```
The public keyring URL is:
```text
https://govoplan.add-ideas.de/catalogs/v1/keyring.json
```
`/mnt/DATA/git/govoplan/tools/release/push-release-tag.sh` can publish the web catalog after module and core
tags have been pushed. It runs the migration release audit in automatic mode:
warning-only before the first recorded migration baseline, strict after a
baseline exists. Add `--strict-migration-audit` when you want to force strict
mode explicitly:
```bash
cd /mnt/DATA/git/govoplan
KEY_DIR="$HOME/.config/govoplan/release-keys"
tools/release/push-release-tag.sh \
--bump subversion \
--strict-migration-audit \
--publish-web-catalog \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \
--build-web-catalog
```
Use `--catalog-signing-key` more than once during a key rotation window. The
catalog will contain multiple signatures and the public keyring will include the
corresponding public keys.
## Release Doctor
Use `/mnt/DATA/git/govoplan/tools/release/release-doctor.py` before release preparation and again before
publishing. It inspects repository state, local versions, release tags,
migration audit state, release package refs, stable catalog/keyring state, and
optionally public catalog availability. The default mode is read-only and
offline. Status output is intentionally concise:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python tools/release/release-doctor.py status --target-version <x.y.z>
```
Use `--details` when the full findings should be printed directly:
```bash
./.venv/bin/python tools/release/release-doctor.py status --target-version <x.y.z> --details
```
For concise guidance, print only suggested next commands:
```bash
./.venv/bin/python tools/release/release-doctor.py next --target-version <x.y.z>
```
For an operator-guided session, run interactive mode. The doctor asks which
suggested command to run; mutating commands require typing `RUN` before they
execute:
```bash
./.venv/bin/python tools/release/release-doctor.py interactive --target-version <x.y.z>
```
Interactive mode starts with a compact summary and waits for an action. It can
open the full report in the configured pager, run one suggested command, repair
Git `safe.directory` dubious-ownership blocks after explicit confirmation, push
all clean repositories that are ahead of their upstream, or commit and push all
dirty repositories. The dirty-repository bulk action asks for a commit message,
skips repositories that are behind upstream, and requires typing
`COMMIT AND PUSH` before it stages, commits, and pushes.
Add `--online` when remote tag and public catalog/keyring reachability should be
checked. Add `--json` when a CI job or another tool should consume the report.
On a GovOPlaN installation that should consume the official stable catalog:
```bash
GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true
GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true
```
For production, copy the public keyring into deployment configuration and pin it
locally. Do not rely on a URL-fetched keyring as the only trust root.
`stable.json` includes a top-level `core_release` section for operator/update
tooling. Core is intentionally not listed as a normal module entry because it
must not be added to saved enabled-module state. Core upgrades should remain an
operator-supervised package update with restart and health checks.
Key rotation for published catalogs:
1. Generate the next private key outside git.
2. Run `/mnt/DATA/git/govoplan/tools/release/publish-release-catalog.sh` with both signing keys.
3. Publish the web catalog/keyring.
4. Roll the new public keyring into installations.
5. Stop signing with the old key after the supported fleet trusts the new key.
6. Mark compromised keys as revoked in the public keyring and publish a higher
sequence catalog signed by a trusted uncompromised key.
## PostgreSQL Release Check
Release candidates should pass a disposable PostgreSQL migration and startup
smoke check before tagging or publishing catalogs. Start the local testbed,
then run the permutation check from the core checkout:
```bash
cd /mnt/DATA/git/govoplan/dev/postgres
cp .env.example .env
docker compose --env-file .env up -d
cd /mnt/DATA/git/govoplan
set -a
. /mnt/DATA/git/govoplan/dev/postgres/.env
set +a
tools/checks/postgres-integration-check.py \
--database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \
--reset-schema
```
The script checks migrations and `/health` startup for core-only, files-only,
mail-only, campaign-only, campaign+files, campaign+mail, and full-product
module sets. `--reset-schema` is destructive and must only be used against a
throwaway database.
## Migration Baselines
Development migrations may be small and numerous while a feature is moving.
GovOPlaN keeps those detailed migrations on an explicit development track and
publishes reviewed release shortcuts on the release track. Before a stable
release, unreleased development migrations may be squashed into a release-level
baseline or release-to-release upgrade migration. After a release tag has
shipped, released migration revision IDs are immutable.
The release policy is:
- unreleased development migrations live in `dev_versions` and are not deleted
when a release shortcut is added;
- released migrations live in `versions` and are never rewritten or deleted;
- future release shortcuts are additive. A new release-to-release migration
starts from the previous recorded release heads, so installations can upgrade
through sane release steps instead of replaying every development revision;
- each stable release records the public migration head revisions in
`docs/migration-release-baselines.json`;
- `heads` records the Alembic dependency-leaf graph heads for a full graph,
while `owner_heads` records the latest migration revision per migration
owner for subset installs and review;
- fresh installations should apply release-level baselines/upgrades, not
unreleased create-then-rename churn;
- release-to-release schema changes should be folded into one reviewed
migration per migration owner where practical.
The default runtime track is `release`:
```bash
cd /mnt/DATA/git/govoplan
GOVOPLAN_MIGRATION_TRACK=release ./.venv/bin/python -m govoplan_core.commands.init_db
```
Use `dev` only for local/disposable development databases that intentionally
need the detailed chain:
```bash
cd /mnt/DATA/git/govoplan
GOVOPLAN_MIGRATION_TRACK=dev ./.venv/bin/python -m govoplan_core.commands.init_db
```
Production, release checks, operator install flows, and the normal development
launcher should stay on the `release` track unless a fresh/disposable database
is intentionally being used to test the detailed development chain.
Audit the current graph during release preparation:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python tools/release/release-migration-audit.py
```
Audit the detailed development graph separately when a squash is prepared:
```bash
./.venv/bin/python tools/release/release-migration-audit.py --track dev
```
Generate the reviewed/manual squash checklist:
```bash
./.venv/bin/python tools/release/release-migration-audit.py --squash-plan
```
After the release migrations have been reviewed and the graph is final, record
the release baseline:
```bash
./.venv/bin/python tools/release/release-migration-audit.py --record-release <x.y.z>
```
Use strict mode to verify that the current heads are recorded:
```bash
./.venv/bin/python tools/release/release-migration-audit.py --strict
```
`/mnt/DATA/git/govoplan/tools/release/push-release-tag.sh` runs the audit by default in automatic mode:
non-strict while no release baseline exists, strict after the first baseline is
recorded. Pass `--warn-migration-audit` for an explicit non-strict audit,
`--strict-migration-audit` to force strict mode, or `--skip-migration-audit`
only for emergency/manual release work.
The first public baseline is v0.1.7. It intentionally adds release-track
shortcuts for the unreleased v0.0.0 -> v0.1.7 development chains while keeping
the detailed chains on the `dev` track. No production installations existed
before that baseline, so pre-v0.1.7 development revisions are not release
upgrade targets. Future release-to-release changes must start from a recorded
release baseline and add a new release-track step-up instead of replacing prior
release shortcuts. The tracking issue is
`add-ideas/govoplan-core#223`.
## Related Operator Documents
- `DEPLOYMENT_OPERATOR_GUIDE.md`: runtime environment, explicit migrations,
backup/restore commands, module installer daemon/supervisor operation, and
rollback drills.
- `REMOTE_WEBUI_BUNDLES.md`: experimental browser-loaded module bundles for
controlled deployments; normal releases use package builds.
## Release Checklist
- Keep Python package versions, WebUI package versions, and git tags aligned.
- Tag core, files, mail, and campaign repositories together.
- Update `requirements-release.txt` and `webui/package.release.json` when the release tag changes.
- Generate the committed full-product release lockfile from `package.release.json` with `scripts/generate-release-lock.sh`.
- Add separate release manifest/lockfile pairs only for module compositions that are shipped as their own products.
- Tag core, access, admin, tenancy, policy, audit, files, mail, campaign,
calendar, and scaffold module repositories together.
- Update meta `requirements-release.txt` and core `webui/package.release.json` when the
release tag changes.
- Generate the committed full-product release lockfile from
`package.release.json` with `/mnt/DATA/git/govoplan/tools/release/generate-release-lock.sh`.
- Run `/mnt/DATA/git/govoplan/tools/release/release-migration-audit.py --strict` after recording a release
baseline.
- Run the PostgreSQL release check against a disposable database.
- Publish the signed catalog through the release catalog publishing flow above.
- Add separate release manifest/lockfile pairs only for module compositions
that are shipped as their own products.
- Do not commit local sibling paths into release manifests.

View File

@@ -0,0 +1,161 @@
# Remote WebUI Bundle Loading
GovOPlaN WebUI modules normally ship through the core WebUI package graph:
install a tagged npm/git dependency, run `npm install`, rebuild the shell, and
restart or reload the served assets. Remote WebUI bundles are an experimental
future path for controlled deployments where a backend-enabled module is not in
the local WebUI package graph and the operator wants the shell to load its
frontend without rebuilding core.
This design defines the target guardrails. The current rebuild/reload path
remains the default production path.
## Current Rebuild Path
The supported release path is:
1. Install or remove backend and WebUI package dependencies through the trusted
installer CLI/daemon.
2. Snapshot package state before mutation.
3. Run `npm install` and optionally `npm run build`.
4. Restart or reload the served WebUI assets.
5. Use installer rollback if package apply, restart, or health checks fail.
Strengths:
- package manager and lockfile semantics stay conventional
- CSP can stay strict because all code is served as built assets
- rollback restores package files and lockfiles
- local development uses sibling workspace dependencies naturally
Costs:
- frontend changes require a rebuild/reload
- hot enabling a module with frontend code is not possible in the running shell
- failed rebuilds happen at install time, not at lazy module-load time
## Remote Bundle Path
Remote loading is only for modules that are enabled by the backend and absent
from `virtual:govoplan-installed-modules`. The backend manifest exposes:
- `FrontendModule.asset_manifest`
- `asset_manifest_integrity`
- `asset_manifest_signature`
- `asset_manifest_public_key_id`
- `asset_manifest_contract_version`
The WebUI shell fetches the asset manifest, verifies manifest integrity and/or
signature, validates the manifest contract, fetches the entry bundle, verifies
the entry integrity, imports the bundle, validates the exported
`PlatformWebModule`, and applies backend metadata before registering routes,
navigation, and UI capabilities.
Unsigned and unhashed manifests are skipped. Entries without integrity are
skipped. A module id mismatch is skipped.
## Asset Manifest Contract
Contract version `1`:
```json
{
"contractVersion": "1",
"moduleId": "files",
"entry": "./files-webui.remote.js",
"entryIntegrity": "SHA-256-<base64-or-hex-digest>",
"moduleExport": "default"
}
```
The backend manifest carries the manifest-level trust metadata. The remote
asset manifest carries the concrete entry URL and entry digest.
## Compatibility Checks
Before a remote bundle is accepted:
- backend module must be enabled
- local WebUI module with the same id must be absent
- manifest contract must be supported by the shell
- manifest `moduleId`, exported module `id`, and backend module id must match
- backend metadata remains authoritative for label, version, dependencies,
nav, and runtime UI capability exposure
- future contract versions must declare required shell capabilities so old
shells fail closed
Remote bundles must not broaden backend permissions. They can only contribute
routes/nav/capabilities that the backend metadata and existing permission
checks allow.
## CSP
The current implementation imports verified entry bytes through a blob URL. A
production CSP for this mode must explicitly allow:
- `connect-src` for the approved asset-manifest and bundle origins
- `script-src` for `blob:` only when remote loading is enabled
- no `unsafe-inline` requirement for remote modules
If an installation cannot allow `blob:` scripts, the follow-up implementation
should use signed same-origin module assets with static URLs instead of blob
imports. Remote loading must be disableable by configuration so strict-CSP
deployments can keep the rebuild path only.
## Cache Invalidation
The shell cache key is:
```text
<module-id>:<asset-manifest-url>:<backend-module-version>
```
Release catalogs should publish immutable manifest and entry URLs or change at
least one cache-key component on every release. Emergency rollback can point the
backend manifest at a previous immutable manifest URL or lower the enabled
module version after the backend package rollback.
Browsers may still cache remote responses. Approved asset servers should use:
- long cache lifetimes only for content-addressed immutable assets
- short cache lifetimes or explicit revalidation for channel/latest manifest
aliases
- `Cache-Control: no-store` for emergency override manifests
## Rollback
Remote WebUI rollback is metadata rollback, not package-manager rollback:
- if the backend package install rolls back, the previous backend frontend
metadata returns
- if only remote assets are bad, publish a new manifest URL or revert the
backend/frontend metadata to the last known good manifest
- failed remote loading must degrade by omitting the remote module frontend,
not by breaking the shell
Installer run records should eventually include remote asset manifest URLs,
integrity, signature key ids, and load-test results when a plan enables a
remote frontend.
## Local And Development Behavior
Local development should keep using workspace/file dependencies and Vite. Remote
loading is useful for integration testing release artifacts, not for day-to-day
module UI development.
Development deployments may use unsigned catalogs and local asset servers only
when signature/integrity enforcement is intentionally disabled. The remote
loader itself still requires an integrity hash or a verifiable signature.
## Follow-Up Slices
1. Add a server-side remote-bundle policy flag and surface whether remote
loading is enabled in platform metadata.
2. Add CSP documentation/config generation for strict rebuild-only mode versus
controlled remote-bundle mode.
3. Add an installer/catalog preflight that validates remote WebUI asset
manifests and records verified identity in installer run records.
4. Add Playwright coverage for a signed test remote bundle, failed digest, bad
module id, cache-key refresh, and fallback when the module is unavailable.
5. Add release tooling to emit immutable remote asset manifests with digest,
signature, key id, and rollback metadata.

15
docs/SECURITY_AUDIT.md Normal file
View File

@@ -0,0 +1,15 @@
# Security Audit Toolchain
The shared GovOPlaN security audit toolbox moved to the meta repository.
Use:
```bash
cd /mnt/DATA/git/govoplan
tools/checks/security-audit/run.sh --mode ci --scope govoplan
tools/checks/security-audit/run.sh --mode full --scope govoplan
```
Canonical documentation:
- `/mnt/DATA/git/govoplan/docs/SECURITY_AUDIT.md`

View File

@@ -0,0 +1,80 @@
# Self-Hosted Installability
GovOPlaN uses a staged self-hosted installability path.
## Packaging Decision
The early packaging approach is a staged combination:
1. Generate an explicit environment template and validate it before startup.
2. Use a Compose-backed production-like development profile for local rehearsal.
3. Use the deployment operator guide as the runbook for migrations, workers,
backups, health checks, and module installer rollback drills.
4. Use the module installer CLI/daemon for package mutation once the runtime is
already installed and under maintenance mode.
This keeps first installation understandable while still preserving the later
goal of install/update/uninstall through signed catalogs and the installer
daemon. The API server must not run package managers from request handlers.
## Config Bootstrap
Generate a self-hosted template:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python -m govoplan_core.commands.config env-template \
--profile self-hosted \
--generate-secrets \
--output .env.self-hosted
```
Validate the current shell environment:
```bash
set -a
. .env.self-hosted
set +a
./.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted
```
The command reports all known blockers at once. Production-like/self-hosted
profiles require explicit `APP_ENV`, `DATABASE_URL`, `MASTER_KEY_B64`,
`ENABLED_MODULES`, and `CORS_ORIGINS`. Production rejects SQLite, development
bootstrap, insecure auth cookies, and unsigned catalog trust roots when a
catalog source is configured.
## Production-Like Dev Stack
Use the local production-like wrapper for repeatable rehearsal:
```bash
cd /mnt/DATA/git/govoplan
tools/launch/production-like-dev.sh validate-config
tools/launch/production-like-dev.sh seed
tools/launch/production-like-dev.sh start
```
Stop Docker dependencies:
```bash
tools/launch/production-like-dev.sh stop
```
Reset all profile data:
```bash
tools/launch/production-like-dev.sh reset --yes
```
The start command delegates to `tools/launch/launch-production-like-dev.sh`, which
runs API, worker, and WebUI in the foreground. Stop those processes with
`Ctrl+C` in the launcher terminal.
## Module Boundary Gate
`govoplan/tools/checks/check_dependency_boundaries.py` is part of the focused verification
path. It checks backend imports and WebUI package/source imports so modules do
not grow hidden runtime dependencies on each other. Feature modules should
integrate through core capabilities, backend APIs/events, route contributions,
or explicit UI extension points.

View File

@@ -0,0 +1,234 @@
# GovOPlaN UI/UX Decision Ledger
This ledger records product UI/UX decisions that affect admin, settings,
configuration, connector, policy, and module-management surfaces. It is a
binding design reference: future implementation should follow these decisions
unless the decision is explicitly revised here and affected screens are updated
to match.
Active tracking issue: `add-ideas/govoplan-core#225`.
## Operating Rule
GovOPlaN must expose advanced platform capability without presenting the user
with every option at once. Non-technical users should be able to complete common
workflows through guided, plain-language flows. Expert and diagnostic detail may
exist, but it must be deliberately layered.
The ethical design doctrine in `INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` is the
normative baseline for this ledger. A screen can be visually quiet and still be
ethically complete only when it preserves context, consequence,
contestability, responsibility, and traceability at the point of action.
## Binding Decisions
| ID | Decision | Status | Applies To |
| --- | --- | --- | --- |
| UX-001 | Use progressive disclosure by default. Common decisions stay visible; advanced or hazardous options live in collapsed panels, later wizard steps, or explicit advanced sections. | Accepted | Admin, settings, connector setup, policy editors, module operations |
| UX-002 | Raw JSON is not a primary configuration editor. Every normal configuration path needs typed controls, validation, and help text. JSON may be shown for import/export, diagnostics, or expert inspection only. | Accepted | All admin/configuration UIs |
| UX-003 | Prefer guided workflows over option dumps for setup and risky changes. Use wizards for connector setup, configuration package import, module install/uninstall, destructive actions, and policy changes with broad impact. | Accepted | Connectors, package import, module lifecycle, governance/policy |
| UX-004 | Discover technical values when possible. Ask for the smallest user-known input, then discover and prefill technical fields for review. | Accepted | File connectors, mail/groupware, public URLs, future external providers |
| UX-005 | Disabled actions and failed steps must explain why they are unavailable, who can fix them, and where to go next. Silent disabled states are not acceptable for primary actions. | Accepted | All primary actions |
| UX-006 | Explanations must be available but quiet. Use short inline text and expose richer explanations through help affordances, side panels, expandable sections, or review steps. | Accepted | All complex forms and flows |
| UX-007 | Creation/editing should prefer modals or focused step flows when it reduces page clutter. Overview and comparison screens remain full-page. | Accepted | Settings/admin surfaces |
| UX-008 | Similar concepts must use shared placement and components: server/credential/policy rows, problem lists, review steps, advanced panels, confirmation modals, and empty/error states. | Accepted | Core WebUI and module WebUIs |
| UX-009 | Preflight and diagnostics are product UX. Validation, policy, permission, dependency, and capability failures must be written for operators before exposing internal details. | Accepted | Installer, connectors, policy, package import |
| UX-010 | Context, decision, and consequence must be visible together for actions that affect rights, duties, records, money, communication, retention, external systems, or workflow state. | Accepted | Workflow, admin, portal, policy, connector, records, payments |
| UX-011 | Navigation is not consent. Route changes, panel switches, and passive selection must not execute consequential actions without an explicit action surface. | Accepted | All WebUI surfaces |
| UX-012 | Automated actions must remain inspectable. The UI must show the system actor, trigger, policy result, observed effects, and failure/manual-intervention state when automation changes administrative state. | Accepted | Workflow, automation, connectors, tasks, audit |
| UX-013 | Contestable decisions must expose provenance. Denials, locks, generated outputs, calculated defaults, policy decisions, access decisions, and retention decisions need a reachable source path. | Accepted | Policy, access, templates, workflow, retention, records |
| UX-014 | Retraction, expiry, undo, rollback, and delete controls must state the real limit of the operation. Corrective or future-only actions must not be described as if they undo already observed effects. | Accepted | Postbox, files, records, installer, workflow, payments |
| UX-015 | Core owns the platform appearance contract. Modules must use shared CSS tokens and shared controls for theme-aware UI; they must not define independent light/dark palette systems. | Accepted | Core shell and all module WebUIs |
## Confirmed Implementation Decisions
These decisions were accepted on 2026-07-09 for the first implementation slice.
They shape reusable components and screen structure. Future changes must revise
this section and update affected screens.
### DUE-001: Primary Admin Configuration Shell
Decision: admin/configuration surfaces should standardize on a two-zone layout.
- Left or top area: searchable overview/list, status, and primary actions.
- Main area: selected item summary and common settings.
- Modal/wizard: create, connect, edit, test, review, and confirm actions.
- Collapsed advanced panels: rarely used technical fields.
Use this for file connectors and mail servers first, then migrate policy,
retention, API keys, and module operations.
### DUE-002: Wizard Step Model
Decision: standard wizard steps and names use this baseline:
1. Choose type or scope.
2. Enter essentials.
3. Discover or test.
4. Configure ownership and policy.
5. Review changes and blockers.
6. Save or submit for operator action.
Not every wizard needs every step, but flows should use these names and order
where applicable.
This applies to explicit assisted setup, onboarding, import, preflight, and
risky multi-step operations. It is not the default shape for ordinary create or
edit dialogs.
### DUE-003: Explanation Placement
Decision: richer explanations should use this placement model:
- Short helper text below labels only when it prevents common mistakes.
- Tooltips for icon-only controls and compact terms.
- Expandable "Why?" or "Details" blocks for contextual explanations.
- Right-side detail panel or review step for preflight/provenance/diagnostics.
Avoid permanently visible paragraphs inside dense admin cards.
### DUE-004: Advanced Options Contract
Decision: advanced options are fields that are needed for control, compatibility,
or diagnostics but are not part of the common setup path.
- protocol-specific endpoints, ports, path overrides, TLS/signing toggles,
timeout/retry tuning, raw headers, migration/destructive flags, and fallback
compatibility settings are advanced.
- names, descriptions, provider type, base URL, ownership, policy mode, and
basic credentials are not advanced.
Advanced fields must still be editable through typed controls.
### DUE-005: Blocker Language Contract
Decision: all disabled or blocked primary actions should use a shared structured
reason object.
Shape:
- `summary`: short plain-language reason.
- `details`: optional explanation.
- `required_action`: what needs to happen.
- `actor`: who can do it, for example system administrator or tenant admin.
- `target`: where to go or which setting/capability is missing.
- `technical_details`: optional expandable developer/operator data.
For simple missing required fields, highlight the fields and put the structured
reason in a compact hover/focus bubble on the disabled primary action instead of
adding a large persistent warning block.
### DUE-006: First Migration Surface
Decision: convert file connectors first, then mail servers. They share the same
server/credential/policy model, are high-value, and will prove the reusable
patterns quickly.
### DUE-007: Adaptive Create/Edit Forms
Decision: ordinary create/edit dialogs should show the full editable state in an
adaptive form, not force a linear wizard.
- The user chooses a type, provider, credential mode, or policy mode.
- The dialog immediately adjusts to show only the fields relevant to that state.
- Editing an existing object uses the same field grouping and layout as creating
it, so users can recognize the state they configured.
- Discovery and test actions must give visible feedback in the same dialog:
directly usable, discovered alternative, credentials needed/rejected, or not
found with the next action the user can take.
- Wizard shells remain available for assisted setup, first-run guidance,
imports, discovery-heavy flows, and operational preflight workflows.
### DUE-008: Platform Theme Contract
Decision: the WebUI shell exposes a small, stable appearance contract based on
shared CSS tokens and persisted user preference selection.
- Core applies `system`, `light`, and `dark` preferences at the document root.
- Core owns shared tokens such as `--bg`, `--bar`, `--panel`, `--surface`,
`--line`, `--line-dark`, `--text`, `--text-strong`, `--muted`, semantic
status colors, radii, shadows, and disabled-control colors.
- Modules must style new UI with these tokens and shared controls. Module-local
CSS may tune layout and spacing, but it must not introduce a separate
appearance system.
- Appearance controls live in user settings first. Tenant defaults and policy
enforcement can be added later without changing the token contract.
- Visual preview in settings is illustrative; it must reflect token families,
not become a second theme implementation.
## Implementation Sequence
| Phase | Scope | Output |
| --- | --- | --- |
| 0 | UX inventory | List every admin/settings/configuration surface, classify it, and record whether it violates a binding decision. |
| 1 | Core primitives | Shared wizard shell, advanced panel, help affordance, blocker callout, problem list, review step, and discovery/test result components. |
| 2 | File connectors | Adaptive create/edit for provider/server, credential, discovery/test, ownership/policy, plus optional assisted wizard later. |
| 3 | Mail servers | Same adaptive pattern as files, adapted to server/credential/policy and test-send/test-login behavior. |
| 4 | Policy/retention editors | Effective value first, provenance visible, override/edit in modal, blocked edits explained. |
| 5 | Module/package operations | Step-based install/uninstall flow with preflight, maintenance, daemon handoff, migration, and rollback explanation. |
| 6 | Remaining settings/admin screens | Apply inventory findings by priority and remove one-off layouts. |
## Current Surface Inventory
This is the phase-0 inventory baseline. It should be extended as each screen is
converted or reviewed.
| Surface | Repository | UX State | Next Action |
| --- | --- | --- | --- |
| File connector settings | `govoplan-files` | First adaptive modal slice started: connections and credentials now use full-state create/edit forms with conditional fields, advanced panels, and blocker primitives. Wizard shell is retained for later assisted setup. Central policy card still needs a layered editor. | Finish provider discovery/test-in-flow, then convert policy editing. |
| Mail server settings | `govoplan-mail` / `govoplan-core` | Uses the shared server/credential model visually, but create/edit still needs the same adaptive pattern as files. | Migrate to adaptive server/credential/policy dialogs, with optional assisted wizard later. |
| Connector policy/effective rows | `govoplan-core`, module UIs | Effective-policy direction exists, but many editors still expose broad option sets. | Put effective value first, move overrides into modal, and explain blocked edits. |
| Admin module management | `govoplan-admin` | Has preflight concepts, but operational choices are still technical and dense. | Convert install/uninstall/package changes to operator wizards. |
| Configuration packages | `govoplan-admin` | Catalog/import work exists, but package editing can still drift toward technical fields. | Add guided import/review/problem-list flow. |
| Retention and privacy | `govoplan-core` | Functional editor exists; consequence language and provenance can be stronger. | Layer advanced retention options and add review for broad changes. |
| API keys | `govoplan-access` / admin UI | Security-sensitive creation needs least-privilege guidance. | Add scoped creation wizard with expiry/owner review. |
| User settings | `govoplan-core` | Preferences persistence exists; interface navigation issue was fixed earlier, but the surface still needs UX review. | Keep simple sections, remove double-click traps, and add quiet explanations. |
## Impact Index
| Surface | Current Risk | Expected Pattern |
| --- | --- | --- |
| File connectors | Too many technical fields and unclear setup order. | Adaptive create/edit form with optional assisted wizard, discovery/test, credential binding, policy review, and advanced protocol panel. |
| Mail servers | Similar server/credential/policy concepts risk diverging from files. | Same tree/list and adaptive server/credential/policy model as file connectors. |
| Connector credentials | Security-sensitive details can overwhelm users. | Separate credential flow with secret-reference language and test result explanation. |
| Policy and effective settings | Users need to know why a value is inherited or locked. | Effective row first, source/provenance, local override action, actionable blocked reason. |
| Module install/uninstall | Operationally risky, currently inherently technical. | Operator wizard with preflight, maintenance, daemon handoff, review, and rollback explanation. |
| Configuration packages | Could become package JSON editing. | Package catalog/import wizard using provider data requirements and problem lists. |
| Retention/privacy | High-risk settings need explanation and provenance. | Layered editor with plain-language consequences and review. |
| Automation/workflow commands | Hidden side effects would undermine accountability. | Action/effect preview, system-actor display, command record, retry/quarantine/manual states, and audit links. |
| Postbox and encrypted communication | Retraction and access can be misunderstood. | Honest key-fetch/decryption state, expiry limits, recipient/device access provenance, and delivery evidence. |
| API keys | Security-sensitive creation and scope selection. | Scoped creation wizard, least-privilege suggestions, clear expiry/owner explanation. |
| User settings | Needs clarity and persistence across profile/interface/preferences. | Simple settings sections with immediate feedback and no double-click navigation traps. |
## Review Checklist
Every new or changed admin/configuration surface should answer:
- What is the common path, and is it visible without noise?
- Which fields are advanced, and are they collapsed by default?
- Is every configuration value editable through typed controls?
- Does the flow avoid JSON as the primary editor?
- Does the screen explain disabled actions and failed validation in plain
language?
- Does it say who can fix a blocker and where?
- Does it reuse existing core patterns for wizard steps, problem lists, modals,
help, and review?
- Is there a review or preflight step before broad, destructive, or risky
changes?
- Does the action surface show consequence, reversibility, and audit evidence
when rights, duties, records, money, communication, external systems, or
workflow state are affected?
- If automation is involved, can the user see the trigger, system actor,
observed effects, and failure/manual-intervention state?
- Are technical details available without being the first thing the user sees?
## Revision Rule
When a UX decision changes:
1. Update this ledger.
2. Update the affected shared components.
3. Update existing screens listed in the impact index.
4. Add or update Gitea issues for any remaining surfaces that still follow the
old decision.
5. Sync the wiki.

View File

@@ -0,0 +1,71 @@
# Dependency Audit - 2026-07-09
Commands:
```bash
cd /mnt/DATA/git/govoplan
GOVOPLAN_CORE_ROOT=/mnt/DATA/git/govoplan-core bash tools/checks/check-dependency-audits.sh
```
Status: remediated.
Initial result:
- Python audit failed: 24 advisories were reported across `cryptography`,
`pip`, `python-multipart`, `pyzipper`, and `starlette`.
- npm production audit passed: `npm audit --omit=dev` reported 0
vulnerabilities.
Python findings:
| Package | Installed | Advisory count | Minimum reported fix |
| --- | ---: | ---: | --- |
| `cryptography` | `44.0.0` | 5 | `48.0.1` |
| `pip` | `26.0.1` | 3 | `26.1.2` |
| `python-multipart` | `0.0.17` | 7 | `0.0.31` |
| `pyzipper` | `0.3.6` | 1 | `0.4.0` |
| `starlette` | `0.41.3` | 8 | `1.3.1` |
Private GovOPlaN packages were skipped by `pip-audit` because they are not
published on PyPI. That is expected for local editable development installs.
The remediation below upgrades those dependencies and records the compatibility
checks run against core, files, and campaign behavior.
## Remediation
Remediation applied on 2026-07-09:
- upgraded core's FastAPI floor to `fastapi>=0.139,<1`, resolving Starlette to
`starlette==1.3.1`
- upgraded core's cryptography floor to `cryptography>=48.0.1,<50`, resolving
to `cryptography==49.0.0`
- declared the files module upload parser dependency as
`python-multipart>=0.0.31,<1`, resolving to `python-multipart==0.0.32`
- upgraded the campaign ZIP dependency to `pyzipper>=0.4,<1`, resolving to
`pyzipper==0.4.0`
- upgraded the local audit environment to `pip==26.1.2`
- removed the obsolete local mailer-module editable install
from the audit environment so the audit reflects the split module product
Post-remediation result:
- `bash tools/checks/check-dependency-audits.sh`: passed, no known Python
vulnerabilities found and npm production audit reported 0 vulnerabilities.
- `python -m pip check`: passed.
- `bash tools/checks/check-module-matrix.sh`: passed.
- `python -m unittest tests.test_api_smoke`: passed.
- campaign encrypted/plain ZIP smoke with `pyzipper==0.4.0`: passed.
Notes:
- `pip-audit` still reports private GovOPlaN packages as skipped because they
are not published on PyPI. That is expected for editable local development
installs.
- `httpx2>=2.5,<3` is included in development requirements so Starlette's
`TestClient` uses the non-deprecated backend. `httpx==0.28.1` remains in dev
requirements for tests that mock connector HTTP responses directly.
- Split-module API routers now use Starlette's renamed
`HTTP_422_UNPROCESSABLE_CONTENT` status constant. This preserves the 422
status code while avoiding the deprecated `HTTP_422_UNPROCESSABLE_ENTITY`
alias.

View File

@@ -0,0 +1,46 @@
{
"channel": "stable",
"packages": [
{
"package_id": "govoplan.application-handling.basic",
"name": "Basic application handling",
"version": "0.1.0",
"description": "Portal form, case workflow, task creation, mail notification, payment setup, and access roles for a simple application process.",
"publisher": "ADD ideas",
"category": "workflow",
"artifact_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-configuration-packages.git@application-handling-basic-v0.1.0",
"artifact_sha256": "<sha256>",
"required_modules": [
{ "module_id": "portal", "version": ">=0.1.0" },
{ "module_id": "forms", "version": ">=0.1.0" },
{ "module_id": "cases", "version": ">=0.1.0" },
{ "module_id": "workflow", "version": ">=0.1.0" },
{ "module_id": "tasks", "version": ">=0.1.0" },
{ "module_id": "templates", "version": ">=0.1.0" },
{ "module_id": "mail", "version": ">=0.1.0" },
{ "module_id": "payments", "version": ">=0.1.0" },
{ "module_id": "access", "version": ">=0.1.0" },
{ "module_id": "audit", "version": ">=0.1.0" }
],
"required_capabilities": [
"configuration.provider",
"access.configuration",
"forms.configuration",
"workflow.configuration",
"mail.configuration",
"payments.configuration"
],
"tags": ["official", "example", "public-service"],
"signature": {
"algorithm": "ed25519",
"key_id": "configuration-release-key-1",
"value": "<base64 signature over the package artifact metadata>"
}
}
],
"signature": {
"algorithm": "ed25519",
"key_id": "configuration-catalog-key-1",
"value": "<base64 signature over this JSON object without the signature field>"
}
}

View File

@@ -0,0 +1,89 @@
{
"releases": [
{
"heads": [
{
"owner": "govoplan-campaign",
"revision": "2c3d4e5f7081"
},
{
"owner": "govoplan-mail",
"revision": "3d4e5f708192"
},
{
"owner": "govoplan-idm",
"revision": "8f9a0b1c2d3e"
},
{
"owner": "govoplan-calendar",
"revision": "9e0f1a2b3c4d"
},
{
"owner": "govoplan-files",
"revision": "a7b8c9d0e1f3"
}
],
"owner_heads": [
{
"owner": "govoplan-access",
"revisions": [
"4a5b6c7d8e9f"
]
},
{
"owner": "govoplan-calendar",
"revisions": [
"9e0f1a2b3c4d"
]
},
{
"owner": "govoplan-campaign",
"revisions": [
"2c3d4e5f7081"
]
},
{
"owner": "govoplan-core",
"revisions": [
"4f2a9c8e7b6d"
]
},
{
"owner": "govoplan-files",
"revisions": [
"a7b8c9d0e1f3"
]
},
{
"owner": "govoplan-identity",
"revisions": [
"5c6d7e8f9a10"
]
},
{
"owner": "govoplan-idm",
"revisions": [
"8f9a0b1c2d3e"
]
},
{
"owner": "govoplan-mail",
"revisions": [
"3d4e5f708192"
]
},
{
"owner": "govoplan-organizations",
"revisions": [
"6d7e8f9a0b1c"
]
}
],
"recorded_at": "2026-07-11T01:39:45Z",
"release": "0.1.7",
"track": "release",
"squash_policy": "reviewed-manual"
}
],
"version": 1
}

View File

@@ -0,0 +1,151 @@
{
"catalog_version": "1",
"channel": "stable",
"sequence": 14,
"generated_at": "2026-07-07T00:00:00Z",
"expires_at": "2030-12-31T23:59:59Z",
"modules": [
{
"module_id": "files",
"name": "Files",
"description": "Managed file spaces and campaign attachment integration.",
"version": "0.1.4",
"action": "install",
"python_package": "govoplan-files",
"python_ref": "govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.4",
"webui_package": "@govoplan/files-webui",
"webui_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git#v0.1.4",
"migration_safety": "forward_only",
"migration_notes": "Database rollback requires restoring the pre-update snapshot.",
"migration_after": ["access"],
"migration_before": ["campaigns"],
"migration_tasks": [
{
"task_id": "backfill_spaces",
"phase": "post_migration_backfill",
"summary": "Backfill file spaces after the schema migration.",
"task_version": "1",
"safety": "forward_only",
"idempotent": true,
"timeout_seconds": 300
}
],
"current_version_min": "0.1.0",
"current_version_max_exclusive": "0.2.0",
"bridge_release": true,
"bridge_notes": "Keeps the 0.1.x campaign attachment interface while introducing the 0.2.x contract.",
"allow_downgrade": false,
"allow_same_version": false,
"recovery_tested": true,
"recovery_notes": "Snapshot restore and forward recovery were rehearsed on the release candidate dataset.",
"provides_interfaces": [
{
"name": "files.campaign_attachments",
"version": "1.4.0"
}
],
"requires_interfaces": [
{
"name": "campaigns.access",
"version_min": "1.0.0",
"version_max_exclusive": "2.0.0",
"optional": true
}
],
"artifact_integrity": {
"python": {
"ref": "govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.4",
"sha256": "<sha256 of the resolved Python artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-files-0.1.4.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-files-0.1.4.intoto.jsonl",
"git_ref": "refs/tags/v0.1.4"
},
"webui": {
"ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git#v0.1.4",
"sha256": "<sha256 of the resolved WebUI package artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-files-webui-0.1.4.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-files-webui-0.1.4.intoto.jsonl",
"git_ref": "refs/tags/v0.1.4"
}
},
"license_features": ["module.files"],
"tags": ["official", "service-module"]
},
{
"module_id": "mail",
"name": "Mail",
"description": "SMTP/IMAP profile management and read-only mailbox access.",
"version": "0.1.4",
"action": "install",
"python_package": "govoplan-mail",
"python_ref": "govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.4",
"webui_package": "@govoplan/mail-webui",
"webui_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git#v0.1.4",
"provides_interfaces": [
{
"name": "mail.campaign_delivery",
"version": "1.4.0"
}
],
"artifact_integrity": {
"python": {
"ref": "govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.4",
"sha256": "<sha256 of the resolved Python artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-mail-0.1.4.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-mail-0.1.4.intoto.jsonl",
"git_ref": "refs/tags/v0.1.4"
},
"webui": {
"ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git#v0.1.4",
"sha256": "<sha256 of the resolved WebUI package artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-mail-webui-0.1.4.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-mail-webui-0.1.4.intoto.jsonl",
"git_ref": "refs/tags/v0.1.4"
}
},
"license_features": ["module.mail"],
"tags": ["official", "service-module"]
},
{
"module_id": "dashboard",
"name": "Dashboard",
"description": "Configurable user home assembled from module-provided widgets.",
"version": "0.1.6",
"action": "install",
"python_package": "govoplan-dashboard",
"python_ref": "govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.6",
"webui_package": "@govoplan/dashboard-webui",
"webui_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git#v0.1.6",
"artifact_integrity": {
"python": {
"ref": "govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.6",
"sha256": "<sha256 of the resolved Python artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-dashboard-0.1.6.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-dashboard-0.1.6.intoto.jsonl",
"git_ref": "refs/tags/v0.1.6"
},
"webui": {
"ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git#v0.1.6",
"sha256": "<sha256 of the resolved WebUI package artifact>",
"sbom_url": "https://govoplan.add-ideas.de/sbom/govoplan-dashboard-webui-0.1.6.spdx.json",
"provenance_url": "https://govoplan.add-ideas.de/provenance/govoplan-dashboard-webui-0.1.6.intoto.jsonl",
"git_ref": "refs/tags/v0.1.6"
}
},
"license_features": ["module.dashboard"],
"tags": ["official", "platform-module"]
}
],
"signatures": [
{
"algorithm": "ed25519",
"key_id": "release-key-1",
"value": "<base64 signature over this JSON object without signature or signatures fields>"
},
{
"algorithm": "ed25519",
"key_id": "release-key-2",
"value": "<optional second signature for key rotation>"
}
]
}

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan-core"
version = "0.1.4"
version = "0.1.8"
description = "Reusable GovOPlaN platform core, access, tenancy, and RBAC components."
readme = "README.md"
requires-python = ">=3.12"
@@ -12,10 +12,10 @@ license = { file = "LICENSE" }
authors = [{ name = "GovOPlaN" }]
dependencies = [
"SQLAlchemy>=2.0,<3",
"fastapi>=0.115,<1",
"fastapi>=0.139,<1",
"pydantic>=2,<3",
"pydantic-settings>=2,<3",
"cryptography>=44,<45",
"cryptography>=48.0.1,<50",
"celery>=5,<6",
"redis>=5,<6",
"alembic>=1,<2",
@@ -28,15 +28,17 @@ where = ["src"]
govoplan_core = ["py.typed"]
[project.scripts]
govoplan-config = "govoplan_core.commands.config:main"
govoplan-devserver = "govoplan_core.devserver:main"
[project.entry-points."govoplan.modules"]
access = "govoplan_core.access.manifest:get_manifest"
govoplan-module-install-plan = "govoplan_core.commands.module_install_plan:main"
govoplan-module-installer = "govoplan_core.commands.module_installer:main"
[project.optional-dependencies]
server = [
"psycopg[binary]>=3.2,<4",
"uvicorn[standard]>=0.32,<1",
]
dev = [
"httpx==0.28.1",
"httpx2>=2.5,<3",
]

View File

@@ -1,5 +0,0 @@
-e .[server]
-e ../govoplan-files
-e ../govoplan-mail
-e ../govoplan-campaign
httpx==0.28.1

View File

@@ -1,6 +0,0 @@
# Release install from tagged module repositories.
# Update GOVOPLAN_RELEASE_TAG together with pyproject/package versions when cutting a release.
.[server]
govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.1
govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.1
govoplan-campaign @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-campaign.git@v0.1.1

View File

@@ -1,60 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="/mnt/DATA/git/govoplan-core"
NODE="/home/zemion/.nvm/versions/node/v22.22.3/bin"
NPM="$NODE/npm"
WEBUI_BIN="$ROOT/webui/node_modules/.bin"
export PATH="$WEBUI_BIN:$NODE:$PATH"
cd "$ROOT"
./.venv/bin/python - <<'PY'
import ast
import pathlib
import sys
roots = [
pathlib.Path("/mnt/DATA/git/govoplan-core/src"),
pathlib.Path("/mnt/DATA/git/govoplan-mail/src"),
pathlib.Path("/mnt/DATA/git/govoplan-files/src"),
pathlib.Path("/mnt/DATA/git/govoplan-campaign/src"),
pathlib.Path("/mnt/DATA/git/govoplan-mail/tests"),
]
errors = []
count = 0
for root in roots:
if not root.exists():
continue
for path in root.rglob("*.py"):
count += 1
try:
ast.parse(path.read_text(), filename=str(path))
except SyntaxError as exc:
errors.append(f"{path}:{exc.lineno}:{exc.offset}: {exc.msg}")
if errors:
print("\n".join(errors))
sys.exit(1)
print(f"AST syntax check passed for {count} Python files")
PY
./.venv/bin/python -c 'import govoplan_core.db.bootstrap; import govoplan_core.admin.service; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")'
./.venv/bin/python -m unittest tests.test_module_system
./.venv/bin/python -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
./.venv/bin/python -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
cd "$ROOT/webui"
"$NPM" run test:mail-components
"$NPM" run test:module-capabilities
"$NPM" run test:module-permutations
cd /mnt/DATA/git/govoplan-mail/webui
"$NPM" run test:mail-ui
cd /mnt/DATA/git/govoplan-campaign/webui
"$NPM" run test:policy-ui
"$NPM" run test:template-preview

View File

@@ -1,81 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'USAGE'
Usage:
scripts/generate-release-lock.sh [options]
Generates webui/package-lock.release.json from webui/package.release.json in a
temporary workspace. The normal development package.json and package-lock.json
are left untouched.
Run this after the module git tags referenced by package.release.json exist and
are reachable, and before tagging the core release commit that should contain
the regenerated release lockfile.
Options:
--npm <path> npm executable to use.
-h, --help Show this help.
USAGE
}
fail() {
echo "error: $*" >&2
exit 1
}
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
WEBUI="$ROOT/webui"
NPM_BIN="${NPM:-}"
if [[ -z "$NPM_BIN" ]]; then
if [[ -x "/home/zemion/.nvm/versions/node/v22.22.3/bin/npm" ]]; then
NPM_BIN="/home/zemion/.nvm/versions/node/v22.22.3/bin/npm"
else
NPM_BIN="npm"
fi
fi
while [[ $# -gt 0 ]]; do
case "$1" in
--npm)
[[ $# -ge 2 ]] || fail "missing value for $1"
NPM_BIN="$2"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
echo "Unknown argument: $1" >&2
usage >&2
exit 2
;;
esac
done
[[ -f "$WEBUI/package.release.json" ]] || fail "missing $WEBUI/package.release.json"
command -v "$NPM_BIN" >/dev/null 2>&1 || fail "npm executable not found: $NPM_BIN"
TMP_DIR="$(mktemp -d "${TMPDIR:-/tmp}/govoplan-release-lock.XXXXXXXX")"
cleanup() {
rm -rf "$TMP_DIR"
}
trap cleanup EXIT
cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"
if [[ -f "$WEBUI/package-lock.release.json" ]]; then
cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"
fi
echo "Generating release lockfile from $WEBUI/package.release.json"
echo "Temporary workspace: $TMP_DIR"
(
cd "$TMP_DIR"
PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts
)
cp "$TMP_DIR/package-lock.json" "$WEBUI/package-lock.release.json"
echo "Updated $WEBUI/package-lock.release.json"

View File

@@ -1,136 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="${GOVOPLAN_CORE_ROOT:-/mnt/DATA/git/govoplan-core}"
WEBUI_ROOT="$ROOT/webui"
PYTHON="${PYTHON:-$ROOT/.venv/bin/python}"
NODE_BIN="${NODE_BIN:-/home/zemion/.nvm/versions/node/v22.22.3/bin}"
NPM="${NPM:-$NODE_BIN/npm}"
BACKEND_HOST="${GOVOPLAN_BACKEND_HOST:-127.0.0.1}"
BACKEND_PORT="${GOVOPLAN_BACKEND_PORT:-8000}"
FRONTEND_HOST="${GOVOPLAN_FRONTEND_HOST:-127.0.0.1}"
FRONTEND_PORT="${GOVOPLAN_FRONTEND_PORT:-5173}"
OPEN_BROWSER="${OPEN_BROWSER:-1}"
LOG_DIR="$ROOT/runtime/dev-launcher"
BACKEND_LOG="$LOG_DIR/backend.log"
FRONTEND_LOG="$LOG_DIR/frontend.log"
BACKEND_URL="http://$BACKEND_HOST:$BACKEND_PORT"
FRONTEND_URL="http://$FRONTEND_HOST:$FRONTEND_PORT"
backend_pid=""
frontend_pid=""
fail() {
printf 'launch-dev: %s\n' "$*" >&2
exit 1
}
port_is_free() {
"$PYTHON" - "$1" "$2" <<'PY'
import socket
import sys
host = sys.argv[1]
port = int(sys.argv[2])
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
sock.bind((host, port))
except OSError:
raise SystemExit(1)
PY
}
wait_for_url() {
"$PYTHON" - "$1" <<'PY'
import sys
import time
import urllib.request
url = sys.argv[1]
deadline = time.monotonic() + 60
last_error = None
while time.monotonic() < deadline:
try:
with urllib.request.urlopen(url, timeout=2) as response:
if 200 <= response.status < 500:
raise SystemExit(0)
except Exception as exc: # noqa: BLE001 - printed only on timeout.
last_error = exc
time.sleep(1)
print(f"Timed out waiting for {url}: {last_error}", file=sys.stderr)
raise SystemExit(1)
PY
}
cleanup() {
if [ -n "${frontend_pid:-}" ] && kill -0 "$frontend_pid" 2>/dev/null; then
kill "$frontend_pid" 2>/dev/null || true
fi
if [ -n "${backend_pid:-}" ] && kill -0 "$backend_pid" 2>/dev/null; then
kill "$backend_pid" 2>/dev/null || true
fi
}
trap cleanup EXIT INT TERM
[ -x "$PYTHON" ] || fail "Python virtualenv not found at $PYTHON. Run: cd $ROOT && ./.venv/bin/python -m pip install -r requirements-dev.txt"
[ -x "$NPM" ] || fail "npm not found at $NPM. Set NODE_BIN or NPM to your Node installation."
[ -f "$WEBUI_ROOT/package.json" ] || fail "WebUI package.json not found at $WEBUI_ROOT/package.json"
[ -d "$WEBUI_ROOT/node_modules" ] || fail "WebUI node_modules missing. Run: cd $WEBUI_ROOT && PATH=$NODE_BIN:\$PATH $NPM install"
mkdir -p "$LOG_DIR"
: > "$BACKEND_LOG"
: > "$FRONTEND_LOG"
port_is_free "$BACKEND_HOST" "$BACKEND_PORT" || fail "$BACKEND_URL is already in use"
port_is_free "$FRONTEND_HOST" "$FRONTEND_PORT" || fail "$FRONTEND_URL is already in use"
printf 'Starting GovOPlaN backend at %s\n' "$BACKEND_URL"
(
cd "$ROOT"
"$PYTHON" -m govoplan_core.devserver --host "$BACKEND_HOST" --port "$BACKEND_PORT"
) >"$BACKEND_LOG" 2>&1 &
backend_pid="$!"
printf 'Waiting for %s/health\n' "$BACKEND_URL"
wait_for_url "$BACKEND_URL/health" || {
tail -n 80 "$BACKEND_LOG" >&2 || true
fail "backend did not become healthy"
}
printf 'Starting GovOPlaN WebUI at %s\n' "$FRONTEND_URL"
(
cd "$WEBUI_ROOT"
PATH="$WEBUI_ROOT/node_modules/.bin:$NODE_BIN:$PATH" \
VITE_API_PROXY_TARGET="$BACKEND_URL" \
"$NPM" run dev
) >"$FRONTEND_LOG" 2>&1 &
frontend_pid="$!"
printf 'Waiting for %s\n' "$FRONTEND_URL"
wait_for_url "$FRONTEND_URL" || {
tail -n 80 "$FRONTEND_LOG" >&2 || true
fail "frontend did not become reachable"
}
if [ "$OPEN_BROWSER" = "1" ] && command -v xdg-open >/dev/null 2>&1; then
xdg-open "$FRONTEND_URL" >/dev/null 2>&1 || true
fi
cat <<EOF
GovOPlaN is running.
Web UI: $FRONTEND_URL
API: $BACKEND_URL/api/v1
Health: $BACKEND_URL/health
Logs:
Backend: $BACKEND_LOG
WebUI: $FRONTEND_LOG
Press Ctrl+C to stop both processes.
EOF
wait

View File

@@ -1,600 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'USAGE'
Usage:
scripts/push-release-tag.sh [options]
Bumps all GovOPlaN package versions, commits all changes in all four repos,
creates an annotated release tag in each repo, and pushes branch+tag.
By default the script asks which version part to bump:
major X.Y.Z -> (X+1).0.0
minor X.Y.Z -> X.(Y+1).0
subversion X.Y.Z -> X.Y.(Z+1)
Options:
--bump <kind> Version bump: major, minor, subversion, or patch.
--version <x.y.z> Explicit target version instead of a bump. May equal
the current repo versions when they were bumped already.
-r, --remote <name> Remote to push to. Defaults to origin.
-b, --branch <name> Branch to update in every repo. Defaults to each current branch.
-m, --message <text> Commit message. Defaults to "Release v<x.y.z>".
--tag-message <text> Annotated tag message. Defaults to the commit message.
--skip-release-lock Do not regenerate core webui/package-lock.release.json.
--npm <path> npm executable for lockfile generation.
-y, --yes Do not prompt before committing, tagging, and pushing.
-n, --dry-run Print intended actions without changing files or git state.
-h, --help Show this help.
Repos:
govoplan-core
govoplan-files
govoplan-mail
govoplan-campaign
Examples:
scripts/push-release-tag.sh
scripts/push-release-tag.sh --bump subversion
scripts/push-release-tag.sh --version 0.2.0 --message "Release v0.2.0"
USAGE
}
fail() {
echo "error: $*" >&2
exit 1
}
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PARENT="$(dirname "$ROOT")"
REPOS=(
"$PARENT/govoplan-files"
"$PARENT/govoplan-mail"
"$PARENT/govoplan-campaign"
"$ROOT"
)
REMOTE="origin"
BRANCH=""
BUMP=""
TARGET_VERSION=""
COMMIT_MESSAGE=""
TAG_MESSAGE=""
DRY_RUN=0
YES=0
GENERATE_RELEASE_LOCK=1
NPM_BIN="${NPM:-}"
if [[ -z "${PYTHON:-}" ]]; then
if [[ -x "$ROOT/.venv/bin/python" ]]; then
PYTHON="$ROOT/.venv/bin/python"
else
PYTHON="python3"
fi
fi
if [[ -z "$NPM_BIN" ]]; then
if [[ -x "/home/zemion/.nvm/versions/node/v22.22.3/bin/npm" ]]; then
NPM_BIN="/home/zemion/.nvm/versions/node/v22.22.3/bin/npm"
else
NPM_BIN="npm"
fi
fi
normalize_bump() {
local value="${1,,}"
case "$value" in
major|minor|subversion)
printf '%s\n' "$value"
;;
patch|sub|sub-version)
printf '%s\n' "subversion"
;;
*)
return 1
;;
esac
}
validate_version() {
[[ "$1" =~ ^[0-9]+[.][0-9]+[.][0-9]+$ ]]
}
version_gt() {
local new_major new_minor new_patch old_major old_minor old_patch
IFS=. read -r new_major new_minor new_patch <<< "$1"
IFS=. read -r old_major old_minor old_patch <<< "$2"
(( new_major > old_major )) && return 0
(( new_major < old_major )) && return 1
(( new_minor > old_minor )) && return 0
(( new_minor < old_minor )) && return 1
(( new_patch > old_patch )) && return 0
return 1
}
bump_version() {
local version="$1"
local bump="$2"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
case "$bump" in
major)
printf '%s.0.0\n' "$((major + 1))"
;;
minor)
printf '%s.%s.0\n' "$major" "$((minor + 1))"
;;
subversion)
printf '%s.%s.%s\n' "$major" "$minor" "$((patch + 1))"
;;
*)
fail "unknown bump kind: $bump"
;;
esac
}
get_project_name() {
"$PYTHON" -c 'import pathlib, sys, tomllib; print(tomllib.loads(pathlib.Path(sys.argv[1]).read_text())["project"]["name"])' "$1/pyproject.toml"
}
get_project_version() {
"$PYTHON" -c 'import pathlib, sys, tomllib; print(tomllib.loads(pathlib.Path(sys.argv[1]).read_text())["project"]["version"])' "$1/pyproject.toml"
}
update_pyproject() {
local repo="$1"
local version="$2"
"$PYTHON" - "$repo/pyproject.toml" "$version" <<'PYCODE'
from __future__ import annotations
import pathlib
import re
import sys
import tomllib
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
data = tomllib.loads(text)
project = data["project"]
name = project["name"]
old_version = project["version"]
text, version_count = re.subn(
r'(?m)^version\s*=\s*["\'][^"\']+["\']\s*$',
f'version = "{new_version}"',
text,
count=1,
)
if version_count != 1:
raise SystemExit(f"could not update project.version in {path}")
if name != "govoplan-core":
text, dependency_count = re.subn(
r'govoplan-core>=\d+\.\d+\.\d+',
f'govoplan-core>={new_version}',
text,
)
if dependency_count < 1:
raise SystemExit(f"could not update govoplan-core dependency in {path}")
path.write_text(text)
print(f"{name}: {old_version} -> {new_version}")
PYCODE
}
update_manifest_version() {
local repo="$1"
local project_name="$2"
local version="$3"
local manifest_path=""
case "$project_name" in
govoplan-core)
manifest_path="$repo/src/govoplan_core/access/manifest.py"
;;
govoplan-files)
manifest_path="$repo/src/govoplan_files/backend/manifest.py"
;;
govoplan-mail)
manifest_path="$repo/src/govoplan_mail/backend/manifest.py"
;;
govoplan-campaign)
manifest_path="$repo/src/govoplan_campaign/backend/manifest.py"
;;
*)
fail "unknown project for manifest version update: $project_name"
;;
esac
[[ -f "$manifest_path" ]] || fail "missing module manifest: $manifest_path"
"$PYTHON" - "$manifest_path" "$version" <<'PYCODE'
from __future__ import annotations
import pathlib
import re
import sys
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(\s*version=)["\'][^"\']+["\'](,?\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update ModuleManifest.version in {path}")
path.write_text(text)
PYCODE
}
update_webui_package() {
local repo="$1"
local project_name="$2"
local version="$3"
local package_path=""
local release_package_path="$repo/webui/package.release.json"
for package_path in "$repo/package.json" "$repo/webui/package.json"; do
[[ -f "$package_path" ]] || continue
"$PYTHON" - "$package_path" "$project_name" "$version" <<'PYCODE'
from __future__ import annotations
import json
import pathlib
import sys
path = pathlib.Path(sys.argv[1])
project_name = sys.argv[2]
new_version = sys.argv[3]
data = json.loads(path.read_text())
data["version"] = new_version
if project_name != "govoplan-core":
peers = data.setdefault("peerDependencies", {})
if "@govoplan/core-webui" in peers:
peers["@govoplan/core-webui"] = f"^{new_version}"
path.write_text(json.dumps(data, indent=2) + "\n")
PYCODE
done
if [[ "$project_name" == "govoplan-core" && -f "$release_package_path" ]]; then
"$PYTHON" - "$release_package_path" "$version" <<'PYCODE'
from __future__ import annotations
import json
import pathlib
import re
import sys
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
data = json.loads(path.read_text())
data["version"] = new_version
dependencies = data.setdefault("dependencies", {})
for name, spec in list(dependencies.items()):
if name in {"@govoplan/files-webui", "@govoplan/mail-webui", "@govoplan/campaign-webui"}:
dependencies[name] = re.sub(r"#v\d+\.\d+\.\d+$", f"#v{new_version}", spec)
path.write_text(json.dumps(data, indent=2) + "\n")
PYCODE
fi
}
update_version_files() {
local repo="$1"
local version="$2"
local project_name="${PROJECT_NAMES[$repo]}"
update_pyproject "$repo" "$version"
update_manifest_version "$repo" "$project_name" "$version"
update_webui_package "$repo" "$project_name" "$version"
}
refresh_development_webui_lock() {
[[ -f "$ROOT/webui/package.json" ]] || return 0
[[ -f "$ROOT/webui/package-lock.json" ]] || return 0
run env PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" --prefix "$ROOT/webui" install --package-lock-only --ignore-scripts
}
generate_release_lock() {
if [[ "$GENERATE_RELEASE_LOCK" -eq 0 ]]; then
return
fi
[[ -x "$ROOT/scripts/generate-release-lock.sh" ]] || fail "missing executable release lock generator: $ROOT/scripts/generate-release-lock.sh"
run "$ROOT/scripts/generate-release-lock.sh" --npm "$NPM_BIN"
}
print_command() {
printf '+'
printf ' %q' "$@"
printf '\n'
}
run() {
print_command "$@"
if [[ "$DRY_RUN" -eq 0 ]]; then
"$@"
fi
}
prompt_for_bump() {
local answer=""
if [[ -n "$BUMP" || -n "$TARGET_VERSION" ]]; then
return
fi
if [[ ! -t 0 ]]; then
fail "pass --bump major|minor|subversion or --version x.y.z for non-interactive use"
fi
echo "Select version increase:"
echo " 1) major X.Y.Z -> (X+1).0.0"
echo " 2) minor X.Y.Z -> X.(Y+1).0"
echo " 3) subversion X.Y.Z -> X.Y.(Z+1)"
printf 'Choice [subversion]: '
read -r answer
case "${answer,,}" in
""|3|subversion|patch|sub|sub-version)
BUMP="subversion"
;;
1|major)
BUMP="major"
;;
2|minor)
BUMP="minor"
;;
*)
fail "unknown version bump: $answer"
;;
esac
}
confirm_release() {
local answer=""
if [[ "$DRY_RUN" -eq 1 || "$YES" -eq 1 ]]; then
return
fi
if [[ ! -t 0 ]]; then
fail "refusing to commit, tag, and push without confirmation; pass --yes for non-interactive use"
fi
printf 'Commit all changes, tag all repos, and push to %s? [y/N] ' "$REMOTE"
read -r answer
case "${answer,,}" in
y|yes)
;;
*)
echo "Aborted."
exit 1
;;
esac
}
while [[ $# -gt 0 ]]; do
case "$1" in
--bump)
[[ $# -ge 2 ]] || fail "missing value for $1"
BUMP="$(normalize_bump "$2")" || fail "unknown bump kind: $2"
shift 2
;;
--version)
[[ $# -ge 2 ]] || fail "missing value for $1"
TARGET_VERSION="$2"
shift 2
;;
-r|--remote)
[[ $# -ge 2 ]] || fail "missing value for $1"
REMOTE="$2"
shift 2
;;
-b|--branch)
[[ $# -ge 2 ]] || fail "missing value for $1"
BRANCH="$2"
shift 2
;;
-m|--message)
[[ $# -ge 2 ]] || fail "missing value for $1"
COMMIT_MESSAGE="$2"
shift 2
;;
--tag-message)
[[ $# -ge 2 ]] || fail "missing value for $1"
TAG_MESSAGE="$2"
shift 2
;;
--skip-release-lock)
GENERATE_RELEASE_LOCK=0
shift
;;
--npm)
[[ $# -ge 2 ]] || fail "missing value for $1"
NPM_BIN="$2"
shift 2
;;
-y|--yes)
YES=1
shift
;;
-n|--dry-run)
DRY_RUN=1
shift
;;
-h|--help)
usage
exit 0
;;
*)
echo "Unknown argument: $1" >&2
usage >&2
exit 2
;;
esac
done
if [[ -n "$BUMP" && -n "$TARGET_VERSION" ]]; then
fail "use either --bump or --version, not both"
fi
prompt_for_bump
if ! command -v "$PYTHON" >/dev/null 2>&1; then
fail "Python interpreter not found: $PYTHON"
fi
if ! command -v "$NPM_BIN" >/dev/null 2>&1; then
fail "npm executable not found: $NPM_BIN"
fi
declare -A PROJECT_NAMES
declare -A OLD_VERSIONS
declare -A BRANCHES
for repo in "${REPOS[@]}"; do
[[ -d "$repo/.git" ]] || fail "not a git repo: $repo"
[[ -f "$repo/pyproject.toml" ]] || fail "missing pyproject.toml: $repo"
PROJECT_NAMES["$repo"]="$(get_project_name "$repo")"
OLD_VERSIONS["$repo"]="$(get_project_version "$repo")"
validate_version "${OLD_VERSIONS[$repo]}" || fail "unsupported version ${OLD_VERSIONS[$repo]} in $repo"
if [[ -n "$BRANCH" ]]; then
BRANCHES["$repo"]="$BRANCH"
else
BRANCHES["$repo"]="$(git -C "$repo" symbolic-ref --quiet --short HEAD || true)"
fi
[[ -n "${BRANCHES[$repo]}" ]] || fail "cannot infer branch for $repo; pass --branch <name>"
git -C "$repo" check-ref-format "refs/heads/${BRANCHES[$repo]}" || fail "invalid branch name ${BRANCHES[$repo]} for $repo"
git -C "$repo" remote get-url "$REMOTE" >/dev/null || fail "remote $REMOTE not found in $repo"
upstream="$(git -C "$repo" rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null || true)"
if [[ -n "$upstream" ]]; then
behind_count="$(git -C "$repo" rev-list --count "HEAD..$upstream")"
[[ "$behind_count" == "0" ]] || fail "$repo is behind $upstream by $behind_count commit(s); pull/rebase first"
fi
done
if [[ -z "$TARGET_VERSION" ]]; then
BASE_VERSION="${OLD_VERSIONS[$ROOT]}"
for repo in "${REPOS[@]}"; do
if [[ "${OLD_VERSIONS[$repo]}" != "$BASE_VERSION" ]]; then
fail "repo versions differ; pass --version x.y.z to choose an explicit target"
fi
done
TARGET_VERSION="$(bump_version "$BASE_VERSION" "$BUMP")"
fi
validate_version "$TARGET_VERSION" || fail "target version must be x.y.z: $TARGET_VERSION"
for repo in "${REPOS[@]}"; do
if [[ "$TARGET_VERSION" == "${OLD_VERSIONS[$repo]}" ]]; then
continue
fi
if ! version_gt "$TARGET_VERSION" "${OLD_VERSIONS[$repo]}"; then
fail "target version $TARGET_VERSION must be greater than ${OLD_VERSIONS[$repo]} for ${PROJECT_NAMES[$repo]}"
fi
done
TAG="v$TARGET_VERSION"
COMMIT_MESSAGE="${COMMIT_MESSAGE:-Release $TAG}"
TAG_MESSAGE="${TAG_MESSAGE:-$COMMIT_MESSAGE}"
git -C "$ROOT" check-ref-format "refs/tags/$TAG" || fail "invalid tag name: $TAG"
for repo in "${REPOS[@]}"; do
if git -C "$repo" rev-parse --quiet --verify "refs/tags/$TAG" >/dev/null; then
fail "local tag already exists in ${PROJECT_NAMES[$repo]}: $TAG"
fi
set +e
git -C "$repo" ls-remote --exit-code --tags "$REMOTE" "refs/tags/$TAG" >/dev/null
ls_remote_status=$?
set -e
if [[ "$ls_remote_status" -eq 0 ]]; then
fail "remote tag already exists for ${PROJECT_NAMES[$repo]} on $REMOTE: $TAG"
elif [[ "$ls_remote_status" -ne 2 ]]; then
fail "could not check remote tags for ${PROJECT_NAMES[$repo]} on $REMOTE"
fi
done
echo "Release plan:"
echo " version: $TARGET_VERSION"
echo " tag: $TAG"
echo " remote: $REMOTE"
echo " commit message: $COMMIT_MESSAGE"
if [[ "$GENERATE_RELEASE_LOCK" -eq 1 ]]; then
echo " release lock: regenerate core webui/package-lock.release.json after module tags are pushed"
else
echo " release lock: skipped"
fi
echo
for repo in "${REPOS[@]}"; do
echo "${PROJECT_NAMES[$repo]}: ${OLD_VERSIONS[$repo]} -> $TARGET_VERSION on ${BRANCHES[$repo]}"
git -C "$repo" status --short
echo
done
confirm_release
for repo in "${REPOS[@]}"; do
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Would update version files in $repo to $TARGET_VERSION"
else
update_version_files "$repo" "$TARGET_VERSION"
fi
done
refresh_development_webui_lock
for repo in "${REPOS[@]:0:3}"; do
run git -C "$repo" add -A
if [[ "$DRY_RUN" -eq 0 ]]; then
if git -C "$repo" diff --cached --quiet; then
fail "no staged changes for ${PROJECT_NAMES[$repo]} after version bump"
fi
fi
run git -C "$repo" commit -m "$COMMIT_MESSAGE"
run git -C "$repo" tag -a "$TAG" -m "$TAG_MESSAGE"
done
for repo in "${REPOS[@]:0:3}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
generate_release_lock
run git -C "$ROOT" add -A
if [[ "$DRY_RUN" -eq 0 ]]; then
if git -C "$ROOT" diff --cached --quiet; then
fail "no staged changes for ${PROJECT_NAMES[$ROOT]} after version bump"
fi
fi
run git -C "$ROOT" commit -m "$COMMIT_MESSAGE"
run git -C "$ROOT" tag -a "$TAG" -m "$TAG_MESSAGE"
run git -C "$ROOT" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$ROOT]}" "refs/tags/$TAG"
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Dry run complete for $TAG across all GovOPlaN repos."
else
echo "Released $TAG across all GovOPlaN repos."
fi

View File

@@ -1,2 +0,0 @@
"""Tenant, identity, auth, RBAC, and datastore access platform module."""

View File

@@ -1,2 +0,0 @@
"""Authentication and principal helpers for platform access."""

View File

@@ -1,28 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
from govoplan_core.access.permissions.evaluator import scopes_grant
AuthMethod = Literal["session", "api_key", "service_account"]
@dataclass(frozen=True, slots=True)
class Principal:
account_id: str
membership_id: str | None
tenant_id: str | None
scopes: frozenset[str]
group_ids: frozenset[str]
auth_method: AuthMethod
api_key_id: str | None = None
session_id: str | None = None
service_account_id: str | None = None
email: str | None = None
display_name: str | None = None
def has(self, required: str) -> bool:
return scopes_grant(self.scopes, required)

View File

@@ -1,82 +0,0 @@
from __future__ import annotations
from sqlalchemy.orm import Session
from govoplan_core.access.db.models import Account, Group, GroupMembership, Membership, Role, RoleBinding
from govoplan_core.access.permissions.evaluator import expand_scopes
from govoplan_core.core.modules import PermissionDefinition, SubjectType
def membership_group_ids(session: Session, *, tenant_id: str, membership_id: str) -> frozenset[str]:
rows = (
session.query(Group.id)
.join(GroupMembership, GroupMembership.group_id == Group.id)
.filter(
GroupMembership.tenant_id == tenant_id,
GroupMembership.membership_id == membership_id,
Group.is_active.is_(True),
)
.all()
)
return frozenset(row[0] for row in rows)
def roles_for_subject(
session: Session,
*,
subject_type: SubjectType,
subject_id: str,
tenant_id: str | None,
) -> list[Role]:
query = (
session.query(Role)
.join(RoleBinding, RoleBinding.role_id == Role.id)
.filter(
RoleBinding.subject_type == subject_type,
RoleBinding.subject_id == subject_id,
)
)
if tenant_id is None:
query = query.filter(RoleBinding.tenant_id.is_(None), Role.tenant_id.is_(None))
else:
query = query.filter(RoleBinding.tenant_id == tenant_id, Role.tenant_id == tenant_id)
return query.order_by(Role.name.asc()).all()
def membership_roles(session: Session, membership: Membership) -> list[Role]:
roles_by_id = {
role.id: role
for role in roles_for_subject(
session,
subject_type="membership",
subject_id=membership.id,
tenant_id=membership.tenant_id,
)
}
for group_id in membership_group_ids(session, tenant_id=membership.tenant_id, membership_id=membership.id):
for role in roles_for_subject(session, subject_type="group", subject_id=group_id, tenant_id=membership.tenant_id):
roles_by_id[role.id] = role
return list(roles_by_id.values())
def account_system_roles(session: Session, account: Account) -> list[Role]:
return roles_for_subject(session, subject_type="account", subject_id=account.id, tenant_id=None)
def principal_scopes(
session: Session,
*,
account: Account,
membership: Membership | None,
include_system: bool,
catalog: dict[str, PermissionDefinition],
) -> list[str]:
scopes: set[str] = set()
if membership is not None:
for role in membership_roles(session, membership):
scopes.update(role.permissions or [])
if include_system:
for role in account_system_roles(session, account):
scopes.update(role.permissions or [])
return expand_scopes(scopes, catalog=catalog)

View File

@@ -1,21 +0,0 @@
from __future__ import annotations
import hashlib
import hmac
import secrets
DEFAULT_RANDOM_BYTES = 32
def generate_secret(prefix: str, *, random_bytes: int = DEFAULT_RANDOM_BYTES) -> str:
return prefix + secrets.token_urlsafe(random_bytes)
def hash_secret(secret: str) -> str:
return hashlib.sha256(secret.encode("utf-8")).hexdigest()
def verify_secret(secret: str, expected_hash: str) -> bool:
return hmac.compare_digest(hash_secret(secret), expected_hash)

View File

@@ -1,2 +0,0 @@
"""Access-platform SQLAlchemy metadata and models."""

View File

@@ -1,21 +0,0 @@
from __future__ import annotations
from typing import Any
from sqlalchemy import JSON, MetaData
from sqlalchemy.orm import DeclarativeBase
from govoplan_core.db.base import NAMING_CONVENTION, TimestampMixin, utcnow
class AccessBase(DeclarativeBase):
metadata = MetaData(naming_convention=NAMING_CONVENTION)
type_annotation_map = {
dict[str, Any]: JSON,
list[dict[str, Any]]: JSON,
list[str]: JSON,
}
__all__ = ["AccessBase", "NAMING_CONVENTION", "TimestampMixin", "utcnow"]

View File

@@ -1,237 +0,0 @@
from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint, text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from govoplan_core.access.db.base import AccessBase, TimestampMixin
def new_uuid() -> str:
return str(uuid.uuid4())
class Account(AccessBase, TimestampMixin):
__tablename__ = "access_accounts"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
email: Mapped[str] = mapped_column(String(320), nullable=False)
normalized_email: Mapped[str] = mapped_column(String(320), nullable=False, unique=True, index=True)
display_name: Mapped[str | None] = mapped_column(String(255))
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
auth_provider: Mapped[str] = mapped_column(String(50), default="local", nullable=False)
password_hash: Mapped[str | None] = mapped_column(String(500))
password_reset_required: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
last_login_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
memberships: Mapped[list[Membership]] = relationship(back_populates="account", cascade="all, delete-orphan")
class Tenant(AccessBase, TimestampMixin):
__tablename__ = "access_tenants"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
slug: Mapped[str] = mapped_column(String(100), nullable=False, unique=True, index=True)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
default_locale: Mapped[str] = mapped_column(String(20), default="en", nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
memberships: Mapped[list[Membership]] = relationship(back_populates="tenant", cascade="all, delete-orphan")
class Membership(AccessBase, TimestampMixin):
__tablename__ = "access_memberships"
__table_args__ = (UniqueConstraint("tenant_id", "account_id", name="uq_access_memberships_tenant_account"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=False, index=True)
account_id: Mapped[str] = mapped_column(ForeignKey("access_accounts.id", ondelete="CASCADE"), nullable=False, index=True)
email_snapshot: Mapped[str | None] = mapped_column(String(320), index=True)
display_name: Mapped[str | None] = mapped_column(String(255))
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
account: Mapped[Account] = relationship(back_populates="memberships")
tenant: Mapped[Tenant] = relationship(back_populates="memberships")
class Group(AccessBase, TimestampMixin):
__tablename__ = "access_groups"
__table_args__ = (UniqueConstraint("tenant_id", "slug", name="uq_access_groups_tenant_slug"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=False, index=True)
slug: Mapped[str] = mapped_column(String(100), nullable=False)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
template_id: Mapped[str | None] = mapped_column(String(100), index=True)
is_protected: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
class GroupMembership(AccessBase, TimestampMixin):
__tablename__ = "access_group_memberships"
__table_args__ = (
UniqueConstraint("tenant_id", "membership_id", "group_id", name="uq_access_group_memberships_member_group"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=False, index=True)
membership_id: Mapped[str] = mapped_column(ForeignKey("access_memberships.id", ondelete="CASCADE"), nullable=False, index=True)
group_id: Mapped[str] = mapped_column(ForeignKey("access_groups.id", ondelete="CASCADE"), nullable=False, index=True)
class Role(AccessBase, TimestampMixin):
__tablename__ = "access_roles"
__table_args__ = (
UniqueConstraint("tenant_id", "slug", name="uq_access_roles_tenant_slug"),
Index(
"uq_access_roles_system_slug",
"slug",
unique=True,
sqlite_where=text("tenant_id IS NULL"),
postgresql_where=text("tenant_id IS NULL"),
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str | None] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=True, index=True)
slug: Mapped[str] = mapped_column(String(100), nullable=False)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
permissions: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
level: Mapped[str] = mapped_column(String(20), default="tenant", nullable=False, index=True)
is_builtin: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
is_assignable: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
template_id: Mapped[str | None] = mapped_column(String(100), index=True)
is_protected: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
class RoleBinding(AccessBase, TimestampMixin):
__tablename__ = "access_role_bindings"
__table_args__ = (
Index("ix_access_role_bindings_subject", "subject_type", "subject_id"),
Index("ix_access_role_bindings_tenant_subject", "tenant_id", "subject_type", "subject_id"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str | None] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=True, index=True)
role_id: Mapped[str] = mapped_column(ForeignKey("access_roles.id", ondelete="CASCADE"), nullable=False, index=True)
subject_type: Mapped[str] = mapped_column(String(50), nullable=False)
subject_id: Mapped[str] = mapped_column(String(36), nullable=False)
created_by_account_id: Mapped[str | None] = mapped_column(String(36), index=True)
created_by_membership_id: Mapped[str | None] = mapped_column(String(36), index=True)
class PermissionCatalogEntry(AccessBase, TimestampMixin):
__tablename__ = "access_permission_catalog"
scope: Mapped[str] = mapped_column(String(255), primary_key=True)
module_id: Mapped[str] = mapped_column(String(100), nullable=False, index=True)
resource: Mapped[str] = mapped_column(String(100), nullable=False)
action: Mapped[str] = mapped_column(String(100), nullable=False)
label: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str] = mapped_column(Text, default="", nullable=False)
category: Mapped[str] = mapped_column(String(100), nullable=False)
level: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
is_deprecated: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
class RoleTemplateModel(AccessBase, TimestampMixin):
__tablename__ = "access_role_templates"
__table_args__ = (UniqueConstraint("module_id", "level", "slug", name="uq_access_role_templates_module_level_slug"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
module_id: Mapped[str] = mapped_column(String(100), nullable=False, index=True)
slug: Mapped[str] = mapped_column(String(100), nullable=False)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
permissions: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
level: Mapped[str] = mapped_column(String(20), default="tenant", nullable=False)
managed: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
protected: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
class ModuleInstallation(AccessBase, TimestampMixin):
__tablename__ = "access_module_installations"
module_id: Mapped[str] = mapped_column(String(100), primary_key=True)
version: Mapped[str] = mapped_column(String(50), nullable=False)
enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
class AuthSession(AccessBase, TimestampMixin):
__tablename__ = "access_auth_sessions"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str | None] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=True, index=True)
membership_id: Mapped[str | None] = mapped_column(ForeignKey("access_memberships.id", ondelete="CASCADE"), nullable=True, index=True)
account_id: Mapped[str] = mapped_column(ForeignKey("access_accounts.id", ondelete="CASCADE"), nullable=False, index=True)
token_hash: Mapped[str] = mapped_column(String(255), nullable=False, unique=True, index=True)
csrf_token_hash: Mapped[str | None] = mapped_column(String(255))
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
last_seen_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
user_agent: Mapped[str | None] = mapped_column(String(500))
ip_address: Mapped[str | None] = mapped_column(String(100))
class ApiKey(AccessBase, TimestampMixin):
__tablename__ = "access_api_keys"
__table_args__ = (Index("ix_access_api_keys_owner", "owner_subject_type", "owner_subject_id"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=False, index=True)
owner_subject_type: Mapped[str] = mapped_column(String(50), nullable=False)
owner_subject_id: Mapped[str] = mapped_column(String(36), nullable=False)
name: Mapped[str] = mapped_column(String(255), nullable=False)
prefix: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
key_hash: Mapped[str] = mapped_column(String(255), nullable=False, unique=True)
scopes: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
class TenantDatastore(AccessBase, TimestampMixin):
__tablename__ = "access_tenant_datastores"
__table_args__ = (UniqueConstraint("tenant_id", "module_id", name="uq_access_tenant_datastores_tenant_module"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), nullable=False, index=True)
module_id: Mapped[str] = mapped_column(String(100), default="*", nullable=False)
mode: Mapped[str] = mapped_column(String(20), default="shared", nullable=False)
database_url_secret_ref: Mapped[str | None] = mapped_column(String(255))
schema_name: Mapped[str | None] = mapped_column(String(100))
storage_bucket: Mapped[str | None] = mapped_column(String(255))
region: Mapped[str | None] = mapped_column(String(100))
is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
class SystemSettings(AccessBase, TimestampMixin):
__tablename__ = "access_system_settings"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default="global")
default_locale: Mapped[str] = mapped_column(String(20), default="en", nullable=False)
allow_tenant_custom_groups: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_custom_roles: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_api_keys: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
class TenantSettings(AccessBase, TimestampMixin):
__tablename__ = "access_tenant_settings"
tenant_id: Mapped[str] = mapped_column(ForeignKey("access_tenants.id", ondelete="CASCADE"), primary_key=True)
allow_custom_groups: Mapped[bool | None] = mapped_column(Boolean)
allow_custom_roles: Mapped[bool | None] = mapped_column(Boolean)
allow_api_keys: Mapped[bool | None] = mapped_column(Boolean)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)

View File

@@ -1,22 +0,0 @@
from __future__ import annotations
from collections.abc import Generator
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, sessionmaker
from govoplan_core.db.session import create_database_engine
def create_access_engine(database_url: str) -> Engine:
return create_database_engine(database_url)
def create_access_session_factory(database_url: str) -> sessionmaker[Session]:
engine = create_access_engine(database_url)
return sessionmaker(bind=engine, autoflush=False, autocommit=False, expire_on_commit=False)
def session_scope(session_factory: sessionmaker[Session]) -> Generator[Session, None, None]:
with session_factory() as session:
yield session

View File

@@ -1,118 +0,0 @@
from __future__ import annotations
from govoplan_core.access.db.base import AccessBase
from govoplan_core.access.db import models as access_models # noqa: F401 - populate access metadata
from govoplan_core.core.modules import MigrationSpec, ModuleManifest, PermissionDefinition, RoleTemplate
def _permission(scope: str, label: str, description: str, category: str, level: str) -> PermissionDefinition:
module_id, resource, action = scope.split(":", 2)
return PermissionDefinition(
scope=scope,
label=label,
description=description,
category=category,
level=level, # type: ignore[arg-type]
module_id=module_id,
resource=resource,
action=action,
)
ACCESS_PERMISSIONS: tuple[PermissionDefinition, ...] = (
_permission("access:tenant:read", "View tenants", "List and inspect tenant registry entries.", "Access", "system"),
_permission("access:tenant:create", "Create tenants", "Create tenant registry entries.", "Access", "system"),
_permission("access:tenant:update", "Update tenants", "Update tenant metadata and activation state.", "Access", "system"),
_permission("access:account:read", "View accounts", "List and inspect global login accounts.", "Access", "system"),
_permission("access:account:create", "Create accounts", "Create global login accounts.", "Access", "system"),
_permission("access:account:update", "Update accounts", "Update or suspend global login accounts.", "Access", "system"),
_permission("access:membership:read", "View memberships", "List tenant memberships and effective access.", "Tenant access", "tenant"),
_permission("access:membership:create", "Create memberships", "Create tenant-local account memberships.", "Tenant access", "tenant"),
_permission("access:membership:update", "Update memberships", "Update or suspend tenant memberships.", "Tenant access", "tenant"),
_permission("access:group:read", "View groups", "List tenant groups and members.", "Tenant access", "tenant"),
_permission("access:group:write", "Manage groups", "Create and update tenant groups.", "Tenant access", "tenant"),
_permission("access:group:manage_members", "Manage group members", "Add and remove memberships from groups.", "Tenant access", "tenant"),
_permission("access:role:read", "View roles", "Inspect tenant and system role definitions.", "Tenant access", "tenant"),
_permission("access:role:write", "Manage roles", "Create and update assignable roles.", "Tenant access", "tenant"),
_permission("access:role:assign", "Assign roles", "Bind roles to memberships, groups, accounts or services.", "Tenant access", "tenant"),
_permission("access:api_key:read", "View API keys", "List API keys without revealing secrets.", "Tenant access", "tenant"),
_permission("access:api_key:create", "Create API keys", "Create tenant API keys within delegation limits.", "Tenant access", "tenant"),
_permission("access:api_key:revoke", "Revoke API keys", "Revoke tenant API keys.", "Tenant access", "tenant"),
_permission("access:setting:read", "View settings", "Read access and governance settings.", "Tenant access", "tenant"),
_permission("access:setting:write", "Manage settings", "Update access and governance settings.", "Tenant access", "tenant"),
_permission("access:governance:read", "View governance", "Inspect managed role and group templates.", "Access", "system"),
_permission("access:governance:write", "Manage governance", "Create and assign managed role and group templates.", "Access", "system"),
)
ACCESS_ROLE_TEMPLATES: tuple[RoleTemplate, ...] = (
RoleTemplate(
slug="system_owner",
name="System owner",
description="Protected full instance-wide administration.",
permissions=("system:*",),
level="system",
managed=True,
protected=True,
),
RoleTemplate(
slug="system_admin",
name="System administrator",
description="Manage tenants, accounts, settings, and governance without protected owner status.",
permissions=(
"access:tenant:read",
"access:tenant:create",
"access:tenant:update",
"access:account:read",
"access:account:create",
"access:account:update",
"access:governance:read",
"access:governance:write",
),
level="system",
managed=True,
protected=False,
),
RoleTemplate(
slug="tenant_owner",
name="Tenant owner",
description="Protected full tenant administration and module access.",
permissions=("tenant:*",),
level="tenant",
managed=True,
protected=True,
),
RoleTemplate(
slug="access_admin",
name="Access administrator",
description="Manage memberships, groups, roles, and API keys within delegation limits.",
permissions=(
"access:membership:read",
"access:membership:create",
"access:membership:update",
"access:group:read",
"access:group:write",
"access:group:manage_members",
"access:role:read",
"access:role:assign",
"access:api_key:read",
"access:api_key:create",
"access:api_key:revoke",
),
level="tenant",
managed=True,
protected=False,
),
)
manifest = ModuleManifest(
id="access",
name="Access",
version="0.1.4",
permissions=ACCESS_PERMISSIONS,
role_templates=ACCESS_ROLE_TEMPLATES,
migration_spec=MigrationSpec(module_id="access", metadata=AccessBase.metadata),
)
def get_manifest() -> ModuleManifest:
return manifest

View File

@@ -1,2 +0,0 @@
"""Permission definitions, evaluation, and registry helpers."""

View File

@@ -1,6 +0,0 @@
from __future__ import annotations
from govoplan_core.core.modules import PermissionDefinition, PermissionLevel, RoleTemplate
__all__ = ["PermissionDefinition", "PermissionLevel", "RoleTemplate"]

View File

@@ -1,52 +0,0 @@
from __future__ import annotations
from collections.abc import Iterable, Mapping
from govoplan_core.core.modules import PermissionDefinition
def scope_grants(granted: str, required: str, *, catalog: Mapping[str, PermissionDefinition] | None = None) -> bool:
if granted == required or granted == "*":
return True
if granted == "tenant:*":
if catalog is None:
return not required.startswith("system:")
definition = catalog.get(required)
return definition is not None and definition.level == "tenant"
if granted == "system:*":
if catalog is None:
return required.startswith("system:")
definition = catalog.get(required)
return definition is not None and definition.level == "system"
if granted.endswith(":*"):
return required.startswith(granted[:-1])
return False
def scopes_grant(
scopes: Iterable[str],
required: str,
*,
catalog: Mapping[str, PermissionDefinition] | None = None,
) -> bool:
return any(scope_grants(scope, required, catalog=catalog) for scope in scopes)
def expand_scopes(
scopes: Iterable[str],
*,
catalog: Mapping[str, PermissionDefinition],
include_unknown: bool = False,
) -> list[str]:
expanded: set[str] = set()
for scope in {str(scope) for scope in scopes if scope}:
matched = {candidate for candidate in catalog if scope_grants(scope, candidate, catalog=catalog)}
if matched:
expanded.update(matched)
if scope.endswith(":*") or scope in {"*", "tenant:*", "system:*"}:
expanded.add(scope)
continue
if include_unknown:
expanded.add(scope)
return sorted(expanded)

View File

@@ -1,38 +0,0 @@
from __future__ import annotations
from collections.abc import Iterable
from govoplan_core.access.permissions.evaluator import expand_scopes, scopes_grant
from govoplan_core.core.modules import PermissionDefinition
class PermissionRegistry:
def __init__(self, permissions: Iterable[PermissionDefinition] = ()) -> None:
self._catalog: dict[str, PermissionDefinition] = {}
for permission in permissions:
self.register(permission)
@property
def catalog(self) -> dict[str, PermissionDefinition]:
return dict(self._catalog)
def register(self, permission: PermissionDefinition) -> None:
if permission.scope in self._catalog:
raise ValueError(f"Duplicate permission scope: {permission.scope}")
self._catalog[permission.scope] = permission
def has(self, scope: str) -> bool:
return scope in self._catalog
def grants(self, scopes: Iterable[str], required: str) -> bool:
return scopes_grant(scopes, required, catalog=self._catalog)
def expand(self, scopes: Iterable[str], *, include_unknown: bool = False) -> list[str]:
return expand_scopes(scopes, catalog=self._catalog, include_unknown=include_unknown)
def validate_known(self, scopes: Iterable[str]) -> list[str]:
unknown = sorted(scope for scope in scopes if scope not in self._catalog and not scope.endswith(":*") and scope not in {"*", "tenant:*", "system:*"})
if unknown:
raise ValueError("Unknown permission scope(s): " + ", ".join(unknown))
return sorted(set(scopes))

View File

@@ -1,2 +0,0 @@
"""Tenant datastore routing abstractions."""

View File

@@ -1,54 +0,0 @@
from __future__ import annotations
from collections.abc import Generator
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Literal, Protocol
from sqlalchemy.orm import Session, sessionmaker
DatastoreMode = Literal["shared", "schema", "database"]
@dataclass(frozen=True, slots=True)
class TenantDatastoreRef:
tenant_id: str
mode: DatastoreMode = "shared"
datastore_id: str | None = None
schema_name: str | None = None
database_url_secret_ref: str | None = None
class TenantDatastore(Protocol):
ref: TenantDatastoreRef
@contextmanager
def session_for(self, module_id: str) -> Generator[Session, None, None]:
...
class DatastoreResolver(Protocol):
def for_tenant(self, tenant_id: str) -> TenantDatastore:
...
class SharedTenantDatastore:
def __init__(self, ref: TenantDatastoreRef, session_factory: sessionmaker[Session]) -> None:
self.ref = ref
self._session_factory = session_factory
@contextmanager
def session_for(self, module_id: str) -> Generator[Session, None, None]:
del module_id
with self._session_factory() as session:
yield session
class SharedDatastoreResolver:
def __init__(self, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
def for_tenant(self, tenant_id: str) -> TenantDatastore:
return SharedTenantDatastore(TenantDatastoreRef(tenant_id=tenant_id), self._session_factory)

View File

@@ -0,0 +1,21 @@
from __future__ import annotations
import re
_SLUG_RE = re.compile(r"[^a-z0-9]+")
class AdminConflictError(ValueError):
pass
class AdminValidationError(ValueError):
pass
def slugify(value: str) -> str:
slug = _SLUG_RE.sub("-", value.strip().casefold()).strip("-")
if not slug:
raise AdminValidationError("A slug must contain at least one letter or number.")
return slug[:100]

View File

@@ -1,328 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from sqlalchemy.orm import Session
from govoplan_core.admin.service import AdminConflictError, AdminValidationError, slugify
from govoplan_core.core.optional import reraise_unless_missing_package
from govoplan_core.db.models import (
ApiKey,
GovernanceTemplate,
GovernanceTemplateAssignment,
Group,
GroupRoleAssignment,
Role,
SystemSettings,
Tenant,
UserGroupMembership,
UserRoleAssignment,
)
from govoplan_core.security.permissions import validate_tenant_permissions
SYSTEM_SETTINGS_ID = "global"
TEMPLATE_KINDS = {"group", "role"}
ASSIGNMENT_MODES = {"available", "required"}
def _file_models():
try:
from govoplan_files.backend.db.models import FileAsset, FileFolder, FileShare
except ModuleNotFoundError as exc:
reraise_unless_missing_package(exc, "govoplan_files")
return None
return FileAsset, FileFolder, FileShare
@dataclass(frozen=True)
class EffectiveTenantGovernance:
allow_custom_groups: bool
allow_custom_roles: bool
allow_api_keys: bool
def get_system_settings(session: Session) -> SystemSettings:
item = session.get(SystemSettings, SYSTEM_SETTINGS_ID)
if item is None:
item = SystemSettings(id=SYSTEM_SETTINGS_ID)
session.add(item)
session.flush()
return item
def _narrowing_bool(system_allows: bool, tenant_override: bool | None) -> bool:
if not system_allows:
return False
return tenant_override is not False
def effective_tenant_governance(session: Session, tenant: Tenant) -> EffectiveTenantGovernance:
defaults = get_system_settings(session)
return EffectiveTenantGovernance(
allow_custom_groups=_narrowing_bool(defaults.allow_tenant_custom_groups, tenant.allow_custom_groups),
allow_custom_roles=_narrowing_bool(defaults.allow_tenant_custom_roles, tenant.allow_custom_roles),
allow_api_keys=_narrowing_bool(defaults.allow_tenant_api_keys, tenant.allow_api_keys),
)
def assert_tenant_governance_override_allowed(session: Session, *, field: str, value: bool | None) -> None:
if value is not True:
return
defaults = get_system_settings(session)
blocked = {
"allow_custom_groups": not defaults.allow_tenant_custom_groups,
"allow_custom_roles": not defaults.allow_tenant_custom_roles,
"allow_api_keys": not defaults.allow_tenant_api_keys,
}
if blocked.get(field):
raise AdminValidationError("Tenant governance cannot explicitly allow a capability denied by system settings.")
def validate_template(kind: str, permissions: list[str]) -> list[str]:
if kind not in TEMPLATE_KINDS:
raise AdminValidationError("Template kind must be group or role.")
if kind == "group":
if permissions:
raise AdminValidationError("Group templates do not contain permissions; assign roles to groups inside each tenant.")
return []
try:
return validate_tenant_permissions(permissions)
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
def create_template(
session: Session,
*,
kind: str,
slug: str,
name: str,
description: str | None,
permissions: list[str],
is_active: bool,
assignments: list[dict[str, str]],
) -> GovernanceTemplate:
normalized_slug = slugify(slug)
permissions = validate_template(kind, permissions)
exists = session.query(GovernanceTemplate).filter(
GovernanceTemplate.kind == kind,
GovernanceTemplate.slug == normalized_slug,
).first()
if exists:
raise AdminConflictError(f"A {kind} template with slug {normalized_slug!r} already exists.")
item = GovernanceTemplate(
kind=kind,
slug=normalized_slug,
name=name.strip(),
description=description,
permissions=permissions,
is_active=is_active,
)
session.add(item)
session.flush()
set_template_assignments(session, item, assignments)
return item
def update_template(
session: Session,
item: GovernanceTemplate,
*,
name: str,
description: str | None,
permissions: list[str],
is_active: bool,
assignments: list[dict[str, str]],
) -> GovernanceTemplate:
item.name = name.strip()
item.description = description
item.permissions = validate_template(item.kind, permissions)
item.is_active = is_active
set_template_assignments(session, item, assignments)
sync_template(session, item)
return item
def set_template_assignments(
session: Session,
item: GovernanceTemplate,
assignments: list[dict[str, str]],
) -> None:
desired: dict[str, str] = {}
for assignment in assignments:
tenant_id = assignment.get("tenant_id", "")
mode = assignment.get("mode", "available")
if mode not in ASSIGNMENT_MODES:
raise AdminValidationError("Template assignment mode must be available or required.")
tenant = session.get(Tenant, tenant_id)
if tenant is None:
raise AdminValidationError(f"Unknown tenant: {tenant_id}")
desired[tenant_id] = mode
existing = {
row.tenant_id: row
for row in session.query(GovernanceTemplateAssignment)
.filter(GovernanceTemplateAssignment.template_id == item.id)
.all()
}
for tenant_id, row in list(existing.items()):
if tenant_id in desired:
row.mode = desired[tenant_id]
continue
_remove_materialized_template(session, item, tenant_id)
session.delete(row)
for tenant_id, mode in desired.items():
if tenant_id not in existing:
session.add(GovernanceTemplateAssignment(template_id=item.id, tenant_id=tenant_id, mode=mode))
session.flush()
sync_template(session, item)
def sync_template(session: Session, item: GovernanceTemplate) -> None:
assignments = session.query(GovernanceTemplateAssignment).filter(
GovernanceTemplateAssignment.template_id == item.id
).all()
for assignment in assignments:
required = assignment.mode == "required"
if item.kind == "group":
group = session.query(Group).filter(
Group.tenant_id == assignment.tenant_id,
Group.system_template_id == item.id,
).first()
if group is None:
group = Group(
tenant_id=assignment.tenant_id,
slug=_available_slug(session, Group, assignment.tenant_id, item.slug),
name=item.name,
description=item.description,
is_active=item.is_active,
system_template_id=item.id,
system_required=required,
)
session.add(group)
else:
group.name = item.name
group.description = item.description
group.system_required = required
if required:
group.is_active = item.is_active
else:
role = session.query(Role).filter(
Role.tenant_id == assignment.tenant_id,
Role.system_template_id == item.id,
).first()
if role is None:
role = Role(
tenant_id=assignment.tenant_id,
slug=_available_slug(session, Role, assignment.tenant_id, item.slug),
name=item.name,
description=item.description,
permissions=item.permissions,
is_builtin=False,
is_assignable=item.is_active,
system_template_id=item.id,
system_required=required,
)
session.add(role)
else:
role.name = item.name
role.description = item.description
role.permissions = item.permissions
role.system_required = required
if required:
role.is_assignable = item.is_active
session.flush()
def delete_template(session: Session, item: GovernanceTemplate) -> None:
assignments = session.query(GovernanceTemplateAssignment).filter(
GovernanceTemplateAssignment.template_id == item.id
).all()
for assignment in assignments:
_remove_materialized_template(session, item, assignment.tenant_id)
session.delete(item)
def _remove_materialized_template(session: Session, item: GovernanceTemplate, tenant_id: str) -> None:
if item.kind == "group":
group = session.query(Group).filter(
Group.tenant_id == tenant_id,
Group.system_template_id == item.id,
).first()
if group is None:
return
membership_count = session.query(UserGroupMembership).filter(UserGroupMembership.group_id == group.id).count()
role_count = session.query(GroupRoleAssignment).filter(GroupRoleAssignment.group_id == group.id).count()
file_models = _file_models()
owned_asset_count = 0
owned_folder_count = 0
shared_asset_count = 0
if file_models is not None:
FileAsset, FileFolder, FileShare = file_models
owned_asset_count = session.query(FileAsset).filter(FileAsset.owner_group_id == group.id).count()
owned_folder_count = session.query(FileFolder).filter(FileFolder.owner_group_id == group.id).count()
shared_asset_count = session.query(FileShare).filter(
FileShare.target_type == "group", FileShare.target_id == group.id
).count()
if membership_count or role_count or owned_asset_count or owned_folder_count or shared_asset_count:
raise AdminConflictError(
f"Cannot remove {item.name!r} from the tenant while its managed group has members, roles, files, folders or shares."
)
session.delete(group)
return
role = session.query(Role).filter(
Role.tenant_id == tenant_id,
Role.system_template_id == item.id,
).first()
if role is None:
return
user_count = session.query(UserRoleAssignment).filter(UserRoleAssignment.role_id == role.id).count()
group_count = session.query(GroupRoleAssignment).filter(GroupRoleAssignment.role_id == role.id).count()
if user_count or group_count:
raise AdminConflictError(
f"Cannot remove {item.name!r} from the tenant while its managed role is assigned to users or groups."
)
session.delete(role)
def _available_slug(session: Session, model: type[Group] | type[Role], tenant_id: str, base: str) -> str:
candidate = base
suffix = 2
while session.query(model).filter(model.tenant_id == tenant_id, model.slug == candidate).first():
candidate = f"{base}-{suffix}"
suffix += 1
return candidate
def ensure_group_mutation_allowed(group: Group, *, requested_active: bool | None = None) -> None:
if group.system_template_id and group.system_required and requested_active is False:
raise AdminConflictError("This centrally required group cannot be deactivated by the tenant.")
def ensure_role_mutation_allowed(role: Role, *, requested_assignable: bool | None = None) -> None:
if role.system_template_id:
if role.system_required and requested_assignable is False:
raise AdminConflictError("This centrally required role cannot be made unavailable by the tenant.")
raise AdminConflictError("Centrally managed role definitions can only be changed in System administration.")
def assert_api_keys_allowed(session: Session, tenant: Tenant) -> None:
if not effective_tenant_governance(session, tenant).allow_api_keys:
raise AdminConflictError("API-key creation is disabled by system or tenant governance.")
def assert_custom_groups_allowed(session: Session, tenant: Tenant) -> None:
if not effective_tenant_governance(session, tenant).allow_custom_groups:
raise AdminConflictError("Custom tenant groups are disabled by system or tenant governance.")
def assert_custom_roles_allowed(session: Session, tenant: Tenant) -> None:
if not effective_tenant_governance(session, tenant).allow_custom_roles:
raise AdminConflictError("Custom tenant roles are disabled by system or tenant governance.")
def active_api_key_count(session: Session, tenant_id: str) -> int:
return session.query(ApiKey).filter(ApiKey.tenant_id == tenant_id, ApiKey.revoked_at.is_(None)).count()

View File

@@ -0,0 +1,23 @@
from __future__ import annotations
from typing import Any
from sqlalchemy import Boolean, JSON, String
from sqlalchemy.orm import Mapped, mapped_column
from govoplan_core.db.base import Base, TimestampMixin
class SystemSettings(Base, TimestampMixin):
__tablename__ = "core_system_settings"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default="global")
default_locale: Mapped[str] = mapped_column(String(20), default="en", nullable=False)
allow_tenant_custom_groups: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_custom_roles: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_api_keys: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
__all__ = ["SystemSettings"]

View File

@@ -1,609 +0,0 @@
from __future__ import annotations
import re
import secrets
import string
from collections.abc import Iterable
from dataclasses import dataclass
from sqlalchemy import func
from sqlalchemy.orm import Session
from govoplan_core.core.optional import reraise_unless_missing_package
from govoplan_core.db.models import (
Account,
ApiKey,
Group,
GroupRoleAssignment,
Role,
SystemRoleAssignment,
Tenant,
User,
UserGroupMembership,
UserRoleAssignment,
)
from govoplan_core.security.passwords import hash_password
from govoplan_core.security.permissions import (
DEFAULT_SYSTEM_ROLES,
DEFAULT_TENANT_ROLES,
normalize_email,
scopes_grant,
validate_system_permissions,
validate_tenant_permissions,
)
_SLUG_RE = re.compile(r"[^a-z0-9]+")
_TEMP_PASSWORD_ALPHABET = string.ascii_letters + string.digits + "-_!@#"
class AdminConflictError(ValueError):
pass
class AdminValidationError(ValueError):
pass
@dataclass(slots=True)
class MembershipCreationResult:
user: User
account: Account
temporary_password: str | None = None
account_created: bool = False
def slugify(value: str) -> str:
slug = _SLUG_RE.sub("-", value.strip().casefold()).strip("-")
if not slug:
raise AdminValidationError("A slug must contain at least one letter or number.")
return slug[:100]
def generate_temporary_password(length: int = 20) -> str:
return "".join(secrets.choice(_TEMP_PASSWORD_ALPHABET) for _ in range(length))
def ensure_default_roles(session: Session, tenant: Tenant | None = None) -> dict[str, Role]:
definitions = DEFAULT_TENANT_ROLES if tenant is not None else DEFAULT_SYSTEM_ROLES
roles: dict[str, Role] = {}
for slug, definition in definitions.items():
query = session.query(Role).filter(Role.slug == slug)
query = query.filter(Role.tenant_id == tenant.id) if tenant is not None else query.filter(Role.tenant_id.is_(None))
role = query.one_or_none()
if role is None:
role = Role(
tenant_id=tenant.id if tenant is not None else None,
slug=slug,
name=str(definition["name"]),
description=str(definition.get("description") or "") or None,
permissions=list(definition["permissions"]),
is_builtin=bool(definition.get("is_builtin", True)),
is_assignable=bool(definition.get("is_assignable", True)),
)
session.add(role)
session.flush()
else:
# Tenant built-ins and explicitly managed system roles remain
# code-defined. Seeded, non-protected system roles are only created
# here and can subsequently be administered in the System roles UI.
managed = tenant is not None or bool(definition.get("managed", False))
if managed:
role.name = str(definition["name"])
role.description = str(definition.get("description") or "") or None
role.permissions = list(definition["permissions"])
role.is_builtin = bool(definition.get("is_builtin", True))
role.is_assignable = bool(definition.get("is_assignable", True))
session.add(role)
roles[slug] = role
return roles
def get_or_create_account(
session: Session,
*,
email: str,
display_name: str | None = None,
password: str | None = None,
password_reset_required: bool = False,
) -> tuple[Account, bool, str | None]:
normalized = normalize_email(email)
if not normalized or "@" not in normalized:
raise AdminValidationError("Enter a valid email address.")
account = session.query(Account).filter(Account.normalized_email == normalized).one_or_none()
if account is not None:
if not account.is_active:
raise AdminConflictError(
"The global account is disabled. A system administrator must reactivate it before adding a tenant membership."
)
return account, False, None
temporary_password = password or generate_temporary_password()
account = Account(
email=email.strip(),
normalized_email=normalized,
display_name=display_name.strip() if display_name else None,
is_active=True,
auth_provider="local",
password_hash=hash_password(temporary_password),
password_reset_required=password_reset_required or password is None,
)
session.add(account)
session.flush()
return account, True, temporary_password
def create_membership(
session: Session,
*,
tenant: Tenant,
email: str,
display_name: str | None = None,
password: str | None = None,
password_reset_required: bool = False,
is_active: bool = True,
) -> MembershipCreationResult:
account, account_created, temporary_password = get_or_create_account(
session,
email=email,
display_name=display_name,
password=password,
password_reset_required=password_reset_required,
)
existing = (
session.query(User)
.filter(User.tenant_id == tenant.id, User.account_id == account.id)
.one_or_none()
)
if existing is not None:
raise AdminConflictError("This account already belongs to the tenant.")
user = User(
tenant_id=tenant.id,
account_id=account.id,
email=account.email,
display_name=display_name.strip() if display_name else account.display_name,
is_active=is_active,
is_tenant_admin=False,
auth_provider=account.auth_provider,
password_hash=account.password_hash,
)
session.add(user)
session.flush()
return MembershipCreationResult(
user=user,
account=account,
temporary_password=temporary_password if account_created else None,
account_created=account_created,
)
def set_user_groups(session: Session, *, user: User, group_ids: Iterable[str]) -> None:
ids = sorted(set(group_ids))
groups = (
session.query(Group)
.filter(Group.tenant_id == user.tenant_id, Group.id.in_(ids))
.all()
if ids else []
)
if len(groups) != len(ids):
raise AdminValidationError("One or more selected groups do not belong to the tenant.")
session.query(UserGroupMembership).filter(
UserGroupMembership.tenant_id == user.tenant_id,
UserGroupMembership.user_id == user.id,
).delete(synchronize_session=False)
for group in groups:
session.add(UserGroupMembership(tenant_id=user.tenant_id, user_id=user.id, group_id=group.id))
session.flush()
def set_user_roles(session: Session, *, user: User, role_ids: Iterable[str]) -> None:
ids = sorted(set(role_ids))
roles = (
session.query(Role)
.filter(Role.tenant_id == user.tenant_id, Role.id.in_(ids), Role.is_assignable.is_(True))
.all()
if ids else []
)
if len(roles) != len(ids):
raise AdminValidationError("One or more selected roles are invalid or not assignable.")
session.query(UserRoleAssignment).filter(
UserRoleAssignment.tenant_id == user.tenant_id,
UserRoleAssignment.user_id == user.id,
).delete(synchronize_session=False)
for role in roles:
session.add(UserRoleAssignment(tenant_id=user.tenant_id, user_id=user.id, role_id=role.id))
session.flush()
def set_group_members(session: Session, *, group: Group, user_ids: Iterable[str]) -> None:
ids = sorted(set(user_ids))
users = (
session.query(User)
.filter(User.tenant_id == group.tenant_id, User.id.in_(ids))
.all()
if ids else []
)
if len(users) != len(ids):
raise AdminValidationError("One or more selected users do not belong to the tenant.")
session.query(UserGroupMembership).filter(
UserGroupMembership.tenant_id == group.tenant_id,
UserGroupMembership.group_id == group.id,
).delete(synchronize_session=False)
for user in users:
session.add(UserGroupMembership(tenant_id=group.tenant_id, user_id=user.id, group_id=group.id))
session.flush()
def set_group_roles(session: Session, *, group: Group, role_ids: Iterable[str]) -> None:
ids = sorted(set(role_ids))
roles = (
session.query(Role)
.filter(Role.tenant_id == group.tenant_id, Role.id.in_(ids), Role.is_assignable.is_(True))
.all()
if ids else []
)
if len(roles) != len(ids):
raise AdminValidationError("One or more selected roles are invalid or not assignable.")
session.query(GroupRoleAssignment).filter(
GroupRoleAssignment.tenant_id == group.tenant_id,
GroupRoleAssignment.group_id == group.id,
).delete(synchronize_session=False)
for role in roles:
session.add(GroupRoleAssignment(tenant_id=group.tenant_id, group_id=group.id, role_id=role.id))
session.flush()
def create_custom_role(
session: Session,
*,
tenant: Tenant,
slug: str,
name: str,
description: str | None,
permissions: Iterable[str],
) -> Role:
normalized_slug = slugify(slug)
if session.query(Role).filter(Role.tenant_id == tenant.id, Role.slug == normalized_slug).count():
raise AdminConflictError("A role with this slug already exists in the tenant.")
try:
validated = validate_tenant_permissions(permissions)
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
role = Role(
tenant_id=tenant.id,
slug=normalized_slug,
name=name.strip(),
description=description.strip() if description else None,
permissions=validated,
is_builtin=False,
is_assignable=True,
)
session.add(role)
session.flush()
return role
def update_custom_role(
session: Session,
*,
role: Role,
name: str,
description: str | None,
permissions: Iterable[str],
is_assignable: bool,
) -> None:
if role.is_builtin:
raise AdminConflictError("Built-in roles are managed by the application and cannot be edited.")
try:
validated = validate_tenant_permissions(permissions)
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
role.name = name.strip()
role.description = description.strip() if description else None
role.permissions = validated
role.is_assignable = is_assignable
session.add(role)
session.flush()
def delete_custom_role(session: Session, role: Role) -> None:
if role.is_builtin:
raise AdminConflictError("Built-in roles cannot be deleted.")
assigned = session.query(UserRoleAssignment).filter(UserRoleAssignment.role_id == role.id).count()
assigned += session.query(GroupRoleAssignment).filter(GroupRoleAssignment.role_id == role.id).count()
if assigned:
raise AdminConflictError("Remove all user and group assignments before deleting this role.")
session.delete(role)
session.flush()
def create_system_role(
session: Session,
*,
slug: str,
name: str,
description: str | None,
permissions: Iterable[str],
) -> Role:
normalized_slug = slugify(slug)
if session.query(Role).filter(Role.tenant_id.is_(None), Role.slug == normalized_slug).count():
raise AdminConflictError("A system role with this slug already exists.")
try:
validated = validate_system_permissions(permissions)
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
if "system:*" in validated:
raise AdminValidationError("Only the protected System owner role may contain system:*.")
role = Role(
tenant_id=None,
slug=normalized_slug,
name=name.strip(),
description=description.strip() if description else None,
permissions=validated,
is_builtin=False,
is_assignable=True,
)
session.add(role)
session.flush()
return role
def update_system_role(
session: Session,
*,
role: Role,
name: str,
description: str | None,
permissions: Iterable[str],
is_assignable: bool,
) -> None:
if role.tenant_id is not None:
raise AdminValidationError("This is not a system role.")
if role.slug == "system_owner":
raise AdminConflictError("The protected System owner role cannot be edited.")
try:
validated = validate_system_permissions(permissions)
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
if "system:*" in validated:
raise AdminValidationError("Only the protected System owner role may contain system:*.")
role.name = name.strip()
role.description = description.strip() if description else None
role.permissions = validated
role.is_assignable = is_assignable
role.is_builtin = False
session.add(role)
session.flush()
def delete_system_role(session: Session, role: Role) -> None:
if role.tenant_id is not None:
raise AdminValidationError("This is not a system role.")
if role.slug == "system_owner":
raise AdminConflictError("The protected System owner role cannot be deleted.")
assigned = session.query(SystemRoleAssignment).filter(SystemRoleAssignment.role_id == role.id).count()
if assigned:
raise AdminConflictError("Remove all account assignments before deleting this system role.")
session.delete(role)
session.flush()
def assert_can_delegate_system_permissions(actor_scopes: Iterable[str], permissions: Iterable[str]) -> None:
"""Prevent a system administrator from defining stronger roles than they hold."""
from govoplan_core.security.permissions import delegateable_system_scopes
requested = set(validate_system_permissions(permissions))
if "system:*" in requested:
if not scopes_grant(actor_scopes, "system:*"):
raise AdminValidationError("Only a System owner may delegate full system access.")
return
allowed = delegateable_system_scopes(actor_scopes)
missing = sorted(scope for scope in requested if scope not in allowed)
if missing:
raise AdminValidationError("You cannot delegate system permissions you do not currently hold: " + ", ".join(missing))
def assert_can_delegate_system_roles(
session: Session,
*,
actor_scopes: Iterable[str],
role_ids: Iterable[str],
) -> None:
ids = sorted(set(role_ids))
if not ids:
return
roles = session.query(Role).filter(Role.tenant_id.is_(None), Role.id.in_(ids)).all()
if len(roles) != len(ids):
raise AdminValidationError("One or more selected system roles are invalid.")
for role in roles:
assert_can_delegate_system_permissions(actor_scopes, role.permissions or [])
def set_system_roles(session: Session, *, account: Account, role_ids: Iterable[str]) -> None:
ids = sorted(set(role_ids))
roles = (
session.query(Role)
.filter(Role.tenant_id.is_(None), Role.id.in_(ids), Role.is_assignable.is_(True))
.all()
if ids else []
)
if len(roles) != len(ids):
raise AdminValidationError("One or more system roles are invalid or not assignable.")
for role in roles:
try:
validate_system_permissions(role.permissions or [])
except ValueError as exc:
raise AdminValidationError(str(exc)) from exc
session.query(SystemRoleAssignment).filter(SystemRoleAssignment.account_id == account.id).delete(synchronize_session=False)
for role in roles:
session.add(SystemRoleAssignment(account_id=account.id, role_id=role.id))
session.flush()
assert_system_owner_exists(session)
def account_has_system_scope(session: Session, account: Account, required: str) -> bool:
roles = (
session.query(Role)
.join(SystemRoleAssignment, SystemRoleAssignment.role_id == Role.id)
.filter(SystemRoleAssignment.account_id == account.id, Role.tenant_id.is_(None))
.all()
)
return any(scopes_grant(role.permissions or [], required) for role in roles)
def tenant_owner_user_ids(session: Session, tenant_id: str) -> set[str]:
"""Return active tenant memberships that currently satisfy the operational-owner invariant."""
users = (
session.query(User)
.join(Account, Account.id == User.account_id)
.filter(
User.tenant_id == tenant_id,
User.is_active.is_(True),
Account.is_active.is_(True),
)
.all()
)
owner_ids: set[str] = set()
user_ids = [user.id for user in users]
if not user_ids:
return owner_ids
roles_by_user_id: dict[str, list[Role]] = {user_id: [] for user_id in user_ids}
direct_role_rows = (
session.query(UserRoleAssignment.user_id, Role)
.join(Role, UserRoleAssignment.role_id == Role.id)
.filter(
UserRoleAssignment.tenant_id == tenant_id,
UserRoleAssignment.user_id.in_(user_ids),
Role.tenant_id == tenant_id,
)
.all()
)
for user_id, role in direct_role_rows:
roles_by_user_id.setdefault(user_id, []).append(role)
group_role_rows = (
session.query(UserGroupMembership.user_id, Role)
.join(GroupRoleAssignment, UserGroupMembership.group_id == GroupRoleAssignment.group_id)
.join(Role, GroupRoleAssignment.role_id == Role.id)
.join(Group, Group.id == UserGroupMembership.group_id)
.filter(
UserGroupMembership.tenant_id == tenant_id,
UserGroupMembership.user_id.in_(user_ids),
Group.is_active.is_(True),
Role.tenant_id == tenant_id,
)
.all()
)
for user_id, role in group_role_rows:
roles_by_user_id.setdefault(user_id, []).append(role)
for user in users:
effective = [
scope
for role in roles_by_user_id.get(user.id, [])
for scope in (role.permissions or [])
]
if scopes_grant(effective, "admin:roles:write") and scopes_grant(effective, "campaign:send"):
owner_ids.add(user.id)
return owner_ids
def tenant_has_owner(session: Session, tenant_id: str) -> bool:
return bool(tenant_owner_user_ids(session, tenant_id))
def assert_tenant_owner_exists(session: Session, tenant_id: str) -> None:
session.flush()
tenant = session.get(Tenant, tenant_id)
if tenant is not None and tenant.is_active and not tenant_has_owner(session, tenant_id):
raise AdminConflictError("An active tenant must retain at least one active owner or administrator with full operational access.")
def assert_system_owner_exists(session: Session) -> None:
"""Keep one active assignment of the protected System owner role.
Other roles may hold broad system permissions, but they are intentionally
not substitutes for the protected break-glass owner identity.
"""
session.flush()
owner_role = (
session.query(Role)
.filter(Role.tenant_id.is_(None), Role.slug == "system_owner")
.one_or_none()
)
if owner_role is None:
raise AdminConflictError("The protected System owner role is missing.")
count = (
session.query(func.count(SystemRoleAssignment.id))
.join(Account, Account.id == SystemRoleAssignment.account_id)
.filter(SystemRoleAssignment.role_id == owner_role.id, Account.is_active.is_(True))
.scalar()
)
if not count:
raise AdminConflictError("At least one active account must retain the protected System owner role.")
def role_assignment_counts(session: Session, role_id: str) -> tuple[int, int]:
return (
session.query(UserRoleAssignment).filter(UserRoleAssignment.role_id == role_id).count(),
session.query(GroupRoleAssignment).filter(GroupRoleAssignment.role_id == role_id).count(),
)
def _optional_model(module_name: str, model_name: str):
try:
module = __import__(module_name, fromlist=[model_name])
except ModuleNotFoundError as exc:
reraise_unless_missing_package(exc, module_name.split(".", 1)[0])
return None
return getattr(module, model_name)
def tenant_counts(session: Session, tenant_id: str) -> dict[str, int]:
Campaign = _optional_model("govoplan_campaign.backend.db.models", "Campaign")
FileAsset = _optional_model("govoplan_files.backend.db.models", "FileAsset")
return {
"users": session.query(User).filter(User.tenant_id == tenant_id).count(),
"active_users": session.query(User).filter(User.tenant_id == tenant_id, User.is_active.is_(True)).count(),
"groups": session.query(Group).filter(Group.tenant_id == tenant_id).count(),
"campaigns": session.query(Campaign).filter(Campaign.tenant_id == tenant_id).count() if Campaign is not None else 0,
"files": session.query(FileAsset).filter(FileAsset.tenant_id == tenant_id).count() if FileAsset is not None else 0,
"api_keys": session.query(ApiKey).filter(ApiKey.tenant_id == tenant_id, ApiKey.revoked_at.is_(None)).count(),
}
def assert_can_delegate_tenant_permissions(actor_scopes: Iterable[str], permissions: Iterable[str]) -> None:
"""Prevent administrators from defining or assigning roles beyond their own effective tenant powers."""
from govoplan_core.security.permissions import delegateable_tenant_scopes
requested = set(validate_tenant_permissions(permissions))
if "tenant:*" in requested:
# Only a tenant owner-equivalent can delegate the wildcard.
if not scopes_grant(actor_scopes, "tenant:*"):
raise AdminValidationError("You cannot delegate full tenant access unless you already have it.")
return
allowed = delegateable_tenant_scopes(actor_scopes)
missing = sorted(scope for scope in requested if scope not in allowed)
if missing:
raise AdminValidationError("You cannot delegate permissions you do not currently hold: " + ", ".join(missing))
def assert_can_delegate_roles(session: Session, *, actor_scopes: Iterable[str], role_ids: Iterable[str], tenant_id: str) -> None:
ids = sorted(set(role_ids))
if not ids:
return
roles = session.query(Role).filter(Role.tenant_id == tenant_id, Role.id.in_(ids)).all()
if len(roles) != len(ids):
raise AdminValidationError("One or more selected roles do not belong to the tenant.")
for role in roles:
assert_can_delegate_tenant_permissions(actor_scopes, role.permissions or [])

View File

@@ -0,0 +1,16 @@
from __future__ import annotations
from sqlalchemy.orm import Session
from govoplan_core.admin.models import SystemSettings
SYSTEM_SETTINGS_ID = "global"
def get_system_settings(session: Session) -> SystemSettings:
item = session.get(SystemSettings, SYSTEM_SETTINGS_ID)
if item is None:
item = SystemSettings(id=SYSTEM_SETTINGS_ID)
session.add(item)
session.flush()
return item

File diff suppressed because it is too large Load Diff

View File

@@ -1,510 +0,0 @@
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator
RETENTION_DAY_KEYS = (
"raw_campaign_json_retention_days",
"generated_eml_retention_days",
"stored_report_detail_retention_days",
"mock_mailbox_retention_days",
"audit_detail_retention_days",
)
RETENTION_POLICY_FIELD_KEYS = (
"store_raw_campaign_json",
*RETENTION_DAY_KEYS,
"audit_detail_level",
)
def default_allow_lower_level_limits() -> dict[str, bool]:
return {key: True for key in RETENTION_POLICY_FIELD_KEYS}
def normalize_allow_lower_level_limits(value: Any, *, fill_defaults: bool) -> dict[str, bool] | None:
if value in (None, ""):
return default_allow_lower_level_limits() if fill_defaults else None
if not isinstance(value, dict):
raise ValueError("allow_lower_level_limits must be an object")
normalized = default_allow_lower_level_limits() if fill_defaults else {}
for key, allowed in value.items():
clean_key = str(key)
if clean_key not in RETENTION_POLICY_FIELD_KEYS:
raise ValueError(f"Unknown retention policy field: {clean_key}")
normalized[clean_key] = bool(allowed)
return normalized
class PermissionItem(BaseModel):
scope: str
label: str
description: str
category: str
level: Literal["tenant", "system"]
class PermissionCatalogResponse(BaseModel):
permissions: list[PermissionItem]
class AdminOverviewResponse(BaseModel):
active_tenant_id: str
active_tenant_name: str
tenant_count: int | None = None
system_account_count: int | None = None
system_group_template_count: int | None = None
system_role_template_count: int | None = None
user_count: int
active_user_count: int
group_count: int
role_count: int
active_api_key_count: int
capabilities: list[str] = Field(default_factory=list)
class TenantAdminItem(BaseModel):
id: str
slug: str = Field(min_length=1, max_length=100)
name: str = Field(min_length=1, max_length=255)
description: str | None = None
default_locale: str = Field(default="en", min_length=1, max_length=20)
settings: dict[str, Any] = Field(default_factory=dict)
allow_custom_groups: bool | None = None
allow_custom_roles: bool | None = None
allow_api_keys: bool | None = None
effective_governance: dict[str, bool] = Field(default_factory=dict)
is_active: bool
counts: dict[str, int] = Field(default_factory=dict)
created_at: datetime
updated_at: datetime
class TenantListResponse(BaseModel):
tenants: list[TenantAdminItem]
class TenantOwnerCandidate(BaseModel):
account_id: str
email: str
display_name: str | None = None
class TenantOwnerCandidateListResponse(BaseModel):
accounts: list[TenantOwnerCandidate]
class TenantCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
slug: str
name: str
owner_account_id: str | None = None
description: str | None = None
default_locale: str = "en"
settings: dict[str, Any] = Field(default_factory=dict)
allow_custom_groups: bool | None = None
allow_custom_roles: bool | None = None
allow_api_keys: bool | None = None
class TenantUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str | None = Field(default=None, min_length=1, max_length=255)
description: str | None = None
default_locale: str | None = Field(default=None, min_length=1, max_length=20)
settings: dict[str, Any] | None = None
allow_custom_groups: bool | None = None
allow_custom_roles: bool | None = None
allow_api_keys: bool | None = None
is_active: bool | None = None
class TenantSettingsItem(BaseModel):
id: str
slug: str
name: str
default_locale: str = Field(default="en", min_length=1, max_length=20)
settings: dict[str, Any] = Field(default_factory=dict)
class TenantSettingsUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
default_locale: str = Field(min_length=1, max_length=20)
class RoleSummary(BaseModel):
id: str
slug: str = Field(min_length=1, max_length=100)
name: str = Field(min_length=1, max_length=255)
description: str | None = None
permissions: list[str] = Field(default_factory=list)
effective_permission_count: int = 0
is_builtin: bool = False
is_assignable: bool = True
user_assignments: int = 0
group_assignments: int = 0
level: Literal["tenant", "system"] = "tenant"
system_template_id: str | None = None
system_required: bool = False
class GroupSummary(BaseModel):
id: str
slug: str = Field(min_length=1, max_length=100)
name: str = Field(min_length=1, max_length=255)
description: str | None = None
is_active: bool = True
member_count: int = 0
member_ids: list[str] = Field(default_factory=list)
roles: list[RoleSummary] = Field(default_factory=list)
created_at: datetime
updated_at: datetime
system_template_id: str | None = None
system_required: bool = False
class UserAdminItem(BaseModel):
id: str
account_id: str
tenant_id: str
email: str = Field(min_length=3, max_length=320)
display_name: str | None = Field(default=None, max_length=255)
is_active: bool
account_is_active: bool
password_reset_required: bool = False
last_login_at: datetime | None = None
groups: list[GroupSummary] = Field(default_factory=list)
roles: list[RoleSummary] = Field(default_factory=list)
effective_scopes: list[str] = Field(default_factory=list)
is_owner: bool = False
is_last_active_owner: bool = False
created_at: datetime
updated_at: datetime
class UserListResponse(BaseModel):
users: list[UserAdminItem]
class UserCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
email: str
display_name: str | None = None
password: str | None = Field(default=None, min_length=10)
password_reset_required: bool = True
is_active: bool = True
group_ids: list[str] = Field(default_factory=list)
role_ids: list[str] = Field(default_factory=list)
class UserCreateResponse(BaseModel):
user: UserAdminItem
account_created: bool
temporary_password: str | None = None
class UserUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
display_name: str | None = Field(default=None, max_length=255)
is_active: bool | None = None
group_ids: list[str] | None = None
role_ids: list[str] | None = None
class GroupListResponse(BaseModel):
groups: list[GroupSummary]
class GroupCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
slug: str
name: str
description: str | None = None
is_active: bool = True
member_ids: list[str] = Field(default_factory=list)
role_ids: list[str] = Field(default_factory=list)
class GroupUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str | None = Field(default=None, min_length=1, max_length=255)
description: str | None = None
is_active: bool | None = None
member_ids: list[str] | None = None
role_ids: list[str] | None = None
class RoleListResponse(BaseModel):
roles: list[RoleSummary]
class RoleCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
slug: str
name: str = Field(min_length=1, max_length=255)
description: str | None = None
permissions: list[str] = Field(default_factory=list)
class RoleUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str
description: str | None = None
permissions: list[str] = Field(default_factory=list)
is_assignable: bool = True
class SystemAccountItem(BaseModel):
account_id: str
email: str
display_name: str | None = None
is_active: bool
memberships: list[dict[str, Any]] = Field(default_factory=list)
roles: list[RoleSummary] = Field(default_factory=list)
last_login_at: datetime | None = None
class SystemAccountListResponse(BaseModel):
accounts: list[SystemAccountItem]
roles: list[RoleSummary]
class SystemAccountUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
display_name: str | None = Field(default=None, max_length=255)
is_active: bool | None = None
role_ids: list[str] | None = None
class SystemAccountRolesUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
role_ids: list[str] = Field(default_factory=list)
class SystemMembershipUpdate(BaseModel):
model_config = ConfigDict(extra="forbid")
tenant_id: str
is_active: bool = True
role_ids: list[str] = Field(default_factory=list)
group_ids: list[str] = Field(default_factory=list)
class SystemAccountMembershipsUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
memberships: list[SystemMembershipUpdate] = Field(default_factory=list)
class SystemAccountCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
email: str
display_name: str | None = None
password: str | None = Field(default=None, min_length=10)
password_reset_required: bool = True
is_active: bool = True
role_ids: list[str] = Field(default_factory=list)
memberships: list[SystemMembershipUpdate] = Field(default_factory=list)
class SystemAccountCreateResponse(BaseModel):
account: SystemAccountItem
temporary_password: str | None = None
class PolicySourceStepItem(BaseModel):
scope_type: str
scope_id: str | None = None
label: str
applied_fields: list[str] = Field(default_factory=list)
policy: dict[str, Any] = Field(default_factory=dict)
class PrivacyRetentionPolicyItem(BaseModel):
model_config = ConfigDict(extra="forbid")
store_raw_campaign_json: bool = True
raw_campaign_json_retention_days: int | None = Field(default=None, ge=0)
generated_eml_retention_days: int | None = Field(default=None, ge=0)
stored_report_detail_retention_days: int | None = Field(default=None, ge=0)
mock_mailbox_retention_days: int | None = Field(default=None, ge=0)
audit_detail_retention_days: int | None = Field(default=None, ge=0)
audit_detail_level: Literal["full", "redacted", "minimal"] = "full"
allow_lower_level_limits: dict[str, bool] = Field(default_factory=default_allow_lower_level_limits)
@field_validator("allow_lower_level_limits", mode="before")
@classmethod
def _normalize_allow_lower_level_limits(cls, value: Any) -> Any:
return normalize_allow_lower_level_limits(value, fill_defaults=True)
class PrivacyRetentionPolicyPatchItem(BaseModel):
model_config = ConfigDict(extra="forbid")
store_raw_campaign_json: bool | None = None
raw_campaign_json_retention_days: int | None = Field(default=None, ge=0)
generated_eml_retention_days: int | None = Field(default=None, ge=0)
stored_report_detail_retention_days: int | None = Field(default=None, ge=0)
mock_mailbox_retention_days: int | None = Field(default=None, ge=0)
audit_detail_retention_days: int | None = Field(default=None, ge=0)
audit_detail_level: Literal["full", "redacted", "minimal"] | None = None
allow_lower_level_limits: dict[str, bool] | None = None
@field_validator("allow_lower_level_limits", mode="before")
@classmethod
def _normalize_allow_lower_level_limits(cls, value: Any) -> Any:
return normalize_allow_lower_level_limits(value, fill_defaults=False)
class PrivacyRetentionPolicyScopeRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
policy: PrivacyRetentionPolicyPatchItem = Field(default_factory=PrivacyRetentionPolicyPatchItem)
class PrivacyRetentionPolicyScopeResponse(BaseModel):
scope_type: Literal["system", "tenant", "user", "group", "campaign"]
scope_id: str | None = None
policy: dict[str, Any]
effective_policy: PrivacyRetentionPolicyItem
parent_policy: PrivacyRetentionPolicyItem | None = None
effective_policy_sources: list[PolicySourceStepItem] = Field(default_factory=list)
parent_policy_sources: list[PolicySourceStepItem] = Field(default_factory=list)
class RetentionRunRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
dry_run: bool = True
class RetentionRunResponse(BaseModel):
result: dict[str, Any]
class SystemSettingsItem(BaseModel):
default_locale: str = "en"
allow_tenant_custom_groups: bool = True
allow_tenant_custom_roles: bool = True
allow_tenant_api_keys: bool = True
privacy_retention_policy: PrivacyRetentionPolicyItem = Field(default_factory=PrivacyRetentionPolicyItem)
settings: dict[str, Any] = Field(default_factory=dict)
class SystemSettingsUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
default_locale: str = Field(min_length=1, max_length=20)
allow_tenant_custom_groups: bool
allow_tenant_custom_roles: bool
allow_tenant_api_keys: bool
privacy_retention_policy: PrivacyRetentionPolicyItem | None = None
class GovernanceAssignment(BaseModel):
tenant_id: str
mode: Literal["available", "required"] = "available"
class GovernanceTemplateItem(BaseModel):
id: str
kind: Literal["group", "role"]
slug: str
name: str
description: str | None = None
permissions: list[str] = Field(default_factory=list)
effective_permission_count: int = 0
is_active: bool = True
assignments: list[GovernanceAssignment] = Field(default_factory=list)
created_at: datetime
updated_at: datetime
class GovernanceTemplateListResponse(BaseModel):
templates: list[GovernanceTemplateItem]
class GovernanceTemplateCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
kind: Literal["group", "role"]
slug: str
name: str = Field(min_length=1, max_length=255)
description: str | None = None
permissions: list[str] = Field(default_factory=list)
is_active: bool = True
assignments: list[GovernanceAssignment] = Field(default_factory=list)
class GovernanceTemplateUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str = Field(min_length=1, max_length=255)
description: str | None = None
permissions: list[str] = Field(default_factory=list)
is_active: bool = True
assignments: list[GovernanceAssignment] = Field(default_factory=list)
class ApiKeyAdminItem(BaseModel):
id: str
user_id: str
user_email: str
name: str
prefix: str
scopes: list[str] = Field(default_factory=list)
expires_at: datetime | None = None
last_used_at: datetime | None = None
revoked_at: datetime | None = None
created_at: datetime
class ApiKeyListResponse(BaseModel):
api_keys: list[ApiKeyAdminItem]
class AdminApiKeyCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str = Field(min_length=1, max_length=255)
user_id: str | None = None
scopes: list[str] = Field(default_factory=list)
expires_at: datetime | None = None
class AdminApiKeyCreateResponse(ApiKeyAdminItem):
secret: str
class AuditAdminItem(BaseModel):
id: str
scope: Literal["tenant", "system"] = "tenant"
tenant_id: str | None = None
actor_email: str | None = None
action: str
object_type: str | None = None
object_id: str | None = None
details: dict[str, Any] = Field(default_factory=dict)
created_at: datetime
class AuditAdminListResponse(BaseModel):
items: list[AuditAdminItem]
total: int
page: int = 1
page_size: int = 100
pages: int = 1

View File

@@ -1,32 +0,0 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from govoplan_core.api.v1.schemas import AuditLogItemResponse, AuditLogListResponse
from govoplan_core.auth.dependencies import ApiPrincipal, require_scope
from govoplan_core.db.models import AuditLog
from govoplan_core.db.session import get_session
router = APIRouter(prefix="/audit", tags=["audit"])
@router.get("", response_model=AuditLogListResponse)
def list_audit_log(
limit: int = Query(default=100, ge=1, le=500),
offset: int = Query(default=0, ge=0),
action: str | None = None,
object_type: str | None = None,
object_id: str | None = None,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(require_scope("audit:read")),
):
query = session.query(AuditLog).filter(AuditLog.tenant_id == principal.tenant_id)
if action:
query = query.filter(AuditLog.action == action)
if object_type:
query = query.filter(AuditLog.object_type == object_type)
if object_id:
query = query.filter(AuditLog.object_id == object_id)
items = query.order_by(AuditLog.created_at.desc()).offset(offset).limit(limit).all()
return AuditLogListResponse(items=[AuditLogItemResponse.model_validate(item) for item in items])

View File

@@ -1,293 +0,0 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
from sqlalchemy.orm import Session
from govoplan_core.api.v1.schemas import (
GroupInfo,
LoginRequest,
LoginResponse,
MeResponse,
ProfileUpdateRequest,
RoleInfo,
SwitchTenantRequest,
TenantInfo,
TenantMembershipInfo,
UserInfo,
)
from govoplan_core.auth.dependencies import ApiPrincipal, get_api_principal
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.db.models import Account, Tenant, User
from govoplan_core.db.session import get_session
from govoplan_core.security.passwords import verify_password
from govoplan_core.security.permissions import normalize_email
from govoplan_core.settings import settings
from govoplan_core.security.sessions import (
collect_system_roles,
collect_tenant_memberships,
collect_user_groups,
collect_user_roles,
collect_user_scopes,
create_auth_session,
switch_auth_session_tenant,
)
from govoplan_core.security.time import utc_now
router = APIRouter(prefix="/auth", tags=["auth"])
def _cookie_samesite() -> str:
value = settings.auth_cookie_samesite.lower().strip()
if value not in {"lax", "strict", "none"}:
return "lax"
if value == "none" and not settings.auth_cookie_secure:
return "lax"
return value
def _set_auth_cookies(response: Response, created) -> None:
max_age = max(0, int((created.model.expires_at - utc_now()).total_seconds()))
common = {
"secure": settings.auth_cookie_secure,
"samesite": _cookie_samesite(),
"max_age": max_age,
"path": "/",
}
if settings.auth_cookie_domain:
common["domain"] = settings.auth_cookie_domain
response.set_cookie(settings.auth_session_cookie_name, created.token, httponly=True, **common)
response.set_cookie(settings.auth_csrf_cookie_name, created.csrf_token, httponly=False, **common)
def _clear_auth_cookies(response: Response) -> None:
kwargs = {"path": "/"}
if settings.auth_cookie_domain:
kwargs["domain"] = settings.auth_cookie_domain
response.delete_cookie(settings.auth_session_cookie_name, **kwargs)
response.delete_cookie(settings.auth_csrf_cookie_name, **kwargs)
def _tenant_info(tenant: Tenant) -> TenantInfo:
return TenantInfo(
id=tenant.id,
slug=tenant.slug,
name=tenant.name,
is_active=tenant.is_active,
default_locale=tenant.default_locale,
)
def _user_info(user: User, account: Account) -> UserInfo:
return UserInfo(
id=user.id,
account_id=account.id,
email=account.email,
display_name=account.display_name or user.display_name,
tenant_display_name=user.display_name,
is_tenant_admin=user.is_tenant_admin,
password_reset_required=account.password_reset_required,
)
def _roles_info(roles, *, level: str = "tenant") -> list[RoleInfo]:
return [
RoleInfo(
id=role.id,
slug=role.slug,
name=role.name,
permissions=role.permissions or [],
level=level,
)
for role in roles
]
def _groups_info(groups) -> list[GroupInfo]:
return [GroupInfo(id=group.id, slug=group.slug, name=group.name) for group in groups]
def _tenant_memberships(session: Session, account: Account) -> list[TenantMembershipInfo]:
memberships: list[TenantMembershipInfo] = []
for user, tenant in collect_tenant_memberships(session, account):
roles = collect_user_roles(session, user)
memberships.append(
TenantMembershipInfo(
id=tenant.id,
slug=tenant.slug,
name=tenant.name,
is_active=tenant.is_active and user.is_active,
default_locale=tenant.default_locale,
roles=[role.slug for role in roles],
)
)
return memberships
def _resolve_login_user(session: Session, payload: LoginRequest) -> tuple[Account, User, Tenant]:
account = (
session.query(Account)
.filter(Account.normalized_email == normalize_email(payload.email), Account.is_active.is_(True))
.one_or_none()
)
if account is None or not verify_password(payload.password, account.password_hash):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid login")
query = (
session.query(User, Tenant)
.join(Tenant, Tenant.id == User.tenant_id)
.filter(
User.account_id == account.id,
User.is_active.is_(True),
Tenant.is_active.is_(True),
)
)
if payload.tenant_slug:
query = query.filter(Tenant.slug == payload.tenant_slug)
row = query.order_by(Tenant.name.asc()).first()
if row is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="No active tenant membership")
return account, row[0], row[1]
def _me_response(
session: Session,
*,
account: Account,
user: User,
tenant: Tenant,
effective_scopes: list[str] | None = None,
include_system: bool = True,
include_all_memberships: bool = True,
) -> MeResponse:
tenant_roles = collect_user_roles(session, user)
system_roles = collect_system_roles(session, account) if include_system else []
groups = collect_user_groups(session, user)
active_tenant = _tenant_info(tenant)
memberships = _tenant_memberships(session, account) if include_all_memberships else [
TenantMembershipInfo(
id=tenant.id,
slug=tenant.slug,
name=tenant.name,
is_active=tenant.is_active and user.is_active,
default_locale=tenant.default_locale,
roles=[role.slug for role in tenant_roles],
)
]
return MeResponse(
user=_user_info(user, account),
tenant=active_tenant,
active_tenant=active_tenant,
tenants=memberships,
scopes=effective_scopes if effective_scopes is not None else collect_user_scopes(session, user, include_system=include_system),
roles=_roles_info(tenant_roles) + _roles_info(system_roles, level="system"),
groups=_groups_info(groups),
)
@router.post("/login", response_model=LoginResponse)
def login(payload: LoginRequest, request: Request, response: Response, session: Session = Depends(get_session)):
account, user, tenant = _resolve_login_user(session, payload)
user_agent = request.headers.get("user-agent")
ip_address = request.client.host if request.client else None
created = create_auth_session(
session,
user=user,
hours=settings.auth_session_hours,
user_agent=user_agent,
ip_address=ip_address,
)
me_payload = _me_response(session, account=account, user=user, tenant=tenant)
session.commit()
_set_auth_cookies(response, created)
return LoginResponse(
access_token=created.token,
expires_at=created.model.expires_at,
**me_payload.model_dump(),
)
@router.get("/me", response_model=MeResponse)
def me(principal: ApiPrincipal = Depends(get_api_principal), session: Session = Depends(get_session)):
tenant = session.get(Tenant, principal.tenant_id)
if tenant is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Active tenant not found")
return _me_response(
session,
account=principal.account,
user=principal.user,
tenant=tenant,
effective_scopes=principal.scopes,
include_system=principal.auth_session is not None,
include_all_memberships=principal.auth_session is not None,
)
@router.patch("/profile", response_model=MeResponse)
def update_profile(
payload: ProfileUpdateRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
if principal.auth_session is None:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="API keys cannot edit an interactive user profile")
if "display_name" in payload.model_fields_set:
principal.account.display_name = payload.display_name.strip() if payload.display_name else None
session.add(principal.account)
if "tenant_display_name" in payload.model_fields_set:
principal.user.display_name = payload.tenant_display_name.strip() if payload.tenant_display_name else None
session.add(principal.user)
audit_from_principal(
session,
principal,
action="profile.updated",
object_type="account",
object_id=principal.account.id,
details={"fields": sorted(payload.model_fields_set)},
)
tenant = session.get(Tenant, principal.tenant_id)
if tenant is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Active tenant not found")
session.commit()
return _me_response(
session,
account=principal.account,
user=principal.user,
tenant=tenant,
include_system=True,
include_all_memberships=True,
)
@router.post("/switch-tenant", response_model=MeResponse)
def switch_tenant(
payload: SwitchTenantRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
if principal.auth_session is None:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="API keys cannot switch tenant context")
try:
membership = switch_auth_session_tenant(session, principal.auth_session, payload.tenant_id)
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(exc)) from exc
tenant = session.get(Tenant, membership.tenant_id)
if tenant is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Tenant not found")
session.commit()
return _me_response(session, account=principal.account, user=membership, tenant=tenant)
@router.post("/logout")
def logout(
response: Response,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
if principal.auth_session is not None:
principal.auth_session.revoked_at = utc_now()
session.add(principal.auth_session)
session.commit()
_clear_auth_cookies(response)
return {"ok": True}

View File

@@ -24,6 +24,21 @@ class AuditLogListResponse(BaseModel):
items: list[AuditLogItemResponse]
class DeltaDeletedItem(BaseModel):
id: str
resource_type: str | None = None
revision: str | None = None
deleted_at: datetime | None = None
class DeltaCollectionResponse(BaseModel):
items: list[dict[str, Any]] = Field(default_factory=list)
deleted: list[DeltaDeletedItem] = Field(default_factory=list)
watermark: str | None = None
has_more: bool = False
full: bool = False
class LoginRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
@@ -46,6 +61,7 @@ class TenantInfo(BaseModel):
name: str
is_active: bool = True
default_locale: str = "en"
enabled_language_codes: list[str] = Field(default_factory=list)
class TenantMembershipInfo(TenantInfo):
@@ -53,6 +69,16 @@ class TenantMembershipInfo(TenantInfo):
is_active: bool = True
class UserUiPreferences(BaseModel):
model_config = ConfigDict(extra="ignore")
compact_tables: bool = False
show_inline_help_hints: bool = True
reduce_motion: bool = False
sticky_section_sidebars: bool = True
theme: Literal["system", "light", "dark"] = "system"
class UserInfo(BaseModel):
id: str
account_id: str
@@ -63,6 +89,31 @@ class UserInfo(BaseModel):
tenant_display_name: str | None = None
is_tenant_admin: bool = False
password_reset_required: bool = False
preferred_language: str | None = None
enabled_language_codes: list[str] = Field(default_factory=list)
ui_preferences: UserUiPreferences = Field(default_factory=UserUiPreferences)
class AuthSessionUserInfo(BaseModel):
id: str
account_id: str
email: str
display_name: str | None = None
tenant_display_name: str | None = None
is_tenant_admin: bool = False
password_reset_required: bool = False
class AuthSessionResponse(BaseModel):
authenticated: bool = True
auth_method: Literal["session", "api_key"] = "session"
user: AuthSessionUserInfo
# Backwards-compatible alias for the active tenant.
tenant: TenantInfo
active_tenant: TenantInfo
session_id: str | None = None
api_key_id: str | None = None
expires_at: datetime | None = None
class ProfileUpdateRequest(BaseModel):
@@ -70,6 +121,15 @@ class ProfileUpdateRequest(BaseModel):
display_name: str | None = Field(default=None, max_length=255)
tenant_display_name: str | None = Field(default=None, max_length=255)
preferred_language: str | None = Field(default=None, max_length=20)
enabled_language_codes: list[str] | None = None
ui_preferences: UserUiPreferences | None = None
class LanguageInfo(BaseModel):
code: str
label: str
native_label: str | None = None
class RoleInfo(BaseModel):
@@ -86,6 +146,59 @@ class GroupInfo(BaseModel):
name: str
class PrincipalContextInfo(BaseModel):
account_id: str
membership_id: str | None = None
tenant_id: str | None = None
identity_id: str | None = None
scopes: list[str] = Field(default_factory=list)
group_ids: list[str] = Field(default_factory=list)
role_ids: list[str] = Field(default_factory=list)
function_assignment_ids: list[str] = Field(default_factory=list)
delegation_ids: list[str] = Field(default_factory=list)
auth_method: Literal["session", "api_key", "service_account"] = "session"
api_key_id: str | None = None
session_id: str | None = None
service_account_id: str | None = None
acting_for_account_id: str | None = None
email: str | None = None
display_name: str | None = None
class AuthShellResponse(BaseModel):
user: UserInfo
# Backwards-compatible alias for the active tenant.
tenant: TenantInfo
active_tenant: TenantInfo
tenants: list[TenantMembershipInfo] = Field(default_factory=list)
scopes: list[str]
principal: PrincipalContextInfo | None = None
profile_loaded: bool = False
roles_loaded: bool = False
groups_loaded: bool = False
class AuthProfileResponse(BaseModel):
user: UserInfo
# Backwards-compatible alias for the active tenant.
tenant: TenantInfo
active_tenant: TenantInfo
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
profile_loaded: bool = True
class AuthRolesResponse(BaseModel):
roles: list[RoleInfo] = Field(default_factory=list)
roles_loaded: bool = True
class AuthGroupsResponse(BaseModel):
groups: list[GroupInfo] = Field(default_factory=list)
groups_loaded: bool = True
class LoginResponse(BaseModel):
access_token: str
token_type: str = "bearer"
@@ -98,6 +211,13 @@ class LoginResponse(BaseModel):
scopes: list[str]
roles: list[RoleInfo] = Field(default_factory=list)
groups: list[GroupInfo] = Field(default_factory=list)
principal: PrincipalContextInfo | None = None
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
profile_loaded: bool = True
roles_loaded: bool = True
groups_loaded: bool = True
class MeResponse(BaseModel):
@@ -109,3 +229,10 @@ class MeResponse(BaseModel):
scopes: list[str]
roles: list[RoleInfo] = Field(default_factory=list)
groups: list[GroupInfo] = Field(default_factory=list)
principal: PrincipalContextInfo | None = None
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
profile_loaded: bool = True
roles_loaded: bool = True
groups_loaded: bool = True

View File

@@ -1,51 +0,0 @@
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status
from govoplan_core.auth.dependencies import ApiPrincipal, require_any_scope
from govoplan_core.core.optional import module_not_found_for
router = APIRouter(prefix="/schemas", tags=["schemas"])
def _load_campaign_schema() -> dict[str, Any]:
try:
from govoplan_campaign.backend.campaign.loader import load_campaign_schema
except ModuleNotFoundError as exc:
if module_not_found_for(exc, "govoplan_campaign"):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Campaign module is not installed") from exc
raise
return load_campaign_schema()
def _load_campaign_schema_ui() -> dict[str, Any]:
try:
from govoplan_campaign.backend.campaign.loader import load_campaign_schema_ui
except ModuleNotFoundError as exc:
if module_not_found_for(exc, "govoplan_campaign"):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Campaign module is not installed") from exc
raise
return load_campaign_schema_ui()
@router.get("/campaign")
def get_campaign_schema(
principal: ApiPrincipal = Depends(require_any_scope("campaigns:campaign:read", "campaigns:campaign:create", "admin:roles:read")),
) -> dict[str, Any]:
"""Return the authoritative campaign JSON Schema used by the backend."""
del principal
return _load_campaign_schema()
@router.get("/campaign/ui")
def get_campaign_schema_ui(
principal: ApiPrincipal = Depends(require_any_scope("campaigns:campaign:read", "campaigns:campaign:create", "admin:roles:read")),
) -> dict[str, Any]:
"""Return UI metadata paired with the campaign JSON Schema."""
del principal
return _load_campaign_schema_ui()

View File

@@ -1,36 +1,197 @@
from __future__ import annotations
from collections.abc import Mapping
from uuid import uuid4
from typing import Any
from sqlalchemy.orm import Session
from govoplan_core.auth.dependencies import ApiPrincipal
from govoplan_core.db.models import AuditLog
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.change_sequence import record_change
from govoplan_core.core.access import AuditEvent, AuditRecordRef, AuditRecorder, CAPABILITY_AUDIT_RECORDER
from govoplan_core.core.events import (
EventActorRef,
EventObjectRef,
EventTenantRef,
PlatformEvent,
current_event_trace,
normalize_trace_id,
publish_platform_event,
)
from govoplan_core.core.runtime import get_registry
from govoplan_core.privacy.retention import sanitize_audit_details_for_policy
from govoplan_core.security.redaction import redact_secret_values
SENSITIVE_DETAIL_KEYS = {
"password",
"smtp_password",
"imap_password",
"secret",
"api_key",
"token",
AUDIT_MODULE_ID = "audit"
AUDIT_TENANT_EVENTS_COLLECTION = "audit.admin.tenant_events"
AUDIT_SYSTEM_EVENTS_COLLECTION = "audit.admin.system_events"
_ACTION_MODULE_PREFIXES = {
"api_key": "access",
"configuration_change": "access",
"configuration_package": "access",
"group": "access",
"idm": "idm",
"profile": "access",
"role": "access",
"system_access": "access",
"system_account": "access",
"system_memberships": "access",
"system_role": "access",
"user": "access",
"tenant": "tenancy",
"privacy_retention": "policy",
"retention_policy": "policy",
"files": "files",
"mail": "mail",
"campaign": "campaigns",
"report": "campaigns",
}
def _optional_text(value: object | None) -> str | None:
if value is None:
return None
candidate = str(value).strip()
return candidate or None
def _compact_trace(trace: Mapping[str, object] | None) -> dict[str, str]:
if not isinstance(trace, Mapping):
return {}
compact: dict[str, str] = {}
correlation_id = _optional_text(trace.get("correlation_id"))
causation_id = _optional_text(trace.get("causation_id"))
clean_correlation_id = normalize_trace_id(correlation_id)
clean_causation_id = normalize_trace_id(causation_id)
if clean_correlation_id:
compact["correlation_id"] = clean_correlation_id
if clean_causation_id:
compact["causation_id"] = clean_causation_id
return compact
def _sanitize_details(value: Any) -> Any:
if isinstance(value, dict):
sanitized: dict[str, Any] = {}
for key, item in value.items():
if str(key).lower() in SENSITIVE_DETAIL_KEYS:
sanitized[key] = "<redacted>"
else:
sanitized[key] = _sanitize_details(item)
return sanitized
if isinstance(value, list):
return [_sanitize_details(item) for item in value]
return value
return redact_secret_values(value)
def audit_operation_context(
*,
module_id: object | None = None,
request_id: object | None = None,
run_id: object | None = None,
outcome: object | None = None,
trace: Mapping[str, object] | None = None,
**details: Any,
) -> dict[str, Any]:
"""Build concise operational audit details for admin and lifecycle actions."""
payload: dict[str, Any] = {}
for key, value in (
("module_id", module_id),
("request_id", request_id),
("run_id", run_id),
("outcome", outcome),
):
text = _optional_text(value)
if text is not None:
payload[key] = text
for key, value in details.items():
if value is not None:
payload[key] = _sanitize_details(value)
compact_trace = _compact_trace(trace)
if compact_trace:
existing = payload.get("_trace")
payload["_trace"] = {**existing, **compact_trace} if isinstance(existing, dict) else compact_trace
return payload
def _trace_details(
details: dict[str, Any],
*,
correlation_id: str | None = None,
causation_id: str | None = None,
) -> tuple[dict[str, Any], dict[str, str]]:
active_trace = current_event_trace()
trace: dict[str, str] = {}
clean_correlation_id = normalize_trace_id(correlation_id)
clean_causation_id = normalize_trace_id(causation_id)
if clean_correlation_id or active_trace:
trace["correlation_id"] = clean_correlation_id or active_trace.correlation_id
if clean_causation_id or (active_trace and active_trace.causation_id):
trace["causation_id"] = clean_causation_id or str(active_trace.causation_id)
if not trace:
return details, {}
merged = dict(details)
existing = merged.get("_trace")
merged["_trace"] = {**existing, **trace} if isinstance(existing, dict) else trace
return merged, trace
def _restore_trace_after_policy(details: dict[str, Any], trace: dict[str, str]) -> dict[str, Any]:
if not trace:
return details
if details.get("_trace") == trace:
return details
restored = dict(details)
existing = restored.get("_trace")
restored["_trace"] = {**existing, **trace} if isinstance(existing, dict) else trace
return restored
def _module_id_for_audit_action(action: str) -> str:
prefix = action.split(".", 1)[0].strip().casefold()
return _ACTION_MODULE_PREFIXES.get(prefix, prefix or AUDIT_MODULE_ID)
def _audit_recorder() -> AuditRecorder:
registry = get_registry()
if registry is None or not hasattr(registry, "has_capability") or not registry.has_capability(CAPABILITY_AUDIT_RECORDER):
return _NullAuditRecorder()
capability = registry.require_capability(CAPABILITY_AUDIT_RECORDER)
if not isinstance(capability, AuditRecorder):
raise RuntimeError("Audit recorder capability is invalid")
return capability
class _NullAuditRecorder:
def record_event(self, session: object, event: AuditEvent) -> AuditRecordRef:
del session
return AuditRecordRef(
id=str(uuid4()),
scope=event.scope,
tenant_id=event.tenant_id,
user_id=event.user_id,
api_key_id=event.api_key_id,
action=event.action,
object_type=event.resource_type,
object_id=event.resource_id,
details=dict(event.details or {}),
)
def _publish_audit_platform_event(item: AuditRecordRef) -> None:
trace = _compact_trace(item.details.get("_trace") if isinstance(item.details, Mapping) else None)
publish_platform_event(
PlatformEvent(
type=item.action,
module_id=_module_id_for_audit_action(item.action),
payload={
"audit_log_id": item.id,
"scope": item.scope,
"details": dict(item.details or {}),
},
correlation_id=trace.get("correlation_id"),
causation_id=trace.get("causation_id"),
actor=EventActorRef(type="user", id=item.user_id) if item.user_id else (
EventActorRef(type="api_key", id=item.api_key_id) if item.api_key_id else None
),
tenant=EventTenantRef(id=item.tenant_id) if item.tenant_id else None,
resource=EventObjectRef(type=item.object_type, id=item.object_id) if item.object_type else None,
classification="internal",
)
)
def audit_event(
@@ -44,8 +205,10 @@ def audit_event(
object_type: str | None = None,
object_id: str | None = None,
details: dict[str, Any] | None = None,
correlation_id: str | None = None,
causation_id: str | None = None,
commit: bool = False,
) -> AuditLog:
) -> AuditRecordRef:
"""Persist one audit event.
The function deliberately accepts primitive IDs so it can be used from API
@@ -56,21 +219,42 @@ def audit_event(
if scope not in {"tenant", "system"}:
raise ValueError(f"Unsupported audit scope: {scope}")
item = AuditLog(
traced_details, trace = _trace_details(
_sanitize_details(details or {}),
correlation_id=correlation_id,
causation_id=causation_id,
)
stored_details = _restore_trace_after_policy(
sanitize_audit_details_for_policy(session, traced_details),
trace,
)
item = _audit_recorder().record_event(session, AuditEvent(
event_type=action,
action=action,
scope=scope,
tenant_id=tenant_id,
user_id=user_id,
api_key_id=api_key_id,
action=action,
object_type=object_type,
object_id=object_id,
details=sanitize_audit_details_for_policy(session, _sanitize_details(details or {})),
resource_type=object_type,
resource_id=object_id,
details=stored_details,
))
record_change(
session,
module_id=AUDIT_MODULE_ID,
collection=AUDIT_SYSTEM_EVENTS_COLLECTION if scope == "system" else AUDIT_TENANT_EVENTS_COLLECTION,
resource_type="audit_log",
resource_id=item.id,
operation="created",
tenant_id=None if scope == "system" else tenant_id,
actor_type="user" if user_id else ("api_key" if api_key_id else None),
actor_id=user_id or api_key_id,
payload={"scope": scope, "action": action, "object_type": object_type, "object_id": object_id},
)
session.add(item)
_publish_audit_platform_event(item)
if commit:
session.commit()
else:
session.flush()
return item
@@ -83,8 +267,10 @@ def audit_from_principal(
object_type: str | None = None,
object_id: str | None = None,
details: dict[str, Any] | None = None,
correlation_id: str | None = None,
causation_id: str | None = None,
commit: bool = False,
) -> AuditLog:
) -> AuditRecordRef:
return audit_event(
session,
tenant_id=principal.tenant_id,
@@ -95,5 +281,7 @@ def audit_from_principal(
object_type=object_type,
object_id=object_id,
details=details,
correlation_id=correlation_id,
causation_id=causation_id,
commit=commit,
)

View File

@@ -0,0 +1,174 @@
from __future__ import annotations
"""Core auth dependency facade.
Routers depend on this module instead of a concrete access-provider package.
The active auth module provides the request principal through the platform
capability registry.
"""
from dataclasses import dataclass
from fastapi import Depends, Header, HTTPException, Request, status
from sqlalchemy.orm import Session
from govoplan_core.core.access import (
CAPABILITY_AUTH_API_PRINCIPAL_PROVIDER,
ApiPrincipalProvider,
PermissionEvaluator,
PrincipalRef,
)
from govoplan_core.core.registry import PlatformRegistry
from govoplan_core.db.session import get_session
from govoplan_core.security.module_permissions import scopes_grant_compatible
@dataclass(slots=True)
class ApiPrincipal:
"""Request principal exposed to API routers.
The stable fields live in ``principal``. ``account``, ``user``,
``api_key``, and ``auth_session`` are provider-owned objects kept for
current routers that still need concrete account/session state.
"""
principal: PrincipalRef
account: object
user: object
api_key: object | None = None
auth_session: object | None = None
permission_evaluator: PermissionEvaluator | None = None
@property
def account_id(self) -> str:
return self.principal.account_id
@property
def membership_id(self) -> str | None:
return self.principal.membership_id
@property
def tenant_id(self) -> str:
if self.principal.tenant_id is None:
raise RuntimeError("Tenant principal has no active tenant id.")
return self.principal.tenant_id
@property
def scopes(self) -> frozenset[str]:
return self.principal.scopes
@property
def group_ids(self) -> frozenset[str]:
return self.principal.group_ids
@property
def role_ids(self) -> frozenset[str]:
return self.principal.role_ids
@property
def function_assignment_ids(self) -> frozenset[str]:
return self.principal.function_assignment_ids
@property
def delegation_ids(self) -> frozenset[str]:
return self.principal.delegation_ids
@property
def identity_id(self) -> str | None:
return self.principal.identity_id
@property
def acting_for_account_id(self) -> str | None:
return self.principal.acting_for_account_id
@property
def auth_method(self) -> str:
return self.principal.auth_method
@property
def api_key_id(self) -> str | None:
return self.principal.api_key_id
@property
def session_id(self) -> str | None:
return self.principal.session_id
@property
def email(self) -> str | None:
return self.principal.email
@property
def display_name(self) -> str | None:
return self.principal.display_name
def has(self, required_scope: str) -> bool:
if self.permission_evaluator is not None:
return self.permission_evaluator.has_scope(self.principal, required_scope)
return scopes_grant_compatible(self.scopes, required_scope)
def to_platform_principal(self) -> PrincipalRef:
return self.principal
def _registry_from_request(request: Request) -> PlatformRegistry | None:
registry = getattr(request.app.state, "govoplan_registry", None)
return registry if isinstance(registry, PlatformRegistry) else None
def _api_principal_provider_from_request(request: Request) -> ApiPrincipalProvider:
registry = _registry_from_request(request)
if registry is None or not registry.has_capability(CAPABILITY_AUTH_API_PRINCIPAL_PROVIDER):
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Auth provider is not available")
capability = registry.require_capability(CAPABILITY_AUTH_API_PRINCIPAL_PROVIDER)
if not isinstance(capability, ApiPrincipalProvider):
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Invalid auth provider capability")
return capability
def get_api_principal(
request: Request,
session: Session = Depends(get_session),
authorization: str | None = Header(default=None),
x_api_key: str | None = Header(default=None, alias="X-API-Key"),
) -> ApiPrincipal:
principal = _api_principal_provider_from_request(request).resolve_api_principal(
request,
session,
authorization=authorization,
x_api_key=x_api_key,
)
if not isinstance(principal, ApiPrincipal):
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Invalid API principal")
return principal
def has_scope(principal: ApiPrincipal, required_scope: str) -> bool:
return principal.has(required_scope)
def require_scope(required_scope: str):
def dependency(principal: ApiPrincipal = Depends(get_api_principal)) -> ApiPrincipal:
if not has_scope(principal, required_scope):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {required_scope}")
return principal
return dependency
def require_any_scope(*required_scopes: str):
def dependency(principal: ApiPrincipal = Depends(get_api_principal)) -> ApiPrincipal:
if not any(has_scope(principal, required) for required in required_scopes):
joined = ", ".join(required_scopes)
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=f"Requires one of: {joined}")
return principal
return dependency
__all__ = [
"ApiPrincipal",
"get_api_principal",
"has_scope",
"require_any_scope",
"require_scope",
]

View File

@@ -1,223 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from fastapi import Depends, Header, HTTPException, Request, status
from sqlalchemy.orm import Session
from govoplan_core.db.models import Account, ApiKey, AuthSession, Tenant, User
from govoplan_core.db.session import get_session
from govoplan_core.security.module_permissions import scopes_grant_compatible
from govoplan_core.access.auth.principals import Principal
from govoplan_core.security.api_keys import authenticate_api_key
from govoplan_core.security.permissions import intersect_api_key_scopes, scopes_grant
from govoplan_core.security.sessions import authenticate_session_token, collect_user_groups, collect_user_scopes, verify_auth_session_csrf
from govoplan_core.settings import settings
@dataclass(slots=True)
class ApiPrincipal:
"""Compatibility wrapper around the platform Principal DTO.
Existing routers still expect ORM ``account`` and ``user`` objects. New
platform/module code should use the stable ID fields or
``to_platform_principal()`` instead.
"""
principal: Principal
account: Account
user: User
api_key: ApiKey | None = None
auth_session: AuthSession | None = None
@property
def account_id(self) -> str:
return self.principal.account_id
@property
def membership_id(self) -> str | None:
return self.principal.membership_id
@property
def tenant_id(self) -> str:
if self.principal.tenant_id is None:
raise RuntimeError("Tenant principal has no active tenant id.")
return self.principal.tenant_id
@property
def scopes(self) -> frozenset[str]:
return self.principal.scopes
@property
def group_ids(self) -> frozenset[str]:
return self.principal.group_ids
@property
def auth_method(self) -> str:
return self.principal.auth_method
@property
def api_key_id(self) -> str | None:
return self.principal.api_key_id
@property
def session_id(self) -> str | None:
return self.principal.session_id
@property
def email(self) -> str | None:
return self.principal.email
@property
def display_name(self) -> str | None:
return self.principal.display_name
def has(self, required_scope: str) -> bool:
# Keep legacy scope aliases alive while current routers still use the
# pre-platform permission catalogue.
return scopes_grant_compatible(self.scopes, required_scope)
def to_platform_principal(self) -> Principal:
return self.principal
def _extract_token(request: Request, authorization: str | None, x_api_key: str | None) -> tuple[str | None, str]:
if x_api_key:
return x_api_key.strip(), "api_key"
if authorization and authorization.lower().startswith("bearer "):
return authorization[7:].strip(), "bearer"
cookie_token = request.cookies.get(settings.auth_session_cookie_name)
if cookie_token:
return cookie_token.strip(), "cookie"
return None, "none"
def _requires_csrf(request: Request) -> bool:
return request.method.upper() not in {"GET", "HEAD", "OPTIONS", "TRACE"}
def _principal_group_ids(session: Session, user: User) -> frozenset[str]:
return frozenset(group.id for group in collect_user_groups(session, user))
def _build_api_principal(
session: Session,
*,
account: Account,
user: User,
tenant_id: str,
scopes: list[str],
auth_method: str,
api_key: ApiKey | None = None,
auth_session: AuthSession | None = None,
) -> ApiPrincipal:
principal = Principal(
account_id=account.id,
membership_id=user.id,
tenant_id=tenant_id,
scopes=frozenset(scopes),
group_ids=_principal_group_ids(session, user),
auth_method=auth_method, # type: ignore[arg-type]
api_key_id=api_key.id if api_key else None,
session_id=auth_session.id if auth_session else None,
email=account.email,
display_name=account.display_name or user.display_name,
)
return ApiPrincipal(
principal=principal,
account=account,
user=user,
api_key=api_key,
auth_session=auth_session,
)
def get_api_principal(
request: Request,
session: Session = Depends(get_session),
authorization: str | None = Header(default=None),
x_api_key: str | None = Header(default=None, alias="X-API-Key"),
) -> ApiPrincipal:
token, source = _extract_token(request, authorization, x_api_key)
if not token:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing API key or session token")
# API keys remain supported for CLI/automation. Their permissions are the
# intersection of the key grant and the owner's current tenant roles.
api_key = authenticate_api_key(session, token)
if api_key:
user = session.get(User, api_key.user_id)
account = session.get(Account, user.account_id) if user else None
tenant = session.get(Tenant, api_key.tenant_id)
if (
not user or not account or not tenant
or not user.is_active or not account.is_active or not tenant.is_active
or user.tenant_id != api_key.tenant_id
):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Inactive or inconsistent API-key principal")
user_scopes = collect_user_scopes(session, user, include_system=False)
effective_scopes = intersect_api_key_scopes(user_scopes, api_key.scopes or [])
session.commit()
return _build_api_principal(
session,
api_key=api_key,
account=account,
user=user,
tenant_id=api_key.tenant_id,
scopes=effective_scopes,
auth_method="api_key",
)
auth_session = authenticate_session_token(session, token)
if auth_session:
user = session.get(User, auth_session.user_id)
account = session.get(Account, auth_session.account_id)
tenant = session.get(Tenant, auth_session.tenant_id)
if (
not user or not account or not tenant
or not user.is_active or not account.is_active or not tenant.is_active
or user.account_id != account.id
or user.tenant_id != tenant.id
):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Inactive or inconsistent session principal")
if source == "cookie" and _requires_csrf(request):
header_token = request.headers.get("x-csrf-token")
cookie_token = request.cookies.get(settings.auth_csrf_cookie_name)
if not header_token or not cookie_token or header_token != cookie_token or not verify_auth_session_csrf(auth_session, header_token):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid or missing CSRF token")
scopes = collect_user_scopes(session, user, include_system=True)
session.commit()
return _build_api_principal(
session,
auth_session=auth_session,
account=account,
user=user,
tenant_id=user.tenant_id,
scopes=scopes,
auth_method="session",
)
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid API key or session token")
def has_scope(principal: ApiPrincipal, required_scope: str) -> bool:
return principal.has(required_scope)
def require_scope(required_scope: str):
def dependency(principal: ApiPrincipal = Depends(get_api_principal)) -> ApiPrincipal:
if not has_scope(principal, required_scope):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {required_scope}")
return principal
return dependency
def require_any_scope(*required_scopes: str):
def dependency(principal: ApiPrincipal = Depends(get_api_principal)) -> ApiPrincipal:
if not any(has_scope(principal, required) for required in required_scopes):
joined = ", ".join(required_scopes)
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=f"Requires one of: {joined}")
return principal
return dependency

Some files were not shown because too many files have changed in this diff Show More