govoplan-files/README.md

govoplan-files

Repository type: module (domain).

GovOPlaN Files is the managed file module. It bundles backend storage APIs and the Files WebUI package so file features can be installed as one module.

Ownership

This repository owns:

• backend module manifest files
• file permissions such as files:file:read, files:file:upload, files:file:download, files:file:organize, and files:file:delete
• file-space, file metadata, share, upload, download, pattern matching, and transfer APIs
• SQLAlchemy file models and module migration registration
• WebUI package @govoplan/files-webui
• /files route and IconRail navigation contribution

Core owns auth, tenants, RBAC evaluation, database/session primitives, CSRF/API helpers, and shell layout.

Development

Install through the core environment:

cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m pip install -r requirements-dev.txt

Run the WebUI from the core host:

cd /mnt/DATA/git/govoplan-core/webui
PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run dev

Module integration

Backend entry point:

[project.entry-points."govoplan.modules"]
files = "govoplan_files.backend.manifest:get_manifest"

Frontend package:

@govoplan/files-webui

The campaign module can integrate with files when both modules are installed, for example for managed attachment selection and campaign file sharing. Files does not import campaign internals; campaign share/existence checks use the core campaigns.access capability registered by the campaign module.

Files also publishes the optional privacy.dsar.files capability. The Core data-subject workflow can use it to collect bounded, tenant-scoped file, version, share, folder, evidence, and non-secret configuration metadata for a direct membership subject. The provider may revoke a subject-targeted share or detach mutable actor references idempotently, but it never exports credential material or raw bytes and never bypasses Files retention, legal hold, evidence, purge approval, audit, or recovery controls. See the handbook's data-subject request coverage section for the review and erasure boundary.

Platform RBAC and governance rules are documented in govoplan-core/docs/.

Managed files can carry source provenance for connector and import workflows. Upload callers may provide source_provenance_json and source_revision; the module stores those values under file metadata, returns normalized source_provenance and source_revision fields in file responses, and carries them into managed campaign attachment matches for frozen execution evidence.

Connector policy preflight is available through POST /api/v1/files/connector-policy/evaluate, and provenance-bearing uploads can pass connector_policy_json to enforce the same policy before file content is read. Policy payloads contain ordered sources; each source has scope_type, optional scope_id, optional label, and a policy object. The policy object supports allow/allowlist/whitelist and deny/denylist/blacklist rules for connectors, providers, external_ids, external_paths, and external_urls. Deny rules win across the hierarchy; allow rules narrow access at each source that defines them.

Connector endpoint settings are exposed through governed connector profiles. Profiles can be supplied as JSON through GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON, or from a JSON file path through GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE. Each profile has an id, provider, endpoint_url, governance scope_type/scope_id, optional capabilities, and deployment-owned credential references such as password_env, token_env, or secret_ref. Environment references require an exact name in the deployment-wide GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST; API-managed profiles cannot select process environment variables and may use only Files-owned encrypted password or token values. API-created secret_ref values fail closed until Files has an ownership contract that can confirm provider-side deletion. Legacy external references are treated as non-owned: deleting a profile or credential detaches and audits the reference but never passes it to an arbitrary secret provider. Profile and credential deletion immediately clears encrypted values, credential identities, deployment references, and private metadata in the same transaction as the non-secret audit record; an audit failure rolls the deletion back. GET /api/v1/files/connectors/profiles returns only profiles visible to the current principal (system, tenant, user, group, or accessible campaign scope) and redacts secret values and environment variable names. Use the returned policy_sources with connector policy preflight before importing files. GET /api/v1/files/connectors/providers exposes provider descriptors for Seafile, Nextcloud, WebDAV, SMB, NFS, and local filesystem connectors. The descriptor declares implementation status, optional dependencies, permission mapping, sync/index strategy, conflict handling, preview behavior, and audit events so provider coverage remains visible without forcing every optional protocol dependency to be installed. GET /api/v1/files/connectors/profiles/{profile_id}/browse provides read-only connector browsing. Browse entries use a shared library/folder/file shape, run profile policy with the browse operation, and support Seafile, WebDAV/Nextcloud, and SMB when the optional smb extra is installed. Seafile profiles browse libraries and directories via Seafile's read-only API; profiles can still opt into WebDAV browsing by setting metadata.webdav_endpoint_url or metadata.browse_protocol to webdav. S3-compatible profiles support bucket and prefix browsing when the optional s3 extra is installed. POST /api/v1/files/connectors/profiles/{profile_id}/import imports a Seafile or WebDAV/Nextcloud/SMB file into managed storage through the same governance, conflict handling, source provenance, and connector audit path as direct uploads. The Seafile provider uses account-token auth and the native file download-link API; Nextcloud and generic WebDAV profiles use authenticated GET requests against the configured WebDAV endpoint. SMB profiles use smb://server[:port]/share[/path] endpoints and deployment-owned or encrypted stored credentials through smbprotocol. Files installs a pinned transport for every initial session, reconnect, alias, and DFS referral target. The deployment private-network policy is re-evaluated immediately before each socket opens.

S3 connector browse/import binds botocore HTTP and HTTPS pools to the same pinned socket policy. Retries, redirects, endpoint discovery, bucket aliases, and new connections are therefore revalidated while TLS keeps the configured hostname for SNI and certificate checks. Connector clients do not use outbound proxies or ambient AWS credential discovery; configure credentials on the governed profile, or explicitly use an anonymous profile for public objects. Keep connector spaces read-only unless a remote write is explicitly needed. An S3 profile with the write capability can back a two-way connector space. POST /api/v1/files/connector-spaces/{space_id}/write-back then conditionally creates or replaces one remote object from a managed file, records durable recovery intent before the provider effect, and verifies request/content markers afterward. Automatic remote delete, rename, move, and ACL propagation remain disabled; other connector providers remain read-only. An incompatible SDK upgrade fails closed before a usable client/session is returned.

Durable platform storage has a separate deployment boundary: installer-owned Garage is accepted only at its exact generated endpoint, while an operator-selected external backend requires a clean HTTPS origin and FILE_STORAGE_S3_ENDPOINT_TRUSTED=true. That flag is deployment configuration, cannot be supplied through a Files connector profile, and does not replace operator responsibility for DNS, certificates, egress, bucket policy, versioning, and recovery.

The actual local/S3 backend implementation is owned by Core so Files, Campaign, and workers resolve the same object namespace. Files owns file metadata and key layout. Node-local storage is supported only for local or one-host host-shared profiles; multi-host shared deployments require S3.

Destructive Files-module retirement applies the same credential lifecycle before dropping tables. Every remaining Files-owned encrypted connector secret is scrubbed and audited first, while legacy non-owned external references are detached and identified as such in the audit record. Retirement does not claim or attempt provider-side deletion for references Files cannot prove it owns.

Local connector development assets live in dev/connectors/. The compose stack boots Nextcloud, Seafile, WebDAV, and SMB endpoints for provider development and manual interoperability testing.

Connector and collaboration ownership boundaries are documented in docs/CONNECTOR_BOUNDARY.md and docs/DOCUMENT_COLLABORATION_BOUNDARY.md. The role-adaptive user, administration, integration, and operator guide is the Files handbook. The Files route, connector surfaces, consequence classes, and pattern-language verification are recorded in Files interface pattern migration.

Archive imports use a two-phase preview and confirmation flow for ZIP, TAR, TAR.GZ, TAR.BZ2, and TAR.XZ. Requests are spooled to bounded temporary files; the server validates paths, entry count, expanded size, and expansion ratio, then returns a 30-minute tenant/user-bound preview token. Confirmation reuploads the original archive and stores only the selected members. Password-protected ZIP passwords remain request-only and are never included in the preview token.

Managed blob writes and applied orphan cleanup use Core's durable recovery ledger. On PostgreSQL, intent, request digests, recovery mode, and a distributed lease are committed before physical storage effects; the Files session commit verifies database and streamed object evidence, while rollback compensates only a newly reserved unreferenced key. Development SQLite records blob intent in the caller transaction to avoid its second-writer deadlock, then verifies on commit or reconstructs compensation evidence after handled rollback. Because a hard loss before that commit can leave an unrecorded object, SQLite requires a complete integrity scan after a crash and is not a production recovery profile. New object keys are opaque and do not retain the uploaded filename. Uncertain or mismatched effects remain visible through Ops.

Operators with files:file:admin can run bounded, resumable integrity scans in Administration. Scan batches and finding actions carry monotonic revisions; stale resume, recheck, or cleanup requests fail before touching object storage. Orphan cleanup always requires a dry-run preview followed by separate confirmation and records recovery-ledger evidence.

Bulk rename and transfer APIs are owner-scoped: callers must provide the active user or group file space with owner_type and owner_id. The storage layer keeps a named legacy file-only helper for historical callers that lack owner context, and regression tests cover its write-access checks.

Optional producer modules can persist generated output through the provider- neutral files.artifact_store capability. Files remains responsible for upload authorization, ownership, path normalization, versioning, and blob storage; producers receive stable file/version references without importing Files internals. See Generated Artifact Store.

Release packaging

The repository root includes a package.json for git-based WebUI installs. It exports the package @govoplan/files-webui from webui/src so release builds can depend on tagged git refs instead of local file: paths.