govoplan-files/README.md

# govoplan-files

<!-- govoplan-repository-type:start -->
**Repository type:** module (domain).
<!-- govoplan-repository-type:end -->

GovOPlaN Files is the managed file module. It bundles backend storage APIs and the Files WebUI package so file features can be installed as one module.

## Ownership

This repository owns:

- backend module manifest `files`
- file permissions such as `files:file:read`, `files:file:upload`, `files:file:download`, `files:file:organize`, and `files:file:delete`
- file-space, file metadata, share, upload, download, pattern matching, and transfer APIs
- SQLAlchemy file models and module migration registration
- WebUI package `@govoplan/files-webui`
- `/files` route and IconRail navigation contribution

Core owns auth, tenants, RBAC evaluation, database/session primitives, CSRF/API helpers, and shell layout.

## Development

Install through the core environment:

```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m pip install -r requirements-dev.txt
```

Run the WebUI from the core host:

```bash
cd /mnt/DATA/git/govoplan-core/webui
PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run dev
```

## Module integration

Backend entry point:

```toml
[project.entry-points."govoplan.modules"]
files = "govoplan_files.backend.manifest:get_manifest"
```

Frontend package:

```text
@govoplan/files-webui
```

The campaign module can integrate with files when both modules are installed,
for example for managed attachment selection and campaign file sharing. Files
does not import campaign internals; campaign share/existence checks use the core
`campaigns.access` capability registered by the campaign module.

Platform RBAC and governance rules are documented in `govoplan-core/docs/`.

Managed files can carry source provenance for connector and import workflows.
Upload callers may provide `source_provenance_json` and `source_revision`; the
module stores those values under file metadata, returns normalized
`source_provenance` and `source_revision` fields in file responses, and carries
them into managed campaign attachment matches for frozen execution evidence.

Connector policy preflight is available through
`POST /api/v1/files/connector-policy/evaluate`, and provenance-bearing uploads
can pass `connector_policy_json` to enforce the same policy before file content
is read. Policy payloads contain ordered `sources`; each source has
`scope_type`, optional `scope_id`, optional `label`, and a `policy` object. The
policy object supports `allow`/`allowlist`/`whitelist` and
`deny`/`denylist`/`blacklist` rules for `connectors`, `providers`,
`external_ids`, `external_paths`, and `external_urls`. Deny rules win across the
hierarchy; allow rules narrow access at each source that defines them.

Connector endpoint settings are exposed through governed connector profiles.
Profiles can be supplied as JSON through
`GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON`, or from a JSON file path through
`GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE`. Each profile has an `id`, `provider`,
`endpoint_url`, governance `scope_type`/`scope_id`, optional `capabilities`, and
deployment-owned credential references such as `password_env`, `token_env`, or
`secret_ref`. Environment references require an exact name in the deployment-wide
`GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST`; API-managed profiles cannot select
process environment variables and may use only Files-owned encrypted password or
token values. API-created `secret_ref` values fail closed until Files has an
ownership contract that can confirm provider-side deletion. Legacy external
references are treated as non-owned: deleting a profile or credential detaches
and audits the reference but never passes it to an arbitrary secret provider.
Profile and credential deletion immediately clears encrypted values, credential
identities, deployment references, and private metadata in the same transaction
as the non-secret audit record; an audit failure rolls the deletion back.
`GET /api/v1/files/connectors/profiles` returns only profiles visible to the
current principal (system, tenant, user, group, or accessible campaign scope) and
redacts secret values and environment variable names. Use the returned
`policy_sources` with connector policy preflight before importing files.
`GET /api/v1/files/connectors/providers` exposes provider descriptors for
Seafile, Nextcloud, WebDAV, SMB, NFS, and local filesystem connectors. The
descriptor declares implementation status, optional dependencies, permission
mapping, sync/index strategy, conflict handling, preview behavior, and audit
events so provider coverage remains visible without forcing every optional
protocol dependency to be installed.
`GET /api/v1/files/connectors/profiles/{profile_id}/browse` provides read-only
connector browsing. Browse entries use a shared `library`/`folder`/`file` shape,
run profile policy with the `browse` operation, and support Seafile,
WebDAV/Nextcloud, and SMB when the optional `smb` extra is installed. Seafile
profiles browse libraries and directories via Seafile's read-only API; profiles
can still opt into WebDAV browsing by setting `metadata.webdav_endpoint_url` or
`metadata.browse_protocol` to `webdav`. S3-compatible profiles support bucket
and prefix browsing when the optional `s3` extra is installed.
`POST /api/v1/files/connectors/profiles/{profile_id}/import` imports a Seafile
or WebDAV/Nextcloud/SMB file into managed storage through the same governance,
conflict handling, source provenance, and connector audit path as direct
uploads. The Seafile provider uses account-token auth and the native file
download-link API; Nextcloud and generic WebDAV profiles use authenticated `GET`
requests against the configured WebDAV endpoint. SMB profiles use
`smb://server[:port]/share[/path]` endpoints and deployment-owned or encrypted
stored credentials through `smbprotocol`. Files installs a pinned transport for
every initial session, reconnect, alias, and DFS referral target. The deployment
private-network policy is re-evaluated immediately before each socket opens.

S3 connector browse/import binds botocore HTTP and HTTPS pools to the same pinned
socket policy. Retries, redirects, endpoint discovery, bucket aliases, and new
connections are therefore revalidated while TLS keeps the configured hostname
for SNI and certificate checks. Connector clients do not use outbound proxies or
ambient AWS credential discovery; configure credentials on the governed profile,
or explicitly use an anonymous profile for public objects. An incompatible SDK
upgrade fails closed before a usable client/session is returned.

Durable platform storage has a separate deployment boundary: installer-owned
Garage is accepted only at its exact generated endpoint, while an
operator-selected external backend requires a clean HTTPS origin and
`FILE_STORAGE_S3_ENDPOINT_TRUSTED=true`. That flag is deployment configuration,
cannot be supplied through a Files connector profile, and does not replace
operator responsibility for DNS, certificates, egress, bucket policy,
versioning, and recovery.

The actual local/S3 backend implementation is owned by Core so Files, Campaign,
and workers resolve the same object namespace. Files owns file metadata and key
layout. Node-local storage is supported only for `local` or one-host
`host-shared` profiles; multi-host `shared` deployments require S3.

Destructive Files-module retirement applies the same credential lifecycle before
dropping tables. Every remaining Files-owned encrypted connector secret is
scrubbed and audited first, while legacy non-owned external references are
detached and identified as such in the audit record. Retirement does not claim
or attempt provider-side deletion for references Files cannot prove it owns.

Local connector development assets live in `dev/connectors/`. The compose stack
boots Nextcloud, Seafile, WebDAV, and SMB endpoints for provider development and
manual interoperability testing.

Connector and collaboration ownership boundaries are documented in
`docs/CONNECTOR_BOUNDARY.md` and `docs/DOCUMENT_COLLABORATION_BOUNDARY.md`.
The role-adaptive user, administration, integration, and operator guide is the
[Files handbook](docs/FILES_HANDBOOK.md).
The Files route, connector surfaces, consequence classes, and pattern-language
verification are recorded in
[Files interface pattern migration](docs/INTERFACE_PATTERN_MIGRATION.md).

Archive imports use a two-phase preview and confirmation flow for ZIP, TAR,
TAR.GZ, TAR.BZ2, and TAR.XZ. Requests are spooled to bounded temporary files;
the server validates paths, entry count, expanded size, and expansion ratio,
then returns a 30-minute tenant/user-bound preview token. Confirmation reuploads
the original archive and stores only the selected members. Password-protected
ZIP passwords remain request-only and are never included in the preview token.

Managed blob writes and applied orphan cleanup use Core's durable recovery
ledger. Intent, request digests, recovery mode, and a distributed lease are
committed before physical storage effects; the Files session commit verifies
database and streamed object evidence, while rollback compensates only a newly
reserved unreferenced key. New object keys are opaque and do not retain the
uploaded filename. Uncertain or mismatched effects remain visible through Ops.

Operators with `files:file:admin` can run bounded, resumable integrity scans in
Administration. Scan batches and finding actions carry monotonic revisions;
stale resume, recheck, or cleanup requests fail before touching object storage.
Orphan cleanup always requires a dry-run preview followed by separate
confirmation and records recovery-ledger evidence.

Bulk rename and transfer APIs are owner-scoped: callers must provide the active
user or group file space with `owner_type` and `owner_id`. The storage layer
keeps a named legacy file-only helper for historical callers that lack owner
context, and regression tests cover its write-access checks.

Optional producer modules can persist generated output through the provider-
neutral `files.artifact_store` capability. Files remains responsible for upload
authorization, ownership, path normalization, versioning, and blob storage;
producers receive stable file/version references without importing Files
internals. See [Generated Artifact Store](docs/GENERATED_ARTIFACT_STORE.md).

## Release packaging

The repository root includes a `package.json` for git-based WebUI installs. It exports the package `@govoplan/files-webui` from `webui/src` so release builds can depend on tagged git refs instead of local `file:` paths.