Compare commits
11 Commits
02ef83ecee
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 85606d5580 | |||
| 632cc6cf7d | |||
| 86a905a3a7 | |||
| 5b868272b9 | |||
| 93eea839c8 | |||
| b667e7ff0e | |||
| 0fd8f6e02a | |||
| 2b34f6e305 | |||
| 58af1a20a7 | |||
| 4722161592 | |||
| 1444ba80a0 |
@@ -56,6 +56,14 @@ The main domain objects are:
|
||||
The Files page is available at `/files`. Actions appear only when the current
|
||||
principal has the required permission and resource access.
|
||||
|
||||
The configured Help Center projects these sections as independently authorized
|
||||
tasks, so upload, ZIP import, organization, download, sharing, and deletion do
|
||||
not disappear merely because an unrelated permission is absent. It states the
|
||||
deployment's actual upload limits. External import appears as a task only when
|
||||
the actor has the required permissions and at least one actor-visible profile
|
||||
is eligible for the current fail-closed browse/import path; endpoint, path, and
|
||||
item policy are still enforced when the operation runs.
|
||||
|
||||
### Choose a space
|
||||
|
||||
Every file user sees **My files**. Group spaces are added for active groups of
|
||||
@@ -727,14 +735,14 @@ returning different content or credentials.
|
||||
|
||||
| Area | Implemented now | Planned or explicitly outside the current boundary |
|
||||
| --- | --- | --- |
|
||||
| Managed storage | Local durable-root backend, fallback read roots, tenant blob deduplication, checksums | Operational S3 after pinned SDK transport ([#34](https://git.add-ideas.de/add-ideas/govoplan-files/issues/34)); automated integrity/orphan reconciliation ([#36](https://git.add-ideas.de/add-ideas/govoplan-files/issues/36)) |
|
||||
| Managed storage | Local durable-root backend, fallback read roots, tenant blob deduplication, checksums | Operational S3 after pinned SDK transport ([#34](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/34)); automated integrity/orphan reconciliation ([#36](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/36)) |
|
||||
| Upload | Bounded direct upload, drag-and-drop UI, ZIP spool/extract limits, explicit conflicts | Malware scanning, quotas, type policy, resumable/chunked upload |
|
||||
| Organization | Folders, bulk rename preview/apply, move/copy, drag-and-drop, ZIP download, pattern resolution | General file-history UI and user-driven append-version/restore |
|
||||
| Sharing | User/group/tenant/campaign grant or permission update through API; campaign linkage display | Share revocation/expiry and complete general share-management UI ([#37](https://git.add-ideas.de/add-ideas/govoplan-files/issues/37)) |
|
||||
| Deletion/retention | Soft-delete assets/folders/spaces; immediate audited connector-secret scrubbing | File restore API, hard purge, retention policy, legal hold, and blob GC ([#38](https://git.add-ideas.de/add-ideas/govoplan-files/issues/38)) |
|
||||
| Sharing | User/group/tenant/campaign grant or permission update through API; campaign linkage display | Share revocation/expiry and complete general share-management UI ([#37](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/37)) |
|
||||
| Deletion/retention | Soft-delete assets/folders/spaces; immediate audited connector-secret scrubbing | File restore API, hard purge, retention policy, legal hold, and blob GC ([#38](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/38)) |
|
||||
| Connector governance | Scoped profiles/credentials/policies, effective source explanation, separate credentials, linked user/group spaces | Provider-owned external secret lifecycle; API `secret_ref` remains rejected |
|
||||
| HTTP connectors | Pinned, bounded, no-redirect Seafile and WebDAV/Nextcloud browse/import/manual sync | Background/folder sync, remote mutation, long-running transfer workers |
|
||||
| SMB and S3 connectors | Provider descriptors and browse/import logic | Live access only after SDK connections plus DFS referrals/redirects are validated and pinned ([SMB #35](https://git.add-ideas.de/add-ideas/govoplan-files/issues/35), [S3 #34](https://git.add-ideas.de/add-ideas/govoplan-files/issues/34)) |
|
||||
| SMB and S3 connectors | Provider descriptors and browse/import logic | Live access only after SDK connections plus DFS referrals/redirects are validated and pinned ([SMB #35](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/35), [S3 #34](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/34)) |
|
||||
| Other providers | Reserved SharePoint/OneDrive keys and NFS/local descriptors | Graph/OAuth/provider paging, NFS deployment integration, DMS connectors |
|
||||
| Connector spaces | User/group link, browse, manual selected-file sync, edit/disable/delete | Background sync, remote writes/deletes, full conflict-reporting jobs |
|
||||
| Profile capabilities | Stored and displayed | Enforce capability flags as an independent operation gate |
|
||||
|
||||
461
src/govoplan_files/backend/documentation.py
Normal file
461
src/govoplan_files/backend/documentation.py
Normal file
@@ -0,0 +1,461 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.campaigns import (
|
||||
CAPABILITY_CAMPAIGNS_ACCESS,
|
||||
CampaignAccessProvider,
|
||||
)
|
||||
from govoplan_core.core.modules import (
|
||||
DocumentationCondition,
|
||||
DocumentationContext,
|
||||
DocumentationLink,
|
||||
DocumentationTopic,
|
||||
)
|
||||
from govoplan_files.backend.storage.archives import ZIP_UPLOAD_MAX_FILES
|
||||
from govoplan_files.backend.storage.access import user_group_ids
|
||||
from govoplan_files.backend.storage.connector_visibility import (
|
||||
connector_profile_usable_for_import,
|
||||
visible_connector_profiles_for_actor,
|
||||
)
|
||||
|
||||
|
||||
_DEFAULT_UPLOAD_MAX_BYTES = 50 * 1024 * 1024
|
||||
_DEFAULT_ZIP_MAX_BYTES = 250 * 1024 * 1024
|
||||
_FILES_READ_SCOPE = "files:file:read"
|
||||
_FILES_UPLOAD_SCOPE = "files:file:upload"
|
||||
|
||||
|
||||
def documentation_topics(
|
||||
context: DocumentationContext,
|
||||
) -> tuple[DocumentationTopic, ...]:
|
||||
if context.documentation_type != "user":
|
||||
return ()
|
||||
|
||||
upload_limit = _configured_positive_int(
|
||||
context.settings,
|
||||
"file_upload_max_bytes",
|
||||
default=_DEFAULT_UPLOAD_MAX_BYTES,
|
||||
)
|
||||
zip_limit = _configured_positive_int(
|
||||
context.settings,
|
||||
"file_upload_zip_max_bytes",
|
||||
default=_DEFAULT_ZIP_MAX_BYTES,
|
||||
)
|
||||
topics: list[DocumentationTopic] = []
|
||||
if upload_limit is not None:
|
||||
topics.append(_upload_topic(upload_limit))
|
||||
if upload_limit is not None and zip_limit is not None:
|
||||
topics.append(_zip_topic(upload_limit, zip_limit))
|
||||
topics.append(_connector_import_topic(context))
|
||||
return tuple(topics)
|
||||
|
||||
|
||||
def _upload_topic(max_bytes: int) -> DocumentationTopic:
|
||||
limit = _format_byte_limit(max_bytes)
|
||||
return DocumentationTopic(
|
||||
id="files.workflow.upload-managed-files",
|
||||
title="Upload managed files",
|
||||
summary=f"Upload files to a personal or accessible group space; each uploaded file may contain at most {limit}.",
|
||||
body=(
|
||||
f"The current deployment accepts at most {limit} for each ordinary upload. "
|
||||
"A name conflict is never resolved silently: reject stops the upload, rename chooses a copy name, and overwrite retires the old asset before creating a new one."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "file_manager", "process_participant"),
|
||||
order=39,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=(_FILES_READ_SCOPE, _FILES_UPLOAD_SCOPE),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(
|
||||
label="Files handbook",
|
||||
href="govoplan-files/docs/FILES_HANDBOOK.md",
|
||||
kind="repository",
|
||||
),
|
||||
),
|
||||
related_modules=("campaigns",),
|
||||
unlocks=(
|
||||
"Users and connected processes can place governed content in managed storage.",
|
||||
),
|
||||
source_module_id="files",
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"You may view and upload managed files.",
|
||||
"The destination personal or group space grants this account write access; upload permission alone does not grant access to every space.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and choose My files or an accessible group space.",
|
||||
"Open the intended destination folder and choose Upload, or drag files into the file list.",
|
||||
"For every name conflict, explicitly reject, rename, overwrite, or skip the affected item.",
|
||||
"Wait for the upload to finish, then open the resulting file details.",
|
||||
],
|
||||
"outcome": "Each accepted file is stored as a governed managed asset in the selected space.",
|
||||
"verification": "Confirm the owner, logical path, size, checksum, and current version in Files.",
|
||||
"constraints": [
|
||||
{
|
||||
"id": "ordinary-upload-size",
|
||||
"label": "Maximum size per file",
|
||||
"description": f"The current safe upload limit is {limit} per file.",
|
||||
"values": [limit],
|
||||
},
|
||||
],
|
||||
"related_topic_ids": [
|
||||
"files.workflow.upload-and-unpack-zip",
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.workflow.find-and-download-files",
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _zip_topic(max_file_bytes: int, max_zip_bytes: int) -> DocumentationTopic:
|
||||
member_limit = _format_byte_limit(max_file_bytes)
|
||||
archive_limit = _format_byte_limit(max_zip_bytes)
|
||||
return DocumentationTopic(
|
||||
id="files.workflow.upload-and-unpack-zip",
|
||||
title="Upload and unpack a ZIP archive",
|
||||
summary=(
|
||||
f"Safely unpack up to {ZIP_UPLOAD_MAX_FILES:,} files from a ZIP whose request and extracted total are each limited to {archive_limit}."
|
||||
),
|
||||
body=(
|
||||
f"The current deployment limits the ZIP request and actual extracted total to {archive_limit}, each member to {member_limit}, "
|
||||
f"and the archive to {ZIP_UPLOAD_MAX_FILES:,} non-directory members. Encrypted archives and unsafe member paths are rejected. "
|
||||
"Actual extracted bytes are counted instead of trusting ZIP headers."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "file_manager", "process_participant"),
|
||||
order=40,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=(_FILES_READ_SCOPE, _FILES_UPLOAD_SCOPE),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(
|
||||
label="Files handbook",
|
||||
href="govoplan-files/docs/FILES_HANDBOOK.md",
|
||||
kind="repository",
|
||||
),
|
||||
),
|
||||
unlocks=(
|
||||
"A bounded archive can become governed managed files without trusting archive paths or size declarations.",
|
||||
),
|
||||
source_module_id="files",
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"You may view and upload managed files.",
|
||||
"The archive is not encrypted and fits the current configured limits.",
|
||||
"The destination personal or group space grants this account write access.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and choose the managed destination space and folder.",
|
||||
"Enable Unpack ZIP uploads, then choose or drag the ZIP archive.",
|
||||
"Resolve every destination conflict explicitly.",
|
||||
"Wait for extraction and finalization to finish before leaving the page.",
|
||||
],
|
||||
"outcome": "Accepted archive members are stored as separate governed managed assets below the selected folder.",
|
||||
"verification": "Confirm the expected member paths and inspect representative file sizes, checksums, and versions.",
|
||||
"constraints": [
|
||||
{
|
||||
"id": "zip-request-and-total",
|
||||
"label": "Maximum ZIP request and extracted total",
|
||||
"description": f"Both the compressed request and the actual extracted total are limited to {archive_limit}.",
|
||||
"values": [archive_limit],
|
||||
},
|
||||
{
|
||||
"id": "zip-member-size",
|
||||
"label": "Maximum extracted member size",
|
||||
"description": f"Each extracted file is limited to {member_limit}.",
|
||||
"values": [member_limit],
|
||||
},
|
||||
{
|
||||
"id": "zip-member-count",
|
||||
"label": "Maximum file count",
|
||||
"description": f"A ZIP may contain at most {ZIP_UPLOAD_MAX_FILES:,} non-directory members.",
|
||||
"values": [f"{ZIP_UPLOAD_MAX_FILES:,} files"],
|
||||
},
|
||||
],
|
||||
"related_topic_ids": [
|
||||
"files.workflow.upload-managed-files",
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.workflow.find-and-download-files",
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _connector_import_topic(context: DocumentationContext) -> DocumentationTopic:
|
||||
principal = context.principal
|
||||
if not _has_all_scopes(principal, (_FILES_READ_SCOPE, _FILES_UPLOAD_SCOPE)):
|
||||
return _connector_import_limitation(
|
||||
"Connector import is not available to this account because both permission to view Files and permission to upload managed files are required."
|
||||
)
|
||||
|
||||
tenant_id = _safe_text_attribute(principal, "tenant_id")
|
||||
user_id = _safe_text_attribute(getattr(principal, "user", None), "id")
|
||||
session = context.session
|
||||
if not tenant_id or not user_id or not isinstance(session, Session):
|
||||
return _connector_import_limitation(
|
||||
"Connector import availability could not be safely evaluated for this request. Try again, or ask a Files administrator to verify an actor-visible connection."
|
||||
)
|
||||
|
||||
try:
|
||||
member_group_ids = _actor_group_ids(
|
||||
session,
|
||||
principal=principal,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
include_admin_groups=False,
|
||||
)
|
||||
connector_group_ids = (
|
||||
_actor_group_ids(
|
||||
session,
|
||||
principal=principal,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
include_admin_groups=True,
|
||||
)
|
||||
if _has_all_scopes(principal, ("files:file:admin",))
|
||||
else member_group_ids
|
||||
)
|
||||
profiles = visible_connector_profiles_for_actor(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=connector_group_ids,
|
||||
settings=context.settings,
|
||||
campaign_visible=_campaign_visibility(
|
||||
context,
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=member_group_ids,
|
||||
),
|
||||
include_effective_policy=True,
|
||||
)
|
||||
usable_profiles = tuple(
|
||||
profile
|
||||
for profile in profiles
|
||||
if connector_profile_usable_for_import(profile)
|
||||
)
|
||||
except Exception:
|
||||
return _connector_import_limitation(
|
||||
"Connector import availability could not be safely evaluated for this request. Try again, or ask a Files administrator to verify an actor-visible connection."
|
||||
)
|
||||
if not usable_profiles:
|
||||
return _connector_import_limitation(
|
||||
"No connection visible to this account is currently eligible to offer an enabled, credential-ready, policy-allowed browse/import path through a pinning-safe provider. Ask a Files administrator to configure or authorize one."
|
||||
)
|
||||
|
||||
return DocumentationTopic(
|
||||
id="files.workflow.import-managed-snapshot",
|
||||
title="Import an external file as a governed snapshot",
|
||||
summary="Browse an authorized connection read-only and import one selected file into managed storage as a frozen, traceable snapshot.",
|
||||
body=(
|
||||
"At least one connection visible to this account is currently eligible to offer the governed browse/import path. "
|
||||
"The selected remote path and item are re-authorized when used, and browse, import, and sync never mutate the remote source. "
|
||||
"The managed snapshot stays unchanged until an explicit manual sync."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "campaign_manager", "report_author"),
|
||||
order=41,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=(_FILES_READ_SCOPE, _FILES_UPLOAD_SCOPE),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(
|
||||
label="Files handbook",
|
||||
href="govoplan-files/docs/FILES_HANDBOOK.md",
|
||||
kind="repository",
|
||||
),
|
||||
),
|
||||
related_modules=("campaigns",),
|
||||
unlocks=(
|
||||
"Campaigns, reports, and workflows can consume a managed snapshot with stable source evidence.",
|
||||
),
|
||||
source_module_id="files",
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list", "files.connector-import"],
|
||||
"prerequisites": [
|
||||
"You may view and upload managed files.",
|
||||
"At least one enabled, credential-ready, policy-allowed connection using a pinning-safe provider is visible to this account.",
|
||||
"The selected remote path and item must pass their operation-time policy checks.",
|
||||
"The managed destination space grants this account write access.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and choose a managed destination space.",
|
||||
"Choose Sync from connection, select an available connection, and browse to the permitted remote file.",
|
||||
"Import the selected file and resolve any destination conflict explicitly.",
|
||||
"Review the managed file's source and current-version details before using it in another task.",
|
||||
],
|
||||
"current_configuration": [
|
||||
"At least one actor-visible connection is eligible to offer a safe browse/import operation; the selected endpoint, path, and item are still checked when used.",
|
||||
"Every selected remote path and item is re-authorized at operation time.",
|
||||
],
|
||||
"outcome": "The external content is a tenant-managed snapshot with a checksum, exact version, and recorded source context.",
|
||||
"verification": "Reopen the managed file and confirm its recorded source context, source revision when available, checksum, and current version.",
|
||||
"related_topic_ids": [
|
||||
"files.governed-connectors-and-provenance",
|
||||
"files.reference.integrity-recovery-and-fail-closed-transports",
|
||||
"files.reference.snapshot-provenance-and-capabilities",
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _connector_import_limitation(message: str) -> DocumentationTopic:
|
||||
return DocumentationTopic(
|
||||
id="files.connector-import-unavailable",
|
||||
title="External file import is not currently available",
|
||||
summary=message,
|
||||
body=(
|
||||
f"{message} Files never exposes connection endpoints, storage paths, credential references, or raw connector policies in user documentation."
|
||||
),
|
||||
layer="available",
|
||||
documentation_types=("user",),
|
||||
order=41,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
any_scopes=(_FILES_READ_SCOPE, _FILES_UPLOAD_SCOPE),
|
||||
),
|
||||
),
|
||||
links=(DocumentationLink(label="Files", href="/files", kind="runtime"),),
|
||||
source_module_id="files",
|
||||
metadata={
|
||||
"kind": "reference",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list", "files.connector-import"],
|
||||
"limitations": [message],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _configured_positive_int(
|
||||
settings: object | None, name: str, *, default: int
|
||||
) -> int | None:
|
||||
raw_value = getattr(settings, name, default) if settings is not None else default
|
||||
try:
|
||||
value = int(raw_value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return value if value > 0 else None
|
||||
|
||||
|
||||
def _format_byte_limit(value: int) -> str:
|
||||
units = ((1024**3, "GiB"), (1024**2, "MiB"), (1024, "KiB"))
|
||||
for divisor, label in units:
|
||||
if value % divisor == 0:
|
||||
return f"{value // divisor:,} {label} ({value:,} bytes)"
|
||||
return f"{value:,} bytes"
|
||||
|
||||
|
||||
def _has_all_scopes(principal: object | None, scopes: tuple[str, ...]) -> bool:
|
||||
checker = getattr(principal, "has", None)
|
||||
if callable(checker):
|
||||
try:
|
||||
return all(bool(checker(scope)) for scope in scopes)
|
||||
except Exception:
|
||||
return False
|
||||
granted = {str(scope) for scope in getattr(principal, "scopes", ())}
|
||||
return all(scope in granted for scope in scopes)
|
||||
|
||||
|
||||
def _safe_text_attribute(value: object | None, name: str) -> str:
|
||||
try:
|
||||
result = getattr(value, name, "")
|
||||
except Exception:
|
||||
return ""
|
||||
return str(result or "")
|
||||
|
||||
|
||||
def _principal_group_ids(principal: object) -> tuple[str, ...]:
|
||||
try:
|
||||
values = getattr(principal, "group_ids", ())
|
||||
except Exception:
|
||||
return ()
|
||||
return tuple(str(value) for value in values if str(value))
|
||||
|
||||
|
||||
def _actor_group_ids(
|
||||
session: Session,
|
||||
*,
|
||||
principal: object,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
include_admin_groups: bool,
|
||||
) -> tuple[str, ...]:
|
||||
try:
|
||||
values = user_group_ids(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
include_admin_groups=include_admin_groups,
|
||||
)
|
||||
except Exception:
|
||||
return _principal_group_ids(principal)
|
||||
return tuple(str(value) for value in values if str(value))
|
||||
|
||||
|
||||
def _campaign_visibility(
|
||||
context: DocumentationContext,
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
group_ids: tuple[str, ...],
|
||||
):
|
||||
principal = context.principal
|
||||
registry = context.registry
|
||||
if not _has_all_scopes(principal, ("campaigns:campaign:read",)):
|
||||
return None
|
||||
try:
|
||||
if not registry.has_capability(CAPABILITY_CAMPAIGNS_ACCESS):
|
||||
return None
|
||||
capability = registry.require_capability(CAPABILITY_CAMPAIGNS_ACCESS)
|
||||
except Exception:
|
||||
return None
|
||||
if not isinstance(capability, CampaignAccessProvider):
|
||||
return None
|
||||
|
||||
def campaign_visible(campaign_id: str) -> bool:
|
||||
try:
|
||||
return capability.campaign_exists(
|
||||
session, tenant_id=tenant_id, campaign_id=campaign_id
|
||||
) and capability.can_read_campaign(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
campaign_id=campaign_id,
|
||||
user_id=user_id,
|
||||
group_ids=group_ids,
|
||||
tenant_admin=_has_all_scopes(principal, ("tenant:*",)),
|
||||
)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
return campaign_visible
|
||||
@@ -13,6 +13,7 @@ from govoplan_core.core.modules import (
|
||||
DocumentationLink,
|
||||
DocumentationTopic,
|
||||
FrontendModule,
|
||||
FrontendRoute,
|
||||
MigrationSpec,
|
||||
ModuleContext,
|
||||
ModuleInterfaceProvider,
|
||||
@@ -22,9 +23,11 @@ from govoplan_core.core.modules import (
|
||||
PermissionDefinition,
|
||||
RoleTemplate,
|
||||
)
|
||||
from govoplan_core.core.views import ViewSurface
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.change_tracking import register_files_change_tracking
|
||||
from govoplan_files.backend.db import models as file_models # noqa: F401 - populate Files ORM metadata
|
||||
from govoplan_files.backend.documentation import documentation_topics
|
||||
|
||||
register_files_change_tracking()
|
||||
|
||||
@@ -137,6 +140,43 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
|
||||
}
|
||||
|
||||
|
||||
def _tenant_summary_batch(session, tenant_ids) -> dict[str, dict[str, int]]:
|
||||
from sqlalchemy import func
|
||||
|
||||
from govoplan_files.backend.db.models import FileAsset, FileConnectorCredential, FileConnectorPolicy, FileConnectorProfile, FileConnectorSpace
|
||||
|
||||
ids = tuple(dict.fromkeys(str(tenant_id) for tenant_id in tenant_ids if tenant_id))
|
||||
if not ids:
|
||||
return {}
|
||||
counts: dict[str, dict[str, int]] = {
|
||||
tenant_id: {
|
||||
"files": 0,
|
||||
"connector_credentials": 0,
|
||||
"connector_policies": 0,
|
||||
"connector_profiles": 0,
|
||||
"connector_spaces": 0,
|
||||
}
|
||||
for tenant_id in ids
|
||||
}
|
||||
models = (
|
||||
("files", FileAsset),
|
||||
("connector_credentials", FileConnectorCredential),
|
||||
("connector_policies", FileConnectorPolicy),
|
||||
("connector_profiles", FileConnectorProfile),
|
||||
("connector_spaces", FileConnectorSpace),
|
||||
)
|
||||
for count_key, model in models:
|
||||
rows = (
|
||||
session.query(model.tenant_id, func.count(model.id))
|
||||
.filter(model.tenant_id.in_(ids))
|
||||
.group_by(model.tenant_id)
|
||||
.all()
|
||||
)
|
||||
for tenant_id, count in rows:
|
||||
counts[tenant_id][count_key] = int(count)
|
||||
return counts
|
||||
|
||||
|
||||
def _veto_group_delete(session, tenant_id: str, group_id: str) -> None:
|
||||
from govoplan_files.backend.db.models import FileAsset, FileConnectorSpace, FileFolder, FileShare
|
||||
|
||||
@@ -186,123 +226,222 @@ manifest = ModuleManifest(
|
||||
route_factory=_files_router,
|
||||
role_templates=ROLE_TEMPLATES,
|
||||
tenant_summary_providers=(_tenant_summary,),
|
||||
tenant_summary_batch_providers=(_tenant_summary_batch,),
|
||||
delete_veto_providers={"group": (_veto_group_delete,)},
|
||||
nav_items=(NavItem(path="/files", label="Files", icon="folder", required_any=("files:file:read",), order=40),),
|
||||
frontend=FrontendModule(
|
||||
module_id="files",
|
||||
package_name="@govoplan/files-webui",
|
||||
routes=(
|
||||
FrontendRoute(
|
||||
path="/files",
|
||||
component="FilesPage",
|
||||
required_any=("files:file:read",),
|
||||
order=40,
|
||||
),
|
||||
),
|
||||
nav_items=(NavItem(path="/files", label="Files", icon="folder", required_any=("files:file:read",), order=40),),
|
||||
view_surfaces=(
|
||||
ViewSurface(id="files.admin.system-connectors", module_id="files", kind="section", label="System file connections", order=75),
|
||||
ViewSurface(id="files.admin.tenant-connectors", module_id="files", kind="section", label="Tenant file connections", order=65),
|
||||
ViewSurface(id="files.admin.group-connectors", module_id="files", kind="section", label="Group file connections", order=65),
|
||||
ViewSurface(id="files.admin.user-connectors", module_id="files", kind="section", label="User file connections", order=65),
|
||||
ViewSurface(id="files.settings.connectors", module_id="files", kind="section", label="Personal file connections", order=20),
|
||||
ViewSurface(id="files.widget.spaces", module_id="files", kind="section", label="File spaces widget", order=35),
|
||||
),
|
||||
),
|
||||
documentation=(
|
||||
DocumentationTopic(
|
||||
id="files.workflow.upload-organize-and-share",
|
||||
title="Upload, organize, and share managed files",
|
||||
summary="Put files in a personal or group space, organize them with explicit conflict handling, and grant or update access through a supported integration or API client.",
|
||||
id="files.workflow.organize-managed-files",
|
||||
title="Organize managed files and folders",
|
||||
summary="Create folders and rename, move, or copy accessible managed content with explicit conflict handling.",
|
||||
body=(
|
||||
"Work in My files or an accessible group space. Uploads, folders, renames, moves, and copies stay inside governed managed storage and require an explicit choice when a target path already exists. "
|
||||
"A caller with share permission can grant or update read, write, or manage access for a user, group, tenant, or campaign without changing ownership. The Files page does not yet provide a general share editor, and the API has no share-revocation route; sharing is currently performed by a supporting workflow or API client."
|
||||
"Organization stays inside governed personal or group spaces. Moves preserve the asset identity, while copies create new assets and versions that reuse immutable blob bytes. "
|
||||
"Every target conflict must be rejected, renamed, overwritten, or skipped explicitly."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "file_manager", "process_participant"),
|
||||
order=39,
|
||||
order=42,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:organize",
|
||||
"files:file:share",
|
||||
),
|
||||
required_scopes=("files:file:read", "files:file:organize"),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(label="Upload API", href="/api/v1/files/upload", kind="api"),
|
||||
DocumentationLink(label="Move or copy API", href="/api/v1/files/transfer", kind="api"),
|
||||
DocumentationLink(label="Grant or update a share", href="/api/v1/files/{file_id}/shares", kind="api"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
related_modules=("campaigns",),
|
||||
unlocks=("Users and connected processes can exchange governed managed files without changing file ownership.",),
|
||||
unlocks=("Managed content can be placed at stable logical paths without bypassing space access.",),
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"Files is installed and you may read, upload, organize, and share managed files.",
|
||||
"You can access the destination personal or group space.",
|
||||
"A supporting workflow or API client is available when a user, group, tenant, or campaign share must be granted or updated.",
|
||||
"You may view and organize managed files.",
|
||||
"You have write or owner access to every source item and destination space used by the operation; the global organize permission alone does not grant resource access.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files, choose My files or an accessible group space, and open the intended destination folder.",
|
||||
"Create folders where needed, then upload or drag files into the destination.",
|
||||
"Resolve every name conflict explicitly by rejecting, renaming, overwriting, or skipping the affected item.",
|
||||
"Use rename, move, or copy to place the managed items at their intended logical paths.",
|
||||
"Review the current version, checksum, owner, and path before granting access.",
|
||||
"Through the supporting workflow or API, grant or update the required read, write, or manage share; do not promise revocation because no revoke route exists yet.",
|
||||
"Have the intended recipient verify access, and verify that an unrelated account remains denied.",
|
||||
"Open Files and select the personal or group space to organize.",
|
||||
"Create the required destination folders or select the files and folders to rename, move, or copy.",
|
||||
"Choose the destination and resolve each target conflict explicitly.",
|
||||
"Apply the operation and reopen the destination folder.",
|
||||
],
|
||||
"outcome": "The content is stored as governed managed assets in the intended space and the requested access has been granted or updated without changing ownership.",
|
||||
"verification": "Reopen the files to confirm owner, logical path, current version, and checksum, then test one intended and one denied access path. A share that must be removed requires an explicit future revocation capability.",
|
||||
"outcome": "The selected content has the intended governed owner and logical path.",
|
||||
"verification": "Confirm each resulting path and owner; for a move, also confirm the old path is gone, and for a copy, confirm the source remains.",
|
||||
"related_topic_ids": [
|
||||
"files.workflow.import-managed-snapshot",
|
||||
"files.assurance.process-and-release-readiness",
|
||||
"files.workflow.upload-managed-files",
|
||||
"files.workflow.find-and-download-files",
|
||||
"files.workflow.delete-managed-files",
|
||||
],
|
||||
},
|
||||
),
|
||||
DocumentationTopic(
|
||||
id="files.workflow.find-and-download-files",
|
||||
title="Find and download managed files",
|
||||
summary="Search accessible managed content and download a current file version or a ZIP archive of a selection.",
|
||||
body=(
|
||||
"Files can be sorted and searched by logical path or name pattern. A download always uses the accessible current version; a multi-file selection can be generated as a temporary ZIP archive. "
|
||||
"There is no dedicated content-preview service yet."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "file_manager", "process_participant"),
|
||||
order=43,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=("files:file:read", "files:file:download"),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
unlocks=("Authorized users can retrieve governed current versions without gaining organization or sharing authority.",),
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": ["You may view and download managed files in the relevant space."],
|
||||
"steps": [
|
||||
"Open Files and select the relevant personal or group space.",
|
||||
"Navigate folders, sort the list, or use a path/name pattern to find the intended content.",
|
||||
"Review the displayed owner, path, size, checksum, and version details.",
|
||||
"Download one current file version, or select several files and choose Download ZIP.",
|
||||
],
|
||||
"outcome": "The authorized current file bytes or generated archive are downloaded to the local device.",
|
||||
"verification": "Confirm the downloaded names and, where integrity matters, compare the file bytes with the displayed checksum.",
|
||||
"related_topic_ids": [
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.reference.snapshot-provenance-and-capabilities",
|
||||
],
|
||||
},
|
||||
),
|
||||
DocumentationTopic(
|
||||
id="files.workflow.import-managed-snapshot",
|
||||
title="Import an external file as a governed snapshot",
|
||||
summary="Browse an authorized connection read-only, import one selected file into managed storage, and use the frozen version in another GovOPlaN task.",
|
||||
id="files.workflow.share-managed-files",
|
||||
title="Grant or update access to managed files",
|
||||
summary="Grant or update read, write, or manage access through a supporting workflow or API client without changing ownership.",
|
||||
body=(
|
||||
"Choose a visible file connection in Files, browse only the permitted remote path, and import the selected content into your personal or group space. "
|
||||
"The managed copy records source provenance and remains unchanged until an explicit manual sync. Browse, import, and sync never mutate the remote source."
|
||||
"The Files service can grant or update a share for a user, group, tenant, or campaign. The Files page does not yet provide a general share editor, and the API has no share-revocation route. "
|
||||
"Do not use this task when access must later be revoked until that explicit capability is implemented."
|
||||
),
|
||||
layer="configured",
|
||||
layer="available",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "campaign_manager", "report_author"),
|
||||
order=40,
|
||||
audience=("file_manager", "process_participant"),
|
||||
order=44,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=("files:file:read", "files:file:upload"),
|
||||
required_scopes=("files:file:read", "files:file:share"),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(label="Visible connector profiles", href="/api/v1/files/connectors/profiles", kind="api"),
|
||||
DocumentationLink(
|
||||
label="Connector import API",
|
||||
href="/api/v1/files/connectors/profiles/{profile_id}/import",
|
||||
kind="api",
|
||||
),
|
||||
DocumentationLink(label="Grant or update a share", href="/api/v1/files/{file_id}/shares", kind="api"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
related_modules=("campaigns",),
|
||||
unlocks=("Campaigns, reports, and workflows can consume a managed snapshot with stable source evidence.",),
|
||||
unlocks=("A supporting process can grant governed file access without changing file ownership.",),
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"Files is installed and you may read and upload files.",
|
||||
"An administrator has configured a connector profile visible in your current user, group, tenant, or campaign scope.",
|
||||
"You may view and share the managed file and have write/manage access to that specific asset; the global share permission alone does not grant resource access.",
|
||||
"A supporting workflow or API client is available; the Files page has no general share editor yet.",
|
||||
"The process does not require share revocation, because no revocation route exists yet.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and choose a managed destination space.",
|
||||
"Choose Sync from connection, select a visible profile, and browse to the permitted remote file.",
|
||||
"Import or sync the selected file and resolve any destination conflict explicitly.",
|
||||
"Review the managed file's source and current-version details before using it in another task.",
|
||||
"Open Files and verify the file owner, logical path, current version, and checksum.",
|
||||
"Through the supporting workflow, identify the intended user, group, tenant, or campaign and choose read, write, or manage access.",
|
||||
"Grant or update the share without changing file ownership.",
|
||||
"Have the intended recipient verify access and an unrelated account verify denial.",
|
||||
],
|
||||
"outcome": "The external content is a tenant-managed snapshot with a checksum, exact version, and recorded source context.",
|
||||
"verification": "Reopen the managed file and confirm its connector/provider, remote identity or path, source revision when available, checksum, and current version.",
|
||||
"limitations": [
|
||||
"The Files page has no general share editor.",
|
||||
"Shares can be granted or updated, but not revoked through the current API.",
|
||||
],
|
||||
"outcome": "The requested access is granted or updated while the managed asset keeps its owner.",
|
||||
"verification": "Test one intended and one denied access path. Do not claim that the share can later be revoked.",
|
||||
"related_topic_ids": [
|
||||
"files.governed-connectors-and-provenance",
|
||||
"files.reference.integrity-recovery-and-fail-closed-transports",
|
||||
"files.reference.snapshot-provenance-and-capabilities",
|
||||
"files.workflow.find-and-download-files",
|
||||
"files.assurance.process-and-release-readiness",
|
||||
],
|
||||
},
|
||||
),
|
||||
DocumentationTopic(
|
||||
id="files.workflow.delete-managed-files",
|
||||
title="Delete managed files and folders",
|
||||
summary="Soft-delete accessible managed files or a folder tree where current policy allows it.",
|
||||
body=(
|
||||
"Deletion hides the selected managed assets rather than hard-purging their stored evidence. Folder deletion is recursive by default and includes child folders and files; a non-recursive request fails for a non-empty folder. "
|
||||
"There is no self-service restore or hard-purge workflow today."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("user",),
|
||||
audience=("file_user", "file_manager", "process_participant"),
|
||||
order=45,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
required_scopes=("files:file:read", "files:file:delete"),
|
||||
),
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
unlocks=("Authorized users can remove obsolete content from active file views while preserving the current soft-delete boundary.",),
|
||||
metadata={
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"You may view and delete the selected managed content and have write or owner access to every affected asset or folder.",
|
||||
"You have reviewed the complete folder tree when deleting recursively.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and select the files or folder to delete.",
|
||||
"Review the selection and, for a folder, all content below it.",
|
||||
"Confirm the delete action.",
|
||||
"Refresh or reopen the space and verify that the selected paths are no longer active.",
|
||||
],
|
||||
"limitations": [
|
||||
"Deletion is soft deletion, not a hard purge.",
|
||||
"There is no self-service restore or hard-purge workflow.",
|
||||
],
|
||||
"outcome": "The selected content is hidden from active Files views under the current soft-delete model.",
|
||||
"verification": "Confirm the deleted paths no longer appear in the active space; do not treat the action as physical erasure.",
|
||||
"related_topic_ids": [
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.assurance.process-and-release-readiness",
|
||||
],
|
||||
},
|
||||
),
|
||||
@@ -317,7 +456,7 @@ manifest = ModuleManifest(
|
||||
layer="configured",
|
||||
documentation_types=("admin",),
|
||||
audience=("file_admin", "tenant_admin", "system_admin", "security_auditor"),
|
||||
order=41,
|
||||
order=50,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
@@ -374,7 +513,7 @@ manifest = ModuleManifest(
|
||||
layer="configured",
|
||||
documentation_types=("admin",),
|
||||
audience=("operator", "system_admin", "security_auditor"),
|
||||
order=42,
|
||||
order=51,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
@@ -428,7 +567,7 @@ manifest = ModuleManifest(
|
||||
layer="available",
|
||||
documentation_types=("admin", "user"),
|
||||
audience=("module_integrator", "file_admin", "campaign_admin", "process_designer"),
|
||||
order=43,
|
||||
order=52,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
@@ -467,7 +606,7 @@ manifest = ModuleManifest(
|
||||
layer="configured",
|
||||
documentation_types=("admin", "user"),
|
||||
audience=("process_owner", "release_manager", "file_admin", "operator", "security_auditor"),
|
||||
order=44,
|
||||
order=53,
|
||||
conditions=(
|
||||
DocumentationCondition(
|
||||
required_modules=("files",),
|
||||
@@ -500,6 +639,7 @@ manifest = ModuleManifest(
|
||||
"kind": "workflow",
|
||||
"route": "/files",
|
||||
"screen": "Files process and release assurance",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"A named process owner has defined the intended users, data classification, retention expectations, and integrations.",
|
||||
"A candidate release is installed on a clean database and an upgrade copy with aligned Python, root package, WebUI package, and module-manifest versions.",
|
||||
@@ -517,7 +657,12 @@ manifest = ModuleManifest(
|
||||
"outcome": "The process or release has an explicit approval record tied to aligned versions, representative evidence, known limitations, and owned residual risks.",
|
||||
"verification": "A reviewer can reproduce the recorded allow/deny, integrity, connector, and recovery checks and can trace each unmet requirement to a documented limitation or accepted compensating control.",
|
||||
"related_topic_ids": [
|
||||
"files.workflow.upload-organize-and-share",
|
||||
"files.workflow.upload-managed-files",
|
||||
"files.workflow.upload-and-unpack-zip",
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.workflow.find-and-download-files",
|
||||
"files.workflow.share-managed-files",
|
||||
"files.workflow.delete-managed-files",
|
||||
"files.workflow.import-managed-snapshot",
|
||||
"files.governed-connectors-and-provenance",
|
||||
"files.reference.integrity-recovery-and-fail-closed-transports",
|
||||
@@ -526,6 +671,7 @@ manifest = ModuleManifest(
|
||||
},
|
||||
),
|
||||
),
|
||||
documentation_providers=(documentation_topics,),
|
||||
migration_spec=MigrationSpec(
|
||||
module_id="files",
|
||||
metadata=Base.metadata,
|
||||
|
||||
1315
src/govoplan_files/backend/route_support.py
Normal file
1315
src/govoplan_files/backend/route_support.py
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1
src/govoplan_files/backend/routes/__init__.py
Normal file
1
src/govoplan_files/backend/routes/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Focused HTTP route modules for the Files API."""
|
||||
134
src/govoplan_files/backend/routes/assets.py
Normal file
134
src/govoplan_files/backend/routes/assets.py
Normal file
@@ -0,0 +1,134 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from io import BytesIO
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
BulkDeleteRequest,
|
||||
BulkDeleteResponse,
|
||||
FileAssetResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
read_asset_bytes,
|
||||
soft_delete_assets,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_attachment_disposition,
|
||||
_audit_connector_event,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/{file_id}", response_model=FileAssetResponse)
|
||||
def get_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return _asset_response(session, asset, include_shares=True)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.get("/{file_id}/download")
|
||||
def download_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:download")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
data, version, blob = read_asset_bytes(session, asset)
|
||||
_audit_connector_event(
|
||||
session,
|
||||
principal,
|
||||
action="files.connector.accessed",
|
||||
asset=asset,
|
||||
version=version,
|
||||
blob=blob,
|
||||
operation="download",
|
||||
commit=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
headers = {"Content-Disposition": _attachment_disposition(asset.filename)}
|
||||
return StreamingResponse(
|
||||
BytesIO(data),
|
||||
media_type=blob.content_type or "application/octet-stream",
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/{file_id}", response_model=BulkDeleteResponse)
|
||||
def delete_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
count = soft_delete_assets(session, [asset])
|
||||
session.commit()
|
||||
return BulkDeleteResponse(deleted_count=count)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.post("/bulk-delete", response_model=BulkDeleteResponse)
|
||||
def bulk_delete_files(
|
||||
payload: BulkDeleteRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
assets = [
|
||||
get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in payload.file_ids
|
||||
]
|
||||
count = soft_delete_assets(session, assets)
|
||||
session.commit()
|
||||
return BulkDeleteResponse(deleted_count=count)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
252
src/govoplan_files/backend/routes/connector_io.py
Normal file
252
src/govoplan_files/backend/routes/connector_io.py
Normal file
@@ -0,0 +1,252 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorBrowseItem,
|
||||
FileConnectorBrowseResponse,
|
||||
FileConnectorImportRequest,
|
||||
FileConnectorSyncResponse,
|
||||
FileUploadResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_browse import (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
browse_connector_profile,
|
||||
normalize_connector_browse_path,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_imports import (
|
||||
ConnectorImportError,
|
||||
ConnectorImportUnsupported,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicyDenied,
|
||||
connector_policy_decision,
|
||||
)
|
||||
from govoplan_files.backend.storage.files import (
|
||||
create_file_asset,
|
||||
sync_file_asset_from_source,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_audit_connector_imports,
|
||||
_audit_connector_sync,
|
||||
_connector_browse_next_token,
|
||||
_connector_policy_error,
|
||||
_download_connector_payload,
|
||||
_ensure_campaign_file_access,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles/{profile_id}/import", response_model=FileUploadResponse
|
||||
)
|
||||
def import_connector_file(
|
||||
profile_id: str,
|
||||
payload: FileConnectorImportRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
try:
|
||||
if payload.campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=payload.campaign_id
|
||||
)
|
||||
_source_path, downloaded, metadata = _download_connector_payload(
|
||||
profile, payload, operation="import"
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
stored = create_file_asset(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=downloaded.filename,
|
||||
data=downloaded.data,
|
||||
folder=payload.target_folder,
|
||||
display_path=payload.target_path,
|
||||
content_type=downloaded.content_type,
|
||||
metadata=metadata,
|
||||
campaign_id=payload.campaign_id,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_audit_connector_imports(session, principal, [stored.asset])
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorImportUnsupported as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (
|
||||
ConnectorImportError,
|
||||
FileStorageError,
|
||||
UnsafeFilePathError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileUploadResponse(
|
||||
files=[_asset_response(session, stored.asset, include_shares=True)]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles/{profile_id}/sync", response_model=FileConnectorSyncResponse
|
||||
)
|
||||
def sync_connector_file(
|
||||
profile_id: str,
|
||||
payload: FileConnectorImportRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
try:
|
||||
if payload.campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=payload.campaign_id
|
||||
)
|
||||
_source_path, downloaded, metadata = _download_connector_payload(
|
||||
profile, payload, operation="sync"
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
stored, sync_action, previous_version_id = sync_file_asset_from_source(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=downloaded.filename,
|
||||
data=downloaded.data,
|
||||
folder=payload.target_folder,
|
||||
display_path=payload.target_path,
|
||||
content_type=downloaded.content_type,
|
||||
metadata=metadata,
|
||||
campaign_id=payload.campaign_id,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_audit_connector_sync(
|
||||
session,
|
||||
principal,
|
||||
stored.asset,
|
||||
sync_action=sync_action,
|
||||
previous_version_id=previous_version_id,
|
||||
)
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorImportUnsupported as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (
|
||||
ConnectorImportError,
|
||||
FileStorageError,
|
||||
UnsafeFilePathError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorSyncResponse(
|
||||
file=_asset_response(session, stored.asset, include_shares=True),
|
||||
action=sync_action,
|
||||
previous_version_id=previous_version_id,
|
||||
current_version_id=stored.version.id,
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/profiles/{profile_id}/browse",
|
||||
response_model=FileConnectorBrowseResponse,
|
||||
)
|
||||
def browse_connector_profile_items(
|
||||
profile_id: str,
|
||||
path: str | None = None,
|
||||
library_id: str | None = None,
|
||||
continuation_token: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=campaign_id
|
||||
)
|
||||
browse_path = normalize_connector_browse_path(path)
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id=profile.id,
|
||||
credential_id=profile.credential_profile_id,
|
||||
provider=profile.provider,
|
||||
external_path=browse_path,
|
||||
external_url=connector_effective_endpoint_url(
|
||||
provider=profile.provider,
|
||||
endpoint_url=profile.endpoint_url,
|
||||
metadata=profile.metadata,
|
||||
),
|
||||
operation="browse",
|
||||
),
|
||||
profile.policy_sources,
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
items = browse_connector_profile(
|
||||
profile,
|
||||
path=browse_path,
|
||||
library_id=library_id,
|
||||
continuation_token=continuation_token,
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorBrowseUnsupported as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (ConnectorBrowseError, OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorBrowseResponse(
|
||||
profile_id=profile.id,
|
||||
provider=profile.provider,
|
||||
path=browse_path,
|
||||
library_id=library_id,
|
||||
next_continuation_token=_connector_browse_next_token(items),
|
||||
has_more=any(bool(item.metadata.get("listing_truncated")) for item in items),
|
||||
decision=decision.to_dict(),
|
||||
items=[FileConnectorBrowseItem(**item.to_response()) for item in items],
|
||||
)
|
||||
261
src/govoplan_files/backend/routes/connector_profiles.py
Normal file
261
src/govoplan_files/backend/routes/connector_profiles.py
Normal file
@@ -0,0 +1,261 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorProfileResponse,
|
||||
FileConnectorProfileUpdateRequest,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_credential_deletion import (
|
||||
delete_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
connector_profile_from_row,
|
||||
get_connector_profile_row,
|
||||
update_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_policy_error,
|
||||
_credential_row_for_profile,
|
||||
_ensure_connector_configuration_allowed,
|
||||
_ensure_connector_local_policy_allowed,
|
||||
_http_error,
|
||||
_record_connector_settings_change,
|
||||
_require_connector_profile_write,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def get_connector_profile(
|
||||
profile_id: str,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session,
|
||||
principal,
|
||||
profile_id,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorProfileResponse(**profile.to_response())
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def update_connector_profile(
|
||||
profile_id: str,
|
||||
payload: FileConnectorProfileUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
profile_id=profile_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_profile_write(principal, row.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
credential_profile_id = (
|
||||
payload.credential_profile_id
|
||||
if payload.credential_profile_id is not None
|
||||
else row.credential_profile_id
|
||||
)
|
||||
provider = payload.provider if payload.provider is not None else row.provider
|
||||
credential_row = _credential_row_for_profile(
|
||||
session,
|
||||
principal,
|
||||
credential_profile_id=credential_profile_id,
|
||||
provider=provider,
|
||||
profile_id=row.id,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=row.id,
|
||||
credential_id=credential_profile_id,
|
||||
provider=provider,
|
||||
endpoint_url=connector_effective_endpoint_url(
|
||||
provider=provider,
|
||||
endpoint_url=payload.endpoint_url
|
||||
if payload.endpoint_url is not None
|
||||
else row.endpoint_url,
|
||||
metadata=payload.metadata
|
||||
if payload.metadata is not None
|
||||
else row.metadata_,
|
||||
),
|
||||
base_path=payload.base_path
|
||||
if payload.base_path is not None
|
||||
else row.base_path,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
operation="configure",
|
||||
)
|
||||
if payload.policy is not None:
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
update_connector_profile_row(
|
||||
session,
|
||||
row,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
enabled=payload.enabled,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username if credentials else None,
|
||||
password=credentials.password if credentials else None,
|
||||
token=credentials.token if credentials else None,
|
||||
password_env=credentials.password_env if credentials else None,
|
||||
token_env=credentials.token_env if credentials else None,
|
||||
secret_ref=credentials.secret_ref if credentials else None,
|
||||
clear_password=payload.clear_password,
|
||||
clear_token=payload.clear_token,
|
||||
capabilities=payload.capabilities,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(
|
||||
row, credential_row=credential_row
|
||||
).to_response()
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def deactivate_connector_profile(
|
||||
profile_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
profile_id=profile_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_profile_write(principal, row.scope_type)
|
||||
try:
|
||||
changed = delete_connector_profile_row(
|
||||
session,
|
||||
row,
|
||||
deletion_reason="api_delete",
|
||||
user_id=principal.user.id,
|
||||
api_key_id=principal.api_key.id if principal.api_key else None,
|
||||
)
|
||||
if changed:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(row).to_response()
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
852
src/govoplan_files/backend/routes/connector_settings.py
Normal file
852
src/govoplan_files/backend/routes/connector_settings.py
Normal file
@@ -0,0 +1,852 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorCredentialCreateRequest,
|
||||
FileConnectorCredentialResponse,
|
||||
FileConnectorCredentialsResponse,
|
||||
FileConnectorCredentialUpdateRequest,
|
||||
FileConnectorDiscoveryRequest,
|
||||
FileConnectorDiscoveryResponse,
|
||||
FileConnectorSettingsDeltaResponse,
|
||||
FileConnectorPolicyEvaluateRequest,
|
||||
FileConnectorPolicyEvaluateResponse,
|
||||
FileConnectorPolicyResponse,
|
||||
FileConnectorPolicyUpdateRequest,
|
||||
FileConnectorProfileCreateRequest,
|
||||
FileConnectorProfileResponse,
|
||||
FileConnectorProfilesResponse,
|
||||
FileConnectorProviderResponse,
|
||||
FileConnectorProvidersResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
create_connector_credential_row,
|
||||
get_connector_credential_row,
|
||||
list_database_connector_credentials,
|
||||
update_connector_credential_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_credential_deletion import (
|
||||
delete_connector_credential_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_browse import (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
browse_connector_profile,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
reject_api_controlled_deployment_references,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
connector_profile_from_row,
|
||||
create_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_providers import (
|
||||
connector_provider_descriptors,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicyDenied,
|
||||
connector_policy_decision,
|
||||
connector_policy_sources_from_payload,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy_store import (
|
||||
connector_policy_response,
|
||||
set_connector_policy,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
FILES_CONNECTOR_POLICY_RESOURCE,
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
_audit_connector_discovery_attempt,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_credential_response,
|
||||
_connector_policy_error,
|
||||
_credential_row_for_profile,
|
||||
_discovery_profile_from_payload,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_connector_configuration_allowed,
|
||||
_ensure_connector_credential_configuration_allowed,
|
||||
_ensure_connector_local_policy_allowed,
|
||||
_file_connector_policy_resource_id,
|
||||
_file_connector_settings_entries,
|
||||
_http_error,
|
||||
_record_connector_settings_change,
|
||||
_require_connector_credential_write,
|
||||
_require_connector_policy_read,
|
||||
_require_connector_profile_write,
|
||||
_same_endpoint,
|
||||
_visible_connector_profiles,
|
||||
_webdav_discovery_candidates,
|
||||
)
|
||||
from govoplan_files.backend.services.connector_settings_delta import (
|
||||
_full_file_connector_settings_delta_response,
|
||||
_incremental_file_connector_settings_delta_response,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connector-policy/evaluate", response_model=FileConnectorPolicyEvaluateResponse
|
||||
)
|
||||
def evaluate_connector_policy(
|
||||
payload: FileConnectorPolicyEvaluateRequest,
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope("files:file:read", "files:file:upload", "files:file:download")
|
||||
),
|
||||
):
|
||||
del principal
|
||||
sources = connector_policy_sources_from_payload(
|
||||
[item.model_dump(mode="json") for item in payload.policy_sources]
|
||||
)
|
||||
request = ConnectorAccessRequest.from_provenance(
|
||||
payload.source_provenance.model_dump(mode="json", exclude_none=True),
|
||||
operation=payload.operation,
|
||||
)
|
||||
return FileConnectorPolicyEvaluateResponse(
|
||||
decision=connector_policy_decision(request, sources).to_dict()
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/settings/delta", response_model=FileConnectorSettingsDeltaResponse
|
||||
)
|
||||
def connector_settings_delta(
|
||||
scope_type: str = Query(default="tenant"),
|
||||
scope_id: str | None = Query(default=None),
|
||||
provider: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
include_inactive: bool = False,
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
since: str | None = None,
|
||||
limit: int = Query(default=100, ge=1, le=500),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
scope_type = scope_type.strip().casefold()
|
||||
_require_connector_policy_read(principal, scope_type)
|
||||
try:
|
||||
if since is None:
|
||||
return _full_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
entries, has_more = _file_connector_settings_entries(
|
||||
session, tenant_id=principal.tenant_id, since=since, limit=limit
|
||||
)
|
||||
if entries is None:
|
||||
return _full_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
return _incremental_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
entries=entries,
|
||||
has_more=has_more,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.get("/connectors/providers", response_model=FileConnectorProvidersResponse)
|
||||
def list_connector_providers(
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope("files:file:read", "files:file:upload", "files:file:admin")
|
||||
),
|
||||
):
|
||||
del principal
|
||||
return FileConnectorProvidersResponse(
|
||||
providers=[
|
||||
FileConnectorProviderResponse(**item.to_response())
|
||||
for item in connector_provider_descriptors()
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post("/connectors/discover", response_model=FileConnectorDiscoveryResponse)
|
||||
def discover_connector_endpoint(
|
||||
payload: FileConnectorDiscoveryRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
reject_api_controlled_deployment_references(
|
||||
password_env=payload.credentials.password_env,
|
||||
token_env=payload.credentials.token_env,
|
||||
secret_ref=payload.credentials.secret_ref,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
if payload.provider not in {"webdav", "nextcloud"}:
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=None,
|
||||
base_path=payload.base_path,
|
||||
status="unsupported",
|
||||
message=f"Discovery is not implemented for {payload.provider} connectors yet",
|
||||
)
|
||||
candidates: list[dict[str, str]] = []
|
||||
for endpoint_url in _webdav_discovery_candidates(payload):
|
||||
profile = _discovery_profile_from_payload(
|
||||
payload, endpoint_url, principal=principal
|
||||
)
|
||||
try:
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=None,
|
||||
credential_id=None,
|
||||
provider=profile.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=profile.base_path,
|
||||
scope_type="tenant",
|
||||
scope_id=principal.tenant_id,
|
||||
operation="discover",
|
||||
)
|
||||
_audit_connector_discovery_attempt(
|
||||
session,
|
||||
principal,
|
||||
provider=profile.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=profile.base_path,
|
||||
)
|
||||
browse_connector_profile(profile, path=payload.base_path or "")
|
||||
except ConnectorPolicyDenied as exc:
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
OSError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
message = str(exc)
|
||||
if "credentials were rejected" in message.casefold():
|
||||
if payload.require_valid_credentials:
|
||||
candidates.append(
|
||||
{
|
||||
"endpoint_url": endpoint_url,
|
||||
"status": "credentials_rejected",
|
||||
"message": "The endpoint exists, but the credentials were rejected.",
|
||||
}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status="credentials_rejected",
|
||||
message="The endpoint was found, but login failed with these credentials.",
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||
)
|
||||
candidates.append(
|
||||
{
|
||||
"endpoint_url": endpoint_url,
|
||||
"status": "found",
|
||||
"message": "The endpoint exists, but credentials are required or were rejected.",
|
||||
}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status="found",
|
||||
message="The endpoint was found. Add working credentials before saving or testing the connection.",
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||
)
|
||||
candidates.append(
|
||||
{"endpoint_url": endpoint_url, "status": "failed", "message": message}
|
||||
)
|
||||
continue
|
||||
status_value = (
|
||||
"usable" if _same_endpoint(endpoint_url, payload.endpoint_url) else "found"
|
||||
)
|
||||
message = (
|
||||
"The supplied URL is directly usable."
|
||||
if status_value == "usable"
|
||||
else "A usable connector endpoint was discovered."
|
||||
)
|
||||
candidates.append(
|
||||
{"endpoint_url": endpoint_url, "status": status_value, "message": message}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status=status_value,
|
||||
message=message,
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-propfind"},
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=None,
|
||||
base_path=payload.base_path,
|
||||
status="not_found",
|
||||
message="No usable WebDAV endpoint was found for this server URL.",
|
||||
candidates=candidates,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/connectors/credentials", response_model=FileConnectorCredentialsResponse)
|
||||
def list_connector_credentials(
|
||||
provider: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
provider_norm = provider.strip().casefold() if provider else None
|
||||
credentials = list_database_connector_credentials(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
)
|
||||
return FileConnectorCredentialsResponse(
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in credentials
|
||||
if provider_norm is None or credential.provider in {None, provider_norm}
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/policies/{scope_type}", response_model=FileConnectorPolicyResponse
|
||||
)
|
||||
def read_connector_policy(
|
||||
scope_type: str,
|
||||
scope_id: str | None = Query(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_policy_read(principal, scope_type)
|
||||
try:
|
||||
return FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.put(
|
||||
"/connectors/policies/{scope_type}", response_model=FileConnectorPolicyResponse
|
||||
)
|
||||
def write_connector_policy(
|
||||
scope_type: str,
|
||||
payload: FileConnectorPolicyUpdateRequest,
|
||||
scope_id: str | None = Query(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_profile_write(principal, scope_type)
|
||||
try:
|
||||
set_connector_policy(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_POLICY_RESOURCE,
|
||||
resource_id=_file_connector_policy_resource_id(scope_type, scope_id),
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=None
|
||||
if scope_type.strip().casefold() == "system"
|
||||
else principal.tenant_id,
|
||||
payload={"scope_type": scope_type.strip().casefold(), "scope_id": scope_id},
|
||||
)
|
||||
session.commit()
|
||||
return FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/credentials",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_connector_credential(
|
||||
payload: FileConnectorCredentialCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_credential_write(principal, payload.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
_ensure_connector_credential_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
credential_id=payload.id,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
operation="configure_credentials",
|
||||
)
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
row = create_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
credential_id=payload.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
enabled=payload.enabled,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username,
|
||||
password=credentials.password,
|
||||
token=credentials.token,
|
||||
password_env=credentials.password_env,
|
||||
token_env=credentials.token_env,
|
||||
secret_ref=credentials.secret_ref,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def get_connector_credential(
|
||||
credential_id: str,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
)
|
||||
return _connector_credential_response(row)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def update_connector_credential(
|
||||
credential_id: str,
|
||||
payload: FileConnectorCredentialUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_credential_write(principal, row.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
provider = payload.provider if payload.provider is not None else row.provider
|
||||
_ensure_connector_credential_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
credential_id=row.id,
|
||||
provider=provider,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
operation="configure_credentials",
|
||||
)
|
||||
if payload.policy is not None:
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
update_connector_credential_row(
|
||||
session,
|
||||
row,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
enabled=payload.enabled,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username if credentials else None,
|
||||
password=credentials.password if credentials else None,
|
||||
token=credentials.token if credentials else None,
|
||||
password_env=credentials.password_env if credentials else None,
|
||||
token_env=credentials.token_env if credentials else None,
|
||||
secret_ref=credentials.secret_ref if credentials else None,
|
||||
clear_password=payload.clear_password,
|
||||
clear_token=payload.clear_token,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def deactivate_connector_credential(
|
||||
credential_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_credential_write(principal, row.scope_type)
|
||||
try:
|
||||
deletion = delete_connector_credential_row(
|
||||
session,
|
||||
row,
|
||||
deletion_reason="api_delete",
|
||||
user_id=principal.user.id,
|
||||
api_key_id=principal.api_key.id if principal.api_key else None,
|
||||
)
|
||||
if deletion.changed:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
for profile in deletion.affected_profiles:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=profile.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=profile.tenant_id,
|
||||
payload={
|
||||
"scope_type": profile.scope_type,
|
||||
"scope_id": profile.scope_id,
|
||||
"provider": profile.provider,
|
||||
"reason": "credential_deleted",
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.get("/connectors/profiles", response_model=FileConnectorProfilesResponse)
|
||||
def list_connector_profiles(
|
||||
provider: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
if campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorProfilesResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in profiles
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles",
|
||||
response_model=FileConnectorProfileResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_connector_profile(
|
||||
payload: FileConnectorProfileCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_profile_write(principal, payload.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
credential_row = _credential_row_for_profile(
|
||||
session,
|
||||
principal,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
provider=payload.provider,
|
||||
profile_id=payload.id,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
)
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=payload.id,
|
||||
credential_id=payload.credential_profile_id,
|
||||
provider=payload.provider,
|
||||
endpoint_url=connector_effective_endpoint_url(
|
||||
provider=payload.provider,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
metadata=payload.metadata,
|
||||
),
|
||||
base_path=payload.base_path,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
operation="configure",
|
||||
)
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
row = create_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
profile_id=payload.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
enabled=payload.enabled,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username,
|
||||
password=credentials.password,
|
||||
token=credentials.token,
|
||||
password_env=credentials.password_env,
|
||||
token_env=credentials.token_env,
|
||||
secret_ref=credentials.secret_ref,
|
||||
capabilities=payload.capabilities,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(
|
||||
row, credential_row=credential_row
|
||||
).to_response()
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
151
src/govoplan_files/backend/routes/folders.py
Normal file
151
src/govoplan_files/backend/routes/folders.py
Normal file
@@ -0,0 +1,151 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileFolderCreateRequest,
|
||||
FileFolderDeleteRequest,
|
||||
FileFolderDeleteResponse,
|
||||
FileFolderResponse,
|
||||
FileFoldersResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.folders import (
|
||||
create_folder,
|
||||
list_folders_for_user,
|
||||
list_folders_for_user_window,
|
||||
soft_delete_folder,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_folder_response,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
from govoplan_files.backend.services.list_queries import (
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
_cursor_page_size,
|
||||
_files_delta_watermark,
|
||||
_folder_cursor_values,
|
||||
_folders_list_fingerprint,
|
||||
_next_folder_list_cursor,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/folders", response_model=FileFoldersResponse)
|
||||
def list_file_folders(
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int | None = Query(default=None, ge=1, le=1000),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
watermark = _files_delta_watermark(session, principal.tenant_id)
|
||||
effective_page_size = _cursor_page_size(
|
||||
FOLDERS_LIST_CURSOR_SCOPE, cursor, page_size
|
||||
)
|
||||
if effective_page_size is None:
|
||||
folders = list_folders_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileFoldersResponse(
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
watermark=watermark,
|
||||
)
|
||||
fingerprint = _folders_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
page_size=effective_page_size,
|
||||
)
|
||||
after_path, after_id = _folder_cursor_values(cursor, fingerprint=fingerprint)
|
||||
folders, has_more = list_folders_for_user_window(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
page_size=effective_page_size,
|
||||
after_path=after_path,
|
||||
after_id=after_id,
|
||||
)
|
||||
return FileFoldersResponse(
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
cursor=cursor,
|
||||
next_cursor=_next_folder_list_cursor(
|
||||
principal,
|
||||
folders,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
page_size=effective_page_size,
|
||||
has_more=has_more,
|
||||
),
|
||||
watermark=watermark,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/folders", response_model=FileFolderResponse)
|
||||
def create_file_folder(
|
||||
payload: FileFolderCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
folder = create_folder(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
user_id=principal.user.id,
|
||||
path=payload.path,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return _folder_response(folder)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/folders/delete", response_model=FileFolderDeleteResponse)
|
||||
def delete_file_folder(
|
||||
payload: FileFolderDeleteRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
deleted_folders, deleted_files = soft_delete_folder(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
user_id=principal.user.id,
|
||||
path=payload.path,
|
||||
recursive=payload.recursive,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return FileFolderDeleteResponse(
|
||||
deleted_folders=deleted_folders, deleted_files=deleted_files
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
167
src/govoplan_files/backend/routes/listing.py
Normal file
167
src/govoplan_files/backend/routes/listing.py
Normal file
@@ -0,0 +1,167 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileDeltaResponse,
|
||||
FileListResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.files import (
|
||||
count_assets_for_user,
|
||||
list_assets_for_user,
|
||||
list_assets_for_user_window,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_list_response,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_list_owner_access,
|
||||
_is_admin,
|
||||
)
|
||||
from govoplan_files.backend.services.list_queries import (
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
_cursor_page_size,
|
||||
_file_cursor_values,
|
||||
_files_delta_response,
|
||||
_files_delta_watermark,
|
||||
_files_list_fingerprint,
|
||||
_full_file_delta_response,
|
||||
_next_file_list_cursor,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/delta", response_model=FileDeltaResponse)
|
||||
def files_delta(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
since: str | None = None,
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, owner_type, owner_id)
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
if since is None:
|
||||
return _full_file_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
return _files_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
since=since,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
|
||||
@router.get("", response_model=FileListResponse)
|
||||
def list_files(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
page_size: int | None = Query(default=None, ge=1, le=1000),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, owner_type, owner_id)
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
watermark = _files_delta_watermark(session, principal.tenant_id)
|
||||
total = count_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
effective_page_size = _cursor_page_size(FILES_LIST_CURSOR_SCOPE, cursor, page_size)
|
||||
if effective_page_size is not None:
|
||||
fingerprint = _files_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=effective_page_size,
|
||||
)
|
||||
after_display_path, after_updated_at, after_id = _file_cursor_values(
|
||||
cursor, fingerprint=fingerprint
|
||||
)
|
||||
assets, has_more = list_assets_for_user_window(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
page_size=effective_page_size,
|
||||
after_display_path=after_display_path,
|
||||
after_updated_at=after_updated_at,
|
||||
after_id=after_id,
|
||||
)
|
||||
return FileListResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
total=total,
|
||||
cursor=cursor,
|
||||
next_cursor=_next_file_list_cursor(
|
||||
principal,
|
||||
assets,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=effective_page_size,
|
||||
has_more=has_more,
|
||||
),
|
||||
watermark=watermark,
|
||||
)
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileListResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
total=total,
|
||||
watermark=watermark,
|
||||
)
|
||||
116
src/govoplan_files/backend/routes/shares.py
Normal file
116
src/govoplan_files/backend/routes/shares.py
Normal file
@@ -0,0 +1,116 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
BulkFileShareRequest,
|
||||
BulkFileShareResponse,
|
||||
FileShareRequest,
|
||||
FileShareResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
share_file,
|
||||
share_files,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post("/{file_id}/shares", response_model=FileShareResponse)
|
||||
def create_share(
|
||||
file_id: str,
|
||||
payload: FileShareRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
share = share_file(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset=asset,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
permission=payload.permission,
|
||||
user_id=principal.user.id,
|
||||
)
|
||||
session.commit()
|
||||
return FileShareResponse(
|
||||
id=share.id,
|
||||
target_type=share.target_type,
|
||||
target_id=share.target_id,
|
||||
permission=share.permission,
|
||||
created_at=share.created_at.isoformat(),
|
||||
revoked_at=share.revoked_at.isoformat() if share.revoked_at else None,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/bulk-shares", response_model=BulkFileShareResponse)
|
||||
def create_bulk_shares(
|
||||
payload: BulkFileShareRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
file_ids = list(dict.fromkeys(payload.file_ids))
|
||||
assets = [
|
||||
get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in file_ids
|
||||
]
|
||||
shares = share_files(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
assets=assets,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
permission=payload.permission,
|
||||
user_id=principal.user.id,
|
||||
)
|
||||
session.commit()
|
||||
return BulkFileShareResponse(
|
||||
shared_count=len(shares),
|
||||
shares=[
|
||||
FileShareResponse(
|
||||
id=share.id,
|
||||
target_type=share.target_type,
|
||||
target_id=share.target_id,
|
||||
permission=share.permission,
|
||||
created_at=share.created_at.isoformat(),
|
||||
revoked_at=share.revoked_at.isoformat()
|
||||
if share.revoked_at
|
||||
else None,
|
||||
)
|
||||
for share in shares
|
||||
],
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
292
src/govoplan_files/backend/routes/spaces.py
Normal file
292
src/govoplan_files/backend/routes/spaces.py
Normal file
@@ -0,0 +1,292 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorSpaceCreateRequest,
|
||||
FileConnectorSpaceResponse,
|
||||
FileConnectorSpacesResponse,
|
||||
FileConnectorSpaceUpdateRequest,
|
||||
FileSpaceResponse,
|
||||
FileSpacesResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.access import group_refs_for_ids, user_group_ids
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_spaces import (
|
||||
connector_space_owner_id,
|
||||
create_connector_space,
|
||||
get_connector_space_for_user,
|
||||
list_connector_spaces_for_user,
|
||||
soft_delete_connector_space,
|
||||
update_connector_space,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
_connector_policy_error,
|
||||
_connector_space_file_space_response,
|
||||
_connector_space_policy_decision,
|
||||
_connector_space_response,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_record_connector_settings_change,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/spaces", response_model=FileSpacesResponse)
|
||||
def list_file_spaces(
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
spaces = [
|
||||
FileSpaceResponse(
|
||||
id=f"user:{principal.user.id}",
|
||||
label="My files",
|
||||
owner_type="user",
|
||||
owner_id=principal.user.id,
|
||||
description="Files owned by your user account.",
|
||||
)
|
||||
]
|
||||
group_ids = user_group_ids(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
include_admin_groups=_is_admin(principal),
|
||||
)
|
||||
if group_ids:
|
||||
groups = group_refs_for_ids(tenant_id=principal.tenant_id, group_ids=group_ids)
|
||||
spaces.extend(
|
||||
FileSpaceResponse(
|
||||
id=f"group:{group.id}",
|
||||
label=f"{group.name} files",
|
||||
owner_type="group",
|
||||
owner_id=group.id,
|
||||
description="Files owned by this group.",
|
||||
)
|
||||
for group in groups
|
||||
)
|
||||
connector_spaces = list_connector_spaces_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
spaces.extend(
|
||||
_connector_space_file_space_response(space) for space in connector_spaces
|
||||
)
|
||||
return FileSpacesResponse(spaces=spaces)
|
||||
|
||||
|
||||
@router.get("/connector-spaces", response_model=FileConnectorSpacesResponse)
|
||||
def list_file_connector_spaces(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
include_inactive: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
spaces = list_connector_spaces_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive and _is_admin(principal),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileConnectorSpacesResponse(
|
||||
spaces=[_connector_space_response(space) for space in spaces]
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connector-spaces",
|
||||
response_model=FileConnectorSpaceResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_file_connector_space(
|
||||
payload: FileConnectorSpaceCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
if payload.owner_type == "group" and not payload.owner_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="owner_id is required for group connector spaces",
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, payload.connector_profile_id
|
||||
)
|
||||
decision = _connector_space_policy_decision(
|
||||
profile,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
operation="link",
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
space = create_connector_space(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
profile=profile,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
sync_mode=payload.sync_mode,
|
||||
metadata=payload.metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.patch("/connector-spaces/{space_id}", response_model=FileConnectorSpaceResponse)
|
||||
def update_file_connector_space(
|
||||
space_id: str,
|
||||
payload: FileConnectorSpaceUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
space = get_connector_space_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
space_id=space_id,
|
||||
include_inactive=_is_admin(principal),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
try:
|
||||
if payload.library_id is not None or payload.remote_path is not None:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, space.connector_profile_id
|
||||
)
|
||||
decision = _connector_space_policy_decision(
|
||||
profile,
|
||||
library_id=payload.library_id
|
||||
if payload.library_id is not None
|
||||
else space.library_id,
|
||||
remote_path=payload.remote_path
|
||||
if payload.remote_path is not None
|
||||
else space.remote_path,
|
||||
operation="link",
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
update_connector_space(
|
||||
session,
|
||||
space,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
sync_mode=payload.sync_mode,
|
||||
is_active=payload.is_active,
|
||||
metadata=payload.metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connector-spaces/{space_id}", response_model=FileConnectorSpaceResponse
|
||||
)
|
||||
def delete_file_connector_space(
|
||||
space_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
space = get_connector_space_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
space_id=space_id,
|
||||
include_inactive=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
soft_delete_connector_space(
|
||||
session, space, user_id=principal.user.id, is_admin=_is_admin(principal)
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
213
src/govoplan_files/backend/routes/transfers.py
Normal file
213
src/govoplan_files/backend/routes/transfers.py
Normal file
@@ -0,0 +1,213 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import FileResponse
|
||||
from starlette.background import BackgroundTask
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
ArchiveRequest,
|
||||
PatternMatchResponse,
|
||||
PatternResolveRequest,
|
||||
PatternResolveResponse,
|
||||
RenamePreviewItem,
|
||||
RenameRequest,
|
||||
RenameResponse,
|
||||
TransferRequest,
|
||||
TransferResponse,
|
||||
_conflict_resolutions,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import (
|
||||
UnsafeFilePathError,
|
||||
filename_from_path,
|
||||
normalize_logical_path,
|
||||
)
|
||||
from govoplan_files.backend.storage.archives import create_zip_file
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
list_assets_for_user,
|
||||
)
|
||||
from govoplan_files.backend.storage.search import resolve_patterns
|
||||
from govoplan_files.backend.storage.transfers import (
|
||||
rename_selection,
|
||||
transfer_selection,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_attachment_disposition,
|
||||
_audit_connector_access,
|
||||
_cleanup_temp_file,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_list_owner_access,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post("/bulk-rename", response_model=RenameResponse)
|
||||
def bulk_rename(
|
||||
payload: RenameRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
plan = rename_selection(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
file_ids=payload.file_ids,
|
||||
folder_paths=payload.folder_paths,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
mode=payload.mode,
|
||||
new_name=payload.new_name,
|
||||
find=payload.find,
|
||||
replacement=payload.replacement,
|
||||
prefix=payload.prefix,
|
||||
suffix=payload.suffix,
|
||||
recursive=payload.recursive,
|
||||
dry_run=payload.dry_run,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if not payload.dry_run:
|
||||
session.commit()
|
||||
return RenameResponse(
|
||||
dry_run=payload.dry_run,
|
||||
items=[
|
||||
RenamePreviewItem(
|
||||
kind=item.kind,
|
||||
id=item.id,
|
||||
file_id=item.id if item.kind == "file" else None,
|
||||
folder_path=item.old_path if item.kind == "folder" else None,
|
||||
old_path=item.old_path,
|
||||
new_path=item.new_path,
|
||||
)
|
||||
for item in plan
|
||||
],
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/transfer", response_model=TransferResponse)
|
||||
def transfer_files(
|
||||
payload: TransferRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
files, folders = transfer_selection(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
operation=payload.operation,
|
||||
file_ids=payload.file_ids,
|
||||
folder_paths=payload.folder_paths,
|
||||
source_owner_type=payload.source_owner_type,
|
||||
source_owner_id=payload.source_owner_id,
|
||||
target_owner_type=payload.target_owner_type,
|
||||
target_owner_id=payload.target_owner_id,
|
||||
target_folder=payload.target_folder,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
conflict_resolutions=_conflict_resolutions(payload.conflict_resolutions),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return TransferResponse(
|
||||
operation=payload.operation, files=files, folders=folders
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/archive.zip")
|
||||
def download_archive(
|
||||
payload: ArchiveRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:download")),
|
||||
):
|
||||
try:
|
||||
assets = [
|
||||
get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in payload.file_ids
|
||||
]
|
||||
tmp = tempfile.NamedTemporaryFile(
|
||||
prefix="govoplan-files-", suffix=".zip", delete=False
|
||||
)
|
||||
tmp_path = tmp.name
|
||||
tmp.close()
|
||||
try:
|
||||
create_zip_file(session, assets, tmp_path)
|
||||
except Exception:
|
||||
_cleanup_temp_file(tmp_path)
|
||||
raise
|
||||
_audit_connector_access(session, principal, assets, operation="archive")
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
filename = filename_from_path(
|
||||
normalize_logical_path(payload.filename, fallback_filename="files.zip")
|
||||
)
|
||||
headers = {"Content-Disposition": _attachment_disposition(filename)}
|
||||
return FileResponse(
|
||||
tmp_path,
|
||||
media_type="application/zip",
|
||||
headers=headers,
|
||||
background=BackgroundTask(_cleanup_temp_file, tmp_path),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/resolve-patterns", response_model=PatternResolveResponse)
|
||||
def resolve_file_patterns(
|
||||
payload: PatternResolveRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, payload.owner_type, payload.owner_id)
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
try:
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
campaign_id=payload.campaign_id,
|
||||
path_prefix=payload.path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
resolved, unmatched = resolve_patterns(
|
||||
assets,
|
||||
payload.patterns,
|
||||
base_path=payload.path_prefix,
|
||||
case_sensitive=payload.case_sensitive,
|
||||
)
|
||||
return PatternResolveResponse(
|
||||
patterns=[
|
||||
PatternMatchResponse(
|
||||
pattern=item.pattern,
|
||||
matches=[_asset_response(session, asset) for asset in item.matches],
|
||||
)
|
||||
for item in resolved
|
||||
],
|
||||
unmatched=[_asset_response(session, asset) for asset in unmatched]
|
||||
if payload.include_unmatched
|
||||
else [],
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
207
src/govoplan_files/backend/routes/uploads.py
Normal file
207
src/govoplan_files/backend/routes/uploads.py
Normal file
@@ -0,0 +1,207 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, File as FastAPIFile, Form, UploadFile
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
ConflictResolutionRequest,
|
||||
FileUploadResponse,
|
||||
_conflict_resolutions,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileAsset
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.runtime import settings
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.archives import extract_zip_upload
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
from govoplan_files.backend.storage.files import (
|
||||
create_file_asset,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_audit_connector_imports,
|
||||
_cleanup_temp_file,
|
||||
_connector_policy_error,
|
||||
_enforce_connector_policy,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_read_limited_upload,
|
||||
_source_metadata_from_form,
|
||||
_spool_limited_upload_to_temp,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post("/upload", response_model=FileUploadResponse)
|
||||
def upload_files(
|
||||
files: list[UploadFile] = FastAPIFile(...),
|
||||
owner_type: Literal["user", "group"] = Form(default="user"),
|
||||
owner_id: str | None = Form(default=None),
|
||||
path: str = Form(default=""),
|
||||
campaign_id: str | None = Form(default=None),
|
||||
unpack_zip: bool = Form(default=False),
|
||||
conflict_strategy: Literal["reject", "overwrite", "rename"] = Form(
|
||||
default="reject"
|
||||
),
|
||||
conflict_resolutions_json: str | None = Form(default=None),
|
||||
source_provenance_json: str | None = Form(default=None),
|
||||
source_revision: str | None = Form(default=None),
|
||||
connector_policy_json: str | None = Form(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
target_owner = owner_id or principal.user.id
|
||||
uploaded_assets: list[FileAsset] = []
|
||||
try:
|
||||
raw_resolutions = (
|
||||
json.loads(conflict_resolutions_json) if conflict_resolutions_json else []
|
||||
)
|
||||
upload_resolutions = _conflict_resolutions(
|
||||
[ConflictResolutionRequest(**item) for item in raw_resolutions]
|
||||
)
|
||||
_enforce_connector_policy(
|
||||
source_provenance_json, connector_policy_json, operation="import"
|
||||
)
|
||||
metadata = _source_metadata_from_form(source_provenance_json, source_revision)
|
||||
for upload in files:
|
||||
filename = upload.filename or "file"
|
||||
content_type = upload.content_type or None
|
||||
upload_limit = (
|
||||
settings.file_upload_zip_max_bytes
|
||||
if unpack_zip and filename.lower().endswith(".zip")
|
||||
else settings.file_upload_max_bytes
|
||||
)
|
||||
if unpack_zip and filename.lower().endswith(".zip"):
|
||||
zip_path = _spool_limited_upload_to_temp(
|
||||
upload, max_bytes=upload_limit, suffix=".zip"
|
||||
)
|
||||
try:
|
||||
extracted = extract_zip_upload(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
zip_data=zip_path,
|
||||
folder=path,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
max_file_bytes=settings.file_upload_max_bytes,
|
||||
max_total_bytes=settings.file_upload_zip_max_bytes,
|
||||
)
|
||||
finally:
|
||||
_cleanup_temp_file(zip_path)
|
||||
uploaded_assets.extend(item.asset for item in extracted)
|
||||
continue
|
||||
data = _read_limited_upload(upload, max_bytes=upload_limit)
|
||||
stored = create_file_asset(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=filename,
|
||||
data=data,
|
||||
folder=path,
|
||||
content_type=content_type,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
uploaded_assets.append(stored.asset)
|
||||
_audit_connector_imports(session, principal, uploaded_assets)
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileUploadResponse(
|
||||
files=[
|
||||
_asset_response(session, asset, include_shares=True)
|
||||
for asset in uploaded_assets
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post("/upload-zip", response_model=FileUploadResponse)
|
||||
def upload_zip(
|
||||
file: UploadFile = FastAPIFile(...),
|
||||
owner_type: Literal["user", "group"] = Form(default="user"),
|
||||
owner_id: str | None = Form(default=None),
|
||||
path: str = Form(default=""),
|
||||
campaign_id: str | None = Form(default=None),
|
||||
conflict_strategy: Literal["reject", "overwrite", "rename"] = Form(
|
||||
default="reject"
|
||||
),
|
||||
conflict_resolutions_json: str | None = Form(default=None),
|
||||
source_provenance_json: str | None = Form(default=None),
|
||||
source_revision: str | None = Form(default=None),
|
||||
connector_policy_json: str | None = Form(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
target_owner = owner_id or principal.user.id
|
||||
zip_path: str | None = None
|
||||
try:
|
||||
raw_resolutions = (
|
||||
json.loads(conflict_resolutions_json) if conflict_resolutions_json else []
|
||||
)
|
||||
upload_resolutions = _conflict_resolutions(
|
||||
[ConflictResolutionRequest(**item) for item in raw_resolutions]
|
||||
)
|
||||
_enforce_connector_policy(
|
||||
source_provenance_json, connector_policy_json, operation="import"
|
||||
)
|
||||
metadata = _source_metadata_from_form(source_provenance_json, source_revision)
|
||||
zip_path = _spool_limited_upload_to_temp(
|
||||
file, max_bytes=settings.file_upload_zip_max_bytes, suffix=".zip"
|
||||
)
|
||||
extracted = extract_zip_upload(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
zip_data=zip_path,
|
||||
folder=path,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
max_file_bytes=settings.file_upload_max_bytes,
|
||||
max_total_bytes=settings.file_upload_zip_max_bytes,
|
||||
)
|
||||
_audit_connector_imports(session, principal, [item.asset for item in extracted])
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
finally:
|
||||
if zip_path:
|
||||
_cleanup_temp_file(zip_path)
|
||||
return FileUploadResponse(
|
||||
files=[
|
||||
_asset_response(session, item.asset, include_shares=True)
|
||||
for item in extracted
|
||||
]
|
||||
)
|
||||
@@ -154,6 +154,7 @@ class FileFolderDeleteResponse(BaseModel):
|
||||
|
||||
class FileListResponse(BaseModel):
|
||||
files: list[FileAssetResponse]
|
||||
total: int
|
||||
cursor: str | None = None
|
||||
next_cursor: str | None = None
|
||||
watermark: str | None = None
|
||||
|
||||
1
src/govoplan_files/backend/services/__init__.py
Normal file
1
src/govoplan_files/backend/services/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Query and response assembly services used by Files routes."""
|
||||
292
src/govoplan_files/backend/services/connector_settings_delta.py
Normal file
292
src/govoplan_files/backend/services/connector_settings_delta.py
Normal file
@@ -0,0 +1,292 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal
|
||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||
from govoplan_core.core.change_sequence import (
|
||||
ChangeSequenceEntry,
|
||||
)
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorCredentialResponse,
|
||||
FileConnectorSettingsDeltaResponse,
|
||||
FileConnectorPolicyResponse,
|
||||
FileConnectorProfileResponse,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileConnectorSpace
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
ConnectorCredential,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||
from govoplan_files.backend.storage.connector_policy_store import (
|
||||
connector_policy_response,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_deleted_item,
|
||||
_connector_space_response,
|
||||
_ensure_campaign_file_access,
|
||||
_file_connector_settings_response_watermark,
|
||||
_file_connector_settings_watermark,
|
||||
_visible_connector_credentials,
|
||||
_visible_connector_profiles,
|
||||
_visible_connector_spaces,
|
||||
)
|
||||
|
||||
|
||||
def _full_file_connector_settings_delta_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
provider: str | None,
|
||||
campaign_id: str | None,
|
||||
include_disabled: bool,
|
||||
include_inactive: bool,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> FileConnectorSettingsDeltaResponse:
|
||||
if campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
credentials = _visible_connector_credentials(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
spaces = _visible_connector_spaces(
|
||||
session,
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive,
|
||||
)
|
||||
return FileConnectorSettingsDeltaResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in profiles
|
||||
],
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in credentials
|
||||
],
|
||||
spaces=[_connector_space_response(space) for space in spaces],
|
||||
policy=FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
),
|
||||
changed_sections=["profiles", "credentials", "spaces", "policy"],
|
||||
deleted=[],
|
||||
watermark=_file_connector_settings_watermark(
|
||||
session, tenant_id=principal.tenant_id
|
||||
),
|
||||
has_more=False,
|
||||
full=True,
|
||||
)
|
||||
|
||||
|
||||
def _changed_file_connector_setting_ids(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
) -> tuple[set[str], set[str], set[str], bool]:
|
||||
changed_profile_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_PROFILES_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_PROFILE_RESOURCE
|
||||
}
|
||||
changed_credential_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_CREDENTIALS_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_CREDENTIAL_RESOURCE
|
||||
}
|
||||
changed_space_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_SPACES_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_SPACE_RESOURCE
|
||||
}
|
||||
policy_changed = any(
|
||||
entry.collection == FILES_CONNECTOR_POLICIES_COLLECTION for entry in entries
|
||||
)
|
||||
return (
|
||||
changed_profile_ids,
|
||||
changed_credential_ids,
|
||||
changed_space_ids,
|
||||
policy_changed,
|
||||
)
|
||||
|
||||
|
||||
def _file_connector_settings_changed_sections(
|
||||
*,
|
||||
changed_profile_ids: set[str],
|
||||
changed_credential_ids: set[str],
|
||||
changed_space_ids: set[str],
|
||||
policy_changed: bool,
|
||||
profiles: list[ConnectorProfile],
|
||||
) -> list[str]:
|
||||
changed_sections = []
|
||||
if changed_profile_ids or any(
|
||||
profile.credential_profile_id
|
||||
and profile.credential_profile_id in changed_credential_ids
|
||||
for profile in profiles
|
||||
):
|
||||
changed_sections.append("profiles")
|
||||
if changed_credential_ids:
|
||||
changed_sections.append("credentials")
|
||||
if changed_space_ids:
|
||||
changed_sections.append("spaces")
|
||||
if policy_changed:
|
||||
changed_sections.append("policy")
|
||||
return changed_sections
|
||||
|
||||
|
||||
def _file_connector_settings_deleted_items(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
*,
|
||||
visible_profiles: dict[str, ConnectorProfile],
|
||||
visible_credentials: dict[str, ConnectorCredential],
|
||||
visible_spaces: dict[str, FileConnectorSpace],
|
||||
) -> list[DeltaDeletedItem]:
|
||||
return [
|
||||
_connector_deleted_item(entry)
|
||||
for entry in entries
|
||||
if (
|
||||
entry.resource_type == FILES_CONNECTOR_PROFILE_RESOURCE
|
||||
and entry.resource_id not in visible_profiles
|
||||
)
|
||||
or (
|
||||
entry.resource_type == FILES_CONNECTOR_CREDENTIAL_RESOURCE
|
||||
and entry.resource_id not in visible_credentials
|
||||
)
|
||||
or (
|
||||
entry.resource_type == FILES_CONNECTOR_SPACE_RESOURCE
|
||||
and entry.resource_id not in visible_spaces
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def _incremental_file_connector_settings_delta_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
entries: list[ChangeSequenceEntry],
|
||||
has_more: bool,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
provider: str | None,
|
||||
campaign_id: str | None,
|
||||
include_disabled: bool,
|
||||
include_inactive: bool,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> FileConnectorSettingsDeltaResponse:
|
||||
changed_profile_ids, changed_credential_ids, changed_space_ids, policy_changed = (
|
||||
_changed_file_connector_setting_ids(entries)
|
||||
)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
visible_profiles = {
|
||||
profile.id: profile
|
||||
for profile in profiles
|
||||
if profile.id in changed_profile_ids
|
||||
or (
|
||||
profile.credential_profile_id
|
||||
and profile.credential_profile_id in changed_credential_ids
|
||||
)
|
||||
}
|
||||
credentials = _visible_connector_credentials(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
visible_credentials = {
|
||||
credential.id: credential
|
||||
for credential in credentials
|
||||
if credential.id in changed_credential_ids
|
||||
}
|
||||
spaces = _visible_connector_spaces(
|
||||
session,
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive,
|
||||
)
|
||||
visible_spaces = {
|
||||
space.id: space for space in spaces if space.id in changed_space_ids
|
||||
}
|
||||
return FileConnectorSettingsDeltaResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in visible_profiles.values()
|
||||
],
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in visible_credentials.values()
|
||||
],
|
||||
spaces=[_connector_space_response(space) for space in visible_spaces.values()],
|
||||
policy=FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
if policy_changed
|
||||
else None,
|
||||
changed_sections=_file_connector_settings_changed_sections(
|
||||
changed_profile_ids=changed_profile_ids,
|
||||
changed_credential_ids=changed_credential_ids,
|
||||
changed_space_ids=changed_space_ids,
|
||||
policy_changed=policy_changed,
|
||||
profiles=profiles,
|
||||
),
|
||||
deleted=_file_connector_settings_deleted_items(
|
||||
entries,
|
||||
visible_profiles=visible_profiles,
|
||||
visible_credentials=visible_credentials,
|
||||
visible_spaces=visible_spaces,
|
||||
),
|
||||
watermark=_file_connector_settings_response_watermark(
|
||||
session, tenant_id=principal.tenant_id, entries=entries, has_more=has_more
|
||||
),
|
||||
has_more=has_more,
|
||||
full=False,
|
||||
)
|
||||
619
src/govoplan_files/backend/services/list_queries.py
Normal file
619
src/govoplan_files/backend/services/list_queries.py
Normal file
@@ -0,0 +1,619 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal
|
||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||
from govoplan_core.core.change_sequence import (
|
||||
ChangeSequenceEntry,
|
||||
decode_sequence_watermark,
|
||||
encode_sequence_watermark,
|
||||
max_sequence_id,
|
||||
sequence_entries_since,
|
||||
sequence_watermark_is_expired,
|
||||
)
|
||||
from govoplan_core.core.pagination import (
|
||||
KeysetCursorError,
|
||||
decode_keyset_cursor,
|
||||
encode_keyset_cursor,
|
||||
keyset_query_fingerprint,
|
||||
)
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_ASSETS_COLLECTION,
|
||||
FILES_FOLDERS_COLLECTION,
|
||||
FILES_MODULE_ID,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileDeltaResponse,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileAsset, FileFolder, FileShare
|
||||
from govoplan_files.backend.storage.access import user_group_ids
|
||||
from govoplan_files.backend.storage.files import (
|
||||
list_assets_for_user,
|
||||
)
|
||||
from govoplan_files.backend.storage.folders import list_folders_for_user
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_list_response,
|
||||
_folder_response,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
_FILES_DELTA_COLLECTIONS = (FILES_ASSETS_COLLECTION, FILES_FOLDERS_COLLECTION)
|
||||
FILES_LIST_CURSOR_SCOPE = "files.list.v1"
|
||||
FOLDERS_LIST_CURSOR_SCOPE = "files.folders.list.v1"
|
||||
DEFAULT_FILE_LIST_PAGE_SIZE = 500
|
||||
|
||||
|
||||
def _cursor_http_error(exc: Exception) -> HTTPException:
|
||||
return HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc))
|
||||
|
||||
|
||||
def _cursor_page_size(
|
||||
scope: str, cursor: str | None, explicit_page_size: int | None
|
||||
) -> int | None:
|
||||
if explicit_page_size is not None:
|
||||
return explicit_page_size
|
||||
if not cursor:
|
||||
return DEFAULT_FILE_LIST_PAGE_SIZE
|
||||
try:
|
||||
values = decode_keyset_cursor(scope, cursor)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
raw_page_size = values.get("page_size")
|
||||
if not isinstance(raw_page_size, int) or raw_page_size < 1 or raw_page_size > 1000:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
return raw_page_size
|
||||
|
||||
|
||||
def _files_list_fingerprint(
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None,
|
||||
audit_relevant: bool | None,
|
||||
page_size: int,
|
||||
) -> str:
|
||||
return keyset_query_fingerprint(
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
{
|
||||
"tenant_id": principal.tenant_id,
|
||||
"actor": "admin" if _is_admin(principal) else principal.user.id,
|
||||
"owner_type": owner_type,
|
||||
"owner_id": owner_id,
|
||||
"campaign_id": campaign_id,
|
||||
"path_prefix": path_prefix or "",
|
||||
"campaign_usage": campaign_usage or "",
|
||||
"audit_relevant": audit_relevant,
|
||||
"sort": "display_path.asc,updated_at.desc,id.asc",
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _folders_list_fingerprint(
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int,
|
||||
) -> str:
|
||||
return keyset_query_fingerprint(
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
{
|
||||
"tenant_id": principal.tenant_id,
|
||||
"actor": "admin" if _is_admin(principal) else principal.user.id,
|
||||
"owner_type": owner_type,
|
||||
"owner_id": owner_id,
|
||||
"sort": "path.asc,id.asc",
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _file_cursor_values(
|
||||
cursor: str | None, *, fingerprint: str
|
||||
) -> tuple[str | None, datetime | None, str | None]:
|
||||
try:
|
||||
values = decode_keyset_cursor(
|
||||
FILES_LIST_CURSOR_SCOPE, cursor, fingerprint=fingerprint
|
||||
)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
if values is None:
|
||||
return None, None, None
|
||||
display_path = values.get("display_path")
|
||||
updated_at = values.get("updated_at")
|
||||
asset_id = values.get("id")
|
||||
if (
|
||||
not isinstance(display_path, str)
|
||||
or not isinstance(updated_at, str)
|
||||
or not isinstance(asset_id, str)
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
try:
|
||||
parsed_updated_at = datetime.fromisoformat(updated_at)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
) from exc
|
||||
return display_path, parsed_updated_at, asset_id
|
||||
|
||||
|
||||
def _folder_cursor_values(
|
||||
cursor: str | None, *, fingerprint: str
|
||||
) -> tuple[str | None, str | None]:
|
||||
try:
|
||||
values = decode_keyset_cursor(
|
||||
FOLDERS_LIST_CURSOR_SCOPE, cursor, fingerprint=fingerprint
|
||||
)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
if values is None:
|
||||
return None, None
|
||||
folder_path = values.get("path")
|
||||
folder_id = values.get("id")
|
||||
if not isinstance(folder_path, str) or not isinstance(folder_id, str):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
return folder_path, folder_id
|
||||
|
||||
|
||||
def _next_file_list_cursor(
|
||||
principal: ApiPrincipal,
|
||||
assets: list[FileAsset],
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None,
|
||||
audit_relevant: bool | None,
|
||||
page_size: int,
|
||||
has_more: bool,
|
||||
) -> str | None:
|
||||
if not has_more or not assets:
|
||||
return None
|
||||
last = assets[-1]
|
||||
return encode_keyset_cursor(
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
fingerprint=_files_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=page_size,
|
||||
),
|
||||
values={
|
||||
"display_path": last.display_path,
|
||||
"updated_at": last.updated_at,
|
||||
"id": last.id,
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _next_folder_list_cursor(
|
||||
principal: ApiPrincipal,
|
||||
folders: list[FileFolder],
|
||||
*,
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int,
|
||||
has_more: bool,
|
||||
) -> str | None:
|
||||
if not has_more or not folders:
|
||||
return None
|
||||
last = folders[-1]
|
||||
return encode_keyset_cursor(
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
fingerprint=_folders_list_fingerprint(
|
||||
principal, owner_type=owner_type, owner_id=owner_id, page_size=page_size
|
||||
),
|
||||
values={"path": last.path, "id": last.id, "page_size": page_size},
|
||||
)
|
||||
|
||||
|
||||
def _files_delta_watermark(session: Session, tenant_id: str) -> str:
|
||||
return encode_sequence_watermark(
|
||||
max_sequence_id(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _full_file_delta_response(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> FileDeltaResponse:
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
folders = _visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
return FileDeltaResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
deleted=[],
|
||||
watermark=_files_delta_watermark(session, principal.tenant_id),
|
||||
has_more=False,
|
||||
full=True,
|
||||
)
|
||||
|
||||
|
||||
def _visible_folders_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> list[FileFolder]:
|
||||
if not owner_type or not owner_id:
|
||||
return []
|
||||
folders = list_folders_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if not path_prefix:
|
||||
return folders
|
||||
normalized = path_prefix.strip().strip("/")
|
||||
if not normalized:
|
||||
return folders
|
||||
return [
|
||||
folder
|
||||
for folder in folders
|
||||
if folder.path == normalized or folder.path.startswith(f"{normalized}/")
|
||||
]
|
||||
|
||||
|
||||
def _entry_path_matches(
|
||||
entry_payload: dict[str, object], path_prefix: str | None
|
||||
) -> bool:
|
||||
if not path_prefix:
|
||||
return True
|
||||
normalized = path_prefix.strip().strip("/")
|
||||
if not normalized:
|
||||
return True
|
||||
for key in ("path", "previous_path"):
|
||||
value = entry_payload.get(key)
|
||||
if isinstance(value, str) and (
|
||||
value == normalized or value.startswith(f"{normalized}/")
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _entry_owner_matches(
|
||||
entry_payload: dict[str, object],
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> bool:
|
||||
if not owner_type or not owner_id:
|
||||
return True
|
||||
return (
|
||||
entry_payload.get("owner_type") == owner_type
|
||||
and entry_payload.get("owner_id") == owner_id
|
||||
)
|
||||
|
||||
|
||||
def _entry_campaign_matches(session: Session, entry, campaign_id: str | None) -> bool:
|
||||
if not campaign_id:
|
||||
return True
|
||||
payload = entry.payload or {}
|
||||
if (
|
||||
payload.get("share_target_type") == "campaign"
|
||||
and payload.get("share_target_id") == campaign_id
|
||||
):
|
||||
return True
|
||||
if entry.resource_type != "file":
|
||||
return False
|
||||
return (
|
||||
session.query(FileShare)
|
||||
.filter(
|
||||
FileShare.tenant_id == entry.tenant_id,
|
||||
FileShare.file_asset_id == entry.resource_id,
|
||||
FileShare.target_type == "campaign",
|
||||
FileShare.target_id == campaign_id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
)
|
||||
.first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _principal_group_ids_for_delta(
|
||||
session: Session, principal: ApiPrincipal
|
||||
) -> set[str]:
|
||||
cache = session.info.setdefault("files_delta_group_ids", {})
|
||||
key = (principal.tenant_id, principal.user.id)
|
||||
if key not in cache:
|
||||
cache[key] = set(
|
||||
user_group_ids(
|
||||
session, tenant_id=principal.tenant_id, user_id=principal.user.id
|
||||
)
|
||||
)
|
||||
return cache[key]
|
||||
|
||||
|
||||
def _entry_subject_matches_principal(
|
||||
session: Session, principal: ApiPrincipal, entry_payload: dict[str, object]
|
||||
) -> bool:
|
||||
if _is_admin(principal):
|
||||
return True
|
||||
owner_type = entry_payload.get("owner_type")
|
||||
owner_id = entry_payload.get("owner_id")
|
||||
if owner_type == "user" and owner_id == principal.user.id:
|
||||
return True
|
||||
if owner_type == "group" and isinstance(owner_id, str):
|
||||
if owner_id in _principal_group_ids_for_delta(session, principal):
|
||||
return True
|
||||
share_target_type = entry_payload.get("share_target_type")
|
||||
share_target_id = entry_payload.get("share_target_id")
|
||||
if share_target_type == "user" and share_target_id == principal.user.id:
|
||||
return True
|
||||
if share_target_type == "tenant" and share_target_id == principal.tenant_id:
|
||||
return True
|
||||
if share_target_type == "group" and isinstance(share_target_id, str):
|
||||
if share_target_id in _principal_group_ids_for_delta(session, principal):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _entry_matches_delta_scope(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
entry,
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> bool:
|
||||
payload = entry.payload or {}
|
||||
if (
|
||||
not owner_type
|
||||
and not campaign_id
|
||||
and not _entry_subject_matches_principal(session, principal, payload)
|
||||
):
|
||||
return False
|
||||
return (
|
||||
_entry_owner_matches(payload, owner_type, owner_id)
|
||||
and _entry_path_matches(payload, path_prefix)
|
||||
and _entry_campaign_matches(session, entry, campaign_id)
|
||||
)
|
||||
|
||||
|
||||
def _decode_files_delta_watermark(since: str) -> int:
|
||||
try:
|
||||
return decode_sequence_watermark(since)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
def _changed_delta_resource_ids(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
) -> tuple[list[str], list[str]]:
|
||||
file_ids = list(
|
||||
dict.fromkeys(
|
||||
entry.resource_id for entry in entries if entry.resource_type == "file"
|
||||
)
|
||||
)
|
||||
folder_ids = list(
|
||||
dict.fromkeys(
|
||||
entry.resource_id for entry in entries if entry.resource_type == "folder"
|
||||
)
|
||||
)
|
||||
return file_ids, folder_ids
|
||||
|
||||
|
||||
def _visible_assets_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
changed_file_ids: list[str],
|
||||
) -> dict[str, FileAsset]:
|
||||
return {
|
||||
asset.id: asset
|
||||
for asset in list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if asset.id in changed_file_ids
|
||||
}
|
||||
|
||||
|
||||
def _changed_visible_folders_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
path_prefix: str | None,
|
||||
changed_folder_ids: list[str],
|
||||
) -> dict[str, FileFolder]:
|
||||
return {
|
||||
folder.id: folder
|
||||
for folder in _visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
if folder.id in changed_folder_ids
|
||||
}
|
||||
|
||||
|
||||
def _deleted_delta_items(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
entries: list[ChangeSequenceEntry],
|
||||
visible_assets: dict[str, FileAsset],
|
||||
visible_folders: dict[str, FileFolder],
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> list[DeltaDeletedItem]:
|
||||
deleted: dict[tuple[str, str], DeltaDeletedItem] = {}
|
||||
for entry in entries:
|
||||
if entry.resource_type == "file" and entry.resource_id in visible_assets:
|
||||
continue
|
||||
if entry.resource_type == "folder" and entry.resource_id in visible_folders:
|
||||
continue
|
||||
if not _entry_matches_delta_scope(
|
||||
session,
|
||||
principal,
|
||||
entry,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
):
|
||||
continue
|
||||
deleted[(entry.resource_type, entry.resource_id)] = DeltaDeletedItem(
|
||||
id=entry.resource_id,
|
||||
resource_type=entry.resource_type,
|
||||
revision=encode_sequence_watermark(entry.id),
|
||||
deleted_at=entry.created_at if entry.operation == "deleted" else None,
|
||||
)
|
||||
return list(deleted.values())
|
||||
|
||||
|
||||
def _files_delta_response(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
since: str,
|
||||
limit: int,
|
||||
) -> FileDeltaResponse:
|
||||
since_sequence = _decode_files_delta_watermark(since)
|
||||
if sequence_watermark_is_expired(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
):
|
||||
return _full_file_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
|
||||
entries_plus_one = sequence_entries_since(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
limit=limit + 1,
|
||||
)
|
||||
has_more = len(entries_plus_one) > limit
|
||||
entries = entries_plus_one[:limit]
|
||||
|
||||
changed_file_ids, changed_folder_ids = _changed_delta_resource_ids(entries)
|
||||
visible_assets = _visible_assets_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
changed_file_ids=changed_file_ids,
|
||||
)
|
||||
visible_folders = _changed_visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
changed_folder_ids=changed_folder_ids,
|
||||
)
|
||||
deleted = _deleted_delta_items(
|
||||
session,
|
||||
principal=principal,
|
||||
entries=entries,
|
||||
visible_assets=visible_assets,
|
||||
visible_folders=visible_folders,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
|
||||
watermark = (
|
||||
encode_sequence_watermark(entries[-1].id)
|
||||
if has_more and entries
|
||||
else _files_delta_watermark(session, principal.tenant_id)
|
||||
)
|
||||
return FileDeltaResponse(
|
||||
files=_asset_list_response(
|
||||
session, list(visible_assets.values()), include_shares=True
|
||||
),
|
||||
folders=[_folder_response(folder) for folder in visible_folders.values()],
|
||||
deleted=deleted,
|
||||
watermark=watermark,
|
||||
has_more=has_more,
|
||||
full=False,
|
||||
)
|
||||
@@ -17,6 +17,7 @@ from govoplan_files.backend.storage.paths import filename_from_path, normalize_f
|
||||
|
||||
|
||||
_ZIP_READ_CHUNK_SIZE = 1024 * 1024
|
||||
ZIP_UPLOAD_MAX_FILES = 1000
|
||||
|
||||
|
||||
def _read_zip_member(
|
||||
@@ -76,7 +77,7 @@ def extract_zip_upload(
|
||||
conflict_resolutions: Iterable[FileConflictResolution] | None = None,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
is_admin: bool = False,
|
||||
max_files: int = 1000,
|
||||
max_files: int = ZIP_UPLOAD_MAX_FILES,
|
||||
max_file_bytes: int = 50 * 1024 * 1024,
|
||||
max_total_bytes: int = 250 * 1024 * 1024,
|
||||
) -> list[UploadedStoredFile]:
|
||||
|
||||
@@ -5,8 +5,17 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import inspect
|
||||
|
||||
from govoplan_core.core.policy import normalize_policy_scope_type, policy_source_path
|
||||
from govoplan_core.security.credential_envelopes import (
|
||||
CredentialAccessContext,
|
||||
CredentialEnvelope,
|
||||
CredentialEnvelopeError,
|
||||
ResolvedCredentialEnvelope,
|
||||
list_credential_envelopes,
|
||||
resolve_credential_envelope,
|
||||
)
|
||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
@@ -14,6 +23,8 @@ from govoplan_files.backend.storage.connector_deployment import reject_api_contr
|
||||
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
||||
from govoplan_files.backend.storage.connector_profiles import supported_connector_providers
|
||||
|
||||
CORE_CREDENTIAL_ENVELOPE_PREFIX = "credential-envelope:"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ConnectorCredential:
|
||||
@@ -33,6 +44,7 @@ class ConnectorCredential:
|
||||
policy_sources: tuple[ConnectorPolicySource, ...] = ()
|
||||
metadata: Mapping[str, Any] | None = None
|
||||
source_kind: str = "database"
|
||||
has_secret: bool = False
|
||||
|
||||
@property
|
||||
def source_path(self) -> str:
|
||||
@@ -58,7 +70,7 @@ class ConnectorCredential:
|
||||
return True
|
||||
# Environment references are deliberately unavailable to API-managed
|
||||
# credential rows. Legacy rows remain visible but fail closed.
|
||||
return bool(self.secret_ref or self.password_value or self.token_value)
|
||||
return bool(self.has_secret or self.secret_ref or self.password_value or self.token_value)
|
||||
|
||||
def to_response(self) -> dict[str, Any]:
|
||||
return {
|
||||
@@ -85,7 +97,159 @@ def list_database_connector_credentials(
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
) -> list[ConnectorCredential]:
|
||||
return [connector_credential_from_row(row) for row in list_connector_credential_rows(session, tenant_id=tenant_id, include_disabled=include_disabled)]
|
||||
local = [
|
||||
connector_credential_from_row(row)
|
||||
for row in list_connector_credential_rows(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
]
|
||||
return [
|
||||
*local,
|
||||
*list_reusable_connector_credentials(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def reusable_credential_reference(credential_id: str) -> str:
|
||||
return f"{CORE_CREDENTIAL_ENVELOPE_PREFIX}{credential_id}"
|
||||
|
||||
|
||||
def reusable_credential_id(credential_ref: str | None) -> str | None:
|
||||
if not credential_ref or not credential_ref.startswith(CORE_CREDENTIAL_ENVELOPE_PREFIX):
|
||||
return None
|
||||
value = credential_ref.removeprefix(CORE_CREDENTIAL_ENVELOPE_PREFIX).strip()
|
||||
return value or None
|
||||
|
||||
|
||||
def file_credential_context(
|
||||
*,
|
||||
tenant_id: str,
|
||||
profile_id: str | None = None,
|
||||
scope_type: str = "tenant",
|
||||
scope_id: str | None = None,
|
||||
administrative: bool = False,
|
||||
) -> CredentialAccessContext:
|
||||
return CredentialAccessContext(
|
||||
tenant_id=tenant_id,
|
||||
user_id=scope_id if scope_type == "user" else None,
|
||||
group_ids=frozenset({scope_id}) if scope_type == "group" and scope_id else frozenset(),
|
||||
target_scope_type=scope_type,
|
||||
target_scope_id=scope_id or tenant_id,
|
||||
module_id="files",
|
||||
server_ref=f"files:{profile_id}" if profile_id else None,
|
||||
administrative=administrative,
|
||||
)
|
||||
|
||||
|
||||
def list_reusable_connector_credentials(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
) -> list[ConnectorCredential]:
|
||||
if not inspect(session.get_bind()).has_table(CredentialEnvelope.__tablename__):
|
||||
return []
|
||||
context = file_credential_context(tenant_id=tenant_id, administrative=True)
|
||||
return [
|
||||
connector_credential_from_envelope(row)
|
||||
for row in list_credential_envelopes(
|
||||
session,
|
||||
context=context,
|
||||
include_inactive=include_disabled,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def resolve_reusable_connector_credential(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
credential_ref: str,
|
||||
profile_id: str,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
) -> ConnectorCredential:
|
||||
credential_id = reusable_credential_id(credential_ref)
|
||||
if credential_id is None:
|
||||
raise FileStorageError("Reusable credential reference is invalid")
|
||||
try:
|
||||
resolved = resolve_credential_envelope(
|
||||
session,
|
||||
credential_id=credential_id,
|
||||
context=file_credential_context(
|
||||
tenant_id=tenant_id,
|
||||
profile_id=profile_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
),
|
||||
)
|
||||
except CredentialEnvelopeError as exc:
|
||||
raise FileStorageError("Reusable credential is unavailable to this file connection") from exc
|
||||
return connector_credential_from_resolved_envelope(
|
||||
resolved,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
|
||||
|
||||
def connector_credential_from_envelope(row: CredentialEnvelope) -> ConnectorCredential:
|
||||
return ConnectorCredential(
|
||||
id=reusable_credential_reference(row.id),
|
||||
label=row.name,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
enabled=row.is_active,
|
||||
credential_mode=_envelope_credential_mode(row.credential_kind, row.secret_keys),
|
||||
username=_clean_public_value(row.public_data, "username"),
|
||||
source_kind="credential_envelope",
|
||||
has_secret=bool(row.secret_data_encrypted),
|
||||
metadata={
|
||||
"credential_envelope_id": row.id,
|
||||
"credential_kind": row.credential_kind,
|
||||
"allowed_modules": list(row.allowed_modules or []),
|
||||
"allowed_server_refs": list(row.allowed_server_refs or []),
|
||||
"inherit_to_lower_scopes": bool(row.inherit_to_lower_scopes),
|
||||
"revision": row.revision,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def connector_credential_from_resolved_envelope(
|
||||
row: ResolvedCredentialEnvelope,
|
||||
*,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
) -> ConnectorCredential:
|
||||
return ConnectorCredential(
|
||||
id=reusable_credential_reference(row.id),
|
||||
label=row.name,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
enabled=True,
|
||||
credential_mode=_envelope_credential_mode(row.credential_kind, row.secret_data),
|
||||
username=_clean_public_value(row.public_data, "username"),
|
||||
password_value=_first_secret(row.secret_data, "password", "secret"),
|
||||
token_value=_first_secret(
|
||||
row.secret_data,
|
||||
"access_token",
|
||||
"bearer_token",
|
||||
"token",
|
||||
"api_key",
|
||||
),
|
||||
source_kind="credential_envelope",
|
||||
has_secret=bool(row.secret_data),
|
||||
metadata={
|
||||
"credential_envelope_id": row.id,
|
||||
"credential_kind": row.credential_kind,
|
||||
"inherit_to_lower_scopes": True,
|
||||
"revision": row.revision,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def list_connector_credential_rows(
|
||||
@@ -348,6 +512,27 @@ def _policy_source_response(source: ConnectorPolicySource) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _envelope_credential_mode(kind: str, secret_values: Mapping[str, Any] | list[str]) -> str:
|
||||
keys = {str(key) for key in secret_values}
|
||||
if kind in {"token", "oauth2", "api_key"} or keys.intersection(
|
||||
{"access_token", "bearer_token", "token", "api_key"}
|
||||
):
|
||||
return "token"
|
||||
return "basic"
|
||||
|
||||
|
||||
def _clean_public_value(values: Mapping[str, Any] | None, key: str) -> str | None:
|
||||
return _clean((values or {}).get(key))
|
||||
|
||||
|
||||
def _first_secret(values: Mapping[str, Any], *keys: str) -> str | None:
|
||||
for key in keys:
|
||||
value = _clean(values.get(key))
|
||||
if value is not None:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def _clean(value: object) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
@@ -246,11 +246,11 @@ def _applied_fields(policy: Mapping[str, Any]) -> tuple[str, ...]:
|
||||
|
||||
def _matches_field(request: ConnectorAccessRequest, field: str, patterns: list[str]) -> bool:
|
||||
if field == "connectors":
|
||||
return _matches_exact(request.connector_id, patterns)
|
||||
return _matches_reference(request.connector_id, patterns)
|
||||
if field == "credentials":
|
||||
return _matches_exact(request.credential_id, patterns)
|
||||
return _matches_reference(request.credential_id, patterns)
|
||||
if field == "providers":
|
||||
return _matches_exact(request.provider, patterns)
|
||||
return _matches_reference(request.provider, patterns)
|
||||
if field == "external_ids":
|
||||
return _matches_glob(request.external_id, patterns)
|
||||
if field == "external_paths":
|
||||
@@ -260,11 +260,11 @@ def _matches_field(request: ConnectorAccessRequest, field: str, patterns: list[s
|
||||
return False
|
||||
|
||||
|
||||
def _matches_exact(value: str | None, patterns: list[str]) -> bool:
|
||||
def _matches_reference(value: str | None, patterns: list[str]) -> bool:
|
||||
if value is None:
|
||||
return False
|
||||
clean = value.casefold()
|
||||
return any(pattern == "*" or clean == pattern.casefold() for pattern in patterns)
|
||||
return any(fnmatchcase(clean, pattern.casefold()) for pattern in patterns)
|
||||
|
||||
|
||||
def _matches_glob(value: str | None, patterns: list[str]) -> bool:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
@@ -10,7 +10,13 @@ from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_deployment import reject_api_controlled_deployment_references
|
||||
from govoplan_files.backend.storage.connector_credential_store import connector_credential_from_row, credential_rows_by_id
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
ConnectorCredential,
|
||||
connector_credential_from_row,
|
||||
credential_rows_by_id,
|
||||
resolve_reusable_connector_credential,
|
||||
reusable_credential_id,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import connector_policy_sources_from_payload
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, supported_connector_providers
|
||||
|
||||
@@ -20,7 +26,26 @@ def list_database_connector_profiles(
|
||||
*,
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
row_visible: Callable[[FileConnectorProfile], bool] | None = None,
|
||||
) -> list[ConnectorProfile]:
|
||||
profiles, _profile_ids = select_database_connector_profiles(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
row_visible=row_visible,
|
||||
)
|
||||
return profiles
|
||||
|
||||
|
||||
def select_database_connector_profiles(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
row_visible: Callable[[FileConnectorProfile], bool] | None = None,
|
||||
) -> tuple[list[ConnectorProfile], set[str]]:
|
||||
"""Return visible profiles and every database id that shadows settings."""
|
||||
|
||||
query = session.query(FileConnectorProfile).filter(
|
||||
(FileConnectorProfile.scope_type == "system")
|
||||
| (FileConnectorProfile.tenant_id == tenant_id)
|
||||
@@ -28,9 +53,33 @@ def list_database_connector_profiles(
|
||||
if not include_disabled:
|
||||
query = query.filter(FileConnectorProfile.enabled.is_(True))
|
||||
rows = query.order_by(FileConnectorProfile.scope_type.asc(), FileConnectorProfile.label.asc()).all()
|
||||
credential_ids = {_clean(row.credential_profile_id) for row in rows if _clean(row.credential_profile_id)}
|
||||
profile_ids = {row.id for row in rows}
|
||||
if row_visible is not None:
|
||||
rows = [row for row in rows if row_visible(row)]
|
||||
credential_ids = {
|
||||
_clean(row.credential_profile_id)
|
||||
for row in rows
|
||||
if _clean(row.credential_profile_id) and not reusable_credential_id(row.credential_profile_id)
|
||||
}
|
||||
credentials = credential_rows_by_id(session, tenant_id=tenant_id, credential_ids={item for item in credential_ids if item}, include_disabled=include_disabled)
|
||||
return [connector_profile_from_row(row, credential_row=credentials.get(row.credential_profile_id or "")) for row in rows]
|
||||
return (
|
||||
[
|
||||
connector_profile_from_row(
|
||||
row,
|
||||
credential_row=(
|
||||
_resolve_profile_reusable_credential(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
row=row,
|
||||
)
|
||||
if reusable_credential_id(row.credential_profile_id)
|
||||
else credentials.get(row.credential_profile_id or "")
|
||||
),
|
||||
)
|
||||
for row in rows
|
||||
],
|
||||
profile_ids,
|
||||
)
|
||||
|
||||
|
||||
def list_connector_profile_rows(
|
||||
@@ -48,7 +97,10 @@ def list_connector_profile_rows(
|
||||
return query.order_by(FileConnectorProfile.scope_type.asc(), FileConnectorProfile.label.asc()).all()
|
||||
|
||||
|
||||
def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileConnectorCredential | None = None) -> ConnectorProfile:
|
||||
def connector_profile_from_row(
|
||||
row: FileConnectorProfile,
|
||||
credential_row: FileConnectorCredential | ConnectorCredential | None = None,
|
||||
) -> ConnectorProfile:
|
||||
policy_sources = []
|
||||
if row.policy:
|
||||
policy_sources = connector_policy_sources_from_payload({
|
||||
@@ -57,9 +109,18 @@ def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileCo
|
||||
"label": row.label,
|
||||
"policy": row.policy,
|
||||
})
|
||||
credential = connector_credential_from_row(credential_row) if credential_row is not None else None
|
||||
credential = (
|
||||
credential_row
|
||||
if isinstance(credential_row, ConnectorCredential)
|
||||
else connector_credential_from_row(credential_row)
|
||||
if credential_row is not None
|
||||
else None
|
||||
)
|
||||
if credential:
|
||||
policy_sources.extend(credential.policy_sources)
|
||||
metadata = dict(row.metadata_ or {})
|
||||
if reusable_credential_id(row.credential_profile_id) and credential is None:
|
||||
metadata["credential_unavailable"] = True
|
||||
return ConnectorProfile(
|
||||
id=row.id,
|
||||
label=row.label,
|
||||
@@ -80,11 +141,30 @@ def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileCo
|
||||
token_value=credential.token_value if credential else decrypt_secret(row.token_encrypted),
|
||||
capabilities=tuple(_string_list(row.capabilities)),
|
||||
policy_sources=tuple(policy_sources),
|
||||
metadata=dict(row.metadata_ or {}),
|
||||
metadata=metadata,
|
||||
source_kind="database",
|
||||
)
|
||||
|
||||
|
||||
def _resolve_profile_reusable_credential(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
row: FileConnectorProfile,
|
||||
) -> ConnectorCredential | None:
|
||||
try:
|
||||
return resolve_reusable_connector_credential(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
credential_ref=row.credential_profile_id or "",
|
||||
profile_id=row.id,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
)
|
||||
except FileStorageError:
|
||||
return None
|
||||
|
||||
|
||||
def get_connector_profile_row(
|
||||
session: Session,
|
||||
*,
|
||||
|
||||
218
src/govoplan_files/backend/storage/connector_visibility.py
Normal file
218
src/govoplan_files/backend/storage/connector_visibility.py
Normal file
@@ -0,0 +1,218 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable
|
||||
from dataclasses import replace
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
select_database_connector_profiles,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profiles import (
|
||||
ConnectorProfile,
|
||||
connector_profiles_from_settings,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
connector_policy_decision,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy_store import (
|
||||
effective_connector_policy_sources,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_providers import (
|
||||
connector_provider_descriptors,
|
||||
)
|
||||
|
||||
|
||||
CampaignVisibility = Callable[[str], bool]
|
||||
|
||||
|
||||
def visible_connector_profiles_for_actor(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
group_ids: Iterable[str],
|
||||
settings: object | None,
|
||||
provider: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
campaign_visible: CampaignVisibility | None = None,
|
||||
include_disabled: bool = False,
|
||||
include_admin_scopes: bool = False,
|
||||
include_effective_policy: bool = True,
|
||||
) -> list[ConnectorProfile]:
|
||||
"""Return the same actor-filtered connector set used by API and docs surfaces."""
|
||||
|
||||
provider_norm = provider.strip().casefold() if provider else None
|
||||
actor_group_ids = {str(group_id) for group_id in group_ids if str(group_id)}
|
||||
database_profiles, database_profile_ids = select_database_connector_profiles(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
row_visible=lambda row: (
|
||||
include_admin_scopes
|
||||
and row.scope_type in {"user", "group", "campaign"}
|
||||
) or _scope_visible_to_actor(
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=actor_group_ids,
|
||||
campaign_id=campaign_id,
|
||||
campaign_visible=campaign_visible,
|
||||
),
|
||||
)
|
||||
configured_profiles = connector_profiles_from_settings(settings)
|
||||
profiles_by_id: dict[str, ConnectorProfile] = {}
|
||||
for profile in database_profiles:
|
||||
if profile.id not in profiles_by_id:
|
||||
profiles_by_id[profile.id] = profile
|
||||
for profile in configured_profiles:
|
||||
if profile.id not in database_profile_ids and profile.id not in profiles_by_id:
|
||||
profiles_by_id[profile.id] = profile
|
||||
|
||||
visible: list[ConnectorProfile] = []
|
||||
for profile in profiles_by_id.values():
|
||||
if not (profile.enabled or include_disabled):
|
||||
continue
|
||||
if provider_norm is not None and profile.provider != provider_norm:
|
||||
continue
|
||||
admin_scope_visible = include_admin_scopes and profile.scope_type in {
|
||||
"user",
|
||||
"group",
|
||||
"campaign",
|
||||
}
|
||||
if not admin_scope_visible and not _profile_visible_to_actor(
|
||||
profile,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=actor_group_ids,
|
||||
campaign_id=campaign_id,
|
||||
campaign_visible=campaign_visible,
|
||||
):
|
||||
continue
|
||||
visible.append(
|
||||
_with_effective_connector_policy(
|
||||
session, tenant_id=tenant_id, profile=profile
|
||||
)
|
||||
if include_effective_policy
|
||||
else profile
|
||||
)
|
||||
return visible
|
||||
|
||||
|
||||
def connector_profile_usable_for_import(profile: ConnectorProfile) -> bool:
|
||||
"""Whether a visible profile can safely offer the current live browse/import task."""
|
||||
|
||||
effective_endpoint_url = connector_effective_endpoint_url(
|
||||
provider=profile.provider,
|
||||
endpoint_url=profile.endpoint_url,
|
||||
metadata=profile.metadata,
|
||||
)
|
||||
if (
|
||||
not profile.enabled
|
||||
or not effective_endpoint_url
|
||||
or not profile.credentials_configured
|
||||
or bool(profile.secret_ref)
|
||||
):
|
||||
return False
|
||||
descriptor = next(
|
||||
(
|
||||
item
|
||||
for item in connector_provider_descriptors()
|
||||
if item.provider == profile.provider
|
||||
),
|
||||
None,
|
||||
)
|
||||
if descriptor is None:
|
||||
return False
|
||||
if not (
|
||||
descriptor.implemented
|
||||
and descriptor.installed
|
||||
and descriptor.browse_supported
|
||||
and descriptor.import_supported
|
||||
):
|
||||
return False
|
||||
# These SDK paths intentionally fail closed until every connection peer and
|
||||
# SDK-managed redirect/referral can be pinned and revalidated.
|
||||
if profile.provider in {"s3", "smb"}:
|
||||
return False
|
||||
|
||||
# Prove that the initial root browse performed by the current Files UI is
|
||||
# policy-allowed. A later selected remote path/item is checked again.
|
||||
return connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id=profile.id,
|
||||
credential_id=profile.credential_profile_id,
|
||||
provider=profile.provider,
|
||||
external_path="",
|
||||
external_url=effective_endpoint_url,
|
||||
operation="import",
|
||||
),
|
||||
profile.policy_sources,
|
||||
).allowed
|
||||
|
||||
|
||||
def _profile_visible_to_actor(
|
||||
profile: ConnectorProfile,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
group_ids: set[str],
|
||||
campaign_id: str | None,
|
||||
campaign_visible: CampaignVisibility | None,
|
||||
) -> bool:
|
||||
return _scope_visible_to_actor(
|
||||
scope_type=profile.scope_type,
|
||||
scope_id=profile.scope_id,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
group_ids=group_ids,
|
||||
campaign_id=campaign_id,
|
||||
campaign_visible=campaign_visible,
|
||||
)
|
||||
|
||||
|
||||
def _scope_visible_to_actor(
|
||||
*,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
group_ids: set[str],
|
||||
campaign_id: str | None,
|
||||
campaign_visible: CampaignVisibility | None,
|
||||
) -> bool:
|
||||
if scope_type == "system":
|
||||
return True
|
||||
if scope_type == "tenant":
|
||||
return scope_id == tenant_id
|
||||
if scope_type == "user":
|
||||
return scope_id == user_id
|
||||
if scope_type == "group":
|
||||
return bool(scope_id and scope_id in group_ids)
|
||||
if scope_type != "campaign" or not scope_id:
|
||||
return False
|
||||
if campaign_id and scope_id != campaign_id:
|
||||
return False
|
||||
return bool(campaign_visible and campaign_visible(scope_id))
|
||||
|
||||
|
||||
def _with_effective_connector_policy(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
profile: ConnectorProfile,
|
||||
) -> ConnectorProfile:
|
||||
sources = effective_connector_policy_sources(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
scope_type=profile.scope_type,
|
||||
scope_id=profile.scope_id,
|
||||
)
|
||||
if not sources:
|
||||
return profile
|
||||
return replace(profile, policy_sources=tuple([*sources, *profile.policy_sources]))
|
||||
@@ -6,7 +6,7 @@ from pathlib import PurePosixPath
|
||||
from typing import Any, Iterable
|
||||
from uuid import uuid4
|
||||
|
||||
from sqlalchemy import and_, or_
|
||||
from sqlalchemy import and_, exists, func, or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.campaigns import CAPABILITY_CAMPAIGNS_ACCESS, CampaignAccessProvider
|
||||
@@ -352,6 +352,8 @@ def list_assets_for_user(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
) -> list[FileAsset]:
|
||||
@@ -363,6 +365,8 @@ def list_assets_for_user(
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
@@ -378,6 +382,8 @@ def list_assets_for_user_window(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
page_size: int,
|
||||
@@ -393,6 +399,8 @@ def list_assets_for_user_window(
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
@@ -417,6 +425,8 @@ def _asset_visibility_query_for_user(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
):
|
||||
@@ -451,7 +461,64 @@ def _asset_visibility_query_for_user(
|
||||
prefix = normalize_folder(path_prefix)
|
||||
if prefix:
|
||||
query = query.filter(FileAsset.display_path.like(f"{prefix}/%"))
|
||||
return query
|
||||
campaign_share_exists = exists().where(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == FileAsset.id,
|
||||
FileShare.target_type == "campaign",
|
||||
FileShare.revoked_at.is_(None),
|
||||
)
|
||||
campaign_use_exists = exists().where(
|
||||
CampaignAttachmentUse.tenant_id == tenant_id,
|
||||
CampaignAttachmentUse.file_asset_id == FileAsset.id,
|
||||
)
|
||||
if campaign_usage == "linked":
|
||||
query = query.filter(or_(campaign_share_exists, campaign_use_exists))
|
||||
elif campaign_usage == "unlinked":
|
||||
query = query.filter(~or_(campaign_share_exists, campaign_use_exists))
|
||||
|
||||
if audit_relevant is not None:
|
||||
sent_use_exists = exists().where(
|
||||
CampaignAttachmentUse.tenant_id == tenant_id,
|
||||
CampaignAttachmentUse.file_asset_id == FileAsset.id,
|
||||
CampaignAttachmentUse.use_stage == "sent",
|
||||
)
|
||||
query = query.filter(sent_use_exists if audit_relevant else ~sent_use_exists)
|
||||
return query.distinct()
|
||||
|
||||
|
||||
def count_assets_for_user(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
owner_type: str | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
) -> int:
|
||||
query = _asset_visibility_query_for_user(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
return int(
|
||||
query.order_by(None)
|
||||
.with_entities(func.count(func.distinct(FileAsset.id)))
|
||||
.scalar()
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def current_version_and_blob(session: Session, asset: FileAsset) -> tuple[FileVersion, FileBlob]:
|
||||
|
||||
@@ -185,14 +185,22 @@ class _OutboundPolicyHTTPTransport(httpx.BaseTransport):
|
||||
self._connection_pool.close()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _stream_connector_request(method: str, url: str, **kwargs: Any) -> Iterator[httpx.Response]:
|
||||
with httpx.Client(
|
||||
_CONNECTOR_HTTP_CLIENT = httpx.Client(
|
||||
transport=_OutboundPolicyHTTPTransport(),
|
||||
follow_redirects=False,
|
||||
timeout=kwargs.pop("timeout", 15.0),
|
||||
) as client:
|
||||
with client.stream(method, url, **kwargs) as response:
|
||||
timeout=15.0,
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _stream_connector_request(method: str, url: str, **kwargs: Any) -> Iterator[httpx.Response]:
|
||||
timeout = kwargs.pop("timeout", 15.0)
|
||||
with _CONNECTOR_HTTP_CLIENT.stream(
|
||||
method,
|
||||
url,
|
||||
timeout=timeout,
|
||||
**kwargs,
|
||||
) as response:
|
||||
yield response
|
||||
|
||||
|
||||
|
||||
@@ -2,14 +2,16 @@ from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.core.access import PrincipalRef
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.capabilities import FilesAccessService, virtual_folder_resource_id
|
||||
from govoplan_files.backend.db.models import FileAsset, FileFolder, FileShare
|
||||
from govoplan_files.backend.storage.files import count_assets_for_user, list_assets_for_user
|
||||
|
||||
|
||||
TENANT_ID = "tenant-1"
|
||||
@@ -75,10 +77,114 @@ class FilesAccessProviderTests(unittest.TestCase):
|
||||
self.assertTrue(any(item.kind == "owner" and item.id == GROUP_ID for item in virtual_items))
|
||||
self.assertEqual("files.not_found", missing_items[0].source)
|
||||
|
||||
def test_file_property_filters_cover_campaign_and_audit_usage(self) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
assets = [
|
||||
FileAsset(
|
||||
id=f"file-{index}",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_type="user",
|
||||
owner_user_id=USER_ID,
|
||||
display_path=f"{index}.pdf",
|
||||
filename=f"{index}.pdf",
|
||||
)
|
||||
for index in range(1, 4)
|
||||
]
|
||||
session.add_all([
|
||||
*assets,
|
||||
FileShare(
|
||||
id="share-campaign",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=assets[0].id,
|
||||
target_type="campaign",
|
||||
target_id="campaign-1",
|
||||
permission="read",
|
||||
),
|
||||
])
|
||||
session.commit()
|
||||
session.execute(
|
||||
text(
|
||||
"INSERT INTO campaign_attachment_uses "
|
||||
"(id, tenant_id, file_asset_id, use_stage) "
|
||||
"VALUES (:id, :tenant_id, :file_asset_id, :use_stage)"
|
||||
),
|
||||
{
|
||||
"id": "attachment-use-1",
|
||||
"tenant_id": TENANT_ID,
|
||||
"file_asset_id": assets[1].id,
|
||||
"use_stage": "sent",
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
|
||||
linked = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="linked",
|
||||
)
|
||||
unlinked = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="unlinked",
|
||||
)
|
||||
audit_relevant = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
audit_relevant=True,
|
||||
)
|
||||
|
||||
self.assertEqual([asset.id for asset in linked], ["file-1", "file-2"])
|
||||
self.assertEqual([asset.id for asset in unlinked], ["file-3"])
|
||||
self.assertEqual([asset.id for asset in audit_relevant], ["file-2"])
|
||||
self.assertEqual(
|
||||
count_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="unlinked",
|
||||
),
|
||||
1,
|
||||
)
|
||||
|
||||
|
||||
def _session():
|
||||
engine = create_engine("sqlite:///:memory:", future=True)
|
||||
Base.metadata.create_all(bind=engine, tables=[Account.__table__, User.__table__, Group.__table__, FileAsset.__table__, FileFolder.__table__, FileShare.__table__])
|
||||
Base.metadata.create_all(
|
||||
bind=engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
FileAsset.__table__,
|
||||
FileFolder.__table__,
|
||||
FileShare.__table__,
|
||||
],
|
||||
)
|
||||
with engine.begin() as connection:
|
||||
connection.execute(
|
||||
text(
|
||||
"CREATE TABLE campaign_attachment_uses ("
|
||||
"id VARCHAR(36) PRIMARY KEY, "
|
||||
"tenant_id VARCHAR(36) NOT NULL, "
|
||||
"file_asset_id VARCHAR(36) NOT NULL, "
|
||||
"use_stage VARCHAR(20) NOT NULL"
|
||||
")"
|
||||
)
|
||||
)
|
||||
return sessionmaker(bind=engine, future=True)()
|
||||
|
||||
|
||||
|
||||
@@ -12,7 +12,8 @@ from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.security.secrets import encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
||||
from govoplan_files.backend.router import deactivate_connector_credential, deactivate_connector_profile
|
||||
from govoplan_files.backend.routes.connector_profiles import deactivate_connector_profile
|
||||
from govoplan_files.backend.routes.connector_settings import deactivate_connector_credential
|
||||
|
||||
|
||||
class Principal:
|
||||
|
||||
@@ -10,7 +10,7 @@ from unittest.mock import MagicMock
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
from govoplan_files.backend.router import discover_connector_endpoint
|
||||
from govoplan_files.backend.routes.connector_settings import discover_connector_endpoint
|
||||
from govoplan_files.backend.schemas import FileConnectorDiscoveryRequest
|
||||
from govoplan_files.backend.storage.connector_browse import ConnectorBrowseError, _profile_password, _s3_verify
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
@@ -182,7 +182,7 @@ class ConnectorDiscoveryBoundaryTests(unittest.TestCase):
|
||||
credential_mode="basic",
|
||||
credentials={"username": "admin", "password_env": "MASTER_KEY_B64"},
|
||||
)
|
||||
with patch("govoplan_files.backend.router.browse_connector_profile") as browse, self.assertRaises(
|
||||
with patch("govoplan_files.backend.routes.connector_settings.browse_connector_profile") as browse, self.assertRaises(
|
||||
HTTPException
|
||||
) as raised:
|
||||
discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||
@@ -212,10 +212,10 @@ class ConnectorDiscoveryBoundaryTests(unittest.TestCase):
|
||||
events.append("io")
|
||||
return []
|
||||
|
||||
with patch("govoplan_files.backend.router._ensure_connector_configuration_allowed", side_effect=ensure), patch(
|
||||
"govoplan_files.backend.router._audit_connector_discovery_attempt",
|
||||
with patch("govoplan_files.backend.routes.connector_settings._ensure_connector_configuration_allowed", side_effect=ensure), patch(
|
||||
"govoplan_files.backend.routes.connector_settings._audit_connector_discovery_attempt",
|
||||
side_effect=audit,
|
||||
), patch("govoplan_files.backend.router.browse_connector_profile", side_effect=browse):
|
||||
), patch("govoplan_files.backend.routes.connector_settings.browse_connector_profile", side_effect=browse):
|
||||
response = discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||
|
||||
self.assertEqual("usable", response.status)
|
||||
|
||||
66
tests/test_connector_policy.py
Normal file
66
tests/test_connector_policy.py
Normal file
@@ -0,0 +1,66 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicySource,
|
||||
connector_policy_decision,
|
||||
)
|
||||
|
||||
|
||||
class ConnectorPolicyPatternTests(unittest.TestCase):
|
||||
def test_resource_reference_allow_patterns_match(self) -> None:
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id="dev-smb",
|
||||
credential_id="credential-primary",
|
||||
provider="smb",
|
||||
),
|
||||
(
|
||||
ConnectorPolicySource(
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
label="Tenant",
|
||||
policy={
|
||||
"allow": {
|
||||
"connectors": ["dev-*"],
|
||||
"credentials": ["credential-*"],
|
||||
"providers": ["s?b"],
|
||||
}
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
self.assertTrue(decision.allowed)
|
||||
|
||||
def test_resource_reference_deny_patterns_take_precedence(self) -> None:
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id="dev-smb",
|
||||
credential_id="credential-legacy",
|
||||
provider="smb",
|
||||
),
|
||||
(
|
||||
ConnectorPolicySource(
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
label="Tenant",
|
||||
policy={
|
||||
"allow": {"connectors": ["dev-*"]},
|
||||
"deny": {"credentials": ["*-legacy"]},
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
self.assertFalse(decision.allowed)
|
||||
self.assertEqual(
|
||||
("connector_policy_denylist",),
|
||||
decision.requirements,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -8,16 +9,26 @@ from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db import models as access_models # noqa: F401 - resolve Files user foreign keys
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.security.credential_envelopes import CredentialEnvelope
|
||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorProfile
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_profile_store import update_connector_profile_row
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
list_database_connector_profiles,
|
||||
update_connector_profile_row,
|
||||
)
|
||||
|
||||
|
||||
class ConnectorProfileStoreUpdateTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.engine = create_engine("sqlite:///:memory:")
|
||||
Base.metadata.create_all(self.engine, tables=[FileConnectorProfile.__table__])
|
||||
Base.metadata.create_all(
|
||||
self.engine,
|
||||
tables=[
|
||||
FileConnectorProfile.__table__,
|
||||
CredentialEnvelope.__table__,
|
||||
],
|
||||
)
|
||||
self.session = sessionmaker(bind=self.engine)()
|
||||
self.row = FileConnectorProfile(
|
||||
id="profile-1",
|
||||
@@ -140,6 +151,78 @@ class ConnectorProfileStoreUpdateTests(unittest.TestCase):
|
||||
self.assertEqual("Original profile", persisted.label)
|
||||
self.assertEqual("original-password", decrypt_secret(persisted.password_encrypted))
|
||||
|
||||
def test_visibility_filter_runs_before_connector_secrets_are_decrypted(self) -> None:
|
||||
invisible = FileConnectorProfile(
|
||||
id="invisible-profile",
|
||||
tenant_id="tenant-1",
|
||||
scope_type="user",
|
||||
scope_id="another-user",
|
||||
label="Invisible profile",
|
||||
provider="webdav",
|
||||
endpoint_url="https://invisible.example.test",
|
||||
enabled=True,
|
||||
credential_mode="basic",
|
||||
password_encrypted="not-a-valid-encrypted-secret",
|
||||
capabilities=["browse"],
|
||||
policy={},
|
||||
metadata_={},
|
||||
)
|
||||
self.session.add(invisible)
|
||||
self.session.commit()
|
||||
|
||||
profiles = list_database_connector_profiles(
|
||||
self.session,
|
||||
tenant_id="tenant-1",
|
||||
row_visible=lambda row: row.id == self.row.id,
|
||||
)
|
||||
|
||||
self.assertEqual([self.row.id], [profile.id for profile in profiles])
|
||||
self.assertEqual("original-password", profiles[0].password_value)
|
||||
|
||||
def test_reusable_credential_resolves_and_revocation_keeps_profile_visible(self) -> None:
|
||||
credential = CredentialEnvelope(
|
||||
id="shared-credential",
|
||||
tenant_id="tenant-1",
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
name="Shared WebDAV login",
|
||||
credential_kind="username_password",
|
||||
public_data={"username": "ada"},
|
||||
secret_data_encrypted=encrypt_secret(json.dumps({"password": "secret"})),
|
||||
secret_keys=["password"],
|
||||
allowed_modules=["files"],
|
||||
allowed_server_refs=[],
|
||||
inherit_to_lower_scopes=True,
|
||||
is_active=True,
|
||||
revision="revision-1",
|
||||
)
|
||||
self.row.credential_profile_id = "credential-envelope:shared-credential"
|
||||
self.row.username = None
|
||||
self.row.password_encrypted = None
|
||||
self.row.token_encrypted = None
|
||||
self.session.add(credential)
|
||||
self.session.commit()
|
||||
|
||||
profile = list_database_connector_profiles(
|
||||
self.session,
|
||||
tenant_id="tenant-1",
|
||||
)[0]
|
||||
|
||||
self.assertEqual("ada", profile.username)
|
||||
self.assertEqual("secret", profile.password_value)
|
||||
self.assertTrue(profile.credentials_configured)
|
||||
|
||||
credential.is_active = False
|
||||
self.session.commit()
|
||||
profile = list_database_connector_profiles(
|
||||
self.session,
|
||||
tenant_id="tenant-1",
|
||||
)[0]
|
||||
|
||||
self.assertIsNone(profile.password_value)
|
||||
self.assertFalse(profile.credentials_configured)
|
||||
self.assertTrue(profile.metadata["credential_unavailable"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
265
tests/test_connector_visibility.py
Normal file
265
tests/test_connector_visibility.py
Normal file
@@ -0,0 +1,265 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from unittest.mock import patch
|
||||
|
||||
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||
from govoplan_files.backend.storage.connector_visibility import (
|
||||
connector_profile_usable_for_import,
|
||||
visible_connector_profiles_for_actor,
|
||||
)
|
||||
|
||||
|
||||
def _profile(
|
||||
profile_id: str,
|
||||
*,
|
||||
scope_type: str = "system",
|
||||
scope_id: str | None = None,
|
||||
provider: str = "webdav",
|
||||
enabled: bool = True,
|
||||
) -> ConnectorProfile:
|
||||
return ConnectorProfile(
|
||||
id=profile_id,
|
||||
label=profile_id,
|
||||
provider=provider,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
endpoint_url=f"https://{profile_id}.example.invalid",
|
||||
enabled=enabled,
|
||||
credential_mode="anonymous",
|
||||
)
|
||||
|
||||
|
||||
class ConnectorVisibilityTests(unittest.TestCase):
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.effective_connector_policy_sources",
|
||||
return_value=[],
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.connector_profiles_from_settings"
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.select_database_connector_profiles"
|
||||
)
|
||||
def test_profiles_are_filtered_to_actor_scopes_and_database_definition_wins(
|
||||
self,
|
||||
database_profiles,
|
||||
configured_profiles,
|
||||
_policy_sources,
|
||||
) -> None:
|
||||
profiles = [
|
||||
_profile("system"),
|
||||
_profile("tenant", scope_type="tenant", scope_id="tenant-1"),
|
||||
_profile("other-tenant", scope_type="tenant", scope_id="tenant-2"),
|
||||
_profile("user", scope_type="user", scope_id="user-1"),
|
||||
_profile("other-user", scope_type="user", scope_id="user-2"),
|
||||
_profile("group", scope_type="group", scope_id="group-1"),
|
||||
_profile("campaign", scope_type="campaign", scope_id="campaign-1"),
|
||||
_profile("disabled", enabled=False),
|
||||
_profile("duplicate", provider="webdav"),
|
||||
]
|
||||
database_profiles.return_value = (profiles, {profile.id for profile in profiles})
|
||||
configured_profiles.return_value = [_profile("duplicate", provider="nextcloud")]
|
||||
|
||||
visible = visible_connector_profiles_for_actor(
|
||||
object(), # type: ignore[arg-type]
|
||||
tenant_id="tenant-1",
|
||||
user_id="user-1",
|
||||
group_ids={"group-1"},
|
||||
settings=object(),
|
||||
campaign_visible=lambda campaign_id: campaign_id == "campaign-1",
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
{"system", "tenant", "user", "group", "campaign", "duplicate"},
|
||||
{profile.id for profile in visible},
|
||||
)
|
||||
duplicate = next(profile for profile in visible if profile.id == "duplicate")
|
||||
self.assertEqual("webdav", duplicate.provider)
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.effective_connector_policy_sources",
|
||||
return_value=[],
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.connector_profiles_from_settings",
|
||||
return_value=[],
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.select_database_connector_profiles"
|
||||
)
|
||||
def test_campaign_and_admin_visibility_remain_explicit(
|
||||
self,
|
||||
database_profiles,
|
||||
_configured_profiles,
|
||||
_policy_sources,
|
||||
) -> None:
|
||||
profiles = [
|
||||
_profile("campaign-a", scope_type="campaign", scope_id="campaign-a"),
|
||||
_profile("campaign-b", scope_type="campaign", scope_id="campaign-b"),
|
||||
_profile("other-user", scope_type="user", scope_id="user-2"),
|
||||
]
|
||||
database_profiles.return_value = (profiles, {profile.id for profile in profiles})
|
||||
|
||||
campaign_only = visible_connector_profiles_for_actor(
|
||||
object(), # type: ignore[arg-type]
|
||||
tenant_id="tenant-1",
|
||||
user_id="user-1",
|
||||
group_ids=(),
|
||||
settings=None,
|
||||
campaign_id="campaign-a",
|
||||
campaign_visible=lambda _campaign_id: True,
|
||||
)
|
||||
self.assertEqual(["campaign-a"], [profile.id for profile in campaign_only])
|
||||
|
||||
admin = visible_connector_profiles_for_actor(
|
||||
object(), # type: ignore[arg-type]
|
||||
tenant_id="tenant-1",
|
||||
user_id="user-1",
|
||||
group_ids=(),
|
||||
settings=None,
|
||||
campaign_visible=None,
|
||||
include_admin_scopes=True,
|
||||
)
|
||||
self.assertEqual(
|
||||
{"campaign-a", "campaign-b", "other-user"},
|
||||
{profile.id for profile in admin},
|
||||
)
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.connector_profiles_from_settings",
|
||||
return_value=[],
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.select_database_connector_profiles"
|
||||
)
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.connector_visibility.effective_connector_policy_sources"
|
||||
)
|
||||
def test_effective_policy_is_attached_without_mutating_profile(
|
||||
self,
|
||||
policy_sources,
|
||||
database_profiles,
|
||||
_configured_profiles,
|
||||
) -> None:
|
||||
profile = _profile("profile")
|
||||
source = ConnectorPolicySource(
|
||||
scope_type="system",
|
||||
label="System",
|
||||
policy={"allow": {"providers": ["webdav"]}},
|
||||
)
|
||||
database_profiles.return_value = ([profile], {profile.id})
|
||||
policy_sources.return_value = [source]
|
||||
|
||||
visible = visible_connector_profiles_for_actor(
|
||||
object(), # type: ignore[arg-type]
|
||||
tenant_id="tenant-1",
|
||||
user_id="user-1",
|
||||
group_ids=(),
|
||||
settings=None,
|
||||
)
|
||||
|
||||
self.assertEqual((), profile.policy_sources)
|
||||
self.assertEqual((source,), visible[0].policy_sources)
|
||||
|
||||
def test_import_usability_requires_safe_provider_credentials_endpoint_and_identity_policy(
|
||||
self,
|
||||
) -> None:
|
||||
self.assertTrue(connector_profile_usable_for_import(_profile("webdav")))
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(_profile("s3", provider="s3"))
|
||||
)
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(_profile("smb", provider="smb"))
|
||||
)
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(
|
||||
_profile("sharepoint", provider="sharepoint")
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(
|
||||
ConnectorProfile(
|
||||
id="no-endpoint",
|
||||
label="No endpoint",
|
||||
provider="webdav",
|
||||
credential_mode="anonymous",
|
||||
)
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(
|
||||
ConnectorProfile(
|
||||
id="no-credentials",
|
||||
label="No credentials",
|
||||
provider="webdav",
|
||||
endpoint_url="https://files.example.invalid",
|
||||
credential_mode="basic",
|
||||
)
|
||||
)
|
||||
)
|
||||
self.assertTrue(
|
||||
connector_profile_usable_for_import(
|
||||
ConnectorProfile(
|
||||
id="metadata-endpoint",
|
||||
label="Metadata endpoint",
|
||||
provider="webdav",
|
||||
credential_mode="anonymous",
|
||||
metadata={"webdav_endpoint_url": "https://files.example.invalid"},
|
||||
)
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
connector_profile_usable_for_import(
|
||||
ConnectorProfile(
|
||||
id="unresolved-secret-reference",
|
||||
label="Unresolved secret reference",
|
||||
provider="webdav",
|
||||
endpoint_url="https://files.example.invalid",
|
||||
credential_mode="basic",
|
||||
secret_ref="vault://files/webdav",
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
denied = replace(
|
||||
_profile("denied"),
|
||||
policy_sources=(
|
||||
ConnectorPolicySource(
|
||||
scope_type="system",
|
||||
label="System",
|
||||
policy={"deny": {"providers": ["webdav"]}},
|
||||
),
|
||||
),
|
||||
)
|
||||
self.assertFalse(connector_profile_usable_for_import(denied))
|
||||
|
||||
path_limited = replace(
|
||||
_profile("path-limited"),
|
||||
policy_sources=(
|
||||
ConnectorPolicySource(
|
||||
scope_type="system",
|
||||
label="System",
|
||||
policy={"allow": {"external_paths": ["/approved/*"]}},
|
||||
),
|
||||
),
|
||||
)
|
||||
self.assertFalse(connector_profile_usable_for_import(path_limited))
|
||||
|
||||
endpoint_allowed = replace(
|
||||
_profile("endpoint-allowed"),
|
||||
policy_sources=(
|
||||
ConnectorPolicySource(
|
||||
scope_type="system",
|
||||
label="System",
|
||||
policy={"allow": {"external_urls": ["https://*.example.invalid"]}},
|
||||
),
|
||||
),
|
||||
)
|
||||
self.assertTrue(connector_profile_usable_for_import(endpoint_allowed))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
241
tests/test_documentation.py
Normal file
241
tests/test_documentation.py
Normal file
@@ -0,0 +1,241 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.modules import DocumentationContext
|
||||
from govoplan_files.backend.documentation import documentation_topics
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||
|
||||
|
||||
class _Principal:
|
||||
tenant_id = "tenant-1"
|
||||
user = SimpleNamespace(id="user-1")
|
||||
group_ids = frozenset({"group-1"})
|
||||
|
||||
def __init__(self, scopes: set[str]) -> None:
|
||||
self.scopes = frozenset(scopes)
|
||||
|
||||
def has(self, scope: str) -> bool:
|
||||
return scope in self.scopes
|
||||
|
||||
|
||||
class FilesRuntimeDocumentationTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.session = Session()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.session.close()
|
||||
|
||||
def context(
|
||||
self,
|
||||
scopes: set[str],
|
||||
*,
|
||||
settings: object | None = None,
|
||||
documentation_type: str = "user",
|
||||
) -> DocumentationContext:
|
||||
return DocumentationContext(
|
||||
registry=object(),
|
||||
principal=_Principal(scopes),
|
||||
settings=settings,
|
||||
session=self.session,
|
||||
documentation_type=documentation_type, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
def test_upload_tasks_state_exact_current_safe_limits(self) -> None:
|
||||
settings = SimpleNamespace(
|
||||
file_upload_max_bytes=7 * 1024 * 1024,
|
||||
file_upload_zip_max_bytes=19 * 1024 * 1024,
|
||||
)
|
||||
with patch(
|
||||
"govoplan_files.backend.documentation.visible_connector_profiles_for_actor",
|
||||
return_value=[],
|
||||
):
|
||||
topics = {
|
||||
topic.id: topic
|
||||
for topic in documentation_topics(
|
||||
self.context(
|
||||
{"files:file:read", "files:file:upload"}, settings=settings
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
upload = topics["files.workflow.upload-managed-files"]
|
||||
self.assertIn("7 MiB (7,340,032 bytes)", upload.body)
|
||||
self.assertEqual(["files.list"], upload.metadata["help_contexts"])
|
||||
self.assertEqual(
|
||||
{"files:file:read", "files:file:upload"},
|
||||
set(upload.conditions[0].required_scopes),
|
||||
)
|
||||
|
||||
archive = topics["files.workflow.upload-and-unpack-zip"]
|
||||
self.assertIn("19 MiB (19,922,944 bytes)", archive.body)
|
||||
self.assertIn("7 MiB (7,340,032 bytes)", archive.body)
|
||||
self.assertIn("1,000", archive.body)
|
||||
self.assertIn("Actual extracted bytes are counted", archive.body)
|
||||
|
||||
def test_connector_task_requires_authority_and_a_visible_usable_profile(
|
||||
self,
|
||||
) -> None:
|
||||
usable = ConnectorProfile(
|
||||
id="private-profile-id",
|
||||
label="Private profile label",
|
||||
provider="webdav",
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
endpoint_url="https://private.example.invalid/secret-root",
|
||||
base_path="/secret-root",
|
||||
credential_mode="anonymous",
|
||||
)
|
||||
context = self.context({"files:file:read", "files:file:upload"})
|
||||
with patch(
|
||||
"govoplan_files.backend.documentation.visible_connector_profiles_for_actor",
|
||||
return_value=[usable],
|
||||
) as visible:
|
||||
topics = {topic.id: topic for topic in documentation_topics(context)}
|
||||
|
||||
self.assertIn("files.workflow.import-managed-snapshot", topics)
|
||||
self.assertNotIn("files.connector-import-unavailable", topics)
|
||||
task = topics["files.workflow.import-managed-snapshot"]
|
||||
self.assertEqual("configured", task.layer)
|
||||
self.assertEqual(
|
||||
["files.list", "files.connector-import"], task.metadata["help_contexts"]
|
||||
)
|
||||
self.assertIn("re-authorized", task.body)
|
||||
self.assertNotIn("private-profile-id", repr(task))
|
||||
self.assertNotIn("private.example.invalid", repr(task))
|
||||
self.assertNotIn("secret-root", repr(task))
|
||||
visible.assert_called_once()
|
||||
self.assertEqual(("group-1",), tuple(visible.call_args.kwargs["group_ids"]))
|
||||
|
||||
without_authority = {
|
||||
topic.id: topic
|
||||
for topic in documentation_topics(self.context({"files:file:read"}))
|
||||
}
|
||||
self.assertNotIn("files.workflow.import-managed-snapshot", without_authority)
|
||||
self.assertIn(
|
||||
"both permission to view Files and permission to upload",
|
||||
without_authority["files.connector-import-unavailable"].body,
|
||||
)
|
||||
|
||||
def test_connector_limitation_is_precise_but_never_exposes_profile_internals(
|
||||
self,
|
||||
) -> None:
|
||||
blocked = ConnectorProfile(
|
||||
id="sensitive-profile-id",
|
||||
label="Sensitive label",
|
||||
provider="s3",
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
endpoint_url="https://internal.example.invalid/private",
|
||||
base_path="/classified",
|
||||
credential_mode="basic",
|
||||
password_value="credential-secret",
|
||||
)
|
||||
with patch(
|
||||
"govoplan_files.backend.documentation.visible_connector_profiles_for_actor",
|
||||
return_value=[blocked],
|
||||
):
|
||||
topics = {
|
||||
topic.id: topic
|
||||
for topic in documentation_topics(
|
||||
self.context({"files:file:read", "files:file:upload"})
|
||||
)
|
||||
}
|
||||
|
||||
self.assertNotIn("files.workflow.import-managed-snapshot", topics)
|
||||
limitation = topics["files.connector-import-unavailable"]
|
||||
self.assertEqual("available", limitation.layer)
|
||||
self.assertIn("pinning-safe provider", limitation.body)
|
||||
payload = repr(limitation)
|
||||
for secret in (
|
||||
"sensitive-profile-id",
|
||||
"Sensitive label",
|
||||
"internal.example.invalid",
|
||||
"classified",
|
||||
"credential-secret",
|
||||
):
|
||||
self.assertNotIn(secret, payload)
|
||||
|
||||
def test_files_admin_connector_groups_do_not_widen_campaign_membership(self) -> None:
|
||||
usable = ConnectorProfile(
|
||||
id="group-profile",
|
||||
label="Group profile",
|
||||
provider="webdav",
|
||||
scope_type="group",
|
||||
scope_id="admin-visible-group",
|
||||
endpoint_url="https://files.example.invalid",
|
||||
credential_mode="anonymous",
|
||||
)
|
||||
|
||||
def groups_for_actor(_session, *, include_admin_groups, **_kwargs):
|
||||
return ["member-group", "admin-visible-group"] if include_admin_groups else ["member-group"]
|
||||
|
||||
with (
|
||||
patch(
|
||||
"govoplan_files.backend.documentation.user_group_ids",
|
||||
side_effect=groups_for_actor,
|
||||
),
|
||||
patch(
|
||||
"govoplan_files.backend.documentation._campaign_visibility",
|
||||
return_value=None,
|
||||
) as campaign_visibility,
|
||||
patch(
|
||||
"govoplan_files.backend.documentation.visible_connector_profiles_for_actor",
|
||||
return_value=[usable],
|
||||
) as visible,
|
||||
):
|
||||
topics = {
|
||||
topic.id: topic
|
||||
for topic in documentation_topics(
|
||||
self.context(
|
||||
{
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:admin",
|
||||
}
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
self.assertIn("files.workflow.import-managed-snapshot", topics)
|
||||
self.assertEqual(
|
||||
("member-group", "admin-visible-group"),
|
||||
tuple(visible.call_args.kwargs["group_ids"]),
|
||||
)
|
||||
self.assertEqual(
|
||||
("member-group",),
|
||||
tuple(campaign_visibility.call_args.kwargs["group_ids"]),
|
||||
)
|
||||
|
||||
def test_connector_evaluation_errors_fail_closed_without_error_details(
|
||||
self,
|
||||
) -> None:
|
||||
with patch(
|
||||
"govoplan_files.backend.documentation.visible_connector_profiles_for_actor",
|
||||
side_effect=RuntimeError("private endpoint /root credential-id"),
|
||||
):
|
||||
topics = {
|
||||
topic.id: topic
|
||||
for topic in documentation_topics(
|
||||
self.context({"files:file:read", "files:file:upload"})
|
||||
)
|
||||
}
|
||||
|
||||
self.assertNotIn("files.workflow.import-managed-snapshot", topics)
|
||||
limitation = topics["files.connector-import-unavailable"]
|
||||
self.assertIn("could not be safely evaluated", limitation.body)
|
||||
self.assertNotIn("private endpoint", repr(limitation))
|
||||
self.assertNotIn("credential-id", repr(limitation))
|
||||
|
||||
def test_runtime_provider_only_emits_user_documentation(self) -> None:
|
||||
self.assertEqual(
|
||||
(), documentation_topics(self.context(set(), documentation_type="admin"))
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -3,14 +3,22 @@ from __future__ import annotations
|
||||
import unittest
|
||||
|
||||
|
||||
TOPIC_IDS = {
|
||||
"files.workflow.upload-organize-and-share",
|
||||
"files.workflow.import-managed-snapshot",
|
||||
STATIC_TOPIC_IDS = {
|
||||
"files.workflow.organize-managed-files",
|
||||
"files.workflow.find-and-download-files",
|
||||
"files.workflow.share-managed-files",
|
||||
"files.workflow.delete-managed-files",
|
||||
"files.governed-connectors-and-provenance",
|
||||
"files.reference.integrity-recovery-and-fail-closed-transports",
|
||||
"files.reference.snapshot-provenance-and-capabilities",
|
||||
"files.assurance.process-and-release-readiness",
|
||||
}
|
||||
RUNTIME_TOPIC_IDS = {
|
||||
"files.workflow.upload-managed-files",
|
||||
"files.workflow.upload-and-unpack-zip",
|
||||
"files.workflow.import-managed-snapshot",
|
||||
"files.connector-import-unavailable",
|
||||
}
|
||||
HANDBOOK_HREF = "govoplan-files/docs/FILES_HANDBOOK.md"
|
||||
|
||||
|
||||
@@ -19,64 +27,98 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
def setUpClass(cls) -> None:
|
||||
from govoplan_files.backend.manifest import manifest
|
||||
|
||||
cls.manifest = manifest
|
||||
cls.topics = {topic.id: topic for topic in manifest.documentation}
|
||||
|
||||
def topic(self, topic_id: str):
|
||||
return self.topics[topic_id]
|
||||
|
||||
def test_adaptive_topics_have_role_scope_module_and_link_contracts(self) -> None:
|
||||
self.assertEqual(TOPIC_IDS, set(self.topics))
|
||||
self.assertEqual({"admin", "user"}, {item for topic in self.topics.values() for item in topic.documentation_types})
|
||||
def test_static_topics_have_role_scope_module_and_link_contracts(self) -> None:
|
||||
self.assertEqual(STATIC_TOPIC_IDS, set(self.topics))
|
||||
self.assertEqual(
|
||||
{"admin", "user"},
|
||||
{
|
||||
item
|
||||
for topic in self.topics.values()
|
||||
for item in topic.documentation_types
|
||||
},
|
||||
)
|
||||
|
||||
for topic in self.topics.values():
|
||||
with self.subTest(topic=topic.id):
|
||||
self.assertTrue(topic.audience)
|
||||
self.assertTrue(topic.conditions)
|
||||
self.assertTrue(any("files" in condition.required_modules for condition in topic.conditions))
|
||||
self.assertTrue(any(condition.any_scopes or condition.required_scopes for condition in topic.conditions))
|
||||
self.assertIn("runtime", {link.kind for link in topic.links})
|
||||
self.assertIn("api", {link.kind for link in topic.links})
|
||||
self.assertIn(HANDBOOK_HREF, {link.href for link in topic.links if link.kind == "repository"})
|
||||
|
||||
def test_import_topic_is_a_complete_user_workflow(self) -> None:
|
||||
topic = self.topic("files.workflow.import-managed-snapshot")
|
||||
|
||||
self.assertEqual(("user",), topic.documentation_types)
|
||||
self.assertEqual("workflow", topic.metadata["kind"])
|
||||
self.assertEqual("/files", topic.metadata["route"])
|
||||
self.assertEqual("Files", topic.metadata["screen"])
|
||||
self.assertEqual(
|
||||
{"files:file:read", "files:file:upload"},
|
||||
set(topic.conditions[0].required_scopes),
|
||||
self.assertTrue(
|
||||
any(
|
||||
"files" in condition.required_modules
|
||||
for condition in topic.conditions
|
||||
)
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
condition.any_scopes or condition.required_scopes
|
||||
for condition in topic.conditions
|
||||
)
|
||||
)
|
||||
self.assertIn("runtime", {link.kind for link in topic.links})
|
||||
self.assertIn(
|
||||
HANDBOOK_HREF,
|
||||
{link.href for link in topic.links if link.kind == "repository"},
|
||||
)
|
||||
for key in ("prerequisites", "steps", "outcome", "verification"):
|
||||
self.assertTrue(topic.metadata[key])
|
||||
self.assertIn("never mutate the remote source", topic.body)
|
||||
self.assertIn("/api/v1/files/connectors/profiles/{profile_id}/import", {link.href for link in topic.links})
|
||||
|
||||
def test_managed_file_workflow_covers_upload_organization_and_current_share_boundary(self) -> None:
|
||||
topic = self.topic("files.workflow.upload-organize-and-share")
|
||||
self.assertIn(
|
||||
"documentation_topics",
|
||||
{provider.__name__ for provider in self.manifest.documentation_providers},
|
||||
)
|
||||
|
||||
self.assertEqual(("user",), topic.documentation_types)
|
||||
self.assertEqual("workflow", topic.metadata["kind"])
|
||||
self.assertEqual(
|
||||
{
|
||||
def test_user_tasks_are_independently_authorized_and_contextual(self) -> None:
|
||||
expected_scopes = {
|
||||
"files.workflow.organize-managed-files": {
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:organize",
|
||||
},
|
||||
"files.workflow.find-and-download-files": {
|
||||
"files:file:read",
|
||||
"files:file:download",
|
||||
},
|
||||
"files.workflow.share-managed-files": {
|
||||
"files:file:read",
|
||||
"files:file:share",
|
||||
},
|
||||
set(topic.conditions[0].required_scopes),
|
||||
)
|
||||
"files.workflow.delete-managed-files": {
|
||||
"files:file:read",
|
||||
"files:file:delete",
|
||||
},
|
||||
}
|
||||
for topic_id, scopes in expected_scopes.items():
|
||||
with self.subTest(topic=topic_id):
|
||||
topic = self.topic(topic_id)
|
||||
self.assertEqual(("user",), topic.documentation_types)
|
||||
self.assertEqual("workflow", topic.metadata["kind"])
|
||||
self.assertEqual(scopes, set(topic.conditions[0].required_scopes))
|
||||
self.assertEqual(["files.list"], topic.metadata["help_contexts"])
|
||||
for key in ("prerequisites", "steps", "outcome", "verification"):
|
||||
self.assertTrue(topic.metadata[key])
|
||||
self.assertIn("does not yet provide a general share editor", topic.body)
|
||||
self.assertIn("no share-revocation route", topic.body)
|
||||
self.assertIn("/api/v1/files/upload", {link.href for link in topic.links})
|
||||
self.assertIn("/api/v1/files/transfer", {link.href for link in topic.links})
|
||||
self.assertIn("/api/v1/files/{file_id}/shares", {link.href for link in topic.links})
|
||||
|
||||
def test_admin_topic_covers_policy_redaction_and_atomic_credential_deletion(self) -> None:
|
||||
def test_share_and_delete_tasks_state_current_boundaries(self) -> None:
|
||||
share = self.topic("files.workflow.share-managed-files")
|
||||
self.assertEqual("available", share.layer)
|
||||
self.assertIn("does not yet provide a general share editor", share.body)
|
||||
self.assertIn("no share-revocation route", share.body)
|
||||
self.assertIn(
|
||||
"/api/v1/files/{file_id}/shares", {link.href for link in share.links}
|
||||
)
|
||||
|
||||
delete = self.topic("files.workflow.delete-managed-files")
|
||||
self.assertIn("Soft-delete", delete.summary)
|
||||
self.assertIn("no self-service restore or hard-purge", delete.body)
|
||||
self.assertTrue(
|
||||
any("not a hard purge" in item for item in delete.metadata["limitations"])
|
||||
)
|
||||
|
||||
def test_admin_topic_covers_policy_redaction_and_atomic_credential_deletion(
|
||||
self,
|
||||
) -> None:
|
||||
topic = self.topic("files.governed-connectors-and-provenance")
|
||||
|
||||
self.assertEqual(("admin",), topic.documentation_types)
|
||||
@@ -86,12 +128,24 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("deny rules win", topic.body)
|
||||
self.assertIn("redact secret values", topic.body)
|
||||
self.assertIn("same transaction", topic.body)
|
||||
self.assertTrue(any("non-owned external references" in item for item in topic.metadata["security_invariants"]))
|
||||
self.assertIn("/api/v1/files/connectors/policies/tenant", {link.href for link in topic.links})
|
||||
self.assertIn("/api/v1/files/connectors/credentials", {link.href for link in topic.links})
|
||||
self.assertTrue(
|
||||
any(
|
||||
"non-owned external references" in item
|
||||
for item in topic.metadata["security_invariants"]
|
||||
)
|
||||
)
|
||||
self.assertIn(
|
||||
"/api/v1/files/connectors/policies/tenant",
|
||||
{link.href for link in topic.links},
|
||||
)
|
||||
self.assertIn(
|
||||
"/api/v1/files/connectors/credentials", {link.href for link in topic.links}
|
||||
)
|
||||
|
||||
def test_operator_topic_covers_recovery_and_fail_closed_s3_smb(self) -> None:
|
||||
topic = self.topic("files.reference.integrity-recovery-and-fail-closed-transports")
|
||||
topic = self.topic(
|
||||
"files.reference.integrity-recovery-and-fail-closed-transports"
|
||||
)
|
||||
|
||||
self.assertEqual(("admin",), topic.documentation_types)
|
||||
self.assertEqual("reference", topic.metadata["kind"])
|
||||
@@ -102,7 +156,9 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("does not remove backend blob objects", topic.body)
|
||||
self.assertIn("MASTER_KEY_B64", topic.metadata["recovery_unit"])
|
||||
self.assertTrue(topic.metadata["verification"])
|
||||
self.assertIn("GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS", topic.configuration_keys)
|
||||
self.assertIn(
|
||||
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS", topic.configuration_keys
|
||||
)
|
||||
|
||||
def test_integrator_topic_exposes_capability_and_provenance_boundary(self) -> None:
|
||||
topic = self.topic("files.reference.snapshot-provenance-and-capabilities")
|
||||
@@ -117,11 +173,14 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("exact asset, version, blob, checksum", topic.body)
|
||||
self.assertIn("collaboration", topic.body)
|
||||
|
||||
def test_process_and_release_assurance_exposes_gates_evidence_and_limits(self) -> None:
|
||||
def test_process_and_release_assurance_exposes_gates_evidence_and_limits(
|
||||
self,
|
||||
) -> None:
|
||||
topic = self.topic("files.assurance.process-and-release-readiness")
|
||||
|
||||
self.assertEqual(("admin", "user"), topic.documentation_types)
|
||||
self.assertEqual("workflow", topic.metadata["kind"])
|
||||
self.assertEqual(["files.list"], topic.metadata["help_contexts"])
|
||||
self.assertIn("process_owner", topic.audience)
|
||||
self.assertIn("release_manager", topic.audience)
|
||||
self.assertIn("versions align", topic.body)
|
||||
@@ -129,16 +188,22 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("no enforced retention or legal hold", topic.body)
|
||||
for key in ("prerequisites", "steps", "outcome", "verification"):
|
||||
self.assertTrue(topic.metadata[key])
|
||||
self.assertTrue(any("meta-repository security" in step for step in topic.metadata["steps"]))
|
||||
self.assertTrue(
|
||||
any("meta-repository security" in step for step in topic.metadata["steps"])
|
||||
)
|
||||
self.assertTrue(any("fails closed" in step for step in topic.metadata["steps"]))
|
||||
self.assertTrue(any("SHA-256" in step for step in topic.metadata["steps"]))
|
||||
self.assertIn(HANDBOOK_HREF, {link.href for link in topic.links if link.kind == "repository"})
|
||||
|
||||
def test_internal_related_topic_ids_resolve(self) -> None:
|
||||
known_ids = STATIC_TOPIC_IDS | RUNTIME_TOPIC_IDS
|
||||
for topic in self.topics.values():
|
||||
for related_id in topic.metadata.get("related_topic_ids", []):
|
||||
if related_id.startswith("files."):
|
||||
self.assertIn(related_id, TOPIC_IDS, f"{topic.id} refers to missing topic {related_id}")
|
||||
self.assertIn(
|
||||
related_id,
|
||||
known_ids,
|
||||
f"{topic.id} refers to missing topic {related_id}",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1,11 +1,57 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from collections import Counter
|
||||
from inspect import signature
|
||||
|
||||
from govoplan_files.backend.router import router
|
||||
from govoplan_files.backend.routes.assets import router as assets_router
|
||||
from govoplan_files.backend.routes.connector_io import router as connector_io_router
|
||||
from govoplan_files.backend.routes.connector_profiles import router as connector_profiles_router
|
||||
from govoplan_files.backend.routes.connector_settings import router as connector_settings_router
|
||||
from govoplan_files.backend.routes.folders import router as folders_router
|
||||
from govoplan_files.backend.routes.listing import router as listing_router
|
||||
from govoplan_files.backend.routes.shares import router as shares_router
|
||||
from govoplan_files.backend.routes.spaces import router as spaces_router
|
||||
from govoplan_files.backend.routes.transfers import router as transfers_router
|
||||
from govoplan_files.backend.routes.uploads import router as uploads_router
|
||||
|
||||
|
||||
class FilesRouterContractTests(unittest.TestCase):
|
||||
@staticmethod
|
||||
def _operation_keys(candidate_router) -> list[tuple[str, str]]:
|
||||
return [
|
||||
(method, route.path)
|
||||
for route in candidate_router.routes
|
||||
for method in sorted(route.methods or ())
|
||||
]
|
||||
|
||||
def test_composed_router_contains_every_workflow_operation_once(self) -> None:
|
||||
workflow_routers = (
|
||||
spaces_router,
|
||||
folders_router,
|
||||
listing_router,
|
||||
uploads_router,
|
||||
connector_settings_router,
|
||||
connector_io_router,
|
||||
connector_profiles_router,
|
||||
assets_router,
|
||||
shares_router,
|
||||
transfers_router,
|
||||
)
|
||||
expected = [
|
||||
operation
|
||||
for workflow_router in workflow_routers
|
||||
for operation in self._operation_keys(workflow_router)
|
||||
]
|
||||
actual = self._operation_keys(router)
|
||||
|
||||
self.assertEqual(expected, actual)
|
||||
self.assertEqual(41, len(actual))
|
||||
self.assertFalse(
|
||||
[operation for operation, count in Counter(actual).items() if count > 1]
|
||||
)
|
||||
|
||||
def test_connector_routes_keep_existing_api_paths(self) -> None:
|
||||
routes = {(tuple(sorted(route.methods or ())), route.path) for route in router.routes}
|
||||
|
||||
@@ -31,6 +77,18 @@ class FilesRouterContractTests(unittest.TestCase):
|
||||
self.assertIn((("POST",), "/files/bulk-rename"), routes)
|
||||
self.assertIn((("POST",), "/files/transfer"), routes)
|
||||
|
||||
def test_file_listing_exposes_structured_property_filters(self) -> None:
|
||||
route = next(
|
||||
route
|
||||
for route in listing_router.routes
|
||||
if route.path == "/files" and "GET" in (route.methods or ())
|
||||
)
|
||||
parameters = signature(route.endpoint).parameters
|
||||
|
||||
self.assertIn("campaign_usage", parameters)
|
||||
self.assertIn("audit_relevant", parameters)
|
||||
self.assertIn("total", route.response_model.model_fields)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
108
tests/test_tenant_summary_batch.py
Normal file
108
tests/test_tenant_summary_batch.py
Normal file
@@ -0,0 +1,108 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.db.models import (
|
||||
FileAsset,
|
||||
FileConnectorCredential,
|
||||
FileConnectorPolicy,
|
||||
FileConnectorProfile,
|
||||
FileConnectorSpace,
|
||||
)
|
||||
from govoplan_files.backend.manifest import _tenant_summary_batch
|
||||
|
||||
|
||||
class FilesTenantSummaryBatchTests(unittest.TestCase):
|
||||
def test_batch_summary_uses_one_grouped_query_per_owned_table(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
Base.metadata.create_all(
|
||||
engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
FileAsset.__table__,
|
||||
FileConnectorCredential.__table__,
|
||||
FileConnectorPolicy.__table__,
|
||||
FileConnectorProfile.__table__,
|
||||
FileConnectorSpace.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
],
|
||||
)
|
||||
try:
|
||||
with Session(engine) as session:
|
||||
session.add_all(
|
||||
[
|
||||
FileAsset(
|
||||
id="file-1",
|
||||
tenant_id="tenant-1",
|
||||
owner_type="tenant",
|
||||
display_path="/one.txt",
|
||||
filename="one.txt",
|
||||
),
|
||||
FileConnectorCredential(
|
||||
id="credential-1",
|
||||
tenant_id="tenant-1",
|
||||
label="Credential",
|
||||
),
|
||||
FileConnectorPolicy(
|
||||
id="policy-1",
|
||||
tenant_id="tenant-1",
|
||||
scope_type="tenant",
|
||||
),
|
||||
FileConnectorProfile(
|
||||
id="profile-1",
|
||||
tenant_id="tenant-1",
|
||||
label="Profile",
|
||||
provider="webdav",
|
||||
),
|
||||
FileConnectorSpace(
|
||||
id="space-1",
|
||||
tenant_id="tenant-1",
|
||||
owner_type="tenant",
|
||||
label="Space",
|
||||
connector_profile_id="profile-1",
|
||||
provider="webdav",
|
||||
),
|
||||
]
|
||||
)
|
||||
session.commit()
|
||||
query_count = 0
|
||||
|
||||
def count_query(*_args: object) -> None:
|
||||
nonlocal query_count
|
||||
query_count += 1
|
||||
|
||||
event.listen(engine, "before_cursor_execute", count_query)
|
||||
try:
|
||||
counts = _tenant_summary_batch(
|
||||
session,
|
||||
["tenant-1", "tenant-empty"],
|
||||
)
|
||||
finally:
|
||||
event.remove(engine, "before_cursor_execute", count_query)
|
||||
|
||||
self.assertEqual(5, query_count)
|
||||
self.assertEqual(
|
||||
{
|
||||
"files": 1,
|
||||
"connector_credentials": 1,
|
||||
"connector_policies": 1,
|
||||
"connector_profiles": 1,
|
||||
"connector_spaces": 1,
|
||||
},
|
||||
counts["tenant-1"],
|
||||
)
|
||||
self.assertEqual(0, counts["tenant-empty"]["files"])
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -14,7 +14,8 @@
|
||||
"./styles/file-manager.css": "./src/styles/file-manager.css"
|
||||
},
|
||||
"scripts": {
|
||||
"test:file-drop-target": "node scripts/test-file-drop-target-structure.mjs"
|
||||
"test:file-drop-target": "node scripts/test-file-drop-target-structure.mjs",
|
||||
"test:file-property-filters": "node scripts/test-file-property-filters-structure.mjs"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@vitejs/plugin-react": "^4.3.4",
|
||||
@@ -22,7 +23,7 @@
|
||||
"typescript": "^5.7.2",
|
||||
"react": "^19.0.0",
|
||||
"react-dom": "^19.0.0",
|
||||
"react-router-dom": "^7.1.1",
|
||||
"react-router-dom": ">=7.18.2 <8",
|
||||
"lucide-react": "^1.23.0",
|
||||
"@govoplan/core-webui": "^0.1.9"
|
||||
},
|
||||
|
||||
36
webui/scripts/test-file-property-filters-structure.mjs
Normal file
36
webui/scripts/test-file-property-filters-structure.mjs
Normal file
@@ -0,0 +1,36 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const page = readFileSync(new URL("../src/features/files/FilesPage.tsx", import.meta.url), "utf8").replace(/\s+/g, " ");
|
||||
const api = readFileSync(new URL("../src/api/files.ts", import.meta.url), "utf8").replace(/\s+/g, " ");
|
||||
|
||||
function assertIncludes(source, fragment, message) {
|
||||
if (!source.includes(fragment.replace(/\s+/g, " "))) throw new Error(message);
|
||||
}
|
||||
|
||||
assertIncludes(
|
||||
api,
|
||||
"campaign_usage?: FileCampaignUsageFilter; audit_relevant?: boolean;",
|
||||
"the Files API must expose structured campaign and audit filters"
|
||||
);
|
||||
assertIncludes(
|
||||
api,
|
||||
"do { const response = await listFiles",
|
||||
"property filtering must traverse every server result page"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
"if (propertyFiltersActive && propertyFilterResults) return propertyFilterResults;",
|
||||
"server-filtered rows must replace, not narrow, the currently loaded explorer window"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
'<option value="unlinked">Not linked or used</option>',
|
||||
"the high-priority unlinked campaign-file filter must remain available"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
"setPropertyFilterTotal(response.total);",
|
||||
"the UI must retain the exact server-side filtered count"
|
||||
);
|
||||
|
||||
console.log("Files property filters cover the complete server result window.");
|
||||
@@ -297,7 +297,8 @@ export type ManagedFile = {
|
||||
shares?: FileShare[];
|
||||
};
|
||||
|
||||
export type FileListResponse = {files: ManagedFile[];cursor?: string | null;next_cursor?: string | null;watermark?: string | null;};
|
||||
export type FileCampaignUsageFilter = "linked" | "unlinked";
|
||||
export type FileListResponse = {files: ManagedFile[];total: number;cursor?: string | null;next_cursor?: string | null;watermark?: string | null;};
|
||||
export type FileDeltaDeletedItem = {id: string;resource_type?: string | null;revision?: string | null;deleted_at?: string | null;};
|
||||
export type FileDeltaResponse = {
|
||||
files: ManagedFile[];
|
||||
@@ -383,15 +384,43 @@ payload: {owner_type: "user" | "group";owner_id: string;path: string;recursive?:
|
||||
return apiFetch<FolderDeleteResponse>(settings, "/api/v1/files/folders/delete", { method: "POST", body: JSON.stringify({ recursive: true, ...payload }) });
|
||||
}
|
||||
|
||||
export function listFiles(settings: ApiSettings, params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;page_size?: number;cursor?: string | null;} = {}): Promise<FileListResponse> {
|
||||
export function listFiles(settings: ApiSettings, params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;campaign_usage?: FileCampaignUsageFilter;audit_relevant?: boolean;page_size?: number;cursor?: string | null;} = {}): Promise<FileListResponse> {
|
||||
const search = new URLSearchParams();
|
||||
for (const [key, value] of Object.entries(params)) {
|
||||
if (value) search.set(key, String(value));
|
||||
if (value !== undefined && value !== null && value !== "") search.set(key, String(value));
|
||||
}
|
||||
const suffix = search.toString() ? `?${search.toString()}` : "";
|
||||
return apiFetch<FileListResponse>(settings, `/api/v1/files${suffix}`);
|
||||
}
|
||||
|
||||
export async function listFilesByProperties(
|
||||
settings: ApiSettings,
|
||||
params: {
|
||||
owner_type: "user" | "group";
|
||||
owner_id: string;
|
||||
path_prefix?: string;
|
||||
campaign_usage?: FileCampaignUsageFilter;
|
||||
audit_relevant?: boolean;
|
||||
page_size?: number;
|
||||
})
|
||||
: Promise<{files: ManagedFile[];total: number;}> {
|
||||
const pageSize = params.page_size ?? DEFAULT_MANAGED_FILE_WINDOW_SIZE;
|
||||
let cursor: string | null | undefined = null;
|
||||
let total = 0;
|
||||
let files: ManagedFile[] = [];
|
||||
do {
|
||||
const response = await listFiles(settings, {
|
||||
...params,
|
||||
page_size: pageSize,
|
||||
cursor
|
||||
});
|
||||
files = files.concat(response.files);
|
||||
total = response.total;
|
||||
cursor = response.next_cursor;
|
||||
} while (cursor);
|
||||
return { files, total };
|
||||
}
|
||||
|
||||
export function listFilesDelta(
|
||||
settings: ApiSettings,
|
||||
params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;since?: string;limit?: number;} = {})
|
||||
|
||||
@@ -14,16 +14,21 @@ import {
|
||||
AdvancedOptionsPanel,
|
||||
LoadingFrame,
|
||||
mergeDeltaRows,
|
||||
ReferenceMultiSelect,
|
||||
SegmentedControl,
|
||||
staticReferenceOptionProvider,
|
||||
StatusBadge,
|
||||
TableActionGroup,
|
||||
ToggleSwitch,
|
||||
useDeltaWatermarks,
|
||||
usePlatformLanguage,
|
||||
useUnsavedDraftGuard,
|
||||
wildcardReferenceOption,
|
||||
type ApiSettings,
|
||||
type ConnectionTreeColumn,
|
||||
type FileConnectorScope,
|
||||
type FileConnectorTargetOption,
|
||||
type ReferenceOption,
|
||||
i18nMessage } from
|
||||
"@govoplan/core-webui";
|
||||
import {
|
||||
@@ -186,6 +191,7 @@ export default function FileConnectorSettingsPanel({
|
||||
const [credentialLoginResult, setCredentialLoginResult] = useState<{ok: boolean;message: string;} | null>(null);
|
||||
const [message, setMessage] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const { translateText } = usePlatformLanguage();
|
||||
const { getDeltaWatermark, setDeltaWatermark, resetDeltaWatermark } = useDeltaWatermarks();
|
||||
|
||||
const requiresTarget = scopeType === "user" || scopeType === "group";
|
||||
@@ -198,6 +204,10 @@ export default function FileConnectorSettingsPanel({
|
||||
const profileDirty = Boolean(draft) && connectorProfileDraftKey(draft) !== savedDraftKey;
|
||||
const credentialDirty = Boolean(credentialDraft) && connectorCredentialDraftKey(credentialDraft) !== savedCredentialDraftKey;
|
||||
const policyDirty = scopeReady && centralPolicyDraftKey(policyDraft) !== savedPolicyDraftKey;
|
||||
const policyConflicts = useMemo(
|
||||
() => policyReferenceConflicts(policyDraft),
|
||||
[policyDraft]
|
||||
);
|
||||
|
||||
useUnsavedDraftGuard({
|
||||
dirty: profileDirty || credentialDirty || policyDirty,
|
||||
@@ -225,6 +235,79 @@ export default function FileConnectorSettingsPanel({
|
||||
() => credentials.filter((credential) => connectorBelongsToEditableScope(credential, scopeType, activeScopeId)),
|
||||
[activeScopeId, credentials, scopeType]
|
||||
);
|
||||
const selectableCredentials = useMemo(
|
||||
() => credentials.filter((credential) =>
|
||||
connectorBelongsToEditableScope(credential, scopeType, activeScopeId) ||
|
||||
credential.source_kind === "credential_envelope" &&
|
||||
reusableCredentialAvailableAtScope(credential, scopeType, activeScopeId)
|
||||
),
|
||||
[activeScopeId, credentials, scopeType]
|
||||
);
|
||||
const connectorReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => profiles.map((profile) => ({
|
||||
value: profile.id,
|
||||
label: profile.label || profile.id,
|
||||
description: [
|
||||
profile.provider,
|
||||
profile.source_path,
|
||||
profile.enabled ? null : "Inactive"
|
||||
].filter(Boolean).join(" · "),
|
||||
kind: "file_connection",
|
||||
availability: profile.enabled ? "available" : "inactive",
|
||||
disabled: !profile.enabled,
|
||||
sourceModule: "files",
|
||||
provenance: {
|
||||
scopeType: profile.scope_type,
|
||||
scopeId: profile.scope_id,
|
||||
sourcePath: profile.source_path
|
||||
}
|
||||
})),
|
||||
[profiles]
|
||||
);
|
||||
const credentialReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => credentials.map((credential) => ({
|
||||
value: credential.id,
|
||||
label: credential.label || credential.id,
|
||||
description: [
|
||||
credential.provider || "shared",
|
||||
credential.source_path,
|
||||
credential.enabled ? null : "Inactive"
|
||||
].filter(Boolean).join(" · "),
|
||||
kind: "file_credential",
|
||||
availability: credential.enabled ? "available" : "inactive",
|
||||
disabled: !credential.enabled,
|
||||
sourceModule: credential.source_kind === "credential_envelope" ? "core" : "files",
|
||||
provenance: {
|
||||
scopeType: credential.scope_type,
|
||||
scopeId: credential.scope_id,
|
||||
sourcePath: credential.source_path,
|
||||
sourceKind: credential.source_kind
|
||||
}
|
||||
})),
|
||||
[credentials]
|
||||
);
|
||||
const providerReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => PROVIDERS.map((provider) => ({
|
||||
value: provider.id,
|
||||
label: translateText(provider.label),
|
||||
description: provider.id,
|
||||
kind: "file_provider",
|
||||
sourceModule: "files"
|
||||
})),
|
||||
[translateText]
|
||||
);
|
||||
const connectorReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(connectorReferenceOptions),
|
||||
[connectorReferenceOptions]
|
||||
);
|
||||
const credentialReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(credentialReferenceOptions),
|
||||
[credentialReferenceOptions]
|
||||
);
|
||||
const providerReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(providerReferenceOptions),
|
||||
[providerReferenceOptions]
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
resetDeltaWatermark(deltaKey);
|
||||
@@ -424,7 +507,7 @@ export default function FileConnectorSettingsPanel({
|
||||
endpoint_url: cleanOrNull(draft.endpointUrl),
|
||||
base_path: cleanOrNull(draft.basePath),
|
||||
enabled: draft.enabled,
|
||||
credential_profile_id: cleanOrNull(credentialProfileIdForDraft(draft, scopedCredentials)),
|
||||
credential_profile_id: cleanOrNull(credentialProfileIdForDraft(draft, selectableCredentials)),
|
||||
credential_mode: draft.credentialMode,
|
||||
capabilities,
|
||||
policy: policyFromDraft(draft),
|
||||
@@ -694,7 +777,7 @@ export default function FileConnectorSettingsPanel({
|
||||
const panelTitle = title ?? i18nMessage("i18n:govoplan-files.value_file_connections", { value0: scopeLabel(scopeType) });
|
||||
const selectedCredentialIds = new Set(scopedProfiles.map((profile) => profile.credential_profile_id).filter((value): value is string => Boolean(value)));
|
||||
const firstCompatibleProfileByCredential = new Map<string, string>();
|
||||
for (const credential of scopedCredentials) {
|
||||
for (const credential of selectableCredentials) {
|
||||
const selectedProfile = scopedProfiles.find((profile) => profile.credential_profile_id === credential.id);
|
||||
const fallbackProfile = selectedProfile ?? scopedProfiles.find((profile) => !selectedCredentialIds.has(credential.id) && credentialMatchesProfile(credential, profile));
|
||||
if (fallbackProfile) firstCompatibleProfileByCredential.set(credential.id, fallbackProfile.id);
|
||||
@@ -717,8 +800,8 @@ export default function FileConnectorSettingsPanel({
|
||||
const credentialTestUnsupported = Boolean(credentialTestProfile && credentialTestProfile.provider !== "webdav" && credentialTestProfile.provider !== "nextcloud");
|
||||
const credentialTestDisabled = !credentialDraft || saving || testingCredentialLogin || !credentialTestProfile || credentialTestUnsupported;
|
||||
const credentialTestTooltip = credentialTestHelpText(credentialTestProfile, credentialTestUnsupported);
|
||||
const profileCredentialOptions = draft ? credentialOptionsForProfileDraft(scopedCredentials, draft) : [];
|
||||
const profileCredentialSelectValue = draft ? credentialProfileIdForDraft(draft, scopedCredentials) : "";
|
||||
const profileCredentialOptions = draft ? credentialOptionsForProfileDraft(selectableCredentials, draft) : [];
|
||||
const profileCredentialSelectValue = draft ? credentialProfileIdForDraft(draft, selectableCredentials) : "";
|
||||
const connectorColumns: ConnectionTreeColumn<ConnectorTreeRow>[] = [
|
||||
{
|
||||
id: "connection",
|
||||
@@ -755,7 +838,7 @@ export default function FileConnectorSettingsPanel({
|
||||
|
||||
function connectorChildren(row: ConnectorTreeRow): ConnectorTreeRow[] {
|
||||
if (row.kind !== "profile") return [];
|
||||
return scopedCredentials.
|
||||
return selectableCredentials.
|
||||
filter((credential) => firstCompatibleProfileByCredential.get(credential.id) === row.profile.id).
|
||||
map((credential) => ({ kind: "credential" as const, id: `credential:${row.profile.id}:${credential.id}`, credential, profile: row.profile }));
|
||||
}
|
||||
@@ -857,22 +940,70 @@ export default function FileConnectorSettingsPanel({
|
||||
<>
|
||||
<div className="form-grid two">
|
||||
<FormField label="i18n:govoplan-files.allowed_connection_ids.0df854c8">
|
||||
<textarea value={policyDraft.allowedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedConnectors: event.target.value })} rows={3} placeholder={"tenant-webdav\nseafile-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedConnectors)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedConnectors: values.join("\n") })}
|
||||
provider={connectorReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file connections"
|
||||
placeholder="Add a connection or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_connection_ids.a95218b4">
|
||||
<textarea value={policyDraft.deniedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedConnectors: event.target.value })} rows={3} placeholder={"legacy-*\nblocked-smb"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedConnectors)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedConnectors: values.join("\n") })}
|
||||
provider={connectorReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file connections"
|
||||
placeholder="Add a connection or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_credential_ids.efcaba2d">
|
||||
<textarea value={policyDraft.allowedCredentials} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedCredentials: event.target.value })} rows={3} placeholder={"tenant-webdav-credentials\nshared-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedCredentials)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedCredentials: values.join("\n") })}
|
||||
provider={credentialReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file credentials"
|
||||
placeholder="Add a credential or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_credential_ids.b6838bc2">
|
||||
<textarea value={policyDraft.deniedCredentials} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedCredentials: event.target.value })} rows={3} placeholder={"personal-*\nblocked-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedCredentials)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedCredentials: values.join("\n") })}
|
||||
provider={credentialReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file credentials"
|
||||
placeholder="Add a credential or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_providers.82a9c23e">
|
||||
<textarea value={policyDraft.allowedProviders} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedProviders: event.target.value })} rows={3} placeholder={"webdav\nnextcloud\nsmb"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedProviders)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedProviders: values.join("\n") })}
|
||||
provider={providerReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file providers"
|
||||
placeholder="Add a provider or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_providers.149b29f4">
|
||||
<textarea value={policyDraft.deniedProviders} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedProviders: event.target.value })} rows={3} placeholder={"generic\nnfs"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedProviders)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedProviders: values.join("\n") })}
|
||||
provider={providerReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file providers"
|
||||
placeholder="Add a provider or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_paths.c953b4be">
|
||||
<textarea value={policyDraft.allowedPaths} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedPaths: event.target.value })} rows={3} placeholder={"GovOPlaN\nreports/*"} />
|
||||
@@ -887,6 +1018,20 @@ export default function FileConnectorSettingsPanel({
|
||||
<textarea value={policyDraft.deniedUrls} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedUrls: event.target.value })} rows={3} placeholder={"http://*\n*://legacy.example.test/*"} />
|
||||
</FormField>
|
||||
</div>
|
||||
{policyConflicts.length ? (
|
||||
<DismissibleAlert
|
||||
tone="warning"
|
||||
resetKey={policyConflicts.join("\u0000")}
|
||||
>
|
||||
Allow and deny rules contain the same values for{" "}
|
||||
{policyConflicts.join("; ")}. Deny rules take precedence.
|
||||
</DismissibleAlert>
|
||||
) : (
|
||||
<p className="muted small-note">
|
||||
When an allow and deny rule both match, the deny rule takes
|
||||
precedence.
|
||||
</p>
|
||||
)}
|
||||
<div className="file-connector-settings-section file-connector-policy-locks">
|
||||
<ToggleSwitch label="i18n:govoplan-files.lower_connection_limits.4f9838bc" checked={policyDraft.allowLowerConnectionLimits} disabled={saving || !canWrite} onChange={(allowLowerConnectionLimits) => patchPolicyDraft({ allowLowerConnectionLimits })} />
|
||||
<ToggleSwitch label="i18n:govoplan-files.lower_credential_limits.ec2b72bc" checked={policyDraft.allowLowerCredentialLimits} disabled={saving || !canWrite} onChange={(allowLowerCredentialLimits) => patchPolicyDraft({ allowLowerCredentialLimits })} />
|
||||
@@ -1432,10 +1577,39 @@ function connectorCredentialDraftKey(draft: ConnectorCredentialDraft): string {
|
||||
function credentialOptionsForProfileDraft(credentials: FileConnectorCredential[], draft: ConnectorProfileDraft): FileConnectorCredential[] {
|
||||
return credentials.filter((credential) =>
|
||||
credential.id === draft.credentialProfileId ||
|
||||
credential.enabled && (!credential.provider || credential.provider === draft.provider)
|
||||
credential.enabled &&
|
||||
(!credential.provider || credential.provider === draft.provider) &&
|
||||
reusableCredentialAllowsProfile(credential, draft.id)
|
||||
);
|
||||
}
|
||||
|
||||
function reusableCredentialAvailableAtScope(
|
||||
credential: FileConnectorCredential,
|
||||
scopeType: ConnectorScope,
|
||||
scopeId: string | null
|
||||
): boolean {
|
||||
if (connectorBelongsToEditableScope(credential, scopeType, scopeId)) return true;
|
||||
if (credential.source_kind !== "credential_envelope") return false;
|
||||
const inherited = credential.metadata?.inherit_to_lower_scopes === true;
|
||||
if (!inherited) return false;
|
||||
if (credential.scope_type === "system") return scopeType !== "system";
|
||||
if (credential.scope_type === "tenant") {
|
||||
return scopeType === "group" || scopeType === "user";
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function reusableCredentialAllowsProfile(
|
||||
credential: FileConnectorCredential,
|
||||
profileId: string
|
||||
): boolean {
|
||||
if (credential.source_kind !== "credential_envelope") return true;
|
||||
const refs = Array.isArray(credential.metadata?.allowed_server_refs)
|
||||
? credential.metadata.allowed_server_refs.filter((value): value is string => typeof value === "string")
|
||||
: [];
|
||||
return refs.length === 0 || refs.includes(`files:${profileId}`);
|
||||
}
|
||||
|
||||
function credentialProfileIdForDraft(draft: ConnectorProfileDraft, credentials: FileConnectorCredential[]): string {
|
||||
const options = credentialOptionsForProfileDraft(credentials, draft);
|
||||
if (draft.credentialProfileId && options.some((credential) => credential.id === draft.credentialProfileId)) {
|
||||
@@ -1656,6 +1830,26 @@ function lines(value: string): string[] {
|
||||
return value.split(/\r?\n/).map((line) => line.trim()).filter(Boolean);
|
||||
}
|
||||
|
||||
function policyReferenceConflicts(
|
||||
draft: CentralConnectorPolicyDraft
|
||||
): string[] {
|
||||
return [
|
||||
["connections", draft.allowedConnectors, draft.deniedConnectors],
|
||||
["credentials", draft.allowedCredentials, draft.deniedCredentials],
|
||||
["providers", draft.allowedProviders, draft.deniedProviders],
|
||||
["paths", draft.allowedPaths, draft.deniedPaths],
|
||||
["endpoint URLs", draft.allowedUrls, draft.deniedUrls]
|
||||
].flatMap(([label, allowed, denied]) => {
|
||||
const deniedValues = new Set(
|
||||
lines(denied).map((value) => value.toLocaleLowerCase())
|
||||
);
|
||||
const overlaps = lines(allowed).filter((value) =>
|
||||
deniedValues.has(value.toLocaleLowerCase())
|
||||
);
|
||||
return overlaps.length ? [`${label}: ${overlaps.join(", ")}`] : [];
|
||||
});
|
||||
}
|
||||
|
||||
function policyRuleList(policy: Record<string, unknown>, kind: "allow" | "deny", field: "connectors" | "providers" | "credentials" | "external_paths" | "external_urls"): string[] {
|
||||
const rule = recordValue(policy[kind]) ??
|
||||
recordValue(policy[kind === "allow" ? "allowlist" : "denylist"]) ??
|
||||
|
||||
98
webui/src/features/files/FileSpacesWidget.tsx
Normal file
98
webui/src/features/files/FileSpacesWidget.tsx
Normal file
@@ -0,0 +1,98 @@
|
||||
import { useCallback } from "react";
|
||||
import { Folder, Network } from "lucide-react";
|
||||
import { Link } from "react-router-dom";
|
||||
import {
|
||||
DashboardWidgetList,
|
||||
DismissibleAlert,
|
||||
LoadingFrame,
|
||||
StatusBadge,
|
||||
useDashboardWidgetData,
|
||||
type ApiSettings,
|
||||
type DashboardWidgetConfiguration
|
||||
} from "@govoplan/core-webui";
|
||||
import {
|
||||
listFileSpaces,
|
||||
type FileSpace
|
||||
} from "../../api/files";
|
||||
|
||||
export default function FileSpacesWidget({
|
||||
settings,
|
||||
refreshKey,
|
||||
configuration
|
||||
}: {
|
||||
settings: ApiSettings;
|
||||
refreshKey: number;
|
||||
configuration: DashboardWidgetConfiguration;
|
||||
}) {
|
||||
const maxItems = numberSetting(configuration.maxItems, 6, 1, 16);
|
||||
const includeConnectors = configuration.includeConnectors !== false;
|
||||
const load = useCallback(async () => {
|
||||
const response = await listFileSpaces(settings);
|
||||
return response.spaces
|
||||
.filter(
|
||||
(space) =>
|
||||
includeConnectors || (space.space_type ?? "managed") === "managed"
|
||||
)
|
||||
.sort((left, right) => left.label.localeCompare(right.label))
|
||||
.slice(0, maxItems);
|
||||
}, [includeConnectors, maxItems, settings]);
|
||||
const { data: spaces, loading, error } = useDashboardWidgetData(
|
||||
load,
|
||||
refreshKey
|
||||
);
|
||||
|
||||
return (
|
||||
<LoadingFrame loading={loading} label="Loading file spaces">
|
||||
{error && (
|
||||
<DismissibleAlert tone="warning" resetKey={error}>
|
||||
{error}
|
||||
</DismissibleAlert>
|
||||
)}
|
||||
<DashboardWidgetList
|
||||
emptyText="No file spaces are available."
|
||||
items={(spaces ?? []).map((space) => ({
|
||||
id: space.id,
|
||||
title: space.label,
|
||||
detail: spaceDescription(space),
|
||||
meta: space.owner_type === "group" ? "Group" : "Personal",
|
||||
leading:
|
||||
space.space_type === "connector" ? (
|
||||
<Network size={17} aria-hidden="true" />
|
||||
) : (
|
||||
<Folder size={17} aria-hidden="true" />
|
||||
),
|
||||
trailing: space.read_only ? (
|
||||
<StatusBadge status="locked" label="Read only" />
|
||||
) : undefined,
|
||||
to: "/files"
|
||||
}))}
|
||||
/>
|
||||
<div className="dashboard-contribution-footer">
|
||||
<Link className="btn btn-secondary" to="/files">
|
||||
Open files
|
||||
</Link>
|
||||
</div>
|
||||
</LoadingFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function spaceDescription(space: FileSpace): string {
|
||||
if (space.space_type !== "connector") {
|
||||
return space.description || "Managed storage";
|
||||
}
|
||||
return [space.provider, space.library_id, space.remote_path]
|
||||
.filter(Boolean)
|
||||
.join(" · ") || "Connected storage";
|
||||
}
|
||||
|
||||
function numberSetting(
|
||||
value: unknown,
|
||||
fallback: number,
|
||||
minimum: number,
|
||||
maximum: number
|
||||
): number {
|
||||
const numeric = typeof value === "number" ? value : Number(value);
|
||||
return Number.isFinite(numeric)
|
||||
? Math.max(minimum, Math.min(maximum, Math.floor(numeric)))
|
||||
: fallback;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { useEffect, useMemo, useRef, useState, type DragEvent as ReactDragEvent, type KeyboardEvent as ReactKeyboardEvent, type MouseEvent as ReactMouseEvent } from "react";
|
||||
import { ArrowUp, ChevronRight, Copy, Download, File, Folder, Home, KeyRound, Link2, MoveRight, Plus, RefreshCw, Search, Trash2, UploadCloud } from "lucide-react";
|
||||
import { ArrowUp, ChevronRight, Copy, Download, File, Folder, Home, KeyRound, Link2, ListFilter, MoveRight, Plus, RefreshCw, Search, Trash2, UploadCloud } from "lucide-react";
|
||||
import {
|
||||
Button,
|
||||
ConfirmDialog,
|
||||
@@ -25,6 +25,7 @@ import {
|
||||
downloadFilesAsZip,
|
||||
fetchResourceAccessExplanation,
|
||||
listFilesDelta,
|
||||
listFilesByProperties,
|
||||
listFileConnectorProfiles,
|
||||
listFileSpaces,
|
||||
listManagedFileSnapshot,
|
||||
@@ -38,6 +39,7 @@ import {
|
||||
type FileConnectorBrowseItem,
|
||||
type FileConnectorProfile,
|
||||
type FileDeltaResponse,
|
||||
type FileCampaignUsageFilter,
|
||||
type FileFolder,
|
||||
type FileSpace,
|
||||
type ManagedFile,
|
||||
@@ -88,6 +90,7 @@ import { useFileDialogs } from "./hooks/useFileDialogs";
|
||||
import { useFileDragDropState } from "./hooks/useFileDragDropState";
|
||||
|
||||
type UploadPhase = "idle" | "uploading" | "unpacking" | "finalizing";
|
||||
type AuditRelevantFilter = "" | "true" | "false";
|
||||
type FileAccessExplanationTarget = {
|
||||
resourceType: "file" | "folder";
|
||||
resourceId: string;
|
||||
@@ -118,6 +121,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const [searchCaseSensitive, setSearchCaseSensitive] = useState(false);
|
||||
const [searchActive, setSearchActive] = useState(false);
|
||||
const [searchResults, setSearchResults] = useState<ManagedFile[] | null>(null);
|
||||
const [campaignUsageFilter, setCampaignUsageFilter] = useState<"" | FileCampaignUsageFilter>("");
|
||||
const [auditRelevantFilter, setAuditRelevantFilter] = useState<AuditRelevantFilter>("");
|
||||
const [propertyFiltersActive, setPropertyFiltersActive] = useState(false);
|
||||
const [propertyFilterResults, setPropertyFilterResults] = useState<ManagedFile[] | null>(null);
|
||||
const [propertyFilterTotal, setPropertyFilterTotal] = useState<number | null>(null);
|
||||
const [sortColumn, setSortColumn] = useState<SortColumn>("name");
|
||||
const [sortDirection, setSortDirection] = useState<SortDirection>("asc");
|
||||
const [unmatchedCount, setUnmatchedCount] = useState<number | null>(null);
|
||||
@@ -176,11 +184,19 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const [renameSuffix, setRenameSuffix] = useState("");
|
||||
const [renameRecursive, setRenameRecursive] = useState(false);
|
||||
const [renamePreview, setRenamePreview] = useState<RenameResponse | null>(null);
|
||||
const visibleFiles = searchActive && searchResults ? searchResults : files;
|
||||
const visibleFiles = useMemo(() => {
|
||||
if (searchActive && searchResults && propertyFiltersActive && propertyFilterResults) {
|
||||
const ids = new Set(propertyFilterResults.map((file) => file.id));
|
||||
return searchResults.filter((file) => ids.has(file.id));
|
||||
}
|
||||
if (propertyFiltersActive && propertyFilterResults) return propertyFilterResults;
|
||||
if (searchActive && searchResults) return searchResults;
|
||||
return files;
|
||||
}, [files, propertyFilterResults, propertyFiltersActive, searchActive, searchResults]);
|
||||
const explorerEntries = useMemo(() => {
|
||||
const entries = buildExplorerEntries(visibleFiles, folders, currentFolder, searchActive);
|
||||
const entries = buildExplorerEntries(visibleFiles, folders, currentFolder, searchActive || propertyFiltersActive);
|
||||
return sortExplorerEntries(entries, sortColumn, sortDirection);
|
||||
}, [visibleFiles, folders, currentFolder, searchActive, sortColumn, sortDirection]);
|
||||
}, [visibleFiles, folders, currentFolder, searchActive, propertyFiltersActive, sortColumn, sortDirection]);
|
||||
const fileListViewportRef = useRef<HTMLDivElement | null>(null);
|
||||
const fileListMeasureRowRef = useRef<HTMLDivElement | null>(null);
|
||||
const managedSpaceLoadsInFlightRef = useRef<Set<string>>(new Set());
|
||||
@@ -285,7 +301,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
if (!viewport) return;
|
||||
viewport.scrollTop = 0;
|
||||
setFileListScrollTop(0);
|
||||
}, [activeSpaceId, currentFolder, searchActive, searchResults, sortColumn, sortDirection]);
|
||||
}, [activeSpaceId, currentFolder, propertyFiltersActive, propertyFilterResults, searchActive, searchResults, sortColumn, sortDirection]);
|
||||
|
||||
async function loadSpaces() {
|
||||
setBusy(true);
|
||||
@@ -373,6 +389,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchActive(false);
|
||||
setSearchPattern("");
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
}
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
@@ -410,6 +431,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchPattern("");
|
||||
setSearchActive(false);
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
setConnectorSpaceSelectedItem(null);
|
||||
@@ -431,6 +457,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchActive(false);
|
||||
setSearchPattern("");
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
}
|
||||
setUnmatchedCount(null);
|
||||
setRenamePreview(null);
|
||||
@@ -1001,6 +1032,50 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSelectedFolderPaths(new Set());
|
||||
}
|
||||
|
||||
async function runPropertyFilters(
|
||||
campaignUsage: "" | FileCampaignUsageFilter,
|
||||
auditRelevant: AuditRelevantFilter
|
||||
) {
|
||||
if (!activeSpace || activeSpaceIsConnector) return;
|
||||
if (!campaignUsage && !auditRelevant) {
|
||||
clearPropertyFilters();
|
||||
return;
|
||||
}
|
||||
setBusy(true);
|
||||
setError("");
|
||||
setMessage("");
|
||||
try {
|
||||
const response = await listFilesByProperties(settings, {
|
||||
owner_type: activeSpace.owner_type,
|
||||
owner_id: activeSpace.owner_id,
|
||||
path_prefix: currentFolder,
|
||||
campaign_usage: campaignUsage || undefined,
|
||||
audit_relevant: auditRelevant ? auditRelevant === "true" : undefined
|
||||
});
|
||||
setCampaignUsageFilter(campaignUsage);
|
||||
setAuditRelevantFilter(auditRelevant);
|
||||
setPropertyFilterResults(response.files);
|
||||
setPropertyFilterTotal(response.total);
|
||||
setPropertyFiltersActive(true);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
function clearPropertyFilters() {
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
}
|
||||
|
||||
function deleteSelected(
|
||||
fileIds: Set<string> = selectedFileIds,
|
||||
folderPaths: Set<string> = selectedFolderPaths,
|
||||
@@ -2067,6 +2142,15 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
disabled={!activeSpace || activeSpaceIsConnector}
|
||||
onSearch={(patternValue, caseSensitiveValue) => void runPatternSearch(patternValue, caseSensitiveValue)}
|
||||
onClear={() => void clearSearch()} />
|
||||
<FilePropertyFilterRow
|
||||
campaignUsage={campaignUsageFilter}
|
||||
auditRelevant={auditRelevantFilter}
|
||||
busy={busy}
|
||||
active={propertyFiltersActive}
|
||||
disabled={!activeSpace || activeSpaceIsConnector}
|
||||
total={propertyFilterTotal}
|
||||
onApply={(campaignUsage, auditRelevant) => void runPropertyFilters(campaignUsage, auditRelevant)}
|
||||
onClear={clearPropertyFilters} />
|
||||
|
||||
<div className="file-list-meta">
|
||||
<span>{activeSpaceIsConnector ? connectorSpaceSelectedItem?.kind === "file" ? connectorSpaceSelectedItem.name : "i18n:govoplan-files.remote_connector_space.d8956863" : selectedSummary}</span>
|
||||
@@ -2509,6 +2593,72 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
|
||||
}
|
||||
|
||||
function FilePropertyFilterRow({
|
||||
campaignUsage,
|
||||
auditRelevant,
|
||||
busy,
|
||||
active,
|
||||
disabled,
|
||||
total,
|
||||
onApply,
|
||||
onClear
|
||||
}: {
|
||||
campaignUsage: "" | FileCampaignUsageFilter;
|
||||
auditRelevant: AuditRelevantFilter;
|
||||
busy: boolean;
|
||||
active: boolean;
|
||||
disabled: boolean;
|
||||
total: number | null;
|
||||
onApply: (campaignUsage: "" | FileCampaignUsageFilter, auditRelevant: AuditRelevantFilter) => void;
|
||||
onClear: () => void;
|
||||
}) {
|
||||
const [campaignUsageDraft, setCampaignUsageDraft] = useState(campaignUsage);
|
||||
const [auditRelevantDraft, setAuditRelevantDraft] = useState(auditRelevant);
|
||||
|
||||
useEffect(() => {
|
||||
setCampaignUsageDraft(campaignUsage);
|
||||
}, [campaignUsage]);
|
||||
|
||||
useEffect(() => {
|
||||
setAuditRelevantDraft(auditRelevant);
|
||||
}, [auditRelevant]);
|
||||
|
||||
return (
|
||||
<div className="file-property-filter-row">
|
||||
<ListFilter size={17} aria-hidden="true" />
|
||||
<label className="file-property-filter-field">
|
||||
<span>Campaign use</span>
|
||||
<select
|
||||
value={campaignUsageDraft}
|
||||
disabled={busy || disabled}
|
||||
onChange={(event) => setCampaignUsageDraft(event.target.value as "" | FileCampaignUsageFilter)}>
|
||||
<option value="">Any</option>
|
||||
<option value="linked">Linked or used</option>
|
||||
<option value="unlinked">Not linked or used</option>
|
||||
</select>
|
||||
</label>
|
||||
<label className="file-property-filter-field">
|
||||
<span>Audit evidence</span>
|
||||
<select
|
||||
value={auditRelevantDraft}
|
||||
disabled={busy || disabled}
|
||||
onChange={(event) => setAuditRelevantDraft(event.target.value as AuditRelevantFilter)}>
|
||||
<option value="">Any</option>
|
||||
<option value="true">Audit relevant</option>
|
||||
<option value="false">Not audit relevant</option>
|
||||
</select>
|
||||
</label>
|
||||
<Button
|
||||
onClick={() => onApply(campaignUsageDraft, auditRelevantDraft)}
|
||||
disabled={busy || disabled || !campaignUsageDraft && !auditRelevantDraft}>
|
||||
Apply filters
|
||||
</Button>
|
||||
{active && <Button onClick={onClear} disabled={busy}>Clear filters</Button>}
|
||||
{active && total !== null && <span className="file-property-filter-count">{total} matching file{total === 1 ? "" : "s"}</span>}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function FileSearchRow({
|
||||
value,
|
||||
caseSensitive,
|
||||
|
||||
@@ -1,8 +1,16 @@
|
||||
import { createElement, lazy } from "react";
|
||||
import { hasScope, type AdminSectionsUiCapability, type FilesConnectorsUiCapability, type FilesFileExplorerUiCapability, type PlatformWebModule } from "@govoplan/core-webui";
|
||||
import {
|
||||
hasScope,
|
||||
type AdminSectionsUiCapability,
|
||||
type DashboardWidgetsUiCapability,
|
||||
type FilesConnectorsUiCapability,
|
||||
type FilesFileExplorerUiCapability,
|
||||
type PlatformWebModule
|
||||
} from "@govoplan/core-webui";
|
||||
import { FolderTree } from "./features/files/components/FileManagerComponents";
|
||||
import FileConnectorSettingsPanel from "./features/files/FileConnectorSettingsPanel";
|
||||
import ManagedFileChooser from "./features/files/components/ManagedFileChooser";
|
||||
import FileSpacesWidget from "./features/files/FileSpacesWidget";
|
||||
import { listFileSpaces, listFiles, listFilesDelta, listFolders, resolveFilePatterns, shareFilesWithTarget } from "./api/files";
|
||||
import { generatedTranslations } from "./i18n/generatedTranslations";
|
||||
import "./styles/file-manager.css";
|
||||
@@ -14,10 +22,55 @@ const translations = {
|
||||
en: generatedTranslations.en,
|
||||
de: generatedTranslations.de
|
||||
};
|
||||
const fileDashboardWidgets: DashboardWidgetsUiCapability = {
|
||||
widgets: [
|
||||
{
|
||||
id: "files.spaces",
|
||||
surfaceId: "files.widget.spaces",
|
||||
title: "File spaces",
|
||||
description: "Managed and connected file spaces available to you.",
|
||||
moduleId: "files",
|
||||
category: "Content",
|
||||
order: 35,
|
||||
defaultVisible: false,
|
||||
defaultSize: "medium",
|
||||
supportedSizes: ["medium", "wide"],
|
||||
anyOf: fileRead,
|
||||
refreshIntervalMs: 60_000,
|
||||
defaultConfiguration: {
|
||||
maxItems: 6,
|
||||
includeConnectors: true
|
||||
},
|
||||
configurationFields: [
|
||||
{
|
||||
id: "maxItems",
|
||||
label: "Maximum spaces",
|
||||
kind: "number",
|
||||
min: 1,
|
||||
max: 16,
|
||||
step: 1,
|
||||
required: true
|
||||
},
|
||||
{
|
||||
id: "includeConnectors",
|
||||
label: "Include connected storage",
|
||||
kind: "boolean"
|
||||
}
|
||||
],
|
||||
render: ({ settings, refreshKey, configuration }) =>
|
||||
createElement(FileSpacesWidget, {
|
||||
settings,
|
||||
refreshKey,
|
||||
configuration
|
||||
})
|
||||
}
|
||||
]
|
||||
};
|
||||
const fileConnectorAdminSections: AdminSectionsUiCapability = {
|
||||
sections: [
|
||||
{
|
||||
id: "system-file-connectors",
|
||||
surfaceId: "files.admin.system-connectors",
|
||||
label: "i18n:govoplan-files.file_connections.1e362326",
|
||||
group: "SYSTEM",
|
||||
order: 75,
|
||||
@@ -30,6 +83,7 @@ const fileConnectorAdminSections: AdminSectionsUiCapability = {
|
||||
},
|
||||
{
|
||||
id: "tenant-file-connectors",
|
||||
surfaceId: "files.admin.tenant-connectors",
|
||||
label: "i18n:govoplan-files.file_connections.1e362326",
|
||||
group: "TENANT",
|
||||
order: 65,
|
||||
@@ -49,6 +103,14 @@ export const filesModule: PlatformWebModule = {
|
||||
version: "1.0.0",
|
||||
dependencies: ["access"],
|
||||
translations,
|
||||
viewSurfaces: [
|
||||
{ id: "files.admin.system-connectors", moduleId: "files", kind: "section", label: "System file connections", order: 75 },
|
||||
{ id: "files.admin.tenant-connectors", moduleId: "files", kind: "section", label: "Tenant file connections", order: 65 },
|
||||
{ id: "files.admin.group-connectors", moduleId: "files", kind: "section", label: "Group file connections", order: 65 },
|
||||
{ id: "files.admin.user-connectors", moduleId: "files", kind: "section", label: "User file connections", order: 65 },
|
||||
{ id: "files.settings.connectors", moduleId: "files", kind: "section", label: "Personal file connections", order: 20 },
|
||||
{ id: "files.widget.spaces", moduleId: "files", kind: "section", label: "File spaces widget", order: 35 }
|
||||
],
|
||||
navItems: [{ to: "/files", label: "i18n:govoplan-files.files.6ce6c512", iconName: "folder", anyOf: fileRead, order: 40 }],
|
||||
routes: [
|
||||
{ path: "/files", anyOf: fileRead, order: 40, render: ({ settings, auth }) => createElement(FilesPage, { settings, auth }) }],
|
||||
@@ -67,7 +129,8 @@ export const filesModule: PlatformWebModule = {
|
||||
"files.connectors": {
|
||||
FileConnectorScopeManager: FileConnectorSettingsPanel
|
||||
} satisfies FilesConnectorsUiCapability,
|
||||
"admin.sections": fileConnectorAdminSections
|
||||
"admin.sections": fileConnectorAdminSections,
|
||||
"dashboard.widgets": fileDashboardWidgets
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -11,6 +11,31 @@
|
||||
padding: 4px 0 10px 14px;
|
||||
}
|
||||
|
||||
.file-property-filter-row {
|
||||
display: flex;
|
||||
align-items: end;
|
||||
gap: 12px;
|
||||
min-width: 0;
|
||||
padding: 0 0 10px 14px;
|
||||
}
|
||||
|
||||
.file-property-filter-field {
|
||||
display: grid;
|
||||
gap: 4px;
|
||||
min-width: 180px;
|
||||
}
|
||||
|
||||
.file-property-filter-field > span,
|
||||
.file-property-filter-count {
|
||||
color: var(--muted);
|
||||
font-size: var(--font-size-sm);
|
||||
}
|
||||
|
||||
.file-property-filter-count {
|
||||
align-self: center;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.file-list-drop-target.is-active,
|
||||
.file-list-drop-target.is-drop-target {
|
||||
background: var(--line);
|
||||
@@ -565,6 +590,15 @@
|
||||
.file-search-row {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.file-property-filter-row {
|
||||
align-items: stretch;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.file-property-filter-count {
|
||||
margin-left: 0;
|
||||
}
|
||||
}
|
||||
|
||||
.file-conflict-summary {
|
||||
|
||||
Reference in New Issue
Block a user