Compare commits
8 Commits
2b34f6e305
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 835eacfc5d | |||
| 85606d5580 | |||
| 632cc6cf7d | |||
| 86a905a3a7 | |||
| 5b868272b9 | |||
| 93eea839c8 | |||
| b667e7ff0e | |||
| 0fd8f6e02a |
@@ -187,9 +187,10 @@ The managed-file flow is:
|
||||
8. Connector-originated operations also emit their connector audit evidence.
|
||||
|
||||
Blob storage is not part of the database transaction. An object may therefore
|
||||
be left without committed metadata after a process or database failure. No
|
||||
automatic orphan reconciliation exists yet; operators must account for this in
|
||||
integrity checks and retention plans.
|
||||
be left without committed metadata after a process or database failure. The
|
||||
operator integrity API scans database blobs and the tenant storage prefix in
|
||||
bounded, resumable phases. It reports orphan objects before any cleanup and
|
||||
never deletes them as part of a scan.
|
||||
|
||||
### Governed connector import
|
||||
|
||||
@@ -463,9 +464,30 @@ include:
|
||||
- deployment-owned connector profile files, referenced CA bundles, and secret
|
||||
environment configuration where those definitions are in use.
|
||||
|
||||
There is no Files backup/restore API and no automated blob integrity or orphan
|
||||
reconciliation job. Use a write quiesce or coordinated snapshots so database
|
||||
references and objects represent the same recovery point.
|
||||
There is no Files backup/restore API. Use a write quiesce or coordinated
|
||||
snapshots so database references and objects represent the same recovery point.
|
||||
The integrity API verifies a restored set, but it does not replace a coordinated
|
||||
backup.
|
||||
|
||||
Create a scan with `POST /api/v1/files/integrity/scans`, then call
|
||||
`POST /api/v1/files/integrity/scans/{scan_id}/run` until it reports
|
||||
`completed`. Each call advances at most the persisted batch size, so a stopped
|
||||
operator or worker can resume from the committed blob/object cursors.
|
||||
|
||||
Findings distinguish:
|
||||
|
||||
- `missing`: metadata references an absent object;
|
||||
- `size_mismatch` or `checksum_mismatch`: bytes do not match immutable blob
|
||||
metadata and the blob is quarantined;
|
||||
- `orphan_object`: an object exists in the tenant Files prefix without a
|
||||
corresponding blob row.
|
||||
|
||||
Missing or corrupt blobs fail closed for ordinary downloads and Campaign
|
||||
attachment materialization. After restoring the expected bytes, use the finding
|
||||
`recheck` action first in dry-run mode and then apply it. Orphan cleanup is also
|
||||
dry-run by default, rechecks that no database reference exists, remains scoped
|
||||
to the scanned tenant prefix, and is idempotent. Both applied and dry-run
|
||||
actions emit audit evidence.
|
||||
|
||||
After restore:
|
||||
|
||||
@@ -493,9 +515,10 @@ Legacy external secret references are detached and identified as non-owned;
|
||||
Files never calls a provider delete operation for them.
|
||||
|
||||
The retirement executor drops database tables but does not delete corresponding
|
||||
objects from the configured blob backend. Operators must include those orphaned
|
||||
objects in the approved retention/destruction plan. Validate the installer
|
||||
snapshot and independent blob backup before retirement.
|
||||
objects from the configured blob backend. Operators must include those objects
|
||||
in the approved retention/destruction plan, report them with an integrity scan,
|
||||
and explicitly approve cleanup. Validate the installer snapshot and independent
|
||||
blob backup before retirement.
|
||||
|
||||
### Operational signals
|
||||
|
||||
@@ -657,8 +680,9 @@ delete against an already scrubbed tombstone does not recreate secret evidence.
|
||||
File versions and blobs are effectively retained indefinitely today. Although a
|
||||
blob has `ref_count` and `retained_until` fields, no complete retention-policy,
|
||||
legal-hold, hard-purge, or garbage-collection service enforces them. There is
|
||||
also no supported user restore endpoint for soft-deleted assets/folders and no
|
||||
automated orphan-object reconciliation.
|
||||
also no supported user restore endpoint for soft-deleted assets/folders.
|
||||
Integrity reconciliation is operator-triggered and is not a retention or
|
||||
automatic garbage-collection policy.
|
||||
|
||||
Do not promise erasure, timed retention, legal hold, or self-service recovery
|
||||
from the current soft-delete behavior. Those require an explicit, auditable
|
||||
@@ -735,14 +759,14 @@ returning different content or credentials.
|
||||
|
||||
| Area | Implemented now | Planned or explicitly outside the current boundary |
|
||||
| --- | --- | --- |
|
||||
| Managed storage | Local durable-root backend, fallback read roots, tenant blob deduplication, checksums | Operational S3 after pinned SDK transport ([#34](https://git.add-ideas.de/add-ideas/govoplan-files/issues/34)); automated integrity/orphan reconciliation ([#36](https://git.add-ideas.de/add-ideas/govoplan-files/issues/36)) |
|
||||
| Managed storage | Local durable-root backend, fallback read roots, tenant blob deduplication, checksums, bounded resumable integrity scans, quarantine, and dry-run-first orphan cleanup | Operational S3 after pinned SDK transport ([#34](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/34)); scheduled scan execution |
|
||||
| Upload | Bounded direct upload, drag-and-drop UI, ZIP spool/extract limits, explicit conflicts | Malware scanning, quotas, type policy, resumable/chunked upload |
|
||||
| Organization | Folders, bulk rename preview/apply, move/copy, drag-and-drop, ZIP download, pattern resolution | General file-history UI and user-driven append-version/restore |
|
||||
| Sharing | User/group/tenant/campaign grant or permission update through API; campaign linkage display | Share revocation/expiry and complete general share-management UI ([#37](https://git.add-ideas.de/add-ideas/govoplan-files/issues/37)) |
|
||||
| Deletion/retention | Soft-delete assets/folders/spaces; immediate audited connector-secret scrubbing | File restore API, hard purge, retention policy, legal hold, and blob GC ([#38](https://git.add-ideas.de/add-ideas/govoplan-files/issues/38)) |
|
||||
| Sharing | User/group/tenant/campaign grants, expiry, idempotent revocation, searchable share-management UI, and campaign linkage display | Richer policy-driven share lifecycles |
|
||||
| Deletion/retention | Soft-delete assets/folders/spaces; immediate audited connector-secret scrubbing | File restore API, hard purge, retention policy, legal hold, and blob GC ([#38](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/38)) |
|
||||
| Connector governance | Scoped profiles/credentials/policies, effective source explanation, separate credentials, linked user/group spaces | Provider-owned external secret lifecycle; API `secret_ref` remains rejected |
|
||||
| HTTP connectors | Pinned, bounded, no-redirect Seafile and WebDAV/Nextcloud browse/import/manual sync | Background/folder sync, remote mutation, long-running transfer workers |
|
||||
| SMB and S3 connectors | Provider descriptors and browse/import logic | Live access only after SDK connections plus DFS referrals/redirects are validated and pinned ([SMB #35](https://git.add-ideas.de/add-ideas/govoplan-files/issues/35), [S3 #34](https://git.add-ideas.de/add-ideas/govoplan-files/issues/34)) |
|
||||
| SMB and S3 connectors | Provider descriptors and browse/import logic | Live access only after SDK connections plus DFS referrals/redirects are validated and pinned ([SMB #35](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/35), [S3 #34](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/34)) |
|
||||
| Other providers | Reserved SharePoint/OneDrive keys and NFS/local descriptors | Graph/OAuth/provider paging, NFS deployment integration, DMS connectors |
|
||||
| Connector spaces | User/group link, browse, manual selected-file sync, edit/disable/delete | Background sync, remote writes/deletes, full conflict-reporting jobs |
|
||||
| Profile capabilities | Stored and displayed | Enforce capability flags as an independent operation gate |
|
||||
|
||||
@@ -22,6 +22,7 @@ from govoplan_files.backend.storage.campaign_attachments import (
|
||||
from govoplan_files.backend.storage.campaign_usage import record_campaign_attachment_uses_for_jobs
|
||||
from govoplan_files.backend.storage.files import current_version_and_blob
|
||||
from govoplan_files.backend.storage.paths import normalize_folder
|
||||
from govoplan_files.backend.storage.share_state import effective_file_share_clause
|
||||
|
||||
|
||||
VIRTUAL_FOLDER_RESOURCE_PREFIX = "virtual-folder:v1"
|
||||
@@ -96,7 +97,7 @@ class FilesAccessService(FileAccessProvider):
|
||||
.filter(
|
||||
FileShare.tenant_id == asset.tenant_id,
|
||||
FileShare.file_asset_id == asset.id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
effective_file_share_clause(),
|
||||
FileShare.permission.in_(sorted(permission_values)),
|
||||
or_(
|
||||
(FileShare.target_type == "user") & (FileShare.target_id == principal.membership_id),
|
||||
|
||||
@@ -117,7 +117,15 @@ def _record_share_visibility_change(session: OrmSession, share: FileShare) -> No
|
||||
return
|
||||
if not state.pending and not has_attr_changes(
|
||||
state,
|
||||
("file_asset_id", "target_type", "target_id", "permission", "revoked_at"),
|
||||
(
|
||||
"file_asset_id",
|
||||
"target_type",
|
||||
"target_id",
|
||||
"permission",
|
||||
"expires_at",
|
||||
"revoked_at",
|
||||
"revoked_by_user_id",
|
||||
),
|
||||
):
|
||||
return
|
||||
_ensure_id(share)
|
||||
@@ -136,7 +144,9 @@ def _record_share_visibility_change(session: OrmSession, share: FileShare) -> No
|
||||
"share_target_type": share.target_type,
|
||||
"share_target_id": share.target_id,
|
||||
"share_permission": share.permission,
|
||||
"share_expires_at": _isoformat(share.expires_at),
|
||||
"share_revoked_at": _isoformat(share.revoked_at),
|
||||
"share_revoked_by_user_id": share.revoked_by_user_id,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@@ -28,6 +28,55 @@ class FileBlob(Base, TimestampMixin):
|
||||
content_type: Mapped[str | None] = mapped_column(String(255))
|
||||
ref_count: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
|
||||
retained_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
integrity_status: Mapped[str] = mapped_column(String(30), default="unchecked", nullable=False, index=True)
|
||||
integrity_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
|
||||
integrity_failure: Mapped[str | None] = mapped_column(String(100), nullable=True)
|
||||
quarantined_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
|
||||
|
||||
|
||||
class FileIntegrityScan(Base, TimestampMixin):
|
||||
__tablename__ = "file_integrity_scans"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||
storage_backend: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||
storage_prefix: Mapped[str] = mapped_column(String(1000), nullable=False)
|
||||
status: Mapped[str] = mapped_column(String(30), default="pending", nullable=False, index=True)
|
||||
phase: Mapped[str] = mapped_column(String(30), default="blobs", nullable=False)
|
||||
verify_checksums: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
|
||||
batch_size: Mapped[int] = mapped_column(Integer, default=100, nullable=False)
|
||||
blob_cursor: Mapped[str | None] = mapped_column(String(36), nullable=True)
|
||||
object_cursor: Mapped[str | None] = mapped_column(String(1000), nullable=True)
|
||||
scanned_blob_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
verified_blob_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
quarantined_blob_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
scanned_object_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
orphan_object_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
last_error: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
|
||||
|
||||
class FileIntegrityFinding(Base, TimestampMixin):
|
||||
__tablename__ = "file_integrity_findings"
|
||||
__table_args__ = (
|
||||
Index("ix_file_integrity_findings_scan_state", "scan_id", "state"),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
|
||||
scan_id: Mapped[str] = mapped_column(ForeignKey("file_integrity_scans.id", ondelete="CASCADE"), nullable=False, index=True)
|
||||
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||
kind: Mapped[str] = mapped_column(String(40), nullable=False, index=True)
|
||||
state: Mapped[str] = mapped_column(String(30), default="open", nullable=False, index=True)
|
||||
blob_id: Mapped[str | None] = mapped_column(ForeignKey("file_blobs.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||
storage_key: Mapped[str] = mapped_column(String(1000), nullable=False)
|
||||
expected_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
observed_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
expected_checksum_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
observed_checksum_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
resolved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
resolved_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||
|
||||
|
||||
class FileFolder(Base, TimestampMixin):
|
||||
@@ -105,7 +154,9 @@ class FileShare(Base, TimestampMixin):
|
||||
target_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
|
||||
permission: Mapped[str] = mapped_column(String(20), default="read", nullable=False)
|
||||
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||
expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
|
||||
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
|
||||
revoked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
|
||||
|
||||
|
||||
class FileConnectorProfile(Base, TimestampMixin):
|
||||
|
||||
@@ -13,6 +13,7 @@ from govoplan_core.core.modules import (
|
||||
DocumentationLink,
|
||||
DocumentationTopic,
|
||||
FrontendModule,
|
||||
FrontendRoute,
|
||||
MigrationSpec,
|
||||
ModuleContext,
|
||||
ModuleInterfaceProvider,
|
||||
@@ -22,6 +23,7 @@ from govoplan_core.core.modules import (
|
||||
PermissionDefinition,
|
||||
RoleTemplate,
|
||||
)
|
||||
from govoplan_core.core.views import ViewSurface
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.change_tracking import register_files_change_tracking
|
||||
from govoplan_files.backend.db import models as file_models # noqa: F401 - populate Files ORM metadata
|
||||
@@ -32,6 +34,8 @@ register_files_change_tracking()
|
||||
|
||||
_files_table_retirement_provider = drop_table_retirement_provider(
|
||||
file_models.FileBlob,
|
||||
file_models.FileIntegrityScan,
|
||||
file_models.FileIntegrityFinding,
|
||||
file_models.FileFolder,
|
||||
file_models.FileAsset,
|
||||
file_models.FileVersion,
|
||||
@@ -98,7 +102,7 @@ PERMISSIONS = (
|
||||
_permission("files:file:download", "Download files", "Download managed files and generated archives."),
|
||||
_permission("files:file:upload", "Upload files", "Upload new managed file versions."),
|
||||
_permission("files:file:organize", "Organize files", "Create folders, rename, move or copy managed files."),
|
||||
_permission("files:file:share", "Share files", "Grant or update managed file shares; revocation is not available yet."),
|
||||
_permission("files:file:share", "Share files", "List, grant, update, expire, and revoke managed file shares."),
|
||||
_permission("files:file:delete", "Delete files", "Delete or hide managed files and folders where policy allows it."),
|
||||
_permission("files:file:admin", "Administer file spaces", "Administer all file spaces in the tenant."),
|
||||
)
|
||||
@@ -138,6 +142,43 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
|
||||
}
|
||||
|
||||
|
||||
def _tenant_summary_batch(session, tenant_ids) -> dict[str, dict[str, int]]:
|
||||
from sqlalchemy import func
|
||||
|
||||
from govoplan_files.backend.db.models import FileAsset, FileConnectorCredential, FileConnectorPolicy, FileConnectorProfile, FileConnectorSpace
|
||||
|
||||
ids = tuple(dict.fromkeys(str(tenant_id) for tenant_id in tenant_ids if tenant_id))
|
||||
if not ids:
|
||||
return {}
|
||||
counts: dict[str, dict[str, int]] = {
|
||||
tenant_id: {
|
||||
"files": 0,
|
||||
"connector_credentials": 0,
|
||||
"connector_policies": 0,
|
||||
"connector_profiles": 0,
|
||||
"connector_spaces": 0,
|
||||
}
|
||||
for tenant_id in ids
|
||||
}
|
||||
models = (
|
||||
("files", FileAsset),
|
||||
("connector_credentials", FileConnectorCredential),
|
||||
("connector_policies", FileConnectorPolicy),
|
||||
("connector_profiles", FileConnectorProfile),
|
||||
("connector_spaces", FileConnectorSpace),
|
||||
)
|
||||
for count_key, model in models:
|
||||
rows = (
|
||||
session.query(model.tenant_id, func.count(model.id))
|
||||
.filter(model.tenant_id.in_(ids))
|
||||
.group_by(model.tenant_id)
|
||||
.all()
|
||||
)
|
||||
for tenant_id, count in rows:
|
||||
counts[tenant_id][count_key] = int(count)
|
||||
return counts
|
||||
|
||||
|
||||
def _veto_group_delete(session, tenant_id: str, group_id: str) -> None:
|
||||
from govoplan_files.backend.db.models import FileAsset, FileConnectorSpace, FileFolder, FileShare
|
||||
|
||||
@@ -187,12 +228,29 @@ manifest = ModuleManifest(
|
||||
route_factory=_files_router,
|
||||
role_templates=ROLE_TEMPLATES,
|
||||
tenant_summary_providers=(_tenant_summary,),
|
||||
tenant_summary_batch_providers=(_tenant_summary_batch,),
|
||||
delete_veto_providers={"group": (_veto_group_delete,)},
|
||||
nav_items=(NavItem(path="/files", label="Files", icon="folder", required_any=("files:file:read",), order=40),),
|
||||
frontend=FrontendModule(
|
||||
module_id="files",
|
||||
package_name="@govoplan/files-webui",
|
||||
routes=(
|
||||
FrontendRoute(
|
||||
path="/files",
|
||||
component="FilesPage",
|
||||
required_any=("files:file:read",),
|
||||
order=40,
|
||||
),
|
||||
),
|
||||
nav_items=(NavItem(path="/files", label="Files", icon="folder", required_any=("files:file:read",), order=40),),
|
||||
view_surfaces=(
|
||||
ViewSurface(id="files.admin.system-connectors", module_id="files", kind="section", label="System file connections", order=75),
|
||||
ViewSurface(id="files.admin.tenant-connectors", module_id="files", kind="section", label="Tenant file connections", order=65),
|
||||
ViewSurface(id="files.admin.group-connectors", module_id="files", kind="section", label="Group file connections", order=65),
|
||||
ViewSurface(id="files.admin.user-connectors", module_id="files", kind="section", label="User file connections", order=65),
|
||||
ViewSurface(id="files.settings.connectors", module_id="files", kind="section", label="Personal file connections", order=20),
|
||||
ViewSurface(id="files.widget.spaces", module_id="files", kind="section", label="File spaces widget", order=35),
|
||||
),
|
||||
),
|
||||
documentation=(
|
||||
DocumentationTopic(
|
||||
@@ -288,11 +346,11 @@ manifest = ModuleManifest(
|
||||
),
|
||||
DocumentationTopic(
|
||||
id="files.workflow.share-managed-files",
|
||||
title="Grant or update access to managed files",
|
||||
summary="Grant or update read, write, or manage access through a supporting workflow or API client without changing ownership.",
|
||||
title="Manage access to managed files",
|
||||
summary="List, grant, update, expire, or revoke direct read, write, and manage access without changing ownership.",
|
||||
body=(
|
||||
"The Files service can grant or update a share for a user, group, tenant, or campaign. The Files page does not yet provide a general share editor, and the API has no share-revocation route. "
|
||||
"Do not use this task when access must later be revoked until that explicit capability is implemented."
|
||||
"File owners and file administrators can manage direct shares for users, groups, the tenant, and Campaign. Expired and revoked grants stop authorizing access immediately while independent active grants remain effective. "
|
||||
"The Files share dialog lists active and historical grants, and revocation is idempotent."
|
||||
),
|
||||
layer="available",
|
||||
documentation_types=("user",),
|
||||
@@ -306,7 +364,7 @@ manifest = ModuleManifest(
|
||||
),
|
||||
links=(
|
||||
DocumentationLink(label="Files", href="/files", kind="runtime"),
|
||||
DocumentationLink(label="Grant or update a share", href="/api/v1/files/{file_id}/shares", kind="api"),
|
||||
DocumentationLink(label="List and manage shares", href="/api/v1/files/{file_id}/shares", kind="api"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
related_modules=("campaigns",),
|
||||
@@ -317,22 +375,21 @@ manifest = ModuleManifest(
|
||||
"screen": "Files",
|
||||
"help_contexts": ["files.list"],
|
||||
"prerequisites": [
|
||||
"You may view and share the managed file and have write/manage access to that specific asset; the global share permission alone does not grant resource access.",
|
||||
"A supporting workflow or API client is available; the Files page has no general share editor yet.",
|
||||
"The process does not require share revocation, because no revocation route exists yet.",
|
||||
"You have the Files share permission and own the file, or administer file spaces for the active tenant.",
|
||||
"The intended user, group, tenant, or Campaign target exists and is active.",
|
||||
],
|
||||
"steps": [
|
||||
"Open Files and verify the file owner, logical path, current version, and checksum.",
|
||||
"Through the supporting workflow, identify the intended user, group, tenant, or campaign and choose read, write, or manage access.",
|
||||
"Select one managed file, choose Manage shares, and review the effective and historical direct grants.",
|
||||
"Select the intended user, group, or tenant and choose read, write, or manage access plus an optional expiry.",
|
||||
"Grant or update the share without changing file ownership.",
|
||||
"Have the intended recipient verify access and an unrelated account verify denial.",
|
||||
"Revoke a grant when it is no longer needed; repeated revocation is a no-op.",
|
||||
],
|
||||
"limitations": [
|
||||
"The Files page has no general share editor.",
|
||||
"Shares can be granted or updated, but not revoked through the current API.",
|
||||
"Campaign-target grants are normally created by the Campaign integration rather than selected manually in the Files dialog.",
|
||||
"A user may retain access through another active direct grant or ownership path after one share is revoked.",
|
||||
],
|
||||
"outcome": "The requested access is granted or updated while the managed asset keeps its owner.",
|
||||
"verification": "Test one intended and one denied access path. Do not claim that the share can later be revoked.",
|
||||
"outcome": "Direct access has the requested permission and lifetime while the managed asset keeps its owner.",
|
||||
"verification": "Test one intended and one denied path, then expire or revoke the grant and verify that only independent access paths remain.",
|
||||
"related_topic_ids": [
|
||||
"files.workflow.find-and-download-files",
|
||||
"files.assurance.process-and-release-readiness",
|
||||
@@ -451,7 +508,7 @@ manifest = ModuleManifest(
|
||||
title="Operate Files integrity, recovery, and connector transport safety",
|
||||
summary="Back up database evidence, blob bytes, and the encryption key as one recovery unit, and keep unsupported SDK transports fail-closed.",
|
||||
body=(
|
||||
"Local durable storage is the operational baseline. Recover Files from a coordinated database/blob snapshot with the original master key, then verify representative checksums and access paths. "
|
||||
"Local durable storage is the operational baseline. Recover Files from a coordinated database/blob snapshot with the original master key, then run the bounded resumable integrity scan and verify representative access paths. Missing or mismatched blobs are quarantined; orphan objects are reported before dry-run-first, explicitly authorized cleanup. "
|
||||
"Live S3 managed storage/connectors and SMB connectors fail closed until botocore redirects/endpoint discovery and SMB initial connections/DFS referrals support connection-time DNS/IP pinning. Destructive module retirement drops database tables but does not remove backend blob objects."
|
||||
),
|
||||
layer="configured",
|
||||
@@ -472,6 +529,7 @@ manifest = ModuleManifest(
|
||||
links=(
|
||||
DocumentationLink(label="System file connections", href="/admin?section=system-file-connectors", kind="runtime"),
|
||||
DocumentationLink(label="Connector provider status", href="/api/v1/files/connectors/providers", kind="api"),
|
||||
DocumentationLink(label="Create an integrity scan", href="/api/v1/files/integrity/scans", kind="api"),
|
||||
DocumentationLink(label="Files handbook", href="govoplan-files/docs/FILES_HANDBOOK.md", kind="repository"),
|
||||
),
|
||||
related_modules=("audit", "ops"),
|
||||
@@ -493,7 +551,7 @@ manifest = ModuleManifest(
|
||||
"screen": "System file connections and deployment operations",
|
||||
"section": "Storage integrity, backup/recovery, and fail-closed transports",
|
||||
"recovery_unit": ["Files database rows", "managed blob namespace", "MASTER_KEY_B64", "deployment-owned connector configuration"],
|
||||
"verification": "After restore, download representative files, compare their bytes with recorded SHA-256 values, verify authorized and denied access, and test one permitted pinned HTTP connector.",
|
||||
"verification": "After restore, complete a checksum-enabled integrity scan, resolve every missing/corrupt finding, approve or retain every reported orphan, verify authorized and denied access, and test one permitted pinned HTTP connector.",
|
||||
"related_topic_ids": [
|
||||
"files.governed-connectors-and-provenance",
|
||||
"files.reference.snapshot-provenance-and-capabilities",
|
||||
@@ -545,7 +603,7 @@ manifest = ModuleManifest(
|
||||
summary="Check a process against the implemented Files boundary, exercise permitted and denied paths, and retain evidence before approving a release or operational use.",
|
||||
body=(
|
||||
"A process owner must distinguish implemented controls from planned capabilities before relying on Files. A release is not ready until all package and manifest versions align and representative authorization, upload limits, conflict handling, download, deletion, connector, and recovery paths have been exercised. "
|
||||
"Current limitations include no general share-management UI or share revocation, no self-service restore or hard purge, no enforced retention or legal hold, and no dedicated canonical audit event for every ordinary Files mutation. Record these limitations in the process assessment instead of treating soft deletion or change-sequence entries as stronger evidence."
|
||||
"Current limitations include no self-service restore or hard purge, no enforced retention or legal hold, and no dedicated canonical audit event for every ordinary Files mutation. Share grant, change, expiry, and revocation operations do emit dedicated audit records. Record remaining limitations in the process assessment instead of treating soft deletion or change-sequence entries as stronger evidence."
|
||||
),
|
||||
layer="configured",
|
||||
documentation_types=("admin", "user"),
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""file share lifecycle
|
||||
|
||||
Revision ID: b8c9d0e1f2a4
|
||||
Revises: a7b8c9d0e1f3
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "b8c9d0e1f2a4"
|
||||
down_revision = "a7b8c9d0e1f3"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("file_shares") as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True)
|
||||
)
|
||||
batch_op.add_column(
|
||||
sa.Column("revoked_by_user_id", sa.String(length=36), nullable=True)
|
||||
)
|
||||
batch_op.create_foreign_key(
|
||||
op.f("fk_file_shares_revoked_by_user_id_access_users"),
|
||||
"access_users",
|
||||
["revoked_by_user_id"],
|
||||
["id"],
|
||||
ondelete="SET NULL",
|
||||
)
|
||||
batch_op.create_index(
|
||||
op.f("ix_file_shares_expires_at"), ["expires_at"], unique=False
|
||||
)
|
||||
batch_op.create_index(
|
||||
op.f("ix_file_shares_revoked_by_user_id"),
|
||||
["revoked_by_user_id"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("file_shares") as batch_op:
|
||||
batch_op.drop_index(op.f("ix_file_shares_revoked_by_user_id"))
|
||||
batch_op.drop_index(op.f("ix_file_shares_expires_at"))
|
||||
batch_op.drop_constraint(
|
||||
op.f("fk_file_shares_revoked_by_user_id_access_users"),
|
||||
type_="foreignkey",
|
||||
)
|
||||
batch_op.drop_column("revoked_by_user_id")
|
||||
batch_op.drop_column("expires_at")
|
||||
@@ -0,0 +1,168 @@
|
||||
"""file integrity reconciliation
|
||||
|
||||
Revision ID: c9d0e1f2a3b5
|
||||
Revises: b8c9d0e1f2a4
|
||||
Create Date: 2026-07-30 00:00:00.000000
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "c9d0e1f2a3b5"
|
||||
down_revision = "b8c9d0e1f2a4"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("file_blobs") as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column(
|
||||
"integrity_status",
|
||||
sa.String(length=30),
|
||||
nullable=False,
|
||||
server_default="unchecked",
|
||||
)
|
||||
)
|
||||
batch_op.add_column(
|
||||
sa.Column("integrity_checked_at", sa.DateTime(timezone=True), nullable=True)
|
||||
)
|
||||
batch_op.add_column(
|
||||
sa.Column("integrity_failure", sa.String(length=100), nullable=True)
|
||||
)
|
||||
batch_op.add_column(
|
||||
sa.Column("quarantined_at", sa.DateTime(timezone=True), nullable=True)
|
||||
)
|
||||
batch_op.create_index(
|
||||
op.f("ix_file_blobs_integrity_status"),
|
||||
["integrity_status"],
|
||||
unique=False,
|
||||
)
|
||||
batch_op.create_index(
|
||||
op.f("ix_file_blobs_integrity_checked_at"),
|
||||
["integrity_checked_at"],
|
||||
unique=False,
|
||||
)
|
||||
batch_op.create_index(
|
||||
op.f("ix_file_blobs_quarantined_at"),
|
||||
["quarantined_at"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"file_integrity_scans",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("storage_backend", sa.String(length=50), nullable=False),
|
||||
sa.Column("storage_prefix", sa.String(length=1000), nullable=False),
|
||||
sa.Column("status", sa.String(length=30), nullable=False),
|
||||
sa.Column("phase", sa.String(length=30), nullable=False),
|
||||
sa.Column("verify_checksums", sa.Boolean(), nullable=False),
|
||||
sa.Column("batch_size", sa.Integer(), nullable=False),
|
||||
sa.Column("blob_cursor", sa.String(length=36), nullable=True),
|
||||
sa.Column("object_cursor", sa.String(length=1000), nullable=True),
|
||||
sa.Column("scanned_blob_count", sa.Integer(), nullable=False),
|
||||
sa.Column("verified_blob_count", sa.Integer(), nullable=False),
|
||||
sa.Column("quarantined_blob_count", sa.Integer(), nullable=False),
|
||||
sa.Column("scanned_object_count", sa.Integer(), nullable=False),
|
||||
sa.Column("orphan_object_count", sa.Integer(), nullable=False),
|
||||
sa.Column("created_by_user_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("last_error", sa.String(length=255), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["created_by_user_id"],
|
||||
["access_users.id"],
|
||||
name=op.f(
|
||||
"fk_file_integrity_scans_created_by_user_id_access_users"
|
||||
),
|
||||
ondelete="SET NULL",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_file_integrity_scans")),
|
||||
)
|
||||
for column in ("tenant_id", "status", "created_by_user_id"):
|
||||
op.create_index(
|
||||
op.f(f"ix_file_integrity_scans_{column}"),
|
||||
"file_integrity_scans",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"file_integrity_findings",
|
||||
sa.Column("id", sa.String(length=36), nullable=False),
|
||||
sa.Column("scan_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("tenant_id", sa.String(length=36), nullable=False),
|
||||
sa.Column("kind", sa.String(length=40), nullable=False),
|
||||
sa.Column("state", sa.String(length=30), nullable=False),
|
||||
sa.Column("blob_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("storage_key", sa.String(length=1000), nullable=False),
|
||||
sa.Column("expected_size_bytes", sa.Integer(), nullable=True),
|
||||
sa.Column("observed_size_bytes", sa.Integer(), nullable=True),
|
||||
sa.Column("expected_checksum_sha256", sa.String(length=64), nullable=True),
|
||||
sa.Column("observed_checksum_sha256", sa.String(length=64), nullable=True),
|
||||
sa.Column("resolved_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("resolved_by_user_id", sa.String(length=36), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["blob_id"],
|
||||
["file_blobs.id"],
|
||||
name=op.f("fk_file_integrity_findings_blob_id_file_blobs"),
|
||||
ondelete="SET NULL",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["resolved_by_user_id"],
|
||||
["access_users.id"],
|
||||
name=op.f(
|
||||
"fk_file_integrity_findings_resolved_by_user_id_access_users"
|
||||
),
|
||||
ondelete="SET NULL",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["scan_id"],
|
||||
["file_integrity_scans.id"],
|
||||
name=op.f(
|
||||
"fk_file_integrity_findings_scan_id_file_integrity_scans"
|
||||
),
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id", name=op.f("pk_file_integrity_findings")),
|
||||
)
|
||||
for column in (
|
||||
"scan_id",
|
||||
"tenant_id",
|
||||
"kind",
|
||||
"state",
|
||||
"blob_id",
|
||||
"resolved_by_user_id",
|
||||
):
|
||||
op.create_index(
|
||||
op.f(f"ix_file_integrity_findings_{column}"),
|
||||
"file_integrity_findings",
|
||||
[column],
|
||||
unique=False,
|
||||
)
|
||||
op.create_index(
|
||||
"ix_file_integrity_findings_scan_state",
|
||||
"file_integrity_findings",
|
||||
["scan_id", "state"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("file_integrity_findings")
|
||||
op.drop_table("file_integrity_scans")
|
||||
with op.batch_alter_table("file_blobs") as batch_op:
|
||||
batch_op.drop_index(op.f("ix_file_blobs_quarantined_at"))
|
||||
batch_op.drop_index(op.f("ix_file_blobs_integrity_checked_at"))
|
||||
batch_op.drop_index(op.f("ix_file_blobs_integrity_status"))
|
||||
batch_op.drop_column("quarantined_at")
|
||||
batch_op.drop_column("integrity_failure")
|
||||
batch_op.drop_column("integrity_checked_at")
|
||||
batch_op.drop_column("integrity_status")
|
||||
1311
src/govoplan_files/backend/route_support.py
Normal file
1311
src/govoplan_files/backend/route_support.py
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1
src/govoplan_files/backend/routes/__init__.py
Normal file
1
src/govoplan_files/backend/routes/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Focused HTTP route modules for the Files API."""
|
||||
134
src/govoplan_files/backend/routes/assets.py
Normal file
134
src/govoplan_files/backend/routes/assets.py
Normal file
@@ -0,0 +1,134 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from io import BytesIO
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
BulkDeleteRequest,
|
||||
BulkDeleteResponse,
|
||||
FileAssetResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
read_asset_bytes,
|
||||
soft_delete_assets,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_attachment_disposition,
|
||||
_audit_connector_event,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/{file_id}", response_model=FileAssetResponse)
|
||||
def get_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return _asset_response(session, asset, include_shares=True)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.get("/{file_id}/download")
|
||||
def download_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:download")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
data, version, blob = read_asset_bytes(session, asset)
|
||||
_audit_connector_event(
|
||||
session,
|
||||
principal,
|
||||
action="files.connector.accessed",
|
||||
asset=asset,
|
||||
version=version,
|
||||
blob=blob,
|
||||
operation="download",
|
||||
commit=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
headers = {"Content-Disposition": _attachment_disposition(asset.filename)}
|
||||
return StreamingResponse(
|
||||
BytesIO(data),
|
||||
media_type=blob.content_type or "application/octet-stream",
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/{file_id}", response_model=BulkDeleteResponse)
|
||||
def delete_file(
|
||||
file_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
count = soft_delete_assets(session, [asset])
|
||||
session.commit()
|
||||
return BulkDeleteResponse(deleted_count=count)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.post("/bulk-delete", response_model=BulkDeleteResponse)
|
||||
def bulk_delete_files(
|
||||
payload: BulkDeleteRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
assets = [
|
||||
get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
require_write=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in payload.file_ids
|
||||
]
|
||||
count = soft_delete_assets(session, assets)
|
||||
session.commit()
|
||||
return BulkDeleteResponse(deleted_count=count)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
252
src/govoplan_files/backend/routes/connector_io.py
Normal file
252
src/govoplan_files/backend/routes/connector_io.py
Normal file
@@ -0,0 +1,252 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorBrowseItem,
|
||||
FileConnectorBrowseResponse,
|
||||
FileConnectorImportRequest,
|
||||
FileConnectorSyncResponse,
|
||||
FileUploadResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_browse import (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
browse_connector_profile,
|
||||
normalize_connector_browse_path,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_imports import (
|
||||
ConnectorImportError,
|
||||
ConnectorImportUnsupported,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicyDenied,
|
||||
connector_policy_decision,
|
||||
)
|
||||
from govoplan_files.backend.storage.files import (
|
||||
create_file_asset,
|
||||
sync_file_asset_from_source,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_audit_connector_imports,
|
||||
_audit_connector_sync,
|
||||
_connector_browse_next_token,
|
||||
_connector_policy_error,
|
||||
_download_connector_payload,
|
||||
_ensure_campaign_file_access,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles/{profile_id}/import", response_model=FileUploadResponse
|
||||
)
|
||||
def import_connector_file(
|
||||
profile_id: str,
|
||||
payload: FileConnectorImportRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
try:
|
||||
if payload.campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=payload.campaign_id
|
||||
)
|
||||
_source_path, downloaded, metadata = _download_connector_payload(
|
||||
profile, payload, operation="import"
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
stored = create_file_asset(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=downloaded.filename,
|
||||
data=downloaded.data,
|
||||
folder=payload.target_folder,
|
||||
display_path=payload.target_path,
|
||||
content_type=downloaded.content_type,
|
||||
metadata=metadata,
|
||||
campaign_id=payload.campaign_id,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_audit_connector_imports(session, principal, [stored.asset])
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorImportUnsupported as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (
|
||||
ConnectorImportError,
|
||||
FileStorageError,
|
||||
UnsafeFilePathError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileUploadResponse(
|
||||
files=[_asset_response(session, stored.asset, include_shares=True)]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles/{profile_id}/sync", response_model=FileConnectorSyncResponse
|
||||
)
|
||||
def sync_connector_file(
|
||||
profile_id: str,
|
||||
payload: FileConnectorImportRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
try:
|
||||
if payload.campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=payload.campaign_id
|
||||
)
|
||||
_source_path, downloaded, metadata = _download_connector_payload(
|
||||
profile, payload, operation="sync"
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
stored, sync_action, previous_version_id = sync_file_asset_from_source(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=downloaded.filename,
|
||||
data=downloaded.data,
|
||||
folder=payload.target_folder,
|
||||
display_path=payload.target_path,
|
||||
content_type=downloaded.content_type,
|
||||
metadata=metadata,
|
||||
campaign_id=payload.campaign_id,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_audit_connector_sync(
|
||||
session,
|
||||
principal,
|
||||
stored.asset,
|
||||
sync_action=sync_action,
|
||||
previous_version_id=previous_version_id,
|
||||
)
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorImportUnsupported as exc:
|
||||
session.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (
|
||||
ConnectorImportError,
|
||||
FileStorageError,
|
||||
UnsafeFilePathError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorSyncResponse(
|
||||
file=_asset_response(session, stored.asset, include_shares=True),
|
||||
action=sync_action,
|
||||
previous_version_id=previous_version_id,
|
||||
current_version_id=stored.version.id,
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/profiles/{profile_id}/browse",
|
||||
response_model=FileConnectorBrowseResponse,
|
||||
)
|
||||
def browse_connector_profile_items(
|
||||
profile_id: str,
|
||||
path: str | None = None,
|
||||
library_id: str | None = None,
|
||||
continuation_token: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, profile_id, campaign_id=campaign_id
|
||||
)
|
||||
browse_path = normalize_connector_browse_path(path)
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id=profile.id,
|
||||
credential_id=profile.credential_profile_id,
|
||||
provider=profile.provider,
|
||||
external_path=browse_path,
|
||||
external_url=connector_effective_endpoint_url(
|
||||
provider=profile.provider,
|
||||
endpoint_url=profile.endpoint_url,
|
||||
metadata=profile.metadata,
|
||||
),
|
||||
operation="browse",
|
||||
),
|
||||
profile.policy_sources,
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
items = browse_connector_profile(
|
||||
profile,
|
||||
path=browse_path,
|
||||
library_id=library_id,
|
||||
continuation_token=continuation_token,
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except ConnectorBrowseUnsupported as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc)
|
||||
) from exc
|
||||
except (ConnectorBrowseError, OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorBrowseResponse(
|
||||
profile_id=profile.id,
|
||||
provider=profile.provider,
|
||||
path=browse_path,
|
||||
library_id=library_id,
|
||||
next_continuation_token=_connector_browse_next_token(items),
|
||||
has_more=any(bool(item.metadata.get("listing_truncated")) for item in items),
|
||||
decision=decision.to_dict(),
|
||||
items=[FileConnectorBrowseItem(**item.to_response()) for item in items],
|
||||
)
|
||||
261
src/govoplan_files/backend/routes/connector_profiles.py
Normal file
261
src/govoplan_files/backend/routes/connector_profiles.py
Normal file
@@ -0,0 +1,261 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorProfileResponse,
|
||||
FileConnectorProfileUpdateRequest,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_credential_deletion import (
|
||||
delete_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
connector_profile_from_row,
|
||||
get_connector_profile_row,
|
||||
update_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_policy_error,
|
||||
_credential_row_for_profile,
|
||||
_ensure_connector_configuration_allowed,
|
||||
_ensure_connector_local_policy_allowed,
|
||||
_http_error,
|
||||
_record_connector_settings_change,
|
||||
_require_connector_profile_write,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def get_connector_profile(
|
||||
profile_id: str,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session,
|
||||
principal,
|
||||
profile_id,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorProfileResponse(**profile.to_response())
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def update_connector_profile(
|
||||
profile_id: str,
|
||||
payload: FileConnectorProfileUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
profile_id=profile_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_profile_write(principal, row.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
credential_profile_id = (
|
||||
payload.credential_profile_id
|
||||
if payload.credential_profile_id is not None
|
||||
else row.credential_profile_id
|
||||
)
|
||||
provider = payload.provider if payload.provider is not None else row.provider
|
||||
credential_row = _credential_row_for_profile(
|
||||
session,
|
||||
principal,
|
||||
credential_profile_id=credential_profile_id,
|
||||
provider=provider,
|
||||
profile_id=row.id,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=row.id,
|
||||
credential_id=credential_profile_id,
|
||||
provider=provider,
|
||||
endpoint_url=connector_effective_endpoint_url(
|
||||
provider=provider,
|
||||
endpoint_url=payload.endpoint_url
|
||||
if payload.endpoint_url is not None
|
||||
else row.endpoint_url,
|
||||
metadata=payload.metadata
|
||||
if payload.metadata is not None
|
||||
else row.metadata_,
|
||||
),
|
||||
base_path=payload.base_path
|
||||
if payload.base_path is not None
|
||||
else row.base_path,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
operation="configure",
|
||||
)
|
||||
if payload.policy is not None:
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
update_connector_profile_row(
|
||||
session,
|
||||
row,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
enabled=payload.enabled,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username if credentials else None,
|
||||
password=credentials.password if credentials else None,
|
||||
token=credentials.token if credentials else None,
|
||||
password_env=credentials.password_env if credentials else None,
|
||||
token_env=credentials.token_env if credentials else None,
|
||||
secret_ref=credentials.secret_ref if credentials else None,
|
||||
clear_password=payload.clear_password,
|
||||
clear_token=payload.clear_token,
|
||||
capabilities=payload.capabilities,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(
|
||||
row, credential_row=credential_row
|
||||
).to_response()
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connectors/profiles/{profile_id}", response_model=FileConnectorProfileResponse
|
||||
)
|
||||
def deactivate_connector_profile(
|
||||
profile_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
profile_id=profile_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_profile_write(principal, row.scope_type)
|
||||
try:
|
||||
changed = delete_connector_profile_row(
|
||||
session,
|
||||
row,
|
||||
deletion_reason="api_delete",
|
||||
user_id=principal.user.id,
|
||||
api_key_id=principal.api_key.id if principal.api_key else None,
|
||||
)
|
||||
if changed:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(row).to_response()
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
852
src/govoplan_files/backend/routes/connector_settings.py
Normal file
852
src/govoplan_files/backend/routes/connector_settings.py
Normal file
@@ -0,0 +1,852 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_any_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorCredentialCreateRequest,
|
||||
FileConnectorCredentialResponse,
|
||||
FileConnectorCredentialsResponse,
|
||||
FileConnectorCredentialUpdateRequest,
|
||||
FileConnectorDiscoveryRequest,
|
||||
FileConnectorDiscoveryResponse,
|
||||
FileConnectorSettingsDeltaResponse,
|
||||
FileConnectorPolicyEvaluateRequest,
|
||||
FileConnectorPolicyEvaluateResponse,
|
||||
FileConnectorPolicyResponse,
|
||||
FileConnectorPolicyUpdateRequest,
|
||||
FileConnectorProfileCreateRequest,
|
||||
FileConnectorProfileResponse,
|
||||
FileConnectorProfilesResponse,
|
||||
FileConnectorProviderResponse,
|
||||
FileConnectorProvidersResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
create_connector_credential_row,
|
||||
get_connector_credential_row,
|
||||
list_database_connector_credentials,
|
||||
update_connector_credential_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_credential_deletion import (
|
||||
delete_connector_credential_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_browse import (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
browse_connector_profile,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
connector_effective_endpoint_url,
|
||||
reject_api_controlled_deployment_references,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profile_store import (
|
||||
connector_profile_from_row,
|
||||
create_connector_profile_row,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_providers import (
|
||||
connector_provider_descriptors,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicyDenied,
|
||||
connector_policy_decision,
|
||||
connector_policy_sources_from_payload,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy_store import (
|
||||
connector_policy_response,
|
||||
set_connector_policy,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
FILES_CONNECTOR_POLICY_RESOURCE,
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
_audit_connector_discovery_attempt,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_credential_response,
|
||||
_connector_policy_error,
|
||||
_credential_row_for_profile,
|
||||
_discovery_profile_from_payload,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_connector_configuration_allowed,
|
||||
_ensure_connector_credential_configuration_allowed,
|
||||
_ensure_connector_local_policy_allowed,
|
||||
_file_connector_policy_resource_id,
|
||||
_file_connector_settings_entries,
|
||||
_http_error,
|
||||
_record_connector_settings_change,
|
||||
_require_connector_credential_write,
|
||||
_require_connector_policy_read,
|
||||
_require_connector_profile_write,
|
||||
_same_endpoint,
|
||||
_visible_connector_profiles,
|
||||
_webdav_discovery_candidates,
|
||||
)
|
||||
from govoplan_files.backend.services.connector_settings_delta import (
|
||||
_full_file_connector_settings_delta_response,
|
||||
_incremental_file_connector_settings_delta_response,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connector-policy/evaluate", response_model=FileConnectorPolicyEvaluateResponse
|
||||
)
|
||||
def evaluate_connector_policy(
|
||||
payload: FileConnectorPolicyEvaluateRequest,
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope("files:file:read", "files:file:upload", "files:file:download")
|
||||
),
|
||||
):
|
||||
del principal
|
||||
sources = connector_policy_sources_from_payload(
|
||||
[item.model_dump(mode="json") for item in payload.policy_sources]
|
||||
)
|
||||
request = ConnectorAccessRequest.from_provenance(
|
||||
payload.source_provenance.model_dump(mode="json", exclude_none=True),
|
||||
operation=payload.operation,
|
||||
)
|
||||
return FileConnectorPolicyEvaluateResponse(
|
||||
decision=connector_policy_decision(request, sources).to_dict()
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/settings/delta", response_model=FileConnectorSettingsDeltaResponse
|
||||
)
|
||||
def connector_settings_delta(
|
||||
scope_type: str = Query(default="tenant"),
|
||||
scope_id: str | None = Query(default=None),
|
||||
provider: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
include_inactive: bool = False,
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
since: str | None = None,
|
||||
limit: int = Query(default=100, ge=1, le=500),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
scope_type = scope_type.strip().casefold()
|
||||
_require_connector_policy_read(principal, scope_type)
|
||||
try:
|
||||
if since is None:
|
||||
return _full_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
entries, has_more = _file_connector_settings_entries(
|
||||
session, tenant_id=principal.tenant_id, since=since, limit=limit
|
||||
)
|
||||
if entries is None:
|
||||
return _full_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
return _incremental_file_connector_settings_delta_response(
|
||||
session,
|
||||
principal,
|
||||
entries=entries,
|
||||
has_more=has_more,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled,
|
||||
include_inactive=include_inactive,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.get("/connectors/providers", response_model=FileConnectorProvidersResponse)
|
||||
def list_connector_providers(
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope("files:file:read", "files:file:upload", "files:file:admin")
|
||||
),
|
||||
):
|
||||
del principal
|
||||
return FileConnectorProvidersResponse(
|
||||
providers=[
|
||||
FileConnectorProviderResponse(**item.to_response())
|
||||
for item in connector_provider_descriptors()
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post("/connectors/discover", response_model=FileConnectorDiscoveryResponse)
|
||||
def discover_connector_endpoint(
|
||||
payload: FileConnectorDiscoveryRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
reject_api_controlled_deployment_references(
|
||||
password_env=payload.credentials.password_env,
|
||||
token_env=payload.credentials.token_env,
|
||||
secret_ref=payload.credentials.secret_ref,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
if payload.provider not in {"webdav", "nextcloud"}:
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=None,
|
||||
base_path=payload.base_path,
|
||||
status="unsupported",
|
||||
message=f"Discovery is not implemented for {payload.provider} connectors yet",
|
||||
)
|
||||
candidates: list[dict[str, str]] = []
|
||||
for endpoint_url in _webdav_discovery_candidates(payload):
|
||||
profile = _discovery_profile_from_payload(
|
||||
payload, endpoint_url, principal=principal
|
||||
)
|
||||
try:
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=None,
|
||||
credential_id=None,
|
||||
provider=profile.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=profile.base_path,
|
||||
scope_type="tenant",
|
||||
scope_id=principal.tenant_id,
|
||||
operation="discover",
|
||||
)
|
||||
_audit_connector_discovery_attempt(
|
||||
session,
|
||||
principal,
|
||||
provider=profile.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=profile.base_path,
|
||||
)
|
||||
browse_connector_profile(profile, path=payload.base_path or "")
|
||||
except ConnectorPolicyDenied as exc:
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (
|
||||
ConnectorBrowseError,
|
||||
ConnectorBrowseUnsupported,
|
||||
OSError,
|
||||
ValueError,
|
||||
json.JSONDecodeError,
|
||||
) as exc:
|
||||
message = str(exc)
|
||||
if "credentials were rejected" in message.casefold():
|
||||
if payload.require_valid_credentials:
|
||||
candidates.append(
|
||||
{
|
||||
"endpoint_url": endpoint_url,
|
||||
"status": "credentials_rejected",
|
||||
"message": "The endpoint exists, but the credentials were rejected.",
|
||||
}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status="credentials_rejected",
|
||||
message="The endpoint was found, but login failed with these credentials.",
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||
)
|
||||
candidates.append(
|
||||
{
|
||||
"endpoint_url": endpoint_url,
|
||||
"status": "found",
|
||||
"message": "The endpoint exists, but credentials are required or were rejected.",
|
||||
}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status="found",
|
||||
message="The endpoint was found. Add working credentials before saving or testing the connection.",
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||
)
|
||||
candidates.append(
|
||||
{"endpoint_url": endpoint_url, "status": "failed", "message": message}
|
||||
)
|
||||
continue
|
||||
status_value = (
|
||||
"usable" if _same_endpoint(endpoint_url, payload.endpoint_url) else "found"
|
||||
)
|
||||
message = (
|
||||
"The supplied URL is directly usable."
|
||||
if status_value == "usable"
|
||||
else "A usable connector endpoint was discovered."
|
||||
)
|
||||
candidates.append(
|
||||
{"endpoint_url": endpoint_url, "status": status_value, "message": message}
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
status=status_value,
|
||||
message=message,
|
||||
candidates=candidates,
|
||||
metadata={"discovered_by": "webdav-propfind"},
|
||||
)
|
||||
return FileConnectorDiscoveryResponse(
|
||||
provider=payload.provider,
|
||||
endpoint_url=None,
|
||||
base_path=payload.base_path,
|
||||
status="not_found",
|
||||
message="No usable WebDAV endpoint was found for this server URL.",
|
||||
candidates=candidates,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/connectors/credentials", response_model=FileConnectorCredentialsResponse)
|
||||
def list_connector_credentials(
|
||||
provider: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
provider_norm = provider.strip().casefold() if provider else None
|
||||
credentials = list_database_connector_credentials(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
)
|
||||
return FileConnectorCredentialsResponse(
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in credentials
|
||||
if provider_norm is None or credential.provider in {None, provider_norm}
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/policies/{scope_type}", response_model=FileConnectorPolicyResponse
|
||||
)
|
||||
def read_connector_policy(
|
||||
scope_type: str,
|
||||
scope_id: str | None = Query(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_policy_read(principal, scope_type)
|
||||
try:
|
||||
return FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.put(
|
||||
"/connectors/policies/{scope_type}", response_model=FileConnectorPolicyResponse
|
||||
)
|
||||
def write_connector_policy(
|
||||
scope_type: str,
|
||||
payload: FileConnectorPolicyUpdateRequest,
|
||||
scope_id: str | None = Query(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_profile_write(principal, scope_type)
|
||||
try:
|
||||
set_connector_policy(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_POLICY_RESOURCE,
|
||||
resource_id=_file_connector_policy_resource_id(scope_type, scope_id),
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=None
|
||||
if scope_type.strip().casefold() == "system"
|
||||
else principal.tenant_id,
|
||||
payload={"scope_type": scope_type.strip().casefold(), "scope_id": scope_id},
|
||||
)
|
||||
session.commit()
|
||||
return FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/credentials",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_connector_credential(
|
||||
payload: FileConnectorCredentialCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_credential_write(principal, payload.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
_ensure_connector_credential_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
credential_id=payload.id,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
operation="configure_credentials",
|
||||
)
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
row = create_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
credential_id=payload.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
enabled=payload.enabled,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username,
|
||||
password=credentials.password,
|
||||
token=credentials.token,
|
||||
password_env=credentials.password_env,
|
||||
token_env=credentials.token_env,
|
||||
secret_ref=credentials.secret_ref,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.get(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def get_connector_credential(
|
||||
credential_id: str,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:read", "admin:settings:read"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
)
|
||||
return _connector_credential_response(row)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def update_connector_credential(
|
||||
credential_id: str,
|
||||
payload: FileConnectorCredentialUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_credential_write(principal, row.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
provider = payload.provider if payload.provider is not None else row.provider
|
||||
_ensure_connector_credential_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
credential_id=row.id,
|
||||
provider=provider,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
operation="configure_credentials",
|
||||
)
|
||||
if payload.policy is not None:
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
update_connector_credential_row(
|
||||
session,
|
||||
row,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
enabled=payload.enabled,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username if credentials else None,
|
||||
password=credentials.password if credentials else None,
|
||||
token=credentials.token if credentials else None,
|
||||
password_env=credentials.password_env if credentials else None,
|
||||
token_env=credentials.token_env if credentials else None,
|
||||
secret_ref=credentials.secret_ref if credentials else None,
|
||||
clear_password=payload.clear_password,
|
||||
clear_token=payload.clear_token,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connectors/credentials/{credential_id}",
|
||||
response_model=FileConnectorCredentialResponse,
|
||||
)
|
||||
def deactivate_connector_credential(
|
||||
credential_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
row = get_connector_credential_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
credential_id=credential_id,
|
||||
include_disabled=True,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
_require_connector_credential_write(principal, row.scope_type)
|
||||
try:
|
||||
deletion = delete_connector_credential_row(
|
||||
session,
|
||||
row,
|
||||
deletion_reason="api_delete",
|
||||
user_id=principal.user.id,
|
||||
api_key_id=principal.api_key.id if principal.api_key else None,
|
||||
)
|
||||
if deletion.changed:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
for profile in deletion.affected_profiles:
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=profile.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=profile.tenant_id,
|
||||
payload={
|
||||
"scope_type": profile.scope_type,
|
||||
"scope_id": profile.scope_id,
|
||||
"provider": profile.provider,
|
||||
"reason": "credential_deleted",
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return _connector_credential_response(row)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.get("/connectors/profiles", response_model=FileConnectorProfilesResponse)
|
||||
def list_connector_profiles(
|
||||
provider: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
include_disabled: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:read",
|
||||
"files:file:upload",
|
||||
"files:file:download",
|
||||
"files:file:admin",
|
||||
"system:settings:read",
|
||||
"admin:settings:read",
|
||||
)
|
||||
),
|
||||
):
|
||||
try:
|
||||
if campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return FileConnectorProfilesResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in profiles
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connectors/profiles",
|
||||
response_model=FileConnectorProfileResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_connector_profile(
|
||||
payload: FileConnectorProfileCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(
|
||||
require_any_scope(
|
||||
"files:file:admin", "system:settings:write", "admin:settings:write"
|
||||
)
|
||||
),
|
||||
):
|
||||
_require_connector_profile_write(principal, payload.scope_type)
|
||||
credentials = payload.credentials
|
||||
try:
|
||||
credential_row = _credential_row_for_profile(
|
||||
session,
|
||||
principal,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
provider=payload.provider,
|
||||
profile_id=payload.id,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
)
|
||||
_ensure_connector_configuration_allowed(
|
||||
session,
|
||||
principal,
|
||||
connector_id=payload.id,
|
||||
credential_id=payload.credential_profile_id,
|
||||
provider=payload.provider,
|
||||
endpoint_url=connector_effective_endpoint_url(
|
||||
provider=payload.provider,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
metadata=payload.metadata,
|
||||
),
|
||||
base_path=payload.base_path,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
operation="configure",
|
||||
)
|
||||
_ensure_connector_local_policy_allowed(
|
||||
session,
|
||||
principal,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
policy=payload.policy,
|
||||
)
|
||||
row = create_connector_profile_row(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
profile_id=payload.id,
|
||||
label=payload.label,
|
||||
provider=payload.provider,
|
||||
scope_type=payload.scope_type,
|
||||
scope_id=payload.scope_id,
|
||||
endpoint_url=payload.endpoint_url,
|
||||
base_path=payload.base_path,
|
||||
enabled=payload.enabled,
|
||||
credential_profile_id=payload.credential_profile_id,
|
||||
credential_mode=payload.credential_mode,
|
||||
username=credentials.username,
|
||||
password=credentials.password,
|
||||
token=credentials.token,
|
||||
password_env=credentials.password_env,
|
||||
token_env=credentials.token_env,
|
||||
secret_ref=credentials.secret_ref,
|
||||
capabilities=payload.capabilities,
|
||||
policy=payload.policy,
|
||||
metadata=payload.metadata,
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
resource_id=row.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=row.tenant_id,
|
||||
payload={
|
||||
"scope_type": row.scope_type,
|
||||
"scope_id": row.scope_id,
|
||||
"provider": row.provider,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
return FileConnectorProfileResponse(
|
||||
**connector_profile_from_row(
|
||||
row, credential_row=credential_row
|
||||
).to_response()
|
||||
)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
151
src/govoplan_files/backend/routes/folders.py
Normal file
151
src/govoplan_files/backend/routes/folders.py
Normal file
@@ -0,0 +1,151 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileFolderCreateRequest,
|
||||
FileFolderDeleteRequest,
|
||||
FileFolderDeleteResponse,
|
||||
FileFolderResponse,
|
||||
FileFoldersResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.folders import (
|
||||
create_folder,
|
||||
list_folders_for_user,
|
||||
list_folders_for_user_window,
|
||||
soft_delete_folder,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_folder_response,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
from govoplan_files.backend.services.list_queries import (
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
_cursor_page_size,
|
||||
_files_delta_watermark,
|
||||
_folder_cursor_values,
|
||||
_folders_list_fingerprint,
|
||||
_next_folder_list_cursor,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/folders", response_model=FileFoldersResponse)
|
||||
def list_file_folders(
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int | None = Query(default=None, ge=1, le=1000),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
watermark = _files_delta_watermark(session, principal.tenant_id)
|
||||
effective_page_size = _cursor_page_size(
|
||||
FOLDERS_LIST_CURSOR_SCOPE, cursor, page_size
|
||||
)
|
||||
if effective_page_size is None:
|
||||
folders = list_folders_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileFoldersResponse(
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
watermark=watermark,
|
||||
)
|
||||
fingerprint = _folders_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
page_size=effective_page_size,
|
||||
)
|
||||
after_path, after_id = _folder_cursor_values(cursor, fingerprint=fingerprint)
|
||||
folders, has_more = list_folders_for_user_window(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
page_size=effective_page_size,
|
||||
after_path=after_path,
|
||||
after_id=after_id,
|
||||
)
|
||||
return FileFoldersResponse(
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
cursor=cursor,
|
||||
next_cursor=_next_folder_list_cursor(
|
||||
principal,
|
||||
folders,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
page_size=effective_page_size,
|
||||
has_more=has_more,
|
||||
),
|
||||
watermark=watermark,
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/folders", response_model=FileFolderResponse)
|
||||
def create_file_folder(
|
||||
payload: FileFolderCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
folder = create_folder(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
user_id=principal.user.id,
|
||||
path=payload.path,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return _folder_response(folder)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/folders/delete", response_model=FileFolderDeleteResponse)
|
||||
def delete_file_folder(
|
||||
payload: FileFolderDeleteRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:delete")),
|
||||
):
|
||||
try:
|
||||
deleted_folders, deleted_files = soft_delete_folder(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
user_id=principal.user.id,
|
||||
path=payload.path,
|
||||
recursive=payload.recursive,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return FileFolderDeleteResponse(
|
||||
deleted_folders=deleted_folders, deleted_files=deleted_files
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
337
src/govoplan_files/backend/routes/integrity.py
Normal file
337
src/govoplan_files/backend/routes/integrity.py
Normal file
@@ -0,0 +1,337 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.db.models import (
|
||||
FileIntegrityFinding,
|
||||
FileIntegrityScan,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileIntegrityActionRequest,
|
||||
FileIntegrityActionResponse,
|
||||
FileIntegrityFindingResponse,
|
||||
FileIntegrityFindingsResponse,
|
||||
FileIntegrityScanCreateRequest,
|
||||
FileIntegrityScanResponse,
|
||||
FileIntegrityScansResponse,
|
||||
)
|
||||
from govoplan_files.backend.storage.backends import StorageBackendError
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.integrity import (
|
||||
cleanup_orphan_finding,
|
||||
create_integrity_scan,
|
||||
mark_integrity_scan_failed,
|
||||
recheck_integrity_finding,
|
||||
run_integrity_scan_batch,
|
||||
)
|
||||
|
||||
|
||||
router = APIRouter(prefix="/files/integrity", tags=["files-integrity"])
|
||||
|
||||
|
||||
@router.get("/scans", response_model=FileIntegrityScansResponse)
|
||||
def list_integrity_scans(
|
||||
limit: int = Query(default=50, ge=1, le=200),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityScansResponse:
|
||||
rows = (
|
||||
session.query(FileIntegrityScan)
|
||||
.filter(FileIntegrityScan.tenant_id == principal.tenant_id)
|
||||
.order_by(FileIntegrityScan.created_at.desc(), FileIntegrityScan.id.desc())
|
||||
.limit(limit)
|
||||
.all()
|
||||
)
|
||||
return FileIntegrityScansResponse(
|
||||
scans=[_scan_response(row) for row in rows]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/scans",
|
||||
response_model=FileIntegrityScanResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_scan(
|
||||
payload: FileIntegrityScanCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityScanResponse:
|
||||
try:
|
||||
scan = create_integrity_scan(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
verify_checksums=payload.verify_checksums,
|
||||
batch_size=payload.batch_size,
|
||||
)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="files.integrity.scan_created",
|
||||
object_type="file_integrity_scan",
|
||||
object_id=scan.id,
|
||||
details={
|
||||
"storage_backend": scan.storage_backend,
|
||||
"verify_checksums": scan.verify_checksums,
|
||||
"batch_size": scan.batch_size,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _scan_response(scan)
|
||||
except (FileStorageError, StorageBackendError) as exc:
|
||||
session.rollback()
|
||||
raise _integrity_http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/scans/{scan_id}/run", response_model=FileIntegrityScanResponse)
|
||||
def run_scan_batch(
|
||||
scan_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityScanResponse:
|
||||
scan = _scan_for_tenant(session, scan_id, principal.tenant_id)
|
||||
previous_status = scan.status
|
||||
try:
|
||||
run_integrity_scan_batch(session, scan)
|
||||
if scan.status == "completed" and previous_status != "completed":
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="files.integrity.scan_completed",
|
||||
object_type="file_integrity_scan",
|
||||
object_id=scan.id,
|
||||
details={
|
||||
"verified_blob_count": scan.verified_blob_count,
|
||||
"quarantined_blob_count": scan.quarantined_blob_count,
|
||||
"orphan_object_count": scan.orphan_object_count,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _scan_response(scan)
|
||||
except (FileStorageError, StorageBackendError) as exc:
|
||||
session.rollback()
|
||||
scan = _scan_for_tenant(session, scan_id, principal.tenant_id)
|
||||
mark_integrity_scan_failed(scan, error=exc)
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action="files.integrity.scan_failed",
|
||||
object_type="file_integrity_scan",
|
||||
object_id=scan.id,
|
||||
details={"error_type": type(exc).__name__},
|
||||
)
|
||||
session.commit()
|
||||
raise _integrity_http_error(exc) from exc
|
||||
|
||||
|
||||
@router.get(
|
||||
"/scans/{scan_id}/findings",
|
||||
response_model=FileIntegrityFindingsResponse,
|
||||
)
|
||||
def list_integrity_findings(
|
||||
scan_id: str,
|
||||
state_filter: str | None = Query(default=None, alias="state"),
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityFindingsResponse:
|
||||
scan = _scan_for_tenant(session, scan_id, principal.tenant_id)
|
||||
query = session.query(FileIntegrityFinding).filter(
|
||||
FileIntegrityFinding.scan_id == scan.id,
|
||||
FileIntegrityFinding.tenant_id == principal.tenant_id,
|
||||
)
|
||||
if state_filter:
|
||||
query = query.filter(FileIntegrityFinding.state == state_filter)
|
||||
rows = (
|
||||
query.order_by(
|
||||
FileIntegrityFinding.created_at.asc(),
|
||||
FileIntegrityFinding.id.asc(),
|
||||
)
|
||||
.limit(limit)
|
||||
.all()
|
||||
)
|
||||
return FileIntegrityFindingsResponse(
|
||||
findings=[_finding_response(row) for row in rows]
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/findings/{finding_id}/recheck",
|
||||
response_model=FileIntegrityActionResponse,
|
||||
)
|
||||
def recheck_finding(
|
||||
finding_id: str,
|
||||
payload: FileIntegrityActionRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityActionResponse:
|
||||
finding = _finding_for_tenant(session, finding_id, principal.tenant_id)
|
||||
try:
|
||||
result = recheck_integrity_finding(
|
||||
session,
|
||||
finding,
|
||||
user_id=principal.user.id,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
_audit_integrity_action(session, principal, result)
|
||||
session.commit()
|
||||
return _action_response(result)
|
||||
except (FileStorageError, StorageBackendError) as exc:
|
||||
session.rollback()
|
||||
raise _integrity_http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/findings/{finding_id}/cleanup",
|
||||
response_model=FileIntegrityActionResponse,
|
||||
)
|
||||
def cleanup_finding(
|
||||
finding_id: str,
|
||||
payload: FileIntegrityActionRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:admin")),
|
||||
) -> FileIntegrityActionResponse:
|
||||
finding = _finding_for_tenant(session, finding_id, principal.tenant_id)
|
||||
try:
|
||||
result = cleanup_orphan_finding(
|
||||
session,
|
||||
finding,
|
||||
user_id=principal.user.id,
|
||||
dry_run=payload.dry_run,
|
||||
)
|
||||
_audit_integrity_action(session, principal, result)
|
||||
session.commit()
|
||||
return _action_response(result)
|
||||
except (FileStorageError, StorageBackendError) as exc:
|
||||
session.rollback()
|
||||
raise _integrity_http_error(exc) from exc
|
||||
|
||||
|
||||
def _scan_for_tenant(
|
||||
session: Session,
|
||||
scan_id: str,
|
||||
tenant_id: str,
|
||||
) -> FileIntegrityScan:
|
||||
scan = session.get(FileIntegrityScan, scan_id)
|
||||
if scan is None or scan.tenant_id != tenant_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Integrity scan not found",
|
||||
)
|
||||
return scan
|
||||
|
||||
|
||||
def _finding_for_tenant(
|
||||
session: Session,
|
||||
finding_id: str,
|
||||
tenant_id: str,
|
||||
) -> FileIntegrityFinding:
|
||||
finding = session.get(FileIntegrityFinding, finding_id)
|
||||
if finding is None or finding.tenant_id != tenant_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Integrity finding not found",
|
||||
)
|
||||
return finding
|
||||
|
||||
|
||||
def _audit_integrity_action(session, principal, result) -> None:
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action=f"files.integrity.{result.action}",
|
||||
object_type="file_integrity_finding",
|
||||
object_id=result.finding.id,
|
||||
details={
|
||||
"scan_id": result.finding.scan_id,
|
||||
"kind": result.finding.kind,
|
||||
"dry_run": result.dry_run,
|
||||
"changed": result.changed,
|
||||
"storage_key_sha256": hashlib.sha256(
|
||||
result.finding.storage_key.encode("utf-8")
|
||||
).hexdigest(),
|
||||
"inspection": result.inspection.kind
|
||||
if result.inspection
|
||||
else None,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _scan_response(scan: FileIntegrityScan) -> FileIntegrityScanResponse:
|
||||
return FileIntegrityScanResponse(
|
||||
id=scan.id,
|
||||
tenant_id=scan.tenant_id,
|
||||
storage_backend=scan.storage_backend,
|
||||
storage_prefix=scan.storage_prefix,
|
||||
status=scan.status,
|
||||
phase=scan.phase,
|
||||
verify_checksums=scan.verify_checksums,
|
||||
batch_size=scan.batch_size,
|
||||
scanned_blob_count=scan.scanned_blob_count,
|
||||
verified_blob_count=scan.verified_blob_count,
|
||||
quarantined_blob_count=scan.quarantined_blob_count,
|
||||
scanned_object_count=scan.scanned_object_count,
|
||||
orphan_object_count=scan.orphan_object_count,
|
||||
created_by_user_id=scan.created_by_user_id,
|
||||
started_at=scan.started_at.isoformat() if scan.started_at else None,
|
||||
completed_at=scan.completed_at.isoformat()
|
||||
if scan.completed_at
|
||||
else None,
|
||||
last_error=scan.last_error,
|
||||
created_at=scan.created_at.isoformat(),
|
||||
updated_at=scan.updated_at.isoformat(),
|
||||
)
|
||||
|
||||
|
||||
def _finding_response(
|
||||
finding: FileIntegrityFinding,
|
||||
) -> FileIntegrityFindingResponse:
|
||||
return FileIntegrityFindingResponse(
|
||||
id=finding.id,
|
||||
scan_id=finding.scan_id,
|
||||
tenant_id=finding.tenant_id,
|
||||
kind=finding.kind,
|
||||
state=finding.state,
|
||||
blob_id=finding.blob_id,
|
||||
storage_key=finding.storage_key,
|
||||
expected_size_bytes=finding.expected_size_bytes,
|
||||
observed_size_bytes=finding.observed_size_bytes,
|
||||
expected_checksum_sha256=finding.expected_checksum_sha256,
|
||||
observed_checksum_sha256=finding.observed_checksum_sha256,
|
||||
resolved_at=finding.resolved_at.isoformat()
|
||||
if finding.resolved_at
|
||||
else None,
|
||||
resolved_by_user_id=finding.resolved_by_user_id,
|
||||
created_at=finding.created_at.isoformat(),
|
||||
updated_at=finding.updated_at.isoformat(),
|
||||
)
|
||||
|
||||
|
||||
def _action_response(result) -> FileIntegrityActionResponse:
|
||||
return FileIntegrityActionResponse(
|
||||
action=result.action,
|
||||
changed=result.changed,
|
||||
dry_run=result.dry_run,
|
||||
finding=_finding_response(result.finding),
|
||||
inspection_kind=result.inspection.kind if result.inspection else None,
|
||||
inspection_valid=result.inspection.valid if result.inspection else None,
|
||||
)
|
||||
|
||||
|
||||
def _integrity_http_error(exc: Exception) -> HTTPException:
|
||||
if isinstance(exc, FileStorageError):
|
||||
return HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=str(exc),
|
||||
)
|
||||
return HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="The configured file storage backend could not complete the integrity operation",
|
||||
)
|
||||
167
src/govoplan_files/backend/routes/listing.py
Normal file
167
src/govoplan_files/backend/routes/listing.py
Normal file
@@ -0,0 +1,167 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileDeltaResponse,
|
||||
FileListResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.files import (
|
||||
count_assets_for_user,
|
||||
list_assets_for_user,
|
||||
list_assets_for_user_window,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_list_response,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_list_owner_access,
|
||||
_is_admin,
|
||||
)
|
||||
from govoplan_files.backend.services.list_queries import (
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
_cursor_page_size,
|
||||
_file_cursor_values,
|
||||
_files_delta_response,
|
||||
_files_delta_watermark,
|
||||
_files_list_fingerprint,
|
||||
_full_file_delta_response,
|
||||
_next_file_list_cursor,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/delta", response_model=FileDeltaResponse)
|
||||
def files_delta(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
since: str | None = None,
|
||||
limit: int = Query(default=500, ge=1, le=1000),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, owner_type, owner_id)
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
if since is None:
|
||||
return _full_file_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
return _files_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
since=since,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
|
||||
@router.get("", response_model=FileListResponse)
|
||||
def list_files(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
page_size: int | None = Query(default=None, ge=1, le=1000),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, owner_type, owner_id)
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
watermark = _files_delta_watermark(session, principal.tenant_id)
|
||||
total = count_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
effective_page_size = _cursor_page_size(FILES_LIST_CURSOR_SCOPE, cursor, page_size)
|
||||
if effective_page_size is not None:
|
||||
fingerprint = _files_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=effective_page_size,
|
||||
)
|
||||
after_display_path, after_updated_at, after_id = _file_cursor_values(
|
||||
cursor, fingerprint=fingerprint
|
||||
)
|
||||
assets, has_more = list_assets_for_user_window(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
page_size=effective_page_size,
|
||||
after_display_path=after_display_path,
|
||||
after_updated_at=after_updated_at,
|
||||
after_id=after_id,
|
||||
)
|
||||
return FileListResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
total=total,
|
||||
cursor=cursor,
|
||||
next_cursor=_next_file_list_cursor(
|
||||
principal,
|
||||
assets,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=effective_page_size,
|
||||
has_more=has_more,
|
||||
),
|
||||
watermark=watermark,
|
||||
)
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileListResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
total=total,
|
||||
watermark=watermark,
|
||||
)
|
||||
330
src/govoplan_files/backend/routes/shares.py
Normal file
330
src/govoplan_files/backend/routes/shares.py
Normal file
@@ -0,0 +1,330 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.api.v1.schemas import (
|
||||
ReferenceOptionListResponse,
|
||||
ReferenceOptionResponse,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_core.audit.logging import audit_from_principal
|
||||
from govoplan_core.core.references import (
|
||||
access_scope_reference_page,
|
||||
access_scope_reference_provider_available,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.runtime import get_registry
|
||||
from govoplan_files.backend.schemas import (
|
||||
BulkFileShareRequest,
|
||||
BulkFileShareResponse,
|
||||
FileShareRequest,
|
||||
FileShareResponse,
|
||||
FileSharesResponse,
|
||||
)
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
current_file_share_for_target,
|
||||
get_asset_for_share_management,
|
||||
list_file_shares,
|
||||
revoke_file_share,
|
||||
share_file,
|
||||
share_files,
|
||||
)
|
||||
from govoplan_files.backend.route_support import (
|
||||
_file_share_response,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{file_id}/share-target-options",
|
||||
response_model=ReferenceOptionListResponse,
|
||||
)
|
||||
def search_share_targets(
|
||||
file_id: str,
|
||||
target_type: str,
|
||||
q: str = "",
|
||||
selected: list[str] = Query(default=[]),
|
||||
limit: int = Query(default=50, ge=1, le=200),
|
||||
cursor: str | None = None,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
) -> ReferenceOptionListResponse:
|
||||
if target_type not in {"user", "group"}:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail="Share target type must be user or group",
|
||||
)
|
||||
try:
|
||||
get_asset_for_share_management(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
registry = get_registry()
|
||||
page = access_scope_reference_page(
|
||||
registry,
|
||||
principal,
|
||||
scope_type=target_type,
|
||||
reference_kind="membership" if target_type == "user" else "group",
|
||||
query=q,
|
||||
selected_values=selected,
|
||||
limit=limit,
|
||||
cursor=cursor,
|
||||
administrative=True,
|
||||
session=session,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
return ReferenceOptionListResponse(
|
||||
options=[
|
||||
ReferenceOptionResponse(**option.to_dict()) for option in page.options
|
||||
],
|
||||
provider_available=access_scope_reference_provider_available(registry),
|
||||
next_cursor=page.next_cursor,
|
||||
has_more=page.has_more,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{file_id}/shares", response_model=FileSharesResponse)
|
||||
def list_shares(
|
||||
file_id: str,
|
||||
include_inactive: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_share_management(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileSharesResponse(
|
||||
shares=[
|
||||
_file_share_response(share)
|
||||
for share in list_file_shares(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset_id=asset.id,
|
||||
include_inactive=include_inactive,
|
||||
)
|
||||
]
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/{file_id}/shares", response_model=FileShareResponse)
|
||||
def create_share(
|
||||
file_id: str,
|
||||
payload: FileShareRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_share_management(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
previous = current_file_share_for_target(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset_id=asset.id,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
)
|
||||
previous_permission = previous.permission if previous else None
|
||||
previous_expiry = previous.expires_at if previous else None
|
||||
share = share_file(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset=asset,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
permission=payload.permission,
|
||||
user_id=principal.user.id,
|
||||
expires_at=payload.expires_at,
|
||||
)
|
||||
session.flush()
|
||||
action = _share_audit_action(
|
||||
existed=previous is not None,
|
||||
previous_permission=previous_permission,
|
||||
permission=share.permission,
|
||||
previous_expiry=previous_expiry,
|
||||
expiry=share.expires_at,
|
||||
)
|
||||
if action:
|
||||
_audit_share_change(session, principal, share, action=action)
|
||||
session.commit()
|
||||
return _file_share_response(share)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/bulk-shares", response_model=BulkFileShareResponse)
|
||||
def create_bulk_shares(
|
||||
payload: BulkFileShareRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
file_ids = list(dict.fromkeys(payload.file_ids))
|
||||
assets = [
|
||||
get_asset_for_share_management(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in file_ids
|
||||
]
|
||||
previous_by_asset = {
|
||||
asset.id: current_file_share_for_target(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset_id=asset.id,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
)
|
||||
for asset in assets
|
||||
}
|
||||
previous_values = {
|
||||
asset_id: (
|
||||
share.permission if share else None,
|
||||
share.expires_at if share else None,
|
||||
)
|
||||
for asset_id, share in previous_by_asset.items()
|
||||
}
|
||||
shares = share_files(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
assets=assets,
|
||||
target_type=payload.target_type,
|
||||
target_id=payload.target_id,
|
||||
permission=payload.permission,
|
||||
user_id=principal.user.id,
|
||||
expires_at=payload.expires_at,
|
||||
)
|
||||
session.flush()
|
||||
for share in shares:
|
||||
previous_permission, previous_expiry = previous_values[share.file_asset_id]
|
||||
action = _share_audit_action(
|
||||
existed=previous_by_asset[share.file_asset_id] is not None,
|
||||
previous_permission=previous_permission,
|
||||
permission=share.permission,
|
||||
previous_expiry=previous_expiry,
|
||||
expiry=share.expires_at,
|
||||
)
|
||||
if action:
|
||||
_audit_share_change(session, principal, share, action=action)
|
||||
session.commit()
|
||||
return BulkFileShareResponse(
|
||||
shared_count=len(shares),
|
||||
shares=[_file_share_response(share) for share in shares],
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete("/{file_id}/shares/{share_id}", response_model=FileShareResponse)
|
||||
def revoke_share(
|
||||
file_id: str,
|
||||
share_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:share")),
|
||||
):
|
||||
try:
|
||||
asset = get_asset_for_share_management(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
share, changed = revoke_file_share(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
asset_id=asset.id,
|
||||
share_id=share_id,
|
||||
user_id=principal.user.id,
|
||||
)
|
||||
if changed:
|
||||
_audit_share_change(
|
||||
session, principal, share, action="files.share.revoked"
|
||||
)
|
||||
session.commit()
|
||||
return _file_share_response(share)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
def _share_audit_action(
|
||||
*,
|
||||
existed: bool,
|
||||
previous_permission: str | None,
|
||||
permission: str,
|
||||
previous_expiry: datetime | None,
|
||||
expiry: datetime | None,
|
||||
) -> str | None:
|
||||
if not existed:
|
||||
return "files.share.granted"
|
||||
permission_changed = previous_permission != permission
|
||||
expiry_changed = _normalized_expiry(previous_expiry) != _normalized_expiry(expiry)
|
||||
if permission_changed:
|
||||
return "files.share.changed"
|
||||
if expiry_changed:
|
||||
return "files.share.expiry_changed"
|
||||
return None
|
||||
|
||||
|
||||
def _audit_share_change(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
share,
|
||||
*,
|
||||
action: str,
|
||||
) -> None:
|
||||
audit_from_principal(
|
||||
session,
|
||||
principal,
|
||||
action=action,
|
||||
object_type="file_share",
|
||||
object_id=share.id,
|
||||
details={
|
||||
"file_asset_id": share.file_asset_id,
|
||||
"target_type": share.target_type,
|
||||
"target_id": share.target_id,
|
||||
"permission": share.permission,
|
||||
"expires_at": _normalized_expiry(share.expires_at),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _normalized_expiry(value: datetime | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
if value.tzinfo is None:
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc).isoformat()
|
||||
292
src/govoplan_files/backend/routes/spaces.py
Normal file
292
src/govoplan_files/backend/routes/spaces.py
Normal file
@@ -0,0 +1,292 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorSpaceCreateRequest,
|
||||
FileConnectorSpaceResponse,
|
||||
FileConnectorSpacesResponse,
|
||||
FileConnectorSpaceUpdateRequest,
|
||||
FileSpaceResponse,
|
||||
FileSpacesResponse,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.access import group_refs_for_ids, user_group_ids
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_spaces import (
|
||||
connector_space_owner_id,
|
||||
create_connector_space,
|
||||
get_connector_space_for_user,
|
||||
list_connector_spaces_for_user,
|
||||
soft_delete_connector_space,
|
||||
update_connector_space,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
_connector_policy_error,
|
||||
_connector_space_file_space_response,
|
||||
_connector_space_policy_decision,
|
||||
_connector_space_response,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_record_connector_settings_change,
|
||||
_visible_connector_profile,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.get("/spaces", response_model=FileSpacesResponse)
|
||||
def list_file_spaces(
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
spaces = [
|
||||
FileSpaceResponse(
|
||||
id=f"user:{principal.user.id}",
|
||||
label="My files",
|
||||
owner_type="user",
|
||||
owner_id=principal.user.id,
|
||||
description="Files owned by your user account.",
|
||||
)
|
||||
]
|
||||
group_ids = user_group_ids(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
include_admin_groups=_is_admin(principal),
|
||||
)
|
||||
if group_ids:
|
||||
groups = group_refs_for_ids(tenant_id=principal.tenant_id, group_ids=group_ids)
|
||||
spaces.extend(
|
||||
FileSpaceResponse(
|
||||
id=f"group:{group.id}",
|
||||
label=f"{group.name} files",
|
||||
owner_type="group",
|
||||
owner_id=group.id,
|
||||
description="Files owned by this group.",
|
||||
)
|
||||
for group in groups
|
||||
)
|
||||
connector_spaces = list_connector_spaces_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
spaces.extend(
|
||||
_connector_space_file_space_response(space) for space in connector_spaces
|
||||
)
|
||||
return FileSpacesResponse(spaces=spaces)
|
||||
|
||||
|
||||
@router.get("/connector-spaces", response_model=FileConnectorSpacesResponse)
|
||||
def list_file_connector_spaces(
|
||||
owner_type: Literal["user", "group"] | None = None,
|
||||
owner_id: str | None = None,
|
||||
include_inactive: bool = False,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
try:
|
||||
spaces = list_connector_spaces_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive and _is_admin(principal),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
return FileConnectorSpacesResponse(
|
||||
spaces=[_connector_space_response(space) for space in spaces]
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post(
|
||||
"/connector-spaces",
|
||||
response_model=FileConnectorSpaceResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_file_connector_space(
|
||||
payload: FileConnectorSpaceCreateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
if payload.owner_type == "group" and not payload.owner_id:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="owner_id is required for group connector spaces",
|
||||
)
|
||||
target_owner = payload.owner_id or principal.user.id
|
||||
try:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, payload.connector_profile_id
|
||||
)
|
||||
decision = _connector_space_policy_decision(
|
||||
profile,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
operation="link",
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
space = create_connector_space(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
profile=profile,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
sync_mode=payload.sync_mode,
|
||||
metadata=payload.metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="created",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.patch("/connector-spaces/{space_id}", response_model=FileConnectorSpaceResponse)
|
||||
def update_file_connector_space(
|
||||
space_id: str,
|
||||
payload: FileConnectorSpaceUpdateRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
space = get_connector_space_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
space_id=space_id,
|
||||
include_inactive=_is_admin(principal),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
try:
|
||||
if payload.library_id is not None or payload.remote_path is not None:
|
||||
profile = _visible_connector_profile(
|
||||
session, principal, space.connector_profile_id
|
||||
)
|
||||
decision = _connector_space_policy_decision(
|
||||
profile,
|
||||
library_id=payload.library_id
|
||||
if payload.library_id is not None
|
||||
else space.library_id,
|
||||
remote_path=payload.remote_path
|
||||
if payload.remote_path is not None
|
||||
else space.remote_path,
|
||||
operation="link",
|
||||
)
|
||||
if not decision.allowed:
|
||||
raise ConnectorPolicyDenied(decision)
|
||||
update_connector_space(
|
||||
session,
|
||||
space,
|
||||
user_id=principal.user.id,
|
||||
label=payload.label,
|
||||
library_id=payload.library_id,
|
||||
remote_path=payload.remote_path,
|
||||
sync_mode=payload.sync_mode,
|
||||
is_active=payload.is_active,
|
||||
metadata=payload.metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="updated",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, ValueError, json.JSONDecodeError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/connector-spaces/{space_id}", response_model=FileConnectorSpaceResponse
|
||||
)
|
||||
def delete_file_connector_space(
|
||||
space_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
space = get_connector_space_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
space_id=space_id,
|
||||
include_inactive=True,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
soft_delete_connector_space(
|
||||
session, space, user_id=principal.user.id, is_admin=_is_admin(principal)
|
||||
)
|
||||
_record_connector_settings_change(
|
||||
session,
|
||||
collection=FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
resource_type=FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
resource_id=space.id,
|
||||
operation="deleted",
|
||||
principal=principal,
|
||||
tenant_id=space.tenant_id,
|
||||
payload={
|
||||
"owner_type": space.owner_type,
|
||||
"owner_id": connector_space_owner_id(space),
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
return _connector_space_response(space)
|
||||
except FileStorageError as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc, not_found=True) from exc
|
||||
213
src/govoplan_files/backend/routes/transfers.py
Normal file
213
src/govoplan_files/backend/routes/transfers.py
Normal file
@@ -0,0 +1,213 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import FileResponse
|
||||
from starlette.background import BackgroundTask
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
ArchiveRequest,
|
||||
PatternMatchResponse,
|
||||
PatternResolveRequest,
|
||||
PatternResolveResponse,
|
||||
RenamePreviewItem,
|
||||
RenameRequest,
|
||||
RenameResponse,
|
||||
TransferRequest,
|
||||
TransferResponse,
|
||||
_conflict_resolutions,
|
||||
)
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.storage.paths import (
|
||||
UnsafeFilePathError,
|
||||
filename_from_path,
|
||||
normalize_logical_path,
|
||||
)
|
||||
from govoplan_files.backend.storage.archives import create_zip_file
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
list_assets_for_user,
|
||||
)
|
||||
from govoplan_files.backend.storage.search import resolve_patterns
|
||||
from govoplan_files.backend.storage.transfers import (
|
||||
rename_selection,
|
||||
transfer_selection,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_attachment_disposition,
|
||||
_audit_connector_access,
|
||||
_cleanup_temp_file,
|
||||
_ensure_campaign_file_access,
|
||||
_ensure_list_owner_access,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post("/bulk-rename", response_model=RenameResponse)
|
||||
def bulk_rename(
|
||||
payload: RenameRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
plan = rename_selection(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
file_ids=payload.file_ids,
|
||||
folder_paths=payload.folder_paths,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
mode=payload.mode,
|
||||
new_name=payload.new_name,
|
||||
find=payload.find,
|
||||
replacement=payload.replacement,
|
||||
prefix=payload.prefix,
|
||||
suffix=payload.suffix,
|
||||
recursive=payload.recursive,
|
||||
dry_run=payload.dry_run,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if not payload.dry_run:
|
||||
session.commit()
|
||||
return RenameResponse(
|
||||
dry_run=payload.dry_run,
|
||||
items=[
|
||||
RenamePreviewItem(
|
||||
kind=item.kind,
|
||||
id=item.id,
|
||||
file_id=item.id if item.kind == "file" else None,
|
||||
folder_path=item.old_path if item.kind == "folder" else None,
|
||||
old_path=item.old_path,
|
||||
new_path=item.new_path,
|
||||
)
|
||||
for item in plan
|
||||
],
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/transfer", response_model=TransferResponse)
|
||||
def transfer_files(
|
||||
payload: TransferRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:organize")),
|
||||
):
|
||||
try:
|
||||
files, folders = transfer_selection(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
operation=payload.operation,
|
||||
file_ids=payload.file_ids,
|
||||
folder_paths=payload.folder_paths,
|
||||
source_owner_type=payload.source_owner_type,
|
||||
source_owner_id=payload.source_owner_id,
|
||||
target_owner_type=payload.target_owner_type,
|
||||
target_owner_id=payload.target_owner_id,
|
||||
target_folder=payload.target_folder,
|
||||
conflict_strategy=payload.conflict_strategy,
|
||||
conflict_resolutions=_conflict_resolutions(payload.conflict_resolutions),
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
session.commit()
|
||||
return TransferResponse(
|
||||
operation=payload.operation, files=files, folders=folders
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
|
||||
|
||||
@router.post("/archive.zip")
|
||||
def download_archive(
|
||||
payload: ArchiveRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:download")),
|
||||
):
|
||||
try:
|
||||
assets = [
|
||||
get_asset_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
asset_id=file_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
for file_id in payload.file_ids
|
||||
]
|
||||
tmp = tempfile.NamedTemporaryFile(
|
||||
prefix="govoplan-files-", suffix=".zip", delete=False
|
||||
)
|
||||
tmp_path = tmp.name
|
||||
tmp.close()
|
||||
try:
|
||||
create_zip_file(session, assets, tmp_path)
|
||||
except Exception:
|
||||
_cleanup_temp_file(tmp_path)
|
||||
raise
|
||||
_audit_connector_access(session, principal, assets, operation="archive")
|
||||
except FileStorageError as exc:
|
||||
raise _http_error(exc) from exc
|
||||
filename = filename_from_path(
|
||||
normalize_logical_path(payload.filename, fallback_filename="files.zip")
|
||||
)
|
||||
headers = {"Content-Disposition": _attachment_disposition(filename)}
|
||||
return FileResponse(
|
||||
tmp_path,
|
||||
media_type="application/zip",
|
||||
headers=headers,
|
||||
background=BackgroundTask(_cleanup_temp_file, tmp_path),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/resolve-patterns", response_model=PatternResolveResponse)
|
||||
def resolve_file_patterns(
|
||||
payload: PatternResolveRequest,
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:read")),
|
||||
):
|
||||
_ensure_list_owner_access(session, principal, payload.owner_type, payload.owner_id)
|
||||
_ensure_campaign_file_access(session, principal, payload.campaign_id)
|
||||
try:
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=payload.owner_type,
|
||||
owner_id=payload.owner_id,
|
||||
campaign_id=payload.campaign_id,
|
||||
path_prefix=payload.path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
resolved, unmatched = resolve_patterns(
|
||||
assets,
|
||||
payload.patterns,
|
||||
base_path=payload.path_prefix,
|
||||
case_sensitive=payload.case_sensitive,
|
||||
)
|
||||
return PatternResolveResponse(
|
||||
patterns=[
|
||||
PatternMatchResponse(
|
||||
pattern=item.pattern,
|
||||
matches=[_asset_response(session, asset) for asset in item.matches],
|
||||
)
|
||||
for item in resolved
|
||||
],
|
||||
unmatched=[_asset_response(session, asset) for asset in unmatched]
|
||||
if payload.include_unmatched
|
||||
else [],
|
||||
)
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
raise _http_error(exc) from exc
|
||||
207
src/govoplan_files/backend/routes/uploads.py
Normal file
207
src/govoplan_files/backend/routes/uploads.py
Normal file
@@ -0,0 +1,207 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Literal
|
||||
from fastapi import APIRouter, Depends, File as FastAPIFile, Form, UploadFile
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal, require_scope
|
||||
from govoplan_files.backend.schemas import (
|
||||
ConflictResolutionRequest,
|
||||
FileUploadResponse,
|
||||
_conflict_resolutions,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileAsset
|
||||
from govoplan_core.db.session import get_session
|
||||
from govoplan_files.backend.runtime import settings
|
||||
from govoplan_files.backend.storage.paths import UnsafeFilePathError
|
||||
from govoplan_files.backend.storage.archives import extract_zip_upload
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorPolicyDenied,
|
||||
)
|
||||
from govoplan_files.backend.storage.files import (
|
||||
create_file_asset,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_response,
|
||||
_audit_connector_imports,
|
||||
_cleanup_temp_file,
|
||||
_connector_policy_error,
|
||||
_enforce_connector_policy,
|
||||
_http_error,
|
||||
_is_admin,
|
||||
_read_limited_upload,
|
||||
_source_metadata_from_form,
|
||||
_spool_limited_upload_to_temp,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/files", tags=["files"])
|
||||
|
||||
|
||||
@router.post("/upload", response_model=FileUploadResponse)
|
||||
def upload_files(
|
||||
files: list[UploadFile] = FastAPIFile(...),
|
||||
owner_type: Literal["user", "group"] = Form(default="user"),
|
||||
owner_id: str | None = Form(default=None),
|
||||
path: str = Form(default=""),
|
||||
campaign_id: str | None = Form(default=None),
|
||||
unpack_zip: bool = Form(default=False),
|
||||
conflict_strategy: Literal["reject", "overwrite", "rename"] = Form(
|
||||
default="reject"
|
||||
),
|
||||
conflict_resolutions_json: str | None = Form(default=None),
|
||||
source_provenance_json: str | None = Form(default=None),
|
||||
source_revision: str | None = Form(default=None),
|
||||
connector_policy_json: str | None = Form(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
target_owner = owner_id or principal.user.id
|
||||
uploaded_assets: list[FileAsset] = []
|
||||
try:
|
||||
raw_resolutions = (
|
||||
json.loads(conflict_resolutions_json) if conflict_resolutions_json else []
|
||||
)
|
||||
upload_resolutions = _conflict_resolutions(
|
||||
[ConflictResolutionRequest(**item) for item in raw_resolutions]
|
||||
)
|
||||
_enforce_connector_policy(
|
||||
source_provenance_json, connector_policy_json, operation="import"
|
||||
)
|
||||
metadata = _source_metadata_from_form(source_provenance_json, source_revision)
|
||||
for upload in files:
|
||||
filename = upload.filename or "file"
|
||||
content_type = upload.content_type or None
|
||||
upload_limit = (
|
||||
settings.file_upload_zip_max_bytes
|
||||
if unpack_zip and filename.lower().endswith(".zip")
|
||||
else settings.file_upload_max_bytes
|
||||
)
|
||||
if unpack_zip and filename.lower().endswith(".zip"):
|
||||
zip_path = _spool_limited_upload_to_temp(
|
||||
upload, max_bytes=upload_limit, suffix=".zip"
|
||||
)
|
||||
try:
|
||||
extracted = extract_zip_upload(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
zip_data=zip_path,
|
||||
folder=path,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
max_file_bytes=settings.file_upload_max_bytes,
|
||||
max_total_bytes=settings.file_upload_zip_max_bytes,
|
||||
)
|
||||
finally:
|
||||
_cleanup_temp_file(zip_path)
|
||||
uploaded_assets.extend(item.asset for item in extracted)
|
||||
continue
|
||||
data = _read_limited_upload(upload, max_bytes=upload_limit)
|
||||
stored = create_file_asset(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
filename=filename,
|
||||
data=data,
|
||||
folder=path,
|
||||
content_type=content_type,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
uploaded_assets.append(stored.asset)
|
||||
_audit_connector_imports(session, principal, uploaded_assets)
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
return FileUploadResponse(
|
||||
files=[
|
||||
_asset_response(session, asset, include_shares=True)
|
||||
for asset in uploaded_assets
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@router.post("/upload-zip", response_model=FileUploadResponse)
|
||||
def upload_zip(
|
||||
file: UploadFile = FastAPIFile(...),
|
||||
owner_type: Literal["user", "group"] = Form(default="user"),
|
||||
owner_id: str | None = Form(default=None),
|
||||
path: str = Form(default=""),
|
||||
campaign_id: str | None = Form(default=None),
|
||||
conflict_strategy: Literal["reject", "overwrite", "rename"] = Form(
|
||||
default="reject"
|
||||
),
|
||||
conflict_resolutions_json: str | None = Form(default=None),
|
||||
source_provenance_json: str | None = Form(default=None),
|
||||
source_revision: str | None = Form(default=None),
|
||||
connector_policy_json: str | None = Form(default=None),
|
||||
session: Session = Depends(get_session),
|
||||
principal: ApiPrincipal = Depends(require_scope("files:file:upload")),
|
||||
):
|
||||
target_owner = owner_id or principal.user.id
|
||||
zip_path: str | None = None
|
||||
try:
|
||||
raw_resolutions = (
|
||||
json.loads(conflict_resolutions_json) if conflict_resolutions_json else []
|
||||
)
|
||||
upload_resolutions = _conflict_resolutions(
|
||||
[ConflictResolutionRequest(**item) for item in raw_resolutions]
|
||||
)
|
||||
_enforce_connector_policy(
|
||||
source_provenance_json, connector_policy_json, operation="import"
|
||||
)
|
||||
metadata = _source_metadata_from_form(source_provenance_json, source_revision)
|
||||
zip_path = _spool_limited_upload_to_temp(
|
||||
file, max_bytes=settings.file_upload_zip_max_bytes, suffix=".zip"
|
||||
)
|
||||
extracted = extract_zip_upload(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=target_owner,
|
||||
user_id=principal.user.id,
|
||||
zip_data=zip_path,
|
||||
folder=path,
|
||||
campaign_id=campaign_id,
|
||||
conflict_strategy=conflict_strategy,
|
||||
conflict_resolutions=upload_resolutions,
|
||||
metadata=metadata,
|
||||
is_admin=_is_admin(principal),
|
||||
max_file_bytes=settings.file_upload_max_bytes,
|
||||
max_total_bytes=settings.file_upload_zip_max_bytes,
|
||||
)
|
||||
_audit_connector_imports(session, principal, [item.asset for item in extracted])
|
||||
session.commit()
|
||||
except ConnectorPolicyDenied as exc:
|
||||
session.rollback()
|
||||
raise _connector_policy_error(exc) from exc
|
||||
except (FileStorageError, UnsafeFilePathError, ValueError) as exc:
|
||||
session.rollback()
|
||||
raise _http_error(exc) from exc
|
||||
finally:
|
||||
if zip_path:
|
||||
_cleanup_temp_file(zip_path)
|
||||
return FileUploadResponse(
|
||||
files=[
|
||||
_asset_response(session, item.asset, include_shares=True)
|
||||
for item in extracted
|
||||
]
|
||||
)
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
@@ -73,11 +74,86 @@ class FileConnectorSpacesResponse(BaseModel):
|
||||
|
||||
class FileShareResponse(BaseModel):
|
||||
id: str
|
||||
file_asset_id: str
|
||||
target_type: str
|
||||
target_id: str
|
||||
permission: str
|
||||
created_by_user_id: str | None = None
|
||||
created_at: str
|
||||
expires_at: str | None = None
|
||||
revoked_at: str | None = None
|
||||
revoked_by_user_id: str | None = None
|
||||
active: bool
|
||||
|
||||
|
||||
class FileSharesResponse(BaseModel):
|
||||
shares: list[FileShareResponse] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FileIntegrityScanCreateRequest(BaseModel):
|
||||
verify_checksums: bool = True
|
||||
batch_size: int = Field(default=100, ge=1, le=1000)
|
||||
|
||||
|
||||
class FileIntegrityScanResponse(BaseModel):
|
||||
id: str
|
||||
tenant_id: str
|
||||
storage_backend: str
|
||||
storage_prefix: str
|
||||
status: str
|
||||
phase: str
|
||||
verify_checksums: bool
|
||||
batch_size: int
|
||||
scanned_blob_count: int
|
||||
verified_blob_count: int
|
||||
quarantined_blob_count: int
|
||||
scanned_object_count: int
|
||||
orphan_object_count: int
|
||||
created_by_user_id: str | None = None
|
||||
started_at: str | None = None
|
||||
completed_at: str | None = None
|
||||
last_error: str | None = None
|
||||
created_at: str
|
||||
updated_at: str
|
||||
|
||||
|
||||
class FileIntegrityScansResponse(BaseModel):
|
||||
scans: list[FileIntegrityScanResponse] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FileIntegrityFindingResponse(BaseModel):
|
||||
id: str
|
||||
scan_id: str
|
||||
tenant_id: str
|
||||
kind: str
|
||||
state: str
|
||||
blob_id: str | None = None
|
||||
storage_key: str
|
||||
expected_size_bytes: int | None = None
|
||||
observed_size_bytes: int | None = None
|
||||
expected_checksum_sha256: str | None = None
|
||||
observed_checksum_sha256: str | None = None
|
||||
resolved_at: str | None = None
|
||||
resolved_by_user_id: str | None = None
|
||||
created_at: str
|
||||
updated_at: str
|
||||
|
||||
|
||||
class FileIntegrityFindingsResponse(BaseModel):
|
||||
findings: list[FileIntegrityFindingResponse] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FileIntegrityActionRequest(BaseModel):
|
||||
dry_run: bool = True
|
||||
|
||||
|
||||
class FileIntegrityActionResponse(BaseModel):
|
||||
action: str
|
||||
changed: bool
|
||||
dry_run: bool
|
||||
finding: FileIntegrityFindingResponse
|
||||
inspection_kind: str | None = None
|
||||
inspection_valid: bool | None = None
|
||||
|
||||
|
||||
class FileSourceProvenance(BaseModel):
|
||||
@@ -154,6 +230,7 @@ class FileFolderDeleteResponse(BaseModel):
|
||||
|
||||
class FileListResponse(BaseModel):
|
||||
files: list[FileAssetResponse]
|
||||
total: int
|
||||
cursor: str | None = None
|
||||
next_cursor: str | None = None
|
||||
watermark: str | None = None
|
||||
@@ -452,6 +529,7 @@ class FileShareRequest(BaseModel):
|
||||
target_type: Literal["user", "group", "campaign", "tenant"]
|
||||
target_id: str
|
||||
permission: Literal["read", "write", "manage"] = "read"
|
||||
expires_at: datetime | None = None
|
||||
|
||||
|
||||
class BulkFileShareRequest(BaseModel):
|
||||
@@ -459,6 +537,7 @@ class BulkFileShareRequest(BaseModel):
|
||||
target_type: Literal["user", "group", "campaign", "tenant"]
|
||||
target_id: str
|
||||
permission: Literal["read", "write", "manage"] = "read"
|
||||
expires_at: datetime | None = None
|
||||
|
||||
|
||||
class BulkFileShareResponse(BaseModel):
|
||||
|
||||
1
src/govoplan_files/backend/services/__init__.py
Normal file
1
src/govoplan_files/backend/services/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Query and response assembly services used by Files routes."""
|
||||
292
src/govoplan_files/backend/services/connector_settings_delta.py
Normal file
292
src/govoplan_files/backend/services/connector_settings_delta.py
Normal file
@@ -0,0 +1,292 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal
|
||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||
from govoplan_core.core.change_sequence import (
|
||||
ChangeSequenceEntry,
|
||||
)
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_CONNECTOR_CREDENTIALS_COLLECTION,
|
||||
FILES_CONNECTOR_POLICIES_COLLECTION,
|
||||
FILES_CONNECTOR_PROFILES_COLLECTION,
|
||||
FILES_CONNECTOR_SPACES_COLLECTION,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileConnectorCredentialResponse,
|
||||
FileConnectorSettingsDeltaResponse,
|
||||
FileConnectorPolicyResponse,
|
||||
FileConnectorProfileResponse,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileConnectorSpace
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
ConnectorCredential,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||
from govoplan_files.backend.storage.connector_policy_store import (
|
||||
connector_policy_response,
|
||||
)
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
FILES_CONNECTOR_CREDENTIAL_RESOURCE,
|
||||
FILES_CONNECTOR_PROFILE_RESOURCE,
|
||||
FILES_CONNECTOR_SPACE_RESOURCE,
|
||||
_can_read_disabled_connector_profiles,
|
||||
_connector_deleted_item,
|
||||
_connector_space_response,
|
||||
_ensure_campaign_file_access,
|
||||
_file_connector_settings_response_watermark,
|
||||
_file_connector_settings_watermark,
|
||||
_visible_connector_credentials,
|
||||
_visible_connector_profiles,
|
||||
_visible_connector_spaces,
|
||||
)
|
||||
|
||||
|
||||
def _full_file_connector_settings_delta_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
provider: str | None,
|
||||
campaign_id: str | None,
|
||||
include_disabled: bool,
|
||||
include_inactive: bool,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> FileConnectorSettingsDeltaResponse:
|
||||
if campaign_id:
|
||||
_ensure_campaign_file_access(session, principal, campaign_id)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
credentials = _visible_connector_credentials(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
spaces = _visible_connector_spaces(
|
||||
session,
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive,
|
||||
)
|
||||
return FileConnectorSettingsDeltaResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in profiles
|
||||
],
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in credentials
|
||||
],
|
||||
spaces=[_connector_space_response(space) for space in spaces],
|
||||
policy=FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
),
|
||||
changed_sections=["profiles", "credentials", "spaces", "policy"],
|
||||
deleted=[],
|
||||
watermark=_file_connector_settings_watermark(
|
||||
session, tenant_id=principal.tenant_id
|
||||
),
|
||||
has_more=False,
|
||||
full=True,
|
||||
)
|
||||
|
||||
|
||||
def _changed_file_connector_setting_ids(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
) -> tuple[set[str], set[str], set[str], bool]:
|
||||
changed_profile_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_PROFILES_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_PROFILE_RESOURCE
|
||||
}
|
||||
changed_credential_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_CREDENTIALS_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_CREDENTIAL_RESOURCE
|
||||
}
|
||||
changed_space_ids = {
|
||||
entry.resource_id
|
||||
for entry in entries
|
||||
if entry.collection == FILES_CONNECTOR_SPACES_COLLECTION
|
||||
and entry.resource_type == FILES_CONNECTOR_SPACE_RESOURCE
|
||||
}
|
||||
policy_changed = any(
|
||||
entry.collection == FILES_CONNECTOR_POLICIES_COLLECTION for entry in entries
|
||||
)
|
||||
return (
|
||||
changed_profile_ids,
|
||||
changed_credential_ids,
|
||||
changed_space_ids,
|
||||
policy_changed,
|
||||
)
|
||||
|
||||
|
||||
def _file_connector_settings_changed_sections(
|
||||
*,
|
||||
changed_profile_ids: set[str],
|
||||
changed_credential_ids: set[str],
|
||||
changed_space_ids: set[str],
|
||||
policy_changed: bool,
|
||||
profiles: list[ConnectorProfile],
|
||||
) -> list[str]:
|
||||
changed_sections = []
|
||||
if changed_profile_ids or any(
|
||||
profile.credential_profile_id
|
||||
and profile.credential_profile_id in changed_credential_ids
|
||||
for profile in profiles
|
||||
):
|
||||
changed_sections.append("profiles")
|
||||
if changed_credential_ids:
|
||||
changed_sections.append("credentials")
|
||||
if changed_space_ids:
|
||||
changed_sections.append("spaces")
|
||||
if policy_changed:
|
||||
changed_sections.append("policy")
|
||||
return changed_sections
|
||||
|
||||
|
||||
def _file_connector_settings_deleted_items(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
*,
|
||||
visible_profiles: dict[str, ConnectorProfile],
|
||||
visible_credentials: dict[str, ConnectorCredential],
|
||||
visible_spaces: dict[str, FileConnectorSpace],
|
||||
) -> list[DeltaDeletedItem]:
|
||||
return [
|
||||
_connector_deleted_item(entry)
|
||||
for entry in entries
|
||||
if (
|
||||
entry.resource_type == FILES_CONNECTOR_PROFILE_RESOURCE
|
||||
and entry.resource_id not in visible_profiles
|
||||
)
|
||||
or (
|
||||
entry.resource_type == FILES_CONNECTOR_CREDENTIAL_RESOURCE
|
||||
and entry.resource_id not in visible_credentials
|
||||
)
|
||||
or (
|
||||
entry.resource_type == FILES_CONNECTOR_SPACE_RESOURCE
|
||||
and entry.resource_id not in visible_spaces
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def _incremental_file_connector_settings_delta_response(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
entries: list[ChangeSequenceEntry],
|
||||
has_more: bool,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
provider: str | None,
|
||||
campaign_id: str | None,
|
||||
include_disabled: bool,
|
||||
include_inactive: bool,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> FileConnectorSettingsDeltaResponse:
|
||||
changed_profile_ids, changed_credential_ids, changed_space_ids, policy_changed = (
|
||||
_changed_file_connector_setting_ids(entries)
|
||||
)
|
||||
profiles = _visible_connector_profiles(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
campaign_id=campaign_id,
|
||||
include_disabled=include_disabled
|
||||
and _can_read_disabled_connector_profiles(principal),
|
||||
include_admin_scopes=_can_read_disabled_connector_profiles(principal),
|
||||
include_effective_policy=False,
|
||||
)
|
||||
visible_profiles = {
|
||||
profile.id: profile
|
||||
for profile in profiles
|
||||
if profile.id in changed_profile_ids
|
||||
or (
|
||||
profile.credential_profile_id
|
||||
and profile.credential_profile_id in changed_credential_ids
|
||||
)
|
||||
}
|
||||
credentials = _visible_connector_credentials(
|
||||
session,
|
||||
principal,
|
||||
provider=provider,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
visible_credentials = {
|
||||
credential.id: credential
|
||||
for credential in credentials
|
||||
if credential.id in changed_credential_ids
|
||||
}
|
||||
spaces = _visible_connector_spaces(
|
||||
session,
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
include_inactive=include_inactive,
|
||||
)
|
||||
visible_spaces = {
|
||||
space.id: space for space in spaces if space.id in changed_space_ids
|
||||
}
|
||||
return FileConnectorSettingsDeltaResponse(
|
||||
profiles=[
|
||||
FileConnectorProfileResponse(**profile.to_response())
|
||||
for profile in visible_profiles.values()
|
||||
],
|
||||
credentials=[
|
||||
FileConnectorCredentialResponse(**credential.to_response())
|
||||
for credential in visible_credentials.values()
|
||||
],
|
||||
spaces=[_connector_space_response(space) for space in visible_spaces.values()],
|
||||
policy=FileConnectorPolicyResponse(
|
||||
**connector_policy_response(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
)
|
||||
if policy_changed
|
||||
else None,
|
||||
changed_sections=_file_connector_settings_changed_sections(
|
||||
changed_profile_ids=changed_profile_ids,
|
||||
changed_credential_ids=changed_credential_ids,
|
||||
changed_space_ids=changed_space_ids,
|
||||
policy_changed=policy_changed,
|
||||
profiles=profiles,
|
||||
),
|
||||
deleted=_file_connector_settings_deleted_items(
|
||||
entries,
|
||||
visible_profiles=visible_profiles,
|
||||
visible_credentials=visible_credentials,
|
||||
visible_spaces=visible_spaces,
|
||||
),
|
||||
watermark=_file_connector_settings_response_watermark(
|
||||
session, tenant_id=principal.tenant_id, entries=entries, has_more=has_more
|
||||
),
|
||||
has_more=has_more,
|
||||
full=False,
|
||||
)
|
||||
620
src/govoplan_files/backend/services/list_queries.py
Normal file
620
src/govoplan_files/backend/services/list_queries.py
Normal file
@@ -0,0 +1,620 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.auth import ApiPrincipal
|
||||
from govoplan_core.api.v1.schemas import DeltaDeletedItem
|
||||
from govoplan_core.core.change_sequence import (
|
||||
ChangeSequenceEntry,
|
||||
decode_sequence_watermark,
|
||||
encode_sequence_watermark,
|
||||
max_sequence_id,
|
||||
sequence_entries_since,
|
||||
sequence_watermark_is_expired,
|
||||
)
|
||||
from govoplan_core.core.pagination import (
|
||||
KeysetCursorError,
|
||||
decode_keyset_cursor,
|
||||
encode_keyset_cursor,
|
||||
keyset_query_fingerprint,
|
||||
)
|
||||
from govoplan_files.backend.change_tracking import (
|
||||
FILES_ASSETS_COLLECTION,
|
||||
FILES_FOLDERS_COLLECTION,
|
||||
FILES_MODULE_ID,
|
||||
)
|
||||
from govoplan_files.backend.schemas import (
|
||||
FileDeltaResponse,
|
||||
)
|
||||
from govoplan_files.backend.db.models import FileAsset, FileFolder, FileShare
|
||||
from govoplan_files.backend.storage.access import user_group_ids
|
||||
from govoplan_files.backend.storage.files import (
|
||||
list_assets_for_user,
|
||||
)
|
||||
from govoplan_files.backend.storage.folders import list_folders_for_user
|
||||
from govoplan_files.backend.storage.share_state import effective_file_share_clause
|
||||
|
||||
|
||||
from govoplan_files.backend.route_support import (
|
||||
_asset_list_response,
|
||||
_folder_response,
|
||||
_is_admin,
|
||||
)
|
||||
|
||||
_FILES_DELTA_COLLECTIONS = (FILES_ASSETS_COLLECTION, FILES_FOLDERS_COLLECTION)
|
||||
FILES_LIST_CURSOR_SCOPE = "files.list.v1"
|
||||
FOLDERS_LIST_CURSOR_SCOPE = "files.folders.list.v1"
|
||||
DEFAULT_FILE_LIST_PAGE_SIZE = 500
|
||||
|
||||
|
||||
def _cursor_http_error(exc: Exception) -> HTTPException:
|
||||
return HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc))
|
||||
|
||||
|
||||
def _cursor_page_size(
|
||||
scope: str, cursor: str | None, explicit_page_size: int | None
|
||||
) -> int | None:
|
||||
if explicit_page_size is not None:
|
||||
return explicit_page_size
|
||||
if not cursor:
|
||||
return DEFAULT_FILE_LIST_PAGE_SIZE
|
||||
try:
|
||||
values = decode_keyset_cursor(scope, cursor)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
raw_page_size = values.get("page_size")
|
||||
if not isinstance(raw_page_size, int) or raw_page_size < 1 or raw_page_size > 1000:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
return raw_page_size
|
||||
|
||||
|
||||
def _files_list_fingerprint(
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None,
|
||||
audit_relevant: bool | None,
|
||||
page_size: int,
|
||||
) -> str:
|
||||
return keyset_query_fingerprint(
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
{
|
||||
"tenant_id": principal.tenant_id,
|
||||
"actor": "admin" if _is_admin(principal) else principal.user.id,
|
||||
"owner_type": owner_type,
|
||||
"owner_id": owner_id,
|
||||
"campaign_id": campaign_id,
|
||||
"path_prefix": path_prefix or "",
|
||||
"campaign_usage": campaign_usage or "",
|
||||
"audit_relevant": audit_relevant,
|
||||
"sort": "display_path.asc,updated_at.desc,id.asc",
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _folders_list_fingerprint(
|
||||
principal: ApiPrincipal,
|
||||
*,
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int,
|
||||
) -> str:
|
||||
return keyset_query_fingerprint(
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
{
|
||||
"tenant_id": principal.tenant_id,
|
||||
"actor": "admin" if _is_admin(principal) else principal.user.id,
|
||||
"owner_type": owner_type,
|
||||
"owner_id": owner_id,
|
||||
"sort": "path.asc,id.asc",
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _file_cursor_values(
|
||||
cursor: str | None, *, fingerprint: str
|
||||
) -> tuple[str | None, datetime | None, str | None]:
|
||||
try:
|
||||
values = decode_keyset_cursor(
|
||||
FILES_LIST_CURSOR_SCOPE, cursor, fingerprint=fingerprint
|
||||
)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
if values is None:
|
||||
return None, None, None
|
||||
display_path = values.get("display_path")
|
||||
updated_at = values.get("updated_at")
|
||||
asset_id = values.get("id")
|
||||
if (
|
||||
not isinstance(display_path, str)
|
||||
or not isinstance(updated_at, str)
|
||||
or not isinstance(asset_id, str)
|
||||
):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
try:
|
||||
parsed_updated_at = datetime.fromisoformat(updated_at)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
) from exc
|
||||
return display_path, parsed_updated_at, asset_id
|
||||
|
||||
|
||||
def _folder_cursor_values(
|
||||
cursor: str | None, *, fingerprint: str
|
||||
) -> tuple[str | None, str | None]:
|
||||
try:
|
||||
values = decode_keyset_cursor(
|
||||
FOLDERS_LIST_CURSOR_SCOPE, cursor, fingerprint=fingerprint
|
||||
)
|
||||
except KeysetCursorError as exc:
|
||||
raise _cursor_http_error(exc) from exc
|
||||
if values is None:
|
||||
return None, None
|
||||
folder_path = values.get("path")
|
||||
folder_id = values.get("id")
|
||||
if not isinstance(folder_path, str) or not isinstance(folder_id, str):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid pagination cursor"
|
||||
)
|
||||
return folder_path, folder_id
|
||||
|
||||
|
||||
def _next_file_list_cursor(
|
||||
principal: ApiPrincipal,
|
||||
assets: list[FileAsset],
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
campaign_usage: Literal["linked", "unlinked"] | None,
|
||||
audit_relevant: bool | None,
|
||||
page_size: int,
|
||||
has_more: bool,
|
||||
) -> str | None:
|
||||
if not has_more or not assets:
|
||||
return None
|
||||
last = assets[-1]
|
||||
return encode_keyset_cursor(
|
||||
FILES_LIST_CURSOR_SCOPE,
|
||||
fingerprint=_files_list_fingerprint(
|
||||
principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
page_size=page_size,
|
||||
),
|
||||
values={
|
||||
"display_path": last.display_path,
|
||||
"updated_at": last.updated_at,
|
||||
"id": last.id,
|
||||
"page_size": page_size,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _next_folder_list_cursor(
|
||||
principal: ApiPrincipal,
|
||||
folders: list[FileFolder],
|
||||
*,
|
||||
owner_type: Literal["user", "group"],
|
||||
owner_id: str,
|
||||
page_size: int,
|
||||
has_more: bool,
|
||||
) -> str | None:
|
||||
if not has_more or not folders:
|
||||
return None
|
||||
last = folders[-1]
|
||||
return encode_keyset_cursor(
|
||||
FOLDERS_LIST_CURSOR_SCOPE,
|
||||
fingerprint=_folders_list_fingerprint(
|
||||
principal, owner_type=owner_type, owner_id=owner_id, page_size=page_size
|
||||
),
|
||||
values={"path": last.path, "id": last.id, "page_size": page_size},
|
||||
)
|
||||
|
||||
|
||||
def _files_delta_watermark(session: Session, tenant_id: str) -> str:
|
||||
return encode_sequence_watermark(
|
||||
max_sequence_id(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _full_file_delta_response(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> FileDeltaResponse:
|
||||
assets = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
folders = _visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
return FileDeltaResponse(
|
||||
files=_asset_list_response(session, assets, include_shares=True),
|
||||
folders=[_folder_response(folder) for folder in folders],
|
||||
deleted=[],
|
||||
watermark=_files_delta_watermark(session, principal.tenant_id),
|
||||
has_more=False,
|
||||
full=True,
|
||||
)
|
||||
|
||||
|
||||
def _visible_folders_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> list[FileFolder]:
|
||||
if not owner_type or not owner_id:
|
||||
return []
|
||||
folders = list_folders_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if not path_prefix:
|
||||
return folders
|
||||
normalized = path_prefix.strip().strip("/")
|
||||
if not normalized:
|
||||
return folders
|
||||
return [
|
||||
folder
|
||||
for folder in folders
|
||||
if folder.path == normalized or folder.path.startswith(f"{normalized}/")
|
||||
]
|
||||
|
||||
|
||||
def _entry_path_matches(
|
||||
entry_payload: dict[str, object], path_prefix: str | None
|
||||
) -> bool:
|
||||
if not path_prefix:
|
||||
return True
|
||||
normalized = path_prefix.strip().strip("/")
|
||||
if not normalized:
|
||||
return True
|
||||
for key in ("path", "previous_path"):
|
||||
value = entry_payload.get(key)
|
||||
if isinstance(value, str) and (
|
||||
value == normalized or value.startswith(f"{normalized}/")
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _entry_owner_matches(
|
||||
entry_payload: dict[str, object],
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
) -> bool:
|
||||
if not owner_type or not owner_id:
|
||||
return True
|
||||
return (
|
||||
entry_payload.get("owner_type") == owner_type
|
||||
and entry_payload.get("owner_id") == owner_id
|
||||
)
|
||||
|
||||
|
||||
def _entry_campaign_matches(session: Session, entry, campaign_id: str | None) -> bool:
|
||||
if not campaign_id:
|
||||
return True
|
||||
payload = entry.payload or {}
|
||||
if (
|
||||
payload.get("share_target_type") == "campaign"
|
||||
and payload.get("share_target_id") == campaign_id
|
||||
):
|
||||
return True
|
||||
if entry.resource_type != "file":
|
||||
return False
|
||||
return (
|
||||
session.query(FileShare)
|
||||
.filter(
|
||||
FileShare.tenant_id == entry.tenant_id,
|
||||
FileShare.file_asset_id == entry.resource_id,
|
||||
FileShare.target_type == "campaign",
|
||||
FileShare.target_id == campaign_id,
|
||||
effective_file_share_clause(),
|
||||
)
|
||||
.first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _principal_group_ids_for_delta(
|
||||
session: Session, principal: ApiPrincipal
|
||||
) -> set[str]:
|
||||
cache = session.info.setdefault("files_delta_group_ids", {})
|
||||
key = (principal.tenant_id, principal.user.id)
|
||||
if key not in cache:
|
||||
cache[key] = set(
|
||||
user_group_ids(
|
||||
session, tenant_id=principal.tenant_id, user_id=principal.user.id
|
||||
)
|
||||
)
|
||||
return cache[key]
|
||||
|
||||
|
||||
def _entry_subject_matches_principal(
|
||||
session: Session, principal: ApiPrincipal, entry_payload: dict[str, object]
|
||||
) -> bool:
|
||||
if _is_admin(principal):
|
||||
return True
|
||||
owner_type = entry_payload.get("owner_type")
|
||||
owner_id = entry_payload.get("owner_id")
|
||||
if owner_type == "user" and owner_id == principal.user.id:
|
||||
return True
|
||||
if owner_type == "group" and isinstance(owner_id, str):
|
||||
if owner_id in _principal_group_ids_for_delta(session, principal):
|
||||
return True
|
||||
share_target_type = entry_payload.get("share_target_type")
|
||||
share_target_id = entry_payload.get("share_target_id")
|
||||
if share_target_type == "user" and share_target_id == principal.user.id:
|
||||
return True
|
||||
if share_target_type == "tenant" and share_target_id == principal.tenant_id:
|
||||
return True
|
||||
if share_target_type == "group" and isinstance(share_target_id, str):
|
||||
if share_target_id in _principal_group_ids_for_delta(session, principal):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _entry_matches_delta_scope(
|
||||
session: Session,
|
||||
principal: ApiPrincipal,
|
||||
entry,
|
||||
*,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> bool:
|
||||
payload = entry.payload or {}
|
||||
if (
|
||||
not owner_type
|
||||
and not campaign_id
|
||||
and not _entry_subject_matches_principal(session, principal, payload)
|
||||
):
|
||||
return False
|
||||
return (
|
||||
_entry_owner_matches(payload, owner_type, owner_id)
|
||||
and _entry_path_matches(payload, path_prefix)
|
||||
and _entry_campaign_matches(session, entry, campaign_id)
|
||||
)
|
||||
|
||||
|
||||
def _decode_files_delta_watermark(since: str) -> int:
|
||||
try:
|
||||
return decode_sequence_watermark(since)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)
|
||||
) from exc
|
||||
|
||||
|
||||
def _changed_delta_resource_ids(
|
||||
entries: list[ChangeSequenceEntry],
|
||||
) -> tuple[list[str], list[str]]:
|
||||
file_ids = list(
|
||||
dict.fromkeys(
|
||||
entry.resource_id for entry in entries if entry.resource_type == "file"
|
||||
)
|
||||
)
|
||||
folder_ids = list(
|
||||
dict.fromkeys(
|
||||
entry.resource_id for entry in entries if entry.resource_type == "folder"
|
||||
)
|
||||
)
|
||||
return file_ids, folder_ids
|
||||
|
||||
|
||||
def _visible_assets_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
changed_file_ids: list[str],
|
||||
) -> dict[str, FileAsset]:
|
||||
return {
|
||||
asset.id: asset
|
||||
for asset in list_assets_for_user(
|
||||
session,
|
||||
tenant_id=principal.tenant_id,
|
||||
user_id=principal.user.id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
is_admin=_is_admin(principal),
|
||||
)
|
||||
if asset.id in changed_file_ids
|
||||
}
|
||||
|
||||
|
||||
def _changed_visible_folders_for_delta(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
path_prefix: str | None,
|
||||
changed_folder_ids: list[str],
|
||||
) -> dict[str, FileFolder]:
|
||||
return {
|
||||
folder.id: folder
|
||||
for folder in _visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
if folder.id in changed_folder_ids
|
||||
}
|
||||
|
||||
|
||||
def _deleted_delta_items(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
entries: list[ChangeSequenceEntry],
|
||||
visible_assets: dict[str, FileAsset],
|
||||
visible_folders: dict[str, FileFolder],
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
) -> list[DeltaDeletedItem]:
|
||||
deleted: dict[tuple[str, str], DeltaDeletedItem] = {}
|
||||
for entry in entries:
|
||||
if entry.resource_type == "file" and entry.resource_id in visible_assets:
|
||||
continue
|
||||
if entry.resource_type == "folder" and entry.resource_id in visible_folders:
|
||||
continue
|
||||
if not _entry_matches_delta_scope(
|
||||
session,
|
||||
principal,
|
||||
entry,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
):
|
||||
continue
|
||||
deleted[(entry.resource_type, entry.resource_id)] = DeltaDeletedItem(
|
||||
id=entry.resource_id,
|
||||
resource_type=entry.resource_type,
|
||||
revision=encode_sequence_watermark(entry.id),
|
||||
deleted_at=entry.created_at if entry.operation == "deleted" else None,
|
||||
)
|
||||
return list(deleted.values())
|
||||
|
||||
|
||||
def _files_delta_response(
|
||||
session: Session,
|
||||
*,
|
||||
principal: ApiPrincipal,
|
||||
owner_type: Literal["user", "group"] | None,
|
||||
owner_id: str | None,
|
||||
campaign_id: str | None,
|
||||
path_prefix: str | None,
|
||||
since: str,
|
||||
limit: int,
|
||||
) -> FileDeltaResponse:
|
||||
since_sequence = _decode_files_delta_watermark(since)
|
||||
if sequence_watermark_is_expired(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
):
|
||||
return _full_file_delta_response(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
|
||||
entries_plus_one = sequence_entries_since(
|
||||
session,
|
||||
since=since_sequence,
|
||||
tenant_id=principal.tenant_id,
|
||||
module_id=FILES_MODULE_ID,
|
||||
collections=_FILES_DELTA_COLLECTIONS,
|
||||
limit=limit + 1,
|
||||
)
|
||||
has_more = len(entries_plus_one) > limit
|
||||
entries = entries_plus_one[:limit]
|
||||
|
||||
changed_file_ids, changed_folder_ids = _changed_delta_resource_ids(entries)
|
||||
visible_assets = _visible_assets_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
changed_file_ids=changed_file_ids,
|
||||
)
|
||||
visible_folders = _changed_visible_folders_for_delta(
|
||||
session,
|
||||
principal=principal,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
path_prefix=path_prefix,
|
||||
changed_folder_ids=changed_folder_ids,
|
||||
)
|
||||
deleted = _deleted_delta_items(
|
||||
session,
|
||||
principal=principal,
|
||||
entries=entries,
|
||||
visible_assets=visible_assets,
|
||||
visible_folders=visible_folders,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
)
|
||||
|
||||
watermark = (
|
||||
encode_sequence_watermark(entries[-1].id)
|
||||
if has_more and entries
|
||||
else _files_delta_watermark(session, principal.tenant_id)
|
||||
)
|
||||
return FileDeltaResponse(
|
||||
files=_asset_list_response(
|
||||
session, list(visible_assets.values()), include_shares=True
|
||||
),
|
||||
folders=[_folder_response(folder) for folder in visible_folders.values()],
|
||||
deleted=deleted,
|
||||
watermark=watermark,
|
||||
has_more=has_more,
|
||||
full=False,
|
||||
)
|
||||
@@ -17,6 +17,22 @@ class StorageBackendError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class StorageObjectMissing(StorageBackendError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StorageObjectInfo:
|
||||
key: str
|
||||
size_bytes: int
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StorageObjectPage:
|
||||
objects: tuple[StorageObjectInfo, ...]
|
||||
next_cursor: str | None = None
|
||||
|
||||
|
||||
class StorageBackend(Protocol):
|
||||
name: str
|
||||
|
||||
@@ -25,6 +41,14 @@ class StorageBackend(Protocol):
|
||||
def iter_bytes(self, key: str, *, chunk_size: int = 1024 * 1024) -> Iterable[bytes]: ...
|
||||
def delete(self, key: str) -> None: ...
|
||||
def exists(self, key: str) -> bool: ...
|
||||
def stat(self, key: str) -> StorageObjectInfo: ...
|
||||
def list_objects(
|
||||
self,
|
||||
*,
|
||||
prefix: str,
|
||||
after: str | None = None,
|
||||
limit: int = 500,
|
||||
) -> StorageObjectPage: ...
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -55,7 +79,7 @@ class LocalFilesystemStorageBackend:
|
||||
candidate = self._path_for_root(root, key)
|
||||
if candidate.exists() and candidate.is_file():
|
||||
return candidate
|
||||
raise StorageBackendError("Stored object does not exist")
|
||||
raise StorageObjectMissing("Stored object does not exist")
|
||||
|
||||
def put_bytes(self, key: str, data: bytes, *, content_type: str | None = None) -> None:
|
||||
path = self._path(key)
|
||||
@@ -86,6 +110,33 @@ class LocalFilesystemStorageBackend:
|
||||
return False
|
||||
return True
|
||||
|
||||
def stat(self, key: str) -> StorageObjectInfo:
|
||||
path = self._readable_path(key)
|
||||
return StorageObjectInfo(key=key, size_bytes=path.stat().st_size)
|
||||
|
||||
def list_objects(
|
||||
self,
|
||||
*,
|
||||
prefix: str,
|
||||
after: str | None = None,
|
||||
limit: int = 500,
|
||||
) -> StorageObjectPage:
|
||||
normalized_limit = max(1, min(int(limit), 5000))
|
||||
candidates: list[StorageObjectInfo] = []
|
||||
for path in _iter_local_files(self.root):
|
||||
key = path.relative_to(self.root).as_posix()
|
||||
if not key.startswith(prefix) or (after is not None and key <= after):
|
||||
continue
|
||||
candidates.append(StorageObjectInfo(key=key, size_bytes=path.stat().st_size))
|
||||
if len(candidates) > normalized_limit:
|
||||
break
|
||||
has_more = len(candidates) > normalized_limit
|
||||
page = tuple(candidates[:normalized_limit])
|
||||
return StorageObjectPage(
|
||||
objects=page,
|
||||
next_cursor=page[-1].key if has_more and page else None,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class S3StorageBackend:
|
||||
@@ -175,6 +226,52 @@ class S3StorageBackend:
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def stat(self, key: str) -> StorageObjectInfo:
|
||||
try:
|
||||
response = self.client.head_object(Bucket=self.bucket, Key=key)
|
||||
except Exception as exc: # pragma: no cover - depends on S3 backend
|
||||
if _s3_missing_error(exc):
|
||||
raise StorageObjectMissing("Stored object does not exist") from exc
|
||||
raise StorageBackendError(str(exc)) from exc
|
||||
try:
|
||||
size = int(response.get("ContentLength"))
|
||||
except (AttributeError, TypeError, ValueError) as exc:
|
||||
raise StorageBackendError("S3 object metadata did not include a valid size") from exc
|
||||
return StorageObjectInfo(key=key, size_bytes=size)
|
||||
|
||||
def list_objects(
|
||||
self,
|
||||
*,
|
||||
prefix: str,
|
||||
after: str | None = None,
|
||||
limit: int = 500,
|
||||
) -> StorageObjectPage:
|
||||
normalized_limit = max(1, min(int(limit), 1000))
|
||||
kwargs: dict[str, object] = {
|
||||
"Bucket": self.bucket,
|
||||
"Prefix": prefix,
|
||||
"MaxKeys": normalized_limit,
|
||||
}
|
||||
if after:
|
||||
kwargs["StartAfter"] = after
|
||||
try:
|
||||
response = self.client.list_objects_v2(**kwargs)
|
||||
except Exception as exc: # pragma: no cover - depends on S3 backend
|
||||
raise StorageBackendError(str(exc)) from exc
|
||||
objects = tuple(
|
||||
StorageObjectInfo(
|
||||
key=str(item["Key"]),
|
||||
size_bytes=int(item.get("Size") or 0),
|
||||
)
|
||||
for item in response.get("Contents", ())
|
||||
if isinstance(item, dict) and item.get("Key")
|
||||
)
|
||||
has_more = bool(response.get("IsTruncated"))
|
||||
return StorageObjectPage(
|
||||
objects=objects,
|
||||
next_cursor=objects[-1].key if has_more and objects else None,
|
||||
)
|
||||
|
||||
|
||||
def _reject_declared_object_size(obj: object, *, max_bytes: int) -> None:
|
||||
if not isinstance(obj, dict):
|
||||
@@ -187,6 +284,25 @@ def _reject_declared_object_size(obj: object, *, max_bytes: int) -> None:
|
||||
raise StorageBackendError(f"Stored object exceeds the deployment limit of {max_bytes} bytes")
|
||||
|
||||
|
||||
def _iter_local_files(root: Path):
|
||||
for entry in sorted(root.iterdir(), key=lambda item: item.name):
|
||||
if entry.is_dir():
|
||||
yield from _iter_local_files(entry)
|
||||
elif entry.is_file():
|
||||
yield entry
|
||||
|
||||
|
||||
def _s3_missing_error(exc: Exception) -> bool:
|
||||
response = getattr(exc, "response", None)
|
||||
if not isinstance(response, dict):
|
||||
return False
|
||||
error = response.get("Error")
|
||||
metadata = response.get("ResponseMetadata")
|
||||
code = str(error.get("Code") if isinstance(error, dict) else "")
|
||||
status_code = metadata.get("HTTPStatusCode") if isinstance(metadata, dict) else None
|
||||
return code in {"404", "NoSuchKey", "NotFound"} or status_code == 404
|
||||
|
||||
|
||||
def _fallback_roots() -> tuple[Path, ...]:
|
||||
raw = getattr(settings, "file_storage_local_fallback_roots", "") or ""
|
||||
return tuple(Path(item.strip()) for item in str(raw).split(",") if item.strip())
|
||||
|
||||
@@ -13,12 +13,17 @@ from typing import Any, Iterator
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_files.backend.db.models import FileAsset, FileBlob, FileShare, FileVersion
|
||||
from govoplan_files.backend.storage.backends import StorageBackend, StorageBackendError, get_storage_backend
|
||||
from govoplan_files.backend.storage.backends import StorageBackend, get_storage_backend
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.files import current_versions_and_blobs, get_asset_for_user, list_assets_for_user, share_files
|
||||
from govoplan_files.backend.storage.access import ensure_owner_access
|
||||
from govoplan_files.backend.storage.paths import normalize_folder, normalize_logical_path, safe_storage_component
|
||||
from govoplan_files.backend.storage.provenance import source_provenance_from_metadata, source_revision_from_metadata
|
||||
from govoplan_files.backend.storage.share_state import effective_file_share_clause
|
||||
from govoplan_files.backend.storage.integrity import (
|
||||
ensure_blob_is_readable,
|
||||
read_verified_blob_bytes,
|
||||
)
|
||||
|
||||
|
||||
MANAGED_SOURCE_PREFIX = "managed:"
|
||||
@@ -165,7 +170,7 @@ def _campaign_linked_asset_ids(
|
||||
FileShare.file_asset_id.in_(chunk),
|
||||
FileShare.target_type == "campaign",
|
||||
FileShare.target_id == campaign_id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
effective_file_share_clause(),
|
||||
)
|
||||
.all()
|
||||
)
|
||||
@@ -301,11 +306,9 @@ def _materialize_managed_asset(
|
||||
target = _safe_local_target(local_root, relative_path)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
version, blob = version_blobs[asset.id]
|
||||
ensure_blob_is_readable(blob)
|
||||
if include_bytes:
|
||||
try:
|
||||
data = backend.get_bytes(blob.storage_key) if backend else b""
|
||||
except StorageBackendError as exc:
|
||||
raise FileStorageError(str(exc)) from exc
|
||||
data = read_verified_blob_bytes(blob, backend=backend) if backend else b""
|
||||
target.write_bytes(data)
|
||||
else:
|
||||
target.touch()
|
||||
|
||||
@@ -5,8 +5,17 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import inspect
|
||||
|
||||
from govoplan_core.core.policy import normalize_policy_scope_type, policy_source_path
|
||||
from govoplan_core.security.credential_envelopes import (
|
||||
CredentialAccessContext,
|
||||
CredentialEnvelope,
|
||||
CredentialEnvelopeError,
|
||||
ResolvedCredentialEnvelope,
|
||||
list_credential_envelopes,
|
||||
resolve_credential_envelope,
|
||||
)
|
||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
@@ -14,6 +23,8 @@ from govoplan_files.backend.storage.connector_deployment import reject_api_contr
|
||||
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
||||
from govoplan_files.backend.storage.connector_profiles import supported_connector_providers
|
||||
|
||||
CORE_CREDENTIAL_ENVELOPE_PREFIX = "credential-envelope:"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ConnectorCredential:
|
||||
@@ -33,6 +44,7 @@ class ConnectorCredential:
|
||||
policy_sources: tuple[ConnectorPolicySource, ...] = ()
|
||||
metadata: Mapping[str, Any] | None = None
|
||||
source_kind: str = "database"
|
||||
has_secret: bool = False
|
||||
|
||||
@property
|
||||
def source_path(self) -> str:
|
||||
@@ -58,7 +70,7 @@ class ConnectorCredential:
|
||||
return True
|
||||
# Environment references are deliberately unavailable to API-managed
|
||||
# credential rows. Legacy rows remain visible but fail closed.
|
||||
return bool(self.secret_ref or self.password_value or self.token_value)
|
||||
return bool(self.has_secret or self.secret_ref or self.password_value or self.token_value)
|
||||
|
||||
def to_response(self) -> dict[str, Any]:
|
||||
return {
|
||||
@@ -85,7 +97,159 @@ def list_database_connector_credentials(
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
) -> list[ConnectorCredential]:
|
||||
return [connector_credential_from_row(row) for row in list_connector_credential_rows(session, tenant_id=tenant_id, include_disabled=include_disabled)]
|
||||
local = [
|
||||
connector_credential_from_row(row)
|
||||
for row in list_connector_credential_rows(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
)
|
||||
]
|
||||
return [
|
||||
*local,
|
||||
*list_reusable_connector_credentials(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
include_disabled=include_disabled,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def reusable_credential_reference(credential_id: str) -> str:
|
||||
return f"{CORE_CREDENTIAL_ENVELOPE_PREFIX}{credential_id}"
|
||||
|
||||
|
||||
def reusable_credential_id(credential_ref: str | None) -> str | None:
|
||||
if not credential_ref or not credential_ref.startswith(CORE_CREDENTIAL_ENVELOPE_PREFIX):
|
||||
return None
|
||||
value = credential_ref.removeprefix(CORE_CREDENTIAL_ENVELOPE_PREFIX).strip()
|
||||
return value or None
|
||||
|
||||
|
||||
def file_credential_context(
|
||||
*,
|
||||
tenant_id: str,
|
||||
profile_id: str | None = None,
|
||||
scope_type: str = "tenant",
|
||||
scope_id: str | None = None,
|
||||
administrative: bool = False,
|
||||
) -> CredentialAccessContext:
|
||||
return CredentialAccessContext(
|
||||
tenant_id=tenant_id,
|
||||
user_id=scope_id if scope_type == "user" else None,
|
||||
group_ids=frozenset({scope_id}) if scope_type == "group" and scope_id else frozenset(),
|
||||
target_scope_type=scope_type,
|
||||
target_scope_id=scope_id or tenant_id,
|
||||
module_id="files",
|
||||
server_ref=f"files:{profile_id}" if profile_id else None,
|
||||
administrative=administrative,
|
||||
)
|
||||
|
||||
|
||||
def list_reusable_connector_credentials(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
include_disabled: bool = False,
|
||||
) -> list[ConnectorCredential]:
|
||||
if not inspect(session.get_bind()).has_table(CredentialEnvelope.__tablename__):
|
||||
return []
|
||||
context = file_credential_context(tenant_id=tenant_id, administrative=True)
|
||||
return [
|
||||
connector_credential_from_envelope(row)
|
||||
for row in list_credential_envelopes(
|
||||
session,
|
||||
context=context,
|
||||
include_inactive=include_disabled,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def resolve_reusable_connector_credential(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
credential_ref: str,
|
||||
profile_id: str,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
) -> ConnectorCredential:
|
||||
credential_id = reusable_credential_id(credential_ref)
|
||||
if credential_id is None:
|
||||
raise FileStorageError("Reusable credential reference is invalid")
|
||||
try:
|
||||
resolved = resolve_credential_envelope(
|
||||
session,
|
||||
credential_id=credential_id,
|
||||
context=file_credential_context(
|
||||
tenant_id=tenant_id,
|
||||
profile_id=profile_id,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
),
|
||||
)
|
||||
except CredentialEnvelopeError as exc:
|
||||
raise FileStorageError("Reusable credential is unavailable to this file connection") from exc
|
||||
return connector_credential_from_resolved_envelope(
|
||||
resolved,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
)
|
||||
|
||||
|
||||
def connector_credential_from_envelope(row: CredentialEnvelope) -> ConnectorCredential:
|
||||
return ConnectorCredential(
|
||||
id=reusable_credential_reference(row.id),
|
||||
label=row.name,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
enabled=row.is_active,
|
||||
credential_mode=_envelope_credential_mode(row.credential_kind, row.secret_keys),
|
||||
username=_clean_public_value(row.public_data, "username"),
|
||||
source_kind="credential_envelope",
|
||||
has_secret=bool(row.secret_data_encrypted),
|
||||
metadata={
|
||||
"credential_envelope_id": row.id,
|
||||
"credential_kind": row.credential_kind,
|
||||
"allowed_modules": list(row.allowed_modules or []),
|
||||
"allowed_server_refs": list(row.allowed_server_refs or []),
|
||||
"inherit_to_lower_scopes": bool(row.inherit_to_lower_scopes),
|
||||
"revision": row.revision,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def connector_credential_from_resolved_envelope(
|
||||
row: ResolvedCredentialEnvelope,
|
||||
*,
|
||||
scope_type: str,
|
||||
scope_id: str | None,
|
||||
) -> ConnectorCredential:
|
||||
return ConnectorCredential(
|
||||
id=reusable_credential_reference(row.id),
|
||||
label=row.name,
|
||||
scope_type=scope_type,
|
||||
scope_id=scope_id,
|
||||
enabled=True,
|
||||
credential_mode=_envelope_credential_mode(row.credential_kind, row.secret_data),
|
||||
username=_clean_public_value(row.public_data, "username"),
|
||||
password_value=_first_secret(row.secret_data, "password", "secret"),
|
||||
token_value=_first_secret(
|
||||
row.secret_data,
|
||||
"access_token",
|
||||
"bearer_token",
|
||||
"token",
|
||||
"api_key",
|
||||
),
|
||||
source_kind="credential_envelope",
|
||||
has_secret=bool(row.secret_data),
|
||||
metadata={
|
||||
"credential_envelope_id": row.id,
|
||||
"credential_kind": row.credential_kind,
|
||||
"inherit_to_lower_scopes": True,
|
||||
"revision": row.revision,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def list_connector_credential_rows(
|
||||
@@ -348,6 +512,27 @@ def _policy_source_response(source: ConnectorPolicySource) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _envelope_credential_mode(kind: str, secret_values: Mapping[str, Any] | list[str]) -> str:
|
||||
keys = {str(key) for key in secret_values}
|
||||
if kind in {"token", "oauth2", "api_key"} or keys.intersection(
|
||||
{"access_token", "bearer_token", "token", "api_key"}
|
||||
):
|
||||
return "token"
|
||||
return "basic"
|
||||
|
||||
|
||||
def _clean_public_value(values: Mapping[str, Any] | None, key: str) -> str | None:
|
||||
return _clean((values or {}).get(key))
|
||||
|
||||
|
||||
def _first_secret(values: Mapping[str, Any], *keys: str) -> str | None:
|
||||
for key in keys:
|
||||
value = _clean(values.get(key))
|
||||
if value is not None:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def _clean(value: object) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
@@ -246,11 +246,11 @@ def _applied_fields(policy: Mapping[str, Any]) -> tuple[str, ...]:
|
||||
|
||||
def _matches_field(request: ConnectorAccessRequest, field: str, patterns: list[str]) -> bool:
|
||||
if field == "connectors":
|
||||
return _matches_exact(request.connector_id, patterns)
|
||||
return _matches_reference(request.connector_id, patterns)
|
||||
if field == "credentials":
|
||||
return _matches_exact(request.credential_id, patterns)
|
||||
return _matches_reference(request.credential_id, patterns)
|
||||
if field == "providers":
|
||||
return _matches_exact(request.provider, patterns)
|
||||
return _matches_reference(request.provider, patterns)
|
||||
if field == "external_ids":
|
||||
return _matches_glob(request.external_id, patterns)
|
||||
if field == "external_paths":
|
||||
@@ -260,11 +260,11 @@ def _matches_field(request: ConnectorAccessRequest, field: str, patterns: list[s
|
||||
return False
|
||||
|
||||
|
||||
def _matches_exact(value: str | None, patterns: list[str]) -> bool:
|
||||
def _matches_reference(value: str | None, patterns: list[str]) -> bool:
|
||||
if value is None:
|
||||
return False
|
||||
clean = value.casefold()
|
||||
return any(pattern == "*" or clean == pattern.casefold() for pattern in patterns)
|
||||
return any(fnmatchcase(clean, pattern.casefold()) for pattern in patterns)
|
||||
|
||||
|
||||
def _matches_glob(value: str | None, patterns: list[str]) -> bool:
|
||||
|
||||
@@ -10,7 +10,13 @@ from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.connector_deployment import reject_api_controlled_deployment_references
|
||||
from govoplan_files.backend.storage.connector_credential_store import connector_credential_from_row, credential_rows_by_id
|
||||
from govoplan_files.backend.storage.connector_credential_store import (
|
||||
ConnectorCredential,
|
||||
connector_credential_from_row,
|
||||
credential_rows_by_id,
|
||||
resolve_reusable_connector_credential,
|
||||
reusable_credential_id,
|
||||
)
|
||||
from govoplan_files.backend.storage.connector_policy import connector_policy_sources_from_payload
|
||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, supported_connector_providers
|
||||
|
||||
@@ -50,10 +56,28 @@ def select_database_connector_profiles(
|
||||
profile_ids = {row.id for row in rows}
|
||||
if row_visible is not None:
|
||||
rows = [row for row in rows if row_visible(row)]
|
||||
credential_ids = {_clean(row.credential_profile_id) for row in rows if _clean(row.credential_profile_id)}
|
||||
credential_ids = {
|
||||
_clean(row.credential_profile_id)
|
||||
for row in rows
|
||||
if _clean(row.credential_profile_id) and not reusable_credential_id(row.credential_profile_id)
|
||||
}
|
||||
credentials = credential_rows_by_id(session, tenant_id=tenant_id, credential_ids={item for item in credential_ids if item}, include_disabled=include_disabled)
|
||||
return (
|
||||
[connector_profile_from_row(row, credential_row=credentials.get(row.credential_profile_id or "")) for row in rows],
|
||||
[
|
||||
connector_profile_from_row(
|
||||
row,
|
||||
credential_row=(
|
||||
_resolve_profile_reusable_credential(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
row=row,
|
||||
)
|
||||
if reusable_credential_id(row.credential_profile_id)
|
||||
else credentials.get(row.credential_profile_id or "")
|
||||
),
|
||||
)
|
||||
for row in rows
|
||||
],
|
||||
profile_ids,
|
||||
)
|
||||
|
||||
@@ -73,7 +97,10 @@ def list_connector_profile_rows(
|
||||
return query.order_by(FileConnectorProfile.scope_type.asc(), FileConnectorProfile.label.asc()).all()
|
||||
|
||||
|
||||
def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileConnectorCredential | None = None) -> ConnectorProfile:
|
||||
def connector_profile_from_row(
|
||||
row: FileConnectorProfile,
|
||||
credential_row: FileConnectorCredential | ConnectorCredential | None = None,
|
||||
) -> ConnectorProfile:
|
||||
policy_sources = []
|
||||
if row.policy:
|
||||
policy_sources = connector_policy_sources_from_payload({
|
||||
@@ -82,9 +109,18 @@ def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileCo
|
||||
"label": row.label,
|
||||
"policy": row.policy,
|
||||
})
|
||||
credential = connector_credential_from_row(credential_row) if credential_row is not None else None
|
||||
credential = (
|
||||
credential_row
|
||||
if isinstance(credential_row, ConnectorCredential)
|
||||
else connector_credential_from_row(credential_row)
|
||||
if credential_row is not None
|
||||
else None
|
||||
)
|
||||
if credential:
|
||||
policy_sources.extend(credential.policy_sources)
|
||||
metadata = dict(row.metadata_ or {})
|
||||
if reusable_credential_id(row.credential_profile_id) and credential is None:
|
||||
metadata["credential_unavailable"] = True
|
||||
return ConnectorProfile(
|
||||
id=row.id,
|
||||
label=row.label,
|
||||
@@ -105,11 +141,30 @@ def connector_profile_from_row(row: FileConnectorProfile, credential_row: FileCo
|
||||
token_value=credential.token_value if credential else decrypt_secret(row.token_encrypted),
|
||||
capabilities=tuple(_string_list(row.capabilities)),
|
||||
policy_sources=tuple(policy_sources),
|
||||
metadata=dict(row.metadata_ or {}),
|
||||
metadata=metadata,
|
||||
source_kind="database",
|
||||
)
|
||||
|
||||
|
||||
def _resolve_profile_reusable_credential(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
row: FileConnectorProfile,
|
||||
) -> ConnectorCredential | None:
|
||||
try:
|
||||
return resolve_reusable_connector_credential(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
credential_ref=row.credential_profile_id or "",
|
||||
profile_id=row.id,
|
||||
scope_type=row.scope_type,
|
||||
scope_id=row.scope_id,
|
||||
)
|
||||
except FileStorageError:
|
||||
return None
|
||||
|
||||
|
||||
def get_connector_profile_row(
|
||||
session: Session,
|
||||
*,
|
||||
|
||||
@@ -2,21 +2,31 @@ from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import mimetypes
|
||||
from datetime import datetime
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Any, Iterable
|
||||
from uuid import uuid4
|
||||
|
||||
from sqlalchemy import and_, or_
|
||||
from sqlalchemy import and_, exists, func, or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.campaigns import CAPABILITY_CAMPAIGNS_ACCESS, CampaignAccessProvider
|
||||
from govoplan_files.backend.db.models import CampaignAttachmentUse, FileAsset, FileBlob, FileShare, FileVersion
|
||||
from govoplan_files.backend.runtime import get_registry, settings
|
||||
from govoplan_files.backend.storage.access import ensure_owner_access, ensure_share_target_exists, user_group_ids
|
||||
from govoplan_files.backend.storage.backends import StorageBackendError, get_storage_backend
|
||||
from govoplan_files.backend.storage.backends import (
|
||||
StorageBackendError,
|
||||
StorageObjectMissing,
|
||||
get_storage_backend,
|
||||
)
|
||||
from govoplan_files.backend.storage.common import FileConflictResolution, FileStorageError, UploadedStoredFile, utcnow
|
||||
from govoplan_files.backend.storage.paths import filename_from_path, join_folder_filename, normalize_folder, normalize_logical_path, safe_storage_component
|
||||
from govoplan_files.backend.storage.provenance import source_provenance_from_metadata
|
||||
from govoplan_files.backend.storage.integrity import (
|
||||
QUARANTINED_BLOB_STATUSES,
|
||||
read_verified_blob_bytes,
|
||||
)
|
||||
from govoplan_files.backend.storage.share_state import effective_file_share_clause
|
||||
|
||||
|
||||
def _campaign_access_provider() -> CampaignAccessProvider:
|
||||
@@ -72,11 +82,25 @@ def _get_or_create_blob(
|
||||
)
|
||||
if blob:
|
||||
backend = get_storage_backend()
|
||||
if not backend.exists(blob.storage_key):
|
||||
repair_required = (
|
||||
blob.integrity_status in QUARANTINED_BLOB_STATUSES
|
||||
or blob.quarantined_at is not None
|
||||
)
|
||||
try:
|
||||
backend.stat(blob.storage_key)
|
||||
except StorageObjectMissing:
|
||||
repair_required = True
|
||||
except StorageBackendError as exc:
|
||||
raise FileStorageError(str(exc)) from exc
|
||||
if repair_required:
|
||||
try:
|
||||
backend.put_bytes(blob.storage_key, data, content_type=content_type)
|
||||
except StorageBackendError as exc:
|
||||
raise FileStorageError(str(exc)) from exc
|
||||
blob.integrity_status = "verified"
|
||||
blob.integrity_checked_at = utcnow()
|
||||
blob.integrity_failure = None
|
||||
blob.quarantined_at = None
|
||||
blob.ref_count += 1
|
||||
session.add(blob)
|
||||
return blob
|
||||
@@ -96,6 +120,8 @@ def _get_or_create_blob(
|
||||
size_bytes=size,
|
||||
content_type=content_type,
|
||||
ref_count=1,
|
||||
integrity_status="verified",
|
||||
integrity_checked_at=utcnow(),
|
||||
)
|
||||
session.add(blob)
|
||||
session.flush()
|
||||
@@ -328,7 +354,7 @@ def get_asset_for_user(session: Session, *, tenant_id: str, user_id: str, asset_
|
||||
.filter(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == asset.id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
effective_file_share_clause(),
|
||||
FileShare.permission.in_(permission_values),
|
||||
or_(
|
||||
(FileShare.target_type == "user") & (FileShare.target_id == user_id),
|
||||
@@ -343,6 +369,32 @@ def get_asset_for_user(session: Session, *, tenant_id: str, user_id: str, asset_
|
||||
return asset
|
||||
|
||||
|
||||
def get_asset_for_share_management(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
asset_id: str,
|
||||
is_admin: bool = False,
|
||||
) -> FileAsset:
|
||||
asset = session.get(FileAsset, asset_id)
|
||||
if not asset or asset.tenant_id != tenant_id or asset.deleted_at is not None:
|
||||
raise FileStorageError("File not found")
|
||||
if is_admin:
|
||||
return asset
|
||||
owns_asset = (
|
||||
asset.owner_type == "user"
|
||||
and asset.owner_user_id == user_id
|
||||
) or (
|
||||
asset.owner_type == "group"
|
||||
and asset.owner_group_id
|
||||
in user_group_ids(session, tenant_id=tenant_id, user_id=user_id)
|
||||
)
|
||||
if not owns_asset:
|
||||
raise FileStorageError("Only file owners and administrators can manage shares")
|
||||
return asset
|
||||
|
||||
|
||||
def list_assets_for_user(
|
||||
session: Session,
|
||||
*,
|
||||
@@ -352,6 +404,8 @@ def list_assets_for_user(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
) -> list[FileAsset]:
|
||||
@@ -363,6 +417,8 @@ def list_assets_for_user(
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
@@ -378,6 +434,8 @@ def list_assets_for_user_window(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
page_size: int,
|
||||
@@ -393,6 +451,8 @@ def list_assets_for_user_window(
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
@@ -417,6 +477,8 @@ def _asset_visibility_query_for_user(
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
):
|
||||
@@ -434,7 +496,7 @@ def _asset_visibility_query_for_user(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.target_type == "campaign",
|
||||
FileShare.target_id == campaign_id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
effective_file_share_clause(),
|
||||
)
|
||||
elif not is_admin and not owner_type:
|
||||
group_ids = user_group_ids(session, tenant_id=tenant_id, user_id=user_id)
|
||||
@@ -442,16 +504,73 @@ def _asset_visibility_query_for_user(
|
||||
or_(
|
||||
(FileAsset.owner_type == "user") & (FileAsset.owner_user_id == user_id),
|
||||
(FileAsset.owner_type == "group") & (FileAsset.owner_group_id.in_(group_ids)),
|
||||
(FileShare.revoked_at.is_(None)) & (FileShare.target_type == "user") & (FileShare.target_id == user_id),
|
||||
(FileShare.revoked_at.is_(None)) & (FileShare.target_type == "group") & (FileShare.target_id.in_(group_ids)),
|
||||
(FileShare.revoked_at.is_(None)) & (FileShare.target_type == "tenant") & (FileShare.target_id == tenant_id),
|
||||
effective_file_share_clause() & (FileShare.target_type == "user") & (FileShare.target_id == user_id),
|
||||
effective_file_share_clause() & (FileShare.target_type == "group") & (FileShare.target_id.in_(group_ids)),
|
||||
effective_file_share_clause() & (FileShare.target_type == "tenant") & (FileShare.target_id == tenant_id),
|
||||
)
|
||||
)
|
||||
if path_prefix:
|
||||
prefix = normalize_folder(path_prefix)
|
||||
if prefix:
|
||||
query = query.filter(FileAsset.display_path.like(f"{prefix}/%"))
|
||||
return query
|
||||
campaign_share_exists = exists().where(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == FileAsset.id,
|
||||
FileShare.target_type == "campaign",
|
||||
effective_file_share_clause(),
|
||||
)
|
||||
campaign_use_exists = exists().where(
|
||||
CampaignAttachmentUse.tenant_id == tenant_id,
|
||||
CampaignAttachmentUse.file_asset_id == FileAsset.id,
|
||||
)
|
||||
if campaign_usage == "linked":
|
||||
query = query.filter(or_(campaign_share_exists, campaign_use_exists))
|
||||
elif campaign_usage == "unlinked":
|
||||
query = query.filter(~or_(campaign_share_exists, campaign_use_exists))
|
||||
|
||||
if audit_relevant is not None:
|
||||
sent_use_exists = exists().where(
|
||||
CampaignAttachmentUse.tenant_id == tenant_id,
|
||||
CampaignAttachmentUse.file_asset_id == FileAsset.id,
|
||||
CampaignAttachmentUse.use_stage == "sent",
|
||||
)
|
||||
query = query.filter(sent_use_exists if audit_relevant else ~sent_use_exists)
|
||||
return query.distinct()
|
||||
|
||||
|
||||
def count_assets_for_user(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
owner_type: str | None = None,
|
||||
owner_id: str | None = None,
|
||||
campaign_id: str | None = None,
|
||||
path_prefix: str | None = None,
|
||||
campaign_usage: str | None = None,
|
||||
audit_relevant: bool | None = None,
|
||||
include_deleted: bool = False,
|
||||
is_admin: bool = False,
|
||||
) -> int:
|
||||
query = _asset_visibility_query_for_user(
|
||||
session,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
owner_type=owner_type,
|
||||
owner_id=owner_id,
|
||||
campaign_id=campaign_id,
|
||||
path_prefix=path_prefix,
|
||||
campaign_usage=campaign_usage,
|
||||
audit_relevant=audit_relevant,
|
||||
include_deleted=include_deleted,
|
||||
is_admin=is_admin,
|
||||
)
|
||||
return int(
|
||||
query.order_by(None)
|
||||
.with_entities(func.count(func.distinct(FileAsset.id)))
|
||||
.scalar()
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def current_version_and_blob(session: Session, asset: FileAsset) -> tuple[FileVersion, FileBlob]:
|
||||
@@ -495,10 +614,7 @@ def current_versions_and_blobs(session: Session, assets: Iterable[FileAsset]) ->
|
||||
def read_asset_bytes(session: Session, asset: FileAsset) -> tuple[bytes, FileVersion, FileBlob]:
|
||||
version, blob = current_version_and_blob(session, asset)
|
||||
backend = get_storage_backend()
|
||||
try:
|
||||
return backend.get_bytes(blob.storage_key), version, blob
|
||||
except StorageBackendError as exc:
|
||||
raise FileStorageError(str(exc)) from exc
|
||||
return read_verified_blob_bytes(blob, backend=backend), version, blob
|
||||
|
||||
|
||||
def share_file(
|
||||
@@ -510,6 +626,7 @@ def share_file(
|
||||
target_id: str,
|
||||
permission: str,
|
||||
user_id: str,
|
||||
expires_at: datetime | None = None,
|
||||
) -> FileShare:
|
||||
target_type = target_type.lower().strip()
|
||||
permission = permission.lower().strip()
|
||||
@@ -519,6 +636,7 @@ def share_file(
|
||||
raise FileStorageError("Unsupported share target")
|
||||
if permission not in {"read", "write", "manage"}:
|
||||
raise FileStorageError("Unsupported file permission")
|
||||
_validate_share_expiry(expires_at)
|
||||
if target_type in {"user", "group", "tenant"}:
|
||||
ensure_share_target_exists(tenant_id=tenant_id, target_type=target_type, target_id=target_id)
|
||||
if target_type == "campaign":
|
||||
@@ -536,6 +654,7 @@ def share_file(
|
||||
)
|
||||
if existing:
|
||||
existing.permission = permission
|
||||
existing.expires_at = expires_at
|
||||
session.add(existing)
|
||||
return existing
|
||||
share = FileShare(
|
||||
@@ -545,6 +664,7 @@ def share_file(
|
||||
target_id=target_id,
|
||||
permission=permission,
|
||||
created_by_user_id=user_id,
|
||||
expires_at=expires_at,
|
||||
)
|
||||
session.add(share)
|
||||
return share
|
||||
@@ -559,6 +679,7 @@ def share_files(
|
||||
target_id: str,
|
||||
permission: str,
|
||||
user_id: str,
|
||||
expires_at: datetime | None = None,
|
||||
) -> list[FileShare]:
|
||||
target_type = target_type.lower().strip()
|
||||
permission = permission.lower().strip()
|
||||
@@ -566,6 +687,7 @@ def share_files(
|
||||
raise FileStorageError("Unsupported share target")
|
||||
if permission not in {"read", "write", "manage"}:
|
||||
raise FileStorageError("Unsupported file permission")
|
||||
_validate_share_expiry(expires_at)
|
||||
if target_type in {"user", "group", "tenant"}:
|
||||
ensure_share_target_exists(tenant_id=tenant_id, target_type=target_type, target_id=target_id)
|
||||
if target_type == "campaign":
|
||||
@@ -593,6 +715,7 @@ def share_files(
|
||||
existing = existing_by_asset.get(asset.id)
|
||||
if existing:
|
||||
existing.permission = permission
|
||||
existing.expires_at = expires_at
|
||||
session.add(existing)
|
||||
shares.append(existing)
|
||||
continue
|
||||
@@ -603,12 +726,88 @@ def share_files(
|
||||
target_id=target_id,
|
||||
permission=permission,
|
||||
created_by_user_id=user_id,
|
||||
expires_at=expires_at,
|
||||
)
|
||||
session.add(share)
|
||||
shares.append(share)
|
||||
return shares
|
||||
|
||||
|
||||
def list_file_shares(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
asset_id: str,
|
||||
include_inactive: bool = False,
|
||||
) -> list[FileShare]:
|
||||
query = session.query(FileShare).filter(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == asset_id,
|
||||
)
|
||||
if not include_inactive:
|
||||
query = query.filter(effective_file_share_clause())
|
||||
return query.order_by(FileShare.created_at.desc(), FileShare.id.desc()).all()
|
||||
|
||||
|
||||
def revoke_file_share(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
asset_id: str,
|
||||
share_id: str,
|
||||
user_id: str,
|
||||
) -> tuple[FileShare, bool]:
|
||||
share = (
|
||||
session.query(FileShare)
|
||||
.filter(
|
||||
FileShare.id == share_id,
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == asset_id,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if share is None:
|
||||
raise FileStorageError("File share not found")
|
||||
if share.revoked_at is not None:
|
||||
return share, False
|
||||
share.revoked_at = utcnow()
|
||||
share.revoked_by_user_id = user_id
|
||||
session.add(share)
|
||||
return share, True
|
||||
|
||||
|
||||
def current_file_share_for_target(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
asset_id: str,
|
||||
target_type: str,
|
||||
target_id: str,
|
||||
) -> FileShare | None:
|
||||
return (
|
||||
session.query(FileShare)
|
||||
.filter(
|
||||
FileShare.tenant_id == tenant_id,
|
||||
FileShare.file_asset_id == asset_id,
|
||||
FileShare.target_type == target_type.lower().strip(),
|
||||
FileShare.target_id == target_id,
|
||||
FileShare.revoked_at.is_(None),
|
||||
)
|
||||
.order_by(FileShare.created_at.desc(), FileShare.id.desc())
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def _validate_share_expiry(expires_at: datetime | None) -> None:
|
||||
if expires_at is None:
|
||||
return
|
||||
from govoplan_files.backend.storage.share_state import file_share_is_active
|
||||
|
||||
candidate = FileShare(expires_at=expires_at)
|
||||
if not file_share_is_active(candidate):
|
||||
raise FileStorageError("File share expiry must be in the future")
|
||||
|
||||
|
||||
|
||||
def soft_delete_assets(session: Session, assets: Iterable[FileAsset]) -> int:
|
||||
count = 0
|
||||
|
||||
@@ -185,14 +185,22 @@ class _OutboundPolicyHTTPTransport(httpx.BaseTransport):
|
||||
self._connection_pool.close()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _stream_connector_request(method: str, url: str, **kwargs: Any) -> Iterator[httpx.Response]:
|
||||
with httpx.Client(
|
||||
_CONNECTOR_HTTP_CLIENT = httpx.Client(
|
||||
transport=_OutboundPolicyHTTPTransport(),
|
||||
follow_redirects=False,
|
||||
timeout=kwargs.pop("timeout", 15.0),
|
||||
) as client:
|
||||
with client.stream(method, url, **kwargs) as response:
|
||||
timeout=15.0,
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _stream_connector_request(method: str, url: str, **kwargs: Any) -> Iterator[httpx.Response]:
|
||||
timeout = kwargs.pop("timeout", 15.0)
|
||||
with _CONNECTOR_HTTP_CLIENT.stream(
|
||||
method,
|
||||
url,
|
||||
timeout=timeout,
|
||||
**kwargs,
|
||||
) as response:
|
||||
yield response
|
||||
|
||||
|
||||
|
||||
512
src/govoplan_files/backend/storage/integrity.py
Normal file
512
src/govoplan_files/backend/storage/integrity.py
Normal file
@@ -0,0 +1,512 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_files.backend.db.models import (
|
||||
FileBlob,
|
||||
FileIntegrityFinding,
|
||||
FileIntegrityScan,
|
||||
)
|
||||
from govoplan_files.backend.storage.backends import (
|
||||
StorageBackend,
|
||||
StorageBackendError,
|
||||
StorageObjectMissing,
|
||||
get_storage_backend,
|
||||
)
|
||||
from govoplan_files.backend.storage.common import FileStorageError, utcnow
|
||||
|
||||
|
||||
TERMINAL_SCAN_STATUSES = {"completed", "cancelled"}
|
||||
QUARANTINED_BLOB_STATUSES = {
|
||||
"missing",
|
||||
"size_mismatch",
|
||||
"checksum_mismatch",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class BlobInspection:
|
||||
valid: bool
|
||||
kind: str
|
||||
observed_size_bytes: int | None = None
|
||||
observed_checksum_sha256: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class IntegrityActionResult:
|
||||
action: str
|
||||
changed: bool
|
||||
dry_run: bool
|
||||
finding: FileIntegrityFinding
|
||||
inspection: BlobInspection | None = None
|
||||
|
||||
|
||||
def storage_prefix_for_tenant(tenant_id: str) -> str:
|
||||
return f"tenants/{tenant_id}/files/"
|
||||
|
||||
|
||||
def create_integrity_scan(
|
||||
session: Session,
|
||||
*,
|
||||
tenant_id: str,
|
||||
user_id: str,
|
||||
verify_checksums: bool = True,
|
||||
batch_size: int = 100,
|
||||
backend: StorageBackend | None = None,
|
||||
) -> FileIntegrityScan:
|
||||
active_backend = backend or get_storage_backend()
|
||||
scan = FileIntegrityScan(
|
||||
tenant_id=tenant_id,
|
||||
storage_backend=active_backend.name,
|
||||
storage_prefix=storage_prefix_for_tenant(tenant_id),
|
||||
verify_checksums=verify_checksums,
|
||||
batch_size=max(1, min(int(batch_size), 1000)),
|
||||
created_by_user_id=user_id,
|
||||
)
|
||||
session.add(scan)
|
||||
session.flush()
|
||||
return scan
|
||||
|
||||
|
||||
def run_integrity_scan_batch(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
*,
|
||||
backend: StorageBackend | None = None,
|
||||
) -> FileIntegrityScan:
|
||||
if scan.status in TERMINAL_SCAN_STATUSES:
|
||||
return scan
|
||||
active_backend = backend or get_storage_backend()
|
||||
if active_backend.name != scan.storage_backend:
|
||||
raise FileStorageError(
|
||||
"The configured storage backend changed after this integrity scan started"
|
||||
)
|
||||
if scan.started_at is None:
|
||||
scan.started_at = utcnow()
|
||||
scan.status = "running"
|
||||
scan.last_error = None
|
||||
if scan.phase == "blobs":
|
||||
_scan_blob_batch(session, scan, active_backend)
|
||||
elif scan.phase == "objects":
|
||||
_scan_object_batch(session, scan, active_backend)
|
||||
else:
|
||||
scan.phase = "completed"
|
||||
scan.status = "completed"
|
||||
scan.completed_at = utcnow()
|
||||
session.add(scan)
|
||||
return scan
|
||||
|
||||
|
||||
def mark_integrity_scan_failed(
|
||||
scan: FileIntegrityScan,
|
||||
*,
|
||||
error: Exception,
|
||||
) -> None:
|
||||
scan.status = "failed"
|
||||
scan.last_error = type(error).__name__[:255]
|
||||
|
||||
|
||||
def inspect_blob(
|
||||
blob: FileBlob,
|
||||
*,
|
||||
backend: StorageBackend,
|
||||
verify_checksum: bool = True,
|
||||
) -> BlobInspection:
|
||||
try:
|
||||
info = backend.stat(blob.storage_key)
|
||||
except StorageObjectMissing:
|
||||
return BlobInspection(valid=False, kind="missing")
|
||||
if info.size_bytes != blob.size_bytes:
|
||||
return BlobInspection(
|
||||
valid=False,
|
||||
kind="size_mismatch",
|
||||
observed_size_bytes=info.size_bytes,
|
||||
)
|
||||
if not verify_checksum:
|
||||
return BlobInspection(
|
||||
valid=True,
|
||||
kind="verified",
|
||||
observed_size_bytes=info.size_bytes,
|
||||
)
|
||||
digest = hashlib.sha256()
|
||||
observed_size = 0
|
||||
for chunk in backend.iter_bytes(blob.storage_key):
|
||||
observed_size += len(chunk)
|
||||
digest.update(chunk)
|
||||
observed_checksum = digest.hexdigest()
|
||||
if observed_size != blob.size_bytes:
|
||||
return BlobInspection(
|
||||
valid=False,
|
||||
kind="size_mismatch",
|
||||
observed_size_bytes=observed_size,
|
||||
observed_checksum_sha256=observed_checksum,
|
||||
)
|
||||
if observed_checksum != blob.checksum_sha256:
|
||||
return BlobInspection(
|
||||
valid=False,
|
||||
kind="checksum_mismatch",
|
||||
observed_size_bytes=observed_size,
|
||||
observed_checksum_sha256=observed_checksum,
|
||||
)
|
||||
return BlobInspection(
|
||||
valid=True,
|
||||
kind="verified",
|
||||
observed_size_bytes=observed_size,
|
||||
observed_checksum_sha256=observed_checksum,
|
||||
)
|
||||
|
||||
|
||||
def apply_blob_inspection(
|
||||
session: Session,
|
||||
blob: FileBlob,
|
||||
inspection: BlobInspection,
|
||||
*,
|
||||
checked_at=None,
|
||||
) -> None:
|
||||
timestamp = checked_at or utcnow()
|
||||
blob.integrity_checked_at = timestamp
|
||||
if inspection.valid:
|
||||
blob.integrity_status = "verified"
|
||||
blob.integrity_failure = None
|
||||
blob.quarantined_at = None
|
||||
else:
|
||||
blob.integrity_status = inspection.kind
|
||||
blob.integrity_failure = inspection.kind
|
||||
blob.quarantined_at = blob.quarantined_at or timestamp
|
||||
session.add(blob)
|
||||
|
||||
|
||||
def ensure_blob_is_readable(blob: FileBlob) -> None:
|
||||
if blob.integrity_status in QUARANTINED_BLOB_STATUSES or blob.quarantined_at:
|
||||
raise FileStorageError(
|
||||
"Managed file content is quarantined because integrity verification failed"
|
||||
)
|
||||
|
||||
|
||||
def read_verified_blob_bytes(
|
||||
blob: FileBlob,
|
||||
*,
|
||||
backend: StorageBackend,
|
||||
) -> bytes:
|
||||
ensure_blob_is_readable(blob)
|
||||
try:
|
||||
data = backend.get_bytes(blob.storage_key)
|
||||
except StorageBackendError as exc:
|
||||
raise FileStorageError("Managed file content is not available") from exc
|
||||
if len(data) != blob.size_bytes:
|
||||
raise FileStorageError(
|
||||
"Managed file content failed its recorded size verification"
|
||||
)
|
||||
if hashlib.sha256(data).hexdigest() != blob.checksum_sha256:
|
||||
raise FileStorageError(
|
||||
"Managed file content failed its recorded checksum verification"
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def recheck_integrity_finding(
|
||||
session: Session,
|
||||
finding: FileIntegrityFinding,
|
||||
*,
|
||||
user_id: str,
|
||||
dry_run: bool,
|
||||
backend: StorageBackend | None = None,
|
||||
) -> IntegrityActionResult:
|
||||
if finding.kind == "orphan_object" or not finding.blob_id:
|
||||
raise FileStorageError("Only blob integrity findings can be rechecked")
|
||||
blob = session.get(FileBlob, finding.blob_id)
|
||||
if blob is None or blob.tenant_id != finding.tenant_id:
|
||||
raise FileStorageError("Integrity finding blob no longer exists")
|
||||
active_backend = backend or get_storage_backend()
|
||||
scan = session.get(FileIntegrityScan, finding.scan_id)
|
||||
if scan is None or scan.storage_backend != active_backend.name:
|
||||
raise FileStorageError(
|
||||
"The configured storage backend does not match the integrity finding"
|
||||
)
|
||||
inspection = inspect_blob(
|
||||
blob,
|
||||
backend=active_backend,
|
||||
verify_checksum=True,
|
||||
)
|
||||
changed = False
|
||||
if not dry_run:
|
||||
previous = (
|
||||
blob.integrity_status,
|
||||
blob.quarantined_at,
|
||||
finding.state,
|
||||
)
|
||||
apply_blob_inspection(session, blob, inspection)
|
||||
_update_finding_from_inspection(finding, inspection)
|
||||
if inspection.valid:
|
||||
finding.state = "resolved"
|
||||
finding.resolved_at = utcnow()
|
||||
finding.resolved_by_user_id = user_id
|
||||
session.add(finding)
|
||||
changed = previous != (
|
||||
blob.integrity_status,
|
||||
blob.quarantined_at,
|
||||
finding.state,
|
||||
)
|
||||
return IntegrityActionResult(
|
||||
action="recheck",
|
||||
changed=changed,
|
||||
dry_run=dry_run,
|
||||
finding=finding,
|
||||
inspection=inspection,
|
||||
)
|
||||
|
||||
|
||||
def cleanup_orphan_finding(
|
||||
session: Session,
|
||||
finding: FileIntegrityFinding,
|
||||
*,
|
||||
user_id: str,
|
||||
dry_run: bool,
|
||||
backend: StorageBackend | None = None,
|
||||
) -> IntegrityActionResult:
|
||||
if finding.kind != "orphan_object":
|
||||
raise FileStorageError("Only orphan-object findings can be cleaned up")
|
||||
scan = session.get(FileIntegrityScan, finding.scan_id)
|
||||
if scan is None or scan.tenant_id != finding.tenant_id:
|
||||
raise FileStorageError("Integrity scan not found")
|
||||
if not finding.storage_key.startswith(scan.storage_prefix):
|
||||
raise FileStorageError("Orphan cleanup is outside the scan storage scope")
|
||||
referenced = (
|
||||
session.query(FileBlob.id)
|
||||
.filter(
|
||||
FileBlob.tenant_id == finding.tenant_id,
|
||||
FileBlob.storage_backend == scan.storage_backend,
|
||||
FileBlob.storage_key == finding.storage_key,
|
||||
)
|
||||
.first()
|
||||
)
|
||||
if referenced:
|
||||
if not dry_run and finding.state != "resolved":
|
||||
finding.state = "resolved"
|
||||
finding.resolved_at = utcnow()
|
||||
finding.resolved_by_user_id = user_id
|
||||
session.add(finding)
|
||||
return IntegrityActionResult(
|
||||
action="retained_referenced",
|
||||
changed=True,
|
||||
dry_run=False,
|
||||
finding=finding,
|
||||
)
|
||||
return IntegrityActionResult(
|
||||
action="retained_referenced",
|
||||
changed=False,
|
||||
dry_run=dry_run,
|
||||
finding=finding,
|
||||
)
|
||||
if finding.state == "deleted":
|
||||
return IntegrityActionResult(
|
||||
action="already_deleted",
|
||||
changed=False,
|
||||
dry_run=dry_run,
|
||||
finding=finding,
|
||||
)
|
||||
if dry_run:
|
||||
return IntegrityActionResult(
|
||||
action="would_delete",
|
||||
changed=False,
|
||||
dry_run=True,
|
||||
finding=finding,
|
||||
)
|
||||
active_backend = backend or get_storage_backend()
|
||||
if active_backend.name != scan.storage_backend:
|
||||
raise FileStorageError(
|
||||
"The configured storage backend does not match the integrity finding"
|
||||
)
|
||||
try:
|
||||
active_backend.stat(finding.storage_key)
|
||||
except StorageObjectMissing:
|
||||
action = "already_absent"
|
||||
else:
|
||||
active_backend.delete(finding.storage_key)
|
||||
action = "deleted"
|
||||
finding.state = "deleted"
|
||||
finding.resolved_at = utcnow()
|
||||
finding.resolved_by_user_id = user_id
|
||||
session.add(finding)
|
||||
return IntegrityActionResult(
|
||||
action=action,
|
||||
changed=True,
|
||||
dry_run=False,
|
||||
finding=finding,
|
||||
)
|
||||
|
||||
|
||||
def _scan_blob_batch(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
backend: StorageBackend,
|
||||
) -> None:
|
||||
query = session.query(FileBlob).filter(
|
||||
FileBlob.tenant_id == scan.tenant_id,
|
||||
FileBlob.storage_backend == scan.storage_backend,
|
||||
)
|
||||
if scan.blob_cursor:
|
||||
query = query.filter(FileBlob.id > scan.blob_cursor)
|
||||
blobs = query.order_by(FileBlob.id.asc()).limit(scan.batch_size).all()
|
||||
for blob in blobs:
|
||||
inspection = inspect_blob(
|
||||
blob,
|
||||
backend=backend,
|
||||
verify_checksum=scan.verify_checksums,
|
||||
)
|
||||
apply_blob_inspection(session, blob, inspection)
|
||||
scan.scanned_blob_count += 1
|
||||
if inspection.valid:
|
||||
scan.verified_blob_count += 1
|
||||
_resolve_scan_blob_findings(session, scan, blob)
|
||||
else:
|
||||
scan.quarantined_blob_count += 1
|
||||
_record_blob_finding(session, scan, blob, inspection)
|
||||
scan.blob_cursor = blob.id
|
||||
if len(blobs) < scan.batch_size:
|
||||
scan.phase = "objects"
|
||||
scan.object_cursor = None
|
||||
|
||||
|
||||
def _scan_object_batch(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
backend: StorageBackend,
|
||||
) -> None:
|
||||
page = backend.list_objects(
|
||||
prefix=scan.storage_prefix,
|
||||
after=scan.object_cursor,
|
||||
limit=scan.batch_size,
|
||||
)
|
||||
keys = [item.key for item in page.objects]
|
||||
referenced_keys = {
|
||||
row[0]
|
||||
for row in session.query(FileBlob.storage_key)
|
||||
.filter(
|
||||
FileBlob.tenant_id == scan.tenant_id,
|
||||
FileBlob.storage_backend == scan.storage_backend,
|
||||
FileBlob.storage_key.in_(keys),
|
||||
)
|
||||
.all()
|
||||
} if keys else set()
|
||||
for item in page.objects:
|
||||
scan.scanned_object_count += 1
|
||||
if item.key in referenced_keys:
|
||||
continue
|
||||
scan.orphan_object_count += 1
|
||||
_record_orphan_finding(
|
||||
session,
|
||||
scan,
|
||||
storage_key=item.key,
|
||||
size_bytes=item.size_bytes,
|
||||
)
|
||||
scan.object_cursor = page.next_cursor
|
||||
if page.next_cursor is None:
|
||||
scan.phase = "completed"
|
||||
scan.status = "completed"
|
||||
scan.completed_at = utcnow()
|
||||
|
||||
|
||||
def _record_blob_finding(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
blob: FileBlob,
|
||||
inspection: BlobInspection,
|
||||
) -> FileIntegrityFinding:
|
||||
finding = (
|
||||
session.query(FileIntegrityFinding)
|
||||
.filter(
|
||||
FileIntegrityFinding.scan_id == scan.id,
|
||||
FileIntegrityFinding.blob_id == blob.id,
|
||||
FileIntegrityFinding.kind == inspection.kind,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if finding is None:
|
||||
finding = FileIntegrityFinding(
|
||||
scan_id=scan.id,
|
||||
tenant_id=scan.tenant_id,
|
||||
kind=inspection.kind,
|
||||
blob_id=blob.id,
|
||||
storage_key=blob.storage_key,
|
||||
expected_size_bytes=blob.size_bytes,
|
||||
expected_checksum_sha256=blob.checksum_sha256,
|
||||
)
|
||||
_update_finding_from_inspection(finding, inspection)
|
||||
session.add(finding)
|
||||
return finding
|
||||
|
||||
|
||||
def _record_orphan_finding(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
*,
|
||||
storage_key: str,
|
||||
size_bytes: int,
|
||||
) -> FileIntegrityFinding:
|
||||
finding = (
|
||||
session.query(FileIntegrityFinding)
|
||||
.filter(
|
||||
FileIntegrityFinding.scan_id == scan.id,
|
||||
FileIntegrityFinding.kind == "orphan_object",
|
||||
FileIntegrityFinding.storage_key == storage_key,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if finding is None:
|
||||
finding = FileIntegrityFinding(
|
||||
scan_id=scan.id,
|
||||
tenant_id=scan.tenant_id,
|
||||
kind="orphan_object",
|
||||
storage_key=storage_key,
|
||||
observed_size_bytes=size_bytes,
|
||||
)
|
||||
session.add(finding)
|
||||
return finding
|
||||
|
||||
|
||||
def _resolve_scan_blob_findings(
|
||||
session: Session,
|
||||
scan: FileIntegrityScan,
|
||||
blob: FileBlob,
|
||||
) -> None:
|
||||
for finding in (
|
||||
session.query(FileIntegrityFinding)
|
||||
.filter(
|
||||
FileIntegrityFinding.scan_id == scan.id,
|
||||
FileIntegrityFinding.blob_id == blob.id,
|
||||
FileIntegrityFinding.state == "open",
|
||||
)
|
||||
.all()
|
||||
):
|
||||
finding.state = "resolved"
|
||||
finding.resolved_at = utcnow()
|
||||
session.add(finding)
|
||||
|
||||
|
||||
def _update_finding_from_inspection(
|
||||
finding: FileIntegrityFinding,
|
||||
inspection: BlobInspection,
|
||||
) -> None:
|
||||
finding.observed_size_bytes = inspection.observed_size_bytes
|
||||
finding.observed_checksum_sha256 = inspection.observed_checksum_sha256
|
||||
|
||||
|
||||
__all__ = [
|
||||
"BlobInspection",
|
||||
"IntegrityActionResult",
|
||||
"QUARANTINED_BLOB_STATUSES",
|
||||
"apply_blob_inspection",
|
||||
"cleanup_orphan_finding",
|
||||
"create_integrity_scan",
|
||||
"ensure_blob_is_readable",
|
||||
"inspect_blob",
|
||||
"mark_integrity_scan_failed",
|
||||
"read_verified_blob_bytes",
|
||||
"recheck_integrity_finding",
|
||||
"run_integrity_scan_batch",
|
||||
"storage_prefix_for_tenant",
|
||||
]
|
||||
34
src/govoplan_files/backend/storage/share_state.py
Normal file
34
src/govoplan_files/backend/storage/share_state.py
Normal file
@@ -0,0 +1,34 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import and_, or_
|
||||
|
||||
from govoplan_files.backend.db.models import FileShare
|
||||
from govoplan_files.backend.storage.common import utcnow
|
||||
|
||||
|
||||
def effective_file_share_clause(*, at: datetime | None = None):
|
||||
checked_at = at or utcnow()
|
||||
return and_(
|
||||
FileShare.revoked_at.is_(None),
|
||||
or_(FileShare.expires_at.is_(None), FileShare.expires_at > checked_at),
|
||||
)
|
||||
|
||||
|
||||
def file_share_is_active(share: FileShare, *, at: datetime | None = None) -> bool:
|
||||
if share.revoked_at is not None:
|
||||
return False
|
||||
if share.expires_at is None:
|
||||
return True
|
||||
checked_at = _aware_utc(at or utcnow())
|
||||
return _aware_utc(share.expires_at) > checked_at
|
||||
|
||||
|
||||
def _aware_utc(value: datetime) -> datetime:
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc)
|
||||
|
||||
|
||||
__all__ = ["effective_file_share_clause", "file_share_is_active"]
|
||||
@@ -1,15 +1,18 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.core.access import PrincipalRef
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.capabilities import FilesAccessService, virtual_folder_resource_id
|
||||
from govoplan_files.backend.db.models import FileAsset, FileFolder, FileShare
|
||||
from govoplan_files.backend.storage.files import count_assets_for_user, list_assets_for_user
|
||||
|
||||
|
||||
TENANT_ID = "tenant-1"
|
||||
@@ -75,10 +78,123 @@ class FilesAccessProviderTests(unittest.TestCase):
|
||||
self.assertTrue(any(item.kind == "owner" and item.id == GROUP_ID for item in virtual_items))
|
||||
self.assertEqual("files.not_found", missing_items[0].source)
|
||||
|
||||
def test_file_property_filters_cover_campaign_and_audit_usage(self) -> None:
|
||||
session = _session()
|
||||
self.addCleanup(_close_session, session)
|
||||
_seed_access_subjects(session)
|
||||
assets = [
|
||||
FileAsset(
|
||||
id=f"file-{index}",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_type="user",
|
||||
owner_user_id=USER_ID,
|
||||
display_path=f"{index}.pdf",
|
||||
filename=f"{index}.pdf",
|
||||
)
|
||||
for index in range(1, 4)
|
||||
]
|
||||
session.add_all([
|
||||
*assets,
|
||||
FileShare(
|
||||
id="share-campaign",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=assets[0].id,
|
||||
target_type="campaign",
|
||||
target_id="campaign-1",
|
||||
permission="read",
|
||||
),
|
||||
FileShare(
|
||||
id="share-campaign-expired",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=assets[2].id,
|
||||
target_type="campaign",
|
||||
target_id="campaign-1",
|
||||
permission="read",
|
||||
expires_at=datetime.now(timezone.utc) - timedelta(seconds=1),
|
||||
),
|
||||
])
|
||||
session.commit()
|
||||
session.execute(
|
||||
text(
|
||||
"INSERT INTO campaign_attachment_uses "
|
||||
"(id, tenant_id, file_asset_id, use_stage) "
|
||||
"VALUES (:id, :tenant_id, :file_asset_id, :use_stage)"
|
||||
),
|
||||
{
|
||||
"id": "attachment-use-1",
|
||||
"tenant_id": TENANT_ID,
|
||||
"file_asset_id": assets[1].id,
|
||||
"use_stage": "sent",
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
|
||||
linked = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="linked",
|
||||
)
|
||||
unlinked = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="unlinked",
|
||||
)
|
||||
audit_relevant = list_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
audit_relevant=True,
|
||||
)
|
||||
|
||||
self.assertEqual([asset.id for asset in linked], ["file-1", "file-2"])
|
||||
self.assertEqual([asset.id for asset in unlinked], ["file-3"])
|
||||
self.assertEqual([asset.id for asset in audit_relevant], ["file-2"])
|
||||
self.assertEqual(
|
||||
count_assets_for_user(
|
||||
session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
owner_type="user",
|
||||
owner_id=USER_ID,
|
||||
campaign_usage="unlinked",
|
||||
),
|
||||
1,
|
||||
)
|
||||
|
||||
|
||||
def _session():
|
||||
engine = create_engine("sqlite:///:memory:", future=True)
|
||||
Base.metadata.create_all(bind=engine, tables=[Account.__table__, User.__table__, Group.__table__, FileAsset.__table__, FileFolder.__table__, FileShare.__table__])
|
||||
Base.metadata.create_all(
|
||||
bind=engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
FileAsset.__table__,
|
||||
FileFolder.__table__,
|
||||
FileShare.__table__,
|
||||
],
|
||||
)
|
||||
with engine.begin() as connection:
|
||||
connection.execute(
|
||||
text(
|
||||
"CREATE TABLE campaign_attachment_uses ("
|
||||
"id VARCHAR(36) PRIMARY KEY, "
|
||||
"tenant_id VARCHAR(36) NOT NULL, "
|
||||
"file_asset_id VARCHAR(36) NOT NULL, "
|
||||
"use_stage VARCHAR(20) NOT NULL"
|
||||
")"
|
||||
)
|
||||
)
|
||||
return sessionmaker(bind=engine, future=True)()
|
||||
|
||||
|
||||
|
||||
@@ -12,7 +12,8 @@ from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.security.secrets import encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
||||
from govoplan_files.backend.router import deactivate_connector_credential, deactivate_connector_profile
|
||||
from govoplan_files.backend.routes.connector_profiles import deactivate_connector_profile
|
||||
from govoplan_files.backend.routes.connector_settings import deactivate_connector_credential
|
||||
|
||||
|
||||
class Principal:
|
||||
|
||||
@@ -10,7 +10,7 @@ from unittest.mock import MagicMock
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
from govoplan_files.backend.router import discover_connector_endpoint
|
||||
from govoplan_files.backend.routes.connector_settings import discover_connector_endpoint
|
||||
from govoplan_files.backend.schemas import FileConnectorDiscoveryRequest
|
||||
from govoplan_files.backend.storage.connector_browse import ConnectorBrowseError, _profile_password, _s3_verify
|
||||
from govoplan_files.backend.storage.connector_deployment import (
|
||||
@@ -182,7 +182,7 @@ class ConnectorDiscoveryBoundaryTests(unittest.TestCase):
|
||||
credential_mode="basic",
|
||||
credentials={"username": "admin", "password_env": "MASTER_KEY_B64"},
|
||||
)
|
||||
with patch("govoplan_files.backend.router.browse_connector_profile") as browse, self.assertRaises(
|
||||
with patch("govoplan_files.backend.routes.connector_settings.browse_connector_profile") as browse, self.assertRaises(
|
||||
HTTPException
|
||||
) as raised:
|
||||
discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||
@@ -212,10 +212,10 @@ class ConnectorDiscoveryBoundaryTests(unittest.TestCase):
|
||||
events.append("io")
|
||||
return []
|
||||
|
||||
with patch("govoplan_files.backend.router._ensure_connector_configuration_allowed", side_effect=ensure), patch(
|
||||
"govoplan_files.backend.router._audit_connector_discovery_attempt",
|
||||
with patch("govoplan_files.backend.routes.connector_settings._ensure_connector_configuration_allowed", side_effect=ensure), patch(
|
||||
"govoplan_files.backend.routes.connector_settings._audit_connector_discovery_attempt",
|
||||
side_effect=audit,
|
||||
), patch("govoplan_files.backend.router.browse_connector_profile", side_effect=browse):
|
||||
), patch("govoplan_files.backend.routes.connector_settings.browse_connector_profile", side_effect=browse):
|
||||
response = discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||
|
||||
self.assertEqual("usable", response.status)
|
||||
|
||||
66
tests/test_connector_policy.py
Normal file
66
tests/test_connector_policy.py
Normal file
@@ -0,0 +1,66 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from govoplan_files.backend.storage.connector_policy import (
|
||||
ConnectorAccessRequest,
|
||||
ConnectorPolicySource,
|
||||
connector_policy_decision,
|
||||
)
|
||||
|
||||
|
||||
class ConnectorPolicyPatternTests(unittest.TestCase):
|
||||
def test_resource_reference_allow_patterns_match(self) -> None:
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id="dev-smb",
|
||||
credential_id="credential-primary",
|
||||
provider="smb",
|
||||
),
|
||||
(
|
||||
ConnectorPolicySource(
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
label="Tenant",
|
||||
policy={
|
||||
"allow": {
|
||||
"connectors": ["dev-*"],
|
||||
"credentials": ["credential-*"],
|
||||
"providers": ["s?b"],
|
||||
}
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
self.assertTrue(decision.allowed)
|
||||
|
||||
def test_resource_reference_deny_patterns_take_precedence(self) -> None:
|
||||
decision = connector_policy_decision(
|
||||
ConnectorAccessRequest(
|
||||
connector_id="dev-smb",
|
||||
credential_id="credential-legacy",
|
||||
provider="smb",
|
||||
),
|
||||
(
|
||||
ConnectorPolicySource(
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
label="Tenant",
|
||||
policy={
|
||||
"allow": {"connectors": ["dev-*"]},
|
||||
"deny": {"credentials": ["*-legacy"]},
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
self.assertFalse(decision.allowed)
|
||||
self.assertEqual(
|
||||
("connector_policy_denylist",),
|
||||
decision.requirements,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -8,6 +9,7 @@ from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db import models as access_models # noqa: F401 - resolve Files user foreign keys
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.security.credential_envelopes import CredentialEnvelope
|
||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||
from govoplan_files.backend.db.models import FileConnectorProfile
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
@@ -20,7 +22,13 @@ from govoplan_files.backend.storage.connector_profile_store import (
|
||||
class ConnectorProfileStoreUpdateTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.engine = create_engine("sqlite:///:memory:")
|
||||
Base.metadata.create_all(self.engine, tables=[FileConnectorProfile.__table__])
|
||||
Base.metadata.create_all(
|
||||
self.engine,
|
||||
tables=[
|
||||
FileConnectorProfile.__table__,
|
||||
CredentialEnvelope.__table__,
|
||||
],
|
||||
)
|
||||
self.session = sessionmaker(bind=self.engine)()
|
||||
self.row = FileConnectorProfile(
|
||||
id="profile-1",
|
||||
@@ -171,6 +179,50 @@ class ConnectorProfileStoreUpdateTests(unittest.TestCase):
|
||||
self.assertEqual([self.row.id], [profile.id for profile in profiles])
|
||||
self.assertEqual("original-password", profiles[0].password_value)
|
||||
|
||||
def test_reusable_credential_resolves_and_revocation_keeps_profile_visible(self) -> None:
|
||||
credential = CredentialEnvelope(
|
||||
id="shared-credential",
|
||||
tenant_id="tenant-1",
|
||||
scope_type="tenant",
|
||||
scope_id="tenant-1",
|
||||
name="Shared WebDAV login",
|
||||
credential_kind="username_password",
|
||||
public_data={"username": "ada"},
|
||||
secret_data_encrypted=encrypt_secret(json.dumps({"password": "secret"})),
|
||||
secret_keys=["password"],
|
||||
allowed_modules=["files"],
|
||||
allowed_server_refs=[],
|
||||
inherit_to_lower_scopes=True,
|
||||
is_active=True,
|
||||
revision="revision-1",
|
||||
)
|
||||
self.row.credential_profile_id = "credential-envelope:shared-credential"
|
||||
self.row.username = None
|
||||
self.row.password_encrypted = None
|
||||
self.row.token_encrypted = None
|
||||
self.session.add(credential)
|
||||
self.session.commit()
|
||||
|
||||
profile = list_database_connector_profiles(
|
||||
self.session,
|
||||
tenant_id="tenant-1",
|
||||
)[0]
|
||||
|
||||
self.assertEqual("ada", profile.username)
|
||||
self.assertEqual("secret", profile.password_value)
|
||||
self.assertTrue(profile.credentials_configured)
|
||||
|
||||
credential.is_active = False
|
||||
self.session.commit()
|
||||
profile = list_database_connector_profiles(
|
||||
self.session,
|
||||
tenant_id="tenant-1",
|
||||
)[0]
|
||||
|
||||
self.assertIsNone(profile.password_value)
|
||||
self.assertFalse(profile.credentials_configured)
|
||||
self.assertTrue(profile.metadata["credential_unavailable"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
196
tests/test_integrity_reconciliation.py
Normal file
196
tests/test_integrity_reconciliation.py
Normal file
@@ -0,0 +1,196 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.db.models import (
|
||||
FileBlob,
|
||||
FileIntegrityFinding,
|
||||
FileIntegrityScan,
|
||||
)
|
||||
from govoplan_files.backend.storage.backends import LocalFilesystemStorageBackend
|
||||
from govoplan_files.backend.storage.common import FileStorageError
|
||||
from govoplan_files.backend.storage.integrity import (
|
||||
cleanup_orphan_finding,
|
||||
create_integrity_scan,
|
||||
read_verified_blob_bytes,
|
||||
recheck_integrity_finding,
|
||||
run_integrity_scan_batch,
|
||||
)
|
||||
|
||||
|
||||
TENANT_ID = "tenant-1"
|
||||
USER_ID = "user-1"
|
||||
|
||||
|
||||
class IntegrityReconciliationTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary_directory = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary_directory.cleanup)
|
||||
self.backend = LocalFilesystemStorageBackend(
|
||||
Path(self.temporary_directory.name)
|
||||
)
|
||||
self.engine = create_engine("sqlite:///:memory:", future=True)
|
||||
Base.metadata.create_all(
|
||||
bind=self.engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
FileBlob.__table__,
|
||||
FileIntegrityScan.__table__,
|
||||
FileIntegrityFinding.__table__,
|
||||
],
|
||||
)
|
||||
self.session = sessionmaker(bind=self.engine, future=True)()
|
||||
self.addCleanup(self._close)
|
||||
|
||||
def _close(self) -> None:
|
||||
self.session.close()
|
||||
self.engine.dispose()
|
||||
|
||||
def test_scan_is_bounded_resumable_and_reconciles_both_orphan_directions(
|
||||
self,
|
||||
) -> None:
|
||||
valid_data = b"valid"
|
||||
restored_data = b"restore-me"
|
||||
expected_corrupt_data = b"expected"
|
||||
stored_corrupt_data = b"corrupt!"
|
||||
valid = _blob("blob-1", "valid.bin", valid_data)
|
||||
missing = _blob("blob-2", "missing.bin", restored_data)
|
||||
corrupt = _blob("blob-3", "corrupt.bin", expected_corrupt_data)
|
||||
self.session.add_all([valid, missing, corrupt])
|
||||
self.session.commit()
|
||||
self.backend.put_bytes(valid.storage_key, valid_data)
|
||||
self.backend.put_bytes(corrupt.storage_key, stored_corrupt_data)
|
||||
orphan_key = f"tenants/{TENANT_ID}/files/orphan.bin"
|
||||
self.backend.put_bytes(orphan_key, b"orphan")
|
||||
|
||||
scan = create_integrity_scan(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=USER_ID,
|
||||
batch_size=1,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.session.commit()
|
||||
|
||||
invocations = 0
|
||||
while scan.status != "completed":
|
||||
run_integrity_scan_batch(
|
||||
self.session,
|
||||
scan,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.session.commit()
|
||||
invocations += 1
|
||||
scan = self.session.get(FileIntegrityScan, scan.id)
|
||||
self.assertIsNotNone(scan)
|
||||
self.session.expire_all()
|
||||
if invocations > 12:
|
||||
self.fail("Integrity scan did not complete")
|
||||
|
||||
self.assertGreater(invocations, 3)
|
||||
self.assertEqual(3, scan.scanned_blob_count)
|
||||
self.assertEqual(1, scan.verified_blob_count)
|
||||
self.assertEqual(2, scan.quarantined_blob_count)
|
||||
self.assertEqual(3, scan.scanned_object_count)
|
||||
self.assertEqual(1, scan.orphan_object_count)
|
||||
findings = (
|
||||
self.session.query(FileIntegrityFinding)
|
||||
.filter(FileIntegrityFinding.scan_id == scan.id)
|
||||
.all()
|
||||
)
|
||||
self.assertEqual(
|
||||
{"missing", "checksum_mismatch", "orphan_object"},
|
||||
{finding.kind for finding in findings},
|
||||
)
|
||||
|
||||
missing = self.session.get(FileBlob, missing.id)
|
||||
corrupt = self.session.get(FileBlob, corrupt.id)
|
||||
self.assertEqual("missing", missing.integrity_status)
|
||||
self.assertEqual("checksum_mismatch", corrupt.integrity_status)
|
||||
with self.assertRaisesRegex(FileStorageError, "quarantined"):
|
||||
read_verified_blob_bytes(missing, backend=self.backend)
|
||||
|
||||
missing_finding = next(
|
||||
finding for finding in findings if finding.kind == "missing"
|
||||
)
|
||||
self.backend.put_bytes(missing.storage_key, restored_data)
|
||||
preview = recheck_integrity_finding(
|
||||
self.session,
|
||||
missing_finding,
|
||||
user_id=USER_ID,
|
||||
dry_run=True,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.assertTrue(preview.inspection.valid)
|
||||
self.assertEqual("open", missing_finding.state)
|
||||
repaired = recheck_integrity_finding(
|
||||
self.session,
|
||||
missing_finding,
|
||||
user_id=USER_ID,
|
||||
dry_run=False,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.session.commit()
|
||||
self.assertTrue(repaired.changed)
|
||||
self.assertEqual("resolved", missing_finding.state)
|
||||
self.assertEqual(
|
||||
restored_data,
|
||||
read_verified_blob_bytes(missing, backend=self.backend),
|
||||
)
|
||||
|
||||
orphan_finding = next(
|
||||
finding for finding in findings if finding.kind == "orphan_object"
|
||||
)
|
||||
preview_cleanup = cleanup_orphan_finding(
|
||||
self.session,
|
||||
orphan_finding,
|
||||
user_id=USER_ID,
|
||||
dry_run=True,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.assertEqual("would_delete", preview_cleanup.action)
|
||||
self.assertTrue(self.backend.exists(orphan_key))
|
||||
cleanup = cleanup_orphan_finding(
|
||||
self.session,
|
||||
orphan_finding,
|
||||
user_id=USER_ID,
|
||||
dry_run=False,
|
||||
backend=self.backend,
|
||||
)
|
||||
repeated = cleanup_orphan_finding(
|
||||
self.session,
|
||||
orphan_finding,
|
||||
user_id=USER_ID,
|
||||
dry_run=False,
|
||||
backend=self.backend,
|
||||
)
|
||||
self.assertEqual("deleted", cleanup.action)
|
||||
self.assertFalse(self.backend.exists(orphan_key))
|
||||
self.assertEqual("already_deleted", repeated.action)
|
||||
self.assertFalse(repeated.changed)
|
||||
|
||||
|
||||
def _blob(blob_id: str, filename: str, expected_data: bytes) -> FileBlob:
|
||||
return FileBlob(
|
||||
id=blob_id,
|
||||
tenant_id=TENANT_ID,
|
||||
storage_backend="local",
|
||||
storage_key=f"tenants/{TENANT_ID}/files/{filename}",
|
||||
checksum_sha256=hashlib.sha256(expected_data).hexdigest(),
|
||||
size_bytes=len(expected_data),
|
||||
ref_count=1,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -103,8 +103,8 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
def test_share_and_delete_tasks_state_current_boundaries(self) -> None:
|
||||
share = self.topic("files.workflow.share-managed-files")
|
||||
self.assertEqual("available", share.layer)
|
||||
self.assertIn("does not yet provide a general share editor", share.body)
|
||||
self.assertIn("no share-revocation route", share.body)
|
||||
self.assertIn("Expired and revoked grants stop authorizing", share.body)
|
||||
self.assertIn("revocation is idempotent", share.body)
|
||||
self.assertIn(
|
||||
"/api/v1/files/{file_id}/shares", {link.href for link in share.links}
|
||||
)
|
||||
@@ -154,8 +154,14 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("fail closed", topic.body)
|
||||
self.assertIn("DFS referrals", topic.body)
|
||||
self.assertIn("does not remove backend blob objects", topic.body)
|
||||
self.assertIn("bounded resumable integrity scan", topic.body)
|
||||
self.assertIn("quarantined", topic.body)
|
||||
self.assertIn("MASTER_KEY_B64", topic.metadata["recovery_unit"])
|
||||
self.assertTrue(topic.metadata["verification"])
|
||||
self.assertIn(
|
||||
"/api/v1/files/integrity/scans",
|
||||
{link.href for link in topic.links},
|
||||
)
|
||||
self.assertIn(
|
||||
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS", topic.configuration_keys
|
||||
)
|
||||
@@ -184,7 +190,7 @@ class FilesManifestDocumentationTests(unittest.TestCase):
|
||||
self.assertIn("process_owner", topic.audience)
|
||||
self.assertIn("release_manager", topic.audience)
|
||||
self.assertIn("versions align", topic.body)
|
||||
self.assertIn("no general share-management UI or share revocation", topic.body)
|
||||
self.assertIn("Share grant, change, expiry, and revocation", topic.body)
|
||||
self.assertIn("no enforced retention or legal hold", topic.body)
|
||||
for key in ("prerequisites", "steps", "outcome", "verification"):
|
||||
self.assertTrue(topic.metadata[key])
|
||||
|
||||
@@ -1,11 +1,59 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from collections import Counter
|
||||
from inspect import signature
|
||||
|
||||
from govoplan_files.backend.router import router
|
||||
from govoplan_files.backend.routes.assets import router as assets_router
|
||||
from govoplan_files.backend.routes.connector_io import router as connector_io_router
|
||||
from govoplan_files.backend.routes.connector_profiles import router as connector_profiles_router
|
||||
from govoplan_files.backend.routes.connector_settings import router as connector_settings_router
|
||||
from govoplan_files.backend.routes.folders import router as folders_router
|
||||
from govoplan_files.backend.routes.integrity import router as integrity_router
|
||||
from govoplan_files.backend.routes.listing import router as listing_router
|
||||
from govoplan_files.backend.routes.shares import router as shares_router
|
||||
from govoplan_files.backend.routes.spaces import router as spaces_router
|
||||
from govoplan_files.backend.routes.transfers import router as transfers_router
|
||||
from govoplan_files.backend.routes.uploads import router as uploads_router
|
||||
|
||||
|
||||
class FilesRouterContractTests(unittest.TestCase):
|
||||
@staticmethod
|
||||
def _operation_keys(candidate_router) -> list[tuple[str, str]]:
|
||||
return [
|
||||
(method, route.path)
|
||||
for route in candidate_router.routes
|
||||
for method in sorted(route.methods or ())
|
||||
]
|
||||
|
||||
def test_composed_router_contains_every_workflow_operation_once(self) -> None:
|
||||
workflow_routers = (
|
||||
spaces_router,
|
||||
folders_router,
|
||||
integrity_router,
|
||||
listing_router,
|
||||
uploads_router,
|
||||
connector_settings_router,
|
||||
connector_io_router,
|
||||
connector_profiles_router,
|
||||
assets_router,
|
||||
shares_router,
|
||||
transfers_router,
|
||||
)
|
||||
expected = [
|
||||
operation
|
||||
for workflow_router in workflow_routers
|
||||
for operation in self._operation_keys(workflow_router)
|
||||
]
|
||||
actual = self._operation_keys(router)
|
||||
|
||||
self.assertEqual(expected, actual)
|
||||
self.assertEqual(50, len(actual))
|
||||
self.assertFalse(
|
||||
[operation for operation, count in Counter(actual).items() if count > 1]
|
||||
)
|
||||
|
||||
def test_connector_routes_keep_existing_api_paths(self) -> None:
|
||||
routes = {(tuple(sorted(route.methods or ())), route.path) for route in router.routes}
|
||||
|
||||
@@ -31,6 +79,26 @@ class FilesRouterContractTests(unittest.TestCase):
|
||||
self.assertIn((("POST",), "/files/bulk-rename"), routes)
|
||||
self.assertIn((("POST",), "/files/transfer"), routes)
|
||||
|
||||
def test_share_lifecycle_routes_are_exposed(self) -> None:
|
||||
routes = {(tuple(sorted(route.methods or ())), route.path) for route in router.routes}
|
||||
|
||||
self.assertIn((("GET",), "/files/{file_id}/shares"), routes)
|
||||
self.assertIn((("POST",), "/files/{file_id}/shares"), routes)
|
||||
self.assertIn((("DELETE",), "/files/{file_id}/shares/{share_id}"), routes)
|
||||
self.assertIn((("GET",), "/files/{file_id}/share-target-options"), routes)
|
||||
|
||||
def test_file_listing_exposes_structured_property_filters(self) -> None:
|
||||
route = next(
|
||||
route
|
||||
for route in listing_router.routes
|
||||
if route.path == "/files" and "GET" in (route.methods or ())
|
||||
)
|
||||
parameters = signature(route.endpoint).parameters
|
||||
|
||||
self.assertIn("campaign_usage", parameters)
|
||||
self.assertIn("audit_relevant", parameters)
|
||||
self.assertIn("total", route.response_model.model_fields)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
219
tests/test_share_lifecycle.py
Normal file
219
tests/test_share_lifecycle.py
Normal file
@@ -0,0 +1,219 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from datetime import timedelta
|
||||
from unittest.mock import patch
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.db.models import FileAsset, FileShare
|
||||
from govoplan_files.backend.storage.common import FileStorageError, utcnow
|
||||
from govoplan_files.backend.storage.files import (
|
||||
get_asset_for_user,
|
||||
list_file_shares,
|
||||
revoke_file_share,
|
||||
share_file,
|
||||
)
|
||||
|
||||
|
||||
TENANT_ID = "tenant-1"
|
||||
OWNER_ID = "owner-1"
|
||||
RECIPIENT_ID = "recipient-1"
|
||||
GROUP_ID = "group-1"
|
||||
|
||||
|
||||
class FileShareLifecycleTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.engine = create_engine("sqlite:///:memory:", future=True)
|
||||
Base.metadata.create_all(
|
||||
bind=self.engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
FileAsset.__table__,
|
||||
FileShare.__table__,
|
||||
],
|
||||
)
|
||||
self.session = sessionmaker(bind=self.engine, future=True)()
|
||||
self.asset = FileAsset(
|
||||
id="file-1",
|
||||
tenant_id=TENANT_ID,
|
||||
owner_type="user",
|
||||
owner_user_id=OWNER_ID,
|
||||
display_path="shared.pdf",
|
||||
filename="shared.pdf",
|
||||
)
|
||||
self.session.add(self.asset)
|
||||
self.session.commit()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.session.close()
|
||||
self.engine.dispose()
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.files.user_group_ids",
|
||||
return_value=[],
|
||||
)
|
||||
def test_expired_share_stops_access_immediately(self, _groups) -> None:
|
||||
self.session.add(
|
||||
FileShare(
|
||||
id="expired-share",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=self.asset.id,
|
||||
target_type="user",
|
||||
target_id=RECIPIENT_ID,
|
||||
permission="read",
|
||||
expires_at=utcnow() - timedelta(seconds=1),
|
||||
)
|
||||
)
|
||||
self.session.commit()
|
||||
|
||||
with self.assertRaisesRegex(FileStorageError, "No access"):
|
||||
get_asset_for_user(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=RECIPIENT_ID,
|
||||
asset_id=self.asset.id,
|
||||
)
|
||||
self.assertEqual(
|
||||
[],
|
||||
list_file_shares(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
),
|
||||
)
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.files.user_group_ids",
|
||||
return_value=[GROUP_ID],
|
||||
)
|
||||
def test_independent_active_grant_survives_other_expiry(self, _groups) -> None:
|
||||
self.session.add_all(
|
||||
[
|
||||
FileShare(
|
||||
id="expired-user-share",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=self.asset.id,
|
||||
target_type="user",
|
||||
target_id=RECIPIENT_ID,
|
||||
permission="read",
|
||||
expires_at=utcnow() - timedelta(seconds=1),
|
||||
),
|
||||
FileShare(
|
||||
id="active-group-share",
|
||||
tenant_id=TENANT_ID,
|
||||
file_asset_id=self.asset.id,
|
||||
target_type="group",
|
||||
target_id=GROUP_ID,
|
||||
permission="read",
|
||||
expires_at=utcnow() + timedelta(hours=1),
|
||||
),
|
||||
]
|
||||
)
|
||||
self.session.commit()
|
||||
|
||||
result = get_asset_for_user(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
user_id=RECIPIENT_ID,
|
||||
asset_id=self.asset.id,
|
||||
)
|
||||
|
||||
self.assertEqual(self.asset.id, result.id)
|
||||
self.assertEqual(
|
||||
["active-group-share"],
|
||||
[
|
||||
share.id
|
||||
for share in list_file_shares(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.files.ensure_share_target_exists",
|
||||
return_value=None,
|
||||
)
|
||||
def test_revoke_is_idempotent_and_future_expiry_is_persisted(
|
||||
self, _target_exists
|
||||
) -> None:
|
||||
expiry = utcnow() + timedelta(days=1)
|
||||
share = share_file(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset=self.asset,
|
||||
target_type="user",
|
||||
target_id=RECIPIENT_ID,
|
||||
permission="read",
|
||||
user_id=OWNER_ID,
|
||||
expires_at=expiry,
|
||||
)
|
||||
self.session.commit()
|
||||
|
||||
revoked, first_changed = revoke_file_share(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
share_id=share.id,
|
||||
user_id=OWNER_ID,
|
||||
)
|
||||
self.session.commit()
|
||||
repeated, second_changed = revoke_file_share(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
share_id=share.id,
|
||||
user_id=OWNER_ID,
|
||||
)
|
||||
|
||||
self.assertTrue(first_changed)
|
||||
self.assertFalse(second_changed)
|
||||
self.assertEqual(OWNER_ID, revoked.revoked_by_user_id)
|
||||
self.assertEqual(revoked.revoked_at, repeated.revoked_at)
|
||||
self.assertEqual([], list_file_shares(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
))
|
||||
self.assertEqual(
|
||||
[share.id],
|
||||
[
|
||||
item.id
|
||||
for item in list_file_shares(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset_id=self.asset.id,
|
||||
include_inactive=True,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@patch(
|
||||
"govoplan_files.backend.storage.files.ensure_share_target_exists",
|
||||
return_value=None,
|
||||
)
|
||||
def test_past_expiry_is_rejected(self, _target_exists) -> None:
|
||||
with self.assertRaisesRegex(FileStorageError, "future"):
|
||||
share_file(
|
||||
self.session,
|
||||
tenant_id=TENANT_ID,
|
||||
asset=self.asset,
|
||||
target_type="user",
|
||||
target_id=RECIPIENT_ID,
|
||||
permission="read",
|
||||
user_id=OWNER_ID,
|
||||
expires_at=utcnow() - timedelta(seconds=1),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
108
tests/test_tenant_summary_batch.py
Normal file
108
tests/test_tenant_summary_batch.py
Normal file
@@ -0,0 +1,108 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from sqlalchemy import create_engine, event
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_access.backend.db.models import Account, Group, User
|
||||
from govoplan_core.core.change_sequence import ChangeSequenceEntry
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_files.backend.db.models import (
|
||||
FileAsset,
|
||||
FileConnectorCredential,
|
||||
FileConnectorPolicy,
|
||||
FileConnectorProfile,
|
||||
FileConnectorSpace,
|
||||
)
|
||||
from govoplan_files.backend.manifest import _tenant_summary_batch
|
||||
|
||||
|
||||
class FilesTenantSummaryBatchTests(unittest.TestCase):
|
||||
def test_batch_summary_uses_one_grouped_query_per_owned_table(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
Base.metadata.create_all(
|
||||
engine,
|
||||
tables=[
|
||||
Account.__table__,
|
||||
User.__table__,
|
||||
Group.__table__,
|
||||
FileAsset.__table__,
|
||||
FileConnectorCredential.__table__,
|
||||
FileConnectorPolicy.__table__,
|
||||
FileConnectorProfile.__table__,
|
||||
FileConnectorSpace.__table__,
|
||||
ChangeSequenceEntry.__table__,
|
||||
],
|
||||
)
|
||||
try:
|
||||
with Session(engine) as session:
|
||||
session.add_all(
|
||||
[
|
||||
FileAsset(
|
||||
id="file-1",
|
||||
tenant_id="tenant-1",
|
||||
owner_type="tenant",
|
||||
display_path="/one.txt",
|
||||
filename="one.txt",
|
||||
),
|
||||
FileConnectorCredential(
|
||||
id="credential-1",
|
||||
tenant_id="tenant-1",
|
||||
label="Credential",
|
||||
),
|
||||
FileConnectorPolicy(
|
||||
id="policy-1",
|
||||
tenant_id="tenant-1",
|
||||
scope_type="tenant",
|
||||
),
|
||||
FileConnectorProfile(
|
||||
id="profile-1",
|
||||
tenant_id="tenant-1",
|
||||
label="Profile",
|
||||
provider="webdav",
|
||||
),
|
||||
FileConnectorSpace(
|
||||
id="space-1",
|
||||
tenant_id="tenant-1",
|
||||
owner_type="tenant",
|
||||
label="Space",
|
||||
connector_profile_id="profile-1",
|
||||
provider="webdav",
|
||||
),
|
||||
]
|
||||
)
|
||||
session.commit()
|
||||
query_count = 0
|
||||
|
||||
def count_query(*_args: object) -> None:
|
||||
nonlocal query_count
|
||||
query_count += 1
|
||||
|
||||
event.listen(engine, "before_cursor_execute", count_query)
|
||||
try:
|
||||
counts = _tenant_summary_batch(
|
||||
session,
|
||||
["tenant-1", "tenant-empty"],
|
||||
)
|
||||
finally:
|
||||
event.remove(engine, "before_cursor_execute", count_query)
|
||||
|
||||
self.assertEqual(5, query_count)
|
||||
self.assertEqual(
|
||||
{
|
||||
"files": 1,
|
||||
"connector_credentials": 1,
|
||||
"connector_policies": 1,
|
||||
"connector_profiles": 1,
|
||||
"connector_spaces": 1,
|
||||
},
|
||||
counts["tenant-1"],
|
||||
)
|
||||
self.assertEqual(0, counts["tenant-empty"]["files"])
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -14,7 +14,8 @@
|
||||
"./styles/file-manager.css": "./src/styles/file-manager.css"
|
||||
},
|
||||
"scripts": {
|
||||
"test:file-drop-target": "node scripts/test-file-drop-target-structure.mjs"
|
||||
"test:file-drop-target": "node scripts/test-file-drop-target-structure.mjs",
|
||||
"test:file-property-filters": "node scripts/test-file-property-filters-structure.mjs"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@vitejs/plugin-react": "^4.3.4",
|
||||
@@ -22,7 +23,7 @@
|
||||
"typescript": "^5.7.2",
|
||||
"react": "^19.0.0",
|
||||
"react-dom": "^19.0.0",
|
||||
"react-router-dom": "^7.1.1",
|
||||
"react-router-dom": ">=7.18.2 <8",
|
||||
"lucide-react": "^1.23.0",
|
||||
"@govoplan/core-webui": "^0.1.9"
|
||||
},
|
||||
|
||||
36
webui/scripts/test-file-property-filters-structure.mjs
Normal file
36
webui/scripts/test-file-property-filters-structure.mjs
Normal file
@@ -0,0 +1,36 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const page = readFileSync(new URL("../src/features/files/FilesPage.tsx", import.meta.url), "utf8").replace(/\s+/g, " ");
|
||||
const api = readFileSync(new URL("../src/api/files.ts", import.meta.url), "utf8").replace(/\s+/g, " ");
|
||||
|
||||
function assertIncludes(source, fragment, message) {
|
||||
if (!source.includes(fragment.replace(/\s+/g, " "))) throw new Error(message);
|
||||
}
|
||||
|
||||
assertIncludes(
|
||||
api,
|
||||
"campaign_usage?: FileCampaignUsageFilter; audit_relevant?: boolean;",
|
||||
"the Files API must expose structured campaign and audit filters"
|
||||
);
|
||||
assertIncludes(
|
||||
api,
|
||||
"do { const response = await listFiles",
|
||||
"property filtering must traverse every server result page"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
"if (propertyFiltersActive && propertyFilterResults) return propertyFilterResults;",
|
||||
"server-filtered rows must replace, not narrow, the currently loaded explorer window"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
'<option value="unlinked">Not linked or used</option>',
|
||||
"the high-priority unlinked campaign-file filter must remain available"
|
||||
);
|
||||
assertIncludes(
|
||||
page,
|
||||
"setPropertyFilterTotal(response.total);",
|
||||
"the UI must retain the exact server-side filtered count"
|
||||
);
|
||||
|
||||
console.log("Files property filters cover the complete server result window.");
|
||||
@@ -1,4 +1,15 @@
|
||||
import { ApiError, apiFetch, apiUrl, authHeaders, csrfToken, type ApiSettings, type DeltaDeletedItem, type FilesManagedFileLinkTarget } from "@govoplan/core-webui";
|
||||
import {
|
||||
ApiError,
|
||||
apiFetch,
|
||||
apiReferenceOptionProvider,
|
||||
apiUrl,
|
||||
authHeaders,
|
||||
csrfToken,
|
||||
type ApiSettings,
|
||||
type DeltaDeletedItem,
|
||||
type FilesManagedFileLinkTarget,
|
||||
type ReferenceOptionProvider
|
||||
} from "@govoplan/core-webui";
|
||||
export { fetchResourceAccessExplanation } from "@govoplan/core-webui";
|
||||
export type {
|
||||
AccessDecisionProvenanceItem,
|
||||
@@ -24,11 +35,16 @@ export type FileSpace = {
|
||||
|
||||
export type FileShare = {
|
||||
id: string;
|
||||
file_asset_id: string;
|
||||
target_type: string;
|
||||
target_id: string;
|
||||
permission: string;
|
||||
created_by_user_id?: string | null;
|
||||
created_at: string;
|
||||
expires_at?: string | null;
|
||||
revoked_at?: string | null;
|
||||
revoked_by_user_id?: string | null;
|
||||
active: boolean;
|
||||
};
|
||||
|
||||
export type FileSourceProvenance = {
|
||||
@@ -297,7 +313,8 @@ export type ManagedFile = {
|
||||
shares?: FileShare[];
|
||||
};
|
||||
|
||||
export type FileListResponse = {files: ManagedFile[];cursor?: string | null;next_cursor?: string | null;watermark?: string | null;};
|
||||
export type FileCampaignUsageFilter = "linked" | "unlinked";
|
||||
export type FileListResponse = {files: ManagedFile[];total: number;cursor?: string | null;next_cursor?: string | null;watermark?: string | null;};
|
||||
export type FileDeltaDeletedItem = {id: string;resource_type?: string | null;revision?: string | null;deleted_at?: string | null;};
|
||||
export type FileDeltaResponse = {
|
||||
files: ManagedFile[];
|
||||
@@ -383,15 +400,43 @@ payload: {owner_type: "user" | "group";owner_id: string;path: string;recursive?:
|
||||
return apiFetch<FolderDeleteResponse>(settings, "/api/v1/files/folders/delete", { method: "POST", body: JSON.stringify({ recursive: true, ...payload }) });
|
||||
}
|
||||
|
||||
export function listFiles(settings: ApiSettings, params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;page_size?: number;cursor?: string | null;} = {}): Promise<FileListResponse> {
|
||||
export function listFiles(settings: ApiSettings, params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;campaign_usage?: FileCampaignUsageFilter;audit_relevant?: boolean;page_size?: number;cursor?: string | null;} = {}): Promise<FileListResponse> {
|
||||
const search = new URLSearchParams();
|
||||
for (const [key, value] of Object.entries(params)) {
|
||||
if (value) search.set(key, String(value));
|
||||
if (value !== undefined && value !== null && value !== "") search.set(key, String(value));
|
||||
}
|
||||
const suffix = search.toString() ? `?${search.toString()}` : "";
|
||||
return apiFetch<FileListResponse>(settings, `/api/v1/files${suffix}`);
|
||||
}
|
||||
|
||||
export async function listFilesByProperties(
|
||||
settings: ApiSettings,
|
||||
params: {
|
||||
owner_type: "user" | "group";
|
||||
owner_id: string;
|
||||
path_prefix?: string;
|
||||
campaign_usage?: FileCampaignUsageFilter;
|
||||
audit_relevant?: boolean;
|
||||
page_size?: number;
|
||||
})
|
||||
: Promise<{files: ManagedFile[];total: number;}> {
|
||||
const pageSize = params.page_size ?? DEFAULT_MANAGED_FILE_WINDOW_SIZE;
|
||||
let cursor: string | null | undefined = null;
|
||||
let total = 0;
|
||||
let files: ManagedFile[] = [];
|
||||
do {
|
||||
const response = await listFiles(settings, {
|
||||
...params,
|
||||
page_size: pageSize,
|
||||
cursor
|
||||
});
|
||||
files = files.concat(response.files);
|
||||
total = response.total;
|
||||
cursor = response.next_cursor;
|
||||
} while (cursor);
|
||||
return { files, total };
|
||||
}
|
||||
|
||||
export function listFilesDelta(
|
||||
settings: ApiSettings,
|
||||
params: {owner_type?: string;owner_id?: string;campaign_id?: string;path_prefix?: string;since?: string;limit?: number;} = {})
|
||||
@@ -564,6 +609,12 @@ export function bulkDeleteFiles(settings: ApiSettings, fileIds: string[]): Promi
|
||||
}
|
||||
|
||||
export type FileBulkShareResponse = {shares: FileShare[];shared_count: number;};
|
||||
export type FileSharePayload = {
|
||||
target_type: "user" | "group" | "tenant" | "campaign";
|
||||
target_id: string;
|
||||
permission: "read" | "write" | "manage";
|
||||
expires_at?: string | null;
|
||||
};
|
||||
|
||||
export function virtualFolderResourceId(options: {tenantId: string;ownerType: "user" | "group";ownerId: string;path: string;}): string {
|
||||
return `virtual-folder:v1:${options.tenantId}:${options.ownerType}:${options.ownerId}:${base64Url(options.path.replace(/\\/g, "/").replace(/^\/+|\/+$/g, ""))}`;
|
||||
@@ -583,6 +634,54 @@ export function shareFilesWithTarget(settings: ApiSettings, fileIds: string[], t
|
||||
});
|
||||
}
|
||||
|
||||
export function listFileShares(
|
||||
settings: ApiSettings,
|
||||
fileId: string,
|
||||
options: { includeInactive?: boolean } = {}
|
||||
): Promise<{ shares: FileShare[] }> {
|
||||
const suffix = options.includeInactive ? "?include_inactive=true" : "";
|
||||
return apiFetch<{ shares: FileShare[] }>(
|
||||
settings,
|
||||
`/api/v1/files/${encodeURIComponent(fileId)}/shares${suffix}`
|
||||
);
|
||||
}
|
||||
|
||||
export function createFileShare(
|
||||
settings: ApiSettings,
|
||||
fileId: string,
|
||||
payload: FileSharePayload
|
||||
): Promise<FileShare> {
|
||||
return apiFetch<FileShare>(
|
||||
settings,
|
||||
`/api/v1/files/${encodeURIComponent(fileId)}/shares`,
|
||||
{ method: "POST", body: JSON.stringify(payload) }
|
||||
);
|
||||
}
|
||||
|
||||
export function revokeFileShare(
|
||||
settings: ApiSettings,
|
||||
fileId: string,
|
||||
shareId: string
|
||||
): Promise<FileShare> {
|
||||
return apiFetch<FileShare>(
|
||||
settings,
|
||||
`/api/v1/files/${encodeURIComponent(fileId)}/shares/${encodeURIComponent(shareId)}`,
|
||||
{ method: "DELETE" }
|
||||
);
|
||||
}
|
||||
|
||||
export function fileShareTargetProvider(
|
||||
settings: ApiSettings,
|
||||
fileId: string,
|
||||
targetType: "user" | "group"
|
||||
): ReferenceOptionProvider {
|
||||
return apiReferenceOptionProvider(
|
||||
settings,
|
||||
`/api/v1/files/${encodeURIComponent(fileId)}/share-target-options`,
|
||||
{ target_type: targetType }
|
||||
);
|
||||
}
|
||||
|
||||
export function evaluateFileConnectorPolicy(
|
||||
settings: ApiSettings,
|
||||
payload: {source_provenance: FileSourceProvenance;operation?: string;policy_sources?: FileConnectorPolicySource[];})
|
||||
|
||||
@@ -14,16 +14,21 @@ import {
|
||||
AdvancedOptionsPanel,
|
||||
LoadingFrame,
|
||||
mergeDeltaRows,
|
||||
ReferenceMultiSelect,
|
||||
SegmentedControl,
|
||||
staticReferenceOptionProvider,
|
||||
StatusBadge,
|
||||
TableActionGroup,
|
||||
ToggleSwitch,
|
||||
useDeltaWatermarks,
|
||||
usePlatformLanguage,
|
||||
useUnsavedDraftGuard,
|
||||
wildcardReferenceOption,
|
||||
type ApiSettings,
|
||||
type ConnectionTreeColumn,
|
||||
type FileConnectorScope,
|
||||
type FileConnectorTargetOption,
|
||||
type ReferenceOption,
|
||||
i18nMessage } from
|
||||
"@govoplan/core-webui";
|
||||
import {
|
||||
@@ -186,6 +191,7 @@ export default function FileConnectorSettingsPanel({
|
||||
const [credentialLoginResult, setCredentialLoginResult] = useState<{ok: boolean;message: string;} | null>(null);
|
||||
const [message, setMessage] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const { translateText } = usePlatformLanguage();
|
||||
const { getDeltaWatermark, setDeltaWatermark, resetDeltaWatermark } = useDeltaWatermarks();
|
||||
|
||||
const requiresTarget = scopeType === "user" || scopeType === "group";
|
||||
@@ -198,6 +204,10 @@ export default function FileConnectorSettingsPanel({
|
||||
const profileDirty = Boolean(draft) && connectorProfileDraftKey(draft) !== savedDraftKey;
|
||||
const credentialDirty = Boolean(credentialDraft) && connectorCredentialDraftKey(credentialDraft) !== savedCredentialDraftKey;
|
||||
const policyDirty = scopeReady && centralPolicyDraftKey(policyDraft) !== savedPolicyDraftKey;
|
||||
const policyConflicts = useMemo(
|
||||
() => policyReferenceConflicts(policyDraft),
|
||||
[policyDraft]
|
||||
);
|
||||
|
||||
useUnsavedDraftGuard({
|
||||
dirty: profileDirty || credentialDirty || policyDirty,
|
||||
@@ -225,6 +235,79 @@ export default function FileConnectorSettingsPanel({
|
||||
() => credentials.filter((credential) => connectorBelongsToEditableScope(credential, scopeType, activeScopeId)),
|
||||
[activeScopeId, credentials, scopeType]
|
||||
);
|
||||
const selectableCredentials = useMemo(
|
||||
() => credentials.filter((credential) =>
|
||||
connectorBelongsToEditableScope(credential, scopeType, activeScopeId) ||
|
||||
credential.source_kind === "credential_envelope" &&
|
||||
reusableCredentialAvailableAtScope(credential, scopeType, activeScopeId)
|
||||
),
|
||||
[activeScopeId, credentials, scopeType]
|
||||
);
|
||||
const connectorReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => profiles.map((profile) => ({
|
||||
value: profile.id,
|
||||
label: profile.label || profile.id,
|
||||
description: [
|
||||
profile.provider,
|
||||
profile.source_path,
|
||||
profile.enabled ? null : "Inactive"
|
||||
].filter(Boolean).join(" · "),
|
||||
kind: "file_connection",
|
||||
availability: profile.enabled ? "available" : "inactive",
|
||||
disabled: !profile.enabled,
|
||||
sourceModule: "files",
|
||||
provenance: {
|
||||
scopeType: profile.scope_type,
|
||||
scopeId: profile.scope_id,
|
||||
sourcePath: profile.source_path
|
||||
}
|
||||
})),
|
||||
[profiles]
|
||||
);
|
||||
const credentialReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => credentials.map((credential) => ({
|
||||
value: credential.id,
|
||||
label: credential.label || credential.id,
|
||||
description: [
|
||||
credential.provider || "shared",
|
||||
credential.source_path,
|
||||
credential.enabled ? null : "Inactive"
|
||||
].filter(Boolean).join(" · "),
|
||||
kind: "file_credential",
|
||||
availability: credential.enabled ? "available" : "inactive",
|
||||
disabled: !credential.enabled,
|
||||
sourceModule: credential.source_kind === "credential_envelope" ? "core" : "files",
|
||||
provenance: {
|
||||
scopeType: credential.scope_type,
|
||||
scopeId: credential.scope_id,
|
||||
sourcePath: credential.source_path,
|
||||
sourceKind: credential.source_kind
|
||||
}
|
||||
})),
|
||||
[credentials]
|
||||
);
|
||||
const providerReferenceOptions = useMemo<ReferenceOption[]>(
|
||||
() => PROVIDERS.map((provider) => ({
|
||||
value: provider.id,
|
||||
label: translateText(provider.label),
|
||||
description: provider.id,
|
||||
kind: "file_provider",
|
||||
sourceModule: "files"
|
||||
})),
|
||||
[translateText]
|
||||
);
|
||||
const connectorReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(connectorReferenceOptions),
|
||||
[connectorReferenceOptions]
|
||||
);
|
||||
const credentialReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(credentialReferenceOptions),
|
||||
[credentialReferenceOptions]
|
||||
);
|
||||
const providerReferenceProvider = useMemo(
|
||||
() => staticReferenceOptionProvider(providerReferenceOptions),
|
||||
[providerReferenceOptions]
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
resetDeltaWatermark(deltaKey);
|
||||
@@ -424,7 +507,7 @@ export default function FileConnectorSettingsPanel({
|
||||
endpoint_url: cleanOrNull(draft.endpointUrl),
|
||||
base_path: cleanOrNull(draft.basePath),
|
||||
enabled: draft.enabled,
|
||||
credential_profile_id: cleanOrNull(credentialProfileIdForDraft(draft, scopedCredentials)),
|
||||
credential_profile_id: cleanOrNull(credentialProfileIdForDraft(draft, selectableCredentials)),
|
||||
credential_mode: draft.credentialMode,
|
||||
capabilities,
|
||||
policy: policyFromDraft(draft),
|
||||
@@ -694,7 +777,7 @@ export default function FileConnectorSettingsPanel({
|
||||
const panelTitle = title ?? i18nMessage("i18n:govoplan-files.value_file_connections", { value0: scopeLabel(scopeType) });
|
||||
const selectedCredentialIds = new Set(scopedProfiles.map((profile) => profile.credential_profile_id).filter((value): value is string => Boolean(value)));
|
||||
const firstCompatibleProfileByCredential = new Map<string, string>();
|
||||
for (const credential of scopedCredentials) {
|
||||
for (const credential of selectableCredentials) {
|
||||
const selectedProfile = scopedProfiles.find((profile) => profile.credential_profile_id === credential.id);
|
||||
const fallbackProfile = selectedProfile ?? scopedProfiles.find((profile) => !selectedCredentialIds.has(credential.id) && credentialMatchesProfile(credential, profile));
|
||||
if (fallbackProfile) firstCompatibleProfileByCredential.set(credential.id, fallbackProfile.id);
|
||||
@@ -717,8 +800,8 @@ export default function FileConnectorSettingsPanel({
|
||||
const credentialTestUnsupported = Boolean(credentialTestProfile && credentialTestProfile.provider !== "webdav" && credentialTestProfile.provider !== "nextcloud");
|
||||
const credentialTestDisabled = !credentialDraft || saving || testingCredentialLogin || !credentialTestProfile || credentialTestUnsupported;
|
||||
const credentialTestTooltip = credentialTestHelpText(credentialTestProfile, credentialTestUnsupported);
|
||||
const profileCredentialOptions = draft ? credentialOptionsForProfileDraft(scopedCredentials, draft) : [];
|
||||
const profileCredentialSelectValue = draft ? credentialProfileIdForDraft(draft, scopedCredentials) : "";
|
||||
const profileCredentialOptions = draft ? credentialOptionsForProfileDraft(selectableCredentials, draft) : [];
|
||||
const profileCredentialSelectValue = draft ? credentialProfileIdForDraft(draft, selectableCredentials) : "";
|
||||
const connectorColumns: ConnectionTreeColumn<ConnectorTreeRow>[] = [
|
||||
{
|
||||
id: "connection",
|
||||
@@ -755,7 +838,7 @@ export default function FileConnectorSettingsPanel({
|
||||
|
||||
function connectorChildren(row: ConnectorTreeRow): ConnectorTreeRow[] {
|
||||
if (row.kind !== "profile") return [];
|
||||
return scopedCredentials.
|
||||
return selectableCredentials.
|
||||
filter((credential) => firstCompatibleProfileByCredential.get(credential.id) === row.profile.id).
|
||||
map((credential) => ({ kind: "credential" as const, id: `credential:${row.profile.id}:${credential.id}`, credential, profile: row.profile }));
|
||||
}
|
||||
@@ -857,22 +940,70 @@ export default function FileConnectorSettingsPanel({
|
||||
<>
|
||||
<div className="form-grid two">
|
||||
<FormField label="i18n:govoplan-files.allowed_connection_ids.0df854c8">
|
||||
<textarea value={policyDraft.allowedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedConnectors: event.target.value })} rows={3} placeholder={"tenant-webdav\nseafile-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedConnectors)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedConnectors: values.join("\n") })}
|
||||
provider={connectorReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file connections"
|
||||
placeholder="Add a connection or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_connection_ids.a95218b4">
|
||||
<textarea value={policyDraft.deniedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedConnectors: event.target.value })} rows={3} placeholder={"legacy-*\nblocked-smb"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedConnectors)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedConnectors: values.join("\n") })}
|
||||
provider={connectorReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file connections"
|
||||
placeholder="Add a connection or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_credential_ids.efcaba2d">
|
||||
<textarea value={policyDraft.allowedCredentials} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedCredentials: event.target.value })} rows={3} placeholder={"tenant-webdav-credentials\nshared-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedCredentials)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedCredentials: values.join("\n") })}
|
||||
provider={credentialReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file credentials"
|
||||
placeholder="Add a credential or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_credential_ids.b6838bc2">
|
||||
<textarea value={policyDraft.deniedCredentials} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedCredentials: event.target.value })} rows={3} placeholder={"personal-*\nblocked-*"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedCredentials)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedCredentials: values.join("\n") })}
|
||||
provider={credentialReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file credentials"
|
||||
placeholder="Add a credential or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_providers.82a9c23e">
|
||||
<textarea value={policyDraft.allowedProviders} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedProviders: event.target.value })} rows={3} placeholder={"webdav\nnextcloud\nsmb"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.allowedProviders)}
|
||||
onChange={(values) => patchPolicyDraft({ allowedProviders: values.join("\n") })}
|
||||
provider={providerReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Allowed file providers"
|
||||
placeholder="Add a provider or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.denied_providers.149b29f4">
|
||||
<textarea value={policyDraft.deniedProviders} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedProviders: event.target.value })} rows={3} placeholder={"generic\nnfs"} />
|
||||
<ReferenceMultiSelect
|
||||
values={lines(policyDraft.deniedProviders)}
|
||||
onChange={(values) => patchPolicyDraft({ deniedProviders: values.join("\n") })}
|
||||
provider={providerReferenceProvider}
|
||||
createCustomOption={(value) => wildcardReferenceOption(value)}
|
||||
aria-label="Denied file providers"
|
||||
placeholder="Add a provider or wildcard pattern"
|
||||
disabled={saving || !canWrite}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="i18n:govoplan-files.allowed_paths.c953b4be">
|
||||
<textarea value={policyDraft.allowedPaths} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedPaths: event.target.value })} rows={3} placeholder={"GovOPlaN\nreports/*"} />
|
||||
@@ -887,6 +1018,20 @@ export default function FileConnectorSettingsPanel({
|
||||
<textarea value={policyDraft.deniedUrls} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ deniedUrls: event.target.value })} rows={3} placeholder={"http://*\n*://legacy.example.test/*"} />
|
||||
</FormField>
|
||||
</div>
|
||||
{policyConflicts.length ? (
|
||||
<DismissibleAlert
|
||||
tone="warning"
|
||||
resetKey={policyConflicts.join("\u0000")}
|
||||
>
|
||||
Allow and deny rules contain the same values for{" "}
|
||||
{policyConflicts.join("; ")}. Deny rules take precedence.
|
||||
</DismissibleAlert>
|
||||
) : (
|
||||
<p className="muted small-note">
|
||||
When an allow and deny rule both match, the deny rule takes
|
||||
precedence.
|
||||
</p>
|
||||
)}
|
||||
<div className="file-connector-settings-section file-connector-policy-locks">
|
||||
<ToggleSwitch label="i18n:govoplan-files.lower_connection_limits.4f9838bc" checked={policyDraft.allowLowerConnectionLimits} disabled={saving || !canWrite} onChange={(allowLowerConnectionLimits) => patchPolicyDraft({ allowLowerConnectionLimits })} />
|
||||
<ToggleSwitch label="i18n:govoplan-files.lower_credential_limits.ec2b72bc" checked={policyDraft.allowLowerCredentialLimits} disabled={saving || !canWrite} onChange={(allowLowerCredentialLimits) => patchPolicyDraft({ allowLowerCredentialLimits })} />
|
||||
@@ -1432,10 +1577,39 @@ function connectorCredentialDraftKey(draft: ConnectorCredentialDraft): string {
|
||||
function credentialOptionsForProfileDraft(credentials: FileConnectorCredential[], draft: ConnectorProfileDraft): FileConnectorCredential[] {
|
||||
return credentials.filter((credential) =>
|
||||
credential.id === draft.credentialProfileId ||
|
||||
credential.enabled && (!credential.provider || credential.provider === draft.provider)
|
||||
credential.enabled &&
|
||||
(!credential.provider || credential.provider === draft.provider) &&
|
||||
reusableCredentialAllowsProfile(credential, draft.id)
|
||||
);
|
||||
}
|
||||
|
||||
function reusableCredentialAvailableAtScope(
|
||||
credential: FileConnectorCredential,
|
||||
scopeType: ConnectorScope,
|
||||
scopeId: string | null
|
||||
): boolean {
|
||||
if (connectorBelongsToEditableScope(credential, scopeType, scopeId)) return true;
|
||||
if (credential.source_kind !== "credential_envelope") return false;
|
||||
const inherited = credential.metadata?.inherit_to_lower_scopes === true;
|
||||
if (!inherited) return false;
|
||||
if (credential.scope_type === "system") return scopeType !== "system";
|
||||
if (credential.scope_type === "tenant") {
|
||||
return scopeType === "group" || scopeType === "user";
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function reusableCredentialAllowsProfile(
|
||||
credential: FileConnectorCredential,
|
||||
profileId: string
|
||||
): boolean {
|
||||
if (credential.source_kind !== "credential_envelope") return true;
|
||||
const refs = Array.isArray(credential.metadata?.allowed_server_refs)
|
||||
? credential.metadata.allowed_server_refs.filter((value): value is string => typeof value === "string")
|
||||
: [];
|
||||
return refs.length === 0 || refs.includes(`files:${profileId}`);
|
||||
}
|
||||
|
||||
function credentialProfileIdForDraft(draft: ConnectorProfileDraft, credentials: FileConnectorCredential[]): string {
|
||||
const options = credentialOptionsForProfileDraft(credentials, draft);
|
||||
if (draft.credentialProfileId && options.some((credential) => credential.id === draft.credentialProfileId)) {
|
||||
@@ -1656,6 +1830,26 @@ function lines(value: string): string[] {
|
||||
return value.split(/\r?\n/).map((line) => line.trim()).filter(Boolean);
|
||||
}
|
||||
|
||||
function policyReferenceConflicts(
|
||||
draft: CentralConnectorPolicyDraft
|
||||
): string[] {
|
||||
return [
|
||||
["connections", draft.allowedConnectors, draft.deniedConnectors],
|
||||
["credentials", draft.allowedCredentials, draft.deniedCredentials],
|
||||
["providers", draft.allowedProviders, draft.deniedProviders],
|
||||
["paths", draft.allowedPaths, draft.deniedPaths],
|
||||
["endpoint URLs", draft.allowedUrls, draft.deniedUrls]
|
||||
].flatMap(([label, allowed, denied]) => {
|
||||
const deniedValues = new Set(
|
||||
lines(denied).map((value) => value.toLocaleLowerCase())
|
||||
);
|
||||
const overlaps = lines(allowed).filter((value) =>
|
||||
deniedValues.has(value.toLocaleLowerCase())
|
||||
);
|
||||
return overlaps.length ? [`${label}: ${overlaps.join(", ")}`] : [];
|
||||
});
|
||||
}
|
||||
|
||||
function policyRuleList(policy: Record<string, unknown>, kind: "allow" | "deny", field: "connectors" | "providers" | "credentials" | "external_paths" | "external_urls"): string[] {
|
||||
const rule = recordValue(policy[kind]) ??
|
||||
recordValue(policy[kind === "allow" ? "allowlist" : "denylist"]) ??
|
||||
|
||||
98
webui/src/features/files/FileSpacesWidget.tsx
Normal file
98
webui/src/features/files/FileSpacesWidget.tsx
Normal file
@@ -0,0 +1,98 @@
|
||||
import { useCallback } from "react";
|
||||
import { Folder, Network } from "lucide-react";
|
||||
import { Link } from "react-router-dom";
|
||||
import {
|
||||
DashboardWidgetList,
|
||||
DismissibleAlert,
|
||||
LoadingFrame,
|
||||
StatusBadge,
|
||||
useDashboardWidgetData,
|
||||
type ApiSettings,
|
||||
type DashboardWidgetConfiguration
|
||||
} from "@govoplan/core-webui";
|
||||
import {
|
||||
listFileSpaces,
|
||||
type FileSpace
|
||||
} from "../../api/files";
|
||||
|
||||
export default function FileSpacesWidget({
|
||||
settings,
|
||||
refreshKey,
|
||||
configuration
|
||||
}: {
|
||||
settings: ApiSettings;
|
||||
refreshKey: number;
|
||||
configuration: DashboardWidgetConfiguration;
|
||||
}) {
|
||||
const maxItems = numberSetting(configuration.maxItems, 6, 1, 16);
|
||||
const includeConnectors = configuration.includeConnectors !== false;
|
||||
const load = useCallback(async () => {
|
||||
const response = await listFileSpaces(settings);
|
||||
return response.spaces
|
||||
.filter(
|
||||
(space) =>
|
||||
includeConnectors || (space.space_type ?? "managed") === "managed"
|
||||
)
|
||||
.sort((left, right) => left.label.localeCompare(right.label))
|
||||
.slice(0, maxItems);
|
||||
}, [includeConnectors, maxItems, settings]);
|
||||
const { data: spaces, loading, error } = useDashboardWidgetData(
|
||||
load,
|
||||
refreshKey
|
||||
);
|
||||
|
||||
return (
|
||||
<LoadingFrame loading={loading} label="Loading file spaces">
|
||||
{error && (
|
||||
<DismissibleAlert tone="warning" resetKey={error}>
|
||||
{error}
|
||||
</DismissibleAlert>
|
||||
)}
|
||||
<DashboardWidgetList
|
||||
emptyText="No file spaces are available."
|
||||
items={(spaces ?? []).map((space) => ({
|
||||
id: space.id,
|
||||
title: space.label,
|
||||
detail: spaceDescription(space),
|
||||
meta: space.owner_type === "group" ? "Group" : "Personal",
|
||||
leading:
|
||||
space.space_type === "connector" ? (
|
||||
<Network size={17} aria-hidden="true" />
|
||||
) : (
|
||||
<Folder size={17} aria-hidden="true" />
|
||||
),
|
||||
trailing: space.read_only ? (
|
||||
<StatusBadge status="locked" label="Read only" />
|
||||
) : undefined,
|
||||
to: "/files"
|
||||
}))}
|
||||
/>
|
||||
<div className="dashboard-contribution-footer">
|
||||
<Link className="btn btn-secondary" to="/files">
|
||||
Open files
|
||||
</Link>
|
||||
</div>
|
||||
</LoadingFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function spaceDescription(space: FileSpace): string {
|
||||
if (space.space_type !== "connector") {
|
||||
return space.description || "Managed storage";
|
||||
}
|
||||
return [space.provider, space.library_id, space.remote_path]
|
||||
.filter(Boolean)
|
||||
.join(" · ") || "Connected storage";
|
||||
}
|
||||
|
||||
function numberSetting(
|
||||
value: unknown,
|
||||
fallback: number,
|
||||
minimum: number,
|
||||
maximum: number
|
||||
): number {
|
||||
const numeric = typeof value === "number" ? value : Number(value);
|
||||
return Number.isFinite(numeric)
|
||||
? Math.max(minimum, Math.min(maximum, Math.floor(numeric)))
|
||||
: fallback;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { useEffect, useMemo, useRef, useState, type DragEvent as ReactDragEvent, type KeyboardEvent as ReactKeyboardEvent, type MouseEvent as ReactMouseEvent } from "react";
|
||||
import { ArrowUp, ChevronRight, Copy, Download, File, Folder, Home, KeyRound, Link2, MoveRight, Plus, RefreshCw, Search, Trash2, UploadCloud } from "lucide-react";
|
||||
import { ArrowUp, ChevronRight, Copy, Download, File, Folder, Home, KeyRound, Link2, ListFilter, MoveRight, Plus, RefreshCw, Search, Share2, Trash2, UploadCloud } from "lucide-react";
|
||||
import {
|
||||
Button,
|
||||
ConfirmDialog,
|
||||
@@ -25,6 +25,7 @@ import {
|
||||
downloadFilesAsZip,
|
||||
fetchResourceAccessExplanation,
|
||||
listFilesDelta,
|
||||
listFilesByProperties,
|
||||
listFileConnectorProfiles,
|
||||
listFileSpaces,
|
||||
listManagedFileSnapshot,
|
||||
@@ -38,6 +39,7 @@ import {
|
||||
type FileConnectorBrowseItem,
|
||||
type FileConnectorProfile,
|
||||
type FileDeltaResponse,
|
||||
type FileCampaignUsageFilter,
|
||||
type FileFolder,
|
||||
type FileSpace,
|
||||
type ManagedFile,
|
||||
@@ -46,6 +48,7 @@ import {
|
||||
"../../api/files";
|
||||
import { EMPTY_FILES, EMPTY_FOLDERS, EMPTY_SPACES, INTERNAL_DRAG_TYPE } from "./constants";
|
||||
import { FileConflictDialog, FileContextMenu, FileDialog, FolderTree, RenamePreviewList, TransferFolderSelector } from "./components/FileManagerComponents";
|
||||
import FileShareDialog from "./components/FileShareDialog";
|
||||
import type {
|
||||
ContextMenuState,
|
||||
DialogKind,
|
||||
@@ -88,6 +91,7 @@ import { useFileDialogs } from "./hooks/useFileDialogs";
|
||||
import { useFileDragDropState } from "./hooks/useFileDragDropState";
|
||||
|
||||
type UploadPhase = "idle" | "uploading" | "unpacking" | "finalizing";
|
||||
type AuditRelevantFilter = "" | "true" | "false";
|
||||
type FileAccessExplanationTarget = {
|
||||
resourceType: "file" | "folder";
|
||||
resourceId: string;
|
||||
@@ -103,6 +107,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const canUpload = hasScope(auth, "files:upload");
|
||||
const canOrganize = hasScope(auth, "files:organize");
|
||||
const canDelete = hasScope(auth, "files:delete");
|
||||
const canShare = hasScope(auth, "files:file:share");
|
||||
const [spaces, setSpaces] = useState<FileSpace[]>(EMPTY_SPACES);
|
||||
const [activeSpaceId, setActiveSpaceId] = useState("");
|
||||
const activeSpace = spaces.find((space) => space.id === activeSpaceId) ?? spaces[0] ?? null;
|
||||
@@ -118,6 +123,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const [searchCaseSensitive, setSearchCaseSensitive] = useState(false);
|
||||
const [searchActive, setSearchActive] = useState(false);
|
||||
const [searchResults, setSearchResults] = useState<ManagedFile[] | null>(null);
|
||||
const [campaignUsageFilter, setCampaignUsageFilter] = useState<"" | FileCampaignUsageFilter>("");
|
||||
const [auditRelevantFilter, setAuditRelevantFilter] = useState<AuditRelevantFilter>("");
|
||||
const [propertyFiltersActive, setPropertyFiltersActive] = useState(false);
|
||||
const [propertyFilterResults, setPropertyFilterResults] = useState<ManagedFile[] | null>(null);
|
||||
const [propertyFilterTotal, setPropertyFilterTotal] = useState<number | null>(null);
|
||||
const [sortColumn, setSortColumn] = useState<SortColumn>("name");
|
||||
const [sortDirection, setSortDirection] = useState<SortDirection>("asc");
|
||||
const [unmatchedCount, setUnmatchedCount] = useState<number | null>(null);
|
||||
@@ -146,6 +156,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const [accessExplanationTarget, setAccessExplanationTarget] = useState<FileAccessExplanationTarget | null>(null);
|
||||
const [resourceAccessExplanation, setResourceAccessExplanation] = useState<ResourceAccessExplanationResponse | null>(null);
|
||||
const [resourceAccessLoading, setResourceAccessLoading] = useState(false);
|
||||
const [shareDialogFile, setShareDialogFile] = useState<ManagedFile | null>(null);
|
||||
const { setDragActive, internalDrag, setInternalDrag, dropTargetKey, setDropTargetKey, clearDropState: clearDragDropState } = useFileDragDropState();
|
||||
const {
|
||||
dialog,
|
||||
@@ -176,11 +187,19 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
const [renameSuffix, setRenameSuffix] = useState("");
|
||||
const [renameRecursive, setRenameRecursive] = useState(false);
|
||||
const [renamePreview, setRenamePreview] = useState<RenameResponse | null>(null);
|
||||
const visibleFiles = searchActive && searchResults ? searchResults : files;
|
||||
const visibleFiles = useMemo(() => {
|
||||
if (searchActive && searchResults && propertyFiltersActive && propertyFilterResults) {
|
||||
const ids = new Set(propertyFilterResults.map((file) => file.id));
|
||||
return searchResults.filter((file) => ids.has(file.id));
|
||||
}
|
||||
if (propertyFiltersActive && propertyFilterResults) return propertyFilterResults;
|
||||
if (searchActive && searchResults) return searchResults;
|
||||
return files;
|
||||
}, [files, propertyFilterResults, propertyFiltersActive, searchActive, searchResults]);
|
||||
const explorerEntries = useMemo(() => {
|
||||
const entries = buildExplorerEntries(visibleFiles, folders, currentFolder, searchActive);
|
||||
const entries = buildExplorerEntries(visibleFiles, folders, currentFolder, searchActive || propertyFiltersActive);
|
||||
return sortExplorerEntries(entries, sortColumn, sortDirection);
|
||||
}, [visibleFiles, folders, currentFolder, searchActive, sortColumn, sortDirection]);
|
||||
}, [visibleFiles, folders, currentFolder, searchActive, propertyFiltersActive, sortColumn, sortDirection]);
|
||||
const fileListViewportRef = useRef<HTMLDivElement | null>(null);
|
||||
const fileListMeasureRowRef = useRef<HTMLDivElement | null>(null);
|
||||
const managedSpaceLoadsInFlightRef = useRef<Set<string>>(new Set());
|
||||
@@ -229,6 +248,14 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
onOpenFolder: openFolder
|
||||
});
|
||||
const selectedFiles = useMemo(() => files.filter((file) => selectedFileIds.has(file.id)), [files, selectedFileIds]);
|
||||
const shareManageableFile = useMemo(() => {
|
||||
if (!canShare || selectedFiles.length !== 1 || selectedFolderPaths.size > 0) return null;
|
||||
const file = selectedFiles[0];
|
||||
const ownsFile = file.owner_type === "user"
|
||||
? file.owner_id === auth.user.id
|
||||
: auth.groups.some((group) => group.id === file.owner_id);
|
||||
return ownsFile || hasScope(auth, "files:file:admin") ? file : null;
|
||||
}, [auth, canShare, selectedFiles, selectedFolderPaths]);
|
||||
const selectedDownloadFileIds = useMemo(() => fileIdsForSelection(files, selectedFileIds, selectedFolderPaths), [files, selectedFileIds, selectedFolderPaths]);
|
||||
const accessExplainableTarget = useMemo(
|
||||
() => accessTargetForSelection(selectedFileIds, selectedFolderPaths, activeSpaceId),
|
||||
@@ -285,7 +312,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
if (!viewport) return;
|
||||
viewport.scrollTop = 0;
|
||||
setFileListScrollTop(0);
|
||||
}, [activeSpaceId, currentFolder, searchActive, searchResults, sortColumn, sortDirection]);
|
||||
}, [activeSpaceId, currentFolder, propertyFiltersActive, propertyFilterResults, searchActive, searchResults, sortColumn, sortDirection]);
|
||||
|
||||
async function loadSpaces() {
|
||||
setBusy(true);
|
||||
@@ -373,6 +400,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchActive(false);
|
||||
setSearchPattern("");
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
}
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
@@ -410,6 +442,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchPattern("");
|
||||
setSearchActive(false);
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
setConnectorSpaceSelectedItem(null);
|
||||
@@ -431,6 +468,11 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSearchActive(false);
|
||||
setSearchPattern("");
|
||||
setSearchResults(null);
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
}
|
||||
setUnmatchedCount(null);
|
||||
setRenamePreview(null);
|
||||
@@ -1001,6 +1043,50 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
setSelectedFolderPaths(new Set());
|
||||
}
|
||||
|
||||
async function runPropertyFilters(
|
||||
campaignUsage: "" | FileCampaignUsageFilter,
|
||||
auditRelevant: AuditRelevantFilter
|
||||
) {
|
||||
if (!activeSpace || activeSpaceIsConnector) return;
|
||||
if (!campaignUsage && !auditRelevant) {
|
||||
clearPropertyFilters();
|
||||
return;
|
||||
}
|
||||
setBusy(true);
|
||||
setError("");
|
||||
setMessage("");
|
||||
try {
|
||||
const response = await listFilesByProperties(settings, {
|
||||
owner_type: activeSpace.owner_type,
|
||||
owner_id: activeSpace.owner_id,
|
||||
path_prefix: currentFolder,
|
||||
campaign_usage: campaignUsage || undefined,
|
||||
audit_relevant: auditRelevant ? auditRelevant === "true" : undefined
|
||||
});
|
||||
setCampaignUsageFilter(campaignUsage);
|
||||
setAuditRelevantFilter(auditRelevant);
|
||||
setPropertyFilterResults(response.files);
|
||||
setPropertyFilterTotal(response.total);
|
||||
setPropertyFiltersActive(true);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
function clearPropertyFilters() {
|
||||
setCampaignUsageFilter("");
|
||||
setAuditRelevantFilter("");
|
||||
setPropertyFiltersActive(false);
|
||||
setPropertyFilterResults(null);
|
||||
setPropertyFilterTotal(null);
|
||||
setSelectedFileIds(new Set());
|
||||
setSelectedFolderPaths(new Set());
|
||||
}
|
||||
|
||||
function deleteSelected(
|
||||
fileIds: Set<string> = selectedFileIds,
|
||||
folderPaths: Set<string> = selectedFolderPaths,
|
||||
@@ -1821,6 +1907,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
<Link2 size={16} aria-hidden="true" /> i18n:govoplan-files.add_space.e4d674d4
|
||||
</Button>
|
||||
<Button onClick={() => void downloadSelection()} disabled={busy || activeSpaceIsConnector || !canDownload || selectedDownloadFileIds.length === 0}><Download size={16} aria-hidden="true" /> {downloadLabel}</Button>
|
||||
<Button onClick={() => shareManageableFile && setShareDialogFile(shareManageableFile)} disabled={busy || activeSpaceIsConnector || !shareManageableFile}><Share2 size={16} aria-hidden="true" /> Manage shares</Button>
|
||||
<Button onClick={() => openDialog("create-folder", toolbarTarget())} disabled={busy || !activeSpace || activeSpaceIsConnector || !canOrganize}><Plus size={16} aria-hidden="true" /> i18n:govoplan-files.create_folder.97bafaba</Button>
|
||||
<Button onClick={() => openTransferDialog("move")} disabled={busy || activeSpaceIsConnector || !canOrganize || !hasSelection}><MoveRight size={16} aria-hidden="true" /> i18n:govoplan-files.move.76cdb950</Button>
|
||||
<Button onClick={() => openTransferDialog("copy")} disabled={busy || activeSpaceIsConnector || !canOrganize || !hasSelection}><Copy size={16} aria-hidden="true" /> i18n:govoplan-files.copy.af74f7c5</Button>
|
||||
@@ -2067,6 +2154,15 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
disabled={!activeSpace || activeSpaceIsConnector}
|
||||
onSearch={(patternValue, caseSensitiveValue) => void runPatternSearch(patternValue, caseSensitiveValue)}
|
||||
onClear={() => void clearSearch()} />
|
||||
<FilePropertyFilterRow
|
||||
campaignUsage={campaignUsageFilter}
|
||||
auditRelevant={auditRelevantFilter}
|
||||
busy={busy}
|
||||
active={propertyFiltersActive}
|
||||
disabled={!activeSpace || activeSpaceIsConnector}
|
||||
total={propertyFilterTotal}
|
||||
onApply={(campaignUsage, auditRelevant) => void runPropertyFilters(campaignUsage, auditRelevant)}
|
||||
onClear={clearPropertyFilters} />
|
||||
|
||||
<div className="file-list-meta">
|
||||
<span>{activeSpaceIsConnector ? connectorSpaceSelectedItem?.kind === "file" ? connectorSpaceSelectedItem.name : "i18n:govoplan-files.remote_connector_space.d8956863" : selectedSummary}</span>
|
||||
@@ -2223,6 +2319,17 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
</FileDialog>
|
||||
}
|
||||
|
||||
{shareDialogFile &&
|
||||
<FileShareDialog
|
||||
settings={settings}
|
||||
file={shareDialogFile}
|
||||
tenantId={auth.active_tenant?.id || auth.tenant.id}
|
||||
onClose={() => setShareDialogFile(null)}
|
||||
onChanged={() => {
|
||||
if (activeSpace) void loadSpaceContents(activeSpace, { silent: true });
|
||||
}} />
|
||||
}
|
||||
|
||||
{dialog === "upload" &&
|
||||
<FileDialog title={i18nMessage("i18n:govoplan-files.upload_to_value_value.b83a34b4", { value0: activeDialogSpace?.label || "i18n:govoplan-files.files.6ce6c512", value1: activeDialogTarget?.folderPath || "i18n:govoplan-files.root.e96857c5" })} onClose={closeDialog}>
|
||||
<FileDropZone
|
||||
@@ -2509,6 +2616,72 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
||||
|
||||
}
|
||||
|
||||
function FilePropertyFilterRow({
|
||||
campaignUsage,
|
||||
auditRelevant,
|
||||
busy,
|
||||
active,
|
||||
disabled,
|
||||
total,
|
||||
onApply,
|
||||
onClear
|
||||
}: {
|
||||
campaignUsage: "" | FileCampaignUsageFilter;
|
||||
auditRelevant: AuditRelevantFilter;
|
||||
busy: boolean;
|
||||
active: boolean;
|
||||
disabled: boolean;
|
||||
total: number | null;
|
||||
onApply: (campaignUsage: "" | FileCampaignUsageFilter, auditRelevant: AuditRelevantFilter) => void;
|
||||
onClear: () => void;
|
||||
}) {
|
||||
const [campaignUsageDraft, setCampaignUsageDraft] = useState(campaignUsage);
|
||||
const [auditRelevantDraft, setAuditRelevantDraft] = useState(auditRelevant);
|
||||
|
||||
useEffect(() => {
|
||||
setCampaignUsageDraft(campaignUsage);
|
||||
}, [campaignUsage]);
|
||||
|
||||
useEffect(() => {
|
||||
setAuditRelevantDraft(auditRelevant);
|
||||
}, [auditRelevant]);
|
||||
|
||||
return (
|
||||
<div className="file-property-filter-row">
|
||||
<ListFilter size={17} aria-hidden="true" />
|
||||
<label className="file-property-filter-field">
|
||||
<span>Campaign use</span>
|
||||
<select
|
||||
value={campaignUsageDraft}
|
||||
disabled={busy || disabled}
|
||||
onChange={(event) => setCampaignUsageDraft(event.target.value as "" | FileCampaignUsageFilter)}>
|
||||
<option value="">Any</option>
|
||||
<option value="linked">Linked or used</option>
|
||||
<option value="unlinked">Not linked or used</option>
|
||||
</select>
|
||||
</label>
|
||||
<label className="file-property-filter-field">
|
||||
<span>Audit evidence</span>
|
||||
<select
|
||||
value={auditRelevantDraft}
|
||||
disabled={busy || disabled}
|
||||
onChange={(event) => setAuditRelevantDraft(event.target.value as AuditRelevantFilter)}>
|
||||
<option value="">Any</option>
|
||||
<option value="true">Audit relevant</option>
|
||||
<option value="false">Not audit relevant</option>
|
||||
</select>
|
||||
</label>
|
||||
<Button
|
||||
onClick={() => onApply(campaignUsageDraft, auditRelevantDraft)}
|
||||
disabled={busy || disabled || !campaignUsageDraft && !auditRelevantDraft}>
|
||||
Apply filters
|
||||
</Button>
|
||||
{active && <Button onClick={onClear} disabled={busy}>Clear filters</Button>}
|
||||
{active && total !== null && <span className="file-property-filter-count">{total} matching file{total === 1 ? "" : "s"}</span>}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function FileSearchRow({
|
||||
value,
|
||||
caseSensitive,
|
||||
@@ -2559,7 +2732,15 @@ function FileSearchRow({
|
||||
}
|
||||
|
||||
function activeCampaignShares(file: ManagedFile) {
|
||||
return (file.shares ?? []).filter((share) => share.target_type === "campaign" && !share.revoked_at);
|
||||
return (file.shares ?? []).filter((share) =>
|
||||
share.target_type === "campaign" && fileShareIsActive(share)
|
||||
);
|
||||
}
|
||||
|
||||
function fileShareIsActive(share: NonNullable<ManagedFile["shares"]>[number]): boolean {
|
||||
if (typeof share.active === "boolean") return share.active;
|
||||
return !share.revoked_at
|
||||
&& (!share.expires_at || new Date(share.expires_at).getTime() > Date.now());
|
||||
}
|
||||
|
||||
function activeCampaignShareCount(file: ManagedFile): number {
|
||||
|
||||
300
webui/src/features/files/components/FileShareDialog.tsx
Normal file
300
webui/src/features/files/components/FileShareDialog.tsx
Normal file
@@ -0,0 +1,300 @@
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { Trash2 } from "lucide-react";
|
||||
import {
|
||||
Button,
|
||||
ConfirmDialog,
|
||||
DataGrid,
|
||||
DismissibleAlert,
|
||||
FormField,
|
||||
ReferenceSelect,
|
||||
StatusBadge,
|
||||
TableActionGroup,
|
||||
type ApiSettings,
|
||||
type DataGridColumn
|
||||
} from "@govoplan/core-webui";
|
||||
import {
|
||||
createFileShare,
|
||||
fileShareTargetProvider,
|
||||
listFileShares,
|
||||
revokeFileShare,
|
||||
type FileShare,
|
||||
type ManagedFile
|
||||
} from "../../../api/files";
|
||||
import { FileDialog } from "./FileManagerComponents";
|
||||
import { formatDate } from "../utils/fileManagerUtils";
|
||||
|
||||
type ShareTargetType = "user" | "group" | "tenant";
|
||||
type SharePermission = "read" | "write" | "manage";
|
||||
|
||||
export default function FileShareDialog({
|
||||
settings,
|
||||
file,
|
||||
tenantId,
|
||||
onClose,
|
||||
onChanged
|
||||
}: {
|
||||
settings: ApiSettings;
|
||||
file: ManagedFile;
|
||||
tenantId: string;
|
||||
onClose: () => void;
|
||||
onChanged: () => void;
|
||||
}) {
|
||||
const [shares, setShares] = useState<FileShare[]>([]);
|
||||
const [targetType, setTargetType] = useState<ShareTargetType>("user");
|
||||
const [targetId, setTargetId] = useState("");
|
||||
const [permission, setPermission] = useState<SharePermission>("read");
|
||||
const [expiresAt, setExpiresAt] = useState("");
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [error, setError] = useState("");
|
||||
const [revokeTarget, setRevokeTarget] = useState<FileShare | null>(null);
|
||||
const targetProvider = useMemo(
|
||||
() => targetType === "tenant"
|
||||
? null
|
||||
: fileShareTargetProvider(settings, file.id, targetType),
|
||||
[file.id, settings, targetType]
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
void refresh();
|
||||
}, [file.id]);
|
||||
|
||||
async function refresh() {
|
||||
setBusy(true);
|
||||
setError("");
|
||||
try {
|
||||
const response = await listFileShares(settings, file.id, {
|
||||
includeInactive: true
|
||||
});
|
||||
setShares(response.shares);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function grant() {
|
||||
const effectiveTargetId = targetType === "tenant" ? tenantId : targetId;
|
||||
if (!effectiveTargetId) return;
|
||||
setBusy(true);
|
||||
setError("");
|
||||
try {
|
||||
await createFileShare(settings, file.id, {
|
||||
target_type: targetType,
|
||||
target_id: effectiveTargetId,
|
||||
permission,
|
||||
expires_at: expiresAt ? new Date(expiresAt).toISOString() : null
|
||||
});
|
||||
setTargetId("");
|
||||
setExpiresAt("");
|
||||
await refresh();
|
||||
onChanged();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function revoke() {
|
||||
if (!revokeTarget) return;
|
||||
setBusy(true);
|
||||
setError("");
|
||||
try {
|
||||
await revokeFileShare(settings, file.id, revokeTarget.id);
|
||||
setRevokeTarget(null);
|
||||
await refresh();
|
||||
onChanged();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : String(err));
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
const columns = useMemo<DataGridColumn<FileShare>[]>(() => [
|
||||
{
|
||||
id: "target",
|
||||
header: "Target",
|
||||
minWidth: 220,
|
||||
resizable: true,
|
||||
sortable: true,
|
||||
filterable: true,
|
||||
value: (share) => `${share.target_type}: ${share.target_id}`
|
||||
},
|
||||
{
|
||||
id: "permission",
|
||||
header: "Permission",
|
||||
width: 115,
|
||||
sortable: true,
|
||||
filterable: true,
|
||||
value: (share) => share.permission
|
||||
},
|
||||
{
|
||||
id: "created",
|
||||
header: "Created",
|
||||
minWidth: 160,
|
||||
resizable: true,
|
||||
sortable: true,
|
||||
value: (share) => formatDate(share.created_at)
|
||||
},
|
||||
{
|
||||
id: "creator",
|
||||
header: "Created by",
|
||||
minWidth: 150,
|
||||
resizable: true,
|
||||
filterable: true,
|
||||
value: (share) => share.created_by_user_id || "System"
|
||||
},
|
||||
{
|
||||
id: "expiry",
|
||||
header: "Expires",
|
||||
minWidth: 160,
|
||||
resizable: true,
|
||||
sortable: true,
|
||||
value: (share) => share.expires_at ? formatDate(share.expires_at) : "No expiry"
|
||||
},
|
||||
{
|
||||
id: "status",
|
||||
header: "Status",
|
||||
width: 115,
|
||||
sortable: true,
|
||||
value: (share) => shareStatus(share),
|
||||
render: (share) => (
|
||||
<StatusBadge
|
||||
status={share.active ? "active" : "inactive"}
|
||||
label={shareStatus(share)}
|
||||
/>
|
||||
)
|
||||
},
|
||||
{
|
||||
id: "actions",
|
||||
header: "Actions",
|
||||
width: 72,
|
||||
sticky: "end",
|
||||
align: "right",
|
||||
render: (share) => (
|
||||
<TableActionGroup
|
||||
minimumSlots={1}
|
||||
actions={[
|
||||
{
|
||||
id: "revoke",
|
||||
label: "Revoke share",
|
||||
icon: <Trash2 size={16} aria-hidden="true" />,
|
||||
variant: "danger",
|
||||
disabled: !share.active || busy,
|
||||
disabledReason: !share.active ? "This share is already inactive." : undefined,
|
||||
onClick: () => setRevokeTarget(share)
|
||||
}
|
||||
]}
|
||||
/>
|
||||
)
|
||||
}
|
||||
], [busy]);
|
||||
|
||||
return (
|
||||
<>
|
||||
<FileDialog title={`Manage shares - ${file.filename}`} onClose={() => !busy && onClose()}>
|
||||
<div className="file-share-dialog-content">
|
||||
{error ? (
|
||||
<DismissibleAlert tone="danger" resetKey={error}>
|
||||
{error}
|
||||
</DismissibleAlert>
|
||||
) : null}
|
||||
<div className="file-share-form">
|
||||
<FormField label="Target type">
|
||||
<select
|
||||
value={targetType}
|
||||
disabled={busy}
|
||||
onChange={(event) => {
|
||||
setTargetType(event.target.value as ShareTargetType);
|
||||
setTargetId("");
|
||||
}}
|
||||
>
|
||||
<option value="user">User</option>
|
||||
<option value="group">Group</option>
|
||||
<option value="tenant">Entire tenant</option>
|
||||
</select>
|
||||
</FormField>
|
||||
<FormField label="Target">
|
||||
{targetType === "tenant" ? (
|
||||
<input value={tenantId} disabled readOnly />
|
||||
) : targetProvider ? (
|
||||
<ReferenceSelect
|
||||
value={targetId}
|
||||
onChange={setTargetId}
|
||||
provider={targetProvider}
|
||||
disabled={busy}
|
||||
placeholder={`Select a ${targetType}`}
|
||||
aria-label={`Share target ${targetType}`}
|
||||
required
|
||||
/>
|
||||
) : null}
|
||||
</FormField>
|
||||
<FormField label="Permission">
|
||||
<select
|
||||
value={permission}
|
||||
disabled={busy}
|
||||
onChange={(event) => setPermission(event.target.value as SharePermission)}
|
||||
>
|
||||
<option value="read">Read</option>
|
||||
<option value="write">Write</option>
|
||||
<option value="manage">Manage</option>
|
||||
</select>
|
||||
</FormField>
|
||||
<FormField label="Expires (optional)">
|
||||
<input
|
||||
type="datetime-local"
|
||||
value={expiresAt}
|
||||
min={minimumLocalDateTime()}
|
||||
disabled={busy}
|
||||
onChange={(event) => setExpiresAt(event.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<Button
|
||||
variant="primary"
|
||||
disabled={busy || (targetType !== "tenant" && !targetId)}
|
||||
onClick={() => void grant()}
|
||||
>
|
||||
Grant access
|
||||
</Button>
|
||||
</div>
|
||||
<DataGrid
|
||||
id={`file-${file.id}-shares`}
|
||||
rows={shares}
|
||||
columns={columns}
|
||||
getRowKey={(share) => share.id}
|
||||
emptyText={busy ? "Loading shares..." : "This file has no direct shares."}
|
||||
initialFit="container"
|
||||
/>
|
||||
<div className="button-row compact-actions align-end">
|
||||
<Button onClick={onClose} disabled={busy}>Close</Button>
|
||||
</div>
|
||||
</div>
|
||||
</FileDialog>
|
||||
<ConfirmDialog
|
||||
open={Boolean(revokeTarget)}
|
||||
title="Revoke file share"
|
||||
message="Revoke this direct grant? Other independent grants remain in effect."
|
||||
confirmLabel="Revoke"
|
||||
cancelLabel="Cancel"
|
||||
tone="danger"
|
||||
busy={busy}
|
||||
onConfirm={() => void revoke()}
|
||||
onCancel={() => setRevokeTarget(null)}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function shareStatus(share: FileShare): string {
|
||||
if (share.revoked_at) return "Revoked";
|
||||
if (share.expires_at && new Date(share.expires_at).getTime() <= Date.now()) {
|
||||
return "Expired";
|
||||
}
|
||||
return "Active";
|
||||
}
|
||||
|
||||
function minimumLocalDateTime(): string {
|
||||
const now = new Date(Date.now() + 60_000);
|
||||
const local = new Date(now.getTime() - now.getTimezoneOffset() * 60_000);
|
||||
return local.toISOString().slice(0, 16);
|
||||
}
|
||||
@@ -1024,7 +1024,14 @@ renderPatternPreview?: FilesManagedFileChooserProps["renderPatternPreview"])
|
||||
|
||||
function isLinkedToTarget(file: ManagedFile, target?: FilesManagedFileLinkTarget): boolean {
|
||||
if (!target) return false;
|
||||
return Boolean(file.shares?.some((share) => share.target_type === target.type && share.target_id === target.id && !share.revoked_at));
|
||||
return Boolean(file.shares?.some((share) =>
|
||||
share.target_type === target.type
|
||||
&& share.target_id === target.id
|
||||
&& (typeof share.active === "boolean"
|
||||
? share.active
|
||||
: !share.revoked_at
|
||||
&& (!share.expires_at || new Date(share.expires_at).getTime() > Date.now()))
|
||||
));
|
||||
}
|
||||
|
||||
function readRememberedState(key: string): RememberedChooserState {
|
||||
|
||||
@@ -1,8 +1,16 @@
|
||||
import { createElement, lazy } from "react";
|
||||
import { hasScope, type AdminSectionsUiCapability, type FilesConnectorsUiCapability, type FilesFileExplorerUiCapability, type PlatformWebModule } from "@govoplan/core-webui";
|
||||
import {
|
||||
hasScope,
|
||||
type AdminSectionsUiCapability,
|
||||
type DashboardWidgetsUiCapability,
|
||||
type FilesConnectorsUiCapability,
|
||||
type FilesFileExplorerUiCapability,
|
||||
type PlatformWebModule
|
||||
} from "@govoplan/core-webui";
|
||||
import { FolderTree } from "./features/files/components/FileManagerComponents";
|
||||
import FileConnectorSettingsPanel from "./features/files/FileConnectorSettingsPanel";
|
||||
import ManagedFileChooser from "./features/files/components/ManagedFileChooser";
|
||||
import FileSpacesWidget from "./features/files/FileSpacesWidget";
|
||||
import { listFileSpaces, listFiles, listFilesDelta, listFolders, resolveFilePatterns, shareFilesWithTarget } from "./api/files";
|
||||
import { generatedTranslations } from "./i18n/generatedTranslations";
|
||||
import "./styles/file-manager.css";
|
||||
@@ -14,10 +22,55 @@ const translations = {
|
||||
en: generatedTranslations.en,
|
||||
de: generatedTranslations.de
|
||||
};
|
||||
const fileDashboardWidgets: DashboardWidgetsUiCapability = {
|
||||
widgets: [
|
||||
{
|
||||
id: "files.spaces",
|
||||
surfaceId: "files.widget.spaces",
|
||||
title: "File spaces",
|
||||
description: "Managed and connected file spaces available to you.",
|
||||
moduleId: "files",
|
||||
category: "Content",
|
||||
order: 35,
|
||||
defaultVisible: false,
|
||||
defaultSize: "medium",
|
||||
supportedSizes: ["medium", "wide"],
|
||||
anyOf: fileRead,
|
||||
refreshIntervalMs: 60_000,
|
||||
defaultConfiguration: {
|
||||
maxItems: 6,
|
||||
includeConnectors: true
|
||||
},
|
||||
configurationFields: [
|
||||
{
|
||||
id: "maxItems",
|
||||
label: "Maximum spaces",
|
||||
kind: "number",
|
||||
min: 1,
|
||||
max: 16,
|
||||
step: 1,
|
||||
required: true
|
||||
},
|
||||
{
|
||||
id: "includeConnectors",
|
||||
label: "Include connected storage",
|
||||
kind: "boolean"
|
||||
}
|
||||
],
|
||||
render: ({ settings, refreshKey, configuration }) =>
|
||||
createElement(FileSpacesWidget, {
|
||||
settings,
|
||||
refreshKey,
|
||||
configuration
|
||||
})
|
||||
}
|
||||
]
|
||||
};
|
||||
const fileConnectorAdminSections: AdminSectionsUiCapability = {
|
||||
sections: [
|
||||
{
|
||||
id: "system-file-connectors",
|
||||
surfaceId: "files.admin.system-connectors",
|
||||
label: "i18n:govoplan-files.file_connections.1e362326",
|
||||
group: "SYSTEM",
|
||||
order: 75,
|
||||
@@ -30,6 +83,7 @@ const fileConnectorAdminSections: AdminSectionsUiCapability = {
|
||||
},
|
||||
{
|
||||
id: "tenant-file-connectors",
|
||||
surfaceId: "files.admin.tenant-connectors",
|
||||
label: "i18n:govoplan-files.file_connections.1e362326",
|
||||
group: "TENANT",
|
||||
order: 65,
|
||||
@@ -49,6 +103,14 @@ export const filesModule: PlatformWebModule = {
|
||||
version: "1.0.0",
|
||||
dependencies: ["access"],
|
||||
translations,
|
||||
viewSurfaces: [
|
||||
{ id: "files.admin.system-connectors", moduleId: "files", kind: "section", label: "System file connections", order: 75 },
|
||||
{ id: "files.admin.tenant-connectors", moduleId: "files", kind: "section", label: "Tenant file connections", order: 65 },
|
||||
{ id: "files.admin.group-connectors", moduleId: "files", kind: "section", label: "Group file connections", order: 65 },
|
||||
{ id: "files.admin.user-connectors", moduleId: "files", kind: "section", label: "User file connections", order: 65 },
|
||||
{ id: "files.settings.connectors", moduleId: "files", kind: "section", label: "Personal file connections", order: 20 },
|
||||
{ id: "files.widget.spaces", moduleId: "files", kind: "section", label: "File spaces widget", order: 35 }
|
||||
],
|
||||
navItems: [{ to: "/files", label: "i18n:govoplan-files.files.6ce6c512", iconName: "folder", anyOf: fileRead, order: 40 }],
|
||||
routes: [
|
||||
{ path: "/files", anyOf: fileRead, order: 40, render: ({ settings, auth }) => createElement(FilesPage, { settings, auth }) }],
|
||||
@@ -67,7 +129,8 @@ export const filesModule: PlatformWebModule = {
|
||||
"files.connectors": {
|
||||
FileConnectorScopeManager: FileConnectorSettingsPanel
|
||||
} satisfies FilesConnectorsUiCapability,
|
||||
"admin.sections": fileConnectorAdminSections
|
||||
"admin.sections": fileConnectorAdminSections,
|
||||
"dashboard.widgets": fileDashboardWidgets
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -11,6 +11,31 @@
|
||||
padding: 4px 0 10px 14px;
|
||||
}
|
||||
|
||||
.file-property-filter-row {
|
||||
display: flex;
|
||||
align-items: end;
|
||||
gap: 12px;
|
||||
min-width: 0;
|
||||
padding: 0 0 10px 14px;
|
||||
}
|
||||
|
||||
.file-property-filter-field {
|
||||
display: grid;
|
||||
gap: 4px;
|
||||
min-width: 180px;
|
||||
}
|
||||
|
||||
.file-property-filter-field > span,
|
||||
.file-property-filter-count {
|
||||
color: var(--muted);
|
||||
font-size: var(--font-size-sm);
|
||||
}
|
||||
|
||||
.file-property-filter-count {
|
||||
align-self: center;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.file-list-drop-target.is-active,
|
||||
.file-list-drop-target.is-drop-target {
|
||||
background: var(--line);
|
||||
@@ -263,6 +288,35 @@
|
||||
width: min(820px, 100%);
|
||||
}
|
||||
|
||||
.file-dialog:has(.file-share-dialog-content) {
|
||||
width: min(1080px, 100%);
|
||||
}
|
||||
|
||||
.file-share-dialog-content {
|
||||
display: grid;
|
||||
min-width: 0;
|
||||
gap: 16px;
|
||||
}
|
||||
|
||||
.file-share-form {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(130px, .7fr) minmax(220px, 1.4fr) minmax(120px, .7fr) minmax(190px, 1fr) auto;
|
||||
gap: 12px;
|
||||
align-items: end;
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
.file-share-form {
|
||||
grid-template-columns: minmax(0, 1fr) minmax(0, 1fr);
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 620px) {
|
||||
.file-share-form {
|
||||
grid-template-columns: minmax(0, 1fr);
|
||||
}
|
||||
}
|
||||
|
||||
.connector-sync-grid {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, .9fr) minmax(0, 1.1fr);
|
||||
@@ -565,6 +619,15 @@
|
||||
.file-search-row {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.file-property-filter-row {
|
||||
align-items: stretch;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.file-property-filter-count {
|
||||
margin-left: 0;
|
||||
}
|
||||
}
|
||||
|
||||
.file-conflict-summary {
|
||||
|
||||
Reference in New Issue
Block a user