Compare commits
8 Commits
3bc1d3489e
...
15ade8df75
| Author | SHA1 | Date | |
|---|---|---|---|
| 15ade8df75 | |||
| f3c485ef61 | |||
| 0e36b20a14 | |||
| 06e6e7191b | |||
| 3449cbc8a5 | |||
| 92950af6f4 | |||
| 8826cf2890 | |||
| f2dfb6c90e |
10
README.md
10
README.md
@@ -78,7 +78,11 @@ Profiles can be supplied as JSON through
|
|||||||
`GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON`, or from a JSON file path through
|
`GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON`, or from a JSON file path through
|
||||||
`GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE`. Each profile has an `id`, `provider`,
|
`GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE`. Each profile has an `id`, `provider`,
|
||||||
`endpoint_url`, governance `scope_type`/`scope_id`, optional `capabilities`, and
|
`endpoint_url`, governance `scope_type`/`scope_id`, optional `capabilities`, and
|
||||||
credential references such as `password_env`, `token_env`, or `secret_ref`.
|
deployment-owned credential references such as `password_env`, `token_env`, or
|
||||||
|
`secret_ref`. Environment references require an exact name in the deployment-wide
|
||||||
|
`GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST`; API-managed profiles cannot select
|
||||||
|
process environment variables and use encrypted stored credentials or scoped
|
||||||
|
secret-provider references instead.
|
||||||
`GET /api/v1/files/connectors/profiles` returns only profiles visible to the
|
`GET /api/v1/files/connectors/profiles` returns only profiles visible to the
|
||||||
current principal (system, tenant, user, group, or accessible campaign scope) and
|
current principal (system, tenant, user, group, or accessible campaign scope) and
|
||||||
redacts secret values and environment variable names. Use the returned
|
redacts secret values and environment variable names. Use the returned
|
||||||
@@ -102,8 +106,8 @@ conflict handling, source provenance, and connector audit path as direct
|
|||||||
uploads. The Seafile provider uses account-token auth and the native file
|
uploads. The Seafile provider uses account-token auth and the native file
|
||||||
download-link API; Nextcloud and generic WebDAV profiles use authenticated `GET`
|
download-link API; Nextcloud and generic WebDAV profiles use authenticated `GET`
|
||||||
requests against the configured WebDAV endpoint. SMB profiles use
|
requests against the configured WebDAV endpoint. SMB profiles use
|
||||||
`smb://server[:port]/share[/path]` endpoints and environment-backed credentials
|
`smb://server[:port]/share[/path]` endpoints and deployment-owned or encrypted
|
||||||
through `smbprotocol`.
|
stored credentials through `smbprotocol`.
|
||||||
|
|
||||||
Local connector development assets live in `dev/connectors/`. The compose stack
|
Local connector development assets live in `dev/connectors/`. The compose stack
|
||||||
boots Nextcloud, Seafile, WebDAV, and SMB endpoints for provider development and
|
boots Nextcloud, Seafile, WebDAV, and SMB endpoints for provider development and
|
||||||
|
|||||||
@@ -38,8 +38,9 @@ The local fixture data under `data/` is ignored by git.
|
|||||||
## GovOPlaN Profile Config
|
## GovOPlaN Profile Config
|
||||||
|
|
||||||
Connector profiles are read from `GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON` or
|
Connector profiles are read from `GOVOPLAN_FILES_CONNECTOR_PROFILES_JSON` or
|
||||||
`GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE`. Credentials should be referenced by
|
`GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE`. This deployment-owned configuration
|
||||||
secret refs or environment variables; profile API responses only expose the
|
may reference environment variables only when their exact names are listed in
|
||||||
|
`GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST`; profile API responses expose only the
|
||||||
credential source and configured state.
|
credential source and configured state.
|
||||||
|
|
||||||
Example local profile file:
|
Example local profile file:
|
||||||
@@ -123,6 +124,8 @@ Start GovOPlaN with:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
export GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE=/mnt/DATA/git/govoplan-files/dev/connectors/profiles.local.json
|
export GOVOPLAN_FILES_CONNECTOR_PROFILES_FILE=/mnt/DATA/git/govoplan-files/dev/connectors/profiles.local.json
|
||||||
|
export GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST=SEAFILE_ADMIN_PASSWORD,NEXTCLOUD_ADMIN_PASSWORD,WEBDAV_PASSWORD,SMB_PASSWORD,MINIO_ROOT_PASSWORD
|
||||||
|
export GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=true
|
||||||
```
|
```
|
||||||
|
|
||||||
## Smoke Test
|
## Smoke Test
|
||||||
|
|||||||
@@ -49,8 +49,9 @@ any remote write behavior.
|
|||||||
Every provider must:
|
Every provider must:
|
||||||
|
|
||||||
- enforce GovOPlaN profile visibility and connector policy before browse/import
|
- enforce GovOPlaN profile visibility and connector policy before browse/import
|
||||||
- keep credentials as environment variables or secret references, never API
|
- keep credentials as encrypted values or scoped secret references, never API
|
||||||
response values
|
response values; process-environment references are allowed only in
|
||||||
|
deployment-owned profiles with an exact deployment allowlist
|
||||||
- import external files into managed storage before they are used in campaigns
|
- import external files into managed storage before they are used in campaigns
|
||||||
or workflows
|
or workflows
|
||||||
- preserve source provenance and revision metadata
|
- preserve source provenance and revision metadata
|
||||||
|
|||||||
@@ -34,8 +34,9 @@ Credential profile:
|
|||||||
|
|
||||||
- provider: a specific provider or any provider
|
- provider: a specific provider or any provider
|
||||||
- scope: `system` or `tenant` for administered credentials
|
- scope: `system` or `tenant` for administered credentials
|
||||||
- credential mode: anonymous, environment reference, secret reference, or
|
- credential mode: anonymous, scoped secret reference, or encrypted stored
|
||||||
encrypted stored password/token
|
password/token; environment references are reserved for deployment-owned JSON
|
||||||
|
profiles and exact deployment allowlisting
|
||||||
- username and redacted secret configuration
|
- username and redacted secret configuration
|
||||||
- credential-local policy for allow/deny rules
|
- credential-local policy for allow/deny rules
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@
|
|||||||
"LICENSE"
|
"LICENSE"
|
||||||
],
|
],
|
||||||
"peerDependencies": {
|
"peerDependencies": {
|
||||||
"@govoplan/core-webui": "^0.1.8",
|
"@govoplan/core-webui": "^0.1.9",
|
||||||
"lucide-react": "^1.23.0",
|
"lucide-react": "^1.23.0",
|
||||||
"react": "^19.0.0",
|
"react": "^19.0.0",
|
||||||
"react-dom": "^19.0.0",
|
"react-dom": "^19.0.0",
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ requires-python = ">=3.12"
|
|||||||
license = { file = "LICENSE" }
|
license = { file = "LICENSE" }
|
||||||
authors = [{ name = "GovOPlaN" }]
|
authors = [{ name = "GovOPlaN" }]
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"govoplan-core>=0.1.8",
|
"govoplan-core>=0.1.9",
|
||||||
"defusedxml>=0.7,<1",
|
"defusedxml>=0.7,<1",
|
||||||
"python-multipart>=0.0.31,<1",
|
"python-multipart>=0.0.31,<1",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ from dataclasses import replace
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from io import BytesIO
|
from io import BytesIO
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
from urllib.parse import quote, urljoin
|
from urllib.parse import quote, urljoin, urlsplit, urlunsplit
|
||||||
from fastapi import APIRouter, Depends, File as FastAPIFile, Form, HTTPException, Query, UploadFile, status
|
from fastapi import APIRouter, Depends, File as FastAPIFile, Form, HTTPException, Query, UploadFile, status
|
||||||
from fastapi.responses import FileResponse, StreamingResponse
|
from fastapi.responses import FileResponse, StreamingResponse
|
||||||
from starlette.background import BackgroundTask
|
from starlette.background import BackgroundTask
|
||||||
@@ -120,6 +120,10 @@ from govoplan_files.backend.storage.connector_browse import (
|
|||||||
normalize_connector_browse_path,
|
normalize_connector_browse_path,
|
||||||
)
|
)
|
||||||
from govoplan_files.backend.storage.connector_imports import ConnectorImportError, ConnectorImportUnsupported, read_connector_file
|
from govoplan_files.backend.storage.connector_imports import ConnectorImportError, ConnectorImportUnsupported, read_connector_file
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import (
|
||||||
|
connector_effective_endpoint_url,
|
||||||
|
reject_api_controlled_deployment_references,
|
||||||
|
)
|
||||||
from govoplan_files.backend.storage.connector_profile_store import (
|
from govoplan_files.backend.storage.connector_profile_store import (
|
||||||
connector_profile_from_row,
|
connector_profile_from_row,
|
||||||
create_connector_profile_row,
|
create_connector_profile_row,
|
||||||
@@ -571,11 +575,48 @@ def _discovery_profile_from_payload(
|
|||||||
password_value=credentials.password,
|
password_value=credentials.password,
|
||||||
token_value=credentials.token,
|
token_value=credentials.token,
|
||||||
capabilities=("browse",),
|
capabilities=("browse",),
|
||||||
metadata=payload.metadata,
|
# Discovery metadata is untrusted API input and must not be able to
|
||||||
|
# replace the candidate endpoint or inject a static/fake listing.
|
||||||
|
metadata={},
|
||||||
source_kind="discovery",
|
source_kind="discovery",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_connector_discovery_attempt(
|
||||||
|
session: Session,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
*,
|
||||||
|
provider: str,
|
||||||
|
endpoint_url: str,
|
||||||
|
base_path: str | None,
|
||||||
|
) -> None:
|
||||||
|
audit_from_principal(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
action="files.connector.discovery_attempted",
|
||||||
|
object_type="connector_endpoint",
|
||||||
|
object_id=provider,
|
||||||
|
details={
|
||||||
|
"provider": provider,
|
||||||
|
"endpoint_origin": _redacted_connector_url(endpoint_url),
|
||||||
|
"base_path": base_path,
|
||||||
|
},
|
||||||
|
commit=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _redacted_connector_url(value: str) -> str:
|
||||||
|
try:
|
||||||
|
parsed = urlsplit(value)
|
||||||
|
hostname = parsed.hostname or ""
|
||||||
|
if ":" in hostname:
|
||||||
|
hostname = f"[{hostname}]"
|
||||||
|
netloc = f"{hostname}:{parsed.port}" if parsed.port is not None else hostname
|
||||||
|
return urlunsplit((parsed.scheme, netloc, "", "", ""))
|
||||||
|
except ValueError:
|
||||||
|
return "<invalid connector URL>"
|
||||||
|
|
||||||
|
|
||||||
def _visible_connector_profile(
|
def _visible_connector_profile(
|
||||||
session: Session,
|
session: Session,
|
||||||
principal: ApiPrincipal,
|
principal: ApiPrincipal,
|
||||||
@@ -768,7 +809,11 @@ def _download_connector_payload(
|
|||||||
provider=profile.provider,
|
provider=profile.provider,
|
||||||
external_id=f"{payload.library_id}:{source_path}",
|
external_id=f"{payload.library_id}:{source_path}",
|
||||||
external_path=source_path,
|
external_path=source_path,
|
||||||
external_url=profile.endpoint_url,
|
external_url=connector_effective_endpoint_url(
|
||||||
|
provider=profile.provider,
|
||||||
|
endpoint_url=profile.endpoint_url,
|
||||||
|
metadata=profile.metadata,
|
||||||
|
),
|
||||||
operation=operation,
|
operation=operation,
|
||||||
),
|
),
|
||||||
profile.policy_sources,
|
profile.policy_sources,
|
||||||
@@ -946,38 +991,8 @@ def _asset_response(session: Session, asset: FileAsset, *, include_shares: bool
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _asset_list_response(session: Session, assets: list[FileAsset], *, include_shares: bool = False) -> list[FileAssetResponse]:
|
def _file_share_response(row: FileShare) -> FileShareResponse:
|
||||||
if not assets:
|
return FileShareResponse(
|
||||||
return []
|
|
||||||
|
|
||||||
asset_ids = [asset.id for asset in assets]
|
|
||||||
version_ids = [asset.current_version_id for asset in assets if asset.current_version_id]
|
|
||||||
if len(version_ids) != len(assets):
|
|
||||||
raise FileStorageError("File has no current version")
|
|
||||||
|
|
||||||
version_blob_rows: list[tuple[FileVersion, FileBlob]] = []
|
|
||||||
for chunk in _chunks(version_ids):
|
|
||||||
version_blob_rows.extend(
|
|
||||||
session.query(FileVersion, FileBlob)
|
|
||||||
.join(FileBlob, FileBlob.id == FileVersion.blob_id)
|
|
||||||
.filter(FileVersion.id.in_(chunk))
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
versions_by_id = {version.id: version for version, _blob in version_blob_rows}
|
|
||||||
blobs_by_version_id = {version.id: blob for version, blob in version_blob_rows}
|
|
||||||
|
|
||||||
shares_by_asset_id: dict[str, list[FileShareResponse]] = {asset_id: [] for asset_id in asset_ids}
|
|
||||||
if include_shares:
|
|
||||||
for chunk in _chunks(asset_ids):
|
|
||||||
share_rows = (
|
|
||||||
session.query(FileShare)
|
|
||||||
.filter(FileShare.file_asset_id.in_(chunk))
|
|
||||||
.order_by(FileShare.created_at.desc())
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
for row in share_rows:
|
|
||||||
shares_by_asset_id.setdefault(row.file_asset_id, []).append(
|
|
||||||
FileShareResponse(
|
|
||||||
id=row.id,
|
id=row.id,
|
||||||
target_type=row.target_type,
|
target_type=row.target_type,
|
||||||
target_id=row.target_id,
|
target_id=row.target_id,
|
||||||
@@ -985,27 +1000,62 @@ def _asset_list_response(session: Session, assets: list[FileAsset], *, include_s
|
|||||||
created_at=row.created_at.isoformat(),
|
created_at=row.created_at.isoformat(),
|
||||||
revoked_at=row.revoked_at.isoformat() if row.revoked_at else None,
|
revoked_at=row.revoked_at.isoformat() if row.revoked_at else None,
|
||||||
)
|
)
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
|
def _asset_versions_and_blobs(session: Session, assets: list[FileAsset]) -> dict[str, tuple[FileVersion, FileBlob]]:
|
||||||
|
version_ids = [asset.current_version_id for asset in assets if asset.current_version_id]
|
||||||
|
if len(version_ids) != len(assets):
|
||||||
|
raise FileStorageError("File has no current version")
|
||||||
|
rows: list[tuple[FileVersion, FileBlob]] = []
|
||||||
|
for chunk in _chunks(version_ids):
|
||||||
|
rows.extend(
|
||||||
|
session.query(FileVersion, FileBlob)
|
||||||
|
.join(FileBlob, FileBlob.id == FileVersion.blob_id)
|
||||||
|
.filter(FileVersion.id.in_(chunk))
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
return {version.id: (version, blob) for version, blob in rows}
|
||||||
|
|
||||||
|
|
||||||
|
def _asset_shares_by_id(session: Session, asset_ids: list[str], *, include_shares: bool) -> dict[str, list[FileShareResponse]]:
|
||||||
|
shares_by_asset_id: dict[str, list[FileShareResponse]] = {asset_id: [] for asset_id in asset_ids}
|
||||||
|
if not include_shares:
|
||||||
|
return shares_by_asset_id
|
||||||
|
for chunk in _chunks(asset_ids):
|
||||||
|
rows = (
|
||||||
|
session.query(FileShare)
|
||||||
|
.filter(FileShare.file_asset_id.in_(chunk))
|
||||||
|
.order_by(FileShare.created_at.desc())
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
for row in rows:
|
||||||
|
shares_by_asset_id.setdefault(row.file_asset_id, []).append(_file_share_response(row))
|
||||||
|
return shares_by_asset_id
|
||||||
|
|
||||||
|
|
||||||
|
def _sent_campaign_asset_ids(session: Session, asset_ids: list[str]) -> set[str]:
|
||||||
sent_asset_ids: set[str] = set()
|
sent_asset_ids: set[str] = set()
|
||||||
for chunk in _chunks(asset_ids):
|
for chunk in _chunks(asset_ids):
|
||||||
sent_rows = (
|
rows = (
|
||||||
session.query(CampaignAttachmentUse.file_asset_id)
|
session.query(CampaignAttachmentUse.file_asset_id)
|
||||||
.filter(CampaignAttachmentUse.file_asset_id.in_(chunk), CampaignAttachmentUse.use_stage == "sent")
|
.filter(CampaignAttachmentUse.file_asset_id.in_(chunk), CampaignAttachmentUse.use_stage == "sent")
|
||||||
.distinct()
|
.distinct()
|
||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
sent_asset_ids.update(row[0] for row in sent_rows)
|
sent_asset_ids.update(row[0] for row in rows)
|
||||||
|
return sent_asset_ids
|
||||||
|
|
||||||
responses: list[FileAssetResponse] = []
|
|
||||||
for asset in assets:
|
def _loaded_asset_response(
|
||||||
version = versions_by_id.get(asset.current_version_id or "")
|
asset: FileAsset,
|
||||||
blob = blobs_by_version_id.get(asset.current_version_id or "")
|
*,
|
||||||
if not version or not blob:
|
version: FileVersion,
|
||||||
raise FileStorageError("File version not found")
|
blob: FileBlob,
|
||||||
|
shares: list[FileShareResponse],
|
||||||
|
sent_asset_ids: set[str],
|
||||||
|
) -> FileAssetResponse:
|
||||||
metadata = asset.metadata_ or {}
|
metadata = asset.metadata_ or {}
|
||||||
responses.append(
|
return FileAssetResponse(
|
||||||
FileAssetResponse(
|
|
||||||
id=asset.id,
|
id=asset.id,
|
||||||
tenant_id=asset.tenant_id,
|
tenant_id=asset.tenant_id,
|
||||||
owner_type=asset.owner_type,
|
owner_type=asset.owner_type,
|
||||||
@@ -1024,7 +1074,32 @@ def _asset_list_response(session: Session, assets: list[FileAsset], *, include_s
|
|||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
source_provenance=source_provenance_from_metadata(metadata),
|
source_provenance=source_provenance_from_metadata(metadata),
|
||||||
source_revision=source_revision_from_metadata(metadata),
|
source_revision=source_revision_from_metadata(metadata),
|
||||||
|
shares=shares,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _asset_list_response(session: Session, assets: list[FileAsset], *, include_shares: bool = False) -> list[FileAssetResponse]:
|
||||||
|
if not assets:
|
||||||
|
return []
|
||||||
|
|
||||||
|
asset_ids = [asset.id for asset in assets]
|
||||||
|
versions_and_blobs = _asset_versions_and_blobs(session, assets)
|
||||||
|
shares_by_asset_id = _asset_shares_by_id(session, asset_ids, include_shares=include_shares)
|
||||||
|
sent_asset_ids = _sent_campaign_asset_ids(session, asset_ids)
|
||||||
|
|
||||||
|
responses: list[FileAssetResponse] = []
|
||||||
|
for asset in assets:
|
||||||
|
version_and_blob = versions_and_blobs.get(asset.current_version_id or "")
|
||||||
|
if version_and_blob is None:
|
||||||
|
raise FileStorageError("File version not found")
|
||||||
|
version, blob = version_and_blob
|
||||||
|
responses.append(
|
||||||
|
_loaded_asset_response(
|
||||||
|
asset,
|
||||||
|
version=version,
|
||||||
|
blob=blob,
|
||||||
shares=shares_by_asset_id.get(asset.id, []),
|
shares=shares_by_asset_id.get(asset.id, []),
|
||||||
|
sent_asset_ids=sent_asset_ids,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return responses
|
return responses
|
||||||
@@ -1108,7 +1183,11 @@ def _connector_space_policy_decision(
|
|||||||
provider=profile.provider,
|
provider=profile.provider,
|
||||||
external_id=external_id,
|
external_id=external_id,
|
||||||
external_path=browse_path,
|
external_path=browse_path,
|
||||||
external_url=profile.endpoint_url,
|
external_url=connector_effective_endpoint_url(
|
||||||
|
provider=profile.provider,
|
||||||
|
endpoint_url=profile.endpoint_url,
|
||||||
|
metadata=profile.metadata,
|
||||||
|
),
|
||||||
operation=operation,
|
operation=operation,
|
||||||
),
|
),
|
||||||
profile.policy_sources,
|
profile.policy_sources,
|
||||||
@@ -1450,6 +1529,104 @@ def _entry_matches_delta_scope(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_files_delta_watermark(since: str) -> int:
|
||||||
|
try:
|
||||||
|
return decode_sequence_watermark(since)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _changed_delta_resource_ids(entries: list[ChangeSequenceEntry]) -> tuple[list[str], list[str]]:
|
||||||
|
file_ids = list(dict.fromkeys(entry.resource_id for entry in entries if entry.resource_type == "file"))
|
||||||
|
folder_ids = list(dict.fromkeys(entry.resource_id for entry in entries if entry.resource_type == "folder"))
|
||||||
|
return file_ids, folder_ids
|
||||||
|
|
||||||
|
|
||||||
|
def _visible_assets_for_delta(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
owner_type: Literal["user", "group"] | None,
|
||||||
|
owner_id: str | None,
|
||||||
|
campaign_id: str | None,
|
||||||
|
path_prefix: str | None,
|
||||||
|
changed_file_ids: list[str],
|
||||||
|
) -> dict[str, FileAsset]:
|
||||||
|
return {
|
||||||
|
asset.id: asset
|
||||||
|
for asset in list_assets_for_user(
|
||||||
|
session,
|
||||||
|
tenant_id=principal.tenant_id,
|
||||||
|
user_id=principal.user.id,
|
||||||
|
owner_type=owner_type,
|
||||||
|
owner_id=owner_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
path_prefix=path_prefix,
|
||||||
|
is_admin=_is_admin(principal),
|
||||||
|
)
|
||||||
|
if asset.id in changed_file_ids
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _changed_visible_folders_for_delta(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
owner_type: Literal["user", "group"] | None,
|
||||||
|
owner_id: str | None,
|
||||||
|
path_prefix: str | None,
|
||||||
|
changed_folder_ids: list[str],
|
||||||
|
) -> dict[str, FileFolder]:
|
||||||
|
return {
|
||||||
|
folder.id: folder
|
||||||
|
for folder in _visible_folders_for_delta(
|
||||||
|
session,
|
||||||
|
principal=principal,
|
||||||
|
owner_type=owner_type,
|
||||||
|
owner_id=owner_id,
|
||||||
|
path_prefix=path_prefix,
|
||||||
|
)
|
||||||
|
if folder.id in changed_folder_ids
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _deleted_delta_items(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
principal: ApiPrincipal,
|
||||||
|
entries: list[ChangeSequenceEntry],
|
||||||
|
visible_assets: dict[str, FileAsset],
|
||||||
|
visible_folders: dict[str, FileFolder],
|
||||||
|
owner_type: Literal["user", "group"] | None,
|
||||||
|
owner_id: str | None,
|
||||||
|
campaign_id: str | None,
|
||||||
|
path_prefix: str | None,
|
||||||
|
) -> list[DeltaDeletedItem]:
|
||||||
|
deleted: dict[tuple[str, str], DeltaDeletedItem] = {}
|
||||||
|
for entry in entries:
|
||||||
|
if entry.resource_type == "file" and entry.resource_id in visible_assets:
|
||||||
|
continue
|
||||||
|
if entry.resource_type == "folder" and entry.resource_id in visible_folders:
|
||||||
|
continue
|
||||||
|
if not _entry_matches_delta_scope(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
entry,
|
||||||
|
owner_type=owner_type,
|
||||||
|
owner_id=owner_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
path_prefix=path_prefix,
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
deleted[(entry.resource_type, entry.resource_id)] = DeltaDeletedItem(
|
||||||
|
id=entry.resource_id,
|
||||||
|
resource_type=entry.resource_type,
|
||||||
|
revision=encode_sequence_watermark(entry.id),
|
||||||
|
deleted_at=entry.created_at if entry.operation == "deleted" else None,
|
||||||
|
)
|
||||||
|
return list(deleted.values())
|
||||||
|
|
||||||
|
|
||||||
def _files_delta_response(
|
def _files_delta_response(
|
||||||
session: Session,
|
session: Session,
|
||||||
*,
|
*,
|
||||||
@@ -1461,10 +1638,7 @@ def _files_delta_response(
|
|||||||
since: str,
|
since: str,
|
||||||
limit: int,
|
limit: int,
|
||||||
) -> FileDeltaResponse:
|
) -> FileDeltaResponse:
|
||||||
try:
|
since_sequence = _decode_files_delta_watermark(since)
|
||||||
since_sequence = decode_sequence_watermark(since)
|
|
||||||
except ValueError as exc:
|
|
||||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc
|
|
||||||
if sequence_watermark_is_expired(
|
if sequence_watermark_is_expired(
|
||||||
session,
|
session,
|
||||||
since=since_sequence,
|
since=since_sequence,
|
||||||
@@ -1492,65 +1666,41 @@ def _files_delta_response(
|
|||||||
has_more = len(entries_plus_one) > limit
|
has_more = len(entries_plus_one) > limit
|
||||||
entries = entries_plus_one[:limit]
|
entries = entries_plus_one[:limit]
|
||||||
|
|
||||||
changed_file_ids = list(dict.fromkeys(entry.resource_id for entry in entries if entry.resource_type == "file"))
|
changed_file_ids, changed_folder_ids = _changed_delta_resource_ids(entries)
|
||||||
changed_folder_ids = list(dict.fromkeys(entry.resource_id for entry in entries if entry.resource_type == "folder"))
|
visible_assets = _visible_assets_for_delta(
|
||||||
|
|
||||||
visible_assets = {
|
|
||||||
asset.id: asset
|
|
||||||
for asset in list_assets_for_user(
|
|
||||||
session,
|
session,
|
||||||
tenant_id=principal.tenant_id,
|
principal=principal,
|
||||||
user_id=principal.user.id,
|
|
||||||
owner_type=owner_type,
|
owner_type=owner_type,
|
||||||
owner_id=owner_id,
|
owner_id=owner_id,
|
||||||
campaign_id=campaign_id,
|
campaign_id=campaign_id,
|
||||||
path_prefix=path_prefix,
|
path_prefix=path_prefix,
|
||||||
is_admin=_is_admin(principal),
|
changed_file_ids=changed_file_ids,
|
||||||
)
|
)
|
||||||
if asset.id in changed_file_ids
|
visible_folders = _changed_visible_folders_for_delta(
|
||||||
}
|
|
||||||
visible_folders = {
|
|
||||||
folder.id: folder
|
|
||||||
for folder in _visible_folders_for_delta(
|
|
||||||
session,
|
session,
|
||||||
principal=principal,
|
principal=principal,
|
||||||
owner_type=owner_type,
|
owner_type=owner_type,
|
||||||
owner_id=owner_id,
|
owner_id=owner_id,
|
||||||
path_prefix=path_prefix,
|
path_prefix=path_prefix,
|
||||||
|
changed_folder_ids=changed_folder_ids,
|
||||||
)
|
)
|
||||||
if folder.id in changed_folder_ids
|
deleted = _deleted_delta_items(
|
||||||
}
|
|
||||||
|
|
||||||
deleted: dict[tuple[str, str], DeltaDeletedItem] = {}
|
|
||||||
for entry in entries:
|
|
||||||
is_visible = (
|
|
||||||
(entry.resource_type == "file" and entry.resource_id in visible_assets)
|
|
||||||
or (entry.resource_type == "folder" and entry.resource_id in visible_folders)
|
|
||||||
)
|
|
||||||
if is_visible:
|
|
||||||
continue
|
|
||||||
if not _entry_matches_delta_scope(
|
|
||||||
session,
|
session,
|
||||||
principal,
|
principal=principal,
|
||||||
entry,
|
entries=entries,
|
||||||
|
visible_assets=visible_assets,
|
||||||
|
visible_folders=visible_folders,
|
||||||
owner_type=owner_type,
|
owner_type=owner_type,
|
||||||
owner_id=owner_id,
|
owner_id=owner_id,
|
||||||
campaign_id=campaign_id,
|
campaign_id=campaign_id,
|
||||||
path_prefix=path_prefix,
|
path_prefix=path_prefix,
|
||||||
):
|
|
||||||
continue
|
|
||||||
deleted[(entry.resource_type, entry.resource_id)] = DeltaDeletedItem(
|
|
||||||
id=entry.resource_id,
|
|
||||||
resource_type=entry.resource_type,
|
|
||||||
revision=encode_sequence_watermark(entry.id),
|
|
||||||
deleted_at=entry.created_at if entry.operation == "deleted" else None,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
watermark = encode_sequence_watermark(entries[-1].id) if has_more and entries else _files_delta_watermark(session, principal.tenant_id)
|
watermark = encode_sequence_watermark(entries[-1].id) if has_more and entries else _files_delta_watermark(session, principal.tenant_id)
|
||||||
return FileDeltaResponse(
|
return FileDeltaResponse(
|
||||||
files=_asset_list_response(session, list(visible_assets.values()), include_shares=True),
|
files=_asset_list_response(session, list(visible_assets.values()), include_shares=True),
|
||||||
folders=[_folder_response(folder) for folder in visible_folders.values()],
|
folders=[_folder_response(folder) for folder in visible_folders.values()],
|
||||||
deleted=list(deleted.values()),
|
deleted=deleted,
|
||||||
watermark=watermark,
|
watermark=watermark,
|
||||||
has_more=has_more,
|
has_more=has_more,
|
||||||
full=False,
|
full=False,
|
||||||
@@ -2373,8 +2523,17 @@ def list_connector_providers(
|
|||||||
@router.post("/connectors/discover", response_model=FileConnectorDiscoveryResponse)
|
@router.post("/connectors/discover", response_model=FileConnectorDiscoveryResponse)
|
||||||
def discover_connector_endpoint(
|
def discover_connector_endpoint(
|
||||||
payload: FileConnectorDiscoveryRequest,
|
payload: FileConnectorDiscoveryRequest,
|
||||||
|
session: Session = Depends(get_session),
|
||||||
principal: ApiPrincipal = Depends(require_any_scope("files:file:admin", "system:settings:write", "admin:settings:write")),
|
principal: ApiPrincipal = Depends(require_any_scope("files:file:admin", "system:settings:write", "admin:settings:write")),
|
||||||
):
|
):
|
||||||
|
try:
|
||||||
|
reject_api_controlled_deployment_references(
|
||||||
|
password_env=payload.credentials.password_env,
|
||||||
|
token_env=payload.credentials.token_env,
|
||||||
|
metadata=payload.metadata,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise _http_error(exc) from exc
|
||||||
if payload.provider not in {"webdav", "nextcloud"}:
|
if payload.provider not in {"webdav", "nextcloud"}:
|
||||||
return FileConnectorDiscoveryResponse(
|
return FileConnectorDiscoveryResponse(
|
||||||
provider=payload.provider,
|
provider=payload.provider,
|
||||||
@@ -2387,7 +2546,28 @@ def discover_connector_endpoint(
|
|||||||
for endpoint_url in _webdav_discovery_candidates(payload):
|
for endpoint_url in _webdav_discovery_candidates(payload):
|
||||||
profile = _discovery_profile_from_payload(payload, endpoint_url, principal=principal)
|
profile = _discovery_profile_from_payload(payload, endpoint_url, principal=principal)
|
||||||
try:
|
try:
|
||||||
|
_ensure_connector_configuration_allowed(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
connector_id=None,
|
||||||
|
credential_id=None,
|
||||||
|
provider=profile.provider,
|
||||||
|
endpoint_url=endpoint_url,
|
||||||
|
base_path=profile.base_path,
|
||||||
|
scope_type="tenant",
|
||||||
|
scope_id=principal.tenant_id,
|
||||||
|
operation="discover",
|
||||||
|
)
|
||||||
|
_audit_connector_discovery_attempt(
|
||||||
|
session,
|
||||||
|
principal,
|
||||||
|
provider=profile.provider,
|
||||||
|
endpoint_url=endpoint_url,
|
||||||
|
base_path=profile.base_path,
|
||||||
|
)
|
||||||
browse_connector_profile(profile, path=payload.base_path or "")
|
browse_connector_profile(profile, path=payload.base_path or "")
|
||||||
|
except ConnectorPolicyDenied as exc:
|
||||||
|
raise _connector_policy_error(exc) from exc
|
||||||
except (ConnectorBrowseError, ConnectorBrowseUnsupported, OSError, ValueError, json.JSONDecodeError) as exc:
|
except (ConnectorBrowseError, ConnectorBrowseUnsupported, OSError, ValueError, json.JSONDecodeError) as exc:
|
||||||
message = str(exc)
|
message = str(exc)
|
||||||
if "credentials were rejected" in message.casefold():
|
if "credentials were rejected" in message.casefold():
|
||||||
@@ -2400,7 +2580,7 @@ def discover_connector_endpoint(
|
|||||||
status="credentials_rejected",
|
status="credentials_rejected",
|
||||||
message="The endpoint was found, but login failed with these credentials.",
|
message="The endpoint was found, but login failed with these credentials.",
|
||||||
candidates=candidates,
|
candidates=candidates,
|
||||||
metadata={**payload.metadata, "discovered_by": "webdav-auth-challenge"},
|
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||||
)
|
)
|
||||||
candidates.append({"endpoint_url": endpoint_url, "status": "found", "message": "The endpoint exists, but credentials are required or were rejected."})
|
candidates.append({"endpoint_url": endpoint_url, "status": "found", "message": "The endpoint exists, but credentials are required or were rejected."})
|
||||||
return FileConnectorDiscoveryResponse(
|
return FileConnectorDiscoveryResponse(
|
||||||
@@ -2410,7 +2590,7 @@ def discover_connector_endpoint(
|
|||||||
status="found",
|
status="found",
|
||||||
message="The endpoint was found. Add working credentials before saving or testing the connection.",
|
message="The endpoint was found. Add working credentials before saving or testing the connection.",
|
||||||
candidates=candidates,
|
candidates=candidates,
|
||||||
metadata={**payload.metadata, "discovered_by": "webdav-auth-challenge"},
|
metadata={"discovered_by": "webdav-auth-challenge"},
|
||||||
)
|
)
|
||||||
candidates.append({"endpoint_url": endpoint_url, "status": "failed", "message": message})
|
candidates.append({"endpoint_url": endpoint_url, "status": "failed", "message": message})
|
||||||
continue
|
continue
|
||||||
@@ -2424,7 +2604,7 @@ def discover_connector_endpoint(
|
|||||||
status=status_value,
|
status=status_value,
|
||||||
message=message,
|
message=message,
|
||||||
candidates=candidates,
|
candidates=candidates,
|
||||||
metadata={**payload.metadata, "discovered_by": "webdav-propfind"},
|
metadata={"discovered_by": "webdav-propfind"},
|
||||||
)
|
)
|
||||||
return FileConnectorDiscoveryResponse(
|
return FileConnectorDiscoveryResponse(
|
||||||
provider=payload.provider,
|
provider=payload.provider,
|
||||||
@@ -2740,8 +2920,12 @@ def create_connector_profile(
|
|||||||
principal,
|
principal,
|
||||||
connector_id=payload.id,
|
connector_id=payload.id,
|
||||||
credential_id=payload.credential_profile_id,
|
credential_id=payload.credential_profile_id,
|
||||||
|
provider=payload.provider,
|
||||||
|
endpoint_url=connector_effective_endpoint_url(
|
||||||
provider=payload.provider,
|
provider=payload.provider,
|
||||||
endpoint_url=payload.endpoint_url,
|
endpoint_url=payload.endpoint_url,
|
||||||
|
metadata=payload.metadata,
|
||||||
|
),
|
||||||
base_path=payload.base_path,
|
base_path=payload.base_path,
|
||||||
scope_type=payload.scope_type,
|
scope_type=payload.scope_type,
|
||||||
scope_id=payload.scope_id,
|
scope_id=payload.scope_id,
|
||||||
@@ -2909,7 +3093,11 @@ def browse_connector_profile_items(
|
|||||||
credential_id=profile.credential_profile_id,
|
credential_id=profile.credential_profile_id,
|
||||||
provider=profile.provider,
|
provider=profile.provider,
|
||||||
external_path=browse_path,
|
external_path=browse_path,
|
||||||
external_url=profile.endpoint_url,
|
external_url=connector_effective_endpoint_url(
|
||||||
|
provider=profile.provider,
|
||||||
|
endpoint_url=profile.endpoint_url,
|
||||||
|
metadata=profile.metadata,
|
||||||
|
),
|
||||||
operation="browse",
|
operation="browse",
|
||||||
),
|
),
|
||||||
profile.policy_sources,
|
profile.policy_sources,
|
||||||
@@ -2985,8 +3173,12 @@ def update_connector_profile(
|
|||||||
principal,
|
principal,
|
||||||
connector_id=row.id,
|
connector_id=row.id,
|
||||||
credential_id=credential_profile_id,
|
credential_id=credential_profile_id,
|
||||||
|
provider=provider,
|
||||||
|
endpoint_url=connector_effective_endpoint_url(
|
||||||
provider=provider,
|
provider=provider,
|
||||||
endpoint_url=payload.endpoint_url if payload.endpoint_url is not None else row.endpoint_url,
|
endpoint_url=payload.endpoint_url if payload.endpoint_url is not None else row.endpoint_url,
|
||||||
|
metadata=payload.metadata if payload.metadata is not None else row.metadata_,
|
||||||
|
),
|
||||||
base_path=payload.base_path if payload.base_path is not None else row.base_path,
|
base_path=payload.base_path if payload.base_path is not None else row.base_path,
|
||||||
scope_type=row.scope_type,
|
scope_type=row.scope_type,
|
||||||
scope_id=row.scope_id,
|
scope_id=row.scope_id,
|
||||||
|
|||||||
@@ -4,6 +4,12 @@ from dataclasses import dataclass, field
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Iterable, Protocol
|
from typing import Iterable, Protocol
|
||||||
|
|
||||||
|
from govoplan_core.security.outbound_http import (
|
||||||
|
OutboundHttpError,
|
||||||
|
response_limit,
|
||||||
|
validate_unpinned_sdk_http_url,
|
||||||
|
)
|
||||||
|
|
||||||
from govoplan_files.backend.runtime import settings
|
from govoplan_files.backend.runtime import settings
|
||||||
|
|
||||||
|
|
||||||
@@ -96,15 +102,25 @@ class S3StorageBackend:
|
|||||||
import boto3
|
import boto3
|
||||||
except ModuleNotFoundError as exc:
|
except ModuleNotFoundError as exc:
|
||||||
raise StorageBackendError("boto3 is required for the S3 storage backend") from exc
|
raise StorageBackendError("boto3 is required for the S3 storage backend") from exc
|
||||||
|
try:
|
||||||
|
endpoint_url = validate_unpinned_sdk_http_url(
|
||||||
|
self.endpoint_url,
|
||||||
|
label="File storage S3 endpoint",
|
||||||
|
)
|
||||||
return boto3.client(
|
return boto3.client(
|
||||||
"s3",
|
"s3",
|
||||||
endpoint_url=self.endpoint_url,
|
endpoint_url=endpoint_url,
|
||||||
region_name=self.region_name,
|
region_name=self.region_name,
|
||||||
aws_access_key_id=self.access_key_id,
|
aws_access_key_id=self.access_key_id,
|
||||||
aws_secret_access_key=self.secret_access_key,
|
aws_secret_access_key=self.secret_access_key,
|
||||||
)
|
)
|
||||||
|
except OutboundHttpError as exc:
|
||||||
|
raise StorageBackendError(str(exc)) from exc
|
||||||
|
|
||||||
def put_bytes(self, key: str, data: bytes, *, content_type: str | None = None) -> None:
|
def put_bytes(self, key: str, data: bytes, *, content_type: str | None = None) -> None:
|
||||||
|
max_bytes = response_limit("file")
|
||||||
|
if len(data) > max_bytes:
|
||||||
|
raise StorageBackendError(f"Stored object exceeds the deployment limit of {max_bytes} bytes")
|
||||||
kwargs = {"Bucket": self.bucket, "Key": key, "Body": data}
|
kwargs = {"Bucket": self.bucket, "Key": key, "Body": data}
|
||||||
if content_type:
|
if content_type:
|
||||||
kwargs["ContentType"] = content_type
|
kwargs["ContentType"] = content_type
|
||||||
@@ -113,19 +129,39 @@ class S3StorageBackend:
|
|||||||
def get_bytes(self, key: str) -> bytes:
|
def get_bytes(self, key: str) -> bytes:
|
||||||
try:
|
try:
|
||||||
obj = self.client.get_object(Bucket=self.bucket, Key=key)
|
obj = self.client.get_object(Bucket=self.bucket, Key=key)
|
||||||
return obj["Body"].read()
|
max_bytes = response_limit("file")
|
||||||
|
body = obj["Body"]
|
||||||
|
try:
|
||||||
|
_reject_declared_object_size(obj, max_bytes=max_bytes)
|
||||||
|
data = body.read(max_bytes + 1)
|
||||||
|
if len(data) > max_bytes:
|
||||||
|
raise StorageBackendError(f"Stored object exceeds the deployment limit of {max_bytes} bytes")
|
||||||
|
return data
|
||||||
|
finally:
|
||||||
|
if hasattr(body, "close"):
|
||||||
|
body.close()
|
||||||
except Exception as exc: # pragma: no cover - depends on S3 backend
|
except Exception as exc: # pragma: no cover - depends on S3 backend
|
||||||
raise StorageBackendError(str(exc)) from exc
|
raise StorageBackendError(str(exc)) from exc
|
||||||
|
|
||||||
def iter_bytes(self, key: str, *, chunk_size: int = 1024 * 1024) -> Iterable[bytes]:
|
def iter_bytes(self, key: str, *, chunk_size: int = 1024 * 1024) -> Iterable[bytes]:
|
||||||
try:
|
try:
|
||||||
obj = self.client.get_object(Bucket=self.bucket, Key=key)
|
obj = self.client.get_object(Bucket=self.bucket, Key=key)
|
||||||
|
max_bytes = response_limit("file")
|
||||||
body = obj["Body"]
|
body = obj["Body"]
|
||||||
|
try:
|
||||||
|
_reject_declared_object_size(obj, max_bytes=max_bytes)
|
||||||
|
total = 0
|
||||||
while True:
|
while True:
|
||||||
chunk = body.read(chunk_size)
|
chunk = body.read(chunk_size)
|
||||||
if not chunk:
|
if not chunk:
|
||||||
break
|
break
|
||||||
|
total += len(chunk)
|
||||||
|
if total > max_bytes:
|
||||||
|
raise StorageBackendError(f"Stored object exceeds the deployment limit of {max_bytes} bytes")
|
||||||
yield chunk
|
yield chunk
|
||||||
|
finally:
|
||||||
|
if hasattr(body, "close"):
|
||||||
|
body.close()
|
||||||
except Exception as exc: # pragma: no cover - depends on S3 backend
|
except Exception as exc: # pragma: no cover - depends on S3 backend
|
||||||
raise StorageBackendError(str(exc)) from exc
|
raise StorageBackendError(str(exc)) from exc
|
||||||
|
|
||||||
@@ -140,6 +176,17 @@ class S3StorageBackend:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_declared_object_size(obj: object, *, max_bytes: int) -> None:
|
||||||
|
if not isinstance(obj, dict):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
declared_size = int(obj.get("ContentLength"))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return
|
||||||
|
if declared_size > max_bytes:
|
||||||
|
raise StorageBackendError(f"Stored object exceeds the deployment limit of {max_bytes} bytes")
|
||||||
|
|
||||||
|
|
||||||
def _fallback_roots() -> tuple[Path, ...]:
|
def _fallback_roots() -> tuple[Path, ...]:
|
||||||
raw = getattr(settings, "file_storage_local_fallback_roots", "") or ""
|
raw = getattr(settings, "file_storage_local_fallback_roots", "") or ""
|
||||||
return tuple(Path(item.strip()) for item in str(raw).split(",") if item.strip())
|
return tuple(Path(item.strip()) for item in str(raw).split(",") if item.strip())
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ from typing import Any, Iterator
|
|||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from govoplan_files.backend.db.models import FileAsset, FileShare
|
from govoplan_files.backend.db.models import FileAsset, FileBlob, FileShare, FileVersion
|
||||||
from govoplan_files.backend.storage.backends import StorageBackendError, get_storage_backend
|
from govoplan_files.backend.storage.backends import StorageBackend, StorageBackendError, get_storage_backend
|
||||||
from govoplan_files.backend.storage.common import FileStorageError
|
from govoplan_files.backend.storage.common import FileStorageError
|
||||||
from govoplan_files.backend.storage.files import current_versions_and_blobs, get_asset_for_user, list_assets_for_user, share_files
|
from govoplan_files.backend.storage.files import current_versions_and_blobs, get_asset_for_user, list_assets_for_user, share_files
|
||||||
from govoplan_files.backend.storage.access import ensure_owner_access
|
from govoplan_files.backend.storage.access import ensure_owner_access
|
||||||
@@ -232,6 +232,177 @@ def _selected_base_path(
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _prepared_attachment_config(raw_json: dict[str, Any]) -> tuple[dict[str, Any], dict[str, Any], list[Any]]:
|
||||||
|
prepared_json = copy.deepcopy(raw_json if isinstance(raw_json, dict) else {})
|
||||||
|
attachments = prepared_json.get("attachments")
|
||||||
|
if not isinstance(attachments, dict):
|
||||||
|
attachments = {}
|
||||||
|
prepared_json["attachments"] = attachments
|
||||||
|
base_paths = attachments.get("base_paths")
|
||||||
|
if not isinstance(base_paths, list):
|
||||||
|
base_paths = []
|
||||||
|
return prepared_json, attachments, base_paths
|
||||||
|
|
||||||
|
|
||||||
|
def _campaign_snapshot_assets(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
campaign_id: str,
|
||||||
|
prepared_json: dict[str, Any],
|
||||||
|
include_unlinked_candidates: bool,
|
||||||
|
user_id: str,
|
||||||
|
is_admin: bool,
|
||||||
|
) -> tuple[list[FileAsset], list[FileAsset], set[str]]:
|
||||||
|
shared_assets = list_assets_for_user(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
user_id="",
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
is_admin=True,
|
||||||
|
)
|
||||||
|
if not include_unlinked_candidates:
|
||||||
|
return shared_assets, shared_assets, {asset.id for asset in shared_assets}
|
||||||
|
candidate_assets, linked_asset_ids = _candidate_assets_for_managed_sources(
|
||||||
|
session,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
campaign_id=campaign_id,
|
||||||
|
raw_json=prepared_json,
|
||||||
|
user_id=user_id,
|
||||||
|
is_admin=is_admin,
|
||||||
|
shared_assets=shared_assets,
|
||||||
|
)
|
||||||
|
return shared_assets, candidate_assets, linked_asset_ids
|
||||||
|
|
||||||
|
|
||||||
|
def _assets_grouped_by_owner(assets: list[FileAsset]) -> dict[tuple[str, str], list[FileAsset]]:
|
||||||
|
assets_by_owner: dict[tuple[str, str], list[FileAsset]] = defaultdict(list)
|
||||||
|
for asset in assets:
|
||||||
|
owner_id = _asset_owner_id(asset)
|
||||||
|
if owner_id:
|
||||||
|
assets_by_owner[(asset.owner_type, owner_id)].append(asset)
|
||||||
|
return assets_by_owner
|
||||||
|
|
||||||
|
|
||||||
|
def _materialize_managed_asset(
|
||||||
|
asset: FileAsset,
|
||||||
|
*,
|
||||||
|
owner_id: str,
|
||||||
|
logical_root: str,
|
||||||
|
local_root: Path,
|
||||||
|
version_blobs: dict[str, tuple[FileVersion, FileBlob]],
|
||||||
|
backend: StorageBackend | None,
|
||||||
|
include_bytes: bool,
|
||||||
|
linked_asset_ids: set[str],
|
||||||
|
) -> tuple[str, ManagedAttachmentFile] | None:
|
||||||
|
relative_path = _relative_asset_path(asset, logical_root)
|
||||||
|
if not relative_path:
|
||||||
|
return None
|
||||||
|
target = _safe_local_target(local_root, relative_path)
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
version, blob = version_blobs[asset.id]
|
||||||
|
if include_bytes:
|
||||||
|
try:
|
||||||
|
data = backend.get_bytes(blob.storage_key) if backend else b""
|
||||||
|
except StorageBackendError as exc:
|
||||||
|
raise FileStorageError(str(exc)) from exc
|
||||||
|
target.write_bytes(data)
|
||||||
|
else:
|
||||||
|
target.touch()
|
||||||
|
local_key = str(target.resolve())
|
||||||
|
return local_key, ManagedAttachmentFile(
|
||||||
|
local_path=local_key,
|
||||||
|
asset_id=asset.id,
|
||||||
|
version_id=version.id,
|
||||||
|
blob_id=blob.id,
|
||||||
|
display_path=asset.display_path,
|
||||||
|
relative_path=normalize_logical_path(relative_path),
|
||||||
|
filename=asset.filename,
|
||||||
|
owner_type=asset.owner_type,
|
||||||
|
owner_id=owner_id,
|
||||||
|
checksum_sha256=blob.checksum_sha256,
|
||||||
|
size_bytes=blob.size_bytes,
|
||||||
|
content_type=blob.content_type,
|
||||||
|
linked_to_campaign=asset.id in linked_asset_ids,
|
||||||
|
source_provenance=source_provenance_from_metadata(asset.metadata_),
|
||||||
|
source_revision=source_revision_from_metadata(asset.metadata_),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _prepare_managed_base_paths(
|
||||||
|
base_paths: list[Any],
|
||||||
|
*,
|
||||||
|
materialized_root: Path,
|
||||||
|
assets_by_owner: dict[tuple[str, str], list[FileAsset]],
|
||||||
|
version_blobs: dict[str, tuple[FileVersion, FileBlob]],
|
||||||
|
backend: StorageBackend | None,
|
||||||
|
include_bytes: bool,
|
||||||
|
linked_asset_ids: set[str],
|
||||||
|
) -> tuple[
|
||||||
|
dict[str, ManagedAttachmentFile],
|
||||||
|
dict[str, tuple[str, str]],
|
||||||
|
dict[str, list[tuple[str, str]]],
|
||||||
|
tuple[str, str] | None,
|
||||||
|
]:
|
||||||
|
manifest: dict[str, ManagedAttachmentFile] = {}
|
||||||
|
prepared_by_id: dict[str, tuple[str, str]] = {}
|
||||||
|
prepared_by_old_path: dict[str, list[tuple[str, str]]] = {}
|
||||||
|
first_prepared: tuple[str, str] | None = None
|
||||||
|
for index, item in enumerate(base_paths):
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
parsed_source = parse_managed_source(item.get("source"))
|
||||||
|
if parsed_source is None:
|
||||||
|
continue
|
||||||
|
owner_type, owner_id = parsed_source
|
||||||
|
old_path = str(item.get("path") or ".").strip() or "."
|
||||||
|
logical_root = "" if old_path in {"", ".", "/"} else normalize_folder(old_path)
|
||||||
|
base_path_id = str(item.get("id") or f"base-path-{index + 1}")
|
||||||
|
local_root = materialized_root / f"{index + 1:03d}-{safe_storage_component(base_path_id)}"
|
||||||
|
local_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
local_root_string = str(local_root.resolve())
|
||||||
|
prepared = (base_path_id, local_root_string)
|
||||||
|
prepared_by_id[base_path_id] = prepared
|
||||||
|
prepared_by_old_path.setdefault(old_path, []).append(prepared)
|
||||||
|
if first_prepared is None:
|
||||||
|
first_prepared = prepared
|
||||||
|
item["path"] = local_root_string
|
||||||
|
for asset in assets_by_owner.get((owner_type, owner_id), []):
|
||||||
|
materialized = _materialize_managed_asset(
|
||||||
|
asset,
|
||||||
|
owner_id=owner_id,
|
||||||
|
logical_root=logical_root,
|
||||||
|
local_root=local_root,
|
||||||
|
version_blobs=version_blobs,
|
||||||
|
backend=backend,
|
||||||
|
include_bytes=include_bytes,
|
||||||
|
linked_asset_ids=linked_asset_ids,
|
||||||
|
)
|
||||||
|
if materialized is not None:
|
||||||
|
local_key, managed_file = materialized
|
||||||
|
manifest[local_key] = managed_file
|
||||||
|
return manifest, prepared_by_id, prepared_by_old_path, first_prepared
|
||||||
|
|
||||||
|
|
||||||
|
def _rewrite_managed_attachment_rules(
|
||||||
|
attachments: dict[str, Any],
|
||||||
|
prepared_json: dict[str, Any],
|
||||||
|
*,
|
||||||
|
prepared_by_id: dict[str, tuple[str, str]],
|
||||||
|
prepared_by_old_path: dict[str, list[tuple[str, str]]],
|
||||||
|
first_prepared: tuple[str, str] | None,
|
||||||
|
) -> None:
|
||||||
|
for rule in _iter_rule_dicts(attachments, prepared_json):
|
||||||
|
selected = _selected_base_path(rule, prepared_by_id, prepared_by_old_path, first_prepared)
|
||||||
|
if selected is None:
|
||||||
|
continue
|
||||||
|
base_path_id, local_root_string = selected
|
||||||
|
rule["base_path_id"] = base_path_id
|
||||||
|
rule["base_dir"] = local_root_string
|
||||||
|
if first_prepared is not None:
|
||||||
|
attachments["base_path"] = first_prepared[1]
|
||||||
|
|
||||||
|
|
||||||
def prepare_campaign_snapshot(
|
def prepare_campaign_snapshot(
|
||||||
session: Session,
|
session: Session,
|
||||||
*,
|
*,
|
||||||
@@ -258,114 +429,35 @@ def prepare_campaign_snapshot(
|
|||||||
materialized_root = destination / "managed-attachments"
|
materialized_root = destination / "managed-attachments"
|
||||||
materialized_root.mkdir(parents=True, exist_ok=True)
|
materialized_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
prepared_json = copy.deepcopy(raw_json if isinstance(raw_json, dict) else {})
|
prepared_json, attachments, base_paths = _prepared_attachment_config(raw_json)
|
||||||
attachments = prepared_json.get("attachments")
|
shared_assets, candidate_assets, linked_asset_ids = _campaign_snapshot_assets(
|
||||||
if not isinstance(attachments, dict):
|
|
||||||
attachments = {}
|
|
||||||
prepared_json["attachments"] = attachments
|
|
||||||
base_paths = attachments.get("base_paths")
|
|
||||||
if not isinstance(base_paths, list):
|
|
||||||
base_paths = []
|
|
||||||
|
|
||||||
shared_assets = list_assets_for_user(
|
|
||||||
session,
|
|
||||||
tenant_id=tenant_id,
|
|
||||||
user_id="",
|
|
||||||
campaign_id=campaign_id,
|
|
||||||
is_admin=True,
|
|
||||||
)
|
|
||||||
candidate_assets = shared_assets
|
|
||||||
linked_asset_ids = {asset.id for asset in shared_assets}
|
|
||||||
if include_unlinked_candidates:
|
|
||||||
candidate_assets, linked_asset_ids = _candidate_assets_for_managed_sources(
|
|
||||||
session,
|
session,
|
||||||
tenant_id=tenant_id,
|
tenant_id=tenant_id,
|
||||||
campaign_id=campaign_id,
|
campaign_id=campaign_id,
|
||||||
raw_json=prepared_json,
|
prepared_json=prepared_json,
|
||||||
|
include_unlinked_candidates=include_unlinked_candidates,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
is_admin=is_admin,
|
is_admin=is_admin,
|
||||||
shared_assets=shared_assets,
|
|
||||||
)
|
)
|
||||||
|
assets_by_owner = _assets_grouped_by_owner(candidate_assets)
|
||||||
assets_by_owner: dict[tuple[str, str], list[FileAsset]] = defaultdict(list)
|
|
||||||
for asset in candidate_assets:
|
|
||||||
owner_id = _asset_owner_id(asset)
|
|
||||||
if owner_id:
|
|
||||||
assets_by_owner[(asset.owner_type, owner_id)].append(asset)
|
|
||||||
version_blobs = current_versions_and_blobs(session, candidate_assets)
|
version_blobs = current_versions_and_blobs(session, candidate_assets)
|
||||||
backend = get_storage_backend() if include_bytes else None
|
backend = get_storage_backend() if include_bytes else None
|
||||||
|
manifest, prepared_by_id, prepared_by_old_path, first_prepared = _prepare_managed_base_paths(
|
||||||
manifest: dict[str, ManagedAttachmentFile] = {}
|
base_paths,
|
||||||
prepared_by_id: dict[str, tuple[str, str]] = {}
|
materialized_root=materialized_root,
|
||||||
prepared_by_old_path: dict[str, list[tuple[str, str]]] = {}
|
assets_by_owner=assets_by_owner,
|
||||||
first_prepared: tuple[str, str] | None = None
|
version_blobs=version_blobs,
|
||||||
|
backend=backend,
|
||||||
for index, item in enumerate(base_paths):
|
include_bytes=include_bytes,
|
||||||
if not isinstance(item, dict):
|
linked_asset_ids=linked_asset_ids,
|
||||||
continue
|
)
|
||||||
parsed_source = parse_managed_source(item.get("source"))
|
_rewrite_managed_attachment_rules(
|
||||||
if parsed_source is None:
|
attachments,
|
||||||
continue
|
prepared_json,
|
||||||
owner_type, owner_id = parsed_source
|
prepared_by_id=prepared_by_id,
|
||||||
old_path = str(item.get("path") or ".").strip() or "."
|
prepared_by_old_path=prepared_by_old_path,
|
||||||
logical_root = "" if old_path in {"", ".", "/"} else normalize_folder(old_path)
|
first_prepared=first_prepared,
|
||||||
base_path_id = str(item.get("id") or f"base-path-{index + 1}")
|
|
||||||
local_root = materialized_root / f"{index + 1:03d}-{safe_storage_component(base_path_id)}"
|
|
||||||
local_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
local_root_string = str(local_root.resolve())
|
|
||||||
|
|
||||||
prepared = (base_path_id, local_root_string)
|
|
||||||
prepared_by_id[base_path_id] = prepared
|
|
||||||
prepared_by_old_path.setdefault(old_path, []).append(prepared)
|
|
||||||
if first_prepared is None:
|
|
||||||
first_prepared = prepared
|
|
||||||
|
|
||||||
item["path"] = local_root_string
|
|
||||||
|
|
||||||
for asset in assets_by_owner.get((owner_type, owner_id), []):
|
|
||||||
relative_path = _relative_asset_path(asset, logical_root)
|
|
||||||
if not relative_path:
|
|
||||||
continue
|
|
||||||
target = _safe_local_target(local_root, relative_path)
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
version, blob = version_blobs[asset.id]
|
|
||||||
if include_bytes:
|
|
||||||
try:
|
|
||||||
data = backend.get_bytes(blob.storage_key) if backend else b""
|
|
||||||
except StorageBackendError as exc:
|
|
||||||
raise FileStorageError(str(exc)) from exc
|
|
||||||
target.write_bytes(data)
|
|
||||||
else:
|
|
||||||
target.touch()
|
|
||||||
local_key = str(target.resolve())
|
|
||||||
manifest[local_key] = ManagedAttachmentFile(
|
|
||||||
local_path=local_key,
|
|
||||||
asset_id=asset.id,
|
|
||||||
version_id=version.id,
|
|
||||||
blob_id=blob.id,
|
|
||||||
display_path=asset.display_path,
|
|
||||||
relative_path=normalize_logical_path(relative_path),
|
|
||||||
filename=asset.filename,
|
|
||||||
owner_type=asset.owner_type,
|
|
||||||
owner_id=owner_id,
|
|
||||||
checksum_sha256=blob.checksum_sha256,
|
|
||||||
size_bytes=blob.size_bytes,
|
|
||||||
content_type=blob.content_type,
|
|
||||||
linked_to_campaign=asset.id in linked_asset_ids,
|
|
||||||
source_provenance=source_provenance_from_metadata(asset.metadata_),
|
|
||||||
source_revision=source_revision_from_metadata(asset.metadata_),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
for rule in _iter_rule_dicts(attachments, prepared_json):
|
|
||||||
selected = _selected_base_path(rule, prepared_by_id, prepared_by_old_path, first_prepared)
|
|
||||||
if selected is None:
|
|
||||||
continue
|
|
||||||
base_path_id, local_root_string = selected
|
|
||||||
rule["base_path_id"] = base_path_id
|
|
||||||
rule["base_dir"] = local_root_string
|
|
||||||
|
|
||||||
if first_prepared is not None:
|
|
||||||
attachments["base_path"] = first_prepared[1]
|
|
||||||
|
|
||||||
snapshot_path = destination / "campaign.json"
|
snapshot_path = destination / "campaign.json"
|
||||||
snapshot_path.write_text(json.dumps(prepared_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
snapshot_path.write_text(json.dumps(prepared_json, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import json
|
||||||
from collections.abc import Mapping
|
from collections.abc import Mapping
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
@@ -12,8 +12,21 @@ from urllib.parse import quote, unquote, urljoin, urlsplit
|
|||||||
import xml.etree.ElementTree as ET # nosec B405 - typing/element creation only; parsing uses defusedxml below.
|
import xml.etree.ElementTree as ET # nosec B405 - typing/element creation only; parsing uses defusedxml below.
|
||||||
|
|
||||||
from defusedxml import ElementTree as SafeElementTree
|
from defusedxml import ElementTree as SafeElementTree
|
||||||
import httpx
|
|
||||||
|
|
||||||
|
from govoplan_core.security.outbound_http import (
|
||||||
|
OutboundHttpError,
|
||||||
|
outbound_http_policy,
|
||||||
|
pinned_outbound_hostname,
|
||||||
|
validate_unpinned_sdk_http_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
from govoplan_files.backend.storage.http_client import ConnectorHttpError, request_connector_bytes
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import (
|
||||||
|
ConnectorDeploymentConfigurationError,
|
||||||
|
connector_ca_bundle_path,
|
||||||
|
connector_secret_env_value,
|
||||||
|
validate_connector_tls_metadata,
|
||||||
|
)
|
||||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||||
|
|
||||||
|
|
||||||
@@ -170,14 +183,22 @@ def _browse_webdav(profile: ConnectorProfile, *, path: str) -> list[ConnectorBro
|
|||||||
</prop>
|
</prop>
|
||||||
</propfind>"""
|
</propfind>"""
|
||||||
try:
|
try:
|
||||||
response = httpx.request("PROPFIND", url, headers=headers, content=body, auth=auth, timeout=15.0)
|
response = request_connector_bytes(
|
||||||
except httpx.HTTPError as exc:
|
"PROPFIND",
|
||||||
|
url,
|
||||||
|
headers=headers,
|
||||||
|
content=body,
|
||||||
|
auth=auth,
|
||||||
|
timeout=15.0,
|
||||||
|
label="WebDAV browse",
|
||||||
|
)
|
||||||
|
except ConnectorHttpError as exc:
|
||||||
raise ConnectorBrowseError(f"Connector browse failed: {exc}") from exc
|
raise ConnectorBrowseError(f"Connector browse failed: {exc}") from exc
|
||||||
if response.status_code in {401, 403}:
|
if response.status_code in {401, 403}:
|
||||||
raise ConnectorBrowseError("Connector credentials were rejected")
|
raise ConnectorBrowseError("Connector credentials were rejected")
|
||||||
if response.status_code not in {200, 207}:
|
if response.status_code not in {200, 207}:
|
||||||
raise ConnectorBrowseError(f"Connector browse failed with HTTP {response.status_code}")
|
raise ConnectorBrowseError(f"Connector browse failed with HTTP {response.status_code}")
|
||||||
return parse_webdav_multistatus(root_url=root_url, current_path=path, payload=response.text)
|
return parse_webdav_multistatus(root_url=root_url, current_path=path, payload=response.content)
|
||||||
|
|
||||||
|
|
||||||
def _browse_smb(profile: ConnectorProfile, *, path: str) -> list[ConnectorBrowseItem]:
|
def _browse_smb(profile: ConnectorProfile, *, path: str) -> list[ConnectorBrowseItem]:
|
||||||
@@ -298,7 +319,17 @@ def _s3_client(profile: ConnectorProfile) -> Any:
|
|||||||
raise ConnectorBrowseUnsupported("S3 connector browsing requires the optional boto3 dependency") from exc
|
raise ConnectorBrowseUnsupported("S3 connector browsing requires the optional boto3 dependency") from exc
|
||||||
kwargs: dict[str, object] = {}
|
kwargs: dict[str, object] = {}
|
||||||
if profile.endpoint_url:
|
if profile.endpoint_url:
|
||||||
kwargs["endpoint_url"] = profile.endpoint_url
|
try:
|
||||||
|
kwargs["endpoint_url"] = validate_unpinned_sdk_http_url(
|
||||||
|
profile.endpoint_url,
|
||||||
|
label="S3 connector endpoint",
|
||||||
|
)
|
||||||
|
except OutboundHttpError as exc:
|
||||||
|
raise ConnectorBrowseError(str(exc)) from exc
|
||||||
|
elif not outbound_http_policy().allow_private_networks:
|
||||||
|
raise ConnectorBrowseError(
|
||||||
|
"S3 connector endpoint discovery cannot pin the SDK connection address while private-network access is disabled"
|
||||||
|
)
|
||||||
region = _metadata_string(profile, "region") or _metadata_string(profile, "aws_region")
|
region = _metadata_string(profile, "region") or _metadata_string(profile, "aws_region")
|
||||||
if region:
|
if region:
|
||||||
kwargs["region_name"] = region
|
kwargs["region_name"] = region
|
||||||
@@ -426,9 +457,16 @@ def _s3_max_keys(profile: ConnectorProfile) -> int:
|
|||||||
|
|
||||||
|
|
||||||
def _s3_verify(profile: ConnectorProfile) -> bool | str | None:
|
def _s3_verify(profile: ConnectorProfile) -> bool | str | None:
|
||||||
|
try:
|
||||||
|
validate_connector_tls_metadata(profile.metadata)
|
||||||
|
except ConnectorDeploymentConfigurationError as exc:
|
||||||
|
raise ConnectorBrowseError(str(exc)) from exc
|
||||||
ca_bundle = _metadata_string(profile, "ca_bundle")
|
ca_bundle = _metadata_string(profile, "ca_bundle")
|
||||||
if ca_bundle:
|
if ca_bundle:
|
||||||
return ca_bundle
|
try:
|
||||||
|
return connector_ca_bundle_path(ca_bundle)
|
||||||
|
except ConnectorDeploymentConfigurationError as exc:
|
||||||
|
raise ConnectorBrowseError(str(exc)) from exc
|
||||||
if "verify_tls" in profile.metadata:
|
if "verify_tls" in profile.metadata:
|
||||||
return _metadata_bool(profile, "verify_tls", default=True)
|
return _metadata_bool(profile, "verify_tls", default=True)
|
||||||
if "tls_verify" in profile.metadata:
|
if "tls_verify" in profile.metadata:
|
||||||
@@ -484,16 +522,24 @@ def _request_json(
|
|||||||
data: Mapping[str, str] | None = None,
|
data: Mapping[str, str] | None = None,
|
||||||
) -> object:
|
) -> object:
|
||||||
try:
|
try:
|
||||||
response = httpx.request(method, url, headers=dict(headers or {}), params=params, data=data, timeout=15.0)
|
response = request_connector_bytes(
|
||||||
except httpx.HTTPError as exc:
|
method,
|
||||||
|
url,
|
||||||
|
headers=dict(headers or {}),
|
||||||
|
params=params,
|
||||||
|
data=data,
|
||||||
|
timeout=15.0,
|
||||||
|
label="Seafile API",
|
||||||
|
)
|
||||||
|
except ConnectorHttpError as exc:
|
||||||
raise ConnectorBrowseError(f"Connector browse failed: {exc}") from exc
|
raise ConnectorBrowseError(f"Connector browse failed: {exc}") from exc
|
||||||
if response.status_code in {401, 403}:
|
if response.status_code in {401, 403}:
|
||||||
raise ConnectorBrowseError("Connector credentials were rejected")
|
raise ConnectorBrowseError("Connector credentials were rejected")
|
||||||
if response.status_code not in {200, 201}:
|
if response.status_code not in {200, 201}:
|
||||||
raise ConnectorBrowseError(f"Connector browse failed with HTTP {response.status_code}")
|
raise ConnectorBrowseError(f"Connector browse failed with HTTP {response.status_code}")
|
||||||
try:
|
try:
|
||||||
return response.json()
|
return json.loads(response.content)
|
||||||
except ValueError as exc:
|
except (UnicodeDecodeError, ValueError) as exc:
|
||||||
raise ConnectorBrowseError("Connector returned invalid JSON") from exc
|
raise ConnectorBrowseError("Connector returned invalid JSON") from exc
|
||||||
|
|
||||||
|
|
||||||
@@ -671,14 +717,14 @@ def _metadata_env(profile: ConnectorProfile, key: str) -> str | None:
|
|||||||
env_name = _metadata_string(profile, key)
|
env_name = _metadata_string(profile, key)
|
||||||
if not env_name:
|
if not env_name:
|
||||||
return None
|
return None
|
||||||
return _env_required(env_name, profile.id)
|
return _env_required(env_name, profile)
|
||||||
|
|
||||||
|
|
||||||
def _env_required(name: str, profile_id: str) -> str:
|
def _env_required(name: str, profile: ConnectorProfile) -> str:
|
||||||
value = _clean(os.environ.get(name))
|
try:
|
||||||
if not value:
|
return connector_secret_env_value(name, source_kind=profile.source_kind)
|
||||||
raise ConnectorBrowseError(f"Connector profile {profile_id} is missing required credential environment")
|
except ConnectorDeploymentConfigurationError as exc:
|
||||||
return value
|
raise ConnectorBrowseError(f"Connector profile {profile.id}: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
def normalize_connector_browse_path(value: object) -> str:
|
def normalize_connector_browse_path(value: object) -> str:
|
||||||
@@ -735,6 +781,11 @@ def _smb_location(profile: ConnectorProfile) -> _SmbLocation:
|
|||||||
server = _clean(parsed.hostname)
|
server = _clean(parsed.hostname)
|
||||||
if not server:
|
if not server:
|
||||||
raise ConnectorBrowseError("SMB connector endpoint_url must include a server")
|
raise ConnectorBrowseError("SMB connector endpoint_url must include a server")
|
||||||
|
port = parsed.port or _int(profile.metadata.get("port")) or 445
|
||||||
|
try:
|
||||||
|
server = pinned_outbound_hostname(server, port=port, label="SMB connector endpoint")
|
||||||
|
except OutboundHttpError as exc:
|
||||||
|
raise ConnectorBrowseError(str(exc)) from exc
|
||||||
path_parts = [part for part in unquote(parsed.path or "").strip("/").split("/") if part]
|
path_parts = [part for part in unquote(parsed.path or "").strip("/").split("/") if part]
|
||||||
if not path_parts:
|
if not path_parts:
|
||||||
raise ConnectorBrowseError("SMB connector endpoint_url must include a share name")
|
raise ConnectorBrowseError("SMB connector endpoint_url must include a share name")
|
||||||
@@ -744,7 +795,7 @@ def _smb_location(profile: ConnectorProfile) -> _SmbLocation:
|
|||||||
return _SmbLocation(
|
return _SmbLocation(
|
||||||
server=server,
|
server=server,
|
||||||
share=path_parts[0],
|
share=path_parts[0],
|
||||||
port=parsed.port or _int(profile.metadata.get("port")) or 445,
|
port=port,
|
||||||
root_path=normalize_connector_browse_path("/".join(root_parts)),
|
root_path=normalize_connector_browse_path("/".join(root_parts)),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -780,7 +831,7 @@ def _profile_password(profile: ConnectorProfile) -> str | None:
|
|||||||
if profile.password_value:
|
if profile.password_value:
|
||||||
return profile.password_value
|
return profile.password_value
|
||||||
if profile.password_env:
|
if profile.password_env:
|
||||||
return _env_required(profile.password_env, profile.id)
|
return _env_required(profile.password_env, profile)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
@@ -788,7 +839,7 @@ def _profile_token(profile: ConnectorProfile) -> str | None:
|
|||||||
if profile.token_value:
|
if profile.token_value:
|
||||||
return profile.token_value
|
return profile.token_value
|
||||||
if profile.token_env:
|
if profile.token_env:
|
||||||
return _env_required(profile.token_env, profile.id)
|
return _env_required(profile.token_env, profile)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ from govoplan_core.core.policy import normalize_policy_scope_type, policy_source
|
|||||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||||
from govoplan_files.backend.db.models import FileConnectorCredential
|
from govoplan_files.backend.db.models import FileConnectorCredential
|
||||||
from govoplan_files.backend.storage.common import FileStorageError
|
from govoplan_files.backend.storage.common import FileStorageError
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import reject_api_controlled_deployment_references
|
||||||
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
||||||
from govoplan_files.backend.storage.connector_profiles import supported_connector_providers
|
from govoplan_files.backend.storage.connector_profiles import supported_connector_providers
|
||||||
|
|
||||||
@@ -53,7 +54,9 @@ class ConnectorCredential:
|
|||||||
def credentials_configured(self) -> bool:
|
def credentials_configured(self) -> bool:
|
||||||
if self.credential_mode.casefold() in {"", "none", "anonymous"}:
|
if self.credential_mode.casefold() in {"", "none", "anonymous"}:
|
||||||
return True
|
return True
|
||||||
return bool(self.secret_ref or self.password_value or self.token_value or self.password_env or self.token_env)
|
# Environment references are deliberately unavailable to API-managed
|
||||||
|
# credential rows. Legacy rows remain visible but fail closed.
|
||||||
|
return bool(self.secret_ref or self.password_value or self.token_value)
|
||||||
|
|
||||||
def to_response(self) -> dict[str, Any]:
|
def to_response(self) -> dict[str, Any]:
|
||||||
return {
|
return {
|
||||||
@@ -182,6 +185,11 @@ def create_connector_credential_row(
|
|||||||
policy: Mapping[str, Any] | None = None,
|
policy: Mapping[str, Any] | None = None,
|
||||||
metadata: Mapping[str, Any] | None = None,
|
metadata: Mapping[str, Any] | None = None,
|
||||||
) -> FileConnectorCredential:
|
) -> FileConnectorCredential:
|
||||||
|
reject_api_controlled_deployment_references(
|
||||||
|
password_env=password_env,
|
||||||
|
token_env=token_env,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
clean_id = _normalize_id(credential_id)
|
clean_id = _normalize_id(credential_id)
|
||||||
if session.get(FileConnectorCredential, clean_id) is not None:
|
if session.get(FileConnectorCredential, clean_id) is not None:
|
||||||
raise FileStorageError(f"Connector credential already exists: {clean_id}")
|
raise FileStorageError(f"Connector credential already exists: {clean_id}")
|
||||||
@@ -231,6 +239,11 @@ def update_connector_credential_row(
|
|||||||
clear_password: bool = False,
|
clear_password: bool = False,
|
||||||
clear_token: bool = False,
|
clear_token: bool = False,
|
||||||
) -> FileConnectorCredential:
|
) -> FileConnectorCredential:
|
||||||
|
reject_api_controlled_deployment_references(
|
||||||
|
password_env=password_env,
|
||||||
|
token_env=token_env,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
if label is not None:
|
if label is not None:
|
||||||
row.label = _normalize_label(label)
|
row.label = _normalize_label(label)
|
||||||
if provider is not None:
|
if provider is not None:
|
||||||
|
|||||||
199
src/govoplan_files/backend/storage/connector_deployment.py
Normal file
199
src/govoplan_files/backend/storage/connector_deployment.py
Normal file
@@ -0,0 +1,199 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
_SECRET_ENV_ALLOWLIST = "GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST" # noqa: S105 # nosec B105 - configuration key.
|
||||||
|
_CA_BUNDLE_ALLOWLIST = "GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST"
|
||||||
|
_ENV_NAME = re.compile(r"[A-Za-z_][A-Za-z0-9_]*\Z")
|
||||||
|
_SECRET_ENV_METADATA_KEYS = frozenset(
|
||||||
|
{
|
||||||
|
"access_key_id_env",
|
||||||
|
"secret_access_key_env",
|
||||||
|
"session_token_env",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
_DEVELOPMENT_ENVIRONMENTS = frozenset({"dev", "development", "local", "test", "testing"})
|
||||||
|
|
||||||
|
|
||||||
|
class ConnectorDeploymentConfigurationError(ValueError):
|
||||||
|
"""Raised when connector data crosses a deployment-owned trust boundary."""
|
||||||
|
|
||||||
|
|
||||||
|
def connector_secret_env_value(name: str, *, source_kind: str) -> str:
|
||||||
|
clean_name = _validate_secret_env_reference(name, source_kind=source_kind)
|
||||||
|
value = os.environ.get(clean_name)
|
||||||
|
if value is None or value == "":
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector credential environment variable is not configured")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def connector_secret_env_available(name: str | None, *, source_kind: str) -> bool:
|
||||||
|
if not name:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
clean_name = _validate_secret_env_reference(name, source_kind=source_kind)
|
||||||
|
except ConnectorDeploymentConfigurationError:
|
||||||
|
return False
|
||||||
|
return bool(os.environ.get(clean_name))
|
||||||
|
|
||||||
|
|
||||||
|
def validate_deployment_connector_references(
|
||||||
|
*,
|
||||||
|
source_kind: str,
|
||||||
|
password_env: str | None = None,
|
||||||
|
token_env: str | None = None,
|
||||||
|
metadata: Mapping[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Validate references in deployment-owned connector configuration."""
|
||||||
|
|
||||||
|
for name in (password_env, token_env):
|
||||||
|
if name:
|
||||||
|
_validate_secret_env_reference(name, source_kind=source_kind)
|
||||||
|
for key in _SECRET_ENV_METADATA_KEYS:
|
||||||
|
name = _clean((metadata or {}).get(key))
|
||||||
|
if name:
|
||||||
|
_validate_secret_env_reference(name, source_kind=source_kind)
|
||||||
|
validate_connector_tls_metadata(metadata)
|
||||||
|
|
||||||
|
|
||||||
|
def reject_api_controlled_deployment_references(
|
||||||
|
*,
|
||||||
|
password_env: str | None = None,
|
||||||
|
token_env: str | None = None,
|
||||||
|
metadata: Mapping[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Reject process-secret selectors controlled through tenant-facing APIs."""
|
||||||
|
|
||||||
|
if _clean(password_env) or _clean(token_env):
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
"Environment-backed credentials may only be declared in deployment-owned connector configuration"
|
||||||
|
)
|
||||||
|
for key, value in (metadata or {}).items():
|
||||||
|
if _clean(value) and (str(key).strip().casefold() in _SECRET_ENV_METADATA_KEYS or str(key).strip().casefold().endswith("_env")):
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
"Environment-backed credentials may only be declared in deployment-owned connector configuration"
|
||||||
|
)
|
||||||
|
validate_connector_tls_metadata(metadata)
|
||||||
|
|
||||||
|
|
||||||
|
def connector_ca_bundle_path(value: str) -> str:
|
||||||
|
clean_value = _clean(value)
|
||||||
|
if not clean_value:
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector CA bundle path is empty")
|
||||||
|
candidate = Path(clean_value)
|
||||||
|
if not candidate.is_absolute():
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector CA bundle paths must be absolute")
|
||||||
|
try:
|
||||||
|
resolved = candidate.resolve(strict=True)
|
||||||
|
except OSError as exc:
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector CA bundle path does not exist") from exc
|
||||||
|
allowed = _allowed_ca_bundle_paths()
|
||||||
|
if resolved not in allowed:
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
f"Connector CA bundle path is not listed in {_CA_BUNDLE_ALLOWLIST}"
|
||||||
|
)
|
||||||
|
if not resolved.is_file():
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector CA bundle path must be a regular file")
|
||||||
|
return str(resolved)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_connector_tls_metadata(metadata: Mapping[str, Any] | None) -> None:
|
||||||
|
values = metadata or {}
|
||||||
|
ca_bundle = _clean(values.get("ca_bundle"))
|
||||||
|
if ca_bundle:
|
||||||
|
connector_ca_bundle_path(ca_bundle)
|
||||||
|
for key in ("verify_tls", "tls_verify"):
|
||||||
|
if key in values and not _as_bool(values.get(key), default=True) and not _development_runtime():
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
"Connector TLS certificate verification may only be disabled in dev/test environments"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def connector_effective_endpoint_url(
|
||||||
|
*,
|
||||||
|
provider: str | None,
|
||||||
|
endpoint_url: str | None,
|
||||||
|
metadata: Mapping[str, Any] | None,
|
||||||
|
) -> str | None:
|
||||||
|
"""Return the endpoint that connector I/O will actually use."""
|
||||||
|
|
||||||
|
clean_provider = (provider or "").strip().casefold()
|
||||||
|
values = metadata or {}
|
||||||
|
webdav_url = _clean(values.get("webdav_endpoint_url"))
|
||||||
|
browse_protocol = (_clean(values.get("browse_protocol")) or "").casefold()
|
||||||
|
if webdav_url and (clean_provider in {"seafile", "webdav", "nextcloud"} or browse_protocol == "webdav"):
|
||||||
|
return webdav_url
|
||||||
|
return _clean(endpoint_url)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_secret_env_reference(name: str, *, source_kind: str) -> str:
|
||||||
|
if source_kind.strip().casefold() != "settings":
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
"Environment-backed credentials may only be used by deployment-owned connector configuration"
|
||||||
|
)
|
||||||
|
clean_name = name.strip()
|
||||||
|
if not _ENV_NAME.fullmatch(clean_name):
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector credential environment variable name is invalid")
|
||||||
|
if clean_name not in _allowed_secret_env_names():
|
||||||
|
raise ConnectorDeploymentConfigurationError(
|
||||||
|
f"Connector credential environment variable is not listed in {_SECRET_ENV_ALLOWLIST}"
|
||||||
|
)
|
||||||
|
return clean_name
|
||||||
|
|
||||||
|
|
||||||
|
def _allowed_secret_env_names() -> frozenset[str]:
|
||||||
|
raw = os.environ.get(_SECRET_ENV_ALLOWLIST, "")
|
||||||
|
names = frozenset(item.strip() for item in raw.split(",") if item.strip())
|
||||||
|
invalid = sorted(name for name in names if not _ENV_NAME.fullmatch(name))
|
||||||
|
if invalid:
|
||||||
|
raise ConnectorDeploymentConfigurationError(f"{_SECRET_ENV_ALLOWLIST} contains an invalid variable name")
|
||||||
|
return names
|
||||||
|
|
||||||
|
|
||||||
|
def _allowed_ca_bundle_paths() -> frozenset[Path]:
|
||||||
|
raw = os.environ.get(_CA_BUNDLE_ALLOWLIST, "")
|
||||||
|
paths: set[Path] = set()
|
||||||
|
for item in (part.strip() for part in raw.split(",")):
|
||||||
|
if not item:
|
||||||
|
continue
|
||||||
|
path = Path(item)
|
||||||
|
if not path.is_absolute():
|
||||||
|
raise ConnectorDeploymentConfigurationError(f"{_CA_BUNDLE_ALLOWLIST} requires absolute paths")
|
||||||
|
try:
|
||||||
|
paths.add(path.resolve(strict=True))
|
||||||
|
except OSError as exc:
|
||||||
|
raise ConnectorDeploymentConfigurationError(f"{_CA_BUNDLE_ALLOWLIST} contains a missing path") from exc
|
||||||
|
return frozenset(paths)
|
||||||
|
|
||||||
|
|
||||||
|
def _development_runtime() -> bool:
|
||||||
|
app_env = os.environ.get("APP_ENV", "dev").strip().casefold()
|
||||||
|
install_profile = os.environ.get("GOVOPLAN_INSTALL_PROFILE", "").strip().casefold()
|
||||||
|
return app_env in _DEVELOPMENT_ENVIRONMENTS and (
|
||||||
|
not install_profile or install_profile in _DEVELOPMENT_ENVIRONMENTS
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_bool(value: object, *, default: bool) -> bool:
|
||||||
|
if value is None:
|
||||||
|
return default
|
||||||
|
if isinstance(value, bool):
|
||||||
|
return value
|
||||||
|
clean = str(value).strip().casefold()
|
||||||
|
if clean in {"1", "true", "yes", "on"}:
|
||||||
|
return True
|
||||||
|
if clean in {"0", "false", "no", "off"}:
|
||||||
|
return False
|
||||||
|
raise ConnectorDeploymentConfigurationError("Connector TLS verification setting must be true or false")
|
||||||
|
|
||||||
|
|
||||||
|
def _clean(value: object) -> str | None:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
clean = str(value).strip()
|
||||||
|
return clean or None
|
||||||
@@ -4,7 +4,7 @@ from dataclasses import dataclass, field
|
|||||||
import mimetypes
|
import mimetypes
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
import httpx
|
from govoplan_core.security.outbound_http import response_limit
|
||||||
|
|
||||||
from govoplan_files.backend.storage.connector_browse import (
|
from govoplan_files.backend.storage.connector_browse import (
|
||||||
ConnectorBrowseError,
|
ConnectorBrowseError,
|
||||||
@@ -30,6 +30,7 @@ from govoplan_files.backend.storage.connector_browse import (
|
|||||||
normalize_connector_browse_path,
|
normalize_connector_browse_path,
|
||||||
)
|
)
|
||||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||||
|
from govoplan_files.backend.storage.http_client import ConnectorHttpError, request_connector_bytes
|
||||||
from govoplan_files.backend.storage.paths import filename_from_path
|
from govoplan_files.backend.storage.paths import filename_from_path
|
||||||
|
|
||||||
|
|
||||||
@@ -59,6 +60,7 @@ def read_connector_file(
|
|||||||
path: str,
|
path: str,
|
||||||
max_bytes: int,
|
max_bytes: int,
|
||||||
) -> ConnectorDownloadedFile:
|
) -> ConnectorDownloadedFile:
|
||||||
|
max_bytes = min(max_bytes, response_limit("file"))
|
||||||
if profile.provider == "seafile":
|
if profile.provider == "seafile":
|
||||||
if _metadata_string(profile, "browse_protocol") == "webdav" or _metadata_string(profile, "webdav_endpoint_url"):
|
if _metadata_string(profile, "browse_protocol") == "webdav" or _metadata_string(profile, "webdav_endpoint_url"):
|
||||||
return _read_webdav_file(profile, path=path, max_bytes=max_bytes)
|
return _read_webdav_file(profile, path=path, max_bytes=max_bytes)
|
||||||
@@ -102,8 +104,15 @@ def _read_seafile_file(profile: ConnectorProfile, *, library_id: str, path: str,
|
|||||||
if not isinstance(download_url, str) or not download_url.strip():
|
if not isinstance(download_url, str) or not download_url.strip():
|
||||||
raise ConnectorImportError("Seafile did not return a file download URL")
|
raise ConnectorImportError("Seafile did not return a file download URL")
|
||||||
try:
|
try:
|
||||||
response = httpx.request("GET", download_url, timeout=30.0)
|
response = request_connector_bytes(
|
||||||
except httpx.HTTPError as exc:
|
"GET",
|
||||||
|
download_url,
|
||||||
|
timeout=30.0,
|
||||||
|
kind="file",
|
||||||
|
max_bytes=max_bytes,
|
||||||
|
label="Seafile file download",
|
||||||
|
)
|
||||||
|
except ConnectorHttpError as exc:
|
||||||
raise ConnectorImportError(f"Seafile file download failed: {exc}") from exc
|
raise ConnectorImportError(f"Seafile file download failed: {exc}") from exc
|
||||||
if response.status_code != 200:
|
if response.status_code != 200:
|
||||||
raise ConnectorImportError(f"Seafile file download failed with HTTP {response.status_code}")
|
raise ConnectorImportError(f"Seafile file download failed with HTTP {response.status_code}")
|
||||||
@@ -149,8 +158,17 @@ def _read_webdav_file(profile: ConnectorProfile, *, path: str, max_bytes: int) -
|
|||||||
elif profile.credential_mode.casefold() not in {"", "none", "anonymous"} and profile.secret_ref:
|
elif profile.credential_mode.casefold() not in {"", "none", "anonymous"} and profile.secret_ref:
|
||||||
raise ConnectorImportError("Secret-ref WebDAV credentials need a runtime secret resolver before live import")
|
raise ConnectorImportError("Secret-ref WebDAV credentials need a runtime secret resolver before live import")
|
||||||
try:
|
try:
|
||||||
response = httpx.request("GET", url, headers=headers, auth=auth, timeout=30.0)
|
response = request_connector_bytes(
|
||||||
except httpx.HTTPError as exc:
|
"GET",
|
||||||
|
url,
|
||||||
|
headers=headers,
|
||||||
|
auth=auth,
|
||||||
|
timeout=30.0,
|
||||||
|
kind="file",
|
||||||
|
max_bytes=max_bytes,
|
||||||
|
label="WebDAV file download",
|
||||||
|
)
|
||||||
|
except ConnectorHttpError as exc:
|
||||||
raise ConnectorImportError(f"WebDAV file download failed: {exc}") from exc
|
raise ConnectorImportError(f"WebDAV file download failed: {exc}") from exc
|
||||||
if response.status_code in {401, 403}:
|
if response.status_code in {401, 403}:
|
||||||
raise ConnectorImportError("Connector credentials were rejected")
|
raise ConnectorImportError("Connector credentials were rejected")
|
||||||
@@ -218,19 +236,16 @@ def _read_smb_file(profile: ConnectorProfile, *, path: str, max_bytes: int) -> C
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _read_s3_file(profile: ConnectorProfile, *, library_id: str, path: str, max_bytes: int) -> ConnectorDownloadedFile:
|
def _s3_import_client(profile: ConnectorProfile) -> Any:
|
||||||
bucket = _s3_bucket(profile, library_id)
|
|
||||||
if not bucket:
|
|
||||||
raise ConnectorImportError("S3 import requires a bucket in library_id or profile metadata")
|
|
||||||
key = _s3_object_key(profile, path)
|
|
||||||
if not key:
|
|
||||||
raise ConnectorImportError("S3 import requires an object key")
|
|
||||||
try:
|
try:
|
||||||
client = _s3_client(profile)
|
return _s3_client(profile)
|
||||||
except ConnectorBrowseUnsupported as exc:
|
except ConnectorBrowseUnsupported as exc:
|
||||||
raise ConnectorImportUnsupported(str(exc)) from exc
|
raise ConnectorImportUnsupported(str(exc)) from exc
|
||||||
except ConnectorBrowseError as exc:
|
except ConnectorBrowseError as exc:
|
||||||
raise ConnectorImportError(str(exc)) from exc
|
raise ConnectorImportError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _s3_object_detail(client: Any, *, bucket: str, key: str, max_bytes: int) -> dict[str, Any]:
|
||||||
try:
|
try:
|
||||||
detail = client.head_object(Bucket=bucket, Key=key)
|
detail = client.head_object(Bucket=bucket, Key=key)
|
||||||
except Exception as exc: # pragma: no cover - concrete exception types are dependency-version specific
|
except Exception as exc: # pragma: no cover - concrete exception types are dependency-version specific
|
||||||
@@ -240,7 +255,17 @@ def _read_s3_file(profile: ConnectorProfile, *, library_id: str, path: str, max_
|
|||||||
size = _int(detail.get("ContentLength"))
|
size = _int(detail.get("ContentLength"))
|
||||||
if size is not None and size > max_bytes:
|
if size is not None and size > max_bytes:
|
||||||
raise ConnectorImportError(f"S3 object exceeds limit of {max_bytes} bytes")
|
raise ConnectorImportError(f"S3 object exceeds limit of {max_bytes} bytes")
|
||||||
version_id = _clean(detail.get("VersionId"))
|
return detail
|
||||||
|
|
||||||
|
|
||||||
|
def _download_s3_object(
|
||||||
|
client: Any,
|
||||||
|
*,
|
||||||
|
bucket: str,
|
||||||
|
key: str,
|
||||||
|
version_id: str | None,
|
||||||
|
max_bytes: int,
|
||||||
|
) -> tuple[Any, bytes]:
|
||||||
request: dict[str, object] = {"Bucket": bucket, "Key": key}
|
request: dict[str, object] = {"Bucket": bucket, "Key": key}
|
||||||
if version_id:
|
if version_id:
|
||||||
request["VersionId"] = version_id
|
request["VersionId"] = version_id
|
||||||
@@ -252,6 +277,52 @@ def _read_s3_file(profile: ConnectorProfile, *, library_id: str, path: str, max_
|
|||||||
raise ConnectorImportError(f"S3 object download failed: {exc}") from exc
|
raise ConnectorImportError(f"S3 object download failed: {exc}") from exc
|
||||||
if len(data) > max_bytes:
|
if len(data) > max_bytes:
|
||||||
raise ConnectorImportError(f"S3 object exceeds limit of {max_bytes} bytes")
|
raise ConnectorImportError(f"S3 object exceeds limit of {max_bytes} bytes")
|
||||||
|
return response, data
|
||||||
|
|
||||||
|
|
||||||
|
def _s3_download_metadata(
|
||||||
|
detail: dict[str, Any],
|
||||||
|
*,
|
||||||
|
bucket: str,
|
||||||
|
key: str,
|
||||||
|
version_id: str | None,
|
||||||
|
etag: str | None,
|
||||||
|
size: int,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
metadata: dict[str, Any] = {
|
||||||
|
"bucket": bucket,
|
||||||
|
"key": key,
|
||||||
|
"version_id": version_id,
|
||||||
|
"etag": etag,
|
||||||
|
"size": size,
|
||||||
|
}
|
||||||
|
for source_key, target_key in (
|
||||||
|
("ChecksumSHA256", "checksum_sha256"),
|
||||||
|
("ChecksumCRC32", "checksum_crc32"),
|
||||||
|
("StorageClass", "storage_class"),
|
||||||
|
):
|
||||||
|
if source_key in detail:
|
||||||
|
metadata[target_key] = detail[source_key]
|
||||||
|
return metadata
|
||||||
|
|
||||||
|
|
||||||
|
def _read_s3_file(profile: ConnectorProfile, *, library_id: str, path: str, max_bytes: int) -> ConnectorDownloadedFile:
|
||||||
|
bucket = _s3_bucket(profile, library_id)
|
||||||
|
if not bucket:
|
||||||
|
raise ConnectorImportError("S3 import requires a bucket in library_id or profile metadata")
|
||||||
|
key = _s3_object_key(profile, path)
|
||||||
|
if not key:
|
||||||
|
raise ConnectorImportError("S3 import requires an object key")
|
||||||
|
client = _s3_import_client(profile)
|
||||||
|
detail = _s3_object_detail(client, bucket=bucket, key=key, max_bytes=max_bytes)
|
||||||
|
version_id = _clean(detail.get("VersionId"))
|
||||||
|
response, data = _download_s3_object(
|
||||||
|
client,
|
||||||
|
bucket=bucket,
|
||||||
|
key=key,
|
||||||
|
version_id=version_id,
|
||||||
|
max_bytes=max_bytes,
|
||||||
|
)
|
||||||
content_type = _clean(response.get("ContentType") if isinstance(response, dict) else None) or _clean(detail.get("ContentType")) or mimetypes.guess_type(key)[0]
|
content_type = _clean(response.get("ContentType") if isinstance(response, dict) else None) or _clean(detail.get("ContentType")) or mimetypes.guess_type(key)[0]
|
||||||
etag = _clean(response.get("ETag") if isinstance(response, dict) else None) or _clean(detail.get("ETag"))
|
etag = _clean(response.get("ETag") if isinstance(response, dict) else None) or _clean(detail.get("ETag"))
|
||||||
filename = filename_from_path(key)
|
filename = filename_from_path(key)
|
||||||
@@ -262,16 +333,14 @@ def _read_s3_file(profile: ConnectorProfile, *, library_id: str, path: str, max_
|
|||||||
revision=version_id or etag or _clean(detail.get("LastModified")),
|
revision=version_id or etag or _clean(detail.get("LastModified")),
|
||||||
external_id=f"{bucket}:{key}",
|
external_id=f"{bucket}:{key}",
|
||||||
external_url=f"s3://{bucket}/{key}",
|
external_url=f"s3://{bucket}/{key}",
|
||||||
metadata={
|
metadata=_s3_download_metadata(
|
||||||
"bucket": bucket,
|
detail,
|
||||||
"key": key,
|
bucket=bucket,
|
||||||
"version_id": version_id,
|
key=key,
|
||||||
"etag": etag,
|
version_id=version_id,
|
||||||
"size": len(data),
|
etag=etag,
|
||||||
**({"checksum_sha256": detail["ChecksumSHA256"]} if "ChecksumSHA256" in detail else {}),
|
size=len(data),
|
||||||
**({"checksum_crc32": detail["ChecksumCRC32"]} if "ChecksumCRC32" in detail else {}),
|
),
|
||||||
**({"storage_class": detail["StorageClass"]} if "StorageClass" in detail else {}),
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from govoplan_core.core.policy import normalize_policy_scope_type
|
|||||||
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
from govoplan_core.security.secrets import decrypt_secret, encrypt_secret
|
||||||
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
from govoplan_files.backend.db.models import FileConnectorCredential, FileConnectorProfile
|
||||||
from govoplan_files.backend.storage.common import FileStorageError
|
from govoplan_files.backend.storage.common import FileStorageError
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import reject_api_controlled_deployment_references
|
||||||
from govoplan_files.backend.storage.connector_credential_store import connector_credential_from_row, credential_rows_by_id
|
from govoplan_files.backend.storage.connector_credential_store import connector_credential_from_row, credential_rows_by_id
|
||||||
from govoplan_files.backend.storage.connector_policy import connector_policy_sources_from_payload
|
from govoplan_files.backend.storage.connector_policy import connector_policy_sources_from_payload
|
||||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, supported_connector_providers
|
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, supported_connector_providers
|
||||||
@@ -124,6 +125,11 @@ def create_connector_profile_row(
|
|||||||
policy: Mapping[str, Any] | None = None,
|
policy: Mapping[str, Any] | None = None,
|
||||||
metadata: Mapping[str, Any] | None = None,
|
metadata: Mapping[str, Any] | None = None,
|
||||||
) -> FileConnectorProfile:
|
) -> FileConnectorProfile:
|
||||||
|
reject_api_controlled_deployment_references(
|
||||||
|
password_env=password_env,
|
||||||
|
token_env=token_env,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
clean_id = _normalize_profile_id(profile_id)
|
clean_id = _normalize_profile_id(profile_id)
|
||||||
if session.get(FileConnectorProfile, clean_id) is not None:
|
if session.get(FileConnectorProfile, clean_id) is not None:
|
||||||
raise FileStorageError(f"Connector profile already exists: {clean_id}")
|
raise FileStorageError(f"Connector profile already exists: {clean_id}")
|
||||||
@@ -181,6 +187,11 @@ def update_connector_profile_row(
|
|||||||
clear_password: bool = False,
|
clear_password: bool = False,
|
||||||
clear_token: bool = False,
|
clear_token: bool = False,
|
||||||
) -> FileConnectorProfile:
|
) -> FileConnectorProfile:
|
||||||
|
reject_api_controlled_deployment_references(
|
||||||
|
password_env=password_env,
|
||||||
|
token_env=token_env,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
if label is not None:
|
if label is not None:
|
||||||
row.label = _normalize_label(label)
|
row.label = _normalize_label(label)
|
||||||
if provider is not None:
|
if provider is not None:
|
||||||
|
|||||||
@@ -7,6 +7,10 @@ from dataclasses import dataclass, field
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from govoplan_core.core.policy import normalize_policy_scope_type, policy_source_path
|
from govoplan_core.core.policy import normalize_policy_scope_type, policy_source_path
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import (
|
||||||
|
connector_secret_env_available,
|
||||||
|
validate_deployment_connector_references,
|
||||||
|
)
|
||||||
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
from govoplan_files.backend.storage.connector_policy import ConnectorPolicySource, connector_policy_sources_from_payload
|
||||||
|
|
||||||
|
|
||||||
@@ -82,9 +86,9 @@ class ConnectorProfile:
|
|||||||
if self.secret_ref or self.has_inline_secret or self.password_value or self.token_value:
|
if self.secret_ref or self.has_inline_secret or self.password_value or self.token_value:
|
||||||
return True
|
return True
|
||||||
if self.token_env:
|
if self.token_env:
|
||||||
return bool(os.environ.get(self.token_env))
|
return connector_secret_env_available(self.token_env, source_kind=self.source_kind)
|
||||||
if self.password_env:
|
if self.password_env:
|
||||||
return bool(os.environ.get(self.password_env))
|
return connector_secret_env_available(self.password_env, source_kind=self.source_kind)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def to_response(self) -> dict[str, Any]:
|
def to_response(self) -> dict[str, Any]:
|
||||||
@@ -106,7 +110,7 @@ class ConnectorProfile:
|
|||||||
"username": self.username,
|
"username": self.username,
|
||||||
"capabilities": list(self.capabilities),
|
"capabilities": list(self.capabilities),
|
||||||
"policy_sources": [_policy_source_response(source) for source in self.policy_sources],
|
"policy_sources": [_policy_source_response(source) for source in self.policy_sources],
|
||||||
"metadata": dict(self.metadata),
|
"metadata": _response_metadata(self.metadata),
|
||||||
"source_kind": self.source_kind,
|
"source_kind": self.source_kind,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -146,7 +150,7 @@ def _profile_from_mapping(value: Mapping[str, Any]) -> ConnectorProfile:
|
|||||||
credential_mode=mode,
|
credential_mode=mode,
|
||||||
credentials=credentials,
|
credentials=credentials,
|
||||||
)
|
)
|
||||||
return ConnectorProfile(
|
result = ConnectorProfile(
|
||||||
id=profile.id,
|
id=profile.id,
|
||||||
label=profile.label,
|
label=profile.label,
|
||||||
provider=profile.provider,
|
provider=profile.provider,
|
||||||
@@ -170,6 +174,13 @@ def _profile_from_mapping(value: Mapping[str, Any]) -> ConnectorProfile:
|
|||||||
metadata=profile.metadata,
|
metadata=profile.metadata,
|
||||||
source_kind="settings",
|
source_kind="settings",
|
||||||
)
|
)
|
||||||
|
validate_deployment_connector_references(
|
||||||
|
source_kind=result.source_kind,
|
||||||
|
password_env=result.password_env,
|
||||||
|
token_env=result.token_env,
|
||||||
|
metadata=result.metadata,
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
def _profile_id_from_mapping(value: Mapping[str, Any]) -> str:
|
def _profile_id_from_mapping(value: Mapping[str, Any]) -> str:
|
||||||
@@ -291,6 +302,16 @@ def _public_metadata(value: object) -> Mapping[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _response_metadata(value: Mapping[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
str(key): item
|
||||||
|
for key, item in value.items()
|
||||||
|
if str(key).strip().casefold() not in _INLINE_SECRET_FIELDS
|
||||||
|
and not str(key).strip().casefold().endswith("_env")
|
||||||
|
and str(key).strip().casefold() != "ca_bundle"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _string_list(value: object) -> list[str]:
|
def _string_list(value: object) -> list[str]:
|
||||||
if value is None:
|
if value is None:
|
||||||
return []
|
return []
|
||||||
|
|||||||
239
src/govoplan_files/backend/storage/http_client.py
Normal file
239
src/govoplan_files/backend/storage/http_client.py
Normal file
@@ -0,0 +1,239 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import socket
|
||||||
|
import ssl
|
||||||
|
import select
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Iterator
|
||||||
|
|
||||||
|
import httpcore
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from govoplan_core.security.outbound_http import (
|
||||||
|
OutboundHttpError,
|
||||||
|
bounded_chunks_bytes,
|
||||||
|
create_outbound_connection,
|
||||||
|
response_limit,
|
||||||
|
validate_outbound_http_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ConnectorHttpError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ConnectorHttpResponse:
|
||||||
|
status_code: int
|
||||||
|
headers: Mapping[str, str]
|
||||||
|
content: bytes
|
||||||
|
|
||||||
|
|
||||||
|
_HTTPCORE_TRANSPORT_ERRORS = (
|
||||||
|
httpcore.TimeoutException,
|
||||||
|
httpcore.NetworkError,
|
||||||
|
httpcore.ProtocolError,
|
||||||
|
httpcore.ProxyError,
|
||||||
|
httpcore.UnsupportedProtocol,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _SocketNetworkStream(httpcore.NetworkStream):
|
||||||
|
"""Public httpcore NetworkStream adapter around an approved socket."""
|
||||||
|
|
||||||
|
def __init__(self, sock: socket.socket) -> None:
|
||||||
|
self._socket = sock
|
||||||
|
|
||||||
|
def read(self, max_bytes: int, timeout: float | None = None) -> bytes:
|
||||||
|
try:
|
||||||
|
self._socket.settimeout(timeout)
|
||||||
|
return self._socket.recv(max_bytes)
|
||||||
|
except socket.timeout as exc:
|
||||||
|
raise httpcore.ReadTimeout(str(exc)) from exc
|
||||||
|
except OSError as exc:
|
||||||
|
raise httpcore.ReadError(str(exc)) from exc
|
||||||
|
|
||||||
|
def write(self, buffer: bytes, timeout: float | None = None) -> None:
|
||||||
|
try:
|
||||||
|
self._socket.settimeout(timeout)
|
||||||
|
self._socket.sendall(buffer)
|
||||||
|
except socket.timeout as exc:
|
||||||
|
raise httpcore.WriteTimeout(str(exc)) from exc
|
||||||
|
except OSError as exc:
|
||||||
|
raise httpcore.WriteError(str(exc)) from exc
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._socket.close()
|
||||||
|
|
||||||
|
def start_tls(
|
||||||
|
self,
|
||||||
|
ssl_context: ssl.SSLContext,
|
||||||
|
server_hostname: str | None = None,
|
||||||
|
timeout: float | None = None,
|
||||||
|
) -> httpcore.NetworkStream:
|
||||||
|
try:
|
||||||
|
self._socket.settimeout(timeout)
|
||||||
|
tls_socket = ssl_context.wrap_socket(self._socket, server_hostname=server_hostname)
|
||||||
|
except socket.timeout as exc:
|
||||||
|
self.close()
|
||||||
|
raise httpcore.ConnectTimeout(str(exc)) from exc
|
||||||
|
except OSError as exc:
|
||||||
|
self.close()
|
||||||
|
raise httpcore.ConnectError(str(exc)) from exc
|
||||||
|
return _SocketNetworkStream(tls_socket)
|
||||||
|
|
||||||
|
def get_extra_info(self, info: str) -> Any:
|
||||||
|
if info == "ssl_object" and isinstance(self._socket, ssl.SSLSocket):
|
||||||
|
return self._socket
|
||||||
|
if info == "client_addr":
|
||||||
|
return self._socket.getsockname()
|
||||||
|
if info == "server_addr":
|
||||||
|
return self._socket.getpeername()
|
||||||
|
if info == "socket":
|
||||||
|
return self._socket
|
||||||
|
if info == "is_readable":
|
||||||
|
try:
|
||||||
|
return bool(select.select([self._socket], [], [], 0)[0])
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return True
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class _OutboundPolicyNetworkBackend(httpcore.NetworkBackend):
|
||||||
|
def connect_tcp(
|
||||||
|
self,
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
timeout: float | None = None,
|
||||||
|
local_address: str | None = None,
|
||||||
|
socket_options: Any = None,
|
||||||
|
) -> httpcore.NetworkStream:
|
||||||
|
source_address = None if local_address is None else (local_address, 0)
|
||||||
|
try:
|
||||||
|
sock = create_outbound_connection(
|
||||||
|
host,
|
||||||
|
port,
|
||||||
|
timeout=timeout,
|
||||||
|
source_address=source_address,
|
||||||
|
socket_options=socket_options,
|
||||||
|
label="File connector HTTP endpoint",
|
||||||
|
)
|
||||||
|
except socket.timeout as exc:
|
||||||
|
raise httpcore.ConnectTimeout(str(exc)) from exc
|
||||||
|
except (OSError, OutboundHttpError) as exc:
|
||||||
|
raise httpcore.ConnectError(str(exc)) from exc
|
||||||
|
return _SocketNetworkStream(sock)
|
||||||
|
|
||||||
|
def connect_unix_socket(self, path: str, timeout: float | None = None, socket_options: Any = None): # type: ignore[no-untyped-def]
|
||||||
|
del path, timeout, socket_options
|
||||||
|
raise httpcore.ConnectError("Unix sockets are not supported for file connectors")
|
||||||
|
|
||||||
|
|
||||||
|
class _HttpcoreResponseStream(httpx.SyncByteStream):
|
||||||
|
def __init__(self, stream: Any, *, request: httpx.Request) -> None:
|
||||||
|
self._stream = stream
|
||||||
|
self._request = request
|
||||||
|
|
||||||
|
def __iter__(self): # type: ignore[no-untyped-def]
|
||||||
|
try:
|
||||||
|
yield from self._stream
|
||||||
|
except _HTTPCORE_TRANSPORT_ERRORS as exc:
|
||||||
|
raise httpx.TransportError(str(exc), request=self._request) from exc
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
if hasattr(self._stream, "close"):
|
||||||
|
self._stream.close()
|
||||||
|
|
||||||
|
|
||||||
|
class _OutboundPolicyHTTPTransport(httpx.BaseTransport):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._connection_pool = httpcore.ConnectionPool(
|
||||||
|
ssl_context=httpx.create_ssl_context(verify=True, trust_env=False),
|
||||||
|
max_connections=100,
|
||||||
|
max_keepalive_connections=20,
|
||||||
|
keepalive_expiry=5.0,
|
||||||
|
network_backend=_OutboundPolicyNetworkBackend(),
|
||||||
|
)
|
||||||
|
|
||||||
|
def handle_request(self, request: httpx.Request) -> httpx.Response:
|
||||||
|
core_request = httpcore.Request(
|
||||||
|
method=request.method,
|
||||||
|
url=httpcore.URL(
|
||||||
|
scheme=request.url.raw_scheme,
|
||||||
|
host=request.url.raw_host,
|
||||||
|
port=request.url.port,
|
||||||
|
target=request.url.raw_path,
|
||||||
|
),
|
||||||
|
headers=request.headers.raw,
|
||||||
|
content=request.stream,
|
||||||
|
extensions=request.extensions,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
response = self._connection_pool.handle_request(core_request)
|
||||||
|
except _HTTPCORE_TRANSPORT_ERRORS as exc:
|
||||||
|
raise httpx.TransportError(str(exc), request=request) from exc
|
||||||
|
return httpx.Response(
|
||||||
|
status_code=response.status,
|
||||||
|
headers=response.headers,
|
||||||
|
stream=_HttpcoreResponseStream(response.stream, request=request),
|
||||||
|
extensions=response.extensions,
|
||||||
|
)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._connection_pool.close()
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _stream_connector_request(method: str, url: str, **kwargs: Any) -> Iterator[httpx.Response]:
|
||||||
|
with httpx.Client(
|
||||||
|
transport=_OutboundPolicyHTTPTransport(),
|
||||||
|
follow_redirects=False,
|
||||||
|
timeout=kwargs.pop("timeout", 15.0),
|
||||||
|
) as client:
|
||||||
|
with client.stream(method, url, **kwargs) as response:
|
||||||
|
yield response
|
||||||
|
|
||||||
|
|
||||||
|
def request_connector_bytes(
|
||||||
|
method: str,
|
||||||
|
url: str,
|
||||||
|
*,
|
||||||
|
headers: Mapping[str, str] | None = None,
|
||||||
|
params: Mapping[str, str] | None = None,
|
||||||
|
data: Mapping[str, str] | bytes | str | None = None,
|
||||||
|
content: bytes | str | None = None,
|
||||||
|
auth: Any = None,
|
||||||
|
timeout: float = 15.0,
|
||||||
|
kind: str = "structured",
|
||||||
|
max_bytes: int | None = None,
|
||||||
|
label: str = "File connector",
|
||||||
|
) -> ConnectorHttpResponse:
|
||||||
|
try:
|
||||||
|
validated_url = validate_outbound_http_url(url, label=f"{label} URL")
|
||||||
|
effective_limit = response_limit(kind) if max_bytes is None else min(int(max_bytes), response_limit(kind))
|
||||||
|
with _stream_connector_request(
|
||||||
|
method,
|
||||||
|
validated_url,
|
||||||
|
headers=dict(headers or {}),
|
||||||
|
params=params,
|
||||||
|
data=data,
|
||||||
|
content=content,
|
||||||
|
auth=auth,
|
||||||
|
timeout=timeout,
|
||||||
|
) as response:
|
||||||
|
body = bounded_chunks_bytes(
|
||||||
|
response.iter_bytes(),
|
||||||
|
headers=response.headers,
|
||||||
|
max_bytes=effective_limit,
|
||||||
|
kind=kind,
|
||||||
|
label=f"{label} response",
|
||||||
|
)
|
||||||
|
return ConnectorHttpResponse(
|
||||||
|
status_code=response.status_code,
|
||||||
|
headers=dict(response.headers),
|
||||||
|
content=body,
|
||||||
|
)
|
||||||
|
except (httpx.HTTPError, OutboundHttpError, ValueError) as exc:
|
||||||
|
raise ConnectorHttpError(str(exc)) from exc
|
||||||
196
tests/test_connector_deployment.py
Normal file
196
tests/test_connector_deployment.py
Normal file
@@ -0,0 +1,196 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
from govoplan_files.backend.router import discover_connector_endpoint
|
||||||
|
from govoplan_files.backend.schemas import FileConnectorDiscoveryRequest
|
||||||
|
from govoplan_files.backend.storage.connector_browse import ConnectorBrowseError, _profile_password, _s3_verify
|
||||||
|
from govoplan_files.backend.storage.connector_deployment import (
|
||||||
|
ConnectorDeploymentConfigurationError,
|
||||||
|
connector_effective_endpoint_url,
|
||||||
|
connector_secret_env_value,
|
||||||
|
reject_api_controlled_deployment_references,
|
||||||
|
)
|
||||||
|
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile
|
||||||
|
from govoplan_files.backend.storage.connector_profile_store import create_connector_profile_row
|
||||||
|
|
||||||
|
|
||||||
|
class ConnectorDeploymentBoundaryTests(unittest.TestCase):
|
||||||
|
def test_database_profile_cannot_read_arbitrary_process_environment(self) -> None:
|
||||||
|
profile = ConnectorProfile(
|
||||||
|
id="tenant-webdav",
|
||||||
|
label="Tenant WebDAV",
|
||||||
|
provider="webdav",
|
||||||
|
password_env="MASTER_KEY_B64",
|
||||||
|
source_kind="database",
|
||||||
|
)
|
||||||
|
with patch.dict(
|
||||||
|
os.environ,
|
||||||
|
{
|
||||||
|
"MASTER_KEY_B64": "must-not-leave-process",
|
||||||
|
"GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST": "MASTER_KEY_B64",
|
||||||
|
},
|
||||||
|
clear=False,
|
||||||
|
), self.assertRaisesRegex(ConnectorBrowseError, "deployment-owned"):
|
||||||
|
_profile_password(profile)
|
||||||
|
|
||||||
|
def test_deployment_profile_requires_exact_secret_allowlist(self) -> None:
|
||||||
|
with patch.dict(
|
||||||
|
os.environ,
|
||||||
|
{
|
||||||
|
"GOVOPLAN_FILES_WEBDAV_PASSWORD": "deployment-secret",
|
||||||
|
"GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST": "GOVOPLAN_FILES_WEBDAV_PASSWORD",
|
||||||
|
},
|
||||||
|
clear=False,
|
||||||
|
):
|
||||||
|
self.assertEqual(
|
||||||
|
"deployment-secret",
|
||||||
|
connector_secret_env_value(
|
||||||
|
"GOVOPLAN_FILES_WEBDAV_PASSWORD",
|
||||||
|
source_kind="settings",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(ConnectorDeploymentConfigurationError, "not listed"):
|
||||||
|
connector_secret_env_value("MASTER_KEY_B64", source_kind="settings")
|
||||||
|
|
||||||
|
def test_api_profiles_cannot_select_secret_environment_names(self) -> None:
|
||||||
|
with self.assertRaisesRegex(ConnectorDeploymentConfigurationError, "deployment-owned"):
|
||||||
|
reject_api_controlled_deployment_references(password_env="DATABASE_URL")
|
||||||
|
with self.assertRaisesRegex(ConnectorDeploymentConfigurationError, "deployment-owned"):
|
||||||
|
reject_api_controlled_deployment_references(metadata={"secret_access_key_env": "MASTER_KEY_B64"})
|
||||||
|
|
||||||
|
session = MagicMock()
|
||||||
|
with self.assertRaisesRegex(ConnectorDeploymentConfigurationError, "deployment-owned"):
|
||||||
|
create_connector_profile_row(
|
||||||
|
session,
|
||||||
|
tenant_id="tenant-1",
|
||||||
|
user_id="user-1",
|
||||||
|
profile_id="unsafe",
|
||||||
|
label="Unsafe",
|
||||||
|
provider="webdav",
|
||||||
|
scope_type="tenant",
|
||||||
|
password_env="DATABASE_URL",
|
||||||
|
)
|
||||||
|
session.get.assert_not_called()
|
||||||
|
|
||||||
|
def test_profile_responses_hide_environment_and_local_ca_references(self) -> None:
|
||||||
|
profile = ConnectorProfile(
|
||||||
|
id="deployment-s3",
|
||||||
|
label="Deployment S3",
|
||||||
|
provider="s3",
|
||||||
|
metadata={
|
||||||
|
"bucket": "documents",
|
||||||
|
"secret_access_key_env": "GOVOPLAN_FILES_S3_SECRET",
|
||||||
|
"ca_bundle": "/etc/govoplan/connector-ca.pem",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual({"bucket": "documents"}, profile.to_response()["metadata"])
|
||||||
|
|
||||||
|
def test_ca_bundle_must_be_an_exact_deployment_allowlisted_file(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as temp_dir:
|
||||||
|
allowed = Path(temp_dir, "connector-ca.pem")
|
||||||
|
other = Path(temp_dir, "other-ca.pem")
|
||||||
|
allowed.write_text("test CA", encoding="utf-8")
|
||||||
|
other.write_text("other CA", encoding="utf-8")
|
||||||
|
with patch.dict(
|
||||||
|
os.environ,
|
||||||
|
{
|
||||||
|
"APP_ENV": "production",
|
||||||
|
"GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST": str(allowed),
|
||||||
|
},
|
||||||
|
clear=False,
|
||||||
|
):
|
||||||
|
profile = ConnectorProfile(
|
||||||
|
id="s3",
|
||||||
|
label="S3",
|
||||||
|
provider="s3",
|
||||||
|
metadata={"ca_bundle": str(allowed)},
|
||||||
|
)
|
||||||
|
self.assertEqual(str(allowed.resolve()), _s3_verify(profile))
|
||||||
|
with self.assertRaisesRegex(ConnectorDeploymentConfigurationError, "not listed"):
|
||||||
|
reject_api_controlled_deployment_references(metadata={"ca_bundle": str(other)})
|
||||||
|
|
||||||
|
def test_tls_verification_can_only_be_disabled_in_development(self) -> None:
|
||||||
|
with patch.dict(os.environ, {"APP_ENV": "production"}, clear=False), self.assertRaisesRegex(
|
||||||
|
ConnectorDeploymentConfigurationError,
|
||||||
|
"dev/test",
|
||||||
|
):
|
||||||
|
reject_api_controlled_deployment_references(metadata={"verify_tls": False})
|
||||||
|
with patch.dict(os.environ, {"APP_ENV": "test"}, clear=False):
|
||||||
|
reject_api_controlled_deployment_references(metadata={"verify_tls": False})
|
||||||
|
|
||||||
|
def test_effective_endpoint_uses_webdav_override(self) -> None:
|
||||||
|
self.assertEqual(
|
||||||
|
"https://dav.example.test/root",
|
||||||
|
connector_effective_endpoint_url(
|
||||||
|
provider="seafile",
|
||||||
|
endpoint_url="https://seafile.example.test",
|
||||||
|
metadata={"webdav_endpoint_url": "https://dav.example.test/root"},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ConnectorDiscoveryBoundaryTests(unittest.TestCase):
|
||||||
|
@staticmethod
|
||||||
|
def _principal() -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(tenant_id="tenant-1", user=SimpleNamespace(id="user-1"), api_key=None)
|
||||||
|
|
||||||
|
def test_discovery_rejects_environment_credentials_before_io(self) -> None:
|
||||||
|
payload = FileConnectorDiscoveryRequest(
|
||||||
|
provider="webdav",
|
||||||
|
endpoint_url="https://dav.example.test",
|
||||||
|
credential_mode="basic",
|
||||||
|
credentials={"username": "admin", "password_env": "MASTER_KEY_B64"},
|
||||||
|
)
|
||||||
|
with patch("govoplan_files.backend.router.browse_connector_profile") as browse, self.assertRaises(
|
||||||
|
HTTPException
|
||||||
|
) as raised:
|
||||||
|
discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||||
|
self.assertEqual(400, raised.exception.status_code)
|
||||||
|
browse.assert_not_called()
|
||||||
|
|
||||||
|
def test_discovery_applies_policy_and_audit_before_each_io_candidate(self) -> None:
|
||||||
|
payload = FileConnectorDiscoveryRequest(
|
||||||
|
provider="webdav",
|
||||||
|
endpoint_url="https://dav.example.test/root",
|
||||||
|
metadata={
|
||||||
|
"webdav_endpoint_url": "https://bypass.example.test",
|
||||||
|
"static_listing": {"": []},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
events: list[str] = []
|
||||||
|
|
||||||
|
def ensure(*_args: object, **kwargs: object) -> None:
|
||||||
|
self.assertEqual("https://dav.example.test/root/", kwargs["endpoint_url"])
|
||||||
|
events.append("policy")
|
||||||
|
|
||||||
|
def audit(*_args: object, **_kwargs: object) -> None:
|
||||||
|
events.append("audit")
|
||||||
|
|
||||||
|
def browse(profile: ConnectorProfile, **_kwargs: object) -> list[object]:
|
||||||
|
self.assertEqual({}, profile.metadata)
|
||||||
|
events.append("io")
|
||||||
|
return []
|
||||||
|
|
||||||
|
with patch("govoplan_files.backend.router._ensure_connector_configuration_allowed", side_effect=ensure), patch(
|
||||||
|
"govoplan_files.backend.router._audit_connector_discovery_attempt",
|
||||||
|
side_effect=audit,
|
||||||
|
), patch("govoplan_files.backend.router.browse_connector_profile", side_effect=browse):
|
||||||
|
response = discover_connector_endpoint(payload, session=object(), principal=self._principal()) # type: ignore[arg-type]
|
||||||
|
|
||||||
|
self.assertEqual("usable", response.status)
|
||||||
|
self.assertEqual(["policy", "audit", "io"], events)
|
||||||
|
self.assertEqual({"discovered_by": "webdav-propfind"}, response.metadata)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -4,7 +4,7 @@ import unittest
|
|||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
from govoplan_files.backend.storage.connector_browse import browse_connector_profile
|
from govoplan_files.backend.storage.connector_browse import _smb_location, browse_connector_profile
|
||||||
from govoplan_files.backend.storage.connector_browse import ConnectorBrowseUnsupported
|
from govoplan_files.backend.storage.connector_browse import ConnectorBrowseUnsupported
|
||||||
from govoplan_files.backend.storage.connector_imports import read_connector_file
|
from govoplan_files.backend.storage.connector_imports import read_connector_file
|
||||||
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, connector_profiles_from_payload
|
from govoplan_files.backend.storage.connector_profiles import ConnectorProfile, connector_profiles_from_payload
|
||||||
@@ -76,6 +76,24 @@ def s3_profile(**overrides: object) -> ConnectorProfile:
|
|||||||
|
|
||||||
|
|
||||||
class ConnectorProviderTests(unittest.TestCase):
|
class ConnectorProviderTests(unittest.TestCase):
|
||||||
|
def test_smb_uses_validated_numeric_target_when_private_networks_are_disabled(self) -> None:
|
||||||
|
profile = ConnectorProfile(
|
||||||
|
id="public-smb",
|
||||||
|
label="Public SMB",
|
||||||
|
provider="smb",
|
||||||
|
endpoint_url="smb://files.example.test/share",
|
||||||
|
)
|
||||||
|
with patch.dict(
|
||||||
|
"os.environ",
|
||||||
|
{"APP_ENV": "production", "GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "false"},
|
||||||
|
), patch(
|
||||||
|
"govoplan_core.security.outbound_http.socket.getaddrinfo",
|
||||||
|
return_value=[(2, 1, 6, "", ("93.184.216.34", 445))],
|
||||||
|
):
|
||||||
|
location = _smb_location(profile)
|
||||||
|
|
||||||
|
self.assertEqual("93.184.216.34", location.server)
|
||||||
|
|
||||||
def test_provider_descriptors_include_s3_and_reserved_microsoft_providers(self) -> None:
|
def test_provider_descriptors_include_s3_and_reserved_microsoft_providers(self) -> None:
|
||||||
descriptors = {descriptor.provider: descriptor for descriptor in connector_provider_descriptors()}
|
descriptors = {descriptor.provider: descriptor for descriptor in connector_provider_descriptors()}
|
||||||
|
|
||||||
|
|||||||
126
tests/test_http_client.py
Normal file
126
tests/test_http_client.py
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from threading import Thread
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import httpcore
|
||||||
|
|
||||||
|
from govoplan_core.security.outbound_http import outbound_http_policy, validate_outbound_http_url
|
||||||
|
from govoplan_files.backend.storage.http_client import (
|
||||||
|
ConnectorHttpError,
|
||||||
|
_OutboundPolicyNetworkBackend,
|
||||||
|
_SocketNetworkStream,
|
||||||
|
request_connector_bytes,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _TestHandler(BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self) -> None: # noqa: N802 - stdlib handler contract
|
||||||
|
body = b"connector-ok"
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, _format: str, *_args: object) -> None:
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
class ConnectorHttpClientTests(unittest.TestCase):
|
||||||
|
def test_public_transport_adapter_performs_a_real_http_request(self) -> None:
|
||||||
|
server = ThreadingHTTPServer(("127.0.0.1", 0), _TestHandler)
|
||||||
|
thread = Thread(target=server.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
try:
|
||||||
|
with patch.dict(
|
||||||
|
"os.environ",
|
||||||
|
{"APP_ENV": "test", "GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true"},
|
||||||
|
clear=False,
|
||||||
|
):
|
||||||
|
result = request_connector_bytes(
|
||||||
|
"GET",
|
||||||
|
f"http://127.0.0.1:{server.server_port}/object",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
server.shutdown()
|
||||||
|
server.server_close()
|
||||||
|
thread.join(timeout=2)
|
||||||
|
self.assertEqual(200, result.status_code)
|
||||||
|
self.assertEqual(b"connector-ok", result.content)
|
||||||
|
|
||||||
|
def test_connector_responses_are_streamed_without_redirects(self) -> None:
|
||||||
|
response = MagicMock()
|
||||||
|
response.status_code = 200
|
||||||
|
response.headers = {"content-length": "6"}
|
||||||
|
response.iter_bytes.return_value = iter((b"abc", b"def"))
|
||||||
|
context = MagicMock()
|
||||||
|
context.__enter__.return_value = response
|
||||||
|
|
||||||
|
with patch("govoplan_files.backend.storage.http_client._stream_connector_request", return_value=context):
|
||||||
|
result = request_connector_bytes("GET", "https://example.test/object", max_bytes=10)
|
||||||
|
|
||||||
|
self.assertEqual(b"abcdef", result.content)
|
||||||
|
|
||||||
|
def test_connector_response_limit_is_enforced_while_streaming(self) -> None:
|
||||||
|
response = MagicMock()
|
||||||
|
response.status_code = 200
|
||||||
|
response.headers = {}
|
||||||
|
response.iter_bytes.return_value = iter((b"12345", b"67890", b"!"))
|
||||||
|
context = MagicMock()
|
||||||
|
context.__enter__.return_value = response
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"govoplan_files.backend.storage.http_client._stream_connector_request",
|
||||||
|
return_value=context,
|
||||||
|
), self.assertRaisesRegex(
|
||||||
|
ConnectorHttpError,
|
||||||
|
"configured limit",
|
||||||
|
):
|
||||||
|
request_connector_bytes("GET", "https://example.test/object", max_bytes=10)
|
||||||
|
|
||||||
|
def test_private_destination_is_blocked_before_http_connection(self) -> None:
|
||||||
|
with patch.dict(
|
||||||
|
"os.environ",
|
||||||
|
{"APP_ENV": "production", "GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "false"},
|
||||||
|
), patch(
|
||||||
|
"govoplan_core.security.outbound_http.socket.getaddrinfo",
|
||||||
|
return_value=[(2, 1, 6, "", ("10.0.0.5", 443))],
|
||||||
|
), patch("govoplan_files.backend.storage.http_client._stream_connector_request") as stream, self.assertRaisesRegex(
|
||||||
|
ConnectorHttpError,
|
||||||
|
"non-public network",
|
||||||
|
):
|
||||||
|
request_connector_bytes("GET", "https://connector.example.test/object")
|
||||||
|
stream.assert_not_called()
|
||||||
|
|
||||||
|
def test_connection_backend_rejects_private_rebinding_before_socket_open(self) -> None:
|
||||||
|
policy = outbound_http_policy({"APP_ENV": "production"})
|
||||||
|
public = [(2, 1, 6, "", ("93.184.216.34", 443))]
|
||||||
|
private = [(2, 1, 6, "", ("127.0.0.1", 443))]
|
||||||
|
with patch.dict(
|
||||||
|
"os.environ",
|
||||||
|
{"APP_ENV": "production", "GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "false"},
|
||||||
|
), patch(
|
||||||
|
"govoplan_core.security.outbound_http.socket.getaddrinfo",
|
||||||
|
side_effect=(public, private),
|
||||||
|
), patch("govoplan_core.security.outbound_http.socket.socket") as socket_factory:
|
||||||
|
validate_outbound_http_url("https://connector.example.test/object", policy=policy)
|
||||||
|
with self.assertRaises(httpcore.ConnectError):
|
||||||
|
_OutboundPolicyNetworkBackend().connect_tcp("connector.example.test", 443)
|
||||||
|
socket_factory.assert_not_called()
|
||||||
|
|
||||||
|
def test_network_backend_returns_public_network_stream_adapter(self) -> None:
|
||||||
|
sock = MagicMock()
|
||||||
|
with patch(
|
||||||
|
"govoplan_files.backend.storage.http_client.create_outbound_connection",
|
||||||
|
return_value=sock,
|
||||||
|
):
|
||||||
|
stream = _OutboundPolicyNetworkBackend().connect_tcp("connector.example.test", 443)
|
||||||
|
|
||||||
|
self.assertIsInstance(stream, _SocketNetworkStream)
|
||||||
|
self.assertIs(stream.get_extra_info("socket"), sock)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
64
tests/test_storage_backends.py
Normal file
64
tests/test_storage_backends.py
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import MagicMock, PropertyMock, patch
|
||||||
|
|
||||||
|
from govoplan_files.backend.storage.backends import S3StorageBackend, StorageBackendError
|
||||||
|
|
||||||
|
|
||||||
|
def _backend() -> S3StorageBackend:
|
||||||
|
return S3StorageBackend(
|
||||||
|
bucket="files",
|
||||||
|
endpoint_url="https://objects.example.test",
|
||||||
|
region_name="test",
|
||||||
|
access_key_id="access",
|
||||||
|
secret_access_key="secret",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class S3StorageBackendTests(unittest.TestCase):
|
||||||
|
def test_get_bytes_rejects_declared_oversize_object_without_reading(self) -> None:
|
||||||
|
body = MagicMock()
|
||||||
|
client = MagicMock()
|
||||||
|
client.get_object.return_value = {"ContentLength": 6, "Body": body}
|
||||||
|
with patch.dict("os.environ", {"GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES": "5"}), patch.object(
|
||||||
|
S3StorageBackend,
|
||||||
|
"client",
|
||||||
|
new_callable=PropertyMock,
|
||||||
|
return_value=client,
|
||||||
|
), self.assertRaisesRegex(StorageBackendError, "deployment limit"):
|
||||||
|
_backend().get_bytes("large.bin")
|
||||||
|
body.read.assert_not_called()
|
||||||
|
body.close.assert_called_once_with()
|
||||||
|
|
||||||
|
def test_iter_bytes_rejects_undeclared_oversize_object(self) -> None:
|
||||||
|
client = MagicMock()
|
||||||
|
client.get_object.return_value = {"Body": io.BytesIO(b"123456")}
|
||||||
|
with patch.dict("os.environ", {"GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES": "5"}), patch.object(
|
||||||
|
S3StorageBackend,
|
||||||
|
"client",
|
||||||
|
new_callable=PropertyMock,
|
||||||
|
return_value=client,
|
||||||
|
), self.assertRaisesRegex(StorageBackendError, "deployment limit"):
|
||||||
|
list(_backend().iter_bytes("large.bin", chunk_size=3))
|
||||||
|
|
||||||
|
def test_iter_bytes_closes_streaming_body_when_consumer_stops_early(self) -> None:
|
||||||
|
body = MagicMock()
|
||||||
|
body.read.side_effect = (b"123", b"456", b"")
|
||||||
|
client = MagicMock()
|
||||||
|
client.get_object.return_value = {"ContentLength": 6, "Body": body}
|
||||||
|
with patch.dict("os.environ", {"GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES": "10"}), patch.object(
|
||||||
|
S3StorageBackend,
|
||||||
|
"client",
|
||||||
|
new_callable=PropertyMock,
|
||||||
|
return_value=client,
|
||||||
|
):
|
||||||
|
chunks = _backend().iter_bytes("object.bin", chunk_size=3)
|
||||||
|
self.assertEqual(b"123", next(chunks))
|
||||||
|
chunks.close()
|
||||||
|
body.close.assert_called_once_with()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -24,7 +24,7 @@
|
|||||||
"react-dom": "^19.0.0",
|
"react-dom": "^19.0.0",
|
||||||
"react-router-dom": "^7.1.1",
|
"react-router-dom": "^7.1.1",
|
||||||
"lucide-react": "^1.23.0",
|
"lucide-react": "^1.23.0",
|
||||||
"@govoplan/core-webui": "^0.1.8"
|
"@govoplan/core-webui": "^0.1.9"
|
||||||
},
|
},
|
||||||
"peerDependenciesMeta": {
|
"peerDependenciesMeta": {
|
||||||
"@govoplan/core-webui": {
|
"@govoplan/core-webui": {
|
||||||
|
|||||||
@@ -1,4 +1,10 @@
|
|||||||
import { ApiError, apiFetch, apiUrl, authHeaders, csrfToken, type ApiSettings, type DeltaDeletedItem, type FilesManagedFileLinkTarget } from "@govoplan/core-webui";
|
import { ApiError, apiFetch, apiUrl, authHeaders, csrfToken, type ApiSettings, type DeltaDeletedItem, type FilesManagedFileLinkTarget } from "@govoplan/core-webui";
|
||||||
|
export { fetchResourceAccessExplanation } from "@govoplan/core-webui";
|
||||||
|
export type {
|
||||||
|
AccessDecisionProvenanceItem,
|
||||||
|
ResourceAccessExplanationUser as AccessExplanationUser,
|
||||||
|
ResourceAccessExplanationResponse
|
||||||
|
} from "@govoplan/core-webui";
|
||||||
|
|
||||||
export type FileSpace = {
|
export type FileSpace = {
|
||||||
id: string;
|
id: string;
|
||||||
@@ -116,8 +122,6 @@ export type FileConnectorCredentialsPayload = {
|
|||||||
username?: string | null;
|
username?: string | null;
|
||||||
password?: string | null;
|
password?: string | null;
|
||||||
token?: string | null;
|
token?: string | null;
|
||||||
password_env?: string | null;
|
|
||||||
token_env?: string | null;
|
|
||||||
secret_ref?: string | null;
|
secret_ref?: string | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -561,44 +565,6 @@ export function bulkDeleteFiles(settings: ApiSettings, fileIds: string[]): Promi
|
|||||||
|
|
||||||
export type FileBulkShareResponse = {shares: FileShare[];shared_count: number;};
|
export type FileBulkShareResponse = {shares: FileShare[];shared_count: number;};
|
||||||
|
|
||||||
export type AccessExplanationUser = {
|
|
||||||
id: string;
|
|
||||||
account_id?: string | null;
|
|
||||||
email?: string | null;
|
|
||||||
display_name?: string | null;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type AccessDecisionProvenanceItem = {
|
|
||||||
kind: string;
|
|
||||||
id?: string | null;
|
|
||||||
label?: string | null;
|
|
||||||
tenant_id?: string | null;
|
|
||||||
source?: string | null;
|
|
||||||
details?: Record<string, unknown>;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type ResourceAccessExplanationResponse = {
|
|
||||||
user: AccessExplanationUser;
|
|
||||||
resource_type: string;
|
|
||||||
resource_id: string;
|
|
||||||
action: string;
|
|
||||||
provenance: AccessDecisionProvenanceItem[];
|
|
||||||
};
|
|
||||||
|
|
||||||
export function fetchResourceAccessExplanation(
|
|
||||||
settings: ApiSettings,
|
|
||||||
options: {userId: string;resourceType: string;resourceId: string;action: string;tenantId?: string | null;})
|
|
||||||
: Promise<ResourceAccessExplanationResponse> {
|
|
||||||
const params = new URLSearchParams({
|
|
||||||
user_id: options.userId,
|
|
||||||
resource_type: options.resourceType,
|
|
||||||
resource_id: options.resourceId,
|
|
||||||
action: options.action
|
|
||||||
});
|
|
||||||
if (options.tenantId) params.set("tenant_id", options.tenantId);
|
|
||||||
return apiFetch<ResourceAccessExplanationResponse>(settings, `/api/v1/admin/access/resource-explanation?${params.toString()}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function virtualFolderResourceId(options: {tenantId: string;ownerType: "user" | "group";ownerId: string;path: string;}): string {
|
export function virtualFolderResourceId(options: {tenantId: string;ownerType: "user" | "group";ownerId: string;path: string;}): string {
|
||||||
return `virtual-folder:v1:${options.tenantId}:${options.ownerType}:${options.ownerId}:${base64Url(options.path.replace(/\\/g, "/").replace(/^\/+|\/+$/g, ""))}`;
|
return `virtual-folder:v1:${options.tenantId}:${options.ownerType}:${options.ownerId}:${base64Url(options.path.replace(/\\/g, "/").replace(/^\/+|\/+$/g, ""))}`;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ import {
|
|||||||
LoadingFrame,
|
LoadingFrame,
|
||||||
mergeDeltaRows,
|
mergeDeltaRows,
|
||||||
SegmentedControl,
|
SegmentedControl,
|
||||||
|
StatusBadge,
|
||||||
|
TableActionGroup,
|
||||||
ToggleSwitch,
|
ToggleSwitch,
|
||||||
useDeltaWatermarks,
|
useDeltaWatermarks,
|
||||||
useUnsavedDraftGuard,
|
useUnsavedDraftGuard,
|
||||||
@@ -65,8 +67,6 @@ type ConnectorProfileDraft = {
|
|||||||
username: string;
|
username: string;
|
||||||
password: string;
|
password: string;
|
||||||
token: string;
|
token: string;
|
||||||
passwordEnv: string;
|
|
||||||
tokenEnv: string;
|
|
||||||
secretRef: string;
|
secretRef: string;
|
||||||
canBrowse: boolean;
|
canBrowse: boolean;
|
||||||
canImport: boolean;
|
canImport: boolean;
|
||||||
@@ -91,8 +91,6 @@ type ConnectorCredentialDraft = {
|
|||||||
username: string;
|
username: string;
|
||||||
password: string;
|
password: string;
|
||||||
token: string;
|
token: string;
|
||||||
passwordEnv: string;
|
|
||||||
tokenEnv: string;
|
|
||||||
secretRef: string;
|
secretRef: string;
|
||||||
policyMode: PolicyMode;
|
policyMode: PolicyMode;
|
||||||
allowedPaths: string;
|
allowedPaths: string;
|
||||||
@@ -491,8 +489,6 @@ export default function FileConnectorSettingsPanel({
|
|||||||
username: cleanOrNull(credentialDraft.username),
|
username: cleanOrNull(credentialDraft.username),
|
||||||
password: cleanOrUndefined(credentialDraft.password),
|
password: cleanOrUndefined(credentialDraft.password),
|
||||||
token: cleanOrUndefined(credentialDraft.token),
|
token: cleanOrUndefined(credentialDraft.token),
|
||||||
password_env: cleanOrNull(credentialDraft.passwordEnv),
|
|
||||||
token_env: cleanOrNull(credentialDraft.tokenEnv),
|
|
||||||
secret_ref: cleanOrNull(credentialDraft.secretRef)
|
secret_ref: cleanOrNull(credentialDraft.secretRef)
|
||||||
};
|
};
|
||||||
const sharedPayload = {
|
const sharedPayload = {
|
||||||
@@ -633,8 +629,6 @@ export default function FileConnectorSettingsPanel({
|
|||||||
username: cleanOrNull(draft.username),
|
username: cleanOrNull(draft.username),
|
||||||
password: cleanOrUndefined(draft.password),
|
password: cleanOrUndefined(draft.password),
|
||||||
token: cleanOrUndefined(draft.token),
|
token: cleanOrUndefined(draft.token),
|
||||||
password_env: cleanOrNull(draft.passwordEnv),
|
|
||||||
token_env: cleanOrNull(draft.tokenEnv),
|
|
||||||
secret_ref: cleanOrNull(draft.secretRef)
|
secret_ref: cleanOrNull(draft.secretRef)
|
||||||
},
|
},
|
||||||
metadata: metadataFromDraft(draft)
|
metadata: metadataFromDraft(draft)
|
||||||
@@ -680,8 +674,6 @@ export default function FileConnectorSettingsPanel({
|
|||||||
username: cleanOrNull(credentialDraft.username),
|
username: cleanOrNull(credentialDraft.username),
|
||||||
password: cleanOrUndefined(credentialDraft.password),
|
password: cleanOrUndefined(credentialDraft.password),
|
||||||
token: cleanOrUndefined(credentialDraft.token),
|
token: cleanOrUndefined(credentialDraft.token),
|
||||||
password_env: cleanOrNull(credentialDraft.passwordEnv),
|
|
||||||
token_env: cleanOrNull(credentialDraft.tokenEnv),
|
|
||||||
secret_ref: cleanOrNull(credentialDraft.secretRef)
|
secret_ref: cleanOrNull(credentialDraft.secretRef)
|
||||||
},
|
},
|
||||||
metadata: profile.metadata ?? {},
|
metadata: profile.metadata ?? {},
|
||||||
@@ -756,7 +748,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
width: "120px",
|
width: "120px",
|
||||||
render: (row) => {
|
render: (row) => {
|
||||||
const enabled = row.kind === "profile" ? row.profile.enabled : row.credential.enabled;
|
const enabled = row.kind === "profile" ? row.profile.enabled : row.credential.enabled;
|
||||||
return <span className={`status-badge ${enabled ? "success" : "neutral"}`}>{enabled ? "i18n:govoplan-files.enabled.df174a3f" : "i18n:govoplan-files.disabled.f4f4473d"}</span>;
|
return <StatusBadge status={enabled ? "success" : "inactive"} label={enabled ? "i18n:govoplan-files.enabled.df174a3f" : "i18n:govoplan-files.disabled.f4f4473d"} />;
|
||||||
}
|
}
|
||||||
}];
|
}];
|
||||||
|
|
||||||
@@ -771,21 +763,19 @@ export default function FileConnectorSettingsPanel({
|
|||||||
function renderConnectorActions(row: ConnectorTreeRow) {
|
function renderConnectorActions(row: ConnectorTreeRow) {
|
||||||
if (row.kind === "credential") {
|
if (row.kind === "credential") {
|
||||||
const readOnly = row.credential.source_kind !== "database";
|
const readOnly = row.credential.source_kind !== "database";
|
||||||
return (
|
return <TableActionGroup actions={[
|
||||||
<div className="admin-icon-actions">
|
{ id: "edit", label: i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.credential.label }), icon: <Edit3 size={15} />, disabled: !canWrite || readOnly || saving, onClick: () => startCredentialEdit(row.credential) },
|
||||||
<Button type="button" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.credential.label })} aria-label={i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.credential.label })} onClick={() => startCredentialEdit(row.credential)} disabled={!canWrite || readOnly || saving}><Edit3 size={15} /></Button>
|
{ id: "disable", label: i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.credential.label }), icon: <Trash2 size={15} />, variant: "danger", applicable: row.credential.enabled, disabled: !canWrite || readOnly || saving, onClick: () => setPendingCredentialDeactivate(row.credential) }
|
||||||
<Button type="button" variant="danger" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.credential.label })} aria-label={i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.credential.label })} onClick={() => setPendingCredentialDeactivate(row.credential)} disabled={!canWrite || readOnly || saving || !row.credential.enabled}><Trash2 size={15} /></Button>
|
]} />;
|
||||||
</div>);
|
|
||||||
|
|
||||||
}
|
}
|
||||||
const readOnly = row.profile.source_kind !== "database";
|
const readOnly = row.profile.source_kind !== "database";
|
||||||
return (
|
return <TableActionGroup actions={[
|
||||||
<div className="admin-icon-actions">
|
{ id: "test", label: i18nMessage("i18n:govoplan-files.test_value.6a5d10a5", { value0: row.profile.label }), icon: <RefreshCw size={15} />, applicable: row.profile.enabled, disabled: saving || testingProfileId === row.profile.id, onClick: () => void testBrowse(row.profile) },
|
||||||
<Button type="button" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.test_value.6a5d10a5", { value0: row.profile.label })} aria-label={i18nMessage("i18n:govoplan-files.test_value.6a5d10a5", { value0: row.profile.label })} onClick={() => void testBrowse(row.profile)} disabled={saving || testingProfileId === row.profile.id || !row.profile.enabled}><RefreshCw size={15} /></Button>
|
{ id: "add-credential", label: i18nMessage("i18n:govoplan-files.add_credential_for_value.0fa9c1fe", { value0: row.profile.label }), icon: <UserRoundKey size={15} />, variant: "primary", disabled: !canWrite || saving, onClick: () => startCredentialCreate(row.profile) },
|
||||||
<Button type="button" variant="primary" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.add_credential_for_value.0fa9c1fe", { value0: row.profile.label })} aria-label={i18nMessage("i18n:govoplan-files.add_credential_for_value.0fa9c1fe", { value0: row.profile.label })} onClick={() => startCredentialCreate(row.profile)} disabled={!canWrite || saving}><UserRoundKey size={15} /></Button>
|
{ id: "edit", label: i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.profile.label }), icon: <Edit3 size={15} />, disabled: !canWrite || readOnly || saving, onClick: () => startEdit(row.profile) },
|
||||||
<Button type="button" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.profile.label })} aria-label={i18nMessage("i18n:govoplan-files.edit_value.fad75899", { value0: row.profile.label })} onClick={() => startEdit(row.profile)} disabled={!canWrite || readOnly || saving}><Edit3 size={15} /></Button>
|
{ id: "disable", label: i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.profile.label }), icon: <Trash2 size={15} />, variant: "danger", applicable: row.profile.enabled, disabled: !canWrite || readOnly || saving, onClick: () => setPendingDeactivate(row.profile) }
|
||||||
<Button type="button" variant="danger" className="admin-icon-button" title={i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.profile.label })} aria-label={i18nMessage("i18n:govoplan-files.disable_value.09485f7f", { value0: row.profile.label })} onClick={() => setPendingDeactivate(row.profile)} disabled={!canWrite || readOnly || saving || !row.profile.enabled}><Trash2 size={15} /></Button>
|
]} />;
|
||||||
</div>);
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -820,7 +810,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
|
|
||||||
{targetSelectionRequired &&
|
{targetSelectionRequired &&
|
||||||
<Card title={i18nMessage("i18n:govoplan-files.value_scope", { value0: targetLabel })}>
|
<Card title={i18nMessage("i18n:govoplan-files.value_scope", { value0: targetLabel })}>
|
||||||
<div className="mail-profile-target-row">
|
<div className="settings-target-row">
|
||||||
<FormField label={targetLabel}>
|
<FormField label={targetLabel}>
|
||||||
<select value={selectedTargetId} onChange={(event) => setSelectedTargetId(event.target.value)} disabled={loading || saving || !hasSelectableTarget}>
|
<select value={selectedTargetId} onChange={(event) => setSelectedTargetId(event.target.value)} disabled={loading || saving || !hasSelectableTarget}>
|
||||||
{!hasSelectableTarget && <option value="">{targetEmptyText}</option>}
|
{!hasSelectableTarget && <option value="">{targetEmptyText}</option>}
|
||||||
@@ -865,7 +855,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
|
|
||||||
<LoadingFrame loading={loading} label="i18n:govoplan-files.loading_connector_policy.f12ab285">
|
<LoadingFrame loading={loading} label="i18n:govoplan-files.loading_connector_policy.f12ab285">
|
||||||
<>
|
<>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.allowed_connection_ids.0df854c8">
|
<FormField label="i18n:govoplan-files.allowed_connection_ids.0df854c8">
|
||||||
<textarea value={policyDraft.allowedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedConnectors: event.target.value })} rows={3} placeholder={"tenant-webdav\nseafile-*"} />
|
<textarea value={policyDraft.allowedConnectors} disabled={saving || !canWrite} onChange={(event) => patchPolicyDraft({ allowedConnectors: event.target.value })} rows={3} placeholder={"tenant-webdav\nseafile-*"} />
|
||||||
</FormField>
|
</FormField>
|
||||||
@@ -940,7 +930,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
<h3>Credential</h3>
|
<h3>Credential</h3>
|
||||||
<p>Choose where this credential can be used and how it signs in.</p>
|
<p>Choose where this credential can be used and how it signs in.</p>
|
||||||
</header>
|
</header>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.credential_id.9432a6e1">
|
<FormField label="i18n:govoplan-files.credential_id.9432a6e1">
|
||||||
<input className={!editingCredentialId && !credentialDraft.id.trim() ? "field-input-missing" : undefined} aria-invalid={!editingCredentialId && !credentialDraft.id.trim() || undefined} value={credentialDraft.id} disabled={Boolean(editingCredentialId) || saving} onChange={(event) => patchCredentialDraft({ id: event.target.value })} placeholder={`${scopeType}-webdav-credentials`} />
|
<input className={!editingCredentialId && !credentialDraft.id.trim() ? "field-input-missing" : undefined} aria-invalid={!editingCredentialId && !credentialDraft.id.trim() || undefined} value={credentialDraft.id} disabled={Boolean(editingCredentialId) || saving} onChange={(event) => patchCredentialDraft({ id: event.target.value })} placeholder={`${scopeType}-webdav-credentials`} />
|
||||||
</FormField>
|
</FormField>
|
||||||
@@ -1018,7 +1008,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
onChange={patchCredentialValues}
|
onChange={patchCredentialValues}
|
||||||
disabled={saving}
|
disabled={saving}
|
||||||
showPassword={false} />
|
showPassword={false} />
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.secret_reference.04ed2221">
|
<FormField label="i18n:govoplan-files.secret_reference.04ed2221">
|
||||||
<input value={credentialDraft.secretRef} disabled={saving} onChange={(event) => patchCredentialDraft({ secretRef: event.target.value })} />
|
<input value={credentialDraft.secretRef} disabled={saving} onChange={(event) => patchCredentialDraft({ secretRef: event.target.value })} />
|
||||||
</FormField>
|
</FormField>
|
||||||
@@ -1049,16 +1039,6 @@ export default function FileConnectorSettingsPanel({
|
|||||||
{credentialLoginResult.message}
|
{credentialLoginResult.message}
|
||||||
</DismissibleAlert>
|
</DismissibleAlert>
|
||||||
}
|
}
|
||||||
<AdvancedOptionsPanel title="Environment and fallback secrets" summary="Use these only when the deployment injects secrets outside the database.">
|
|
||||||
<div className="form-grid two-column-form-grid">
|
|
||||||
<FormField label="i18n:govoplan-files.password_environment_variable.0e77673f">
|
|
||||||
<input value={credentialDraft.passwordEnv} disabled={saving} onChange={(event) => patchCredentialDraft({ passwordEnv: event.target.value })} />
|
|
||||||
</FormField>
|
|
||||||
<FormField label="i18n:govoplan-files.token_environment_variable.5af4a79e">
|
|
||||||
<input value={credentialDraft.tokenEnv} disabled={saving} onChange={(event) => patchCredentialDraft({ tokenEnv: event.target.value })} />
|
|
||||||
</FormField>
|
|
||||||
</div>
|
|
||||||
</AdvancedOptionsPanel>
|
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
<section className="adaptive-config-section">
|
<section className="adaptive-config-section">
|
||||||
@@ -1066,7 +1046,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
<h3>Policy</h3>
|
<h3>Policy</h3>
|
||||||
<p>Limit where lower levels may use this credential.</p>
|
<p>Limit where lower levels may use this credential.</p>
|
||||||
</header>
|
</header>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.policy.bb9cf141">
|
<FormField label="i18n:govoplan-files.policy.bb9cf141">
|
||||||
<select value={credentialDraft.policyMode} disabled={saving} onChange={(event) => patchCredentialDraft({ policyMode: event.target.value as PolicyMode })}>
|
<select value={credentialDraft.policyMode} disabled={saving} onChange={(event) => patchCredentialDraft({ policyMode: event.target.value as PolicyMode })}>
|
||||||
<option value="allow-provider">i18n:govoplan-files.can_use_this_credential.f4a41971</option>
|
<option value="allow-provider">i18n:govoplan-files.can_use_this_credential.f4a41971</option>
|
||||||
@@ -1118,7 +1098,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
<h3>Connection</h3>
|
<h3>Connection</h3>
|
||||||
<p>Name the file server connection and choose the provider.</p>
|
<p>Name the file server connection and choose the provider.</p>
|
||||||
</header>
|
</header>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.profile_id.25961d68">
|
<FormField label="i18n:govoplan-files.profile_id.25961d68">
|
||||||
<input className={!editingProfileId && !draft.id.trim() ? "field-input-missing" : undefined} aria-invalid={!editingProfileId && !draft.id.trim() || undefined} value={draft.id} disabled={Boolean(editingProfileId) || saving} onChange={(event) => patchDraft({ id: event.target.value })} placeholder={`${scopeType}-webdav`} />
|
<input className={!editingProfileId && !draft.id.trim() ? "field-input-missing" : undefined} aria-invalid={!editingProfileId && !draft.id.trim() || undefined} value={draft.id} disabled={Boolean(editingProfileId) || saving} onChange={(event) => patchDraft({ id: event.target.value })} placeholder={`${scopeType}-webdav`} />
|
||||||
</FormField>
|
</FormField>
|
||||||
@@ -1151,7 +1131,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
<h3>Location and credentials</h3>
|
<h3>Location and credentials</h3>
|
||||||
<p>Only fields relevant for the selected provider and credential mode are shown.</p>
|
<p>Only fields relevant for the selected provider and credential mode are shown.</p>
|
||||||
</header>
|
</header>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.endpoint_url.65aaaa45">
|
<FormField label="i18n:govoplan-files.endpoint_url.65aaaa45">
|
||||||
<input value={draft.endpointUrl} disabled={saving} onChange={(event) => patchDraft({ endpointUrl: event.target.value })} placeholder={ENDPOINT_PLACEHOLDERS[draft.provider]} />
|
<input value={draft.endpointUrl} disabled={saving} onChange={(event) => patchDraft({ endpointUrl: event.target.value })} placeholder={ENDPOINT_PLACEHOLDERS[draft.provider]} />
|
||||||
</FormField>
|
</FormField>
|
||||||
@@ -1188,7 +1168,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
</DismissibleAlert>
|
</DismissibleAlert>
|
||||||
}
|
}
|
||||||
<AdvancedOptionsPanel title="Protocol and compatibility options" summary="Change these only when the provider or network requires an override.">
|
<AdvancedOptionsPanel title="Protocol and compatibility options" summary="Change these only when the provider or network requires an override.">
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.browse_protocol.d1f27c90">
|
<FormField label="i18n:govoplan-files.browse_protocol.d1f27c90">
|
||||||
<select value={draft.browseProtocol} disabled={saving} onChange={(event) => patchDraft({ browseProtocol: event.target.value })}>
|
<select value={draft.browseProtocol} disabled={saving} onChange={(event) => patchDraft({ browseProtocol: event.target.value })}>
|
||||||
<option value="">i18n:govoplan-files.native_default.ba32f7b6</option>
|
<option value="">i18n:govoplan-files.native_default.ba32f7b6</option>
|
||||||
@@ -1222,7 +1202,7 @@ export default function FileConnectorSettingsPanel({
|
|||||||
<ToggleSwitch label="i18n:govoplan-files.import.d6fbc9d2" checked={draft.canImport} disabled={saving} onChange={(canImport) => patchDraft({ canImport })} />
|
<ToggleSwitch label="i18n:govoplan-files.import.d6fbc9d2" checked={draft.canImport} disabled={saving} onChange={(canImport) => patchDraft({ canImport })} />
|
||||||
<ToggleSwitch label="i18n:govoplan-files.sync.905f6309" checked={draft.canSync} disabled={saving} onChange={(canSync) => patchDraft({ canSync })} />
|
<ToggleSwitch label="i18n:govoplan-files.sync.905f6309" checked={draft.canSync} disabled={saving} onChange={(canSync) => patchDraft({ canSync })} />
|
||||||
</div>
|
</div>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.policy.bb9cf141">
|
<FormField label="i18n:govoplan-files.policy.bb9cf141">
|
||||||
<select value={draft.policyMode} disabled={saving} onChange={(event) => patchDraft({ policyMode: event.target.value as PolicyMode })}>
|
<select value={draft.policyMode} disabled={saving} onChange={(event) => patchDraft({ policyMode: event.target.value as PolicyMode })}>
|
||||||
<option value="allow-provider">i18n:govoplan-files.can_use_this_connection.5091a74c</option>
|
<option value="allow-provider">i18n:govoplan-files.can_use_this_connection.5091a74c</option>
|
||||||
@@ -1373,8 +1353,6 @@ function emptyDraft(scopeType: ConnectorScope): ConnectorProfileDraft {
|
|||||||
username: "",
|
username: "",
|
||||||
password: "",
|
password: "",
|
||||||
token: "",
|
token: "",
|
||||||
passwordEnv: "",
|
|
||||||
tokenEnv: "",
|
|
||||||
secretRef: "",
|
secretRef: "",
|
||||||
canBrowse: true,
|
canBrowse: true,
|
||||||
canImport: true,
|
canImport: true,
|
||||||
@@ -1437,8 +1415,6 @@ function emptyCredentialDraft(scopeType: ConnectorScope): ConnectorCredentialDra
|
|||||||
username: "",
|
username: "",
|
||||||
password: "",
|
password: "",
|
||||||
token: "",
|
token: "",
|
||||||
passwordEnv: "",
|
|
||||||
tokenEnv: "",
|
|
||||||
secretRef: "",
|
secretRef: "",
|
||||||
policyMode: "allow-provider",
|
policyMode: "allow-provider",
|
||||||
allowedPaths: "",
|
allowedPaths: "",
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import {
|
|||||||
FileDropZone,
|
FileDropZone,
|
||||||
FormField,
|
FormField,
|
||||||
LoadingIndicator,
|
LoadingIndicator,
|
||||||
|
ResourceAccessExplanation,
|
||||||
ToggleSwitch,
|
ToggleSwitch,
|
||||||
hasScope,
|
hasScope,
|
||||||
type ApiSettings,
|
type ApiSettings,
|
||||||
@@ -1984,7 +1985,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
|||||||
const currentFolderDropActive = currentFolderDropTarget ? dropTargetKey === dropTargetId(currentFolderDropTarget) : false;
|
const currentFolderDropActive = currentFolderDropTarget ? dropTargetKey === dropTargetId(currentFolderDropTarget) : false;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="workspace-data-page module-entry-page file-manager-page file-manager-fullscreen">
|
<div className="workspace-data-page module-entry-page file-manager-page file-manager-fullscreen files-page">
|
||||||
{error &&
|
{error &&
|
||||||
<DismissibleAlert tone="danger" resetKey={error} floating>{error}</DismissibleAlert>
|
<DismissibleAlert tone="danger" resetKey={error} floating>{error}</DismissibleAlert>
|
||||||
}
|
}
|
||||||
@@ -2211,8 +2212,8 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
|||||||
}
|
}
|
||||||
|
|
||||||
{accessExplanationTarget &&
|
{accessExplanationTarget &&
|
||||||
<FileDialog title="i18n:govoplan-files.access_explanation.75ee7f62" onClose={() => {if (!resourceAccessLoading) {setAccessExplanationTarget(null);setResourceAccessExplanation(null);}}}>
|
<FileDialog title="i18n:govoplan-core.access_explanation.75ee7f62" onClose={() => {if (!resourceAccessLoading) {setAccessExplanationTarget(null);setResourceAccessExplanation(null);}}}>
|
||||||
<ResourceAccessExplanationContent
|
<ResourceAccessExplanation
|
||||||
loading={resourceAccessLoading}
|
loading={resourceAccessLoading}
|
||||||
explanation={resourceAccessExplanation}
|
explanation={resourceAccessExplanation}
|
||||||
fallbackResourceLabel={accessExplanationTarget.label} />
|
fallbackResourceLabel={accessExplanationTarget.label} />
|
||||||
@@ -2433,7 +2434,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
|||||||
{dialog === "transfer" && transferDialogState &&
|
{dialog === "transfer" && transferDialogState &&
|
||||||
<FileDialog title={`${transferMode === "copy" ? "i18n:govoplan-files.copy.af74f7c5" : "i18n:govoplan-files.move.76cdb950"} selection`} onClose={closeDialog}>
|
<FileDialog title={`${transferMode === "copy" ? "i18n:govoplan-files.copy.af74f7c5" : "i18n:govoplan-files.move.76cdb950"} selection`} onClose={closeDialog}>
|
||||||
<p className="muted">i18n:govoplan-files.choose_a_destination_space_and_folder_folders_ar.45158643</p>
|
<p className="muted">i18n:govoplan-files.choose_a_destination_space_and_folder_folders_ar.45158643</p>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.destination_space.92b63970" help="i18n:govoplan-files.select_the_space_that_will_receive_the_selected_.0a8bea8f">
|
<FormField label="i18n:govoplan-files.destination_space.92b63970" help="i18n:govoplan-files.select_the_space_that_will_receive_the_selected_.0a8bea8f">
|
||||||
<select value={transferDialogState.targetSpaceId} onChange={(event) => setTransferDialogState((current) => current ? { ...current, targetSpaceId: event.target.value, targetFolder: "" } : current)}>
|
<select value={transferDialogState.targetSpaceId} onChange={(event) => setTransferDialogState((current) => current ? { ...current, targetSpaceId: event.target.value, targetFolder: "" } : current)}>
|
||||||
{spaces.filter((space) => !isConnectorSpace(space)).map((space) => <option key={space.id} value={space.id}>{space.label}</option>)}
|
{spaces.filter((space) => !isConnectorSpace(space)).map((space) => <option key={space.id} value={space.id}>{space.label}</option>)}
|
||||||
@@ -2472,7 +2473,7 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
|||||||
{dialog === "rename" &&
|
{dialog === "rename" &&
|
||||||
<FileDialog title="i18n:govoplan-files.bulk_rename_selected_items.5e79d694" onClose={closeDialog}>
|
<FileDialog title="i18n:govoplan-files.bulk_rename_selected_items.5e79d694" onClose={closeDialog}>
|
||||||
<p className="muted">i18n:govoplan-files.bulk_rename_changes_managed_display_paths_only_i.8cf34a6b</p>
|
<p className="muted">i18n:govoplan-files.bulk_rename_changes_managed_display_paths_only_i.8cf34a6b</p>
|
||||||
<div className="form-grid two-column-form-grid">
|
<div className="form-grid two">
|
||||||
<FormField label="i18n:govoplan-files.mode.a7b93d21" help="i18n:govoplan-files.choose_how_the_selected_names_should_be_changed.0231854f">
|
<FormField label="i18n:govoplan-files.mode.a7b93d21" help="i18n:govoplan-files.choose_how_the_selected_names_should_be_changed.0231854f">
|
||||||
<select value={renameMode} onChange={(event) => setRenameMode(event.target.value as RenameMode)}>
|
<select value={renameMode} onChange={(event) => setRenameMode(event.target.value as RenameMode)}>
|
||||||
<option value="prefix">i18n:govoplan-files.add_prefix.672452bc</option>
|
<option value="prefix">i18n:govoplan-files.add_prefix.672452bc</option>
|
||||||
@@ -2508,71 +2509,6 @@ export default function FilesPage({ settings, auth }: {settings: ApiSettings;aut
|
|||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function ResourceAccessExplanationContent({
|
|
||||||
loading,
|
|
||||||
explanation,
|
|
||||||
fallbackResourceLabel
|
|
||||||
}: {loading: boolean;explanation: ResourceAccessExplanationResponse | null;fallbackResourceLabel: string;}) {
|
|
||||||
if (loading) return <p className="muted small-note">i18n:govoplan-files.loading_access_explanation.04a7c934</p>;
|
|
||||||
if (!explanation) return null;
|
|
||||||
const userLabel = explanation.user.display_name || explanation.user.email || explanation.user.id;
|
|
||||||
const resourceLabel = explanation.provenance.find((item) => item.kind === "resource")?.label || fallbackResourceLabel || explanation.resource_id;
|
|
||||||
return (
|
|
||||||
<>
|
|
||||||
<div className="form-grid compact responsive-form-grid">
|
|
||||||
<div><span className="form-label">i18n:govoplan-files.user.9f8a2389</span><p>{userLabel}</p></div>
|
|
||||||
<div><span className="form-label">i18n:govoplan-files.resource.d1c626a9</span><p>{resourceLabel}</p></div>
|
|
||||||
<div><span className="form-label">i18n:govoplan-files.action.97c89a4d</span><p><code>{explanation.action}</code></p></div>
|
|
||||||
<div><span className="form-label">i18n:govoplan-files.evidence.8487d192</span><p>{explanation.provenance.length}</p></div>
|
|
||||||
</div>
|
|
||||||
{explanation.provenance.length === 0 ?
|
|
||||||
<p className="muted small-note">i18n:govoplan-files.no_access_evidence_was_returned.84a21e4e</p> :
|
|
||||||
<div className="admin-assignment-grid">
|
|
||||||
{explanation.provenance.map((item, index) =>
|
|
||||||
<div key={`${item.kind}:${item.id ?? index}`}>
|
|
||||||
<strong>{provenanceKindLabel(item.kind)}</strong>
|
|
||||||
<div className="muted small-note">
|
|
||||||
{item.source || "i18n:govoplan-files.no_source.6dcf9723"}
|
|
||||||
{item.id && <> · <code>{item.id}</code></>}
|
|
||||||
</div>
|
|
||||||
{item.label && <p>{item.label}</p>}
|
|
||||||
{Object.keys(item.details ?? {}).length > 0 && <p className="muted small-note">{formatProvenanceDetails(item.details ?? {})}</p>}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
}
|
|
||||||
</>);
|
|
||||||
}
|
|
||||||
|
|
||||||
function provenanceKindLabel(kind: string): string {
|
|
||||||
switch (kind) {
|
|
||||||
case "resource":
|
|
||||||
return "i18n:govoplan-files.resource.d1c626a9";
|
|
||||||
case "owner":
|
|
||||||
return "i18n:govoplan-files.owner.89ff3122";
|
|
||||||
case "share":
|
|
||||||
return "i18n:govoplan-files.share.09ca55ca";
|
|
||||||
case "policy":
|
|
||||||
return "i18n:govoplan-files.policy.0b779a05";
|
|
||||||
case "role":
|
|
||||||
return "i18n:govoplan-files.role.b5b4a5a2";
|
|
||||||
case "right":
|
|
||||||
return "i18n:govoplan-files.permission.2f81a22d";
|
|
||||||
default:
|
|
||||||
return kind;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function formatProvenanceDetails(details: Record<string, unknown>): string {
|
|
||||||
return Object.entries(details).map(([key, value]) => `${key}: ${formatProvenanceValue(value)}`).join("; ");
|
|
||||||
}
|
|
||||||
|
|
||||||
function formatProvenanceValue(value: unknown): string {
|
|
||||||
if (value === null || value === undefined) return "i18n:govoplan-files.none.6eef6648";
|
|
||||||
if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") return String(value);
|
|
||||||
return JSON.stringify(value);
|
|
||||||
}
|
|
||||||
|
|
||||||
function FileSearchRow({
|
function FileSearchRow({
|
||||||
value,
|
value,
|
||||||
caseSensitive,
|
caseSensitive,
|
||||||
|
|||||||
@@ -252,7 +252,6 @@ export const generatedTranslations: PlatformTranslations = {
|
|||||||
"i18n:govoplan-files.overwrite.0625c54e": "Overwrite",
|
"i18n:govoplan-files.overwrite.0625c54e": "Overwrite",
|
||||||
"i18n:govoplan-files.owner_space.364c4b62": "Owner space",
|
"i18n:govoplan-files.owner_space.364c4b62": "Owner space",
|
||||||
"i18n:govoplan-files.parent_folder.d8ed4c2a": "Parent folder",
|
"i18n:govoplan-files.parent_folder.d8ed4c2a": "Parent folder",
|
||||||
"i18n:govoplan-files.password_environment_variable.0e77673f": "Password environment variable",
|
|
||||||
"i18n:govoplan-files.password.8be3c943": "Password",
|
"i18n:govoplan-files.password.8be3c943": "Password",
|
||||||
"i18n:govoplan-files.path_limited.d32cbef0": "Path-limited",
|
"i18n:govoplan-files.path_limited.d32cbef0": "Path-limited",
|
||||||
"i18n:govoplan-files.pattern_preview_results.56cea56c": "Pattern preview results",
|
"i18n:govoplan-files.pattern_preview_results.56cea56c": "Pattern preview results",
|
||||||
@@ -318,7 +317,6 @@ export const generatedTranslations: PlatformTranslations = {
|
|||||||
"i18n:govoplan-files.this_file_credential.695efb90": "this file credential",
|
"i18n:govoplan-files.this_file_credential.695efb90": "this file credential",
|
||||||
"i18n:govoplan-files.this_folder_is_empty_upload_files_in_the_top_lev.cb6c3a1a": "This folder is empty. Upload files in the top-level Files module.",
|
"i18n:govoplan-files.this_folder_is_empty_upload_files_in_the_top_lev.cb6c3a1a": "This folder is empty. Upload files in the top-level Files module.",
|
||||||
"i18n:govoplan-files.this_folder_is_empty_use_upload_or_create_folder.5977d784": "This folder is empty. Use Upload or Create Folder to add content.",
|
"i18n:govoplan-files.this_folder_is_empty_use_upload_or_create_folder.5977d784": "This folder is empty. Use Upload or Create Folder to add content.",
|
||||||
"i18n:govoplan-files.token_environment_variable.5af4a79e": "Token environment variable",
|
|
||||||
"i18n:govoplan-files.token.a1141eb9": "Token",
|
"i18n:govoplan-files.token.a1141eb9": "Token",
|
||||||
"i18n:govoplan-files.unpack_zip_uploads.256fead4": "Unpack ZIP uploads",
|
"i18n:govoplan-files.unpack_zip_uploads.256fead4": "Unpack ZIP uploads",
|
||||||
"i18n:govoplan-files.unpacking_zip_archive.698095f4": "Unpacking ZIP archive",
|
"i18n:govoplan-files.unpacking_zip_archive.698095f4": "Unpacking ZIP archive",
|
||||||
@@ -609,7 +607,6 @@ export const generatedTranslations: PlatformTranslations = {
|
|||||||
"i18n:govoplan-files.overwrite.0625c54e": "Overwrite",
|
"i18n:govoplan-files.overwrite.0625c54e": "Overwrite",
|
||||||
"i18n:govoplan-files.owner_space.364c4b62": "Owner space",
|
"i18n:govoplan-files.owner_space.364c4b62": "Owner space",
|
||||||
"i18n:govoplan-files.parent_folder.d8ed4c2a": "Parent folder",
|
"i18n:govoplan-files.parent_folder.d8ed4c2a": "Parent folder",
|
||||||
"i18n:govoplan-files.password_environment_variable.0e77673f": "Password environment variable",
|
|
||||||
"i18n:govoplan-files.password.8be3c943": "Passwort",
|
"i18n:govoplan-files.password.8be3c943": "Passwort",
|
||||||
"i18n:govoplan-files.path_limited.d32cbef0": "Path-limited",
|
"i18n:govoplan-files.path_limited.d32cbef0": "Path-limited",
|
||||||
"i18n:govoplan-files.pattern_preview_results.56cea56c": "Pattern preview results",
|
"i18n:govoplan-files.pattern_preview_results.56cea56c": "Pattern preview results",
|
||||||
@@ -675,7 +672,6 @@ export const generatedTranslations: PlatformTranslations = {
|
|||||||
"i18n:govoplan-files.this_file_credential.695efb90": "this file credential",
|
"i18n:govoplan-files.this_file_credential.695efb90": "this file credential",
|
||||||
"i18n:govoplan-files.this_folder_is_empty_upload_files_in_the_top_lev.cb6c3a1a": "This folder is empty. Upload files in the top-level Files module.",
|
"i18n:govoplan-files.this_folder_is_empty_upload_files_in_the_top_lev.cb6c3a1a": "This folder is empty. Upload files in the top-level Files module.",
|
||||||
"i18n:govoplan-files.this_folder_is_empty_use_upload_or_create_folder.5977d784": "This folder is empty. Use Upload or Create Folder to add content.",
|
"i18n:govoplan-files.this_folder_is_empty_use_upload_or_create_folder.5977d784": "This folder is empty. Use Upload or Create Folder to add content.",
|
||||||
"i18n:govoplan-files.token_environment_variable.5af4a79e": "Token environment variable",
|
|
||||||
"i18n:govoplan-files.token.a1141eb9": "Token",
|
"i18n:govoplan-files.token.a1141eb9": "Token",
|
||||||
"i18n:govoplan-files.unpack_zip_uploads.256fead4": "Unpack ZIP uploads",
|
"i18n:govoplan-files.unpack_zip_uploads.256fead4": "Unpack ZIP uploads",
|
||||||
"i18n:govoplan-files.unpacking_zip_archive.698095f4": "Unpacking ZIP archive",
|
"i18n:govoplan-files.unpacking_zip_archive.698095f4": "Unpacking ZIP archive",
|
||||||
|
|||||||
@@ -1,85 +1,6 @@
|
|||||||
/* Files manager */
|
/* Files manager */
|
||||||
.file-manager-page.file-manager-fullscreen {
|
.files-page .file-manager-shell {
|
||||||
position: relative;
|
|
||||||
display: grid;
|
|
||||||
grid-template-rows: 1fr;
|
|
||||||
height: calc(100vh - 115px);
|
|
||||||
padding: 0;
|
|
||||||
overflow: hidden;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-manager-toolbar {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 8px;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
padding: 10px 14px;
|
|
||||||
border-bottom: var(--border-line);
|
|
||||||
background: var(--panel-header);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-manager-toolbar .button {
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 7px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-manager-shell {
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: minmax(240px, 300px) minmax(0, 1fr);
|
grid-template-columns: minmax(240px, 300px) minmax(0, 1fr);
|
||||||
min-height: 0;
|
|
||||||
height: 100%;
|
|
||||||
border: var(--border-line);
|
|
||||||
border-radius: 0;
|
|
||||||
overflow: hidden;
|
|
||||||
background: var(--panel);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-panel {
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
min-width: 0;
|
|
||||||
min-height: 0;
|
|
||||||
background: var(--panel);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-sticky {
|
|
||||||
flex: 0 0 auto;
|
|
||||||
z-index: 2;
|
|
||||||
border-bottom: var(--border-line);
|
|
||||||
background: var(--panel-soft);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-breadcrumbs {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
gap: 6px;
|
|
||||||
min-height: 32px;
|
|
||||||
padding: 10px 14px 6px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-breadcrumb,
|
|
||||||
.file-breadcrumb-segment {
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 5px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-breadcrumb {
|
|
||||||
border: 0;
|
|
||||||
background: transparent;
|
|
||||||
color: var(--text-strong);
|
|
||||||
cursor: pointer;
|
|
||||||
font-weight: 800;
|
|
||||||
padding: 4px 6px;
|
|
||||||
border-radius: 7px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-breadcrumb:hover:not(:disabled),
|
|
||||||
.file-breadcrumb:focus-visible {
|
|
||||||
background: var(--panel);
|
|
||||||
outline: none;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-search-row {
|
.file-search-row {
|
||||||
@@ -90,53 +11,21 @@
|
|||||||
padding: 4px 0 10px 14px;
|
padding: 4px 0 10px 14px;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-meta {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
gap: 12px;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
padding: 8px 14px;
|
|
||||||
border-top: var(--border-line);
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-drop-target {
|
|
||||||
position: relative;
|
|
||||||
flex: 1 1 auto;
|
|
||||||
min-height: 0;
|
|
||||||
overflow: auto;
|
|
||||||
transition: background-color .16s ease, box-shadow .16s ease;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-drop-target.is-active,
|
.file-list-drop-target.is-active,
|
||||||
.file-list-drop-target.is-drop-target {
|
.file-list-drop-target.is-drop-target {
|
||||||
background: var(--line);
|
background: var(--line);
|
||||||
box-shadow: inset 0 0 0 2px var(--line-dark);
|
box-shadow: inset 0 0 0 2px var(--line-dark);
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-table {
|
.files-page .file-list-table {
|
||||||
min-width: 760px;
|
min-width: 760px;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-table-head,
|
.files-page .file-list-table-head,
|
||||||
.file-list-row {
|
.files-page .file-list-row,
|
||||||
display: grid;
|
.managed-pattern-results .file-list-table-head,
|
||||||
|
.managed-pattern-results .file-list-row {
|
||||||
grid-template-columns: minmax(280px, 1fr) 120px 240px;
|
grid-template-columns: minmax(280px, 1fr) 120px 240px;
|
||||||
gap: 12px;
|
|
||||||
align-items: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-table-head {
|
|
||||||
padding: 10px 14px;
|
|
||||||
border-top: var(--border-line);
|
|
||||||
background: var(--panel);
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 12px;
|
|
||||||
font-weight: 800;
|
|
||||||
letter-spacing: .04em;
|
|
||||||
text-transform: uppercase;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-table-head button {
|
.file-list-table-head button {
|
||||||
@@ -158,24 +47,6 @@
|
|||||||
outline: none;
|
outline: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-row {
|
|
||||||
min-height: 58px;
|
|
||||||
padding: 9px 14px;
|
|
||||||
border-bottom: var(--border-line);
|
|
||||||
cursor: default;
|
|
||||||
user-select: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-row:focus-visible {
|
|
||||||
outline: 2px solid var(--line-dark);
|
|
||||||
outline-offset: -2px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-row:hover,
|
|
||||||
.file-list-row.is-selected {
|
|
||||||
background: var(--line);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-row.is-drop-target {
|
.file-list-row.is-drop-target {
|
||||||
background: var(--line);
|
background: var(--line);
|
||||||
box-shadow: inset 0 0 0 2px var(--line-dark);
|
box-shadow: inset 0 0 0 2px var(--line-dark);
|
||||||
@@ -191,43 +62,6 @@
|
|||||||
background: transparent;
|
background: transparent;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-name-cell {
|
|
||||||
min-width: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-name {
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 10px;
|
|
||||||
max-width: 100%;
|
|
||||||
min-width: 0;
|
|
||||||
border: 0;
|
|
||||||
background: transparent;
|
|
||||||
color: var(--text);
|
|
||||||
cursor: default;
|
|
||||||
font: inherit;
|
|
||||||
text-align: left;
|
|
||||||
padding: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-name > span {
|
|
||||||
display: grid;
|
|
||||||
min-width: 0;
|
|
||||||
gap: 2px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-name strong,
|
|
||||||
.file-list-name small {
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-name small {
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-linkage-status {
|
.file-linkage-status {
|
||||||
display: inline-flex;
|
display: inline-flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
@@ -238,31 +72,10 @@
|
|||||||
color: var(--text-strong);
|
color: var(--text-strong);
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-row-icon {
|
|
||||||
color: var(--muted);
|
|
||||||
flex: 0 0 auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.folder-row .file-row-icon {
|
.folder-row .file-row-icon {
|
||||||
color: var(--warning-deep);
|
color: var(--warning-deep);
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-row-tail {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
gap: 10px;
|
|
||||||
min-width: 0;
|
|
||||||
color: var(--muted);
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-empty {
|
|
||||||
padding: 36px 20px;
|
|
||||||
color: var(--muted);
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-list-virtual-spacer {
|
.file-list-virtual-spacer {
|
||||||
min-height: 0;
|
min-height: 0;
|
||||||
pointer-events: none;
|
pointer-events: none;
|
||||||
@@ -724,25 +537,27 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
@media (max-width: 1050px) {
|
@media (max-width: 1050px) {
|
||||||
.file-manager-shell {
|
.files-page .file-manager-shell {
|
||||||
grid-template-columns: 1fr;
|
grid-template-columns: 1fr;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-tree-panel {
|
.files-page .file-tree-panel {
|
||||||
max-height: 260px;
|
max-height: 260px;
|
||||||
border-right: 0;
|
border-right: 0;
|
||||||
border-bottom: var(--border-line);
|
border-bottom: var(--border-line);
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-table-head {
|
.files-page .file-list-table-head,
|
||||||
|
.managed-pattern-results .file-list-table-head {
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-table {
|
.files-page .file-list-table {
|
||||||
min-width: 0;
|
min-width: 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-list-row {
|
.files-page .file-list-row,
|
||||||
|
.managed-pattern-results .file-list-row {
|
||||||
grid-template-columns: 1fr;
|
grid-template-columns: 1fr;
|
||||||
gap: 8px;
|
gap: 8px;
|
||||||
}
|
}
|
||||||
@@ -752,23 +567,6 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
.file-manager-shell.is-loading .file-tree-panel,
|
|
||||||
.file-manager-shell.is-loading .file-list-panel {
|
|
||||||
filter: blur(1.5px);
|
|
||||||
pointer-events: none;
|
|
||||||
user-select: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-manager-loading-overlay {
|
|
||||||
position: absolute;
|
|
||||||
inset: 0;
|
|
||||||
z-index: 35;
|
|
||||||
display: grid;
|
|
||||||
place-items: center;
|
|
||||||
background: var(--panel-glass);
|
|
||||||
backdrop-filter: blur(1px);
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-conflict-summary {
|
.file-conflict-summary {
|
||||||
display: grid;
|
display: grid;
|
||||||
gap: 3px;
|
gap: 3px;
|
||||||
@@ -960,7 +758,7 @@
|
|||||||
border: 1px solid transparent;
|
border: 1px solid transparent;
|
||||||
border-radius: 9px;
|
border-radius: 9px;
|
||||||
background: transparent;
|
background: transparent;
|
||||||
color: var(--ink);
|
color: var(--text-strong);
|
||||||
text-align: left;
|
text-align: left;
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
}
|
}
|
||||||
@@ -1038,7 +836,7 @@
|
|||||||
border: 0;
|
border: 0;
|
||||||
border-radius: 6px;
|
border-radius: 6px;
|
||||||
background: transparent;
|
background: transparent;
|
||||||
color: var(--ink);
|
color: var(--text-strong);
|
||||||
padding: 5px 6px;
|
padding: 5px 6px;
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
}
|
}
|
||||||
@@ -1137,7 +935,7 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.managed-file-entry:disabled {
|
.managed-file-entry:disabled {
|
||||||
color: var(--ink);
|
color: var(--text-strong);
|
||||||
opacity: 1;
|
opacity: 1;
|
||||||
cursor: default;
|
cursor: default;
|
||||||
}
|
}
|
||||||
@@ -1245,7 +1043,7 @@
|
|||||||
border-right: 0;
|
border-right: 0;
|
||||||
border-left: 0;
|
border-left: 0;
|
||||||
background: transparent;
|
background: transparent;
|
||||||
color: var(--ink);
|
color: var(--text-strong);
|
||||||
text-align: left;
|
text-align: left;
|
||||||
font: inherit;
|
font: inherit;
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
@@ -1272,27 +1070,6 @@
|
|||||||
color: var(--muted);
|
color: var(--muted);
|
||||||
}
|
}
|
||||||
|
|
||||||
.split-field-action {
|
|
||||||
gap: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.split-field-action > input,
|
|
||||||
.split-field-action > select,
|
|
||||||
.split-field-action > textarea {
|
|
||||||
border-top-right-radius: 0 !important;
|
|
||||||
border-bottom-right-radius: 0 !important;
|
|
||||||
}
|
|
||||||
|
|
||||||
.split-field-action > button,
|
|
||||||
.split-field-action > .button,
|
|
||||||
.split-field-action > .btn {
|
|
||||||
align-self: stretch;
|
|
||||||
margin-left: -1px;
|
|
||||||
border-top-left-radius: 0;
|
|
||||||
border-bottom-left-radius: 0;
|
|
||||||
box-shadow: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
@media (max-width: 850px) {
|
@media (max-width: 850px) {
|
||||||
.managed-file-entry-head,
|
.managed-file-entry-head,
|
||||||
.managed-file-entry {
|
.managed-file-entry {
|
||||||
|
|||||||
Reference in New Issue
Block a user