feat(files): orchestrate connector folder sync
Module Package Release / publish-packages (push) Successful in 12s
Module Package Release / publish-packages (push) Successful in 12s
This commit is contained in:
+34
-7
@@ -107,8 +107,11 @@ Files resources when their owner is specified. Linked connector spaces appear
|
||||
beside managed spaces when they are active and visible to the user.
|
||||
|
||||
A connector space is read-only by default. It is a view of an approved remote
|
||||
location and a starting point for importing or synchronizing selected files
|
||||
into managed storage. An administrator may opt an S3 space into two-way mode
|
||||
location and a starting point for importing a selected file or synchronizing a
|
||||
selected file or bounded folder tree into managed storage. Folder sync preserves
|
||||
relative remote paths in the linked owner's managed space and returns a
|
||||
reviewable result for every discovered or deliberately skipped item. An
|
||||
administrator may opt an S3 space into two-way mode
|
||||
only after enabling the profile's `write` capability. Even then, Files exposes
|
||||
explicit, conditional file write-back—not a mounted filesystem. Automatic
|
||||
remote delete, rename, move, and ACL propagation stay disabled.
|
||||
@@ -281,6 +284,19 @@ inside the chosen owner space:
|
||||
- identical checksum and size updates provenance and returns `unchanged`;
|
||||
- changed bytes append a version and return `updated`.
|
||||
|
||||
Folder-level manual sync uses
|
||||
`POST /api/v1/files/connector-spaces/{space_id}/sync`. The interactive request
|
||||
is bounded by file count and traversal depth, follows provider pagination, and
|
||||
can include or exclude subfolders. Every file is isolated in a database
|
||||
savepoint, so one conflict, policy denial, invalid provider result, or transport
|
||||
failure remains reviewable without discarding successful siblings. Results are
|
||||
classified as `created`, `updated`, `unchanged`, `skipped`, `conflict`,
|
||||
`policy_denied`, or `failed`. The UI defaults unrelated target collisions to
|
||||
`skip`; `rename`, `reject`, and `overwrite` require an explicit operator choice.
|
||||
The response marks a run as truncated when its bound is reached so the operator
|
||||
can continue with a narrower remote folder. This is interactive orchestration,
|
||||
not background scheduling.
|
||||
|
||||
Browse, import, and inbound sync never mutate the remote source. An S3 space in
|
||||
explicit two-way mode can write a selected managed file to one remote object
|
||||
path through `POST /api/v1/files/connector-spaces/{space_id}/write-back`.
|
||||
@@ -394,7 +410,7 @@ those relationships first.
|
||||
| --- | --- |
|
||||
| `files:file:read` | List and inspect accessible files, folders, spaces, and visible connectors |
|
||||
| `files:file:download` | Download an accessible current version or ZIP archive |
|
||||
| `files:file:upload` | Upload managed assets and import/sync selected connector files |
|
||||
| `files:file:upload` | Upload managed assets and import/sync selected connector files or bounded connector folders |
|
||||
| `files:file:organize` | Create folders, rename, move/copy, and manage linked connector spaces |
|
||||
| `files:file:share` | Create or update file shares |
|
||||
| `files:file:delete` | Soft-delete accessible writable files and folders |
|
||||
@@ -813,7 +829,7 @@ All routes below are under `/api/v1/files`.
|
||||
| File access | `GET /{file_id}`, `GET /{file_id}/download`, `DELETE /{file_id}`, `POST /bulk-delete` |
|
||||
| Organization | `POST /bulk-rename`, `POST /transfer`, `POST /archive.zip`, `POST /resolve-patterns` |
|
||||
| Sharing | `POST /{file_id}/shares`, `POST /bulk-shares` |
|
||||
| Connector spaces | `GET/POST /connector-spaces`, `PATCH/DELETE /connector-spaces/{space_id}`, `POST /connector-spaces/{space_id}/restore`, `POST /connector-spaces/{space_id}/write-back` |
|
||||
| Connector spaces | `GET/POST /connector-spaces`, `PATCH/DELETE /connector-spaces/{space_id}`, `POST /connector-spaces/{space_id}/restore`, `POST /connector-spaces/{space_id}/sync`, `POST /connector-spaces/{space_id}/write-back` |
|
||||
| Connector catalog/discovery | `GET /connectors/providers`, `POST /connectors/discover` |
|
||||
| Connector profiles | `GET/POST /connectors/profiles`, `GET/PATCH/DELETE /connectors/profiles/{profile_id}` |
|
||||
| Browse/import/sync | `GET /connectors/profiles/{profile_id}/browse`, `POST /connectors/profiles/{profile_id}/import`, `POST /connectors/profiles/{profile_id}/sync` |
|
||||
@@ -916,7 +932,7 @@ events. External metadata is provider/user input and is not a digital signature.
|
||||
Files records canonical audit events for:
|
||||
|
||||
- connector discovery attempts, before the attempted external I/O;
|
||||
- connector imports and manual syncs;
|
||||
- connector imports, selected-file syncs, and bounded folder sync summaries;
|
||||
- download/archive access to connector-originated managed files;
|
||||
- immediate connector profile and credential deletion/scrubbing;
|
||||
- credential scrubbing during destructive module retirement.
|
||||
@@ -924,6 +940,10 @@ Files records canonical audit events for:
|
||||
Connector audit details include the managed asset/version/blob, checksum, size,
|
||||
operation, source revision, and provenance where applicable. Deletion audit
|
||||
details name secret/reference kinds but never the secret values.
|
||||
`files.connector.folder_synced` additionally records the connector profile,
|
||||
provider, library/root, managed target, traversal mode, truncation state, typed
|
||||
counts, and bounded per-item source/target/action references. It does not embed
|
||||
downloaded bytes, credentials, or transport content.
|
||||
|
||||
Assets, folders, shares, profiles, credentials, policies, and connector spaces
|
||||
also feed Core's incremental change sequence for UI synchronization. A change
|
||||
@@ -1021,6 +1041,13 @@ must carry source identity, revision when available, current version, checksum,
|
||||
and size. Re-syncing identical bytes must return `unchanged`; changed bytes must
|
||||
create a higher version while preserving the previous version.
|
||||
|
||||
Given a permitted linked connector folder contains changed, unchanged,
|
||||
colliding, and policy-denied files, a manual folder sync must preserve remote
|
||||
relative paths, commit the allowed siblings, return one typed outcome for every
|
||||
discovered/skipped item, and emit a matching summary audit event. Reaching the
|
||||
configured file bound must mark the run truncated rather than imply complete
|
||||
coverage.
|
||||
|
||||
### Campaign freeze
|
||||
|
||||
Given a campaign snapshot selected version V1, when the managed asset later
|
||||
@@ -1053,10 +1080,10 @@ returning different content or credentials.
|
||||
| Deletion/retention | Soft-delete and restore assets/folders/spaces; optimistic retention and legal-hold controls; preview-bound, approval-referenced hard purge; reference-checked blob GC; immediate audited connector-secret scrubbing | Automatic time-based purge scheduling and richer lifecycle administration UI |
|
||||
| Privacy requests | Tenant-scoped bounded DSAR metadata search; retained/manual/revoke/detach planning; idempotent share revocation and mutable actor-reference detachment; explicit separation from byte purge | Content-specific automated redaction and policy-specific approval remain manual or belong to the owning process |
|
||||
| Connector governance | Scoped profiles/credentials/policies, effective source explanation, separate credentials, linked user/group spaces | Provider-owned external secret lifecycle; API `secret_ref` remains rejected |
|
||||
| HTTP connectors | Pinned, bounded, no-redirect Seafile and WebDAV/Nextcloud browse/import/manual sync | Background/folder sync, remote mutation, long-running transfer workers |
|
||||
| HTTP connectors | Pinned, bounded, no-redirect Seafile and WebDAV/Nextcloud browse/import, selected-file sync, and folder sync | Background sync, remote mutation, long-running transfer workers |
|
||||
| SMB and S3 connectors | Provider descriptors, browse/import/manual sync, pinned SDK transports, redirect/retry/referral transport-contract tests, and explicit conditional S3 write-back with Core-ledger recovery | Live topology smoke evidence, provider-specific OAuth, additional provider writes, and background indexing remain separate deployment or connector-module concerns |
|
||||
| Other providers | Reserved SharePoint/OneDrive keys and NFS/local descriptors | Graph/OAuth/provider paging, NFS deployment integration, DMS connectors |
|
||||
| Connector spaces | User/group link, browse, manual selected-file sync, edit/disable/delete/restore, read-only default, and opt-in S3 two-way mode | Background sync and automatic remote delete/rename/move/ACL propagation |
|
||||
| Connector spaces | User/group link, browse, bounded manual file/folder sync, edit/disable/delete/restore, read-only default, and opt-in S3 two-way mode | Background sync and automatic remote delete/rename/move/ACL propagation |
|
||||
| Profile capabilities | Stored, displayed, and enforced for explicit connector write-back | Broader provider-specific capability negotiation |
|
||||
| Audit | Connector discovery/import/sync/access and connector deletion; campaign exact-use evidence | Dedicated canonical audit events for every ordinary Files mutation |
|
||||
| Preview | File metadata and attachment download | Dedicated safe content-preview service |
|
||||
|
||||
Reference in New Issue
Block a user