Pin S3 and SMB connector peers

docs/CONNECTOR_BOUNDARY.md

@@ -19,6 +19,7 @@ normal product workflows and can share the same governance model:
 - Nextcloud through WebDAV
 - generic WebDAV
 - SMB through the optional `smb` extra
+- S3-compatible stores through the optional `s3` extra
 
 These providers are surfaced through connector descriptors at
 `GET /api/v1/files/connectors/providers`. Provider descriptors declare whether
@@ -60,11 +61,14 @@ Every provider must:
 - use a transport that pins every connection to a policy-validated DNS/IP answer
   and revalidates redirects; SDK transports without that guarantee fail closed
 
-Live SMB access is disabled in all modes until `smbprotocol` initial connections
-and DFS referral targets can be pinned and policy-validated. An explicit IP is
-not sufficient because the server may still issue a referral to another peer.
-Live S3 access is likewise disabled until `boto3`/botocore can be bound to the
-pinned transport, including SDK-managed redirects and endpoint discovery.
+SMB initial connections, reconnects, aliases, and DFS referral targets are
+created through a Files-owned pinned `smbprotocol` transport. S3 HTTP/HTTPS pools
+use the equivalent botocore adapter for every connection selected by retries,
+redirects, endpoint discovery, and virtual-host addressing. Both adapters apply
+the deployment-wide private-network policy at socket creation. S3 keeps the
+configured hostname for TLS SNI and certificate verification, but does not use
+outbound proxies or ambient AWS credential discovery. An incompatible optional
+SDK release fails closed before a usable session or client is returned.
 
 ## Non-Goals For Files
 

docs/FILES_HANDBOOK.md

@@ -405,15 +405,15 @@ in a development/test runtime.
 | Seafile | Read-only native API browse/download-link import and manual sync implemented using the pinned HTTP transport; WebDAV opt-in supported |
 | Nextcloud | Read-only WebDAV browse/import/manual sync implemented using the pinned HTTP transport |
 | Generic WebDAV | Read-only browse/import/manual sync implemented using the pinned HTTP transport |
-| SMB | Browse/import code and descriptor implemented, but every live connection fails closed until initial connections and DFS referrals can be policy-validated and pinned |
-| S3 connector | Browse/import code and descriptor implemented, but every live SDK connection fails closed until botocore connections, redirects, and endpoint discovery can be validated and pinned |
+| SMB | Read-only browse/import/manual sync implemented through a pinned smbprotocol transport for initial peers, reconnects, aliases, and DFS referral targets |
+| S3 connector | Read-only bucket/prefix browse, import, and manual sync implemented through pinned botocore pools covering retries, redirects, endpoint discovery, and provider aliases |
 | SharePoint and OneDrive | Provider keys/descriptors reserved; live Microsoft Graph browse/import is planned |
 | NFS and local connector | Described as optional future providers; the local managed-storage backend is a different feature |
 
 Provider descriptors are available from
 `GET /api/v1/files/connectors/providers`. Use their `implemented`, `installed`,
-and support fields for display, but expect a fail-closed transport error where
-the table above says live access is disabled.
+and support fields for display. An incompatible optional SDK release fails closed
+before it can return a usable client or session.
 
 ## Operator runbook
 
@@ -478,17 +478,29 @@ The built-in HTTP transport:
 - bounds structured responses to 16 MiB and file transfers to 512 MiB by
   default.
 
+The S3 SDK adapter applies the same socket rule to every botocore pool selected
+for a retry, redirect, discovered endpoint, or virtual-host bucket alias. The
+original authority remains in the request and TLS SNI/certificate check. S3
+connector clients use no outbound proxy and never discover ambient AWS
+credentials: configure both access and secret keys on the governed profile, or
+use an anonymous profile for a public source.
+
+The SMB adapter owns a separate connection cache and replaces smbprotocol's TCP
+factory process-wide with the stricter pinned socket. Initial peers, reconnects,
+server aliases, domain-controller connections, and DFS referral targets therefore
+pass the same policy at connection time. Signing is required by default; enable
+SMB encryption on the profile where the server supports it.
+
 Override the connector response limits with
 `GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES` and
 `GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES`. The smaller applicable limit wins
 when an import is also subject to `FILE_UPLOAD_MAX_BYTES`.
 
 Never work around a connector pinning failure by adding a raw IP, disabling TLS,
-or enabling private networks. SMB may redirect through DFS, and user-configured
-S3 connectors may perform their own redirects or endpoint discovery; those
-connector transports remain disabled until every connection peer can be
-governed. The separately configured platform S3 backend is trusted only by the
-deployment owner and is not selectable by a user or connector profile.
+or enabling private networks. A failure means the peer policy rejected an actual
+connection destination or the installed SDK no longer exposes the verified
+transport seam. The separately configured platform S3 backend is trusted only by
+the deployment owner and is not selectable by a user or connector profile.
 
 ### Backup and restore
 
@@ -620,7 +632,7 @@ Use these symptoms as routing hints:
 | `Stored object does not exist` | Database/blob restore mismatch, wrong root, or missing shared storage |
 | `Stored secret cannot be decrypted` | Wrong or rotated `MASTER_KEY_B64` |
 | Private/non-public endpoint blocked | Deployment-wide egress policy is public-only or DNS returned a forbidden answer |
-| SDK cannot pin redirects/referrals | Expected fail-closed S3/SMB boundary, not a transient connector outage |
+| SDK peer-pinning seam is unavailable | Optional S3/SMB SDK is incompatible; keep access fail-closed and validate the supported dependency range before upgrade |
 | Connector response exceeds limit | Remote payload exceeds connector or upload limit |
 | Profile is not visible | Scope, disabled state, campaign access, or policy mismatch |
 | Group removal is vetoed | The group still owns a file/folder/connector space or is a share target |
@@ -824,9 +836,11 @@ bypass the same rule.
 ### Pinned connector transport
 
 Given public-only mode and DNS returns any private address, the connection must
-be rejected before a socket opens. Given private mode, the HTTP connection must
-still use the validated address and reject redirects. SMB and S3 must fail
-before their SDK clients connect until all SDK-managed peers can be pinned.
+be rejected before a socket opens. Given private mode, every HTTP, S3, and SMB
+connection must still use an address from the answer validated for that exact
+attempt. Botocore retries, redirects, endpoint discovery, and aliases, plus SMB
+reconnects and DFS referrals, must pass through the pinned factories. A changed
+or unsupported SDK seam must fail before a usable client/session is returned.
 
 ### Imported evidence and sync
 
@@ -867,7 +881,7 @@ returning different content or credentials.
 | Deletion/retention | Soft-delete assets/folders/spaces; immediate audited connector-secret scrubbing | File restore API, hard purge, retention policy, legal hold, and blob GC ([#38](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/38)) |
 | Connector governance | Scoped profiles/credentials/policies, effective source explanation, separate credentials, linked user/group spaces | Provider-owned external secret lifecycle; API `secret_ref` remains rejected |
 | HTTP connectors | Pinned, bounded, no-redirect Seafile and WebDAV/Nextcloud browse/import/manual sync | Background/folder sync, remote mutation, long-running transfer workers |
-| SMB and S3 connectors | Provider descriptors and browse/import logic | Live access only after SDK connections plus DFS referrals/redirects are validated and pinned ([SMB #35](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/35), [S3 #34](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/34)) |
+| SMB and S3 connectors | Provider descriptors, browse/import/manual sync, pinned SDK transports, and redirect/retry/referral transport-contract tests | Live topology smoke evidence, provider-specific OAuth, remote writes, and background indexing remain separate deployment or connector-module concerns |
 | Other providers | Reserved SharePoint/OneDrive keys and NFS/local descriptors | Graph/OAuth/provider paging, NFS deployment integration, DMS connectors |
 | Connector spaces | User/group link, browse, manual selected-file sync, edit/disable/delete | Background sync, remote writes/deletes, full conflict-reporting jobs |
 | Profile capabilities | Stored and displayed | Enforce capability flags as an independent operation gate |
@@ -890,8 +904,8 @@ Before releasing Files:
 6. Exercise upload, ZIP bounds, conflict handling, download, and soft deletion.
 7. Exercise connector policy explanation and one pinned HTTP provider where
    configured; verify the deployment-managed or trusted external S3 backend if
-   selected, and verify user-configured S3 and SMB connector peers still fail
-   closed.
+   selected, and verify one configured S3 and SMB connector while recording the
+   actual target topology and private-network policy.
 8. Verify credential deletion scrubs dependents and produces audit evidence.
 9. Verify a campaign attachment snapshot still identifies its exact version and
    checksum after the current file changes.