fix(files): isolate archive workers and make handoff event driven

This commit is contained in:
2026-09-08 07:47:18 +02:00
parent ff84812f7f
commit 13433514b9
11 changed files with 945 additions and 86 deletions
+48
View File
@@ -682,6 +682,54 @@ and restart workers after installation. Both paths retain complete header and
path validation, selection and actual output limits; the native path also
independently checks size and CRC and never extracts to filesystem paths.
All ZIP/TAR metadata parsing (including ZIP central directories and TAR PAX
headers) and decoding now run in fresh, credential-stripped child processes.
Preview/password checks have 120 seconds wall time, 90 CPU seconds and 512 MiB
address space. Confirmation uses one child for the entire archive, with 600
seconds wall time and 300 CPU seconds. Its address-space limit is the larger of
512 MiB or `3 × configured member limit + 128 MiB`. A member limit must be positive
and at most 2 GiB; the default 50 MiB member limit and the existing 250 MiB request,
10,000-entry, 2 GiB expanded and 100:1 limits are unchanged. The kernel file-size
ceiling is `max(member limit, 16 KiB)`; actual member/cumulative bytes are checked
independently. Metadata transport has a 64 MiB ceiling, final extraction receipts
64 KiB, and status records 16 KiB. Raw archive/member bytes are not pipe DTOs.
`GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` is shared with other isolated module work:
one admitted operation per API/worker process by default, configurable from one
to 16. Admission covers source snapshot preparation before starting the child,
so a busy request does not first copy its source. Capacity is not global across
replicas. Include each replica's address-space and temporary-disk budgets when
sizing the deployment. Resource controls are required; there is no inline parser
fallback. These are process resource limits, not a filesystem/network sandbox or
an aggregate cgroup memory guarantee.
Each admitted operation creates a separate local OS-temporary `0700` directory
and `0600` source snapshot, beyond the upload-once work-root quota described
above. Plan additional capacity for one compressed source copy and at most one
extracted member per admitted archive. Snapshot copying is bounded to the initial
regular-file size and rejects changed sources. The staging wrapper writes only
numbered internal filenames, never archive names. The parent rechecks selected
logical paths, regular/no-symlink file identity, size, digest and monotonic
progress, persists the member with the existing Files authority/transaction, and
deletes it before acknowledging the child to decode the next member. A private
`0600` FIFO carries one eight-byte sequence acknowledgement; completed members
wake the parent immediately, with no fixed per-member sleep. Wake notifications
are capped at 32 KiB; configurations allowing more than 32,768 members use Core's
ordinary progress polling after that budget, still within the wall-time limit.
Passwords remain request-only and are never written to these files. Handled success/error
paths reap the child and remove the private directory. Abrupt host/parent loss
can leave private temporary files; use the deployment's OS-temporary cleanup
policy without deleting active work.
The confirmation wall budget includes time waiting for parent storage; a slow
destination may exhaust it after earlier members were staged for persistence.
Existing rollback and blob recovery remain authoritative. Busy capacity, missing
controls, CPU/memory/time/output limits, or invalid staging records produce a
controlled failure; split the archive or explicitly retry after resolving the
cause. There is no resumable worker or automatic confirmation retry. An optional
UI progress-write failure remains non-fatal, but a failed private worker status
or acknowledgement channel must abort safely.
Verified members are read and stored one at a time, avoiding a whole expanded
archive in memory. Storage-client reuse is confined to one archive operation;
authorization, tenant isolation and policy are not cached. Batched response