fix(files): isolate archive workers and make handoff event driven
This commit is contained in:
@@ -682,6 +682,54 @@ and restart workers after installation. Both paths retain complete header and
|
||||
path validation, selection and actual output limits; the native path also
|
||||
independently checks size and CRC and never extracts to filesystem paths.
|
||||
|
||||
All ZIP/TAR metadata parsing (including ZIP central directories and TAR PAX
|
||||
headers) and decoding now run in fresh, credential-stripped child processes.
|
||||
Preview/password checks have 120 seconds wall time, 90 CPU seconds and 512 MiB
|
||||
address space. Confirmation uses one child for the entire archive, with 600
|
||||
seconds wall time and 300 CPU seconds. Its address-space limit is the larger of
|
||||
512 MiB or `3 × configured member limit + 128 MiB`. A member limit must be positive
|
||||
and at most 2 GiB; the default 50 MiB member limit and the existing 250 MiB request,
|
||||
10,000-entry, 2 GiB expanded and 100:1 limits are unchanged. The kernel file-size
|
||||
ceiling is `max(member limit, 16 KiB)`; actual member/cumulative bytes are checked
|
||||
independently. Metadata transport has a 64 MiB ceiling, final extraction receipts
|
||||
64 KiB, and status records 16 KiB. Raw archive/member bytes are not pipe DTOs.
|
||||
|
||||
`GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` is shared with other isolated module work:
|
||||
one admitted operation per API/worker process by default, configurable from one
|
||||
to 16. Admission covers source snapshot preparation before starting the child,
|
||||
so a busy request does not first copy its source. Capacity is not global across
|
||||
replicas. Include each replica's address-space and temporary-disk budgets when
|
||||
sizing the deployment. Resource controls are required; there is no inline parser
|
||||
fallback. These are process resource limits, not a filesystem/network sandbox or
|
||||
an aggregate cgroup memory guarantee.
|
||||
|
||||
Each admitted operation creates a separate local OS-temporary `0700` directory
|
||||
and `0600` source snapshot, beyond the upload-once work-root quota described
|
||||
above. Plan additional capacity for one compressed source copy and at most one
|
||||
extracted member per admitted archive. Snapshot copying is bounded to the initial
|
||||
regular-file size and rejects changed sources. The staging wrapper writes only
|
||||
numbered internal filenames, never archive names. The parent rechecks selected
|
||||
logical paths, regular/no-symlink file identity, size, digest and monotonic
|
||||
progress, persists the member with the existing Files authority/transaction, and
|
||||
deletes it before acknowledging the child to decode the next member. A private
|
||||
`0600` FIFO carries one eight-byte sequence acknowledgement; completed members
|
||||
wake the parent immediately, with no fixed per-member sleep. Wake notifications
|
||||
are capped at 32 KiB; configurations allowing more than 32,768 members use Core's
|
||||
ordinary progress polling after that budget, still within the wall-time limit.
|
||||
Passwords remain request-only and are never written to these files. Handled success/error
|
||||
paths reap the child and remove the private directory. Abrupt host/parent loss
|
||||
can leave private temporary files; use the deployment's OS-temporary cleanup
|
||||
policy without deleting active work.
|
||||
|
||||
The confirmation wall budget includes time waiting for parent storage; a slow
|
||||
destination may exhaust it after earlier members were staged for persistence.
|
||||
Existing rollback and blob recovery remain authoritative. Busy capacity, missing
|
||||
controls, CPU/memory/time/output limits, or invalid staging records produce a
|
||||
controlled failure; split the archive or explicitly retry after resolving the
|
||||
cause. There is no resumable worker or automatic confirmation retry. An optional
|
||||
UI progress-write failure remains non-fatal, but a failed private worker status
|
||||
or acknowledgement channel must abort safely.
|
||||
|
||||
Verified members are read and stored one at a time, avoiding a whole expanded
|
||||
archive in memory. Storage-client reuse is confined to one archive operation;
|
||||
authorization, tenant isolation and policy are not cached. Batched response
|
||||
|
||||
Reference in New Issue
Block a user