82 lines
5.1 KiB
Markdown
82 lines
5.1 KiB
Markdown
# Disposable resource-bounded operations
|
||
|
||
`security.bounded_process.run_bounded_operation` runs a trusted, importable,
|
||
module-level `bytes -> bytes` function in a fresh interpreter. Core owns the
|
||
process lifecycle, not the business parser. Owners retain authorization,
|
||
sessions, provider reads, idempotency and persistence in the parent and pass
|
||
only explicit bounded data. Never accept the operation, module or source path
|
||
from a client. Use `security.worker_payload` for typed values; it does not use
|
||
pickle, arbitrary constructors or JSON object hooks.
|
||
|
||
The runner requires POSIX process groups, `waitid(WNOWAIT)` and resource limits.
|
||
Unsupported controls fail closed; there is no in-process fallback. The child
|
||
uses `-I -B`, a fixed minimal environment, `/` as working directory, closed
|
||
inherited descriptors and a new process session. Limits are installed before
|
||
the owning module is imported. Installed dependencies must support isolated
|
||
Python imports; development `PYTHONPATH` alone is insufficient.
|
||
|
||
`ProcessLimits` specifies wall-clock seconds (including child startup), CPU
|
||
seconds, virtual address space, input/output pipe bytes and maximum regular-file
|
||
size. Defaults are 10 seconds wall/CPU, 256 MiB address space, 8 MiB input and
|
||
output, and no regular-file output. Wall/CPU limits are at most 600 seconds,
|
||
memory 64 MiB–8 GiB, pipe limits 1 byte–256 MiB, and file size 0–2 GiB. Owners
|
||
must document their tighter functional limits; a transport cap does not replace
|
||
row, archive expansion, item-count or artifact limits.
|
||
|
||
Admission is non-queuing. `GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` defaults to 1
|
||
(range 1–16) and applies across these operations **within each API/worker
|
||
process**. Multiply capacity and memory budgets by the number of API/worker
|
||
processes when sizing an installation. This is not a fleet-wide semaphore,
|
||
cgroup quota, filesystem/network sandbox or permission to run arbitrary code.
|
||
`RLIMIT_FSIZE` is per file, not a total disk quota. Owners creating staged files
|
||
must enforce cumulative quotas and clean up their own private directories.
|
||
|
||
Owners preparing bounded local snapshots can enter
|
||
`bounded_operation_admission()` before preparation and pass its token as
|
||
`admission=` to the runner. This reuses shared capacity rather than reserving a
|
||
second slot. Tokens belong to their active context, thread and process; expired,
|
||
cross-thread and overlapping reuse fail. Preparation exceptions release the
|
||
slot without launching a child. Never hold admission while waiting for a user;
|
||
parent-side preparation still requires explicit I/O and byte bounds.
|
||
|
||
The parent concurrently drains stdout/stderr while writing input. Output is
|
||
bounded during reading, stderr is discarded and capped at 64 KiB, and raw child
|
||
tracebacks are never returned. Every success, exception, timeout, cancellation
|
||
and callback failure kills the owned process group before reaping its leader,
|
||
including descendants which close their inherited pipes. A module-level child
|
||
handler must return bytes; it must not print logs/progress to stdout.
|
||
|
||
The optional `cancelled` callback runs in the parent at most roughly every
|
||
50 ms while waiting. It must be fast and must not return an awaitable. It may
|
||
also service a module-owned bounded progress protocol; exceptions terminate
|
||
the child and propagate. There is no fabricated progress for killed work.
|
||
`ProcessBudgetError.code` distinguishes busy, cancelled, timeout, CPU, memory,
|
||
input/output limits, unavailable controls and worker failure. Owners map these
|
||
to their existing structured diagnostics and recovery semantics.
|
||
|
||
The private typed-data codec supports null, booleans, strings, bytes, integers,
|
||
floats, Decimal, UUID, date/time/datetime, lists, tuples and string-keyed maps.
|
||
It rejects unsupported objects, malformed/trailing bytes, duplicate keys,
|
||
excess depth (64) and node counts (1,000,000). Operation DTOs remain owner
|
||
contracts and require owner validation. Do not persist this private wire format
|
||
or use it as a public API.
|
||
|
||
Tests use real child processes for catastrophic regex, memory exhaustion,
|
||
noisy output, exact limits, cancellation, closed-pipe hangs and descendant
|
||
cleanup. These are local process regression tests, not production concurrent
|
||
load certification. Operators still need target Linux/cgroup, cancellation,
|
||
worker-count, memory and disk-quota evidence before raising concurrency.
|
||
|
||
## Deutsche Betriebszusammenfassung
|
||
|
||
Rechenintensive, vertrauenswürdige Moduloperationen laufen in einem frischen
|
||
Prozess mit harten Laufzeit-, CPU-, Speicher- und Ausgabegrenzen. Berechtigungen,
|
||
Sitzungen, Zugangsdaten und Datenbankänderungen bleiben im Hauptprozess. Fehlende
|
||
Betriebssystemkontrollen führen zu einer Diagnose, nicht zu ungeschützter
|
||
Ausführung. `GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` begrenzt die gemeinsame
|
||
Zulassung je API-/Worker-Prozess, standardmäßig auf 1. Mehrere Prozesse haben
|
||
jeweils eigene Grenzen; systemweite Speicher- und Festplattenquoten müssen
|
||
Betreiber zusätzlich konfigurieren und auf der Zielinstallation prüfen. Die
|
||
Schnittstelle ist keine Sandbox für beliebigen Code. Modul-Dokumentation nennt
|
||
die jeweiligen fachlichen Grenzen, Fortschritts- und Wiederholungsregeln.
|