2 Commits
Author SHA1 Message Date
zemion 5c9802eb4e fix(ui): align contextual documentation with headings
Verified with the coordinated workspace changes by devkit full run
2026-09-08T225814-186389-0000-3e3ed7cd (all seven phases passed).
This shared UI pass does not mark the individual module reviews complete.
2026-09-09 02:04:22 +02:00
zemion 1b1e3e0f92 fix(security): isolate template rendering with hard resource limits 2026-09-08 07:47:18 +02:00
7 changed files with 240 additions and 5 deletions
+18
View File
@@ -36,3 +36,21 @@ Apply the module Alembic migration before startup. Monitor rejected renders for
contract drift, output limits, missing Files permission, and reused idempotency contract drift, output limits, missing Files permission, and reused idempotency
keys. HTML is designed for browser/OS printing; do not treat it as a signed PDF keys. HTML is designed for browser/OS printing; do not treat it as a signed PDF
or proof of physical printer delivery. or proof of physical printer delivery.
Pure rendering runs in a disposable Core worker after authorization, revision
selection and idempotency lookup. Each worker has a 15-second wall limit,
10 CPU seconds, 512 MiB address space and no file output. Typed input transport
is limited to 32 MiB and result transport to 8 MiB, including serialization
overhead; nesting is limited to 64 levels and 1,000,000 value nodes. The existing
5 MiB final output and 5,000-item limits remain. Only data DTOs cross the
boundary; principals, SQL sessions, credentials and artifact writes stay in the
parent. Completed idempotent renders return before a worker is started.
`GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` in Core controls active isolated work per
API/worker process without queuing. Busy capacity produces a sanitized retryable
render error. CPU, memory, transport or time failures stop the entire render
before artifact persistence; reduce the workload before retrying. POSIX process
resource controls are required. Missing controls, cancellation or worker failure
fails closed with no in-process fallback. Budget aggregate memory across all
slots and API/worker replicas; monitor limit and overload errors separately
from template compatibility errors.
+10
View File
@@ -21,3 +21,13 @@ Open **Templates** to create or select a reusable definition.
Render evidence shows the exact revision and abbreviated template, input, and Render evidence shows the exact revision and abbreviated template, input, and
output hashes. A consumer such as Campaign can submit many frozen recipients; output hashes. A consumer such as Campaign can submit many frozen recipients;
the UI sample intentionally validates one representative item. the UI sample intentionally validates one representative item.
Rendering is also limited to 15 seconds elapsed time, 10 CPU seconds, 512 MiB
process memory, 32 MiB serialized input and 8 MiB serialized result transport;
the final output remains limited to 5 MiB and 5,000 items. Deeply nested data
or more than 1,000,000 transported value nodes is rejected. A limit error creates
no partial output or render evidence: reduce the selected input or template
complexity and retry. If processing capacity is busy, retry later. Unavailable
worker controls or a worker failure require operator investigation. Existing
successful output with the same idempotency key is reused without rendering
again, provided its input and authorization still match.
+44 -2
View File
@@ -102,11 +102,50 @@ ROLE_TEMPLATES = (
) )
DOCUMENTATION = ( DOCUMENTATION = (
DocumentationTopic(
id="templates.render-worker-limits",
title="Rendering resource and capacity limits",
summary="Render in a disposable process and reject incomplete output before persistence.",
body=(
"Template substitution and document composition run in a fresh process with a 15-second wall limit, 10 CPU seconds, "
"512 MiB address space, no file output, 32 MiB typed input and 8 MiB typed result transport. The existing final output "
"limit remains 5 MiB and the item limit remains 5,000. Transport permits only explicit data types, at most 64 nesting "
"levels and 1,000,000 value nodes. Exceeding a limit fails the entire render before Files or render evidence is written; "
"reduce the input or template complexity and retry. Core GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY limits simultaneous work per "
"API/worker process without queuing; a busy error can be retried later. Missing POSIX resource controls, cancellation "
"and worker failures are explicit sanitized render errors, with no in-process fallback. Authorization, revision "
"selection, idempotency lookup, SQL sessions and artifact persistence remain in the parent. Existing completed "
"idempotent output is reused before starting a worker. Operators must budget aggregate memory across process slots."
),
layer="static",
documentation_types=("user", "admin"),
audience=("user", "module_admin", "operator"),
order=30,
translations={"de": {
"title": "Ressourcen- und Kapazitätsgrenzen beim Rendern",
"summary": "In einem kurzlebigen Prozess rendern und unvollständige Ausgaben vor dem Speichern ablehnen.",
"body": (
"Platzhalterersetzung und Dokumentaufbau laufen in einem neuen Prozess mit höchstens 15 Sekunden Gesamtdauer, "
"10 CPU-Sekunden, 512 MiB Adressraum, keiner Dateiausgabe sowie 32 MiB typisierter Eingabe und 8 MiB Ergebnistransport. "
"Die endgültige Ausgabe bleibt auf 5 MiB und die Elementzahl auf 5.000 begrenzt. Der Transport erlaubt nur ausdrückliche "
"Datentypen, höchstens 64 Verschachtelungsebenen und 1.000.000 Wertknoten. Eine Grenzverletzung bricht den gesamten "
"Vorgang vor dem Speichern in Files oder als Render-Nachweis ab; reduzieren Sie Eingaben oder Vorlagenkomplexität "
"und versuchen Sie es erneut. Core GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY begrenzt gleichzeitige Arbeiten pro API-/Worker-Prozess "
"ohne Warteschlange; bei belegter Kapazität kann später erneut versucht werden. Fehlende POSIX-Ressourcenbegrenzungen, "
"Abbruch und Worker-Fehler sind ausdrückliche bereinigte Render-Fehler ohne Rückfall auf den Elternprozess. "
"Berechtigungsprüfung, Revisionsauswahl, Idempotenzprüfung, SQL-Sitzungen und Artefaktspeicherung bleiben im Elternprozess. "
"Bereits abgeschlossene idempotente Ausgaben werden vor einem Worker-Start wiederverwendet. Betreiber müssen den "
"Gesamtspeicher aller Prozessplätze berücksichtigen."
),
}},
),
DocumentationTopic( DocumentationTopic(
id="templates.workspace-layout", id="templates.workspace-layout",
title="Templates workspace actions", title="Templates workspace actions",
summary="Find collection-wide commands in their consistent workspace position.", summary="Find collection-wide commands in their consistent workspace position.",
body="Reload and Add template use the persistent full-width workspace header at the upper right; Reload sits immediately before creation. Selecting a record, changing filters, or opening an editor does not move these collection-wide commands into the left pane. Template editing, saving, publishing, and rendering remain scoped to the selected template. Existing permissions, disabled-state rules, and unsaved-change guards still apply. Administrators configure authority through the existing permission system; no new permission or automatic operation is introduced.", body="Documentation books sit immediately beside the visible heading or contextual label for "
"Templates, not among operational action buttons. Field help remains beside its label. "
"Reload and Add template use the persistent full-width workspace header at the upper right; Reload sits immediately before creation. Selecting a record, changing filters, or opening an editor does not move these collection-wide commands into the left pane. Template editing, saving, publishing, and rendering remain scoped to the selected template. Existing permissions, disabled-state rules, and unsaved-change guards still apply. Administrators configure authority through the existing permission system; no new permission or automatic operation is introduced.",
layer="static", layer="static",
documentation_types=("user", "admin"), documentation_types=("user", "admin"),
audience=("user", "module_admin", "operator"), audience=("user", "module_admin", "operator"),
@@ -114,7 +153,10 @@ DOCUMENTATION = (
translations={"de": { translations={"de": {
"title": "Vorlagen: Aktionen im Arbeitsbereich", "title": "Vorlagen: Aktionen im Arbeitsbereich",
"summary": "Sammlungsweite Aktionen an ihrer einheitlichen Position im Arbeitsbereich finden.", "summary": "Sammlungsweite Aktionen an ihrer einheitlichen Position im Arbeitsbereich finden.",
"body": "Neu laden und Vorlage hinzufügen stehen oben rechts in der dauerhaft sichtbaren, arbeitsbereichsweiten Leiste; Neu laden steht unmittelbar vor dem Anlegen. Auswahl, Filterwechsel und Bearbeitung verschieben diese sammlungsweiten Aktionen nicht in den linken Bereich. Bearbeiten, Speichern, Veröffentlichen und Rendern bleiben der ausgewählten Vorlage zugeordnet. Bestehende Berechtigungen, Deaktivierungsregeln und der Schutz ungespeicherter Änderungen gelten weiterhin. Administratoren konfigurieren Rechte im bestehenden Berechtigungssystem; es entstehen weder neue Rechte noch automatische Vorgänge.", "body": "Dokumentationsbücher stehen unmittelbar neben der sichtbaren Überschrift oder "
"Kontextbezeichnung für Vorlagen, nicht zwischen ausführbaren Aktionsschaltflächen. Feldhilfe "
"bleibt neben der Feldbezeichnung. "
"Neu laden und Vorlage hinzufügen stehen oben rechts in der dauerhaft sichtbaren, arbeitsbereichsweiten Leiste; Neu laden steht unmittelbar vor dem Anlegen. Auswahl, Filterwechsel und Bearbeitung verschieben diese sammlungsweiten Aktionen nicht in den linken Bereich. Bearbeiten, Speichern, Veröffentlichen und Rendern bleiben der ausgewählten Vorlage zugeordnet. Bestehende Berechtigungen, Deaktivierungsregeln und der Schutz ungespeicherter Änderungen gelten weiterhin. Administratoren konfigurieren Rechte im bestehenden Berechtigungssystem; es entstehen weder neue Rechte noch automatische Vorgänge.",
}}, }},
), ),
DocumentationTopic( DocumentationTopic(
@@ -25,6 +25,12 @@ from govoplan_core.core.templates import (
TemplateRenderRequest, TemplateRenderRequest,
TemplateRenderResult, TemplateRenderResult,
) )
from govoplan_core.security.bounded_process import (
ProcessBudgetError,
ProcessLimits,
run_bounded_operation,
)
from govoplan_core.security.worker_payload import decode_worker_payload, encode_worker_payload
from govoplan_templates.backend.db.models import ( from govoplan_templates.backend.db.models import (
TemplateDefinition, TemplateDefinition,
TemplateRender, TemplateRender,
@@ -42,9 +48,36 @@ from govoplan_templates.backend.service import (
RENDERER_VERSION = "templates-html-1" RENDERER_VERSION = "templates-html-1"
MAX_OUTPUT_BYTES = 5 * 1024 * 1024 MAX_OUTPUT_BYTES = 5 * 1024 * 1024
MAX_ITEMS = 5_000 MAX_ITEMS = 5_000
RENDER_PROCESS_LIMITS = ProcessLimits(
wall_seconds=15,
cpu_seconds=10,
memory_bytes=512 * 1024 * 1024,
input_bytes=32 * 1024 * 1024,
output_bytes=8 * 1024 * 1024,
)
_TOKEN_PATTERN = re.compile(r"{{\s*([A-Za-z_][A-Za-z0-9_.-]*)\s*}}") _TOKEN_PATTERN = re.compile(r"{{\s*([A-Za-z_][A-Za-z0-9_.-]*)\s*}}")
@dataclasses.dataclass(frozen=True, slots=True)
class _RenderDefinition:
name: str
@dataclasses.dataclass(frozen=True, slots=True)
class _RenderRevision:
content_text: str | None
content_html: str | None
template_type: str
layout: dict[str, object]
output_profiles: list[dict[str, object]]
@dataclasses.dataclass(frozen=True, slots=True)
class _RenderRequest:
output_format: str
parameters: dict[str, object]
class SqlTemplateRenderer: class SqlTemplateRenderer:
def __init__(self, registry: object | None = None) -> None: def __init__(self, registry: object | None = None) -> None:
self.registry = registry self.registry = registry
@@ -359,6 +392,68 @@ def _render_payload(
*, *,
request: TemplateRenderRequest, request: TemplateRenderRequest,
items: Sequence[Mapping[str, object]], items: Sequence[Mapping[str, object]],
) -> tuple[bytes, str, int]:
# Authoritative selection/idempotency precede this call. Transfer only the
# fields needed for pure rendering, never ORM objects, principals or Files.
try:
data = encode_worker_payload({
"definition": {"name": definition.name},
"revision": {
"content_text": revision.content_text,
"content_html": revision.content_html,
"template_type": revision.template_type,
"layout": dict(revision.layout),
"output_profiles": list(revision.output_profiles),
},
"request": {
"output_format": request.output_format,
"parameters": dict(request.parameters),
},
"items": tuple(dict(item) for item in items),
}, max_bytes=RENDER_PROCESS_LIMITS.input_bytes)
result = decode_worker_payload(
run_bounded_operation(_render_payload_worker, data, limits=RENDER_PROCESS_LIMITS),
max_bytes=RENDER_PROCESS_LIMITS.output_bytes,
)
except ProcessBudgetError as exc:
raise TemplateRenderError(f"Template rendering failed ({exc.code}): {exc}") from exc
except (TypeError, ValueError, RecursionError) as exc:
raise TemplateRenderError("Template input or output could not be safely transferred.") from exc
if not isinstance(result, dict):
raise TemplateRenderError("Template worker returned an invalid result.")
if "render_error" in result:
raise TemplateRenderError(str(result["render_error"]))
payload, content_type, page_count = result.get("payload"), result.get("content_type"), result.get("page_count")
if not isinstance(payload, bytes) or not isinstance(content_type, str) or type(page_count) is not int:
raise TemplateRenderError("Template worker returned an invalid result.")
if len(payload) > MAX_OUTPUT_BYTES:
_output_limit_exceeded()
return payload, content_type, page_count
def _render_payload_worker(payload: bytes) -> bytes:
data = decode_worker_payload(payload, max_bytes=RENDER_PROCESS_LIMITS.input_bytes)
definition = _RenderDefinition(**data["definition"])
revision = _RenderRevision(**data["revision"])
request = _RenderRequest(**data["request"])
try:
payload, content_type, page_count = _render_payload_content(
definition, revision, request=request, items=data["items"]
)
except TemplateRenderError as exc:
return encode_worker_payload({"render_error": str(exc)})
return encode_worker_payload(
{"payload": payload, "content_type": content_type, "page_count": page_count},
max_bytes=RENDER_PROCESS_LIMITS.output_bytes,
)
def _render_payload_content(
definition: _RenderDefinition,
revision: _RenderRevision,
*,
request: _RenderRequest,
items: Sequence[Mapping[str, object]],
) -> tuple[bytes, str, int]: ) -> tuple[bytes, str, int]:
if request.output_format == "text": if request.output_format == "text":
body = revision.content_text or _html_to_text(revision.content_html or "") body = revision.content_text or _html_to_text(revision.content_html or "")
+38
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import unittest import unittest
from dataclasses import replace
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import patch from unittest.mock import patch
@@ -47,6 +48,43 @@ class TemplateRenderLimitTests(unittest.TestCase):
with self.assertRaises(TemplateRenderError): with self.assertRaises(TemplateRenderError):
rendering._render_payload(definition, revision, request=request, items=[{}]) rendering._render_payload(definition, revision, request=request, items=[{}])
def test_fresh_worker_preserves_exact_text_bytes_and_page_count(self):
definition = SimpleNamespace(name="Example")
revision = SimpleNamespace(content_html=None, content_text="Hi {{name}}: {{flag}}", template_type="serial_letter", layout={}, output_profiles=[])
request = SimpleNamespace(output_format="text", parameters={"flag": True})
with patch.object(rendering, "_render_payload_content", side_effect=AssertionError("parent renderer ran")):
result = rendering._render_payload(definition, revision, request=request, items=[{"name": "Äda"}, {"name": "Grace"}])
self.assertEqual(("Hi Äda: true\n\n---\n\nHi Grace: true".encode(), "text/plain; charset=utf-8", 2), result)
def test_real_worker_timeout_fails_without_parent_fallback(self):
definition = SimpleNamespace(name="Example")
revision = SimpleNamespace(content_html=None, content_text="Hello", template_type="serial_letter", layout={}, output_profiles=[])
request = SimpleNamespace(output_format="text", parameters={})
limits = replace(rendering.RENDER_PROCESS_LIMITS, wall_seconds=0.001)
with patch.object(rendering, "RENDER_PROCESS_LIMITS", limits), patch.object(
rendering, "_render_payload_content", side_effect=AssertionError("parent renderer ran")
):
with self.assertRaisesRegex(TemplateRenderError, "timeout"):
rendering._render_payload(definition, revision, request=request, items=[{}])
def test_worker_html_bytes_match_existing_composition(self):
definition = SimpleNamespace(name="Letters <archive>")
revision = SimpleNamespace(content_html="<p>{{name}}: {{settings}}</p>", content_text=None, template_type="label_sheet", layout={"page_size": "A5", "columns": 2, "rows": 2}, output_profiles=[])
request = SimpleNamespace(output_format="html", parameters={"settings": {"b": True, "a": "ü"}})
items = [{"name": "<Ada>"}, {"name": "Grace & Co"}]
expected = rendering._render_payload_content(definition, revision, request=request, items=items)
actual = rendering._render_payload(definition, revision, request=request, items=items)
self.assertEqual(expected, actual)
def test_real_worker_output_transport_limit_is_enforced(self):
definition = SimpleNamespace(name="Example")
revision = SimpleNamespace(content_html=None, content_text="x" * 4096, template_type="serial_letter", layout={}, output_profiles=[])
request = SimpleNamespace(output_format="text", parameters={})
limits = replace(rendering.RENDER_PROCESS_LIMITS, output_bytes=1024)
with patch.object(rendering, "RENDER_PROCESS_LIMITS", limits):
with self.assertRaisesRegex(TemplateRenderError, "output_limit"):
rendering._render_payload(definition, revision, request=request, items=[{}])
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+31 -1
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import hashlib import hashlib
import unittest import unittest
from dataclasses import replace
from unittest.mock import patch from unittest.mock import patch
from govoplan_core.auth import ApiPrincipal from govoplan_core.auth import ApiPrincipal
@@ -19,6 +20,7 @@ from govoplan_core.core.templates import (
) )
from govoplan_core.db.base import Base from govoplan_core.db.base import Base
from govoplan_core.db.session import configure_database, reset_database from govoplan_core.db.session import configure_database, reset_database
from govoplan_templates.backend import rendering
from govoplan_templates.backend.capabilities import ( from govoplan_templates.backend.capabilities import (
SqlTemplateCatalog, SqlTemplateCatalog,
SqlTemplateContentLibrary, SqlTemplateContentLibrary,
@@ -231,7 +233,8 @@ class TemplateServiceTests(unittest.TestCase):
idempotency_key="campaign-1:postal-output-1", idempotency_key="campaign-1:postal-output-1",
) )
first = render_template(session, principal(), registry=_Registry(), request=request) first = render_template(session, principal(), registry=_Registry(), request=request)
second = render_template(session, principal(), registry=_Registry(), request=request) with patch.object(rendering, "run_bounded_operation", side_effect=AssertionError("idempotent render started a worker")):
second = render_template(session, principal(), registry=_Registry(), request=request)
session.commit() session.commit()
self.assertEqual(first.render_id, second.render_id) self.assertEqual(first.render_id, second.render_id)
@@ -243,6 +246,33 @@ class TemplateServiceTests(unittest.TestCase):
self.assertIn(b"Ada", first.payload) self.assertIn(b"Ada", first.payload)
self.assertIn(b"Grace", first.payload) self.assertIn(b"Grace", first.payload)
def test_real_worker_timeout_writes_neither_artifact_nor_render_evidence(self) -> None:
store = _ArtifactStore()
with self.database.session() as session:
item, _ = create_template(session, principal(), payload())
request = TemplateRenderRequest(
template_id=item.id,
usage="campaign.postal",
items=({"name": "Ada", "postal": {"address": "Street 1"}},),
persist_to_files=True,
)
limits = replace(rendering.RENDER_PROCESS_LIMITS, wall_seconds=0.001)
with patch.object(rendering, "RENDER_PROCESS_LIMITS", limits):
with self.assertRaisesRegex(TemplateRenderError, "timeout"):
render_template(session, principal(), registry=_Registry(store), request=request)
self.assertIsNone(store.request)
self.assertEqual([], list_renders(session, principal()))
def test_incompatible_input_is_rejected_before_worker_admission(self) -> None:
with self.database.session() as session:
item, _ = create_template(session, principal(), payload())
request = TemplateRenderRequest(
template_id=item.id, usage="campaign.postal", items=({"name": "Missing address"},),
)
with patch.object(rendering, "run_bounded_operation", side_effect=AssertionError("incompatible render started a worker")):
with self.assertRaises(TemplateCompatibilityError):
render_template(session, principal(), registry=_Registry(), request=request)
def test_bounded_render_history_and_payload_are_owner_scoped(self) -> None: def test_bounded_render_history_and_payload_are_owner_scoped(self) -> None:
owner = principal(admin=False) owner = principal(admin=False)
other = principal(account_id="account-2", admin=False) other = principal(account_id="account-2", admin=False)
@@ -18,6 +18,7 @@ import { FormGrid, ActionToolbar,
Dialog, Dialog,
DialogSection, DialogSection,
DocumentationHelpLink, DocumentationHelpLink,
TextWithHelp,
DismissibleAlert, DismissibleAlert,
FilterBar, FilterBar,
FormField, FormField,
@@ -349,10 +350,11 @@ export default function TemplatesPage({ settings, auth }: Props) {
state={busy ? "saving" : dirty ? "dirty" : "clean"} state={busy ? "saving" : dirty ? "dirty" : "clean"}
className="templates-workspace-toolbar" className="templates-workspace-toolbar"
contextActions={<span className="templates-current-title"> contextActions={<span className="templates-current-title">
<strong>{selected?.name ?? "Select a template"}</strong> <TextWithHelp help={<DocumentationHelpLink reference={TEMPLATES_DOCUMENTATION} />}>
<strong>{selected?.name ?? "Select a template"}</strong>
</TextWithHelp>
<small>{selected ? `${typeLabel(selected.template_type)} · ${selected.revision.locale}` : ""}</small> <small>{selected ? `${typeLabel(selected.template_type)} · ${selected.revision.locale}` : ""}</small>
</span>} </span>}
helpAction={<DocumentationHelpLink reference={TEMPLATES_DOCUMENTATION} />}
primaryActions={<> primaryActions={<>
<Button helpContextId="templates.action.publish" helpModuleId="templates" disabled={!selected || !canPublish || dirty || busy} disabledReason={busy ? TEMPLATES_I18N.busy : !selected ? TEMPLATES_I18N.noSelection : !canPublish ? TEMPLATES_I18N.publishReason : dirty ? TEMPLATES_I18N.saveBeforeAction : undefined} onClick={() => setPublishOpen(true)}><FileCheck2 size={16} /> Publish</Button> <Button helpContextId="templates.action.publish" helpModuleId="templates" disabled={!selected || !canPublish || dirty || busy} disabledReason={busy ? TEMPLATES_I18N.busy : !selected ? TEMPLATES_I18N.noSelection : !canPublish ? TEMPLATES_I18N.publishReason : dirty ? TEMPLATES_I18N.saveBeforeAction : undefined} onClick={() => setPublishOpen(true)}><FileCheck2 size={16} /> Publish</Button>
<SegmentedControl <SegmentedControl