Enforce declared platform interface inventory
Dependency Audit / dependency-audit (push) Failing after 1m36s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m2s

This commit is contained in:
2026-08-04 05:20:47 +02:00
parent f7590a7b8b
commit 9bb2c808a6
9 changed files with 818 additions and 25 deletions
+2 -2
View File
@@ -55,9 +55,9 @@ jobs:
- name: Install WebUI release dependencies with test scripts - name: Install WebUI release dependencies with test scripts
working-directory: govoplan working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
- name: Validate platform endpoint surface declarations - name: Validate platform interface and endpoint declarations
working-directory: govoplan working-directory: govoplan
run: .venv/bin/python tools/inventory/platform-interface-inventory.py --strict-endpoints run: .venv/bin/python tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
- name: Validate Search against PostgreSQL - name: Validate Search against PostgreSQL
working-directory: govoplan working-directory: govoplan
env: env:
+1 -1
View File
@@ -345,7 +345,7 @@ make every module symmetrical.
| 11 | Configured-system pattern help | Role/config-aware workflow, reference, pattern, and system topics are projected by Docs; shared route, field, blocker, and action links resolve to configured Docs or the hosted fallback | Stable configured-system guidance without feature-to-Docs imports | Docs #15 | Docs suite, shared component tests, Campaign review tests, 46 module permutations, full-product bundle budget | Complete 2026-08-03 (Docs `abe2f78`; Core `b823a22`; Campaign `d635f3a`) | | 11 | Configured-system pattern help | Role/config-aware workflow, reference, pattern, and system topics are projected by Docs; shared route, field, blocker, and action links resolve to configured Docs or the hosted fallback | Stable configured-system guidance without feature-to-Docs imports | Docs #15 | Docs suite, shared component tests, Campaign review tests, 46 module permutations, full-product bundle budget | Complete 2026-08-03 (Docs `abe2f78`; Core `b823a22`; Campaign `d635f3a`) |
| 12 | Admin/configuration family | Core host/settings/credential/retention contracts, shared primitives, module lifecycle, Files, Mail, Policy, Access, Admin, Tenancy, Views, and Organizations are integrated and verified | Continue the same consequence/provenance grammar only through bounded module-owned migrations | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Core #225 complete `fa32cca`; Access `1409dbf`; Files `d8ae506`; Mail `7844d9c`; Policy `f964ed7`; Admin `d428f33`; Tenancy `e76fe16`; Views `c125f33`; Organizations `97acfcb` | | 12 | Admin/configuration family | Core host/settings/credential/retention contracts, shared primitives, module lifecycle, Files, Mail, Policy, Access, Admin, Tenancy, Views, and Organizations are integrated and verified | Continue the same consequence/provenance grammar only through bounded module-owned migrations | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Core #225 complete `fa32cca`; Access `1409dbf`; Files `d8ae506`; Mail `7844d9c`; Policy `f964ed7`; Admin `d428f33`; Tenancy `e76fe16`; Views `c125f33`; Organizations `97acfcb` |
| 13 | Remaining module surfaces | 33 bounded module-owned issues cover every WebUI contributor not already tracked by Campaign #74 or completed Docs #15 | Per-module audit and migration, ordered by user task and consequence rather than a bulk rewrite | Issues linked in the direct-route and composed-surface sections | Module-focused tests, manifest shapes, contextual Docs, and applicable definition-of-done gates | Complete: prior 28 recorded commits plus Workflow #15, Search #4, Reporting #8, Projects #2 and Portal #2 verified 2026-08-03 | | 13 | Remaining module surfaces | 33 bounded module-owned issues cover every WebUI contributor not already tracked by Campaign #74 or completed Docs #15 | Per-module audit and migration, ordered by user task and consequence rather than a bulk rewrite | Issues linked in the direct-route and composed-surface sections | Module-focused tests, manifest shapes, contextual Docs, and applicable definition-of-done gates | Complete: prior 28 recorded commits plus Workflow #15, Search #4, Reporting #8, Projects #2 and Portal #2 verified 2026-08-03 |
| 14 | Manifest/runtime alignment | Authenticated canonical routes align; public signed-token and compatibility routes are explicit exceptions | Reconcile stable declarations with source and runtime continuously | [Meta #25](https://git.add-ideas.de/GovOPlaN/govoplan/issues/25) | Generated static/runtime comparison and duplicate/stale declaration CI | Independent control-plane hardening follow-up; not a rollout closure blocker | | 14 | Manifest/runtime alignment | Authenticated canonical routes align; public signed-token and compatibility routes are explicit exceptions | Stable declarations reconcile with source and any effective runtime module combination | [Meta #25](https://git.add-ideas.de/GovOPlaN/govoplan/issues/25) | Strict duplicate/stale/undeclared declaration CI, per-module digests, and authorized read-only runtime inventory | Complete 2026-08-04 |
Workflow remains outside this rollout matrix because it has its own runtime and Workflow remains outside this rollout matrix because it has its own runtime and
editor workstream, not because it is postponed. Focused views can be specified, editor workstream, not because it is postponed. Focused views can be specified,
+61 -11
View File
@@ -25,6 +25,7 @@ releases, module boundaries, migrations, and security controls.
| Labels and translations | Generated translation catalogs plus source usage | | Labels and translations | Generated translation catalogs plus source usage |
| Fields and help coverage | Shared form components plus generated TypeScript AST inventory | | Fields and help coverage | Shared form components plus generated TypeScript AST inventory |
| API use by the WebUI | Typed API clients plus generated static reference inventory | | API use by the WebUI | Typed API clients plus generated static reference inventory |
| Stable platform interface IDs | Typed manifest/WebUI declarations plus line-independent source anchors for low-level controls |
| Effective configuration | Owning module data plus Policy provenance | | Effective configuration | Owning module data plus Policy provenance |
Runtime introspection is authoritative for an installed system. Static source Runtime introspection is authoritative for an installed system. Static source
@@ -45,7 +46,9 @@ The command writes:
- `audit-reports/platform-inventory/platform-interface-inventory.json` - `audit-reports/platform-inventory/platform-interface-inventory.json`
- `audit-reports/platform-inventory/platform-interface-inventory.md` - `audit-reports/platform-inventory/platform-interface-inventory.md`
Use `--strict` for the combined translation and endpoint audit. Use Use `--strict` for the combined translation, endpoint, and declaration audit.
Use `--strict-declarations` for duplicate/stale/undeclared interface checks
without making existing translation coverage a release blocker. Use
`--strict-endpoints` in the endpoint-surface CI gate so unrelated translation `--strict-endpoints` in the endpoint-surface CI gate so unrelated translation
catalog work cannot disable route classification enforcement. Both strict modes catalog work cannot disable route classification enforcement. Both strict modes
require every backend endpoint without a statically visible WebUI path to have require every backend endpoint without a statically visible WebUI path to have
@@ -75,6 +78,16 @@ It combines:
2. TypeScript AST extraction of fields, label attributes, visible text, 2. TypeScript AST extraction of fields, label attributes, visible text,
translations, frontend routes, navigation, capabilities, and API references translations, frontend routes, navigation, capabilities, and API references
3. Python AST extraction of FastAPI route decorators and router prefixes 3. Python AST extraction of FastAPI route decorators and router prefixes
4. normalized runtime declarations from every loaded `ModuleManifest`
The declaration set covers routes, navigation, View surfaces, fields, actions,
help references, translations, admin/settings sections, widgets, search
objects, permissions, provided interfaces, and backend capabilities. Typed
module contributions keep their declared IDs. Shared controls may declare
`interfaceId` and `helpTopicId`; otherwise the extractor assigns a deterministic
source anchor based on repository, file, component context, control type, and
semantic label rather than a line number. The JSON records which identity
source was used.
The JSON includes exact repository, file, and line evidence. A missing-help The JSON includes exact repository, file, and line evidence. A missing-help
entry is a review candidate because dynamic parent components may supply help. entry is a review candidate because dynamic parent components may supply help.
@@ -82,11 +95,41 @@ A backend route without a static frontend reference is also a review candidate:
public APIs, workers, callbacks, health checks, connectors, and dynamic URL public APIs, workers, callbacks, health checks, connectors, and dynamic URL
assembly are valid explanations. assembly are valid explanations.
The module matrix enforces endpoint declarations with `--strict-endpoints`. The module matrix enforces endpoint and interface declarations with
`--strict-endpoints --strict-declarations`.
Combined `--strict` additionally fails when used translation keys are absent Combined `--strict` additionally fails when used translation keys are absent
from generated locale catalogs. Help-text findings remain review candidates from generated locale catalogs. Help-text findings remain review candidates
rather than a release gate because dynamic parent components can supply help. rather than a release gate because dynamic parent components can supply help.
## Runtime Comparison
Core exposes a sanitized read-only catalog at
`GET /api/v1/platform/interface-catalog`. Access requires
`admin:module:read` or `system:settings:read`. Tenant module entitlements are
applied before serialization, so the response describes only the effective
installed combination. It contains IDs, paths, authorization metadata,
versions, counts, and canonical digests; it excludes factories, callbacks,
credentials, and mutable runtime state.
Capture and compare a running installation:
```bash
curl --fail --silent \
-H "Authorization: Bearer $GOVOPLAN_ACCESS_TOKEN" \
"$GOVOPLAN_URL/api/v1/platform/interface-catalog" \
> /tmp/govoplan-runtime-interface.json
./.venv/bin/python tools/inventory/platform-interface-inventory.py \
--runtime-snapshot /tmp/govoplan-runtime-interface.json \
--strict-declarations \
--strict-endpoints
```
The comparison accepts any installed subset. Every module present in the
runtime response must have the same contract version, module version, and
declaration digest as the static release inventory. Unknown, duplicate, or
mismatched runtime modules fail strict declaration mode.
## Admin Information Architecture ## Admin Information Architecture
The Admin host uses a tree because system, tenant, group, user, and module The Admin host uses a tree because system, tenant, group, user, and module
@@ -139,13 +182,20 @@ Custom code, new routes, arbitrary SQL, and executable workflow nodes remain
release artifacts. Modeling them as ordinary configuration would create an release artifacts. Modeling them as ordinary configuration would create an
unreviewed code-execution and migration channel. unreviewed code-execution and migration channel.
## Next Enforcement Slices ## Enforced Contract
1. Require every WebUI module route and admin/settings contribution to have 1. Public WebUI routes and View surfaces must reconcile with runtime manifest
matching manifest metadata or a reviewed exception. metadata; stale runtime routes and source-only public surfaces fail CI.
2. Add stable field IDs and optional help-topic IDs to shared field components. 2. Duplicate stable IDs fail CI. Shared controls support explicit field/action
3. Classify each statically unreferenced backend endpoint by consumer type. and help-topic identities; fallback anchors remain visible review evidence.
4. Compare a running installation's OpenAPI and module registry against the 3. Every statically unreferenced backend endpoint has an exact reviewed
release inventory. consumer classification, and stale classifications fail CI.
5. Publish the sanitized installed-system structure through Ops/Docs for 4. Runtime module combinations can be compared exactly with static release
authorized administrators. evidence through versioned per-module digests.
5. Runtime introspection is authorized, tenant-filtered, and read-only. It is
safe for Ops/Docs projection but is not a generic configuration or code
mutation channel.
Generated JSON and Markdown remain build/audit artifacts. Do not hand-edit or
use them as a backlog; change the owning manifest, typed WebUI contribution,
translation/help declaration, or exact endpoint classification instead.
+108
View File
@@ -199,6 +199,114 @@ def read_item(item_id: str):
}, },
) )
def test_source_declarations_normalize_stable_control_and_contribution_ids(
self,
) -> None:
webui = {
"fields": [
{
"repository": "govoplan-example",
"file": "webui/src/Example.tsx",
"line": 12,
"column": 3,
"id": "govoplan-example.field.example.name.abc123",
"idSource": "source_anchor",
"explicitId": None,
"context": "Example",
"helpId": "govoplan-example.field.example.name.abc123.help",
"helpDynamic": False,
}
],
"actions": [],
"contributions": [
{
"repository": "govoplan-example",
"file": "webui/src/module.ts",
"line": 20,
"column": 5,
"kind": "frontend_route",
"id": "/examples/:exampleId",
"path": "/examples/:exampleId",
}
],
"translationCatalog": {"en": {}, "de": {}},
}
manifests = [{"repository": "govoplan-example", "id": "examples"}]
declarations = inventory._source_interface_declarations(webui, manifests)
keys = {item["key"] for item in declarations}
self.assertIn("field:examples.field.example.name.abc123", keys)
self.assertIn(
"help:examples.field.example.name.abc123.help",
keys,
)
self.assertIn(
"frontend_route:examples.route.examples.exampleid",
keys,
)
def test_declaration_health_rejects_duplicate_and_undeclared_source_ids(
self,
) -> None:
declaration = {
"key": "frontend_route:example.route.unlisted",
"id": "example.route.unlisted",
"module_id": "example",
"kind": "frontend_route",
"origin": "webui_contribution",
}
manifests = [
{
"id": "example",
"repository": "govoplan-example",
"interface_catalog": {"declarations": []},
}
]
health = inventory._declaration_health(
[declaration, dict(declaration)],
manifests,
)
self.assertEqual(1, len(health["duplicate_ids"]))
self.assertEqual(1, len(health["undeclared_source_surfaces"]))
def test_runtime_snapshot_comparison_accepts_an_installed_subset(self) -> None:
manifests = [
{
"id": "one",
"interface_catalog": {
"contract_version": "1",
"module_id": "one",
"module_version": "1.0.0",
"digest": "sha256:one",
},
},
{
"id": "two",
"interface_catalog": {
"contract_version": "1",
"module_id": "two",
"module_version": "1.0.0",
"digest": "sha256:two",
},
},
]
snapshot = {
"contract_version": "1",
"modules": [dict(manifests[1]["interface_catalog"])],
}
comparison = inventory._compare_runtime_snapshot(snapshot, manifests)
self.assertEqual(["two"], comparison["matched_modules"])
self.assertEqual([], comparison["mismatches"])
snapshot["modules"][0]["digest"] = "sha256:changed"
comparison = inventory._compare_runtime_snapshot(snapshot, manifests)
self.assertEqual("digest_mismatch", comparison["mismatches"][0]["reason"])
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+1
View File
@@ -40,6 +40,7 @@ GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
cd "$META_ROOT" cd "$META_ROOT"
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
"$PYTHON" tools/repo/sync-module-package-workflows.py --check "$PYTHON" tools/repo/sync-module-package-workflows.py --check
"$PYTHON" tools/release/generate-developer-meta-package.py --check "$PYTHON" tools/release/generate-developer-meta-package.py --check
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release "$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
@@ -182,6 +182,11 @@ run_step "Validate installed module manifests and registry"
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \ "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
--requirements "$META_ROOT/requirements-release.txt" --requirements "$META_ROOT/requirements-release.txt"
run_step "Validate platform interface and endpoint declarations"
"$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \
--strict-declarations \
--strict-endpoints
run_step "Generate release dependency provenance" run_step "Generate release dependency provenance"
"$PYTHON" "$META_ROOT/tools/release/generate-release-sbom.py" \ "$PYTHON" "$META_ROOT/tools/release/generate-release-sbom.py" \
--python "$PYTHON" \ --python "$PYTHON" \
@@ -567,6 +567,13 @@
"rationale": "The shared ownership UI uses a dynamic transfer-action path that the static string scan cannot resolve.", "rationale": "The shared ownership UI uses a dynamic transfer-action path that the static string scan cannot resolve.",
"repository": "govoplan-core" "repository": "govoplan-core"
}, },
{
"category": "intentionally_headless",
"method": "GET",
"path": "/platform/interface-catalog",
"rationale": "Authorized module and system administrators consume the read-only control-plane inventory directly or through Ops/Docs projections.",
"repository": "govoplan-core"
},
{ {
"category": "compatibility", "category": "compatibility",
"method": "GET", "method": "GET",
+230 -7
View File
@@ -2,6 +2,7 @@
import fs from "node:fs"; import fs from "node:fs";
import path from "node:path"; import path from "node:path";
import { createHash } from "node:crypto";
import { pathToFileURL } from "node:url"; import { pathToFileURL } from "node:url";
const [metaRootArgument] = process.argv.slice(2); const [metaRootArgument] = process.argv.slice(2);
@@ -60,9 +61,19 @@ const helpAttributes = new Set([
"helperText", "helperText",
"helpText" "helpText"
]); ]);
const actionComponentPattern = /(?:Action|Button|Link)$/;
const contributionTypes = new Map([
["AdminSectionsUiCapability", "admin_section"],
["DashboardWidgetsUiCapability", "widget"],
["OrganizationFunctionActionsUiCapability", "action"],
["SearchContextsUiCapability", "search_object"],
["SettingsSectionsUiCapability", "setting"],
["WizardDirectoriesUiCapability", "workflow_directory"]
]);
const result = { const result = {
fields: [], fields: [],
actions: [],
labels: [], labels: [],
visibleText: [], visibleText: [],
translationCatalog: {}, translationCatalog: {},
@@ -71,7 +82,8 @@ const result = {
routes: [], routes: [],
navigation: [], navigation: [],
frontendApiReferences: [], frontendApiReferences: [],
uiCapabilities: [] uiCapabilities: [],
contributions: []
}; };
for (const repository of repositoryCatalog.repositories) { for (const repository of repositoryCatalog.repositories) {
@@ -115,6 +127,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS
); );
const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath); const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath);
const identityCounters = new Map();
function location(node) { function location(node) {
const position = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile)); const position = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile));
@@ -178,6 +191,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
const isField = const isField =
fieldComponents.has(component) || fieldComponents.has(component) ||
(fieldComponentPattern.test(component) && component !== "FormField"); (fieldComponentPattern.test(component) && component !== "FormField");
inspectAction(node, component, attributes, locate);
if (!isField) return; if (!isField) return;
const parentFormField = nearestFormField(node); const parentFormField = nearestFormField(node);
@@ -191,8 +205,25 @@ function inspectSource(repository, sourceRoot, sourcePath) {
null; null;
const help = firstAttribute(attributes, helpAttributes) ?? const help = firstAttribute(attributes, helpAttributes) ??
firstAttribute(parentAttributes, helpAttributes); firstAttribute(parentAttributes, helpAttributes);
const hasHelp = hasAnyAttribute(attributes, helpAttributes) ||
hasAnyAttribute(parentAttributes, helpAttributes);
const explicitId = firstAttribute(
attributes,
new Set(["interfaceId", "data-interface-id", "id", "name", "field"])
);
const context = nearestNamedContext(node);
const stableId = sourceIdentity(
"field",
node,
component,
explicitId ?? label ?? attributes.get("placeholder") ?? "field"
);
result.fields.push({ result.fields.push({
...locate(node), ...locate(node),
id: stableId,
explicitId,
idSource: explicitId === null ? "source_anchor" : "explicit",
context,
component, component,
name: name:
attributes.get("name") ?? attributes.get("name") ??
@@ -202,8 +233,102 @@ function inspectSource(repository, sourceRoot, sourcePath) {
label, label,
placeholder: attributes.get("placeholder") ?? null, placeholder: attributes.get("placeholder") ?? null,
help: help ?? null, help: help ?? null,
helpCandidate: help === null helpId: hasHelp ? `${stableId}.help` : null,
helpDynamic: hasHelp && help === null,
helpCandidate: !hasHelp
}); });
}
function inspectAction(node, component, attributes, locate) {
const lowerComponent = component.toLowerCase();
const inputType = attributes.get("type")?.toLowerCase();
const isAction = lowerComponent === "button" ||
lowerComponent === "a" ||
actionComponentPattern.test(component) ||
(lowerComponent === "input" && ["button", "reset", "submit"].includes(inputType));
if (!isAction) return;
const label = attributes.get("aria-label") ??
attributes.get("title") ??
attributes.get("label") ??
staticJsxChildText(node) ??
null;
const explicitId = firstAttribute(
attributes,
new Set(["interfaceId", "data-interface-id", "id", "name"])
);
const context = nearestNamedContext(node);
result.actions.push({
...locate(node),
id: sourceIdentity(
"action",
node,
component,
explicitId ?? label ?? "action"
),
explicitId,
idSource: explicitId === null ? "source_anchor" : "explicit",
context,
component,
label
});
}
function nearestNamedContext(node) {
let current = node.parent;
while (current) {
if (ts.isFunctionDeclaration(current) && current.name) {
return current.name.text;
}
if (ts.isMethodDeclaration(current) && current.name) {
return current.name.getText(sourceFile);
}
if (
(ts.isArrowFunction(current) || ts.isFunctionExpression(current)) &&
ts.isVariableDeclaration(current.parent) &&
ts.isIdentifier(current.parent.name)
) {
return current.parent.name.text;
}
if (
(ts.isArrowFunction(current) || ts.isFunctionExpression(current)) &&
ts.isPropertyAssignment(current.parent)
) {
return propertyNameText(current.parent.name) ?? "anonymous";
}
current = current.parent;
}
return path.basename(relativeFile).replace(/\.[^.]+$/, "");
}
function sourceIdentity(kind, node, component, semantic) {
const context = nearestNamedContext(node);
const normalizedSemantic = slug(String(semantic));
const counterKey = `${kind}:${context}:${component}:${normalizedSemantic}`;
const occurrence = (identityCounters.get(counterKey) ?? 0) + 1;
identityCounters.set(counterKey, occurrence);
const anchor = [relativeFile, context, component, normalizedSemantic, occurrence].join(":");
const digest = createHash("sha256").update(anchor).digest("hex").slice(0, 12);
return `${repository}.${kind}.${slug(context)}.${normalizedSemantic}.${digest}`;
}
function staticJsxChildText(node) {
if (!ts.isJsxOpeningElement(node) || !ts.isJsxElement(node.parent)) return null;
const values = [];
for (const child of node.parent.children) {
if (ts.isJsxText(child)) {
const value = child.getText(sourceFile).replace(/\s+/g, " ").trim();
if (value) values.push(value);
} else if (
ts.isJsxExpression(child) &&
child.expression &&
ts.isStringLiteralLike(child.expression)
) {
values.push(child.expression.text);
}
}
return values.length > 0 ? values.join(" ") : null;
} }
function nearestFormField(node) { function nearestFormField(node) {
@@ -275,22 +400,103 @@ function inspectSource(repository, sourceRoot, sourcePath) {
function inspectProperty(node, locate) { function inspectProperty(node, locate) {
const propertyName = propertyNameText(node.name); const propertyName = propertyNameText(node.name);
if (isDirectUiCapabilityProperty(node) && typeof propertyName === "string") {
result.uiCapabilities.push({ ...locate(node), name: propertyName });
result.contributions.push({
...locate(node),
kind: "ui_capability",
id: propertyName
});
}
const value = staticExpressionText(node.initializer); const value = staticExpressionText(node.initializer);
if (value === null) return; if (value === null) return;
if (propertyName === "path" && value.startsWith("/") && !value.includes("/api/")) { if (propertyName === "path" && value.startsWith("/") && !value.includes("/api/")) {
result.routes.push({ ...locate(node), path: value }); result.routes.push({ ...locate(node), path: value });
const routeCollection = nearestCollectionProperty(node);
if (routeCollection === "routes" || routeCollection === "publicRoutes") {
result.contributions.push({
...locate(node),
kind: routeCollection === "publicRoutes" ? "public_route" : "frontend_route",
id: value,
path: value
});
}
} }
if (propertyName === "to" && value.startsWith("/")) { if (propertyName === "to" && value.startsWith("/")) {
result.navigation.push({ ...locate(node), path: value }); result.navigation.push({ ...locate(node), path: value });
if (nearestCollectionProperty(node) === "navItems") {
result.contributions.push({
...locate(node),
kind: "navigation",
id: value,
path: value
});
}
} }
if ( if (propertyName === "id") {
ancestorPropertyName(node, "uiCapabilities") && const contributionKind = contributionKindFor(node);
typeof propertyName === "string" if (contributionKind !== null) {
) { result.contributions.push({
result.uiCapabilities.push({ ...locate(node), name: propertyName }); ...locate(node),
kind: contributionKind,
id: value
});
}
} }
} }
function contributionKindFor(node) {
const collection = nearestCollectionProperty(node);
if (collection === "viewSurfaces") return "view_surface";
if (collection === "widgets") return "widget";
if (collection === "contexts") return "search_object";
if (collection === "actions") return "action";
if (collection === "directories") return "workflow_directory";
if (collection !== "sections") return null;
const variableType = nearestVariableType(node);
for (const [typeName, kind] of contributionTypes) {
if (variableType.includes(typeName)) return kind;
}
return ancestorPropertyName(node, "admin.sections")
? "admin_section"
: ancestorPropertyName(node, "settings.sections")
? "setting"
: "section";
}
function nearestCollectionProperty(node) {
let current = node.parent;
while (current) {
if (
ts.isArrayLiteralExpression(current) &&
ts.isPropertyAssignment(current.parent)
) {
return propertyNameText(current.parent.name);
}
current = current.parent;
}
return null;
}
function nearestVariableType(node) {
let current = node.parent;
while (current) {
if (ts.isVariableDeclaration(current)) {
return current.type?.getText(sourceFile) ?? "";
}
current = current.parent;
}
return "";
}
function isDirectUiCapabilityProperty(node) {
const parent = node.parent;
const uiCapabilities = parent?.parent;
return ts.isObjectLiteralExpression(parent) &&
ts.isPropertyAssignment(uiCapabilities) &&
propertyNameText(uiCapabilities.name) === "uiCapabilities";
}
function inspectTranslationProperty(node, locate) { function inspectTranslationProperty(node, locate) {
const key = propertyNameText(node.name); const key = propertyNameText(node.name);
if (!key?.startsWith("i18n:")) return; if (!key?.startsWith("i18n:")) return;
@@ -343,6 +549,23 @@ function firstAttribute(attributes, names) {
return null; return null;
} }
function hasAnyAttribute(attributes, names) {
for (const name of names) {
if (attributes.has(name)) return true;
}
return false;
}
function slug(value) {
const normalized = value
.toLowerCase()
.replace(/^i18n:/, "")
.replace(/\.[a-f0-9]{8}$/, "")
.replace(/[^a-z0-9]+/g, ".")
.replace(/^\.+|\.+$/g, "");
return (normalized || "unnamed").slice(0, 72);
}
function propertyNameText(name) { function propertyNameText(name) {
if ( if (
ts.isIdentifier(name) || ts.isIdentifier(name) ||
+403 -4
View File
@@ -50,6 +50,23 @@ def main() -> int:
action="store_true", action="store_true",
help="Fail only on incomplete or stale endpoint-surface declarations.", help="Fail only on incomplete or stale endpoint-surface declarations.",
) )
parser.add_argument(
"--strict-declarations",
action="store_true",
help=(
"Fail on duplicate stable IDs, WebUI surfaces absent from runtime "
"metadata, or stale runtime route declarations."
),
)
parser.add_argument(
"--runtime-snapshot",
type=Path,
help=(
"Compare a saved /api/v1/platform/interface-catalog response with "
"the static manifest inventory. Any installed module combination "
"is accepted; every module present in the snapshot must match."
),
)
parser.add_argument( parser.add_argument(
"--endpoint-declarations", "--endpoint-declarations",
type=Path, type=Path,
@@ -71,6 +88,11 @@ def main() -> int:
backend_endpoints=backend_endpoints, backend_endpoints=backend_endpoints,
manifests=manifests, manifests=manifests,
endpoint_declarations=endpoint_declarations, endpoint_declarations=endpoint_declarations,
runtime_snapshot=(
_load_runtime_snapshot(args.runtime_snapshot.resolve())
if args.runtime_snapshot is not None
else None
),
) )
output_dir = args.output_dir.resolve() output_dir = args.output_dir.resolve()
@@ -85,11 +107,12 @@ def main() -> int:
print(f"Platform inventory JSON: {json_path}") print(f"Platform inventory JSON: {json_path}")
print(f"Platform inventory summary: {markdown_path}") print(f"Platform inventory summary: {markdown_path}")
if args.strict or args.strict_endpoints: if args.strict or args.strict_endpoints or args.strict_declarations:
failures = _strict_failures( failures = _strict_failures(
inventory, inventory,
check_translations=args.strict, check_translations=args.strict,
check_endpoints=True, check_endpoints=args.strict or args.strict_endpoints,
check_declarations=args.strict or args.strict_declarations,
) )
if failures: if failures:
print( print(
@@ -105,6 +128,7 @@ def _strict_failures(
*, *,
check_translations: bool, check_translations: bool,
check_endpoints: bool, check_endpoints: bool,
check_declarations: bool = False,
) -> list[str]: ) -> list[str]:
failures: list[str] = [] failures: list[str] = []
if ( if (
@@ -122,6 +146,32 @@ def _strict_failures(
f"{len(inventory['api']['stale_endpoint_declarations'])} " f"{len(inventory['api']['stale_endpoint_declarations'])} "
"endpoint declarations do not match a backend endpoint" "endpoint declarations do not match a backend endpoint"
) )
declaration_health = inventory.get("declaration_health", {})
if check_declarations and declaration_health.get("duplicate_ids"):
failures.append(
f"{len(declaration_health['duplicate_ids'])} platform interface "
"IDs are declared more than once"
)
if check_declarations and declaration_health.get("undeclared_source_surfaces"):
failures.append(
f"{len(declaration_health['undeclared_source_surfaces'])} public "
"WebUI surfaces have no runtime manifest declaration"
)
if check_declarations and declaration_health.get("stale_runtime_routes"):
failures.append(
f"{len(declaration_health['stale_runtime_routes'])} runtime route "
"declarations have no WebUI implementation"
)
runtime_comparison = inventory.get("runtime_comparison")
if (
check_declarations
and runtime_comparison is not None
and runtime_comparison.get("mismatches")
):
failures.append(
f"{len(runtime_comparison['mismatches'])} runtime catalog entries "
"do not match the release inventory"
)
return failures return failures
@@ -265,6 +315,10 @@ def _extract_manifests(
if (workspace_root / repository["path"] / "src").is_dir() if (workspace_root / repository["path"] / "src").is_dir()
] ]
sys.path[:0] = [str(path) for path in source_roots] sys.path[:0] = [str(path) for path in source_roots]
from govoplan_core.core.platform_interfaces import ( # noqa: PLC0415
manifest_interface_catalog,
)
manifests: list[dict[str, Any]] = [] manifests: list[dict[str, Any]] = []
for repository in catalog["repositories"]: for repository in catalog["repositories"]:
source_root = workspace_root / repository["path"] / "src" source_root = workspace_root / repository["path"] / "src"
@@ -299,15 +353,24 @@ def _extract_manifests(
} }
for permission in manifest.permissions for permission in manifest.permissions
], ],
"interface_catalog": manifest_interface_catalog(manifest),
"frontend": ( "frontend": (
{ {
"package": frontend.package_name, "package": frontend.package_name,
"routes": [ "routes": [
_plain_value(route) for route in frontend.routes _plain_value(route) for route in frontend.routes
], ],
"public_routes": [
_plain_value(route)
for route in frontend.public_routes
],
"nav_items": [ "nav_items": [
_plain_value(item) for item in frontend.nav_items _plain_value(item) for item in frontend.nav_items
], ],
"settings_routes": [
_plain_value(route)
for route in frontend.settings_routes
],
"view_surfaces": [ "view_surfaces": [
_plain_value(surface) _plain_value(surface)
for surface in frontend.view_surfaces for surface in frontend.view_surfaces
@@ -327,6 +390,7 @@ def _assemble_inventory(
backend_endpoints: list[dict[str, Any]], backend_endpoints: list[dict[str, Any]],
manifests: list[dict[str, Any]], manifests: list[dict[str, Any]],
endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]], endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]],
runtime_snapshot: dict[str, Any] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
frontend_refs = webui["frontendApiReferences"] frontend_refs = webui["frontendApiReferences"]
frontend_paths = { frontend_paths = {
@@ -396,25 +460,40 @@ def _assemble_inventory(
] ]
fields = webui["fields"] fields = webui["fields"]
help_candidates = [field for field in fields if field["helpCandidate"]] help_candidates = [field for field in fields if field["helpCandidate"]]
dynamic_help = [field for field in fields if field.get("helpDynamic")]
source_declarations = _source_interface_declarations(webui, manifests)
declaration_health = _declaration_health(source_declarations, manifests)
runtime_comparison = (
_compare_runtime_snapshot(runtime_snapshot, manifests)
if runtime_snapshot is not None
else None
)
return { return {
"schema_version": 1, "schema_version": 2,
"scope": { "scope": {
"source": "local GovOPlaN repository catalog", "source": "local GovOPlaN repository catalog",
"limitations": [ "limitations": [
"Static extraction cannot resolve runtime-computed labels, routes, or API paths.", "Static extraction cannot resolve runtime-computed labels, routes, or API paths.",
"A backend endpoint without a static WebUI reference may intentionally serve public clients, workers, connectors, or external integrations.", "A backend endpoint without a static WebUI reference may intentionally serve public clients, workers, connectors, or external integrations.",
"A field marked as a help candidate may receive contextual help from a surrounding dynamic component.", "A field marked as a help candidate may receive contextual help from a surrounding dynamic component.",
"Low-level field and action IDs use line-independent source anchors unless an explicit interfaceId, DOM id, or name is declared.",
], ],
}, },
"modules": manifests, "modules": manifests,
"interface_declarations": source_declarations,
"declaration_health": declaration_health,
"runtime_comparison": runtime_comparison,
"ui": { "ui": {
"fields": fields, "fields": fields,
"actions": webui.get("actions", []),
"labels": webui["labels"], "labels": webui["labels"],
"visible_text": webui["visibleText"], "visible_text": webui["visibleText"],
"routes": webui["routes"], "routes": webui["routes"],
"navigation": webui["navigation"], "navigation": webui["navigation"],
"ui_capabilities": webui["uiCapabilities"], "ui_capabilities": webui["uiCapabilities"],
"help_candidates": help_candidates, "help_candidates": help_candidates,
"dynamic_help": dynamic_help,
"contributions": webui.get("contributions", []),
}, },
"translations": { "translations": {
"catalog": catalogs, "catalog": catalogs,
@@ -439,6 +518,16 @@ def _assemble_inventory(
"ui_fields": len(fields), "ui_fields": len(fields),
"ui_fields_with_static_help": len(fields) - len(help_candidates), "ui_fields_with_static_help": len(fields) - len(help_candidates),
"help_review_candidates": len(help_candidates), "help_review_candidates": len(help_candidates),
"dynamic_help_references": len(dynamic_help),
"ui_actions": len(webui.get("actions", [])),
"interface_declarations": len(source_declarations),
"duplicate_interface_ids": len(declaration_health["duplicate_ids"]),
"undeclared_source_surfaces": len(
declaration_health["undeclared_source_surfaces"]
),
"stale_runtime_routes": len(
declaration_health["stale_runtime_routes"]
),
"label_attributes": len(webui["labels"]), "label_attributes": len(webui["labels"]),
"visible_text_nodes": len(webui["visibleText"]), "visible_text_nodes": len(webui["visibleText"]),
"frontend_routes": len(webui["routes"]), "frontend_routes": len(webui["routes"]),
@@ -451,6 +540,299 @@ def _assemble_inventory(
} }
def _source_interface_declarations(
webui: dict[str, Any],
manifests: list[dict[str, Any]],
) -> list[dict[str, Any]]:
module_by_repository = {
str(manifest["repository"]): str(manifest["id"])
for manifest in manifests
}
declarations: list[dict[str, Any]] = []
def module_id(repository: str) -> str:
if repository in module_by_repository:
return module_by_repository[repository]
if repository == "govoplan-core":
return "core"
return repository.removeprefix("govoplan-").replace("-", "_")
def source_evidence(item: dict[str, Any]) -> dict[str, Any]:
return {
key: item[key]
for key in ("repository", "file", "line", "column")
if key in item
}
for kind, items in (
("field", webui["fields"]),
("action", webui.get("actions", [])),
):
for item in items:
repository = str(item["repository"])
owner = module_id(repository)
raw_id = str(item["id"])
stable_id = (
f"{owner}.{raw_id[len(repository) + 1:]}"
if raw_id.startswith(f"{repository}.")
else _namespaced_interface_id(owner, kind, raw_id)
)
declarations.append(
{
"key": f"{kind}:{stable_id}",
"id": stable_id,
"module_id": owner,
"kind": kind,
"origin": "webui_source",
"id_source": item.get("idSource", "source_anchor"),
"explicit_id": item.get("explicitId"),
"context": item.get("context"),
**source_evidence(item),
}
)
if kind == "field" and item.get("helpId"):
help_raw_id = str(item["helpId"])
help_id = (
f"{owner}.{help_raw_id[len(repository) + 1:]}"
if help_raw_id.startswith(f"{repository}.")
else _namespaced_interface_id(owner, "help", help_raw_id)
)
declarations.append(
{
"key": f"help:{help_id}",
"id": help_id,
"module_id": owner,
"kind": "help",
"origin": "webui_source",
"field_id": stable_id,
"dynamic": bool(item.get("helpDynamic")),
**source_evidence(item),
}
)
for item in webui.get("contributions", []):
repository = str(item["repository"])
owner = module_id(repository)
kind = str(item["kind"])
raw_id = str(item["id"])
path = item.get("path")
if kind == "frontend_route" and isinstance(path, str):
stable_id = f"{owner}.route.{_surface_slug(path)}"
elif kind == "public_route" and isinstance(path, str):
stable_id = f"{owner}.public.{_surface_slug(path)}"
elif kind == "navigation" and isinstance(path, str):
stable_id = f"{owner}.nav.{_surface_slug(path)}"
else:
stable_id = _namespaced_interface_id(owner, kind, raw_id)
declarations.append(
{
"key": f"{kind}:{stable_id}",
"id": stable_id,
"module_id": owner,
"kind": kind,
"origin": "webui_contribution",
"declared_value": raw_id,
**({"path": path} if isinstance(path, str) else {}),
**source_evidence(item),
}
)
translation_entries: dict[tuple[str, str], dict[str, Any]] = {}
for locale, entries in webui["translationCatalog"].items():
for key, item in entries.items():
repository = str(item["repository"])
owner = module_id(repository)
declaration_key = (owner, str(key))
declaration = translation_entries.setdefault(
declaration_key,
{
"key": f"translation:{key}",
"id": key,
"module_id": owner,
"kind": "translation",
"origin": "translation_catalog",
"locales": [],
**source_evidence(item),
},
)
declaration["locales"].append(locale)
declarations.extend(translation_entries.values())
return sorted(
declarations,
key=lambda item: (
item["module_id"],
item["kind"],
item["id"],
item.get("file", ""),
item.get("line", 0),
),
)
def _declaration_health(
source_declarations: list[dict[str, Any]],
manifests: list[dict[str, Any]],
) -> dict[str, Any]:
grouped: dict[tuple[str, str, str], list[dict[str, Any]]] = {}
for declaration in source_declarations:
key = (
str(declaration["module_id"]),
str(declaration["kind"]),
str(declaration["id"]),
)
grouped.setdefault(key, []).append(declaration)
duplicate_ids = [
{
"module_id": key[0],
"kind": key[1],
"id": key[2],
"evidence": values,
}
for key, values in sorted(grouped.items())
if len(values) > 1
]
comparable_kinds = {
"frontend_route",
"navigation",
"public_route",
"view_surface",
}
source_surfaces = {
(str(item["module_id"]), str(item["kind"]), str(item["id"])): item
for item in source_declarations
if item["origin"] == "webui_contribution"
and item["kind"] in comparable_kinds
}
runtime_surfaces: dict[tuple[str, str, str], dict[str, Any]] = {}
for manifest in manifests:
catalog = manifest["interface_catalog"]
for declaration in catalog["declarations"]:
if declaration["kind"] not in comparable_kinds:
continue
key = (
str(manifest["id"]),
str(declaration["kind"]),
str(declaration["id"]),
)
runtime_surfaces[key] = {
"repository": manifest["repository"],
**declaration,
}
surface_id = declaration.get("metadata", {}).get("surface_id")
if (
declaration["kind"]
in {"frontend_route", "navigation", "settings_route"}
and isinstance(surface_id, str)
and surface_id
):
runtime_surfaces[
(str(manifest["id"]), "view_surface", surface_id)
] = {
"repository": manifest["repository"],
"key": f"view_surface:{surface_id}",
"id": surface_id,
"module_id": manifest["id"],
"kind": "view_surface",
"metadata": {
"derived_from": declaration["kind"],
"path": declaration.get("path"),
},
}
undeclared_source_surfaces = [
source_surfaces[key]
for key in sorted(source_surfaces.keys() - runtime_surfaces.keys())
]
route_kinds = {"frontend_route", "public_route"}
stale_runtime_routes = [
runtime_surfaces[key]
for key in sorted(runtime_surfaces.keys() - source_surfaces.keys())
if key[1] in route_kinds
]
return {
"duplicate_ids": duplicate_ids,
"undeclared_source_surfaces": undeclared_source_surfaces,
"stale_runtime_routes": stale_runtime_routes,
"source_declaration_count": len(source_declarations),
"runtime_declaration_count": sum(
len(manifest["interface_catalog"]["declarations"])
for manifest in manifests
),
}
def _compare_runtime_snapshot(
snapshot: dict[str, Any],
manifests: list[dict[str, Any]],
) -> dict[str, Any]:
static_by_module = {
str(manifest["id"]): manifest["interface_catalog"]
for manifest in manifests
}
modules = snapshot.get("modules")
if not isinstance(modules, list):
raise ValueError("Runtime interface snapshot must contain a modules list.")
mismatches: list[dict[str, Any]] = []
seen: set[str] = set()
matched: list[str] = []
for item in modules:
if not isinstance(item, dict) or not isinstance(item.get("module_id"), str):
raise ValueError("Runtime interface snapshot has an invalid module entry.")
module_id = item["module_id"]
if module_id in seen:
mismatches.append({"module_id": module_id, "reason": "duplicate_module"})
continue
seen.add(module_id)
expected = static_by_module.get(module_id)
if expected is None:
mismatches.append({"module_id": module_id, "reason": "unknown_module"})
continue
for field in ("contract_version", "module_version", "digest"):
if item.get(field) != expected.get(field):
mismatches.append(
{
"module_id": module_id,
"reason": f"{field}_mismatch",
"expected": expected.get(field),
"actual": item.get(field),
}
)
if not any(
mismatch["module_id"] == module_id for mismatch in mismatches
):
matched.append(module_id)
return {
"contract_version": snapshot.get("contract_version"),
"matched_modules": sorted(matched),
"mismatches": mismatches,
}
def _load_runtime_snapshot(path: Path) -> dict[str, Any]:
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise ValueError(f"Runtime interface snapshot does not exist: {path}") from exc
except json.JSONDecodeError as exc:
raise ValueError(f"Runtime interface snapshot is invalid JSON: {exc}") from exc
if not isinstance(payload, dict):
raise ValueError("Runtime interface snapshot must be a JSON object.")
return payload
def _namespaced_interface_id(module_id: str, kind: str, value: str) -> str:
if value.startswith(f"{module_id}."):
return value
return f"{module_id}.{kind}.{_surface_slug(value)}"
def _surface_slug(value: str) -> str:
normalized = re.sub(r"[^a-z0-9]+", ".", value.strip().lower()).strip(".")
return normalized or "root"
def _render_markdown(inventory: dict[str, Any]) -> str: def _render_markdown(inventory: dict[str, Any]) -> str:
summary = inventory["summary"] summary = inventory["summary"]
missing = inventory["translation_health"]["missing_catalog_entries"] missing = inventory["translation_health"]["missing_catalog_entries"]
@@ -472,8 +854,14 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
"", "",
f"- Modules: {summary['modules']}", f"- Modules: {summary['modules']}",
f"- UI fields: {summary['ui_fields']}", f"- UI fields: {summary['ui_fields']}",
f"- UI actions: {summary['ui_actions']}",
f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}", f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}",
f"- Fields with dynamic help references: {summary['dynamic_help_references']}",
f"- Help review candidates: {summary['help_review_candidates']}", f"- Help review candidates: {summary['help_review_candidates']}",
f"- Stable interface declarations: {summary['interface_declarations']}",
f"- Duplicate interface IDs: {summary['duplicate_interface_ids']}",
f"- WebUI surfaces missing runtime declarations: {summary['undeclared_source_surfaces']}",
f"- Runtime routes missing WebUI implementations: {summary['stale_runtime_routes']}",
f"- Label attributes: {summary['label_attributes']}", f"- Label attributes: {summary['label_attributes']}",
f"- Frontend routes: {summary['frontend_routes']}", f"- Frontend routes: {summary['frontend_routes']}",
f"- Backend endpoints: {summary['backend_endpoints']}", f"- Backend endpoints: {summary['backend_endpoints']}",
@@ -527,13 +915,24 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
) )
lines.extend( lines.extend(
[ [
"",
"## Declaration Reconciliation",
"",
"Routes, navigation, View surfaces, fields, actions, help references,",
"translations, settings, widgets, search objects, and backend",
"capabilities use normalized stable IDs. CI rejects duplicate IDs,",
"WebUI public surfaces absent from runtime metadata, and runtime routes",
"without a WebUI implementation.",
"",
"", "",
"## Interpretation", "## Interpretation",
"", "",
"Use the JSON artifact for exact file and line evidence. Missing help is", "Use the JSON artifact for exact file and line evidence. Missing help is",
"a triage list, not an automatic defect. Endpoint coverage requires an", "a triage list, not an automatic defect. Endpoint coverage requires an",
"owner classification before enforcement. Runtime-computed structures", "owner classification before enforcement. Runtime-computed structures",
"need explicit manifest metadata to become canonically visible.", "need explicit manifest or typed PlatformWebModule metadata to become",
"canonically visible. The generated files are release evidence, not an",
"editable source of platform behavior.",
"", "",
] ]
) )