feat: add platform interface inventory and encryption registry
This commit is contained in:
@@ -9,6 +9,12 @@ The applicable design contract is
|
||||
|
||||
## Snapshot And Method
|
||||
|
||||
The source-derived inventory command is documented in
|
||||
[`PLATFORM_CONTROL_PLANE.md`](PLATFORM_CONTROL_PLANE.md). It produces
|
||||
machine-readable field, label, translation, route, API-reference, and module
|
||||
manifest evidence. This hand-maintained document remains the reviewed product
|
||||
interpretation and rollout ledger; generated evidence does not replace it.
|
||||
|
||||
Snapshot refreshed: 2026-07-22.
|
||||
|
||||
Evidence was read from tracked Git `HEAD` in the local GovOPlaN checkouts:
|
||||
@@ -215,10 +221,11 @@ backend-only modules may remain intentionally headless.
|
||||
| 13 | Remaining direct routes | Routes are contributed; most are unreviewed | Per-module bounded audit and migration plan | New module issues derived from this inventory | Applicable definition-of-done gates | P2 after Campaign, not a bulk rewrite |
|
||||
| 14 | Manifest/runtime alignment | Several executable routes are absent from manifest metadata | Declared alignment or explicit validated exception | Core contract issue to create | Automated manifest/module route check and configured Docs verification | Discovery follow-up |
|
||||
|
||||
Workflow/user-story implementation is postponed. It is not on the critical path
|
||||
for this rollout matrix. Focused views can be specified, manually selected, and
|
||||
tested through core composition contracts; a later workflow step may become one
|
||||
activation source without changing the proven surface patterns.
|
||||
Workflow remains outside this rollout matrix because it has its own runtime and
|
||||
editor workstream, not because it is postponed. Focused views can be specified,
|
||||
manually selected, and tested through core composition contracts today.
|
||||
Workflow steps may activate those views through the same contract without
|
||||
changing the proven surface patterns.
|
||||
|
||||
## Inventory Maintenance
|
||||
|
||||
|
||||
126
docs/PLATFORM_CONTROL_PLANE.md
Normal file
126
docs/PLATFORM_CONTROL_PLANE.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Platform Control Plane And Self-Description
|
||||
|
||||
## Objective
|
||||
|
||||
GovOPlaN should be able to describe its installed structure without becoming a
|
||||
self-modifying application. The platform model is a declarative control plane:
|
||||
module manifests, UI contributions, schemas, policy provenance, runtime
|
||||
capabilities, and generated source evidence describe what can be configured.
|
||||
Ordinary administrators edit validated data through those contracts; they do
|
||||
not edit Python, TypeScript, routes, or database code from the product UI.
|
||||
|
||||
This distinction provides the requested overview while preserving reviewable
|
||||
releases, module boundaries, migrations, and security controls.
|
||||
|
||||
## Canonical Sources
|
||||
|
||||
| Concern | Canonical source |
|
||||
| --- | --- |
|
||||
| Installed modules and dependency graph | Runtime `ModuleManifest` registry |
|
||||
| Backend routes | Registered FastAPI application; Python AST is build-time evidence |
|
||||
| Frontend routes and navigation | `PlatformWebModule` contributions |
|
||||
| View-filterable regions | Versioned `viewSurfaces` declarations |
|
||||
| Admin sections and module settings | `admin.sections`, including `moduleId`, `kind`, scope group, permission guards, and surface ID |
|
||||
| User settings | `settings.sections` and core settings schemas |
|
||||
| Labels and translations | Generated translation catalogs plus source usage |
|
||||
| Fields and help coverage | Shared form components plus generated TypeScript AST inventory |
|
||||
| API use by the WebUI | Typed API clients plus generated static reference inventory |
|
||||
| Effective configuration | Owning module data plus Policy provenance |
|
||||
|
||||
Runtime introspection is authoritative for an installed system. Static source
|
||||
inventory is authoritative evidence for a checkout or release candidate. The
|
||||
two should be compared in CI and by Ops, not conflated.
|
||||
|
||||
## Generated Inventory
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /mnt/DATA/git/govoplan
|
||||
./.venv/bin/python tools/inventory/platform-interface-inventory.py
|
||||
```
|
||||
|
||||
The command writes:
|
||||
|
||||
- `audit-reports/platform-inventory/platform-interface-inventory.json`
|
||||
- `audit-reports/platform-inventory/platform-interface-inventory.md`
|
||||
|
||||
It combines:
|
||||
|
||||
1. loaded module manifests
|
||||
2. TypeScript AST extraction of fields, label attributes, visible text,
|
||||
translations, frontend routes, navigation, capabilities, and API references
|
||||
3. Python AST extraction of FastAPI route decorators and router prefixes
|
||||
|
||||
The JSON includes exact repository, file, and line evidence. A missing-help
|
||||
entry is a review candidate because dynamic parent components may supply help.
|
||||
A backend route without a static frontend reference is also a review candidate:
|
||||
public APIs, workers, callbacks, health checks, connectors, and dynamic URL
|
||||
assembly are valid explanations.
|
||||
|
||||
`--strict` currently enforces only translation-catalog completeness. Endpoint
|
||||
and help classifications need narrow reviewed baselines before they can become
|
||||
release gates.
|
||||
|
||||
## Admin Information Architecture
|
||||
|
||||
The Admin host uses a tree because system, tenant, group, user, and module
|
||||
settings form a hierarchy rather than one flat list. Every contributed section
|
||||
can identify:
|
||||
|
||||
- its owning `moduleId`
|
||||
- whether it is `management` or `settings`
|
||||
- its system/tenant/group/user scope group
|
||||
- an optional future `parentId`
|
||||
- permission and View visibility requirements
|
||||
|
||||
Existing panels remain their own render owners. The tree only changes discovery
|
||||
and grouping. A later embedded-settings contract may add named slots inside an
|
||||
owning page; it must not allow one module to import another module's private
|
||||
component.
|
||||
|
||||
## Navigation And Workflow
|
||||
|
||||
The intended maximum visible navigation stack is:
|
||||
|
||||
1. global shell context
|
||||
2. one task/object navigation surface
|
||||
3. one workflow stage surface when a workflow is active
|
||||
|
||||
Workflow instance pages should reuse the Campaign stage language: clear stage
|
||||
state, optional/skipped/blocked semantics, partial progress, and a stable current
|
||||
step. Workflow definition pages remain graph editors. Views may activate a
|
||||
focused workflow view that suppresses unrelated shell and module surfaces while
|
||||
retaining an explicit way out.
|
||||
|
||||
Nested module submenus should not be added merely because a data hierarchy
|
||||
exists. Prefer a tree inside configuration/directory surfaces, tabs for sibling
|
||||
views, and the workflow stage rail for ordered work.
|
||||
|
||||
## Safe Meta-Configuration
|
||||
|
||||
The platform can eventually render many configuration editors from versioned
|
||||
JSON Schema and UI Schema supplied by modules. Generated editors remain bounded
|
||||
by:
|
||||
|
||||
- explicit typed schemas and migrations
|
||||
- module-owned validation and preview
|
||||
- Policy locks and provenance
|
||||
- permission and View filtering
|
||||
- preflight, consequence, and rollback information
|
||||
- auditable apply operations
|
||||
|
||||
Custom code, new routes, arbitrary SQL, and executable workflow nodes remain
|
||||
release artifacts. Modeling them as ordinary configuration would create an
|
||||
unreviewed code-execution and migration channel.
|
||||
|
||||
## Next Enforcement Slices
|
||||
|
||||
1. Require every WebUI module route and admin/settings contribution to have
|
||||
matching manifest metadata or a reviewed exception.
|
||||
2. Add stable field IDs and optional help-topic IDs to shared field components.
|
||||
3. Classify each statically unreferenced backend endpoint by consumer type.
|
||||
4. Compare a running installation's OpenAPI and module registry against the
|
||||
release inventory.
|
||||
5. Publish the sanitized installed-system structure through Ops/Docs for
|
||||
authorized administrators.
|
||||
@@ -34,6 +34,7 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
|
||||
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/GovOPlaN/govoplan-dms) |
|
||||
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists) |
|
||||
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/GovOPlaN/govoplan-docs) |
|
||||
| `govoplan-encryption` | `platform` | `../govoplan-encryption` | [govoplan-encryption](https://git.add-ideas.de/GovOPlaN/govoplan-encryption) |
|
||||
| `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/GovOPlaN/govoplan-erp) |
|
||||
| `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/GovOPlaN/govoplan-evaluation) |
|
||||
| `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/GovOPlaN/govoplan-facilities) |
|
||||
|
||||
@@ -191,6 +191,12 @@
|
||||
"description": "GovOPlaN Erp module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/encryption",
|
||||
"color": "b60205",
|
||||
"description": "GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/evaluation",
|
||||
"color": "bfdadc",
|
||||
|
||||
@@ -27,6 +27,7 @@
|
||||
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dms.git", "path": "govoplan-dms"},
|
||||
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
|
||||
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-docs.git", "path": "govoplan-docs"},
|
||||
{"name": "govoplan-encryption", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-encryption.git", "path": "govoplan-encryption"},
|
||||
{"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-erp.git", "path": "govoplan-erp"},
|
||||
{"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-evaluation.git", "path": "govoplan-evaluation"},
|
||||
{"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-facilities.git", "path": "govoplan-facilities"},
|
||||
|
||||
@@ -32,6 +32,7 @@
|
||||
-e ../govoplan-notifications
|
||||
-e ../govoplan-evaluation
|
||||
-e ../govoplan-docs
|
||||
-e ../govoplan-encryption
|
||||
-e ../govoplan-ops
|
||||
httpx==0.28.1
|
||||
httpx2>=2.5,<3
|
||||
|
||||
66
tests/test_platform_interface_inventory.py
Normal file
66
tests/test_platform_interface_inventory.py
Normal file
@@ -0,0 +1,66 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
|
||||
SCRIPT = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "tools"
|
||||
/ "inventory"
|
||||
/ "platform-interface-inventory.py"
|
||||
)
|
||||
SPEC = importlib.util.spec_from_file_location("platform_interface_inventory", SCRIPT)
|
||||
assert SPEC is not None and SPEC.loader is not None
|
||||
inventory = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(inventory)
|
||||
|
||||
|
||||
class PlatformInterfaceInventoryTests(unittest.TestCase):
|
||||
def test_canonical_api_path_normalizes_versions_and_parameters(self) -> None:
|
||||
self.assertEqual(
|
||||
inventory.canonical_api_path(
|
||||
"http://localhost/api/v1/campaigns/${campaignId}?limit=10"
|
||||
),
|
||||
"/campaigns/{}",
|
||||
)
|
||||
self.assertEqual(
|
||||
inventory.canonical_api_path("/api/v2/campaigns/{campaign_id}"),
|
||||
"/campaigns/{}",
|
||||
)
|
||||
|
||||
def test_fastapi_route_scanner_includes_router_prefix(self) -> None:
|
||||
tree = ast.parse(
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
router = APIRouter(prefix="/api/v1/items")
|
||||
|
||||
@router.get("/{item_id}")
|
||||
def read_item(item_id: str):
|
||||
return item_id
|
||||
"""
|
||||
)
|
||||
prefixes = inventory._router_prefixes(tree)
|
||||
function = next(
|
||||
node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)
|
||||
)
|
||||
|
||||
route = inventory._endpoint_from_decorator(
|
||||
function.decorator_list[0],
|
||||
prefixes=prefixes,
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
route,
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/items/{item_id}",
|
||||
"router": "router",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
366
tools/inventory/extract-webui-structure.mjs
Normal file
366
tools/inventory/extract-webui-structure.mjs
Normal file
@@ -0,0 +1,366 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const [metaRootArgument] = process.argv.slice(2);
|
||||
if (!metaRootArgument) {
|
||||
throw new Error("Usage: extract-webui-structure.mjs META_ROOT");
|
||||
}
|
||||
|
||||
const metaRoot = path.resolve(metaRootArgument);
|
||||
const repositoryCatalog = JSON.parse(
|
||||
fs.readFileSync(path.join(metaRoot, "repositories.json"), "utf8")
|
||||
);
|
||||
const workspaceRoot = path.resolve(repositoryCatalog.default_parent);
|
||||
const typescriptPath = path.join(
|
||||
workspaceRoot,
|
||||
"govoplan-core",
|
||||
"webui",
|
||||
"node_modules",
|
||||
"typescript",
|
||||
"lib",
|
||||
"typescript.js"
|
||||
);
|
||||
const ts = await import(pathToFileURL(typescriptPath).href);
|
||||
|
||||
const fieldComponents = new Set([
|
||||
"input",
|
||||
"select",
|
||||
"textarea",
|
||||
"Checkbox",
|
||||
"DateTimeField",
|
||||
"EmailAddressInput",
|
||||
"ReferenceSelect",
|
||||
"SearchableSelect",
|
||||
"ToggleSwitch"
|
||||
]);
|
||||
const fieldComponentPattern =
|
||||
/(?:Input|Field|Select|Picker|Toggle|Checkbox|Radio|Editor)$/;
|
||||
const labelAttributes = new Set([
|
||||
"aria-label",
|
||||
"description",
|
||||
"help",
|
||||
"helperText",
|
||||
"helpText",
|
||||
"label",
|
||||
"placeholder",
|
||||
"title"
|
||||
]);
|
||||
const helpAttributes = new Set([
|
||||
"description",
|
||||
"help",
|
||||
"helperText",
|
||||
"helpText"
|
||||
]);
|
||||
|
||||
const result = {
|
||||
fields: [],
|
||||
labels: [],
|
||||
visibleText: [],
|
||||
translationCatalog: {},
|
||||
translationUsages: [],
|
||||
dynamicTranslationUsages: [],
|
||||
routes: [],
|
||||
navigation: [],
|
||||
frontendApiReferences: [],
|
||||
uiCapabilities: []
|
||||
};
|
||||
|
||||
for (const repository of repositoryCatalog.repositories) {
|
||||
const sourceRoot = path.join(workspaceRoot, repository.path, "webui", "src");
|
||||
if (!fs.existsSync(sourceRoot)) continue;
|
||||
for (const sourcePath of sourceFiles(sourceRoot)) {
|
||||
inspectSource(repository.name, sourceRoot, sourcePath);
|
||||
}
|
||||
}
|
||||
|
||||
process.stdout.write(`${JSON.stringify(result)}\n`);
|
||||
|
||||
function sourceFiles(root) {
|
||||
const files = [];
|
||||
const pending = [root];
|
||||
while (pending.length > 0) {
|
||||
const current = pending.pop();
|
||||
for (const entry of fs.readdirSync(current, { withFileTypes: true })) {
|
||||
if (
|
||||
entry.name === "node_modules" ||
|
||||
entry.name.startsWith(".") ||
|
||||
entry.name === "dist"
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const candidate = path.join(current, entry.name);
|
||||
if (entry.isDirectory()) pending.push(candidate);
|
||||
else if (/\.(?:ts|tsx)$/.test(entry.name)) files.push(candidate);
|
||||
}
|
||||
}
|
||||
return files.sort();
|
||||
}
|
||||
|
||||
function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
const sourceText = fs.readFileSync(sourcePath, "utf8");
|
||||
const sourceFile = ts.createSourceFile(
|
||||
sourcePath,
|
||||
sourceText,
|
||||
ts.ScriptTarget.Latest,
|
||||
true,
|
||||
sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS
|
||||
);
|
||||
const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath);
|
||||
|
||||
function location(node) {
|
||||
const position = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile));
|
||||
return {
|
||||
repository,
|
||||
file: relativeFile,
|
||||
line: position.line + 1,
|
||||
column: position.character + 1
|
||||
};
|
||||
}
|
||||
|
||||
function visit(node) {
|
||||
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
|
||||
inspectJsxOpening(node, location);
|
||||
}
|
||||
if (ts.isJsxText(node)) {
|
||||
const value = node.getText(sourceFile).replace(/\s+/g, " ").trim();
|
||||
if (value) {
|
||||
result.visibleText.push({
|
||||
...location(node),
|
||||
value,
|
||||
translationKey: value.startsWith("i18n:") ? value : null
|
||||
});
|
||||
}
|
||||
}
|
||||
if (ts.isStringLiteralLike(node)) {
|
||||
inspectString(node.text, node, location);
|
||||
} else if (ts.isTemplateExpression(node)) {
|
||||
inspectString(templateText(node), node, location);
|
||||
}
|
||||
if (ts.isPropertyAssignment(node)) {
|
||||
inspectProperty(node, location);
|
||||
inspectTranslationProperty(node, location);
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
|
||||
visit(sourceFile);
|
||||
|
||||
function inspectJsxOpening(node, locate) {
|
||||
const component = node.tagName.getText(sourceFile);
|
||||
const attributes = new Map();
|
||||
for (const attribute of node.attributes.properties) {
|
||||
if (!ts.isJsxAttribute(attribute)) continue;
|
||||
attributes.set(
|
||||
attribute.name.getText(sourceFile),
|
||||
jsxAttributeValue(attribute)
|
||||
);
|
||||
}
|
||||
for (const [attribute, value] of attributes) {
|
||||
if (!labelAttributes.has(attribute) || value === null) continue;
|
||||
result.labels.push({
|
||||
...locate(node),
|
||||
component,
|
||||
attribute,
|
||||
value,
|
||||
translationKey: value.startsWith("i18n:") ? value : null
|
||||
});
|
||||
}
|
||||
|
||||
const isField =
|
||||
fieldComponents.has(component) ||
|
||||
(fieldComponentPattern.test(component) && component !== "FormField");
|
||||
if (!isField) return;
|
||||
|
||||
const parentFormField = nearestFormField(node);
|
||||
const parentAttributes = parentFormField
|
||||
? jsxAttributes(parentFormField)
|
||||
: new Map();
|
||||
const label =
|
||||
attributes.get("label") ??
|
||||
attributes.get("aria-label") ??
|
||||
parentAttributes.get("label") ??
|
||||
null;
|
||||
const help = firstAttribute(attributes, helpAttributes) ??
|
||||
firstAttribute(parentAttributes, helpAttributes);
|
||||
result.fields.push({
|
||||
...locate(node),
|
||||
component,
|
||||
name:
|
||||
attributes.get("name") ??
|
||||
attributes.get("id") ??
|
||||
attributes.get("field") ??
|
||||
null,
|
||||
label,
|
||||
placeholder: attributes.get("placeholder") ?? null,
|
||||
help: help ?? null,
|
||||
helpCandidate: help === null
|
||||
});
|
||||
}
|
||||
|
||||
function nearestFormField(node) {
|
||||
let current = node.parent;
|
||||
while (current) {
|
||||
if (
|
||||
ts.isJsxElement(current) &&
|
||||
current.openingElement.tagName.getText(sourceFile) === "FormField"
|
||||
) {
|
||||
return current.openingElement;
|
||||
}
|
||||
if (
|
||||
ts.isJsxOpeningElement(current) ||
|
||||
ts.isJsxSelfClosingElement(current)
|
||||
) {
|
||||
const name = current.tagName.getText(sourceFile);
|
||||
if (name !== "FormField") return null;
|
||||
}
|
||||
current = current.parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function jsxAttributes(node) {
|
||||
const mapped = new Map();
|
||||
for (const attribute of node.attributes.properties) {
|
||||
if (!ts.isJsxAttribute(attribute)) continue;
|
||||
mapped.set(
|
||||
attribute.name.getText(sourceFile),
|
||||
jsxAttributeValue(attribute)
|
||||
);
|
||||
}
|
||||
return mapped;
|
||||
}
|
||||
|
||||
function jsxAttributeValue(attribute) {
|
||||
if (!attribute.initializer) return "true";
|
||||
if (ts.isStringLiteral(attribute.initializer)) return attribute.initializer.text;
|
||||
if (!ts.isJsxExpression(attribute.initializer)) return null;
|
||||
const expression = attribute.initializer.expression;
|
||||
if (!expression) return null;
|
||||
if (ts.isStringLiteralLike(expression)) return expression.text;
|
||||
if (ts.isNoSubstitutionTemplateLiteral(expression)) return expression.text;
|
||||
if (ts.isTemplateExpression(expression)) return templateText(expression);
|
||||
return null;
|
||||
}
|
||||
|
||||
function inspectString(value, node, locate) {
|
||||
if (value.startsWith("i18n:") && value.length > "i18n:".length) {
|
||||
const target = value.includes("${}") || isStringPrefixCheck(node)
|
||||
? result.dynamicTranslationUsages
|
||||
: result.translationUsages;
|
||||
target.push({ ...locate(node), key: value });
|
||||
}
|
||||
if (value.includes("/api/")) {
|
||||
result.frontendApiReferences.push({ ...locate(node), path: value });
|
||||
}
|
||||
}
|
||||
|
||||
function isStringPrefixCheck(node) {
|
||||
const call = node.parent;
|
||||
if (!ts.isCallExpression(call) || !ts.isPropertyAccessExpression(call.expression)) {
|
||||
return false;
|
||||
}
|
||||
return ["endsWith", "includes", "startsWith"].includes(
|
||||
call.expression.name.text
|
||||
);
|
||||
}
|
||||
|
||||
function inspectProperty(node, locate) {
|
||||
const propertyName = propertyNameText(node.name);
|
||||
const value = staticExpressionText(node.initializer);
|
||||
if (value === null) return;
|
||||
if (propertyName === "path" && value.startsWith("/") && !value.includes("/api/")) {
|
||||
result.routes.push({ ...locate(node), path: value });
|
||||
}
|
||||
if (propertyName === "to" && value.startsWith("/")) {
|
||||
result.navigation.push({ ...locate(node), path: value });
|
||||
}
|
||||
if (
|
||||
ancestorPropertyName(node, "uiCapabilities") &&
|
||||
typeof propertyName === "string"
|
||||
) {
|
||||
result.uiCapabilities.push({ ...locate(node), name: propertyName });
|
||||
}
|
||||
}
|
||||
|
||||
function inspectTranslationProperty(node, locate) {
|
||||
const key = propertyNameText(node.name);
|
||||
if (!key?.startsWith("i18n:")) return;
|
||||
const value = staticExpressionText(node.initializer);
|
||||
if (value === null) return;
|
||||
const locale = nearestLocaleProperty(node);
|
||||
if (!locale) return;
|
||||
result.translationCatalog[locale] ??= {};
|
||||
result.translationCatalog[locale][key] = {
|
||||
value,
|
||||
...locate(node)
|
||||
};
|
||||
}
|
||||
|
||||
function nearestLocaleProperty(node) {
|
||||
let current = node.parent;
|
||||
while (current) {
|
||||
if (
|
||||
ts.isPropertyAssignment(current) &&
|
||||
(propertyNameText(current.name) === "en" ||
|
||||
propertyNameText(current.name) === "de")
|
||||
) {
|
||||
return propertyNameText(current.name);
|
||||
}
|
||||
current = current.parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function ancestorPropertyName(node, expected) {
|
||||
let current = node.parent;
|
||||
while (current) {
|
||||
if (
|
||||
ts.isPropertyAssignment(current) &&
|
||||
propertyNameText(current.name) === expected
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
current = current.parent;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function firstAttribute(attributes, names) {
|
||||
for (const name of names) {
|
||||
const value = attributes.get(name);
|
||||
if (value !== undefined && value !== null) return value;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function propertyNameText(name) {
|
||||
if (
|
||||
ts.isIdentifier(name) ||
|
||||
ts.isStringLiteral(name) ||
|
||||
ts.isNumericLiteral(name)
|
||||
) {
|
||||
return name.text;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function staticExpressionText(node) {
|
||||
if (ts.isStringLiteralLike(node) || ts.isNoSubstitutionTemplateLiteral(node)) {
|
||||
return node.text;
|
||||
}
|
||||
if (ts.isTemplateExpression(node)) return templateText(node);
|
||||
return null;
|
||||
}
|
||||
|
||||
function templateText(node) {
|
||||
return (
|
||||
node.head.text +
|
||||
node.templateSpans
|
||||
.map((span) => "${}" + span.literal.text)
|
||||
.join("")
|
||||
);
|
||||
}
|
||||
462
tools/inventory/platform-interface-inventory.py
Normal file
462
tools/inventory/platform-interface-inventory.py
Normal file
@@ -0,0 +1,462 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract a source-derived GovOPlaN UI, translation, module, and API inventory."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
from collections import Counter
|
||||
from dataclasses import asdict, is_dataclass
|
||||
import importlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Any
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
HTTP_METHODS = {"delete", "get", "head", "options", "patch", "post", "put"}
|
||||
PATH_PARAMETER = re.compile(r"\$\{[^{}]*\}|\{[^{}]+\}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
default=META_ROOT / "audit-reports" / "platform-inventory",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--strict",
|
||||
action="store_true",
|
||||
help="Fail when a used translation key is absent from a generated locale catalog.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
catalog = json.loads(
|
||||
(META_ROOT / "repositories.json").read_text(encoding="utf-8")
|
||||
)
|
||||
workspace_root = Path(catalog["default_parent"]).resolve()
|
||||
webui = _extract_webui()
|
||||
backend_endpoints = _extract_backend_endpoints(catalog, workspace_root)
|
||||
manifests = _extract_manifests(catalog, workspace_root)
|
||||
inventory = _assemble_inventory(
|
||||
webui=webui,
|
||||
backend_endpoints=backend_endpoints,
|
||||
manifests=manifests,
|
||||
)
|
||||
|
||||
output_dir = args.output_dir.resolve()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
json_path = output_dir / "platform-interface-inventory.json"
|
||||
markdown_path = output_dir / "platform-interface-inventory.md"
|
||||
json_path.write_text(
|
||||
json.dumps(inventory, indent=2, sort_keys=True) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
markdown_path.write_text(_render_markdown(inventory), encoding="utf-8")
|
||||
print(f"Platform inventory JSON: {json_path}")
|
||||
print(f"Platform inventory summary: {markdown_path}")
|
||||
|
||||
if args.strict and inventory["translation_health"]["missing_catalog_entries"]:
|
||||
print(
|
||||
"Used translation keys are missing from generated catalogs.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def _extract_webui() -> dict[str, Any]:
|
||||
helper = META_ROOT / "tools" / "inventory" / "extract-webui-structure.mjs"
|
||||
completed = subprocess.run(
|
||||
["node", str(helper), str(META_ROOT)],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return json.loads(completed.stdout)
|
||||
|
||||
|
||||
def _extract_backend_endpoints(
|
||||
catalog: dict[str, Any],
|
||||
workspace_root: Path,
|
||||
) -> list[dict[str, Any]]:
|
||||
endpoints: list[dict[str, Any]] = []
|
||||
for repository in catalog["repositories"]:
|
||||
repository_root = workspace_root / repository["path"]
|
||||
source_root = repository_root / "src"
|
||||
if not source_root.is_dir():
|
||||
continue
|
||||
for source_path in sorted(source_root.rglob("*.py")):
|
||||
try:
|
||||
tree = ast.parse(
|
||||
source_path.read_text(encoding="utf-8"),
|
||||
filename=str(source_path),
|
||||
)
|
||||
except (OSError, SyntaxError, UnicodeDecodeError):
|
||||
continue
|
||||
prefixes = _router_prefixes(tree)
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
continue
|
||||
for decorator in node.decorator_list:
|
||||
endpoint = _endpoint_from_decorator(
|
||||
decorator,
|
||||
prefixes=prefixes,
|
||||
)
|
||||
if endpoint is None:
|
||||
continue
|
||||
endpoints.append(
|
||||
{
|
||||
"repository": repository["name"],
|
||||
"file": str(source_path.relative_to(repository_root)),
|
||||
"line": node.lineno,
|
||||
"handler": node.name,
|
||||
**endpoint,
|
||||
}
|
||||
)
|
||||
return sorted(
|
||||
endpoints,
|
||||
key=lambda item: (
|
||||
item["repository"],
|
||||
item["path"],
|
||||
item["method"],
|
||||
item["file"],
|
||||
item["line"],
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _router_prefixes(tree: ast.AST) -> dict[str, str]:
|
||||
prefixes: dict[str, str] = {}
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, (ast.Assign, ast.AnnAssign)):
|
||||
continue
|
||||
value = node.value
|
||||
if not isinstance(value, ast.Call):
|
||||
continue
|
||||
function_name = _call_name(value.func)
|
||||
if function_name not in {"APIRouter", "fastapi.APIRouter"}:
|
||||
continue
|
||||
prefix = ""
|
||||
for keyword in value.keywords:
|
||||
if keyword.arg == "prefix":
|
||||
prefix = _static_string(keyword.value) or ""
|
||||
targets = node.targets if isinstance(node, ast.Assign) else [node.target]
|
||||
for target in targets:
|
||||
if isinstance(target, ast.Name):
|
||||
prefixes[target.id] = prefix
|
||||
return prefixes
|
||||
|
||||
|
||||
def _endpoint_from_decorator(
|
||||
decorator: ast.expr,
|
||||
*,
|
||||
prefixes: dict[str, str],
|
||||
) -> dict[str, str] | None:
|
||||
if not isinstance(decorator, ast.Call) or not isinstance(
|
||||
decorator.func, ast.Attribute
|
||||
):
|
||||
return None
|
||||
method = decorator.func.attr.lower()
|
||||
if method not in HTTP_METHODS or not decorator.args:
|
||||
return None
|
||||
route = _static_string(decorator.args[0])
|
||||
if route is None:
|
||||
return None
|
||||
owner = decorator.func.value.id if isinstance(decorator.func.value, ast.Name) else ""
|
||||
prefix = prefixes.get(owner, "")
|
||||
return {
|
||||
"method": method.upper(),
|
||||
"path": _join_route(prefix, route),
|
||||
"router": owner,
|
||||
}
|
||||
|
||||
|
||||
def _extract_manifests(
|
||||
catalog: dict[str, Any],
|
||||
workspace_root: Path,
|
||||
) -> list[dict[str, Any]]:
|
||||
source_roots = [
|
||||
workspace_root / repository["path"] / "src"
|
||||
for repository in catalog["repositories"]
|
||||
if (workspace_root / repository["path"] / "src").is_dir()
|
||||
]
|
||||
sys.path[:0] = [str(path) for path in source_roots]
|
||||
manifests: list[dict[str, Any]] = []
|
||||
for repository in catalog["repositories"]:
|
||||
source_root = workspace_root / repository["path"] / "src"
|
||||
if not source_root.is_dir():
|
||||
continue
|
||||
for manifest_path in sorted(source_root.glob("*/backend/manifest.py")):
|
||||
module_name = ".".join(
|
||||
manifest_path.relative_to(source_root).with_suffix("").parts
|
||||
)
|
||||
loaded = importlib.import_module(module_name)
|
||||
manifest = loaded.get_manifest()
|
||||
frontend = manifest.frontend
|
||||
manifests.append(
|
||||
{
|
||||
"repository": repository["name"],
|
||||
"id": manifest.id,
|
||||
"name": manifest.name,
|
||||
"version": manifest.version,
|
||||
"dependencies": list(manifest.dependencies),
|
||||
"optional_dependencies": list(manifest.optional_dependencies),
|
||||
"required_capabilities": list(manifest.required_capabilities),
|
||||
"provided_interfaces": [
|
||||
{"name": item.name, "version": item.version}
|
||||
for item in manifest.provides_interfaces
|
||||
],
|
||||
"runtime_capabilities": sorted(manifest.capability_factories),
|
||||
"permissions": [
|
||||
{
|
||||
"scope": permission.scope,
|
||||
"label": permission.label,
|
||||
"level": permission.level,
|
||||
}
|
||||
for permission in manifest.permissions
|
||||
],
|
||||
"frontend": (
|
||||
{
|
||||
"package": frontend.package_name,
|
||||
"routes": [
|
||||
_plain_value(route) for route in frontend.routes
|
||||
],
|
||||
"nav_items": [
|
||||
_plain_value(item) for item in frontend.nav_items
|
||||
],
|
||||
"view_surfaces": [
|
||||
_plain_value(surface)
|
||||
for surface in frontend.view_surfaces
|
||||
],
|
||||
}
|
||||
if frontend is not None
|
||||
else None
|
||||
),
|
||||
}
|
||||
)
|
||||
return sorted(manifests, key=lambda item: item["id"])
|
||||
|
||||
|
||||
def _assemble_inventory(
|
||||
*,
|
||||
webui: dict[str, Any],
|
||||
backend_endpoints: list[dict[str, Any]],
|
||||
manifests: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
frontend_refs = webui["frontendApiReferences"]
|
||||
frontend_paths = {
|
||||
canonical_api_path(reference["path"])
|
||||
for reference in frontend_refs
|
||||
if canonical_api_path(reference["path"])
|
||||
}
|
||||
unreferenced = [
|
||||
endpoint
|
||||
for endpoint in backend_endpoints
|
||||
if canonical_api_path(endpoint["path"]) not in frontend_paths
|
||||
]
|
||||
usages = {item["key"] for item in webui["translationUsages"]}
|
||||
catalogs = webui["translationCatalog"]
|
||||
catalog_keys = {
|
||||
locale: set(entries)
|
||||
for locale, entries in catalogs.items()
|
||||
}
|
||||
expected_locales = sorted(catalog_keys)
|
||||
missing_catalog_entries = [
|
||||
{
|
||||
"key": key,
|
||||
"missing_locales": [
|
||||
locale
|
||||
for locale in expected_locales
|
||||
if key not in catalog_keys[locale]
|
||||
],
|
||||
}
|
||||
for key in sorted(usages)
|
||||
if any(key not in catalog_keys[locale] for locale in expected_locales)
|
||||
]
|
||||
fields = webui["fields"]
|
||||
help_candidates = [field for field in fields if field["helpCandidate"]]
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"scope": {
|
||||
"source": "local GovOPlaN repository catalog",
|
||||
"limitations": [
|
||||
"Static extraction cannot resolve runtime-computed labels, routes, or API paths.",
|
||||
"A backend endpoint without a static WebUI reference may intentionally serve public clients, workers, connectors, or external integrations.",
|
||||
"A field marked as a help candidate may receive contextual help from a surrounding dynamic component.",
|
||||
],
|
||||
},
|
||||
"modules": manifests,
|
||||
"ui": {
|
||||
"fields": fields,
|
||||
"labels": webui["labels"],
|
||||
"visible_text": webui["visibleText"],
|
||||
"routes": webui["routes"],
|
||||
"navigation": webui["navigation"],
|
||||
"ui_capabilities": webui["uiCapabilities"],
|
||||
"help_candidates": help_candidates,
|
||||
},
|
||||
"translations": {
|
||||
"catalog": catalogs,
|
||||
"usages": webui["translationUsages"],
|
||||
"dynamic_usages": webui["dynamicTranslationUsages"],
|
||||
},
|
||||
"translation_health": {
|
||||
"locales": expected_locales,
|
||||
"used_keys": len(usages),
|
||||
"missing_catalog_entries": missing_catalog_entries,
|
||||
},
|
||||
"api": {
|
||||
"backend_endpoints": backend_endpoints,
|
||||
"frontend_references": frontend_refs,
|
||||
"unreferenced_by_static_webui_scan": unreferenced,
|
||||
},
|
||||
"summary": {
|
||||
"modules": len(manifests),
|
||||
"ui_fields": len(fields),
|
||||
"ui_fields_with_static_help": len(fields) - len(help_candidates),
|
||||
"help_review_candidates": len(help_candidates),
|
||||
"label_attributes": len(webui["labels"]),
|
||||
"visible_text_nodes": len(webui["visibleText"]),
|
||||
"frontend_routes": len(webui["routes"]),
|
||||
"backend_endpoints": len(backend_endpoints),
|
||||
"frontend_api_references": len(frontend_refs),
|
||||
"backend_endpoints_without_static_webui_reference": len(unreferenced),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _render_markdown(inventory: dict[str, Any]) -> str:
|
||||
summary = inventory["summary"]
|
||||
missing = inventory["translation_health"]["missing_catalog_entries"]
|
||||
help_by_repository = Counter(
|
||||
item["repository"] for item in inventory["ui"]["help_candidates"]
|
||||
)
|
||||
endpoint_by_repository = Counter(
|
||||
item["repository"]
|
||||
for item in inventory["api"]["unreferenced_by_static_webui_scan"]
|
||||
)
|
||||
lines = [
|
||||
"# GovOPlaN Platform Interface Inventory",
|
||||
"",
|
||||
"Generated from runtime module manifests plus TypeScript and Python ASTs.",
|
||||
"Counts are source evidence, not a claim that every surface is enabled or reachable.",
|
||||
"",
|
||||
"## Summary",
|
||||
"",
|
||||
f"- Modules: {summary['modules']}",
|
||||
f"- UI fields: {summary['ui_fields']}",
|
||||
f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}",
|
||||
f"- Help review candidates: {summary['help_review_candidates']}",
|
||||
f"- Label attributes: {summary['label_attributes']}",
|
||||
f"- Frontend routes: {summary['frontend_routes']}",
|
||||
f"- Backend endpoints: {summary['backend_endpoints']}",
|
||||
f"- Frontend API references: {summary['frontend_api_references']}",
|
||||
(
|
||||
"- Backend endpoints without a static WebUI reference: "
|
||||
f"{summary['backend_endpoints_without_static_webui_reference']}"
|
||||
),
|
||||
f"- Used translation keys missing from a locale catalog: {len(missing)}",
|
||||
"",
|
||||
"## Help Review Candidates",
|
||||
"",
|
||||
"| Repository | Fields |",
|
||||
"| --- | ---: |",
|
||||
]
|
||||
lines.extend(
|
||||
f"| `{repository}` | {count} |"
|
||||
for repository, count in sorted(help_by_repository.items())
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"## Endpoints Without Static WebUI References",
|
||||
"",
|
||||
"These are review candidates. Public APIs, worker callbacks, connector",
|
||||
"endpoints, health checks, and dynamically assembled paths are legitimate",
|
||||
"reasons for appearing here.",
|
||||
"",
|
||||
"| Repository | Endpoints |",
|
||||
"| --- | ---: |",
|
||||
]
|
||||
)
|
||||
lines.extend(
|
||||
f"| `{repository}` | {count} |"
|
||||
for repository, count in sorted(endpoint_by_repository.items())
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"## Interpretation",
|
||||
"",
|
||||
"Use the JSON artifact for exact file and line evidence. Missing help is",
|
||||
"a triage list, not an automatic defect. Endpoint coverage requires an",
|
||||
"owner classification before enforcement. Runtime-computed structures",
|
||||
"need explicit manifest metadata to become canonically visible.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def canonical_api_path(value: str) -> str:
|
||||
path = value.split("?", 1)[0].strip()
|
||||
if "/api/" in path:
|
||||
path = path[path.index("/api/") :]
|
||||
path = re.sub(r"^/api/v\d+", "", path)
|
||||
path = PATH_PARAMETER.sub("{}", path)
|
||||
path = re.sub(r"/+", "/", path)
|
||||
return path.rstrip("/") or "/"
|
||||
|
||||
|
||||
def _join_route(prefix: str, route: str) -> str:
|
||||
return f"/{prefix.strip('/')}/{route.strip('/')}".replace("//", "/")
|
||||
|
||||
|
||||
def _static_string(node: ast.AST) -> str | None:
|
||||
if isinstance(node, ast.Constant) and isinstance(node.value, str):
|
||||
return node.value
|
||||
if isinstance(node, ast.JoinedStr):
|
||||
pieces: list[str] = []
|
||||
for value in node.values:
|
||||
if isinstance(value, ast.Constant) and isinstance(value.value, str):
|
||||
pieces.append(value.value)
|
||||
elif isinstance(value, ast.FormattedValue):
|
||||
pieces.append("${}")
|
||||
else:
|
||||
return None
|
||||
return "".join(pieces)
|
||||
return None
|
||||
|
||||
|
||||
def _call_name(node: ast.AST) -> str:
|
||||
if isinstance(node, ast.Name):
|
||||
return node.id
|
||||
if isinstance(node, ast.Attribute):
|
||||
parent = _call_name(node.value)
|
||||
return f"{parent}.{node.attr}" if parent else node.attr
|
||||
return ""
|
||||
|
||||
|
||||
def _plain_value(value: Any) -> Any:
|
||||
if is_dataclass(value):
|
||||
return {
|
||||
key: _plain_value(item)
|
||||
for key, item in asdict(value).items()
|
||||
}
|
||||
if isinstance(value, tuple):
|
||||
return [_plain_value(item) for item in value]
|
||||
if isinstance(value, dict):
|
||||
return {
|
||||
str(key): _plain_value(item)
|
||||
for key, item in value.items()
|
||||
}
|
||||
return value
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -91,6 +91,7 @@ PACKAGE_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-dataflow"
|
||||
"$PARENT/govoplan-datasources"
|
||||
"$PARENT/govoplan-docs"
|
||||
"$PARENT/govoplan-encryption"
|
||||
"$PARENT/govoplan-facilities"
|
||||
"$PARENT/govoplan-files"
|
||||
"$PARENT/govoplan-forms-runtime"
|
||||
|
||||
Reference in New Issue
Block a user