feat(devkit): add resumable workspace automation and UI review tooling
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m30s

Verified with the coordinated workspace changes by devkit full run
2026-09-08T225814-186389-0000-3e3ed7cd (all seven phases passed).
This shared UI pass does not mark the individual module reviews complete.
This commit is contained in:
2026-09-09 02:03:17 +02:00
parent 14b19fbead
commit 2ffdb23f69
67 changed files with 17306 additions and 94 deletions
+7
View File
@@ -15,6 +15,13 @@ platform behavior remains owned by the corresponding module repository.
## Working Rules ## Working Rules
- Start repeated workflows with `./devkit commands`; use `context --changed`,
`check --profile quick --changed --dry-run`, and `review MODULE` instead of
reconstructing repository/check inventories. See `docs/operations/DEVKIT.md`.
- Prefer compact check receipts (`status`, `summary`, `logs`) to repeated full
log dumps. `check --profile full` retains the required focused gate; targeted
profiles or cached results do not waive required verification or approvals.
- Treat Gitea issues as the canonical backlog and state log. - Treat Gitea issues as the canonical backlog and state log.
- Preserve optional module boundaries and use Core contracts or capabilities for integrations. - Preserve optional module boundaries and use Core contracts or capabilities for integrations.
- Prefer targeted checks before full workspace scans. - Prefer targeted checks before full workspace scans.
+11
View File
@@ -19,6 +19,17 @@ installed modules/connectors discovered by core.
## Common Commands ## Common Commands
For repeated development and review workflows, start with the unified command
suite. [Developer command guide](docs/operations/DEVKIT.md) documents profiles,
evidence, safe Git/release operations and reuse in other projects.
```sh
./devkit commands
./devkit context --changed
./devkit check --profile quick --changed --dry-run
./devkit review campaign
```
Create the whole-product development virtualenv in this meta repository: Create the whole-product development virtualenv in this meta repository:
```sh ```sh
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# One stable entry point; no environment activation, installation or server startup.
set -eu
DEVKIT_ROOT=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
if [ -n "${PYTHON:-}" ]; then
DEVKIT_PYTHON=$(command -v "$PYTHON") || { echo "devkit: configured PYTHON is unavailable" >&2; exit 127; }
elif [ -x "$DEVKIT_ROOT/.venv/bin/python" ]; then
DEVKIT_PYTHON="$DEVKIT_ROOT/.venv/bin/python"
else
DEVKIT_PYTHON=$(command -v python3)
fi
exec "$DEVKIT_PYTHON" "$DEVKIT_ROOT/tools/devkit/devkit.py" "$@"
+3
View File
@@ -64,6 +64,7 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
| Evidence collection and promotion | [Target Maturity Evidence Runbook](operations/TARGET_MATURITY_EVIDENCE_RUNBOOK.md) | | Evidence collection and promotion | [Target Maturity Evidence Runbook](operations/TARGET_MATURITY_EVIDENCE_RUNBOOK.md) |
| Package publication and consumption | [Package Registry Releases](operations/PACKAGE_REGISTRY_RELEASES.md) | | Package publication and consumption | [Package Registry Releases](operations/PACKAGE_REGISTRY_RELEASES.md) |
| Release-console operation | [Release Console](operations/RELEASE_CONSOLE.md) | | Release-console operation | [Release Console](operations/RELEASE_CONSOLE.md) |
| Repeated development, verification and evidence commands | [Developer Command Suite](operations/DEVKIT.md) |
| Module compatibility and install behavior | [Module Contracts and Installs](operations/MODULE_CONTRACTS_AND_INSTALLS.md) | | Module compatibility and install behavior | [Module Contracts and Installs](operations/MODULE_CONTRACTS_AND_INSTALLS.md) |
| Security-audit toolchain | [Security Audit](operations/SECURITY_AUDIT.md) | | Security-audit toolchain | [Security Audit](operations/SECURITY_AUDIT.md) |
@@ -75,6 +76,8 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
meta, module, deployment, and website content. meta, module, deployment, and website content.
- [Gitea Issues](project/GITEA_ISSUES.md) defines labels, templates, import, and - [Gitea Issues](project/GITEA_ISSUES.md) defines labels, templates, import, and
state-update conventions. state-update conventions.
- [UI Review Program](project/UI_REVIEW_PROGRAM.md) defines the principle-led,
per-module review process and links the canonical Gitea review inventory.
## Evidence And Archive ## Evidence And Archive
+300
View File
@@ -0,0 +1,300 @@
# Developer command suite
`./devkit` is the maintained entry point for repeated development work. It uses
ordinary local programs, not an AI service. People, CI and coding agents use the
same commands and results. It composes existing GovOPlaN checks and release
services rather than defining parallel product rules.
## Start here
From the Meta checkout:
```sh
./devkit commands
./devkit doctor --repo core
./devkit context --changed
./devkit check --profile quick --repo campaign --dry-run
./devkit coverage --profile ui --repo campaign
./devkit check --profile quick --repo campaign
./devkit latest
./devkit review campaign
```
Use `--help` on each command. `--json` (or `--format json`) returns structured
results; the default summary intentionally omits full logs and source contents.
Global options work before or after the command. `--workspace-root` is the parent
directory of the registered repositories, not the Core checkout. Unknown
repository filters fail rather than silently widening/narrowing scope.
The launcher prefers Meta's `.venv/bin/python`, or the explicit `PYTHON`
executable, with `python3` as the no-venv fallback. Checks resolve `NODE` and
`NPM` from explicit environment settings or `PATH`; no username-specific Node
installation path is required. Doctor reports repairs but never installs
packages, changes configuration, kills an occupied port or starts a server.
## Command contracts
| Command | Purpose | Effects |
| --- | --- | --- |
| `context [--changed] [--repo NAME]` | Offline Git state, instruction/documentation paths and review links | Reads only; upstream counts are not freshly fetched |
| `doctor [--repo NAME] [--profile PROFILE]` | Interpreter, dependency, browser and test-resource preflight | Reads only; repair suggestions are not executed |
| `check --profile PROFILE` | Registered checks with explanations, bounded logs and a receipt | Runs trusted tests/builds; `--dry-run` only plans |
| `coverage --profile PROFILE [--repo NAME]` | Declared suite dispositions and explicit coverage limits | Reads only; not execution evidence |
| `runs [--limit 10] [--before RUN]`, `latest` | Discover recent runs without looking up state-directory paths | Reads only; malformed latest evidence is not replaced with an older pass |
| `status RUN`, `summary RUN` | Stage counts and failure log locations | Reads only |
| `logs RUN --stage STAGE --tail 40 [--final-only]` | Provisional live output or a hash-verified final stage log | Reads only; provisional output never proves a pass |
| `resume RUN` | Replan the previous check selection and reuse eligible successful stages | Runs remaining checks, creating a new receipt |
| `recover RUN [--apply]` | Recover an abandoned check-run record after acquiring its OS lock | No test replay; apply changes only the local record |
| `review MODULE` | Source inventory, principle revision, issue links, check plan and walkthrough checklist | Does not perform or complete a module review |
| `docs audit [--repo NAME] [--changed]` | Owning manifest/help/translation checks and visible-label gaps | Local checks only; never translates or edits content |
| `issues note …` | Preview and explicitly append deduplicated evidence | Remote comment creation only with `--apply` |
| `release …` | Existing durable release planning/execution/recovery | Explicit `--apply`, request IDs and existing step confirmations |
| `git …` | Frozen, explicit-path maintenance commit and exact branch push | Preview first; mutations require `--apply`; no stage-all, force or tags |
Git maintenance is a separate, selected-path workflow; see
[Git maintenance](DEVKIT_MAINTENANCE.md). Issue notes and module bundles are
documented in [Evidence and review commands](DEVKIT_EVIDENCE.md). The complete
release command syntax, limitations and examples are in
[Headless release operations](DEVKIT_RELEASE.md).
## Check profiles and their limits
- `quick`: existing manifest/interface invariants, import/help guards and selected
module source/structure tests. It does not compile component suites or run the
full browser matrix.
- `ui`: quick/source contracts plus the shared Core component batch, compiled
once in an isolated directory. This is not a full visual/usability review.
- `backend`: existing manifest/interface checks and selected repository backend
test suites. Shared backend-test state is serialized.
- `full`: the canonical `tools/checks/check-focused.sh`, including optional-module
build permutations and integrated browser conformance. Repository filters do
not reduce this required cross-module completion gate.
`--changed` includes uncommitted work, commits ahead of the locally cached
upstream and repositories with commits but no configured upstream.
Missing/unreadable repositories remain visible as errors. For the
GovOPlaN profiles, Core or Meta changes conservatively select every registered
consumer; other changes expand through declared interface consumers. This is
not complete semantic dependency analysis. Use the full gate for cross-module
completion as required by `AGENTS.md`.
The module catalog reuses declared direct-Node tests and established structural
script names. Arbitrary shell chains are not guessed or rewritten. Unsupported
declared scripts remain visible with a reason; only exact known Core component
aliases are covered by the shared component runner. Invalid package metadata is
an error, not an empty test inventory.
`coverage` inventories declared suites as `planned`, `covered_elsewhere`,
`excluded` or `unsupported`, naming their covering stage where applicable. The
same inventory accompanies check dry runs and saved receipts. **Full means the
canonical focused gate, not every test in every package.** Its inventory follows
explicit canonical commands conservatively; it does not infer arbitrary nested
shell commands or npm hooks. Add new deterministic checks to the owning
package/canonical gate and the catalog's tests as appropriate. Coverage planning
does not execute the scripts it inventories or assert that they passed. See
[Coverage and validation details](DEVKIT_COVERAGE.md) for examples and boundaries.
Independent checks can run with `--jobs 1..8` (default 2). Shared resources use
OS locks for the current user, including across runs with different evidence
directories. Browser/port locks are shared across that user's workspaces. Full runs
reserve the shared WebUI/backend/browser resources. Standalone Core component
commands use isolated temporary build directories; `npm run test:components --
page-layout documentation-help` compiles once for that batch. Managed temporary
output is cleaned; no user worktree is reset or restored.
## Evidence, interruption and safe reuse
A check announces its run ID and saves a `preparing` receipt before source and
environment fingerprinting. Progress on stderr shows elapsed time, stage counts
and active stages; it also distinguishes `checking` from final snapshot
verification (`finalizing`). `--json` keeps stdout as one final JSON result and
emits structured progress events on stderr; `--quiet` suppresses these events.
Use `latest`, `status RUN` and `logs RUN --stage STAGE` from another terminal
while a check runs. Counts are stages, not a guessed percentage of test effort.
Live snapshots are private, bounded, explicitly provisional and not accepted for
reuse or issue evidence. A completed stage log can be hash-verified before the
whole run finishes, but only the final run snapshot verifies the overall result.
`--final-only` rejects unfinished logs. Run history supports cursor pagination
(up to 100 rows per page), bounds directory scanning, and never deletes evidence.
An explicitly selected `--project` also filters run discovery by project file.
Default records are private below
`$XDG_STATE_HOME/govoplan/devkit/workspace-<identity>/runs/<run>/` (fallback
`~/.local/state`). `--state-dir` selects another base; workspace scoping still
applies. Status changes are atomically persisted with restrictive file modes;
symlinked evidence paths are rejected. A record contains source/environment/plan
fingerprints, stage reasons, status, exit code, timing and log identities.
Stages distinguish pending, running, passed, failed, timed out, blocked, skipped,
stale and interrupted. A skipped or unexecuted stage is never a pass. A source or
environment change during verification marks the overall result **stale**, even
when individual subprocesses returned zero. Such a result is not passing evidence.
`resume RUN` replans the current selection and compares each phase independently.
Matching **verified checkpoints** are reused; changed/new phases and phases
without a checkpoint run again. A checkpoint binds the exact command and working
directory, declared repository inputs, transitive data-dependency identities,
devkit implementation/schema, tool/dependency metadata and environment. Logs must
retain their recorded hashes; a missing or changed cached log fails closed.
The summary reports how many checkpoints were reused; JSON records `input_scope`,
`cache_key`, `checkpoint_verified`, `reused_from` and the reuse/rerun reason.
Each checkpoint is saved while its resource locks remain held, only after exit
zero, a verified final log, and matching before/after inputs and environment.
A phase that changes its own inputs is stale, even if another phase later restores
those bytes. Final verification rechecks all passed/reused phases against current
inputs before the new aggregate can pass. An interrupted, failed or stale run may
donate an independently verified matching checkpoint, but never becomes passing
evidence itself. Old receipts remain readable: legacy monolithic logs and receipts
without these checkpoints cannot be retroactively split or reused as verified phases.
After a hard crash, `recover RUN` shows the abandoned owner and guidance. Inspect the
recorded commands and manually stop any surviving test/build processes before
using `recover RUN --apply --confirm-processes-stopped`. The command verifies the
run lock is free before marking the record interrupted. A free parent lock alone
does **not** prove children stopped after a hard kill. Recovery does not replay
processes or infer that a partially completed operation succeeded.
### Canonical phases and narrower inputs
The full gate has seven explicit phases: `preflight`, `tooling`, `backend`,
`core-ui`, `module-builds`, `browser`, and `module-ui`. Devkit stage IDs are
`focused.<phase>`. The authoritative Bash bodies remain in
`tools/checks/check-focused.sh`; the bounded metadata in
`tools/checks/focused-phases.json` names their order and resources. The direct
shell command still runs every phase in the original fail-fast order. For inspection:
```sh
tools/checks/check-focused.sh --list-phases
./devkit check --profile full --jobs 1
./devkit resume RUN_ID --jobs 1
```
For example, a browser failure no longer forces successful backend/build phases
to run again if their inputs still match. Standalone `--phase browser` is available
for diagnostics, but is not a substitute for the complete gate or its receipt.
Scopes are whole repositories, not inferred file globs. A trusted check may declare
`"inputs": {"repos": ["canonical-repository-name"]}`; no declaration means the whole
registered workspace. Dry runs show that boundary. Native source-discovery checks
retain broad scopes where cross-module dependencies cannot safely be narrowed.
Changes outside an explicit scope do not invalidate it, but tool/environment
changes remain conservatively global. Scope declarations are a correctness
contract: include every source repository that the check reads, not only its cwd.
Native discovery ownership/presence is rechecked at phase boundaries, so a newly
appearing module or WebUI directory invalidates the in-flight environment identity.
Unregistered native sibling sources disable reuse with an explicit coverage note;
register them before relying on cached results. Ordinary generated files inside
existing source directories do not change this directory-shape identity.
Every snapshot rereads repository presence, HEAD, index/flags and tracked plus
non-ignored untracked membership, including assume-unchanged files. File hashes
use a bounded **in-process-only** memo with file identity/ctime/mtime/size checks
before and after an open file descriptor; no persistent mtime cache is trusted.
In-repository regular-file symlink targets are included; escaping or directory
symlinks fail closed. Ignored files, undeclared external data and service state
are not source evidence and need explicit verification.
Environment probes stream bounded regular-file contents and reject concurrent
replacement, growth and symlink retargeting. Stable venv executable and dependency
directory symlinks remain supported without changing the executable path; absent
optional metadata remains optional. Environment identities are not persistently
cached and remain deliberately global.
These checkpoints cache **verification results, not output artifacts**. The full
gate's phases were audited so later phases do not require an earlier phase's
retained build output. A custom setup/build that produces files consumed later
must declare `"reuse": "never"`; its consumers must use `deps`. Until output
manifests/restoration exist, do not assume a cached successful build recreates
deleted ignored artifacts.
Check output is drained without unbounded memory growth. Stored output retains
the beginning and actual final tail within an 8 MiB bound per stage, with a small
truncation marker and an omitted-byte count; output truncation does not change
the subprocess exit result. `logs` exposes at most 200 lines and 16 KiB. Live
snapshots retain at most 64 KiB after redaction. Incomplete live lines and cut
retention-boundary lines are withheld to avoid exposing fragments of secrets;
very long single-line output may therefore be absent from live views.
Known environment credentials and common authorization patterns are
redacted as display hygiene. **Never pass credentials as check arguments or print
them from tests.** Redaction is not a general secret-classification guarantee.
JSON command output is a redacted presentation; the receipt file at the reported
path is the canonical local record. Cancellation and deadlines terminate the
owned process group; a parent leaving running descendants does not pass.
During source/environment fingerprinting, cancellation is observed between
probes; the current bounded probe may finish first. Cancellation during final
verification is recorded as interrupted, never as a passing run.
Deliberately detached processes are outside that group: this is not a sandbox.
These local hash-bound records detect accidental changes; they are not signed
attestations, a security certification, proof of complete test coverage, or
permission to publish. External service state and undeclared dependencies still
need explicit verification. No automated result closes a Gitea issue or marks a
module reviewed. Detailed logs remain local unless deliberately shared. Old run
directories are not automatically deleted; review retention before removing any
evidence referenced by an issue.
## Reuse in another project
The runner, repository context, doctor and evidence primitives are usable with an
explicit local JSON project manifest. The example at
`tools/devkit/examples/project.json` registers ordinary Python tests and a Git
whitespace check; the format is described by
`tools/devkit/project.schema.json`.
```sh
/path/to/govoplan/devkit --workspace-root /path/to/project \
--project /path/to/project/devkit-project.json check --profile quick --dry-run
```
Repository paths and check working directories must remain inside the selected
workspace. Check arguments are arrays, not shell-evaluated strings. The supported
tool placeholders are `{python}`, `{node}`, `{npm}` and `{workspace}`. Check
dependencies form an acyclic graph; resources serialize incompatible tasks.
`deps` declares actual data dependencies: consumers rerun whenever a dependency
runs again. `after` declares only fail-fast execution ordering: a failed predecessor
skips the follower, but a successfully rerun independent predecessor does not
invalidate the follower's inputs. Both edge types are validated together for
cycles, missing IDs and overlap. `reuse` is `verified` by default or `never` for
setup/output-producing checks. Repository `inputs` names are canonical, unique,
nonempty and validated even for unselected checks.
The complete custom manifest is validated, including unselected checks/profiles:
unknown fields, misspellings, invalid bounds/types and unresolved dependencies
fail early. This prevents silently ignored resource or timeout declarations.
Portable `doctor` always requires its Python runtime, plus tools needed by the
selected checks and their dependencies and tools explicitly configured in the
manifest. Without `--profile`, all declared profiles are considered. Unused
Node/npm are marked `not_required` and do not block Python-only projects. Declare
indirect tool dependencies explicitly: arbitrary script contents are not
analyzed. Native GovOPlaN retains its Python/Node/npm requirements.
Only use project manifests and scripts you trust: running a registered test is
ordinary code execution, not a sandbox. Platform-specific release and Docs
commands reject generic project manifests; put another project's checks in its
own profiles rather than pretending its release policy is GovOPlaN's.
The initial runtime targets POSIX environments with Python 3.11+ and OS advisory
locks; it does not claim Windows support. No globally installed service or Codex
plugin is needed. Keep this implementation versioned and reuse it; avoid copying
diverging helper implementations into each project.
Devkit is not a complete build system: it does not infer semantic dependencies,
restore output artifacts, sandbox checks, or share signed remote caches. Repository
scopes intentionally stop short of file/glob narrowing; environment identities
remain global. These are explicit boundaries, not claims of exhaustive coverage.
## Development and conformance
```sh
./.venv/bin/python -m pytest -q tests/test_devkit_*.py
./.venv/bin/python -m pytest -q tests/test_focused_phases.py
node --test tests/test-devkit-display-labels.mjs
./.venv/bin/python -m unittest tests.test_documentation_structure
```
The focused gate includes the Python command-suite regression tests. Fixture
tests use temporary repositories/processes and mocked issue/release transports;
they do not commit user work, send mail, publish artifacts or change live issues.
Documentation is part of each command change: update `--help`, this guide and the
corresponding targeted contract tests together.
+72
View File
@@ -0,0 +1,72 @@
# Check configuration and suite coverage
`devkit coverage` explains what a selected plan intends to run. It does not run
tests, start servers, or report successful verification:
```sh
./devkit coverage --profile quick --repo campaign
./devkit coverage --profile ui --repo portal --json
./devkit coverage --profile full --json
./devkit --project /path/to/project.json coverage --profile quick
```
The inventory accounts for declared `test` and `test:*` scripts in each
registered repository's root and `webui/package.json`, discovered UI structural
checks, and explicitly configured project checks. It is a suite inventory, not
an enumeration of every test function or recursive dependency. Unselected suites
remain visible; every row has a disposition and reason:
| Disposition | Meaning |
| --- | --- |
| `planned` | A selected stage directly invokes the suite. |
| `covered_elsewhere` | An exact equivalent invocation or an explicitly selected shared batch owns it; the covering stage is named. |
| `excluded` | The suite is outside the actual selected plan. |
| `unsupported` | Scoped discovery cannot safely interpret or locate the command; no command is guessed or executed. |
`full` means the existing canonical `check-focused.sh` gate. It does **not**
mean every package script or every component suite. Coverage reads the current
script's explicit npm/Node commands inside its seven registered marked phase
bodies, without executing shell code or inferring arbitrary functions, nested
scripts, npm hooks, branches or here-document contents. Each covered suite names
the `focused.<phase>` that owns it. The
current gate names four of the sixteen Core component suites: layout primitives,
page layout, DataGrid actions/sizing, and Mail components. The other twelve stay
explicitly excluded from that gate's component batch. `ui` runs the shared
sixteen-suite batch once; `quick` does not compile component tests.
Only Core's exact known component aliases receive that shared-batch treatment.
A different repository using the same filename, an unknown alias, or a compound
command containing the runner is not silently credited or launched. Ordinary
scoped source discovery accepts bounded direct `node`/`node --test` commands
targeting regular package-owned `.mjs` files in `scripts` or `tests`; shell
chains, extra flags, missing targets and symlinked targets are not rewritten.
An explicit invocation already present in the canonical full gate remains part
of that gate, even when the narrower discovery profile does not support it.
Coverage rows retain a command hash and, for supported commands, redacted argv.
Unsupported shell bodies are not copied into the inventory. This is display
hygiene, not permission to put secrets into project commands. Coverage is
attached to check plans/receipts; it never turns an excluded suite into a pass.
## Portable-project validation
The runtime validates custom project metadata against the published
[`project.schema.json`](../../tools/devkit/project.schema.json) using a small
dependency-free validator. All checks and profiles are validated before
selection, including checks the chosen profile does not execute. Unknown nested
fields such as `resource` or `timout_seconds`, malformed types, oversized values,
duplicate references, missing dependencies, unknown profiles and dependency
cycles fail with controlled errors. Repository aliases and resolved paths must
be unambiguous and confined to the workspace. Check `cwd` defaults to `.` when
omitted; configured paths cannot escape the workspace.
Package JSON is bounded to 1 MiB and 512 scripts; duplicate JSON keys, excessive
nesting and malformed script objects fail before planning. Command strings are
bounded to 8,192 characters. This validates configuration, not the safety of its
code: run only project manifests and test scripts you trust.
Checks may declare repository-scoped `inputs`, true data `deps`, order-only
`after`, and `reuse: "never"` for setup/output-producing work. Unknown repositories,
empty/duplicate scopes, overlapping edge types and cycles fail before execution.
Missing inputs deliberately fall back to all registered repositories. See
[checkpoint and input contracts](DEVKIT.md#canonical-phases-and-narrower-inputs).
+171
View File
@@ -0,0 +1,171 @@
# Devkit review bundles and issue evidence
Gitea remains the canonical backlog and review state log. These commands assemble
guidance and append evidence; they do not create a second progress tracker, close
issues, rewrite issue bodies, tick checkboxes, or certify a module review.
## Prepare a module review
```sh
./devkit review campaign
./devkit review campaigns --json --output /tmp/campaign-review.json
./devkit review campaign --evidence RUN_ID
```
Repository names, repository aliases and scope IDs from the
[issue discovery inventory](../project/ui-review-issue-inventory.json) are accepted.
`review bundle campaign` is an equivalent spelling. A bundle contains the current
source inventory and fingerprint, canonical module/central issue links, the
Core principle revision and content hash, applicable **planned** check stages,
and a manual checklist. No checks or application servers are started. The
optional output file is a local snapshot, not an authoritative state record.
The filename inventory is a starting point, not proof that all runtime surfaces
were found. It does not import application manifests or optional modules. Review
routes, panes, dialogs, settings, widgets, public forms, contributed/headless
interfaces, missing-module behavior, permissions, English/German and narrow/wide
layouts manually. Missing source, no UI files or no planned checks is never an
automatic pass or N/A. Record exceptions and remaining work in the module issue.
Apply [UI-01 and UI-02, and the other Core principles](../../../govoplan-core/docs/UI_DESIGN_PRINCIPLES.md):
books belong beside meaningful visible text; normal screens display compact
data, with intentional scoped dialog editing. A bulk editing mode needs a
documented reason. Record new principles and their back-propagation to previously
reviewed modules through the [central review program](../project/UI_REVIEW_PROGRAM.md).
An explicit portable `--project project.json` does not inherit GovOPlaN issue
links or principles. Its optional configuration is:
```json
{
"review": {
"issue_inventory": "meta/docs/review-issues.json",
"principles": "core/docs/UI_DESIGN_PRINCIPLES.md"
}
}
```
Paths are relative to `--workspace-root` and cannot escape it or traverse
symlinks. The inventory uses schema version 1, `issues[]` with `scope_id`,
`repository`, `number`, `url`, and optional `name`/`kind`; an optional `epic`
contains `repository`, `number`, `url`. Snapshot issue-state fields are not
reported as current state. Custom checks come from the project's selected
profile (`--profile ui` by default).
## Preview and append evidence
```sh
./devkit issues note --root govoplan-campaign --issue 103 \
--evidence RUN_ID --summary 'Targeted checks passed.' \
--next 'Manual module review and remaining fixes are still required.' --json
./devkit issues note --root govoplan-campaign --issue 103 \
--evidence RUN_ID --summary 'Targeted checks passed.' \
--next 'Manual module review and remaining fixes are still required.' \
--env-file /private/gitea.env --apply
```
`--root` is resolved relative to the selected **workspace root**, not the shell's
current directory. In the normal siblings workspace, use `--root govoplan-campaign`
or an absolute checkout path; do not use a path escaping the workspace.
The default is a completely offline dry run; it does not load credentials or
contact Gitea. `--apply` explicitly authorizes comment creation. Tokens are read
from `GITEA_TOKEN` or the explicit `--env-file` (environment wins); there is no
implicit `.env` search. The credential file is parsed, never sourced, and is not
printed or uploaded. Git's selected remote (default `origin`) determines the
exact target: ambient `GITEA_OWNER`, `GITEA_REPO` and `GITEA_URL` cannot retarget
the request. Credential-bearing HTTP Git remote URLs are rejected.
Use repeatable `--summary`/`--next`, `--body-file` for Markdown, or `--note-file`
with structured JSON `{ "summary": ["..."], "next": ["..."], "body": "..." }`.
Do not put secrets in notes, paths, check arguments or logs. Display redaction is
best-effort protection, not permission to include confidential data.
For a bounded multi-issue operation, replace `--root`/`--issue` with
`--target-plan targets.json`:
```json
{
"schema_version": 1,
"targets": [
{
"root": "govoplan-campaign",
"issue": 103,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103"
}
]
}
```
Targets must be unique, inside the workspace and bound exactly to their Git
remote/issue URLs, all on one exact Gitea base URL. Optional `issue_id` binds
Gitea's immutable numeric ID as well. Every target, complete comment history,
marker collision and local journal binding is checked before the first POST.
Comments are then posted serially and read back against the exact issue. A
closed issue stays closed; existing bodies, comments and checklists are untouched.
Each note has a stable hidden marker binding its issue, `--key` purpose
(`verification` by default), and evidence run ID. Identical already-published
notes are read back instead of reposted; changed content with the same marker
is a collision, not an update. Use a distinct intentional `--key` for a separate
follow-up. Without a receipt, the note's structured content determines identity.
A lost POST response is **not** retried automatically. The publisher scans all
comment pages and reads back any matching comment. If it cannot confirm the
result, it records `uncertain`, stops later posts, and exits nonzero. The next
invocation reconciles again without replaying the POST. Only after inspecting
the issue and local result, use `--apply --retry-uncertain` to permit one new
attempt when the marker is still absent. Local journals and OS-released locks
live in the workspace-scoped devkit state directory; do not delete them to bypass
uncertainty. These are retry receipts, not issue state. Independent machines or
different state directories cannot share a lock: coordinate publishers, because
Gitea comment creation does not provide an atomic idempotency key.
## What a receipt does and does not prove
`--evidence RUN_ID` reads the workspace-scoped runner receipt and checks local
integrity. `--evidence /path/to/receipt.json` accepts bounded validated external
receipt metadata, explicitly labeled `external-unverified`. A self-consistent
digest is not a signature or independent verification. Foreign-workspace,
malformed and inconsistent successful receipts are rejected. Receipt-provided
commands are never run; log files are not opened or uploaded, only referenced.
Successful receipts must explicitly record a verified source snapshot; an
external file's flag is still a reported claim, not an independent attestation.
An aggregate containing a skipped stage cannot claim success. New versioned
receipts additionally require independent verified checkpoints for passed phases.
Failed/interrupted/stale aggregates remain labeled as such even when they contain
reusable successful phases; reuse creates a new run and preserves the original.
Stage coverage limitations are validated and retained in the note, including
checks omitted by scoped profiles. Compact review output shows the first eight
limitations and points to the complete list in its JSON bundle. A passing stage
never implies that omitted compiler, browser, permutation or manual checks ran.
Documentation audit limitations are saved with their stage receipts, so later
review bundles and issue notes retain the same scope as the original command.
Managed inventory collectors use the explicit selected workspace throughout;
an incomplete checkout never borrows sources or missing module imports from a
fuller default workspace. Missing required Core/parser dependencies fail the
audit. Legacy direct inventory commands without `--workspace-root` retain their
discovery behavior; use the explicit option for reproducible evidence.
The note distinguishes `matches-current`, `historical-source-differs`,
`different-project-not-compared` and unavailable comparisons. Matching source
is a comparison at bundle/note preparation time, not a live test run or proof
that a person completed review. If the source or environment has changed, run
the appropriate checks again and keep historical evidence labeled honestly.
Versioned repository-input receipts compare the recorded repository scope, not a
legacy whole-workspace hash. Partial scopes remain explicitly labeled in evidence
notes. This comparison reads source only; receipt commands are never executed.
Legacy receipts retain their original comparison format. A matching source scope
does not prove that ignored build outputs, the environment or live services match.
Fixtures cover dry-run isolation, exact target bindings, capped/repeated
pagination, collisions, append-only idempotency, uncertain POST reconciliation,
secret-safe errors, receipt validation, portable projects and optional-module
review boundaries:
```sh
python3 -m pytest -q tests/test_devkit_issues.py tests/test_devkit_review.py
```
+97
View File
@@ -0,0 +1,97 @@
# Selected-path Git maintenance
The optional `git` namespace of `tools/devkit/devkit.py` is a deliberately
narrow maintenance workflow. It is not a replacement for normal Git, the
release executor, required human review, or Gitea's canonical issue state.
From the Meta repository, preview and then save a plan for explicit files:
```sh
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example"
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example" --apply
```
Use the saved `git-…` plan ID for the remaining commands. Each mutation needs
its own `--apply`; an unqualified command only previews the operation:
```sh
python3 tools/devkit/devkit.py git commit git-PLAN_ID
python3 tools/devkit/devkit.py git commit git-PLAN_ID --apply
python3 tools/devkit/devkit.py git push git-PLAN_ID
python3 tools/devkit/devkit.py git push git-PLAN_ID --apply
python3 tools/devkit/devkit.py git status git-PLAN_ID
```
The immutable local plan binds the registered repository, HEAD, branch, origin
fetch/push URL hashes, Git configuration, complete index identity, explicit
selected file paths, working-tree hashes, expected Git blobs, and commit message.
Raw remote URLs and file contents are not stored. A changed input blocks the
operation; inspect it and create another plan instead of weakening the check.
Receipts are private local records with integrity checks, not signed approval
or a security attestation.
Commit captures each selected file once, verifies the captured bytes and Git
blob against the plan, and builds an isolated index/tree from the frozen HEAD
plus only those blobs. Git's normal identity rules create the commit object;
an atomic branch compare-and-swap publishes it only if the planned parent is
still current. Active hooks/signing remain refused rather than bypassed.
An editor racing with this operation cannot substitute newer working-tree
bytes: those edits remain uncommitted, and the helper never overwrites files.
The real index is protected by Git's standard index lock. Only selected entries
are updated from a prepared private copy; unrelated staging and index flags are
preserved. If the index changes independently, its new data is not overwritten
and the receipt requires reconciliation. The resulting parent, message,
changed paths, blobs/modes and unrelated index entries are verified.
Git refs and index files are separately atomic, not one filesystem transaction.
Independent writers that ignore the standard index lock can still change refs
during publication. The helper rechecks the exact recorded candidate before
index recovery and before recording success; it refuses an uncertain result
instead of adopting a newer HEAD or claiming that competing work was its own.
Directories, noncanonical paths, implicit globs, staging-all, amend,
force-push and extra-tag publication are not supported. A selected file with
different staged and working-tree changes is refused; decide explicitly which
version to commit using normal Git first.
Push requires the recorded commit to remain current, an unchanged origin and
one standard-transport push destination. It requests a normal, non-forced push
of that commit to the recorded branch and checks the remote branch afterward.
The push preview does not contact a remote. A normal push may run the remote's
usual CI or server-side hooks; this is an external effect of the separately
authorized push, not of planning or committing.
Local active hooks (including index-change hooks), filter/encoding attributes,
signing, external fsmonitor/SSH configuration, Git environment overrides,
in-progress merge/rebase/cherry-pick operations, submodules, symlinks and
assume-unchanged/skip-worktree entries, split indexes and replacement/graft
history are deliberately
unsupported. The helper
refuses these cases instead of disabling hooks, signatures or filters. An
unused globally installed filter definition alone does not block maintenance;
active attributes are checked across tracked files as well as selected files.
Custom remote receive-pack/upload-pack/helper commands, recursive submodule
pushes and partial-clone lazy fetch are also refused. Of inherited `GIT_*`
variables, only `GIT_OPTIONAL_LOCKS`, `GIT_TERMINAL_PROMPT` and `GIT_PAGER` are
allowed (every command explicitly passes `--no-pager`, so the pager is inert);
namespace, identity, alternate-index/object-directory and unknown overrides
are not silently removed. Git subprocesses have bounded input/output and
deadlines; cancellation terminates their owned process group, including
transport helpers. Network failures can still leave a remote effect uncertain,
which is why receipts require explicit reconciliation rather than blind retry.
An interruption or uncertain failure is recorded before any retry. Use
`git reconcile git-PLAN_ID` to preview, then add `--apply` to verify an
already-existing result. Reconciliation never makes a commit or push. It may
finish the selected real-index update after an interrupted commit publication,
but only when the original index fingerprint still matches; independent staged
work is preserved and requires normal Git resolution. For an
uncertain push it may read the frozen remote, but only with `--apply`. If no
planned commit is current, the helper reports that state and requires a fresh
plan after inspection. A forcibly killed process may leave its private scratch
index or an owned Git index lock; the helper never guesses that an existing
lock is safe to delete. Blob/commit preparation can leave unreferenced Git
objects for ordinary Git garbage collection. It never automatically resets,
restores, deletes, or rolls back user work.
The maintenance tests use disposable local repositories and local bare remotes.
They never commit or push the user's workspace repositories.
+121
View File
@@ -0,0 +1,121 @@
# Headless durable releases
The devkit `release` commands use the **same release application, request
validation, run store and bounded executors** as the
[release console](RELEASE_CONSOLE.md). They invoke its ASGI application inside
the current process, with an ephemeral internal token. No listening socket,
background server, browser session or externally supplied console URL is used.
The development environment's FastAPI/HTTPX dependencies load only when a
release command runs; ordinary devkit help does not import them.
This is a GovOPlaN-specific provider. Repository and origin authority still comes
from the release service's registered catalog. `--project` is explicitly rejected
for release commands; a portable project configuration must not silently override
that authority. A different workspace path does not authorize arbitrary release
repositories, remotes, commands, signing policies or source bindings.
The default compact output includes the overall status, selected repository
states and versions, source-preflight readiness, bounded gate findings and the
recommended next action. Run inspection also shows step-state counts and the
first outstanding steps. These are projections of the service response, not new
readiness checks. Use `--json` for the complete existing plan or receipt; compact
output deliberately omits executor commands, arguments and source bindings.
## Commands and boundaries
| Command | Purpose | Effect boundary |
| --- | --- | --- |
| `release status` | Current release dashboard | Offline by default; explicit remote/catalog flags enable their checks |
| `release plan` | Selective repository/version plan | Inspection only; does not create a run |
| `release list` | Bounded run history, with `--limit` and `--cursor` | Existing workspace-scoped store |
| `release show RUN_ID` | Exact verified run, steps, receipts and required confirmations | No release execution |
| `release create` | Freeze a selected repository/version plan | Preview by default; `--apply` persists a run |
| `release preview RUN_ID STEP_ID` | Frozen step and its current constraints | No step claim; catalog publication additionally uses the existing validated-candidate preview |
| `release execute RUN_ID STEP_ID` | One available durable step | Preview by default; `--apply`, explicit request ID and the step's exact confirmation are required |
| `release resume RUN_ID` | Mark persisted running attempts interrupted | Requires `--apply`; never assumes an effect succeeded |
| `release retry RUN_ID STEP_ID` | Prepare an eligible failed/read-only-interrupted step | Requires `--apply`; does not execute it |
| `release reconcile RUN_ID STEP_ID` | Record a proven uncertain-write outcome | Requires `--apply`, `--confirm RECONCILE` and an explicit outcome |
Create and transition commands require caller-chosen `--request-id` values.
Retain the same ID for an uncertain replay of the **same** command; changing
inputs under an existing ID fails closed. No command automatically retries a
mutation. These commands perform one explicit transition, not an implicit
"release everything" loop.
Read commands do not commit, tag, push, publish or apply database migrations.
The existing private run store can initialize its lock directory or upgrade a
legacy record while inspecting it; this does not advance release steps.
`--include-migrations` requests the existing audits, never migration application.
`--online`, `--remote-tags` and `--public-catalog` are explicit network-check
choices. A generic step preview describes frozen intent; it is not a claim that
live preflight, remote identity or release acceptance has passed.
## Example: review, freeze and execute one step
Run the devkit through the workspace `./devkit` entrypoint. These examples put
global options before `release`:
```sh
./devkit --workspace-root /path/to/workspace --format json release plan \
--repo-version govoplan-files=0.1.9
./devkit --workspace-root /path/to/workspace release create \
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001
./devkit --workspace-root /path/to/workspace release create \
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001 --apply
./devkit --workspace-root /path/to/workspace release show RUN_ID
./devkit --workspace-root /path/to/workspace release preview RUN_ID STEP_ID
./devkit --workspace-root /path/to/workspace release execute RUN_ID STEP_ID \
--request-id files-release-step-0001 --confirm REQUIRED_CONFIRMATION --apply
```
`RUN_ID`, `STEP_ID` and `REQUIRED_CONFIRMATION` stand for the actual values
returned by the frozen run. For read-only preflight/alignment/install-verification
steps, omit `--confirm` when the service reports an empty confirmation. Metadata,
commit, release-lock, source-tag, source-push, candidate-generation and catalog
publication steps keep the existing `UPDATE`, `COMMIT`, `LOCK`, `TAG`, `PUBLISH`,
`GENERATE` and `PUSH` confirmations respectively. Prerequisite ordering is enforced
by the same store; a later step cannot be forced through this adapter.
Version selection supports repeated `--repo-version REPO=VERSION`, or repeated
`--repo REPO` with `--target-version VERSION`. Every created run needs explicit
versions for all selected repositories. Conflicting version assignments are
rejected. Planning is selective; repositories are never silently selected merely
because their worktrees are dirty.
## Candidate publication and recovery
Use the frozen run's `catalog:selective-generator` and
`catalog:validate-sign-publish` steps, not the disabled legacy mutation endpoints.
Generation accepts repeated `--signing-key KEY_ID=PRIVATE_KEY_FILE`; these are
operator-owned key-file references, never inline key material. Signing arguments
are not emitted in devkit JSON or persisted by the adapter. Publication consumes
the candidate receipt created by that run, not an arbitrary candidate directory.
The existing source/runtime trust, immutable tag/remote identity, private
candidate checks and exact commit-delta checks remain authoritative.
After an interrupted write, inspect the run and the actual local/remote effect.
Use `resume --apply` if an attempt was left running. Then use `reconcile` with
`effect_absent`, `effect_succeeded` or `unresolved`, `--confirm RECONCILE`, a new
reconciliation request ID and `--apply`. Successful reconciliation still performs
the service's independent receipt checks. An unresolved effect must not be
retried or papered over by generating a new request ID. A failed/read-only
attempt may use `retry --apply` and a new explicit execute attempt only when
the existing lifecycle makes that safe.
## State and limitations
Every successful response exposes `state_location` and `candidate_location`.
Without `--state-dir`, the adapter uses the console's existing default private,
workspace-fingerprinted state directory. With `--state-dir PATH`, it uses
`PATH/release-console/workspace-<fingerprint>/release-runs`. Workspaces cannot
read or resume one another's runs. Existing ownership, symlink, permission,
retention and record-integrity checks are unchanged.
Release preparation can commit only the recognized, receipt-bound metadata
changes its executors produced. This command does **not** stage arbitrary source
changes, use `git add -A`, force-push, retarget tags or enable the disabled generic
prepare/sync/push endpoints. Normal selected-path maintenance requires its own
explicit reviewed-change workflow; it is not silently folded into release.
+11 -2
View File
@@ -354,8 +354,17 @@ such as `0.2.0` or `0.2.0-alpha1`, but requires the first three version numbers
to move forward. to move forward.
Plain repository pushes are separate from catalog publication. `Preview Push` Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in shows the selected repository push commands, but generic push, sync and prepare
the repository push confirmation field. mutation endpoints are disabled: they require a separate durable, receipt-bound
maintenance workflow and cannot be enabled by typing a confirmation. Source
release branch/tag publication uses the durable release-run steps described
above; do not route ordinary dirty worktrees through the legacy all-repository
stage/commit/tag helper.
The [headless devkit release commands](DEVKIT_RELEASE.md) invoke this same
application in-process without starting a server. They expose selective planning,
bounded status/history, create/show/preview/execute and explicit recovery while
retaining the same request IDs, confirmations, source bindings and receipts.
The source release panel retains `Preview Tag + Publish` as a non-mutating The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
+164
View File
@@ -0,0 +1,164 @@
# Product-wide UI review program
The [central Gitea epic](https://git.add-ideas.de/GovOPlaN/govoplan/issues/56)
coordinates the review; its linked module issues are the canonical backlog,
progress and evidence log. The [issue inventory](ui-review-issue-inventory.json)
is a discovery/link snapshot, **not a second progress tracker**. All reviews were
initialized pending. Implementing a shared component or moving a help icon does
not by itself complete a module review.
The visual and interaction rules live in Core's
[UI design principles](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/UI_DESIGN_PRINCIPLES.md).
This document defines how to apply and verify those principles across repositories;
it does not fork their definitions. The usual [Gitea issue contract](GITEA_ISSUES.md)
still applies. At the user's request, this cross-product program is coordinated in
Meta; shared primitives and their implementation remain owned by Core.
## Scope and initial inventory
The September 2026 source/catalog baseline contains **77 review tracks**:
- **73 implemented scopes:** Core/shared shell and 72 manifest-backed modules.
- **4 registered placeholders:** Appointments, Ledger, XÖV and XTA/OSCI. They have
README-only repositories at initialization. Their separate readiness tracks
remain pending for future interfaces; they are not counted as implemented or
reviewed modules.
Meta and the public website are not business-module review scopes. Module IDs
come from actual manifests, not repository-name guesses: for example,
`govoplan-campaign` owns `campaigns` and `govoplan-dist-lists` owns `dist_lists`.
The inventory is derived from `repositories.json`, trusted module manifests and
the owning repository's source files. Changes to the catalog must be reconciled
with the central epic rather than silently dropping a module.
A module with no standalone WebUI is still in scope. Review its configuration
and administrator experience, contributed operator/public surfaces, widgets,
documentation, permissions, errors and interactions exposed through other
modules. A missing standalone page is not evidence of conformance. A reviewer
may mark a particular principle not applicable only with a concrete explanation
and source/runtime evidence. A placeholder likewise needs an explicit future-UI
gate, not a fabricated page inventory or an automatic green review.
## Work order and ownership
Prioritize user-visible defects, usability and consistent interactions before
broader features. Start with Core/shared-shell contracts and Campaign's complete
workflow, then Files, Mail, Templates, Notifications, Docs/Help, Dashboard, Quick
Access and Views. The remaining queue is ordered by observed defects, shared
dependencies and risk. This is a starting order, not a claim that those reviews
have started or finished; the central issue records the current focus.
For Campaign, use a compact, read-only campaign settings overview with explicit,
scoped edit dialogs. Large recipient or attachment grids may use the UI-02 bulk
editing exception: clearly entered edit mode, explicit Save and Cancel, dirty
navigation protection, and no silent loss or accidental save. This direction
must be implemented and verified in the Campaign review, not marked complete by
the program bootstrap.
Fix a repeated problem in the owning shared primitive or semantic contract first,
then migrate consumers and verify representative actual-module fixtures. Keep
module business behavior in its own repository and preserve optional module
boundaries. Do not replace concrete user evidence with a generic fixture alone.
Parallel reviewers may own different modules, but they must coordinate shared
Core files and browser fixtures instead of racing the same resources.
## Per-module review workflow
1. **Inventory.** Verify and extend the issue's source-derived seed: navigation,
pages, nested routes, dialogs, labels/forms, tables/trees, admin/system/tenant/
user settings, public/operator interfaces, widgets and optional-module
contributions. Follow actual module entrypoints and runtime contributions;
filename heuristics and manifest routes are a starting point, not an exhaustive
runtime audit. Note roles, permission boundaries and installed optional modules.
2. **Observe.** Exercise realistic narrow and wide viewports, German and English,
keyboard navigation and focus, empty/loading/error/success states, clean/dirty
edit modes, reload, navigation away and return, and restoration of personal
preferences. Include realistic row counts, long labels, horizontal overflow,
pagination and two-way column resizing with fixed columns between text fields.
3. **Record.** Keep a findings/TODO/done ledger in the module issue. Every finding
names the surface, reproduction, expected principle, impact, owner and linked
implementation/evidence. A source inventory, planned fix or green generic
test is not a completed finding. State product decisions and manual checks
explicitly with enough context to resolve them.
4. **Implement.** Prefer reusable shared primitives and action/page contracts.
Keep display mode readable and editing intentional. Preserve authorization,
server validation, save/cancel behavior and data integrity. Never send mail,
delete records, change live configuration or trigger other irreversible effects
merely to obtain UI evidence; use authorized fixtures or a safe test context.
5. **Verify.** Run proportionate unit, structure and actual-module browser tests.
Record commands, results, source revision and safe runtime evidence. Update the
owning module's manifest-driven EN/DE documentation for behavior changes; run
`tools/checks/check-manifest-shapes.py` and relevant cross-module checks.
6. **Conclude honestly.** Complete the principle matrix, evidence and manual
checks. Link unresolved follow-ups and blockers; do not close a supposedly
complete review while required work is still pending. Closing an issue requires
explicit reviewed scope and the applicable design-principle revision.
## Principle ledger and back-propagation
Use the stable IDs from Core in each issue. The initial revision covers UI-01
heading/label help placement; UI-02 display-first/scoped editing; UI-03 actions;
UI-04 table/card geometry; UI-05 loading/error/progress; UI-06 tree interaction;
UI-07 accessibility and German; UI-08 data integrity; and UI-09 propagation of
revised principles. Core remains the authoritative definition.
Each module maintains this matrix, initially entirely pending:
| Principle/revision | Applicable surfaces / justified N/A | Applied / remaining work | Evidence | Exception / owner / follow-up |
| --- | --- | --- | --- | --- |
| UI-01 … UI-09 | Pending inventory | Pending review | Not yet recorded | None approved |
Whenever a principle is added or changed:
1. Update Core's rule with its rationale, stable ID and revision/change reference.
2. List **all already-reviewed modules**, not just the current implementation's
consumers. Check whether the changed rule applies to each.
3. Record unchanged conformance with evidence, or reopen the review / create a
linked follow-up with owner, exact affected surfaces and required checks.
4. Update the central epic's propagation ledger. Previously reviewed modules with
outstanding applicable work are **follow-up required**, not silently green.
5. Resolve the propagation sweep only after every affected module has current
evidence or an explicit, owned and justified exception. Exceptions do not
silently change the shared rule.
Suggested central ledger:
| Principle change | Already-reviewed modules checked | Conformant evidence | Reopened / follow-up required | Exceptions / owner | Sweep state |
| --- | --- | --- | --- | --- | --- |
| Initial UI-01UI-09 baseline | None reviewed at initialization | None | All module reviews pending | None approved | Pending |
## Safe inventory/bootstrap automation
`tools/gitea/gitea-ui-review-program.py` derives the current scopes from the
catalog and source manifests, inventories routes and UI source entrypoints,
then deduplicates issues by a stable hidden marker and normalized exact title.
It includes closed issues in duplicate checks and never reopens, closes or
overwrites existing module issues. Ambiguous or unmanaged duplicates stop the
operation for review. Existing issue bodies, labels and review progress are
preserved. A failed POST is not automatically retried; a subsequent run checks
the marker again before deciding whether another create is needed.
Dry-run is the default. An authenticated dry-run performs only Gitea reads:
```sh
.venv/bin/python tools/gitea/gitea-ui-review-program.py \
--env-file /home/zemion/.config/gitea/gitea.env --epic 56
```
Creating missing review issues requires explicit `--apply`. Initial population
of the central epic's module-link block additionally requires
`--initialize-links`; it is accepted only for the untouched bootstrap placeholder.
Later runs must not rewrite checkboxes or human-maintained progress. Existing
complete links are verified read-only. The token is loaded through the shared
Gitea helpers and is never emitted in reports. A scoped `--ipv4` option works
around host-specific IPv6 connectivity while retaining HTTPS certificate and
hostname verification.
The tool emits a JSON result to stdout with source counts and issue links. The
checked-in inventory is a reviewed initialization snapshot of that result;
refreshing links does not authorize replacing live review status with the
snapshot. Test the automation offline with:
```sh
.venv/bin/python -m pytest tests/test_ui_review_program.py -q
```
+866
View File
@@ -0,0 +1,866 @@
{
"schema_version": 1,
"snapshot_purpose": "Issue discovery links; live Gitea issues own review state and evidence.",
"epic": {
"repository": "govoplan",
"number": 56,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan/issues/56"
},
"scope_count": 77,
"manifest_modules": 72,
"implemented_scopes": 73,
"catalogued_placeholders": 4,
"created": 77,
"missing": 0,
"issues": [
{
"scope_id": "core",
"name": "Core / shared shell",
"repository": "govoplan-core",
"kind": "core",
"ui_source_count": 111,
"number": 301,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/301",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "access",
"name": "Access",
"repository": "govoplan-access",
"kind": "manifest",
"ui_source_count": 18,
"number": 23,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/23",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "addresses",
"name": "Addresses",
"repository": "govoplan-addresses",
"kind": "manifest",
"ui_source_count": 1,
"number": 26,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/26",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "admin",
"name": "Admin",
"repository": "govoplan-admin",
"kind": "manifest",
"ui_source_count": 9,
"number": 11,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-admin/issues/11",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "approvals",
"name": "Approvals",
"repository": "govoplan-approvals",
"kind": "manifest",
"ui_source_count": 4,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "assets",
"name": "Assets",
"repository": "govoplan-assets",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-assets/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "audit",
"name": "Audit",
"repository": "govoplan-audit",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-audit/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "booking",
"name": "Booking",
"repository": "govoplan-booking",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-booking/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "calendar",
"name": "Calendar",
"repository": "govoplan-calendar",
"kind": "manifest",
"ui_source_count": 10,
"number": 26,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-calendar/issues/26",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "campaigns",
"name": "Campaigns",
"repository": "govoplan-campaign",
"kind": "manifest",
"ui_source_count": 60,
"number": 103,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "cases",
"name": "Cases",
"repository": "govoplan-cases",
"kind": "manifest",
"ui_source_count": 4,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-cases/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "certificates",
"name": "Certificates",
"repository": "govoplan-certificates",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-certificates/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "committee",
"name": "Committee",
"repository": "govoplan-committee",
"kind": "manifest",
"ui_source_count": 3,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "connectors",
"name": "Connectors",
"repository": "govoplan-connectors",
"kind": "manifest",
"ui_source_count": 3,
"number": 20,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/20",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "consultation",
"name": "Consultation",
"repository": "govoplan-consultation",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-consultation/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "contracts",
"name": "Contracts",
"repository": "govoplan-contracts",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-contracts/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dashboard",
"name": "Dashboard",
"repository": "govoplan-dashboard",
"kind": "manifest",
"ui_source_count": 5,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dashboard/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dataflow",
"name": "Dataflow",
"repository": "govoplan-dataflow",
"kind": "manifest",
"ui_source_count": 5,
"number": 24,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/24",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "datasources",
"name": "Datasources",
"repository": "govoplan-datasources",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "decisions",
"name": "Decisions",
"repository": "govoplan-decisions",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-decisions/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dist_lists",
"name": "Distribution Lists",
"repository": "govoplan-dist-lists",
"kind": "manifest",
"ui_source_count": 2,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dms",
"name": "DMS",
"repository": "govoplan-dms",
"kind": "manifest",
"ui_source_count": 0,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dms/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "docs",
"name": "Docs",
"repository": "govoplan-docs",
"kind": "manifest",
"ui_source_count": 2,
"number": 23,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/23",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "encryption",
"name": "Encryption",
"repository": "govoplan-encryption",
"kind": "manifest",
"ui_source_count": 1,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "erp",
"name": "ERP",
"repository": "govoplan-erp",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-erp/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "evaluation",
"name": "Evaluation",
"repository": "govoplan-evaluation",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-evaluation/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "facilities",
"name": "Facilities",
"repository": "govoplan-facilities",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-facilities/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "files",
"name": "Files",
"repository": "govoplan-files",
"kind": "manifest",
"ui_source_count": 8,
"number": 48,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/48",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "fit_connect",
"name": "FIT-Connect",
"repository": "govoplan-fit-connect",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-fit-connect/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "forms",
"name": "Forms",
"repository": "govoplan-forms",
"kind": "manifest",
"ui_source_count": 2,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "forms_runtime",
"name": "Forms Runtime",
"repository": "govoplan-forms-runtime",
"kind": "manifest",
"ui_source_count": 6,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "grants",
"name": "Grants",
"repository": "govoplan-grants",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-grants/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "helpdesk",
"name": "Helpdesk",
"repository": "govoplan-helpdesk",
"kind": "manifest",
"ui_source_count": 1,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-helpdesk/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "identity",
"name": "Identity",
"repository": "govoplan-identity",
"kind": "manifest",
"ui_source_count": 1,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "identity_trust",
"name": "Identity Trust",
"repository": "govoplan-identity-trust",
"kind": "manifest",
"ui_source_count": 1,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "idm",
"name": "IDM",
"repository": "govoplan-idm",
"kind": "manifest",
"ui_source_count": 3,
"number": 14,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-idm/issues/14",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "inspections",
"name": "Inspections",
"repository": "govoplan-inspections",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-inspections/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "learning",
"name": "Learning",
"repository": "govoplan-learning",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-learning/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "mail",
"name": "Mail",
"repository": "govoplan-mail",
"kind": "manifest",
"ui_source_count": 6,
"number": 25,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/25",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "mandates",
"name": "Mandates",
"repository": "govoplan-mandates",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mandates/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "notifications",
"name": "Notifications",
"repository": "govoplan-notifications",
"kind": "manifest",
"ui_source_count": 3,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "ops",
"name": "Ops",
"repository": "govoplan-ops",
"kind": "manifest",
"ui_source_count": 2,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ops/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "organizations",
"name": "Organizations",
"repository": "govoplan-organizations",
"kind": "manifest",
"ui_source_count": 4,
"number": 9,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-organizations/issues/9",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "parties",
"name": "Parties",
"repository": "govoplan-parties",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-parties/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "payments",
"name": "Payments",
"repository": "govoplan-payments",
"kind": "manifest",
"ui_source_count": 3,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-payments/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "permits",
"name": "Permits",
"repository": "govoplan-permits",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-permits/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "policy",
"name": "Policy",
"repository": "govoplan-policy",
"kind": "manifest",
"ui_source_count": 3,
"number": 14,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-policy/issues/14",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "poll",
"name": "Poll",
"repository": "govoplan-poll",
"kind": "manifest",
"ui_source_count": 0,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "portal",
"name": "Portal",
"repository": "govoplan-portal",
"kind": "manifest",
"ui_source_count": 2,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-portal/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "postbox",
"name": "Postbox",
"repository": "govoplan-postbox",
"kind": "manifest",
"ui_source_count": 4,
"number": 29,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-postbox/issues/29",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "procurement",
"name": "Procurement",
"repository": "govoplan-procurement",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-procurement/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "projects",
"name": "Projects",
"repository": "govoplan-projects",
"kind": "manifest",
"ui_source_count": 1,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-projects/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "quick_access",
"name": "Quick Access",
"repository": "govoplan-quick-access",
"kind": "manifest",
"ui_source_count": 2,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-quick-access/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "records",
"name": "Records",
"repository": "govoplan-records",
"kind": "manifest",
"ui_source_count": 4,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-records/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "reporting",
"name": "Reporting",
"repository": "govoplan-reporting",
"kind": "manifest",
"ui_source_count": 3,
"number": 11,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/11",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "resources",
"name": "Resources",
"repository": "govoplan-resources",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-resources/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "rest",
"name": "REST Connector",
"repository": "govoplan-rest",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-rest/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "risk_compliance",
"name": "Risk Compliance",
"repository": "govoplan-risk-compliance",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "scheduling",
"name": "Scheduling",
"repository": "govoplan-scheduling",
"kind": "manifest",
"ui_source_count": 4,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-scheduling/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "search",
"name": "Search",
"repository": "govoplan-search",
"kind": "manifest",
"ui_source_count": 4,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "services",
"name": "Services",
"repository": "govoplan-services",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-services/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "soap",
"name": "SOAP Connector",
"repository": "govoplan-soap",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-soap/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tasks",
"name": "Tasks",
"repository": "govoplan-tasks",
"kind": "manifest",
"ui_source_count": 3,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tasks/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "templates",
"name": "Templates",
"repository": "govoplan-templates",
"kind": "manifest",
"ui_source_count": 1,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tenancy",
"name": "Tenancy",
"repository": "govoplan-tenancy",
"kind": "manifest",
"ui_source_count": 2,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tenancy/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tickets",
"name": "Tickets",
"repository": "govoplan-tickets",
"kind": "manifest",
"ui_source_count": 1,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tickets/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "transparency",
"name": "Transparency",
"repository": "govoplan-transparency",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-transparency/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "views",
"name": "Views",
"repository": "govoplan-views",
"kind": "manifest",
"ui_source_count": 3,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-views/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "voting",
"name": "Voting",
"repository": "govoplan-voting",
"kind": "manifest",
"ui_source_count": 2,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-voting/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "wiki",
"name": "Wiki",
"repository": "govoplan-wiki",
"kind": "manifest",
"ui_source_count": 1,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-wiki/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "workflow",
"name": "Workflow",
"repository": "govoplan-workflow",
"kind": "manifest",
"ui_source_count": 6,
"number": 17,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow/issues/17",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "workflow_engine",
"name": "Workflow Engine",
"repository": "govoplan-workflow-engine",
"kind": "manifest",
"ui_source_count": 0,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "xrechnung",
"name": "XRechnung",
"repository": "govoplan-xrechnung",
"kind": "manifest",
"ui_source_count": 0,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-appointments",
"name": "Appointments",
"repository": "govoplan-appointments",
"kind": "placeholder",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-appointments/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-ledger",
"name": "Ledger",
"repository": "govoplan-ledger",
"kind": "placeholder",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ledger/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-xoev",
"name": "Xoev",
"repository": "govoplan-xoev",
"kind": "placeholder",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xoev/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-xta-osci",
"name": "Xta Osci",
"repository": "govoplan-xta-osci",
"kind": "placeholder",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xta-osci/issues/1",
"state_at_verification": "open",
"operation": "created"
}
],
"snapshot_date": "2026-09-08",
"initial_review_status": "pending for every scope; this snapshot does not track subsequent issue progress"
}
+55
View File
@@ -0,0 +1,55 @@
import assert from "node:assert/strict";
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { test } from "node:test";
import { auditRepository } from "../tools/devkit/audit-display-labels.mjs";
const require = createRequire(resolve(import.meta.dirname, "../../govoplan-core/webui/package.json"));
const ts = require("typescript");
function fixture(fn) {
const root = mkdtempSync(join(tmpdir(), "govoplan-label-audit-"));
const write = (relative, content) => { const target = join(root, relative); mkdirSync(dirname(target), { recursive: true }); writeFileSync(target, content); };
try {
write("core/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Shared: "Shared" }, de: { Shared: "Gemeinsam" } };');
return fn(root, write);
} finally { rmSync(root, { recursive: true, force: true }); }
}
test("plain labels resolve owning registered catalogs, aliases, constants and Core defaults", () => fixture((root, write) => {
write("module/webui/src/i18n/generatedTranslations.ts", 'const en = { Projects: "Projects" }; const de = { Projects: "Projekte" }; export const generatedTranslations = { en, de };');
write("module/webui/src/module.ts", 'import { generatedTranslations as words } from "./i18n/generatedTranslations"; export const exampleModule: PlatformWebModule = { translations: words };');
write("module/webui/src/Page.tsx", 'import { PageLayout as Layout, PageTitle } from "@govoplan/core-webui"; const title = "Projects"; export const page = <><Layout title={title} /><PageTitle>Shared</PageTitle></>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.registration, "registered");
assert.equal(result.labels.length, 2);
assert.deepEqual(result.findings, []);
}));
test("a catalog file without module registration cannot mask untranslated titles", () => fixture((root, write) => {
write("module/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Projects: "Projects" }, de: { Projects: "Projekte" } };');
write("module/webui/src/module.ts", 'export const exampleModule = { id: "example" };');
write("module/webui/src/Page.tsx", 'import * as Core from "@govoplan/core-webui"; export const page = <Core.PageLayout title="Projects" />;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert(result.findings.some((item) => item.code === "catalog-not-registered"));
assert.deepEqual(result.labels[0].missing_locales, ["en", "de"]);
}));
test("only known static display slots fail; markers and runtime data remain separate", () => fixture((root, write) => {
write("module/webui/src/Page.tsx", 'import { PageLayout } from "@govoplan/core-webui"; export const page = <><PageLayout title="Untranslated" /><PageLayout title={campaign.name} /><PageLayout title="i18n:known.key" /><CustomThing title="Not a known slot" /></>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.findings.length, 1);
assert.equal(result.findings[0].text, "Untranslated");
assert.equal(result.review.length, 1);
assert.equal(result.review[0].code, "dynamic-display-slot");
}));
test("dynamic translation registration is reported for review, not falsely missing", () => fixture((root, write) => {
write("module/webui/src/module.ts", 'export const exampleModule: PlatformWebModule = { translations: configuredCatalog() };');
write("module/webui/src/Page.tsx", 'export const page = <PageTitle>Example</PageTitle>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.registration, "dynamic");
assert.deepEqual(result.findings, []);
assert(result.review.some((item) => item.code === "dynamic-catalog-review"));
}));
+64
View File
@@ -0,0 +1,64 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { findDetachedDocumentation } from "../tools/checks/check-heading-help.mjs";
import "./test-heading-translations.mjs";
const inspect = (source) => findDetachedDocumentation([{ path: "/fixture/Page.tsx", source }]).findings;
const book = '<DocumentationHelpLink reference={topic} />';
test("heading and text contracts accept contextual help", () => {
for (const component of ["PageHeader", "PageLayout", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]) {
assert.equal(inspect(`const Page=()=> <${component} title="Topic" titleHelp={${book}} />`).length, 0, component);
}
assert.equal(inspect(`const Page=()=> <PageTitle titleHelp={${book}}>Topic</PageTitle>`).length, 0);
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}>Existing label</TextWithHelp>`).length, 0);
});
test("detached action and body links are rejected", () => {
for (const source of [`<PageActionBar helpAction={${book}} />`, `<Card title="Topic" actions={${book}} />`, `<div>${book}</div>`, `<Card title={${book}} />`]) {
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
}
});
test("aliases, conditional help and simple local references remain checked", () => {
assert.equal(inspect(`import {DocumentationHelpLink as Book, Card as Box} from '@govoplan/core-webui'; const Page=()=> <Box title="Topic" titleHelp={<Book reference={topic}/>} />`).length, 0);
assert.equal(inspect(`import {DocumentationHelpLink as Book} from '@govoplan/core-webui'; const Page=()=> <div><Book reference={topic}/></div>`).length, 1);
assert.equal(inspect(`const help=${book}; const Page=()=> <Card title="Topic" titleHelp={enabled ? help : null}/>`).length, 0);
assert.equal(inspect(`const help=${book}; const Page=()=> <><Card title="Topic" titleHelp={help}/><PageActionBar helpAction={help}/></>`).length, 1);
assert.equal(inspect(`const help=${book}; const again=help; const Page=()=> <Card title="Topic" titleHelp={again}/>`).length, 0);
});
test("empty anchors, unknown contracts and nested interactive elements fail", () => {
for (const source of [`<Card titleHelp={${book}}/>`, `<TextWithHelp help={${book}}/>`, `<TextWithHelp help={${book}}> </TextWithHelp>`, `<Unknown title="Topic" titleHelp={${book}}/>`, `<Card title="Topic" titleHelp={<button>${book}</button>}/>`]) {
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
}
});
test("namespace and explicit component default imports cannot bypass placement", () => {
for (const source of [
`import * as UI from '@govoplan/core-webui'; const Page=()=> <div><UI.DocumentationHelpLink reference={topic}/></div>`,
`import Book from './components/help/DocumentationHelpLink'; const Page=()=> <div><Book reference={topic}/></div>`,
]) assert.equal(inspect(source).length, 1, source);
assert.equal(inspect(`import * as UI from '@govoplan/core-webui'; const Page=()=> <UI.Card title="Topic" titleHelp={<UI.DocumentationHelpLink reference={topic}/>}/>`).length, 0);
assert.equal(inspect(`import Book from './components/help/DocumentationHelpLink'; import Label from './components/help/TextWithHelp'; const Page=()=> <Label help={<Book reference={topic}/>}>Topic</Label>`).length, 0);
assert.equal(findDetachedDocumentation([{ path: "/fixture/Page.tsx", source: `import Book from './business/Book'; const Page=()=> <div><Book/></div>` }]).links, 0);
});
test("outer interactive containers and statically absent text are rejected", () => {
for (const source of [
`<button><TextWithHelp help={${book}}>Topic</TextWithHelp></button>`,
`<a href="/other"><Card title="Topic" titleHelp={${book}}/></a>`,
`<Card title="" titleHelp={${book}}/>`,
`<Card title={false} titleHelp={${book}}/>`,
`<PageTitle titleHelp={${book}}>{null}</PageTitle>`,
`<TextWithHelp help={${book}}>{/* Topic */}</TextWithHelp>`,
`<TextWithHelp help={${book}}>{undefined}</TextWithHelp>`,
`<TextWithHelp help={${book}}><span hidden>Topic</span></TextWithHelp>`,
`<TextWithHelp hidden help={${book}}>Topic</TextWithHelp>`,
]) assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
assert.equal(inspect(`const help=${book}; const Page=()=> <Button><TextWithHelp help={help}>Topic</TextWithHelp></Button>`).length, 1);
assert.equal(inspect(`const title=null; const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 1);
assert.equal(inspect(`const Page=()=> <Card title={translateText(title)} titleHelp={${book}}/>`).length, 0);
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}><TranslatedTitle/></TextWithHelp>`).length, 0);
assert.equal(inspect(`let title=""; title=translateText(key); const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 0);
});
+80
View File
@@ -0,0 +1,80 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { createRequire } from "node:module";
import { resolve } from "node:path";
import { test } from "node:test";
const root = resolve(import.meta.dirname, "../..");
const require = createRequire(resolve(root, "govoplan-core/webui/package.json"));
const { buildSync } = require("esbuild");
const labels = {
approvals: { "Approval requests": "Genehmigungsanträge" },
forms: { "Form meaning": "Formularbedeutung" },
"forms-runtime": { "Form instance": "Formularinstanz", "Forms runtime": "Formularlaufzeit" },
helpdesk: { "Helpdesk profiles": "Helpdesk-Profile" },
portal: { "Service directory": "Leistungsverzeichnis" },
projects: { Projects: "Projekte" },
reporting: { Reporting: "Reporting" },
"risk-compliance": { "Risk Compliance": "Risiko und Compliance" },
scheduling: { Scheduling: "Terminplanung" },
tickets: { Tickets: "Tickets" },
voting: { Ballots: "Abstimmungen" },
wiki: { Wiki: "Wiki" },
workflow: { Workflows: "Workflows", Workflow: "Workflow" },
idm: {
"Direct changes to this governed function are": "Direkte Änderungen an dieser gesteuerten Funktion sind",
"emergency overrides": "Notfallübersteuerungen",
". Use a request or grant above for the normal process.": ". Verwenden Sie für den regulären Prozess einen Antrag oder eine Vergabe.",
},
};
// Render the actual Core heading/locale contract with one owning module at a
// time. The in-memory bundle creates no shared component-build artifacts.
const names = Object.keys(labels);
const imports = names.map((name, index) => `import { generatedTranslations as t${index} } from ${JSON.stringify(resolve(root, `govoplan-${name}/webui/src/i18n/generatedTranslations.ts`))};`).join("\n");
const { outputFiles } = buildSync({
stdin: {
contents: `${imports}
import { renderToStaticMarkup } from 'react-dom/server';
import { PlatformLanguageProvider } from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/i18n/LanguageContext.tsx"))};
import PageTitle from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/PageTitle.tsx"))};
import DocumentationHelpLink from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/help/DocumentationHelpLink.tsx"))};
export const catalogs = [${names.map((_, index) => `t${index}`).join(",")}];
export function heading(index, language, label) {
return renderToStaticMarkup(<PlatformLanguageProvider preferredLanguageCode={language} moduleTranslations={[catalogs[index]]}>
<PageTitle titleHelp={<DocumentationHelpLink reference={{ contextId: "heading-test" }} />}>{label}</PageTitle>
</PlatformLanguageProvider>);
}`,
loader: "tsx",
resolveDir: resolve(root, "govoplan-core/webui"),
},
bundle: true,
write: false,
platform: "node",
format: "cjs",
jsx: "automatic",
external: ["react", "react-dom/server"],
logLevel: "silent",
});
const compiled = { exports: {} };
new Function("require", "module", "exports", outputFiles[0].text)(require, compiled, compiled.exports);
const { catalogs, heading } = compiled.exports;
test("new contextual labels render from their owning EN/DE catalogues", async (context) => {
for (const [index, name] of names.entries()) {
await context.test(name, () => {
const moduleSource = readFileSync(resolve(root, `govoplan-${name}/webui/src/module.ts`), "utf8");
assert.match(moduleSource, /import\s*\{\s*generatedTranslations\s*\}\s*from\s*["']\.\/i18n\/generatedTranslations["']/);
assert.match(moduleSource, /\btranslations(?:\s*:\s*generatedTranslations)?\s*,/);
for (const [english, german] of Object.entries(labels[name])) {
assert.equal(catalogs[index].en[english], english);
assert.equal(catalogs[index].de[english], german);
for (const [language, expected] of [["en", english], ["de", german]]) {
const markup = heading(index, language, english);
assert(markup.includes(`<h1>${expected}</h1>`), `${name}: ${language} contextual heading`);
assert(markup.includes(language === "de" ? "Benutzerdokumentation öffnen" : "Open user documentation"));
}
}
});
}
});
+438
View File
@@ -0,0 +1,438 @@
"""Audit the selected fixture workspace, never a fuller neighboring checkout."""
from argparse import Namespace
import importlib.util
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
from types import ModuleType, SimpleNamespace
import pytest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools/inventory/platform-interface-inventory.py"
WEBUI_SCRIPT = META_ROOT / "tools/inventory/extract-webui-structure.mjs"
SPEC = importlib.util.spec_from_file_location("audit_scope_inventory", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
inventory = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(inventory)
sys.path.insert(0, str(META_ROOT / "tools/devkit"))
from govoplan_devkit import docs, issues, runner # noqa: E402
from govoplan_devkit.workspace import Project, Repository # noqa: E402
def write(path, content):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
@pytest.fixture
def workspaces(tmp_path):
selected = tmp_path / "selected"
default = tmp_path / "fuller-default"
meta = tmp_path / "legacy-siblings/govoplan"
selected.mkdir()
meta.mkdir(parents=True)
catalog = {
"default_parent": str(default),
"repositories": [
{"name": "govoplan-core", "path": "govoplan-core"},
{"name": "govoplan-example", "path": "nested/example"},
{"name": "govoplan-optional", "path": "govoplan-optional"},
],
}
write(meta / "repositories.json", json.dumps(catalog))
write(
selected / "nested/example/webui/src/Page.tsx",
"export const page = <h1>SELECTED_ONLY</h1>;\n",
)
write(
default / "nested/example/webui/src/Page.tsx",
"export const page = <h1>DEFAULT_ONLY</h1>;\n",
)
write(
default / "govoplan-optional/webui/src/Page.tsx",
"export const page = <h1>DEFAULT_OPTIONAL</h1>;\n",
)
for item in catalog["repositories"]:
(default / item["path"] / "src").mkdir(parents=True)
return selected, default, meta, catalog
def test_explicit_partial_root_beats_fuller_legacy_discovery(workspaces, monkeypatch):
selected, default, meta, catalog = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
assert inventory._resolve_workspace_root(catalog) == default
assert inventory._resolve_workspace_root(catalog, selected) == selected
with pytest.raises(ValueError, match="existing directory"):
inventory._resolve_workspace_root(catalog, selected / "missing")
@pytest.mark.parametrize("relative", ["../fuller-default", "/absolute/path"])
def test_repository_paths_cannot_escape_explicit_root(workspaces, relative):
selected, _, _, _ = workspaces
with pytest.raises(ValueError, match="inside the selected workspace"):
inventory._validate_repository_roots(
{"repositories": [{"path": relative}]}, selected
)
def test_linked_repository_cannot_borrow_default_sources(workspaces):
selected, default, _, catalog = workspaces
(selected / "govoplan-core").symlink_to(
default / "govoplan-core", target_is_directory=True
)
with pytest.raises(ValueError, match="escapes the selected workspace"):
inventory._validate_repository_roots(catalog, selected)
def test_python_forwards_selected_root_and_configured_node(workspaces, monkeypatch):
selected, _, meta, _ = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
monkeypatch.setenv("NODE", "/selected/toolchain/node")
calls = []
def run(argv, **kwargs):
calls.append((argv, kwargs))
return SimpleNamespace(stdout=json.dumps({"workspaceRoot": str(selected)}))
monkeypatch.setattr(inventory.subprocess, "run", run)
assert inventory._extract_webui(selected)["workspaceRoot"] == str(selected)
argv, options = calls[0]
assert argv == [
"/selected/toolchain/node",
str(meta / "tools/inventory/extract-webui-structure.mjs"),
str(meta),
"--workspace-root",
str(selected),
]
assert options == {"check": True, "capture_output": True, "text": True}
def test_python_rejects_webui_result_from_another_root(workspaces, monkeypatch):
selected, default, _, _ = workspaces
monkeypatch.setattr(
inventory.subprocess,
"run",
lambda *a, **k: SimpleNamespace(
stdout=json.dumps({"workspaceRoot": str(default)})
),
)
with pytest.raises(ValueError, match="did not confirm"):
inventory._extract_webui(selected)
def test_main_forwards_same_root_to_every_collector(workspaces, monkeypatch):
selected, _, meta, catalog = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
roots = []
monkeypatch.setattr(
inventory, "_extract_webui", lambda root: roots.append(root) or {}
)
monkeypatch.setattr(
inventory,
"_extract_backend_endpoints",
lambda data, root: roots.append(root) or [],
)
monkeypatch.setattr(
inventory, "_extract_manifests", lambda data, root: roots.append(root) or []
)
monkeypatch.setattr(inventory, "_load_endpoint_declarations", lambda _: {})
monkeypatch.setattr(inventory, "_load_high_risk_help_baseline", lambda _: {})
monkeypatch.setattr(inventory, "_assemble_inventory", lambda **kwargs: {})
monkeypatch.setattr(inventory, "_render_markdown", lambda data: "fixture\n")
output = selected / "output"
monkeypatch.setattr(
sys,
"argv",
[str(SCRIPT), "--workspace-root", str(selected), "--output-dir", str(output)],
)
assert inventory.main() == 0
assert roots == [selected, selected, selected]
report = json.loads((output / "platform-interface-inventory.json").read_text())
assert report["workspace_root"] == str(selected)
assert report["workspace_selection"] == "explicit"
def install_fixture_compiler(root):
compiler = META_ROOT.parent / "govoplan-core/webui/node_modules/typescript"
if not compiler.is_dir() or not shutil.which("node"):
pytest.skip("Node and the installed Core TypeScript parser are required")
target = root / "govoplan-core/webui/node_modules/typescript"
target.parent.mkdir(parents=True, exist_ok=True)
# Parser dependencies may be shared; audited source checkouts may not.
target.symlink_to(compiler, target_is_directory=True)
def collect_webui(meta, selected=None):
argv = [shutil.which("node") or "node", str(WEBUI_SCRIPT), str(meta)]
if selected is not None:
argv += ["--workspace-root", str(selected)]
return subprocess.run(argv, capture_output=True, text=True, timeout=30)
def test_javascript_explicit_root_is_authoritative_and_legacy_cli_still_works(
workspaces,
):
selected, default, meta, _ = workspaces
install_fixture_compiler(selected)
install_fixture_compiler(default)
explicit = collect_webui(meta, selected)
assert explicit.returncode == 0, explicit.stderr
report = json.loads(explicit.stdout)
assert report["workspaceRoot"] == str(selected)
assert [item["value"] for item in report["visibleText"]] == ["SELECTED_ONLY"]
assert report["visibleText"][0]["file"] == "webui/src/Page.tsx"
legacy = collect_webui(meta)
assert legacy.returncode == 0, legacy.stderr
assert json.loads(legacy.stdout)["workspaceRoot"] == str(default)
assert {item["value"] for item in json.loads(legacy.stdout)["visibleText"]} == {
"DEFAULT_ONLY",
"DEFAULT_OPTIONAL",
}
def test_javascript_does_not_borrow_missing_core_dependencies(workspaces):
selected, default, meta, _ = workspaces
install_fixture_compiler(default)
result = collect_webui(meta, selected)
assert result.returncode != 0
assert (
str(selected / "govoplan-core/webui/node_modules/typescript") in result.stderr
)
assert not result.stdout
def test_javascript_rejects_source_root_linked_outside_workspace(workspaces):
selected, default, meta, _ = workspaces
install_fixture_compiler(selected)
linked = selected / "govoplan-optional/webui/src"
linked.parent.mkdir(parents=True)
linked.symlink_to(default / "govoplan-optional/webui/src", target_is_directory=True)
result = collect_webui(meta, selected)
assert result.returncode != 0
assert "source root escapes the selected workspace" in result.stderr
def test_backend_does_not_borrow_optional_endpoints(workspaces):
selected, default, _, catalog = workspaces
source = "from fastapi import APIRouter\nrouter = APIRouter()\n@router.get('/selected')\ndef endpoint(): pass\n"
write(selected / "nested/example/src/routes.py", source)
write(
default / "govoplan-optional/src/routes.py", source.replace("selected", "other")
)
endpoints = inventory._extract_backend_endpoints(catalog, selected)
assert [item["path"] for item in endpoints] == ["/selected"]
(selected / "nested/example/src/foreign.py").symlink_to(
default / "govoplan-optional/src/routes.py"
)
with pytest.raises(ValueError, match="Backend source path escapes"):
inventory._extract_backend_endpoints(catalog, selected)
def test_manifests_require_selected_core_sources(workspaces, monkeypatch):
selected, default, _, catalog = workspaces
write(
default / "govoplan-core/src/govoplan_core/core/platform_interfaces.py",
"raise AssertionError('foreign source imported')",
)
monkeypatch.syspath_prepend(str(default / "govoplan-core/src"))
with pytest.raises(ValueError, match="requires Core interface sources"):
inventory._extract_manifests(catalog, selected)
def test_cached_application_import_cannot_replace_missing_checkout(
workspaces, monkeypatch
):
selected, default, _, _ = workspaces
# Meta's own devkit/release packages may audit another workspace.
tooling = ModuleType("govoplan_release")
tooling.__file__ = str(default / "tools/release/govoplan_release/__init__.py")
modules = {"govoplan_release": tooling, "govoplan_devkit.docs": docs}
# Isolate the fixture from application packages collected by unrelated
# suites, without removing or replacing those real cached imports.
monkeypatch.setattr(inventory, "sys", SimpleNamespace(modules=modules))
inventory._assert_workspace_imports(selected)
application = ModuleType("govoplan_audit_fixture")
application.__file__ = str(
default / "govoplan-optional/src/govoplan_audit_fixture/__init__.py"
)
modules["govoplan_audit_fixture"] = application
with pytest.raises(ValueError, match="outside the selected inventory workspace"):
inventory._assert_workspace_imports(selected)
def test_partial_manifest_collection_uses_selected_sources_in_fresh_process(workspaces):
selected, default, _, catalog = workspaces
core = selected / "govoplan-core/src/govoplan_core"
write(core / "__init__.py", "")
write(core / "core/__init__.py", "")
write(
core / "core/platform_interfaces.py",
"def manifest_interface_catalog(manifest): return {'selected': True}\n",
)
write(
default / "govoplan-core/src/govoplan_core/__init__.py",
"raise AssertionError('foreign Core loaded')\n",
)
module = selected / "nested/example/src/govoplan_example"
write(module / "__init__.py", "")
write(module / "backend/__init__.py", "")
manifest = (
"from types import SimpleNamespace as S\n"
"def get_manifest():\n"
" return S(id='selected', name='Selected', version='1', dependencies=(), "
"optional_dependencies=(), required_capabilities=(), provides_interfaces=(), "
"capability_factories={}, permissions=(), documentation=(), architecture=None, "
"information_governance=S(to_dict=lambda: {}), frontend=None)\n"
)
write(module / "backend/manifest.py", manifest)
write(
default / "govoplan-optional/src/govoplan_optional/backend/manifest.py",
manifest.replace("selected", "foreign"),
)
code = (
"import importlib.util, json; from pathlib import Path; "
f"spec=importlib.util.spec_from_file_location('fixture', {str(SCRIPT)!r}); "
"module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module); "
f"print(json.dumps(module._extract_manifests({catalog!r}, Path({str(selected)!r}))))"
)
env = {
**os.environ,
"PYTHONPATH": os.pathsep.join(
str(default / item["path"] / "src") for item in catalog["repositories"]
),
}
result = subprocess.run(
[sys.executable, "-c", code],
env=env,
capture_output=True,
text=True,
timeout=30,
)
assert result.returncode == 0, result.stderr
manifests = json.loads(result.stdout)
assert [manifest["id"] for manifest in manifests] == ["selected"]
assert manifests[0]["interface_catalog"] == {"selected": True}
def doc_plan(workspace, monkeypatch):
repos = tuple(
Repository(name, workspace / name)
for name in ("govoplan", "govoplan-core", "govoplan-example")
)
project = Project("fixture", repos, {})
monkeypatch.setattr(docs, "load_project", lambda *a: project)
args = Namespace(
workspace_root=workspace,
project=None,
state_dir=workspace.parent / "state",
repo=["govoplan-example"],
changed=False,
)
return docs.build_doc_stages(args)
def test_docs_plan_forwards_root_and_persists_all_limitations(workspaces, monkeypatch):
selected, _, _, _ = workspaces
stages = doc_plan(selected, monkeypatch)
by_id = {stage["id"]: stage for stage in stages}
for stage_id in (
"docs.manifests",
"docs.interface-inventory",
"docs.plain-display-labels",
):
argv = by_id[stage_id]["argv"]
assert argv[argv.index("--workspace-root") + 1] == str(selected)
assert by_id["docs.translation-structure"]["argv"][1] == str(
selected / "govoplan-core/webui/scripts/audit-i18n-structural.mjs"
)
assert issues.coverage_notes(stages) == docs.LIMITATIONS
assert by_id["docs.plain-display-labels"]["argv"][-2:] == [
"--repo",
"govoplan-example",
]
def test_docs_limitations_survive_real_receipt_and_issue_evidence(
workspaces, monkeypatch, tmp_path
):
selected, _, _, _ = workspaces
stages = doc_plan(selected, monkeypatch)
repo = selected / "example"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True, timeout=10)
write(repo / "source.txt", "fixture\n")
subprocess.run(
["git", "-C", str(repo), "add", "source.txt"], check=True, timeout=10
)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
timeout=10,
)
project = tmp_path / "project.json"
write(
project,
json.dumps(
{
"schema_version": 1,
"name": "fixture",
"repositories": [{"name": "example", "path": "example"}],
"checks": [],
"profiles": {},
}
),
)
args = Namespace(
workspace_root=selected,
project=project,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="docs",
resume=None,
)
for stage in stages:
stage.update(
argv=[sys.executable, "-c", "print('fixture audit')"],
cwd=str(repo),
deps=[],
resources=[],
timeout_seconds=10,
)
monkeypatch.setattr(docs, "build_doc_stages", lambda _: stages)
monkeypatch.setattr(
runner, "environment_fingerprint", lambda *a, **k: "fixture-env"
)
result = docs.audit(args)
assert result["status"] == "passed", result
assert all(result["summary"].count(note) == 1 for note in docs.LIMITATIONS)
receipt = runner.read_receipt(selected, args.state_dir, result["run_id"])
assert issues.coverage_notes(receipt["stages"]) == docs.LIMITATIONS
evidence = issues.evidence_record(result["run_id"], args)
assert evidence["coverage_notes"] == docs.LIMITATIONS
assert evidence["source_state"] == "matches-current"
target = issues.NoteTarget(
repo, "https://gitea.example.invalid", "fixture", "example", 1
)
_, body = issues.render_note(
{"summary": [], "next": [], "body": ""}, evidence, target, "fixture"
)
assert all(note in body for note in docs.LIMITATIONS)
+88
View File
@@ -0,0 +1,88 @@
"""Signals during snapshot probes must not become passing run evidence."""
import os
import signal
import pytest
import test_devkit_runner as fixtures
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
example = fixtures.example
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
@pytest.mark.parametrize("probe", ["source", "environment"])
def test_interrupt_during_final_snapshot_is_not_a_passing_run(
example, monkeypatch, interruption, probe
):
args, repo = example
calls = {"source": 0, "environment": 0}
events, interruptions = [], []
args.on_progress = events.append
original = Checkpoints.source
def reached(kind):
calls[kind] += 1
if kind == probe and events and events[-1]["phase"] == "finalizing":
# Intermediate checkpoint probes are also real verification. Target
# finalization explicitly, while the runner owns signal handlers.
interruptions.append(kind)
os.kill(os.getpid(), interruption)
def source(self, *values, **kwargs):
reached("source")
return original(self, *values, **kwargs)
def environment(*_args, **_kwargs):
reached("environment")
return "fixture-environment"
monkeypatch.setattr(Checkpoints, "source", source)
monkeypatch.setattr(runner, "environment_fingerprint", environment)
result = runner.run_checks(args, [fixtures.stage(repo)])
assert interruptions == [probe]
assert calls[probe] >= 2
assert result["stages"][0]["status"] == "passed"
assert result["status"] == "interrupted"
assert result["_exit_code"] != 0
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["status"] == "interrupted"
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
@pytest.mark.parametrize("probe", ["source", "environment"])
def test_preparation_interrupt_stops_at_probe_boundary_without_running_checks(
example, monkeypatch, interruption, probe
):
args, repo = example
calls = []
original = Checkpoints.source
def reached(kind):
calls.append(kind)
if kind == probe:
os.kill(os.getpid(), interruption)
def source(self, *values, **kwargs):
reached("source")
return original(self, *values, **kwargs)
def environment(*_args, **_kwargs):
reached("environment")
return "fixture-environment"
def must_not_execute(*args, **kwargs):
pytest.fail("A cancelled preparation must not start a check")
monkeypatch.setattr(Checkpoints, "source", source)
monkeypatch.setattr(runner, "environment_fingerprint", environment)
monkeypatch.setattr(runner, "execute_stage", must_not_execute)
result = runner.run_checks(args, [fixtures.stage(repo)])
assert calls == (["source"] if probe == "source" else ["source", "environment"])
assert result["status"] == "interrupted"
assert result["snapshot_verified"] is False
assert result["_exit_code"] != 0
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["status"] == "interrupted"
+451
View File
@@ -0,0 +1,451 @@
"""Bounded planning fixtures: no application imports, compilers or servers."""
from __future__ import annotations
import argparse
import json
import shutil
from pathlib import Path
import sys
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.catalog import (
_expanded_repositories,
build_stages,
module_ui_stages,
)
from govoplan_devkit.docs import build_doc_stages
from govoplan_devkit.workspace import Project, Repository
class CatalogTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-catalog-")
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.core = Repository("govoplan-core", self.root / "govoplan-core", ("core",))
self.meta = Repository("govoplan", self.root / "govoplan", ("meta",))
self.module = Repository(
"govoplan-example", self.root / "govoplan-example", ("example",)
)
self.project = Project("fixture", (self.meta, self.core, self.module), {})
for repo in self.project.repositories:
repo.path.mkdir()
(self.meta.path / "tools/checks").mkdir(parents=True)
shutil.copyfile(
Path(__file__).resolve().parents[1] / "tools/checks/focused-phases.json",
self.meta.path / "tools/checks/focused-phases.json",
)
def write(self, relative, content):
path = self.root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
def plan(self, profile, repos=None, changed=False):
with patch("govoplan_devkit.workspace.load_project", return_value=self.project):
return build_stages(self.root, profile, repos or [], changed)
def test_ui_compiles_once_and_quick_does_not_compile(self):
self.write(
"govoplan-core/webui/package.json",
json.dumps(
{"scripts": {"test:components": "node scripts/run-component-tests.mjs"}}
),
)
self.write("govoplan-core/webui/scripts/run-component-tests.mjs", "// fixture")
self.assertFalse(
any(
"run-component-tests.mjs" in " ".join(item["argv"])
for item in self.plan("quick")
)
)
matches = [
item
for item in self.plan("ui")
if "run-component-tests.mjs" in " ".join(item["argv"])
]
self.assertEqual(len(matches), 1)
self.assertEqual(matches[0]["id"], "core.component-batch")
def test_module_script_metadata_is_bounded_and_deduplicated(self):
self.write(
"govoplan-example/webui/package.json",
json.dumps(
{
"scripts": {
"test:interface-pattern": "node scripts/test-interface-pattern-language.mjs",
"test:source": "node --test tests/source.test.mjs",
"test:dangerous-chain": "node tests/source.test.mjs && npm run dev",
"test:flags": "node --eval 'startServer()'",
}
}
),
)
self.write(
"govoplan-example/webui/scripts/test-interface-pattern-language.mjs",
"// source-only fixture",
)
self.write(
"govoplan-example/webui/tests/source.test.mjs", "// source-only fixture"
)
stages = module_ui_stages(self.module, reason="fixture")
self.assertEqual(len(stages), 2)
self.assertEqual(len({item["id"] for item in stages}), 2)
self.assertTrue(all(item["argv"][0] == "{node}" for item in stages))
self.assertFalse(any("&&" in item["argv"] for item in stages))
def test_full_is_never_narrowed_by_repo_filter(self):
result = self.plan("full", ["example"])
self.assertEqual(
[item["id"] for item in result],
[
"focused." + identity
for identity in (
"preflight",
"tooling",
"backend",
"core-ui",
"module-builds",
"browser",
"module-ui",
)
],
)
self.assertEqual(
[item["after"] for item in result],
[[], *[[item["id"]] for item in result[:-1]]],
)
self.assertTrue(all(item["deps"] == [] for item in result))
self.assertTrue(
all(
item["argv"]
== [
"bash",
str(self.meta.path / "tools/checks/check-focused.sh"),
"--phase",
item["id"].removeprefix("focused."),
]
for item in result
)
)
self.assertIn("webui:govoplan-example", result[-1]["resources"])
self.assertIn("backend:test-state", result[0]["resources"])
def test_changed_empty_is_not_full_verification(self):
with (
patch("govoplan_devkit.workspace.load_project", return_value=self.project),
patch("govoplan_devkit.workspace.selected_repositories", return_value=[]),
):
self.assertEqual(build_stages(self.root, "quick", [], True), [])
self.assertEqual(len(build_stages(self.root, "full", [], True)), 7)
def test_full_ui_scope_excludes_backend_only_but_includes_all_ui_owners(self):
backend = Repository(
"govoplan-backend-only", self.root / "govoplan-backend-only"
)
backend.path.mkdir()
(self.module.path / "webui").mkdir()
(self.core.path / "webui").mkdir()
project = Project("fixture", (*self.project.repositories, backend), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "full", ["example"], False)
for check in checks[:3]:
self.assertNotIn("inputs", check)
for check in checks[3:]:
self.assertEqual(
check["inputs"]["repos"],
["govoplan", "govoplan-core", "govoplan-example"],
)
(backend.path / "webui").mkdir()
with patch("govoplan_devkit.workspace.load_project", return_value=project):
replanned = build_stages(self.root, "full", [], False)
self.assertIn(backend.name, replanned[3]["inputs"]["repos"])
def test_full_missing_checkout_explicitly_falls_back_to_workspace_inputs(self):
missing = Repository("govoplan-missing", self.root / "govoplan-missing")
project = Project("fixture", (*self.project.repositories, missing), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "full", [], False)
self.assertTrue(all("inputs" not in check for check in checks))
self.assertTrue(
all(
any("workspace-wide" in note for note in check["coverage_notes"])
for check in checks[3:]
)
)
def test_full_requires_authoritative_phase_metadata(self):
(self.meta.path / "tools/checks/focused-phases.json").write_text("{}")
with self.assertRaisesRegex(ValueError, "phase metadata"):
self.plan("full")
def test_native_ui_guards_and_component_batch_share_safe_ui_owner_scope(self):
backend = Repository(
"govoplan-backend-only", self.root / "govoplan-backend-only"
)
backend.path.mkdir()
(self.module.path / "webui").mkdir()
(self.core.path / "webui").mkdir()
project = Project("fixture", (*self.project.repositories, backend), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "ui", ["example"], False)
for check in checks:
if check["id"] in {
"jsx-value-imports",
"heading-help",
"core.component-batch",
}:
self.assertEqual(
check["inputs"]["repos"],
["govoplan", "govoplan-core", "govoplan-example"],
)
else:
self.assertNotIn("inputs", check)
def test_unregistered_src_or_webui_disables_all_native_reuse(self):
for directory in ("src", "webui"):
with self.subTest(directory=directory):
unknown = self.root / "govoplan-unregistered" / directory
unknown.mkdir(parents=True)
try:
for profile in ("quick", "ui", "backend", "full"):
checks = self.plan(profile)
self.assertTrue(checks)
self.assertTrue(
all(check["reuse"] == "never" for check in checks)
)
self.assertTrue(
all(
any(
"Unregistered sibling" in note
for note in check["coverage_notes"]
)
for check in checks
)
)
finally:
unknown.rmdir()
unknown.parent.rmdir()
(self.root / "govoplan-unused-empty").mkdir()
self.assertTrue(all("reuse" not in check for check in self.plan("full")))
def test_unregistered_broken_source_symlink_does_not_allow_reuse(self):
unknown = self.root / "govoplan-unregistered"
unknown.mkdir()
(unknown / "src").symlink_to(self.root / "missing")
self.assertTrue(all(check["reuse"] == "never" for check in self.plan("full")))
def test_unknown_repo_is_not_silently_ignored(self):
with self.assertRaisesRegex(ValueError, "Unknown repository"):
self.plan("ui", ["not-a-repo"])
def test_changed_provider_selects_transitive_declared_consumers(self):
other = Repository("govoplan-other", self.root / "govoplan-other")
final = Repository("govoplan-final", self.root / "govoplan-final")
project = Project("fixture", (self.meta, self.module, other, final), {})
self.write("govoplan/tools/release/govoplan_release/contracts.py", "# fixture")
for name in ("example", "other", "final"):
self.write(f"govoplan-{name}/src/fixture/backend/manifest.py", "# fixture")
def contract(_path, repo_name):
gives = {
self.module.name: ["first"],
other.name: ["second"],
final.name: [],
}[repo_name]
needs = {
self.module.name: [],
other.name: ["first"],
final.name: ["second"],
}[repo_name]
return SimpleNamespace(
repo=repo_name,
provides_interfaces=[SimpleNamespace(name=value) for value in gives],
requires_interfaces=[SimpleNamespace(name=value) for value in needs],
)
with patch.dict(
sys.modules,
{
"govoplan_release.contracts": SimpleNamespace(
parse_manifest_contract=contract
)
},
):
selected, reason = _expanded_repositories(
project, [self.module], changed=True
)
self.assertEqual(
[repo.name for repo in selected], [self.module.name, other.name, final.name]
)
self.assertIn("declared interface consumers", reason)
def test_generic_dependency_closure_includes_filtered_prerequisites(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [
{"name": "one", "path": "one"},
{"name": "two", "path": "two"},
],
"checks": [
{
"id": "compile",
"argv": ["{node}", "compile.mjs"],
"cwd": "one",
"repos": ["two"],
},
{
"id": "test",
"argv": ["{node}", "test.mjs"],
"cwd": "one",
"deps": ["compile"],
"repos": ["one"],
},
{
"id": "other",
"argv": ["{python}", "test.py"],
"cwd": "two",
"repos": ["two"],
},
],
"profiles": {"quick": ["test", "other"]},
}
self.write("project.json", json.dumps(config))
result = build_stages(
self.root, "quick", ["one"], False, self.root / "project.json"
)
self.assertEqual([item["id"] for item in result], ["compile", "test"])
self.assertEqual(result[1]["deps"], ["compile"])
self.assertEqual(result[1]["cwd"], str(self.root / "one"))
def test_generic_cycles_and_duplicate_ids_fail(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [{"name": "one", "path": "one"}],
"checks": [
{"id": "one", "argv": ["test"], "deps": ["two"]},
{"id": "two", "argv": ["test"], "deps": ["one"]},
],
"profiles": {"quick": ["one"]},
}
self.write("project.json", json.dumps(config))
with self.assertRaisesRegex(ValueError, "Cyclic"):
build_stages(self.root, "quick", [], False, self.root / "project.json")
config["checks"][1]["id"] = "one"
self.write("project.json", json.dumps(config))
with self.assertRaisesRegex(ValueError, "Duplicate"):
build_stages(self.root, "quick", [], False, self.root / "project.json")
def test_generic_order_prerequisites_inputs_and_reuse_survive_planning(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [
{"name": "one", "path": "one"},
{"name": "two", "path": "two"},
],
"checks": [
{
"id": "prepare",
"argv": ["prepare"],
"repos": ["two"],
"reuse": "never",
"inputs": {"repos": ["two"]},
},
{
"id": "test",
"argv": ["test"],
"repos": ["one"],
"after": ["prepare"],
"reuse": "verified",
"inputs": {"repos": ["one"]},
},
{"id": "broad", "argv": ["check"], "repos": ["one"]},
],
"profiles": {"quick": ["test", "broad"]},
}
self.write("project.json", json.dumps(config))
result = build_stages(
self.root, "quick", ["one"], False, self.root / "project.json"
)
self.assertEqual([item["id"] for item in result], ["prepare", "test", "broad"])
self.assertEqual(result[1]["after"], ["prepare"])
self.assertEqual(result[1]["deps"], [])
self.assertEqual(result[0]["reuse"], "never")
self.assertEqual(result[1]["reuse"], "verified")
self.assertEqual(result[0]["inputs"], {"repos": ["two"]})
self.assertEqual(result[1]["inputs"], {"repos": ["one"]})
self.assertNotIn("inputs", result[2])
def test_generic_original_types_are_validated_before_coercion(self):
for field, invalid in (
("argv", "false"),
("argv", None),
("resources", "shared"),
("cwd", None),
("cwd", "../outside"),
("title", 5),
("timeout_seconds", True),
):
with self.subTest(field=field, invalid=invalid):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [{"name": "one", "path": "one"}],
"checks": [{"id": "test", "argv": ["test"], field: invalid}],
"profiles": {"quick": ["test"]},
}
self.write("project.json", json.dumps(config))
with self.assertRaises(ValueError):
build_stages(
self.root, "quick", [], False, self.root / "project.json"
)
def test_ui_coverage_notes_make_excluded_suites_explicit_and_discover_tests_folder(
self,
):
self.write(
"govoplan-example/webui/package.json",
json.dumps({"scripts": {"test:full-ui": "tsc && node tests/full-ui.js"}}),
)
self.write(
"govoplan-example/webui/tests/aggregate-report-structure.test.mjs",
"// fixture",
)
stages = self.plan("ui", ["example"])
self.assertTrue(
any("aggregate-report-structure" in item["id"] for item in stages)
)
self.assertTrue(
any("test:full-ui" in note for note in stages[0]["coverage_notes"])
)
def test_docs_reuses_existing_guards_and_only_narrows_plain_labels(self):
args = argparse.Namespace(
workspace_root=self.root,
state_dir=self.root / "state",
project=None,
repo=["example"],
changed=False,
)
with patch("govoplan_devkit.docs.load_project", return_value=self.project):
stages = build_doc_stages(args)
self.assertEqual(len(stages), 4)
self.assertIn("check-manifest-shapes.py", " ".join(stages[0]["argv"]))
self.assertIn("platform-interface-inventory.py", " ".join(stages[1]["argv"]))
self.assertIn("--strict-declarations", stages[1]["argv"])
self.assertEqual(stages[3]["argv"][-2:], ["--repo", "govoplan-example"])
self.assertFalse(
(self.root / "state").exists(), "Planning must not create artifacts"
)
if __name__ == "__main__":
unittest.main()
+158
View File
@@ -0,0 +1,158 @@
"""Public CLI contracts and an executable portable-project smoke fixture."""
import json
from pathlib import Path
import subprocess
import sys
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import cli, doctor, runner
from govoplan_devkit.common import META_ROOT
@pytest.mark.parametrize(
"arguments",
[["--json", "commands"], ["commands", "--json"], ["commands", "--format", "json"]],
)
def test_global_output_flags_work_on_either_side_of_command(arguments, capsys):
assert cli.main(arguments) == 0
output = json.loads(capsys.readouterr().out)
names = {item["command"] for item in output["commands"]}
assert {"context", "check", "doctor", "docs", "issues", "release", "git"} <= names
@pytest.mark.parametrize(
"command",
[
"context",
"doctor",
"check",
"resume",
"recover",
"docs",
"review",
"issues",
"release",
"git",
],
)
def test_command_help_does_not_require_live_services(command, capsys):
with pytest.raises(SystemExit) as stopped:
cli.main([command, "--help"])
assert stopped.value.code == 0
assert "usage:" in capsys.readouterr().out
def test_portable_project_executes_registered_commands_and_reads_receipt(
tmp_path, capsys, monkeypatch
):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "state"))
repo = tmp_path / "example"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"--allow-empty",
"-qm",
"fixture",
],
check=True,
)
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Portable",
"repositories": [{"name": "example", "path": "example"}],
"checks": [
{
"id": "test",
"argv": ["{python}", "-c", "print('portable ok')"],
"cwd": "example",
}
],
"profiles": {"quick": ["test"]},
}
)
)
common = ["--workspace-root", str(tmp_path), "--project", str(project), "--json"]
with patch.object(runner, "environment_fingerprint", return_value="fixture"):
assert cli.main(common + ["check", "--profile", "quick"]) == 0
result = json.loads(capsys.readouterr().out)
assert result["status"] == "passed"
assert cli.main(["status", result["run_id"], *common]) == 0
status = json.loads(capsys.readouterr().out)
assert status["snapshot_verified"] is True
assert cli.main(["logs", result["run_id"], "--stage", "test", *common]) == 0
assert "portable ok" in capsys.readouterr().out
def test_portable_example_matches_published_schema():
import jsonschema
jsonschema.validate(
json.loads((META_ROOT / "tools/devkit/examples/project.json").read_text()),
json.loads((META_ROOT / "tools/devkit/project.schema.json").read_text()),
)
def test_malformed_project_is_a_controlled_json_error(tmp_path, capsys):
project = tmp_path / "bad.json"
project.write_text('{"schema_version":true,"repositories":[]}')
assert (
cli.main(
[
"context",
"--workspace-root",
str(tmp_path),
"--project",
str(project),
"--json",
]
)
== 2
)
assert json.loads(capsys.readouterr().out)["status"] == "error"
def test_doctor_is_read_only_and_preserves_dependency_warnings(tmp_path, capsys):
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{"schema_version": 1, "repositories": [{"name": "example", "path": "."}]}
)
)
(tmp_path / "package.json").write_text("{}")
before = set(tmp_path.iterdir())
with (
patch.object(doctor, "tool_version", return_value="fixture"),
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
assert (
cli.main(
[
"doctor",
"--workspace-root",
str(tmp_path),
"--project",
str(project),
"--json",
]
)
== 0
)
output = json.loads(capsys.readouterr().out)
assert any(item["status"] == "warning" for item in output["checks"])
assert set(tmp_path.iterdir()) == before
+339
View File
@@ -0,0 +1,339 @@
"""Coverage inventory is explicit intent, never execution or guessed completion."""
import json
from pathlib import Path
import re
import shutil
import sys
from unittest.mock import patch
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/devkit"))
from govoplan_devkit.catalog import build_coverage, build_stages, module_ui_stages # noqa: E402
from govoplan_devkit.coverage import canonical_invocations # noqa: E402
from govoplan_devkit.package_tests import ( # noqa: E402
CORE_COMPONENT_SUITES,
declared_tests,
read_package,
)
from govoplan_devkit.workspace import Project, Repository # noqa: E402
@pytest.fixture
def fixture(tmp_path):
core = Repository("govoplan-core", tmp_path / "govoplan-core", ("core",))
meta = Repository("govoplan", tmp_path / "govoplan", ("meta",))
module = Repository("govoplan-example", tmp_path / "govoplan-example", ("example",))
for repo in (core, meta, module):
(repo.path / "webui/scripts").mkdir(parents=True)
(repo.path / "webui/tests").mkdir()
scripts = {
"test:components": "node scripts/run-component-tests.mjs",
**{
f"test:{name}": f"node scripts/run-component-tests.mjs {name}"
for name in CORE_COMPONENT_SUITES
},
}
(core.path / "webui/package.json").write_text(json.dumps({"scripts": scripts}))
(core.path / "webui/scripts/run-component-tests.mjs").write_text("// fixture")
(module.path / "webui/package.json").write_text(
json.dumps(
{
"scripts": {
"test:safe": "node --test tests/source.test.mjs",
"test:compound": "tsc && node test.js",
"test:bad-quote": "node 'broken",
}
}
)
)
(module.path / "webui/tests/source.test.mjs").write_text("// fixture")
script = meta.path / "tools/checks/check-focused.sh"
script.parent.mkdir(parents=True)
phase_metadata = ROOT / "tools/checks/focused-phases.json"
shutil.copyfile(phase_metadata, script.with_name("focused-phases.json"))
bodies = {
"core-ui": 'cd "$ROOT/webui"\n"$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components\n',
"module-ui": 'cd "$WORKSPACE_ROOT/govoplan-example/webui"\n"$NPM" run test:compound\n',
}
script.write_text(
"\n".join(
f"focused_phase_{phase['id'].replace('-', '_')}() {{\n# devkit-phase: {phase['id']} begin\n"
+ bodies.get(phase["id"], 'cd "$ROOT"\n')
+ f"# devkit-phase: {phase['id']} end\n}}\n"
for phase in json.loads(phase_metadata.read_text())["phases"]
)
)
project = Project("Fixture", (meta, core, module), {})
with (
patch("govoplan_devkit.workspace.load_project", return_value=project),
patch("govoplan_devkit.coverage.load_project", return_value=project),
):
yield tmp_path, core, module, project
def rows(coverage, repo="govoplan-core"):
return {row["name"]: row for row in coverage["suites"] if row["repo"] == repo}
def test_core_aliases_are_explicitly_excluded_in_quick_and_covered_by_one_ui_stage(
fixture,
):
root, _, _, _ = fixture
quick = rows(build_coverage(root, "quick", [], False))
ui = rows(build_coverage(root, "ui", [], False))
for suite in CORE_COMPONENT_SUITES:
assert quick["test:" + suite]["disposition"] == "excluded"
assert "quick" in quick["test:" + suite]["reason"]
assert ui["test:" + suite]["disposition"] == "covered_elsewhere"
assert ui["test:" + suite]["covering_stage"] == "core.component-batch"
assert ui["test:components"]["covered_components"] == list(CORE_COMPONENT_SUITES)
def test_full_reports_only_four_of_sixteen_components_and_exact_shell_suite(fixture):
root, _, _, _ = fixture
result = build_coverage(root, "full", ["example"], False)
core = rows(result)
aliases = [core["test:" + suite] for suite in CORE_COMPONENT_SUITES]
assert sum(item["disposition"] == "covered_elsewhere" for item in aliases) == 4
assert sum(item["disposition"] == "excluded" for item in aliases) == 12
assert len(core["test:components"]["covered_components"]) == 4
assert "4/16" in core["test:components"]["reason"]
module = rows(result, "govoplan-example")
assert module["test:compound"]["disposition"] == "planned"
assert module["test:safe"]["disposition"] == "excluded"
assert result["stages"] == [
"focused." + phase["id"]
for phase in json.loads(
(ROOT / "tools/checks/focused-phases.json").read_text()
)["phases"]
]
assert core["test:components"]["covering_stage"] == "focused.core-ui"
assert module["test:compound"]["covering_stage"] == "focused.module-ui"
def test_prebuilt_plan_avoids_replanning(fixture):
root, _, _, _ = fixture
stages = build_stages(root, "quick", [], False)
with patch(
"govoplan_devkit.catalog.build_stages",
side_effect=AssertionError("must not replan"),
):
result = build_coverage(root, "quick", [], False, stages=stages)
assert rows(result, "govoplan-example")["test:safe"]["disposition"] == "planned"
assert sum(result["counts"].values()) == result["suite_count"]
assert all(item["reason"] for item in result["suites"])
def test_missing_or_spoofed_phase_stage_does_not_grant_component_coverage(fixture):
root, _, _, _ = fixture
stages = build_stages(root, "full", [], False)
without_core = [item for item in stages if item["id"] != "focused.core-ui"]
result = build_coverage(root, "full", [], False, stages=without_core)
assert rows(result)["test:components"]["covered_components"] == []
assert (
rows(result, "govoplan-example")["test:compound"]["covering_stage"]
== "focused.module-ui"
)
next(item for item in stages if item["id"] == "focused.core-ui")["argv"] = ["true"]
spoofed = build_coverage(root, "full", [], False, stages=stages)
assert rows(spoofed)["test:components"]["covered_components"] == []
@pytest.mark.parametrize("wrapper", ["heredoc", "function", "conditional"])
def test_lookalike_phase_wrappers_outside_top_level_do_not_grant_coverage(
fixture, wrapper
):
root, core, _, project = fixture
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
path = meta / "tools/checks/check-focused.sh"
fake = 'focused_phase_core_ui() {\n# devkit-phase: core-ui begin\ncd "$ROOT/webui"\n"$NPM" run test:spoof\n# devkit-phase: core-ui end\n}\n'
prefix = {
"heredoc": "cat <<'BODY'\n" + fake + "BODY\n",
"function": "unused() {\n" + fake + "}\n",
"conditional": "if false; then\n" + fake + "fi\n",
}[wrapper]
path.write_text(prefix + path.read_text())
result = canonical_invocations(root, meta, core.path)
assert result["notes"] == []
assert [item["name"] for item in result["npm"]] == [
"test:components",
"test:compound",
]
@pytest.mark.parametrize("damage", ["missing", "duplicate", "bad-end"])
def test_invalid_marked_phase_bodies_do_not_infer_coverage(fixture, damage):
root, core, _, project = fixture
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
path = meta / "tools/checks/check-focused.sh"
text = path.read_text()
if damage == "missing":
text = text.replace("focused_phase_core_ui()", "unregistered_core_ui()")
elif damage == "duplicate":
text += text
else:
text = text.replace(
"# devkit-phase: core-ui end", "# devkit-phase: another end"
)
path.write_text(text)
result = canonical_invocations(root, meta, core.path)
assert result["npm"] == [] and result["node"] == []
assert "no phase coverage inferred" in result["notes"][0]
@pytest.mark.parametrize(
"repo_name,name,command",
[
("govoplan-example", "test:components", "node scripts/run-component-tests.mjs"),
(
"govoplan-core",
"test:dialog-focus",
"node scripts/run-component-tests.mjs dialog-focus && npm run dev",
),
(
"govoplan-core",
"test:unrecognized",
"node scripts/run-component-tests.mjs dialog-focus",
),
],
)
def test_component_alias_exemption_is_exact_and_core_only(
tmp_path, repo_name, name, command
):
repo = Repository(repo_name, tmp_path / repo_name)
package = repo.path / "webui/package.json"
package.parent.mkdir(parents=True)
package.write_text(json.dumps({"scripts": {name: command}}))
declared = declared_tests(repo, package)
assert declared[0]["component_suite"] is None
assert declared[0]["_argv"] is None
assert module_ui_stages(repo, reason="fixture") == []
@pytest.mark.parametrize(
"body",
[
"[]",
"null",
'{"scripts":[]}',
'{"scripts":{"test:bad":42}}',
'{"scripts":{"test:one":"node a","test:one":"node b"}}',
"[" * 2000 + "]" * 2000,
],
)
def test_malformed_package_metadata_is_a_controlled_error(tmp_path, body):
package = tmp_path / "package.json"
package.write_text(body)
with pytest.raises(ValueError):
read_package(package)
def test_oversize_package_and_symlinked_test_are_not_discovered(tmp_path):
repo = Repository("example", tmp_path)
webui = tmp_path / "webui"
(webui / "tests").mkdir(parents=True)
package = webui / "package.json"
package.write_text(" " * (1024 * 1024 + 1))
with pytest.raises(ValueError):
read_package(package)
package.write_text(
json.dumps({"scripts": {"test:escape": "node tests/escape.mjs"}})
)
(webui / "tests/escape.mjs").symlink_to(tmp_path / "outside.mjs")
(tmp_path / "outside.mjs").write_text("// fixture")
assert module_ui_stages(repo, reason="fixture") == []
def test_unparseable_and_sensitive_commands_do_not_leak_into_coverage(fixture):
root, _, module, _ = fixture
package = module.path / "webui/package.json"
package.write_text(
json.dumps(
{
"scripts": {
"test:secret": "node tests/source.test.mjs --token unknown-private-value",
"test:broken": "node 'unknown-other-secret",
}
}
)
)
result = build_coverage(root, "quick", [], False)
encoded = json.dumps(result)
assert (
"unknown-private-value" not in encoded and "unknown-other-secret" not in encoded
)
assert "_argv" not in encoded and '"command"' not in encoded
assert all(
item["disposition"] == "unsupported"
for item in rows(result, "govoplan-example").values()
)
def test_custom_check_coverage_redacts_separate_token_and_includes_unselected(tmp_path):
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"repositories": [{"name": "app", "path": "."}],
"checks": [
{
"id": "one",
"argv": ["check", "--token", "unknown-private-value"],
},
{"id": "two", "argv": ["true"]},
],
"profiles": {"quick": ["one"]},
}
)
)
result = build_coverage(tmp_path, "quick", [], False, project)
assert "unknown-private-value" not in json.dumps(result)
assert result["counts"]["planned"] == 1 and result["counts"]["excluded"] == 1
def test_canonical_parser_does_not_credit_comments_heredocs_conditionals_or_chains(
tmp_path,
):
meta, core = tmp_path / "govoplan", tmp_path / "govoplan-core"
path = meta / "tools/checks/check-focused.sh"
path.parent.mkdir(parents=True)
path.write_text(
'cd "$ROOT/webui"\n# "$NPM" run test:comment\n"$PYTHON" - <<\'PY\'\n"$NPM" run test:heredoc\nPY\nif false; then\n"$NPM" run test:conditional\nfi\n"$NPM" run test:compound && true\n"$NPM" run test:real\n'
)
result = canonical_invocations(tmp_path, meta, core)
assert [item["name"] for item in result["npm"]] == ["test:real"]
def test_real_canonical_gate_has_four_explicit_component_suites():
result = canonical_invocations(ROOT.parent, ROOT, ROOT.parent / "govoplan-core")
calls = [item for item in result["npm"] if item["name"] == "test:components"]
assert len(calls) == 1
assert calls[0]["args"] == [
"layout-primitives",
"page-layout",
"data-grid-actions",
"mail-components",
]
assert calls[0]["phase"] == "core-ui"
for name, phase in (
("test:module-permutations", "module-builds"),
("test:conformance", "browser"),
):
matching = [item for item in result["npm"] if item["name"] == name]
assert len(matching) == 1 and matching[0]["phase"] == phase
def test_known_core_component_aliases_match_the_owned_runner_registry():
runner = (
ROOT.parent / "govoplan-core/webui/scripts/run-component-tests.mjs"
).read_text()
block = runner.split("export const componentSuites = Object.freeze({", 1)[1].split(
"});", 1
)[0]
assert set(re.findall(r'^ "([a-z0-9-]+)":', block, re.M)) == set(
CORE_COMPONENT_SUITES
)
+164
View File
@@ -0,0 +1,164 @@
"""Portable preflight inference uses fixtures only and never installs or starts tools."""
from argparse import Namespace
from pathlib import Path
import sys
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import doctor
from govoplan_devkit.workspace import Project, Repository
def project(tmp_path, checks, *, tools=None, profiles=None):
return Project(
"Fixture",
(Repository("app", tmp_path / "app"), Repository("other", tmp_path / "other")),
{
"checks": checks,
"profiles": profiles
if profiles is not None
else {"quick": [item["id"] for item in checks]},
"tools": tools or {},
},
)
def check(identity, executable, *, repos=None, deps=None):
return {
"id": identity,
"argv": [executable, "--version"],
"cwd": ".",
"repos": repos or [],
"deps": deps or [],
}
def diagnose(tmp_path, configured, *, profile=None, repos=None, versions=None):
args = Namespace(
workspace_root=tmp_path,
project=tmp_path / "not-read.json",
repo=repos or [],
profile=profile,
)
tools = {"python": sys.executable, "node": "fixture-node", "npm": "fixture-npm"}
version_map = versions or {
sys.executable: "Python 3",
"fixture-node": "unavailable",
"fixture-npm": "unavailable",
}
with (
patch.object(doctor, "load_project", return_value=configured),
patch.object(doctor, "resolve_tools", return_value=tools),
patch.object(
doctor,
"tool_version",
side_effect=lambda executable, _env: version_map[executable],
) as probe,
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
result = doctor.diagnose(args)
return result, probe
def test_python_only_portable_project_does_not_probe_or_block_on_unused_node(tmp_path):
configured = project(tmp_path, [check("python-test", "{python}")])
result, probe = diagnose(tmp_path, configured)
assert result["_exit_code"] == 0
assert result["required_tools"] == ["python"]
assert probe.call_count == 1
assert {
item["id"] for item in result["checks"] if item["status"] == "not_required"
} == {"node", "npm"}
def test_selected_profile_excludes_other_profile_tool_requirements(tmp_path):
configured = project(
tmp_path,
[check("py", "{python}"), check("js", "{npm}")],
profiles={"quick": ["py"], "ui": ["js"]},
)
result, _ = diagnose(tmp_path, configured, profile="quick")
assert result["required_tools"] == ["python"]
result, _ = diagnose(tmp_path, configured)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
def test_selected_repository_includes_dependency_tool_requirements(tmp_path):
configured = project(
tmp_path,
[
check("build", "{npm}", repos=["other"]),
check("test", "{python}", repos=["app"], deps=["build"]),
],
)
result, _ = diagnose(tmp_path, configured, repos=["app"])
assert result["required_tools"] == ["node", "npm", "python"]
def test_unselected_repository_does_not_require_its_tool(tmp_path):
configured = project(
tmp_path,
[check("js", "{npm}", repos=["other"]), check("py", "{python}", repos=["app"])],
)
result, _ = diagnose(tmp_path, configured, repos=["app"])
assert result["required_tools"] == ["python"]
def test_explicit_tool_configuration_declares_indirect_script_dependency(tmp_path):
configured = project(tmp_path, [check("shell", "sh")], tools={"npm": "fixture-npm"})
result, _ = diagnose(tmp_path, configured)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
@pytest.mark.parametrize(
"executable,expected",
[
("/opt/node/bin/node", ["node", "python"]),
("npm", ["node", "npm", "python"]),
("npx", ["node", "npm", "python"]),
],
)
def test_direct_tool_names_are_inferred(tmp_path, executable, expected):
result, _ = diagnose(tmp_path, project(tmp_path, [check("test", executable)]))
assert result["required_tools"] == expected
def test_context_only_project_needs_no_node_and_makes_no_files(tmp_path):
before = set(tmp_path.iterdir())
result, _ = diagnose(tmp_path, project(tmp_path, [], profiles={}))
assert result["required_tools"] == ["python"]
assert set(tmp_path.iterdir()) == before
def test_missing_profile_is_not_silently_widened(tmp_path):
with pytest.raises(ValueError, match="does not declare profile"):
diagnose(tmp_path, project(tmp_path, []), profile="ui")
def test_native_govoplan_still_requires_all_three_tools(tmp_path):
configured = project(tmp_path, [])
args = Namespace(workspace_root=tmp_path, project=None, repo=[], profile=None)
with (
patch.object(doctor, "load_project", return_value=configured),
patch.object(
doctor,
"resolve_tools",
return_value={"python": sys.executable, "node": "node", "npm": "npm"},
),
patch.object(
doctor,
"tool_version",
side_effect=lambda executable, _env: (
"Python" if executable == sys.executable else "unavailable"
),
),
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
result = doctor.diagnose(args)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
+378
View File
@@ -0,0 +1,378 @@
"""Environment probes read bounded stable files; all fixtures are local."""
import hashlib
import os
from pathlib import Path
import sys
from types import SimpleNamespace
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import environment
from govoplan_devkit.common import digest
from govoplan_devkit.workspace import Project, Repository
def mutate_during_read(monkeypatch, callback):
original = hashlib.sha256
mutated = False
class MutatingHasher:
def __init__(self):
self.hasher = original()
def update(self, chunk):
nonlocal mutated
self.hasher.update(chunk)
if not mutated:
mutated = True
callback()
def hexdigest(self):
return self.hasher.hexdigest()
monkeypatch.setattr(environment.hashlib, "sha256", MutatingHasher)
@pytest.mark.parametrize("content", [b"", b"stable", b"x" * (1024 * 1024 + 3)])
def test_file_hash_keeps_existing_digest_and_exact_size_boundary(tmp_path, content):
path = tmp_path / "input"
path.write_bytes(content)
assert (
environment._environment_file_hash(path, len(content))
== hashlib.sha256(content).hexdigest()
)
def test_missing_optional_file_remains_optional_without_a_persistent_cache(tmp_path):
path = tmp_path / "input"
assert environment._environment_file_hash(path, 10) is None
path.write_bytes(b"first")
first = environment._environment_file_hash(path, 10)
path.write_bytes(b"other")
assert environment._environment_file_hash(path, 10) != first
@pytest.mark.parametrize("kind", ["fifo", "directory", "oversized", "dangling"])
def test_present_unsafe_input_is_rejected_without_blocking(tmp_path, kind):
path = tmp_path / "input"
if kind == "fifo":
os.mkfifo(path)
elif kind == "directory":
path.mkdir()
elif kind == "oversized":
path.write_bytes(b"too large")
else:
path.symlink_to(tmp_path / "missing")
with pytest.raises(ValueError, match="Environment input"):
environment._environment_file_hash(path, 2)
def test_stable_venv_executable_symlink_keeps_original_path_and_identity(tmp_path):
target = tmp_path / "real-python"
target.write_bytes(b"fixture binary")
executable = tmp_path / "venv" / "bin" / "python"
executable.parent.mkdir(parents=True)
executable.symlink_to(target)
repo = tmp_path / "repo"
metadata = repo / "node_modules" / ".package-lock.json"
metadata.parent.mkdir(parents=True)
metadata.write_bytes(b'{"fixture":true}')
project = Project("Fixture", (Repository("repo", repo),), {})
tools = {"python": str(executable)}
env = {"PATH": "fixture", "PWD": "ignored"}
distributions = b'[["fixture", "1"]]\n'
with (
patch.object(environment, "tool_version", return_value="fixture-version"),
patch.object(
environment,
"require_capture",
return_value=SimpleNamespace(returncode=0, stdout=distributions),
) as capture,
):
actual = environment.environment_fingerprint(tmp_path, project, tools, env)
assert actual == digest(
{
"environment": {"PATH": "fixture"},
"tools": {
"python": {
"path": str(executable),
"version": "fixture-version",
"sha256": hashlib.sha256(target.read_bytes()).hexdigest(),
}
},
"installed": {
str(metadata): hashlib.sha256(metadata.read_bytes()).hexdigest(),
"python_distributions": hashlib.sha256(distributions).hexdigest(),
},
}
)
assert capture.call_args.args[0][0] == str(executable)
def test_symlinked_package_directory_is_allowed_when_stable(tmp_path):
actual = tmp_path / "packages"
actual.mkdir()
(actual / ".package-lock.json").write_bytes(b"fixture")
link = tmp_path / "node_modules"
link.symlink_to(actual, target_is_directory=True)
assert environment._environment_file_hash(link / ".package-lock.json", 10)
def test_fifo_replacement_between_inspection_and_open_does_not_block(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"fixture")
real_open = os.open
def replace_before_open(target, flags):
path.unlink()
os.mkfifo(path)
assert flags & os.O_NONBLOCK
return real_open(target, flags)
monkeypatch.setattr(environment.os, "open", replace_before_open)
with pytest.raises(ValueError, match="bounded regular"):
environment._environment_file_hash(path, 10)
def test_growth_between_inspection_and_open_cannot_bypass_size_bound(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"a")
real_open = os.open
def grow_before_open(target, flags):
path.write_bytes(b"x" * 11)
return real_open(target, flags)
monkeypatch.setattr(environment.os, "open", grow_before_open)
with pytest.raises(ValueError, match="bounded regular"):
environment._environment_file_hash(path, 10)
def test_growth_during_read_cannot_bypass_size_bound(tmp_path, monkeypatch):
path = tmp_path / "input"
path.write_bytes(b"a")
mutate_during_read(monkeypatch, lambda: path.write_bytes(b"x" * 11))
with pytest.raises(ValueError, match="grew beyond"):
environment._environment_file_hash(path, 10)
def test_same_size_edit_with_restored_mtime_is_not_a_stable_identity(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"first")
metadata = path.stat()
def mutate():
path.write_bytes(b"other")
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
mutate_during_read(monkeypatch, mutate)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(path, 10)
def test_replacement_with_identical_bytes_is_not_a_stable_identity(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"fixture")
replacement = tmp_path / "replacement"
replacement.write_bytes(b"fixture")
mutate_during_read(monkeypatch, lambda: replacement.replace(path))
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(path, 10)
def test_symlink_retarget_with_identical_bytes_is_not_a_stable_identity(
tmp_path, monkeypatch
):
first, second, link = (tmp_path / name for name in ("first", "second", "python"))
first.write_bytes(b"fixture")
second.write_bytes(b"fixture")
link.symlink_to(first)
def retarget():
link.unlink()
link.symlink_to(second)
mutate_during_read(monkeypatch, retarget)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(link, 10)
def test_source_disappearing_during_read_fails_closed(tmp_path, monkeypatch):
path = tmp_path / "input"
path.write_bytes(b"fixture")
mutate_during_read(monkeypatch, path.unlink)
with pytest.raises(ValueError, match="Environment input"):
environment._environment_file_hash(path, 10)
def test_parent_symlink_retarget_is_rejected_even_for_the_same_target_inode(
tmp_path, monkeypatch
):
first, second, link = (tmp_path / name for name in ("first", "second", "bin"))
first.mkdir()
second.mkdir()
(first / "python").write_bytes(b"fixture")
os.link(first / "python", second / "python")
link.symlink_to(first, target_is_directory=True)
def retarget():
link.unlink()
link.symlink_to(second, target_is_directory=True)
mutate_during_read(monkeypatch, retarget)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(link / "python", 10)
def test_invalid_executable_is_rejected_before_any_version_process(tmp_path):
executable = tmp_path / "python"
os.mkfifo(executable)
project = Project("Fixture", (), {})
with patch.object(environment, "tool_version") as version:
with pytest.raises(ValueError, match="bounded regular"):
environment.environment_fingerprint(
tmp_path, project, {"python": str(executable)}, {}
)
version.assert_not_called()
def discovery_project(root, *, path="govoplan-backend"):
repo = root / path
(repo / "src").mkdir(parents=True)
return Project(
"GovOPlaN",
(Repository("govoplan-backend", repo),),
{"organization": "GovOPlaN"},
)
def test_native_shape_ignores_build_files_and_directory_timestamps(tmp_path):
project = discovery_project(tmp_path)
repo = project.repositories[0].path
(repo / "webui").mkdir()
before = environment._native_discovery_fingerprint(tmp_path, project)
for directory in (repo, repo / "src", repo / "webui"):
(directory / "temporary-build-output").write_text("changed")
(directory / "temporary-build-directory").mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == before
(repo / "webui" / "temporary-build-output").unlink()
assert environment._native_discovery_fingerprint(tmp_path, project) == before
def test_native_shape_detects_unknown_sibling_and_source_addition_removal(tmp_path):
project = discovery_project(tmp_path)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
sibling = tmp_path / "govoplan-unregistered"
sibling.mkdir()
empty = environment._native_discovery_fingerprint(tmp_path, project)
assert empty != baseline
for name in ("src", "webui"):
(sibling / name).mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != empty
(sibling / name).rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == empty
sibling.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
@pytest.mark.parametrize("repo_path", ["govoplan-backend", "nonstandard-layout"])
def test_native_shape_detects_registered_backend_gaining_webui(tmp_path, repo_path):
project = discovery_project(tmp_path, path=repo_path)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
webui = project.repositories[0].path / "webui"
webui.mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != baseline
webui.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
def test_native_shape_binds_registered_ownership_names(tmp_path):
project = discovery_project(tmp_path, path="nonstandard-layout")
renamed = Project(
project.name,
(Repository("govoplan-renamed", project.repositories[0].path),),
project.config,
)
assert environment._native_discovery_fingerprint(
tmp_path, project
) != environment._native_discovery_fingerprint(tmp_path, renamed)
def test_native_shape_detects_source_link_retarget_and_dangling_target(tmp_path):
project = discovery_project(tmp_path)
first, second = tmp_path / "first", tmp_path / "second"
first.mkdir()
second.mkdir()
link = project.repositories[0].path / "webui"
link.symlink_to(first, target_is_directory=True)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
link.unlink()
link.symlink_to(second, target_is_directory=True)
changed = environment._native_discovery_fingerprint(tmp_path, project)
assert changed != baseline
second.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != changed
def test_native_shape_encodes_non_directory_source_entries(tmp_path):
project = discovery_project(tmp_path)
webui = project.repositories[0].path / "webui"
baseline = environment._native_discovery_fingerprint(tmp_path, project)
webui.write_text("not a directory")
file_shape = environment._native_discovery_fingerprint(tmp_path, project)
assert file_shape != baseline
webui.unlink()
webui.mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != file_shape
def test_native_shape_audit_is_bounded_including_nonmatching_children(
tmp_path, monkeypatch
):
project = discovery_project(tmp_path)
(tmp_path / "unrelated-one").mkdir()
(tmp_path / "unrelated-two").mkdir()
monkeypatch.setattr(environment, "MAX_DISCOVERY_CHILDREN", 2)
with pytest.raises(ValueError, match="bounded ownership"):
environment._native_discovery_fingerprint(tmp_path, project)
def test_native_shape_resolution_loop_fails_closed(tmp_path):
project = discovery_project(tmp_path)
link = project.repositories[0].path / "webui"
link.symlink_to(link)
with pytest.raises(ValueError, match="discovery cannot be resolved"):
environment._native_discovery_fingerprint(tmp_path, project)
@pytest.mark.parametrize("portable", [False, True])
def test_only_native_environment_identity_binds_discovery_shape(tmp_path, portable):
project = discovery_project(tmp_path)
if portable:
project.config["schema_version"] = 1
executable = tmp_path / "fixture-python"
executable.write_text("not executed")
tools = {"python": str(executable)}
with (
patch.object(environment, "tool_version", return_value="fixture"),
patch.object(
environment,
"require_capture",
return_value=SimpleNamespace(returncode=0, stdout=b"[]"),
),
):
baseline = environment.environment_fingerprint(tmp_path, project, tools, {})
(tmp_path / "govoplan-new" / "src").mkdir(parents=True)
changed = environment.environment_fingerprint(tmp_path, project, tools, {})
assert (baseline == changed) is portable
+601
View File
@@ -0,0 +1,601 @@
"""Incremental checkpoints use isolated local repositories, never product or remote state."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import subprocess
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import atomic_json, read_json, resource_lock, state_root
@pytest.fixture
def workspace(tmp_path, monkeypatch):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg"))
root = tmp_path / "workspace"
repos = {}
for name in ("alpha", "beta"):
repo = root / name
repo.mkdir(parents=True)
subprocess.run(["git", "init", "-q", str(repo)], check=True)
(repo / "source.txt").write_text(name + " original\n")
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
)
repos[name] = repo
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Incremental fixture",
"repositories": [{"name": name, "path": name} for name in repos],
"checks": [],
"profiles": {},
}
)
)
traces = tmp_path / "traces"
traces.mkdir()
args = Namespace(
workspace_root=root,
project=config,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="quick",
resume=None,
repo=[],
changed=False,
)
return args, repos, traces
def stage(workspace, identity, repo="alpha", *, inputs=True, body="", **extra):
_, repos, traces = workspace
counter = traces / identity
code = (
"from pathlib import Path; "
f"counter=Path({str(counter)!r}); "
"counter.write_text(str(int(counter.read_text())+1 if counter.exists() else 1)); "
f"print({identity!r},flush=True); " + body
)
result = {
"id": identity,
"title": identity,
"argv": [sys.executable, "-c", code],
"cwd": str(repos[repo]),
"timeout_seconds": 5,
**extra,
}
if inputs:
result["inputs"] = {"repos": [repo]}
return result
def run(workspace, stages, *, environment="e" * 64):
args, _, _ = workspace
with patch.object(runner, "environment_fingerprint", return_value=environment):
return runner.run_checks(args, stages)
def count(workspace, identity):
path = workspace[2] / identity
return int(path.read_text()) if path.exists() else 0
def stages_by_id(result):
return {item["id"]: item for item in result["stages"]}
def test_checkpoint_probes_and_durable_persistence_hold_stage_resource_lock(workspace):
args, _, _ = workspace
locks = state_root(args.workspace_root) / "resource-locks"
original_identity, original_write = Checkpoints.identity, runner.atomic_json
probes, persisted = [], []
def assert_held():
with pytest.raises(RuntimeError, match="busy"):
with resource_lock(locks, "checkpoint-fixture"):
pass
def identity(self, selected):
assert_held()
probes.append(selected["id"])
return original_identity(self, selected)
def write(path, payload):
original_write(path, payload)
if (
payload.get("phase") == "checking"
and payload["stages"][0].get("checkpoint_verified") is True
and not persisted
):
assert_held()
assert read_json(path)["stages"][0]["checkpoint_verified"] is True
persisted.append(path)
with (
patch.object(Checkpoints, "identity", identity),
patch.object(runner, "atomic_json", side_effect=write),
):
result = run(
workspace, [stage(workspace, "a", resources=["checkpoint-fixture"])]
)
assert result["status"] == "passed"
assert len(probes) >= 2 and len(persisted) == 1
def test_checkpoint_save_failure_never_releases_dependent_execution(workspace):
args, _, _ = workspace
original_write, original_command, original_wait = (
runner.atomic_json,
runner._execute_stage_command,
runner.wait,
)
scheduler_waiting = threading.Event()
checkpoint_stalled = threading.Event()
scheduler_rechecked = threading.Event()
release_failure = threading.Event()
failed_write_paths, results, errors = [], [], []
def command(selected, *values, **kwargs):
if selected["id"] == "producer":
# Let the scheduler finish its initial running-state save and enter
# its wait loop before the command publishes a passing checkpoint.
assert scheduler_waiting.wait(timeout=5)
return original_command(selected, *values, **kwargs)
def wait(*values, **kwargs):
scheduler_waiting.set()
result = original_wait(*values, **kwargs)
if checkpoint_stalled.is_set():
scheduler_rechecked.set()
return result
def write(path, payload):
producer = next(
(item for item in payload.get("stages", []) if item["id"] == "producer"),
None,
)
if (
producer
and producer.get("checkpoint_verified") is True
and not failed_write_paths
):
failed_write_paths.append(path)
checkpoint_stalled.set()
assert release_failure.wait(timeout=5)
raise OSError("fixture checkpoint persistence failed")
return original_write(path, payload)
def execute():
try:
results.append(
run(
workspace,
[
stage(workspace, "producer"),
stage(workspace, "dependent", "beta", deps=["producer"]),
],
)
)
except BaseException as exc:
errors.append(exc)
worker = threading.Thread(target=execute)
with (
patch.object(runner, "atomic_json", side_effect=write),
patch.object(runner, "_execute_stage_command", side_effect=command),
patch.object(runner, "wait", side_effect=wait),
):
worker.start()
try:
assert checkpoint_stalled.wait(timeout=5)
assert scheduler_rechecked.wait(timeout=5)
# Keep persistence blocked across a scheduling turn: the in-memory
# producer status must not grant authority to start a consumer.
time.sleep(0.15)
assert count(workspace, "dependent") == 0
durable = read_json(failed_write_paths[0])
assert (
stages_by_id(durable)["producer"].get("checkpoint_verified") is not True
)
finally:
release_failure.set()
worker.join(timeout=8)
assert not worker.is_alive() and not errors
assert len(results) == 1 and results[0]["status"] == "failed"
assert count(workspace, "producer") == 1 and count(workspace, "dependent") == 0
by_id = stages_by_id(results[0])
assert by_id["producer"]["status"] == "failed"
assert by_id["producer"]["checkpoint_verified"] is False
assert "fixture checkpoint persistence failed" in by_id["producer"]["error"]
assert by_id["dependent"]["status"] == "skipped"
persisted = runner.read_receipt(
args.workspace_root, args.state_dir, results[0]["run_id"]
)
assert stages_by_id(persisted)["producer"]["checkpoint_verified"] is False
def test_verified_checkpoint_reuses_unchanged_stage_after_other_repo_changes(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
first = run(workspace, plan)
old_bytes = Path(first["receipt_path"]).read_bytes()
(repos["beta"] / "source.txt").write_text("beta changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
by_id = stages_by_id(second)
assert second["status"] == "passed" and second["snapshot_verified"] is True
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
assert by_id["a"]["reused_from"] == first["run_id"]
assert "reused_from" not in by_id["b"]
assert by_id["a"]["checkpoint_verified"] is True
assert by_id["a"]["checkpoint_version"] == 1
assert isinstance(by_id["a"]["cache_key"], str) and by_id["a"]["cache_key"]
assert second["source_fingerprint"] != first["source_fingerprint"]
assert Path(first["receipt_path"]).read_bytes() == old_bytes
def test_failed_post_execution_probe_preserves_actual_log_without_certifying_it(
workspace,
):
from govoplan_devkit.checkpoints import Checkpoints
identity = Checkpoints.identity
calls = 0
def probe(self, selected):
nonlocal calls
calls += 1
if calls == 2:
raise ValueError("fixture input became unreadable")
return identity(self, selected)
with patch.object(Checkpoints, "identity", probe):
result = run(workspace, [stage(workspace, "a")])
selected = result["stages"][0]
assert result["status"] == "stale"
assert selected["status"] == "stale" and selected["exit_code"] == 0
assert selected["checkpoint_verified"] is False
assert Path(selected["log_path"]).read_text() == "a\n"
assert "fixture input became unreadable" in selected["error"]
def test_changed_and_new_commands_run_without_discarding_unrelated_checkpoint(
workspace,
):
args, _, _ = workspace
first = run(workspace, [stage(workspace, "a"), stage(workspace, "b", "beta")])
args.resume = first["run_id"]
second = run(
workspace,
[
stage(workspace, "a"),
stage(workspace, "b", "beta", body="print('changed command')"),
stage(workspace, "new", "beta"),
],
)
assert second["status"] == "passed"
assert count(workspace, "a") == 1
assert count(workspace, "b") == 2
assert count(workspace, "new") == 1
assert first["plan_fingerprint"] != second["plan_fingerprint"]
def test_unrelated_profile_edit_does_not_change_existing_stage_execution_identity(
workspace,
):
args, _, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
config = json.loads(args.project.read_text())
config["checks"] = [
{
"id": "extra",
"argv": [sys.executable, "-c", "print('extra')"],
"cwd": "beta",
"repos": ["beta"],
}
]
config["profiles"] = {"backend": ["extra"]}
args.project.write_text(json.dumps(config))
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1
def test_added_dependency_edge_invalidates_consumer_even_when_both_repo_bytes_match(
workspace,
):
args, _, _ = workspace
a, b = stage(workspace, "a"), stage(workspace, "b", "beta")
first = run(workspace, [a, b])
args.resume = first["run_id"]
second = run(workspace, [a, {**b, "deps": ["a"]}])
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
def test_failed_run_reuses_successful_phase_but_retries_failed_phase(workspace):
args, _, traces = workspace
ready = traces / "ready"
plan = [
stage(workspace, "a"),
stage(
workspace,
"b",
"beta",
after=["a"],
body=f"raise SystemExit(0 if Path({str(ready)!r}).exists() else 3)",
),
]
first = run(workspace, plan)
assert first["status"] == "failed"
assert stages_by_id(first)["a"]["checkpoint_verified"] is True
ready.touch()
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
def test_changed_dependency_invalidates_transitive_consumers_but_not_independent_stage(
workspace,
):
args, repos, _ = workspace
plan = [
stage(workspace, "a"),
stage(workspace, "b", "beta", deps=["a"]),
stage(workspace, "c", "beta", deps=["b"]),
stage(workspace, "independent", "beta"),
]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert {
identity: count(workspace, identity)
for identity in ("a", "b", "c", "independent")
} == {"a": 2, "b": 2, "c": 2, "independent": 1}
def test_order_only_predecessor_change_does_not_invalidate_independent_stage(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 2 and count(workspace, "b") == 1
assert stages_by_id(second)["b"]["reused_from"] == first["run_id"]
def test_order_only_failure_prevents_new_downstream_execution(workspace):
plan = [
stage(workspace, "a", body="raise SystemExit(8)"),
stage(workspace, "b", "beta", after=["a"]),
]
result = run(workspace, plan)
assert result["status"] == "failed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 0
assert stages_by_id(result)["b"]["status"] == "skipped"
def test_never_reused_stage_and_actual_consumers_rerun_but_order_only_stage_can_reuse(
workspace,
):
args, _, _ = workspace
plan = [
stage(workspace, "producer", reuse="never"),
stage(workspace, "consumer", "beta", deps=["producer"]),
stage(workspace, "independent", "beta", after=["producer"]),
]
first = run(workspace, plan)
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "producer") == 2
assert count(workspace, "consumer") == 2
assert count(workspace, "independent") == 1
def test_unspecified_input_scope_stays_conservatively_workspace_wide(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "broad", "beta", inputs=False)]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "broad") == 2
def test_global_environment_change_invalidates_all_scoped_checkpoints(workspace):
args, _, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
first = run(workspace, plan)
args.resume = first["run_id"]
second = run(workspace, plan, environment="f" * 64)
assert second["status"] == "passed"
assert count(workspace, "a") == 2 and count(workspace, "b") == 2
assert second["environment_fingerprint"] != first["environment_fingerprint"]
def test_changed_declared_scope_prevents_same_id_reuse(workspace):
args, _, _ = workspace
selected = stage(workspace, "a")
first = run(workspace, [selected])
args.resume = first["run_id"]
selected = {**selected, "inputs": {"repos": ["alpha", "beta"]}}
second = run(workspace, [selected])
assert second["status"] == "passed" and count(workspace, "a") == 2
def test_input_mutation_during_passing_command_never_creates_reusable_checkpoint(
workspace,
):
args, repos, _ = workspace
plan = [
stage(
workspace,
"mutates",
body="Path('source.txt').write_text('mutated during stage\\n')",
)
]
first = run(workspace, plan)
assert first["status"] != "passed"
assert stages_by_id(first)["mutates"].get("checkpoint_verified") is not True
args.resume = first["run_id"]
# The same command is now stable against the already-mutated current bytes;
# its unverified first result still cannot be skipped.
second = run(workspace, plan)
assert count(workspace, "mutates") == 2
assert second["status"] == "passed"
assert (repos["alpha"] / "source.txt").read_text() == "mutated during stage\n"
def test_later_restoration_of_bytes_cannot_turn_invalid_phase_into_overall_pass(
workspace,
):
_, repos, _ = workspace
source = repos["alpha"] / "source.txt"
original = source.read_text()
plan = [
stage(
workspace,
"mutates",
body="Path('source.txt').write_text('temporary change\\n')",
),
stage(
workspace,
"restores",
"beta",
after=["mutates"],
body=f"Path({str(source)!r}).write_text({original!r})",
),
]
result = run(workspace, plan)
assert result["status"] != "passed"
assert stages_by_id(result)["mutates"].get("checkpoint_verified") is not True
def test_recovered_interruption_reuses_only_verified_durable_checkpoint(workspace):
args, _, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
first = run(workspace, plan)
receipt = read_json(Path(first["receipt_path"]))
receipt.update(
status="running", phase="checking", snapshot_verified=False, finished_at=None
)
unfinished = stages_by_id(receipt)["b"]
unfinished.update(status="running", exit_code=None, checkpoint_verified=False)
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers(dest="command", required=True))
recovered = parser.parse_args(
["recover", first["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=Namespace(**vars(args)),
)
recovery = recovered.handler(recovered)
assert recovery["status"] == "interrupted"
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
assert stages_by_id(second)["a"]["reused_from"] == first["run_id"]
@pytest.mark.parametrize("change", ["tampered", "missing"])
def test_cached_log_tamper_or_loss_is_not_accepted_as_verified_evidence(
workspace, change
):
args, _, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
path = Path(first["stages"][0]["log_path"])
if change == "missing":
path.unlink()
else:
path.write_text("tampered evidence\n")
args.resume = first["run_id"]
result = run(workspace, plan)
assert result["status"] == "failed"
assert (
result["snapshot_verified"] is not True
or result["stages"][0]["status"] == "failed"
)
assert result["stages"][0].get("checkpoint_verified") is not True
assert "reused_from" not in result["stages"][0]
assert "error" in result["stages"][0]
assert count(workspace, "a") == 1
def test_stale_run_donates_only_checkpoints_matching_final_current_inputs(workspace):
args, repos, _ = workspace
source = repos["alpha"] / "source.txt"
plan = [
stage(workspace, "a"),
stage(
workspace,
"changes-alpha",
"beta",
after=["a"],
body=f"Path({str(source)!r}).write_text('new alpha bytes\\n')",
),
]
first = run(workspace, plan)
assert first["status"] != "passed"
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 2
# This order-only phase did not consume alpha and its own scoped inputs are
# still identical; it may retain its independently verified checkpoint.
assert count(workspace, "changes-alpha") == 1
def test_legacy_checkpointless_receipt_does_not_gain_incremental_authority(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
receipt = read_json(Path(first["receipt_path"]))
receipt.pop("fingerprint_version", None)
for item in receipt["stages"]:
for field in ("checkpoint_version", "checkpoint_verified", "cache_key"):
item.pop(field, None)
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
(repos["beta"] / "source.txt").write_text("unrelated changed source\n")
args.resume = first["run_id"]
try:
second = run(workspace, plan)
except ValueError:
return # Rejecting legacy incremental reuse is also safely fail-closed.
assert second["status"] == "passed"
assert count(workspace, "a") == 2
+393
View File
@@ -0,0 +1,393 @@
"""Repository input identities use disposable Git fixtures, never remote effects."""
from copy import deepcopy
import os
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import inputs
from govoplan_devkit.inputs import InputSnapshotter, validate_input_declaration
from govoplan_devkit.workspace import Project, Repository, source_fingerprint
def git(repo, *arguments):
return subprocess.run(
["git", "-C", str(repo), *arguments], capture_output=True, check=True
).stdout
@pytest.fixture
def fixture(tmp_path, monkeypatch):
repositories = []
for name in ("alpha", "beta"):
path = tmp_path / name
path.mkdir()
git(path, "init", "-q")
(path / "source.txt").write_text("initial\n")
git(path, "add", "source.txt")
git(
path,
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
)
repositories.append(Repository(name, path, ("alias-" + name,)))
project = Project(
"Fixture",
tuple(repositories),
{"tools": {}, "profiles": {"quick": ["one"]}, "checks": []},
)
# Avoid incidental edits by other agents affecting these source-input tests.
# The actual tool-source hashing implementation is checked separately.
monkeypatch.setattr(
InputSnapshotter, "_tooling_identity", lambda *_: "fixture-devkit-source"
)
return tmp_path, project, InputSnapshotter(project, workspace_root=tmp_path)
def stage(identity="one", repos=None, **extra):
return {
"id": identity,
"argv": ["true"],
**({"inputs": {"repos": repos}} if repos is not None else {}),
**extra,
}
def fingerprint(snapshot, name="one"):
return snapshot["stages"][name]["fingerprint"]
def test_undeclared_inputs_remain_whole_workspace(fixture):
root, _, engine = fixture
result = engine.snapshot([stage()])
assert result["complete_workspace"] is True
assert result["observed_scope"]["repos"] == ["alpha", "beta"]
assert result["stages"]["one"]["scope"] == {
"version": 1,
"kind": "workspace",
"declared": False,
"repos": ["alpha", "beta"],
}
(root / "beta/source.txt").write_text("changed\n")
assert fingerprint(engine.snapshot([stage()])) != fingerprint(result)
def test_scoped_input_does_not_read_unrelated_repository_bytes(fixture, monkeypatch):
root, _, engine = fixture
original = inputs.os.open
def guarded(path, *args, **kwargs):
if Path(path).is_relative_to(root / "beta"):
raise AssertionError("Unrelated repository input was opened")
return original(path, *args, **kwargs)
monkeypatch.setattr(inputs.os, "open", guarded)
first = engine.snapshot([stage(repos=["alpha"])])
(root / "beta/source.txt").write_text("unrelated change")
second = engine.snapshot([stage(repos=["alpha"])])
assert fingerprint(first) == fingerprint(second)
assert second["scan_stats"]["repositories"] == 1
assert second["scan_stats"]["git_calls"] == 2
assert second["complete_workspace"] is False
def test_repository_union_scans_once_per_snapshot(fixture):
_, _, engine = fixture
result = engine.snapshot(
[
stage("one", ["alpha"]),
stage("two", ["alpha"]),
stage("three", ["beta", "alpha"]),
]
)
assert result["scan_stats"]["repositories"] == 2
assert result["scan_stats"]["git_calls"] == 4
assert result["stages"]["three"]["scope"]["repos"] == ["alpha", "beta"]
def test_session_rechecks_metadata_but_reuses_stable_file_content(fixture):
_, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
after = engine.snapshot(plan)
assert fingerprint(before) == fingerprint(after)
assert before["scan_stats"]["bytes"] == len("initial\n")
assert after["scan_stats"]["bytes"] == 0
assert after["scan_stats"]["cache_hits"] == 1
assert after["scan_stats"]["git_calls"] == 2
def test_changed_bytes_with_restored_mtime_are_not_reused(fixture):
root, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
path = root / "alpha/source.txt"
metadata = path.stat()
path.write_text("changed\n")
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
after = engine.snapshot(plan)
assert fingerprint(after) != fingerprint(before)
assert after["scan_stats"]["bytes"] == len("changed\n")
def test_inode_replacement_with_same_size_and_mtime_is_not_reused(fixture):
root, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
path = root / "alpha/source.txt"
metadata = path.stat()
replacement = root / "replacement.txt"
replacement.write_text("changed\n")
os.utime(replacement, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
replacement.replace(path)
after = engine.snapshot(plan)
assert fingerprint(before) != fingerprint(after)
assert after["scan_stats"]["cache_hits"] == 0
@pytest.mark.parametrize("flag", ["assume-unchanged", "skip-worktree"])
def test_hidden_index_flags_do_not_hide_worktree_changes(fixture, flag):
root, _, engine = fixture
git(root / "alpha", "update-index", "--" + flag, "source.txt")
before = engine.source_snapshot(["alpha"])
(root / "alpha/source.txt").write_text("hidden change\n")
after = engine.source_snapshot(["alpha"])
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
def test_index_head_and_new_deleted_files_are_bound(fixture):
root, _, engine = fixture
repo = root / "alpha"
identities = []
def record():
identities.append(
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
)
record()
(repo / "source.txt").write_text("changed\n")
record()
git(repo, "add", "source.txt")
record()
git(
repo,
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"change",
)
record()
(repo / "new.txt").write_text("new")
record()
(repo / "source.txt").unlink()
record()
assert len(set(identities)) == len(identities)
def test_missing_registered_repository_becoming_present_invalidates(fixture):
root, project, _ = fixture
extra = Repository("missing", root / "missing")
extended = Project("Fixture", (*project.repositories, extra), project.config)
engine = InputSnapshotter(extended, workspace_root=root)
before = engine.source_snapshot(["missing"])
extra.path.mkdir()
git(extra.path, "init", "-q")
after = engine.source_snapshot(["missing"])
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
def test_unrelated_plan_config_edits_do_not_invalidate_scoped_stage(fixture):
root, project, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
updated = deepcopy(project.config)
updated["profiles"]["full"] = ["unrelated"]
updated["checks"].append({"id": "unrelated", "argv": ["false"]})
other = InputSnapshotter(
Project("Changed label", project.repositories, updated), workspace_root=root
)
assert fingerprint(other.snapshot(plan)) == fingerprint(before)
def test_scope_command_tools_and_tooling_change_invalidate(fixture):
root, project, engine = fixture
first = engine.snapshot([stage(repos=["alpha"])])
assert fingerprint(engine.snapshot([stage(repos=["beta"])])) != fingerprint(first)
assert fingerprint(
engine.snapshot([stage(repos=["alpha"], argv=["false"])])
) != fingerprint(first)
assert fingerprint(
engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="new-env")
) != fingerprint(first)
changed = deepcopy(project.config)
changed["tools"] = {"node": "/another/node"}
other = InputSnapshotter(
Project("Fixture", project.repositories, changed), workspace_root=root
)
assert fingerprint(other.snapshot([stage(repos=["alpha"])])) != fingerprint(first)
def test_source_only_attestation_is_independent_of_plan_and_tooling(fixture):
_, _, engine = fixture
one = engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="env-one")
two = engine.snapshot(
[stage("different", ["alpha"], argv=["false"])], tooling_fingerprint="env-two"
)
source = engine.source_snapshot(["alpha"])
assert (
one["observed_source_fingerprint"]
== two["observed_source_fingerprint"]
== source["observed_source_fingerprint"]
)
assert (
one["stages"]["one"]["source_fingerprint"]
== source["observed_source_fingerprint"]
)
assert source["fingerprint_version"] == inputs.FINGERPRINT_VERSION
@pytest.mark.parametrize(
"value",
[
None,
{},
{"paths": ["src/**"]},
{"repos": []},
{"repos": "alpha"},
{"repos": ["unknown"]},
{"repos": ["alias-alpha"]},
{"repos": ["alpha", "alpha"]},
{"repos": [""]},
{"repos": ["alpha"], "extra": True},
],
)
def test_synthetic_stage_scopes_fail_closed(fixture, value):
_, _, engine = fixture
with pytest.raises(ValueError):
engine.snapshot([stage(inputs=value)])
def test_duplicate_stage_ids_and_escaped_repository_paths_fail(fixture):
root, project, engine = fixture
with pytest.raises(ValueError, match="Duplicate"):
engine.snapshot([stage(), stage()])
bad = Project("Bad", (Repository("outside", root.parent),), project.config)
with pytest.raises(ValueError, match="escapes"):
InputSnapshotter(bad, workspace_root=root)
def test_declaration_validation_is_pure_before_dry_run(monkeypatch):
monkeypatch.setattr(
inputs,
"git_bytes",
lambda *_a, **_k: pytest.fail("Planning validation ran Git"),
)
assert validate_input_declaration(
{"repos": ["beta", "alpha"]}, {"alpha", "beta"}
) == {"repos": ["alpha", "beta"]}
with pytest.raises(ValueError):
validate_input_declaration({"repos": ["unknown"]}, {"alpha", "beta"})
def test_runtime_records_are_not_hashed_as_execution_plans(fixture):
_, _, engine = fixture
with pytest.raises(ValueError, match="freshly planned"):
engine.snapshot([stage(repos=["alpha"], status="passed")])
def test_membership_count_is_bounded_before_entry_hashing(fixture, monkeypatch):
root, _, engine = fixture
(root / "alpha/new.txt").write_text("extra")
monkeypatch.setattr(inputs, "MAX_REPOSITORY_ENTRIES", 1)
with pytest.raises(ValueError, match="entry count"):
engine.source_snapshot(["alpha"])
def test_symlink_to_ignored_file_inside_repository_binds_target(fixture):
root, _, engine = fixture
repo = root / "alpha"
(repo / ".gitignore").write_text("ignored.txt\n")
(repo / "ignored.txt").write_text("first")
(repo / "linked.txt").symlink_to("ignored.txt")
before = engine.source_snapshot(["alpha"])
(repo / "ignored.txt").write_text("other")
assert (
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
!= before["observed_source_fingerprint"]
)
def test_dangling_symlink_target_creation_is_observed(fixture):
root, _, engine = fixture
repo = root / "alpha"
(repo / "linked.txt").symlink_to("future.txt")
before = engine.source_snapshot(["alpha"])
(repo / "future.txt").write_text("created")
assert (
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
!= before["observed_source_fingerprint"]
)
def test_cross_repository_and_directory_symlinks_are_not_silently_reused(fixture):
root, _, engine = fixture
link = root / "alpha/linked.txt"
link.symlink_to(root / "beta/source.txt")
with pytest.raises(ValueError, match="escapes"):
engine.source_snapshot(["alpha"])
link.unlink()
link.symlink_to(".")
with pytest.raises(ValueError, match="regular file"):
engine.source_snapshot(["alpha"])
def test_file_change_during_hash_is_rejected(fixture, monkeypatch):
root, _, engine = fixture
path = root / "alpha/source.txt"
original = inputs.os.fstat
changed = False
def mutate(descriptor):
nonlocal changed
metadata = original(descriptor)
if not changed:
changed = True
path.write_text("changed during read")
return metadata
monkeypatch.setattr(inputs.os, "fstat", mutate)
with pytest.raises(ValueError, match="changed"):
engine._file_hash(path, inputs._stats())
def test_legacy_whole_project_api_remains_independent(fixture):
_, project, engine = fixture
legacy = source_fingerprint(project)
engine.snapshot([stage(repos=["alpha"])])
assert source_fingerprint(project) == legacy
def test_real_tooling_inventory_is_bound_and_memoized(tmp_path):
repo = Repository("example", tmp_path)
project = Project("Fixture", (repo,), {"tools": {}})
engine = InputSnapshotter(project, workspace_root=tmp_path)
before_stats, after_stats = inputs._stats(), inputs._stats()
before = engine._tooling_identity(before_stats)
after = engine._tooling_identity(after_stats)
assert before == after
assert before_stats["tooling_files"] > 15 and before_stats["tooling_bytes"] > 0
assert after_stats["tooling_files"] == after_stats["tooling_cache_hits"]
assert after_stats["tooling_bytes"] == 0
+387
View File
@@ -0,0 +1,387 @@
import argparse
from copy import deepcopy
import json
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import issues
from govoplan_devkit.common import atomic_json, digest, state_root
from govoplan_devkit.workspace import load_project, source_fingerprint
@pytest.fixture
def args(tmp_path, monkeypatch):
monkeypatch.delenv("GITEA_TOKEN", raising=False)
root = tmp_path / "repo"
root.mkdir()
subprocess.run(["git", "init", "-q", str(root)], check=True)
subprocess.run(["git", "-C", str(root), "remote", "add", "origin", "https://gitea.invalid/team/repo.git"], check=True)
project = tmp_path / "project.json"
project.write_text(json.dumps({"schema_version": 1, "name": "Fixture", "repositories": [{"name": "repo", "path": "repo"}]}))
env_file = tmp_path / "private.env"
env_file.write_text("GITEA_TOKEN=fixture-private-token\nGITEA_OWNER=wrong-owner\n")
return argparse.Namespace(workspace_root=tmp_path, state_dir=tmp_path / "state", project=project,
root=root, issue=7, target_plan=None, remote="origin", env_file=env_file, evidence=None,
key="verification", note_summary=["Verified scoped changes."], next_steps=["Manual review remains."],
body_file=None, note_file=None, apply=False, retry_uncertain=False)
class FakeClient:
def __init__(self, target):
self.target = target
self.calls = []
self.comments = []
self.issue_id = 700
self.post_mode = "normal"
self.bad_binding = False
self.repeat_pages = False
self.bad_issue = False
self.fail = False
self.page_size = 2 # A server may enforce a cap smaller than the requested limit.
def close(self):
pass
def request_json(self, method, path, body=None, query=None):
self.calls.append((method, path, deepcopy(body), deepcopy(query)))
if self.fail:
raise RuntimeError("Remote echoed fixture-private-token")
if method == "GET" and path == self.target.path:
return {"id": self.issue_id, "number": self.target.issue, "html_url": self.target.url + ("wrong" if self.bad_issue else ""),
"state": "closed", "body": "- [ ] Preserve this checklist"}
if method == "GET" and path == self.target.path + "/comments":
page = 1 if self.repeat_pages else query["page"]
return deepcopy(self.comments[(page - 1) * self.page_size:page * self.page_size])
if method == "GET" and "/issues/comments/" in path:
comment = deepcopy(next(item for item in self.comments if str(item["id"]) == path.rsplit("/", 1)[1]))
comment["html_url"] = self.target.url + ("-other" if self.bad_binding else "") + "#issuecomment-" + str(comment["id"])
return comment
if method == "POST" and path == self.target.path + "/comments":
if self.post_mode == "timeout-before-commit":
raise TimeoutError("fixture-private-token")
comment = {"id": 1000 + len(self.comments), "body": body["body"]}
self.comments.append(comment)
if self.post_mode == "timeout-after-commit":
raise TimeoutError("fixture-private-token")
return deepcopy(comment)
raise AssertionError((method, path))
@property
def posts(self):
return [call for call in self.calls if call[0] == "POST"]
@pytest.fixture
def client(args, monkeypatch):
client = FakeClient(issues.resolve_target(args.root, args.issue, args.workspace_root))
monkeypatch.setattr(issues, "make_client", lambda target, token: client)
return client
def test_default_dry_run_is_offline_and_ignores_ambient_target_overrides(args, monkeypatch):
monkeypatch.setenv("GITEA_OWNER", "wrong")
monkeypatch.setenv("GITEA_REPO", "wrong")
monkeypatch.setenv("GITEA_URL", "https://wrong.invalid")
monkeypatch.setattr(issues, "make_client", lambda *_: pytest.fail("No network in preview"))
args.env_file = args.workspace_root / "does-not-exist.env"
result = issues.handle_note(args)
assert result["targets"][0]["url"] == "https://gitea.invalid/team/repo/issues/7"
assert result["targets"][0]["status"] == "would-post"
assert not args.state_dir.exists()
def test_scoped_checkpoint_evidence_compares_only_recorded_sources(args, monkeypatch):
from govoplan_devkit import runner
other = args.workspace_root / "other"
other.mkdir()
subprocess.run(["git", "init", "-q", str(other)], check=True)
declaration = json.loads(args.project.read_text())
declaration["repositories"].append({"name": "other", "path": "other"})
args.project.write_text(json.dumps(declaration))
monkeypatch.setattr(runner, "environment_fingerprint", lambda *_: "fixture-env")
run_args = argparse.Namespace(**{**vars(args), "jobs": 1, "profile": "quick", "dry_run": False})
result = runner.run_checks(run_args, [{"id": "scoped", "argv": [sys.executable, "-c", "print('ok')"],
"cwd": str(args.root), "inputs": {"repos": ["repo"]}}])
assert result["status"] == "passed"
evidence = issues.evidence_record(result["run_id"], args)
assert evidence["source_state"] == "matches-current"
assert evidence["source_scope"]["repos"] == ["repo"]
assert any("recorded repository input scope" in note for note in evidence["coverage_notes"])
(other / "unrelated.txt").write_text("Outside the recorded scope")
assert issues.evidence_record(result["run_id"], args)["source_state"] == "matches-current"
(args.root / "changed.txt").write_text("Inside the recorded scope")
assert issues.evidence_record(result["run_id"], args)["source_state"] == "historical-source-differs"
def test_passing_aggregate_cannot_hide_skipped_checks(args):
payload = receipt(args)
payload["stages"].append({"id": "skipped", "status": "skipped", "exit_code": None})
with pytest.raises(ValueError, match="inconsistent"):
issues.validate_receipt(payload, args.workspace_root)
def test_apply_is_append_only_read_back_and_idempotent(args, client):
args.apply = True
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "posted-verified"
assert len(client.posts) == 1
again = issues.handle_note(args)
assert again["targets"][0]["status"] == "existing-verified"
assert len(client.posts) == 1
assert {call[0] for call in client.calls} == {"GET", "POST"}
assert all(call[1].endswith("/comments") for call in client.posts)
def test_complete_pagination_continues_past_short_pages(args, client):
body = issues.handle_note(args)["targets"][0]["body"]
client.comments = [{"id": n, "body": "unrelated"} for n in range(1, 6)] + [{"id": 6, "body": body}]
args.apply = True
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "existing-verified"
assert not client.posts
assert max(call[3]["page"] for call in client.calls if call[3]) == 4
@pytest.mark.parametrize("collision", ["different-body", "duplicate", "repeating-pagination"])
def test_collisions_and_incomplete_pagination_refuse_post(args, client, collision):
body = issues.handle_note(args)["targets"][0]["body"]
client.comments = [{"id": 1, "body": body}]
if collision == "different-body":
client.comments[0]["body"] += "changed"
elif collision == "duplicate":
client.comments.append({"id": 2, "body": body})
else:
client.repeat_pages = True
args.apply = True
assert issues.handle_note(args)["_exit_code"] == 2
assert not client.posts
def test_timeout_after_commit_reconciles_without_replaying_post(args, client):
args.apply = True
client.post_mode = "timeout-after-commit"
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "reconciled-verified"
assert len(client.posts) == 1
assert "fixture-private-token" not in json.dumps(result)
def test_uncertain_post_requires_explicit_retry_after_reconciliation(args, client):
args.apply = True
client.post_mode = "timeout-before-commit"
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
client.post_mode = "normal"
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain-retry-required"
assert len(client.posts) == 1
args.retry_uncertain = True
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
assert len(client.posts) == 2
journals = list(state_root(args.workspace_root, args.state_dir).glob("issue-notes/*.json"))
assert journals and all("fixture-private-token" not in path.read_text() for path in journals)
def test_uncertain_readback_is_not_reported_verified_and_later_reconciles(args, client):
args.apply = True
client.bad_binding = True
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
client.bad_binding = False
assert issues.handle_note(args)["targets"][0]["status"] == "existing-verified"
assert len(client.posts) == 1
def test_journal_binds_immutable_issue_id(args, client):
args.apply = True
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
client.issue_id += 1
client.comments.clear()
args.retry_uncertain = True
assert issues.handle_note(args)["_exit_code"] == 2
assert len(client.posts) == 1
def _plan(args, rows):
args.root = args.issue = None
args.target_plan = args.workspace_root / "targets.json"
args.target_plan.write_text(json.dumps({"schema_version": 1, "targets": rows}))
@pytest.mark.parametrize("change", ["wrong-url", "duplicate", "outside", "mixed-origin", "credentials"])
def test_target_plans_require_exact_unique_workspace_bindings(args, change):
row = {"root": "repo", "issue": 7, "url": "https://gitea.invalid/team/repo/issues/7"}
if change == "wrong-url":
row["url"] = "https://gitea.invalid/team/other/issues/7"
elif change == "outside":
row["root"] = "../other"
elif change == "credentials":
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", "https://user:private@gitea.invalid/team/repo.git"], check=True)
rows = [row, deepcopy(row)] if change == "duplicate" else [row]
if change == "mixed-origin":
other = args.workspace_root / "other"
other.mkdir()
subprocess.run(["git", "init", "-q", str(other)], check=True)
subprocess.run(["git", "-C", str(other), "remote", "add", "origin", "http://gitea.invalid/team/other.git"], check=True)
rows.append({"root": "other", "issue": 8, "url": "http://gitea.invalid/team/other/issues/8"})
_plan(args, rows)
with pytest.raises(ValueError):
issues.handle_note(args)
def test_all_targets_preflight_before_first_post_and_posts_are_serial(args, client, monkeypatch):
first = client.target.record()
second = {**first, "issue": 8, "url": first["url"].rsplit("/", 1)[0] + "/8"}
_plan(args, [first, second])
second_client = FakeClient(issues.resolve_target(Path(first["root"]), 8, args.workspace_root))
second_client.bad_issue = True
monkeypatch.setattr(issues, "make_client", lambda target, token: client if target.issue == 7 else second_client)
args.apply = True
assert issues.handle_note(args)["_exit_code"] == 2
assert not client.posts and not second_client.posts
second_client.bad_issue = False
result = issues.handle_note(args)
assert [row["status"] for row in result["targets"]] == ["posted-verified", "posted-verified"]
assert len(client.posts) == len(second_client.posts) == 1
def test_credentials_and_response_errors_are_not_returned(args, client):
args.apply = True
client.fail = True
result = issues.handle_note(args)
assert result["_exit_code"] == 2
assert "fixture-private-token" not in json.dumps(result)
args.note_summary = ["fixture-private-token"]
with pytest.raises(ValueError, match="credential"):
issues.handle_note(args)
def test_marker_injection_and_symlink_inputs_are_rejected(args):
args.note_summary = [issues.MARKER_PREFIX + "fake -->"]
with pytest.raises(ValueError, match="reserved"):
issues.handle_note(args)
args.note_summary = ["safe"]
args.body_file = args.workspace_root / "alias.md"
args.body_file.symlink_to(args.env_file)
with pytest.raises(ValueError, match="symlink"):
issues.handle_note(args)
@pytest.mark.parametrize("remote", ["https://gitea.invalid/team/repo.git?token=secret", "https://gitea.invalid/team/repo.git#other", "https://gitea.invalid:bad/team/repo.git"])
def test_ambiguous_git_remote_is_not_silently_reinterpreted(args, remote):
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", remote], check=True)
with pytest.raises(ValueError):
issues.handle_note(args)
def test_structured_inputs_are_merged_as_data_not_executed(args):
args.note_file = args.workspace_root / "note.json"
args.note_file.write_text(json.dumps({"summary": ["Recorded earlier"], "next": ["Still pending"], "body": "$(never-execute)"}))
args.body_file = args.workspace_root / "body.md"
args.body_file.write_text("`never-run-this-either`")
body = issues.handle_note(args)["targets"][0]["body"]
assert "Recorded earlier" in body and "Verified scoped changes" in body
assert "$(never-execute)" in body and "`never-run-this-either`" in body
def receipt(args):
return {"schema_version": 1, "run_id": "fixture-run", "workspace_root": str(args.workspace_root),
"project_file": str(args.project), "source_fingerprint": source_fingerprint(load_project(args.workspace_root, args.project)),
"status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08T12:00:00Z", "finished_at": "2026-09-08T12:00:01Z",
"stages": [{"id": "fixture", "status": "passed", "exit_code": 0, "duration_seconds": 1,
"log_path": "/private/log-not-opened", "argv": ["never-execute-this"]}]}
def test_external_receipts_are_unverified_metadata_with_source_comparison(args):
path = args.workspace_root / "external.json"
path.write_text(json.dumps(receipt(args)))
args.evidence = str(path)
result = issues.handle_note(args)
evidence = result["evidence"]
assert evidence["origin"] == "external-unverified"
assert evidence["source_state"] == "matches-current"
assert "argv" not in evidence["stages"][0]
(args.root / "dirty.txt").write_text("changed")
assert issues.handle_note(args)["evidence"]["source_state"] == "historical-source-differs"
@pytest.mark.parametrize("mutation", ["foreign", "bad-fingerprint", "false-pass", "duplicate-stage", "unknown-status", "noninteger-exit", "invalid-state-shape", "invalid-stage-state-shape", "bool-schema"])
def test_invalid_receipt_cannot_supply_evidence(args, mutation):
payload = receipt(args)
if mutation == "foreign":
payload["workspace_root"] = str(args.workspace_root.parent)
elif mutation == "bad-fingerprint":
payload["source_fingerprint"] = "claimed-green"
elif mutation == "false-pass":
payload["stages"][0]["exit_code"] = 1
elif mutation == "duplicate-stage":
payload["stages"] *= 2
elif mutation == "unknown-status":
payload["status"] = "complete-review"
elif mutation == "invalid-state-shape":
payload["status"] = {}
elif mutation == "invalid-stage-state-shape":
payload["stages"][0]["status"] = []
elif mutation == "bool-schema":
payload["schema_version"] = True
else:
payload["stages"][0]["exit_code"] = False
with pytest.raises(ValueError):
issues.validate_receipt(payload, args.workspace_root)
def test_local_receipt_integrity_is_checked_but_not_an_attestation(args):
payload = receipt(args)
payload["integrity_sha256"] = digest(payload)
path = state_root(args.workspace_root, args.state_dir) / "runs/fixture-run/receipt.json"
atomic_json(path, payload)
args.evidence = "fixture-run"
result = issues.handle_note(args)
assert result["evidence"]["origin"] == "local-integrity-checked"
assert "not an independent attestation" in result["evidence"]["attestation"]
payload["status"] = "failed"
atomic_json(path, payload)
with pytest.raises(ValueError, match="integrity"):
issues.handle_note(args)
def test_timed_out_stage_is_reportable_without_claiming_success(args):
payload = receipt(args)
payload["status"] = "failed"
payload["stages"][0].update(status="timed_out", exit_code=-15)
assert issues.validate_receipt(payload, args.workspace_root)["status"] == "failed"
def test_scoped_coverage_limits_are_preserved_in_evidence_and_rendered_note(args, monkeypatch):
payload = receipt(args)
monkeypatch.setenv("FIXTURE_SECRET", "fixture-secret-value")
payload["stages"][0]["coverage_notes"] = ["Compiler/chained suite not run.", "Manual <script>not executed</script> fixture-secret-value"]
path = args.workspace_root / "coverage.json"
path.write_text(json.dumps(payload))
args.evidence = str(path)
result = issues.handle_note(args)
evidence = result["evidence"]
assert evidence["coverage_notes"] == evidence["stages"][0]["coverage_notes"]
assert "Compiler/chained suite not run." in result["targets"][0]["body"]
assert "omitted checks ran" in result["targets"][0]["body"]
assert "<script>" not in result["targets"][0]["body"]
assert "fixture-secret-value" not in json.dumps(result)
assert any("coverage limitation" in line for line in result["summary"])
@pytest.mark.parametrize("notes", ["not-a-list", [None], [""], ["x" * 4097], ["limit"] * 2049, [issues.MARKER_PREFIX + "injection"]])
def test_malformed_coverage_metadata_is_rejected(args, notes):
payload = receipt(args)
payload["stages"][0]["coverage_notes"] = notes
with pytest.raises(ValueError):
issues.validate_receipt(payload, args.workspace_root)
@pytest.mark.parametrize("snapshot_verified", [None, False, "true", 1])
def test_external_pass_requires_verified_snapshot_flag(args, snapshot_verified):
payload = receipt(args)
payload["snapshot_verified"] = snapshot_verified
with pytest.raises(ValueError, match="verified source snapshot"):
issues.validate_receipt(payload, args.workspace_root)
+539
View File
@@ -0,0 +1,539 @@
"""Git maintenance runs only in disposable local repositories/bare fixtures."""
from __future__ import annotations
import argparse
from contextlib import contextmanager
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import time
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import maintenance
class MaintenanceTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-git-")
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.repo = self.root / "repo"
self.repo.mkdir()
self.git("init", "-b", "main")
self.git("config", "user.name", "Fixture")
self.git("config", "user.email", "fixture@example.invalid")
self.git("config", "commit.gpgsign", "false")
self.write("selected.txt", "base selected\n")
self.write("unrelated.txt", "base unrelated\n")
self.git("add", "--", "selected.txt", "unrelated.txt")
self.git("commit", "-m", "fixture base")
self.base = self.git("rev-parse", "HEAD").strip()
self.project = self.root / "project.json"
self.project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "fixture",
"repositories": [{"name": "repo", "path": "repo"}],
}
)
)
self.args = argparse.Namespace(
workspace_root=self.root,
state_dir=self.root / "state",
project=self.project,
repo="repo",
path=["selected.txt"],
message="selected change",
apply=False,
)
def git(self, *args):
return subprocess.run(
["git", "-C", str(self.repo), *args],
check=True,
capture_output=True,
text=True,
).stdout
def write(self, name, value):
(self.repo / name).write_text(value)
def save_plan(self):
self.args.apply = True
result = maintenance.plan(self.args)
self.args.plan_id = result["plan"]["plan_id"]
return result
def test_preview_does_not_write_a_plan_or_stage(self):
self.write("selected.txt", "changed\n")
before = self.git("ls-files", "--stage")
result = maintenance.plan(self.args)
self.assertEqual(result["state"]["status"], "preview")
self.assertFalse((self.root / "state").exists())
self.assertEqual(self.git("ls-files", "--stage"), before)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_selected_commit_preserves_unrelated_staged_and_dirty_changes(self):
self.write("selected.txt", "selected final\n")
self.write("unrelated.txt", "unrelated staged\n")
self.git("add", "--", "unrelated.txt")
self.write("unrelated.txt", "unrelated unstaged\n")
staged = self.git("show", ":unrelated.txt")
self.save_plan()
before_plan = (
maintenance._receipt_path(self.args, self.args.plan_id)
.joinpath("plan.json")
.read_bytes()
)
self.args.apply = False
self.assertEqual(maintenance.commit(self.args)["state"]["status"], "preview")
self.args.apply = True
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(
self.git(
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
).strip(),
"selected.txt",
)
self.assertEqual(self.git("show", ":unrelated.txt"), staged)
self.assertEqual(
(self.repo / "unrelated.txt").read_text(), "unrelated unstaged\n"
)
self.assertEqual(
maintenance.commit(self.args)["state"]["commit"], result["state"]["commit"]
)
self.assertEqual(
maintenance._receipt_path(self.args, self.args.plan_id)
.joinpath("plan.json")
.read_bytes(),
before_plan,
)
def test_new_selected_file_and_deletion_are_supported(self):
self.write("new file.txt", "new\n")
(self.repo / "selected.txt").unlink()
self.git("add", "--", "selected.txt")
self.args.path = ["new file.txt", "selected.txt"]
self.save_plan()
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(
set(
self.git(
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
).splitlines()
),
{"new file.txt", "selected.txt"},
)
def test_selected_partial_staging_is_not_overwritten(self):
self.write("selected.txt", "staged\n")
self.git("add", "--", "selected.txt")
self.write("selected.txt", "unstaged\n")
with self.assertRaisesRegex(ValueError, "different staged"):
self.save_plan()
self.assertEqual(self.git("show", ":selected.txt"), "staged\n")
def test_stale_content_index_and_origin_block_commit(self):
for change in ("content", "index", "origin"):
with self.subTest(change=change):
self.write("selected.txt", f"selected {change}\n")
self.save_plan()
if change == "content":
self.write("selected.txt", "different content\n")
elif change == "index":
self.write("unrelated.txt", "changed index\n")
self.git("add", "--", "unrelated.txt")
else:
self.git("remote", "add", "origin", str(self.root / "other.git"))
with self.assertRaisesRegex(ValueError, "changed; prepare a new plan"):
maintenance.commit(self.args)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_active_hooks_and_filters_are_refused_not_bypassed(self):
self.write("selected.txt", "changed\n")
hook = self.repo / ".git/hooks/pre-commit"
hook.write_text("#!/bin/sh\nexit 99\n")
hook.chmod(0o700)
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
self.save_plan()
hook.unlink()
self.git("config", "filter.example.clean", "some-external-command")
self.write(".gitattributes", "*.txt filter=example\n")
with self.assertRaisesRegex(ValueError, "Active Git filter"):
self.save_plan()
def test_directory_traversal_and_symlinks_are_refused(self):
self.write("selected.txt", "changed\n")
(self.repo / "linked.txt").symlink_to(self.repo / "selected.txt")
for name in (".", "../outside", ".git/config", "linked.txt"):
with self.subTest(name=name):
self.args.path = [name]
with self.assertRaises(ValueError):
self.save_plan()
def test_post_index_change_hook_is_refused_before_it_can_run(self):
self.write("selected.txt", "changed\n")
marker = self.root / "hook-executed"
hook = self.repo / ".git/hooks/post-index-change"
hook.write_text(f"#!/bin/sh\ntouch '{marker}'\n")
hook.chmod(0o700)
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
self.save_plan()
self.assertFalse(marker.exists())
def test_remote_helpers_and_recursive_operations_are_refused(self):
self.write("selected.txt", "changed\n")
for setting in (
"remote.origin.receivepack",
"remote.origin.uploadpack",
"remote.origin.vcs",
"core.gitProxy",
"core.alternateRefsCommand",
"push.recurseSubmodules",
"submodule.recurse",
"remote.origin.promisor",
):
with self.subTest(setting=setting):
self.git("config", setting, "true")
with self.assertRaises(ValueError):
self.save_plan()
self.git("config", "--unset", setting)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_git_namespace_identity_and_repository_overrides_are_refused(self):
self.write("selected.txt", "changed\n")
for key in (
"GIT_NAMESPACE",
"GIT_COMMON_DIR",
"GIT_QUARANTINE_PATH",
"GIT_SHALLOW_FILE",
"GIT_REPLACE_REF_BASE",
"GIT_ALTERNATE_OBJECT_DIRECTORIES",
"GIT_AUTHOR_NAME",
"GIT_CONFIG_COUNT",
"GIT_UNKNOWN_OVERRIDE",
):
with self.subTest(key=key), patch.dict(os.environ, {key: "unexpected"}):
with self.assertRaisesRegex(ValueError, "environment overrides"):
self.save_plan()
self.assertFalse((self.root / "state").exists())
def test_noncanonical_paths_are_rejected_before_commit(self):
(self.repo / "dir").mkdir()
self.write("dir/file.txt", "new\n")
self.args.path = ["dir//file.txt"]
with self.assertRaisesRegex(ValueError, "explicit relative"):
self.save_plan()
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_replacement_history_is_not_treated_as_the_real_frozen_tree(self):
self.write("selected.txt", "changed\n")
tree = self.git("rev-parse", "HEAD^{tree}").strip()
replacement = self.git("commit-tree", tree, "-m", "replacement fixture").strip()
self.git("replace", self.base, replacement)
with self.assertRaisesRegex(ValueError, "replacement/graft"):
self.save_plan()
def test_normal_push_is_explicit_and_verified_against_a_local_bare_fixture(self):
remote = self.root / "origin.git"
subprocess.run(
["git", "init", "--bare", str(remote)], check=True, capture_output=True
)
self.git("remote", "add", "origin", str(remote))
self.write("selected.txt", "selected final\n")
frozen = self.save_plan()
self.assertNotIn(str(remote), json.dumps(frozen))
result = maintenance.commit(self.args)
self.args.apply = False
self.assertEqual(maintenance.push(self.args)["state"]["status"], "preview")
self.assertEqual(
self.git("ls-remote", "--refs", "origin", "refs/heads/main"), ""
)
self.args.apply = True
pushed = maintenance.push(self.args)
self.assertEqual(pushed["state"]["status"], "pushed")
self.assertTrue(
self.git("ls-remote", "--refs", "origin", "refs/heads/main").startswith(
result["state"]["commit"]
)
)
def test_interrupted_commit_is_reconciled_without_creating_another_commit(self):
self.write("selected.txt", "selected final\n")
self.save_plan()
original = maintenance._write_state
def fail_receipt(directory, state, **updates):
if updates.get("status") == "committed":
raise OSError("fixture interruption after commit")
return original(directory, state, **updates)
with patch.object(maintenance, "_write_state", side_effect=fail_receipt):
with self.assertRaises(OSError):
maintenance.commit(self.args)
head = self.git("rev-parse", "HEAD").strip()
self.assertNotEqual(head, self.base)
with self.assertRaisesRegex(ValueError, "reconcile"):
maintenance.commit(self.args)
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
def test_lost_push_receipt_is_reconciled_without_repeating_push(self):
remote = self.root / "origin.git"
subprocess.run(
["git", "init", "--bare", str(remote)], check=True, capture_output=True
)
self.git("remote", "add", "origin", str(remote))
self.write("selected.txt", "selected final\n")
self.save_plan()
maintenance.commit(self.args)
original = maintenance._git
pushes = []
def lose_response(repo, *argv, **kwargs):
result = original(repo, *argv, **kwargs)
if argv[0] == "push":
pushes.append(argv)
raise ValueError("fixture response lost after remote update")
return result
with patch.object(maintenance, "_git", side_effect=lose_response):
with self.assertRaises(ValueError):
maintenance.push(self.args)
with self.assertRaisesRegex(ValueError, "reconcile"):
maintenance.push(self.args)
reconciled = maintenance.reconcile(self.args)
self.assertEqual(reconciled["state"]["status"], "pushed")
self.assertEqual(len(pushes), 1)
def test_reconciliation_rechecks_state_after_acquiring_lock(self):
self.write("selected.txt", "selected final\n")
self.save_plan()
maintenance.commit(self.args)
payload, state, directory = maintenance._read(self.args, self.args.plan_id)
maintenance._write_state(
directory, state, status="needs_reconcile", operation="commit"
)
@contextmanager
def another_reconciliation(_args, _repo):
maintenance._write_state(directory, state, status="committed")
yield
with patch.object(maintenance, "_locked", another_reconciliation):
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertTrue(
any("no replay or downgrade" in line for line in result["summary"])
)
def test_worktree_edit_racing_with_commit_remains_uncommitted(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._git
def race(repo, *argv, **kwargs):
if argv[0] == "commit-tree":
self.write("selected.txt", "new editor bytes\n")
return original(repo, *argv, **kwargs)
with patch.object(maintenance, "_git", side_effect=race):
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("show", "HEAD:selected.txt"), "approved bytes\n")
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
self.assertEqual((self.repo / "selected.txt").read_text(), "new editor bytes\n")
def test_branch_compare_and_swap_never_overwrites_a_racing_commit(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._git
raced = []
def race(repo, *argv, **kwargs):
if argv[0] == "update-ref" and not raced:
tree = self.git("rev-parse", "HEAD^{tree}").strip()
other = self.git(
"commit-tree", tree, "-p", self.base, "-m", "other writer"
).strip()
self.git("update-ref", "refs/heads/main", other, self.base)
raced.append(other)
return original(repo, *argv, **kwargs)
with patch.object(maintenance, "_git", side_effect=race):
with self.assertRaises(ValueError):
maintenance.commit(self.args)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), raced[0])
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
def test_interrupted_index_publication_reconciles_without_another_commit(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
with patch.object(
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
):
with self.assertRaises(OSError):
maintenance.commit(self.args)
head = self.git("rev-parse", "HEAD").strip()
self.assertNotEqual(head, self.base)
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
def test_concurrent_index_changes_are_preserved_instead_of_overwritten(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._publish_index
def race(repo, locked, prepared, expected):
# Model an external writer that does not honor Git's index.lock.
outside = self.root / "outside.index"
outside.write_bytes((self.repo / ".git/index").read_bytes())
self.write("unrelated.txt", "new independent staging\n")
subprocess.run(
["git", "-C", str(self.repo), "add", "--", "unrelated.txt"],
env={**os.environ, "GIT_INDEX_FILE": str(outside)},
check=True,
capture_output=True,
)
(self.repo / ".git/index").write_bytes(outside.read_bytes())
return original(repo, locked, prepared, expected)
with patch.object(maintenance, "_publish_index", side_effect=race):
with self.assertRaisesRegex(ValueError, "index changed concurrently"):
maintenance.commit(self.args)
self.assertEqual(
self.git("show", ":unrelated.txt"), "new independent staging\n"
)
with self.assertRaises(ValueError):
maintenance.reconcile(self.args)
self.assertEqual(
self.git("show", ":unrelated.txt"), "new independent staging\n"
)
def test_selected_fifo_swap_cannot_block_capture(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._capture_blob
def race(repo, item):
(self.repo / "selected.txt").unlink()
os.mkfifo(self.repo / "selected.txt")
return original(repo, item)
started = time.monotonic()
with patch.object(maintenance, "_capture_blob", side_effect=race):
with self.assertRaises(ValueError):
maintenance.commit(self.args)
self.assertLess(time.monotonic() - started, 3)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_reconcile_never_adopts_a_head_that_races_with_index_preparation(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
with patch.object(
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
):
with self.assertRaises(OSError):
maintenance.commit(self.args)
candidate = self.git("rev-parse", "HEAD").strip()
original = maintenance._selected_index
raced = []
def race(repo, source, target, files):
result = original(repo, source, target, files)
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
other = self.git(
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
).strip()
self.git("update-ref", "refs/heads/main", other, candidate)
raced.append(other)
return result
with patch.object(maintenance, "_selected_index", side_effect=race):
with self.assertRaisesRegex(
ValueError, "changed during index reconciliation"
):
maintenance.reconcile(self.args)
recorded = maintenance.status(self.args)["state"]
self.assertEqual(recorded["status"], "needs_reconcile")
self.assertEqual(recorded["commit"], candidate)
self.assertNotEqual(recorded["commit"], raced[0])
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
def test_final_commit_receipt_cannot_bind_to_a_later_head(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._publish_index
raced = []
def race(repo, locked, prepared, expected):
original(repo, locked, prepared, expected)
candidate = self.git("rev-parse", "HEAD").strip()
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
other = self.git(
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
).strip()
self.git("update-ref", "refs/heads/main", other, candidate)
raced.append(other)
with patch.object(maintenance, "_publish_index", side_effect=race):
with self.assertRaisesRegex(
ValueError, "changed before the exact commit result"
):
maintenance.commit(self.args)
recorded = maintenance.status(self.args)["state"]
self.assertEqual(recorded["status"], "needs_reconcile")
self.assertNotEqual(recorded["commit"], raced[0])
def test_git_output_is_bounded_during_execution(self):
executable = self.root / "git"
executable.write_text(f"#!{sys.executable}\nprint('x' * 100000)\n")
executable.chmod(0o700)
with (
patch.dict(
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
),
patch.object(maintenance, "MAX_GIT_OUTPUT_BYTES", 256),
):
with self.assertRaisesRegex(ValueError, "output_limit"):
maintenance._git(self.repo, "fixture")
def test_timeout_terminates_git_helper_process_group(self):
executable = self.root / "git"
marker = self.root / "helper-ran"
child = (
"import signal,time,pathlib; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(1); pathlib.Path("
+ repr(str(marker))
+ ").write_text('unexpected')"
)
executable.write_text(
f"#!{sys.executable}\nimport subprocess,sys,time\nsubprocess.Popen([sys.executable, '-c', {child!r}])\ntime.sleep(20)\n"
)
executable.chmod(0o700)
with patch.dict(
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
):
with self.assertRaisesRegex(ValueError, "timed_out"):
maintenance._git(self.repo, "fixture", timeout=0.2)
time.sleep(1.1)
self.assertFalse(
marker.exists(),
"An orphaned helper must not finish the operation after timeout",
)
if __name__ == "__main__":
unittest.main()
+319
View File
@@ -0,0 +1,319 @@
"""Live/provisional monitoring and discovery never substitute for verified evidence."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
sys.path.insert(0, str(Path(__file__).resolve().parent))
from test_devkit_runner import example as example, run, stage
from govoplan_devkit import catalog, cli, monitoring, runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import atomic_json, read_json, state_root
from govoplan_devkit.process import OutputSnapshot
def command(args, *argv):
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers(dest="command", required=True))
selected = parser.parse_args(argv, namespace=Namespace(**vars(args)))
return selected.handler(selected)
def test_preparing_receipt_and_run_id_exist_before_source_fingerprinting(example):
args, repo = example
events = []
args.on_progress = events.append
original = Checkpoints.source
def fingerprint(self, *values, **kwargs):
assert events and events[0]["phase"] == "preparing"
first = events[0]
assert Path(first["receipt_path"]).is_file()
record = runner.read_receipt(
args.workspace_root, args.state_dir, first["run_id"]
)
if record["phase"] == "preparing":
assert record["status"] == "running"
assert record["snapshot_verified"] is False
assert record["source_fingerprint"] is None
return original(self, *values, **kwargs)
with patch.object(Checkpoints, "source", fingerprint):
result = run(args, [stage(repo)])
assert result["status"] == "passed"
assert events[-1]["phase"] == "finished"
assert events[-1]["run_id"] == events[0]["run_id"] == result["run_id"]
def test_preflight_failure_persists_discoverable_nonpassing_receipt(example):
args, repo = example
events = []
args.on_progress = events.append
with patch.object(
Checkpoints,
"source",
side_effect=ValueError("fixture fingerprint unavailable"),
):
with pytest.raises(ValueError, match="fixture fingerprint unavailable"):
run(args, [stage(repo)])
assert events[0]["phase"] == "preparing"
record = runner.read_receipt(
args.workspace_root, args.state_dir, events[0]["run_id"]
)
assert record["status"] == "failed"
assert record["phase"] == "finished"
assert record["snapshot_verified"] is False
assert all(item["status"] != "passed" for item in record["stages"])
assert "fixture fingerprint unavailable" in record["error"]
assert monitoring.latest_run(args)["status"] == "failed"
def test_run_history_is_read_only_and_cursor_pagination_has_no_duplicates(example):
args, repo = example
created = [run(args, [stage(repo)])["run_id"] for _ in range(3)]
base = state_root(args.workspace_root, args.state_dir)
before = {
path: (path.stat().st_mtime_ns, path.read_bytes())
for path in base.rglob("*")
if path.is_file()
}
page = monitoring.list_runs(Namespace(**vars(args), limit=2, before=None))
assert [item["run_id"] for item in page["runs"]] == sorted(created, reverse=True)[
:2
]
assert page["next_cursor"]
next_page = monitoring.list_runs(
Namespace(**vars(args), limit=2, before=page["next_cursor"])
)
assert [item["run_id"] for item in next_page["runs"]] == sorted(
created, reverse=True
)[2:]
assert next_page["next_cursor"] is None
assert monitoring.latest_run(args)["run_id"] == max(created)
assert before == {
path: (path.stat().st_mtime_ns, path.read_bytes())
for path in base.rglob("*")
if path.is_file()
}
def test_empty_run_history_does_not_create_a_state_directory(example):
args, _ = example
assert not args.state_dir.exists()
result = monitoring.list_runs(args)
assert result["runs"] == []
assert monitoring.latest_run(args)["status"] == "not_found"
assert not args.state_dir.exists()
def test_invalid_newest_run_is_visible_and_never_replaced_with_older_pass(example):
args, repo = example
older = run(args, [stage(repo)])["run_id"]
invalid = "zz-invalid-newest"
path = (
state_root(args.workspace_root, args.state_dir)
/ "runs"
/ invalid
/ "receipt.json"
)
atomic_json(path, {"malformed": True})
rows = monitoring.list_runs(args)
assert rows["runs"][0]["run_id"] == invalid
assert rows["runs"][0]["status"] == "invalid"
assert any(row["run_id"] == older for row in rows["runs"])
assert rows["_exit_code"] == 1
with pytest.raises(ValueError, match="Latest run is invalid"):
monitoring.latest_run(args)
@pytest.mark.parametrize(
"changes", [{"limit": 0}, {"limit": 101}, {"limit": True}, {"before": "../escape"}]
)
def test_history_rejects_invalid_bounds_and_cursor(example, changes):
args, _ = example
with pytest.raises(ValueError):
monitoring.list_runs(Namespace(**{**vars(args), **changes}))
def test_live_logs_are_available_before_completion_but_final_only_refuses_them(example):
args, repo = example
events, results, errors = [], [], []
args.on_progress = events.append
def execute():
try:
results.append(
run(
args,
[
stage(
repo,
code="import time; print('live ready',flush=True); time.sleep(1.5); print('finished')",
)
],
)
)
except BaseException as exc:
errors.append(exc)
worker = threading.Thread(target=execute)
worker.start()
try:
deadline = time.monotonic() + 4
live = None
while time.monotonic() < deadline:
if events:
live = command(args, "logs", events[0]["run_id"], "--stage", "one")
if "live ready" in live["excerpt"]:
break
time.sleep(0.02)
assert live is not None and "live ready" in live["excerpt"]
assert live["provisional"] is True
assert live["log_verified"] is False
assert live["snapshot_verified"] is False
assert live["run_status"] == "running"
with pytest.raises(ValueError, match="No finalized stage log"):
command(args, "logs", events[0]["run_id"], "--stage", "one", "--final-only")
finally:
worker.join(timeout=5)
assert not worker.is_alive() and not errors
final = command(
args, "logs", results[0]["run_id"], "--stage", "one", "--final-only"
)
assert final["provisional"] is False and final["log_verified"] is True
assert final["snapshot_verified"] is True
assert "finished" in final["excerpt"]
@pytest.mark.parametrize("quiet", [False, True])
def test_cli_keeps_json_stdout_clean_and_progress_on_stderr_or_quiet(
tmp_path, capsys, quiet
):
event = {
"event": "check_progress",
"run_id": "fixture",
"phase": "preparing",
"status": "running",
"counts": {"pending": 1},
"total_stages": 1,
"elapsed_seconds": 0,
"active_stages": [],
"receipt_path": str(tmp_path / "receipt.json"),
}
def check(args, _stages):
args.on_progress(event)
return {"status": "passed", "summary": ["fixture complete"]}
with (
patch.object(catalog, "build_stages", return_value=[]),
patch.object(runner, "run_checks", side_effect=check),
):
assert (
cli.main(
[
"check",
"--workspace-root",
str(tmp_path),
"--json",
*(["--quiet"] if quiet else []),
]
)
== 0
)
output = capsys.readouterr()
assert json.loads(output.out)["status"] == "passed"
if quiet:
assert output.err == ""
else:
assert json.loads(output.err)["event"] == "check_progress"
def snapshot(data, *, split=0, omitted=0, final=False):
return OutputSnapshot(data, b"", bool(omitted), omitted, 0, split, 0, final)
def test_provisional_output_withholds_incomplete_secret_line(monkeypatch):
monkeypatch.setenv("FIXTURE_SECRET", "credential-known-to-redaction")
data = b"public line\nAPI_KEY=credential-known-to-"
live = monitoring.capture_text(snapshot(data), provisional=True)
assert live == "public line\n"
final = monitoring.capture_text(
snapshot(b"public line\nAPI_KEY=credential-known-to-redaction", final=True)
)
assert "credential-known" not in final
assert "[redacted]" in final
def test_truncated_boundaries_cannot_expose_cut_authorization_or_secret_fragments():
head = b"safe head\nAuthorization: Bearer partial-secret-head"
tail = b"partial-secret-tail\nsafe final line\n"
for provisional in (False, True):
text = monitoring.capture_text(
snapshot(head + tail, split=len(head), omitted=90), provisional=provisional
)
assert "safe head" in text and "safe final line" in text
assert "partial-secret" not in text
assert "90 bytes omitted" in text
def test_final_failure_tail_survives_retention_and_stays_redacted(example, monkeypatch):
args, repo = example
monkeypatch.setenv("FIXTURE_SECRET", "sensitive-final-credential")
with patch.object(runner, "MAX_LOG_BYTES", 512):
result = run(
args,
[
stage(
repo,
code="import os; print('begin'); print('x'*20000); print(os.environ['FIXTURE_SECRET']); print('FINAL IMPORTANT ERROR'); raise SystemExit(9)",
)
],
)
final = command(
args, "logs", result["run_id"], "--stage", "one", "--final-only"
)
assert result["status"] == "failed"
assert final["log_verified"] is True
assert "FINAL IMPORTANT ERROR" in final["excerpt"]
assert "sensitive-final-credential" not in final["excerpt"]
assert "[redacted]" in final["excerpt"]
assert result["stages"][0]["omitted_output_bytes"] > 0
@pytest.mark.parametrize("maximum", [1, 2, 3, 7, 75, 76, 77, 80, 81, 82, 255])
@pytest.mark.parametrize("tail_only", [False, True])
def test_tiny_display_bounds_preserve_valid_utf8_and_byte_limit(maximum, tail_only):
value = "Ä😊 " * 200 + "FINAL"
text = monitoring.bounded_display(value, maximum, tail_only=tail_only)
assert len(text.encode("utf-8", errors="strict")) <= maximum
assert text.endswith("L")
if maximum >= 5 and (tail_only or maximum >= 255):
assert text.endswith("FINAL")
def test_live_record_is_bounded_provisional_and_rejects_mismatched_identity(example):
args, _ = example
run_id, stage_id = "fixture-run", "fixture-stage"
path = (
state_root(args.workspace_root, args.state_dir)
/ "runs"
/ run_id
/ (stage_id + ".log")
)
monitoring.write_live(path, stage_id, snapshot(b"line\n"), time.monotonic())
live = monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
assert live["provisional"] is True and live["excerpt"] == "line\n"
record = read_json(path.with_suffix(".live.json"))
record["run_id"] = "different-run"
atomic_json(path.with_suffix(".live.json"), record)
with pytest.raises(ValueError, match="Invalid provisional log"):
monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
+249
View File
@@ -0,0 +1,249 @@
"""Bounded fixture processes only; no project servers or external transports."""
from pathlib import Path
from dataclasses import FrozenInstanceError
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.process import _OutputBuffer, require_capture, run_captured
def python(code, **kwargs):
return run_captured([sys.executable, "-c", code], **kwargs)
def test_large_stdin_and_separate_outputs_are_multiplexed():
result = python(
"import sys; sys.stderr.write('diagnostic'); print(len(sys.stdin.buffer.read()))",
input_bytes=b"x" * 400000,
)
assert result.status == "passed"
assert result.stdout == b"400000\n"
assert result.stderr == b"diagnostic"
def test_live_output_limit_terminates_unbounded_producer():
result = python(
"import os;\nwhile True: os.write(1, b'x'*65536)", max_stdout=2048, timeout=3
)
assert result.status == "output_limit"
assert result.truncated
assert len(result.stdout) == 2048
def test_drain_mode_caps_memory_without_turning_success_into_failure():
result = python("print('x'*100000)", max_stdout=1024, terminate_on_limit=False)
assert result.status == "passed"
assert result.truncated and len(result.stdout) == 1024
def test_deadline_applies_after_command_closes_both_output_streams():
started = time.monotonic()
result = python(
"import os,time; os.close(1); os.close(2); time.sleep(30)", timeout=0.15
)
assert result.status == "timed_out"
assert time.monotonic() - started < 5
def test_cancellation_stops_owned_process():
cancelled = threading.Event()
timer = threading.Timer(0.1, cancelled.set)
timer.start()
try:
result = python("import time; time.sleep(30)", cancelled=cancelled)
finally:
timer.cancel()
timer.join()
assert result.status == "interrupted"
@pytest.mark.parametrize("redirected", [False, True])
def test_outliving_child_is_not_success_and_cannot_keep_running(redirected):
code = (
"import subprocess,sys; child=subprocess.Popen([sys.executable,'-c','import time; time.sleep(30)']"
+ (",stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL" if redirected else "")
+ "); print(child.pid,flush=True)"
)
result = python(code, timeout=4)
assert result.status == "leaked_process"
child_pid = int(result.stdout.strip())
# A terminated adopted child may remain a zombie until the host's init reaps it.
proc = Path(f"/proc/{child_pid}/stat")
if proc.exists():
assert proc.read_text().split(")", 1)[1].strip().split()[0] == "Z"
def test_required_capture_rejects_nonordinary_completion():
with pytest.raises(ValueError, match="output_limit"):
require_capture([sys.executable, "-c", "print('x'*10000)"], max_stdout=32)
def test_ordinary_nonzero_exit_retains_diagnostics():
result = python("import sys; print('problem',file=sys.stderr); sys.exit(7)")
assert (result.status, result.returncode, result.stderr) == (
"failed",
7,
b"problem\n",
)
def test_selector_setup_failure_still_terminates_spawned_process():
from govoplan_devkit import process
original = process.subprocess.Popen
spawned = []
def capture(*args, **kwargs):
child = original(*args, **kwargs)
spawned.append(child)
return child
with (
patch.object(process.subprocess, "Popen", side_effect=capture),
patch.object(
process.selectors,
"DefaultSelector",
side_effect=OSError("fixture selector unavailable"),
),
):
with pytest.raises(OSError, match="selector unavailable"):
python("import time; time.sleep(30)")
assert len(spawned) == 1 and spawned[0].poll() is not None
def test_head_tail_keeps_the_actual_failure_tail_within_original_bound():
output = b"BEGIN" + b"middle" * 100 + b"FINAL ERROR"
result = python(
f"import os; os.write(1, {output!r}); raise SystemExit(7)",
max_stdout=32,
capture_mode="head_tail",
terminate_on_limit=False,
)
assert (result.status, result.returncode) == ("failed", 7)
assert result.stdout == output[:16] + output[-16:]
assert result.stdout_head_bytes == 16
assert result.omitted_stdout_bytes == len(output) - 32
assert result.snapshot().final
assert "FINAL ERROR" in result.snapshot().text()
assert str(len(output) - 32) + " output bytes omitted" in result.snapshot().text()
@pytest.mark.parametrize("limit", [0, 1, 2, 3, 31, 1024])
def test_rolling_buffers_remain_bounded_for_every_chunk(limit):
buffer = _OutputBuffer(limit, "head_tail")
original = b""
for data in (b"a", b"bcdef", b"x" * 65536, b"last error"):
original += data
buffer.append(data)
assert len(buffer.head) + len(buffer.tail) <= limit
assert buffer.omitted == max(0, len(original) - limit)
if len(original) > limit:
head = limit // 2
tail = limit - head
expected = original[:head] + (original[-tail:] if tail else b"")
assert buffer.value() == expected
def test_prefix_capture_remains_exact_and_counts_omitted_bytes():
result = python(
"import os; os.write(1,b'0123456789')", max_stdout=4, terminate_on_limit=False
)
assert result.stdout == b"0123"
assert result.stdout_head_bytes == 4
assert result.omitted_stdout_bytes == 6
def test_callback_exposes_bounded_immutable_initial_and_final_snapshots():
snapshots = []
result = python(
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
"os.write(1,b'x'*10000+b'FINAL\\n'); time.sleep(.1)",
max_stdout=32,
capture_mode="head_tail",
terminate_on_limit=False,
on_output=snapshots.append,
)
assert not snapshots[0].final
assert snapshots[-1] == result.snapshot()
assert snapshots[0].stdout == b"first\n"
assert snapshots[-1].stdout.endswith(b"FINAL\n")
assert all(len(item.stdout) <= 32 for item in snapshots)
with pytest.raises(FrozenInstanceError):
snapshots[0].final = True
def test_callback_updates_dirty_output_while_child_becomes_quiet():
snapshots = []
python(
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
"os.write(1,b'second\\n'); time.sleep(1.15)",
on_output=lambda item: snapshots.append((time.monotonic(), item)),
)
assert len(snapshots) >= 3
assert snapshots[0][1].stdout == b"first\n"
assert not snapshots[1][1].final
assert snapshots[1][1].stdout.endswith(b"second\n")
assert snapshots[1][0] - snapshots[0][0] >= 0.95
assert snapshots[-1][1].final
def test_quiet_process_has_one_final_empty_snapshot():
snapshots = []
python("pass", on_output=snapshots.append)
assert len(snapshots) == 1
assert snapshots[0].final and snapshots[0].stdout == b""
def test_snapshot_text_handles_cut_and_invalid_utf8_without_malformed_strings():
result = python(
"import os; os.write(1, 'Ä😊Z'.encode()*20+b'\\xff\\xfe')",
max_stdout=9,
capture_mode="head_tail",
terminate_on_limit=False,
)
text = result.snapshot().text()
text.encode("utf-8", errors="strict")
assert "output bytes omitted" in text
assert len(result.stdout) == 9
with pytest.raises(ValueError, match="stream"):
result.snapshot().text("other")
def test_callback_failure_terminates_owned_process_and_propagates():
from govoplan_devkit import process
original = process.subprocess.Popen
spawned = []
def capture(*args, **kwargs):
child = original(*args, **kwargs)
spawned.append(child)
return child
def fail(_snapshot):
raise ValueError("fixture callback failed")
with patch.object(process.subprocess, "Popen", side_effect=capture):
with pytest.raises(ValueError, match="fixture callback failed"):
python(
"import time; print('ready',flush=True); time.sleep(30)", on_output=fail
)
assert len(spawned) == 1 and spawned[0].poll() is not None
@pytest.mark.parametrize(
"kwargs", [{"capture_mode": "other"}, {"max_stdout": -1}, {"max_stderr": True}]
)
def test_invalid_capture_configuration_fails_before_starting_a_process(kwargs):
from govoplan_devkit import process
with patch.object(process.subprocess, "Popen") as popen:
with pytest.raises(ValueError):
python("pass", **kwargs)
popen.assert_not_called()
+524
View File
@@ -0,0 +1,524 @@
"""Fixture-only devkit coverage of the real durable release API/store boundary."""
from __future__ import annotations
import argparse
from copy import deepcopy
import json
from pathlib import Path
import socket
import subprocess
import sys
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
ROOT = Path(__file__).resolve().parents[1]
for path in (ROOT / "tools/devkit", ROOT / "tools/release"):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
from govoplan_devkit import release # noqa: E402
from govoplan_release.release_execution import ReleaseExecutionAmbiguous, ReleaseExecutionBlocked # noqa: E402
from govoplan_release.release_run import ReleaseRunCorrupt, ReleaseRunStore # noqa: E402
from govoplan_release.candidate_artifact import ( # noqa: E402
candidate_output_path, harden_private_candidate_tree, issue_candidate_receipt,
)
from server import app as api # noqa: E402
def receipt(repo="govoplan-core", *, target_tag="v1.2.3"):
return {
"kind": "repository_state", "repo": repo, "head": "a" * 40,
"branch": "main", "remote": "origin", "remote_sha256": "b" * 64,
"worktree_clean": True, "target_tag": target_tag, "tag_object": None,
}
def plan(*, catalog=False):
steps = [
("core:preflight", "govoplan-core", False),
("core:tag", "govoplan-core", True),
] if not catalog else [
("catalog:selective-generator", None, True),
("catalog:validate-sign-publish", None, True),
]
return {
"generated_at": "2026-09-08T00:00:00Z", "target_channel": "stable",
"status": "attention", "units": [{"repo": "govoplan-core", "target_version": "1.2.3"}],
"compatibility": [], "gate_findings": [], "recommended_action": {},
"source_preflight_ready": True, "notes": [],
"dry_run_steps": [
{"id": identity, "repo": repo, "mutating": mutating, "title": identity,
"detail": "Fixture-only operation.", "command": "fixture never executed",
"cwd": "/fixture", "status": "planned"}
for identity, repo, mutating in steps
],
}
def namespace(workspace, state, *arguments):
parser = argparse.ArgumentParser()
parser.set_defaults(workspace_root=workspace, state_dir=state, format="json", project=None)
release.register(parser.add_subparsers(required=True))
return parser.parse_args(["release", *arguments])
@pytest.fixture
def environment(tmp_path, monkeypatch):
workspace = tmp_path / "workspace"
workspace.mkdir()
state = tmp_path / "private-state"
plans = [plan()]
dashboard = Mock(return_value={"summary": {"status": "ready"}, "repositories": []})
planner = Mock(side_effect=lambda *args, **kwargs: deepcopy(plans[0]))
execute = Mock(return_value=({"status": "inspected"}, receipt()))
def bind(**kwargs):
result = kwargs["plan"]
for step in result["dry_run_steps"]:
if step.get("repo"):
step["source_binding"] = receipt(step["repo"])
elif step["id"] == "catalog:validate-sign-publish":
step["source_binding"] = receipt("addideas-govoplan-website", target_tag="")
return result
monkeypatch.setattr(api, "build_dashboard", dashboard)
monkeypatch.setattr(api, "build_selective_release_plan", planner)
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock())
monkeypatch.setattr(api, "verify_release_runtime_binding", Mock())
monkeypatch.setattr(api, "bind_plan_source_states", bind)
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(return_value=receipt()))
monkeypatch.setattr(api, "verify_repository_step_precondition", Mock(return_value=receipt()))
monkeypatch.setattr(api, "execute_repository_step", execute)
monkeypatch.setattr(api, "default_signing_keys", lambda: ())
def forbidden(*args, **kwargs):
raise AssertionError("Fixture test attempted a real subprocess or network connection")
monkeypatch.setattr(subprocess, "run", forbidden)
monkeypatch.setattr(socket, "create_connection", forbidden)
def call(*args, state_dir=state, workspace_root=workspace):
values = namespace(workspace_root, state_dir, *args)
return values.handler(values)
def create(request="devkit-create-request-0001"):
result = call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", request, "--apply")
assert result["_exit_code"] == 0, result
return result
return SimpleNamespace(
workspace=workspace, state=state, plans=plans, dashboard=dashboard,
planner=planner, executor=execute, call=call, create=create,
)
def test_registration_does_not_import_heavy_dependencies():
script = """
import argparse, builtins, sys
sys.path.insert(0, sys.argv[1])
original = builtins.__import__
def guarded(name, *args, **kwargs):
if name.split('.')[0] in {'httpx', 'fastapi', 'govoplan_core', 'govoplan_release'}:
raise AssertionError('Heavy import during help registration: ' + name)
return original(name, *args, **kwargs)
builtins.__import__ = guarded
from govoplan_devkit.release import register
parser = argparse.ArgumentParser()
register(parser.add_subparsers())
assert 'release' in parser.format_help()
"""
result = subprocess.run([sys.executable, "-c", script, str(ROOT / "tools/devkit")], capture_output=True, text=True, check=False)
assert result.returncode == 0, result.stderr
def test_plan_is_selective_offline_and_does_not_create_run(environment):
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
assert result["_exit_code"] == 0
assert result["dry_run"] is True
assert environment.planner.call_args.kwargs["selected_repos"] == ("govoplan-core",)
arguments = environment.dashboard.call_args.kwargs
assert arguments["online"] is False
assert arguments["check_remote_tags"] is False
assert arguments["check_public_catalog"] is False
assert arguments["include_migrations"] is False
assert not list(environment.state.rglob("rr-*.json"))
assert "release-console/workspace-" in result["state_location"]
environment.executor.assert_not_called()
def test_status_reports_blocked_exit_and_explicit_check_flags(environment):
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
result = environment.call("status", "--online", "--include-migrations", "--include-website")
assert result["_exit_code"] == 1
assert environment.dashboard.call_args.kwargs["check_remote_tags"] is True
assert environment.dashboard.call_args.kwargs["check_public_catalog"] is True
assert environment.dashboard.call_args.kwargs["include_migrations"] is True
def test_plan_summary_exposes_attention_readiness_gates_and_next_action(environment):
fixture_plan = environment.plans[0]
fixture_plan["source_preflight_ready"] = False
fixture_plan["units"][0]["status"] = "attention"
fixture_plan["gate_findings"] = [{"code": "worktree_dirty", "repo": "govoplan-core", "message": "Uncommitted source changes need review."}]
fixture_plan["recommended_action"] = {"id": "prepare_changes", "title": "Prepare source changes", "remediation": "Review and commit the selected changes before preparing the release."}
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
summary = "\n".join(result["summary"])
assert "Release plan: attention." in summary
assert "Source preflight ready: false." in summary
assert "govoplan-core: attention; target 1.2.3." in summary
assert "Gate worktree_dirty (govoplan-core)" in summary
assert "Next: prepare_changes" in summary
assert "no run was created" in summary
assert "fixture never executed" not in summary
def test_summary_is_bounded_and_redacts_display_only_fields(monkeypatch):
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "fixture-value-not-for-display")
fixture_plan = plan()
fixture_plan["gate_findings"] = [{"code": "fixture", "message": "fixture-value-not-for-display " + "x" * 1000}] * 20
fixture_plan["units"] = [{"repo": f"repo-{index}", "target_version": "1.2.3", "status": "ready"} for index in range(50)]
summary = "\n".join(release._summary_lines("plan", fixture_plan))
assert "38 more selected repositories" in summary
assert "16 more gate findings" in summary
assert "fixture-value-not-for-display" not in summary
assert "[redacted]" in summary
assert len(summary) < 3000
def test_run_and_preview_summary_exposes_steps_and_next_action(environment):
run = environment.create()["result"]
shown = environment.call("show", run["run_id"])
summary = "\n".join(shown["summary"])
assert "Steps: 2 pending." in summary
assert "Step core:preflight: pending." in summary
assert "Next: execute_step [core:preflight]" in summary
preview = environment.call("preview", run["run_id"], "core:tag")
assert "Release preview: pending." in preview["summary"]
assert any("Complete core:preflight first." in line for line in preview["summary"])
def test_status_summary_exposes_repository_counts(environment):
environment.dashboard.return_value = {"summary": {"status": "attention", "repository_count": 77, "dirty_count": 42, "ahead_count": 1, "behind_count": 0, "error_count": 0}}
result = environment.call("status")
assert "Release status: attention." in result["summary"]
assert any("77 repositories, 42 dirty, 1 ahead, 0 behind, 0 errors" in line for line in result["summary"])
def test_display_redaction_cannot_change_semantic_failure_exit(environment, monkeypatch):
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "blocked")
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
result = environment.call("status")
assert result["_exit_code"] == 1
assert "Release status: [redacted]." in result["summary"]
def test_create_defaults_to_preview_without_run_record(environment):
result = environment.call("create", "--repo", "govoplan-core", "--target-version", "1.2.3", "--request-id", "preview-create-request-0001")
assert result["_exit_code"] == 0 and result["dry_run"]
assert not list(environment.state.rglob("rr-*.json"))
environment.executor.assert_not_called()
def test_create_show_and_same_id_replay_use_durable_service(environment):
created = environment.create()
run_id = created["result"]["run_id"]
repeated = environment.create()
assert repeated["result"]["run_id"] == run_id
environment.planner.assert_called_once()
shown = environment.call("show", run_id)
assert shown["result"]["immutable"] == created["result"]["immutable"]
assert shown["result"]["state"]["steps"][0]["executor"]["confirmation"] == ""
assert len(list(environment.state.rglob("rr-*.json"))) == 1
def test_create_replay_rejects_changed_inputs(environment):
environment.create()
result = environment.call("create", "--repo-version", "govoplan-core=1.2.4", "--request-id", "devkit-create-request-0001", "--apply")
assert result["http_status"] == 409
environment.planner.assert_called_once()
@pytest.mark.parametrize("arguments", [
("plan",),
("create", "--repo", "govoplan-core", "--request-id", "missing-version-request"),
("plan", "--repo-version", "govoplan-core=1.2.3", "--repo-version", "govoplan-core=1.2.4"),
("plan", "--repo-version", "govoplan-core=not-a-version"),
])
def test_invalid_selection_is_rejected_before_service_collection(environment, arguments):
result = environment.call(*arguments)
assert result["_exit_code"] == 2
environment.dashboard.assert_not_called()
environment.executor.assert_not_called()
def test_runtime_trust_guard_cannot_be_bypassed_by_cli(environment, monkeypatch):
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock(side_effect=ReleaseExecutionBlocked("Fixture untrusted runtime")))
result = environment.call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", "trust-create-request-0001", "--apply")
assert result["http_status"] == 409
assert not list(environment.state.rglob("rr-*.json"))
environment.planner.assert_not_called()
def test_dry_execute_and_generic_preview_never_claim_or_execute(environment):
run = environment.create()["result"]
dry = environment.call("execute", run["run_id"], "core:preflight", "--request-id", "dry-execute-request-0001")
preview = environment.call("preview", run["run_id"], "core:tag")
assert dry["dry_run"] and preview["dry_run"]
assert preview["result"]["state_step"]["executor"]["confirmation"] == "TAG"
assert preview["result"]["state_step"]["available"] is False
environment.executor.assert_not_called()
assert environment.call("show", run["run_id"])["result"]["state"]["steps"][0]["attempt_count"] == 0
def test_prerequisite_and_confirmation_guards_remain_enforced(environment):
run_id = environment.create()["result"]["run_id"]
blocked = environment.call("execute", run_id, "core:tag", "--request-id", "ordered-tag-request-0001", "--confirm", "TAG", "--apply")
assert blocked["http_status"] == 409
environment.executor.assert_not_called()
assert environment.call("execute", run_id, "core:preflight", "--request-id", "preflight-request-0001", "--apply")["_exit_code"] == 0
missing = environment.call("execute", run_id, "core:tag", "--request-id", "confirmed-tag-request-0001", "--apply")
assert missing["http_status"] == 409
assert environment.executor.call_count == 1
def test_execute_same_attempt_replays_without_repeating_effect(environment):
run_id = environment.create()["result"]["run_id"]
arguments = ("execute", run_id, "core:preflight", "--request-id", "exact-attempt-request-0001", "--apply")
first = environment.call(*arguments)
second = environment.call(*arguments)
assert first["_exit_code"] == second["_exit_code"] == 0
assert second["result"]["execution_result"]["status"] == "replayed"
environment.executor.assert_called_once()
assert environment.executor.call_args.kwargs["remote"] == "origin"
def test_lost_finish_write_is_interrupted_without_reexecuting_effect(environment, monkeypatch):
run_id = environment.create()["result"]["run_id"]
finish = ReleaseRunStore.finish_step
monkeypatch.setattr(ReleaseRunStore, "finish_step", Mock(side_effect=ReleaseRunCorrupt("Fixture durable write failure")))
arguments = ("execute", run_id, "core:preflight", "--request-id", "lost-finish-request-0001", "--apply")
interrupted = environment.call(*arguments)
assert interrupted["http_status"] == 409
monkeypatch.setattr(ReleaseRunStore, "finish_step", finish)
assert environment.call(*arguments)["http_status"] == 409
environment.executor.assert_called_once()
shown = environment.call("show", run_id)["result"]
assert shown["state"]["steps"][0]["state"] == "interrupted"
def test_source_guard_failure_does_not_call_executor(environment, monkeypatch):
run_id = environment.create()["result"]["run_id"]
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(side_effect=ReleaseExecutionBlocked("Frozen HEAD/remote changed")))
result = environment.call("execute", run_id, "core:preflight", "--request-id", "changed-source-request-0001", "--apply")
assert result["_exit_code"] == 1
environment.executor.assert_not_called()
def test_interrupted_write_requires_reconciliation_not_retry(environment, monkeypatch):
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture remote outcome uncertain")
run_id = environment.create()["result"]["run_id"]
execute = ("execute", run_id, "core:tag", "--request-id", "uncertain-tag-request-0001", "--confirm", "TAG", "--apply")
uncertain = environment.call(*execute)
assert uncertain["http_status"] == 409
assert environment.call(*execute)["http_status"] == 409
retry = environment.call("retry", run_id, "core:tag", "--request-id", "unsafe-retry-request-0001", "--apply")
assert retry["http_status"] == 409
environment.executor.assert_called_once()
invalid = environment.call("reconcile", run_id, "core:tag", "--request-id", "bad-reconcile-request-0001", "--outcome", "effect_absent", "--apply")
assert invalid["http_status"] == 409
reconciled = environment.call("reconcile", run_id, "core:tag", "--request-id", "reconcile-absent-request-0001", "--outcome", "effect_absent", "--confirm", "RECONCILE", "--apply")
assert reconciled["_exit_code"] == 0
assert reconciled["result"]["state"]["steps"][0]["state"] == "pending"
def test_effect_succeeded_reconciliation_keeps_independent_receipt_guard(environment, monkeypatch):
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture interruption")
run_id = environment.create()["result"]["run_id"]
environment.call("execute", run_id, "core:tag", "--request-id", "receipt-tag-request-0001", "--confirm", "TAG", "--apply")
guard = Mock(side_effect=ReleaseExecutionBlocked("Remote annotation mismatch"))
monkeypatch.setattr(api, "reconciled_repository_receipt", guard)
result = environment.call("reconcile", run_id, "core:tag", "--request-id", "receipt-success-request-0001", "--outcome", "effect_succeeded", "--confirm", "RECONCILE", "--apply")
assert result["http_status"] == 409
guard.assert_called_once()
environment.executor.assert_called_once()
def test_resume_and_retry_reuse_the_existing_running_attempt_rules(environment):
created = environment.create()
run_id = created["result"]["run_id"]
store = ReleaseRunStore(Path(created["state_location"]), expected_workspace_fingerprint=api.release_workspace_fingerprint(environment.workspace))
store.claim_step(run_id, "core:preflight", attempt_id="lost-process-request-0001")
dry = environment.call("resume", run_id, "--request-id", "resume-process-request-0001")
assert dry["dry_run"]
assert store.get(run_id)["state"]["steps"][0]["state"] == "running"
resumed = environment.call("resume", run_id, "--request-id", "resume-process-request-0001", "--apply")
assert resumed["result"]["state"]["steps"][0]["state"] == "interrupted"
retried = environment.call("retry", run_id, "core:preflight", "--request-id", "retry-readonly-request-0001", "--apply")
assert retried["result"]["state"]["steps"][0]["state"] == "pending"
environment.executor.assert_not_called()
def test_workspace_scoping_and_corrupt_records_fail_closed(environment, tmp_path):
created = environment.create()
run_id = created["result"]["run_id"]
another = tmp_path / "another-workspace"
another.mkdir()
foreign = environment.call("show", run_id, workspace_root=another)
assert foreign["http_status"] == 404
path = Path(created["state_location"]) / f"{run_id}.json"
record = json.loads(path.read_text())
record["immutable"]["input"]["repo_versions"]["govoplan-core"] = "9.9.9"
path.write_text(json.dumps(record))
assert environment.call("show", run_id)["http_status"] == 409
def test_run_list_keeps_cursor_pagination(environment):
environment.create("page-create-request-0001")
environment.create("page-create-request-0002")
first = environment.call("list", "--limit", "1")["result"]
second = environment.call("list", "--limit", "1", "--cursor", first["next_cursor"])["result"]
assert len(first["runs"]) == len(second["runs"]) == 1
assert first["runs"][0]["run_id"] != second["runs"][0]["run_id"]
assert second["next_cursor"] is None
def test_catalog_preview_calls_only_existing_receipt_bound_preview(environment, monkeypatch, tmp_path):
environment.plans[0] = plan(catalog=True)
run_id = environment.create()["result"]["run_id"]
candidate = tmp_path / "candidate"
verify = Mock(return_value=candidate)
publish = Mock(return_value={"status": "planned", "apply": False})
monkeypatch.setattr(api, "verified_run_candidate", verify)
monkeypatch.setattr(api, "publish_catalog_candidate", publish)
result = environment.call("preview", run_id, "catalog:validate-sign-publish")
assert result["_exit_code"] == 0
verify.assert_called_once()
assert publish.call_args.kwargs["apply"] is False
assert publish.call_args.kwargs["candidate_dir"] == candidate
assert publish.call_args.kwargs["remote"] == "origin"
environment.executor.assert_not_called()
def test_catalog_generation_and_publication_use_exact_durable_receipts(environment, monkeypatch):
environment.plans[0] = plan(catalog=True)
created = environment.create()
run_id = created["result"]["run_id"]
candidate_root = Path(created["candidate_location"])
def generated(**kwargs):
candidate_id = kwargs["candidate_id"]
candidate = candidate_output_path(candidate_root, candidate_id)
channels = candidate / "channels"
channels.mkdir(parents=True)
(channels / "stable.json").write_text(json.dumps({"channel": "stable", "signatures": [{}]}))
harden_private_candidate_tree(candidate)
return {"status": "ready"}, issue_candidate_receipt(root=candidate_root, candidate_id=candidate_id, channel="stable")
def published(**kwargs):
candidate = kwargs["candidate_receipt"]
website = kwargs["expected_website_receipt"]
return {"status": "published"}, {
"kind": "catalog_publication", "candidate_id": candidate["candidate_id"],
"catalog_sha256": candidate["catalog_sha256"], "keyring_sha256": "c" * 64,
"publication_commit_sha": "d" * 40, "publication_tag_object_sha": "e" * 40,
"publication_tag_commit_sha": "d" * 40, "branch": website["branch"],
"tag_name": "catalog-stable-1", "remote": "origin", "remote_sha256": website["remote_sha256"],
}
generator = Mock(side_effect=generated)
publisher = Mock(side_effect=published)
website = receipt("addideas-govoplan-website", target_tag="")
monkeypatch.setattr(api, "generate_catalog_candidate", generator)
monkeypatch.setattr(api, "publish_received_candidate", publisher)
monkeypatch.setattr(api, "verify_catalog_publication_precondition", Mock(return_value=website))
generate = ("execute", run_id, "catalog:selective-generator", "--request-id", "generate-candidate-request-0001", "--confirm", "GENERATE", "--apply")
first = environment.call(*generate, "--signing-key", "fixture-key=/private/fixture-key.pem")
assert first["_exit_code"] == 0, first
replayed = environment.call(*generate)
assert replayed["result"]["execution_result"]["status"] == "replayed"
generator.assert_called_once()
assert generator.call_args.kwargs["signing_keys"] == ("fixture-key=/private/fixture-key.pem",)
assert "/private/fixture-key.pem" not in json.dumps(first)
record_text = next(environment.state.rglob("rr-*.json")).read_text()
assert "/private/fixture-key.pem" not in record_text
missing_confirmation = environment.call("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--apply")
assert missing_confirmation["http_status"] == 409
publisher.assert_not_called()
publish = ("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--confirm", "PUSH", "--apply")
complete = environment.call(*publish)
assert complete["_exit_code"] == 0, complete
assert complete["result"]["state"]["status"] == "completed"
frozen = first["result"]["state"]["steps"][0]["result_receipt"]
assert publisher.call_args.kwargs["candidate_path"] == candidate_root / frozen["candidate_id"]
assert publisher.call_args.kwargs["candidate_receipt"] == frozen
assert publisher.call_args.kwargs["expected_website_receipt"] == website
assert publisher.call_args.kwargs["remote"] == "origin"
assert environment.call(*publish)["result"]["execution_result"]["status"] == "replayed"
publisher.assert_called_once()
def test_portable_project_cannot_override_release_catalog(environment):
args = namespace(environment.workspace, environment.state, "plan", "--repo-version", "govoplan-core=1.2.3")
args.project = environment.workspace / "custom-project.json"
result = release.handle(args)
assert result["_exit_code"] == 2
assert "does not accept --project" in result["summary"][0]
environment.dashboard.assert_not_called()
def test_foreign_cached_release_package_is_rejected_before_import(environment, monkeypatch, tmp_path):
monkeypatch.setitem(sys.modules, "server.app", SimpleNamespace(__file__=str(tmp_path / "foreign/app.py")))
importer = Mock(side_effect=AssertionError("Foreign module import must not occur"))
monkeypatch.setattr(release.importlib, "import_module", importer)
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
assert result["_exit_code"] == 2
assert "foreign module" in result["summary"][0]
importer.assert_not_called()
environment.dashboard.assert_not_called()
def test_no_arbitrary_remote_or_legacy_publication_flags():
run_id = "rr-request-" + "a" * 64
with pytest.raises(SystemExit):
namespace(Path("/fixture"), None, "execute", run_id, "core:tag", "--request-id", "arbitrary-remote-request", "--remote", "untrusted")
with pytest.raises(SystemExit):
namespace(Path("/fixture"), None, "publish-candidate", "--candidate-dir", "/untrusted")
def test_unknown_step_and_malformed_paths_do_not_become_other_routes(environment):
run_id = environment.create()["result"]["run_id"]
assert environment.call("preview", run_id, "unknown:step")["http_status"] == 404
with pytest.raises(SystemExit):
namespace(environment.workspace, environment.state, "preview", run_id, "../repositories/push")
def test_signing_material_is_not_echoed_by_validation(environment):
run_id = environment.create()["result"]["run_id"]
secret = "PRIVATE-KEY-MATERIAL\nDO-NOT-ECHO"
result = environment.call("execute", run_id, "core:preflight", "--request-id", "secret-input-request-0001", "--signing-key", secret, "--apply")
assert result["_exit_code"] == 2
assert secret not in json.dumps(result)
assert "SECRET" not in release._error_detail({"detail": [{"loc": ["body", "signing_keys"], "msg": "Invalid input", "input": "SECRET"}]})
environment.executor.assert_not_called()
def test_default_state_matches_console_and_token_never_appears_in_output(environment, monkeypatch, tmp_path):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
monkeypatch.setattr(release.secrets, "token_urlsafe", lambda size: "ephemeral-do-not-print-token")
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3", state_dir=None)
assert result["state_location"] == str(api.default_release_run_root(environment.workspace))
assert "ephemeral-do-not-print-token" not in json.dumps(result)
def test_documentation_keeps_disabled_generic_mutations_and_recovery_explicit():
console = (ROOT / "docs/operations/RELEASE_CONSOLE.md").read_text()
usage = (ROOT / "docs/operations/DEVKIT_RELEASE.md").read_text()
assert "generic push, sync and prepare\nmutation endpoints are disabled" in console
assert "ASGI application inside" in usage
assert "effect_absent" in usage and "effect_succeeded" in usage and "unresolved" in usage
assert "does **not** stage arbitrary source" in usage
+175
View File
@@ -0,0 +1,175 @@
import argparse
import json
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import review
from govoplan_devkit.cli import main
@pytest.fixture
def args(tmp_path):
repo = tmp_path / "optional-feature"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
paths = ["webui/src/pages/CampaignPage.tsx", "webui/src/dialogs/SettingsDialog.tsx",
"webui/src/settings/AdminSettings.tsx", "webui/src/widgets/SummaryWidget.tsx", "webui/src/module.ts"]
for relative in paths:
path = repo / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("// fixture only\n")
manifest = repo / "src/optional_feature/backend/manifest.py"
manifest.parent.mkdir(parents=True)
manifest.write_text("raise RuntimeError('MUST NEVER IMPORT MODULE CODE')\n")
(tmp_path / "principles.md").write_text("# Principles\nRevision **UI-2026-09-08**\n\n## UI-01 — Help beside text\n\n## UI-02 — Display first; edit deliberately\n")
inventory = {"schema_version": 1, "epic": {"repository": "meta", "number": 4, "url": "https://gitea.invalid/team/meta/issues/4"},
"issues": [{"scope_id": "campaigns", "name": "Campaign", "kind": "manifest", "repository": "optional-feature", "number": 8,
"url": "https://gitea.invalid/team/optional-feature/issues/8", "state_at_verification": "closed", "operation": "created"}]}
(tmp_path / "issues.json").write_text(json.dumps(inventory))
project = tmp_path / "project.json"
project.write_text(json.dumps({"schema_version": 1, "name": "Portable project", "repositories": [
{"name": "optional-feature", "path": "optional-feature", "aliases": ["campaign"]},
{"name": "absent-optional", "path": "absent-optional", "aliases": ["absent"]}],
"review": {"issue_inventory": "issues.json", "principles": "principles.md"},
"checks": [{"id": "ui-check", "argv": ["never-execute-this"], "repos": ["optional-feature"], "cwd": "optional-feature"}],
"profiles": {"ui": ["ui-check"]}}))
return argparse.Namespace(workspace_root=tmp_path, project=project, state_dir=tmp_path / "state",
module="campaign", bundle_module=None, profile="ui", evidence=None, output=None)
def test_bundle_contains_guidance_links_revision_and_unexecuted_check_plan(args):
result = review.handle_review(args)
assert result["module"] == "optional-feature"
assert result["inventory"]["ui_source_count"] == 5
assert not result["inventory"]["module_code_imported"]
assert result["inventory"]["manifest_paths"] == ["src/optional_feature/backend/manifest.py"]
assert result["principles"]["revision"] == "UI-2026-09-08"
assert [rule["id"] for rule in result["principles"]["rules"]] == ["UI-01", "UI-02"]
assert result["check_plan"]["stages"][0]["argv"] == ["never-execute-this"]
assert not result["check_plan"]["executed"]
assert result["review_completion"].startswith("Not assessed")
assert len(result["manual_checklist"]) >= 9
assert all("checked" not in item for item in result["manual_checklist"])
assert "state_at_verification" not in json.dumps(result)
assert "operation" not in result["issue_links"][0]
assert result["central_issue"]["number"] == 4
assert not args.state_dir.exists()
@pytest.mark.parametrize("name", ["campaign", "campaigns", "optional-feature"])
def test_alias_scope_id_and_repository_resolve_to_same_registered_module(args, name):
args.module = name
assert review.handle_review(args)["module"] == "optional-feature"
def test_bundle_alias_and_direct_cli_work_with_global_options_anywhere(args, capsys):
assert main(["--workspace-root", str(args.workspace_root), "review", "campaign", "--project", str(args.project), "--json"]) == 0
direct = json.loads(capsys.readouterr().out)
assert direct["module"] == "optional-feature"
assert main(["review", "bundle", "campaigns", "--workspace-root", str(args.workspace_root), "--project", str(args.project), "--format", "json"]) == 0
assert json.loads(capsys.readouterr().out)["module"] == direct["module"]
def test_absent_optional_repository_and_no_ui_do_not_complete_review(args):
args.module = "absent"
result = review.handle_review(args)
assert not result["repository"]["exists"]
assert result["repository"]["errors"]
assert result["inventory"]["ui_source_count"] == 0
assert result["check_plan"]["stages"] == []
assert result["review_completion"].startswith("Not assessed")
assert any("not automatic N/A" in warning for warning in result["warnings"])
assert any("not a passing" in warning for warning in result["warnings"])
def test_portable_project_does_not_inherit_govoplan_links_or_principles(args):
payload = json.loads(args.project.read_text())
payload.pop("review")
args.project.write_text(json.dumps(payload))
result = review.handle_review(args)
assert result["issue_links"] == [] and result["central_issue"] is None
assert not result["principles"]["available"]
assert "git.add-ideas.de" not in json.dumps(result)
@pytest.mark.parametrize("kind", ["symlink", "outside", "bad-url", "duplicate-scope", "foreign-repository"])
def test_review_inputs_cannot_escape_or_silently_misbind_scope(args, kind):
if kind in {"symlink", "outside"}:
payload = json.loads(args.project.read_text())
if kind == "symlink":
alias = args.workspace_root / "alias.json"
alias.symlink_to(args.workspace_root / "issues.json")
payload["review"]["issue_inventory"] = "alias.json"
else:
payload["review"]["principles"] = "../outside.md"
args.project.write_text(json.dumps(payload))
else:
path = args.workspace_root / "issues.json"
payload = json.loads(path.read_text())
if kind == "bad-url":
payload["issues"][0]["url"] = "https://gitea.invalid/team/wrong/issues/8"
elif kind == "duplicate-scope":
payload["issues"].append(payload["issues"][0])
else:
payload["issues"][0].update(repository="foreign", url="https://gitea.invalid/team/foreign/issues/8")
args.module = "campaigns"
path.write_text(json.dumps(payload))
with pytest.raises(ValueError):
review.handle_review(args)
def test_source_inventory_never_follows_symlinked_ancestors(args):
root = args.workspace_root / "indirect"
root.mkdir()
(root / "webui").symlink_to(args.workspace_root / "optional-feature/webui", target_is_directory=True)
result = review.source_inventory(root)
assert result["ui_source_count"] == 0
assert result["skipped_symlinks"] == ["webui/src"]
def test_explicit_artifact_is_only_local_and_preserves_issue_discovery_snapshot(args):
inventory = args.workspace_root / "issues.json"
original = inventory.read_bytes()
args.output = args.workspace_root / "artifacts/review.json"
result = review.handle_review(args)
assert json.loads(args.output.read_text()) == result
assert inventory.read_bytes() == original
assert args.output.stat().st_mode & 0o777 == 0o600
assert not args.state_dir.exists()
def test_attached_historical_receipt_stays_separate_from_current_review(args):
path = args.workspace_root / "historical.json"
path.write_text(json.dumps({"schema_version": 1, "run_id": "historical", "workspace_root": str(args.workspace_root),
"project_file": str(args.project), "source_fingerprint": "a" * 64, "status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08",
"finished_at": "2026-09-08", "stages": [{"id": "old-check", "status": "passed", "exit_code": 0, "duration_seconds": 1, "log_path": "/not-read"}]}))
args.evidence = str(path)
result = review.handle_review(args)
assert result["evidence"]["origin"] == "external-unverified"
assert result["evidence"]["source_state"] == "historical-source-differs"
assert result["review_completion"].startswith("Not assessed")
assert not result["check_plan"]["executed"]
def test_unknown_module_and_extra_positionals_are_errors(args):
args.module = "typo"
with pytest.raises(ValueError, match="Unknown repository"):
review.handle_review(args)
args.module, args.bundle_module = "campaign", "extra"
with pytest.raises(ValueError, match="one module"):
review.handle_review(args)
def test_compact_review_exposes_scoped_coverage_limits_without_hiding_full_list(args, monkeypatch):
notes = [f"Separate suite {number} was not included." for number in range(12)]
monkeypatch.setattr("govoplan_devkit.catalog.build_stages", lambda *_args, **_kwargs: [
{"id": "source-only", "argv": ["never-execute"], "coverage_notes": notes}])
result = review.handle_review(args)
assert result["coverage_notes"] == notes
assert sum(line.startswith("Coverage limitation:") for line in result["summary"]) == 8
assert any("4 additional coverage limitations" in line for line in result["summary"])
assert not result["check_plan"]["executed"]
+466
View File
@@ -0,0 +1,466 @@
"""Real fixture processes and Git worktrees; never invoke product/remote mutations."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import subprocess
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import (
atomic_json,
digest,
read_json,
resource_lock,
state_root,
)
from govoplan_devkit.workspace import load_project, source_fingerprint
@pytest.fixture
def example(tmp_path, monkeypatch):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
workspace = tmp_path / "workspace"
repo = workspace / "example"
repo.mkdir(parents=True)
subprocess.run(["git", "init", "-q", str(repo)], check=True)
(repo / "source.txt").write_text("first\n")
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
)
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Example",
"repositories": [{"name": "example", "path": "example"}],
"checks": [],
"profiles": {},
}
)
)
args = Namespace(
workspace_root=workspace,
project=project,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="quick",
resume=None,
)
return args, repo
def stage(repo, name="one", code="print('ok')", **extra):
return {
"id": name,
"title": name,
"argv": [sys.executable, "-c", code],
"cwd": str(repo),
"timeout_seconds": 10,
**extra,
}
def run(args, stages):
# Environment inspection is tested separately; keep fixture executions cheap/deterministic.
with patch.object(runner, "environment_fingerprint", return_value="fixture-env"):
return runner.run_checks(args, stages)
def test_success_receipt_and_compact_summary(example):
args, repo = example
result = run(args, [stage(repo)])
assert result["status"] == "passed"
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["stages"][0]["exit_code"] == 0
assert Path(receipt["stages"][0]["log_path"]).read_text() == "ok\n"
assert runner.summarize(receipt)["counts"] == {"passed": 1}
assert Path(result["receipt_path"]).stat().st_mode & 0o777 == 0o600
def test_native_environment_binds_selected_checkout(tmp_path, monkeypatch):
from govoplan_devkit.environment import execution_environment
monkeypatch.setenv("GOVOPLAN_WORKSPACE_ROOT", "/another/workspace")
monkeypatch.setenv("GOVOPLAN_CORE_ROOT", "/another/core")
monkeypatch.setenv("GOVOPLAN_CORE_SOURCE_ROOT", "/another/source")
project = load_project(tmp_path)
env = execution_environment(
tmp_path, project, {"python": sys.executable, "node": "node", "npm": "npm"}
)
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(tmp_path)
assert env["GOVOPLAN_CORE_ROOT"] == str(tmp_path / "govoplan-core")
assert env["GOVOPLAN_CORE_SOURCE_ROOT"] == str(tmp_path / "govoplan-core")
def test_portable_environment_does_not_inject_govoplan_scope(example, monkeypatch):
from govoplan_devkit.environment import execution_environment
args, _ = example
for key in (
"GOVOPLAN_WORKSPACE_ROOT",
"GOVOPLAN_CORE_ROOT",
"GOVOPLAN_CORE_SOURCE_ROOT",
):
monkeypatch.delenv(key, raising=False)
env = execution_environment(
args.workspace_root,
load_project(args.workspace_root, args.project),
{"python": sys.executable, "node": "node", "npm": "npm"},
)
assert "GOVOPLAN_WORKSPACE_ROOT" not in env
def test_failure_skips_dependents_but_runs_independent_check(example):
args, repo = example
result = run(
args,
[
stage(repo, "bad", "raise SystemExit(4)"),
stage(repo, "dependent", deps=["bad"]),
stage(repo, "independent"),
],
)
states = {item["id"]: item["status"] for item in result["stages"]}
assert states == {"bad": "failed", "dependent": "skipped", "independent": "passed"}
assert result["_exit_code"] == 1
def test_timeout_is_not_a_pass(example):
args, repo = example
result = run(
args, [stage(repo, code="import time; time.sleep(10)", timeout_seconds=0.1)]
)
assert result["stages"][0]["status"] == "timed_out"
def test_dry_run_does_not_create_state(example):
args, repo = example
args.dry_run = True
result = run(args, [stage(repo, code="raise SystemExit(9)")])
assert result["status"] == "planned"
assert not args.state_dir.exists()
def test_empty_plan_is_not_claimed_as_verified(example):
args, _ = example
assert run(args, [])["status"] == "not_run"
def test_plan_validation_rejects_cycles_unknowns_duplicate_ids_escape(example):
args, repo = example
invalid = [
[stage(repo, deps=["unknown"])],
[stage(repo, "a", deps=["b"]), stage(repo, "b", deps=["a"])],
[stage(repo), stage(repo)],
[stage(repo, "../escape")],
[stage(repo.parent.parent)],
]
for plan in invalid:
with pytest.raises(ValueError):
runner.validate_stages(plan, args.workspace_root, {})
def test_source_identity_includes_staged_unstaged_and_untracked_bytes(example):
args, repo = example
project = load_project(args.workspace_root, args.project)
first = source_fingerprint(project)
(repo / "source.txt").write_text("second\n")
second = source_fingerprint(project)
assert second != first
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
third = source_fingerprint(project)
assert third != second
(repo / "extra.txt").write_text("extra")
assert source_fingerprint(project) != third
def test_source_mutation_during_run_invalidates_receipt(example):
args, repo = example
result = run(
args,
[
stage(
repo,
code="from pathlib import Path; Path('source.txt').write_text('changed')",
)
],
)
assert result["stages"][0]["status"] == "stale"
assert result["stages"][0]["checkpoint_verified"] is False
assert result["status"] == "stale"
def test_resume_reuses_only_identical_plan_and_inputs(example):
args, repo = example
plan = [stage(repo)]
first = run(args, plan)
args.resume = first["run_id"]
second = run(args, plan)
assert second["stages"][0]["reused_from"] == first["run_id"]
changed_plan = run(args, [stage(repo, code="print('different')")])
assert changed_plan["status"] == "passed"
assert "reused_from" not in changed_plan["stages"][0]
assert Path(changed_plan["stages"][0]["log_path"]).read_text() == "different\n"
(repo / "source.txt").write_text("changed")
changed_source = run(args, plan)
assert changed_source["status"] == "passed"
assert "reused_from" not in changed_source["stages"][0]
assert changed_source["stages"][0]["cache_key"] != first["stages"][0]["cache_key"]
def test_receipt_integrity_and_foreign_id_are_rejected(example):
args, repo = example
result = run(args, [stage(repo)])
path = Path(result["receipt_path"])
record = read_json(path)
record["status"] = "fabricated"
atomic_json(path, record)
with pytest.raises(ValueError, match="integrity"):
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
with pytest.raises(ValueError):
runner.read_receipt(args.workspace_root, args.state_dir, "../other")
def test_shared_resources_serialize_stages(example):
args, repo = example
active = 0
maximum = 0
lock = threading.Lock()
original = runner._execute_stage_command
def execute(*args, **kwargs):
nonlocal active, maximum
with lock:
active += 1
maximum = max(maximum, active)
try:
time.sleep(0.04)
return original(*args, **kwargs)
finally:
with lock:
active -= 1
with patch.object(runner, "_execute_stage_command", side_effect=execute):
result = run(
args,
[
stage(repo, "a", resources=["shared"]),
stage(repo, "b", resources=["shared"]),
],
)
assert result["status"] == "passed"
assert maximum == 1
def test_cross_process_resource_conflict_is_explicit(example):
args, repo = example
locks = state_root(args.workspace_root) / "resource-locks"
with resource_lock(locks, "shared"):
result = run(args, [stage(repo, resources=["shared"])])
assert result["stages"][0]["status"] == "blocked"
def test_output_is_bounded_and_known_secrets_redacted(example, monkeypatch):
args, repo = example
monkeypatch.setenv("FIXTURE_SECRET", "private-fixture-credential")
with patch.object(runner, "MAX_LOG_BYTES", 1024):
result = run(
args,
[
stage(
repo,
code="import os; print(os.environ['FIXTURE_SECRET']); print('x'*10000)",
)
],
)
record = result["stages"][0]
output = Path(record["log_path"]).read_text()
assert record["output_truncated"] is True
assert "private-fixture-credential" not in output
assert "[redacted]" in output
assert len(output) < 2000
def test_symlinked_state_is_rejected(example, tmp_path):
args, repo = example
target = tmp_path / "target"
target.mkdir()
args.state_dir.symlink_to(target, target_is_directory=True)
with pytest.raises(ValueError, match="symlink"):
run(args, [stage(repo)])
@pytest.mark.parametrize("flag", ["--assume-unchanged", "--skip-worktree"])
def test_hidden_tracked_edits_invalidate_source_identity(example, flag):
args, repo = example
subprocess.run(
["git", "-C", str(repo), "update-index", flag, "source.txt"], check=True
)
project = load_project(args.workspace_root, args.project)
initial = source_fingerprint(project)
(repo / "source.txt").write_text("hidden edit\n")
assert source_fingerprint(project) != initial
def test_failed_final_snapshot_never_persists_a_passing_receipt(example):
args, repo = example
events = []
args.on_progress = events.append
original = Checkpoints.source
def fingerprint(self, *values, **kwargs):
if events and events[-1]["phase"] == "finalizing":
raise ValueError("unreadable source")
return original(self, *values, **kwargs)
with patch.object(Checkpoints, "source", fingerprint):
result = run(args, [stage(repo)])
assert result["status"] == "stale"
assert result["snapshot_verified"] is False
assert result["_exit_code"] == 1
assert "unreadable source" in result["invalidated_reason"]
assert (
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])[
"status"
]
== "stale"
)
def test_resume_rejects_a_modified_cached_log(example):
args, repo = example
plan = [stage(repo)]
first = run(args, plan)
Path(first["stages"][0]["log_path"]).write_text("altered")
args.resume = first["run_id"]
result = run(args, plan)
assert result["status"] == "failed"
assert result["stages"][0]["status"] == "failed"
assert "log integrity" in result["stages"][0]["error"]
assert "reused_from" not in result["stages"][0]
def test_receipt_argument_redaction_applies_to_unknown_separate_values(example):
args, repo = example
plan = stage(repo)
plan["argv"].extend(["--password", "fixture-not-from-environment"])
result = run(args, [plan])
assert (
"fixture-not-from-environment" not in Path(result["receipt_path"]).read_text()
)
def recovery_parser():
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers())
return parser
def test_recovery_requires_manual_survivor_confirmation(example):
args, repo = example
result = run(args, [stage(repo)])
record = read_json(Path(result["receipt_path"]))
record["status"] = "running"
record["snapshot_verified"] = False
atomic_json(Path(result["receipt_path"]), runner._seal(record))
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply"], namespace=args
)
with pytest.raises(ValueError, match="manual verification"):
parsed.handler(parsed)
def test_recovery_does_not_overwrite_completion_between_reads(example):
args, repo = example
result = run(args, [stage(repo)])
completed = runner.read_receipt(
args.workspace_root, args.state_dir, result["run_id"]
)
initial = {**completed, "status": "running"}
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=args,
)
with (
patch.object(runner, "read_receipt", side_effect=[initial, completed]),
patch.object(runner, "atomic_json") as write,
):
assert parsed.handler(parsed)["status"] == "unchanged"
write.assert_not_called()
def test_active_owner_lock_blocks_recovery(example):
args, repo = example
result = run(args, [stage(repo)])
record = read_json(Path(result["receipt_path"]))
record.update(status="running", snapshot_verified=False)
atomic_json(Path(result["receipt_path"]), runner._seal(record))
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=args,
)
with resource_lock(
state_root(args.workspace_root, args.state_dir) / "locks",
"run:" + result["run_id"],
):
with pytest.raises(RuntimeError, match="busy"):
parsed.handler(parsed)
@pytest.mark.parametrize(
"mutation",
[
"unknown_status",
"unverified_pass",
"duplicate_stage",
"nonboolean_verification",
"nonpassing_stage",
],
)
def test_even_correctly_hashed_receipts_must_have_consistent_states(example, mutation):
args, repo = example
result = run(args, [stage(repo)])
path = Path(result["receipt_path"])
record = read_json(path)
if mutation == "unknown_status":
record["status"] = "wonderful"
elif mutation == "unverified_pass":
record["snapshot_verified"] = False
elif mutation == "duplicate_stage":
record["stages"].append(record["stages"][0])
elif mutation == "nonboolean_verification":
record["snapshot_verified"] = "true"
else:
record["stages"][0]["status"] = "failed"
# Deliberately construct an externally corrupted but correctly hashed record;
# the runner's writer now rejects inconsistent checkpoint state before sealing.
record["integrity_sha256"] = digest(
{key: value for key, value in record.items() if key != "integrity_sha256"}
)
atomic_json(path, record)
with pytest.raises(ValueError):
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
+251
View File
@@ -0,0 +1,251 @@
"""Published-schema parity and all-declaration validation, without commands."""
from copy import deepcopy
import json
from pathlib import Path
import sys
import jsonschema
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/devkit"))
from govoplan_devkit.workspace import load_project # noqa: E402
from govoplan_devkit.validation import _schema_value # noqa: E402
def valid():
return {
"schema_version": 1,
"repositories": [{"name": "example", "path": "example"}],
"checks": [
{"id": "selected", "argv": ["true"]},
{"id": "unselected", "argv": ["true"]},
],
"profiles": {"quick": ["selected"], "full": ["unselected"]},
}
def check(tmp_path, value):
path = tmp_path / "project.json"
path.write_text(json.dumps(value))
return load_project(tmp_path, path)
@pytest.mark.parametrize(
"field,value",
[
("resource", ["shared"]),
("timout_seconds", 20),
("cwd", "../escape"),
("cwd", "/outside"),
("argv", [""]),
("argv", ["true", "x" * 8193]),
("argv", ["true"] * 257),
("argv", ["true", "bad\0value"]),
("resources", ["a", "a"]),
("resources", [""]),
("resources", ["x" * 257]),
("resources", ["bad\nname"]),
("resources", True),
("deps", ["selected", "selected"]),
("repos", ["example", "example"]),
("title", 10),
("title", "x" * 1025),
("timeout_seconds", True),
("timeout_seconds", 43201),
("timeout_seconds", 0),
("id", "invalid\n"),
],
)
def test_unselected_checks_obey_the_published_schema(tmp_path, field, value):
project = valid()
project["checks"][1][field] = value
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"mutation",
[
lambda value: value["profiles"].update(typo=[]),
lambda value: value["profiles"].update(full=["unselected", "unselected"]),
lambda value: value["tools"].update(pyton="python"),
lambda value: value["tools"].update(python="bad\0tool"),
lambda value: value["review"].update(principle="rules.md"),
lambda value: value["review"].update(principles="../rules.md"),
lambda value: value["repositories"][0].update(alias="other"),
lambda value: value["repositories"][0].update(aliases=["alias", "alias"]),
lambda value: value.update(checks=value["checks"] * 257),
lambda value: value.update(schema_version=True),
],
)
def test_all_nested_shape_constraints_match_schema(tmp_path, mutation):
project = valid()
project.update(tools={}, review={})
mutation(project)
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"mutation,reason",
[
(
lambda value: value["checks"][1].update(deps=["missing"]),
"Unknown check dependency",
),
(lambda value: value["checks"][1].update(deps=["unselected"]), "Cyclic"),
(
lambda value: value["checks"][1].update(repos=["missing"]),
"Unknown repository",
),
(lambda value: value["profiles"].update(full=["missing"]), "Unknown profile"),
(lambda value: value["checks"][1].update(id="selected"), "Duplicate"),
(
lambda value: value["repositories"].append(
{"name": "other", "path": "example"}
),
"Duplicate project repository path",
),
],
)
def test_all_semantic_references_are_checked_even_outside_selected_profile(
tmp_path, mutation, reason
):
project = valid()
mutation(project)
with pytest.raises(ValueError, match=reason):
check(tmp_path, project)
def test_valid_defaults_boundaries_and_dependency_order(tmp_path):
project = valid()
project["checks"][0].update(
deps=["unselected"], argv=["true", ""], timeout_seconds=0.1
)
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
jsonschema.validate(project, schema)
assert check(tmp_path, project).config == project
def test_runtime_shape_uses_only_implemented_schema_keywords():
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
permitted = {
"$schema",
"$defs",
"$ref",
"title",
"description",
"type",
"const",
"properties",
"additionalProperties",
"required",
"minItems",
"maxItems",
"uniqueItems",
"prefixItems",
"items",
"minLength",
"maxLength",
"pattern",
"maximum",
"exclusiveMinimum",
"enum",
}
def visit(node):
assert set(node) <= permitted
for group in ("properties", "$defs"):
for child in node.get(group, {}).values():
visit(child)
if "items" in node:
visit(node["items"])
for child in node.get("prefixItems", []):
visit(child)
visit(schema)
_schema_value(valid(), schema, schema["$defs"], "project")
def test_huge_numeric_input_is_a_controlled_bound_error(tmp_path):
project = deepcopy(valid())
project["checks"][1]["timeout_seconds"] = 10**1000
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"field,value",
[
("inputs", {}),
("inputs", None),
("inputs", {"repos": []}),
("inputs", {"paths": ["src/**"]}),
("inputs", {"repos": ["example", "example"]}),
("inputs", {"repos": [""]}),
("inputs", {"repos": ["example"], "glob": "*"}),
("after", ["selected", "selected"]),
("after", [""]),
("after", "selected"),
("reuse", True),
("reuse", "always"),
],
)
def test_input_order_and_reuse_shapes_apply_to_unselected_checks(
tmp_path, field, value
):
project = valid()
project["checks"][1][field] = value
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"change,reason",
[
({"inputs": {"repos": ["unknown"]}}, "Unknown input repository"),
({"after": ["unknown"]}, "Unknown check ordering"),
({"after": ["unselected"]}, "Cyclic"),
(
{"deps": ["selected"], "after": ["selected"]},
"Duplicate dependency/ordering",
),
],
)
def test_input_and_order_semantics_are_validated_before_selection(
tmp_path, change, reason
):
project = valid()
project["checks"][1].update(change)
with pytest.raises(ValueError, match=reason):
check(tmp_path, project)
def test_dependencies_and_order_only_edges_share_cycle_validation(tmp_path):
project = valid()
project["checks"][0]["after"] = ["unselected"]
project["checks"][1]["deps"] = ["selected"]
with pytest.raises(ValueError, match="Cyclic"):
check(tmp_path, project)
def test_valid_explicit_inputs_after_and_reuse_match_published_schema(tmp_path):
project = valid()
project["checks"][0].update(
inputs={"repos": ["example"]}, after=["unselected"], reuse="verified"
)
project["checks"][1]["reuse"] = "never"
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
jsonschema.validate(project, schema)
assert check(tmp_path, project).config == project
+151
View File
@@ -0,0 +1,151 @@
import json
import os
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.common import atomic_json, read_json, redact, safe_output
from govoplan_devkit.context import build_context
from govoplan_devkit.workspace import (
load_project,
selected_repositories,
git_bytes,
inspect_repository,
Repository,
)
def test_portable_project_rejects_escape_alias_collisions_and_duplicate_json(tmp_path):
config = tmp_path / "project.json"
for repositories in (
[{"name": "repo", "path": "../outside"}],
[
{"name": "repo", "path": "repo", "aliases": ["other"]},
{"name": "other", "path": "other"},
],
):
config.write_text(
json.dumps({"schema_version": 1, "repositories": repositories})
)
with pytest.raises(ValueError):
load_project(tmp_path, config)
config.write_text('{"schema_version":1,"schema_version":2}')
with pytest.raises(ValueError, match="Duplicate"):
read_json(config)
def test_missing_repository_is_error_not_clean(tmp_path):
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Test",
"repositories": [{"name": "missing", "path": "missing"}],
}
)
)
result = build_context(tmp_path, config, [], True)
assert result["_exit_code"] == 1
assert result["repositories"][0]["errors"]
assert not result["remote_checked"]
def test_unknown_repo_filter_is_not_silently_ignored(tmp_path):
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{"schema_version": 1, "repositories": [{"name": "repo", "path": "repo"}]}
)
)
with pytest.raises(ValueError, match="Unknown"):
selected_repositories(load_project(tmp_path, config), ["typo"])
def test_atomic_output_does_not_chmod_existing_parent(tmp_path):
folder = tmp_path / "public"
folder.mkdir(mode=0o755)
atomic_json(folder / "private.json", {"safe": True})
assert folder.stat().st_mode & 0o777 == 0o755
assert (folder / "private.json").stat().st_mode & 0o777 == 0o600
def test_read_rejects_fifo_and_symlink(tmp_path):
fifo = tmp_path / "fifo"
os.mkfifo(fifo)
with pytest.raises(ValueError, match="regular"):
read_json(fifo)
target = tmp_path / "target.json"
target.write_text("{}")
alias = tmp_path / "alias.json"
alias.symlink_to(target)
with pytest.raises(ValueError, match="symlink"):
read_json(alias)
def test_generic_secret_display_hygiene():
assert "topsecret" not in redact(
"Authorization: Bearer topsecret\nhttps://user:topsecret@example.invalid/"
)
def test_separate_secret_arguments_are_redacted_in_nested_json():
value = {
"stages": [{"argv": ["check", "--token", "hidden-value"]}],
"password": "hidden-password",
}
output = json.dumps(safe_output(value))
assert "hidden-value" not in output and "hidden-password" not in output
@pytest.mark.parametrize(
"name",
[
"GIT_DIR",
"GIT_NAMESPACE",
"GIT_COMMON_DIR",
"GIT_CONFIG_PARAMETERS",
"GIT_AUTHOR_NAME",
],
)
def test_inherited_git_redirects_are_rejected(tmp_path, monkeypatch, name):
monkeypatch.setenv(name, "fixture")
assert inspect_repository(Repository("example", tmp_path))["errors"]
with pytest.raises(ValueError, match="overrides"):
git_bytes(tmp_path, "status")
def test_deeply_nested_json_fails_as_controlled_input_error(tmp_path):
path = tmp_path / "deep.json"
path.write_text("[" * 10000 + "]" * 10000)
with pytest.raises(ValueError, match="nesting"):
read_json(path)
def test_clean_commit_without_upstream_is_still_changed(tmp_path):
subprocess.run(["git", "init", "-q", str(tmp_path)], check=True)
subprocess.run(
[
"git",
"-C",
str(tmp_path),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"--allow-empty",
"-qm",
"fixture",
],
check=True,
)
from govoplan_devkit.workspace import Project
repo = Repository("example", tmp_path)
assert selected_repositories(Project("Example", (repo,), {}), [], changed=True) == [
repo
]
+393
View File
@@ -0,0 +1,393 @@
"""Canonical phase dispatch tested with inert tools in disposable workspaces."""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import pytest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools/checks/check-focused.sh"
METADATA = META_ROOT / "tools/checks/focused-phases.json"
PHASE_IDS = [
"preflight",
"tooling",
"backend",
"core-ui",
"module-builds",
"browser",
"module-ui",
]
# These hashes bind the original working-copy check bodies at the phase split.
# Only two explicit Core/webui cd lines were added for independent invocation.
# Future deliberate changes to the canonical gate must update this contract.
LEGACY_BODY_SHA256 = {
"preflight": "b9c90fd3c84df788de5f2c001443672f683a9918459e1a7955ed4a225e6f20dd",
"tooling": "3d9a5bf32acbe97134f51327ed4b2457a69ad23b723c15a2b9bd0dce7f82396b",
"backend": "dd57c33919f06bd240516bbe47861be022e0b1c047334eedc7f6c596c2c49632",
"core-ui": "613f806a602970f1dbfb243f4c96ccc6ca4836849bb1f1a2c4cff97aea6f3aeb",
"module-builds": "1f2cd1e2c336f748fbf2972a2da87a0efec58ef9b084a9433511f794a456fa84",
"browser": "aa20d1e1d4ec9f00bc27c06cdee7ffa2456d3d3b8a7155da4e888a7e2209306d",
"module-ui": "c47794d82eb7bb47de114e86a95264e5898886f743c31eb7bdf82a079c827c9f",
}
EXTRA_TEST_COMMAND = '"$PYTHON" -m pytest -q tests/test_focused_phases.py\n'
def definitions():
source = SCRIPT.read_text()
return dict(
re.findall(
r"^# devkit-phase: ([a-z-]+) begin\n(.*?)^# devkit-phase: \1 end$",
source,
re.MULTILINE | re.DOTALL,
)
)
def test_original_commands_remain_exactly_once_in_original_order():
bodies = definitions()
metadata = json.loads(METADATA.read_text())
assert list(bodies) == PHASE_IDS
assert [phase["id"] for phase in metadata["phases"]] == PHASE_IDS
assert SCRIPT.read_text().count(EXTRA_TEST_COMMAND) == 1
for identity, body in bodies.items():
assert SCRIPT.read_text().count(f"# devkit-phase: {identity} begin") == 1
if identity == "tooling":
assert body.count(EXTRA_TEST_COMMAND) == 1
body = body.replace(EXTRA_TEST_COMMAND, "")
assert hashlib.sha256(body.encode()).hexdigest() == LEGACY_BODY_SHA256[identity]
def test_ordering_and_artifact_dependencies_are_distinct():
phases = json.loads(METADATA.read_text())["phases"]
assert [phase["order_after"] for phase in phases] == [
[],
*[[identity] for identity in PHASE_IDS[:-1]],
]
assert all(phase["depends_on"] == [] for phase in phases)
by_id = {phase["id"]: phase for phase in phases}
assert "{core}/webui/dist" in by_id["module-builds"]["outputs"]
assert any(
"does not serve or require module-builds dist" in note
for note in by_id["browser"]["notes"]
)
assert "port:4174" in by_id["browser"]["resources"]
cwd_lines = {
"core": 'cd "$ROOT"',
"meta": 'cd "$META_ROOT"',
"core-webui": 'cd "$ROOT/webui"',
"access-webui": 'cd "${WORKSPACE_ROOT}/govoplan-access/webui"',
}
for phase in phases:
assert definitions()[phase["id"]].splitlines()[0] == cwd_lines[phase["cwd"]]
FAKE_TOOL = r"""
import json
import os
from pathlib import Path
import sys
log = Path(os.environ["FOCUSED_FIXTURE_LOG"])
record = {
"tool": Path(sys.argv[0]).name,
"argv": sys.argv[1:],
"cwd": os.getcwd(),
"env": {key: os.environ.get(key) for key in (
"GOVOPLAN_WORKSPACE_ROOT", "NPM_CONFIG_USERCONFIG", "GOVOPLAN_NPM_USERCONFIG",
"NPM_CONFIG_TMP", "npm_config_tmp", "PYTHONPATH", "PATH",
)},
}
if "-" in sys.argv[1:]:
record["stdin"] = sys.stdin.read()
with log.open("a") as handle:
handle.write(json.dumps(record) + "\n")
if os.environ.get("FOCUSED_FIXTURE_FAIL_TOKEN") in sys.argv[1:]:
raise SystemExit(7)
"""
@pytest.fixture
def fixture_workspace(tmp_path):
workspace = tmp_path / "workspace"
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
copied = meta / "tools/checks/check-focused.sh"
copied.parent.mkdir(parents=True)
shutil.copyfile(SCRIPT, copied)
shutil.copyfile(METADATA, copied.with_name("focused-phases.json"))
for name in (
"govoplan",
"govoplan-core",
"govoplan-access",
"govoplan-payments",
"govoplan-dataflow",
"govoplan-datasources",
"govoplan-workflow",
"govoplan-dashboard",
"govoplan-approvals",
"govoplan-postbox",
"govoplan-mail",
"govoplan-files",
"govoplan-campaign",
"govoplan-policy",
"govoplan-wiki",
):
(workspace / name / "webui").mkdir(parents=True, exist_ok=True)
(workspace / name / "src").mkdir(exist_ok=True)
(meta / "tests").mkdir()
for name in ("test_devkit_alpha.py", "test_devkit_beta.py"):
(meta / "tests" / name).write_text("# inert glob fixture\n")
fake_bin = tmp_path / "bin"
fake_bin.mkdir()
for target in [
*(fake_bin / name for name in ("python-check", "node", "npm", "bash")),
core / "webui/node_modules/.bin/tsc",
meta / "tools/checks/check_dependency_boundaries.py",
]:
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(f"#!{sys.executable}\n" + FAKE_TOOL)
target.chmod(0o700)
temporary = tmp_path / "temporary"
temporary.mkdir()
log = tmp_path / "calls.jsonl"
env = {
**os.environ,
"PATH": str(fake_bin) + os.pathsep + os.environ["PATH"],
"GOVOPLAN_WORKSPACE_ROOT": str(workspace),
"GOVOPLAN_CORE_ROOT": str(core),
"PYTHON": str(fake_bin / "python-check"),
"NODE": str(fake_bin / "node"),
"NPM": str(fake_bin / "npm"),
"TMPDIR": str(temporary),
"FOCUSED_FIXTURE_LOG": str(log),
"PYTHONPATH": "inherited-fixture-tail",
"NPM_CONFIG_TMP": "must-be-unset",
"npm_config_tmp": "must-be-unset",
}
env.pop("FOCUSED_FIXTURE_FAIL_TOKEN", None)
return {
"workspace": workspace,
"meta": meta,
"core": core,
"script": copied,
"env": env,
"log": log,
"temporary": temporary,
}
def invoke(fixture, *arguments, env=None):
return subprocess.run(
["/bin/bash", str(fixture["script"]), *arguments],
cwd=fixture["workspace"].parent,
env=env or fixture["env"],
capture_output=True,
text=True,
timeout=30,
)
def records(fixture):
path = fixture["log"]
return (
[json.loads(line) for line in path.read_text().splitlines()]
if path.exists()
else []
)
def normalized(calls):
result = []
for call in calls:
call = json.loads(json.dumps(call))
call["env"].pop("NPM_CONFIG_USERCONFIG")
call["env"].pop("GOVOPLAN_NPM_USERCONFIG")
result.append(call)
return result
def test_default_gate_equals_sequential_independent_phases(fixture_workspace):
fixture = fixture_workspace
full = invoke(fixture)
assert full.returncode == 0, full.stderr
original = records(fixture)
offset = len(original)
for identity in PHASE_IDS:
isolated = invoke(fixture, "--phase", identity)
assert isolated.returncode == 0, isolated.stderr
assert normalized(original) == normalized(records(fixture)[offset:])
assert len({call["env"]["NPM_CONFIG_USERCONFIG"] for call in original}) == len(
PHASE_IDS
)
assert not list(fixture["temporary"].iterdir())
assert sum("test:conformance" in call["argv"] for call in original) == 1
assert sum("test:module-permutations" in call["argv"] for call in original) == 1
assert sum("tests/test_focused_phases.py" in call["argv"] for call in original) == 1
# The here-document remains one Python invocation, not executed shell text.
ast_scan = [call for call in original if "stdin" in call]
assert len(ast_scan) == 1
assert "AST syntax check passed for" in ast_scan[0]["stdin"]
@pytest.mark.parametrize("identity", PHASE_IDS)
def test_each_phase_initializes_its_own_cwd_and_environment(
fixture_workspace, identity
):
fixture = fixture_workspace
result = invoke(fixture, "--phase", identity)
assert result.returncode == 0, result.stderr
calls = records(fixture)
assert calls
phase = next(
item
for item in json.loads(METADATA.read_text())["phases"]
if item["id"] == identity
)
starts = {
"core": fixture["core"],
"meta": fixture["meta"],
"core-webui": fixture["core"] / "webui",
"access-webui": fixture["workspace"] / "govoplan-access/webui",
}
assert calls[0]["cwd"] == str(starts[phase["cwd"]])
for call in calls:
env = call["env"]
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(fixture["workspace"])
assert env["NPM_CONFIG_USERCONFIG"] == env["GOVOPLAN_NPM_USERCONFIG"]
assert Path(env["NPM_CONFIG_USERCONFIG"]).parent == fixture["temporary"]
assert not Path(env["NPM_CONFIG_USERCONFIG"]).exists()
assert env["NPM_CONFIG_TMP"] is None and env["npm_config_tmp"] is None
assert env["PATH"].split(os.pathsep)[0] == str(
fixture["core"] / "webui/node_modules/.bin"
)
assert env["PYTHONPATH"].endswith(os.pathsep + "inherited-fixture-tail")
assert str(fixture["core"] / "src") in env["PYTHONPATH"].split(os.pathsep)
assert not list(fixture["temporary"].iterdir())
def test_full_is_fail_fast_and_failure_cleans_setup(fixture_workspace):
fixture = fixture_workspace
result = invoke(
fixture,
env={
**fixture["env"],
"FOCUSED_FIXTURE_FAIL_TOKEN": "test:module-permutations",
},
)
assert result.returncode == 7
calls = records(fixture)
assert calls[-1]["argv"] == ["run", "test:module-permutations"]
assert not any("test:conformance" in call["argv"] for call in calls)
assert not any("test:passwords" in call["argv"] for call in calls)
assert not list(fixture["temporary"].iterdir())
@pytest.mark.parametrize(
"arguments",
[
["--unknown"],
["--phase"],
["--phase", ""],
["--phase", "not-a-phase"],
["--phase", "browser", "--phase", "tooling"],
["--list-phases", "--phase", "browser"],
["--phase", "browser", "--list-phases"],
["--list-phases", "--list-phases"],
["--json"],
["--list-phases", "--json", "--json"],
["--phase", "browser; touch unexpected"],
],
)
def test_invalid_selection_is_rejected_before_setup(fixture_workspace, arguments):
fixture = fixture_workspace
env = {
**fixture["env"],
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
"NODE": "absent-node",
"NPM": "absent-npm",
}
result = invoke(fixture, *arguments, env=env)
assert result.returncode == 2
assert "check-focused:" in result.stderr
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
@pytest.mark.parametrize(
"arguments",
[["--list-phases"], ["--list-phases", "--json"], ["--json", "--list-phases"]],
)
def test_listing_is_read_only_without_product_tools(fixture_workspace, arguments):
fixture = fixture_workspace
env = {
**fixture["env"],
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
"PYTHON": "/absent-python",
"NODE": "absent-node",
"NPM": "absent-npm",
}
result = invoke(fixture, *arguments, env=env)
assert result.returncode == 0, result.stderr
if "--json" in arguments:
assert json.loads(result.stdout) == json.loads(METADATA.read_text())
else:
assert [line.split("\t")[0] for line in result.stdout.splitlines()] == PHASE_IDS
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
def test_metadata_loading_never_imports_project_python_modules(fixture_workspace):
fixture = fixture_workspace
foreign = fixture["workspace"].parent / "json.py"
foreign.write_text(
"raise AssertionError('project module imported during listing')\n"
)
result = invoke(
fixture,
"--list-phases",
"--json",
env={**fixture["env"], "PYTHONPATH": str(foreign.parent)},
)
assert result.returncode == 0, result.stderr
assert json.loads(result.stdout)["schema_version"] == 1
assert not fixture["log"].exists()
@pytest.mark.parametrize(
"change",
[
"unknown-fields",
"duplicate-id",
"unknown-cwd",
"later-prerequisite",
"invalid-version",
],
)
def test_malformed_metadata_fails_before_any_check(fixture_workspace, change):
fixture = fixture_workspace
path = fixture["script"].with_name("focused-phases.json")
catalog = json.loads(path.read_text())
if change == "unknown-fields":
catalog["phases"][0]["shell"] = "touch should-not-run"
elif change == "duplicate-id":
catalog["phases"][1]["id"] = catalog["phases"][0]["id"]
elif change == "unknown-cwd":
catalog["phases"][0]["cwd"] = "elsewhere"
elif change == "later-prerequisite":
catalog["phases"][0]["depends_on"] = ["browser"]
else:
catalog["schema_version"] = True
path.write_text(json.dumps(catalog))
result = invoke(fixture, "--phase", "browser")
assert result.returncode == 2
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
+194
View File
@@ -0,0 +1,194 @@
"""Offline safety and completeness checks for the cross-product UI review program."""
import importlib.util
from pathlib import Path
import socket
import sys
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/gitea"))
SPEC = importlib.util.spec_from_file_location("ui_review_program", ROOT / "tools/gitea/gitea-ui-review-program.py")
assert SPEC and SPEC.loader
program = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = program
SPEC.loader.exec_module(program)
def scope(scope_id="campaigns", kind="manifest"):
return {
"scope_id": scope_id, "name": "Campaigns", "repository": "govoplan-campaign",
"kind": kind, "manifest_paths": ["src/govoplan_campaign/backend/manifest.py"],
"frontend": {
"routes": [{"path": "/campaigns/:campaignId/*", "component": "CampaignWorkspace"}],
"public_routes": [{"path": "/public/example", "component": "PublicExample"}],
"settings_routes": [{"path": "/settings/example", "component": "ExampleSettings"}],
"nav_items": [{"path": "/campaigns", "label": "Campaigns"}],
"view_surfaces": [{"id": "campaigns.widget.activity", "label": "Activity widget"}],
},
"source_groups": {"Dialogs and embedded editing surfaces": ["webui/src/ExampleDialog.tsx"]},
"ui_source_count": 1,
}
def test_catalog_includes_core_all_manifests_and_each_placeholder(tmp_path):
names = ["govoplan", "govoplan-core", "govoplan-campaign", "govoplan-ledger", "govoplan-xoev", "website"]
for name in names:
(tmp_path / name).mkdir()
catalog = {"repositories": [
{"name": name, "path": name, "category": "website" if name == "website" else "system" if name in {"govoplan", "govoplan-core"} else "connector" if name == "govoplan-xoev" else "module"}
for name in names
]}
manifests = [{"id": "campaigns", "name": "Campaigns", "repository": "govoplan-campaign", "frontend": None}]
scopes = program.build_scopes(catalog, tmp_path, manifests)
assert {item["scope_id"] for item in scopes} == {"core", "campaigns", "catalog:govoplan-ledger", "catalog:govoplan-xoev"}
assert sum(item["kind"] == "placeholder" for item in scopes) == 2
assert next(item for item in scopes if item["scope_id"] == "campaigns")["kind"] == "manifest"
def test_missing_checkout_or_implementation_without_manifest_is_not_called_placeholder(tmp_path):
catalog = {"repositories": [{"name": "govoplan-example", "path": "govoplan-example", "category": "module"}]}
with pytest.raises(program.GiteaError, match="Missing source checkout"):
program.build_scopes(catalog, tmp_path, [])
root = tmp_path / "govoplan-example"
root.mkdir()
(root / "pyproject.toml").touch()
with pytest.raises(program.GiteaError, match="implementation but no extracted manifest"):
program.build_scopes(catalog, tmp_path, [])
def test_duplicate_or_uncatalogued_manifest_is_rejected(tmp_path):
manifest = {"id": "example", "name": "Example", "repository": "govoplan-example", "frontend": None}
with pytest.raises(program.GiteaError, match="Duplicate source manifest"):
program.build_scopes({"repositories": []}, tmp_path, [manifest, manifest])
with pytest.raises(program.GiteaError, match="absent from the module review catalog"):
program.build_scopes({"repositories": []}, tmp_path, [manifest])
def test_existing_closed_issue_is_found_and_unchanged():
issue = {"number": 4, "title": "Reviewer renamed it", "state": "closed", "body": program.marker("campaigns") + "\nHuman findings\n- [x] Done"}
before = issue.copy()
assert program.find_existing([issue], "campaigns", program.issue_title(scope())) is issue
assert issue == before
def test_gitea_null_pull_request_field_is_an_ordinary_issue():
issue = {"number": 25, "title": "UI review", "state": "open", "body": program.marker("mail"), "pull_request": None}
assert program.find_existing([issue], "mail", "UI review") is issue
def test_unmanaged_title_and_ambiguous_marker_stop_without_overwrite():
title = program.issue_title(scope())
with pytest.raises(program.GiteaError, match="Unmanaged exact-title"):
program.find_existing([{"title": " " + title.upper() + " ", "body": "user content"}], "campaigns", title)
duplicate = {"title": title, "body": program.marker("campaigns")}
with pytest.raises(program.GiteaError, match="Ambiguous"):
program.find_existing([duplicate, duplicate.copy()], "campaigns", title)
def test_pr_not_used_as_matching_issue():
issue = {"title": program.issue_title(scope()), "body": program.marker("campaigns"), "pull_request": {}}
assert program.find_existing([issue], "campaigns", issue["title"]) is None
def test_child_inventory_and_all_principles_start_pending():
body = program.issue_body(scope(), "https://example.test/epic/56")
assert "**Pending / not reviewed.**" in body
assert "https://example.test/epic/56" in body
assert "/campaigns/:campaignId/*" in body
assert "/public/example" in body
assert "/settings/example" in body
assert "campaigns.widget.activity" in body
assert "webui/src/ExampleDialog.tsx" in body
assert "compact read-only campaign settings dashboard" in body
assert "Save/Cancel" in body and "dirty-state protection" in body
assert "- [x]" not in body
assert body.count("| Pending inventory | Pending review | Not yet recorded | None approved |") == 9
for identity, _ in program.PRINCIPLES:
assert identity in body
assert "Reopen this issue or link an owned follow-up" in body
def test_headless_and_placeholder_scopes_are_not_automatic_completions():
headless = scope("rest")
headless["frontend"] = None
assert "No standalone frontend is declared" in program.issue_body(headless, "epic")
assert "**The review is still pending:**" in program.issue_body(headless, "epic")
placeholder = scope("catalog:govoplan-ledger", "placeholder")
body = program.issue_body(placeholder, "epic")
assert "there is no runtime module ID, manifest or standalone WebUI" in body
assert "Keep the future interface review pending" in body
assert "- [x]" not in body
def test_source_grouping_uses_real_files_and_clearly_bounds_large_seeds(tmp_path):
source = tmp_path / "webui/src"
source.mkdir(parents=True)
for name in ["ExamplePage.tsx", "EditDialog.tsx", "TenantSettings.tsx", "ActivityWidget.tsx", "Button.tsx"]:
(source / name).touch()
groups = program.source_groups(tmp_path)
assert sum(map(len, groups.values())) == 5
assert groups["Dialogs and embedded editing surfaces"] == ["webui/src/EditDialog.tsx"]
large = scope()
large["source_groups"] = {"Pages": [f"webui/src/Page{index}.tsx" for index in range(45)]}
large["ui_source_count"] = 45
seed = program.source_seed(large)
assert "5 further files" in seed
assert "not a completed runtime audit" in seed
def test_epic_initialization_preserves_surrounding_text_and_keeps_all_unchecked():
body = program.EPIC_MARKER + "\nHuman introduction\n" + program.LIST_START + "\n" + program.INITIAL_LIST + "\n" + program.LIST_END + "\nHuman evidence"
records = [
{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"},
{"name": "Ledger", "scope_id": "catalog:govoplan-ledger", "repository": "govoplan-ledger", "kind": "placeholder", "url": "https://example.test/ledger/2"},
]
result = program.initialized_epic_body(body, records)
assert "Human introduction" in result and "Human evidence" in result
assert result.count("- [ ]") == 2
assert "Catalogued placeholders" in result
assert program.initialized_epic_body(result, records) == result
human_progress = result.replace("- [ ] [Core]", "- [x] [Core]")
assert program.initialized_epic_body(human_progress, records) == human_progress
def test_epic_edited_or_ambiguous_lists_are_never_overwritten():
records = [{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"}]
body = program.EPIC_MARKER + program.LIST_START + "User-managed content" + program.LIST_END
with pytest.raises(program.GiteaError, match="already edited"):
program.initialized_epic_body(body, records)
with pytest.raises(program.GiteaError, match="absent or ambiguous"):
program.initialized_epic_body(body + program.LIST_START, records)
with pytest.raises(program.GiteaError, match="incomplete issue link inventory"):
program.render_links([{**records[0], "url": None}])
def test_ipv4_override_is_host_scoped_and_restored(monkeypatch):
calls = []
def original(host, port, family=0, type=0, proto=0, flags=0):
calls.append((host, family))
return []
monkeypatch.setattr(socket, "getaddrinfo", original)
with program.ipv4_for_target(True):
socket.getaddrinfo("git.add-ideas.de", 443)
socket.getaddrinfo("unrelated.example", 443)
assert calls == [("git.add-ideas.de", socket.AF_INET), ("unrelated.example", 0)]
assert socket.getaddrinfo is original
def test_snapshot_covers_every_current_catalog_module():
import json
catalog = json.loads((ROOT / "repositories.json").read_text())
inventory_path = ROOT / "docs/project/ui-review-issue-inventory.json"
snapshot = json.loads(inventory_path.read_text())
expected = {repo["name"] for repo in catalog["repositories"] if repo["category"] in {"module", "connector"} or repo["name"] == "govoplan-core"}
assert {issue["repository"] for issue in snapshot["issues"]} == expected
assert len(snapshot["issues"]) == len(expected)
assert len({issue["url"] for issue in snapshot["issues"]}) == len(expected)
assert snapshot["scope_count"] == 77
assert snapshot["implemented_scopes"] == 73
assert snapshot["manifest_modules"] == 72
assert snapshot["catalogued_placeholders"] == 4
assert all(issue["number"] and issue["url"].startswith("https://git.add-ideas.de/GovOPlaN/") for issue in snapshot["issues"])
+270 -76
View File
@@ -1,13 +1,151 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
# The JSON catalog is the only phase order/metadata authority. Check commands
# remain in the marked functions below; devkit invokes this script, not copies.
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
FOCUSED_MODE=run
FOCUSED_PHASE=""
FOCUSED_JSON=0
focused_usage() {
echo "Usage: check-focused.sh [--phase ID] | --list-phases [--json]"
echo "Without --phase, run every canonical phase in catalog order (fail fast)."
}
while [[ $# -gt 0 ]]; do
case "$1" in
--phase)
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" || $# -lt 2 || -z "$2" || "$2" == --* ]]; then
echo "check-focused: --phase requires one ID and cannot be combined with listing." >&2
exit 2
fi
FOCUSED_PHASE="$2"
shift 2
;;
--list-phases)
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" ]]; then
echo "check-focused: duplicate or incompatible phase selection." >&2
exit 2
fi
FOCUSED_MODE=list
shift
;;
--json)
if [[ "$FOCUSED_JSON" == 1 ]]; then
echo "check-focused: duplicate --json." >&2
exit 2
fi
FOCUSED_JSON=1
shift
;;
--help|-h)
focused_usage
exit 0
;;
*)
echo "check-focused: unknown argument: $1" >&2
focused_usage >&2
exit 2
;;
esac
done
if [[ "$FOCUSED_JSON" == 1 && "$FOCUSED_MODE" != list ]]; then
echo "check-focused: --json is supported only with --list-phases." >&2
exit 2
fi
# Metadata-only operations need standard Python, not an installed product venv,
# Node, npm, a Core checkout, or any temporary/state directory.
FOCUSED_METADATA_PYTHON="$(command -v python3)" || {
echo "check-focused: Python 3 is required to read phase metadata." >&2
exit 127
}
FOCUSED_SELECTION="$(
PYTHONDONTWRITEBYTECODE=1 "$FOCUSED_METADATA_PYTHON" -I -S - \
"$META_ROOT/tools/checks/focused-phases.json" "$FOCUSED_MODE" "$FOCUSED_PHASE" "$FOCUSED_JSON" <<'PY'
import json
import os
from pathlib import Path
import re
import stat
import sys
try:
path = Path(sys.argv[1])
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
status = os.fstat(handle.fileno())
if not stat.S_ISREG(status.st_mode) or status.st_size > 1024 * 1024:
raise ValueError("phase metadata must be a bounded regular file")
encoded = handle.read(1024 * 1024 + 1)
if len(encoded) > 1024 * 1024:
raise ValueError("phase metadata exceeds its size bound")
def unique(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError("duplicate phase metadata key")
result[key] = value
return result
catalog = json.loads(encoded, object_pairs_hook=unique)
if not isinstance(catalog, dict) or set(catalog) != {"schema_version", "phases"} or type(catalog["schema_version"]) is not int or catalog["schema_version"] != 1:
raise ValueError("unsupported phase metadata schema")
phases = catalog["phases"]
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
raise ValueError("phase catalog requires 132 definitions")
seen = set()
fields = {"id", "title", "cwd", "order_after", "depends_on", "resources", "outputs", "notes"}
for phase in phases:
if not isinstance(phase, dict) or set(phase) != fields:
raise ValueError("invalid phase metadata fields")
identity = phase["id"]
if not isinstance(identity, str) or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity) or identity in seen:
raise ValueError("invalid or duplicate phase ID")
if not isinstance(phase["title"], str) or not phase["title"].strip() or len(phase["title"]) > 256 or any(ord(char) < 32 for char in phase["title"]):
raise ValueError("invalid phase title")
if phase["cwd"] not in {"core", "meta", "core-webui", "access-webui"}:
raise ValueError("unsupported phase working directory")
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
values = phase[field]
if not isinstance(values, list) or len(values) > 64 or any(not isinstance(value, str) or not value or len(value) > 4096 or "\0" in value for value in values):
raise ValueError("invalid phase list: " + field)
if len(set(values)) != len(values):
raise ValueError("duplicate phase list value: " + field)
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
raise ValueError("phase prerequisites must precede their consumer")
seen.add(identity)
mode, selected, as_json = sys.argv[2:]
if selected and selected not in seen:
raise ValueError("unknown phase: " + selected)
if mode == "list":
print(json.dumps(catalog, indent=2) if as_json == "1" else "\n".join(phase["id"] + "\t" + phase["title"] for phase in phases))
else:
print("\n".join(phase["id"] for phase in phases if not selected or phase["id"] == selected))
except (OSError, ValueError, TypeError, RecursionError) as exc:
print("check-focused: " + str(exc), file=sys.stderr)
raise SystemExit(2)
PY
)"
if [[ "$FOCUSED_MODE" == list ]]; then
printf '%s\n' "$FOCUSED_SELECTION"
exit 0
fi
focused_setup() {
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$META_ROOT")}"
export GOVOPLAN_WORKSPACE_ROOT="$WORKSPACE_ROOT"
ROOT="${GOVOPLAN_CORE_ROOT:-$WORKSPACE_ROOT/govoplan-core}"
ROOT="$(cd "$ROOT" && pwd)" ROOT="$(cd "$ROOT" && pwd)"
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}" VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}" PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
NODE="/home/zemion/.nvm/versions/node/v22.22.3/bin" NODE="$(command -v "${NODE:-node}")" || { echo "check-focused: Node is unavailable; run ./devkit doctor." >&2; exit 127; }
NPM="$NODE/npm" NPM="$(command -v "${NPM:-npm}")" || { echo "check-focused: npm is unavailable; run ./devkit doctor." >&2; exit 127; }
NODE_BIN="$(dirname "$NODE")"
WEBUI_BIN="$ROOT/webui/node_modules/.bin" WEBUI_BIN="$ROOT/webui/node_modules/.bin"
NPM_USERCONFIG="$(mktemp "${TMPDIR:-/tmp}/govoplan-npmrc.XXXXXXXX")" NPM_USERCONFIG="$(mktemp "${TMPDIR:-/tmp}/govoplan-npmrc.XXXXXXXX")"
@@ -19,7 +157,7 @@ trap 'rm -f "$NPM_USERCONFIG"' EXIT
exit 127 exit 127
} }
export PATH="$WEBUI_BIN:$NODE:$PATH" export PATH="$WEBUI_BIN:$NODE_BIN:$PATH"
export NPM_CONFIG_USERCONFIG="$NPM_USERCONFIG" export NPM_CONFIG_USERCONFIG="$NPM_USERCONFIG"
export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG" export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG"
unset npm_config_tmp NPM_CONFIG_TMP unset npm_config_tmp NPM_CONFIG_TMP
@@ -27,28 +165,41 @@ unset npm_config_tmp NPM_CONFIG_TMP
# Validate the current sibling checkouts even when a newly added module has not # Validate the current sibling checkouts even when a newly added module has not
# yet been installed into an existing development virtualenv. # yet been installed into an existing development virtualenv.
SOURCE_PYTHONPATH="" SOURCE_PYTHONPATH=""
for source_dir in "$META_ROOT"/../govoplan*/src; do for source_dir in "$WORKSPACE_ROOT"/govoplan*/src; do
[ -d "$source_dir" ] || continue [ -d "$source_dir" ] || continue
SOURCE_PYTHONPATH="${SOURCE_PYTHONPATH:+$SOURCE_PYTHONPATH:}$source_dir" SOURCE_PYTHONPATH="${SOURCE_PYTHONPATH:+$SOURCE_PYTHONPATH:}$source_dir"
done done
export PYTHONPATH="${SOURCE_PYTHONPATH}${PYTHONPATH:+:$PYTHONPATH}" export PYTHONPATH="${SOURCE_PYTHONPATH}${PYTHONPATH:+:$PYTHONPATH}"
}
focused_phase_preflight() {
# devkit-phase: preflight begin
cd "$ROOT" cd "$ROOT"
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh" GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py" PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
"$NODE/node" "$META_ROOT/tests/test-jsx-value-imports.mjs" "$NODE" "$META_ROOT/tests/test-jsx-value-imports.mjs"
"$NODE/node" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs" "$NODE" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
"$NODE/node" "$META_ROOT/../govoplan-files/webui/scripts/test-archive-client.mjs" "$NODE" "$META_ROOT/tests/test-heading-help.mjs"
"$NODE" "$META_ROOT/tools/checks/check-heading-help.mjs"
"$NODE" --test "$META_ROOT/tests/test-devkit-display-labels.mjs"
"$NODE" --test "$ROOT/webui/tests/component-test-runner.test.mjs"
"$NODE" "$WORKSPACE_ROOT/govoplan-files/webui/scripts/test-archive-client.mjs"
# devkit-phase: preflight end
}
focused_phase_tooling() {
# devkit-phase: tooling begin
cd "$META_ROOT" cd "$META_ROOT"
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints "$PYTHON" tools/inventory/platform-interface-inventory.py --workspace-root "$WORKSPACE_ROOT" --strict-declarations --strict-endpoints
"$PYTHON" tools/repo/sync-module-package-workflows.py --check "$PYTHON" tools/repo/sync-module-package-workflows.py --check
"$PYTHON" tools/release/generate-developer-meta-package.py --check "$PYTHON" tools/release/generate-developer-meta-package.py --check
"$PYTHON" tools/checks/check-webui-package-facades.py "$PYTHON" tools/checks/check-webui-package-facades.py
"$PYTHON" -m unittest tests.test_webui_package_facades "$PYTHON" -m unittest tests.test_webui_package_facades
"$PYTHON" -m pytest -q tests/test_ui_review_program.py "$META_ROOT"/tests/test_devkit_*.py
"$PYTHON" -m pytest -q tests/test_focused_phases.py
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release "$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
"$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries "$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries
"$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py "$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py
@@ -59,14 +210,20 @@ cd "$META_ROOT"
"$PYTHON" -m unittest tests.test_configuration_package_artifacts "$PYTHON" -m unittest tests.test_configuration_package_artifacts
"$PYTHON" -m unittest tests.test_institutional_governance_journey "$PYTHON" -m unittest tests.test_institutional_governance_journey
"$PYTHON" -m unittest tests.test_institutional_service_journey "$PYTHON" -m unittest tests.test_institutional_service_journey
# devkit-phase: tooling end
}
focused_phase_backend() {
# devkit-phase: backend begin
cd "$ROOT" cd "$ROOT"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
import ast import ast
import pathlib import pathlib
import os
import sys import sys
repos_root = pathlib.Path("/mnt/DATA/git") repos_root = pathlib.Path(os.environ["GOVOPLAN_WORKSPACE_ROOT"])
roots = [ roots = [
repo / "src" repo / "src"
for repo in sorted(repos_root.glob("govoplan*")) for repo in sorted(repos_root.glob("govoplan*"))
@@ -107,107 +264,127 @@ PY
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api "$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves "$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
"$PYTHON" -m pytest -q \ "$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-files/tests/test_managed_archives.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_managed_archives.py" \
/mnt/DATA/git/govoplan-files/tests/test_archive_work.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_work.py" \
/mnt/DATA/git/govoplan-files/tests/test_archive_staging.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_staging.py" \
/mnt/DATA/git/govoplan-files/tests/test_upload_response_batching.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_upload_response_batching.py" \
/mnt/DATA/git/govoplan-files/tests/test_archive_performance.py "${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_performance.py"
"$PYTHON" -m pytest -q \ "$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-files/tests/test_archive_workers.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_workers.py" \
/mnt/DATA/git/govoplan-files/tests/test_archive_inspection_bounds.py \ "${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_inspection_bounds.py" \
/mnt/DATA/git/govoplan-files/tests/test_archives.py "${WORKSPACE_ROOT}/govoplan-files/tests/test_archives.py"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests/test_external_function_mapping_migration.py "$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests/test_external_function_mapping_migration.py"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests "$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-templates/tests "$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-templates/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-connectors/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-datasources/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dataflow/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow-engine/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-views/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-quick-access/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-quick-access/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dashboard/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-postbox/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-portal/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-payments/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-payments/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms-runtime/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms-runtime/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-cases/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-cases/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-committee/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-committee/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-voting/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-voting/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-approvals/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-approvals/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-identity-trust/tests"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-encryption/tests"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests "$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-wiki/tests"
"$PYTHON" -m pytest -q \ "$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_approval_gate.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_editor_state_security.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_editor_state_security.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_mail_profile_boundary.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_mail_profile_boundary.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_independent_configuration_repairs.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_independent_configuration_repairs.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_incremental_review_persistence.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_incremental_review_persistence.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_reviewed_build_mock.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_reviewed_build_mock.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_delivery_policy_settings.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_delivery_policy_settings.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_synchronous_delivery_policy.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_synchronous_delivery_policy.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_workerless_recovery.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_workerless_recovery.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_imap_batch_integration.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_imap_batch_integration.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_testbed_claim_recovery.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_testbed_claim_recovery.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py" \
/mnt/DATA/git/govoplan-campaign/tests/test_archive_encryption_governance.py \ "${WORKSPACE_ROOT}/govoplan-campaign/tests/test_archive_encryption_governance.py" \
/mnt/DATA/git/govoplan-policy/tests/test_campaign_archive_encryption.py \ "${WORKSPACE_ROOT}/govoplan-policy/tests/test_campaign_archive_encryption.py" \
/mnt/DATA/git/govoplan-policy/tests/test_archive_encryption_api.py "${WORKSPACE_ROOT}/govoplan-policy/tests/test_archive_encryption_api.py"
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py" "$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py" "$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-mail/tests/test_campaign_protocol_authorization.py /mnt/DATA/git/govoplan-mail/tests/test_campaign_imap_batch.py "$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_protocol_authorization.py" "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_imap_batch.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests "$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-mail/tests"
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count "$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
"$PYTHON" -m unittest \ "$PYTHON" -m unittest \
tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \ tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \
tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \ tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \
tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry
# devkit-phase: backend end
}
focused_phase_core_ui() {
# devkit-phase: core-ui begin
cd "$ROOT/webui" cd "$ROOT/webui"
"$NPM" run test:api-client-cache "$NPM" run test:api-client-cache
"$NPM" run test:auth-action-state "$NPM" run test:auth-action-state
"$NPM" run test:dependency-security "$NPM" run test:dependency-security
"$NPM" run test:layout-primitives "$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components
"$NPM" run test:mail-components "$NODE" --test tests/breadcrumb-bar.test.mjs
"$NPM" run test:module-capabilities "$NPM" run test:module-capabilities
"$NPM" run test:module-permutations # devkit-phase: core-ui end
"$NPM" run test:conformance }
cd /mnt/DATA/git/govoplan-access/webui focused_phase_module_builds() {
# devkit-phase: module-builds begin
cd "$ROOT/webui"
"$NPM" run test:module-permutations
# devkit-phase: module-builds end
}
focused_phase_browser() {
# devkit-phase: browser begin
cd "$ROOT/webui"
"$NPM" run test:conformance
# devkit-phase: browser end
}
focused_phase_module_ui() {
# devkit-phase: module-ui begin
cd "${WORKSPACE_ROOT}/govoplan-access/webui"
"$NPM" run test:passwords "$NPM" run test:passwords
"$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json "$WEBUI_BIN/tsc" -p "${WORKSPACE_ROOT}/govoplan-payments/webui/tsconfig.json"
cd /mnt/DATA/git/govoplan-payments/webui cd "${WORKSPACE_ROOT}/govoplan-payments/webui"
"$NPM" run test:interface-pattern "$NPM" run test:interface-pattern
cd /mnt/DATA/git/govoplan-dataflow/webui cd "${WORKSPACE_ROOT}/govoplan-dataflow/webui"
"$NPM" run test:structure "$NPM" run test:structure
cd /mnt/DATA/git/govoplan-datasources/webui cd "${WORKSPACE_ROOT}/govoplan-datasources/webui"
"$NPM" run typecheck "$NPM" run typecheck
cd /mnt/DATA/git/govoplan-workflow/webui cd "${WORKSPACE_ROOT}/govoplan-workflow/webui"
"$NPM" run typecheck "$NPM" run typecheck
cd /mnt/DATA/git/govoplan-dashboard/webui cd "${WORKSPACE_ROOT}/govoplan-dashboard/webui"
"$NPM" run test:dashboard-layout "$NPM" run test:dashboard-layout
cd /mnt/DATA/git/govoplan-approvals/webui cd "${WORKSPACE_ROOT}/govoplan-approvals/webui"
"$NPM" run test:workspace-layout "$NPM" run test:workspace-layout
cd /mnt/DATA/git/govoplan-postbox/webui cd "${WORKSPACE_ROOT}/govoplan-postbox/webui"
"$NPM" run test:ui-structure "$NPM" run test:ui-structure
cd /mnt/DATA/git/govoplan-mail/webui cd "${WORKSPACE_ROOT}/govoplan-mail/webui"
"$NPM" run test:mail-ui "$NPM" run test:mail-ui
cd /mnt/DATA/git/govoplan-files/webui cd "${WORKSPACE_ROOT}/govoplan-files/webui"
"$NPM" run test:managed-archive "$NPM" run test:managed-archive
cd /mnt/DATA/git/govoplan-campaign/webui cd "${WORKSPACE_ROOT}/govoplan-campaign/webui"
"$NPM" run test:policy-ui "$NPM" run test:policy-ui
"$NPM" run test:template-preview "$NPM" run test:template-preview
"$NPM" run test:review-workflow "$NPM" run test:review-workflow
@@ -215,8 +392,25 @@ cd /mnt/DATA/git/govoplan-campaign/webui
"$NPM" run test:campaign-collaboration "$NPM" run test:campaign-collaboration
"$NPM" run test:campaign-work "$NPM" run test:campaign-work
cd /mnt/DATA/git/govoplan-policy/webui cd "${WORKSPACE_ROOT}/govoplan-policy/webui"
"$NPM" run test:archive-encryption "$NPM" run test:archive-encryption
cd /mnt/DATA/git/govoplan-wiki/webui cd "${WORKSPACE_ROOT}/govoplan-wiki/webui"
"$NPM" run test:interface-pattern "$NPM" run test:interface-pattern
# devkit-phase: module-ui end
}
# Validate every selected implementation before setup can create a temp npmrc.
while IFS= read -r phase_id; do
if ! declare -F "focused_phase_${phase_id//-/_}" >/dev/null; then
echo "check-focused: missing phase implementation: $phase_id" >&2
exit 2
fi
done <<< "$FOCUSED_SELECTION"
while IFS= read -r phase_id; do
(
focused_setup
"focused_phase_${phase_id//-/_}"
)
done <<< "$FOCUSED_SELECTION"
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env node
/** UI-01: contextual documentation belongs beside text, never in action slots. */
import { existsSync, readFileSync, readdirSync } from "node:fs";
import { createRequire } from "node:module";
import { relative, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const workspaceRoot = resolve(import.meta.dirname, "../../..");
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
const ts = require("typescript");
const titleOwners = new Set(["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]);
const interactiveOwners = new Set(["a", "button", "Button", "IconButton"]);
// Existing domain dialog adapter; its owning structural tests must retain
// forwarding to Core Dialog.titleHelp (not a module-local heading definition).
const domainTitleOwners = new Map([["FileDialog", "/govoplan-files/webui/src/"]]);
export function findDetachedDocumentation(sources) {
const files = new Map(sources.map(({ path, source }) => [resolve(path),
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
const host = ts.createCompilerHost(options);
host.getSourceFile = (path) => files.get(resolve(path));
const checker = ts.createProgram([...files.keys()], options, host).getTypeChecker();
const findings = [];
let links = 0;
for (const [path, source] of files) {
const identifiers = [];
const helpNodes = [];
function importedName(node) {
if (ts.isPropertyAccessExpression(node) && ts.isIdentifier(node.expression)) {
const namespace = checker.getSymbolAtLocation(node.expression)?.declarations?.find(ts.isNamespaceImport);
if (namespace) return node.name.text;
}
const declarations = checker.getSymbolAtLocation(node)?.declarations ?? [];
const declaration = declarations.find(ts.isImportSpecifier);
if (declaration) return (declaration.propertyName ?? declaration.name).text;
const defaultImport = declarations.find(ts.isImportClause);
if (defaultImport && ts.isStringLiteral(defaultImport.parent.moduleSpecifier)) {
const component = defaultImport.parent.moduleSpecifier.text.split("/").at(-1).replace(/\.[cm]?[jt]sx?$/, "");
if (component === "DocumentationHelpLink" || titleOwners.has(component) || component === "TextWithHelp" || interactiveOwners.has(component)) return component;
}
return node.getText(source);
}
function tag(node) {
const opening = ts.isJsxElement(node) ? node.openingElement : ts.isJsxSelfClosingElement(node) ? node : null;
return opening ? importedName(opening.tagName) : null;
}
function collect(node) {
if (ts.isIdentifier(node)) identifiers.push(node);
if ((ts.isJsxSelfClosingElement(node) || ts.isJsxElement(node)) && tag(node) === "DocumentationHelpLink") helpNodes.push(node);
ts.forEachChild(node, collect);
}
collect(source);
function staticallyHidden(opening) {
const hidden = opening.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "hidden");
if (!hidden) return false;
if (!hidden.initializer || ts.isStringLiteral(hidden.initializer)) return true;
return ts.isJsxExpression(hidden.initializer) && hidden.initializer.expression?.kind === ts.SyntaxKind.TrueKeyword;
}
// Reject text that is definitely absent while preserving dynamic translated
// titles and components whose rendered text cannot be established statically.
function emptyText(node, seen = new Set()) {
if (!node) return true;
if (ts.isJsxText(node)) return !node.getText(source).trim();
if (ts.isJsxExpression(node)) return emptyText(node.expression, seen);
if (ts.isStringLiteralLike(node)) return !node.text.trim();
if ([ts.SyntaxKind.NullKeyword, ts.SyntaxKind.FalseKeyword, ts.SyntaxKind.TrueKeyword].includes(node.kind) || ts.isVoidExpression(node)) return true;
if (ts.isParenthesizedExpression(node) || ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isNonNullExpression(node)) return emptyText(node.expression, seen);
if (ts.isIdentifier(node)) {
const symbol = checker.getSymbolAtLocation(node);
const declaration = symbol?.declarations?.find(ts.isVariableDeclaration);
const immutable = declaration && ts.isVariableDeclarationList(declaration.parent) && Boolean(declaration.parent.flags & ts.NodeFlags.Const);
if (immutable && declaration.initializer && !seen.has(symbol)) return emptyText(declaration.initializer, new Set(seen).add(symbol));
return node.text === "undefined" && !symbol?.declarations?.length;
}
if (ts.isConditionalExpression(node)) return emptyText(node.whenTrue, seen) && emptyText(node.whenFalse, seen);
if (ts.isJsxFragment(node)) return node.children.every((child) => emptyText(child, seen));
if (ts.isArrayLiteralExpression(node)) return node.elements.every((child) => emptyText(child, seen));
if (ts.isJsxElement(node) || ts.isJsxSelfClosingElement(node)) {
const opening = ts.isJsxElement(node) ? node.openingElement : node;
if (staticallyHidden(opening)) return true;
if (/^[a-z]/.test(opening.tagName.getText(source))) return !ts.isJsxElement(node) || node.children.every((child) => emptyText(child, seen));
}
return false;
}
function isAnchored(node, seen = new Set()) {
// Check the complete rendered ancestry before returning at a recognized
// slot: the entire heading/text contract may itself be inside a button.
for (let parent = node.parent; parent; parent = parent.parent) {
if (interactiveOwners.has(tag(parent))) return false;
const opening = ts.isJsxElement(parent) ? parent.openingElement : ts.isJsxSelfClosingElement(parent) ? parent : null;
if (opening && staticallyHidden(opening)) return false;
}
for (let parent = node.parent; parent; parent = parent.parent) {
if (ts.isJsxAttribute(parent)) {
const owner = parent.parent.parent;
const name = importedName(owner.tagName);
const slot = parent.name.getText(source);
if (slot === "titleHelp" && (titleOwners.has(name) || (domainTitleOwners.has(name) && path.includes(domainTitleOwners.get(name))))) {
if (name === "PageTitle") {
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
return Boolean(element?.children.some((child) => !emptyText(child)));
}
const title = owner.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "title");
return Boolean(title && !emptyText(title.initializer));
}
if (slot === "help" && name === "TextWithHelp") {
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
return Boolean(element?.children.some((child) => !emptyText(child)));
}
return false;
}
if (ts.isVariableDeclaration(parent) && ts.isIdentifier(parent.name)) {
const symbol = checker.getSymbolAtLocation(parent.name);
if (!symbol || seen.has(symbol)) return false;
const next = new Set(seen).add(symbol);
const references = identifiers.filter((identifier) => identifier !== parent.name && checker.getSymbolAtLocation(identifier) === symbol);
return references.length > 0 && references.every((reference) => isAnchored(reference, next));
}
}
return false;
}
for (const node of helpNodes) {
links += 1;
// FieldLabel is the central label+book implementation; its browser/component
// contract verifies sibling text and prevents nested interactive controls.
if (path.endsWith("/govoplan-core/webui/src/components/help/FieldLabel.tsx")) continue;
if (!isAnchored(node)) {
const position = source.getLineAndCharacterOfPosition(node.getStart(source));
findings.push({ path, line: position.line + 1, column: position.character + 1 });
}
}
}
return { findings, links };
}
function sourceFiles(directory) {
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
const path = resolve(directory, entry.name);
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
});
}
export function checkWorkspace(root = workspaceRoot) {
const modules = readdirSync(root, { withFileTypes: true })
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
const paths = modules.flatMap(sourceFiles);
const { findings, links } = findDetachedDocumentation(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
for (const finding of findings) {
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: UI-01 documentation must use a heading's titleHelp or TextWithHelp beside visible text, not an action slot or detached row.`);
}
if (!findings.length) console.log(`Heading-help contract passed: ${links} documentation links in ${paths.length} TSX files across ${modules.length} WebUI modules.`);
return findings.length ? 1 : 0;
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) process.exitCode = checkWorkspace();
@@ -4,6 +4,7 @@ set -euo pipefail
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}" ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
ROOT="$(cd "$ROOT" && pwd)" ROOT="$(cd "$ROOT" && pwd)"
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$ROOT")}"
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}" VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}" PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
NPM="${NPM:-/home/zemion/.nvm/versions/node/v22.22.3/bin/npm}" NPM="${NPM:-/home/zemion/.nvm/versions/node/v22.22.3/bin/npm}"
@@ -184,6 +185,7 @@ run_step "Validate installed module manifests and registry"
run_step "Validate platform interface and endpoint declarations" run_step "Validate platform interface and endpoint declarations"
"$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \ "$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \
--workspace-root "$WORKSPACE_ROOT" \
--strict-declarations \ --strict-declarations \
--strict-endpoints --strict-endpoints
+142
View File
@@ -0,0 +1,142 @@
{
"schema_version": 1,
"phases": [
{
"id": "preflight",
"title": "Dependency and shared contract preflight",
"cwd": "core",
"order_after": [],
"depends_on": [],
"resources": [
"backend:test-state"
],
"outputs": [],
"notes": [
"Checks dependency hygiene, cross-module contracts, manifests and static WebUI conventions."
]
},
{
"id": "tooling",
"title": "Workspace inventories and development/release tooling tests",
"cwd": "meta",
"order_after": [
"preflight"
],
"depends_on": [],
"resources": [
"backend:test-state",
"artifact:platform-inventory"
],
"outputs": [
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.json",
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.md"
],
"notes": [
"Inventory reports are regenerated here; no later phase consumes them. Reused check evidence does not promise these reports still exist."
]
},
{
"id": "backend",
"title": "Workspace syntax, backend modules and integration checks",
"cwd": "core",
"order_after": [
"tooling"
],
"depends_on": [],
"resources": [
"backend:test-state"
],
"outputs": [],
"notes": [
"Tests may create their own temporary fixtures or test caches; no generated artifact is required by a later phase."
]
},
{
"id": "core-ui",
"title": "Core source tests and selected component contracts",
"cwd": "core-webui",
"order_after": [
"backend"
],
"depends_on": [],
"resources": [
"webui:govoplan-core"
],
"outputs": [
"{core}/webui/.module-test-build"
],
"notes": [
"The selected component batch creates and removes a private compilation directory. Module-capability tests rebuild .module-test-build themselves; later phases do not consume it."
]
},
{
"id": "module-builds",
"title": "Optional-module build permutations and bundle checks",
"cwd": "core-webui",
"order_after": [
"core-ui"
],
"depends_on": [],
"resources": [
"webui:govoplan-core"
],
"outputs": [
"{core}/webui/dist",
"{core}/webui/dist/module-permutation-bundle-metrics.json"
],
"notes": [
"Each permutation builds and reads its own fresh bundle metrics. Final dist and aggregate metrics persist, but browser/module-ui phases do not consume them. Reused check evidence is not artifact or deployment attestation."
]
},
{
"id": "browser",
"title": "Core browser conformance",
"cwd": "core-webui",
"order_after": [
"module-builds"
],
"depends_on": [],
"resources": [
"webui:govoplan-core",
"browser:chromium",
"port:4174"
],
"outputs": [
"{core}/webui/test-results",
"{core}/webui/node_modules/.vite/govoplan-conformance"
],
"notes": [
"Playwright starts its own Vite server from conformance sources with an isolated cache. It does not serve or require module-builds dist. The server is owned by the phase and stopped on completion."
]
},
{
"id": "module-ui",
"title": "Owning-module UI and type checks",
"cwd": "access-webui",
"order_after": [
"browser"
],
"depends_on": [],
"resources": [
"webui:govoplan-core",
"webui:govoplan-access",
"webui:govoplan-payments",
"webui:govoplan-dataflow",
"webui:govoplan-datasources",
"webui:govoplan-workflow",
"webui:govoplan-dashboard",
"webui:govoplan-approvals",
"webui:govoplan-postbox",
"webui:govoplan-mail",
"webui:govoplan-files",
"webui:govoplan-campaign",
"webui:govoplan-policy",
"webui:govoplan-wiki"
],
"outputs": [],
"notes": [
"The original 19-command tail retains each owning package's cwd. Its scripts prepare any private test output they need; no earlier phase artifact is a prerequisite."
]
}
]
}
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env node
// Complement the existing i18n-marker inventory with known plain-text display
// slots. Parse source only: never execute module registration/catalog code.
import { existsSync, readFileSync, readdirSync, realpathSync } from "node:fs";
import { createRequire } from "node:module";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const displayProps = new Map([
...["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"].map((name) => [name, new Set(["title", "subtitle"])]),
["FieldLabel", new Set(["label"])], ["MetricCard", new Set(["label"])],
]);
const childOwners = new Set(["PageTitle", "TextWithHelp", "h1", "h2", "h3", "h4", "h5", "h6"]);
const unknown = Symbol("dynamic");
export function createReader(ts, allowedRoot) {
const sources = new Map();
function source(file) {
file = resolve(file);
if (file !== allowedRoot && !file.startsWith(`${resolve(allowedRoot)}/`)) return null;
if (!existsSync(file)) return null;
if (!realpathSync(file).startsWith(`${resolve(allowedRoot)}/`)) return null;
if (!sources.has(file)) sources.set(file, ts.createSourceFile(file, readFileSync(file, "utf8"), ts.ScriptTarget.Latest, true, file.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS));
return sources.get(file);
}
function unwrap(node) {
while (node && (ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isParenthesizedExpression(node))) node = node.expression;
return node;
}
function imported(sf, identifier) {
for (const item of sf.statements) {
if (!ts.isImportDeclaration(item) || !item.importClause || !ts.isStringLiteral(item.moduleSpecifier)) continue;
const names = item.importClause.namedBindings;
if (names && ts.isNamedImports(names)) {
const match = names.elements.find((entry) => entry.name.text === identifier);
if (match) return { path: item.moduleSpecifier.text, name: match.propertyName?.text ?? match.name.text };
}
if (item.importClause.name?.text === identifier) return { path: item.moduleSpecifier.text, name: "default" };
}
return null;
}
function targetFile(sf, specifier) {
if (!specifier.startsWith(".")) return null;
const base = resolve(dirname(sf.fileName), specifier);
return [base, `${base}.ts`, `${base}.tsx`, join(base, "index.ts")].find((file) => existsSync(file) && /\.tsx?$/.test(file)) ?? null;
}
function declaration(sf, name) {
for (const statement of sf.statements) {
if (ts.isVariableStatement(statement) && statement.declarationList.flags & ts.NodeFlags.Const) {
const item = statement.declarationList.declarations.find((node) => ts.isIdentifier(node.name) && node.name.text === name);
if (item) return item.initializer;
}
if (name === "default" && ts.isExportAssignment(statement)) return statement.expression;
}
return null;
}
function value(node, sf, seen = new Set()) {
node = unwrap(node);
if (!node) return unknown;
if (ts.isStringLiteralLike(node)) return node.text;
if (ts.isIdentifier(node)) {
const key = `${sf.fileName}:${node.text}`;
if (seen.has(key)) return unknown;
const visited = new Set([...seen, key]);
const local = declaration(sf, node.text);
if (local) return value(local, sf, visited);
const external = imported(sf, node.text);
const target = external && targetFile(sf, external.path);
const loaded = target && source(target);
return loaded ? value(declaration(loaded, external.name), loaded, visited) : unknown;
}
if (ts.isObjectLiteralExpression(node)) {
const result = {};
for (const property of node.properties) {
if (ts.isSpreadAssignment(property)) {
const spread = value(property.expression, sf, seen);
if (spread !== unknown && spread && typeof spread === "object") Object.assign(result, spread);
else result.__dynamicSpread = true;
} else if (ts.isPropertyAssignment(property)) {
result[property.name.text ?? property.name.getText(sf)] = value(property.initializer, sf, seen);
} else if (ts.isShorthandPropertyAssignment(property)) result[property.name.text] = value(property.name, sf, seen);
}
return result;
}
return unknown;
}
return { source, value, declaration, imported, unwrap };
}
function componentName(ts, reader, sf, node) {
const written = node.getText(sf);
if (/^h[1-6]$/.test(written)) return written;
if (ts.isIdentifier(node)) {
const imported = reader.imported(sf, written);
if (imported?.name === "default") return imported.path.split("/").at(-1).replace(/\.[tj]sx?$/, "");
return imported?.name ?? written;
}
if (ts.isPropertyAccessExpression(node)) {
const owner = node.expression.getText(sf);
const namespace = sf.statements.find((item) => ts.isImportDeclaration(item) && item.importClause?.namedBindings && ts.isNamespaceImport(item.importClause.namedBindings) && item.importClause.namedBindings.name.text === owner);
if (namespace) return node.name.text;
}
return written;
}
function sourceFiles(directory) {
if (!existsSync(directory)) return [];
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
if (entry.name.startsWith(".") || entry.name === "node_modules") return [];
const file = join(directory, entry.name);
if (entry.isDirectory()) return sourceFiles(file);
return entry.isFile() && /\.[tj]sx?$/.test(file) ? [file] : [];
});
}
export function auditRepository(ts, repositoryRoot, coreRoot) {
const reader = createReader(ts, repositoryRoot);
const coreReader = createReader(ts, coreRoot);
const coreFile = coreReader.source(join(coreRoot, "webui/src/i18n/generatedTranslations.ts"));
const coreCatalog = coreFile ? coreReader.value(coreReader.declaration(coreFile, "generatedTranslations"), coreFile) : {};
const moduleFile = reader.source(join(repositoryRoot, "webui/src/module.ts"));
let moduleCatalog = {};
let registration = repositoryRoot === coreRoot ? "core-default" : "absent";
if (moduleFile) {
for (const statement of moduleFile.statements) {
if (!ts.isVariableStatement(statement) || !statement.modifiers?.some((item) => item.kind === ts.SyntaxKind.ExportKeyword)) continue;
for (const node of statement.declarationList.declarations) {
if (!node.initializer || !(/PlatformWebModule/.test(node.type?.getText(moduleFile) ?? "") || /Module$/.test(node.name.getText(moduleFile)))) continue;
const evaluated = reader.value(node.initializer, moduleFile);
if (evaluated && typeof evaluated === "object" && Object.hasOwn(evaluated, "translations")) {
moduleCatalog = evaluated.translations;
registration = moduleCatalog !== unknown && moduleCatalog && typeof moduleCatalog === "object" &&
!moduleCatalog.__dynamicSpread && !moduleCatalog.en?.__dynamicSpread && !moduleCatalog.de?.__dynamicSpread ? "registered" : "dynamic";
}
}
}
}
const findings = [], review = [], labels = [];
const add = (text, node, sf, slot) => {
const position = sf.getLineAndCharacterOfPosition(node.getStart(sf));
const location = { file: sf.fileName, line: position.line + 1, slot };
if (typeof text !== "string") {
review.push({ ...location, code: "dynamic-display-slot", message: "Runtime data or computed text: verify with the owning module/locale context." });
return;
}
text = text.replace(/\s+/g, " ").trim();
if (!text || text.startsWith("i18n:") || !/[\p{L}]/u.test(text)) return;
const missing = ["en", "de"].filter((locale) => {
const translated = moduleCatalog?.[locale]?.[text] ?? coreCatalog?.[locale]?.[text];
return typeof translated !== "string" || !translated.trim();
});
labels.push({ ...location, text, missing_locales: missing });
if (missing.length) {
const item = { ...location, code: "plain-label-missing-translation", text, missing_locales: missing, registration };
if (registration === "dynamic") review.push({ ...item, code: "dynamic-catalog-review" });
else findings.push(item);
}
};
for (const file of sourceFiles(join(repositoryRoot, "webui/src"))) {
if (file.includes("/i18n/")) continue;
const sf = reader.source(file);
function textChild(node, owner) {
if (ts.isJsxText(node)) add(node.text, node, sf, `${owner}.children`);
else if (ts.isJsxExpression(node)) { if (node.expression) add(reader.value(node.expression, sf), node, sf, `${owner}.children`); }
else if (ts.isJsxElement(node) || ts.isJsxFragment(node)) for (const child of node.children) textChild(child, owner);
}
function visit(node) {
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
const owner = componentName(ts, reader, sf, node.tagName);
for (const property of node.attributes.properties) {
if (!ts.isJsxAttribute(property) || !displayProps.get(owner)?.has(property.name.text) || !property.initializer) continue;
const expression = ts.isJsxExpression(property.initializer) ? property.initializer.expression : property.initializer;
add(reader.value(expression, sf), property, sf, `${owner}.${property.name.text}`);
}
}
if (ts.isJsxElement(node)) {
const owner = componentName(ts, reader, sf, node.openingElement.tagName);
if (childOwners.has(owner)) for (const child of node.children) textChild(child, owner);
}
ts.forEachChild(node, visit);
}
visit(sf);
}
const hasCatalog = sourceFiles(join(repositoryRoot, "webui/src/i18n")).some((file) => /Translations\.ts$/.test(file));
if (hasCatalog && registration === "absent") findings.push({ file: moduleFile?.fileName ?? join(repositoryRoot, "webui/src/module.ts"), line: 1, code: "catalog-not-registered", message: "Module-owned catalog exists but no static module translations registration was found." });
return { repository: repositoryRoot, registration, labels, findings, review };
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
const args = process.argv.slice(2);
let workspace = resolve(dirname(fileURLToPath(import.meta.url)), "../../../..");
const repos = [];
for (let index = 0; index < args.length; index++) {
if (args[index] === "--workspace-root") workspace = resolve(args[++index]);
else if (args[index] === "--repo") repos.push(args[++index]);
else throw new Error(`Unknown argument: ${args[index]}`);
}
const core = join(workspace, "govoplan-core");
const require = createRequire(join(core, "webui/package.json"));
const ts = require("typescript");
const catalog = JSON.parse(readFileSync(join(workspace, "govoplan/repositories.json"), "utf8"));
const selected = catalog.repositories.filter((repo) => !repos.length || repos.includes(repo.name));
if (repos.some((name) => !selected.some((repo) => repo.name === name))) throw new Error("Unknown repository selection");
for (const repo of selected) {
if (typeof repo.path !== "string" || !resolve(workspace, repo.path).startsWith(`${workspace}/`)) throw new Error("Repository path escapes the workspace");
}
const results = selected.filter((repo) => existsSync(join(workspace, repo.path, "webui/src"))).map((repo) => auditRepository(ts, join(workspace, repo.path), core));
const findings = results.reduce((total, item) => total + item.findings.length, 0);
process.stdout.write(JSON.stringify({ schema_version: 1, results, finding_count: findings,
limitations: ["Known display slots only; this is not a complete UI or linguistic review.", "Runtime data, computed labels and dynamic registrations require manual review.", "Core defaults plus each owning module are checked; optional sibling catalogs cannot mask missing registration.", "Explicit i18n markers are checked by the existing platform interface inventory."] }) + "\n");
process.exitCode = findings ? 1 : 0;
}
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env python3
"""Command-line entry point; use the repository-root ./devkit launcher."""
from govoplan_devkit.cli import main
if __name__ == "__main__":
raise SystemExit(main())
+32
View File
@@ -0,0 +1,32 @@
{
"schema_version": 1,
"name": "Example Python project",
"repositories": [{"name": "app", "path": "."}],
"checks": [
{
"id": "whitespace",
"title": "Git whitespace check",
"argv": ["git", "diff", "--check"],
"cwd": ".",
"repos": ["app"],
"inputs": {"repos": ["app"]},
"timeout_seconds": 30
},
{
"id": "unit-tests",
"title": "Python unit tests",
"argv": ["{python}", "-m", "unittest", "discover", "-s", "tests"],
"cwd": ".",
"repos": ["app"],
"inputs": {"repos": ["app"]},
"after": ["whitespace"],
"resources": ["test-database"],
"timeout_seconds": 300
}
],
"profiles": {
"quick": ["whitespace", "unit-tests"],
"backend": ["unit-tests"],
"full": ["whitespace", "unit-tests"]
}
}
+3
View File
@@ -0,0 +1,3 @@
"""Small, deterministic development commands for people, CI and coding agents."""
__version__ = "1.0.0"
+566
View File
@@ -0,0 +1,566 @@
"""Test planning over existing checks; planning never executes a test or server."""
from __future__ import annotations
from pathlib import Path
from copy import deepcopy
from itertools import islice
import re
from .common import read_json
from .package_tests import declared_tests, discovered_sources, source_stage_id
PROFILES = ("quick", "ui", "backend", "full")
def stage(
identity: str,
title: str,
argv: list[str],
cwd: Path,
*,
reason: str,
deps: list[str] | None = None,
after: list[str] | None = None,
resources: list[str] | None = None,
timeout_seconds: int = 300,
) -> dict:
return {
"id": identity,
"title": title,
"argv": argv,
"cwd": str(cwd),
"deps": deps or [],
"after": after or [],
"resources": resources or [],
"timeout_seconds": timeout_seconds,
"reason": reason,
}
def _custom_stages(
project, workspace_root: Path, profile: str, selected, filtered: bool
) -> list[dict]:
# load_project validates every nested declaration before selection.
records = project.config.get("checks", [])
checks = {item["id"]: item for item in records}
profiles = project.config.get("profiles", {})
if profile not in profiles:
raise ValueError(
f"Project {project.name!r} does not declare profile {profile!r}"
)
selected_names = {repo.name for repo in selected}
wanted = set()
def include(identity: str, visiting: frozenset[str] = frozenset()) -> None:
if identity not in checks:
raise ValueError(f"Unknown check dependency: {identity}")
if identity in visiting:
raise ValueError(f"Cyclic check dependency: {identity}")
if identity in wanted:
return
for dependency in [
*checks[identity].get("deps", []),
*checks[identity].get("after", []),
]:
include(dependency, visiting | {identity})
wanted.add(identity)
for identity in profiles[profile]:
if identity not in checks:
raise ValueError(f"Unknown profile check: {identity}")
owned = set(checks[identity].get("repos", []))
if not filtered or not owned or selected_names & owned:
include(identity)
result = []
# A dependency may precede/follow its consumer in the configuration: the
# execution engine owns scheduling, not JSON declaration order.
for identity, item in checks.items():
if identity in wanted:
result.append(
stage(
identity,
item.get("title", identity),
list(item["argv"]),
workspace_root / item.get("cwd", "."),
deps=list(item.get("deps", [])),
after=list(item.get("after", [])),
resources=list(item.get("resources", [])),
timeout_seconds=item.get("timeout_seconds", 300),
reason=f"Project profile {profile}",
)
)
for field in ("inputs", "reuse"):
if field in item:
result[-1][field] = deepcopy(item[field])
return result
def focused_phases(meta: Path) -> list[dict]:
"""Read the same bounded, ordered phase metadata as the standalone gate."""
value = read_json(meta / "tools/checks/focused-phases.json", max_bytes=1024 * 1024)
if (
not isinstance(value, dict)
or set(value) != {"schema_version", "phases"}
or type(value["schema_version"]) is not int
or value["schema_version"] != 1
):
raise ValueError("Unsupported focused phase metadata schema")
phases = value["phases"]
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
raise ValueError("Focused phase catalog requires 132 definitions")
seen = set()
fields = {
"id",
"title",
"cwd",
"order_after",
"depends_on",
"resources",
"outputs",
"notes",
}
for phase in phases:
if not isinstance(phase, dict) or set(phase) != fields:
raise ValueError("Invalid focused phase metadata fields")
identity = phase["id"]
if (
not isinstance(identity, str)
or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity)
or identity in seen
):
raise ValueError("Invalid or duplicate focused phase ID")
if (
not isinstance(phase["title"], str)
or not phase["title"].strip()
or len(phase["title"]) > 256
or any(ord(char) < 32 for char in phase["title"])
):
raise ValueError("Invalid focused phase title")
if not isinstance(phase["cwd"], str) or phase["cwd"] not in {
"core",
"meta",
"core-webui",
"access-webui",
}:
raise ValueError("Unsupported focused phase working directory")
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
values = phase[field]
if (
not isinstance(values, list)
or len(values) > 64
or any(
not isinstance(item, str)
or not item
or len(item) > 4096
or "\0" in item
for item in values
)
or len(set(values)) != len(values)
):
raise ValueError("Invalid focused phase list: " + field)
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
raise ValueError("Focused prerequisites must precede their consumer")
seen.add(identity)
return phases
def focused_phase_bodies(text: str, phases: list[dict]) -> dict[str, str]:
"""Only exact top-level registered wrappers are authoritative phase bodies.
Do not extract lookalike markers from heredocs, conditionals or unrelated
shell functions. This recognizes the maintained wrapper convention, not
arbitrary executable shell semantics.
"""
registered = {
"focused_phase_" + phase["id"].replace("-", "_"): phase["id"]
for phase in phases
}
lines, bodies, index, depth, heredoc = text.splitlines(), {}, 0, 0, None
while index < len(lines):
line = lines[index]
stripped = line.strip()
if heredoc is not None:
if stripped == heredoc:
heredoc = None
index += 1
continue
match = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
if match:
heredoc = match[1]
index += 1
continue
function = re.fullmatch(r"(focused_phase_[a-z0-9_]+)\(\) \{", line)
if depth == 0 and function and function[1] in registered:
identity = registered[function[1]]
if (
identity in bodies
or index + 1 >= len(lines)
or lines[index + 1] != f"# devkit-phase: {identity} begin"
):
raise ValueError("Invalid or duplicate focused phase wrapper")
end = index + 2
while end < len(lines) and lines[end] != f"# devkit-phase: {identity} end":
end += 1
if end + 1 >= len(lines) or lines[end + 1] != "}":
raise ValueError("Unclosed focused phase wrapper")
bodies[identity] = "\n".join(lines[index + 2 : end]) + "\n"
index = end + 2
continue
if re.match(
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
stripped,
):
depth += 1
elif re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
depth = max(0, depth - 1)
index += 1
if set(bodies) != {phase["id"] for phase in phases}:
raise ValueError("Focused phase metadata and marked implementations differ")
return {phase["id"]: bodies[phase["id"]] for phase in phases}
def _undeclared_source_note(project, workspace_root: Path) -> str | None:
"""Directory discovery is wider than registered Git input ownership."""
registered_paths = {repo.path.resolve() for repo in project.repositories}
children = list(islice(workspace_root.iterdir(), 4097))
if len(children) > 4096:
return "Workspace discovery exceeds its bounded ownership audit; native reuse is disabled."
for child in children:
if not child.name.startswith("govoplan") or child.resolve() in registered_paths:
continue
if any(
(child / name).exists() or (child / name).is_symlink()
for name in ("src", "webui")
):
return "Unregistered sibling src/WebUI inputs may be consumed by workspace discovery or PYTHONPATH; native reuse is disabled until their repository ownership is declared."
return None
def _apply_undeclared_source_note(checks: list[dict], note: str | None) -> None:
if note:
for check in checks:
check["reuse"] = "never"
check.setdefault("coverage_notes", []).append(note)
def _focused_ui_inputs(project) -> tuple[list[str] | None, str]:
names = {repo.name for repo in project.repositories}
if not {"govoplan", "govoplan-core"} <= names:
return None, "Missing Core/Meta ownership; input scope stays workspace-wide."
selected = {"govoplan", "govoplan-core"}
for repo in project.repositories:
root, webui = repo.path, repo.path / "webui"
if (
not root.is_dir()
or root.is_symlink()
or webui.is_symlink()
or webui.exists()
and not webui.is_dir()
):
return (
None,
"Missing or ambiguous repository/WebUI layout; input scope stays workspace-wide.",
)
if webui.is_dir():
selected.add(repo.name)
return (
sorted(selected),
"UI input scope includes whole Core/Meta and every registered WebUI repository, including helpers/configuration; it is not per-file dependency inference.",
)
def _apply_ui_inputs(check: dict, scope: tuple[list[str] | None, str]) -> None:
names, note = scope
if names is not None:
check["inputs"] = {"repos": list(names)}
check.setdefault("coverage_notes", []).append(note)
def _full_stages(project, workspace_root: Path, meta: Path, core: Path) -> list[dict]:
phases = focused_phases(meta)
ui_scope = _focused_ui_inputs(project)
directories = {
"core": core,
"meta": meta,
"core-webui": core / "webui",
"access-webui": workspace_root / "govoplan-access/webui",
}
checks = []
for phase in phases:
check = stage(
"focused." + phase["id"],
phase["title"],
[
"bash",
str(meta / "tools/checks/check-focused.sh"),
"--phase",
phase["id"],
],
directories[phase["cwd"]],
reason="Full retains every canonical phase in order; repository filters never narrow the required gate.",
deps=["focused." + value for value in phase["depends_on"]],
after=["focused." + value for value in phase["order_after"]],
resources=list(dict.fromkeys(["workspace:focused", *phase["resources"]])),
timeout_seconds=14400,
)
check["coverage_notes"] = list(phase["notes"])
check["phase_outputs"] = list(phase["outputs"])
if phase["id"] in {"core-ui", "module-builds", "browser", "module-ui"}:
check["resources"] = list(
dict.fromkeys(
[
*check["resources"],
*[f"webui:{repo.name}" for repo in project.repositories],
]
)
)
_apply_ui_inputs(check, ui_scope)
else:
check["coverage_notes"].append(
"Cross-module backend/tooling checks retain conservative whole-workspace inputs."
)
checks.append(check)
_apply_undeclared_source_note(
checks, _undeclared_source_note(project, workspace_root)
)
return checks
def _expanded_repositories(project, selected, *, changed: bool):
"""Conservative shared changes; declared interface consumers otherwise.
This is a selection aid, not a claim of exhaustive runtime dependency
analysis. The full profile always retains the canonical workspace gate.
"""
if not changed or not selected:
return selected, "explicit selection" if selected else "no changed repositories"
names = {repo.name for repo in selected}
if names & {"govoplan", "govoplan-core"}:
return list(
project.repositories
), "Core/Meta changed; all registered consumers conservatively selected"
# Reuse the release contract parser without importing module application
# code. If available declarations cannot be parsed, broaden selection.
import sys
meta = next(
(repo.path for repo in project.repositories if repo.name == "govoplan"), None
)
if meta is None:
return selected, "changed repositories; no GovOPlaN contract catalog"
release = meta / "tools" / "release"
if not (release / "govoplan_release" / "contracts.py").is_file():
return selected, "changed repositories; contract parser unavailable"
sys.path.insert(0, str(release))
try:
from govoplan_release.contracts import parse_manifest_contract
contracts = []
for repo in project.repositories:
for manifest in sorted((repo.path / "src").glob("*/backend/manifest.py")):
parsed = parse_manifest_contract(manifest, repo_name=repo.name)
if parsed is None:
return list(
project.repositories
), "unresolved manifest contract; conservative workspace selection"
contracts.append(parsed)
while True:
providers = {
item.name
for contract in contracts
if contract.repo in names
for item in contract.provides_interfaces
}
consumers = {
contract.repo
for contract in contracts
if any(item.name in providers for item in contract.requires_interfaces)
}
added = consumers - names
if not added:
break
names.update(added)
except (ImportError, AttributeError, OSError, SyntaxError, ValueError):
return list(
project.repositories
), "contract analysis unavailable; conservative workspace selection"
finally:
sys.path.remove(str(release))
return [
repo for repo in project.repositories if repo.name in names
], "changed repositories plus declared interface consumers"
def _module_ui_plan(repo, *, reason: str) -> tuple[list[dict], list[str]]:
"""Use package-owned direct Node test metadata, excluding shell/build chains.
Source structural scripts are also discoverable by the existing established
names. Unknown shell commands are deliberately not guessed or rewritten.
"""
webui = repo.path / "webui"
package_path = webui / "package.json"
if not package_path.is_file():
return [], []
scripts: dict[tuple[str, ...], list[str]] = {}
omitted = []
for item in [*declared_tests(repo, package_path), *discovered_sources(repo)]:
if item["component_suite"] is not None:
omitted.append(
f"{repo.name} {item['name']}: only covered by the explicit UI component batch; quick does not compile components"
)
elif item["name"] in {"test:module-permutations", "test:vite-cache-isolation"}:
omitted.append(
f"{repo.name} {item['name']}: separate environment/permutation verification, not run by this scoped profile"
)
elif item["_argv"]:
scripts[tuple(item["_argv"])] = item["_argv"]
else:
omitted.append(f"{repo.name} {item['name']}: {item['reason']}")
return [
stage(
source_stage_id(repo.name, argv),
f"{repo.name}: {Path(argv[-1]).stem}",
argv,
webui,
reason=reason,
resources=[f"webui:{repo.name}"],
)
for _, argv in sorted(scripts.items())
], omitted
def module_ui_stages(repo, *, reason: str) -> list[dict]:
return _module_ui_plan(repo, reason=reason)[0]
def build_stages(
workspace_root: Path,
profile: str,
repos: list[str],
changed: bool,
project: Path | None = None,
) -> list[dict]:
from .workspace import load_project, selected_repositories
if profile not in PROFILES:
raise ValueError(f"Unknown check profile: {profile}")
workspace_root = workspace_root.resolve()
loaded = load_project(workspace_root, project)
selected = selected_repositories(loaded, repos, changed=changed)
if project is not None:
return _custom_stages(
loaded, workspace_root, profile, selected, bool(repos or changed)
)
selected, reason = _expanded_repositories(loaded, selected, changed=changed)
mapping = {repo.name: repo.path for repo in loaded.repositories}
meta = mapping.get("govoplan", workspace_root / "govoplan")
core = mapping.get("govoplan-core", workspace_root / "govoplan-core")
if profile == "full":
return _full_stages(loaded, workspace_root, meta, core)
if changed and not selected:
return []
checks = []
for identity, command in (
(
"contracts",
[
"{python}",
str(meta / "tools/checks/check-contracts.py"),
"--workspace-root",
str(workspace_root),
"--no-impact",
],
),
(
"manifests",
[
"{python}",
str(meta / "tools/checks/check-manifest-shapes.py"),
"--workspace-root",
str(workspace_root),
"--require-architecture",
],
),
):
checks.append(
stage(
identity,
f"Workspace {identity}",
command,
meta,
reason="Shared manifest/interface invariants",
timeout_seconds=600,
)
)
if profile in {"quick", "ui"}:
ui_scope = _focused_ui_inputs(loaded)
coverage_notes = [
"Scoped source/component checks are not a complete module review or the full focused gate."
]
for identity in ("jsx-value-imports", "heading-help"):
checks.append(
stage(
identity,
f"Shared {identity} contract",
["{node}", str(meta / f"tools/checks/check-{identity}.mjs")],
meta,
reason="Existing source-only cross-module guard",
)
)
_apply_ui_inputs(checks[-1], ui_scope)
for repo in selected:
stages, omissions = _module_ui_plan(repo, reason=reason)
checks.extend(stages)
coverage_notes.extend(omissions)
checks[0]["coverage_notes"] = coverage_notes
if profile == "ui":
checks.append(
stage(
"core.component-batch",
"Core component suites (compile once)",
["{node}", str(core / "webui/scripts/run-component-tests.mjs")],
core / "webui",
reason="Shared components affect every UI consumer",
timeout_seconds=900,
)
)
_apply_ui_inputs(checks[-1], ui_scope)
if profile == "backend":
for repo in selected:
if (repo.path / "tests").is_dir():
checks.append(
stage(
f"{repo.name}.backend",
f"{repo.name} backend tests",
["{python}", "-m", "pytest", "-q", str(repo.path / "tests")],
repo.path,
reason=reason,
resources=["backend:test-state"],
timeout_seconds=1800,
)
)
_apply_undeclared_source_note(
checks, _undeclared_source_note(loaded, workspace_root)
)
return checks
def build_coverage(
workspace_root: Path,
profile: str,
repos: list[str],
changed: bool,
project: Path | None = None,
*,
stages: list[dict] | None = None,
) -> dict:
"""Read-only suite inventory; prebuilt stages avoid repeating selection queries."""
from .coverage import coverage_inventory
if profile not in PROFILES:
raise ValueError(f"Unknown check profile: {profile}")
if stages is None:
stages = build_stages(workspace_root, profile, repos, changed, project)
return coverage_inventory(workspace_root, profile, project, stages)
+244
View File
@@ -0,0 +1,244 @@
"""Independently verified check results; never an artifact/build-output cache."""
from __future__ import annotations
from copy import deepcopy
import threading
import re
from .common import digest, now
from .inputs import InputSnapshotter, FINGERPRINT_VERSION
CHECKPOINT_VERSION = 1
def validate_checkpoint_receipt(receipt):
"""New checkpoints are explicit; legacy status alone never certifies a phase."""
version = receipt.get("fingerprint_version")
if version is None:
return
if version != FINGERPRINT_VERSION:
raise ValueError("Unsupported check fingerprint version")
for stage in receipt["stages"]:
verified = stage.get("checkpoint_verified")
if type(verified) is not bool:
raise ValueError("Stage checkpoint verification must be boolean")
if stage["status"] == "passed" and not verified:
raise ValueError(
"Passing stage requires an independently verified checkpoint"
)
if not verified:
continue
if (
stage["status"] != "passed"
or stage.get("checkpoint_version") != CHECKPOINT_VERSION
):
raise ValueError("Invalid verified checkpoint state or version")
for field in (
"cache_key",
"input_fingerprint",
"stage_plan_fingerprint",
"log_sha256",
):
if not isinstance(stage.get(field), str) or not re.fullmatch(
r"[a-f0-9]{64}", stage[field]
):
raise ValueError(
"Verified checkpoint requires bounded content identities"
)
if (
not isinstance(stage.get("checkpoint_at"), str)
or not stage["checkpoint_at"]
):
raise ValueError("Verified checkpoint requires its recording time")
scope = stage.get("input_scope")
if (
not isinstance(scope, dict)
or scope.get("version") != 1
or scope.get("kind") not in {"workspace", "repositories"}
or type(scope.get("declared")) is not bool
):
raise ValueError("Verified checkpoint requires a versioned input scope")
names = scope.get("repos")
if (
not isinstance(names, list)
or not 1 <= len(names) <= 256
or any(not isinstance(name, str) for name in names)
or len(set(names)) != len(names)
):
raise ValueError(
"Verified checkpoint requires bounded repository identities"
)
class Checkpoints:
def __init__(self, project, workspace, plan, environment_probe, cancelled):
self.scanner = InputSnapshotter(project, workspace_root=workspace)
self.plan = {stage["id"]: stage for stage in plan}
self.environment_probe = environment_probe
self.cancelled = cancelled
self.lock = threading.RLock()
self.initial = None
self.environment = None
def source(self, stages=None):
with self.lock:
return self.scanner.snapshot(
list(self.plan.values()) if stages is None else stages
)
def probe_environment(self):
value = self.environment_probe()
self.check_cancelled()
return value
def check_cancelled(self):
if self.cancelled.is_set():
raise InterruptedError("Check cancelled during input verification")
def initialize(self):
self.initial = self.source()
self.check_cancelled()
self.environment = self.probe_environment()
return self.initial, self.environment
def closure(self, stage):
selected = {}
def include(item):
if item["id"] in selected:
return
selected[item["id"]] = item
for identity in item["deps"]:
include(self.plan[identity])
include(self.plan[stage["id"]])
return [selected[key] for key in sorted(selected)]
def identity(self, stage):
with self.lock:
closure = self.closure(stage)
snapshot = self.source(closure)
self.check_cancelled()
environment = self.probe_environment()
own = snapshot["stages"][stage["id"]]
key = digest(
{
"checkpoint_version": CHECKPOINT_VERSION,
"inputs": {
identity: value["fingerprint"]
for identity, value in snapshot["stages"].items()
},
"environment": environment,
}
)
return {
"checkpoint_version": CHECKPOINT_VERSION,
"cache_key": key,
"input_fingerprint": own["fingerprint"],
"input_scope": own["scope"],
"stage_plan_fingerprint": own["plan_fingerprint"],
"stage_environment_fingerprint": environment,
"dependency_input_fingerprints": {
item["id"]: snapshot["stages"][item["id"]]["fingerprint"]
for item in closure
if item["id"] != stage["id"]
},
}
def prepare(self, stage, prior, allow_reuse, verify_log):
before = self.identity(stage)
reason = "No previous verified checkpoint"
if stage.get("reuse", "verified") == "never":
reason = (
"This stage explicitly disables reuse (outputs/setup must be recreated)"
)
elif not allow_reuse:
reason = "A data dependency ran again; its consumers must run again"
elif (
prior
and prior.get("status") == "passed"
and prior.get("checkpoint_verified") is True
and prior.get("checkpoint_version") == CHECKPOINT_VERSION
):
if prior.get("cache_key") == before["cache_key"]:
# Receipt command text is never executed. Only the freshly planned
# stage runs; a cached log must independently match its content hash.
verify_log(prior)
result = {
**before,
**{
key: deepcopy(prior[key])
for key in (
"status",
"exit_code",
"duration_seconds",
"log_path",
"log_sha256",
"output_truncated",
"omitted_output_bytes",
"checkpoint_at",
)
if key in prior
},
}
result.update(
checkpoint_verified=True,
reuse_reason="Verified checkpoint matches current inputs, command, dependencies and environment",
)
return before, result
reason = (
"Declared inputs, command, dependency inputs or environment changed"
)
before["reuse_reason"] = reason
return before, None
def finish(self, stage, before, result):
result = {**result, **before, "checkpoint_verified": False}
if result["status"] != "passed" or result.get("exit_code") != 0:
return result
after = self.identity(stage)
if before["cache_key"] != after["cache_key"]:
result.update(
status="stale",
error="Stage inputs or environment changed during execution; no reusable checkpoint was recorded",
)
else:
result.update(checkpoint_verified=True, checkpoint_at=now())
return result
def finalize(self, stages):
final = self.source()
self.check_cancelled()
environment = self.probe_environment()
valid = (
final["observed_source_fingerprint"]
== self.initial["observed_source_fingerprint"]
and environment == self.environment
)
for stage in stages:
if stage["status"] != "passed":
valid = valid and stage["status"] != "stale"
continue
closure = self.closure(self.plan[stage["id"]])
key = digest(
{
"checkpoint_version": CHECKPOINT_VERSION,
"inputs": {
item["id"]: final["stages"][item["id"]]["fingerprint"]
for item in closure
},
"environment": environment,
}
)
if (
stage.get("checkpoint_verified") is not True
or stage.get("cache_key") != key
):
stage.update(
status="stale",
checkpoint_verified=False,
error="Final inputs no longer match this checkpoint",
)
valid = False
return valid, final, environment
+212
View File
@@ -0,0 +1,212 @@
"""A compact command catalog over maintained project tools."""
from __future__ import annotations
import argparse
import importlib
import json
from pathlib import Path
import sys
from . import __version__
from .common import META_ROOT, redact, safe_output
COMMAND_MODULES = (
"context",
"doctor",
"runner",
"review",
"docs",
"issues",
"release",
"maintenance",
)
def main(argv: list[str] | None = None) -> int:
argv = sys.argv[1:] if argv is None else argv
common = argparse.ArgumentParser(add_help=False, allow_abbrev=False)
common.add_argument(
"--workspace-root",
type=Path,
default=META_ROOT.parent,
help="Directory containing registered repositories",
)
common.add_argument(
"--project", type=Path, help="Explicit trusted portable-project JSON manifest"
)
common.add_argument(
"--state-dir",
type=Path,
help="Private evidence-state base (scoped again by workspace)",
)
common.add_argument("--format", choices=("summary", "json"), default="summary")
common.add_argument(
"--quiet",
action="store_true",
help="Suppress live progress on stderr; retain the final result",
)
common.add_argument(
"--json",
dest="format",
action="store_const",
const="json",
help="Alias for --format json",
)
parser = argparse.ArgumentParser(
prog="devkit",
description="Deterministic development workflows. Read-only previews by default for remote writes.",
parents=[common],
allow_abbrev=False,
)
parser.add_argument("--version", action="version", version=f"devkit {__version__}")
subparsers = parser.add_subparsers(dest="command", required=True)
for name in COMMAND_MODULES:
importlib.import_module("." + name, __package__).register(subparsers)
catalog = subparsers.add_parser("commands", help="Show the compact command catalog")
def commands(_):
entries = [
{
"command": "context",
"purpose": "Offline repository changes, ownership and instructions",
"effects": "read only",
},
{
"command": "doctor",
"purpose": "Local tool/dependency preflight and repair guidance",
"effects": "read only",
},
{
"command": "check",
"purpose": "Registered verification profiles with logs and source-bound receipts",
"effects": "tests/builds; --dry-run previews",
},
{
"command": "runs / latest / status / summary / logs",
"purpose": "Find runs and read progress, compact results and bounded live/final logs",
"effects": "read only",
},
{
"command": "coverage",
"purpose": "Explain declared suite coverage and exclusions for a check profile",
"effects": "read only",
},
{
"command": "resume / recover",
"purpose": "Resume verified identical work or recover an abandoned check run",
"effects": "local checks/state only",
},
{
"command": "review",
"purpose": "Module UI-review inventory and manual evidence checklist",
"effects": "local bundle only",
},
{
"command": "docs",
"purpose": "Existing documentation and translation audits",
"effects": "local checks/evidence",
},
{
"command": "issues",
"purpose": "Preview and explicitly publish deduplicated issue evidence",
"effects": "remote only with --apply",
},
{
"command": "release",
"purpose": "Existing durable release lifecycle, receipts and confirmations",
"effects": "explicit --apply and step confirmation",
},
{
"command": "git",
"purpose": "Frozen explicit-path commit and branch-push maintenance",
"effects": "explicit --apply; no bulk staging, force or tags",
},
]
return {
"commands": entries,
"summary": [
f"{item['command']}: {item['purpose']} ({item['effects']})"
for item in entries
],
}
catalog.set_defaults(handler=commands)
# Global flags work before or after the command, without copying defaults to every parser.
global_args, remaining = common.parse_known_args(argv)
args = parser.parse_args(remaining, namespace=global_args)
args.workspace_root = args.workspace_root.expanduser().resolve()
if args.project:
args.project = args.project.expanduser().absolute()
def progress(event):
if args.quiet:
return
if args.format == "json":
print(
json.dumps(safe_output(event), sort_keys=True, allow_nan=False),
file=sys.stderr,
flush=True,
)
else:
counts = event["counts"]
completed = sum(
count
for state, count in counts.items()
if state not in {"pending", "running"}
)
active = ", ".join(event["active_stages"])
print(
redact(
f"Run {event['run_id']}: {event['phase']} · {completed}/{event['total_stages']} stages · {event['elapsed_seconds']}s"
+ (f" · {active}" if active else "")
),
file=sys.stderr,
flush=True,
)
args.on_progress = progress
try:
result = args.handler(args)
if not isinstance(result, dict):
raise ValueError("Command did not return a result object")
code = int(result.pop("_exit_code", 0))
if args.format == "json":
print(
json.dumps(
safe_output(result),
sort_keys=True,
indent=2,
ensure_ascii=True,
allow_nan=False,
)
)
else:
summary = result.get("summary", [str(result.get("status", "Completed"))])
print(
"\n".join(redact(str(line)) for line in summary)
if isinstance(summary, list)
else redact(str(summary))
)
return code
except (ValueError, OSError, RuntimeError, ImportError) as exc:
message = redact(str(exc))
if args.format == "json":
print(
json.dumps(
{
"status": "error",
"error": message,
"error_type": type(exc).__name__,
}
)
)
else:
print(f"devkit: {message}", file=sys.stderr)
return 2
except KeyboardInterrupt:
print(
"devkit: interrupted; inspect the saved run before retrying",
file=sys.stderr,
)
return 130
+232
View File
@@ -0,0 +1,232 @@
"""Bounded local records and predictable output; no network or AI dependency."""
from __future__ import annotations
from contextlib import contextmanager
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import stat
import tempfile
import time
META_ROOT = Path(__file__).resolve().parents[3]
MAX_JSON_BYTES = 8 * 1024 * 1024
IDENTIFIER = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}\Z")
def now() -> str:
return datetime.now(timezone.utc).isoformat()
def canonical(value: object) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True, allow_nan=False
).encode()
def digest(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def identifier(value: str) -> str:
if (
not isinstance(value, str)
or not IDENTIFIER.fullmatch(value)
or value in {".", ".."}
):
raise ValueError("Invalid record identifier")
return value
def state_root(workspace_root: Path, state_dir: Path | None = None) -> Path:
base = (
state_dir
or Path(os.environ.get("XDG_STATE_HOME", str(Path.home() / ".local/state")))
/ "govoplan/devkit"
)
# Preserve spelling until symlink validation; resolving here would hide an unsafe alias.
base = Path(os.path.abspath(os.fspath(base.expanduser())))
return base / ("workspace-" + digest(str(workspace_root.resolve()))[:24])
def reject_symlinks(path: Path) -> None:
for item in (path, *path.parents):
if item.is_symlink():
raise ValueError("State and evidence paths must not contain symlinks")
def private_directory(path: Path) -> None:
reject_symlinks(path)
path.mkdir(parents=True, exist_ok=True, mode=0o700)
reject_symlinks(path)
metadata = path.stat()
if not stat.S_ISDIR(metadata.st_mode) or (
hasattr(os, "getuid") and metadata.st_uid != os.getuid()
):
raise ValueError("State directory must be owned by the current user")
path.chmod(0o700)
def read_bounded_bytes(path: Path, max_bytes: int = MAX_JSON_BYTES) -> bytes:
reject_symlinks(path)
descriptor = os.open(
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
)
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > max_bytes:
raise ValueError("Evidence must be a bounded regular file")
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes:
raise ValueError("Evidence file exceeds its size bound")
return encoded
def read_json(path: Path, max_bytes: int = MAX_JSON_BYTES) -> object:
encoded = read_bounded_bytes(path, max_bytes)
def unique(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError("Duplicate JSON keys are not accepted")
result[key] = value
return result
try:
return json.loads(
encoded,
object_pairs_hook=unique,
parse_constant=lambda _: (_ for _ in ()).throw(
ValueError("Non-finite JSON number")
),
)
except (RecursionError, UnicodeError) as exc:
raise ValueError("JSON nesting or encoding is unsupported") from exc
def atomic_text(path: Path, value: str, max_bytes: int = MAX_JSON_BYTES) -> None:
encoded = value.encode("utf-8")
if len(encoded) > max_bytes:
raise ValueError("Output exceeds its size bound")
reject_symlinks(path.parent)
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
reject_symlinks(path.parent)
reject_symlinks(path)
if path.exists() and not path.is_file():
raise ValueError("Output target must be a regular file")
descriptor, temporary = tempfile.mkstemp(prefix=".devkit-", dir=path.parent)
try:
with os.fdopen(descriptor, "wb") as handle:
os.fchmod(handle.fileno(), 0o600)
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory = os.open(path.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(directory)
finally:
os.close(directory)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def atomic_json(path: Path, payload: object) -> None:
atomic_text(
path,
json.dumps(
payload, indent=2, sort_keys=True, ensure_ascii=True, allow_nan=False
)
+ "\n",
)
def redact(text: str) -> str:
"""Best-effort display hygiene, not permission to include secrets in commands."""
for key, value in os.environ.items():
if (
re.search(r"TOKEN|SECRET|PASSWORD|API_KEY|PRIVATE_KEY", key, re.I)
and len(value) >= 4
):
text = text.replace(value, "[redacted]")
text = re.sub(r"(?im)(authorization\s*[:=]\s*)([^\r\n]+)", r"\1[redacted]", text)
text = re.sub(r"(?i)(https?://)[^/\s:@]+:[^/\s@]+@", r"\1[redacted]@", text)
text = re.sub(
r"(?i)((?:token|password|secret|api[_-]?key)\s*[=:]\s*)[^\s,;]+",
r"\1[redacted]",
text,
)
return text
def redact_argv(argv: list[str]) -> list[str]:
result, hide_next = [], False
for argument in argv:
if hide_next:
result.append("[redacted]")
hide_next = False
continue
if re.fullmatch(
r"--?(?:password|passwd|token|secret|api[-_]key|access[-_]token|authorization)",
argument,
re.I,
):
hide_next = True
result.append(redact(argument))
return result
def safe_output(value):
"""Redact presentation, not immutable identity hashes or execution inputs."""
if isinstance(value, dict):
return {
key: redact_argv(item)
if key == "argv"
and isinstance(item, list)
and all(isinstance(arg, str) for arg in item)
else "[redacted]"
if re.fullmatch(
r"password|passwd|token|secret|api[_-]?key|authorization",
str(key),
re.I,
)
else safe_output(item)
for key, item in value.items()
}
if isinstance(value, list):
return [safe_output(item) for item in value]
return redact(value) if isinstance(value, str) else value
@contextmanager
def resource_lock(directory: Path, name: str, timeout: float = 0):
"""Host-local advisory lock, released by the OS even after a process crash."""
import fcntl
private_directory(directory)
path = directory / (hashlib.sha256(name.encode()).hexdigest() + ".lock")
reject_symlinks(path)
descriptor = os.open(
path, os.O_CREAT | os.O_RDWR | getattr(os, "O_NOFOLLOW", 0), 0o600
)
try:
if not stat.S_ISREG(os.fstat(descriptor).st_mode):
raise ValueError("Lock target must be a regular file")
deadline = time.monotonic() + timeout
while True:
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
if time.monotonic() >= deadline:
raise RuntimeError(f"Resource is busy: {name}") from None
time.sleep(min(0.1, max(0, deadline - time.monotonic())))
yield
finally:
os.close(descriptor)
+90
View File
@@ -0,0 +1,90 @@
"""Small read-only context bundles; no automatic source/credential dumping."""
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from .common import META_ROOT, now, read_json
from .workspace import inspect_repository, load_project, selected_repositories
def build_context(
workspace_root: Path, project_file: Path | None, names: list[str], changed: bool
) -> dict:
project = load_project(workspace_root, project_file)
repos = selected_repositories(project, names)
with ThreadPoolExecutor(max_workers=8) as pool:
states = list(pool.map(inspect_repository, repos))
if changed:
states = [
state
for state in states
if state["errors"]
or state["dirty_entries"]
or state["ahead"]
or (state["head"] and not state["upstream"])
]
inventory = {}
if not project_file:
path = META_ROOT / "docs/project/ui-review-issue-inventory.json"
if path.is_file():
payload = read_json(path)
for item in payload.get("issues", []) if isinstance(payload, dict) else []:
if isinstance(item, dict):
inventory[item.get("repository")] = item
for state in states:
root = Path(state["path"])
state["instructions"] = [
str(path) for path in (root / "AGENTS.md",) if path.is_file()
]
state["documentation"] = [
str(path)
for path in (
root / "README.md",
root / "docs/README.md",
root / "docs/MODULE_ARCHITECTURE.md",
)
if path.is_file()
]
state["change_entry_count"] = len(state["dirty_entries"])
state["review_issue"] = inventory.get(state["name"], {}).get("url")
state["suggested_check"] = (
f"./devkit check --repo {state['name']} --profile quick --dry-run"
)
summary = [
f"{project.name}: {len(states)} repositories selected (offline; upstream counts may be stale)."
]
for state in states:
errors = f" ERROR: {'; '.join(state['errors'])}" if state["errors"] else ""
summary.append(
f"{state['name']}: {state['branch'] or '(detached/unborn)'}; {state['change_entry_count']} change entries; ahead={state['ahead']} behind={state['behind']}{errors}"
)
return {
"schema_version": 1,
"generated_at": now(),
"workspace_root": str(workspace_root),
"project": project.name,
"remote_checked": False,
"repositories": states,
"summary": summary,
"_exit_code": 1 if any(state["errors"] for state in states) else 0,
}
def register(subparsers):
parser = subparsers.add_parser(
"context",
help="Offline repository changes, ownership and relevant instruction paths",
)
parser.add_argument("--repo", action="append", default=[])
parser.add_argument(
"--changed",
action="store_true",
help="Show dirty or locally ahead repositories, retaining inspection errors",
)
parser.set_defaults(
handler=lambda args: build_context(
args.workspace_root, args.project, args.repo, args.changed
)
)
+453
View File
@@ -0,0 +1,453 @@
"""Explicit suite-plan coverage; never execute or guess nested shell/npm flows."""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import re
import shlex
import stat
from .common import digest, redact_argv, reject_symlinks
from .package_tests import CORE_COMPONENT_SUITES, declared_tests, discovered_sources
from .workspace import load_project
DISPOSITIONS = ("planned", "covered_elsewhere", "excluded", "unsupported")
MAX_SUITES = 4096
def _read_canonical(path: Path) -> str:
reject_symlinks(path)
descriptor = os.open(
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
)
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 1024 * 1024:
raise ValueError("Canonical focused gate must be a bounded regular file")
encoded = handle.read(1024 * 1024 + 1)
if len(encoded) > 1024 * 1024:
raise ValueError("Canonical focused gate exceeds its size bound")
return encoded.decode("utf-8")
def _node_argv(argv: list[str], cwd: Path) -> tuple[str, ...] | None:
if (
not argv
or argv[0] not in {"node", "{node}", "$NODE"}
and Path(argv[0]).name not in {"node", "nodejs"}
):
return None
offset = 2 if len(argv) > 1 and argv[1] == "--test" else 1
if len(argv) != offset + 1:
return None
target = Path(argv[offset])
if "$" in str(target):
return None
return ("node", *argv[1:offset], str((cwd / target).resolve()))
def canonical_invocations(workspace_root: Path, meta: Path, core: Path) -> dict:
"""Recognize direct commands in exact registered phase wrappers or legacy top level.
Shell is not evaluated. Bodies, conditionals, loops, functions, npm hooks and
recursive commands do not grant coverage. Unknown cwd invalidates matches.
The registered marked wrappers are the only function bodies admitted.
"""
path = meta / "tools/checks/check-focused.sh"
try:
text = _read_canonical(path)
except (OSError, ValueError):
return {
"path": str(path),
"sha256": None,
"npm": [],
"node": [],
"node_phases": [],
"notes": [
"Canonical focused script is unavailable/unreadable; no package coverage inferred."
],
}
values = {
"WORKSPACE_ROOT": str(workspace_root),
"META_ROOT": str(meta),
"ROOT": str(core),
}
def expand(value: str) -> str | None:
value = re.sub(
r"\$\{(WORKSPACE_ROOT|META_ROOT|ROOT)\}|\$(WORKSPACE_ROOT|META_ROOT|ROOT)(?![A-Za-z0-9_])",
lambda match: values[match[1] or match[2]],
value,
)
return None if "$" in value or "`" in value else value
result = {
"path": str(path),
"sha256": hashlib.sha256(text.encode()).hexdigest(),
"npm": [],
"node": [],
"node_phases": [],
"notes": [],
}
metadata = meta / "tools/checks/focused-phases.json"
blocks = {None: text}
if metadata.exists() or metadata.is_symlink() or "# devkit-phase:" in text:
from .catalog import focused_phases, focused_phase_bodies
try:
phases = focused_phases(meta)
blocks = focused_phase_bodies(text, phases)
result["phase_ids"] = [phase["id"] for phase in phases]
result["metadata_sha256"] = digest(phases)
except (OSError, ValueError):
result["notes"].append(
"Focused phase metadata/marked bodies are invalid or unavailable; no phase coverage inferred."
)
return result
cwd: Path | None = meta
body_end, nesting = None, 0
current_phase = object()
for phase_id, line in (
(phase_id, line)
for phase_id, block in blocks.items()
for line in block.replace("\\\n", " ").splitlines()
):
if current_phase != phase_id:
cwd, body_end, nesting = meta, None, 0
current_phase = phase_id
stripped = line.strip()
if body_end is not None:
if stripped == body_end:
body_end = None
continue
heredoc = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
if heredoc:
body_end = heredoc[1]
continue
if not stripped or stripped.startswith("#"):
continue
if re.match(
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
stripped,
):
nesting += 1
continue
if re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
nesting = max(0, nesting - 1)
continue
if nesting:
if re.search(r"\bcd\s", stripped):
cwd = None
continue
try:
parts = shlex.split(line, comments=True)
except ValueError:
result["notes"].append(
"An unparseable shell line grants no suite coverage."
)
continue
if not parts:
continue
if parts[0] == "cd":
destination = expand(parts[1]) if len(parts) == 2 else None
candidate = (
(cwd / destination).resolve()
if destination and cwd
else Path(destination).resolve()
if destination and Path(destination).is_absolute()
else None
)
cwd = (
candidate
if candidate and candidate.is_relative_to(workspace_root)
else None
)
continue
if cwd is None:
continue
if (
parts[0] in {"$NPM", "${NPM}", "npm"}
and len(parts) >= 3
and parts[1] == "run"
and re.fullmatch(r"test(?::[A-Za-z0-9_.:-]+)?", parts[2])
):
arguments = parts[3:]
if arguments and (
arguments[0] != "--"
or any(
not re.fullmatch(r"[A-Za-z0-9_.-]+", item) for item in arguments[1:]
)
):
continue
result["npm"].append(
{
"cwd": str(cwd),
"name": parts[2],
"args": arguments[1:] if arguments else [],
"phase": phase_id,
}
)
elif parts[0] in {"$NODE", "${NODE}", "node"}:
expanded = [expand(part) for part in parts[1:]]
if all(part is not None for part in expanded):
command = _node_argv(["node", *expanded], cwd)
if command:
result["node"].append(command)
result["node_phases"].append({"argv": command, "phase": phase_id})
result["notes"] = list(dict.fromkeys(result["notes"]))[:8]
return result
def coverage_inventory(
workspace_root: Path, profile: str, project_file: Path | None, stages: list[dict]
) -> dict:
workspace_root = workspace_root.resolve()
project = load_project(workspace_root, project_file)
stage_map = {item["id"]: item for item in stages}
direct = {}
explicit_npm = {}
for stage in stages:
argv, cwd = stage["argv"], Path(stage["cwd"])
normalized = _node_argv(argv, cwd)
if normalized:
direct[normalized] = stage["id"]
if (
len(argv) == 3
and (argv[0] in {"npm", "{npm}"} or Path(argv[0]).name == "npm")
and argv[1] == "run"
):
explicit_npm[(str(cwd.resolve()), argv[2])] = stage["id"]
paths = {repo.name: repo.path for repo in project.repositories}
canonical = (
canonical_invocations(
workspace_root,
paths.get("govoplan", workspace_root / "govoplan"),
paths.get("govoplan-core", workspace_root / "govoplan-core"),
)
if project_file is None
else None
)
canonical_stages = {}
if canonical is not None:
legacy = stage_map.get("focused-workspace")
if legacy and legacy["argv"] == ["bash", canonical["path"]]:
canonical_stages[None] = "focused-workspace"
for identity in canonical.get("phase_ids", []):
check = stage_map.get("focused." + identity)
if check and check["argv"] == [
"bash",
canonical["path"],
"--phase",
identity,
]:
canonical_stages[identity] = check["id"]
full = bool(canonical_stages)
invocations = (
[
{**item, "covering_stage": canonical_stages[item.get("phase")]}
for item in canonical["npm"]
if item.get("phase") in canonical_stages
]
if canonical
else []
)
canonical_nodes = (
{
tuple(item["argv"]): canonical_stages[item.get("phase")]
for item in canonical.get("node_phases", [])
if item.get("phase") in canonical_stages
}
if canonical
else {}
)
suites, notes = [], []
if canonical:
notes.extend(canonical["notes"])
core_package = (
paths.get("govoplan-core", workspace_root / "govoplan-core")
/ "webui/package.json"
)
core_requests = [
item
for item in invocations
if item["cwd"] == str(core_package.parent) and item["name"] == "test:components"
]
requested_components = set()
for item in core_requests:
requested_components.update(
CORE_COMPONENT_SUITES
if not item["args"] or item["args"] == ["all"]
else item["args"]
)
requested_components.intersection_update(CORE_COMPONENT_SUITES)
for repo in project.repositories:
for location in ("package.json", "webui/package.json"):
package = repo.path / location
if not package.exists() and not package.is_symlink():
continue
entries = declared_tests(repo, package)
if location == "webui/package.json":
declared_argv = {
tuple(item["_argv"]) for item in entries if item["_argv"]
}
entries.extend(
item
for item in discovered_sources(repo)
if not item["_argv"] or tuple(item["_argv"]) not in declared_argv
)
for item in entries:
private_argv = item.pop("_argv")
component = item["component_suite"]
identity = (str(package.parent), item["name"])
normalized = (
_node_argv(private_argv, package.parent) if private_argv else None
)
matching = [
value
for value in invocations
if (value["cwd"], value["name"]) == identity
]
item.update(disposition="excluded", covering_stage=None)
if component is not None:
all_or_selected = (
component == "all" or component in requested_components
)
if "core.component-batch" in stage_map:
item.update(
disposition="planned"
if component == "all"
else "covered_elsewhere",
covering_stage="core.component-batch",
reason="UI explicitly runs the shared component batch; quick never compiles it.",
)
elif full and core_requests and all_or_selected:
item.update(
disposition="planned"
if component == "all"
else "covered_elsewhere",
covering_stage=core_requests[0]["covering_stage"],
reason=f"Canonical focused gate explicitly selects {len(requested_components)}/{len(CORE_COMPONENT_SUITES)} component suites; this is not the complete component batch.",
)
else:
item["reason"] = (
"No component compilation in quick/backend; UI runs all components. Full runs only its explicitly named subset."
)
if component == "all":
selected = (
list(CORE_COMPONENT_SUITES)
if "core.component-batch" in stage_map
else sorted(requested_components)
)
item.update(
covered_components=selected,
excluded_components=[
name
for name in CORE_COMPONENT_SUITES
if name not in selected
],
)
elif identity in explicit_npm:
item.update(
disposition="planned",
covering_stage=explicit_npm[identity],
reason="Explicit project check invokes this exact package suite.",
)
elif matching:
if any(not value["args"] for value in matching):
item.update(
disposition="planned",
covering_stage=next(
value["covering_stage"]
for value in matching
if not value["args"]
),
reason="The canonical focused script directly invokes this exact package suite.",
)
else:
item["reason"] = (
"Canonical invocation supplies arguments; complete suite coverage cannot be inferred."
)
elif normalized in direct:
item.update(
disposition="planned",
covering_stage=direct[normalized],
reason="A selected stage runs this exact direct source-test command.",
)
elif normalized in canonical_nodes:
item.update(
disposition="covered_elsewhere",
covering_stage=canonical_nodes[normalized],
reason="Canonical focused script directly runs this suite's exact Node target.",
)
elif private_argv is None:
item["disposition"] = "unsupported"
elif item["name"] in {
"test:module-permutations",
"test:vite-cache-isolation",
}:
item["reason"] = (
"Separate environment/permutation suite; absent from the selected stage plan."
)
else:
item["reason"] = (
"Not directly present in this profile's stage plan; nested npm scripts/hooks are not inferred."
)
suites.append(item)
if len(suites) > MAX_SUITES:
raise ValueError("Coverage inventory exceeds its bounded suite count")
if project_file is not None:
for check in project.config.get("checks", []):
identity = check["id"]
suites.append(
{
"repo": ",".join(check.get("repos", [])) or "project",
"name": identity,
"kind": "project-check",
"package_path": None,
"argv": redact_argv(check["argv"]),
"command_sha256": digest(check["argv"]),
"disposition": "planned" if identity in stage_map else "excluded",
"covering_stage": identity if identity in stage_map else None,
"reason": "Selected declared check or dependency."
if identity in stage_map
else "Declared check is outside this profile/selection.",
}
)
if len(suites) > MAX_SUITES:
raise ValueError("Coverage inventory exceeds its bounded suite count")
counts = {
kind: sum(item["disposition"] == kind for item in suites)
for kind in DISPOSITIONS
}
notes.append(
"Coverage describes planned suite invocations, not passing tests, per-test coverage, or completed UI review."
)
if full:
notes.append(
"Full is the canonical focused gate, not every declared package test; recursive commands and npm hooks are deliberately not guessed."
)
return {
"schema_version": 1,
"profile": profile,
"scope": "Declared root/webui package test scripts, discovered UI structural checks, and custom project checks across registered repositories.",
"suite_count": len(suites),
"stages": list(stage_map),
"counts": counts,
"suites": suites,
"canonical_gate": {
key: canonical[key]
for key in ("path", "sha256", "metadata_sha256")
if key in canonical
}
if canonical
else None,
"notes": notes,
"summary": [
"Suite coverage: "
+ ", ".join(f"{value} {key}" for key, value in counts.items()),
*notes,
],
}
+133
View File
@@ -0,0 +1,133 @@
"""Recorded documentation checks built from existing owning-module contracts."""
from __future__ import annotations
from pathlib import Path
import uuid
from .catalog import stage
from .common import state_root
from .workspace import load_project, selected_repositories
LIMITATIONS = [
"Static baseline, marker coverage and known display slots do not prove complete workflow or linguistic coverage.",
"Computed labels, configured documentation providers and runtime module/permission combinations still require manual review.",
"Audit findings do not edit documentation, translate text, publish evidence or close a review issue.",
]
def build_doc_stages(args) -> list[dict]:
workspace = Path(args.workspace_root).resolve()
project = load_project(workspace, getattr(args, "project", None))
if getattr(args, "project", None):
raise ValueError(
"The docs audit currently targets GovOPlaN manifest/locale contracts. Register another project's documentation checks in its check profiles."
)
selected = selected_repositories(
project, getattr(args, "repo", []), changed=getattr(args, "changed", False)
)
if getattr(args, "changed", False) and not selected:
return []
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
core = next(
repo.path for repo in project.repositories if repo.name == "govoplan-core"
)
output = (
state_root(workspace, getattr(args, "state_dir", None))
/ "artifacts"
/ f"docs-{uuid.uuid4().hex}"
)
reason = "Reuse the existing owning-module documentation and translation checks"
labels = [
"{node}",
str(meta / "tools/devkit/audit-display-labels.mjs"),
"--workspace-root",
str(workspace),
]
for repo in selected:
labels.extend(["--repo", repo.name])
stages = [
stage(
"docs.manifests",
"Static user/admin documentation and manifest contracts",
[
"{python}",
str(meta / "tools/checks/check-manifest-shapes.py"),
"--workspace-root",
str(workspace),
"--require-architecture",
],
meta,
reason=reason,
timeout_seconds=600,
),
stage(
"docs.interface-inventory",
"Existing EN/DE markers, high-risk help and interface declarations",
[
"{python}",
str(meta / "tools/inventory/platform-interface-inventory.py"),
"--workspace-root",
str(workspace),
"--strict",
"--strict-declarations",
"--strict-endpoints",
"--output-dir",
str(output),
],
meta,
reason="Shared inventory remains workspace-wide; --repo narrows only the additional plain-label audit",
timeout_seconds=600,
),
stage(
"docs.translation-structure",
"Existing translation key/structural-value guard",
["{node}", str(core / "webui/scripts/audit-i18n-structural.mjs")],
core / "webui",
reason=reason,
),
stage(
"docs.plain-display-labels",
"Plain display labels and owning catalog registration",
labels,
meta,
reason="Supplement the marker inventory with known display slots; dynamic cases are review candidates",
timeout_seconds=600,
),
]
# These constraints belong to the saved evidence, not only the immediate
# CLI response. The publisher already validates bounded coverage_notes.
stages[0]["coverage_notes"] = list(LIMITATIONS)
return stages
def audit(args) -> dict:
from .runner import run_checks
result = run_checks(args, build_doc_stages(args))
result["limitations"] = list(LIMITATIONS)
result.setdefault("_exit_code", 0)
summary = result.setdefault("summary", [])
summary.extend(note for note in LIMITATIONS if note not in summary)
return result
def register(subparsers) -> None:
docs = subparsers.add_parser(
"docs", help="Audit existing documentation and translation contracts"
)
commands = docs.add_subparsers(dest="docs_command", required=True)
command = commands.add_parser(
"audit", help="Run recorded checks; never edit or generate translations"
)
command.add_argument(
"--repo",
action="append",
default=[],
help="Repository/alias for plain-label checks; shared checks stay workspace-wide",
)
command.add_argument("--changed", action="store_true")
command.add_argument("--jobs", type=int, default=2)
command.add_argument("--dry-run", action="store_true")
command.set_defaults(handler=audit, profile="docs")
+232
View File
@@ -0,0 +1,232 @@
"""Read-only environment preflight with actionable, never automatic repairs."""
from __future__ import annotations
from dataclasses import replace
import os
from pathlib import Path
import socket
from .common import now
from .environment import execution_environment, resolve_tools, tool_version
from .workspace import inspect_repository, load_project, selected_repositories
from .process import require_capture
def _required_tools(project, workspace_root, selected, *, filtered, profile=None):
"""Use the catalog's selected commands, including dependencies, without running them."""
from .catalog import _custom_stages
if profile is None:
# Doctor covers all declared profiles by default, or all declared checks
# when the project only uses the context/doctor commands so far.
profiles = project.config.get("profiles", {})
records = project.config.get("checks", [])
if not isinstance(profiles, dict) or not isinstance(records, list):
raise ValueError(
"Project profiles and checks must have their declared shapes"
)
identities = []
if profiles:
for values in profiles.values():
if not isinstance(values, list) or any(
not isinstance(value, str) for value in values
):
raise ValueError("A project profile must be a list of check IDs")
identities.extend(values)
else:
if any(not isinstance(record, dict) for record in records):
raise ValueError("Project checks must be objects")
identities = [record.get("id") for record in records]
if any(not isinstance(identity, str) for identity in identities):
raise ValueError("Project checks must have string IDs")
project = replace(
project,
config={
**project.config,
"profiles": {"quick": list(dict.fromkeys(identities))},
},
)
profile = "quick"
stages = _custom_stages(project, workspace_root, profile, selected, filtered)
# Python is required by the runner and its environment fingerprint even if
# all selected check commands use another interpreter. Explicitly configured
# tools also declare requirements for commands hidden inside project scripts.
required = {"python", *project.config.get("tools", {})}
for stage in stages:
argv = stage["argv"]
for name in ("node", "npm", "python"):
if any("{" + name + "}" in value for value in argv):
required.add(name)
executable = Path(argv[0]).name
if executable in {"node", "nodejs"}:
required.add("node")
elif executable in {"npm", "npx"}:
required.update({"npm", "node"})
if "npm" in required:
required.add("node")
return required
def diagnose(args) -> dict:
project = load_project(args.workspace_root, args.project)
tools = resolve_tools(args.workspace_root, project)
env = execution_environment(args.workspace_root, project, tools)
selected = selected_repositories(project, args.repo)
required = (
_required_tools(
project,
args.workspace_root,
selected,
filtered=bool(args.repo),
profile=getattr(args, "profile", None),
)
if args.project
else set(tools)
)
checks = []
for name, executable in tools.items():
if name not in required:
checks.append(
{
"id": name,
"status": "not_required",
"detail": "Not required by the selected declared project checks; not probed.",
"path": executable,
"repair": "For indirect dependencies inside scripts, declare the tool under project tools.",
}
)
continue
version = tool_version(executable, env)
checks.append(
{
"id": name,
"status": "passed" if version != "unavailable" else "blocked",
"detail": version,
"path": executable,
"repair": f"Install/configure {name}; set {name.upper()} or project tools.{name}. No installation was attempted.",
}
)
for repo in selected:
snapshot = inspect_repository(repo)
checks.append(
{
"id": repo.name,
"status": "blocked" if snapshot["errors"] else "passed",
"detail": "; ".join(snapshot["errors"])
or "Git checkout readable (dirty work is allowed).",
"repair": "Use tools/repo/bootstrap-repositories.py --check; review missing checkouts before cloning.",
}
)
for package_dir in (repo.path, repo.path / "webui"):
if (package_dir / "package.json").is_file():
present = (package_dir / "node_modules").is_dir()
checks.append(
{
"id": f"dependencies:{repo.name}:{package_dir.name}",
"status": "passed" if present else "warning",
"detail": "Dependency directory exists; availability is not a full dependency audit."
if present
else "No local node_modules directory; workspace-hoisted packages may still resolve.",
"repair": "Inspect the owning package lock and install instructions before running npm ci.",
}
)
if not args.project:
script = args.workspace_root / "govoplan/tools/repo/sync-python-environment.py"
if script.is_file():
result = require_capture(
[
tools["python"],
str(script),
"--check",
"--requirements",
str(args.workspace_root / "govoplan/requirements-dev.txt"),
"--python",
tools["python"],
],
timeout=60,
max_stdout=65536,
env=env,
)
checks.append(
{
"id": "python-environment-sync",
"status": "passed" if result.returncode == 0 else "warning",
"detail": "Environment synchronization fingerprint is current."
if result.returncode == 0
else "Environment sync check did not pass; inspect its --dry-run output.",
"repair": "./.venv/bin/python tools/repo/sync-python-environment.py --dry-run --requirements requirements-dev.txt --python ./.venv/bin/python",
}
)
browser_roots = [
Path.home() / ".cache/ms-playwright",
Path.home() / ".var/app/com.vscodium.codium/cache/ms-playwright",
]
explicit_browser = os.environ.get("PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH")
browser_available = (
(Path(explicit_browser).is_file() and os.access(explicit_browser, os.X_OK))
if explicit_browser
else any(
root.is_dir() and any(root.glob("chromium-*/chrome-linux*/chrome"))
for root in browser_roots
)
)
checks.append(
{
"id": "browser",
"status": "passed" if browser_available else "warning",
"detail": "Configured/cached Chromium found."
if browser_available
else "Chromium executable not found; check PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH or install the pinned Playwright browser.",
"repair": "Follow Core WebUI conformance setup; do not start a development server to repair this.",
}
)
with socket.socket() as probe:
probe.settimeout(0.25)
busy = probe.connect_ex(("127.0.0.1", 4174)) == 0
checks.append(
{
"id": "browser-test-port",
"status": "warning" if busy else "passed",
"detail": "Port 4174 is occupied; do not kill another run."
if busy
else "Port 4174 is free now (not a reservation).",
"repair": "Wait for the owning check run; use devkit resource coordination.",
}
)
summary = [
f"Environment preflight: {sum(item['status'] == 'blocked' for item in checks)} blockers, {sum(item['status'] == 'warning' for item in checks)} warnings."
]
summary.extend(
f"{item['id']}: {item['status']}{item['detail']}"
for item in checks
if item["status"] != "passed"
)
summary.append(
"Read-only: no installations, configuration repairs or servers started."
)
if args.project:
summary.append(
"Required tools follow declared checks and explicit project tools; indirect script dependencies must be declared explicitly."
)
return {
"schema_version": 1,
"generated_at": now(),
"checks": checks,
"required_tools": sorted(required),
"summary": summary,
"_exit_code": 1 if any(item["status"] == "blocked" for item in checks) else 0,
}
def register(subparsers):
parser = subparsers.add_parser(
"doctor", help="Read-only environment preflight and exact repair guidance"
)
parser.add_argument("--repo", action="append", default=[])
parser.add_argument(
"--profile",
choices=("quick", "ui", "backend", "full"),
help="For portable projects, limit tool requirements to this check profile (default: all declared profiles)",
)
parser.set_defaults(handler=diagnose)
+271
View File
@@ -0,0 +1,271 @@
"""Resolve local tools once, without installation, network access or server startup."""
from __future__ import annotations
import hashlib
from itertools import islice
import os
from pathlib import Path
import shutil
import stat
import subprocess
import sys
from .common import digest
from .workspace import Project
from .process import require_capture
MAX_DISCOVERY_CHILDREN = 4096
def resolve_tools(workspace_root: Path, project: Project) -> dict[str, str]:
configured = project.config.get("tools", {})
if not isinstance(configured, dict) or set(configured) - {"python", "node", "npm"}:
raise ValueError("Project tools must configure only python, node and npm")
tools = {}
python_env = (
Path(
os.environ.get("GOVOPLAN_VENV_ROOT", str(workspace_root / "govoplan/.venv"))
)
/ "bin/python"
)
for name in ("python", "node", "npm"):
default = (
str(python_env)
if name == "python" and python_env.is_file()
else sys.executable
if name == "python"
else name
)
value = configured.get(name) or os.environ.get(name.upper()) or default
if not isinstance(value, str) or not value or "\0" in value:
raise ValueError(f"Invalid executable configuration for {name}")
executable = shutil.which(value)
if executable:
# Keep venv executable symlinks: resolving them loses Python's environment.
tools[name] = os.path.abspath(executable)
else:
tools[name] = value
return tools
def execution_environment(
workspace_root: Path, project: Project, tools: dict[str, str]
) -> dict[str, str]:
env = dict(os.environ)
if (
project.config.get("organization") == "GovOPlaN"
and "schema_version" not in project.config
):
# Managed GovOPlaN checks must not inherit another checkout's scope.
env["GOVOPLAN_WORKSPACE_ROOT"] = str(workspace_root.resolve())
core = next(
repo.path for repo in project.repositories if repo.name == "govoplan-core"
)
env["GOVOPLAN_CORE_ROOT"] = str(core)
env["GOVOPLAN_CORE_SOURCE_ROOT"] = str(core)
directories = [
str(Path(tools[name]).parent)
for name in ("python", "node", "npm")
if Path(tools[name]).is_absolute()
]
core_bins = workspace_root / "govoplan-core/webui/node_modules/.bin"
if core_bins.is_dir():
directories.insert(0, str(core_bins))
env["PATH"] = os.pathsep.join([*directories, env.get("PATH", "")])
env.update({name.upper(): value for name, value in tools.items()})
sources = [
str(repo.path / "src")
for repo in project.repositories
if (repo.path / "src").is_dir()
]
if sources:
env["PYTHONPATH"] = os.pathsep.join(
sources + ([env["PYTHONPATH"]] if env.get("PYTHONPATH") else [])
)
env["GIT_OPTIONAL_LOCKS"] = "0"
env["PYTHONDONTWRITEBYTECODE"] = "1"
return env
def tool_version(executable: str, env: dict[str, str]) -> str:
try:
result = require_capture(
[executable, "--version"],
timeout=15,
max_stdout=4096,
max_stderr=4096,
env=env,
)
return (
(result.stdout or result.stderr).decode(errors="replace").strip()[:256]
if result.returncode == 0
else "unavailable"
)
except (OSError, subprocess.TimeoutExpired, ValueError):
return "unavailable"
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
return (
metadata.st_dev,
metadata.st_ino,
metadata.st_mode,
metadata.st_size,
metadata.st_ctime_ns,
metadata.st_mtime_ns,
)
def _environment_file_hash(path: Path, maximum: int) -> str | None:
"""Read a stable, bounded regular target without changing executable spelling.
Venv executables and package directories may legitimately be symlinks. Only
the read target is resolved; both names and the open descriptor are verified
again afterwards. Missing optional metadata stays absent, but a present
malformed or concurrently changing entry cannot certify an environment.
"""
try:
original = path.lstat()
except FileNotFoundError:
return None
try:
target = path.resolve(strict=True)
target_metadata = target.lstat()
descriptor = os.open(
target,
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
)
with os.fdopen(descriptor, "rb") as handle:
before = os.fstat(handle.fileno())
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
raise ValueError("Environment input must be a bounded regular file")
identity = _file_identity(before)
if identity != _file_identity(target_metadata):
raise ValueError("Environment input changed before fingerprinting")
hasher, size = hashlib.sha256(), 0
for chunk in iter(
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
):
size += len(chunk)
if size > maximum:
raise ValueError("Environment input grew beyond its size bound")
hasher.update(chunk)
if (
_file_identity(os.fstat(handle.fileno())) != identity
or _file_identity(target.lstat()) != identity
or _file_identity(path.lstat()) != _file_identity(original)
or path.resolve(strict=True) != target
):
raise ValueError("Environment input changed during fingerprinting")
return hasher.hexdigest()
except (OSError, RuntimeError) as exc:
raise ValueError("Environment input cannot be fingerprinted safely") from exc
def _discovery_path_shape(path: Path) -> dict:
"""Directory membership, not timestamps changed by ordinary build outputs."""
try:
metadata = path.lstat()
except (FileNotFoundError, NotADirectoryError):
return {"kind": "missing"}
try:
resolved = path.resolve()
try:
target_kind = stat.S_IFMT(resolved.lstat().st_mode)
except (FileNotFoundError, NotADirectoryError):
target_kind = "missing"
if target_kind == stat.S_IFLNK:
# Newer pathlib versions can retain a loop with strict=False.
raise ValueError("Native source discovery cannot be resolved safely")
return {
"kind": stat.S_IFMT(metadata.st_mode),
"resolved": str(resolved),
"target_kind": target_kind,
}
except (OSError, RuntimeError) as exc:
raise ValueError("Native source discovery cannot be resolved safely") from exc
def _native_discovery_fingerprint(workspace_root: Path, project: Project) -> str:
"""Bind glob discovery and registered ownership omitted by narrow Git scopes."""
children = list(islice(workspace_root.iterdir(), MAX_DISCOVERY_CHILDREN + 1))
if len(children) > MAX_DISCOVERY_CHILDREN:
raise ValueError("Native source discovery exceeds its bounded ownership audit")
def shape(path):
return {
name: _discovery_path_shape(path / name if name else path)
for name in ("", "src", "webui")
}
return digest(
{
"version": 1,
"siblings": [
{"name": child.name, "shape": shape(child)}
for child in sorted(children)
if child.name.startswith("govoplan")
],
"registered": [
{"name": repo.name, "path": str(repo.path), "shape": shape(repo.path)}
for repo in sorted(project.repositories, key=lambda item: item.name)
],
}
)
def environment_fingerprint(
workspace_root: Path, project: Project, tools: dict[str, str], env: dict[str, str]
) -> str:
identity = {
"environment": {
key: value
for key, value in env.items()
if key not in {"_", "SHLVL", "PWD", "OLDPWD"}
},
"tools": {},
}
if (
project.config.get("organization") == "GovOPlaN"
and "schema_version" not in project.config
):
identity["native_source_discovery"] = _native_discovery_fingerprint(
workspace_root, project
)
for name, executable in tools.items():
binary_hash = _environment_file_hash(Path(executable), 512 * 1024 * 1024)
identity["tools"][name] = {
"path": executable,
"version": tool_version(executable, env),
"sha256": binary_hash,
}
installed = {}
for repo in project.repositories:
for suffix in (
"node_modules/.package-lock.json",
"webui/node_modules/.package-lock.json",
".venv/pyvenv.cfg",
):
path = repo.path / suffix
value = _environment_file_hash(path, 32 * 1024 * 1024)
if value is not None:
installed[str(path)] = value
try:
result = require_capture(
[
tools["python"],
"-c",
"import importlib.metadata,json; print(json.dumps(sorted((d.metadata['Name'],d.version) for d in importlib.metadata.distributions())))",
],
timeout=30,
max_stdout=1024 * 1024,
env=env,
)
if result.returncode:
raise ValueError("Cannot fingerprint installed Python distributions")
installed["python_distributions"] = hashlib.sha256(result.stdout).hexdigest()
except (OSError, subprocess.TimeoutExpired) as exc:
raise ValueError("Cannot fingerprint Python environment") from exc
identity["installed"] = installed
return digest(identity)
+417
View File
@@ -0,0 +1,417 @@
"""Versioned repository-scoped input identities with run-local content memoization.
No stored receipt supplies commands to this engine. Every snapshot re-reads Git
HEAD/index/flags/membership and opens its inputs; only stable file-content hashes
are memoized, never repository snapshots or prior-run results.
"""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import re
import stat
from .common import META_ROOT, canonical, digest, identifier
from .workspace import Project, git_bytes
FINGERPRINT_VERSION = "repository-inputs-v1"
MAX_SOURCE_BYTES = 64 * 1024 * 1024
MAX_CACHE_FILES = 200_000
MAX_REPOSITORY_ENTRIES = 200_000
_INDEX_RECORD = re.compile(rb"[A-Za-z] [0-7]{6} [a-fA-F0-9]{40,64} [0-3]\t(.*)\Z", re.S)
_RUNTIME_FIELDS = {
"status",
"exit_code",
"duration_seconds",
"log_path",
"log_sha256",
"error",
"started_at",
"finished_at",
"reused_from",
"output_truncated",
"checkpoint_verified",
"checkpoint_at",
"cache_key",
"input_fingerprint",
"reuse_reason",
"omitted_output_bytes",
}
def validate_input_declaration(value: object, repository_names) -> dict:
"""Pure validation shared with planning; does not inspect files or run Git."""
if not isinstance(value, dict) or set(value) != {"repos"}:
raise ValueError(
"Stage inputs must contain only the required repos declaration"
)
names = value["repos"]
if not isinstance(names, list) or not 1 <= len(names) <= 256:
raise ValueError(
"Input repos must be a nonempty bounded list of canonical repository names"
)
for name in names:
identifier(name)
if len(set(names)) != len(names):
raise ValueError("Duplicate input repository reference")
if set(names) - set(repository_names):
raise ValueError(
"Input repos reference an unknown/noncanonical repository name"
)
return {"repos": sorted(names)}
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
return (
metadata.st_dev,
metadata.st_ino,
metadata.st_mode,
metadata.st_size,
metadata.st_ctime_ns,
metadata.st_mtime_ns,
)
def _stats() -> dict[str, int]:
return {
name: 0
for name in (
"repositories",
"git_calls",
"entries",
"files",
"bytes",
"cache_hits",
"tooling_files",
"tooling_bytes",
"tooling_cache_hits",
)
}
class InputSnapshotter:
"""One run's in-memory memo; create a fresh instance for each invocation."""
def __init__(self, project: Project, *, workspace_root: Path):
self.project = project
self.workspace_root = Path(workspace_root).resolve()
self.repositories = {}
for repo in project.repositories:
identifier(repo.name)
if repo.name in self.repositories:
raise ValueError("Duplicate input repository name")
if not repo.path.is_absolute() or not repo.path.resolve().is_relative_to(
self.workspace_root
):
raise ValueError("Input repository path escapes its workspace")
self.repositories[repo.name] = repo
if not self.repositories or len(self.repositories) > 256:
raise ValueError("Input project requires 1256 repositories")
# One most recent stable identity per path. This is deliberately not
# serializable/persisted, and hashes are never reused on mtime alone.
self._files: dict[Path, tuple[tuple[int, ...], bytes]] = {}
def _repo_names(self, names: object) -> list[str]:
return validate_input_declaration({"repos": names}, self.repositories)["repos"]
def scope(self, stage: dict) -> dict:
"""Validate one explicit scope; absent inputs means the whole workspace."""
if "inputs" not in stage:
return {
"version": 1,
"kind": "workspace",
"declared": False,
"repos": sorted(self.repositories),
}
value = validate_input_declaration(stage["inputs"], self.repositories)
return {
"version": 1,
"kind": "repositories",
"declared": True,
"repos": value["repos"],
}
def _file_hash(
self, path: Path, statistics: dict, *, expected=None, tooling: bool = False
) -> bytes:
prefix = "tooling_" if tooling else ""
maximum = 4 * 1024 * 1024 if tooling else MAX_SOURCE_BYTES
descriptor = os.open(
path,
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
)
with os.fdopen(descriptor, "rb") as handle:
before = os.fstat(handle.fileno())
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
raise ValueError("Input must be a bounded regular file")
identity = _file_identity(before)
if expected is not None and _file_identity(expected) != identity:
raise ValueError(
"Input changed before its source identity could be established"
)
statistics[prefix + "files"] += 1
cached = self._files.get(path)
if cached is not None and cached[0] == identity:
value = cached[1]
statistics[prefix + "cache_hits"] += 1
else:
hasher, size = hashlib.sha256(), 0
for chunk in iter(
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
):
size += len(chunk)
if size > maximum:
raise ValueError("Input grew beyond its source identity limit")
hasher.update(chunk)
statistics[prefix + "bytes"] += size
value = hasher.digest()
after = os.fstat(handle.fileno())
# Rechecking the pathname also rejects replacement after the open;
# a stable old FD is not evidence about a newly replaced source file.
current = path.lstat()
if _file_identity(after) != identity or _file_identity(current) != identity:
raise ValueError("Input changed while calculating its source identity")
if path not in self._files and len(self._files) >= MAX_CACHE_FILES:
self._files.clear() # Eviction only causes extra reads, never reuse.
self._files[path] = (identity, value)
return value
def _entry(self, repo, encoded_name: bytes, statistics: dict) -> dict:
relative = Path(os.fsdecode(encoded_name))
if relative.is_absolute() or ".." in relative.parts:
raise ValueError("Unsafe source membership path")
path = repo.path / relative
statistics["entries"] += 1
try:
metadata = path.lstat()
except FileNotFoundError:
return {"kind": "missing"}
if not path.parent.resolve().is_relative_to(repo.path.resolve()):
raise ValueError("Input path escapes its declared repository")
result = {"mode": stat.S_IMODE(metadata.st_mode)}
if stat.S_ISLNK(metadata.st_mode):
target_name = os.readlink(path)
try:
target = path.resolve()
except (OSError, RuntimeError) as exc:
raise ValueError("Input symlink cannot be resolved safely") from exc
if not target.is_relative_to(repo.path.resolve()):
raise ValueError(
"Input symlink target escapes its repository; undeclared target bytes cannot be reused"
)
result.update(
kind="symlink",
target_sha256=hashlib.sha256(os.fsencode(target_name)).hexdigest(),
)
try:
target_metadata = target.lstat()
except FileNotFoundError:
result["target_state"] = "missing"
else:
if not stat.S_ISREG(target_metadata.st_mode):
raise ValueError(
"Input symlink must target a regular file inside its repository"
)
result.update(
target_state="file",
target_mode=stat.S_IMODE(target_metadata.st_mode),
target_content=self._file_hash(
target, statistics, expected=target_metadata
).hex(),
)
if (
_file_identity(path.lstat()) != _file_identity(metadata)
or os.readlink(path) != target_name
):
raise ValueError(
"Input symlink changed while calculating source identity"
)
elif stat.S_ISREG(metadata.st_mode):
result.update(
kind="file",
content=self._file_hash(path, statistics, expected=metadata).hex(),
)
else:
raise ValueError(
"Unsupported source entry; scoped repository inputs require files or in-repository file symlinks"
)
return result
def _repository(self, name: str, statistics: dict) -> str:
repo = self.repositories[name]
statistics["repositories"] += 1
identity = {
"version": FINGERPRINT_VERSION,
"repo": name,
"path": str(repo.path),
}
if not repo.path.exists():
return digest({**identity, "state": "missing"})
if (
not repo.path.resolve().is_relative_to(self.workspace_root)
or not (repo.path / ".git").exists()
):
raise ValueError(
"Cannot establish scoped source identity for an escaped/non-Git repository"
)
statistics["git_calls"] += 1
head = git_bytes(
repo.path, "rev-parse", "--verify", "HEAD", allow_failure=True
).strip()
statistics["git_calls"] += 1
# This preserves stage numbers and assume-unchanged/skip-worktree flags,
# while also listing untracked membership in the same bounded process.
index = git_bytes(
repo.path,
"ls-files",
"--stage",
"-v",
"--cached",
"--others",
"--exclude-standard",
"-z",
)
members = set()
for record in index.split(b"\0"):
if not record:
continue
if record.startswith(b"? "):
members.add(record[2:])
else:
match = _INDEX_RECORD.fullmatch(record)
if not match:
raise ValueError(
"Unsupported Git membership record; cannot establish scoped source identity"
)
members.add(match[1])
if len(members) > MAX_REPOSITORY_ENTRIES:
raise ValueError(
"Repository source membership exceeds its bounded entry count"
)
hasher = hashlib.sha256(
canonical(
{
**identity,
"head_sha256": hashlib.sha256(head).hexdigest(),
"index_sha256": hashlib.sha256(index).hexdigest(),
}
)
)
for member in sorted(members):
hasher.update(
canonical(
{
"name_sha256": hashlib.sha256(member).hexdigest(),
"value": self._entry(repo, member, statistics),
}
)
)
hasher.update(b"\0")
return hasher.hexdigest()
def _source_identity(self, names: list[str], identities: dict[str, str]) -> str:
return digest(
{
"version": FINGERPRINT_VERSION,
"workspace_root": str(self.workspace_root),
"repositories": {name: identities[name] for name in names},
}
)
def _observe(self, names: list[str]) -> dict:
statistics = _stats()
identities = {name: self._repository(name, statistics) for name in names}
complete = set(names) == self.repositories.keys()
return {
"schema_version": 1,
"fingerprint_version": FINGERPRINT_VERSION,
"workspace_root": str(self.workspace_root),
"repository_fingerprints": identities,
"observed_source_fingerprint": self._source_identity(names, identities),
"observed_scope": {
"version": 1,
"kind": "workspace" if complete else "repositories",
"repos": names,
},
"complete_workspace": complete,
"scan_stats": statistics,
}
def source_snapshot(self, repos: list[str] | None = None) -> dict:
"""Compare observed source only, without interpreting any stored commands."""
return self._observe(
sorted(self.repositories) if repos is None else self._repo_names(repos)
)
def _tooling_identity(self, statistics: dict) -> str:
package = Path(__file__).resolve().parent
paths = set()
for path in package.rglob("*.py"):
paths.add(path)
if len(paths) > 256:
raise ValueError("Devkit tooling source exceeds its bounded inventory")
paths.update(
{
package.parent / "devkit.py",
package.parent / "project.schema.json",
META_ROOT / "devkit",
}
)
return digest(
{
str(path.relative_to(META_ROOT)): self._file_hash(
path, statistics, tooling=True
).hex()
for path in sorted(paths)
}
)
def snapshot(self, stages: list[dict], *, tooling_fingerprint: str = "") -> dict:
if not isinstance(stages, list) or len(stages) > 512:
raise ValueError("Input snapshot requires a bounded stage list")
if not isinstance(tooling_fingerprint, str) or len(tooling_fingerprint) > 4096:
raise ValueError("Invalid supplied tooling identity")
scopes = {}
for stage in stages:
if not isinstance(stage, dict):
raise ValueError("Input stages must be objects")
if set(stage) & _RUNTIME_FIELDS:
raise ValueError(
"Input identities require freshly planned stages, not mutable receipt/runtime fields"
)
identity = identifier(stage.get("id"))
if identity in scopes:
raise ValueError("Duplicate input stage identity")
scopes[identity] = self.scope(stage)
names = sorted({name for scope in scopes.values() for name in scope["repos"]})
observed = self._observe(names)
tools = digest(
{
"declared_tools": self.project.config.get("tools", {}),
"devkit_source": self._tooling_identity(observed["scan_stats"]),
"supplied_tooling": tooling_fingerprint,
}
)
entries = {}
for stage in stages:
scope = scopes[stage["id"]]
source = self._source_identity(
scope["repos"], observed["repository_fingerprints"]
)
plan = digest(stage)
entries[stage["id"]] = {
"source_fingerprint": source,
"plan_fingerprint": plan,
"scope": scope,
"fingerprint": digest(
{
"version": FINGERPRINT_VERSION,
"source": source,
"plan": plan,
"tooling": tools,
}
),
}
return {**observed, "tooling_fingerprint": tools, "stages": entries}
+547
View File
@@ -0,0 +1,547 @@
"""Append-only, idempotent Gitea evidence notes; preview is entirely offline."""
from __future__ import annotations
from contextlib import ExitStack
from dataclasses import dataclass
import hashlib
import html
import os
from pathlib import Path
import re
import subprocess
import sys
from typing import Any
from urllib.parse import urlsplit
from .common import META_ROOT, atomic_json, digest, identifier, read_json, redact, resource_lock, state_root
from .inputs import FINGERPRINT_VERSION
_GITEA_PATH = str(META_ROOT / "tools/gitea")
if _GITEA_PATH not in sys.path:
sys.path.insert(0, _GITEA_PATH)
from gitea_common import GiteaClient, RepoTarget, _parse_remote, repo_path # noqa: E402
MAX_NOTE_BYTES = 128 * 1024
MAX_TARGETS = 256
MAX_COVERAGE_NOTES = 2048
MARKER_PREFIX = "<!-- govoplan-devkit:evidence:v1:"
STATUSES = {"planned", "pending", "running", "passed", "failed", "timed_out", "skipped", "cancelled", "interrupted", "blocked", "partial", "stale"}
@dataclass(frozen=True)
class NoteTarget:
root: Path
base_url: str
owner: str
repository: str
issue: int
issue_id: int | None = None
@property
def url(self) -> str:
return f"{self.base_url}/{self.owner}/{self.repository}/issues/{self.issue}"
@property
def path(self) -> str:
return repo_path(self.owner, self.repository, f"/issues/{self.issue}")
def record(self) -> dict:
return {"root": str(self.root), "base_url": self.base_url, "owner": self.owner,
"repository": self.repository, "issue": self.issue, "url": self.url,
**({"issue_id": self.issue_id} if self.issue_id is not None else {})}
def register(subparsers) -> None:
parser = subparsers.add_parser("issues", help="Preview or append evidence to exact Gitea issues")
commands = parser.add_subparsers(dest="issues_command", required=True)
note = commands.add_parser("note", help="Append a deduplicated note; offline dry run by default")
note.add_argument("--root", type=Path, help="Target repository root or child directory")
note.add_argument("--issue", type=int)
note.add_argument("--target-plan", type=Path, help="JSON with schema_version=1 and exact root/issue/url targets")
note.add_argument("--remote", default="origin")
note.add_argument("--env-file", type=Path, help="Private GITEA_TOKEN dotenv file; target overrides are ignored")
note.add_argument("--evidence", help="Local run ID or explicit receipt JSON path")
note.add_argument("--key", default="verification", help="Stable note purpose within an evidence run")
note.add_argument("--summary", dest="note_summary", action="append", default=[])
note.add_argument("--next", dest="next_steps", action="append", default=[])
note.add_argument("--body-file", type=Path, help="Additional Markdown, never executed")
note.add_argument("--note-file", type=Path, help="Structured JSON: summary[], next[], body")
note.add_argument("--apply", action="store_true", help="Explicitly authorize serial comment creation")
note.add_argument("--retry-uncertain", action="store_true", help="After reconciliation, explicitly retry a still-unconfirmed earlier POST")
note.set_defaults(handler=handle_note)
def _text(value: Any, *, maximum: int = 16384) -> str:
if not isinstance(value, str) or len(value.encode("utf-8")) > maximum:
raise ValueError("Note/evidence text must be a bounded string")
if any(ord(character) < 32 and character not in "\n\t\r" for character in value):
raise ValueError("Note/evidence text contains control characters")
if MARKER_PREFIX in value:
raise ValueError("Evidence markers are reserved for the publisher")
return redact(value)
def _positive(value: Any) -> int:
if type(value) is not int or value <= 0:
raise ValueError("Issue and comment identities must be positive integers")
return value
def _base_url(value: str) -> str:
parsed = urlsplit(value)
if (parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username
or parsed.password or parsed.query or parsed.fragment or "\\" in value
or any(ord(char) < 33 for char in value)):
raise ValueError("Gitea target must be a credential-free HTTP(S) URL")
if any(part in {".", ".."} or "%" in part for part in parsed.path.split("/")):
raise ValueError("Gitea base URL contains an ambiguous path")
try:
parsed.port
except ValueError as exc:
raise ValueError("Invalid Gitea port") from exc
return value.rstrip("/")
def _name(value: str) -> str:
if not isinstance(value, str) or not re.fullmatch(r"[A-Za-z0-9_.-]+", value) or value in {".", ".."}:
raise ValueError("Invalid Gitea owner or repository name")
return value
def resolve_target(root: Path, issue: int, workspace_root: Path, *, remote: str = "origin",
expected_url: str | None = None, issue_id: int | None = None) -> NoteTarget:
requested = root if root.is_absolute() else workspace_root / root
resolved = requested.resolve()
if not resolved.is_relative_to(workspace_root.resolve()) or not resolved.is_dir():
raise ValueError("Issue target must be an existing repository inside the selected workspace")
if not re.fullmatch(r"[A-Za-z0-9_.-]+", remote):
raise ValueError("Invalid Git remote name")
command = subprocess.run(["git", "-C", str(resolved), "rev-parse", "--show-toplevel"],
capture_output=True, text=True, timeout=15)
if command.returncode:
raise ValueError("Issue target is not a Git checkout")
actual = Path(command.stdout.strip()).resolve()
if not actual.is_relative_to(workspace_root.resolve()):
raise ValueError("Resolved issue repository escapes the workspace")
result = subprocess.run(["git", "-C", str(actual), "remote", "get-url", remote],
capture_output=True, text=True, timeout=15)
if result.returncode or len(result.stdout) > 8192:
raise ValueError("Target repository has no usable Git remote")
remote_url = result.stdout.strip()
parsed = urlsplit(remote_url)
if (any(ord(char) < 33 for char in remote_url) or "\\" in remote_url
or parsed.query or parsed.fragment):
raise ValueError("Ambiguous Git remote URL cannot bind an issue target")
if parsed.scheme in {"http", "https"} and (parsed.username or parsed.password):
raise ValueError("Credential-bearing Git remotes cannot be used for issue publishing")
base, owner, repository = _parse_remote(remote_url)
target = NoteTarget(actual, _base_url(base), _name(owner), _name(repository), _positive(issue),
_positive(issue_id) if issue_id is not None else None)
if expected_url is not None and expected_url != target.url:
raise ValueError("Target plan issue URL does not match the exact repository remote and issue number")
return target
def _targets(args) -> list[NoteTarget]:
workspace_root = Path(args.workspace_root).resolve()
if args.target_plan:
if args.root is not None or args.issue is not None:
raise ValueError("Use either --root/--issue or --target-plan")
payload = read_json(args.target_plan, max_bytes=1024 * 1024)
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
raise ValueError("Target plan requires schema_version 1")
records = payload.get("targets")
if not isinstance(records, list) or not 1 <= len(records) <= MAX_TARGETS:
raise ValueError("Target plan must contain 1256 exact targets")
targets = []
for record in records:
if (not isinstance(record, dict) or not isinstance(record.get("root"), str)
or not isinstance(record.get("url"), str)):
raise ValueError("Each target plan entry requires root, issue and URL")
target = resolve_target(Path(record["root"]), record.get("issue"), workspace_root,
remote=args.remote, expected_url=record["url"], issue_id=record.get("issue_id"))
for key in ("base_url", "owner", "repository"):
if key in record and record[key] != getattr(target, key):
raise ValueError("Target plan repository identity is inconsistent")
targets.append(target)
else:
if args.root is None or args.issue is None:
raise ValueError("An exact --root and --issue are required")
targets = [resolve_target(args.root, args.issue, workspace_root, remote=args.remote)]
if len({target.url for target in targets}) != len(targets):
raise ValueError("Duplicate issue targets are not accepted")
if len({target.base_url for target in targets}) != 1:
raise ValueError("A credentialed target plan must be confined to one exact Gitea base URL")
return targets
def validate_receipt(payload: Any, workspace_root: Path) -> dict:
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
raise ValueError("Evidence requires receipt schema_version 1")
identifier(payload.get("run_id"))
recorded_workspace = payload.get("workspace_root")
if not isinstance(recorded_workspace, str) or Path(recorded_workspace).resolve() != workspace_root.resolve():
raise ValueError("Evidence belongs to another workspace")
fingerprint = payload.get("source_fingerprint")
if not isinstance(fingerprint, str) or not re.fullmatch(r"[a-f0-9]{64}", fingerprint):
raise ValueError("Evidence requires a recorded source fingerprint")
if not isinstance(payload.get("status"), str) or payload["status"] not in STATUSES:
raise ValueError("Unknown evidence status")
_text(payload.get("generated_at"), maximum=128)
if payload.get("finished_at") is not None:
_text(payload["finished_at"], maximum=128)
stages = payload.get("stages")
if not isinstance(stages, list) or len(stages) > 1000:
raise ValueError("Evidence requires a bounded stage list")
seen = set()
for stage in stages:
if not isinstance(stage, dict):
raise ValueError("Invalid evidence stage")
identity = _text(stage.get("id"), maximum=256)
if not identity or identity in seen or "\n" in identity:
raise ValueError("Evidence stage IDs must be unique")
seen.add(identity)
if not isinstance(stage.get("status"), str) or stage["status"] not in STATUSES:
raise ValueError("Unknown evidence stage status")
code = stage.get("exit_code")
if code is not None and type(code) is not int:
raise ValueError("Evidence exit code must be an integer or null")
duration = stage.get("duration_seconds")
if duration is not None and (type(duration) not in {int, float} or not 0 <= duration < 31536000):
raise ValueError("Invalid evidence stage duration")
if stage.get("log_path") is not None:
_text(stage["log_path"], maximum=4096)
if stage["status"] == "passed" and code != 0:
raise ValueError("Passed evidence stage must have exit code zero")
if payload["status"] == "passed" and (not stages or any(stage["status"] != "passed" for stage in stages)):
raise ValueError("Passed evidence is inconsistent with its stages")
if payload["status"] == "passed" and payload.get("snapshot_verified") is not True:
raise ValueError("Passed evidence requires a verified source snapshot")
coverage_notes(stages)
from .checkpoints import validate_checkpoint_receipt
validate_checkpoint_receipt(payload)
return payload
def coverage_notes(stages: list[dict]) -> list[str]:
"""Keep declared coverage limits visible without interpreting them as commands."""
notes, total, size = [], 0, 0
for stage in stages:
values = stage.get("coverage_notes", [])
if not isinstance(values, list):
raise ValueError("Evidence coverage_notes must be a bounded list of strings")
total += len(values)
if total > MAX_COVERAGE_NOTES:
raise ValueError("Evidence coverage_notes exceed their count bound")
for value in values:
text = _text(value, maximum=4096).strip()
if not text:
raise ValueError("Evidence coverage notes cannot be empty")
size += len(text.encode())
if size > MAX_NOTE_BYTES:
raise ValueError("Evidence coverage_notes exceed their size bound")
notes.append(text)
return list(dict.fromkeys(notes))
def evidence_record(value: str | None, args) -> dict | None:
if not value:
return None
workspace_root = Path(args.workspace_root).resolve()
external = value.endswith(".json") or "/" in value or "\\" in value
if external:
path = Path(value).expanduser().absolute()
payload = read_json(path)
origin = "external-unverified"
else:
from .runner import read_receipt
identity = identifier(value)
payload = read_receipt(workspace_root, args.state_dir, identity)
path = state_root(workspace_root, args.state_dir) / "runs" / identity / "receipt.json"
origin = "local-integrity-checked"
receipt = validate_receipt(payload, workspace_root)
from .workspace import load_project, source_fingerprint
expected_project = str(args.project.resolve()) if getattr(args, "project", None) else None
source_state = "not-compared"
current = None
if receipt.get("project_file") == expected_project:
try:
project = load_project(workspace_root, getattr(args, "project", None))
if receipt.get("fingerprint_version") == FINGERPRINT_VERSION:
from .inputs import InputSnapshotter
scope = receipt.get("source_scope", {})
if not isinstance(scope, dict) or not isinstance(scope.get("repos"), list):
raise ValueError("Scoped evidence requires an explicit recorded repository scope")
current = InputSnapshotter(project, workspace_root=workspace_root).source_snapshot(scope["repos"])["observed_source_fingerprint"]
else:
current = source_fingerprint(project)
source_state = "matches-current" if current == receipt["source_fingerprint"] else "historical-source-differs"
except (OSError, ValueError, subprocess.SubprocessError):
source_state = "current-source-unavailable"
else:
source_state = "different-project-not-compared"
return {"run_id": redact(receipt["run_id"]), "status": receipt["status"], "origin": origin,
"receipt_path": redact(str(path)), "source_fingerprint": receipt["source_fingerprint"],
"current_source_fingerprint": current, "source_state": source_state,
"snapshot_verified": receipt.get("snapshot_verified") is True,
"coverage_notes": coverage_notes(receipt["stages"]) + ([
"Source comparison is limited to the recorded repository input scope; this is not a whole-workspace or artifact verification."
] if receipt.get("fingerprint_version") == FINGERPRINT_VERSION and not receipt.get("source_scope", {}).get("complete_workspace") else []),
"fingerprint_version": receipt.get("fingerprint_version"),
"source_scope": receipt.get("source_scope"),
"generated_at": redact(receipt["generated_at"]), "finished_at": redact(receipt["finished_at"]) if receipt.get("finished_at") else None,
"stages": [{**{key: redact(stage[key]) if isinstance(stage.get(key), str) else stage.get(key)
for key in ("id", "status", "exit_code", "duration_seconds", "log_path")},
"coverage_notes": coverage_notes([stage])} for stage in receipt["stages"]],
"attestation": "Receipt metadata is not an independent attestation, a live verification, or a completed module review."}
def _structured_note(args) -> dict:
payload = read_json(args.note_file, max_bytes=MAX_NOTE_BYTES) if args.note_file else {}
if not isinstance(payload, dict) or set(payload) - {"summary", "next", "body"}:
raise ValueError("Structured note accepts only summary[], next[] and body")
result = {}
for key, values in (("summary", args.note_summary), ("next", args.next_steps)):
supplied = payload.get(key, [])
if not isinstance(supplied, list) or len(supplied) + len(values) > 100:
raise ValueError("Summary and next steps must be bounded lists")
result[key] = [_text(item).strip() for item in [*supplied, *values]]
body = _text(payload.get("body", ""), maximum=MAX_NOTE_BYTES)
if args.body_file:
# Reuse the bounded, no-symlink file reader by wrapping no content in code.
from .common import reject_symlinks
import stat
reject_symlinks(args.body_file)
descriptor = os.open(args.body_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > MAX_NOTE_BYTES:
raise ValueError("Additional body must be a bounded regular file")
text = handle.read(MAX_NOTE_BYTES + 1).decode("utf-8")
body += "\n\n" + _text(text, maximum=MAX_NOTE_BYTES)
result["body"] = body.strip()
return result
def _cell(value: Any) -> str:
return html.escape(str(value if value is not None else "")).replace("|", "&#124;").replace("`", "&#96;").replace("\n", " ")
def render_note(note: dict, evidence: dict | None, target: NoteTarget, key: str) -> tuple[str, str]:
identity = {"target": target.url, "key": identifier(key),
"evidence": evidence["run_id"] if evidence else digest(note)}
marker = MARKER_PREFIX + digest(identity) + " -->"
lines = [marker, "## Development evidence", "", f"Issue: {target.url}", ""]
for field, title in (("summary", "Summary"), ("next", "Next / remaining")):
if note[field]:
lines += [f"### {title}", "", *["- " + item for item in note[field]], ""]
if note["body"]:
lines += [note["body"], ""]
if evidence:
lines += ["### Recorded check evidence", "", f"Run: `{evidence['run_id']}`; reported result: **{evidence['status']}**.",
f"Receipt origin: `{evidence['origin']}`; source comparison: `{evidence['source_state']}`.",
f"Recorded source fingerprint: `{evidence['source_fingerprint']}`.",
f"Finished: {_cell(evidence['finished_at'])}; receipt: `{_cell(evidence['receipt_path'])}`.", "",
"| Stage | Reported status | Exit | Seconds | Local log reference |", "| --- | --- | --- | --- | --- |"]
for stage in evidence["stages"]:
lines.append("| " + " | ".join(_cell(stage.get(field)) for field in ("id", "status", "exit_code", "duration_seconds", "log_path")) + " |")
if evidence.get("coverage_notes"):
lines += ["", "### Coverage limitations / checks not included", "",
"A passing recorded stage does not mean these omitted checks ran.", "",
*["- " + _cell(value) for value in evidence["coverage_notes"]]]
lines += ["", evidence["attestation"], "Local logs are referenced only; their content has not been read or uploaded.", ""]
lines += ["This comment does not close the issue, complete its review, or change its checklist."]
body = redact("\n".join(lines).rstrip() + "\n")
if len(body.encode()) > MAX_NOTE_BYTES:
raise ValueError("Rendered note exceeds its size bound")
return marker, body
def _token(env_file: Path | None) -> str:
values = {}
if env_file is not None:
from .common import reject_symlinks
reject_symlinks(env_file)
import stat
descriptor = os.open(env_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 65536:
raise ValueError("Credential file must be a bounded regular file")
content = handle.read(65537)
if len(content) > 65536:
raise ValueError("Credential file exceeds its size limit")
for line in content.decode("utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if line.startswith("export "):
line = line[7:].strip()
key, separator, value = line.partition("=")
if not separator or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", key.strip()):
raise ValueError("Invalid credential file format")
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
value = value[1:-1]
if key.strip() in values:
raise ValueError("Duplicate credential file keys")
values[key.strip()] = value
token = os.environ.get("GITEA_TOKEN") or values.get("GITEA_TOKEN")
if not token or len(token) > 8192 or any(char.isspace() for char in token):
raise ValueError("GITEA_TOKEN is required for --apply; use the environment or --env-file")
return token
def make_client(target: NoteTarget, token: str):
return GiteaClient(RepoTarget(target.base_url, target.owner, target.repository), token)
def _issue(client, target: NoteTarget) -> dict:
issue = client.request_json("GET", target.path)
if (not isinstance(issue, dict) or issue.get("number") != target.issue or issue.get("html_url") != target.url
or issue.get("pull_request") is not None):
raise ValueError("Remote issue identity does not match the exact target")
_positive(issue.get("id"))
if target.issue_id is not None and issue["id"] != target.issue_id:
raise ValueError("Remote issue ID changed from the target plan")
return issue
def _comments(client, target: NoteTarget) -> list[dict]:
comments, seen = [], set()
for page in range(1, 10001):
values = client.request_json("GET", target.path + "/comments", query={"page": page, "limit": 50})
if not isinstance(values, list):
raise ValueError("Remote comment pagination did not return a list")
if not values:
return comments
for comment in values:
if not isinstance(comment, dict) or not isinstance(comment.get("body"), str):
raise ValueError("Remote comment has an invalid shape")
identity = _positive(comment.get("id"))
if identity in seen:
raise ValueError("Repeated comment pagination; cannot establish a complete duplicate check")
seen.add(identity)
comments.append(comment)
if len(comments) > 100000:
raise ValueError("Remote comments exceed the bounded duplicate-check limit")
raise ValueError("Remote comment pagination did not terminate")
def _existing(comments: list[dict], marker: str, body: str) -> dict | None:
matches = [comment for comment in comments if marker in comment["body"]]
if len(matches) > 1 or (matches and matches[0]["body"] != body):
raise ValueError("Evidence marker collision; existing comments are preserved")
return matches[0] if matches else None
def _readback(client, target: NoteTarget, comment: dict, body: str) -> dict:
identity = _positive(comment.get("id"))
fresh = client.request_json("GET", repo_path(target.owner, target.repository, f"/issues/comments/{identity}"))
issue_api_url = target.base_url + "/api/v1" + target.path
if not isinstance(fresh, dict) or fresh.get("id") != identity or fresh.get("body") != body:
raise ValueError("Posted comment read-back did not match")
if not fresh.get("html_url") and not fresh.get("issue_url"):
raise ValueError("Comment read-back has no issue binding")
if fresh.get("issue_url") and fresh["issue_url"] != issue_api_url:
raise ValueError("Comment read-back belongs to another issue")
if fresh.get("html_url") and fresh["html_url"].split("#", 1)[0] != target.url:
raise ValueError("Comment read-back URL belongs to another issue")
return fresh
def handle_note(args) -> dict:
targets = _targets(args)
note = _structured_note(args)
evidence = evidence_record(args.evidence, args)
if not evidence and not any((note["summary"], note["next"], note["body"])):
raise ValueError("Provide evidence or a nonempty structured note")
if args.retry_uncertain and not args.apply:
raise ValueError("--retry-uncertain requires --apply")
prepared = [(target, *render_note(note, evidence, target, args.key)) for target in targets]
result = {"schema_version": 1, "operation": "issues.note", "apply": bool(args.apply),
"evidence": evidence, "targets": [{**target.record(), "marker": marker, "body": body, "status": "would-post"} for target, marker, body in prepared],
"summary": [f"{'Apply' if args.apply else 'Offline dry run'}: {len(targets)} exact issue target(s); issue bodies and states are preserved."]}
if evidence and evidence["coverage_notes"]:
result["summary"].append(f"Evidence has {len(evidence['coverage_notes'])} coverage limitation(s), retained in the note; omitted checks are not claimed as passed.")
if not args.apply:
return result
token = _token(args.env_file)
# Tokens loaded from an explicit file are not inserted into the process environment.
for record in result["targets"]:
if token in record["body"]:
raise ValueError("A credential occurs in note content; refusing publication")
state = state_root(Path(args.workspace_root), args.state_dir)
with ExitStack() as stack:
for target, marker, _body in sorted(prepared, key=lambda item: item[0].url):
stack.enter_context(resource_lock(state / "locks", "issues.note:" + marker))
clients, bindings, journals = {}, {}, {}
try:
# Validate every target and every existing marker before the first write.
for target, marker, body in prepared:
client = make_client(target, token)
stack.callback(client.close)
clients[target.url] = client
bindings[target.url] = _issue(client, target)["id"]
_existing(_comments(client, target), marker, body)
journal_path = state / "issue-notes" / (hashlib.sha256(marker.encode()).hexdigest() + ".json")
prior = read_json(journal_path) if journal_path.exists() else None
if prior is not None and (not isinstance(prior, dict) or prior.get("schema_version") != 1
or prior.get("target") != target.url or prior.get("body_digest") != digest(body)
or prior.get("issue_id") != bindings[target.url]
or prior.get("status") not in {"posting", "uncertain", "verified"}):
raise ValueError("Local evidence journal identity collision; inspect before retrying")
journals[target.url] = (journal_path, prior)
for index, (target, marker, body) in enumerate(prepared):
client = clients[target.url]
output = result["targets"][index]
journal_path, prior = journals[target.url]
issue = _issue(client, target)
if issue["id"] != bindings[target.url]:
raise ValueError("Issue identity changed after preflight")
existing = _existing(_comments(client, target), marker, body)
if existing:
verified = _readback(client, target, existing, body)
output.update(status="existing-verified", comment_id=verified["id"])
atomic_json(journal_path, {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "verified", "comment_id": verified["id"]})
continue
if prior and prior.get("status") in {"uncertain", "posting", "verified"} and not args.retry_uncertain:
output["status"] = "uncertain-retry-required"
result["_exit_code"] = 2
result["summary"].append("An earlier POST is not visible after complete reconciliation; explicit --retry-uncertain is required. No further posts attempted.")
break
journal = {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "posting"}
atomic_json(journal_path, journal)
try:
posted = client.request_json("POST", target.path + "/comments", body={"body": body})
verified = _readback(client, target, posted, body)
unique = _existing(_comments(client, target), marker, body)
if unique is None or unique["id"] != verified["id"]:
raise ValueError("New comment is not uniquely visible in its issue")
output.update(status="posted-verified", comment_id=verified["id"])
except Exception:
# POST is never replayed automatically, even after a timeout or bad response.
atomic_json(journal_path, {**journal, "status": "uncertain"})
try:
observed = _existing(_comments(client, target), marker, body)
verified = _readback(client, target, observed, body) if observed else None
except Exception:
verified = None
if verified is None:
output["status"] = "uncertain"
result["_exit_code"] = 2
result["summary"].append("POST/read-back could not be confirmed. No automatic retry and no further posts; reconcile this target before continuing.")
break
output.update(status="reconciled-verified", comment_id=verified["id"])
atomic_json(journal_path, {**journal, "status": "verified", "comment_id": output["comment_id"]})
except Exception:
# HTTP response/error bodies may contain secrets; never return them to a report.
result["_exit_code"] = 2
result["summary"].append("Gitea validation or local journal checks failed; existing issues/comments were preserved. No further posts attempted.")
finally:
token = ""
for record in result["targets"]:
if record["status"] == "would-post":
record["status"] = "not-attempted"
result["summary"].append("; ".join(f"{record['url']}: {record['status']}" for record in result["targets"]))
return result
File diff suppressed because it is too large Load Diff
+251
View File
@@ -0,0 +1,251 @@
"""Read-only run discovery and bounded provisional output, never passing evidence."""
from __future__ import annotations
from datetime import datetime
import os
from pathlib import Path
import re
import time
from .common import (
atomic_json,
identifier,
now,
read_json,
redact,
reject_symlinks,
state_root,
)
MAX_LIVE_BYTES = 65536
MAX_HISTORY_ENTRIES = 10000
MAX_HISTORY_SCAN = 500
ANSI = re.compile(r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\))")
CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
def display_text(value: str) -> str:
return redact(CONTROLS.sub("", ANSI.sub("", value)))
def _closed_lines(value: bytes) -> bytes:
boundary = value.rfind(b"\n")
return value[: boundary + 1] if boundary >= 0 else b""
def capture_text(snapshot, *, provisional: bool = False) -> str:
"""Do not expose partial secret/context fragments at live or retention boundaries."""
data = snapshot.stdout
omitted = snapshot.omitted_stdout_bytes
if omitted:
head = _closed_lines(data[: snapshot.stdout_head_bytes])
tail = data[snapshot.stdout_head_bytes :]
# A rolling tail can start inside Authorization or a secret: drop that fragment.
tail = tail.partition(b"\n")[2]
if provisional:
tail = _closed_lines(tail)
return (
display_text(head.decode("utf-8", errors="replace"))
+ f"\n[Output truncated: {omitted} bytes omitted between retained head and tail; cut boundary lines are withheld.]\n"
+ display_text(tail.decode("utf-8", errors="replace"))
)
if provisional:
data = _closed_lines(data)
return display_text(data.decode("utf-8", errors="replace"))
def bounded_display(value: str, maximum: int, *, tail_only: bool = False) -> str:
if type(maximum) is not int or maximum < 1:
raise ValueError("Display bound must be a positive integer")
encoded = value.encode("utf-8")
if len(encoded) <= maximum:
return value
if tail_only:
return encoded[-maximum:].decode("utf-8", errors="ignore")
marker = b"\n[Display bound: middle omitted; retained beginning and final output follow.]\n"
if maximum <= len(marker):
return encoded[-maximum:].decode("utf-8", errors="ignore")
available = max(0, maximum - len(marker))
head = available // 2
return (
encoded[:head].decode("utf-8", errors="ignore")
+ marker.decode()
+ encoded[-(available - head) :].decode("utf-8", errors="ignore")
)
def write_live(log_path: Path, stage_id: str, snapshot, started: float) -> None:
# Complete lines only, even at final callback: the separately finalized log
# may include an unterminated final line after normal redaction.
excerpt = bounded_display(
capture_text(snapshot, provisional=True), MAX_LIVE_BYTES, tail_only=True
)
atomic_json(
log_path.with_suffix(".live.json"),
{
"schema_version": 1,
"run_id": log_path.parent.name,
"stage_id": stage_id,
"provisional": True,
"updated_at": now(),
"elapsed_seconds": round(time.monotonic() - started, 3),
"output_truncated": snapshot.truncated,
"omitted_bytes": snapshot.omitted_stdout_bytes,
"excerpt": excerpt,
},
)
def read_live(
workspace_root: Path, state_dir: Path | None, run_id: str, stage_id: str
) -> dict:
identifier(run_id)
identifier(stage_id)
path = (
state_root(workspace_root, state_dir)
/ "runs"
/ run_id
/ (stage_id + ".live.json")
)
if not path.exists() and not path.is_symlink():
return {"provisional": True, "excerpt": "", "live_available": False}
payload = read_json(path, max_bytes=MAX_LIVE_BYTES * 6 + 8192)
if (
not isinstance(payload, dict)
or payload.get("schema_version") != 1
or payload.get("run_id") != run_id
or payload.get("stage_id") != stage_id
or payload.get("provisional") is not True
or not isinstance(payload.get("excerpt"), str)
or len(payload["excerpt"].encode("utf-8")) > MAX_LIVE_BYTES
or not isinstance(payload.get("updated_at"), str)
or type(payload.get("elapsed_seconds")) not in {int, float}
or payload["elapsed_seconds"] < 0
or type(payload.get("output_truncated")) is not bool
):
raise ValueError("Invalid provisional log snapshot")
return {
"provisional": True,
"live_available": True,
"excerpt": display_text(payload["excerpt"]),
"updated_at": payload.get("updated_at"),
"elapsed_seconds": payload.get("elapsed_seconds"),
"output_truncated": payload.get("output_truncated"),
}
def elapsed_seconds(receipt: dict) -> float | None:
try:
started = datetime.fromisoformat(receipt["generated_at"])
finished = datetime.fromisoformat(receipt.get("finished_at") or now())
return max(0, round((finished - started).total_seconds(), 3))
except (KeyError, TypeError, ValueError):
return None
def list_runs(args) -> dict:
from .runner import read_receipt
limit = getattr(args, "limit", 10)
before = getattr(args, "before", None)
if type(limit) is not int or not 1 <= limit <= 100:
raise ValueError("Run history limit must be between 1 and 100")
if before:
identifier(before)
base = state_root(args.workspace_root, args.state_dir) / "runs"
reject_symlinks(base)
names = []
if base.exists():
with os.scandir(base) as entries:
for index, entry in enumerate(entries):
if index >= MAX_HISTORY_ENTRIES:
raise ValueError(
"Run history exceeds its directory bound; archive old evidence explicitly before listing"
)
if not entry.name.startswith("."):
identifier(entry.name)
names.append(entry.name)
names = sorted(
(name for name in names if before is None or name < before), reverse=True
)
rows, examined, cursor = [], 0, None
wanted_project = (
str(args.project.resolve()) if getattr(args, "project", None) else None
)
for identity in names:
examined += 1
cursor = identity
try:
record = read_receipt(args.workspace_root, args.state_dir, identity)
if wanted_project and record.get("project_file") != wanted_project:
if examined >= MAX_HISTORY_SCAN:
break
continue
rows.append(
{
"run_id": identity,
"status": record["status"],
"phase": record.get("phase"),
"profile": record.get("profile"),
"generated_at": record.get("generated_at"),
"elapsed_seconds": elapsed_seconds(record),
"snapshot_verified": record["snapshot_verified"],
"passed_stages": sum(
item["status"] == "passed" for item in record["stages"]
),
"total_stages": len(record["stages"]),
}
)
except (OSError, ValueError) as exc:
# Corrupt/newest evidence is visible, never silently replaced with an older pass.
rows.append(
{
"run_id": display_text(identity),
"status": "invalid",
"error": display_text(str(exc)),
}
)
if len(rows) >= limit or examined >= MAX_HISTORY_SCAN:
break
next_cursor = cursor if examined < len(names) else None
lines = [
f"{item['run_id']}: {item['status']}"
+ (
f" ({item.get('profile')}; {item.get('elapsed_seconds')}s)"
if item["status"] != "invalid"
else "" + item["error"]
)
for item in rows
]
if not lines:
lines = ["No matching check runs found; no verification is implied."]
if next_cursor:
lines.append("More history: use --before " + display_text(next_cursor))
return {
"runs": rows,
"next_cursor": next_cursor,
"examined": examined,
"summary": lines,
"_exit_code": 1 if any(item["status"] == "invalid" for item in rows) else 0,
}
def latest_run(args) -> dict:
from argparse import Namespace
from .runner import read_receipt, summarize
selected = list_runs(Namespace(**{**vars(args), "limit": 1}))
if not selected["runs"]:
return {
"status": "not_found",
"summary": selected["summary"],
"next_cursor": selected["next_cursor"],
"_exit_code": 1,
}
row = selected["runs"][0]
if row["status"] == "invalid":
raise ValueError(
"Latest run is invalid; inspect run history instead of assuming an older pass"
)
return summarize(read_receipt(args.workspace_root, args.state_dir, row["run_id"]))
+173
View File
@@ -0,0 +1,173 @@
"""Bounded package metadata and conservative direct-Node test discovery."""
from __future__ import annotations
from pathlib import Path
import re
import shlex
from .common import digest, read_json, redact_argv, reject_symlinks
MAX_PACKAGE_BYTES = 1024 * 1024
MAX_SCRIPTS = 512
CORE_COMPONENT_SUITES = (
"data-grid-actions",
"dialog-focus",
"explorer-tree",
"icon-button",
"layout-primitives",
"mail-components",
"metric-card",
"page-layout",
"workspace-layout",
"people-picker",
"password-field",
"resource-access",
"action-blocker",
"documentation-help",
"selection-list",
"wysiwyg-editor",
)
CORE_RUNNER = "scripts/run-component-tests.mjs"
def read_package(path: Path) -> dict:
try:
package = read_json(path, max_bytes=MAX_PACKAGE_BYTES)
except (OSError, ValueError) as exc:
raise ValueError(f"Cannot safely read package metadata: {path}") from exc
if not isinstance(package, dict):
raise ValueError(f"Package metadata must be an object: {path}")
scripts = package.get("scripts", {})
if not isinstance(scripts, dict) or len(scripts) > MAX_SCRIPTS:
raise ValueError(f"Package scripts must be a bounded object: {path}")
for name, command in scripts.items():
if (
not isinstance(name, str)
or not 1 <= len(name) <= 128
or any(char in name for char in "\0\r\n")
):
raise ValueError(f"Package script names must be bounded strings: {path}")
if (
not isinstance(command, str)
or not 1 <= len(command) <= 8192
or "\0" in command
):
raise ValueError(
f"Package script commands must be bounded nonempty strings: {path}"
)
return package
def core_component_alias(
repo_name: str, name: str, command: str, package_path: Path
) -> str | None:
if repo_name != "govoplan-core" or package_path.parent.name != "webui":
return None
if name == "test:components" and command == f"node {CORE_RUNNER}":
return "all"
for suite in CORE_COMPONENT_SUITES:
if name == "test:" + suite and command == f"node {CORE_RUNNER} {suite}":
return suite
return None
def direct_node(command: str, package_path: Path) -> tuple[list[str] | None, str]:
try:
parts = shlex.split(command)
except ValueError:
return None, "Malformed command quoting; no command was guessed or executed."
if not parts or parts[0] != "node":
return (
None,
"Not a direct Node source test; use its explicitly reviewed owning workflow.",
)
offset = 2 if len(parts) > 1 and parts[1] == "--test" else 1
if len(parts) != offset + 1 or not re.fullmatch(
r"(?:scripts|tests)/[A-Za-z0-9_.-]+\.mjs", parts[offset]
):
return (
None,
"Compound command, flags or arguments are unsupported by scoped source discovery.",
)
if parts[offset] == CORE_RUNNER:
return (
None,
"Only exact known Core component aliases belong to the shared UI batch.",
)
target = package_path.parent / parts[offset]
try:
reject_symlinks(target)
if not target.is_file() or not target.resolve().is_relative_to(
package_path.parent.resolve()
):
return None, "Declared test target is missing or escapes its package."
except (OSError, ValueError):
return None, "Declared test target is not a safe regular package file."
return [
"{node}",
*parts[1:offset],
str(target),
], "Direct package-owned source test."
def declared_tests(repo, package_path: Path) -> list[dict]:
package = read_package(package_path)
result = []
for name, command in sorted(package.get("scripts", {}).items()):
if name != "test" and not name.startswith("test:"):
continue
component = core_component_alias(repo.name, name, command, package_path)
argv, reason = (
direct_node(command, package_path)
if component is None
else (None, "Exact known Core component alias.")
)
result.append(
{
"repo": repo.name,
"package_path": str(package_path),
"name": name,
"command_sha256": digest(command),
"argv": redact_argv(argv) if argv else None,
"component_suite": component,
"reason": reason,
"_argv": argv,
}
)
return result
def discovered_sources(repo) -> list[dict]:
webui = repo.path / "webui"
found = {}
for folder in ("scripts", "tests"):
for pattern in ("test-interface-pattern*.mjs", "*structure*.mjs"):
for target in (webui / folder).glob(pattern):
command = (
"node "
+ ("--test " if target.name.endswith(".test.mjs") else "")
+ target.relative_to(webui).as_posix()
)
argv, reason = direct_node(command, webui / "package.json")
found[str(target)] = {
"repo": repo.name,
"package_path": str(webui / "package.json"),
"name": "file:" + target.relative_to(webui).as_posix(),
"discovered": True,
"command_sha256": digest(command),
"argv": redact_argv(argv) if argv else None,
"component_suite": None,
"reason": reason,
"_argv": argv,
}
if len(found) > MAX_SCRIPTS:
raise ValueError(
f"Too many discovered source checks in {repo.name}"
)
return [found[key] for key in sorted(found)]
def source_stage_id(repo_name: str, argv: list[str]) -> str:
stem = Path(argv[-1]).stem
return f"{repo_name}.{stem}"[:110] + "." + digest(argv)[:12]
+300
View File
@@ -0,0 +1,300 @@
"""Bounded subprocess capture shared by execution, discovery and environment probes."""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
import selectors
import signal
import subprocess
import threading
import time
from typing import Callable
@dataclass(frozen=True)
class OutputSnapshot:
"""Immutable bounded output; raw bytes are decoded only at the display boundary."""
stdout: bytes
stderr: bytes
truncated: bool
omitted_stdout_bytes: int
omitted_stderr_bytes: int
stdout_head_bytes: int
stderr_head_bytes: int
final: bool
def text(self, stream: str = "stdout") -> str:
"""Render each retained segment separately, never joining cut UTF-8 sequences.
This is not secret redaction. Callers must redact before publishing or
persisting a snapshot, including withholding partial live lines as needed.
"""
if stream not in {"stdout", "stderr"}:
raise ValueError("Output stream must be stdout or stderr")
data = getattr(self, stream)
omitted = getattr(self, "omitted_" + stream + "_bytes")
if not omitted:
return data.decode("utf-8", errors="replace")
split = getattr(self, stream + "_head_bytes")
return (
data[:split].decode("utf-8", errors="replace")
+ f"\n[{omitted} output bytes omitted between retained segments]\n"
+ data[split:].decode("utf-8", errors="replace")
)
@dataclass(frozen=True)
class Capture:
stdout: bytes
stderr: bytes
returncode: int
status: str
truncated: bool
omitted_stdout_bytes: int = 0
omitted_stderr_bytes: int = 0
stdout_head_bytes: int = 0
stderr_head_bytes: int = 0
def snapshot(self) -> OutputSnapshot:
return OutputSnapshot(
self.stdout,
self.stderr,
self.truncated,
self.omitted_stdout_bytes,
self.omitted_stderr_bytes,
self.stdout_head_bytes,
self.stderr_head_bytes,
True,
)
class _OutputBuffer:
def __init__(self, limit: int, mode: str):
self.limit, self.mode, self.total = limit, mode, 0
self.head, self.tail = bytearray(), bytearray()
self.head_limit = limit if mode == "prefix" else limit // 2
self.tail_limit = limit - self.head_limit
def append(self, data: bytes) -> None:
self.total += len(data)
count = min(len(data), self.head_limit - len(self.head))
self.head.extend(data[:count])
remaining = data[count:]
if self.tail_limit and remaining:
if len(remaining) >= self.tail_limit:
self.tail[:] = remaining[-self.tail_limit :]
else:
excess = max(0, len(self.tail) + len(remaining) - self.tail_limit)
del self.tail[:excess]
self.tail.extend(remaining)
@property
def omitted(self) -> int:
return self.total - len(self.head) - len(self.tail)
def value(self) -> bytes:
return bytes(self.head) + bytes(self.tail)
def group_exists(pid: int) -> bool:
try:
os.killpg(pid, 0)
return True
except ProcessLookupError:
return False
def stop_group(process: subprocess.Popen) -> None:
try:
os.killpg(process.pid, signal.SIGTERM)
except ProcessLookupError:
return
deadline = time.monotonic() + 0.5
while time.monotonic() < deadline and group_exists(process.pid):
process.poll()
time.sleep(0.02)
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
process.wait(timeout=3)
def run_captured(
argv: list[str],
*,
cwd: Path | str | None = None,
env: dict[str, str] | None = None,
timeout: float = 30,
max_stdout: int = 1024 * 1024,
max_stderr: int = 65536,
input_bytes: bytes | None = None,
cancelled: threading.Event | None = None,
merge_stderr: bool = False,
terminate_on_limit: bool = True,
capture_mode: str = "prefix",
on_output: Callable[[OutputSnapshot], None] | None = None,
) -> Capture:
"""No shell, capped memory, finite deadline, owned process-group cleanup.
Prefix capture preserves probe semantics. Head/tail capture retains the
beginning and actual latest output within the same byte bound. Optional
callbacks receive a first-data snapshot, at most one dirty update per second,
and a final snapshot. Callback failures propagate after owned-process cleanup.
This is not a sandbox: a deliberately detached new process session is outside
the original process group. Only run trusted project commands.
"""
if capture_mode not in {"prefix", "head_tail"}:
raise ValueError("Capture mode must be prefix or head_tail")
if any(type(value) is not int or value < 0 for value in (max_stdout, max_stderr)):
raise ValueError("Output bounds must be nonnegative integers")
buffers = {
"stdout": _OutputBuffer(max_stdout, capture_mode),
"stderr": _OutputBuffer(max_stderr, capture_mode),
}
last_notified, notified_total = None, -1
def snapshot(final: bool) -> OutputSnapshot:
out, err = buffers["stdout"], buffers["stderr"]
return OutputSnapshot(
out.value(),
err.value(),
bool(out.omitted or err.omitted),
out.omitted,
err.omitted,
len(out.head),
len(err.head),
final,
)
def notify(final: bool = False) -> None:
nonlocal last_notified, notified_total
if on_output is None:
return
total = sum(buffer.total for buffer in buffers.values())
instant = time.monotonic()
if final or (
total > 0
and total != notified_total
and (last_notified is None or instant - last_notified >= 1)
):
on_output(snapshot(final))
last_notified, notified_total = instant, total
process = subprocess.Popen(
argv,
cwd=cwd,
env=env,
stdin=subprocess.PIPE if input_bytes is not None else subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT if merge_stderr else subprocess.PIPE,
start_new_session=True,
)
pending_input = memoryview(input_bytes or b"")
selector = None
try:
selector = selectors.DefaultSelector()
assert process.stdout is not None
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
if process.stderr:
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
if process.stdin:
if pending_input:
selector.register(process.stdin, selectors.EVENT_WRITE, "stdin")
else:
process.stdin.close()
except BaseException:
stop_group(process)
if selector is not None:
selector.close()
for handle in (process.stdin, process.stdout, process.stderr):
if handle and not handle.closed:
handle.close()
raise
deadline = time.monotonic() + timeout
exited_at = None
state = None
try:
while selector.get_map() or process.poll() is None:
if cancelled and cancelled.is_set():
state = "interrupted"
break
if time.monotonic() >= deadline:
state = "timed_out"
break
for key, _ in selector.select(
timeout=min(0.1, max(0, deadline - time.monotonic()))
):
if key.data == "stdin":
try:
written = os.write(key.fd, pending_input[:4096])
pending_input = pending_input[written:]
except BrokenPipeError:
pending_input = memoryview(b"")
if not pending_input:
selector.unregister(key.fileobj)
key.fileobj.close()
continue
data = os.read(key.fd, 65536)
if not data:
selector.unregister(key.fileobj)
continue
target = buffers[key.data]
target.append(data)
if target.omitted and terminate_on_limit:
state = "output_limit"
break
notify()
if state:
break
if process.poll() is not None:
exited_at = exited_at or time.monotonic()
if selector.get_map() and time.monotonic() - exited_at >= 0.5:
state = "leaked_process"
break
if state:
stop_group(process)
else:
process.wait(timeout=3)
# A child can redirect all output then outlive an otherwise successful parent.
grace = time.monotonic() + 0.1
while group_exists(process.pid) and time.monotonic() < grace:
time.sleep(0.01)
if group_exists(process.pid):
state = "leaked_process"
stop_group(process)
else:
state = "passed" if process.returncode == 0 else "failed"
notify(final=True)
finally:
if process.poll() is None or group_exists(process.pid):
stop_group(process)
selector.close()
for handle in (process.stdin, process.stdout, process.stderr):
if handle and not handle.closed:
handle.close()
output = snapshot(True)
return Capture(
output.stdout,
output.stderr,
process.returncode,
state,
output.truncated,
output.omitted_stdout_bytes,
output.omitted_stderr_bytes,
output.stdout_head_bytes,
output.stderr_head_bytes,
)
def require_capture(argv: list[str], **kwargs) -> Capture:
result = run_captured(argv, **kwargs)
if result.status in {"timed_out", "interrupted", "output_limit", "leaked_process"}:
raise ValueError(
f"Bounded subprocess did not complete normally: {result.status}"
)
return result
+366
View File
@@ -0,0 +1,366 @@
"""Headless adapter to the existing, receipt-bound release console lifecycle.
The ASGI application runs in this process; no HTTP socket or background server
is started. Critical release orchestration remains in the existing service.
"""
from __future__ import annotations
import argparse
import asyncio
import importlib
from pathlib import Path
import re
import secrets
import sys
from typing import Any
from urllib.parse import quote
from .common import redact
META_ROOT = Path(__file__).resolve().parents[3]
RELEASE_ROOT = META_ROOT / "tools/release"
_REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}\Z")
_VERSION = re.compile(r"[0-9]+\.[0-9]+\.[0-9]+(?:[-+][0-9A-Za-z][0-9A-Za-z.-]*)?\Z")
_RUN = re.compile(r"rr-(?:[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}|request-[0-9a-f]{64})\Z")
_STEP = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:@-]{0,159}\Z")
_REQUEST = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:@/-]{7,127}\Z")
def _typed(pattern: re.Pattern[str], label: str):
def parse(value: str) -> str:
if not pattern.fullmatch(value):
raise argparse.ArgumentTypeError(f"Invalid {label}.")
return value
return parse
def _limit(value: str) -> int:
try:
result = int(value)
except ValueError as exc:
raise argparse.ArgumentTypeError("Limit must be an integer from 1 to 100.") from exc
if not 1 <= result <= 100:
raise argparse.ArgumentTypeError("Limit must be an integer from 1 to 100.")
return result
def _planning_options(parser: argparse.ArgumentParser, *, selection: bool) -> None:
if selection:
parser.add_argument("--repo", action="append", default=[], type=_typed(_REPOSITORY, "repository name"))
parser.add_argument("--repo-version", action="append", default=[], metavar="REPO=VERSION")
parser.add_argument("--target-version", type=_typed(_VERSION, "target version"))
parser.add_argument("--channel", default="stable")
parser.add_argument("--online", action="store_true", help="Allow the existing remote/catalog checks.")
parser.add_argument("--remote-tags", action="store_true", help="Explicitly inspect remote Git tags.")
parser.add_argument("--public-catalog", action="store_true", help="Explicitly inspect the public catalog.")
parser.add_argument("--include-migrations", action="store_true", help="Run migration audits; never applies migrations.")
def register(subparsers: Any) -> None:
"""Register release commands without importing FastAPI, HTTPX or Core."""
parser = subparsers.add_parser("release", help="Plan and operate durable GovOPlaN release runs.")
commands = parser.add_subparsers(dest="release_command", required=True)
for name in ("plan", "status", "create"):
command = commands.add_parser(name)
_planning_options(command, selection=name != "status")
if name == "status":
command.add_argument("--include-website", action="store_true")
if name == "create":
command.add_argument("--request-id", required=True, type=_typed(_REQUEST, "request ID"))
command.add_argument("--apply", action="store_true", help="Persist the frozen run; otherwise preview only.")
command.set_defaults(handler=handle)
listing = commands.add_parser("list", help="List bounded, workspace-scoped durable run history.")
listing.add_argument("--limit", type=_limit, default=20)
listing.add_argument("--cursor")
listing.set_defaults(handler=handle)
for name in ("show", "preview", "execute", "resume", "retry", "reconcile"):
command = commands.add_parser(name)
command.add_argument("run_id", type=_typed(_RUN, "run ID"))
if name in {"preview", "execute", "retry", "reconcile"}:
command.add_argument("step_id", type=_typed(_STEP, "step ID"))
if name in {"execute", "resume", "retry", "reconcile"}:
command.add_argument("--request-id", required=True, type=_typed(_REQUEST, "request ID"))
command.add_argument("--apply", action="store_true", help="Apply this explicit durable transition; otherwise preview only.")
if name in {"execute", "reconcile"}:
command.add_argument("--confirm", default="", help="Exact confirmation required by the existing release service.")
if name == "execute":
command.add_argument("--signing-key", action="append", default=[], metavar="KEY_ID=PRIVATE_KEY_FILE")
if name == "reconcile":
command.add_argument("--outcome", required=True, choices=("effect_absent", "effect_succeeded", "unresolved"))
command.set_defaults(handler=handle)
def _selection(args: argparse.Namespace) -> tuple[list[str], dict[str, str]]:
versions: dict[str, str] = {}
for item in args.repo_version:
repo, separator, version = item.partition("=")
if not separator or not _REPOSITORY.fullmatch(repo) or not _VERSION.fullmatch(version):
raise ValueError("--repo-version must be REPO=VERSION with a valid repository and version.")
if repo in versions and versions[repo] != version:
raise ValueError(f"Conflicting target versions for {repo}.")
versions[repo] = version
repos = list(dict.fromkeys([*args.repo, *versions]))
if not repos:
raise ValueError("Select at least one --repo or --repo-version explicitly.")
for repo in repos:
if repo not in versions and args.target_version:
versions[repo] = args.target_version
if args.release_command == "create" and any(repo not in versions for repo in repos):
raise ValueError("Creating a run requires an explicit version for every selected repository.")
return repos, versions
def _planning_query(args: argparse.Namespace) -> dict[str, Any]:
return {
"channel": args.channel,
"online": args.online,
"remote_tags": args.remote_tags,
"public_catalog": args.public_catalog or args.online,
"include_migrations": args.include_migrations,
**({"target_version": args.target_version} if args.target_version else {}),
}
def _load_application(args: argparse.Namespace) -> tuple[Any, str]:
# The generic CLI/help path stays dependency-light. The backend still checks
# its operator-controlled runtime and registered source origins on apply.
for name, loaded in list(sys.modules.items()):
if name in {"server", "govoplan_release"} or name.startswith(("server.", "govoplan_release.")):
expected = RELEASE_ROOT / ("server" if name.startswith("server") else "govoplan_release")
source = getattr(loaded, "__file__", None)
if not isinstance(source, str) or not Path(source).resolve().is_relative_to(expected.resolve()):
raise ValueError("A foreign module shadows the trusted GovOPlaN release service.")
# Reprioritize even when the path was added previously below an unrelated
# working directory. Validate cached packages before importing any submodule.
sys.path[:] = [str(RELEASE_ROOT), *(path for path in sys.path if path != str(RELEASE_ROOT))]
module = importlib.import_module("server.app")
if Path(module.__file__).resolve() != (RELEASE_ROOT / "server/app.py").resolve():
raise ValueError("A foreign server.app module shadows the GovOPlaN release service.")
token = secrets.token_urlsafe(32)
state_dir = getattr(args, "state_dir", None)
app = module.create_app(
workspace_root=Path(args.workspace_root).expanduser().resolve(),
token=token,
run_state_root=Path(state_dir) / "release-console" if state_dir is not None else None,
)
return app, token
def _error_detail(payload: Any) -> str:
detail = payload.get("detail") if isinstance(payload, dict) else None
if isinstance(detail, str):
return detail
if isinstance(detail, list):
# Validation input may contain signing-key arguments. Never echo it.
return "; ".join(
".".join(str(part) for part in item.get("loc", [])) + ": " + str(item.get("msg", "Invalid request"))
for item in detail if isinstance(item, dict)
)
return "The release service did not return a valid successful response."
class _ServiceError(Exception):
def __init__(self, status: int, payload: Any):
super().__init__(_error_detail(payload))
self.status = status
def _brief(value: Any, *, limit: int = 240) -> str:
"""Bound display-only fields; never serialize an executor or its arguments."""
if not isinstance(value, (str, int, float, bool)):
return "unknown"
text = " ".join(redact(str(value)).split())
return text if len(text) <= limit else text[:limit - 1] + ""
def _status(payload: dict[str, Any]) -> str:
status = payload.get("status")
for key in ("summary", "state"):
if isinstance(payload.get(key), dict):
status = payload[key].get("status", status)
if isinstance(payload.get("state_step"), dict):
status = payload["state_step"].get("state", status)
# Return the semantic value unchanged: display redaction must never affect
# failure exit codes, even when an environment secret happens to equal it.
return status if isinstance(status, str) else "ok"
def _summary_lines(name: str, payload: dict[str, Any], note: str | None = None) -> list[str]:
"""Compact, bounded projection; the unchanged JSON result retains details."""
lines = [f"Release {name}: {_brief(_status(payload))}."]
if note:
lines.append(note)
plan = payload.get("immutable", {}).get("plan", {}) if isinstance(payload.get("immutable"), dict) else payload
if not isinstance(plan, dict):
plan = {}
if isinstance(plan.get("source_preflight_ready"), bool):
prefix = "Frozen plan source" if "immutable" in payload else "Source"
lines.append(f"{prefix} preflight ready: {str(plan['source_preflight_ready']).lower()}.")
units = plan.get("units", [])
if isinstance(units, list):
for unit in units[:12]:
if isinstance(unit, dict):
lines.append(f"{_brief(unit.get('repo'))}: {_brief(unit.get('status', 'planned'))}; target {_brief(unit.get('target_version'))}.")
if len(units) > 12:
lines.append(f"{len(units) - 12} more selected repositories; use --json for every repository.")
dashboard = payload.get("summary")
if isinstance(dashboard, dict):
counts = [f"{dashboard[key]} {label}" for key, label in (
("repository_count", "repositories"), ("missing_count", "missing"),
("dirty_count", "dirty"), ("ahead_count", "ahead"),
("behind_count", "behind"), ("error_count", "errors"),
) if isinstance(dashboard.get(key), int)]
if counts:
lines.append("Repository status: " + ", ".join(counts) + ".")
findings = plan.get("gate_findings", payload.get("collection_errors", []))
if isinstance(findings, list):
for finding in findings[:4]:
if isinstance(finding, dict):
scope = f" ({_brief(finding['repo'])})" if finding.get("repo") else ""
lines.append(f"Gate {_brief(finding.get('code'))}{scope}: {_brief(finding.get('message'))}")
if len(findings) > 4:
lines.append(f"{len(findings) - 4} more gate findings; use --json for details.")
state = payload.get("state", {})
steps = state.get("steps", []) if isinstance(state, dict) else []
if isinstance(payload.get("state_step"), dict):
steps = [payload["state_step"]]
if isinstance(steps, list) and steps:
counts: dict[str, int] = {}
for step in steps:
if isinstance(step, dict):
status = _brief(step.get("state", "unknown"))
counts[status] = counts.get(status, 0) + 1
lines.append("Steps: " + ", ".join(f"{count} {state}" for state, count in sorted(counts.items())) + ".")
relevant = [step for step in steps if isinstance(step, dict) and step.get("state") != "succeeded"]
for step in relevant[:3]:
lines.append(f"Step {_brief(step.get('id'))}: {_brief(step.get('state'))}." +
(f" {_brief(step['disabled_reason'])}" if step.get("disabled_reason") else ""))
execution = payload.get("execution_result")
if isinstance(execution, dict):
lines.append(f"Executor result: {_brief(execution.get('status', 'recorded'))}.")
recommendation = payload.get("recommended_next", plan.get("recommended_action"))
if isinstance(recommendation, dict) and recommendation:
step = f" [{_brief(recommendation['step_id'])}]" if recommendation.get("step_id") else ""
lines.append(f"Next: {_brief(recommendation.get('id'))}{step}{_brief(recommendation.get('title'))}.")
if recommendation.get("remediation"):
lines.append(_brief(recommendation["remediation"]))
runs = payload.get("runs")
if isinstance(runs, list):
lines.append(f"{len(runs)} release runs in this page.")
for run in runs[:12]:
if isinstance(run, dict):
lines.append(f"{_brief(run.get('run_id'))}: {_brief(_status(run))}.")
if payload.get("next_cursor"):
lines.append("More history available; use --json for the next cursor.")
return lines
async def _run(args: argparse.Namespace) -> dict[str, Any]:
# Resolve CLI-only validation before loading the service or creating state.
if getattr(args, "project", None) is not None:
raise ValueError("Release uses the authoritative GovOPlaN catalog and does not accept --project overrides; select the registered --workspace-root instead.")
name = args.release_command
selection = _selection(args) if name in {"plan", "create"} else None
keys = getattr(args, "signing_key", [])
if len(keys) > 8 or any(not re.fullmatch(r"[A-Za-z0-9._-]{1,128}=.+", key) or "-----BEGIN" in key or "\n" in key or "\r" in key or len(key) > 4096 for key in keys):
raise ValueError("Provide at most eight --signing-key KEY_ID=PRIVATE_KEY_FILE arguments, never key material.")
import httpx
app, token = _load_application(args)
metadata = {
"workspace_root": str(app.state.workspace_root),
"state_location": str(app.state.release_runs.root),
"candidate_location": str(app.state.release_candidate_root),
}
transport = httpx.ASGITransport(app=app, raise_app_exceptions=False)
async with httpx.AsyncClient(
transport=transport, base_url="http://govoplan-devkit.invalid",
headers={"X-Release-Console-Token": token}, timeout=None,
follow_redirects=False,
) as client:
async def request(method: str, path: str, **kwargs: Any) -> dict[str, Any]:
response = await client.request(method, path, **kwargs)
try:
payload = response.json()
except ValueError:
payload = None
if response.status_code >= 400 or not isinstance(payload, dict):
raise _ServiceError(response.status_code, payload)
return payload
def result(payload: dict[str, Any], *, dry_run: bool = False, summary: str | None = None) -> dict[str, Any]:
status = _status(payload)
execution_failed = isinstance(payload.get("execution_result"), dict) and payload["execution_result"].get("status") == "failed"
return {
**metadata, "operation": name, "dry_run": dry_run,
"result": payload,
"_exit_code": 1 if execution_failed or status in {"blocked", "failed", "interrupted"} else 0,
"summary": [*_summary_lines(name, payload, summary), f"Durable state: {metadata['state_location']}"],
}
async def preview(run: dict[str, Any], step_id: str) -> dict[str, Any]:
plan = run.get("immutable", {}).get("plan", {})
plan_step = next((step for step in plan.get("dry_run_steps", []) if step.get("id") == step_id), None)
state_step = next((step for step in run.get("state", {}).get("steps", []) if step.get("id") == step_id), None)
if plan_step is None or state_step is None:
raise _ServiceError(404, {"detail": "Release run step was not found."})
if state_step.get("executor", {}).get("kind") == "catalog_publish":
return await request("POST", f"{run_path}/steps/{quote(step_id, safe='')}/preview", json={"remote": "origin"})
return {
"run_id": run["run_id"], "plan_step": plan_step, "state_step": state_step,
"note": "Frozen-plan inspection only; no executor was called and no live preflight is claimed.",
}
if name == "status":
payload = await request("GET", "/api/dashboard", params={**_planning_query(args), "include_website": args.include_website})
return result(payload)
if name == "list":
params = {"limit": args.limit, **({"cursor": args.cursor} if args.cursor else {})}
return result(await request("GET", "/api/release-runs", params=params))
if name in {"plan", "create"}:
assert selection is not None
repos, versions = selection
query = {
**_planning_query(args), "repos": ",".join(repos),
"repo_versions": ",".join(f"{repo}={version}" for repo, version in versions.items()),
}
if name == "plan" or not args.apply:
payload = await request("GET", "/api/selective-plan", params=query)
return result(payload, dry_run=True, summary="Release plan inspected; no run was created and no release step executed.")
body = {key: value for key, value in _planning_query(args).items() if key != "target_version"}
payload = await request("POST", "/api/release-runs", json={**body, "request_id": args.request_id, "repo_versions": versions})
return result(payload)
run_path = "/api/release-runs/" + quote(args.run_id, safe="")
if name == "show":
return result(await request("GET", run_path))
if name == "preview" or not args.apply:
run = await request("GET", run_path)
payload = await preview(run, args.step_id) if name in {"preview", "execute"} else run
return result(payload, dry_run=True, summary=f"Release {name} inspected; no durable transition or executor was invoked.")
body = {"request_id": args.request_id}
if name in {"execute", "reconcile"}:
body["confirm"] = args.confirm
if name == "execute":
body.update({"remote": "origin", "signing_keys": keys})
if name == "reconcile":
body["outcome"] = args.outcome
path = f"{run_path}/resume" if name == "resume" else f"{run_path}/steps/{quote(args.step_id, safe='')}/{name}"
return result(await request("POST", path, json=body))
def handle(args: argparse.Namespace) -> dict[str, Any]:
"""Return the common devkit JSON/summary envelope; never retry mutations."""
try:
return asyncio.run(_run(args))
except _ServiceError as exc:
summary = [f"Release {args.release_command} failed (HTTP {exc.status}): {exc}"]
if args.release_command in {"create", "execute", "resume", "retry", "reconcile"}:
summary.append("No automatic retry occurred. Inspect the run; reuse the same request ID for a known replay, or resume/reconcile an uncertain effect before a new attempt.")
return {"_exit_code": 1, "status": "error", "http_status": exc.status, "summary": summary}
except ModuleNotFoundError as exc:
return {"_exit_code": 2, "status": "unavailable", "summary": [f"Release commands need the GovOPlaN development dependencies ({exc.name} is unavailable)."]}
except ValueError as exc:
return {"_exit_code": 2, "status": "invalid", "summary": [str(exc)]}
+244
View File
@@ -0,0 +1,244 @@
"""Local, source-derived review guidance. Gitea remains the only review state log."""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import re
import stat
from urllib.parse import urlsplit
from .common import atomic_json, now, read_json, redact, reject_symlinks
from .issues import coverage_notes, evidence_record, _base_url, _name, _positive
from .workspace import inspect_repository, load_project, selected_repositories, source_fingerprint
MAX_SOURCES = 10000
MANUAL_CHECKS = (
("surfaces", "Confirm every route, pane, dialog, settings surface, widget, public form and contributed interface; source discovery is only a starting list."),
("display-edit", "Show compact readable data first; edit coherent groups in scoped dialogs. Record a reason for a deliberate bulk editing mode or other exception."),
("help", "Check documentation books beside meaningful visible headings/labels, including widgets, loading and configuration states; preserve optional Docs fallback."),
("actions", "Check predictable action order, Save/Cancel, unsaved drafts, destructive consequences and truthful unavailable-action explanations."),
("geometry", "Check shared cards/tables/dialogs, column resizing, pagination, long data, narrow layouts, zoom and the reported wide-window configuration."),
("states", "Exercise loading, empty, partial, error, stale, conflict and permission-denied states; verify no accidental mutation on read or cancel."),
("accessibility", "Exercise keyboard order, visible focus, accessible names, dialog focus restoration and non-color-only feedback."),
("language-docs", "Review English and German, long headings and module-owned user/admin documentation for every changed workflow and limitation."),
("boundaries", "Exercise tenant/authorization boundaries and optional-module absence; inspect headless modules' contributed interfaces rather than marking them complete automatically."),
("propagation", "Record applied principle revision, exceptions, fixes, evidence and remaining work in the module issue; propagate new rules to already-reviewed modules through the central issue."),
)
def register(subparsers) -> None:
parser = subparsers.add_parser("review", help="Assemble a module's local UI-review guidance (does not complete a review)")
parser.add_argument("module", help="Repository name, alias, or review inventory scope ID")
parser.add_argument("bundle_module", nargs="?", help="Also accepts: review bundle MODULE")
parser.add_argument("--profile", default="ui", help="Plan this registered check profile, without running it")
parser.add_argument("--evidence", help="Existing local run ID or explicit receipt JSON path")
parser.add_argument("--output", type=Path, help="Optional local JSON artifact; not a progress tracker")
parser.set_defaults(handler=handle_review)
def _project_path(root: Path, value: str) -> Path:
if not isinstance(value, str):
raise ValueError("Review inventory/principles paths must be strings")
raw = Path(value)
if raw.is_absolute() or ".." in raw.parts:
raise ValueError("Review configuration paths must be relative within the workspace")
path = root / raw
reject_symlinks(path)
if not path.resolve().is_relative_to(root.resolve()):
raise ValueError("Review input escapes the workspace")
return path
def _link(record: dict) -> dict:
if not isinstance(record, dict):
raise ValueError("Review issue link must be an object")
repo = _name(record.get("repository"))
number = _positive(record.get("number"))
url = record.get("url")
if not isinstance(url, str):
raise ValueError("Review issue URL is missing")
parsed = urlsplit(_base_url(url))
parts = parsed.path.rstrip("/").split("/")
if len(parts) < 5 or parts[-3:] != [repo, "issues", str(number)]:
raise ValueError("Review issue URL does not match its repository and number")
_name(parts[-4])
# Snapshot status/operation fields are deliberately not projected as live state.
return {"repository": repo, "number": number, "url": url}
def _issue_inventory(path: Path | None) -> tuple[list[dict], dict | None]:
if path is None or not path.exists():
return [], None
payload = read_json(path, max_bytes=2 * 1024 * 1024)
if not isinstance(payload, dict) or payload.get("schema_version") != 1:
raise ValueError("Review issue inventory requires schema_version 1")
rows = payload.get("issues")
if not isinstance(rows, list) or len(rows) > 1024:
raise ValueError("Review issue inventory must contain a bounded issues list")
issues, scopes, urls = [], set(), set()
for row in rows:
link = _link(row)
scope = row.get("scope_id")
if not isinstance(scope, str) or not re.fullmatch(r"[A-Za-z0-9_:.-]{1,128}", scope) or scope in scopes or link["url"] in urls:
raise ValueError("Review issue inventory has an invalid or duplicate scope/issue")
scopes.add(scope)
urls.add(link["url"])
issues.append({**link, "scope_id": scope, "name": str(row.get("name", scope)), "kind": str(row.get("kind", "unspecified"))})
epic = _link(payload["epic"]) if payload.get("epic") else None
return issues, epic
def _read_text(path: Path) -> str:
reject_symlinks(path)
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 1024 * 1024:
raise ValueError("Principles must be a bounded regular text file")
encoded = handle.read(1024 * 1024 + 1)
if len(encoded) > 1024 * 1024:
raise ValueError("Principles exceed the file-size bound")
return encoded.decode("utf-8")
def _principles(path: Path | None) -> dict:
if path is None or not path.exists():
return {"path": str(path) if path else None, "available": False, "revision": None, "rules": []}
text = _read_text(path)
revision = re.search(r"\bUI-\d{4}-\d{2}-\d{2}\b", text)
rules = [{"id": match.group(1), "title": match.group(2).strip()}
for match in re.finditer(r"^##\s+(UI-\d{2})\s*[—–:-]\s*(.+)$", text, re.MULTILINE)]
return {"path": str(path), "available": True, "revision": revision.group(0) if revision else None,
"content_sha256": hashlib.sha256(text.encode()).hexdigest(), "rules": rules}
def source_inventory(root: Path) -> dict:
"""Enumerate filenames only; never execute module manifests or import optional modules."""
groups = {name: [] for name in ("pages-and-navigation", "dialogs-and-embedded-editors", "settings-and-administration", "widgets-and-public-surfaces", "other-ui-sources")}
manifests, skipped = [], []
count = 0
for base, mode in ((root / "webui/src", "ui"), (root / "src", "backend")):
try:
reject_symlinks(base)
except ValueError:
skipped.append(str(base.relative_to(root)))
continue
for directory, dirs, names in os.walk(base, followlinks=False):
folder = Path(directory)
safe_dirs = []
for name in sorted(dirs):
child = folder / name
if child.is_symlink():
skipped.append(str(child.relative_to(root)))
elif name not in {"node_modules", ".git", "__pycache__", ".venv", "dist"}:
safe_dirs.append(name)
dirs[:] = safe_dirs
for name in sorted(names):
path = folder / name
if path.is_symlink():
skipped.append(str(path.relative_to(root)))
continue
if mode == "backend":
if name == "manifest.py":
manifests.append(str(path.relative_to(root)))
continue
if path.suffix not in {".tsx", ".jsx", ".vue", ".svelte"} and name not in {"index.ts", "module.ts", "routes.ts"}:
continue
count += 1
if count > MAX_SOURCES:
raise ValueError("Module UI source inventory exceeds its size bound")
relative = str(path.relative_to(root))
lower = relative.lower()
group = ("dialogs-and-embedded-editors" if any(part in lower for part in ("dialog", "modal", "editor")) else
"settings-and-administration" if any(part in lower for part in ("setting", "admin")) else
"widgets-and-public-surfaces" if any(part in lower for part in ("widget", "public")) else
"pages-and-navigation" if any(part in lower for part in ("/pages/", "page.", "navigation", "routes.ts")) else "other-ui-sources")
groups[group].append(relative)
return {"basis": "Static filename discovery, not a complete runtime surface inventory or a review result.",
"ui_source_count": count, "groups": groups, "manifest_paths": manifests,
"skipped_symlinks": skipped, "module_code_imported": False}
def _redact_tree(value):
if isinstance(value, str):
return redact(value)
if isinstance(value, list):
return [_redact_tree(item) for item in value]
if isinstance(value, dict):
return {key: _redact_tree(item) for key, item in value.items()}
return value
def handle_review(args) -> dict:
name = args.module
if name == "bundle":
if not args.bundle_module:
raise ValueError("review bundle requires a module")
name = args.bundle_module
elif args.bundle_module:
raise ValueError("Review accepts one module; use review MODULE")
workspace_root = Path(args.workspace_root).resolve()
project = load_project(workspace_root, args.project)
configured = project.config.get("review", {}) if args.project else {}
if not isinstance(configured, dict) or set(configured) - {"issue_inventory", "principles"}:
raise ValueError("Project review configuration accepts issue_inventory and principles paths")
meta = next((repo.path for repo in project.repositories if repo.name == "govoplan"), None)
core = next((repo.path for repo in project.repositories if repo.name == "govoplan-core"), None)
inventory_path = (_project_path(workspace_root, configured["issue_inventory"]) if "issue_inventory" in configured else
meta / "docs/project/ui-review-issue-inventory.json" if meta and not args.project else None)
principles_path = (_project_path(workspace_root, configured["principles"]) if "principles" in configured else
core / "docs/UI_DESIGN_PRINCIPLES.md" if core and not args.project else None)
issues, epic = _issue_inventory(inventory_path)
scope_matches = [item for item in issues if item["scope_id"] == name]
selected = selected_repositories(project, [scope_matches[0]["repository"] if scope_matches else name])
if len(selected) != 1:
raise ValueError("Review must resolve to exactly one registered repository")
repo = selected[0]
links = [item for item in issues if item["repository"] == repo.name]
inventory = source_inventory(repo.path)
principles = _principles(principles_path)
state = inspect_repository(repo)
warnings = []
if not links:
warnings.append("No module issue discovery link is configured; locate/create the canonical Gitea review issue before recording review work.")
if not principles["available"] or not principles["revision"]:
warnings.append("The principle document or dated UI revision is unavailable; confirm the governing rules before reviewing.")
if not inventory["ui_source_count"]:
warnings.append("No UI source filenames were discovered. Check contributed/headless/placeholder scope manually; this is not automatic N/A or completion.")
if inventory["skipped_symlinks"]:
warnings.append("Symlinked source paths were not traversed; the source starting inventory is incomplete.")
if state["errors"]:
warnings.append("Repository inspection reported errors; missing or unreadable source is not a clean review.")
try:
fingerprint = source_fingerprint(project)
except (OSError, ValueError):
fingerprint = None
warnings.append("Current workspace source identity could not be established; do not present attached historical evidence as current verification.")
from .catalog import build_stages
checks = build_stages(workspace_root, args.profile, [repo.name], False, project=args.project)
if not checks:
warnings.append("No automated stages are planned for this selection; this is not a passing verification result.")
evidence = evidence_record(args.evidence, args)
limits = list(dict.fromkeys([*coverage_notes(checks), *(evidence["coverage_notes"] if evidence else [])]))
warnings.extend("Coverage limitation: " + note for note in limits[:8])
if len(limits) > 8:
warnings.append(f"{len(limits) - 8} additional coverage limitations are retained in the JSON bundle.")
result = {"schema_version": 1, "operation": "review.bundle", "generated_at": now(), "workspace_root": str(workspace_root),
"project": project.name, "module": repo.name, "source_fingerprint": fingerprint, "repository": state,
"issue_links": links, "central_issue": epic, "issue_inventory_path": str(inventory_path) if inventory_path else None,
"state_authority": "Live Gitea issues are the canonical backlog and review state log. Discovery links do not report current issue state.",
"inventory": inventory, "principles": principles, "check_plan": {"profile": args.profile, "executed": False, "stages": checks},
"manual_checklist": [{"id": identity, "prompt": prompt} for identity, prompt in MANUAL_CHECKS],
"review_completion": "Not assessed. Automated checks and this bundle never complete a module review or update issue checklists.",
"evidence": evidence, "coverage_notes": limits, "warnings": warnings,
"summary": [f"Review bundle: {repo.name}; {inventory['ui_source_count']} UI source files, {len(checks)} planned checks (not executed).",
"Principle revision: " + (principles["revision"] or "unavailable"),
*["Module issue: " + item["url"] for item in links],
*(["Central issue: " + epic["url"]] if epic else []),
"Manual review remains unassessed; record outcomes and remaining work in Gitea.", *warnings]}
result = _redact_tree(result)
if args.output:
result["summary"].append("Local bundle artifact: " + redact(str(args.output)))
atomic_json(args.output, result)
return result
File diff suppressed because it is too large Load Diff
+142
View File
@@ -0,0 +1,142 @@
"""Dependency-free validation against the published portable-project schema."""
from __future__ import annotations
import math
from pathlib import Path
import re
from .common import META_ROOT, canonical, read_json
def _schema_value(value, schema: dict, definitions: dict, path: str) -> None:
if "$ref" in schema:
return _schema_value(
value,
definitions[schema["$ref"].removeprefix("#/$defs/")],
definitions,
path,
)
kind = schema.get("type")
numeric = type(value) is int or type(value) is float and math.isfinite(value)
matches = {
"object": isinstance(value, dict),
"array": isinstance(value, list),
"string": isinstance(value, str),
"number": numeric,
"integer": numeric and int(value) == value,
}
if kind and not matches[kind]:
raise ValueError(f"{path} must be a {kind}")
if "const" in schema and value != schema["const"]:
raise ValueError(f"{path} has an unsupported value")
if "enum" in schema and value not in schema["enum"]:
raise ValueError(f"{path} has an unsupported value")
if isinstance(value, dict):
properties = schema.get("properties", {})
unknown = set(value) - set(properties)
if schema.get("additionalProperties") is False and unknown:
names = ", ".join(key[:80] for key in sorted(unknown)[:4])
raise ValueError(f"Unknown field in {path}: {names}")
if set(schema.get("required", [])) - set(value):
raise ValueError(f"{path} is missing a required field")
for key, item in value.items():
if key in properties:
_schema_value(item, properties[key], definitions, f"{path}.{key}")
if isinstance(value, list):
if (
not schema.get("minItems", 0)
<= len(value)
<= schema.get("maxItems", len(value))
):
raise ValueError(f"{path} exceeds its item bounds")
if schema.get("uniqueItems") and len(
{canonical(item) for item in value}
) != len(value):
raise ValueError(f"Duplicate value in {path}")
prefix = schema.get("prefixItems", [])
for index, item in enumerate(value):
_schema_value(
item,
prefix[index] if index < len(prefix) else schema.get("items", {}),
definitions,
f"{path}[{index}]",
)
if isinstance(value, str):
if (
not schema.get("minLength", 0)
<= len(value)
<= schema.get("maxLength", len(value))
):
raise ValueError(f"{path} exceeds its string bounds")
if "pattern" in schema and re.search(schema["pattern"], value) is None:
raise ValueError(
f"{path} contains an invalid identifier, path or character"
)
if type(value) in {int, float}:
if (
"maximum" in schema
and value > schema["maximum"]
or "exclusiveMinimum" in schema
and value <= schema["exclusiveMinimum"]
):
raise ValueError(f"{path} exceeds its numeric bounds")
def validate_project(payload: dict, workspace_root: Path) -> None:
"""Validate every declaration, not just the selected profile/dependencies."""
schema = read_json(META_ROOT / "tools/devkit/project.schema.json")
_schema_value(payload, schema, schema["$defs"], "project")
root = workspace_root.resolve()
records = payload["repositories"]
names = {record["name"] for record in records}
if len(names) != len(records):
raise ValueError("Duplicate project repository name")
paths, aliases = set(), set()
for record in records:
path = (root / record["path"]).resolve()
if not path.is_relative_to(root):
raise ValueError("Repository path escapes the workspace")
if path in paths:
raise ValueError("Duplicate project repository path")
paths.add(path)
keys = {record["name"], *record.get("aliases", [])}
if aliases & keys:
raise ValueError("Repository names and aliases must be unambiguous")
aliases.update(keys)
checks = payload.get("checks", [])
identities = {item["id"] for item in checks}
if len(identities) != len(checks):
raise ValueError("Duplicate project check ID")
for item in checks:
if set(item.get("repos", [])) - names:
raise ValueError(f"Unknown repository reference in check {item['id']}")
if set(item.get("deps", [])) - identities:
raise ValueError(f"Unknown check dependency in {item['id']}")
if set(item.get("after", [])) - identities:
raise ValueError(f"Unknown check ordering reference in {item['id']}")
if set(item.get("deps", [])) & set(item.get("after", [])):
raise ValueError(f"Duplicate dependency/ordering reference in {item['id']}")
if set(item.get("inputs", {}).get("repos", [])) - names:
raise ValueError(f"Unknown input repository reference in {item['id']}")
if not (root / item.get("cwd", ".")).resolve().is_relative_to(root):
raise ValueError(f"Check {item['id']} cwd escapes the workspace")
for name, selected in payload.get("profiles", {}).items():
if set(selected) - identities:
raise ValueError(f"Unknown profile check in {name}")
remaining = {
item["id"]: set(item.get("deps", [])) | set(item.get("after", []))
for item in checks
}
while remaining:
ready = {identity for identity, deps in remaining.items() if not deps}
if not ready:
raise ValueError("Cyclic check dependency in project declarations")
remaining = {
identity: deps - ready
for identity, deps in remaining.items()
if identity not in ready
}
for value in payload.get("review", {}).values():
if not (root / value).resolve().is_relative_to(root):
raise ValueError("Project review path escapes the workspace")
+273
View File
@@ -0,0 +1,273 @@
"""Repository discovery, offline snapshots and exact source fingerprints."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import os
from pathlib import Path
import stat
import sys
from .common import META_ROOT, digest, identifier, read_json
from .process import require_capture
@dataclass(frozen=True)
class Repository:
name: str
path: Path
aliases: tuple[str, ...] = ()
@dataclass(frozen=True)
class Project:
name: str
repositories: tuple[Repository, ...]
config: dict
def load_project(workspace_root: Path, project: Path | None = None) -> Project:
root = workspace_root.resolve()
payload = read_json(project or META_ROOT / "repositories.json")
if not isinstance(payload, dict):
raise ValueError("Project manifest must be an object")
if project:
from .validation import validate_project
validate_project(payload, root)
records = payload.get("repositories")
if not isinstance(records, list) or not records or len(records) > 256:
raise ValueError("Project manifest requires 1256 repositories")
repos, names = [], set()
for record in records:
if not isinstance(record, dict) or not isinstance(record.get("path"), str):
raise ValueError("Invalid repository record")
if not record["path"] or (
project and set(record) - {"name", "path", "aliases"}
):
raise ValueError("Repository requires a nonempty path and known fields")
name = identifier(record.get("name"))
raw = Path(record["path"])
if raw.is_absolute() or ".." in raw.parts:
raise ValueError("Repository paths must be relative within the workspace")
path = (root / raw).resolve()
if not path.is_relative_to(root):
raise ValueError("Repository path escapes the workspace")
aliases = (
record.get("aliases", []) if project else [name.removeprefix("govoplan-")]
)
if not isinstance(aliases, list) or any(
not isinstance(alias, str) for alias in aliases
):
raise ValueError("Repository aliases must be strings")
if len(aliases) != len(set(aliases)):
raise ValueError("Duplicate repository alias")
keys = {name, *[identifier(alias) for alias in aliases]}
if names.intersection(keys):
raise ValueError("Repository names and aliases must be unambiguous")
names.update(keys)
repos.append(Repository(name, path, tuple(aliases)))
return Project(
payload.get("name", "Project" if project else "GovOPlaN"), tuple(repos), payload
)
def inspect_repository(repo: Repository) -> dict:
# Reuse the established read-only model, including distinct Git errors.
sys.path.insert(0, str(META_ROOT / "tools/release")) if str(
META_ROOT / "tools/release"
) not in sys.path else None
from govoplan_release.git_state import collect_repository_snapshot
from govoplan_release.model import RepositorySpec
unsafe = unsafe_git_environment()
if unsafe:
return {
"name": repo.name,
"path": str(repo.path),
"exists": repo.path.exists(),
"is_git": (repo.path / ".git").exists(),
"branch": None,
"head": None,
"upstream": None,
"ahead": None,
"behind": None,
"remote_checked": False,
"dirty_entries": [],
"errors": ["Git environment overrides prevent scoped inspection"],
"safe_directory_required": False,
}
snapshot = collect_repository_snapshot(
RepositorySpec(
name=repo.name,
category="module",
subtype="",
remote="",
path=str(repo.path),
),
workspace_root=repo.path.parent,
target_tag=None,
online=False,
)
return {
"name": repo.name,
"path": str(repo.path),
"exists": snapshot.exists,
"is_git": snapshot.is_git,
"branch": snapshot.branch,
"head": snapshot.head,
"upstream": snapshot.upstream,
"ahead": snapshot.ahead,
"behind": snapshot.behind,
"remote_checked": False,
"dirty_entries": list(snapshot.dirty_entries),
"errors": list(snapshot.errors),
"safe_directory_required": snapshot.safe_directory_required,
}
def selected_repositories(
project: Project, names: list[str], changed: bool = False
) -> list[Repository]:
selected = list(project.repositories)
if names:
wanted = set(names)
known = {key for repo in selected for key in (repo.name, *repo.aliases)}
if wanted - known:
raise ValueError(
"Unknown repository filter: " + ", ".join(sorted(wanted - known))
)
selected = [
repo for repo in selected if wanted.intersection((repo.name, *repo.aliases))
]
if changed:
result = []
for repo in selected:
state = inspect_repository(repo)
if (
state["errors"]
or state["dirty_entries"]
or state["ahead"]
or (state["head"] and not state["upstream"])
):
result.append(repo)
selected = result
return selected
def unsafe_git_environment() -> set[str]:
return {
key
for key in os.environ
if key.startswith("GIT_")
and key not in {"GIT_OPTIONAL_LOCKS", "GIT_TERMINAL_PROMPT", "GIT_PAGER"}
}
def git_bytes(path: Path, *argv: str, allow_failure: bool = False) -> bytes:
if unsafe_git_environment():
raise ValueError(
"Git environment overrides prevent a scoped source fingerprint"
)
result = require_capture(
["git", "--no-pager", "-C", str(path), *argv],
timeout=30,
max_stdout=32 * 1024 * 1024,
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0", "GIT_TERMINAL_PROMPT": "0"},
)
if result.returncode:
if allow_failure:
return b"unborn"
raise ValueError(
f"Could not fingerprint Git state in {path.name}; inspect context first"
)
return result.stdout
def source_fingerprint(project: Project) -> str:
"""Bind HEAD, index and every tracked/untracked working file, including hidden changes."""
overall = hashlib.sha256()
overall.update(digest(project.config).encode())
for repo in sorted(project.repositories, key=lambda item: item.name):
overall.update(repo.name.encode() + b"\0" + str(repo.path).encode() + b"\0")
if not repo.path.exists():
overall.update(b"missing\0")
continue
if not (repo.path / ".git").exists():
raise ValueError(
f"Cannot establish source identity for non-Git repository {repo.name}"
)
overall.update(
git_bytes(
repo.path, "rev-parse", "--verify", "HEAD", allow_failure=True
).strip()
)
overall.update(git_bytes(repo.path, "ls-files", "--stage", "-z"))
overall.update(git_bytes(repo.path, "ls-files", "-v", "-z"))
tracked = git_bytes(repo.path, "ls-files", "--cached", "-z")
untracked = git_bytes(
repo.path, "ls-files", "--others", "--exclude-standard", "-z"
)
for encoded_name in sorted(
set(tracked.split(b"\0") + untracked.split(b"\0")) - {b""}
):
name = os.fsdecode(encoded_name)
relative = Path(name)
if relative.is_absolute() or ".." in relative.parts:
raise ValueError("Unsafe repository file name")
path = repo.path / relative
overall.update(encoded_name + b"\0")
if not path.exists() and not path.is_symlink():
overall.update(b"deleted\0")
continue
metadata = path.lstat()
overall.update(str(stat.S_IMODE(metadata.st_mode)).encode() + b"\0")
if path.is_symlink():
overall.update(os.fsencode(os.readlink(path)))
elif path.is_file():
if not path.resolve().is_relative_to(repo.path):
raise ValueError("Fingerprint input escapes its repository")
maximum = 64 * 1024 * 1024
descriptor = os.open(
path,
os.O_RDONLY
| getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_NONBLOCK", 0),
)
with os.fdopen(descriptor, "rb") as handle:
before = os.fstat(handle.fileno())
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
raise ValueError(
f"Fingerprint input must be a bounded regular file in {repo.name}"
)
file_hash, count = hashlib.sha256(), 0
for chunk in iter(
lambda: handle.read(min(1024 * 1024, maximum - count + 1)), b""
):
count += len(chunk)
if count > maximum:
raise ValueError("File grew beyond fingerprint limit")
file_hash.update(chunk)
after = os.fstat(handle.fileno())
if (
before.st_ino,
before.st_size,
before.st_mtime_ns,
before.st_ctime_ns,
) != (
after.st_ino,
after.st_size,
after.st_mtime_ns,
after.st_ctime_ns,
):
raise ValueError(
"File changed while calculating source identity"
)
overall.update(file_hash.digest())
else:
raise ValueError(
"Unsupported changed-file type; cannot establish source identity"
)
overall.update(b"\0")
return overall.hexdigest()
+67
View File
@@ -0,0 +1,67 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Devkit portable project configuration",
"description": "The runtime additionally checks reference existence, cycles, resolved path confinement and cross-repository alias/path uniqueness. cwd defaults to the workspace when omitted.",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "repositories"],
"properties": {
"schema_version": {"type": "integer", "const": 1},
"name": {"type": "string", "minLength": 1, "maxLength": 128, "pattern": "^[^\\u0000\\r\\n]+$"},
"repositories": {
"type": "array", "minItems": 1, "maxItems": 256,
"items": {
"type": "object", "additionalProperties": false, "required": ["name", "path"],
"properties": {
"name": {"$ref": "#/$defs/identifier"},
"path": {"$ref": "#/$defs/relativePath"},
"aliases": {"type": "array", "maxItems": 128, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}
}
}
},
"tools": {
"type": "object", "additionalProperties": false,
"properties": {"python": {"$ref": "#/$defs/executable"}, "node": {"$ref": "#/$defs/executable"}, "npm": {"$ref": "#/$defs/executable"}}
},
"review": {
"type": "object", "additionalProperties": false,
"properties": {
"issue_inventory": {"$ref": "#/$defs/relativePath"},
"principles": {"$ref": "#/$defs/relativePath"}
}
},
"checks": {
"type": "array", "maxItems": 512,
"items": {
"type": "object", "additionalProperties": false, "required": ["id", "argv"],
"properties": {
"id": {"$ref": "#/$defs/identifier"},
"title": {"type": "string", "maxLength": 1024, "pattern": "^[^\\u0000]*$"},
"argv": {"type": "array", "minItems": 1, "maxItems": 256, "prefixItems": [{"type": "string", "minLength": 1, "maxLength": 8192, "pattern": "^[^\\u0000]+$"}], "items": {"type": "string", "maxLength": 8192, "pattern": "^[^\\u0000]*$"}},
"cwd": {"$ref": "#/$defs/relativePath"},
"deps": {"$ref": "#/$defs/profile"},
"after": {"$ref": "#/$defs/profile"},
"reuse": {"type": "string", "enum": ["verified", "never"]},
"inputs": {"$ref": "#/$defs/inputs"},
"resources": {"type": "array", "maxItems": 256, "uniqueItems": true, "items": {"type": "string", "minLength": 1, "maxLength": 256, "pattern": "^[^\\u0000\\r\\n]+$"}},
"repos": {"type": "array", "maxItems": 256, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}},
"timeout_seconds": {"type": "number", "exclusiveMinimum": 0, "maximum": 43200}
}
}
},
"profiles": {
"type": "object", "additionalProperties": false,
"properties": {
"quick": {"$ref": "#/$defs/profile"}, "ui": {"$ref": "#/$defs/profile"},
"backend": {"$ref": "#/$defs/profile"}, "full": {"$ref": "#/$defs/profile"}
}
}
},
"$defs": {
"inputs": {"type": "object", "additionalProperties": false, "required": ["repos"], "properties": {"repos": {"type": "array", "minItems": 1, "maxItems": 256, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}}},
"identifier": {"type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$(?![\\s\\S])"},
"executable": {"type": "string", "minLength": 1, "maxLength": 4096, "pattern": "^[^\\u0000\\r\\n]+$"},
"relativePath": {"type": "string", "minLength": 1, "maxLength": 4096, "pattern": "^(?!/)(?!\\.\\.(?:/|$))(?!.*?/\\.\\.(?:/|$))[^\\u0000\\r\\n]+$"},
"profile": {"type": "array", "maxItems": 512, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}
}
}
+386
View File
@@ -0,0 +1,386 @@
#!/usr/bin/env python3
"""Inventory all module UI review scopes and safely create missing Gitea tracks.
Dry-run by default. Existing issues are never rewritten or closed. The optional
one-time epic link initialization refuses to replace an edited or completed list.
"""
from __future__ import annotations
import argparse
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
import importlib.util
import json
from pathlib import Path
import re
import socket
import sys
from typing import Any, Iterator
from gitea_common import (
GiteaClient, GiteaError, RepoTarget, load_dotenv,
org_path, repo_path, require_token,
)
META_ROOT = Path(__file__).resolve().parents[2]
BASE_URL = "https://git.add-ideas.de"
OWNER = "GovOPlaN"
EPIC_MARKER = "<!-- govoplan-ui-review:v1:epic -->"
LIST_START = "<!-- govoplan-ui-review:module-list:start -->"
LIST_END = "<!-- govoplan-ui-review:module-list:end -->"
INITIAL_LIST = "The linked inventory is being initialized. All 77 tracks are pending; no checkboxes are complete."
PRINCIPLES_URL = f"{BASE_URL}/{OWNER}/govoplan-core/src/branch/main/docs/UI_DESIGN_PRINCIPLES.md"
PROCESS_URL = f"{BASE_URL}/{OWNER}/govoplan/src/branch/main/docs/project/UI_REVIEW_PROGRAM.md"
PRINCIPLES = (
("UI-01", "Help icons beside the relevant heading/label, not action-button rows"),
("UI-02", "Display-first with scoped edit dialogs; explicit bulk-grid editing exception"),
("UI-03", "Shared page actions, Reload/New ordering, Save/Cancel and destructive separation"),
("UI-04", "Full-width table/card geometry, visible actions, pagination and two-way resizing"),
("UI-05", "Scoped loading/error feedback, useful progress and retained state"),
("UI-06", "Predictable tree selection, expansion, grouping and reordering"),
("UI-07", "Keyboard/focus/accessibility, responsive layouts and understandable German"),
("UI-08", "Authorization, optional-module boundaries and save/cancel/retry data integrity"),
("UI-09", "Revision evidence and retroactive checks for changed design principles"),
)
def source_url(repository: str, path: str) -> str:
return f"{BASE_URL}/{OWNER}/{repository}/src/branch/main/{path}"
def marker(scope_id: str) -> str:
return f"<!-- govoplan-ui-review:v1:module:{scope_id} -->"
def normalized_title(value: str) -> str:
return " ".join(value.casefold().split())
def find_existing(issues: list[dict[str, Any]], scope_id: str, title: str) -> dict[str, Any] | None:
matches = [
issue for issue in issues if issue.get("pull_request") is None and (
marker(scope_id) in (issue.get("body") or "")
or normalized_title(issue.get("title") or "") == normalized_title(title)
)
]
if len(matches) > 1:
raise GiteaError(f"Ambiguous review issue matches for {scope_id}; inspect before making changes.")
if matches and marker(scope_id) not in (matches[0].get("body") or ""):
raise GiteaError(f"Unmanaged exact-title issue for {scope_id}; preserve it and resolve the duplicate manually.")
return matches[0] if matches else None
def extract_manifests(catalog: dict[str, Any], workspace_root: Path) -> list[dict[str, Any]]:
path = META_ROOT / "tools/inventory/platform-interface-inventory.py"
spec = importlib.util.spec_from_file_location("ui_review_source_inventory", path)
assert spec and spec.loader
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module._extract_manifests(catalog, workspace_root)
def source_groups(root: Path) -> dict[str, list[str]]:
groups: dict[str, list[str]] = {
"Pages and navigation entrypoints": [],
"Dialogs and embedded editing surfaces": [],
"Settings and administrator surfaces": [],
"Widgets, public/operator and contributed surfaces": [],
"Shared components and other UI entrypoints": [],
}
source_root = root / "webui/src"
for path in sorted(source_root.rglob("*.tsx")):
relative = path.relative_to(root).as_posix()
lower = relative.casefold()
name = path.stem.casefold()
if "page" in name or "navigation" in name or name in {"app", "routes", "index"}:
groups["Pages and navigation entrypoints"].append(relative)
elif re.search(r"dialog|modal|drawer|chooser|overlay", name):
groups["Dialogs and embedded editing surfaces"].append(relative)
elif re.search(r"settings|configur|admin", lower):
groups["Settings and administrator surfaces"].append(relative)
elif re.search(r"widget|public|operator|contribution", lower):
groups["Widgets, public/operator and contributed surfaces"].append(relative)
else:
groups["Shared components and other UI entrypoints"].append(relative)
return groups
def build_scopes(
catalog: dict[str, Any], workspace_root: Path, manifests: list[dict[str, Any]],
) -> list[dict[str, Any]]:
selected = [
repo for repo in catalog["repositories"]
if repo["category"] in {"module", "connector"} or repo["name"] == "govoplan-core"
]
by_repository: dict[str, dict[str, Any]] = {}
ids: set[str] = set()
for manifest in manifests:
if manifest["repository"] in by_repository or manifest["id"] in ids:
raise GiteaError("Duplicate source manifest repository or module ID.")
by_repository[manifest["repository"]] = manifest
ids.add(manifest["id"])
selected_names = {repo["name"] for repo in selected}
if set(by_repository) - selected_names:
raise GiteaError("Source manifests contain repositories absent from the module review catalog.")
scopes: list[dict[str, Any]] = []
for repo in selected:
root = workspace_root / repo["path"]
if not root.is_dir():
raise GiteaError(f"Missing source checkout for {repo['name']}; cannot infer review scope safely.")
manifest = by_repository.get(repo["name"])
paths = sorted(path.relative_to(root).as_posix() for path in root.glob("src/*/backend/manifest.py"))
if repo["name"] == "govoplan-core":
scope_id, name, kind = "core", "Core / shared shell", "core"
elif manifest:
scope_id, name, kind = manifest["id"], manifest["name"], "manifest"
else:
if paths or (root / "pyproject.toml").exists() or (root / "webui/package.json").exists():
raise GiteaError(f"{repo['name']} has implementation but no extracted manifest; inspect instead of calling it a placeholder.")
scope_id = "catalog:" + repo["name"]
name = repo["name"].removeprefix("govoplan-").replace("-", " ").title()
kind = "placeholder"
groups = source_groups(root)
scopes.append({
"scope_id": scope_id, "name": name, "repository": repo["name"],
"kind": kind, "manifest_paths": paths,
"frontend": manifest.get("frontend") if manifest else None,
"source_groups": groups,
"ui_source_count": sum(len(paths) for paths in groups.values()),
})
if len({scope["scope_id"] for scope in scopes}) != len(scopes):
raise GiteaError("Duplicate review scope IDs.")
return sorted(scopes, key=lambda scope: (scope["kind"] == "placeholder", scope["scope_id"] != "core", scope["name"].casefold()))
def issue_title(scope: dict[str, Any]) -> str:
suffix = "readiness and future UI review" if scope["kind"] == "placeholder" else "visual and interaction conformance"
return f"[UI review] {scope['name']}: {suffix}"
def source_seed(scope: dict[str, Any]) -> str:
repo = scope["repository"]
lines = [
"This is a **source-derived starting inventory, not a completed runtime audit**. Verify nested routes, embedded dialogs and contributions in the installed module context; add missing surfaces to this issue.",
"", f"Repository: [{repo}]({BASE_URL}/{OWNER}/{repo}).",
]
if scope["kind"] == "placeholder":
lines += [
"", f"Catalog entry `{repo}` is currently README-only; there is no runtime module ID, manifest or standalone WebUI to claim as reviewed. Source: [README]({source_url(repo, 'README.md')}).",
"", "- [ ] Confirm the catalog/readiness scope and record prerequisites for the first implementation.",
"- [ ] Keep the future interface review pending until actual configuration, public/operator or UI surfaces exist; do not manufacture N/A evidence to close this track.",
]
return "\n".join(lines)
for path in scope["manifest_paths"]:
lines.append(f"Manifest source: [{path}]({source_url(repo, path)}).")
if scope["kind"] == "core":
lines += [
"", "Core/shared shell additionally owns navigation/rail/breadcrumbs, module routing, page/action archetypes, cards/tables/forms/dialogs, loading and error surfaces, help affordances, authentication and user settings. Review optional-module and permission contexts, not just standalone primitives.",
]
frontend = scope["frontend"]
if frontend:
for key, label in (
("routes", "Declared routes"), ("public_routes", "Declared public routes"),
("settings_routes", "Declared settings routes"), ("nav_items", "Declared navigation"),
("view_surfaces", "Declared view, settings and contributed surfaces"),
):
entries = frontend.get(key) or []
lines += ["", f"**{label} ({len(entries)}):**"]
if not entries:
lines.append("None declared in this manifest; verify indirect/contributed surfaces before marking anything not applicable.")
for entry in entries:
identity = entry.get("path") or entry.get("id") or entry.get("component") or "unnamed declaration"
detail = entry.get("component") or entry.get("label") or entry.get("kind") or ""
lines.append(f"- `{identity}`" + (f"{detail}" if detail else ""))
elif scope["kind"] != "core":
lines += [
"", "No standalone frontend is declared. **The review is still pending:** inspect owned configuration/admin workflows, manifest documentation, errors and any interfaces contributed through host modules, public routes or operator tools. Record concrete evidence before claiming a principle does not apply.",
]
lines += ["", f"<details><summary>Source entrypoint seed ({scope['ui_source_count']} TSX files; classification is heuristic)</summary>", ""]
for label, paths in scope["source_groups"].items():
if not paths:
continue
lines += [f"**{label} ({len(paths)}):**", ""]
for path in paths[:40]:
lines.append(f"- [{path}]({source_url(repo, path)})")
if len(paths) > 40:
lines.append(f"- {len(paths) - 40} further files: inspect [the source tree]({source_url(repo, 'webui/src')}); expand the issue inventory during review.")
lines.append("")
lines += ["</details>"]
return "\n".join(lines)
def issue_body(scope: dict[str, Any], epic_url: str) -> str:
lines = [
marker(scope["scope_id"]), "## Status and objective", "",
f"**Pending / not reviewed.** This module track belongs to the [product-wide UI review epic]({epic_url}).",
f"Apply the [Core design principles]({PRINCIPLES_URL}) using the [shared review process]({PROCESS_URL}). The current heading-help icon pass and any existing isolated fixes are preparation, not evidence that this whole module is complete. Source documents are being prepared in the current working tree; this issue does not claim they are released.",
"", "Prioritize usability defects, consistent interaction and shared-component fixes before broader features. Preserve permissions, security, optional-module boundaries and data integrity.",
"", "## Source inventory to verify", "", source_seed(scope),
"", "## Review and implementation TODO", "",
"- [ ] Confirm every actual page, nested route, dialog, field/form, table/tree, settings level, public/operator surface, widget and cross-module contribution; document role/module prerequisites.",
"- [ ] Exercise EN/DE, keyboard/focus, narrow and wide layouts, long content, empty/loading/error states and realistic datasets.",
"- [ ] Check UI-01 heading/label help placement and UI-02 display-first/scoped editing; document any justified large-grid bulk-edit exception with explicit mode, Save/Cancel and dirty-navigation guard.",
"- [ ] Verify consistent top-right actions (Reload left of New), clean/dirty Save/Cancel behavior, destructive separation and safe navigation/reload.",
"- [ ] Verify full-width cards/tables, visible last-column actions, pagination, initial sizing, two-way pointer/keyboard resizing and preference reload with fixed columns and horizontal overflow.",
"- [ ] Check scoped progress/error feedback and predictable tree selection versus expansion; no unnecessary global blocking or repeated background reload.",
"- [ ] Record findings and implement shared-contract corrections plus all affected consumers, not local CSS/action-row exceptions without justification.",
"- [ ] Verify save/cancel/retry and partial-failure behavior without unintended writes, sends, deletes or loss of persisted/unsaved data.",
"- [ ] Update owning manifest-driven EN/DE documentation; record targeted automated checks and manual evidence against actual module surfaces.",
"- [ ] Complete the principle matrix, list unresolved decisions/manual checks and link follow-ups before proposing closure.",
]
if scope["scope_id"] == "campaigns":
lines += [
"", "### Campaign-specific starting direction", "",
"- [ ] Present a compact read-only campaign settings dashboard/overview and use explicit scoped edit dialogs for settings instead of a permanently editable form wall.",
"- [ ] Keep large recipient/attachment tables practical through an explicit bulk-edit mode where appropriate, with Save/Cancel, dirty-state protection and reload persistence; this is the documented UI-02 exception, not silent autosave.",
"- [ ] Review the complete compose → attachments → validation/review → delivery/report/operator workflow, including mail-profile migration, ZIP policy, multiple recipients and SMTP/IMAP progress, without sending live messages just to collect UI evidence.",
]
lines += [
"", "## Principle applicability / application / evidence / exceptions", "",
"Reviewed Core principle revision: **not yet recorded**. No exceptions approved.", "",
"| Principle | Applicable surfaces / justified N/A | Applied / remaining work | Evidence | Exception / owner / follow-up |",
"| --- | --- | --- | --- | --- |",
]
for identity, description in PRINCIPLES:
lines.append(f"| {identity}: {description} | Pending inventory | Pending review | Not yet recorded | None approved |")
lines += [
"", "When a principle changes after this review, re-check applicability and record current evidence. Reopen this issue or link an owned follow-up for outstanding work; notify the central epic. A past review must not silently remain green against an obsolete rule.",
"", "## Findings / TODO / done ledger", "",
"| Finding / surface / reproduction | Principle and expected behavior | TODO / implementation or follow-up | Verified done evidence |",
"| --- | --- | --- | --- |",
"| Full review not started | UI-01UI-09 | Inventory and review pending | None; no completed review claimed |",
"", "## Manual checks, decisions and closure evidence", "",
"- Manual work pending: safe actual-module walkthrough in both languages and realistic viewports, keyboard/focus, permissions/optional-module contexts, loading/error/empty states, edits/Save/Cancel/navigation/reload, tables/trees and progress.",
"- Decisions: none invented by this bootstrap. Record any product or policy choice with context and a recommendation; isolate independent implementation work from blocked decisions.",
"- Automated evidence: not yet recorded for this complete module review. Existing targeted fixes/tests may be linked as partial evidence only.",
"- Closure gate: verified inventory, complete current principle matrix, resolved required findings, owning EN/DE documentation and automated/manual evidence. Unimplemented placeholders remain pending until real surfaces can be reviewed or an explicit catalog/product decision changes scope.",
]
return "\n".join(lines) + "\n"
def result_record(scope: dict[str, Any], issue: dict[str, Any] | None, action: str) -> dict[str, Any]:
return {
"scope_id": scope["scope_id"], "name": scope["name"], "repository": scope["repository"],
"kind": scope["kind"], "ui_source_count": scope["ui_source_count"],
"number": issue.get("number") if issue else None,
"url": issue.get("html_url") if issue else None,
"state_at_verification": issue.get("state") if issue else None,
"operation": action,
}
def render_links(records: list[dict[str, Any]]) -> str:
lines = ["### Implemented scopes — pending review", ""]
for placeholder in (False, True):
if placeholder:
lines += ["", "### Catalogued placeholders — pending readiness / future UI review", ""]
for item in records:
if (item["kind"] == "placeholder") == placeholder:
if not item["url"]:
raise GiteaError("Cannot initialize an incomplete issue link inventory.")
lines.append(f"- [ ] [{item['name']}]({item['url']}) — `{item['scope_id']}` / `{item['repository']}`; pending.")
return "\n".join(lines)
def initialized_epic_body(body: str, records: list[dict[str, Any]]) -> str:
if body.count(LIST_START) != 1 or body.count(LIST_END) != 1 or EPIC_MARKER not in body:
raise GiteaError("Epic managed-list markers are absent or ambiguous; preserve its body.")
before, tail = body.split(LIST_START, 1)
current, after = tail.split(LIST_END, 1)
if current.strip() != INITIAL_LIST:
if all(item["url"] and f"]({item['url']})" in current for item in records):
return body # Never reset human checkboxes, evidence or subsequent edits.
raise GiteaError("Epic list was already edited; update missing links manually without replacing progress.")
return before + LIST_START + "\n" + render_links(records) + "\n" + LIST_END + after
@contextmanager
def ipv4_for_target(enabled: bool) -> Iterator[None]:
original = socket.getaddrinfo
def scoped(host: Any, port: Any, family: int = 0, type: int = 0, proto: int = 0, flags: int = 0) -> Any:
return original(host, port, socket.AF_INET if host == "git.add-ideas.de" else family, type, proto, flags)
if enabled:
socket.getaddrinfo = scoped
try:
yield
finally:
socket.getaddrinfo = original
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--env-file", type=Path)
parser.add_argument("--epic", type=int, required=True, help="Existing managed UI-review epic number in GovOPlaN/govoplan")
parser.add_argument("--apply", action="store_true", help="Create missing review issues; existing issues remain untouched")
parser.add_argument("--initialize-links", action="store_true", help="One-time initialization of the untouched epic module-list placeholder")
parser.add_argument("--ipv4", action="store_true", help="Prefer IPv4 only for git.add-ideas.de; keep HTTPS verification")
args = parser.parse_args()
if args.epic <= 0 or (args.initialize_links and not args.apply):
parser.error("A positive --epic is required; --initialize-links requires --apply.")
try:
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
workspace_root = Path(catalog["default_parent"])
scopes = build_scopes(catalog, workspace_root, extract_manifests(catalog, workspace_root))
load_dotenv(args.env_file)
token = require_token()
target = RepoTarget(BASE_URL, OWNER, "govoplan")
with ipv4_for_target(args.ipv4), GiteaClient(target, token) as central:
epic = central.request_json("GET", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"))
if EPIC_MARKER not in (epic.get("body") or "") or epic.get("state") != "open":
raise GiteaError("Expected an open, managed UI-review epic; no child issues created.")
epic_url = f"{BASE_URL}/{OWNER}/govoplan/issues/{args.epic}"
org_labels = {item["name"]: item["id"] for item in central.paginate(org_path(OWNER, "/labels"))}
def reconcile(scope: dict[str, Any]) -> dict[str, Any]:
repo = scope["repository"]
with GiteaClient(RepoTarget(BASE_URL, OWNER, repo), token) as client:
issues = client.paginate(repo_path(OWNER, repo, "/issues"), query={"state": "all", "type": "issues"})
existing = find_existing(issues, scope["scope_id"], issue_title(scope))
if existing:
return result_record(scope, existing, "existing")
if not args.apply:
return result_record(scope, None, "would-create")
labels = dict(org_labels)
labels.update({item["name"]: item["id"] for item in client.paginate(repo_path(OWNER, repo, "/labels"))})
desired = ["type/task", "area/webui", "area/docs", "priority/p2", f"module/{repo.removeprefix('govoplan-')}"]
desired += ["status/triage"] if scope["kind"] == "placeholder" else ["status/ready", "codex/ready"]
issue = client.request_json("POST", repo_path(OWNER, repo, "/issues"), body={
"title": issue_title(scope), "body": issue_body(scope, epic_url),
"labels": [labels[name] for name in desired if name in labels],
})
verified = client.request_json("GET", repo_path(OWNER, repo, f"/issues/{issue['number']}"))
if marker(scope["scope_id"]) not in (verified.get("body") or "") or verified.get("state") != "open":
raise GiteaError(f"New review issue verification failed for {scope['scope_id']}.")
print(f"Created {repo}#{verified['number']} (pending)", file=sys.stderr, flush=True)
return result_record(scope, verified, "created")
with ThreadPoolExecutor(max_workers=4) as executor:
records = list(executor.map(reconcile, scopes))
if args.initialize_links:
fresh = central.request_json("GET", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"))
body = initialized_epic_body(fresh["body"], records)
if body != fresh["body"]:
central.request_json("PATCH", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"), body={"body": body})
summary = {
"schema_version": 1, "snapshot_purpose": "Issue discovery links; live Gitea issues own review state and evidence.",
"epic": {"repository": "govoplan", "number": args.epic, "url": epic_url},
"scope_count": len(scopes),
"manifest_modules": sum(scope["kind"] == "manifest" for scope in scopes),
"implemented_scopes": sum(scope["kind"] != "placeholder" for scope in scopes),
"catalogued_placeholders": sum(scope["kind"] == "placeholder" for scope in scopes),
"created": sum(item["operation"] == "created" for item in records),
"missing": sum(item["operation"] == "would-create" for item in records),
"issues": records,
}
print(json.dumps(summary, ensure_ascii=False, indent=2))
return 0
except (GiteaError, OSError, ValueError) as exc:
print(f"UI review program: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+28 -10
View File
@@ -5,9 +5,16 @@ import path from "node:path";
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { pathToFileURL } from "node:url"; import { pathToFileURL } from "node:url";
const [metaRootArgument] = process.argv.slice(2); const [metaRootArgument, ...argumentsRest] = process.argv.slice(2);
if (!metaRootArgument) { if (!metaRootArgument) {
throw new Error("Usage: extract-webui-structure.mjs META_ROOT"); throw new Error("Usage: extract-webui-structure.mjs META_ROOT [--workspace-root ROOT]");
}
let explicitWorkspaceRoot;
for (let index = 0; index < argumentsRest.length; index++) {
if (argumentsRest[index] !== "--workspace-root" || explicitWorkspaceRoot !== undefined || !argumentsRest[index + 1]) {
throw new Error("Expected one --workspace-root ROOT option");
}
explicitWorkspaceRoot = path.resolve(argumentsRest[++index]);
} }
const metaRoot = path.resolve(metaRootArgument); const metaRoot = path.resolve(metaRootArgument);
@@ -15,12 +22,18 @@ const repositoryCatalog = JSON.parse(
fs.readFileSync(path.join(metaRoot, "repositories.json"), "utf8") fs.readFileSync(path.join(metaRoot, "repositories.json"), "utf8")
); );
const siblingWorkspaceRoot = path.dirname(metaRoot); const siblingWorkspaceRoot = path.dirname(metaRoot);
const configuredWorkspaceRoot = path.resolve(repositoryCatalog.default_parent); // Discovery is retained only for old direct callers. An explicit root wins
const workspaceRoot = fs.existsSync( // even when another configured checkout contains more optional modules.
const workspaceRoot = explicitWorkspaceRoot ?? (fs.existsSync(
path.join(siblingWorkspaceRoot, "govoplan-core", "webui") path.join(siblingWorkspaceRoot, "govoplan-core", "webui")
) ) ? siblingWorkspaceRoot : path.resolve(repositoryCatalog.default_parent));
? siblingWorkspaceRoot if (!fs.statSync(workspaceRoot).isDirectory()) throw new Error("Inventory workspace root must be an existing directory");
: configuredWorkspaceRoot; const actualWorkspaceRoot = fs.realpathSync(workspaceRoot);
function withinWorkspace(candidate) {
const relative = path.relative(actualWorkspaceRoot, fs.realpathSync(candidate));
return relative === "" || (!relative.startsWith(`..${path.sep}`) && relative !== ".." && !path.isAbsolute(relative));
}
if (!Array.isArray(repositoryCatalog.repositories)) throw new Error("Repository catalog requires a repositories array");
const typescriptPath = path.join( const typescriptPath = path.join(
workspaceRoot, workspaceRoot,
"govoplan-core", "govoplan-core",
@@ -96,6 +109,7 @@ const contributionTypes = new Map([
]); ]);
const result = { const result = {
workspaceRoot: actualWorkspaceRoot,
fields: [], fields: [],
actions: [], actions: [],
labels: [], labels: [],
@@ -111,8 +125,12 @@ const result = {
}; };
for (const repository of repositoryCatalog.repositories) { for (const repository of repositoryCatalog.repositories) {
const sourceRoot = path.join(workspaceRoot, repository.path, "webui", "src"); if (!repository || typeof repository.path !== "string" || !repository.path || path.isAbsolute(repository.path) || repository.path.split(/[\\/]/).includes("..")) {
throw new Error("Inventory repository paths must remain inside the selected workspace");
}
const sourceRoot = path.join(actualWorkspaceRoot, repository.path, "webui", "src");
if (!fs.existsSync(sourceRoot)) continue; if (!fs.existsSync(sourceRoot)) continue;
if (!withinWorkspace(sourceRoot)) throw new Error("Inventory source root escapes the selected workspace");
for (const sourcePath of sourceFiles(sourceRoot)) { for (const sourcePath of sourceFiles(sourceRoot)) {
inspectSource(repository.name, sourceRoot, sourcePath); inspectSource(repository.name, sourceRoot, sourcePath);
} }
@@ -135,7 +153,7 @@ function sourceFiles(root) {
} }
const candidate = path.join(current, entry.name); const candidate = path.join(current, entry.name);
if (entry.isDirectory()) pending.push(candidate); if (entry.isDirectory()) pending.push(candidate);
else if (/\.(?:ts|tsx)$/.test(entry.name)) files.push(candidate); else if (entry.isFile() && /\.(?:ts|tsx)$/.test(entry.name)) files.push(candidate);
} }
} }
return files.sort(); return files.sort();
@@ -150,7 +168,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
true, true,
sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS
); );
const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath); const relativeFile = path.relative(path.resolve(sourceRoot, "..", ".."), sourcePath);
const identityCounters = new Map(); const identityCounters = new Map();
function location(node) { function location(node) {
@@ -9,6 +9,7 @@ from collections import Counter
from dataclasses import asdict, is_dataclass from dataclasses import asdict, is_dataclass
import importlib import importlib
import json import json
import os
from pathlib import Path from pathlib import Path
import re import re
import subprocess import subprocess
@@ -40,6 +41,11 @@ REFERENCE_LOCALE = "de"
def main() -> int: def main() -> int:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--workspace-root",
type=Path,
help="Authoritative directory containing registered checkouts; never falls back to another workspace.",
)
parser.add_argument( parser.add_argument(
"--output-dir", "--output-dir",
type=Path, type=Path,
@@ -87,8 +93,9 @@ def main() -> int:
args = parser.parse_args() args = parser.parse_args()
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8")) catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
workspace_root = _resolve_workspace_root(catalog) workspace_root = _resolve_workspace_root(catalog, args.workspace_root)
webui = _extract_webui() _validate_repository_roots(catalog, workspace_root)
webui = _extract_webui(workspace_root)
backend_endpoints = _extract_backend_endpoints(catalog, workspace_root) backend_endpoints = _extract_backend_endpoints(catalog, workspace_root)
manifests = _extract_manifests(catalog, workspace_root) manifests = _extract_manifests(catalog, workspace_root)
endpoint_declarations = _load_endpoint_declarations( endpoint_declarations = _load_endpoint_declarations(
@@ -109,6 +116,10 @@ def main() -> int:
else None else None
), ),
) )
inventory["workspace_root"] = str(workspace_root)
inventory["workspace_selection"] = (
"explicit" if args.workspace_root is not None else "legacy-discovery"
)
output_dir = args.output_dir.resolve() output_dir = args.output_dir.resolve()
output_dir.mkdir(parents=True, exist_ok=True) output_dir.mkdir(parents=True, exist_ok=True)
@@ -212,7 +223,16 @@ def _strict_failures(
return failures return failures
def _resolve_workspace_root(catalog: dict[str, Any]) -> Path: def _resolve_workspace_root(
catalog: dict[str, Any], explicit_root: Path | None = None
) -> Path:
if explicit_root is not None:
root = explicit_root.expanduser().resolve()
if not root.is_dir():
raise ValueError("The explicit inventory workspace root must be an existing directory")
return root
# Compatibility for direct legacy callers only. Managed callers always
# supply their selected root; a partial checkout must not borrow sources.
sibling_root = META_ROOT.parent.resolve() sibling_root = META_ROOT.parent.resolve()
configured_root = Path(str(catalog["default_parent"])).expanduser().resolve() configured_root = Path(str(catalog["default_parent"])).expanduser().resolve()
repositories = catalog.get("repositories") repositories = catalog.get("repositories")
@@ -233,15 +253,44 @@ def _resolve_workspace_root(catalog: dict[str, Any]) -> Path:
return sibling_root if sibling_count >= configured_count else configured_root return sibling_root if sibling_count >= configured_count else configured_root
def _extract_webui() -> dict[str, Any]: def _validate_repository_roots(catalog: dict[str, Any], workspace_root: Path) -> None:
repositories = catalog.get("repositories")
if not isinstance(repositories, list):
raise ValueError("repository catalog has no repositories array")
for repository in repositories:
if not isinstance(repository, dict) or not isinstance(repository.get("path"), str):
raise ValueError("Invalid inventory repository path")
relative = Path(repository["path"])
if not repository["path"] or relative.is_absolute() or ".." in relative.parts:
raise ValueError("Inventory repository paths must remain inside the selected workspace")
root = workspace_root / relative
# Missing optional checkouts are allowed; links to another checkout are
# not evidence for the selected workspace.
if not root.resolve().is_relative_to(workspace_root):
raise ValueError("Inventory repository path escapes the selected workspace")
for source in (root / "src", root / "webui/src"):
if not source.resolve().is_relative_to(workspace_root):
raise ValueError("Inventory source root escapes the selected workspace")
def _extract_webui(workspace_root: Path | None = None) -> dict[str, Any]:
helper = META_ROOT / "tools" / "inventory" / "extract-webui-structure.mjs" helper = META_ROOT / "tools" / "inventory" / "extract-webui-structure.mjs"
argv = [os.environ.get("NODE", "node"), str(helper), str(META_ROOT)]
if workspace_root is not None:
argv.extend(["--workspace-root", str(workspace_root)])
completed = subprocess.run( completed = subprocess.run(
["node", str(helper), str(META_ROOT)], argv,
check=True, check=True,
capture_output=True, capture_output=True,
text=True, text=True,
) )
return json.loads(completed.stdout) result = json.loads(completed.stdout)
if workspace_root is not None and (
not isinstance(result, dict)
or result.get("workspaceRoot") != str(workspace_root.resolve())
):
raise ValueError("WebUI collector did not confirm the selected inventory workspace")
return result
def _extract_backend_endpoints( def _extract_backend_endpoints(
@@ -255,6 +304,8 @@ def _extract_backend_endpoints(
if not source_root.is_dir(): if not source_root.is_dir():
continue continue
for source_path in sorted(source_root.rglob("*.py")): for source_path in sorted(source_root.rglob("*.py")):
if not source_path.resolve().is_relative_to(workspace_root):
raise ValueError("Backend source path escapes the selected workspace")
try: try:
tree = ast.parse( tree = ast.parse(
source_path.read_text(encoding="utf-8"), source_path.read_text(encoding="utf-8"),
@@ -346,15 +397,26 @@ def _extract_manifests(
catalog: dict[str, Any], catalog: dict[str, Any],
workspace_root: Path, workspace_root: Path,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
_validate_repository_roots(catalog, workspace_root)
source_roots = [ source_roots = [
workspace_root / repository["path"] / "src" workspace_root / repository["path"] / "src"
for repository in catalog["repositories"] for repository in catalog["repositories"]
if (workspace_root / repository["path"] / "src").is_dir() if (workspace_root / repository["path"] / "src").is_dir()
] ]
core_root = next(
(workspace_root / repository["path"] / "src"
for repository in catalog["repositories"]
if repository.get("name") == "govoplan-core"),
workspace_root / "govoplan-core/src",
)
if not (core_root / "govoplan_core/core/platform_interfaces.py").is_file():
raise ValueError("Inventory requires Core interface sources in the selected workspace")
_assert_workspace_imports(workspace_root)
sys.path[:0] = [str(path) for path in source_roots] sys.path[:0] = [str(path) for path in source_roots]
from govoplan_core.core.platform_interfaces import ( # noqa: PLC0415 from govoplan_core.core.platform_interfaces import ( # noqa: PLC0415
manifest_interface_catalog, manifest_interface_catalog,
) )
_assert_workspace_imports(workspace_root)
manifests: list[dict[str, Any]] = [] manifests: list[dict[str, Any]] = []
for repository in catalog["repositories"]: for repository in catalog["repositories"]:
@@ -366,7 +428,12 @@ def _extract_manifests(
manifest_path.relative_to(source_root).with_suffix("").parts manifest_path.relative_to(source_root).with_suffix("").parts
) )
loaded = importlib.import_module(module_name) loaded = importlib.import_module(module_name)
source = getattr(loaded, "__file__", None)
if not isinstance(source, str) or Path(source).resolve() != manifest_path.resolve():
raise ValueError("Manifest import did not resolve to its selected workspace source")
_assert_workspace_imports(workspace_root)
manifest = loaded.get_manifest() manifest = loaded.get_manifest()
_assert_workspace_imports(workspace_root)
frontend = manifest.frontend frontend = manifest.frontend
manifests.append( manifests.append(
{ {
@@ -452,6 +519,26 @@ def _extract_manifests(
return sorted(manifests, key=lambda item: item["id"]) return sorted(manifests, key=lambda item: item["id"])
def _assert_workspace_imports(workspace_root: Path) -> None:
# An editable installation or cached import must not stand in for a missing
# optional checkout. Direct callers with another workspace use a fresh
# process instead of replacing already-loaded application packages.
for name, module in list(sys.modules.items()):
# The Meta tools may audit a separate checkout; they are not module
# contributions and must not be confused with application packages.
package = name.partition(".")[0]
if not package.startswith("govoplan_") or package in {
"govoplan_devkit", "govoplan_release"
}:
continue
filename = getattr(module, "__file__", None)
locations = list(getattr(module, "__path__", ()))
if isinstance(filename, str):
locations.append(filename)
if any(not Path(location).resolve().is_relative_to(workspace_root) for location in locations):
raise ValueError("A GovOPlaN import originates outside the selected inventory workspace")
def _assemble_inventory( def _assemble_inventory(
*, *,
webui: dict[str, Any], webui: dict[str, Any],