14 Commits
Author SHA1 Message Date
zemion 2ffdb23f69 feat(devkit): add resumable workspace automation and UI review tooling
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m30s
Verified with the coordinated workspace changes by devkit full run
2026-09-08T225814-186389-0000-3e3ed7cd (all seven phases passed).
This shared UI pass does not mark the individual module reviews complete.
2026-09-09 02:03:17 +02:00
zemion 14b19fbead chore(release): coordinate integrity and performance source updates
Dependency Audit / dependency-audit (push) Successful in 2m9s
Deployment Installer / deployment-installer (push) Successful in 8s
Security Audit / security-audit (push) Successful in 14m36s
Developer Meta-package Release / publish-package (push) Successful in 10s
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:53:09 +02:00
zemion 845dcbafdb chore(release): coordinate integrity and performance source updates
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:36:36 +02:00
zemion 58d320d9b3 Harden source release preparation and record verified security follow-up
Dependency Audit / dependency-audit (push) Successful in 1m51s
Deployment Installer / deployment-installer (push) Successful in 8s
Security Audit / security-audit (push) Successful in 12m32s
2026-09-08 08:04:12 +02:00
zemion 9554657bb5 fix(release): preserve failed installer retry status
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Successful in 11m21s
Cover all retry call sites with isolated regressions in focused checks and installer CI. Add EN/DE operating guidance and record the unreleased Xrechnung and installer audit follow-ups without changing immutable release artifacts.

Refs #54
2026-09-08 05:36:10 +02:00
zemion 88b685ff5e test(release): follow relocated operations runbook
Developer Meta-package Release / publish-package (push) Successful in 9s
Dependency Audit / dependency-audit (push) Successful in 1m41s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 14m12s
2026-09-08 03:02:31 +02:00
zemion be57a1823a fix(release): support validated per-run SSH address family 2026-09-08 02:50:59 +02:00
zemion 6bcb75f577 feat(release): build verified full-registry catalog candidates 2026-09-08 02:26:19 +02:00
zemion 32fe4b7238 chore(release): document hardening and verify complete source package composition 2026-09-08 02:06:35 +02:00
zemion 6a8f53b87d fix(release): prepare aligned 0.1.45 composition and guarded candidate sequencing 2026-09-08 01:52:08 +02:00
zemion 1cec4ee1d8 chore: compose governed Cases access
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m23s
2026-08-24 20:26:44 +02:00
zemion 29d03aa2ca chore: compose purpose-bound Records access
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m45s
2026-08-24 19:28:37 +02:00
zemion 6fb928d6cf release: publish stable product destinations
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m26s
Developer Meta-package Release / publish-package (push) Successful in 9s
2026-08-24 18:30:01 +02:00
zemion 6edaaadf37 release: align governed tenant erasure
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m4s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-24 16:33:16 +02:00
131 changed files with 24579 additions and 358 deletions
@@ -23,6 +23,9 @@ jobs:
- name: Test declarative deployment bundle
working-directory: govoplan
run: python -m unittest -v tests.test_deployment_installer
- name: Test WebUI installer retry failures
working-directory: govoplan
run: python -m unittest -v tests.test_webui_release_dependency_retries
- name: Build single-file deployer artifact
working-directory: govoplan
run: |
+7
View File
@@ -15,6 +15,13 @@ platform behavior remains owned by the corresponding module repository.
## Working Rules
- Start repeated workflows with `./devkit commands`; use `context --changed`,
`check --profile quick --changed --dry-run`, and `review MODULE` instead of
reconstructing repository/check inventories. See `docs/operations/DEVKIT.md`.
- Prefer compact check receipts (`status`, `summary`, `logs`) to repeated full
log dumps. `check --profile full` retains the required focused gate; targeted
profiles or cached results do not waive required verification or approvals.
- Treat Gitea issues as the canonical backlog and state log.
- Preserve optional module boundaries and use Core contracts or capabilities for integrations.
- Prefer targeted checks before full workspace scans.
+11
View File
@@ -19,6 +19,17 @@ installed modules/connectors discovered by core.
## Common Commands
For repeated development and review workflows, start with the unified command
suite. [Developer command guide](docs/operations/DEVKIT.md) documents profiles,
evidence, safe Git/release operations and reuse in other projects.
```sh
./devkit commands
./devkit context --changed
./devkit check --profile quick --changed --dry-run
./devkit review campaign
```
Create the whole-product development virtualenv in this meta repository:
```sh
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# One stable entry point; no environment activation, installation or server startup.
set -eu
DEVKIT_ROOT=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
if [ -n "${PYTHON:-}" ]; then
DEVKIT_PYTHON=$(command -v "$PYTHON") || { echo "devkit: configured PYTHON is unavailable" >&2; exit 127; }
elif [ -x "$DEVKIT_ROOT/.venv/bin/python" ]; then
DEVKIT_PYTHON="$DEVKIT_ROOT/.venv/bin/python"
else
DEVKIT_PYTHON=$(command -v python3)
fi
exec "$DEVKIT_PYTHON" "$DEVKIT_ROOT/tools/devkit/devkit.py" "$@"
+3
View File
@@ -64,6 +64,7 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
| Evidence collection and promotion | [Target Maturity Evidence Runbook](operations/TARGET_MATURITY_EVIDENCE_RUNBOOK.md) |
| Package publication and consumption | [Package Registry Releases](operations/PACKAGE_REGISTRY_RELEASES.md) |
| Release-console operation | [Release Console](operations/RELEASE_CONSOLE.md) |
| Repeated development, verification and evidence commands | [Developer Command Suite](operations/DEVKIT.md) |
| Module compatibility and install behavior | [Module Contracts and Installs](operations/MODULE_CONTRACTS_AND_INSTALLS.md) |
| Security-audit toolchain | [Security Audit](operations/SECURITY_AUDIT.md) |
@@ -75,6 +76,8 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
meta, module, deployment, and website content.
- [Gitea Issues](project/GITEA_ISSUES.md) defines labels, templates, import, and
state-update conventions.
- [UI Review Program](project/UI_REVIEW_PROGRAM.md) defines the principle-led,
per-module review process and links the canonical Gitea review inventory.
## Evidence And Archive
@@ -72,7 +72,7 @@ Each WebUI module should be able to announce:
The contract references surfaces. It does not permit Core or a product package
to import their implementation.
The first versioned `product_surfaces` slice is now implemented in Core. It
The versioned `product_surfaces` slice is implemented in Core. It
binds a stable product identity and entry path to one or more owner routes,
View surfaces, presentations, capabilities, search sources, help contexts and
documentation topics. It also carries standard unavailable/degraded
@@ -80,8 +80,22 @@ explanations and migration aliases. Mail and Postbox contribute the first
shared identity, `communication.messages`: `/messages` and the migration alias
`/inbox` select the first currently authorized, View-visible owner while the
underlying `/mail` and `/postbox` deep links, custody and permissions remain
unchanged. Alias resolution emits a bounded client telemetry event before the
redirect.
unchanged. Tasks, Calendar and Files contribute the corresponding single-owner
identities:
| Product identity | Stable destination | Compatible owner route |
| --- | --- | --- |
| Work | `/work` | `/tasks` |
| Calendar | `/agenda` | `/calendar` |
| Messages | `/messages` (`/inbox` alias) | `/mail`, `/postbox` |
| Files | `/documents` | `/files` |
Core replaces those owner entries in the ordinary rail with the stable product
destinations. A collapsed **All available tools** catalogue retains every
authorized technical owner route independently of View focus; unauthorized
entries are never disclosed. The original deep links remain valid, and all
contributing owner paths keep the corresponding product entry active. Alias
resolution emits a bounded client telemetry event before the redirect.
Core's `ProductAvailabilityState` is the shared presentation primitive for
authorization, Policy, configuration, disabled, missing-capability, offline and
@@ -105,10 +119,11 @@ People and Responsibility. They are configurable system/tenant defaults and
Views projections, not hard-coded repository groups. Empty areas disappear;
single-destination areas may link directly; familiar tools may remain pinned.
The complete permission-derived module rail remains available as **All
available tools**. Its ability to scroll is useful and is not itself the
product defect. The defect is requiring people to infer a task or outcome from
repository topology.
The complete permission-derived module rail is available as the collapsed
**All available tools** escape. It is deliberately independent of the active
View while still enforcing authorization. Its ability to scroll is useful and
is not itself the product defect. The defect is requiring people to infer a
task or outcome from repository topology.
Task-local Work, Calendar, Messages and Files tools may be contributed to the
optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and
@@ -125,6 +140,12 @@ sections, commands, widgets, and fields. A view must not grant a permission or
change data semantics. Policy can force, allow, or prohibit a surface at system,
tenant, group, or user scope.
Core browser conformance exercises the German Anwohnerparkausweis reference
context with Work, Calendar, Messages and Files entries, verifies that package
owner labels are absent from the primary rail, expands the technical catalogue,
and runs WCAG 2 A/AA checks over the result. Unit permutations cover two-owner,
one-owner, unauthorized-owner and focused-View compositions.
## Error And Provenance Language
Normal errors answer:
@@ -39,16 +39,24 @@ The first production-shaped slice is implemented:
order and optional labels. Scoped Views therefore configure product
presentation for system, tenant, group, user and Workflow contexts;
- the expanded left rail groups classified destinations while retaining
Dashboard and every authorized unclassified destination under More tools.
Dashboard and every authorized unclassified destination under More tools;
- Core promotes Work (`/work`), Calendar (`/agenda`), Messages (`/messages`)
and Files (`/documents`) into stable primary destinations and collapses the
compatible owner routes under **All available tools**;
- **All available tools** is permission-derived but independent of the active
View, providing a deliberate escape without granting access or discarding
the original `/tasks`, `/calendar`, `/mail`, `/postbox` and `/files` links.
The baseline classification is now manifest-declared for every ordinary
user-facing module and enforced by the workspace manifest check. A separately
The baseline classification and the four initial stable destinations are now
manifest-declared. The area classification covers every ordinary user-facing
module and is enforced by the workspace manifest check. A separately
versioned launch-context contract carries bounded active-object, acting,
temporal, View and return references into full-page Quick Access fallbacks;
Cases publishes the first active-object reference. The remaining rollout is to
add useful bounded tools and active-object publishers only where a maintained
journey benefits, and to extend browser evidence to a pinned reference
composition. Authorized global and technical routes remain visible through
journey benefits. The pinned German Anwohnerparkausweis browser composition
verifies stable product labels, technical escape, keyboard access and WCAG
conformance. Authorized global and technical routes remain visible through
their dedicated shell entry or **All available tools**.
## Quick Access Boundary
@@ -166,9 +174,10 @@ areas, and users may personalize them within Policy ceilings. An empty area is
omitted. An area with one destination may open it directly. A multi-destination
area provides a useful work/recent/action surface rather than another menu.
Familiar product nouns such as Calendar, Mail or Files may remain directly
pinned. The objective is not to hide every module name; it is to prevent
repository topology from determining a person's workflow.
Familiar product nouns such as Calendar or Files remain direct product
destinations. The objective is not to hide every implementation name from
administrators; it is to prevent repository topology from determining a
person's workflow.
The initial module classification is deliberately outcome-oriented:
@@ -6,8 +6,8 @@ Every other active module requires a reviewed explanation of why it owns no
persistent subject-data store. Adding a migration invalidates that explanation.
- Active modules: 72
- Registered and documented DSAR providers: 48
- Reviewed no-store rationales: 24
- Registered and documented DSAR providers: 49
- Reviewed no-store rationales: 23
- Unexplained coverage gaps: 0
| Module | Repository | Persistence | Coverage | Rationale |
@@ -75,7 +75,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
| `soap` | `govoplan-soap` | No module migration | Reviewed no-store rationale | Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store. |
| `tasks` | `govoplan-tasks` | Migration-owned | Provider | Provider `privacy.dsar.tasks` is registered and documented. |
| `templates` | `govoplan-templates` | Migration-owned | Provider | Provider `privacy.dsar.templates` is registered and documented. |
| `tenancy` | `govoplan-tenancy` | No module migration | Reviewed no-store rationale | Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data. |
| `tenancy` | `govoplan-tenancy` | Migration-owned | Provider | Provider `privacy.dsar.tenancy` is registered and documented. |
| `tickets` | `govoplan-tickets` | Migration-owned | Provider | Provider `privacy.dsar.tickets` is registered and documented. |
| `transparency` | `govoplan-transparency` | No module migration | Reviewed no-store rationale | Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store. |
| `views` | `govoplan-views` | Migration-owned | Provider | Provider `privacy.dsar.views` is registered and documented. |
+300
View File
@@ -0,0 +1,300 @@
# Developer command suite
`./devkit` is the maintained entry point for repeated development work. It uses
ordinary local programs, not an AI service. People, CI and coding agents use the
same commands and results. It composes existing GovOPlaN checks and release
services rather than defining parallel product rules.
## Start here
From the Meta checkout:
```sh
./devkit commands
./devkit doctor --repo core
./devkit context --changed
./devkit check --profile quick --repo campaign --dry-run
./devkit coverage --profile ui --repo campaign
./devkit check --profile quick --repo campaign
./devkit latest
./devkit review campaign
```
Use `--help` on each command. `--json` (or `--format json`) returns structured
results; the default summary intentionally omits full logs and source contents.
Global options work before or after the command. `--workspace-root` is the parent
directory of the registered repositories, not the Core checkout. Unknown
repository filters fail rather than silently widening/narrowing scope.
The launcher prefers Meta's `.venv/bin/python`, or the explicit `PYTHON`
executable, with `python3` as the no-venv fallback. Checks resolve `NODE` and
`NPM` from explicit environment settings or `PATH`; no username-specific Node
installation path is required. Doctor reports repairs but never installs
packages, changes configuration, kills an occupied port or starts a server.
## Command contracts
| Command | Purpose | Effects |
| --- | --- | --- |
| `context [--changed] [--repo NAME]` | Offline Git state, instruction/documentation paths and review links | Reads only; upstream counts are not freshly fetched |
| `doctor [--repo NAME] [--profile PROFILE]` | Interpreter, dependency, browser and test-resource preflight | Reads only; repair suggestions are not executed |
| `check --profile PROFILE` | Registered checks with explanations, bounded logs and a receipt | Runs trusted tests/builds; `--dry-run` only plans |
| `coverage --profile PROFILE [--repo NAME]` | Declared suite dispositions and explicit coverage limits | Reads only; not execution evidence |
| `runs [--limit 10] [--before RUN]`, `latest` | Discover recent runs without looking up state-directory paths | Reads only; malformed latest evidence is not replaced with an older pass |
| `status RUN`, `summary RUN` | Stage counts and failure log locations | Reads only |
| `logs RUN --stage STAGE --tail 40 [--final-only]` | Provisional live output or a hash-verified final stage log | Reads only; provisional output never proves a pass |
| `resume RUN` | Replan the previous check selection and reuse eligible successful stages | Runs remaining checks, creating a new receipt |
| `recover RUN [--apply]` | Recover an abandoned check-run record after acquiring its OS lock | No test replay; apply changes only the local record |
| `review MODULE` | Source inventory, principle revision, issue links, check plan and walkthrough checklist | Does not perform or complete a module review |
| `docs audit [--repo NAME] [--changed]` | Owning manifest/help/translation checks and visible-label gaps | Local checks only; never translates or edits content |
| `issues note …` | Preview and explicitly append deduplicated evidence | Remote comment creation only with `--apply` |
| `release …` | Existing durable release planning/execution/recovery | Explicit `--apply`, request IDs and existing step confirmations |
| `git …` | Frozen, explicit-path maintenance commit and exact branch push | Preview first; mutations require `--apply`; no stage-all, force or tags |
Git maintenance is a separate, selected-path workflow; see
[Git maintenance](DEVKIT_MAINTENANCE.md). Issue notes and module bundles are
documented in [Evidence and review commands](DEVKIT_EVIDENCE.md). The complete
release command syntax, limitations and examples are in
[Headless release operations](DEVKIT_RELEASE.md).
## Check profiles and their limits
- `quick`: existing manifest/interface invariants, import/help guards and selected
module source/structure tests. It does not compile component suites or run the
full browser matrix.
- `ui`: quick/source contracts plus the shared Core component batch, compiled
once in an isolated directory. This is not a full visual/usability review.
- `backend`: existing manifest/interface checks and selected repository backend
test suites. Shared backend-test state is serialized.
- `full`: the canonical `tools/checks/check-focused.sh`, including optional-module
build permutations and integrated browser conformance. Repository filters do
not reduce this required cross-module completion gate.
`--changed` includes uncommitted work, commits ahead of the locally cached
upstream and repositories with commits but no configured upstream.
Missing/unreadable repositories remain visible as errors. For the
GovOPlaN profiles, Core or Meta changes conservatively select every registered
consumer; other changes expand through declared interface consumers. This is
not complete semantic dependency analysis. Use the full gate for cross-module
completion as required by `AGENTS.md`.
The module catalog reuses declared direct-Node tests and established structural
script names. Arbitrary shell chains are not guessed or rewritten. Unsupported
declared scripts remain visible with a reason; only exact known Core component
aliases are covered by the shared component runner. Invalid package metadata is
an error, not an empty test inventory.
`coverage` inventories declared suites as `planned`, `covered_elsewhere`,
`excluded` or `unsupported`, naming their covering stage where applicable. The
same inventory accompanies check dry runs and saved receipts. **Full means the
canonical focused gate, not every test in every package.** Its inventory follows
explicit canonical commands conservatively; it does not infer arbitrary nested
shell commands or npm hooks. Add new deterministic checks to the owning
package/canonical gate and the catalog's tests as appropriate. Coverage planning
does not execute the scripts it inventories or assert that they passed. See
[Coverage and validation details](DEVKIT_COVERAGE.md) for examples and boundaries.
Independent checks can run with `--jobs 1..8` (default 2). Shared resources use
OS locks for the current user, including across runs with different evidence
directories. Browser/port locks are shared across that user's workspaces. Full runs
reserve the shared WebUI/backend/browser resources. Standalone Core component
commands use isolated temporary build directories; `npm run test:components --
page-layout documentation-help` compiles once for that batch. Managed temporary
output is cleaned; no user worktree is reset or restored.
## Evidence, interruption and safe reuse
A check announces its run ID and saves a `preparing` receipt before source and
environment fingerprinting. Progress on stderr shows elapsed time, stage counts
and active stages; it also distinguishes `checking` from final snapshot
verification (`finalizing`). `--json` keeps stdout as one final JSON result and
emits structured progress events on stderr; `--quiet` suppresses these events.
Use `latest`, `status RUN` and `logs RUN --stage STAGE` from another terminal
while a check runs. Counts are stages, not a guessed percentage of test effort.
Live snapshots are private, bounded, explicitly provisional and not accepted for
reuse or issue evidence. A completed stage log can be hash-verified before the
whole run finishes, but only the final run snapshot verifies the overall result.
`--final-only` rejects unfinished logs. Run history supports cursor pagination
(up to 100 rows per page), bounds directory scanning, and never deletes evidence.
An explicitly selected `--project` also filters run discovery by project file.
Default records are private below
`$XDG_STATE_HOME/govoplan/devkit/workspace-<identity>/runs/<run>/` (fallback
`~/.local/state`). `--state-dir` selects another base; workspace scoping still
applies. Status changes are atomically persisted with restrictive file modes;
symlinked evidence paths are rejected. A record contains source/environment/plan
fingerprints, stage reasons, status, exit code, timing and log identities.
Stages distinguish pending, running, passed, failed, timed out, blocked, skipped,
stale and interrupted. A skipped or unexecuted stage is never a pass. A source or
environment change during verification marks the overall result **stale**, even
when individual subprocesses returned zero. Such a result is not passing evidence.
`resume RUN` replans the current selection and compares each phase independently.
Matching **verified checkpoints** are reused; changed/new phases and phases
without a checkpoint run again. A checkpoint binds the exact command and working
directory, declared repository inputs, transitive data-dependency identities,
devkit implementation/schema, tool/dependency metadata and environment. Logs must
retain their recorded hashes; a missing or changed cached log fails closed.
The summary reports how many checkpoints were reused; JSON records `input_scope`,
`cache_key`, `checkpoint_verified`, `reused_from` and the reuse/rerun reason.
Each checkpoint is saved while its resource locks remain held, only after exit
zero, a verified final log, and matching before/after inputs and environment.
A phase that changes its own inputs is stale, even if another phase later restores
those bytes. Final verification rechecks all passed/reused phases against current
inputs before the new aggregate can pass. An interrupted, failed or stale run may
donate an independently verified matching checkpoint, but never becomes passing
evidence itself. Old receipts remain readable: legacy monolithic logs and receipts
without these checkpoints cannot be retroactively split or reused as verified phases.
After a hard crash, `recover RUN` shows the abandoned owner and guidance. Inspect the
recorded commands and manually stop any surviving test/build processes before
using `recover RUN --apply --confirm-processes-stopped`. The command verifies the
run lock is free before marking the record interrupted. A free parent lock alone
does **not** prove children stopped after a hard kill. Recovery does not replay
processes or infer that a partially completed operation succeeded.
### Canonical phases and narrower inputs
The full gate has seven explicit phases: `preflight`, `tooling`, `backend`,
`core-ui`, `module-builds`, `browser`, and `module-ui`. Devkit stage IDs are
`focused.<phase>`. The authoritative Bash bodies remain in
`tools/checks/check-focused.sh`; the bounded metadata in
`tools/checks/focused-phases.json` names their order and resources. The direct
shell command still runs every phase in the original fail-fast order. For inspection:
```sh
tools/checks/check-focused.sh --list-phases
./devkit check --profile full --jobs 1
./devkit resume RUN_ID --jobs 1
```
For example, a browser failure no longer forces successful backend/build phases
to run again if their inputs still match. Standalone `--phase browser` is available
for diagnostics, but is not a substitute for the complete gate or its receipt.
Scopes are whole repositories, not inferred file globs. A trusted check may declare
`"inputs": {"repos": ["canonical-repository-name"]}`; no declaration means the whole
registered workspace. Dry runs show that boundary. Native source-discovery checks
retain broad scopes where cross-module dependencies cannot safely be narrowed.
Changes outside an explicit scope do not invalidate it, but tool/environment
changes remain conservatively global. Scope declarations are a correctness
contract: include every source repository that the check reads, not only its cwd.
Native discovery ownership/presence is rechecked at phase boundaries, so a newly
appearing module or WebUI directory invalidates the in-flight environment identity.
Unregistered native sibling sources disable reuse with an explicit coverage note;
register them before relying on cached results. Ordinary generated files inside
existing source directories do not change this directory-shape identity.
Every snapshot rereads repository presence, HEAD, index/flags and tracked plus
non-ignored untracked membership, including assume-unchanged files. File hashes
use a bounded **in-process-only** memo with file identity/ctime/mtime/size checks
before and after an open file descriptor; no persistent mtime cache is trusted.
In-repository regular-file symlink targets are included; escaping or directory
symlinks fail closed. Ignored files, undeclared external data and service state
are not source evidence and need explicit verification.
Environment probes stream bounded regular-file contents and reject concurrent
replacement, growth and symlink retargeting. Stable venv executable and dependency
directory symlinks remain supported without changing the executable path; absent
optional metadata remains optional. Environment identities are not persistently
cached and remain deliberately global.
These checkpoints cache **verification results, not output artifacts**. The full
gate's phases were audited so later phases do not require an earlier phase's
retained build output. A custom setup/build that produces files consumed later
must declare `"reuse": "never"`; its consumers must use `deps`. Until output
manifests/restoration exist, do not assume a cached successful build recreates
deleted ignored artifacts.
Check output is drained without unbounded memory growth. Stored output retains
the beginning and actual final tail within an 8 MiB bound per stage, with a small
truncation marker and an omitted-byte count; output truncation does not change
the subprocess exit result. `logs` exposes at most 200 lines and 16 KiB. Live
snapshots retain at most 64 KiB after redaction. Incomplete live lines and cut
retention-boundary lines are withheld to avoid exposing fragments of secrets;
very long single-line output may therefore be absent from live views.
Known environment credentials and common authorization patterns are
redacted as display hygiene. **Never pass credentials as check arguments or print
them from tests.** Redaction is not a general secret-classification guarantee.
JSON command output is a redacted presentation; the receipt file at the reported
path is the canonical local record. Cancellation and deadlines terminate the
owned process group; a parent leaving running descendants does not pass.
During source/environment fingerprinting, cancellation is observed between
probes; the current bounded probe may finish first. Cancellation during final
verification is recorded as interrupted, never as a passing run.
Deliberately detached processes are outside that group: this is not a sandbox.
These local hash-bound records detect accidental changes; they are not signed
attestations, a security certification, proof of complete test coverage, or
permission to publish. External service state and undeclared dependencies still
need explicit verification. No automated result closes a Gitea issue or marks a
module reviewed. Detailed logs remain local unless deliberately shared. Old run
directories are not automatically deleted; review retention before removing any
evidence referenced by an issue.
## Reuse in another project
The runner, repository context, doctor and evidence primitives are usable with an
explicit local JSON project manifest. The example at
`tools/devkit/examples/project.json` registers ordinary Python tests and a Git
whitespace check; the format is described by
`tools/devkit/project.schema.json`.
```sh
/path/to/govoplan/devkit --workspace-root /path/to/project \
--project /path/to/project/devkit-project.json check --profile quick --dry-run
```
Repository paths and check working directories must remain inside the selected
workspace. Check arguments are arrays, not shell-evaluated strings. The supported
tool placeholders are `{python}`, `{node}`, `{npm}` and `{workspace}`. Check
dependencies form an acyclic graph; resources serialize incompatible tasks.
`deps` declares actual data dependencies: consumers rerun whenever a dependency
runs again. `after` declares only fail-fast execution ordering: a failed predecessor
skips the follower, but a successfully rerun independent predecessor does not
invalidate the follower's inputs. Both edge types are validated together for
cycles, missing IDs and overlap. `reuse` is `verified` by default or `never` for
setup/output-producing checks. Repository `inputs` names are canonical, unique,
nonempty and validated even for unselected checks.
The complete custom manifest is validated, including unselected checks/profiles:
unknown fields, misspellings, invalid bounds/types and unresolved dependencies
fail early. This prevents silently ignored resource or timeout declarations.
Portable `doctor` always requires its Python runtime, plus tools needed by the
selected checks and their dependencies and tools explicitly configured in the
manifest. Without `--profile`, all declared profiles are considered. Unused
Node/npm are marked `not_required` and do not block Python-only projects. Declare
indirect tool dependencies explicitly: arbitrary script contents are not
analyzed. Native GovOPlaN retains its Python/Node/npm requirements.
Only use project manifests and scripts you trust: running a registered test is
ordinary code execution, not a sandbox. Platform-specific release and Docs
commands reject generic project manifests; put another project's checks in its
own profiles rather than pretending its release policy is GovOPlaN's.
The initial runtime targets POSIX environments with Python 3.11+ and OS advisory
locks; it does not claim Windows support. No globally installed service or Codex
plugin is needed. Keep this implementation versioned and reuse it; avoid copying
diverging helper implementations into each project.
Devkit is not a complete build system: it does not infer semantic dependencies,
restore output artifacts, sandbox checks, or share signed remote caches. Repository
scopes intentionally stop short of file/glob narrowing; environment identities
remain global. These are explicit boundaries, not claims of exhaustive coverage.
## Development and conformance
```sh
./.venv/bin/python -m pytest -q tests/test_devkit_*.py
./.venv/bin/python -m pytest -q tests/test_focused_phases.py
node --test tests/test-devkit-display-labels.mjs
./.venv/bin/python -m unittest tests.test_documentation_structure
```
The focused gate includes the Python command-suite regression tests. Fixture
tests use temporary repositories/processes and mocked issue/release transports;
they do not commit user work, send mail, publish artifacts or change live issues.
Documentation is part of each command change: update `--help`, this guide and the
corresponding targeted contract tests together.
+72
View File
@@ -0,0 +1,72 @@
# Check configuration and suite coverage
`devkit coverage` explains what a selected plan intends to run. It does not run
tests, start servers, or report successful verification:
```sh
./devkit coverage --profile quick --repo campaign
./devkit coverage --profile ui --repo portal --json
./devkit coverage --profile full --json
./devkit --project /path/to/project.json coverage --profile quick
```
The inventory accounts for declared `test` and `test:*` scripts in each
registered repository's root and `webui/package.json`, discovered UI structural
checks, and explicitly configured project checks. It is a suite inventory, not
an enumeration of every test function or recursive dependency. Unselected suites
remain visible; every row has a disposition and reason:
| Disposition | Meaning |
| --- | --- |
| `planned` | A selected stage directly invokes the suite. |
| `covered_elsewhere` | An exact equivalent invocation or an explicitly selected shared batch owns it; the covering stage is named. |
| `excluded` | The suite is outside the actual selected plan. |
| `unsupported` | Scoped discovery cannot safely interpret or locate the command; no command is guessed or executed. |
`full` means the existing canonical `check-focused.sh` gate. It does **not**
mean every package script or every component suite. Coverage reads the current
script's explicit npm/Node commands inside its seven registered marked phase
bodies, without executing shell code or inferring arbitrary functions, nested
scripts, npm hooks, branches or here-document contents. Each covered suite names
the `focused.<phase>` that owns it. The
current gate names four of the sixteen Core component suites: layout primitives,
page layout, DataGrid actions/sizing, and Mail components. The other twelve stay
explicitly excluded from that gate's component batch. `ui` runs the shared
sixteen-suite batch once; `quick` does not compile component tests.
Only Core's exact known component aliases receive that shared-batch treatment.
A different repository using the same filename, an unknown alias, or a compound
command containing the runner is not silently credited or launched. Ordinary
scoped source discovery accepts bounded direct `node`/`node --test` commands
targeting regular package-owned `.mjs` files in `scripts` or `tests`; shell
chains, extra flags, missing targets and symlinked targets are not rewritten.
An explicit invocation already present in the canonical full gate remains part
of that gate, even when the narrower discovery profile does not support it.
Coverage rows retain a command hash and, for supported commands, redacted argv.
Unsupported shell bodies are not copied into the inventory. This is display
hygiene, not permission to put secrets into project commands. Coverage is
attached to check plans/receipts; it never turns an excluded suite into a pass.
## Portable-project validation
The runtime validates custom project metadata against the published
[`project.schema.json`](../../tools/devkit/project.schema.json) using a small
dependency-free validator. All checks and profiles are validated before
selection, including checks the chosen profile does not execute. Unknown nested
fields such as `resource` or `timout_seconds`, malformed types, oversized values,
duplicate references, missing dependencies, unknown profiles and dependency
cycles fail with controlled errors. Repository aliases and resolved paths must
be unambiguous and confined to the workspace. Check `cwd` defaults to `.` when
omitted; configured paths cannot escape the workspace.
Package JSON is bounded to 1 MiB and 512 scripts; duplicate JSON keys, excessive
nesting and malformed script objects fail before planning. Command strings are
bounded to 8,192 characters. This validates configuration, not the safety of its
code: run only project manifests and test scripts you trust.
Checks may declare repository-scoped `inputs`, true data `deps`, order-only
`after`, and `reuse: "never"` for setup/output-producing work. Unknown repositories,
empty/duplicate scopes, overlapping edge types and cycles fail before execution.
Missing inputs deliberately fall back to all registered repositories. See
[checkpoint and input contracts](DEVKIT.md#canonical-phases-and-narrower-inputs).
+171
View File
@@ -0,0 +1,171 @@
# Devkit review bundles and issue evidence
Gitea remains the canonical backlog and review state log. These commands assemble
guidance and append evidence; they do not create a second progress tracker, close
issues, rewrite issue bodies, tick checkboxes, or certify a module review.
## Prepare a module review
```sh
./devkit review campaign
./devkit review campaigns --json --output /tmp/campaign-review.json
./devkit review campaign --evidence RUN_ID
```
Repository names, repository aliases and scope IDs from the
[issue discovery inventory](../project/ui-review-issue-inventory.json) are accepted.
`review bundle campaign` is an equivalent spelling. A bundle contains the current
source inventory and fingerprint, canonical module/central issue links, the
Core principle revision and content hash, applicable **planned** check stages,
and a manual checklist. No checks or application servers are started. The
optional output file is a local snapshot, not an authoritative state record.
The filename inventory is a starting point, not proof that all runtime surfaces
were found. It does not import application manifests or optional modules. Review
routes, panes, dialogs, settings, widgets, public forms, contributed/headless
interfaces, missing-module behavior, permissions, English/German and narrow/wide
layouts manually. Missing source, no UI files or no planned checks is never an
automatic pass or N/A. Record exceptions and remaining work in the module issue.
Apply [UI-01 and UI-02, and the other Core principles](../../../govoplan-core/docs/UI_DESIGN_PRINCIPLES.md):
books belong beside meaningful visible text; normal screens display compact
data, with intentional scoped dialog editing. A bulk editing mode needs a
documented reason. Record new principles and their back-propagation to previously
reviewed modules through the [central review program](../project/UI_REVIEW_PROGRAM.md).
An explicit portable `--project project.json` does not inherit GovOPlaN issue
links or principles. Its optional configuration is:
```json
{
"review": {
"issue_inventory": "meta/docs/review-issues.json",
"principles": "core/docs/UI_DESIGN_PRINCIPLES.md"
}
}
```
Paths are relative to `--workspace-root` and cannot escape it or traverse
symlinks. The inventory uses schema version 1, `issues[]` with `scope_id`,
`repository`, `number`, `url`, and optional `name`/`kind`; an optional `epic`
contains `repository`, `number`, `url`. Snapshot issue-state fields are not
reported as current state. Custom checks come from the project's selected
profile (`--profile ui` by default).
## Preview and append evidence
```sh
./devkit issues note --root govoplan-campaign --issue 103 \
--evidence RUN_ID --summary 'Targeted checks passed.' \
--next 'Manual module review and remaining fixes are still required.' --json
./devkit issues note --root govoplan-campaign --issue 103 \
--evidence RUN_ID --summary 'Targeted checks passed.' \
--next 'Manual module review and remaining fixes are still required.' \
--env-file /private/gitea.env --apply
```
`--root` is resolved relative to the selected **workspace root**, not the shell's
current directory. In the normal siblings workspace, use `--root govoplan-campaign`
or an absolute checkout path; do not use a path escaping the workspace.
The default is a completely offline dry run; it does not load credentials or
contact Gitea. `--apply` explicitly authorizes comment creation. Tokens are read
from `GITEA_TOKEN` or the explicit `--env-file` (environment wins); there is no
implicit `.env` search. The credential file is parsed, never sourced, and is not
printed or uploaded. Git's selected remote (default `origin`) determines the
exact target: ambient `GITEA_OWNER`, `GITEA_REPO` and `GITEA_URL` cannot retarget
the request. Credential-bearing HTTP Git remote URLs are rejected.
Use repeatable `--summary`/`--next`, `--body-file` for Markdown, or `--note-file`
with structured JSON `{ "summary": ["..."], "next": ["..."], "body": "..." }`.
Do not put secrets in notes, paths, check arguments or logs. Display redaction is
best-effort protection, not permission to include confidential data.
For a bounded multi-issue operation, replace `--root`/`--issue` with
`--target-plan targets.json`:
```json
{
"schema_version": 1,
"targets": [
{
"root": "govoplan-campaign",
"issue": 103,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103"
}
]
}
```
Targets must be unique, inside the workspace and bound exactly to their Git
remote/issue URLs, all on one exact Gitea base URL. Optional `issue_id` binds
Gitea's immutable numeric ID as well. Every target, complete comment history,
marker collision and local journal binding is checked before the first POST.
Comments are then posted serially and read back against the exact issue. A
closed issue stays closed; existing bodies, comments and checklists are untouched.
Each note has a stable hidden marker binding its issue, `--key` purpose
(`verification` by default), and evidence run ID. Identical already-published
notes are read back instead of reposted; changed content with the same marker
is a collision, not an update. Use a distinct intentional `--key` for a separate
follow-up. Without a receipt, the note's structured content determines identity.
A lost POST response is **not** retried automatically. The publisher scans all
comment pages and reads back any matching comment. If it cannot confirm the
result, it records `uncertain`, stops later posts, and exits nonzero. The next
invocation reconciles again without replaying the POST. Only after inspecting
the issue and local result, use `--apply --retry-uncertain` to permit one new
attempt when the marker is still absent. Local journals and OS-released locks
live in the workspace-scoped devkit state directory; do not delete them to bypass
uncertainty. These are retry receipts, not issue state. Independent machines or
different state directories cannot share a lock: coordinate publishers, because
Gitea comment creation does not provide an atomic idempotency key.
## What a receipt does and does not prove
`--evidence RUN_ID` reads the workspace-scoped runner receipt and checks local
integrity. `--evidence /path/to/receipt.json` accepts bounded validated external
receipt metadata, explicitly labeled `external-unverified`. A self-consistent
digest is not a signature or independent verification. Foreign-workspace,
malformed and inconsistent successful receipts are rejected. Receipt-provided
commands are never run; log files are not opened or uploaded, only referenced.
Successful receipts must explicitly record a verified source snapshot; an
external file's flag is still a reported claim, not an independent attestation.
An aggregate containing a skipped stage cannot claim success. New versioned
receipts additionally require independent verified checkpoints for passed phases.
Failed/interrupted/stale aggregates remain labeled as such even when they contain
reusable successful phases; reuse creates a new run and preserves the original.
Stage coverage limitations are validated and retained in the note, including
checks omitted by scoped profiles. Compact review output shows the first eight
limitations and points to the complete list in its JSON bundle. A passing stage
never implies that omitted compiler, browser, permutation or manual checks ran.
Documentation audit limitations are saved with their stage receipts, so later
review bundles and issue notes retain the same scope as the original command.
Managed inventory collectors use the explicit selected workspace throughout;
an incomplete checkout never borrows sources or missing module imports from a
fuller default workspace. Missing required Core/parser dependencies fail the
audit. Legacy direct inventory commands without `--workspace-root` retain their
discovery behavior; use the explicit option for reproducible evidence.
The note distinguishes `matches-current`, `historical-source-differs`,
`different-project-not-compared` and unavailable comparisons. Matching source
is a comparison at bundle/note preparation time, not a live test run or proof
that a person completed review. If the source or environment has changed, run
the appropriate checks again and keep historical evidence labeled honestly.
Versioned repository-input receipts compare the recorded repository scope, not a
legacy whole-workspace hash. Partial scopes remain explicitly labeled in evidence
notes. This comparison reads source only; receipt commands are never executed.
Legacy receipts retain their original comparison format. A matching source scope
does not prove that ignored build outputs, the environment or live services match.
Fixtures cover dry-run isolation, exact target bindings, capped/repeated
pagination, collisions, append-only idempotency, uncertain POST reconciliation,
secret-safe errors, receipt validation, portable projects and optional-module
review boundaries:
```sh
python3 -m pytest -q tests/test_devkit_issues.py tests/test_devkit_review.py
```
+97
View File
@@ -0,0 +1,97 @@
# Selected-path Git maintenance
The optional `git` namespace of `tools/devkit/devkit.py` is a deliberately
narrow maintenance workflow. It is not a replacement for normal Git, the
release executor, required human review, or Gitea's canonical issue state.
From the Meta repository, preview and then save a plan for explicit files:
```sh
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example"
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example" --apply
```
Use the saved `git-…` plan ID for the remaining commands. Each mutation needs
its own `--apply`; an unqualified command only previews the operation:
```sh
python3 tools/devkit/devkit.py git commit git-PLAN_ID
python3 tools/devkit/devkit.py git commit git-PLAN_ID --apply
python3 tools/devkit/devkit.py git push git-PLAN_ID
python3 tools/devkit/devkit.py git push git-PLAN_ID --apply
python3 tools/devkit/devkit.py git status git-PLAN_ID
```
The immutable local plan binds the registered repository, HEAD, branch, origin
fetch/push URL hashes, Git configuration, complete index identity, explicit
selected file paths, working-tree hashes, expected Git blobs, and commit message.
Raw remote URLs and file contents are not stored. A changed input blocks the
operation; inspect it and create another plan instead of weakening the check.
Receipts are private local records with integrity checks, not signed approval
or a security attestation.
Commit captures each selected file once, verifies the captured bytes and Git
blob against the plan, and builds an isolated index/tree from the frozen HEAD
plus only those blobs. Git's normal identity rules create the commit object;
an atomic branch compare-and-swap publishes it only if the planned parent is
still current. Active hooks/signing remain refused rather than bypassed.
An editor racing with this operation cannot substitute newer working-tree
bytes: those edits remain uncommitted, and the helper never overwrites files.
The real index is protected by Git's standard index lock. Only selected entries
are updated from a prepared private copy; unrelated staging and index flags are
preserved. If the index changes independently, its new data is not overwritten
and the receipt requires reconciliation. The resulting parent, message,
changed paths, blobs/modes and unrelated index entries are verified.
Git refs and index files are separately atomic, not one filesystem transaction.
Independent writers that ignore the standard index lock can still change refs
during publication. The helper rechecks the exact recorded candidate before
index recovery and before recording success; it refuses an uncertain result
instead of adopting a newer HEAD or claiming that competing work was its own.
Directories, noncanonical paths, implicit globs, staging-all, amend,
force-push and extra-tag publication are not supported. A selected file with
different staged and working-tree changes is refused; decide explicitly which
version to commit using normal Git first.
Push requires the recorded commit to remain current, an unchanged origin and
one standard-transport push destination. It requests a normal, non-forced push
of that commit to the recorded branch and checks the remote branch afterward.
The push preview does not contact a remote. A normal push may run the remote's
usual CI or server-side hooks; this is an external effect of the separately
authorized push, not of planning or committing.
Local active hooks (including index-change hooks), filter/encoding attributes,
signing, external fsmonitor/SSH configuration, Git environment overrides,
in-progress merge/rebase/cherry-pick operations, submodules, symlinks and
assume-unchanged/skip-worktree entries, split indexes and replacement/graft
history are deliberately
unsupported. The helper
refuses these cases instead of disabling hooks, signatures or filters. An
unused globally installed filter definition alone does not block maintenance;
active attributes are checked across tracked files as well as selected files.
Custom remote receive-pack/upload-pack/helper commands, recursive submodule
pushes and partial-clone lazy fetch are also refused. Of inherited `GIT_*`
variables, only `GIT_OPTIONAL_LOCKS`, `GIT_TERMINAL_PROMPT` and `GIT_PAGER` are
allowed (every command explicitly passes `--no-pager`, so the pager is inert);
namespace, identity, alternate-index/object-directory and unknown overrides
are not silently removed. Git subprocesses have bounded input/output and
deadlines; cancellation terminates their owned process group, including
transport helpers. Network failures can still leave a remote effect uncertain,
which is why receipts require explicit reconciliation rather than blind retry.
An interruption or uncertain failure is recorded before any retry. Use
`git reconcile git-PLAN_ID` to preview, then add `--apply` to verify an
already-existing result. Reconciliation never makes a commit or push. It may
finish the selected real-index update after an interrupted commit publication,
but only when the original index fingerprint still matches; independent staged
work is preserved and requires normal Git resolution. For an
uncertain push it may read the frozen remote, but only with `--apply`. If no
planned commit is current, the helper reports that state and requires a fresh
plan after inspection. A forcibly killed process may leave its private scratch
index or an owned Git index lock; the helper never guesses that an existing
lock is safe to delete. Blob/commit preparation can leave unreferenced Git
objects for ordinary Git garbage collection. It never automatically resets,
restores, deletes, or rolls back user work.
The maintenance tests use disposable local repositories and local bare remotes.
They never commit or push the user's workspace repositories.
+121
View File
@@ -0,0 +1,121 @@
# Headless durable releases
The devkit `release` commands use the **same release application, request
validation, run store and bounded executors** as the
[release console](RELEASE_CONSOLE.md). They invoke its ASGI application inside
the current process, with an ephemeral internal token. No listening socket,
background server, browser session or externally supplied console URL is used.
The development environment's FastAPI/HTTPX dependencies load only when a
release command runs; ordinary devkit help does not import them.
This is a GovOPlaN-specific provider. Repository and origin authority still comes
from the release service's registered catalog. `--project` is explicitly rejected
for release commands; a portable project configuration must not silently override
that authority. A different workspace path does not authorize arbitrary release
repositories, remotes, commands, signing policies or source bindings.
The default compact output includes the overall status, selected repository
states and versions, source-preflight readiness, bounded gate findings and the
recommended next action. Run inspection also shows step-state counts and the
first outstanding steps. These are projections of the service response, not new
readiness checks. Use `--json` for the complete existing plan or receipt; compact
output deliberately omits executor commands, arguments and source bindings.
## Commands and boundaries
| Command | Purpose | Effect boundary |
| --- | --- | --- |
| `release status` | Current release dashboard | Offline by default; explicit remote/catalog flags enable their checks |
| `release plan` | Selective repository/version plan | Inspection only; does not create a run |
| `release list` | Bounded run history, with `--limit` and `--cursor` | Existing workspace-scoped store |
| `release show RUN_ID` | Exact verified run, steps, receipts and required confirmations | No release execution |
| `release create` | Freeze a selected repository/version plan | Preview by default; `--apply` persists a run |
| `release preview RUN_ID STEP_ID` | Frozen step and its current constraints | No step claim; catalog publication additionally uses the existing validated-candidate preview |
| `release execute RUN_ID STEP_ID` | One available durable step | Preview by default; `--apply`, explicit request ID and the step's exact confirmation are required |
| `release resume RUN_ID` | Mark persisted running attempts interrupted | Requires `--apply`; never assumes an effect succeeded |
| `release retry RUN_ID STEP_ID` | Prepare an eligible failed/read-only-interrupted step | Requires `--apply`; does not execute it |
| `release reconcile RUN_ID STEP_ID` | Record a proven uncertain-write outcome | Requires `--apply`, `--confirm RECONCILE` and an explicit outcome |
Create and transition commands require caller-chosen `--request-id` values.
Retain the same ID for an uncertain replay of the **same** command; changing
inputs under an existing ID fails closed. No command automatically retries a
mutation. These commands perform one explicit transition, not an implicit
"release everything" loop.
Read commands do not commit, tag, push, publish or apply database migrations.
The existing private run store can initialize its lock directory or upgrade a
legacy record while inspecting it; this does not advance release steps.
`--include-migrations` requests the existing audits, never migration application.
`--online`, `--remote-tags` and `--public-catalog` are explicit network-check
choices. A generic step preview describes frozen intent; it is not a claim that
live preflight, remote identity or release acceptance has passed.
## Example: review, freeze and execute one step
Run the devkit through the workspace `./devkit` entrypoint. These examples put
global options before `release`:
```sh
./devkit --workspace-root /path/to/workspace --format json release plan \
--repo-version govoplan-files=0.1.9
./devkit --workspace-root /path/to/workspace release create \
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001
./devkit --workspace-root /path/to/workspace release create \
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001 --apply
./devkit --workspace-root /path/to/workspace release show RUN_ID
./devkit --workspace-root /path/to/workspace release preview RUN_ID STEP_ID
./devkit --workspace-root /path/to/workspace release execute RUN_ID STEP_ID \
--request-id files-release-step-0001 --confirm REQUIRED_CONFIRMATION --apply
```
`RUN_ID`, `STEP_ID` and `REQUIRED_CONFIRMATION` stand for the actual values
returned by the frozen run. For read-only preflight/alignment/install-verification
steps, omit `--confirm` when the service reports an empty confirmation. Metadata,
commit, release-lock, source-tag, source-push, candidate-generation and catalog
publication steps keep the existing `UPDATE`, `COMMIT`, `LOCK`, `TAG`, `PUBLISH`,
`GENERATE` and `PUSH` confirmations respectively. Prerequisite ordering is enforced
by the same store; a later step cannot be forced through this adapter.
Version selection supports repeated `--repo-version REPO=VERSION`, or repeated
`--repo REPO` with `--target-version VERSION`. Every created run needs explicit
versions for all selected repositories. Conflicting version assignments are
rejected. Planning is selective; repositories are never silently selected merely
because their worktrees are dirty.
## Candidate publication and recovery
Use the frozen run's `catalog:selective-generator` and
`catalog:validate-sign-publish` steps, not the disabled legacy mutation endpoints.
Generation accepts repeated `--signing-key KEY_ID=PRIVATE_KEY_FILE`; these are
operator-owned key-file references, never inline key material. Signing arguments
are not emitted in devkit JSON or persisted by the adapter. Publication consumes
the candidate receipt created by that run, not an arbitrary candidate directory.
The existing source/runtime trust, immutable tag/remote identity, private
candidate checks and exact commit-delta checks remain authoritative.
After an interrupted write, inspect the run and the actual local/remote effect.
Use `resume --apply` if an attempt was left running. Then use `reconcile` with
`effect_absent`, `effect_succeeded` or `unresolved`, `--confirm RECONCILE`, a new
reconciliation request ID and `--apply`. Successful reconciliation still performs
the service's independent receipt checks. An unresolved effect must not be
retried or papered over by generating a new request ID. A failed/read-only
attempt may use `retry --apply` and a new explicit execute attempt only when
the existing lifecycle makes that safe.
## State and limitations
Every successful response exposes `state_location` and `candidate_location`.
Without `--state-dir`, the adapter uses the console's existing default private,
workspace-fingerprinted state directory. With `--state-dir PATH`, it uses
`PATH/release-console/workspace-<fingerprint>/release-runs`. Workspaces cannot
read or resume one another's runs. Existing ownership, symlink, permission,
retention and record-integrity checks are unchanged.
Release preparation can commit only the recognized, receipt-bound metadata
changes its executors produced. This command does **not** stage arbitrary source
changes, use `git add -A`, force-push, retarget tags or enable the disabled generic
prepare/sync/push endpoints. Normal selected-path maintenance requires its own
explicit reviewed-change workflow; it is not silently folded into release.
+138
View File
@@ -0,0 +1,138 @@
# Full registry candidates / Vollständige Registry-Kandidaten
## Operator workflow (EN)
The canonical `release-catalog.py full-registry` command takes `--package-set`,
`--package-lock`, `--wheelhouse`, `--webui-packages`, `--output-dir`, and a
configured `--catalog-signing-key`. Generate the package set with
`generate-release-package-set.py --profile full` and download its exact artifacts
with `resolve-package-artifacts.py`; do not substitute locally rebuilt wheels.
The candidate compares the package set with the exact developer meta-package
pins, checks archive bytes and package metadata against the lock, and synthesizes
every entry from its immutable tagged manifest. Native package publication and
its CI authority remain trusted: this verifies the published artifact identity,
not independent reproducible-build equivalence to source.
Pass `--selected-repository` once for each newly released repository, including
Core when it changes. These selected units must have clean, version-aligned
named branches whose HEAD equals the annotated local and remote release tag.
Other full-profile packages retain their exact older annotated tags; a later
workflow-only commit on `main` does not relabel those package contents or force
a version bump. Every source fetch/push endpoint must match the registered
origin, and all entries bind their source commit and annotated tag object.
Git replacement objects, caller Git configuration, and executable-path
redirection cannot substitute another tagged manifest tree.
The fixed existing website catalog and keyring are authenticated before signing.
An older catalog without a signed keyring hash can be migrated only through this
complete rebuild, only when its signature verifies and its entire keyring
exactly matches the configured known signers. No old entries or artifact hashes
are reused. The new catalog signs the exact unchanged website keyring hash;
key rotation remains a separate reviewed operation. Selective candidates still
reject unpinned base keyrings. New candidate directories are private and
exclusive: a retry must choose a new directory, not overwrite a reviewed one.
Run these commands on the trusted host, with an operator-private source workspace
and artifact directory. `RELEASE_PYTHON` must select its private environment and
`RELEASE_NPM` an absolute npm executable with a trusted sibling Node 22 binary.
In Flatpak, execute host commands through `flatpak-spawn --host`; sandbox and
host UID mappings are not interchangeable. Do not weaken trust gates or change
system-wide permissions.
```sh
# These paths identify previously prepared private operator resources.
RELEASE_WORKSPACE=/path/to/private/workspace
RELEASE_PYTHON="$RELEASE_WORKSPACE/govoplan/.host-venv/bin/python"
RELEASE_NPM=/path/to/private/node22/bin/npm
ARTIFACT_ROOT=/path/to/private/artifacts
RELEASE_CANDIDATE=/path/to/private/new-candidate
RELEASE_VERSION=0.1.45
RELEASE_TOOLS="$RELEASE_WORKSPACE/govoplan/tools/release"
umask 077
"$RELEASE_PYTHON" "$RELEASE_TOOLS/generate-release-package-set.py" \
--version "$RELEASE_VERSION" --profile full \
--workspace "$RELEASE_WORKSPACE" --output "$ARTIFACT_ROOT/packages.json"
PATH="$(dirname "$RELEASE_NPM"):/usr/bin:/bin" \
"$RELEASE_PYTHON" "$RELEASE_TOOLS/resolve-package-artifacts.py" \
--package-set "$ARTIFACT_ROOT/packages.json" \
--wheelhouse "$ARTIFACT_ROOT/wheels" \
--webui-packages "$ARTIFACT_ROOT/webui" \
--lock-output "$ARTIFACT_ROOT/artifacts.lock.json" \
--python "$RELEASE_PYTHON" --npm "$RELEASE_NPM"
# Repeat --selected-repository for EVERY newly released unit, not just Core.
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" full-registry \
--workspace-root "$RELEASE_WORKSPACE" \
--package-set "$ARTIFACT_ROOT/packages.json" \
--package-lock "$ARTIFACT_ROOT/artifacts.lock.json" \
--wheelhouse "$ARTIFACT_ROOT/wheels" --webui-packages "$ARTIFACT_ROOT/webui" \
--output-dir "$RELEASE_CANDIDATE" --selected-repository govoplan-core \
--catalog-signing-key known-key=/path/to/private/known-key.pem --json
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" publish-candidate \
--workspace-root "$RELEASE_WORKSPACE" --candidate-dir "$RELEASE_CANDIDATE" \
--channel stable --npm "$RELEASE_NPM" --build-web \
--commit --tag --push --tag-name "catalog-v$RELEASE_VERSION" --json
```
The last command is a strict non-mutating preview because `--apply` is absent.
Review its output, then repeat it with `--apply` to publish. The website's locked
build dependencies must already be installed before `--build-web`. The publisher
sanitizes the build and Git environments and pushes the verified immutable
website commit/tag. Source tag publication and registry package availability
must be complete before candidate generation.
Source tags, registry packages, and a signed module catalog do not imply that a
new runtime distribution exists. While runtime images are held, leave the Meta
Gitea runtime Release held too: a normal source-only Release can replace Gitea's
`releases/latest` discovery result despite having no deployment assets. The
deployer still requires an explicit signed manifest, digest, and trusted
keyring; it does not deploy a tag or module catalog directly.
## Betriebsablauf (DE)
`release-catalog.py full-registry` übernimmt den vollständigen Paketbestand,
die Registry-Sperrdatei, das Wheel-Verzeichnis, die WebUI-Archive und den
konfigurierten Signaturschlüssel. Zuerst mit
`generate-release-package-set.py --profile full` die exakten Meta-Paketversionen
ermitteln und mit `resolve-package-artifacts.py` die veröffentlichten Artefakte
herunterladen. Lokal neu gebaute Wheels sind kein Ersatz. Der Kandidat prüft
Paketidentitäten, Dateigrößen und Hashes und erzeugt alle Einträge aus den
unveränderlichen getaggten Manifesten. Die Registry und ihre veröffentlichende
CI bleiben eine Vertrauensgrundlage; dies ist kein unabhängiger Nachweis eines
reproduzierbaren Builds aus dem Quellcode.
Jedes neu veröffentlichte Repository wird mit `--selected-repository`
angegeben. Nur diese Auswahl muss mit dem sauberen, versionsgleichen HEAD eines
benannten Branches und dem annotierten lokalen und entfernten Tag übereinstimmen.
Unveränderte Pakete behalten ihren ursprünglichen Tag, auch wenn auf `main`
bereits eine spätere Workflow-Korrektur liegt. Alle Quelladressen müssen dem
registrierten Ursprung entsprechen; Commit und annotiertes Tag-Objekt werden
für jeden Eintrag gebunden. Git-Ersetzungsobjekte oder fremde Git-Konfiguration
können dabei keinen anderen Manifestbaum unterschieben.
Vor dem Signieren werden der bestehende Website-Katalog und sein Schlüsselbund
geprüft. Ein alter Katalog ohne signierten Schlüsselbund-Hash darf ausschließlich
durch diesen vollständigen Neuaufbau migriert werden: Seine Signatur muss gültig
sein und der gesamte Schlüsselbund exakt den konfigurierten bekannten Signierern
entsprechen. Alte Einträge oder Artefakt-Hashes werden nicht übernommen. Der neue
Katalog bindet den unveränderten Schlüsselbund-Hash; ein Schlüsselwechsel bleibt
ein eigener geprüfter Vorgang. Selektive Kandidaten verlangen weiterhin einen
bereits gebundenen Schlüsselbund. Kandidaten werden nur in neuen privaten
Verzeichnissen erzeugt und niemals überschrieben.
Das obige Befehlsbeispiel wird auf dem vertrauenswürdigen Host ausgeführt. Dafür
die private Python-Umgebung und einen absoluten `--npm`-Pfad zu Node 22 verwenden;
unter Flatpak die Host-Werkzeuge über `flatpak-spawn --host` aufrufen. Die
gesperrten Website-Build-Abhängigkeiten vorher installieren. Keine
Vertrauensprüfung umgehen und keine globalen Rechte ändern. Die Artefaktordner
müssen privat und bei der Auflösung leer sein. Vor der Kandidatenerzeugung
müssen Quell-Tags und Registry-Pakete vollständig veröffentlicht sein.
Die Veröffentlichung zunächst mit `publish-candidate --commit --tag --push
--build-web` ohne `--apply` prüfen und erst nach Prüfung mit `--apply` ausführen.
Solange Laufzeit-Images zurückgestellt sind, bleibt auch das Meta-Gitea-Runtime-
Release zurückgestellt: Ein reines Quellcode-Release könnte sonst als neuestes
Release erscheinen. Eine Installation benötigt weiterhin ein signiertes
Laufzeitmanifest, dessen Digest und einen explizit vertrauenswürdigen Schlüsselbund.
@@ -361,6 +361,15 @@ and WebUI API proxy traffic across API replicas. The WebUI and API services do
not publish host ports. HAProxy has no Docker socket and discovers only the
bounded replica slots rendered into `load-balancer.cfg`.
New installation specifications default to HAProxy `3.2.23-alpine`, pinned to
registry index `sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
This patch remains on HAProxy 3.2 LTS and Alpine 3.24.1. Loading an existing
specification preserves its explicit image; it does not perform an upgrade.
The [runtime remediation evidence](../security/RUNTIME_IMAGE_REMEDIATION_2026-09-08.md)
records both architecture scans and the pending binary/configuration, runtime,
inventory and final-image checks. The source default is not a release approval:
runtime publication remains held in Meta #52.
Replica counts are desired state:
```sh
@@ -0,0 +1,91 @@
# Integrity and performance source release — September 2026
Coordinated tracking: [Core #298](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298).
This source publication advances only the affected packages; it is not a signed
catalog publication, runtime-image release, or remote production rollout.
## Package set
| Package | Version |
| --- | --- |
| Core / developer meta-package | 0.1.46 |
| Addresses | 0.1.23 |
| Calendar | 0.1.24 |
| Campaign | 0.1.29 |
| Cases | 0.1.25 |
| Committee | 0.1.22 |
| Connectors | 0.1.27 |
| Dataflow | 0.1.25 |
| Datasources | 0.1.26 |
| Files | 0.1.27 |
| Forms Runtime | 0.1.22 |
| IDM | 0.1.26 |
| Mail | 0.1.28 |
| Reporting | 0.1.22 |
| Tickets | 0.1.23 |
Consumers of new Core helpers require Core 0.1.46 or later. Dataflow and
Datasources also require the matching Core WebUI contract. The release manifests,
immutable Git lock, and developer package describe this coordinated composition.
Unchanged packages retain their independent versions.
## Database and data integrity
The reviewed additive heads are Connectors `d2a4c6e8f0b1`, Datasources
`e2b8d4a0f6c3`, Files `a2b3c4d5e701`, and Mail `b5d6e7f8091a`.
Back up the target deployment and rehearse its normal upgrade before rollout.
Never downgrade away retained CSV originals without a separate recovery plan.
Files preserves historical duplicate copies; Mail leaves legacy maildrop identity
unset rather than guessing an account. A schema upgrade does not authorize POP3
retrieval/reconciliation, provider deletion, or message resending.
Development startup may automatically apply pending migrations after a watched
source change. Therefore, inspect actual Alembic heads and schema before assuming
a live development database is still at its pre-change state. A backup taken
after such an upgrade is a current-state recovery copy, not a pre-upgrade backup.
On 8 September, the local PostgreSQL development database already contained all
four heads. Schema and constraint inspection passed; recomputing the Files
identity backfill checked 7,385 rows with zero mismatches. A private current-state
database backup was created. Per-instance backup paths and row contents are not
published in this repository. The implementation's initial receipt claiming no
live migration occurred was incorrect: watched development-server restarts had
applied the migration files automatically. Restoring the backup into an isolated
PostgreSQL 16 cluster and running the normal upgrade preserved all 281 public
tables, 142,287 rows, and migration heads exactly. Both Datasources PostgreSQL
concurrency regressions passed. The test-only cluster was then stopped.
Release preparation additionally corrected Alembic's ConfigParser handling of
percent-escaped connection URLs, preserving the exact database URL. The static
migration auditor now recognizes the existing reviewed development-wrapper
aliases and peer filenames without executing migration code; historical
migrations are unchanged.
## Verification and boundaries
The implementation passed the required focused workspace gate, including 63
frontend build configurations and 230 browser conformance tests. Owning-module
regressions cover exact import/rollback evidence, authorization-before-pagination,
bounded recurrence and response processing, collision-safe attachment naming,
and stale asynchronous UI completion. English/German feature documentation stays
in each owning module; Core documents shared integrity contracts.
Mock-provider tests and local work-count measurements do not establish production
throughput or provider race behavior. Real S3/SMB/Seafile and POP3 acceptance,
representative load testing, and deployment authentication checks remain separate
operational validation. Source tags trigger package workflows; a pushed source
tag alone does not prove a registry artifact or signed catalog is published.
## Deutsch
Dieses koordinierte Quellrelease veröffentlicht nur die betroffenen Pakete.
Produktions-Images, signierter Modulkatalog und entfernte Produktivinstanzen
werden dadurch nicht ausgerollt. Die vier Migrationen bewahren bestehende Daten;
historische POP3-Zuordnungen werden nicht geraten. POP3 ist ein eigener
Import-Arbeitsablauf innerhalb von **Mail**, kein separat installierbares Modul.
Der Entwicklungsserver kann Migrationen beim automatischen Neustart nach einer
Quelländerung bereits anwenden. Tatsächliche Schema-Stände prüfen; eine danach
erstellte Sicherung enthält den aktuellen Stand und ist keine Sicherung vor dem
Upgrade. Externe Transportaktionen, erneutes Versenden und Löschungen werden
durch die Migration oder Quellveröffentlichung nicht ausgelöst.
+204 -2
View File
@@ -86,6 +86,16 @@ contract. The coordinated release synchronizes `peerDependencies` and
tag, then synchronizes each lockfile root from the final package metadata. A
distinct root package remains independent.
Every module referenced by Core's Git-based `package.release.json` must expose
its WebUI identity at the repository root, including matching peer requirements
and `webui/`-prefixed entry exports (also CSS subpaths). npm resolves Git
dependencies from the repository root, while the native-package workflow packs
`webui/`; success in one path does not verify the other. Run
`python tools/checks/check-webui-package-facades.py` after changing either
manifest or the release composition. The focused gate also runs this check.
Adding or correcting a facade in an already published repository requires a
new patch tag; never repair an existing immutable tag in place.
It builds one wheel and, where applicable, one npm tarball. The workflow records
the source tag, source commit, filename, size, and SHA-256 in
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
@@ -179,8 +189,13 @@ than invoking `pip`, `npm`, or Git on the target host.
## Public module directory
`tools/release/publish-release-catalog.sh` resolves the selected package set and
registry lock before it creates a catalog. Catalog entries are synthesized from
For an operator-reviewed full publication, use
`tools/release/release-catalog.py full-registry` followed by the same tool's
`publish-candidate` command. Resolve the package set and registry lock first;
the older direct-write shell wrapper is not the strict candidate publication
path. See [Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
for the private host runtime, exact artifact checks, and legacy keyring transition.
Catalog entries are synthesized from
the exact tagged module manifests, never from a hand-maintained module list or
the current workspace. Each entry binds its Python wheel and optional WebUI
tarball to the registry URL, filename, size, SHA-256, package identity, source
@@ -248,11 +263,198 @@ python tools/release/generate-developer-meta-package.py
python tools/release/generate-developer-meta-package.py --check
```
The direct generator is a development synchronization tool, not a receipt-gated
release executor. For release preparation, use the guarded out-of-run stage below.
`push-release-tag.sh` performs this synchronization before release commits and
tags. The meta-package is for editable/developer setup and composition tests. It
does not enable modules, apply migrations, provision services, or establish
backup and recovery evidence.
### Shared source-tag contract and Meta composition
The shared version collector names Meta's real
`packages/govoplan-meta/pyproject.toml` separately from root `pyproject.toml`.
Only the registered `govoplan` system/meta repository with nested project name
`govoplan` receives this contract. Missing, unknown, or misidentified metadata
does not become a versionless exception. Version alignment compares the complete
nested file with the canonical operator-tool generator output: its version must
match Core, and dependencies and `full` composition must match the reviewed
requirements and workspace package versions. Validation never executes a
generator from a selected checkout. The shared trusted manifest checker can
load reviewed application manifests; these checks are not a code sandbox.
Meta's complete generated file is recognized by shared version-mutation discovery,
but the generic durable version executor deliberately cannot write it. A durable
run freezes the release console's own Meta checkout as trusted runtime code;
changing it in place would invalidate that run. The planner therefore places Meta
after Core and exposes only non-executable support preparation/publication steps,
not misleading automatic Meta version, commit, tag, or push actions. A missing or
different Core target produces an actionable preparation prerequisite.
Prepare Core and the intended module inputs first, commit their reviewed state,
then stop active durable runs for the target workspace. Use trusted operator tools
against a separate registered, private source checkout, never the running operator
Meta directory. Preview outside any selected source checkout, for example:
```sh
python tools/release/prepare-developer-meta-package.py \
--workspace /private/release-workspace --target-version X.Y.Z \
> /private/operator/meta-preview.json
python tools/release/prepare-developer-meta-package.py \
--workspace /private/release-workspace --target-version X.Y.Z \
--receipt /private/operator/meta-preview.json --apply --confirm-out-of-run
```
The explicit confirmation attests that no durable run is active for that target
workspace; the helper does not discover or stop other processes. Preview/apply
requires registered clean main sources, matching origins and live-main ancestry,
Core already aligned at the target, the exact nested identity, and no existing or
unverifiable target Meta tag. Frozen receipts cover source HEADs/filesystem
identities, release requirements, every discovered registered full-composition
pyproject, the trusted generator snapshot, and the resulting full-file hash.
Inputs are limited to 128 selected files, 2 MiB per file and 16 MiB aggregate.
The canonical generator renders copied bounded data in a temporary directory;
no generator from the selected checkout executes. Changed receipts block before
the file effect. Apply writes only `packages/govoplan-meta/pyproject.toml`, then
rechecks the other sources and exact output. A write or post-check failure that
may have changed the file reports `needs-reconciliation` and leaves that bounded
delta for explicit review; it never retries, rolls back, commits or publishes.
Review the complete generated composition and manually commit the resulting file.
Complete matching Core publication before guarded Meta source tagging/publication,
then start a fresh durable run from reviewed, clean, published operator tooling.
Hot self-updating durable Meta release execution remains explicitly unsupported;
this out-of-run preparation is the existing developer-meta support contract.
For every `tag_repositories` batch, strict checks apply to every selected
repository before any tag creation, fetch, or push: registered checkout and
origin/push URL, a clean `main` tracking `origin/main`, live remote-main ancestry,
exact frozen HEADs, and immutable annotated local/remote tag objects. Git metadata
must remain inside the operator's trusted workspace. Missing local knowledge of
live remote main is a blocker; fetch and review it separately. Unknown repositories
and non-registered remote aliases fail closed. If selected, Meta runs last.
For Meta only, the matching annotated Core release must exist before its effect; Core may be
an earlier selected repository, or an already tagged dependency. Publication
requires that Core's exact tag and main commit are already remote.
Non-Meta selections do not acquire Meta's composition or Core-tag prerequisite.
Local module-candidate tags still work before Core's final release lock or tag.
The existing Core WebUI bundle gate still applies to module publication and
batches selecting Core: relevant Core release-package and release-lock inputs
must be operator-owned regular files, at most 16 MiB each, and their identities
and content hashes are frozen before preflight and rechecked before every effect.
When Core is unselected, this does not require its checkout to be clean or tagged;
reviewed pending composition inputs retain their previous meaning. Backend-only
selections never read irrelevant Core WebUI files.
Before even read-only Git commands, source ancestry must be owned by root or the
current operator and must not be group/world writable. A sticky shared ancestor
such as `/tmp` is permitted only above an owned, protected child; the workspace
and checkouts receive no writable-directory exception. The current operator must
own source inputs and actual Git/worktree/common metadata, which must be regular
files/directories, non-symlinked, and non-writable by other users. Metadata walks
are bounded to 500,000 entries and 128 levels, and tracked inputs to 100,000 paths
and 16 MiB of listing text. Read-only Git targets, object alternates/grafts and
hidden/sparse/unmerged index entries are blocked. Frozen receipts include actual
checkout/Git directory paths, devices, inodes, owners and modes, so replacing Git
metadata with the same HEAD is still detected. All selected version/composition
inputs must be tracked, including root and WebUI package/lock metadata, discovered
module manifests and package initializers, and Meta's nested package and release
requirements; ignored working files cannot supply declarations absent from a tag.
No chmod, ownership repair or
global Git trust change is performed. A shared writable workspace must first be
recreated or reviewed in the operator's protected release area by an explicitly
authorized preparation workflow.
Preview is read-only. Local-tag mode creates only pinned annotated tags (or
retrieves an identical published annotation); it does not publish main or tags.
Publish mode atomically pushes the frozen main commit and annotation object,
without force, retagging, fallback, or automatic retry. The complete source
receipt is rechecked before every effect and afterward; remote main and the
exact annotated tag must both match, not merely the Git exit status. Changes
after preflight stop the remaining batch. Atomicity is per repository, not
across repositories: earlier successful publications and a newly created local
tag can remain after a later failure. Inspect reported receipts and obtain a new
review before retrying; do not move immutable tags.
Whole-batch revalidation deliberately repeats source and live-remote checks around
each repository effect; the number of checks can grow quadratically with batch
size. Plan release time accordingly rather than bypassing trust checks. The
shared internal preflight is read-only and exposes no legacy mutation path.
The fixture suite covers Meta and non-Meta preview/local-tag/
publication using temporary local bare remotes, including stale compositions,
unsafe origins, divergent branches, damaged tag identity, changed receipts and
false publication success. This is local tooling evidence, not a real release
publication or production permission check.
Deutsch: Die gemeinsamen Helfer erkennen ausschließlich das registrierte
Meta-Repository mit dem echten Paket `packages/govoplan-meta/pyproject.toml`
(Projektname `govoplan`). Version und vollständige Zusammensetzung müssen dem
kanonischen Generator, Core und den geprüften Anforderungen entsprechen; der
Generator stammt niemals aus dem ausgewählten Checkout. Der gemeinsame
Manifestprüfer kann geprüften Anwendungscode laden und ist keine Sandbox.
Die gemeinsame Änderungsplanung erkennt die vollständig generierte Paketdatei,
aber der dauerhafte Versionsausführer darf Meta nicht selbst verändern: sein
eingefrorener Lauf bindet den Meta-Checkout als vertrauenswürdigen Programmstand.
Meta erscheint deshalb nach Core ausschließlich mit nicht automatisch ausführbaren
Vorbereitungs-/Veröffentlichungsschritten. Zuerst Core und Modulquellen vorbereiten
und geprüft committen; bei abweichender Core-Zielversion nennt der Plan diese
Voraussetzung ausdrücklich. Aktive dauerhafte Läufe des Ziel-Workspaces beenden.
Mit `prepare-developer-meta-package.py` zunächst eine Vorschau außerhalb der
Quell-Checkouts speichern, dann deren JSON über `--receipt` zusammen mit `--apply`
und `--confirm-out-of-run` bestätigen. Das Ziel muss ein separater registrierter
privater Checkout sein, niemals das laufende Operator-Meta. Die Bestätigung ist
eine Betreibererklärung; der Helfer sucht oder beendet keine fremden Prozesse.
Quell-HEADs, Pfadidentitäten, Anforderungen, alle registrierten vollständigen
Paket-Eingaben, der vertrauenswürdige Generator und der vollständige Ausgabehash
werden eingefroren. Es gelten höchstens 128 Quelldateien, 2 MiB je Datei und
16 MiB insgesamt. Core muss bereits vollständig zur Zielversion passen; vorhandene
oder nicht verifizierbare Meta-Zieltags sperren die Vorbereitung. Geänderte
Nachweise stoppen vor dem Schreiben. Ausschließlich die verschachtelte Paketdatei
wird vollständig generiert und danach geprüft; ein Fehler nach dem Schreiben
meldet `needs-reconciliation` und erfordert die manuelle Prüfung dieser begrenzten
Änderung, ohne automatisches Zurücksetzen. Kein automatischer
Commit, Push oder Wiederholungsversuch findet statt. Zusammensetzung prüfen,
manuell committen, Core zuerst veröffentlichen, dann die geschützte Meta-Tag-Route
verwenden und einen neuen dauerhaften Lauf starten. Eine Selbstaktualisierung
des aktiven dauerhaften Meta-Laufs bleibt ausdrücklich nicht unterstützt.
Für jeden Tag-Stapel, auch ohne Meta, gelten
Vertrauens-, Origin-, saubere Main- und Live-Abstammungsprüfungen für die gesamte
Auswahl vor jeder Änderung. Unbekannte Repositories und nicht registrierte
Remote-Aliase sind gesperrt. Nur bei ausgewähltem Meta gelten zusätzlich dessen
Zusammensetzungsprüfung und der passende annotierte Core-Tag als Voraussetzung;
Meta folgt zuletzt. Für Metas Veröffentlichung müssen Core-Tag und Main-Commit
bereits auf dem Remote vorliegen. Lokale Modul-Kandidatentags bleiben vor Cores
abschließendem Release-Lock und Tag möglich. Die vorhandene Core-WebUI-Bundleprüfung
bleibt bei Modulveröffentlichung und Core-Auswahl erhalten. Relevante Core-Paket-
und Lockdateien müssen eigene reguläre Dateien mit höchstens je 16 MiB sein;
Identität und Inhaltshash werden eingefroren und vor jeder Aktion erneut geprüft.
Nicht ausgewähltes Core benötigt dafür weder einen sauberen Checkout noch einen
Tag. Reine Backend-Auswahlen lesen keine irrelevanten Core-WebUI-Dateien.
Vor Git-Aufrufen werden Eigentümer, Schreibrechte, sichere
Pfadabstammung und echte Git-/Worktree-Metadaten geprüft; veränderbare gemeinsame
Verzeichnisse, fremde Eigentümer, Alternates, Grafts und versteckte Indexeinträge
sind gesperrt. Ein Sticky-Bit-Vorfahr wie `/tmp` ist nur oberhalb eines eigenen
geschützten Unterverzeichnisses zulässig. Es erfolgen weder Rechtereparaturen
noch globale Git-Vertrauensänderungen. Ausgewählte Versions- und Zusammensetzungs-
dateien müssen versioniert sein: Paket-/Lockdateien, Modulmanifeste und
Paketinitialisierer sowie Metas verschachteltes Paket und Release-Anforderungen.
Ignorierte Arbeitsdateien dürfen keine vom Tag abweichenden Angaben liefern.
Die Vorschau schreibt nichts, lokale Tags veröffentlichen nichts,
und die Veröffentlichung überträgt Main und den exakten annotierten Tag atomar
je Repository. Unmittelbar vor und nach den Aktionen werden die eingefrorenen
Quellnachweise erneut geprüft, einschließlich entferntem Main und Tag-Objekt.
Bei Änderungen oder Fehlern stoppt der Rest des Stapels ohne automatischen
Wiederholungsversuch. Frühere Veröffentlichungen und neu erzeugte lokale Tags
können bestehen bleiben: vor einem neuen Versuch Nachweise prüfen und erneut
freigeben, niemals unveränderliche Tags verschieben. Die vollständigen Quell- und
Live-Remote-Prüfungen werden um jede Aktion wiederholt; bei großen Stapeln kann
deren Anzahl quadratisch wachsen. Diese konservativen Prüfkosten gehören zur
Release-Planung. Der interne Vorprüfer ist ausschließlich lesend und besitzt
keinen alten Änderungspfad. Tests für Auswahlen mit und ohne Meta verwenden
nur temporäre lokale Remotes und ersetzen keine echte Veröffentlichungsprüfung.
If the tag-triggered developer meta-package job fails before publication, rerun
`publish-developer-meta-package.yml` with the existing protected version. The
manual path validates that tag against `main`, checks out its exact commit, and
+76 -4
View File
@@ -58,6 +58,45 @@ checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace
before releasing.
For a host with a confirmed IPv6 connection timeout, set
`GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` only for the release-tool invocation.
When unset, the original SSH command is preserved, including the trusted
operator's per-host `AddressFamily` configuration (normally `any`). Explicit
values accepted by the shared source/tag Git helper are exactly `any`, `inet`
(IPv4 only), and `inet6` (IPv6 only). Empty, misspelled, whitespace-padded, or
injected values fail before Git starts. The selector only adds the corresponding
fixed SSH `AddressFamily` option: it does not change DNS, host-key verification,
the registered remote, authentication, `BatchMode=yes`, or `ConnectTimeout=8`.
Arbitrary `GIT_SSH_COMMAND` overrides remain ignored. For example, start a
single local console invocation with:
```sh
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet \
./.venv/bin/python tools/release/release-console.py
```
The same process-scoped setting applies to canonical source/tag readbacks and
registry-candidate source verification. Under Flatpak, pass it explicitly to
the host invocation with `flatpak-spawn --host /usr/bin/env
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet ...`. It is not a global SSH setting
and does not affect the website publisher's separate transport sanitizer, npm,
or HTTP downloads. An IPv4-only setting cannot reach IPv6-only hosts; omit it
or use `any` when the diagnosed restriction no longer applies.
Deutsch: Bei einem bestätigten IPv6-Verbindungs-Timeout kann für genau einen
Release-Werkzeugaufruf `GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` gesetzt werden.
Ohne diese Variable bleibt der bisherige SSH-Befehl einschließlich der
vertrauenswürdigen Host-Konfiguration unverändert (normalerweise `any`).
Explizit zulässig sind ausschließlich `any`, `inet` (nur IPv4) und `inet6`
(nur IPv6). Andere oder leere Werte werden vor dem Git-Aufruf abgewiesen.
DNS, Hostschlüsselprüfung, registrierte Quelladresse, Authentifizierung und
Zeitlimit bleiben unverändert; frei vorgegebene SSH-Befehle bleiben gesperrt.
Unter Flatpak die Variable ausdrücklich an den Host-Aufruf übergeben. Die
Auswahl gilt für den gemeinsamen Git-Helfer der Quell-/Tag-Prüfungen, nicht
für den separaten Website-Publisher, npm oder HTTP-Downloads. Sie ändert keine
globale Konfiguration. Nach Behebung des Netzwerkproblems die Variable
weglassen oder auf `any` setzen; IPv4-only erreicht keine IPv6-only-Ziele.
The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
@@ -221,6 +260,9 @@ Repository capabilities are frozen into each plan unit (`python-package`,
`core-release-bundle`, and the universal `git-source`) and determine which
steps appear. Internally aligned version changes are rendered deterministically
from recognized TOML, JSON, lockfile, manifest, and package declarations.
The manifest may use a literal version or a top-level literal `MODULE_VERSION`;
the latter is updated without rewriting independently versioned interfaces.
Computed or missing version declarations fail before any metadata is written.
Pre-existing dirty worktrees remain visible but have no commit executor; the
console never absorbs unrelated operator changes.
@@ -232,6 +274,17 @@ runs a receipt-bound alignment gate before exposing any atomic branch/tag push.
A failed step stops later steps while preserving prior receipts for explicit
retry or reconciliation.
Local module candidate creation deliberately does not require those candidates
to be resolved already in Core's release lock: their annotated tags are inputs
to the next lock-generation step. The internal tag helper applies this ordering
only when no Core repository is selected and remote publication is disabled.
Module version/lock consistency, manifest validity, clean/non-behind worktrees,
and local/remote tag immutability checks still apply. Core candidate tagging
continues to validate its own complete bundle, and every remote-publication
preview and execution requires the selected modules to match Core's release
input and resolved lock. A local candidate is therefore not publication
approval; a stale Core lock blocks publication without changing remote refs.
The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the
@@ -301,8 +354,17 @@ such as `0.2.0` or `0.2.0-alpha1`, but requires the first three version numbers
to move forward.
Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in
the repository push confirmation field.
shows the selected repository push commands, but generic push, sync and prepare
mutation endpoints are disabled: they require a separate durable, receipt-bound
maintenance workflow and cannot be enabled by typing a confirmation. Source
release branch/tag publication uses the durable release-run steps described
above; do not route ordinary dirty worktrees through the legacy all-repository
stage/commit/tag helper.
The [headless devkit release commands](DEVKIT_RELEASE.md) invoke this same
application in-process without starting a server. They expose selective planning,
bounded status/history, create/show/preview/execute and explicit recovery while
retaining the same request IDs, confirmations, source bindings and receipts.
The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
@@ -503,6 +565,15 @@ tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`.
For a full registry-backed release, first build a fresh private candidate using
`release-catalog.py full-registry`. Pass `--selected-repository` for newly
released HEAD-bound units, not every unchanged package in the full profile.
The command independently checks all full-profile registry bytes and annotated
tag provenance, then feeds this same strict `publish-candidate` transaction.
It does not create Gitea runtime Releases or dispatch image builds. See
[Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
for the complete EN/DE workflow and the narrowly scoped legacy keyring transition.
Published channels are expected below the public catalog base URL:
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
@@ -515,8 +586,9 @@ updated catalog, and keep the published keyring healthy.
When a selected module exposes a WebUI package, its requested version must also
match Core's `webui/package.release.json` input and the resolved
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
catalog-candidate writer all enforce this composition boundary. Pins for modules
`package-lock.release.json` entry. The source-publication preflight, selective
plan, and catalog-candidate writer all enforce this composition boundary;
module-only local candidate tags use the staged order described above. Pins for modules
that are not part of the selective release remain unchanged.
Release integration also enforces repository and composition version alignment
@@ -0,0 +1,91 @@
# WebUI release dependency installer retries
## English
This operational note covers
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
and the exit-status repair tracked in
[Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54).
It applies to release administrators using the legacy runtime WebUI installer;
there are no new application settings, permissions, or end-user workflows.
Each retried npm install or Git clone has at most three attempts. The installer
waits 10 seconds after the first failure and 20 seconds after the second, and
continues immediately after success. If all attempts fail, it exits with the
last command's nonzero status. Its `set -e` execution stops before subsequent
installation stages; callers using `set -e` also stop before subsequent work.
Previously, the retry helper could report success after three failures because
it captured the status of a completed `if` statement instead of the command.
On exhaustion, inspect the npm or Git error and correct the reported cause
before rerunning the installation. The temporary dependency workspace is
removed on exit. Earlier changes to `package.json`, removal of `package-lock.json`,
cache cleaning, and completed dependency installations are not rolled back;
prepare a fresh disposable release workspace when a clean retry is required.
The repair preserves the existing retry count, backoff, cache behavior, and
peer-resolution flags. It does not lift the runtime publication hold tracked in
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
Review the historical `--legacy-peer-deps` workaround separately before lifting
that hold. Strict disposable Git-release and signed catalog verification do not
use this installer; strict release verification must not bypass peer checks.
See [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md) for release context.
Run the isolated regression suite from the meta repository:
```sh
python3 -m unittest -v tests.test_webui_release_dependency_retries
```
The suite executes the actual Bash installer and a caller using `set -e`, with
local npm, Git, Node, and sleep stubs. It covers success on attempts one, two, and
three, final failure status, backoff, and termination at each retry call site.
It performs no network access, real waiting, or changes to the real npm cache.
It checks shell control flow, not package resolution or runtime publication.
## Deutsch
Dieser Betriebshinweis beschreibt
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
und die unter [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54)
erfasste Korrektur des Rückgabestatus. Er richtet sich an Release-Administratoren,
die den bisherigen WebUI-Installer für Laufzeit-Releases verwenden. Neue
Anwendungseinstellungen, Berechtigungen oder Endanwenderabläufe entstehen nicht.
Jede wiederholte npm-Installation und jeder Git-Klon erhält höchstens drei
Versuche. Nach dem ersten Fehlschlag wartet der Installer 10 Sekunden, nach dem
zweiten 20 Sekunden; nach einem Erfolg fährt er sofort fort. Scheitern alle
Versuche, endet er mit dem letzten von null verschiedenen Rückgabestatus.
Durch `set -e` werden nachfolgende Installationsschritte nicht ausgeführt;
auch aufrufende Skripte mit `set -e` brechen vor ihren nächsten Schritten ab.
Bisher konnte die Hilfsfunktion nach drei Fehlschlägen Erfolg melden, weil sie
den Status der abgeschlossenen `if`-Anweisung statt des Befehls übernahm.
Prüfen Sie nach dem Abbruch die npm- oder Git-Fehlermeldung und beheben Sie deren
Ursache vor einem erneuten Installationslauf. Das temporäre Verzeichnis für
Abhängigkeiten wird beim Beenden entfernt. Vorherige Änderungen an `package.json`,
das Entfernen von `package-lock.json`, die Cache-Bereinigung und abgeschlossene
Installationen werden nicht zurückgerollt. Bereiten Sie bei Bedarf einen neuen
temporären Release-Arbeitsbereich für einen sauberen Wiederholungslauf vor.
Die Korrektur erhält Anzahl und Wartezeiten der Versuche, Cache-Verhalten und
Optionen zur Peer-Auflösung. Die Sperre für Laufzeitveröffentlichungen aus
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52) bleibt bestehen.
Der bisherige Einsatz von `--legacy-peer-deps` muss vor ihrer Aufhebung gesondert
geprüft werden. Die strenge Git-Release-Prüfung in einem temporären Arbeitsbereich
und die Prüfung signierter Kataloge verwenden diesen Installer nicht; die strenge
Release-Prüfung darf Peer-Prüfungen nicht umgehen. Weitere Zusammenhänge erläutert
[Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md).
Führen Sie die isolierten Regressionstests im Meta-Repository aus:
```sh
python3 -m unittest -v tests.test_webui_release_dependency_retries
```
Die Tests führen den tatsächlichen Bash-Installer und ein aufrufendes Skript mit
`set -e` aus. Lokale Testprogramme ersetzen npm, Git, Node und sleep. Geprüft werden
Erfolge im ersten, zweiten und dritten Versuch, der letzte Fehlerstatus,
Warteintervalle und der Abbruch an jeder Aufrufstelle. Es gibt keine
Netzwerkzugriffe, echten Wartezeiten oder Änderungen am tatsächlichen npm-Cache.
Die Tests prüfen den Shell-Ablauf, nicht die Paketauflösung oder Veröffentlichung.
+164
View File
@@ -0,0 +1,164 @@
# Product-wide UI review program
The [central Gitea epic](https://git.add-ideas.de/GovOPlaN/govoplan/issues/56)
coordinates the review; its linked module issues are the canonical backlog,
progress and evidence log. The [issue inventory](ui-review-issue-inventory.json)
is a discovery/link snapshot, **not a second progress tracker**. All reviews were
initialized pending. Implementing a shared component or moving a help icon does
not by itself complete a module review.
The visual and interaction rules live in Core's
[UI design principles](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/UI_DESIGN_PRINCIPLES.md).
This document defines how to apply and verify those principles across repositories;
it does not fork their definitions. The usual [Gitea issue contract](GITEA_ISSUES.md)
still applies. At the user's request, this cross-product program is coordinated in
Meta; shared primitives and their implementation remain owned by Core.
## Scope and initial inventory
The September 2026 source/catalog baseline contains **77 review tracks**:
- **73 implemented scopes:** Core/shared shell and 72 manifest-backed modules.
- **4 registered placeholders:** Appointments, Ledger, XÖV and XTA/OSCI. They have
README-only repositories at initialization. Their separate readiness tracks
remain pending for future interfaces; they are not counted as implemented or
reviewed modules.
Meta and the public website are not business-module review scopes. Module IDs
come from actual manifests, not repository-name guesses: for example,
`govoplan-campaign` owns `campaigns` and `govoplan-dist-lists` owns `dist_lists`.
The inventory is derived from `repositories.json`, trusted module manifests and
the owning repository's source files. Changes to the catalog must be reconciled
with the central epic rather than silently dropping a module.
A module with no standalone WebUI is still in scope. Review its configuration
and administrator experience, contributed operator/public surfaces, widgets,
documentation, permissions, errors and interactions exposed through other
modules. A missing standalone page is not evidence of conformance. A reviewer
may mark a particular principle not applicable only with a concrete explanation
and source/runtime evidence. A placeholder likewise needs an explicit future-UI
gate, not a fabricated page inventory or an automatic green review.
## Work order and ownership
Prioritize user-visible defects, usability and consistent interactions before
broader features. Start with Core/shared-shell contracts and Campaign's complete
workflow, then Files, Mail, Templates, Notifications, Docs/Help, Dashboard, Quick
Access and Views. The remaining queue is ordered by observed defects, shared
dependencies and risk. This is a starting order, not a claim that those reviews
have started or finished; the central issue records the current focus.
For Campaign, use a compact, read-only campaign settings overview with explicit,
scoped edit dialogs. Large recipient or attachment grids may use the UI-02 bulk
editing exception: clearly entered edit mode, explicit Save and Cancel, dirty
navigation protection, and no silent loss or accidental save. This direction
must be implemented and verified in the Campaign review, not marked complete by
the program bootstrap.
Fix a repeated problem in the owning shared primitive or semantic contract first,
then migrate consumers and verify representative actual-module fixtures. Keep
module business behavior in its own repository and preserve optional module
boundaries. Do not replace concrete user evidence with a generic fixture alone.
Parallel reviewers may own different modules, but they must coordinate shared
Core files and browser fixtures instead of racing the same resources.
## Per-module review workflow
1. **Inventory.** Verify and extend the issue's source-derived seed: navigation,
pages, nested routes, dialogs, labels/forms, tables/trees, admin/system/tenant/
user settings, public/operator interfaces, widgets and optional-module
contributions. Follow actual module entrypoints and runtime contributions;
filename heuristics and manifest routes are a starting point, not an exhaustive
runtime audit. Note roles, permission boundaries and installed optional modules.
2. **Observe.** Exercise realistic narrow and wide viewports, German and English,
keyboard navigation and focus, empty/loading/error/success states, clean/dirty
edit modes, reload, navigation away and return, and restoration of personal
preferences. Include realistic row counts, long labels, horizontal overflow,
pagination and two-way column resizing with fixed columns between text fields.
3. **Record.** Keep a findings/TODO/done ledger in the module issue. Every finding
names the surface, reproduction, expected principle, impact, owner and linked
implementation/evidence. A source inventory, planned fix or green generic
test is not a completed finding. State product decisions and manual checks
explicitly with enough context to resolve them.
4. **Implement.** Prefer reusable shared primitives and action/page contracts.
Keep display mode readable and editing intentional. Preserve authorization,
server validation, save/cancel behavior and data integrity. Never send mail,
delete records, change live configuration or trigger other irreversible effects
merely to obtain UI evidence; use authorized fixtures or a safe test context.
5. **Verify.** Run proportionate unit, structure and actual-module browser tests.
Record commands, results, source revision and safe runtime evidence. Update the
owning module's manifest-driven EN/DE documentation for behavior changes; run
`tools/checks/check-manifest-shapes.py` and relevant cross-module checks.
6. **Conclude honestly.** Complete the principle matrix, evidence and manual
checks. Link unresolved follow-ups and blockers; do not close a supposedly
complete review while required work is still pending. Closing an issue requires
explicit reviewed scope and the applicable design-principle revision.
## Principle ledger and back-propagation
Use the stable IDs from Core in each issue. The initial revision covers UI-01
heading/label help placement; UI-02 display-first/scoped editing; UI-03 actions;
UI-04 table/card geometry; UI-05 loading/error/progress; UI-06 tree interaction;
UI-07 accessibility and German; UI-08 data integrity; and UI-09 propagation of
revised principles. Core remains the authoritative definition.
Each module maintains this matrix, initially entirely pending:
| Principle/revision | Applicable surfaces / justified N/A | Applied / remaining work | Evidence | Exception / owner / follow-up |
| --- | --- | --- | --- | --- |
| UI-01 … UI-09 | Pending inventory | Pending review | Not yet recorded | None approved |
Whenever a principle is added or changed:
1. Update Core's rule with its rationale, stable ID and revision/change reference.
2. List **all already-reviewed modules**, not just the current implementation's
consumers. Check whether the changed rule applies to each.
3. Record unchanged conformance with evidence, or reopen the review / create a
linked follow-up with owner, exact affected surfaces and required checks.
4. Update the central epic's propagation ledger. Previously reviewed modules with
outstanding applicable work are **follow-up required**, not silently green.
5. Resolve the propagation sweep only after every affected module has current
evidence or an explicit, owned and justified exception. Exceptions do not
silently change the shared rule.
Suggested central ledger:
| Principle change | Already-reviewed modules checked | Conformant evidence | Reopened / follow-up required | Exceptions / owner | Sweep state |
| --- | --- | --- | --- | --- | --- |
| Initial UI-01UI-09 baseline | None reviewed at initialization | None | All module reviews pending | None approved | Pending |
## Safe inventory/bootstrap automation
`tools/gitea/gitea-ui-review-program.py` derives the current scopes from the
catalog and source manifests, inventories routes and UI source entrypoints,
then deduplicates issues by a stable hidden marker and normalized exact title.
It includes closed issues in duplicate checks and never reopens, closes or
overwrites existing module issues. Ambiguous or unmanaged duplicates stop the
operation for review. Existing issue bodies, labels and review progress are
preserved. A failed POST is not automatically retried; a subsequent run checks
the marker again before deciding whether another create is needed.
Dry-run is the default. An authenticated dry-run performs only Gitea reads:
```sh
.venv/bin/python tools/gitea/gitea-ui-review-program.py \
--env-file /home/zemion/.config/gitea/gitea.env --epic 56
```
Creating missing review issues requires explicit `--apply`. Initial population
of the central epic's module-link block additionally requires
`--initialize-links`; it is accepted only for the untouched bootstrap placeholder.
Later runs must not rewrite checkboxes or human-maintained progress. Existing
complete links are verified read-only. The token is loaded through the shared
Gitea helpers and is never emitted in reports. A scoped `--ipv4` option works
around host-specific IPv6 connectivity while retaining HTTPS certificate and
hostname verification.
The tool emits a JSON result to stdout with source counts and issue links. The
checked-in inventory is a reviewed initialization snapshot of that result;
refreshing links does not authorize replacing live review status with the
snapshot. Test the automation offline with:
```sh
.venv/bin/python -m pytest tests/test_ui_review_program.py -q
```
+866
View File
@@ -0,0 +1,866 @@
{
"schema_version": 1,
"snapshot_purpose": "Issue discovery links; live Gitea issues own review state and evidence.",
"epic": {
"repository": "govoplan",
"number": 56,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan/issues/56"
},
"scope_count": 77,
"manifest_modules": 72,
"implemented_scopes": 73,
"catalogued_placeholders": 4,
"created": 77,
"missing": 0,
"issues": [
{
"scope_id": "core",
"name": "Core / shared shell",
"repository": "govoplan-core",
"kind": "core",
"ui_source_count": 111,
"number": 301,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/301",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "access",
"name": "Access",
"repository": "govoplan-access",
"kind": "manifest",
"ui_source_count": 18,
"number": 23,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/23",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "addresses",
"name": "Addresses",
"repository": "govoplan-addresses",
"kind": "manifest",
"ui_source_count": 1,
"number": 26,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/26",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "admin",
"name": "Admin",
"repository": "govoplan-admin",
"kind": "manifest",
"ui_source_count": 9,
"number": 11,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-admin/issues/11",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "approvals",
"name": "Approvals",
"repository": "govoplan-approvals",
"kind": "manifest",
"ui_source_count": 4,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "assets",
"name": "Assets",
"repository": "govoplan-assets",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-assets/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "audit",
"name": "Audit",
"repository": "govoplan-audit",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-audit/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "booking",
"name": "Booking",
"repository": "govoplan-booking",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-booking/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "calendar",
"name": "Calendar",
"repository": "govoplan-calendar",
"kind": "manifest",
"ui_source_count": 10,
"number": 26,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-calendar/issues/26",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "campaigns",
"name": "Campaigns",
"repository": "govoplan-campaign",
"kind": "manifest",
"ui_source_count": 60,
"number": 103,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "cases",
"name": "Cases",
"repository": "govoplan-cases",
"kind": "manifest",
"ui_source_count": 4,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-cases/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "certificates",
"name": "Certificates",
"repository": "govoplan-certificates",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-certificates/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "committee",
"name": "Committee",
"repository": "govoplan-committee",
"kind": "manifest",
"ui_source_count": 3,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "connectors",
"name": "Connectors",
"repository": "govoplan-connectors",
"kind": "manifest",
"ui_source_count": 3,
"number": 20,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/20",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "consultation",
"name": "Consultation",
"repository": "govoplan-consultation",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-consultation/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "contracts",
"name": "Contracts",
"repository": "govoplan-contracts",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-contracts/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dashboard",
"name": "Dashboard",
"repository": "govoplan-dashboard",
"kind": "manifest",
"ui_source_count": 5,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dashboard/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dataflow",
"name": "Dataflow",
"repository": "govoplan-dataflow",
"kind": "manifest",
"ui_source_count": 5,
"number": 24,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/24",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "datasources",
"name": "Datasources",
"repository": "govoplan-datasources",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "decisions",
"name": "Decisions",
"repository": "govoplan-decisions",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-decisions/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dist_lists",
"name": "Distribution Lists",
"repository": "govoplan-dist-lists",
"kind": "manifest",
"ui_source_count": 2,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "dms",
"name": "DMS",
"repository": "govoplan-dms",
"kind": "manifest",
"ui_source_count": 0,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dms/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "docs",
"name": "Docs",
"repository": "govoplan-docs",
"kind": "manifest",
"ui_source_count": 2,
"number": 23,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/23",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "encryption",
"name": "Encryption",
"repository": "govoplan-encryption",
"kind": "manifest",
"ui_source_count": 1,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "erp",
"name": "ERP",
"repository": "govoplan-erp",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-erp/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "evaluation",
"name": "Evaluation",
"repository": "govoplan-evaluation",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-evaluation/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "facilities",
"name": "Facilities",
"repository": "govoplan-facilities",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-facilities/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "files",
"name": "Files",
"repository": "govoplan-files",
"kind": "manifest",
"ui_source_count": 8,
"number": 48,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/48",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "fit_connect",
"name": "FIT-Connect",
"repository": "govoplan-fit-connect",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-fit-connect/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "forms",
"name": "Forms",
"repository": "govoplan-forms",
"kind": "manifest",
"ui_source_count": 2,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "forms_runtime",
"name": "Forms Runtime",
"repository": "govoplan-forms-runtime",
"kind": "manifest",
"ui_source_count": 6,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "grants",
"name": "Grants",
"repository": "govoplan-grants",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-grants/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "helpdesk",
"name": "Helpdesk",
"repository": "govoplan-helpdesk",
"kind": "manifest",
"ui_source_count": 1,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-helpdesk/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "identity",
"name": "Identity",
"repository": "govoplan-identity",
"kind": "manifest",
"ui_source_count": 1,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "identity_trust",
"name": "Identity Trust",
"repository": "govoplan-identity-trust",
"kind": "manifest",
"ui_source_count": 1,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "idm",
"name": "IDM",
"repository": "govoplan-idm",
"kind": "manifest",
"ui_source_count": 3,
"number": 14,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-idm/issues/14",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "inspections",
"name": "Inspections",
"repository": "govoplan-inspections",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-inspections/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "learning",
"name": "Learning",
"repository": "govoplan-learning",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-learning/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "mail",
"name": "Mail",
"repository": "govoplan-mail",
"kind": "manifest",
"ui_source_count": 6,
"number": 25,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/25",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "mandates",
"name": "Mandates",
"repository": "govoplan-mandates",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mandates/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "notifications",
"name": "Notifications",
"repository": "govoplan-notifications",
"kind": "manifest",
"ui_source_count": 3,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "ops",
"name": "Ops",
"repository": "govoplan-ops",
"kind": "manifest",
"ui_source_count": 2,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ops/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "organizations",
"name": "Organizations",
"repository": "govoplan-organizations",
"kind": "manifest",
"ui_source_count": 4,
"number": 9,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-organizations/issues/9",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "parties",
"name": "Parties",
"repository": "govoplan-parties",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-parties/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "payments",
"name": "Payments",
"repository": "govoplan-payments",
"kind": "manifest",
"ui_source_count": 3,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-payments/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "permits",
"name": "Permits",
"repository": "govoplan-permits",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-permits/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "policy",
"name": "Policy",
"repository": "govoplan-policy",
"kind": "manifest",
"ui_source_count": 3,
"number": 14,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-policy/issues/14",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "poll",
"name": "Poll",
"repository": "govoplan-poll",
"kind": "manifest",
"ui_source_count": 0,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "portal",
"name": "Portal",
"repository": "govoplan-portal",
"kind": "manifest",
"ui_source_count": 2,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-portal/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "postbox",
"name": "Postbox",
"repository": "govoplan-postbox",
"kind": "manifest",
"ui_source_count": 4,
"number": 29,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-postbox/issues/29",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "procurement",
"name": "Procurement",
"repository": "govoplan-procurement",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-procurement/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "projects",
"name": "Projects",
"repository": "govoplan-projects",
"kind": "manifest",
"ui_source_count": 1,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-projects/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "quick_access",
"name": "Quick Access",
"repository": "govoplan-quick-access",
"kind": "manifest",
"ui_source_count": 2,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-quick-access/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "records",
"name": "Records",
"repository": "govoplan-records",
"kind": "manifest",
"ui_source_count": 4,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-records/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "reporting",
"name": "Reporting",
"repository": "govoplan-reporting",
"kind": "manifest",
"ui_source_count": 3,
"number": 11,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/11",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "resources",
"name": "Resources",
"repository": "govoplan-resources",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-resources/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "rest",
"name": "REST Connector",
"repository": "govoplan-rest",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-rest/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "risk_compliance",
"name": "Risk Compliance",
"repository": "govoplan-risk-compliance",
"kind": "manifest",
"ui_source_count": 1,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "scheduling",
"name": "Scheduling",
"repository": "govoplan-scheduling",
"kind": "manifest",
"ui_source_count": 4,
"number": 10,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-scheduling/issues/10",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "search",
"name": "Search",
"repository": "govoplan-search",
"kind": "manifest",
"ui_source_count": 4,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "services",
"name": "Services",
"repository": "govoplan-services",
"kind": "manifest",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-services/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "soap",
"name": "SOAP Connector",
"repository": "govoplan-soap",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-soap/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tasks",
"name": "Tasks",
"repository": "govoplan-tasks",
"kind": "manifest",
"ui_source_count": 3,
"number": 5,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tasks/issues/5",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "templates",
"name": "Templates",
"repository": "govoplan-templates",
"kind": "manifest",
"ui_source_count": 1,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tenancy",
"name": "Tenancy",
"repository": "govoplan-tenancy",
"kind": "manifest",
"ui_source_count": 2,
"number": 7,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tenancy/issues/7",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "tickets",
"name": "Tickets",
"repository": "govoplan-tickets",
"kind": "manifest",
"ui_source_count": 1,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tickets/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "transparency",
"name": "Transparency",
"repository": "govoplan-transparency",
"kind": "manifest",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-transparency/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "views",
"name": "Views",
"repository": "govoplan-views",
"kind": "manifest",
"ui_source_count": 3,
"number": 6,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-views/issues/6",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "voting",
"name": "Voting",
"repository": "govoplan-voting",
"kind": "manifest",
"ui_source_count": 2,
"number": 8,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-voting/issues/8",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "wiki",
"name": "Wiki",
"repository": "govoplan-wiki",
"kind": "manifest",
"ui_source_count": 1,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-wiki/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "workflow",
"name": "Workflow",
"repository": "govoplan-workflow",
"kind": "manifest",
"ui_source_count": 6,
"number": 17,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow/issues/17",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "workflow_engine",
"name": "Workflow Engine",
"repository": "govoplan-workflow-engine",
"kind": "manifest",
"ui_source_count": 0,
"number": 4,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/4",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "xrechnung",
"name": "XRechnung",
"repository": "govoplan-xrechnung",
"kind": "manifest",
"ui_source_count": 0,
"number": 3,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/3",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-appointments",
"name": "Appointments",
"repository": "govoplan-appointments",
"kind": "placeholder",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-appointments/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-ledger",
"name": "Ledger",
"repository": "govoplan-ledger",
"kind": "placeholder",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ledger/issues/1",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-xoev",
"name": "Xoev",
"repository": "govoplan-xoev",
"kind": "placeholder",
"ui_source_count": 0,
"number": 2,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xoev/issues/2",
"state_at_verification": "open",
"operation": "created"
},
{
"scope_id": "catalog:govoplan-xta-osci",
"name": "Xta Osci",
"repository": "govoplan-xta-osci",
"kind": "placeholder",
"ui_source_count": 0,
"number": 1,
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xta-osci/issues/1",
"state_at_verification": "open",
"operation": "created"
}
],
"snapshot_date": "2026-09-08",
"initial_review_status": "pending for every scope; this snapshot does not track subsequent issue progress"
}
+80
View File
@@ -0,0 +1,80 @@
# GovOPlaN 0.1.45 — usability, reliability and security hardening
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
The exact independently versioned composition is recorded in
`packages/govoplan-meta/pyproject.toml`; unchanged modules retain their versions.
This source release does not by itself establish a deployed or independently
approved production environment. Package, signed catalog and runtime publication
results are recorded separately in the coordination issue.
## Runtime publication hold
The [runtime image audit](../security/RUNTIME_IMAGE_AUDIT_2026-09-08.md) completed
eleven registry-only amd64 scans, but found unresolved vulnerabilities and
inventory gaps. Runtime publication remains held separately from this source
release. Patch-only image updates are insufficient; maintained minor-line
changes, narrowly evidenced finding decisions, arm64/final-layer scans and
deployment checks remain necessary. No audited candidate was automatically
adopted and no image was executed during those scans.
The remaining gates are tracked in
[GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
## Included changes
- Shared page/action placement, reusable navigation grouping/editing, table and
dialog sizing, field alignment, multi-select filters and predictable tree
selection. Files, Mail, Search, Notifications and domain pages use the same
contracts, with browser regression coverage.
- Campaign draft saving and independent Mail/ZIP-policy repair, persistent and
bulk message review, clearer delivery eligibility, bounded configurable
synchronous delivery, guarded workerless recovery, lightweight SMTP/IMAP
progress, reused IMAP connections and recipient-complete reporting.
- Files archive staging/reuse, unpacking previously uploaded archives, numeric
progress and bounded traversal. Optional native archive acceleration retains
the same validation rules; portable fallbacks remain available.
- Mail credential references and IMAP folder-name decoding; help topics can be
found by area and tags without expanding every occurrence of the same topic.
- Authentication provenance/scope and browser-cache hardening, patched rich-text
dependencies, spreadsheet/archive/template/Dataflow resource limits, batched
Docs/Notifications queries and safe Reporting bind names. See the
[security/performance review](../security/SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
for measurements, test evidence and remaining limitations.
- A deterministic governance-journey clock fixture, fresh-process Campaign
import coverage, and a new Cases patch aligning its root npm facade with its
Python/WebUI package. Historical published tags are not rewritten.
- Git-root WebUI package facades are aligned with their owning packages, with
a cross-composition parity check. Tasks is included in default module
discovery; it remains subject to enabled modules and normal permissions.
## Upgrade and verification
Back up the database and file storage before upgrading. Apply the complete
selected migration graph before starting the new API/workers. This release
includes additive repair migrations `c58a2d7e9f10` (Core ownership history) and
`d8f1b4e7a0c3` (Access external-function mappings), plus Campaign delivery-state
migrations. Existing business evidence is retained; a schema downgrade is not
a substitute for a reviewed backup/restore plan. Restart API and worker
processes together after upgrading their matching packages.
Updated UI consumers require Core 0.1.45 where they use its new shared contracts.
Tenant keys that previously relied on unintended system permissions/wildcards
must be corrected; the release does not preserve that unsafe behavior. Extremely
sparse spreadsheets, oversized generated output and excessive archive paths
can now fail early with a diagnostic.
For archive staging across multiple hosts, provide shared POSIX storage with
working locks or sticky routing. Background delivery still needs configured
workers; increasing the synchronous limit does not create a worker or guarantee
delivery after a process failure. An unknown SMTP outcome must be reconciled,
not automatically resent.
After deployment, manually verify login/logout and least-privilege API keys,
Campaign Settings and independent Mail/ZIP saves, archive upload/unpack,
recipient-complete reports, and SMTP/IMAP progress with an explicitly approved
test mailbox. No release verification sends real campaign mail automatically.
Hard process isolation, forced-password-change/recovery enforcement, bounded
Xrechnung subprocess output and large-history pagination remain separate open
issues. This release is not a claim that all security or performance debt is
resolved. Production-image scans and multi-host evidence must refer to the
actual signed runtime being deployed.
@@ -0,0 +1,125 @@
# Runtime image candidate audit — 8 September 2026
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
Canonical remediation: [GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
This follow-up to the [source security/performance review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
records registry-only scans of nine proposed runtime dependencies and two
same-minor patch candidates. **Runtime publication is held:** patch-only updates
do not resolve the baseline. Source/package publication is a separate outcome.
No images were executed, rebuilt, selected for CI, or published by this audit.
## Method and reproducible evidence
Official Trivy **0.74.0** was installed only in a private local task directory,
without sudo or Docker access. Its Linux-64bit release archive matched both the
official checksums file and GitHub release asset metadata:
- Archive SHA256: `2ae6fe3ee734b7fdf11335663e18c75ea12dccc76062f09f164a3b0f8be4371a`.
- Checksums-file SHA256: `bc701c3c3ee8b9acbea2c23257e41381e3854888f51281616a6ba5dc96963821`.
- Vulnerability database schema 2, updated `2026-09-07T19:06:01.154199452Z`,
downloaded from `mirror.gcr.io/aquasec/trivy-db:2`.
- Scan flags: `--image-src remote --platform linux/amd64 --scanners vuln
--format json --no-progress --timeout 8m --max-image-size 2GB --exit-code 0`.
Findings were counted from validated JSON; exit zero did not mean clean.
- Existing Docker credentials were not read; no private keys or secrets were
used. Checksums over official HTTPS metadata were verified, not independent
Sigstore signatures. See the [official release](https://github.com/aquasecurity/trivy/releases/tag/v0.74.0)
and [registry-only scan documentation](https://trivy.dev/docs/latest/target/container_image/).
Raw evidence is retained locally, not committed:
`/home/zemion/.cache/govoplan-trivy-remote.yKZgjDOg/scan/`.
It contains eleven `reports/*-amd64.json` reports/logs, scanner scripts,
`patch-candidate-inspection.json`, exact successor registry indices, and
`evidence-checksums.json`. Summary SHA256 values:
- `summary.json`: `f2785a731d637452ab9c0b1f5399772c0f8828a63ca83d5fa7496abdad1c757a`.
- `patch-summary.json`: `b3fc6273fcdad98864040ccdf3477ecf379afd46e9f94444b1f2910f48c1d85b`.
All eleven executions succeeded without timeout/rate-limit failure. Initial
summary fields distinguish `scan_execution_complete: true` from
`coverage_complete: false`: Garage has no detectable package inventory.
Checksums preserve evidence identity, not indefinite storage availability.
## Exact requested pins and results
All references below use `docker.io/`. Counts are package-vulnerability records,
not distinct CVEs or confirmed exploitable application defects. A vulnerability
can appear against several installed packages. Unfixed/unknown records remain.
| Image tag | Exact index SHA256 | Critical / High / Medium / Low / Unknown | Fixable C/H |
| --- | --- | --- | ---: |
| `library/python:3.12-slim-bookworm` | `782412e85d0f0984994c290652577d4018aff08145c85b262bb63dc0c7522254` | 5 / 55 / 102 / 103 / 5 | 0 |
| `library/postgres:16-alpine` | `cf78e76683b9ca8c5733cbbdce6c9262b45b6767934dd0a95e671f9a0fc20685` | 1 / 30 / 28 / 14 / 1 | 31 |
| `library/redis:7-alpine` | `ff02b58f971e7d7d156a1267e283fcbbeee91773b6aa36c49dac28ecfe28eadf` | 0 / 0 / 0 / 0 / 0 | 0 |
| `nginxinc/nginx-unprivileged:1.29-alpine` | `0c79d56aee561a1d81c63f00eee5fb5fe29279560cdc55e91425133104c7fbe6` | 0 / 33 / 74 / 37 / 20 | 33 |
| `library/haproxy:3.2.21-alpine` | `66e25cc9a8332635f4e897f7f4b1e5622c25f09f0ee23cddc6ce9bdb3a24772a` | 0 / 2 / 6 / 12 / 0 | 2 |
| `library/caddy:2.10.2-alpine` | `4c6e91c6ed0e2fa03efd5b44747b625fec79bc9cd06ac5235a779726618e530d` | 7 / 75 / 67 / 37 / 4 | 82 |
| `dxflrs/garage:v2.3.0` | `866bd13ed2038ba7e7190e840482bc27234c4afaf77be8cfa439ae088c1e4690` | **Unknown: no inventory** | — |
| `greenmail/standalone:2.1.9` | `3ac5a83dd6727cf95e4d50e18907fb8ee7bbf5f67e8534714dee2fb1b5b2e1d4` | 0 / 0 / 116 / 35 / 0 | 0 |
| `tonistiigi/binfmt:qemu-v10.2.3-68` | `400a4873b838d1b89194d982c45e5fb3cda4593fbfd7e08a02e76b03b21166f0` | 0 / 9 / 2 / 1 / 1 | 9 |
## Patch-only options and limits
Complete publisher tag listings were inspected for nginx 1.29, Caddy 2.10,
HAProxy 3.2, GreenMail 2.1 and binfmt qemu10.2. Two newer candidates were
scanned; their registry index bytes matched both registry and publisher digests,
and contained amd64 and arm64 manifests:
- `library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`:
same Alpine 3.24.1, 24 detected OS packages, zero reported findings. This is
a useful candidate, not a completed compatibility test or application audit.
- `greenmail/standalone:2.1.13@sha256:3df66b7edd01c8a301343ca5e3601d8674760d4708655573560c24745e624fb2`:
upstream changes Ubuntu 22.04 to Debian 13.6; **3 C / 80 H / 98 M / 85 L /
5 unknown**, 30 fixable C/H records. Not selected as a no-base-change update.
- nginx's newest matching Alpine patch is already 1.29.8 at the scanned pin;
Caddy 2.10 remains 2.10.2; binfmt qemu10.2 remains 10.2.3-68. No newer matching
publisher images were found. The current [official Caddy image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy)
uses 2.11.4; switching minor lines requires new scans and compatibility checks.
Priority remediation: Caddy's own seven HIGH records require fixes through
2.11.4, with additional bundled Go/library fixes that must be re-scanned;
nginx's packages include curl/libcurl fixes through 8.22.0-r0, OpenSSL 3.5.8-r0,
c-ares 1.34.8-r0, expat 2.8.1-r0 and libuuid 2.41.6-r1. PostgreSQL's OS records
require OpenSSL 3.5.8-r0 and libuuid 2.42.3-r1; its CRITICAL plus 21 HIGH Go
records concern the **gosu helper**, not PostgreSQL server code. binfmt's nine
HIGH records concern its Go 1.26.4 build, with fixes through 1.26.6. Package
presence does not establish vulnerable-symbol reachability. No unscanned tag
is claimed to meet every fix requirement.
## Python triage and coverage caveats
Python image metadata identifies CPython 3.12.14, but Trivy inventories only
Debian packages and pip, **not CPython/stdlib**. All 60 C/H records concern
Debian packages: 21 CVEs, 50 `affected` records, 9 `fix_deferred`, 1
`will_not_fix`, without a recorded fixed Bookworm version. Five util-linux CVEs
repeat across eight binary packages. These remain installed; they are not all
removed build dependencies. Pip 25.0.1 separately has five MEDIUM/one LOW
records, with fixes through 26.2.0; it is install tooling, and the API image uses
an offline `--no-index` wheelhouse rather than an arbitrary package index.
Narrow triage examples, **not blanket exemptions**:
- Debian states [CVE-2023-45853](https://security-tracker.debian.org/tracker/CVE-2023-45853)
does not affect the built Bookworm zlib binaries because vulnerable minizip
code is not included. Other bundled minizip implementations are separate.
- [CVE-2026-8376](https://security-tracker.debian.org/tracker/CVE-2026-8376)
explicitly requires 32-bit Perl; this scan targets amd64.
- [CVE-2025-7458](https://security-tracker.debian.org/tracker/CVE-2025-7458)
requires crafted arbitrary SQLite SQL; the managed runtime uses PostgreSQL,
but alternate SQLite use must be reviewed.
- Perl's regex and Archive::Tar records need exact binary/module applicability
checks; vendor-deferred status alone is not a finding dismissal.
Only amd64 was scanned. arm64, newly built GovOPlaN API/Web layers and optional
dependency combinations remain unverified. Garage has no inventory; Redis,
HAProxy and PostgreSQL source-built executables, CPython and QEMU static
binaries need supplemental SBOM/source coverage. Zero detected OS findings is
not zero application vulnerabilities. Trivy also lacks Alpine 3.24 EOL metadata
and nginx CVE-2026-80256 detail; unknowns are retained. There were no runtime,
exploitability, secret, misconfiguration, malware or signature-policy checks.
Before lifting the runtime hold: approve and test maintained image-line changes
where necessary, fix or narrowly disposition findings with evidence, close
inventory gaps, scan both architectures and final runtime layers, then run
deployment/ingress smoke checks. Do not silently change base OS, use unpinned
`latest`, rebuild third-party images, or accept all HIGH/CRITICAL findings.
@@ -0,0 +1,139 @@
# Runtime image remediation follow-up — 8 September 2026
Canonical tracking: [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
and [website #9](https://git.add-ideas.de/add-ideas/addideas-govoplan-website/issues/9).
This addendum supplements the [original audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md);
it does not replace that historical baseline or lift either publication or
deployment gate. The immutable 0.1.45 release and `catalog-v0.1.45` are unchanged.
## Source change and candidate decisions
New installer specifications now use
`haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
This is a patch update from 3.2.21 within the supported
[3.2 LTS branch](https://www.haproxy.org/), keeping Alpine 3.24.1.
The [publisher's exact build source](https://github.com/docker-library/haproxy/blob/7a5c202cde713867a737033dca56e7a211a8b8df/3.2/alpine/Dockerfile)
and scanned image configuration retain the non-root `haproxy` user,
`/usr/local/etc/haproxy/haproxy.cfg`, entrypoint and graceful-stop signal.
The [upstream changelog](https://www.haproxy.org/download/3.2/src/CHANGELOG)
includes HTTP parsing, TLS and memory-safety fixes in 3.2.22/3.2.23.
Existing specifications retain their explicit image, including an older pin;
this source change does not update a running installation.
| Candidate | OS | C / H / M / L / Unknown, per architecture | Disposition |
| --- | --- | --- | --- |
| HAProxy `3.2.23-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Installer source default updated; binary/runtime checks pending |
| nginx-unprivileged `1.30.4-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Candidate only; compatibility and website OS upgrade review pending |
| Caddy `2.11.4-alpine` | Alpine 3.23.5 | 1 / 38 / 41 / 12 / 23 | Not selected; all 39 C/H records have recorded fixes |
| Node `24-alpine` (24.20.0) | Alpine 3.24.1 | 0 / 6 / 11 / 12 / 0 | Not selected; major change and six fixable HIGH records |
Each row was scanned separately for **linux/amd64 and linux/arm64**, with the
same counts on both. Counts are package-vulnerability records, not distinct
CVEs or proven exploits. The [machine-readable evidence](runtime-image-candidates-2026-09-08.json)
contains exact index, platform-manifest, config and report digests, inventory
counts, scanner bounds and decisions. It is audit data, not an accepted release
manifest or an installer input.
nginx's candidate reference is
`docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce`.
It inventories 70 Alpine packages including nginx/NJS, uses UID 101 and exposes
8080. The [upstream stable release](https://nginx.org/en/download.html) and
[security advisories](https://nginx.org/en/security_advisories.html) include the
1.30.4 fixes. The publisher retains its
[unprivileged port and temporary-path contract](https://github.com/nginx/docker-nginx-unprivileged).
For the website, this changes nginx 1.27.5 to 1.30.4, NJS 0.8.10 to 1.0.1 and
Alpine 3.21.3 to 3.24.1. These changes are explicit review items; the website
Dockerfile has not been changed. The GovOPlaN Web image still requires an
explicit verified `NGINX_IMAGE` build argument.
Caddy's candidate reference is
`docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648`.
Although this is the current
[official image line](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy),
its inventory still includes Go 1.26.3, `x/crypto` 0.52.0, `x/net` 0.55.0,
`x/text` 0.37.0 and gRPC 1.81.0. Recorded fixes include Go 1.26.6,
`x/crypto` 0.55.0, `x/net` 0.56.0, `x/text` 0.39.0 and gRPC 1.83.1; Alpine
findings also remain in c-ares, curl/libcurl and OpenSSL. The CRITICAL
`CVE-2026-56854` concerns `x/crypto/ssh` source-address enforcement. A module
record alone does not establish that this binary exposes that SSH path; exact
binary symbol/reachability analysis is still required for a disposition.
The [official Node image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/node)
still maps Node 22 Alpine to 22.23.2 and the previously scanned digest. Node 24's
candidate is
`docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf`.
Its HIGH records remain in two OpenSSL packages and npm dependencies
`brace-expansion`, `ip-address` and `tar`; fixing the earlier critical tar
record alone is insufficient. The website builder stays on Node 22 pending
a reviewed build-tool remedy and a final builder scan.
## Method, verification and retained evidence
The existing Trivy 0.74.0 executable was rehashed against the previously verified
archive member: `d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791`.
The same schema-2 vulnerability database was used, updated
`2026-09-07T19:06:01.154199452Z`. No tool installation or database refresh occurred.
Index bytes matched both the registry digest header and Docker Hub publisher
metadata; both platform-manifest byte hashes matched the index. All eight
registry-only scans completed successfully with validated JSON, `--list-all-pkgs`,
`--scanners vuln`, an eight-minute/2GB image bound, an empty Docker configuration
and no inherited credentials. Exit zero means execution succeeded. Private
temporary paths and in-memory artifact cache isolated this follow-up from the
earlier scanner's artifact cache; its vulnerability database was read only.
Raw reports, logs, manifests, publisher metadata and the scanner script are in
`/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh/`:
- `summary.json` SHA-256: `0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56`.
- `frozen-images.json` SHA-256: `d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467`.
- `scan_successors.py` SHA-256: `f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07`.
Local retention is not permanent artifact hosting; preserve this evidence with
the eventual reviewed release. The JSON evidence records compressed registry
layer sizes; these are not expanded filesystem limits or final GovOPlaN sizes.
Installer regression checks cover the new generated image pin, legacy
specification fallback, preserved explicit images, generated topology and
configuration: `python -I -m unittest discover -s tests -p
test_deployment_installer.py` ran 45 tests successfully with one skip because
Core was not importable in that isolated test environment. The skipped Core
startup-configuration integration was subsequently rerun in the shared development
environment with Core available: all 45 installer tests passed with no skips,
including generated-environment startup validation. This is configuration
validation, not execution of the candidate image.
Both repositories passed `git diff --check`; the audit JSON and all eight
report hashes were checked against the retained evidence.
**Docker, Podman and HAProxy executables are unavailable on
this host**, so no image or HAProxy configuration was executed and no daemon was
installed. Publisher metadata and installer tests support the scoped source
patch; they do not establish binary or deployed compatibility.
## Gates that remain open
- Validate `haproxy -c` on generated local, existing-proxy and managed-ingress
configurations using the exact pinned image and target architectures. Run
bounded isolated checks without live mounts, secrets, privilege or external
network access. Then verify DNS discovery, readiness, forwarded headers,
replica routing and graceful termination in the intended runtime.
- Test the nginx candidate with both the website configuration and GovOPlaN
WebUI entrypoint/proxy configuration, including UID 101, writable temporary
paths, health paths, cache headers and static catalog bytes. Approve the
website nginx/NJS/Alpine version changes before changing its Dockerfile.
- Resolve Caddy, Node build-tool and all unchanged baseline dependencies with
updated publisher images or narrow reviewed applicability evidence. No
severity-wide exceptions or custom third-party rebuilds were introduced.
- Close the original source-built/static inventory gaps. HAProxy's 24-package
OS inventory still omits the source-built HAProxy executable. Node's npm
inventory still omits the Node executable/stdlib. Garage, CPython, Redis,
PostgreSQL and QEMU gaps are unchanged. Alpine 3.24 EOL metadata is still
missing from this scanner; zero findings is not complete coverage.
- Scan **final built** API/Web/website layers and the selected managed
dependencies on both architectures, then perform migration, worker,
readiness and ingress smoke checks. Record failure and unknown states.
Secrets, misconfiguration and image signature policy need separate checks.
- Obtain the website deployment host/operator and rebuild/restart authority,
preserving the exact immutable catalog/keyring/module-directory bytes and
verifying fresh public responses after an authorized rollout.
No images were built, executed, published or deployed; no running service,
release tag, signed manifest, CI image input or live infrastructure was changed.
@@ -0,0 +1,219 @@
# Security and performance follow-up — 8 September 2026
This follows the [original review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
and its post-release issue reconciliation. It describes new source work after
the frozen 0.1.45 release; it does not change published tags, packages, signed
catalogs or deployed images. Gitea remains the canonical state log.
## Implemented source slices
- [Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
a shared disposable-process runner enforces wall/CPU/address-space/input/output
limits, bounded stderr, process-group cleanup and non-queuing per-process
admission. A private binary codec bounds decoding before allocating a full
object graph and preserves explicitly supported data types without pickle.
Read the owning Core `docs/BOUNDED_PROCESS_CONTRACT.md` before adding callers.
- Connectors XLSX parsing, Templates rendering, Files ZIP/TAR inspection and
extraction, and Dataflow reference previews/development execution now use
that boundary. Existing authorization, sessions, provider credentials,
idempotency and persistence remain in the parent. No unprotected inline
fallback is used. Each module contributes static EN/DE user/admin limits and
operational consequences through its manifest.
- Files snapshots authorized sources inside shared admission, validates private
staged members, and acknowledges each persisted member before decoding the
next. Numeric progress remains available. The acknowledgement is event-driven,
not a fixed sleep per member. Reads allocate by validated actual file size,
not by the configured ceiling. Failures reap children, clear private staging
and retain the existing transaction/blob cleanup and explicit retry behavior.
- Dataflow's normal reference preview formerly bypassed the backend wrapper;
it now enters the worker too. Nested source configurations cannot collide
merely because subflows reuse node IDs. Combined reference-source data is
checked before creating further columnar copies, while individual providers
retain their own authorized-read bounds. Staging/production still require
DuckDB; this change does not replace that separate backend.
- [Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
current-password change, session/CSRF rotation, cross-tenant session and human
API-key revocation, and optional administrator-assisted recovery. Recovery
codes are hashed, single-use, expire after 15 minutes, require a current local
System owner and explicit identity verification, and recheck current account,
membership, tenant and issuer authority at redemption. A password change also
invalidates outstanding codes issued by that account for other people. Audit
evidence and validation/error responses do not contain passwords or codes.
External-provider and service-account rules remain separate.
- The Access UI provides first-login/required change, self-service change,
policy-aware sign-in help, public code redemption and eligible owner issuance.
Core consumes an optional lazy auth-action capability rather than importing
Access internals. The required-action gate fails closed if its UI is missing.
- [Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
full-history lists batch pinned revisions, while new summary and bounded
step/event endpoints preserve authorization and explicit pagination. Existing
full-history responses are not silently truncated. Exact inbox total semantics
are retained and their counting cost is documented.
- [Meta #55](https://git.add-ideas.de/GovOPlaN/govoplan/issues/55):
shared version/planning helpers recognize the existing nested developer
package, not invented root metadata. All tag batches enforce trusted private
source ownership, registered origins and clean main/upstream state. Meta
batches additionally require exact composition and matching Core evidence.
Whole-batch preflight,
frozen source receipts, annotated immutable tags, object-pinned atomic
publication and post-effect remote checks are covered with temporary local
repositories. Hidden Git index flags, unsafe ancestry, alternates and changed
Git-directory identities are rejected. Selected version/composition metadata
must be tracked, so ignored files cannot describe bytes absent from a tag.
Applicable unselected Core WebUI inputs have bounded, frozen read receipts;
backend-only releases do not read them. The existing local module-candidate
exception remains intact. The weaker legacy mutation path was removed.
Canonical whole-package preview and receipt-bound apply now cover Meta's
version preparation too. Core must already match the target. Preparation
requires a separate trusted checkout, explicit out-of-run confirmation and
unchanged source/tooling receipts; it cannot rewrite the running operator.
Plans place Meta after Core and explain the manual preparation/publication
steps instead of promising a durable self-update. Ambiguous partial writes
require reconciliation, without automatic rollback or retry.
- [Runtime-image follow-up](RUNTIME_IMAGE_REMEDIATION_2026-09-08.md): eight new
registry-only scans cover four exact candidates on amd64 and arm64. New
installer specifications select the patched same-line HAProxy digest;
existing specifications retain their explicit image. Other candidates and
unresolved inventory/deployment gates remain visible, not blanket-approved.
## Verification record
Targeted checks include actual child execution, catastrophic regex CPU,
aggregate memory exhaustion, TAR extension metadata, noisy output, malformed
transport/staging data, Unicode allocation limits, cancellation/callback
failures, descendant cleanup, rollback and explicit retries. Local mixed-owner
composition tests completed nine real children, rejected six overlapping
requests as busy, observed at most one unreaped child and recovered all slots.
This is local admission evidence, not a target deployment load certification.
Workflow fixtures serialize 40 different pinned revisions with five SQL reads;
summary lists use one query for 40 ordinary rows. Exact inbox totals for
40/400/4,000 candidates used one query, with measured local costs approximately
0.011/0.057/0.492 seconds. These are fixture measurements, not production SLOs.
The broader Core API smoke suite exposed three stale campaign assertions.
All three failures were reproduced against the unchanged private frozen 0.1.45
sources. Updated fixtures verify recipient-summary projection, detailed payload
separation and explicit fenced recovery of a confirmed stopped runtime; observing
SENDING alone must not make a claim recoverable. All 76 smoke tests then passed.
No production Campaign behavior was changed to satisfy these tests.
The final release-tool suite passed 279 tests and 68 subtests, including
temporary local remotes and adversarial source/tag/receipt changes. Rechecking
the whole batch before effects is deliberately conservative: its repeated
filesystem/Git/remote work grows quadratically with batch size. It is not a
new unattended publication path or permission to execute unreviewed source.
Strict interface inventory now reports no unclassified endpoints and exact
contextual help for all 133 high-risk controls. Seventeen password browser cases
include actual F1 help from the restricted screen, empty workspace scopes,
EN/DE layouts, Unicode boundaries and no credential values in help URLs.
The initial production bundle remains within the unchanged limits (512,036
raw bytes and 162,415 gzip bytes; 1,713 gzip bytes below its ceiling), with
46 optional descriptors and no eager optional-module imports.
The focused checker now includes the new Core process, mixed-owner admission,
Access password, Templates and Files worker tests, the repaired campaign smoke
cases, and browser-side auth/password transport contracts. The full focused run
passed, including 63 production module/build permutations and all 230 browser
cases. Its two opt-in Datasources PostgreSQL cases were skipped in that run
and subsequently passed against the isolated real database described below.
The final Meta preparation gate was added after that full run and verified
with the owning release-tool suite and the focused release-gate command.
Manifest validation passed for all 72 modules. The full focused log is
`/mnt/DATA/tmp/govoplan-security-followup-20260908-focused.log`.
The first follow-up quick audit captured an unchanged 79-repository snapshot
in `/mnt/DATA/tmp/govoplan-security-followup-quick-20260908-7s8Sgh/`.
All four required scanners completed, with zero missing/execution reports;
all 168 report checksums and 163 machine-readable reports were validated.
Gitleaks found no secrets in all 79 histories and 79 worktrees. Local Semgrep
rules reported zero findings. Production Bandit reported 65 low and four medium
warnings, and production Ruff retained 54 warnings. The two added Bandit
warnings identify the new Core subprocess import and invocation: trusted
server-owned arguments, no shell, and the documented resource/process boundary
were reviewed; warnings remain visible. This is report-only evidence, not a
warning-free audit or a penetration test. A final snapshot follows the
cross-module declaration/contextual-help corrections and release-tool checks.
That final audit completed on 8 September, 05:59:4606:02:18 UTC, in
`/mnt/DATA/tmp/govoplan-security-final-quick-20260908-vAwQIh/`. All 79 start/end
source fingerprints were identical; all four scanners completed, all 168
registered report checksums matched, and all 163 JSON/SARIF reports parsed.
There were no missing reports or scanner execution errors. Semgrep and both
Gitleaks scopes again reported zero findings. Production counts were unchanged
from the first follow-up: Bandit 65 low/four medium and Ruff 54. Test-only
counts were Bandit 140 low/34 medium and Ruff 136. A separate frozen scan of
all ten changed Meta release/deployment Python files reported seven low Bandit
and four Ruff S603 warnings, with no execution errors. Its four argv-only
subprocess sites were reviewed; the preparation additions introduced no new
warnings. No findings were hidden or severity-wide exceptions added.
The audit manifest SHA-256 is
`a997b786239cd11443cb665d5f9041a968cc38f9d49171e68bb868bf2bd73310`;
its report-checksum list SHA-256 is
`dc590ca5b0a4e445019a05536d410226088d67b40d61cd7657bdef4a4eae56d8`.
The audit includes the eight committed feature/website source changes and the
final uncommitted Meta source. Only this evidence document was updated after
the source freeze ended; the final Meta commit and remote publication are
recorded in the linked Gitea issues, not inferred from local audit completion.
Fresh dependency audits are retained in
`/mnt/DATA/tmp/govoplan-dependency-final-20260908-d24LEK/`: all four full npm
lockfile audits (Core WebUI, Mail root/WebUI and website) report zero known
vulnerabilities. Installed Python auditing covers 137 distributions with zero
known vulnerabilities; 51 local GovOPlaN distributions lack PyPI advisory
coverage. Core's 46 linked packages are likewise not claimed covered by public
registry advisories. All 12 dependency-file hashes and the installed inventory
were unchanged. No packages were installed or automatically fixed.
Managed PostgreSQL 16.15 fixtures used private Unix sockets, synthetic roles
and databases, no TCP listener, per-case schemas and bounded SQL/lock waits.
Both previously skipped Datasources races passed. Twenty-one existing Access
password HTTP tests and four additional races passed on PostgreSQL: single-use
redemption, stale-session/password replacement, competing issuance, and issuer
password revocation during redemption. Four release/development migration checks
also passed for Access and Workflow, including credential preservation and
idempotent indexes. The four races are now owning opt-in Access regressions;
see `govoplan-access/docs/PASSWORD_RECOVERY_POSTGRES_TESTS.md`. These local
database checks do not certify a deployment, fleet load or external recovery
handover. With both explicit PostgreSQL test URLs enabled, the full Access suite
passed 122 tests and 18 subtests, and the full Datasources suite passed 57 tests,
without skips. Access retained 12 existing SQLite datetime-adapter warnings in
its separate SQLite migration cases. Both temporary PostgreSQL fixtures were
stopped and independently verified: no server process, private socket,
generated schema or synthetic cluster remains. Scripts, logs and shutdown
receipts are retained under `/home/zemion/.cache/govoplan-pg-security-20260908.RAUitg/`
and `/home/zemion/.cache/govoplan-pg-promoted-20260908.JZcIKL/`.
## Adoption and remaining gates
1. `AUTH_LOCAL_PASSWORD_RECOVERY_ENABLED` remains **false** by default. The
existing flag is still advisory until an operator explicitly adopts and
enables the complete recovery policy. Confirm who verifies identity and how
the one-time code is handed over; automated email recovery is not enabled.
Test first-login, lost-password, code expiry and administrator availability
in the target environment before enforcement.
2. Access migration `e9a2c5f8b1d4` adds recovery evidence; Workflow migration
`9e6b3f8a2c7d` adds summary-pagination indexes. Use normal backed-up upgrade
procedures and account for index-build cost. No manual live migration or
server restart was performed during this work. The user's existing devserver
has automatic reload, so live schema state must not be assumed unchanged.
3. Release preparation must assign new source/package versions and require a
Core version containing the new worker/auth contracts in the affected module
metadata, including matching WebUI assets. The old immutable release must
not be relabelled or treated as containing these APIs.
4. Resource limits are not an arbitrary-code, filesystem or network sandbox.
Admission is per API/worker process, not fleet-wide. Validate Linux/cgroup
memory, disk quotas, process counts, cancellation and legitimate large-file
workloads on the intended runtime before increasing concurrency. Core #297
retains this target-evidence follow-up.
5. Meta #52 and website #9 retain runtime-image/publication/deployment holds.
Docker/Podman/HAProxy executables are unavailable here. Final built images,
binary/source inventories, ingress behavior, migration/readiness/worker
smoke checks and the website's target/operator authority remain outstanding.
Zero findings in a detected package inventory is not full image coverage.
No real messages, IMAP appends, password resets, provider operations or deployment
actions were used as test fixtures. Development tests use temporary databases,
private temporary files, mock transports and managed test-browser servers.
@@ -0,0 +1,207 @@
# Security and performance review — 8 September 2026
Coordinated status: [Core #296](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/296).
This records a workspace-wide automated scan, targeted manual boundary review,
and a verified implementation pass. It is not a penetration test, an exhaustive
line-by-line review, or a security certification. The audit was completed on
local, unpublished changes, preserving existing worktree changes. Subsequent
release preparation/publication is tracked in
[GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51) and the
[0.1.45 release notes](../releases/0.1.45.md).
## Implemented findings
| Area | Finding and change | Evidence / ownership |
| --- | --- | --- |
| Authentication — high | Preserve service-account provenance and current scope ceilings instead of recalculating them as ordinary membership permissions. Tenant API keys cannot retain canonical system permissions or unsafe wildcard grants. | Previously failing isolated regressions; [Access #21](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/21). |
| Authentication — medium | Warm-cache API keys must follow the same explicit-header credential rules as cold authentication. A session cookie cannot turn an API key into a session credential. | Regression covering source-dependent authentication. |
| Browser authority/cache — medium | Clear reusable data on auth changes and write settlement; fence late 200/304 writes and obsolete 401 side effects. Honor server no-store/no-cache and explicit fresh-read requests. Interactive login/logout remove retained automation keys that could shadow cookie-session identity. | 23 real-client regressions. Unchanged settings keep their object identity, preventing profile-fetch loops. Core `docs/API_CLIENT_CACHE_CONTRACT.md`; owning Access EN/DE session/field documentation. |
| Spreadsheet resource exhaustion | Validate actual XLSX coordinates before openpyxl traversal; ignore misleading declared dimensions; count blank row gaps toward the existing limits. | [Connectors #18](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/18), 13 tests and 2 subtests. |
| Template resource exhaustion | Enforce the existing 5 MiB output budget during substitution and item construction, including UTF-8, HTML escaping and separators. | [Templates #7](https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/7), full 20 tests; independent 3,000-case valid-output comparison. |
| Archive resource exhaustion | Inspect regular TAR member limits before traversing payloads. Limit archive paths to 4,096 UTF-8 bytes / 128 components and count derived directories against entry limits. | [Files #46](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/46), 55 archive and 15 documentation tests. Extension-header decoding still needs stronger isolation. |
| Dataflow resource exhaustion | Reject LPAD/RPAD target lengths above the existing 1,000,000-byte preview budget before fill evaluation/allocation. Preserve final serialized-byte checks. | [Dataflow #22](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/22), full 104 tests and 39 subtests; 7 new guard tests independently rerun. |
| Docs performance / defense in depth | Batch revision reads per request, avoid loading pending draft bodies for readers, and validate tenant/entry/publication consistency while retaining owner/audience checks. | [Docs #22](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/22), full 39 tests. |
| Notifications performance / defense in depth | Batch delivery-attempt loading while preserving recipient checks and rejecting inconsistent attempt references, including already-loaded relationships. | [Notifications #6](https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/6), full 23 tests. |
| Session-list performance | Apply active/expiry predicates and the existing 100-row cap in SQL, before loading session history. | Query-shape regression in Access. |
| Reporting correctness | Use structural bind-name suffixes for recursive calculated measures, preserving valid dotted/hyphenated public keys and parameter uniqueness. | [Reporting #10](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/10), full 29 tests. |
| Audit hygiene | Redact Gitleaks logs and machine reports on current, history and legacy scanner paths. | 13 audit-wrapper tests enforce the flag. |
All changed module workflows/limits have owning EN/DE DocumentationTopic updates.
Independent review found no concrete regression in the backend changes.
## Measured performance changes
These are SQL-query counts in isolated 40-item fixtures, not production latency
or throughput claims. Authorization is still evaluated for each request.
| Projection | Before | After |
| --- | ---: | ---: |
| Docs reader entries | 41 SELECTs | 2 SELECTs |
| Docs editor entries | 81 SELECTs | 2 SELECTs |
| Notification list with attempts | 41 SELECTs | 2 SELECTs |
The Docs 401-entry batching regression uses 3 SELECTs. Resource guards reject
oversized work before the formerly expensive allocation/traversal. This does
not make every legitimate upload or campaign faster. Honoring no-cache can
increase server validation requests; ETags still avoid retransmitting unchanged
bodies. That authorization/freshness trade-off is deliberate.
The original audit snapshot measured 517,380 initial JavaScript bytes and
164,119 gzip bytes. Release preparation's pure-defaults split reduces this to
516,730 initial bytes and 163,908 gzip bytes. Restoring the missing Tasks
descriptor then measures 516,987 initial / 163,976 gzip bytes with all 46 module
descriptors lazy, within the unchanged 524,288 / 164,128 caps. The gzip margin is still small; future
startup work should reduce eager dependencies, not raise the cap automatically.
The full 209-case browser suite passed before the split, followed by 13 focused
browser checks after it. Radon recorded 238 rank-D-or-higher entries; complexity
is a review-priority signal, not a performance measurement.
## Dependency remediation
Core's full npm audit went from 30 affected package entries to zero. Most initial
entries were transitive effects of the same Tiptap advisory, not 30 independent
application exploits. The website went from two affected entries to zero; both
Mail lockfiles also report zero.
- Tiptap packages are aligned at 3.31.3, with direct minimum ranges raised to
3.30.4 in both development and release manifests, with a parity regression.
Added an actual installed-library prototype-attribute regression for
the [maintainer's security advisory](https://github.com/ueberdosis/tiptap/security/advisories/GHSA-cp6q-959q-f8rh).
- Core now resolves xmldom 0.9.12, browserslist 4.28.9 and nanoid 3.3.18.
The website's affected browserslist/nanoid dependencies are patched too.
- Development/audit requirements now require pip >=26.2; the local development
environment uses 26.2.1. The installed audit originally flagged
[CVE-2026-13346](https://github.com/advisories/GHSA-qwm4-qh6w-59xr), requiring an
attacker-controlled package index. This is an installation-tool vulnerability,
not evidence of an exposed application endpoint.
The final installed Python audit enumerated 188 distributions: 137 were
auditable with zero known vulnerabilities, and 51 local distributions were not
available in PyPI. Those skips are covered by source review, not by a claim of
dependency-advisory coverage. Production images and every optional dependency
combination were not independently resolved or scanned.
## Scan coverage and limitations
Evidence directory:
`/mnt/DATA/tmp/govoplan-security-performance-20260908-gsk8jn/`.
The final `final-quick/manifest.json` captures 79 repositories, tool versions,
start/end repository fingerprints, report checksums, 168 report artifacts and
163 validated JSON/SARIF reports. It records an unchanged workspace, complete
coverage for its four required scanners, no execution errors and no missing
reports. It ran in report-only mode: exit zero does **not** mean zero warnings.
- Final production Bandit: 447,008 Python lines; 67 warnings (63 low, 4 medium),
no high findings. Ruff security rules: 54 warnings. SQL-construction warnings
were reviewed against identifier/operator validation and bound values in
DuckDB/Reporting; no injection fix was warranted there. XML import warnings
were checked: feed/BPMN input parsing uses defusedxml; stdlib imports support
types/output construction. Operator-owned fenced-run argv is not a public
arbitrary-command endpoint. Xrechnung output buffering remains a follow-up.
Assertions and error-swallowing markers remain review/maintenance warnings,
not proof that all such code is harmless.
- Final local Semgrep rules: no findings. The broader OWASP-rule pass applied
272 rules to 4,169 tracked targets. Its seven warnings recommended weakening
owner-only 0700 permissions; they were rejected as false positives. One
Calendar rule timeout was rerun with a 60-second budget: zero findings/errors.
Bash and conformance TypeScript checks passed despite two scanner-specific
parser limitations. Ignored/dependency/generated paths are not a complete
line-by-line source audit.
- Gitleaks: 79 Git histories plus 79 worktrees, 158 redacted reports, zero
detected secrets. This does not establish that deployed credentials are safe
or that formerly exposed credentials have been rotated.
- Tool versions included Semgrep 1.176.1, Bandit 1.9.4, Ruff 0.15.21 and
Gitleaks 8.30.1. The downloaded Gitleaks binary archive matched the official
release SHA-256 before execution.
- The containerized full-toolbox path could not access Docker's daemon. Its
full-mode Trivy/misconfiguration and additional OSV scans were **not** run.
A subsequent [registry-only runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md)
successfully scanned nine pinned candidates and two same-minor successors
for amd64 without Docker. It found unresolved vulnerabilities and inventory
gaps; runtime publication is held. This does not complete full-toolbox,
arm64, final-runtime-image or deployment coverage.
No live application probes, database changes, file operations, mail sends,
IMAP appends, imports, notification delivery, deployments, commits or pushes
were performed. Browser tests used isolated mocked fixtures. Package installs,
builds and temporary audit-tool installation were local development operations.
## Verification and remaining work
- 209/209 browser conformance tests pass; production Core/website builds,
conformance TypeScript, 24 Core client/dependency regressions, 4 real-client
Files reload checks, and 72/72 manifest checks pass.
- Access's full 91-test suite passed before the final documentation-only update;
the final documentation suite passed all 4 tests. Other module counts appear
above. The new authentication/resource tests include demonstrated pre-fix
failures rather than only structural assertions.
- The original focused workspace run stopped at the institutional
governance/Portal fixture (`tests/test_institutional_governance_journey.py:223`,
`IndexError`). Release preparation fixes its mixed clocks using the existing
temporal context, retaining validity-boundary exclusions; 7 journey tests and
ambient-year checks pass. Tracked in
[Meta #50](https://git.add-ideas.de/GovOPlaN/govoplan/issues/50).
- Campaign's apparent host-path issue was ruled out by existing tracked
API/build/snapshot guards and 11 passing tests under normal initialization.
Release preparation fixes the standalone import cycle through a deferred
resolver import without changing validation rules. Fresh-process coverage,
all 11 path tests and Campaign's full 611-test suite pass.
Next coordinated work:
1. [Hard resource isolation — Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
regex CPU, aggregate allocation, TAR extension metadata, bounded workers and
cancellation, followed by production-like concurrent load tests.
2. [Forced password change/recovery — Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
the current flag is advisory only. Do not enable enforcement without a usable
local-password/recovery flow and external-provider rules.
3. [Bound subprocess output — Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2):
enforce the existing 2 MiB limit while draining stdout/stderr, not afterwards.
4. [Workflow revision batching/history projection — Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
batch evidence lookups; separately define explicit history pagination and
authorized-total semantics. Docs/notification history volumes also remain.
5. Resolve the [runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md) findings
tracked in [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
and coverage gaps before lifting its publication hold; complete deployment
audits, review exposed development credentials and worker quotas, and
benchmark realistic tenant sizes/concurrency. The sanctions
transport's fixed HTTPS/redirect allowlist is not a demonstrated arbitrary-URL
issue, but migration to Core's pinned egress transport remains desirable.
Operational compatibility: tenant keys relying on accidental system/wildcard
permissions must be corrected rather than weakening the guard. Extreme sparse
spreadsheets, overly deep/long archive paths and oversized padding intermediates
can now fail early with diagnostics. No stored documents or configurations were
deleted or silently migrated.
## Post-release follow-up — 2026-09-08
The findings and scanner counts above describe the original audit snapshot.
The following source fixes are subsequent to the frozen `0.1.45` composition;
they do not change its immutable tags or published package bytes.
- [Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2)
now enforces the existing shared 2 MiB stdout/stderr limit during execution
and kills/reaps the direct validator on overflow, timeout or cancellation.
Report reads are bounded to 16 MiB plus one probe byte before interpretation.
The 30-test module suite passes; noisy-child and report-read regressions were
also demonstrated to fail against the previous source. Owning EN/DE static
documentation is updated. POSIX pipe capture is required; disk quotas,
descendant isolation and process-level CPU/memory limits remain separate work.
- [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54) now preserves
the last command's failure status after exhausted installer retries. Twelve
isolated stage/scenario combinations cover every retry call site, success,
backoff and caller termination under `set -e`. The test is included in the
focused checks and installer CI. See the bilingual
[installer retry note](../operations/WEBUI_RELEASE_DEPENDENCY_RETRIES.md).
These are unreleased follow-up source changes, not a new runtime release or
deployment. The runtime-image hold under Meta #52 remains in force; the
historical peer-dependency workaround still needs its separate review.
Further implementation and adoption gates are tracked in the
[security follow-up](SECURITY_FOLLOWUP_2026-09-08.md), including disposable
parsing/execution workers, opt-in password recovery, workflow read projections
and the newer runtime-image evidence. The original scanner counts above remain
historical and are not silently replaced by later test results.
@@ -0,0 +1,305 @@
{
"schema_version": 1,
"purpose": "Audit evidence only; not a release manifest or active installer configuration.",
"observed_at": "2026-09-08T03:56:47.666808+00:00",
"runtime_publication_held": true,
"website_deployment_held": true,
"scan_execution_complete": true,
"coverage_complete": false,
"scanner": {
"name": "Trivy",
"version": "0.74.0",
"binary_sha256": "d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791",
"database_metadata": {
"Version": 2,
"NextUpdate": "2026-09-08T19:06:01.154199291Z",
"UpdatedAt": "2026-09-07T19:06:01.154199452Z",
"DownloadedAt": "2026-09-07T23:30:53.198039224Z"
},
"source": "remote",
"scanners": [
"vuln"
],
"list_all_packages": true,
"images_executed": false,
"existing_docker_credentials_used": false,
"timeout": "8m",
"maximum_image_size": "2GB"
},
"evidence": {
"private_directory": "/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh",
"summary_sha256": "0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56",
"frozen_images_sha256": "d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467",
"scanner_script_sha256": "f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07"
},
"candidates": [
{
"name": "haproxy",
"image": "docker.io/library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
"disposition": "source_default_updated_binary_runtime_verification_pending",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:0666a2c2f41d341084ed2da85392b48cdcd766adfa28231f31305724ed5c6ea5",
"config_digest": "sha256:9621d75e50a8f26d3738ae3cdbf15e98c6aa5cd48e6baca0699492de502156f5",
"compressed_layer_bytes": 20516844,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 24
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "40cfc3db74e29f09723f38698660ccdd50ac935c8d6e1e75c6e0555b3e9361fb",
"log_sha256": "46881f695780f89c037d5b0b4dc0ded9ea4e459077c7658d80b786efc5754084"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:cd20b9dc6b4713956a2a043997001a1948167d345e7c8d5bd5ff2e667166651f",
"config_digest": "sha256:19796bff8905d4a46c9463c576203b20cac8984d41b7b01ea9cbff55e8422c34",
"compressed_layer_bytes": 20970772,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 24
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "0e1326c58abf358d592fa55c94333228ce1094edf4e489fcc4ece6e32d3320f6",
"log_sha256": "397c2fbf1044cf50733d68b4b9cc4eb68211d96f1f302d5e9f8af0d11fb0de1c"
}
]
},
{
"name": "nginx-stable",
"image": "docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce",
"disposition": "candidate_pending_compatibility",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:b8c179cd3c2ae222a873dd59fbae240fadc03836cae5198afc9e9c19919c3880",
"config_digest": "sha256:8b5953dae38d27a76bca22373bb920fd6ce8d9d7da21578d2926e678002de8a0",
"compressed_layer_bytes": 25526590,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 70
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "3ad164dae3cdf891b12e41451b8a8e65a5e1688824a7c01d848e1274363e6bf9",
"log_sha256": "0f99ff3d9b4396de48655bf8299df30c14ba0c579f480d37baa7d2f6a4c11f1d"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:b6742a0cbd749add25346658991c3da06a8e38796949df120fed78db8c512576",
"config_digest": "sha256:4d8b10f5d2ff99e7aa5f161930d8a4693a86a26f288ec8c0d4cd47f2ef5af179",
"compressed_layer_bytes": 25892370,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 70
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "506fdeb97525ed8e4d9ae538c42bab7aa2217f662a7135f0f12d16d20410c7a6",
"log_sha256": "c41ef9ea091d00f18c7a097404672591bee85e7477ae17d8f5ca771d8e42b2bb"
}
]
},
{
"name": "caddy",
"image": "docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648",
"disposition": "not_selected_remaining_fixable_findings",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:98eb57d882ccd5213d1688764db10c1ca2c58a1ca3a6717a3411ad798f7a423a",
"config_digest": "sha256:af555904a0961945f16bb323a501457b13a4f7e9bde969b145b97da80b38ecbe",
"compressed_layer_bytes": 23907283,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.23.5"
},
"inventory": [
{
"type": "alpine",
"packages": 32
},
{
"type": "gobinary",
"packages": 146
}
],
"counts": {
"CRITICAL": 1,
"HIGH": 38,
"MEDIUM": 41,
"LOW": 12,
"UNKNOWN": 23
},
"fixable_high_critical": 39,
"unique_cves": 68,
"report_sha256": "e483352a1d5b9b97950dcf92e4dfcf4600f5b09306af3d0640b10ca229a07779",
"log_sha256": "9cd599d6dd8c101b421fdeb57036cec1f69f815d765a8bf1f850ec60061036f4"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:1172d4213087d3fc30bafc7ff2c2896180eb0c41ff7f75f315568fb36cabdcba",
"config_digest": "sha256:6b08c1b9858ca9a7d99c1da13c3695081e0e604c6cf214ca26a7ce0e2c4fd9b4",
"compressed_layer_bytes": 22722712,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.23.5"
},
"inventory": [
{
"type": "alpine",
"packages": 32
},
{
"type": "gobinary",
"packages": 146
}
],
"counts": {
"CRITICAL": 1,
"HIGH": 38,
"MEDIUM": 41,
"LOW": 12,
"UNKNOWN": 23
},
"fixable_high_critical": 39,
"unique_cves": 68,
"report_sha256": "20e04d820e3b27d57575ea177e523e23109fd83344cdf57e184a4d2fad27e803",
"log_sha256": "a1d9c37aa7948c137db64040d95e5930c5859a8acd71ac5bc41ff168e2b270c5"
}
]
},
{
"name": "node-lts",
"image": "docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf",
"disposition": "not_selected_remaining_fixable_findings",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:4caaaf42195bcd6f6f3559a413b20cb8f8ad089e231ee874cf7701643966689f",
"config_digest": "sha256:ee289c69ed1ac50a5a042112ea97f132800e2dd53e832da27784f00e45b3289c",
"compressed_layer_bytes": 58486244,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 18
},
{
"type": "node-pkg",
"packages": 146
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 6,
"MEDIUM": 11,
"LOW": 12,
"UNKNOWN": 0
},
"fixable_high_critical": 6,
"unique_cves": 19,
"report_sha256": "c927d995dde700c92027f6328dc6c273f0f1445cd8154301480757cb962e02b2",
"log_sha256": "c7dc1900cbe39c9f91e228b2770bcbd394ec2914d2b3879478295fc7f8f77ab3"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:d3724e44ee368606d753e0027eb8d2a94fc1f275e5d9e4620178a12edb655f5f",
"config_digest": "sha256:722cc1507731edf58a4c0bc3e29553c44ce5774d0849242c1b237abc79926a0a",
"compressed_layer_bytes": 58935654,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 18
},
{
"type": "node-pkg",
"packages": 146
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 6,
"MEDIUM": 11,
"LOW": 12,
"UNKNOWN": 0
},
"fixable_high_critical": 6,
"unique_cves": 19,
"report_sha256": "e5f7799761f23cefc36929381b4186eeb3afb46a2a559c789d12a7eea5dc30d0",
"log_sha256": "fd24671f0da4d3b20dc8bcc2718531e6f6e19155d49bed15958965e21dd10813"
}
]
}
]
}
+44 -44
View File
@@ -4,87 +4,87 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan"
version = "0.1.42"
version = "0.1.46"
description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" }
dependencies = [
"govoplan-core[server]==0.1.42",
"govoplan-tenancy==0.1.20",
"govoplan-organizations==0.1.20",
"govoplan-identity==0.1.20",
"govoplan-idm==0.1.24",
"govoplan-access==0.1.23",
"govoplan-admin==0.1.22",
"govoplan-policy==0.1.22",
"govoplan-core[server]==0.1.46",
"govoplan-tenancy==0.1.22",
"govoplan-organizations==0.1.21",
"govoplan-identity==0.1.21",
"govoplan-idm==0.1.26",
"govoplan-access==0.1.25",
"govoplan-admin==0.1.23",
"govoplan-policy==0.1.23",
"govoplan-audit==0.1.20",
"govoplan-dashboard==0.1.20",
"govoplan-files==0.1.24",
"govoplan-mail==0.1.26",
"govoplan-campaign==0.1.27",
"govoplan-calendar==0.1.22",
"govoplan-docs==0.1.22",
"govoplan-ops==0.1.21",
"govoplan-files==0.1.27",
"govoplan-mail==0.1.28",
"govoplan-campaign==0.1.29",
"govoplan-calendar==0.1.24",
"govoplan-docs==0.1.23",
"govoplan-ops==0.1.22",
]
[project.optional-dependencies]
full = [
"govoplan-addresses==0.1.21",
"govoplan-approvals==0.1.20",
"govoplan-addresses==0.1.23",
"govoplan-approvals==0.1.21",
"govoplan-assets==0.1.20",
"govoplan-booking==0.1.20",
"govoplan-cases==0.1.22",
"govoplan-cases==0.1.25",
"govoplan-certificates==0.1.20",
"govoplan-committee==0.1.20",
"govoplan-connectors==0.1.25",
"govoplan-committee==0.1.22",
"govoplan-connectors==0.1.27",
"govoplan-consultation==0.1.20",
"govoplan-contracts==0.1.20",
"govoplan-dataflow==0.1.23",
"govoplan-datasources==0.1.24",
"govoplan-dataflow==0.1.25",
"govoplan-datasources==0.1.26",
"govoplan-decisions==0.1.19",
"govoplan-dist-lists==0.1.20",
"govoplan-dist-lists==0.1.21",
"govoplan-dms==0.1.20",
"govoplan-encryption==0.1.19",
"govoplan-encryption==0.1.20",
"govoplan-erp==0.1.20",
"govoplan-evaluation==0.1.20",
"govoplan-facilities==0.1.20",
"govoplan-fit-connect==0.1.20",
"govoplan-forms==0.1.22",
"govoplan-forms-runtime==0.1.20",
"govoplan-forms==0.1.23",
"govoplan-forms-runtime==0.1.22",
"govoplan-grants==0.1.20",
"govoplan-helpdesk==0.1.21",
"govoplan-identity-trust==0.1.20",
"govoplan-identity-trust==0.1.21",
"govoplan-inspections==0.1.20",
"govoplan-learning==0.1.20",
"govoplan-mandates==0.1.19",
"govoplan-notifications==0.1.19",
"govoplan-notifications==0.1.20",
"govoplan-parties==0.1.19",
"govoplan-payments==0.1.21",
"govoplan-payments==0.1.22",
"govoplan-permits==0.1.20",
"govoplan-poll==0.1.20",
"govoplan-portal==0.1.21",
"govoplan-postbox==0.1.22",
"govoplan-portal==0.1.22",
"govoplan-postbox==0.1.23",
"govoplan-procurement==0.1.20",
"govoplan-projects==0.1.19",
"govoplan-quick-access==0.1.20",
"govoplan-records==0.1.22",
"govoplan-reporting==0.1.20",
"govoplan-projects==0.1.20",
"govoplan-quick-access==0.1.21",
"govoplan-records==0.1.24",
"govoplan-reporting==0.1.22",
"govoplan-resources==0.1.20",
"govoplan-rest==0.1.19",
"govoplan-risk-compliance==0.1.20",
"govoplan-scheduling==0.1.21",
"govoplan-search==0.1.19",
"govoplan-risk-compliance==0.1.21",
"govoplan-scheduling==0.1.22",
"govoplan-search==0.1.20",
"govoplan-services==0.1.19",
"govoplan-soap==0.1.19",
"govoplan-tasks==0.1.21",
"govoplan-templates==0.1.21",
"govoplan-tickets==0.1.22",
"govoplan-tasks==0.1.23",
"govoplan-templates==0.1.22",
"govoplan-tickets==0.1.23",
"govoplan-transparency==0.1.20",
"govoplan-views==0.1.21",
"govoplan-voting==0.1.20",
"govoplan-views==0.1.22",
"govoplan-voting==0.1.21",
"govoplan-wiki==0.1.22",
"govoplan-workflow==0.1.22",
"govoplan-workflow==0.1.23",
"govoplan-workflow-engine==0.1.21",
"govoplan-xrechnung==0.1.21",
]
+1 -1
View File
@@ -2,7 +2,7 @@ bandit>=1.8,<2
click>=8.3.3
filelock>=3.20.3
idna>=3.15
pip>=26.1.2
pip>=26.2
pip-audit>=2.9,<3
python-multipart>=0.0.31
radon>=6,<7
+1 -1
View File
@@ -56,7 +56,7 @@ httpx2>=2.5,<3
filelock>=3.20.3
idna>=3.15
jsonschema>=4,<5
pip>=26.1.2
pip>=26.2
pip-audit>=2.9,<3
pytest>=9.0.3,<10
pygments>=2.20,<3
+13 -13
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published.
../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.20
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.20
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.20
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.24
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.23
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.22
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.22
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.26
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.24
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.26
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.27
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.22
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.22
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.21
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.27
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.28
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.29
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.24
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
+55
View File
@@ -0,0 +1,55 @@
import assert from "node:assert/strict";
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { test } from "node:test";
import { auditRepository } from "../tools/devkit/audit-display-labels.mjs";
const require = createRequire(resolve(import.meta.dirname, "../../govoplan-core/webui/package.json"));
const ts = require("typescript");
function fixture(fn) {
const root = mkdtempSync(join(tmpdir(), "govoplan-label-audit-"));
const write = (relative, content) => { const target = join(root, relative); mkdirSync(dirname(target), { recursive: true }); writeFileSync(target, content); };
try {
write("core/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Shared: "Shared" }, de: { Shared: "Gemeinsam" } };');
return fn(root, write);
} finally { rmSync(root, { recursive: true, force: true }); }
}
test("plain labels resolve owning registered catalogs, aliases, constants and Core defaults", () => fixture((root, write) => {
write("module/webui/src/i18n/generatedTranslations.ts", 'const en = { Projects: "Projects" }; const de = { Projects: "Projekte" }; export const generatedTranslations = { en, de };');
write("module/webui/src/module.ts", 'import { generatedTranslations as words } from "./i18n/generatedTranslations"; export const exampleModule: PlatformWebModule = { translations: words };');
write("module/webui/src/Page.tsx", 'import { PageLayout as Layout, PageTitle } from "@govoplan/core-webui"; const title = "Projects"; export const page = <><Layout title={title} /><PageTitle>Shared</PageTitle></>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.registration, "registered");
assert.equal(result.labels.length, 2);
assert.deepEqual(result.findings, []);
}));
test("a catalog file without module registration cannot mask untranslated titles", () => fixture((root, write) => {
write("module/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Projects: "Projects" }, de: { Projects: "Projekte" } };');
write("module/webui/src/module.ts", 'export const exampleModule = { id: "example" };');
write("module/webui/src/Page.tsx", 'import * as Core from "@govoplan/core-webui"; export const page = <Core.PageLayout title="Projects" />;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert(result.findings.some((item) => item.code === "catalog-not-registered"));
assert.deepEqual(result.labels[0].missing_locales, ["en", "de"]);
}));
test("only known static display slots fail; markers and runtime data remain separate", () => fixture((root, write) => {
write("module/webui/src/Page.tsx", 'import { PageLayout } from "@govoplan/core-webui"; export const page = <><PageLayout title="Untranslated" /><PageLayout title={campaign.name} /><PageLayout title="i18n:known.key" /><CustomThing title="Not a known slot" /></>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.findings.length, 1);
assert.equal(result.findings[0].text, "Untranslated");
assert.equal(result.review.length, 1);
assert.equal(result.review[0].code, "dynamic-display-slot");
}));
test("dynamic translation registration is reported for review, not falsely missing", () => fixture((root, write) => {
write("module/webui/src/module.ts", 'export const exampleModule: PlatformWebModule = { translations: configuredCatalog() };');
write("module/webui/src/Page.tsx", 'export const page = <PageTitle>Example</PageTitle>;');
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
assert.equal(result.registration, "dynamic");
assert.deepEqual(result.findings, []);
assert(result.review.some((item) => item.code === "dynamic-catalog-review"));
}));
+64
View File
@@ -0,0 +1,64 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { findDetachedDocumentation } from "../tools/checks/check-heading-help.mjs";
import "./test-heading-translations.mjs";
const inspect = (source) => findDetachedDocumentation([{ path: "/fixture/Page.tsx", source }]).findings;
const book = '<DocumentationHelpLink reference={topic} />';
test("heading and text contracts accept contextual help", () => {
for (const component of ["PageHeader", "PageLayout", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]) {
assert.equal(inspect(`const Page=()=> <${component} title="Topic" titleHelp={${book}} />`).length, 0, component);
}
assert.equal(inspect(`const Page=()=> <PageTitle titleHelp={${book}}>Topic</PageTitle>`).length, 0);
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}>Existing label</TextWithHelp>`).length, 0);
});
test("detached action and body links are rejected", () => {
for (const source of [`<PageActionBar helpAction={${book}} />`, `<Card title="Topic" actions={${book}} />`, `<div>${book}</div>`, `<Card title={${book}} />`]) {
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
}
});
test("aliases, conditional help and simple local references remain checked", () => {
assert.equal(inspect(`import {DocumentationHelpLink as Book, Card as Box} from '@govoplan/core-webui'; const Page=()=> <Box title="Topic" titleHelp={<Book reference={topic}/>} />`).length, 0);
assert.equal(inspect(`import {DocumentationHelpLink as Book} from '@govoplan/core-webui'; const Page=()=> <div><Book reference={topic}/></div>`).length, 1);
assert.equal(inspect(`const help=${book}; const Page=()=> <Card title="Topic" titleHelp={enabled ? help : null}/>`).length, 0);
assert.equal(inspect(`const help=${book}; const Page=()=> <><Card title="Topic" titleHelp={help}/><PageActionBar helpAction={help}/></>`).length, 1);
assert.equal(inspect(`const help=${book}; const again=help; const Page=()=> <Card title="Topic" titleHelp={again}/>`).length, 0);
});
test("empty anchors, unknown contracts and nested interactive elements fail", () => {
for (const source of [`<Card titleHelp={${book}}/>`, `<TextWithHelp help={${book}}/>`, `<TextWithHelp help={${book}}> </TextWithHelp>`, `<Unknown title="Topic" titleHelp={${book}}/>`, `<Card title="Topic" titleHelp={<button>${book}</button>}/>`]) {
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
}
});
test("namespace and explicit component default imports cannot bypass placement", () => {
for (const source of [
`import * as UI from '@govoplan/core-webui'; const Page=()=> <div><UI.DocumentationHelpLink reference={topic}/></div>`,
`import Book from './components/help/DocumentationHelpLink'; const Page=()=> <div><Book reference={topic}/></div>`,
]) assert.equal(inspect(source).length, 1, source);
assert.equal(inspect(`import * as UI from '@govoplan/core-webui'; const Page=()=> <UI.Card title="Topic" titleHelp={<UI.DocumentationHelpLink reference={topic}/>}/>`).length, 0);
assert.equal(inspect(`import Book from './components/help/DocumentationHelpLink'; import Label from './components/help/TextWithHelp'; const Page=()=> <Label help={<Book reference={topic}/>}>Topic</Label>`).length, 0);
assert.equal(findDetachedDocumentation([{ path: "/fixture/Page.tsx", source: `import Book from './business/Book'; const Page=()=> <div><Book/></div>` }]).links, 0);
});
test("outer interactive containers and statically absent text are rejected", () => {
for (const source of [
`<button><TextWithHelp help={${book}}>Topic</TextWithHelp></button>`,
`<a href="/other"><Card title="Topic" titleHelp={${book}}/></a>`,
`<Card title="" titleHelp={${book}}/>`,
`<Card title={false} titleHelp={${book}}/>`,
`<PageTitle titleHelp={${book}}>{null}</PageTitle>`,
`<TextWithHelp help={${book}}>{/* Topic */}</TextWithHelp>`,
`<TextWithHelp help={${book}}>{undefined}</TextWithHelp>`,
`<TextWithHelp help={${book}}><span hidden>Topic</span></TextWithHelp>`,
`<TextWithHelp hidden help={${book}}>Topic</TextWithHelp>`,
]) assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
assert.equal(inspect(`const help=${book}; const Page=()=> <Button><TextWithHelp help={help}>Topic</TextWithHelp></Button>`).length, 1);
assert.equal(inspect(`const title=null; const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 1);
assert.equal(inspect(`const Page=()=> <Card title={translateText(title)} titleHelp={${book}}/>`).length, 0);
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}><TranslatedTitle/></TextWithHelp>`).length, 0);
assert.equal(inspect(`let title=""; title=translateText(key); const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 0);
});
+80
View File
@@ -0,0 +1,80 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { createRequire } from "node:module";
import { resolve } from "node:path";
import { test } from "node:test";
const root = resolve(import.meta.dirname, "../..");
const require = createRequire(resolve(root, "govoplan-core/webui/package.json"));
const { buildSync } = require("esbuild");
const labels = {
approvals: { "Approval requests": "Genehmigungsanträge" },
forms: { "Form meaning": "Formularbedeutung" },
"forms-runtime": { "Form instance": "Formularinstanz", "Forms runtime": "Formularlaufzeit" },
helpdesk: { "Helpdesk profiles": "Helpdesk-Profile" },
portal: { "Service directory": "Leistungsverzeichnis" },
projects: { Projects: "Projekte" },
reporting: { Reporting: "Reporting" },
"risk-compliance": { "Risk Compliance": "Risiko und Compliance" },
scheduling: { Scheduling: "Terminplanung" },
tickets: { Tickets: "Tickets" },
voting: { Ballots: "Abstimmungen" },
wiki: { Wiki: "Wiki" },
workflow: { Workflows: "Workflows", Workflow: "Workflow" },
idm: {
"Direct changes to this governed function are": "Direkte Änderungen an dieser gesteuerten Funktion sind",
"emergency overrides": "Notfallübersteuerungen",
". Use a request or grant above for the normal process.": ". Verwenden Sie für den regulären Prozess einen Antrag oder eine Vergabe.",
},
};
// Render the actual Core heading/locale contract with one owning module at a
// time. The in-memory bundle creates no shared component-build artifacts.
const names = Object.keys(labels);
const imports = names.map((name, index) => `import { generatedTranslations as t${index} } from ${JSON.stringify(resolve(root, `govoplan-${name}/webui/src/i18n/generatedTranslations.ts`))};`).join("\n");
const { outputFiles } = buildSync({
stdin: {
contents: `${imports}
import { renderToStaticMarkup } from 'react-dom/server';
import { PlatformLanguageProvider } from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/i18n/LanguageContext.tsx"))};
import PageTitle from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/PageTitle.tsx"))};
import DocumentationHelpLink from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/help/DocumentationHelpLink.tsx"))};
export const catalogs = [${names.map((_, index) => `t${index}`).join(",")}];
export function heading(index, language, label) {
return renderToStaticMarkup(<PlatformLanguageProvider preferredLanguageCode={language} moduleTranslations={[catalogs[index]]}>
<PageTitle titleHelp={<DocumentationHelpLink reference={{ contextId: "heading-test" }} />}>{label}</PageTitle>
</PlatformLanguageProvider>);
}`,
loader: "tsx",
resolveDir: resolve(root, "govoplan-core/webui"),
},
bundle: true,
write: false,
platform: "node",
format: "cjs",
jsx: "automatic",
external: ["react", "react-dom/server"],
logLevel: "silent",
});
const compiled = { exports: {} };
new Function("require", "module", "exports", outputFiles[0].text)(require, compiled, compiled.exports);
const { catalogs, heading } = compiled.exports;
test("new contextual labels render from their owning EN/DE catalogues", async (context) => {
for (const [index, name] of names.entries()) {
await context.test(name, () => {
const moduleSource = readFileSync(resolve(root, `govoplan-${name}/webui/src/module.ts`), "utf8");
assert.match(moduleSource, /import\s*\{\s*generatedTranslations\s*\}\s*from\s*["']\.\/i18n\/generatedTranslations["']/);
assert.match(moduleSource, /\btranslations(?:\s*:\s*generatedTranslations)?\s*,/);
for (const [english, german] of Object.entries(labels[name])) {
assert.equal(catalogs[index].en[english], english);
assert.equal(catalogs[index].de[english], german);
for (const [language, expected] of [["en", english], ["de", german]]) {
const markup = heading(index, language, english);
assert(markup.includes(`<h1>${expected}</h1>`), `${name}: ${language} contextual heading`);
assert(markup.includes(language === "de" ? "Benutzerdokumentation öffnen" : "Open user documentation"));
}
}
});
}
});
+15
View File
@@ -0,0 +1,15 @@
import assert from "node:assert/strict";
import { findTypeOnlyJsxImports } from "../tools/checks/check-jsx-value-imports.mjs";
const findings = (source) => findTypeOnlyJsxImports([{ path: "/fixture.tsx", source }]).map((item) => item.component);
assert.deepEqual(findings('import type { FormGrid, AuthInfo } from "@govoplan/core-webui"; const page = <Dialog><FormGrid /></Dialog>;'), ["FormGrid"]);
assert.deepEqual(findings('import { type FormGrid as Layout } from "ui"; const page = <Layout>Content</Layout>;'), ["Layout"]);
assert.deepEqual(findings('import type Layout from "ui"; const page = <Layout />;'), ["Layout"]);
assert.deepEqual(findings('import type * as ui from "ui"; const page = <ui.Layout />;'), ["ui.Layout"]);
assert.deepEqual(findings('import type { FormGrid } from "ui"; const page = <div title={<FormGrid />} />;'), ["FormGrid"]);
assert.deepEqual(findings('import { FormGrid, type AuthInfo } from "ui"; const page = <FormGrid />;'), []);
assert.deepEqual(findings('import type { FormGrid } from "ui"; function Page({ FormGrid }: Props) { return <FormGrid />; }'), []);
assert.deepEqual(findings('import type * as ui from "ui"; function Page(ui: RuntimeControls) { return <ui.Layout />; }'), []);
assert.deepEqual(findings('import type { Layout } from "ui"; const page: Layout = {};'), []);
assert.deepEqual(findings('import type { input } from "ui"; const page = <input />;'), []);
console.log("JSX runtime-import AST regression tests passed (10 cases).");
+23
View File
@@ -780,6 +780,11 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertIn("redis", compose["services"])
self.assertIn("worker", compose["services"])
self.assertIn("load-balancer", compose["services"])
self.assertEqual(
"haproxy:3.2.23-alpine@sha256:"
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
compose["services"]["load-balancer"]["image"],
)
self.assertNotIn("test-mail", compose["services"])
self.assertEqual(
["127.0.0.1:8080:8080"],
@@ -1145,8 +1150,26 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertEqual(1, parsed.replicas.web)
self.assertEqual(1, parsed.replicas.worker)
self.assertEqual("managed", parsed.components.load_balancer.mode)
self.assertEqual(
default_spec().components.load_balancer.image,
parsed.components.load_balancer.image,
)
self.assertEqual("local", parsed.ingress.mode)
def test_load_balancer_patch_does_not_rewrite_an_existing_image(self) -> None:
for image in (
"haproxy:3.2.21-alpine",
"registry.example.test/haproxy@sha256:" + "a" * 64,
):
with self.subTest(image=image):
saved = default_spec(load_balancer_image=image).to_dict()
restored = parse_spec(json.loads(json.dumps(saved)))
compose = render_compose(restored)
self.assertEqual(image, restored.components.load_balancer.image)
self.assertEqual(image, compose["services"]["load-balancer"]["image"])
def test_compose_contains_no_secret_values(self) -> None:
spec = default_spec()
values = initial_secrets(spec)
+438
View File
@@ -0,0 +1,438 @@
"""Audit the selected fixture workspace, never a fuller neighboring checkout."""
from argparse import Namespace
import importlib.util
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
from types import ModuleType, SimpleNamespace
import pytest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools/inventory/platform-interface-inventory.py"
WEBUI_SCRIPT = META_ROOT / "tools/inventory/extract-webui-structure.mjs"
SPEC = importlib.util.spec_from_file_location("audit_scope_inventory", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
inventory = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(inventory)
sys.path.insert(0, str(META_ROOT / "tools/devkit"))
from govoplan_devkit import docs, issues, runner # noqa: E402
from govoplan_devkit.workspace import Project, Repository # noqa: E402
def write(path, content):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
@pytest.fixture
def workspaces(tmp_path):
selected = tmp_path / "selected"
default = tmp_path / "fuller-default"
meta = tmp_path / "legacy-siblings/govoplan"
selected.mkdir()
meta.mkdir(parents=True)
catalog = {
"default_parent": str(default),
"repositories": [
{"name": "govoplan-core", "path": "govoplan-core"},
{"name": "govoplan-example", "path": "nested/example"},
{"name": "govoplan-optional", "path": "govoplan-optional"},
],
}
write(meta / "repositories.json", json.dumps(catalog))
write(
selected / "nested/example/webui/src/Page.tsx",
"export const page = <h1>SELECTED_ONLY</h1>;\n",
)
write(
default / "nested/example/webui/src/Page.tsx",
"export const page = <h1>DEFAULT_ONLY</h1>;\n",
)
write(
default / "govoplan-optional/webui/src/Page.tsx",
"export const page = <h1>DEFAULT_OPTIONAL</h1>;\n",
)
for item in catalog["repositories"]:
(default / item["path"] / "src").mkdir(parents=True)
return selected, default, meta, catalog
def test_explicit_partial_root_beats_fuller_legacy_discovery(workspaces, monkeypatch):
selected, default, meta, catalog = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
assert inventory._resolve_workspace_root(catalog) == default
assert inventory._resolve_workspace_root(catalog, selected) == selected
with pytest.raises(ValueError, match="existing directory"):
inventory._resolve_workspace_root(catalog, selected / "missing")
@pytest.mark.parametrize("relative", ["../fuller-default", "/absolute/path"])
def test_repository_paths_cannot_escape_explicit_root(workspaces, relative):
selected, _, _, _ = workspaces
with pytest.raises(ValueError, match="inside the selected workspace"):
inventory._validate_repository_roots(
{"repositories": [{"path": relative}]}, selected
)
def test_linked_repository_cannot_borrow_default_sources(workspaces):
selected, default, _, catalog = workspaces
(selected / "govoplan-core").symlink_to(
default / "govoplan-core", target_is_directory=True
)
with pytest.raises(ValueError, match="escapes the selected workspace"):
inventory._validate_repository_roots(catalog, selected)
def test_python_forwards_selected_root_and_configured_node(workspaces, monkeypatch):
selected, _, meta, _ = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
monkeypatch.setenv("NODE", "/selected/toolchain/node")
calls = []
def run(argv, **kwargs):
calls.append((argv, kwargs))
return SimpleNamespace(stdout=json.dumps({"workspaceRoot": str(selected)}))
monkeypatch.setattr(inventory.subprocess, "run", run)
assert inventory._extract_webui(selected)["workspaceRoot"] == str(selected)
argv, options = calls[0]
assert argv == [
"/selected/toolchain/node",
str(meta / "tools/inventory/extract-webui-structure.mjs"),
str(meta),
"--workspace-root",
str(selected),
]
assert options == {"check": True, "capture_output": True, "text": True}
def test_python_rejects_webui_result_from_another_root(workspaces, monkeypatch):
selected, default, _, _ = workspaces
monkeypatch.setattr(
inventory.subprocess,
"run",
lambda *a, **k: SimpleNamespace(
stdout=json.dumps({"workspaceRoot": str(default)})
),
)
with pytest.raises(ValueError, match="did not confirm"):
inventory._extract_webui(selected)
def test_main_forwards_same_root_to_every_collector(workspaces, monkeypatch):
selected, _, meta, catalog = workspaces
monkeypatch.setattr(inventory, "META_ROOT", meta)
roots = []
monkeypatch.setattr(
inventory, "_extract_webui", lambda root: roots.append(root) or {}
)
monkeypatch.setattr(
inventory,
"_extract_backend_endpoints",
lambda data, root: roots.append(root) or [],
)
monkeypatch.setattr(
inventory, "_extract_manifests", lambda data, root: roots.append(root) or []
)
monkeypatch.setattr(inventory, "_load_endpoint_declarations", lambda _: {})
monkeypatch.setattr(inventory, "_load_high_risk_help_baseline", lambda _: {})
monkeypatch.setattr(inventory, "_assemble_inventory", lambda **kwargs: {})
monkeypatch.setattr(inventory, "_render_markdown", lambda data: "fixture\n")
output = selected / "output"
monkeypatch.setattr(
sys,
"argv",
[str(SCRIPT), "--workspace-root", str(selected), "--output-dir", str(output)],
)
assert inventory.main() == 0
assert roots == [selected, selected, selected]
report = json.loads((output / "platform-interface-inventory.json").read_text())
assert report["workspace_root"] == str(selected)
assert report["workspace_selection"] == "explicit"
def install_fixture_compiler(root):
compiler = META_ROOT.parent / "govoplan-core/webui/node_modules/typescript"
if not compiler.is_dir() or not shutil.which("node"):
pytest.skip("Node and the installed Core TypeScript parser are required")
target = root / "govoplan-core/webui/node_modules/typescript"
target.parent.mkdir(parents=True, exist_ok=True)
# Parser dependencies may be shared; audited source checkouts may not.
target.symlink_to(compiler, target_is_directory=True)
def collect_webui(meta, selected=None):
argv = [shutil.which("node") or "node", str(WEBUI_SCRIPT), str(meta)]
if selected is not None:
argv += ["--workspace-root", str(selected)]
return subprocess.run(argv, capture_output=True, text=True, timeout=30)
def test_javascript_explicit_root_is_authoritative_and_legacy_cli_still_works(
workspaces,
):
selected, default, meta, _ = workspaces
install_fixture_compiler(selected)
install_fixture_compiler(default)
explicit = collect_webui(meta, selected)
assert explicit.returncode == 0, explicit.stderr
report = json.loads(explicit.stdout)
assert report["workspaceRoot"] == str(selected)
assert [item["value"] for item in report["visibleText"]] == ["SELECTED_ONLY"]
assert report["visibleText"][0]["file"] == "webui/src/Page.tsx"
legacy = collect_webui(meta)
assert legacy.returncode == 0, legacy.stderr
assert json.loads(legacy.stdout)["workspaceRoot"] == str(default)
assert {item["value"] for item in json.loads(legacy.stdout)["visibleText"]} == {
"DEFAULT_ONLY",
"DEFAULT_OPTIONAL",
}
def test_javascript_does_not_borrow_missing_core_dependencies(workspaces):
selected, default, meta, _ = workspaces
install_fixture_compiler(default)
result = collect_webui(meta, selected)
assert result.returncode != 0
assert (
str(selected / "govoplan-core/webui/node_modules/typescript") in result.stderr
)
assert not result.stdout
def test_javascript_rejects_source_root_linked_outside_workspace(workspaces):
selected, default, meta, _ = workspaces
install_fixture_compiler(selected)
linked = selected / "govoplan-optional/webui/src"
linked.parent.mkdir(parents=True)
linked.symlink_to(default / "govoplan-optional/webui/src", target_is_directory=True)
result = collect_webui(meta, selected)
assert result.returncode != 0
assert "source root escapes the selected workspace" in result.stderr
def test_backend_does_not_borrow_optional_endpoints(workspaces):
selected, default, _, catalog = workspaces
source = "from fastapi import APIRouter\nrouter = APIRouter()\n@router.get('/selected')\ndef endpoint(): pass\n"
write(selected / "nested/example/src/routes.py", source)
write(
default / "govoplan-optional/src/routes.py", source.replace("selected", "other")
)
endpoints = inventory._extract_backend_endpoints(catalog, selected)
assert [item["path"] for item in endpoints] == ["/selected"]
(selected / "nested/example/src/foreign.py").symlink_to(
default / "govoplan-optional/src/routes.py"
)
with pytest.raises(ValueError, match="Backend source path escapes"):
inventory._extract_backend_endpoints(catalog, selected)
def test_manifests_require_selected_core_sources(workspaces, monkeypatch):
selected, default, _, catalog = workspaces
write(
default / "govoplan-core/src/govoplan_core/core/platform_interfaces.py",
"raise AssertionError('foreign source imported')",
)
monkeypatch.syspath_prepend(str(default / "govoplan-core/src"))
with pytest.raises(ValueError, match="requires Core interface sources"):
inventory._extract_manifests(catalog, selected)
def test_cached_application_import_cannot_replace_missing_checkout(
workspaces, monkeypatch
):
selected, default, _, _ = workspaces
# Meta's own devkit/release packages may audit another workspace.
tooling = ModuleType("govoplan_release")
tooling.__file__ = str(default / "tools/release/govoplan_release/__init__.py")
modules = {"govoplan_release": tooling, "govoplan_devkit.docs": docs}
# Isolate the fixture from application packages collected by unrelated
# suites, without removing or replacing those real cached imports.
monkeypatch.setattr(inventory, "sys", SimpleNamespace(modules=modules))
inventory._assert_workspace_imports(selected)
application = ModuleType("govoplan_audit_fixture")
application.__file__ = str(
default / "govoplan-optional/src/govoplan_audit_fixture/__init__.py"
)
modules["govoplan_audit_fixture"] = application
with pytest.raises(ValueError, match="outside the selected inventory workspace"):
inventory._assert_workspace_imports(selected)
def test_partial_manifest_collection_uses_selected_sources_in_fresh_process(workspaces):
selected, default, _, catalog = workspaces
core = selected / "govoplan-core/src/govoplan_core"
write(core / "__init__.py", "")
write(core / "core/__init__.py", "")
write(
core / "core/platform_interfaces.py",
"def manifest_interface_catalog(manifest): return {'selected': True}\n",
)
write(
default / "govoplan-core/src/govoplan_core/__init__.py",
"raise AssertionError('foreign Core loaded')\n",
)
module = selected / "nested/example/src/govoplan_example"
write(module / "__init__.py", "")
write(module / "backend/__init__.py", "")
manifest = (
"from types import SimpleNamespace as S\n"
"def get_manifest():\n"
" return S(id='selected', name='Selected', version='1', dependencies=(), "
"optional_dependencies=(), required_capabilities=(), provides_interfaces=(), "
"capability_factories={}, permissions=(), documentation=(), architecture=None, "
"information_governance=S(to_dict=lambda: {}), frontend=None)\n"
)
write(module / "backend/manifest.py", manifest)
write(
default / "govoplan-optional/src/govoplan_optional/backend/manifest.py",
manifest.replace("selected", "foreign"),
)
code = (
"import importlib.util, json; from pathlib import Path; "
f"spec=importlib.util.spec_from_file_location('fixture', {str(SCRIPT)!r}); "
"module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module); "
f"print(json.dumps(module._extract_manifests({catalog!r}, Path({str(selected)!r}))))"
)
env = {
**os.environ,
"PYTHONPATH": os.pathsep.join(
str(default / item["path"] / "src") for item in catalog["repositories"]
),
}
result = subprocess.run(
[sys.executable, "-c", code],
env=env,
capture_output=True,
text=True,
timeout=30,
)
assert result.returncode == 0, result.stderr
manifests = json.loads(result.stdout)
assert [manifest["id"] for manifest in manifests] == ["selected"]
assert manifests[0]["interface_catalog"] == {"selected": True}
def doc_plan(workspace, monkeypatch):
repos = tuple(
Repository(name, workspace / name)
for name in ("govoplan", "govoplan-core", "govoplan-example")
)
project = Project("fixture", repos, {})
monkeypatch.setattr(docs, "load_project", lambda *a: project)
args = Namespace(
workspace_root=workspace,
project=None,
state_dir=workspace.parent / "state",
repo=["govoplan-example"],
changed=False,
)
return docs.build_doc_stages(args)
def test_docs_plan_forwards_root_and_persists_all_limitations(workspaces, monkeypatch):
selected, _, _, _ = workspaces
stages = doc_plan(selected, monkeypatch)
by_id = {stage["id"]: stage for stage in stages}
for stage_id in (
"docs.manifests",
"docs.interface-inventory",
"docs.plain-display-labels",
):
argv = by_id[stage_id]["argv"]
assert argv[argv.index("--workspace-root") + 1] == str(selected)
assert by_id["docs.translation-structure"]["argv"][1] == str(
selected / "govoplan-core/webui/scripts/audit-i18n-structural.mjs"
)
assert issues.coverage_notes(stages) == docs.LIMITATIONS
assert by_id["docs.plain-display-labels"]["argv"][-2:] == [
"--repo",
"govoplan-example",
]
def test_docs_limitations_survive_real_receipt_and_issue_evidence(
workspaces, monkeypatch, tmp_path
):
selected, _, _, _ = workspaces
stages = doc_plan(selected, monkeypatch)
repo = selected / "example"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True, timeout=10)
write(repo / "source.txt", "fixture\n")
subprocess.run(
["git", "-C", str(repo), "add", "source.txt"], check=True, timeout=10
)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
timeout=10,
)
project = tmp_path / "project.json"
write(
project,
json.dumps(
{
"schema_version": 1,
"name": "fixture",
"repositories": [{"name": "example", "path": "example"}],
"checks": [],
"profiles": {},
}
),
)
args = Namespace(
workspace_root=selected,
project=project,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="docs",
resume=None,
)
for stage in stages:
stage.update(
argv=[sys.executable, "-c", "print('fixture audit')"],
cwd=str(repo),
deps=[],
resources=[],
timeout_seconds=10,
)
monkeypatch.setattr(docs, "build_doc_stages", lambda _: stages)
monkeypatch.setattr(
runner, "environment_fingerprint", lambda *a, **k: "fixture-env"
)
result = docs.audit(args)
assert result["status"] == "passed", result
assert all(result["summary"].count(note) == 1 for note in docs.LIMITATIONS)
receipt = runner.read_receipt(selected, args.state_dir, result["run_id"])
assert issues.coverage_notes(receipt["stages"]) == docs.LIMITATIONS
evidence = issues.evidence_record(result["run_id"], args)
assert evidence["coverage_notes"] == docs.LIMITATIONS
assert evidence["source_state"] == "matches-current"
target = issues.NoteTarget(
repo, "https://gitea.example.invalid", "fixture", "example", 1
)
_, body = issues.render_note(
{"summary": [], "next": [], "body": ""}, evidence, target, "fixture"
)
assert all(note in body for note in docs.LIMITATIONS)
+88
View File
@@ -0,0 +1,88 @@
"""Signals during snapshot probes must not become passing run evidence."""
import os
import signal
import pytest
import test_devkit_runner as fixtures
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
example = fixtures.example
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
@pytest.mark.parametrize("probe", ["source", "environment"])
def test_interrupt_during_final_snapshot_is_not_a_passing_run(
example, monkeypatch, interruption, probe
):
args, repo = example
calls = {"source": 0, "environment": 0}
events, interruptions = [], []
args.on_progress = events.append
original = Checkpoints.source
def reached(kind):
calls[kind] += 1
if kind == probe and events and events[-1]["phase"] == "finalizing":
# Intermediate checkpoint probes are also real verification. Target
# finalization explicitly, while the runner owns signal handlers.
interruptions.append(kind)
os.kill(os.getpid(), interruption)
def source(self, *values, **kwargs):
reached("source")
return original(self, *values, **kwargs)
def environment(*_args, **_kwargs):
reached("environment")
return "fixture-environment"
monkeypatch.setattr(Checkpoints, "source", source)
monkeypatch.setattr(runner, "environment_fingerprint", environment)
result = runner.run_checks(args, [fixtures.stage(repo)])
assert interruptions == [probe]
assert calls[probe] >= 2
assert result["stages"][0]["status"] == "passed"
assert result["status"] == "interrupted"
assert result["_exit_code"] != 0
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["status"] == "interrupted"
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
@pytest.mark.parametrize("probe", ["source", "environment"])
def test_preparation_interrupt_stops_at_probe_boundary_without_running_checks(
example, monkeypatch, interruption, probe
):
args, repo = example
calls = []
original = Checkpoints.source
def reached(kind):
calls.append(kind)
if kind == probe:
os.kill(os.getpid(), interruption)
def source(self, *values, **kwargs):
reached("source")
return original(self, *values, **kwargs)
def environment(*_args, **_kwargs):
reached("environment")
return "fixture-environment"
def must_not_execute(*args, **kwargs):
pytest.fail("A cancelled preparation must not start a check")
monkeypatch.setattr(Checkpoints, "source", source)
monkeypatch.setattr(runner, "environment_fingerprint", environment)
monkeypatch.setattr(runner, "execute_stage", must_not_execute)
result = runner.run_checks(args, [fixtures.stage(repo)])
assert calls == (["source"] if probe == "source" else ["source", "environment"])
assert result["status"] == "interrupted"
assert result["snapshot_verified"] is False
assert result["_exit_code"] != 0
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["status"] == "interrupted"
+451
View File
@@ -0,0 +1,451 @@
"""Bounded planning fixtures: no application imports, compilers or servers."""
from __future__ import annotations
import argparse
import json
import shutil
from pathlib import Path
import sys
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.catalog import (
_expanded_repositories,
build_stages,
module_ui_stages,
)
from govoplan_devkit.docs import build_doc_stages
from govoplan_devkit.workspace import Project, Repository
class CatalogTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-catalog-")
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.core = Repository("govoplan-core", self.root / "govoplan-core", ("core",))
self.meta = Repository("govoplan", self.root / "govoplan", ("meta",))
self.module = Repository(
"govoplan-example", self.root / "govoplan-example", ("example",)
)
self.project = Project("fixture", (self.meta, self.core, self.module), {})
for repo in self.project.repositories:
repo.path.mkdir()
(self.meta.path / "tools/checks").mkdir(parents=True)
shutil.copyfile(
Path(__file__).resolve().parents[1] / "tools/checks/focused-phases.json",
self.meta.path / "tools/checks/focused-phases.json",
)
def write(self, relative, content):
path = self.root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
def plan(self, profile, repos=None, changed=False):
with patch("govoplan_devkit.workspace.load_project", return_value=self.project):
return build_stages(self.root, profile, repos or [], changed)
def test_ui_compiles_once_and_quick_does_not_compile(self):
self.write(
"govoplan-core/webui/package.json",
json.dumps(
{"scripts": {"test:components": "node scripts/run-component-tests.mjs"}}
),
)
self.write("govoplan-core/webui/scripts/run-component-tests.mjs", "// fixture")
self.assertFalse(
any(
"run-component-tests.mjs" in " ".join(item["argv"])
for item in self.plan("quick")
)
)
matches = [
item
for item in self.plan("ui")
if "run-component-tests.mjs" in " ".join(item["argv"])
]
self.assertEqual(len(matches), 1)
self.assertEqual(matches[0]["id"], "core.component-batch")
def test_module_script_metadata_is_bounded_and_deduplicated(self):
self.write(
"govoplan-example/webui/package.json",
json.dumps(
{
"scripts": {
"test:interface-pattern": "node scripts/test-interface-pattern-language.mjs",
"test:source": "node --test tests/source.test.mjs",
"test:dangerous-chain": "node tests/source.test.mjs && npm run dev",
"test:flags": "node --eval 'startServer()'",
}
}
),
)
self.write(
"govoplan-example/webui/scripts/test-interface-pattern-language.mjs",
"// source-only fixture",
)
self.write(
"govoplan-example/webui/tests/source.test.mjs", "// source-only fixture"
)
stages = module_ui_stages(self.module, reason="fixture")
self.assertEqual(len(stages), 2)
self.assertEqual(len({item["id"] for item in stages}), 2)
self.assertTrue(all(item["argv"][0] == "{node}" for item in stages))
self.assertFalse(any("&&" in item["argv"] for item in stages))
def test_full_is_never_narrowed_by_repo_filter(self):
result = self.plan("full", ["example"])
self.assertEqual(
[item["id"] for item in result],
[
"focused." + identity
for identity in (
"preflight",
"tooling",
"backend",
"core-ui",
"module-builds",
"browser",
"module-ui",
)
],
)
self.assertEqual(
[item["after"] for item in result],
[[], *[[item["id"]] for item in result[:-1]]],
)
self.assertTrue(all(item["deps"] == [] for item in result))
self.assertTrue(
all(
item["argv"]
== [
"bash",
str(self.meta.path / "tools/checks/check-focused.sh"),
"--phase",
item["id"].removeprefix("focused."),
]
for item in result
)
)
self.assertIn("webui:govoplan-example", result[-1]["resources"])
self.assertIn("backend:test-state", result[0]["resources"])
def test_changed_empty_is_not_full_verification(self):
with (
patch("govoplan_devkit.workspace.load_project", return_value=self.project),
patch("govoplan_devkit.workspace.selected_repositories", return_value=[]),
):
self.assertEqual(build_stages(self.root, "quick", [], True), [])
self.assertEqual(len(build_stages(self.root, "full", [], True)), 7)
def test_full_ui_scope_excludes_backend_only_but_includes_all_ui_owners(self):
backend = Repository(
"govoplan-backend-only", self.root / "govoplan-backend-only"
)
backend.path.mkdir()
(self.module.path / "webui").mkdir()
(self.core.path / "webui").mkdir()
project = Project("fixture", (*self.project.repositories, backend), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "full", ["example"], False)
for check in checks[:3]:
self.assertNotIn("inputs", check)
for check in checks[3:]:
self.assertEqual(
check["inputs"]["repos"],
["govoplan", "govoplan-core", "govoplan-example"],
)
(backend.path / "webui").mkdir()
with patch("govoplan_devkit.workspace.load_project", return_value=project):
replanned = build_stages(self.root, "full", [], False)
self.assertIn(backend.name, replanned[3]["inputs"]["repos"])
def test_full_missing_checkout_explicitly_falls_back_to_workspace_inputs(self):
missing = Repository("govoplan-missing", self.root / "govoplan-missing")
project = Project("fixture", (*self.project.repositories, missing), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "full", [], False)
self.assertTrue(all("inputs" not in check for check in checks))
self.assertTrue(
all(
any("workspace-wide" in note for note in check["coverage_notes"])
for check in checks[3:]
)
)
def test_full_requires_authoritative_phase_metadata(self):
(self.meta.path / "tools/checks/focused-phases.json").write_text("{}")
with self.assertRaisesRegex(ValueError, "phase metadata"):
self.plan("full")
def test_native_ui_guards_and_component_batch_share_safe_ui_owner_scope(self):
backend = Repository(
"govoplan-backend-only", self.root / "govoplan-backend-only"
)
backend.path.mkdir()
(self.module.path / "webui").mkdir()
(self.core.path / "webui").mkdir()
project = Project("fixture", (*self.project.repositories, backend), {})
with patch("govoplan_devkit.workspace.load_project", return_value=project):
checks = build_stages(self.root, "ui", ["example"], False)
for check in checks:
if check["id"] in {
"jsx-value-imports",
"heading-help",
"core.component-batch",
}:
self.assertEqual(
check["inputs"]["repos"],
["govoplan", "govoplan-core", "govoplan-example"],
)
else:
self.assertNotIn("inputs", check)
def test_unregistered_src_or_webui_disables_all_native_reuse(self):
for directory in ("src", "webui"):
with self.subTest(directory=directory):
unknown = self.root / "govoplan-unregistered" / directory
unknown.mkdir(parents=True)
try:
for profile in ("quick", "ui", "backend", "full"):
checks = self.plan(profile)
self.assertTrue(checks)
self.assertTrue(
all(check["reuse"] == "never" for check in checks)
)
self.assertTrue(
all(
any(
"Unregistered sibling" in note
for note in check["coverage_notes"]
)
for check in checks
)
)
finally:
unknown.rmdir()
unknown.parent.rmdir()
(self.root / "govoplan-unused-empty").mkdir()
self.assertTrue(all("reuse" not in check for check in self.plan("full")))
def test_unregistered_broken_source_symlink_does_not_allow_reuse(self):
unknown = self.root / "govoplan-unregistered"
unknown.mkdir()
(unknown / "src").symlink_to(self.root / "missing")
self.assertTrue(all(check["reuse"] == "never" for check in self.plan("full")))
def test_unknown_repo_is_not_silently_ignored(self):
with self.assertRaisesRegex(ValueError, "Unknown repository"):
self.plan("ui", ["not-a-repo"])
def test_changed_provider_selects_transitive_declared_consumers(self):
other = Repository("govoplan-other", self.root / "govoplan-other")
final = Repository("govoplan-final", self.root / "govoplan-final")
project = Project("fixture", (self.meta, self.module, other, final), {})
self.write("govoplan/tools/release/govoplan_release/contracts.py", "# fixture")
for name in ("example", "other", "final"):
self.write(f"govoplan-{name}/src/fixture/backend/manifest.py", "# fixture")
def contract(_path, repo_name):
gives = {
self.module.name: ["first"],
other.name: ["second"],
final.name: [],
}[repo_name]
needs = {
self.module.name: [],
other.name: ["first"],
final.name: ["second"],
}[repo_name]
return SimpleNamespace(
repo=repo_name,
provides_interfaces=[SimpleNamespace(name=value) for value in gives],
requires_interfaces=[SimpleNamespace(name=value) for value in needs],
)
with patch.dict(
sys.modules,
{
"govoplan_release.contracts": SimpleNamespace(
parse_manifest_contract=contract
)
},
):
selected, reason = _expanded_repositories(
project, [self.module], changed=True
)
self.assertEqual(
[repo.name for repo in selected], [self.module.name, other.name, final.name]
)
self.assertIn("declared interface consumers", reason)
def test_generic_dependency_closure_includes_filtered_prerequisites(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [
{"name": "one", "path": "one"},
{"name": "two", "path": "two"},
],
"checks": [
{
"id": "compile",
"argv": ["{node}", "compile.mjs"],
"cwd": "one",
"repos": ["two"],
},
{
"id": "test",
"argv": ["{node}", "test.mjs"],
"cwd": "one",
"deps": ["compile"],
"repos": ["one"],
},
{
"id": "other",
"argv": ["{python}", "test.py"],
"cwd": "two",
"repos": ["two"],
},
],
"profiles": {"quick": ["test", "other"]},
}
self.write("project.json", json.dumps(config))
result = build_stages(
self.root, "quick", ["one"], False, self.root / "project.json"
)
self.assertEqual([item["id"] for item in result], ["compile", "test"])
self.assertEqual(result[1]["deps"], ["compile"])
self.assertEqual(result[1]["cwd"], str(self.root / "one"))
def test_generic_cycles_and_duplicate_ids_fail(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [{"name": "one", "path": "one"}],
"checks": [
{"id": "one", "argv": ["test"], "deps": ["two"]},
{"id": "two", "argv": ["test"], "deps": ["one"]},
],
"profiles": {"quick": ["one"]},
}
self.write("project.json", json.dumps(config))
with self.assertRaisesRegex(ValueError, "Cyclic"):
build_stages(self.root, "quick", [], False, self.root / "project.json")
config["checks"][1]["id"] = "one"
self.write("project.json", json.dumps(config))
with self.assertRaisesRegex(ValueError, "Duplicate"):
build_stages(self.root, "quick", [], False, self.root / "project.json")
def test_generic_order_prerequisites_inputs_and_reuse_survive_planning(self):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [
{"name": "one", "path": "one"},
{"name": "two", "path": "two"},
],
"checks": [
{
"id": "prepare",
"argv": ["prepare"],
"repos": ["two"],
"reuse": "never",
"inputs": {"repos": ["two"]},
},
{
"id": "test",
"argv": ["test"],
"repos": ["one"],
"after": ["prepare"],
"reuse": "verified",
"inputs": {"repos": ["one"]},
},
{"id": "broad", "argv": ["check"], "repos": ["one"]},
],
"profiles": {"quick": ["test", "broad"]},
}
self.write("project.json", json.dumps(config))
result = build_stages(
self.root, "quick", ["one"], False, self.root / "project.json"
)
self.assertEqual([item["id"] for item in result], ["prepare", "test", "broad"])
self.assertEqual(result[1]["after"], ["prepare"])
self.assertEqual(result[1]["deps"], [])
self.assertEqual(result[0]["reuse"], "never")
self.assertEqual(result[1]["reuse"], "verified")
self.assertEqual(result[0]["inputs"], {"repos": ["two"]})
self.assertEqual(result[1]["inputs"], {"repos": ["one"]})
self.assertNotIn("inputs", result[2])
def test_generic_original_types_are_validated_before_coercion(self):
for field, invalid in (
("argv", "false"),
("argv", None),
("resources", "shared"),
("cwd", None),
("cwd", "../outside"),
("title", 5),
("timeout_seconds", True),
):
with self.subTest(field=field, invalid=invalid):
config = {
"schema_version": 1,
"name": "generic",
"repositories": [{"name": "one", "path": "one"}],
"checks": [{"id": "test", "argv": ["test"], field: invalid}],
"profiles": {"quick": ["test"]},
}
self.write("project.json", json.dumps(config))
with self.assertRaises(ValueError):
build_stages(
self.root, "quick", [], False, self.root / "project.json"
)
def test_ui_coverage_notes_make_excluded_suites_explicit_and_discover_tests_folder(
self,
):
self.write(
"govoplan-example/webui/package.json",
json.dumps({"scripts": {"test:full-ui": "tsc && node tests/full-ui.js"}}),
)
self.write(
"govoplan-example/webui/tests/aggregate-report-structure.test.mjs",
"// fixture",
)
stages = self.plan("ui", ["example"])
self.assertTrue(
any("aggregate-report-structure" in item["id"] for item in stages)
)
self.assertTrue(
any("test:full-ui" in note for note in stages[0]["coverage_notes"])
)
def test_docs_reuses_existing_guards_and_only_narrows_plain_labels(self):
args = argparse.Namespace(
workspace_root=self.root,
state_dir=self.root / "state",
project=None,
repo=["example"],
changed=False,
)
with patch("govoplan_devkit.docs.load_project", return_value=self.project):
stages = build_doc_stages(args)
self.assertEqual(len(stages), 4)
self.assertIn("check-manifest-shapes.py", " ".join(stages[0]["argv"]))
self.assertIn("platform-interface-inventory.py", " ".join(stages[1]["argv"]))
self.assertIn("--strict-declarations", stages[1]["argv"])
self.assertEqual(stages[3]["argv"][-2:], ["--repo", "govoplan-example"])
self.assertFalse(
(self.root / "state").exists(), "Planning must not create artifacts"
)
if __name__ == "__main__":
unittest.main()
+158
View File
@@ -0,0 +1,158 @@
"""Public CLI contracts and an executable portable-project smoke fixture."""
import json
from pathlib import Path
import subprocess
import sys
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import cli, doctor, runner
from govoplan_devkit.common import META_ROOT
@pytest.mark.parametrize(
"arguments",
[["--json", "commands"], ["commands", "--json"], ["commands", "--format", "json"]],
)
def test_global_output_flags_work_on_either_side_of_command(arguments, capsys):
assert cli.main(arguments) == 0
output = json.loads(capsys.readouterr().out)
names = {item["command"] for item in output["commands"]}
assert {"context", "check", "doctor", "docs", "issues", "release", "git"} <= names
@pytest.mark.parametrize(
"command",
[
"context",
"doctor",
"check",
"resume",
"recover",
"docs",
"review",
"issues",
"release",
"git",
],
)
def test_command_help_does_not_require_live_services(command, capsys):
with pytest.raises(SystemExit) as stopped:
cli.main([command, "--help"])
assert stopped.value.code == 0
assert "usage:" in capsys.readouterr().out
def test_portable_project_executes_registered_commands_and_reads_receipt(
tmp_path, capsys, monkeypatch
):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "state"))
repo = tmp_path / "example"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"--allow-empty",
"-qm",
"fixture",
],
check=True,
)
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Portable",
"repositories": [{"name": "example", "path": "example"}],
"checks": [
{
"id": "test",
"argv": ["{python}", "-c", "print('portable ok')"],
"cwd": "example",
}
],
"profiles": {"quick": ["test"]},
}
)
)
common = ["--workspace-root", str(tmp_path), "--project", str(project), "--json"]
with patch.object(runner, "environment_fingerprint", return_value="fixture"):
assert cli.main(common + ["check", "--profile", "quick"]) == 0
result = json.loads(capsys.readouterr().out)
assert result["status"] == "passed"
assert cli.main(["status", result["run_id"], *common]) == 0
status = json.loads(capsys.readouterr().out)
assert status["snapshot_verified"] is True
assert cli.main(["logs", result["run_id"], "--stage", "test", *common]) == 0
assert "portable ok" in capsys.readouterr().out
def test_portable_example_matches_published_schema():
import jsonschema
jsonschema.validate(
json.loads((META_ROOT / "tools/devkit/examples/project.json").read_text()),
json.loads((META_ROOT / "tools/devkit/project.schema.json").read_text()),
)
def test_malformed_project_is_a_controlled_json_error(tmp_path, capsys):
project = tmp_path / "bad.json"
project.write_text('{"schema_version":true,"repositories":[]}')
assert (
cli.main(
[
"context",
"--workspace-root",
str(tmp_path),
"--project",
str(project),
"--json",
]
)
== 2
)
assert json.loads(capsys.readouterr().out)["status"] == "error"
def test_doctor_is_read_only_and_preserves_dependency_warnings(tmp_path, capsys):
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{"schema_version": 1, "repositories": [{"name": "example", "path": "."}]}
)
)
(tmp_path / "package.json").write_text("{}")
before = set(tmp_path.iterdir())
with (
patch.object(doctor, "tool_version", return_value="fixture"),
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
assert (
cli.main(
[
"doctor",
"--workspace-root",
str(tmp_path),
"--project",
str(project),
"--json",
]
)
== 0
)
output = json.loads(capsys.readouterr().out)
assert any(item["status"] == "warning" for item in output["checks"])
assert set(tmp_path.iterdir()) == before
+339
View File
@@ -0,0 +1,339 @@
"""Coverage inventory is explicit intent, never execution or guessed completion."""
import json
from pathlib import Path
import re
import shutil
import sys
from unittest.mock import patch
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/devkit"))
from govoplan_devkit.catalog import build_coverage, build_stages, module_ui_stages # noqa: E402
from govoplan_devkit.coverage import canonical_invocations # noqa: E402
from govoplan_devkit.package_tests import ( # noqa: E402
CORE_COMPONENT_SUITES,
declared_tests,
read_package,
)
from govoplan_devkit.workspace import Project, Repository # noqa: E402
@pytest.fixture
def fixture(tmp_path):
core = Repository("govoplan-core", tmp_path / "govoplan-core", ("core",))
meta = Repository("govoplan", tmp_path / "govoplan", ("meta",))
module = Repository("govoplan-example", tmp_path / "govoplan-example", ("example",))
for repo in (core, meta, module):
(repo.path / "webui/scripts").mkdir(parents=True)
(repo.path / "webui/tests").mkdir()
scripts = {
"test:components": "node scripts/run-component-tests.mjs",
**{
f"test:{name}": f"node scripts/run-component-tests.mjs {name}"
for name in CORE_COMPONENT_SUITES
},
}
(core.path / "webui/package.json").write_text(json.dumps({"scripts": scripts}))
(core.path / "webui/scripts/run-component-tests.mjs").write_text("// fixture")
(module.path / "webui/package.json").write_text(
json.dumps(
{
"scripts": {
"test:safe": "node --test tests/source.test.mjs",
"test:compound": "tsc && node test.js",
"test:bad-quote": "node 'broken",
}
}
)
)
(module.path / "webui/tests/source.test.mjs").write_text("// fixture")
script = meta.path / "tools/checks/check-focused.sh"
script.parent.mkdir(parents=True)
phase_metadata = ROOT / "tools/checks/focused-phases.json"
shutil.copyfile(phase_metadata, script.with_name("focused-phases.json"))
bodies = {
"core-ui": 'cd "$ROOT/webui"\n"$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components\n',
"module-ui": 'cd "$WORKSPACE_ROOT/govoplan-example/webui"\n"$NPM" run test:compound\n',
}
script.write_text(
"\n".join(
f"focused_phase_{phase['id'].replace('-', '_')}() {{\n# devkit-phase: {phase['id']} begin\n"
+ bodies.get(phase["id"], 'cd "$ROOT"\n')
+ f"# devkit-phase: {phase['id']} end\n}}\n"
for phase in json.loads(phase_metadata.read_text())["phases"]
)
)
project = Project("Fixture", (meta, core, module), {})
with (
patch("govoplan_devkit.workspace.load_project", return_value=project),
patch("govoplan_devkit.coverage.load_project", return_value=project),
):
yield tmp_path, core, module, project
def rows(coverage, repo="govoplan-core"):
return {row["name"]: row for row in coverage["suites"] if row["repo"] == repo}
def test_core_aliases_are_explicitly_excluded_in_quick_and_covered_by_one_ui_stage(
fixture,
):
root, _, _, _ = fixture
quick = rows(build_coverage(root, "quick", [], False))
ui = rows(build_coverage(root, "ui", [], False))
for suite in CORE_COMPONENT_SUITES:
assert quick["test:" + suite]["disposition"] == "excluded"
assert "quick" in quick["test:" + suite]["reason"]
assert ui["test:" + suite]["disposition"] == "covered_elsewhere"
assert ui["test:" + suite]["covering_stage"] == "core.component-batch"
assert ui["test:components"]["covered_components"] == list(CORE_COMPONENT_SUITES)
def test_full_reports_only_four_of_sixteen_components_and_exact_shell_suite(fixture):
root, _, _, _ = fixture
result = build_coverage(root, "full", ["example"], False)
core = rows(result)
aliases = [core["test:" + suite] for suite in CORE_COMPONENT_SUITES]
assert sum(item["disposition"] == "covered_elsewhere" for item in aliases) == 4
assert sum(item["disposition"] == "excluded" for item in aliases) == 12
assert len(core["test:components"]["covered_components"]) == 4
assert "4/16" in core["test:components"]["reason"]
module = rows(result, "govoplan-example")
assert module["test:compound"]["disposition"] == "planned"
assert module["test:safe"]["disposition"] == "excluded"
assert result["stages"] == [
"focused." + phase["id"]
for phase in json.loads(
(ROOT / "tools/checks/focused-phases.json").read_text()
)["phases"]
]
assert core["test:components"]["covering_stage"] == "focused.core-ui"
assert module["test:compound"]["covering_stage"] == "focused.module-ui"
def test_prebuilt_plan_avoids_replanning(fixture):
root, _, _, _ = fixture
stages = build_stages(root, "quick", [], False)
with patch(
"govoplan_devkit.catalog.build_stages",
side_effect=AssertionError("must not replan"),
):
result = build_coverage(root, "quick", [], False, stages=stages)
assert rows(result, "govoplan-example")["test:safe"]["disposition"] == "planned"
assert sum(result["counts"].values()) == result["suite_count"]
assert all(item["reason"] for item in result["suites"])
def test_missing_or_spoofed_phase_stage_does_not_grant_component_coverage(fixture):
root, _, _, _ = fixture
stages = build_stages(root, "full", [], False)
without_core = [item for item in stages if item["id"] != "focused.core-ui"]
result = build_coverage(root, "full", [], False, stages=without_core)
assert rows(result)["test:components"]["covered_components"] == []
assert (
rows(result, "govoplan-example")["test:compound"]["covering_stage"]
== "focused.module-ui"
)
next(item for item in stages if item["id"] == "focused.core-ui")["argv"] = ["true"]
spoofed = build_coverage(root, "full", [], False, stages=stages)
assert rows(spoofed)["test:components"]["covered_components"] == []
@pytest.mark.parametrize("wrapper", ["heredoc", "function", "conditional"])
def test_lookalike_phase_wrappers_outside_top_level_do_not_grant_coverage(
fixture, wrapper
):
root, core, _, project = fixture
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
path = meta / "tools/checks/check-focused.sh"
fake = 'focused_phase_core_ui() {\n# devkit-phase: core-ui begin\ncd "$ROOT/webui"\n"$NPM" run test:spoof\n# devkit-phase: core-ui end\n}\n'
prefix = {
"heredoc": "cat <<'BODY'\n" + fake + "BODY\n",
"function": "unused() {\n" + fake + "}\n",
"conditional": "if false; then\n" + fake + "fi\n",
}[wrapper]
path.write_text(prefix + path.read_text())
result = canonical_invocations(root, meta, core.path)
assert result["notes"] == []
assert [item["name"] for item in result["npm"]] == [
"test:components",
"test:compound",
]
@pytest.mark.parametrize("damage", ["missing", "duplicate", "bad-end"])
def test_invalid_marked_phase_bodies_do_not_infer_coverage(fixture, damage):
root, core, _, project = fixture
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
path = meta / "tools/checks/check-focused.sh"
text = path.read_text()
if damage == "missing":
text = text.replace("focused_phase_core_ui()", "unregistered_core_ui()")
elif damage == "duplicate":
text += text
else:
text = text.replace(
"# devkit-phase: core-ui end", "# devkit-phase: another end"
)
path.write_text(text)
result = canonical_invocations(root, meta, core.path)
assert result["npm"] == [] and result["node"] == []
assert "no phase coverage inferred" in result["notes"][0]
@pytest.mark.parametrize(
"repo_name,name,command",
[
("govoplan-example", "test:components", "node scripts/run-component-tests.mjs"),
(
"govoplan-core",
"test:dialog-focus",
"node scripts/run-component-tests.mjs dialog-focus && npm run dev",
),
(
"govoplan-core",
"test:unrecognized",
"node scripts/run-component-tests.mjs dialog-focus",
),
],
)
def test_component_alias_exemption_is_exact_and_core_only(
tmp_path, repo_name, name, command
):
repo = Repository(repo_name, tmp_path / repo_name)
package = repo.path / "webui/package.json"
package.parent.mkdir(parents=True)
package.write_text(json.dumps({"scripts": {name: command}}))
declared = declared_tests(repo, package)
assert declared[0]["component_suite"] is None
assert declared[0]["_argv"] is None
assert module_ui_stages(repo, reason="fixture") == []
@pytest.mark.parametrize(
"body",
[
"[]",
"null",
'{"scripts":[]}',
'{"scripts":{"test:bad":42}}',
'{"scripts":{"test:one":"node a","test:one":"node b"}}',
"[" * 2000 + "]" * 2000,
],
)
def test_malformed_package_metadata_is_a_controlled_error(tmp_path, body):
package = tmp_path / "package.json"
package.write_text(body)
with pytest.raises(ValueError):
read_package(package)
def test_oversize_package_and_symlinked_test_are_not_discovered(tmp_path):
repo = Repository("example", tmp_path)
webui = tmp_path / "webui"
(webui / "tests").mkdir(parents=True)
package = webui / "package.json"
package.write_text(" " * (1024 * 1024 + 1))
with pytest.raises(ValueError):
read_package(package)
package.write_text(
json.dumps({"scripts": {"test:escape": "node tests/escape.mjs"}})
)
(webui / "tests/escape.mjs").symlink_to(tmp_path / "outside.mjs")
(tmp_path / "outside.mjs").write_text("// fixture")
assert module_ui_stages(repo, reason="fixture") == []
def test_unparseable_and_sensitive_commands_do_not_leak_into_coverage(fixture):
root, _, module, _ = fixture
package = module.path / "webui/package.json"
package.write_text(
json.dumps(
{
"scripts": {
"test:secret": "node tests/source.test.mjs --token unknown-private-value",
"test:broken": "node 'unknown-other-secret",
}
}
)
)
result = build_coverage(root, "quick", [], False)
encoded = json.dumps(result)
assert (
"unknown-private-value" not in encoded and "unknown-other-secret" not in encoded
)
assert "_argv" not in encoded and '"command"' not in encoded
assert all(
item["disposition"] == "unsupported"
for item in rows(result, "govoplan-example").values()
)
def test_custom_check_coverage_redacts_separate_token_and_includes_unselected(tmp_path):
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"repositories": [{"name": "app", "path": "."}],
"checks": [
{
"id": "one",
"argv": ["check", "--token", "unknown-private-value"],
},
{"id": "two", "argv": ["true"]},
],
"profiles": {"quick": ["one"]},
}
)
)
result = build_coverage(tmp_path, "quick", [], False, project)
assert "unknown-private-value" not in json.dumps(result)
assert result["counts"]["planned"] == 1 and result["counts"]["excluded"] == 1
def test_canonical_parser_does_not_credit_comments_heredocs_conditionals_or_chains(
tmp_path,
):
meta, core = tmp_path / "govoplan", tmp_path / "govoplan-core"
path = meta / "tools/checks/check-focused.sh"
path.parent.mkdir(parents=True)
path.write_text(
'cd "$ROOT/webui"\n# "$NPM" run test:comment\n"$PYTHON" - <<\'PY\'\n"$NPM" run test:heredoc\nPY\nif false; then\n"$NPM" run test:conditional\nfi\n"$NPM" run test:compound && true\n"$NPM" run test:real\n'
)
result = canonical_invocations(tmp_path, meta, core)
assert [item["name"] for item in result["npm"]] == ["test:real"]
def test_real_canonical_gate_has_four_explicit_component_suites():
result = canonical_invocations(ROOT.parent, ROOT, ROOT.parent / "govoplan-core")
calls = [item for item in result["npm"] if item["name"] == "test:components"]
assert len(calls) == 1
assert calls[0]["args"] == [
"layout-primitives",
"page-layout",
"data-grid-actions",
"mail-components",
]
assert calls[0]["phase"] == "core-ui"
for name, phase in (
("test:module-permutations", "module-builds"),
("test:conformance", "browser"),
):
matching = [item for item in result["npm"] if item["name"] == name]
assert len(matching) == 1 and matching[0]["phase"] == phase
def test_known_core_component_aliases_match_the_owned_runner_registry():
runner = (
ROOT.parent / "govoplan-core/webui/scripts/run-component-tests.mjs"
).read_text()
block = runner.split("export const componentSuites = Object.freeze({", 1)[1].split(
"});", 1
)[0]
assert set(re.findall(r'^ "([a-z0-9-]+)":', block, re.M)) == set(
CORE_COMPONENT_SUITES
)
+164
View File
@@ -0,0 +1,164 @@
"""Portable preflight inference uses fixtures only and never installs or starts tools."""
from argparse import Namespace
from pathlib import Path
import sys
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import doctor
from govoplan_devkit.workspace import Project, Repository
def project(tmp_path, checks, *, tools=None, profiles=None):
return Project(
"Fixture",
(Repository("app", tmp_path / "app"), Repository("other", tmp_path / "other")),
{
"checks": checks,
"profiles": profiles
if profiles is not None
else {"quick": [item["id"] for item in checks]},
"tools": tools or {},
},
)
def check(identity, executable, *, repos=None, deps=None):
return {
"id": identity,
"argv": [executable, "--version"],
"cwd": ".",
"repos": repos or [],
"deps": deps or [],
}
def diagnose(tmp_path, configured, *, profile=None, repos=None, versions=None):
args = Namespace(
workspace_root=tmp_path,
project=tmp_path / "not-read.json",
repo=repos or [],
profile=profile,
)
tools = {"python": sys.executable, "node": "fixture-node", "npm": "fixture-npm"}
version_map = versions or {
sys.executable: "Python 3",
"fixture-node": "unavailable",
"fixture-npm": "unavailable",
}
with (
patch.object(doctor, "load_project", return_value=configured),
patch.object(doctor, "resolve_tools", return_value=tools),
patch.object(
doctor,
"tool_version",
side_effect=lambda executable, _env: version_map[executable],
) as probe,
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
result = doctor.diagnose(args)
return result, probe
def test_python_only_portable_project_does_not_probe_or_block_on_unused_node(tmp_path):
configured = project(tmp_path, [check("python-test", "{python}")])
result, probe = diagnose(tmp_path, configured)
assert result["_exit_code"] == 0
assert result["required_tools"] == ["python"]
assert probe.call_count == 1
assert {
item["id"] for item in result["checks"] if item["status"] == "not_required"
} == {"node", "npm"}
def test_selected_profile_excludes_other_profile_tool_requirements(tmp_path):
configured = project(
tmp_path,
[check("py", "{python}"), check("js", "{npm}")],
profiles={"quick": ["py"], "ui": ["js"]},
)
result, _ = diagnose(tmp_path, configured, profile="quick")
assert result["required_tools"] == ["python"]
result, _ = diagnose(tmp_path, configured)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
def test_selected_repository_includes_dependency_tool_requirements(tmp_path):
configured = project(
tmp_path,
[
check("build", "{npm}", repos=["other"]),
check("test", "{python}", repos=["app"], deps=["build"]),
],
)
result, _ = diagnose(tmp_path, configured, repos=["app"])
assert result["required_tools"] == ["node", "npm", "python"]
def test_unselected_repository_does_not_require_its_tool(tmp_path):
configured = project(
tmp_path,
[check("js", "{npm}", repos=["other"]), check("py", "{python}", repos=["app"])],
)
result, _ = diagnose(tmp_path, configured, repos=["app"])
assert result["required_tools"] == ["python"]
def test_explicit_tool_configuration_declares_indirect_script_dependency(tmp_path):
configured = project(tmp_path, [check("shell", "sh")], tools={"npm": "fixture-npm"})
result, _ = diagnose(tmp_path, configured)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
@pytest.mark.parametrize(
"executable,expected",
[
("/opt/node/bin/node", ["node", "python"]),
("npm", ["node", "npm", "python"]),
("npx", ["node", "npm", "python"]),
],
)
def test_direct_tool_names_are_inferred(tmp_path, executable, expected):
result, _ = diagnose(tmp_path, project(tmp_path, [check("test", executable)]))
assert result["required_tools"] == expected
def test_context_only_project_needs_no_node_and_makes_no_files(tmp_path):
before = set(tmp_path.iterdir())
result, _ = diagnose(tmp_path, project(tmp_path, [], profiles={}))
assert result["required_tools"] == ["python"]
assert set(tmp_path.iterdir()) == before
def test_missing_profile_is_not_silently_widened(tmp_path):
with pytest.raises(ValueError, match="does not declare profile"):
diagnose(tmp_path, project(tmp_path, []), profile="ui")
def test_native_govoplan_still_requires_all_three_tools(tmp_path):
configured = project(tmp_path, [])
args = Namespace(workspace_root=tmp_path, project=None, repo=[], profile=None)
with (
patch.object(doctor, "load_project", return_value=configured),
patch.object(
doctor,
"resolve_tools",
return_value={"python": sys.executable, "node": "node", "npm": "npm"},
),
patch.object(
doctor,
"tool_version",
side_effect=lambda executable, _env: (
"Python" if executable == sys.executable else "unavailable"
),
),
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
):
result = doctor.diagnose(args)
assert result["required_tools"] == ["node", "npm", "python"]
assert result["_exit_code"] == 1
+378
View File
@@ -0,0 +1,378 @@
"""Environment probes read bounded stable files; all fixtures are local."""
import hashlib
import os
from pathlib import Path
import sys
from types import SimpleNamespace
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import environment
from govoplan_devkit.common import digest
from govoplan_devkit.workspace import Project, Repository
def mutate_during_read(monkeypatch, callback):
original = hashlib.sha256
mutated = False
class MutatingHasher:
def __init__(self):
self.hasher = original()
def update(self, chunk):
nonlocal mutated
self.hasher.update(chunk)
if not mutated:
mutated = True
callback()
def hexdigest(self):
return self.hasher.hexdigest()
monkeypatch.setattr(environment.hashlib, "sha256", MutatingHasher)
@pytest.mark.parametrize("content", [b"", b"stable", b"x" * (1024 * 1024 + 3)])
def test_file_hash_keeps_existing_digest_and_exact_size_boundary(tmp_path, content):
path = tmp_path / "input"
path.write_bytes(content)
assert (
environment._environment_file_hash(path, len(content))
== hashlib.sha256(content).hexdigest()
)
def test_missing_optional_file_remains_optional_without_a_persistent_cache(tmp_path):
path = tmp_path / "input"
assert environment._environment_file_hash(path, 10) is None
path.write_bytes(b"first")
first = environment._environment_file_hash(path, 10)
path.write_bytes(b"other")
assert environment._environment_file_hash(path, 10) != first
@pytest.mark.parametrize("kind", ["fifo", "directory", "oversized", "dangling"])
def test_present_unsafe_input_is_rejected_without_blocking(tmp_path, kind):
path = tmp_path / "input"
if kind == "fifo":
os.mkfifo(path)
elif kind == "directory":
path.mkdir()
elif kind == "oversized":
path.write_bytes(b"too large")
else:
path.symlink_to(tmp_path / "missing")
with pytest.raises(ValueError, match="Environment input"):
environment._environment_file_hash(path, 2)
def test_stable_venv_executable_symlink_keeps_original_path_and_identity(tmp_path):
target = tmp_path / "real-python"
target.write_bytes(b"fixture binary")
executable = tmp_path / "venv" / "bin" / "python"
executable.parent.mkdir(parents=True)
executable.symlink_to(target)
repo = tmp_path / "repo"
metadata = repo / "node_modules" / ".package-lock.json"
metadata.parent.mkdir(parents=True)
metadata.write_bytes(b'{"fixture":true}')
project = Project("Fixture", (Repository("repo", repo),), {})
tools = {"python": str(executable)}
env = {"PATH": "fixture", "PWD": "ignored"}
distributions = b'[["fixture", "1"]]\n'
with (
patch.object(environment, "tool_version", return_value="fixture-version"),
patch.object(
environment,
"require_capture",
return_value=SimpleNamespace(returncode=0, stdout=distributions),
) as capture,
):
actual = environment.environment_fingerprint(tmp_path, project, tools, env)
assert actual == digest(
{
"environment": {"PATH": "fixture"},
"tools": {
"python": {
"path": str(executable),
"version": "fixture-version",
"sha256": hashlib.sha256(target.read_bytes()).hexdigest(),
}
},
"installed": {
str(metadata): hashlib.sha256(metadata.read_bytes()).hexdigest(),
"python_distributions": hashlib.sha256(distributions).hexdigest(),
},
}
)
assert capture.call_args.args[0][0] == str(executable)
def test_symlinked_package_directory_is_allowed_when_stable(tmp_path):
actual = tmp_path / "packages"
actual.mkdir()
(actual / ".package-lock.json").write_bytes(b"fixture")
link = tmp_path / "node_modules"
link.symlink_to(actual, target_is_directory=True)
assert environment._environment_file_hash(link / ".package-lock.json", 10)
def test_fifo_replacement_between_inspection_and_open_does_not_block(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"fixture")
real_open = os.open
def replace_before_open(target, flags):
path.unlink()
os.mkfifo(path)
assert flags & os.O_NONBLOCK
return real_open(target, flags)
monkeypatch.setattr(environment.os, "open", replace_before_open)
with pytest.raises(ValueError, match="bounded regular"):
environment._environment_file_hash(path, 10)
def test_growth_between_inspection_and_open_cannot_bypass_size_bound(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"a")
real_open = os.open
def grow_before_open(target, flags):
path.write_bytes(b"x" * 11)
return real_open(target, flags)
monkeypatch.setattr(environment.os, "open", grow_before_open)
with pytest.raises(ValueError, match="bounded regular"):
environment._environment_file_hash(path, 10)
def test_growth_during_read_cannot_bypass_size_bound(tmp_path, monkeypatch):
path = tmp_path / "input"
path.write_bytes(b"a")
mutate_during_read(monkeypatch, lambda: path.write_bytes(b"x" * 11))
with pytest.raises(ValueError, match="grew beyond"):
environment._environment_file_hash(path, 10)
def test_same_size_edit_with_restored_mtime_is_not_a_stable_identity(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"first")
metadata = path.stat()
def mutate():
path.write_bytes(b"other")
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
mutate_during_read(monkeypatch, mutate)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(path, 10)
def test_replacement_with_identical_bytes_is_not_a_stable_identity(
tmp_path, monkeypatch
):
path = tmp_path / "input"
path.write_bytes(b"fixture")
replacement = tmp_path / "replacement"
replacement.write_bytes(b"fixture")
mutate_during_read(monkeypatch, lambda: replacement.replace(path))
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(path, 10)
def test_symlink_retarget_with_identical_bytes_is_not_a_stable_identity(
tmp_path, monkeypatch
):
first, second, link = (tmp_path / name for name in ("first", "second", "python"))
first.write_bytes(b"fixture")
second.write_bytes(b"fixture")
link.symlink_to(first)
def retarget():
link.unlink()
link.symlink_to(second)
mutate_during_read(monkeypatch, retarget)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(link, 10)
def test_source_disappearing_during_read_fails_closed(tmp_path, monkeypatch):
path = tmp_path / "input"
path.write_bytes(b"fixture")
mutate_during_read(monkeypatch, path.unlink)
with pytest.raises(ValueError, match="Environment input"):
environment._environment_file_hash(path, 10)
def test_parent_symlink_retarget_is_rejected_even_for_the_same_target_inode(
tmp_path, monkeypatch
):
first, second, link = (tmp_path / name for name in ("first", "second", "bin"))
first.mkdir()
second.mkdir()
(first / "python").write_bytes(b"fixture")
os.link(first / "python", second / "python")
link.symlink_to(first, target_is_directory=True)
def retarget():
link.unlink()
link.symlink_to(second, target_is_directory=True)
mutate_during_read(monkeypatch, retarget)
with pytest.raises(ValueError, match="changed during"):
environment._environment_file_hash(link / "python", 10)
def test_invalid_executable_is_rejected_before_any_version_process(tmp_path):
executable = tmp_path / "python"
os.mkfifo(executable)
project = Project("Fixture", (), {})
with patch.object(environment, "tool_version") as version:
with pytest.raises(ValueError, match="bounded regular"):
environment.environment_fingerprint(
tmp_path, project, {"python": str(executable)}, {}
)
version.assert_not_called()
def discovery_project(root, *, path="govoplan-backend"):
repo = root / path
(repo / "src").mkdir(parents=True)
return Project(
"GovOPlaN",
(Repository("govoplan-backend", repo),),
{"organization": "GovOPlaN"},
)
def test_native_shape_ignores_build_files_and_directory_timestamps(tmp_path):
project = discovery_project(tmp_path)
repo = project.repositories[0].path
(repo / "webui").mkdir()
before = environment._native_discovery_fingerprint(tmp_path, project)
for directory in (repo, repo / "src", repo / "webui"):
(directory / "temporary-build-output").write_text("changed")
(directory / "temporary-build-directory").mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == before
(repo / "webui" / "temporary-build-output").unlink()
assert environment._native_discovery_fingerprint(tmp_path, project) == before
def test_native_shape_detects_unknown_sibling_and_source_addition_removal(tmp_path):
project = discovery_project(tmp_path)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
sibling = tmp_path / "govoplan-unregistered"
sibling.mkdir()
empty = environment._native_discovery_fingerprint(tmp_path, project)
assert empty != baseline
for name in ("src", "webui"):
(sibling / name).mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != empty
(sibling / name).rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == empty
sibling.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
@pytest.mark.parametrize("repo_path", ["govoplan-backend", "nonstandard-layout"])
def test_native_shape_detects_registered_backend_gaining_webui(tmp_path, repo_path):
project = discovery_project(tmp_path, path=repo_path)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
webui = project.repositories[0].path / "webui"
webui.mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != baseline
webui.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
def test_native_shape_binds_registered_ownership_names(tmp_path):
project = discovery_project(tmp_path, path="nonstandard-layout")
renamed = Project(
project.name,
(Repository("govoplan-renamed", project.repositories[0].path),),
project.config,
)
assert environment._native_discovery_fingerprint(
tmp_path, project
) != environment._native_discovery_fingerprint(tmp_path, renamed)
def test_native_shape_detects_source_link_retarget_and_dangling_target(tmp_path):
project = discovery_project(tmp_path)
first, second = tmp_path / "first", tmp_path / "second"
first.mkdir()
second.mkdir()
link = project.repositories[0].path / "webui"
link.symlink_to(first, target_is_directory=True)
baseline = environment._native_discovery_fingerprint(tmp_path, project)
link.unlink()
link.symlink_to(second, target_is_directory=True)
changed = environment._native_discovery_fingerprint(tmp_path, project)
assert changed != baseline
second.rmdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != changed
def test_native_shape_encodes_non_directory_source_entries(tmp_path):
project = discovery_project(tmp_path)
webui = project.repositories[0].path / "webui"
baseline = environment._native_discovery_fingerprint(tmp_path, project)
webui.write_text("not a directory")
file_shape = environment._native_discovery_fingerprint(tmp_path, project)
assert file_shape != baseline
webui.unlink()
webui.mkdir()
assert environment._native_discovery_fingerprint(tmp_path, project) != file_shape
def test_native_shape_audit_is_bounded_including_nonmatching_children(
tmp_path, monkeypatch
):
project = discovery_project(tmp_path)
(tmp_path / "unrelated-one").mkdir()
(tmp_path / "unrelated-two").mkdir()
monkeypatch.setattr(environment, "MAX_DISCOVERY_CHILDREN", 2)
with pytest.raises(ValueError, match="bounded ownership"):
environment._native_discovery_fingerprint(tmp_path, project)
def test_native_shape_resolution_loop_fails_closed(tmp_path):
project = discovery_project(tmp_path)
link = project.repositories[0].path / "webui"
link.symlink_to(link)
with pytest.raises(ValueError, match="discovery cannot be resolved"):
environment._native_discovery_fingerprint(tmp_path, project)
@pytest.mark.parametrize("portable", [False, True])
def test_only_native_environment_identity_binds_discovery_shape(tmp_path, portable):
project = discovery_project(tmp_path)
if portable:
project.config["schema_version"] = 1
executable = tmp_path / "fixture-python"
executable.write_text("not executed")
tools = {"python": str(executable)}
with (
patch.object(environment, "tool_version", return_value="fixture"),
patch.object(
environment,
"require_capture",
return_value=SimpleNamespace(returncode=0, stdout=b"[]"),
),
):
baseline = environment.environment_fingerprint(tmp_path, project, tools, {})
(tmp_path / "govoplan-new" / "src").mkdir(parents=True)
changed = environment.environment_fingerprint(tmp_path, project, tools, {})
assert (baseline == changed) is portable
+601
View File
@@ -0,0 +1,601 @@
"""Incremental checkpoints use isolated local repositories, never product or remote state."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import subprocess
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import atomic_json, read_json, resource_lock, state_root
@pytest.fixture
def workspace(tmp_path, monkeypatch):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg"))
root = tmp_path / "workspace"
repos = {}
for name in ("alpha", "beta"):
repo = root / name
repo.mkdir(parents=True)
subprocess.run(["git", "init", "-q", str(repo)], check=True)
(repo / "source.txt").write_text(name + " original\n")
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
)
repos[name] = repo
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Incremental fixture",
"repositories": [{"name": name, "path": name} for name in repos],
"checks": [],
"profiles": {},
}
)
)
traces = tmp_path / "traces"
traces.mkdir()
args = Namespace(
workspace_root=root,
project=config,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="quick",
resume=None,
repo=[],
changed=False,
)
return args, repos, traces
def stage(workspace, identity, repo="alpha", *, inputs=True, body="", **extra):
_, repos, traces = workspace
counter = traces / identity
code = (
"from pathlib import Path; "
f"counter=Path({str(counter)!r}); "
"counter.write_text(str(int(counter.read_text())+1 if counter.exists() else 1)); "
f"print({identity!r},flush=True); " + body
)
result = {
"id": identity,
"title": identity,
"argv": [sys.executable, "-c", code],
"cwd": str(repos[repo]),
"timeout_seconds": 5,
**extra,
}
if inputs:
result["inputs"] = {"repos": [repo]}
return result
def run(workspace, stages, *, environment="e" * 64):
args, _, _ = workspace
with patch.object(runner, "environment_fingerprint", return_value=environment):
return runner.run_checks(args, stages)
def count(workspace, identity):
path = workspace[2] / identity
return int(path.read_text()) if path.exists() else 0
def stages_by_id(result):
return {item["id"]: item for item in result["stages"]}
def test_checkpoint_probes_and_durable_persistence_hold_stage_resource_lock(workspace):
args, _, _ = workspace
locks = state_root(args.workspace_root) / "resource-locks"
original_identity, original_write = Checkpoints.identity, runner.atomic_json
probes, persisted = [], []
def assert_held():
with pytest.raises(RuntimeError, match="busy"):
with resource_lock(locks, "checkpoint-fixture"):
pass
def identity(self, selected):
assert_held()
probes.append(selected["id"])
return original_identity(self, selected)
def write(path, payload):
original_write(path, payload)
if (
payload.get("phase") == "checking"
and payload["stages"][0].get("checkpoint_verified") is True
and not persisted
):
assert_held()
assert read_json(path)["stages"][0]["checkpoint_verified"] is True
persisted.append(path)
with (
patch.object(Checkpoints, "identity", identity),
patch.object(runner, "atomic_json", side_effect=write),
):
result = run(
workspace, [stage(workspace, "a", resources=["checkpoint-fixture"])]
)
assert result["status"] == "passed"
assert len(probes) >= 2 and len(persisted) == 1
def test_checkpoint_save_failure_never_releases_dependent_execution(workspace):
args, _, _ = workspace
original_write, original_command, original_wait = (
runner.atomic_json,
runner._execute_stage_command,
runner.wait,
)
scheduler_waiting = threading.Event()
checkpoint_stalled = threading.Event()
scheduler_rechecked = threading.Event()
release_failure = threading.Event()
failed_write_paths, results, errors = [], [], []
def command(selected, *values, **kwargs):
if selected["id"] == "producer":
# Let the scheduler finish its initial running-state save and enter
# its wait loop before the command publishes a passing checkpoint.
assert scheduler_waiting.wait(timeout=5)
return original_command(selected, *values, **kwargs)
def wait(*values, **kwargs):
scheduler_waiting.set()
result = original_wait(*values, **kwargs)
if checkpoint_stalled.is_set():
scheduler_rechecked.set()
return result
def write(path, payload):
producer = next(
(item for item in payload.get("stages", []) if item["id"] == "producer"),
None,
)
if (
producer
and producer.get("checkpoint_verified") is True
and not failed_write_paths
):
failed_write_paths.append(path)
checkpoint_stalled.set()
assert release_failure.wait(timeout=5)
raise OSError("fixture checkpoint persistence failed")
return original_write(path, payload)
def execute():
try:
results.append(
run(
workspace,
[
stage(workspace, "producer"),
stage(workspace, "dependent", "beta", deps=["producer"]),
],
)
)
except BaseException as exc:
errors.append(exc)
worker = threading.Thread(target=execute)
with (
patch.object(runner, "atomic_json", side_effect=write),
patch.object(runner, "_execute_stage_command", side_effect=command),
patch.object(runner, "wait", side_effect=wait),
):
worker.start()
try:
assert checkpoint_stalled.wait(timeout=5)
assert scheduler_rechecked.wait(timeout=5)
# Keep persistence blocked across a scheduling turn: the in-memory
# producer status must not grant authority to start a consumer.
time.sleep(0.15)
assert count(workspace, "dependent") == 0
durable = read_json(failed_write_paths[0])
assert (
stages_by_id(durable)["producer"].get("checkpoint_verified") is not True
)
finally:
release_failure.set()
worker.join(timeout=8)
assert not worker.is_alive() and not errors
assert len(results) == 1 and results[0]["status"] == "failed"
assert count(workspace, "producer") == 1 and count(workspace, "dependent") == 0
by_id = stages_by_id(results[0])
assert by_id["producer"]["status"] == "failed"
assert by_id["producer"]["checkpoint_verified"] is False
assert "fixture checkpoint persistence failed" in by_id["producer"]["error"]
assert by_id["dependent"]["status"] == "skipped"
persisted = runner.read_receipt(
args.workspace_root, args.state_dir, results[0]["run_id"]
)
assert stages_by_id(persisted)["producer"]["checkpoint_verified"] is False
def test_verified_checkpoint_reuses_unchanged_stage_after_other_repo_changes(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
first = run(workspace, plan)
old_bytes = Path(first["receipt_path"]).read_bytes()
(repos["beta"] / "source.txt").write_text("beta changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
by_id = stages_by_id(second)
assert second["status"] == "passed" and second["snapshot_verified"] is True
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
assert by_id["a"]["reused_from"] == first["run_id"]
assert "reused_from" not in by_id["b"]
assert by_id["a"]["checkpoint_verified"] is True
assert by_id["a"]["checkpoint_version"] == 1
assert isinstance(by_id["a"]["cache_key"], str) and by_id["a"]["cache_key"]
assert second["source_fingerprint"] != first["source_fingerprint"]
assert Path(first["receipt_path"]).read_bytes() == old_bytes
def test_failed_post_execution_probe_preserves_actual_log_without_certifying_it(
workspace,
):
from govoplan_devkit.checkpoints import Checkpoints
identity = Checkpoints.identity
calls = 0
def probe(self, selected):
nonlocal calls
calls += 1
if calls == 2:
raise ValueError("fixture input became unreadable")
return identity(self, selected)
with patch.object(Checkpoints, "identity", probe):
result = run(workspace, [stage(workspace, "a")])
selected = result["stages"][0]
assert result["status"] == "stale"
assert selected["status"] == "stale" and selected["exit_code"] == 0
assert selected["checkpoint_verified"] is False
assert Path(selected["log_path"]).read_text() == "a\n"
assert "fixture input became unreadable" in selected["error"]
def test_changed_and_new_commands_run_without_discarding_unrelated_checkpoint(
workspace,
):
args, _, _ = workspace
first = run(workspace, [stage(workspace, "a"), stage(workspace, "b", "beta")])
args.resume = first["run_id"]
second = run(
workspace,
[
stage(workspace, "a"),
stage(workspace, "b", "beta", body="print('changed command')"),
stage(workspace, "new", "beta"),
],
)
assert second["status"] == "passed"
assert count(workspace, "a") == 1
assert count(workspace, "b") == 2
assert count(workspace, "new") == 1
assert first["plan_fingerprint"] != second["plan_fingerprint"]
def test_unrelated_profile_edit_does_not_change_existing_stage_execution_identity(
workspace,
):
args, _, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
config = json.loads(args.project.read_text())
config["checks"] = [
{
"id": "extra",
"argv": [sys.executable, "-c", "print('extra')"],
"cwd": "beta",
"repos": ["beta"],
}
]
config["profiles"] = {"backend": ["extra"]}
args.project.write_text(json.dumps(config))
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1
def test_added_dependency_edge_invalidates_consumer_even_when_both_repo_bytes_match(
workspace,
):
args, _, _ = workspace
a, b = stage(workspace, "a"), stage(workspace, "b", "beta")
first = run(workspace, [a, b])
args.resume = first["run_id"]
second = run(workspace, [a, {**b, "deps": ["a"]}])
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
def test_failed_run_reuses_successful_phase_but_retries_failed_phase(workspace):
args, _, traces = workspace
ready = traces / "ready"
plan = [
stage(workspace, "a"),
stage(
workspace,
"b",
"beta",
after=["a"],
body=f"raise SystemExit(0 if Path({str(ready)!r}).exists() else 3)",
),
]
first = run(workspace, plan)
assert first["status"] == "failed"
assert stages_by_id(first)["a"]["checkpoint_verified"] is True
ready.touch()
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
def test_changed_dependency_invalidates_transitive_consumers_but_not_independent_stage(
workspace,
):
args, repos, _ = workspace
plan = [
stage(workspace, "a"),
stage(workspace, "b", "beta", deps=["a"]),
stage(workspace, "c", "beta", deps=["b"]),
stage(workspace, "independent", "beta"),
]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert {
identity: count(workspace, identity)
for identity in ("a", "b", "c", "independent")
} == {"a": 2, "b": 2, "c": 2, "independent": 1}
def test_order_only_predecessor_change_does_not_invalidate_independent_stage(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 2 and count(workspace, "b") == 1
assert stages_by_id(second)["b"]["reused_from"] == first["run_id"]
def test_order_only_failure_prevents_new_downstream_execution(workspace):
plan = [
stage(workspace, "a", body="raise SystemExit(8)"),
stage(workspace, "b", "beta", after=["a"]),
]
result = run(workspace, plan)
assert result["status"] == "failed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 0
assert stages_by_id(result)["b"]["status"] == "skipped"
def test_never_reused_stage_and_actual_consumers_rerun_but_order_only_stage_can_reuse(
workspace,
):
args, _, _ = workspace
plan = [
stage(workspace, "producer", reuse="never"),
stage(workspace, "consumer", "beta", deps=["producer"]),
stage(workspace, "independent", "beta", after=["producer"]),
]
first = run(workspace, plan)
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "producer") == 2
assert count(workspace, "consumer") == 2
assert count(workspace, "independent") == 1
def test_unspecified_input_scope_stays_conservatively_workspace_wide(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "broad", "beta", inputs=False)]
first = run(workspace, plan)
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "broad") == 2
def test_global_environment_change_invalidates_all_scoped_checkpoints(workspace):
args, _, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
first = run(workspace, plan)
args.resume = first["run_id"]
second = run(workspace, plan, environment="f" * 64)
assert second["status"] == "passed"
assert count(workspace, "a") == 2 and count(workspace, "b") == 2
assert second["environment_fingerprint"] != first["environment_fingerprint"]
def test_changed_declared_scope_prevents_same_id_reuse(workspace):
args, _, _ = workspace
selected = stage(workspace, "a")
first = run(workspace, [selected])
args.resume = first["run_id"]
selected = {**selected, "inputs": {"repos": ["alpha", "beta"]}}
second = run(workspace, [selected])
assert second["status"] == "passed" and count(workspace, "a") == 2
def test_input_mutation_during_passing_command_never_creates_reusable_checkpoint(
workspace,
):
args, repos, _ = workspace
plan = [
stage(
workspace,
"mutates",
body="Path('source.txt').write_text('mutated during stage\\n')",
)
]
first = run(workspace, plan)
assert first["status"] != "passed"
assert stages_by_id(first)["mutates"].get("checkpoint_verified") is not True
args.resume = first["run_id"]
# The same command is now stable against the already-mutated current bytes;
# its unverified first result still cannot be skipped.
second = run(workspace, plan)
assert count(workspace, "mutates") == 2
assert second["status"] == "passed"
assert (repos["alpha"] / "source.txt").read_text() == "mutated during stage\n"
def test_later_restoration_of_bytes_cannot_turn_invalid_phase_into_overall_pass(
workspace,
):
_, repos, _ = workspace
source = repos["alpha"] / "source.txt"
original = source.read_text()
plan = [
stage(
workspace,
"mutates",
body="Path('source.txt').write_text('temporary change\\n')",
),
stage(
workspace,
"restores",
"beta",
after=["mutates"],
body=f"Path({str(source)!r}).write_text({original!r})",
),
]
result = run(workspace, plan)
assert result["status"] != "passed"
assert stages_by_id(result)["mutates"].get("checkpoint_verified") is not True
def test_recovered_interruption_reuses_only_verified_durable_checkpoint(workspace):
args, _, _ = workspace
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
first = run(workspace, plan)
receipt = read_json(Path(first["receipt_path"]))
receipt.update(
status="running", phase="checking", snapshot_verified=False, finished_at=None
)
unfinished = stages_by_id(receipt)["b"]
unfinished.update(status="running", exit_code=None, checkpoint_verified=False)
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers(dest="command", required=True))
recovered = parser.parse_args(
["recover", first["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=Namespace(**vars(args)),
)
recovery = recovered.handler(recovered)
assert recovery["status"] == "interrupted"
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
assert stages_by_id(second)["a"]["reused_from"] == first["run_id"]
@pytest.mark.parametrize("change", ["tampered", "missing"])
def test_cached_log_tamper_or_loss_is_not_accepted_as_verified_evidence(
workspace, change
):
args, _, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
path = Path(first["stages"][0]["log_path"])
if change == "missing":
path.unlink()
else:
path.write_text("tampered evidence\n")
args.resume = first["run_id"]
result = run(workspace, plan)
assert result["status"] == "failed"
assert (
result["snapshot_verified"] is not True
or result["stages"][0]["status"] == "failed"
)
assert result["stages"][0].get("checkpoint_verified") is not True
assert "reused_from" not in result["stages"][0]
assert "error" in result["stages"][0]
assert count(workspace, "a") == 1
def test_stale_run_donates_only_checkpoints_matching_final_current_inputs(workspace):
args, repos, _ = workspace
source = repos["alpha"] / "source.txt"
plan = [
stage(workspace, "a"),
stage(
workspace,
"changes-alpha",
"beta",
after=["a"],
body=f"Path({str(source)!r}).write_text('new alpha bytes\\n')",
),
]
first = run(workspace, plan)
assert first["status"] != "passed"
args.resume = first["run_id"]
second = run(workspace, plan)
assert second["status"] == "passed"
assert count(workspace, "a") == 2
# This order-only phase did not consume alpha and its own scoped inputs are
# still identical; it may retain its independently verified checkpoint.
assert count(workspace, "changes-alpha") == 1
def test_legacy_checkpointless_receipt_does_not_gain_incremental_authority(workspace):
args, repos, _ = workspace
plan = [stage(workspace, "a")]
first = run(workspace, plan)
receipt = read_json(Path(first["receipt_path"]))
receipt.pop("fingerprint_version", None)
for item in receipt["stages"]:
for field in ("checkpoint_version", "checkpoint_verified", "cache_key"):
item.pop(field, None)
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
(repos["beta"] / "source.txt").write_text("unrelated changed source\n")
args.resume = first["run_id"]
try:
second = run(workspace, plan)
except ValueError:
return # Rejecting legacy incremental reuse is also safely fail-closed.
assert second["status"] == "passed"
assert count(workspace, "a") == 2
+393
View File
@@ -0,0 +1,393 @@
"""Repository input identities use disposable Git fixtures, never remote effects."""
from copy import deepcopy
import os
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import inputs
from govoplan_devkit.inputs import InputSnapshotter, validate_input_declaration
from govoplan_devkit.workspace import Project, Repository, source_fingerprint
def git(repo, *arguments):
return subprocess.run(
["git", "-C", str(repo), *arguments], capture_output=True, check=True
).stdout
@pytest.fixture
def fixture(tmp_path, monkeypatch):
repositories = []
for name in ("alpha", "beta"):
path = tmp_path / name
path.mkdir()
git(path, "init", "-q")
(path / "source.txt").write_text("initial\n")
git(path, "add", "source.txt")
git(
path,
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
)
repositories.append(Repository(name, path, ("alias-" + name,)))
project = Project(
"Fixture",
tuple(repositories),
{"tools": {}, "profiles": {"quick": ["one"]}, "checks": []},
)
# Avoid incidental edits by other agents affecting these source-input tests.
# The actual tool-source hashing implementation is checked separately.
monkeypatch.setattr(
InputSnapshotter, "_tooling_identity", lambda *_: "fixture-devkit-source"
)
return tmp_path, project, InputSnapshotter(project, workspace_root=tmp_path)
def stage(identity="one", repos=None, **extra):
return {
"id": identity,
"argv": ["true"],
**({"inputs": {"repos": repos}} if repos is not None else {}),
**extra,
}
def fingerprint(snapshot, name="one"):
return snapshot["stages"][name]["fingerprint"]
def test_undeclared_inputs_remain_whole_workspace(fixture):
root, _, engine = fixture
result = engine.snapshot([stage()])
assert result["complete_workspace"] is True
assert result["observed_scope"]["repos"] == ["alpha", "beta"]
assert result["stages"]["one"]["scope"] == {
"version": 1,
"kind": "workspace",
"declared": False,
"repos": ["alpha", "beta"],
}
(root / "beta/source.txt").write_text("changed\n")
assert fingerprint(engine.snapshot([stage()])) != fingerprint(result)
def test_scoped_input_does_not_read_unrelated_repository_bytes(fixture, monkeypatch):
root, _, engine = fixture
original = inputs.os.open
def guarded(path, *args, **kwargs):
if Path(path).is_relative_to(root / "beta"):
raise AssertionError("Unrelated repository input was opened")
return original(path, *args, **kwargs)
monkeypatch.setattr(inputs.os, "open", guarded)
first = engine.snapshot([stage(repos=["alpha"])])
(root / "beta/source.txt").write_text("unrelated change")
second = engine.snapshot([stage(repos=["alpha"])])
assert fingerprint(first) == fingerprint(second)
assert second["scan_stats"]["repositories"] == 1
assert second["scan_stats"]["git_calls"] == 2
assert second["complete_workspace"] is False
def test_repository_union_scans_once_per_snapshot(fixture):
_, _, engine = fixture
result = engine.snapshot(
[
stage("one", ["alpha"]),
stage("two", ["alpha"]),
stage("three", ["beta", "alpha"]),
]
)
assert result["scan_stats"]["repositories"] == 2
assert result["scan_stats"]["git_calls"] == 4
assert result["stages"]["three"]["scope"]["repos"] == ["alpha", "beta"]
def test_session_rechecks_metadata_but_reuses_stable_file_content(fixture):
_, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
after = engine.snapshot(plan)
assert fingerprint(before) == fingerprint(after)
assert before["scan_stats"]["bytes"] == len("initial\n")
assert after["scan_stats"]["bytes"] == 0
assert after["scan_stats"]["cache_hits"] == 1
assert after["scan_stats"]["git_calls"] == 2
def test_changed_bytes_with_restored_mtime_are_not_reused(fixture):
root, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
path = root / "alpha/source.txt"
metadata = path.stat()
path.write_text("changed\n")
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
after = engine.snapshot(plan)
assert fingerprint(after) != fingerprint(before)
assert after["scan_stats"]["bytes"] == len("changed\n")
def test_inode_replacement_with_same_size_and_mtime_is_not_reused(fixture):
root, _, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
path = root / "alpha/source.txt"
metadata = path.stat()
replacement = root / "replacement.txt"
replacement.write_text("changed\n")
os.utime(replacement, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
replacement.replace(path)
after = engine.snapshot(plan)
assert fingerprint(before) != fingerprint(after)
assert after["scan_stats"]["cache_hits"] == 0
@pytest.mark.parametrize("flag", ["assume-unchanged", "skip-worktree"])
def test_hidden_index_flags_do_not_hide_worktree_changes(fixture, flag):
root, _, engine = fixture
git(root / "alpha", "update-index", "--" + flag, "source.txt")
before = engine.source_snapshot(["alpha"])
(root / "alpha/source.txt").write_text("hidden change\n")
after = engine.source_snapshot(["alpha"])
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
def test_index_head_and_new_deleted_files_are_bound(fixture):
root, _, engine = fixture
repo = root / "alpha"
identities = []
def record():
identities.append(
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
)
record()
(repo / "source.txt").write_text("changed\n")
record()
git(repo, "add", "source.txt")
record()
git(
repo,
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"change",
)
record()
(repo / "new.txt").write_text("new")
record()
(repo / "source.txt").unlink()
record()
assert len(set(identities)) == len(identities)
def test_missing_registered_repository_becoming_present_invalidates(fixture):
root, project, _ = fixture
extra = Repository("missing", root / "missing")
extended = Project("Fixture", (*project.repositories, extra), project.config)
engine = InputSnapshotter(extended, workspace_root=root)
before = engine.source_snapshot(["missing"])
extra.path.mkdir()
git(extra.path, "init", "-q")
after = engine.source_snapshot(["missing"])
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
def test_unrelated_plan_config_edits_do_not_invalidate_scoped_stage(fixture):
root, project, engine = fixture
plan = [stage(repos=["alpha"])]
before = engine.snapshot(plan)
updated = deepcopy(project.config)
updated["profiles"]["full"] = ["unrelated"]
updated["checks"].append({"id": "unrelated", "argv": ["false"]})
other = InputSnapshotter(
Project("Changed label", project.repositories, updated), workspace_root=root
)
assert fingerprint(other.snapshot(plan)) == fingerprint(before)
def test_scope_command_tools_and_tooling_change_invalidate(fixture):
root, project, engine = fixture
first = engine.snapshot([stage(repos=["alpha"])])
assert fingerprint(engine.snapshot([stage(repos=["beta"])])) != fingerprint(first)
assert fingerprint(
engine.snapshot([stage(repos=["alpha"], argv=["false"])])
) != fingerprint(first)
assert fingerprint(
engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="new-env")
) != fingerprint(first)
changed = deepcopy(project.config)
changed["tools"] = {"node": "/another/node"}
other = InputSnapshotter(
Project("Fixture", project.repositories, changed), workspace_root=root
)
assert fingerprint(other.snapshot([stage(repos=["alpha"])])) != fingerprint(first)
def test_source_only_attestation_is_independent_of_plan_and_tooling(fixture):
_, _, engine = fixture
one = engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="env-one")
two = engine.snapshot(
[stage("different", ["alpha"], argv=["false"])], tooling_fingerprint="env-two"
)
source = engine.source_snapshot(["alpha"])
assert (
one["observed_source_fingerprint"]
== two["observed_source_fingerprint"]
== source["observed_source_fingerprint"]
)
assert (
one["stages"]["one"]["source_fingerprint"]
== source["observed_source_fingerprint"]
)
assert source["fingerprint_version"] == inputs.FINGERPRINT_VERSION
@pytest.mark.parametrize(
"value",
[
None,
{},
{"paths": ["src/**"]},
{"repos": []},
{"repos": "alpha"},
{"repos": ["unknown"]},
{"repos": ["alias-alpha"]},
{"repos": ["alpha", "alpha"]},
{"repos": [""]},
{"repos": ["alpha"], "extra": True},
],
)
def test_synthetic_stage_scopes_fail_closed(fixture, value):
_, _, engine = fixture
with pytest.raises(ValueError):
engine.snapshot([stage(inputs=value)])
def test_duplicate_stage_ids_and_escaped_repository_paths_fail(fixture):
root, project, engine = fixture
with pytest.raises(ValueError, match="Duplicate"):
engine.snapshot([stage(), stage()])
bad = Project("Bad", (Repository("outside", root.parent),), project.config)
with pytest.raises(ValueError, match="escapes"):
InputSnapshotter(bad, workspace_root=root)
def test_declaration_validation_is_pure_before_dry_run(monkeypatch):
monkeypatch.setattr(
inputs,
"git_bytes",
lambda *_a, **_k: pytest.fail("Planning validation ran Git"),
)
assert validate_input_declaration(
{"repos": ["beta", "alpha"]}, {"alpha", "beta"}
) == {"repos": ["alpha", "beta"]}
with pytest.raises(ValueError):
validate_input_declaration({"repos": ["unknown"]}, {"alpha", "beta"})
def test_runtime_records_are_not_hashed_as_execution_plans(fixture):
_, _, engine = fixture
with pytest.raises(ValueError, match="freshly planned"):
engine.snapshot([stage(repos=["alpha"], status="passed")])
def test_membership_count_is_bounded_before_entry_hashing(fixture, monkeypatch):
root, _, engine = fixture
(root / "alpha/new.txt").write_text("extra")
monkeypatch.setattr(inputs, "MAX_REPOSITORY_ENTRIES", 1)
with pytest.raises(ValueError, match="entry count"):
engine.source_snapshot(["alpha"])
def test_symlink_to_ignored_file_inside_repository_binds_target(fixture):
root, _, engine = fixture
repo = root / "alpha"
(repo / ".gitignore").write_text("ignored.txt\n")
(repo / "ignored.txt").write_text("first")
(repo / "linked.txt").symlink_to("ignored.txt")
before = engine.source_snapshot(["alpha"])
(repo / "ignored.txt").write_text("other")
assert (
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
!= before["observed_source_fingerprint"]
)
def test_dangling_symlink_target_creation_is_observed(fixture):
root, _, engine = fixture
repo = root / "alpha"
(repo / "linked.txt").symlink_to("future.txt")
before = engine.source_snapshot(["alpha"])
(repo / "future.txt").write_text("created")
assert (
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
!= before["observed_source_fingerprint"]
)
def test_cross_repository_and_directory_symlinks_are_not_silently_reused(fixture):
root, _, engine = fixture
link = root / "alpha/linked.txt"
link.symlink_to(root / "beta/source.txt")
with pytest.raises(ValueError, match="escapes"):
engine.source_snapshot(["alpha"])
link.unlink()
link.symlink_to(".")
with pytest.raises(ValueError, match="regular file"):
engine.source_snapshot(["alpha"])
def test_file_change_during_hash_is_rejected(fixture, monkeypatch):
root, _, engine = fixture
path = root / "alpha/source.txt"
original = inputs.os.fstat
changed = False
def mutate(descriptor):
nonlocal changed
metadata = original(descriptor)
if not changed:
changed = True
path.write_text("changed during read")
return metadata
monkeypatch.setattr(inputs.os, "fstat", mutate)
with pytest.raises(ValueError, match="changed"):
engine._file_hash(path, inputs._stats())
def test_legacy_whole_project_api_remains_independent(fixture):
_, project, engine = fixture
legacy = source_fingerprint(project)
engine.snapshot([stage(repos=["alpha"])])
assert source_fingerprint(project) == legacy
def test_real_tooling_inventory_is_bound_and_memoized(tmp_path):
repo = Repository("example", tmp_path)
project = Project("Fixture", (repo,), {"tools": {}})
engine = InputSnapshotter(project, workspace_root=tmp_path)
before_stats, after_stats = inputs._stats(), inputs._stats()
before = engine._tooling_identity(before_stats)
after = engine._tooling_identity(after_stats)
assert before == after
assert before_stats["tooling_files"] > 15 and before_stats["tooling_bytes"] > 0
assert after_stats["tooling_files"] == after_stats["tooling_cache_hits"]
assert after_stats["tooling_bytes"] == 0
+387
View File
@@ -0,0 +1,387 @@
import argparse
from copy import deepcopy
import json
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import issues
from govoplan_devkit.common import atomic_json, digest, state_root
from govoplan_devkit.workspace import load_project, source_fingerprint
@pytest.fixture
def args(tmp_path, monkeypatch):
monkeypatch.delenv("GITEA_TOKEN", raising=False)
root = tmp_path / "repo"
root.mkdir()
subprocess.run(["git", "init", "-q", str(root)], check=True)
subprocess.run(["git", "-C", str(root), "remote", "add", "origin", "https://gitea.invalid/team/repo.git"], check=True)
project = tmp_path / "project.json"
project.write_text(json.dumps({"schema_version": 1, "name": "Fixture", "repositories": [{"name": "repo", "path": "repo"}]}))
env_file = tmp_path / "private.env"
env_file.write_text("GITEA_TOKEN=fixture-private-token\nGITEA_OWNER=wrong-owner\n")
return argparse.Namespace(workspace_root=tmp_path, state_dir=tmp_path / "state", project=project,
root=root, issue=7, target_plan=None, remote="origin", env_file=env_file, evidence=None,
key="verification", note_summary=["Verified scoped changes."], next_steps=["Manual review remains."],
body_file=None, note_file=None, apply=False, retry_uncertain=False)
class FakeClient:
def __init__(self, target):
self.target = target
self.calls = []
self.comments = []
self.issue_id = 700
self.post_mode = "normal"
self.bad_binding = False
self.repeat_pages = False
self.bad_issue = False
self.fail = False
self.page_size = 2 # A server may enforce a cap smaller than the requested limit.
def close(self):
pass
def request_json(self, method, path, body=None, query=None):
self.calls.append((method, path, deepcopy(body), deepcopy(query)))
if self.fail:
raise RuntimeError("Remote echoed fixture-private-token")
if method == "GET" and path == self.target.path:
return {"id": self.issue_id, "number": self.target.issue, "html_url": self.target.url + ("wrong" if self.bad_issue else ""),
"state": "closed", "body": "- [ ] Preserve this checklist"}
if method == "GET" and path == self.target.path + "/comments":
page = 1 if self.repeat_pages else query["page"]
return deepcopy(self.comments[(page - 1) * self.page_size:page * self.page_size])
if method == "GET" and "/issues/comments/" in path:
comment = deepcopy(next(item for item in self.comments if str(item["id"]) == path.rsplit("/", 1)[1]))
comment["html_url"] = self.target.url + ("-other" if self.bad_binding else "") + "#issuecomment-" + str(comment["id"])
return comment
if method == "POST" and path == self.target.path + "/comments":
if self.post_mode == "timeout-before-commit":
raise TimeoutError("fixture-private-token")
comment = {"id": 1000 + len(self.comments), "body": body["body"]}
self.comments.append(comment)
if self.post_mode == "timeout-after-commit":
raise TimeoutError("fixture-private-token")
return deepcopy(comment)
raise AssertionError((method, path))
@property
def posts(self):
return [call for call in self.calls if call[0] == "POST"]
@pytest.fixture
def client(args, monkeypatch):
client = FakeClient(issues.resolve_target(args.root, args.issue, args.workspace_root))
monkeypatch.setattr(issues, "make_client", lambda target, token: client)
return client
def test_default_dry_run_is_offline_and_ignores_ambient_target_overrides(args, monkeypatch):
monkeypatch.setenv("GITEA_OWNER", "wrong")
monkeypatch.setenv("GITEA_REPO", "wrong")
monkeypatch.setenv("GITEA_URL", "https://wrong.invalid")
monkeypatch.setattr(issues, "make_client", lambda *_: pytest.fail("No network in preview"))
args.env_file = args.workspace_root / "does-not-exist.env"
result = issues.handle_note(args)
assert result["targets"][0]["url"] == "https://gitea.invalid/team/repo/issues/7"
assert result["targets"][0]["status"] == "would-post"
assert not args.state_dir.exists()
def test_scoped_checkpoint_evidence_compares_only_recorded_sources(args, monkeypatch):
from govoplan_devkit import runner
other = args.workspace_root / "other"
other.mkdir()
subprocess.run(["git", "init", "-q", str(other)], check=True)
declaration = json.loads(args.project.read_text())
declaration["repositories"].append({"name": "other", "path": "other"})
args.project.write_text(json.dumps(declaration))
monkeypatch.setattr(runner, "environment_fingerprint", lambda *_: "fixture-env")
run_args = argparse.Namespace(**{**vars(args), "jobs": 1, "profile": "quick", "dry_run": False})
result = runner.run_checks(run_args, [{"id": "scoped", "argv": [sys.executable, "-c", "print('ok')"],
"cwd": str(args.root), "inputs": {"repos": ["repo"]}}])
assert result["status"] == "passed"
evidence = issues.evidence_record(result["run_id"], args)
assert evidence["source_state"] == "matches-current"
assert evidence["source_scope"]["repos"] == ["repo"]
assert any("recorded repository input scope" in note for note in evidence["coverage_notes"])
(other / "unrelated.txt").write_text("Outside the recorded scope")
assert issues.evidence_record(result["run_id"], args)["source_state"] == "matches-current"
(args.root / "changed.txt").write_text("Inside the recorded scope")
assert issues.evidence_record(result["run_id"], args)["source_state"] == "historical-source-differs"
def test_passing_aggregate_cannot_hide_skipped_checks(args):
payload = receipt(args)
payload["stages"].append({"id": "skipped", "status": "skipped", "exit_code": None})
with pytest.raises(ValueError, match="inconsistent"):
issues.validate_receipt(payload, args.workspace_root)
def test_apply_is_append_only_read_back_and_idempotent(args, client):
args.apply = True
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "posted-verified"
assert len(client.posts) == 1
again = issues.handle_note(args)
assert again["targets"][0]["status"] == "existing-verified"
assert len(client.posts) == 1
assert {call[0] for call in client.calls} == {"GET", "POST"}
assert all(call[1].endswith("/comments") for call in client.posts)
def test_complete_pagination_continues_past_short_pages(args, client):
body = issues.handle_note(args)["targets"][0]["body"]
client.comments = [{"id": n, "body": "unrelated"} for n in range(1, 6)] + [{"id": 6, "body": body}]
args.apply = True
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "existing-verified"
assert not client.posts
assert max(call[3]["page"] for call in client.calls if call[3]) == 4
@pytest.mark.parametrize("collision", ["different-body", "duplicate", "repeating-pagination"])
def test_collisions_and_incomplete_pagination_refuse_post(args, client, collision):
body = issues.handle_note(args)["targets"][0]["body"]
client.comments = [{"id": 1, "body": body}]
if collision == "different-body":
client.comments[0]["body"] += "changed"
elif collision == "duplicate":
client.comments.append({"id": 2, "body": body})
else:
client.repeat_pages = True
args.apply = True
assert issues.handle_note(args)["_exit_code"] == 2
assert not client.posts
def test_timeout_after_commit_reconciles_without_replaying_post(args, client):
args.apply = True
client.post_mode = "timeout-after-commit"
result = issues.handle_note(args)
assert result["targets"][0]["status"] == "reconciled-verified"
assert len(client.posts) == 1
assert "fixture-private-token" not in json.dumps(result)
def test_uncertain_post_requires_explicit_retry_after_reconciliation(args, client):
args.apply = True
client.post_mode = "timeout-before-commit"
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
client.post_mode = "normal"
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain-retry-required"
assert len(client.posts) == 1
args.retry_uncertain = True
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
assert len(client.posts) == 2
journals = list(state_root(args.workspace_root, args.state_dir).glob("issue-notes/*.json"))
assert journals and all("fixture-private-token" not in path.read_text() for path in journals)
def test_uncertain_readback_is_not_reported_verified_and_later_reconciles(args, client):
args.apply = True
client.bad_binding = True
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
client.bad_binding = False
assert issues.handle_note(args)["targets"][0]["status"] == "existing-verified"
assert len(client.posts) == 1
def test_journal_binds_immutable_issue_id(args, client):
args.apply = True
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
client.issue_id += 1
client.comments.clear()
args.retry_uncertain = True
assert issues.handle_note(args)["_exit_code"] == 2
assert len(client.posts) == 1
def _plan(args, rows):
args.root = args.issue = None
args.target_plan = args.workspace_root / "targets.json"
args.target_plan.write_text(json.dumps({"schema_version": 1, "targets": rows}))
@pytest.mark.parametrize("change", ["wrong-url", "duplicate", "outside", "mixed-origin", "credentials"])
def test_target_plans_require_exact_unique_workspace_bindings(args, change):
row = {"root": "repo", "issue": 7, "url": "https://gitea.invalid/team/repo/issues/7"}
if change == "wrong-url":
row["url"] = "https://gitea.invalid/team/other/issues/7"
elif change == "outside":
row["root"] = "../other"
elif change == "credentials":
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", "https://user:private@gitea.invalid/team/repo.git"], check=True)
rows = [row, deepcopy(row)] if change == "duplicate" else [row]
if change == "mixed-origin":
other = args.workspace_root / "other"
other.mkdir()
subprocess.run(["git", "init", "-q", str(other)], check=True)
subprocess.run(["git", "-C", str(other), "remote", "add", "origin", "http://gitea.invalid/team/other.git"], check=True)
rows.append({"root": "other", "issue": 8, "url": "http://gitea.invalid/team/other/issues/8"})
_plan(args, rows)
with pytest.raises(ValueError):
issues.handle_note(args)
def test_all_targets_preflight_before_first_post_and_posts_are_serial(args, client, monkeypatch):
first = client.target.record()
second = {**first, "issue": 8, "url": first["url"].rsplit("/", 1)[0] + "/8"}
_plan(args, [first, second])
second_client = FakeClient(issues.resolve_target(Path(first["root"]), 8, args.workspace_root))
second_client.bad_issue = True
monkeypatch.setattr(issues, "make_client", lambda target, token: client if target.issue == 7 else second_client)
args.apply = True
assert issues.handle_note(args)["_exit_code"] == 2
assert not client.posts and not second_client.posts
second_client.bad_issue = False
result = issues.handle_note(args)
assert [row["status"] for row in result["targets"]] == ["posted-verified", "posted-verified"]
assert len(client.posts) == len(second_client.posts) == 1
def test_credentials_and_response_errors_are_not_returned(args, client):
args.apply = True
client.fail = True
result = issues.handle_note(args)
assert result["_exit_code"] == 2
assert "fixture-private-token" not in json.dumps(result)
args.note_summary = ["fixture-private-token"]
with pytest.raises(ValueError, match="credential"):
issues.handle_note(args)
def test_marker_injection_and_symlink_inputs_are_rejected(args):
args.note_summary = [issues.MARKER_PREFIX + "fake -->"]
with pytest.raises(ValueError, match="reserved"):
issues.handle_note(args)
args.note_summary = ["safe"]
args.body_file = args.workspace_root / "alias.md"
args.body_file.symlink_to(args.env_file)
with pytest.raises(ValueError, match="symlink"):
issues.handle_note(args)
@pytest.mark.parametrize("remote", ["https://gitea.invalid/team/repo.git?token=secret", "https://gitea.invalid/team/repo.git#other", "https://gitea.invalid:bad/team/repo.git"])
def test_ambiguous_git_remote_is_not_silently_reinterpreted(args, remote):
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", remote], check=True)
with pytest.raises(ValueError):
issues.handle_note(args)
def test_structured_inputs_are_merged_as_data_not_executed(args):
args.note_file = args.workspace_root / "note.json"
args.note_file.write_text(json.dumps({"summary": ["Recorded earlier"], "next": ["Still pending"], "body": "$(never-execute)"}))
args.body_file = args.workspace_root / "body.md"
args.body_file.write_text("`never-run-this-either`")
body = issues.handle_note(args)["targets"][0]["body"]
assert "Recorded earlier" in body and "Verified scoped changes" in body
assert "$(never-execute)" in body and "`never-run-this-either`" in body
def receipt(args):
return {"schema_version": 1, "run_id": "fixture-run", "workspace_root": str(args.workspace_root),
"project_file": str(args.project), "source_fingerprint": source_fingerprint(load_project(args.workspace_root, args.project)),
"status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08T12:00:00Z", "finished_at": "2026-09-08T12:00:01Z",
"stages": [{"id": "fixture", "status": "passed", "exit_code": 0, "duration_seconds": 1,
"log_path": "/private/log-not-opened", "argv": ["never-execute-this"]}]}
def test_external_receipts_are_unverified_metadata_with_source_comparison(args):
path = args.workspace_root / "external.json"
path.write_text(json.dumps(receipt(args)))
args.evidence = str(path)
result = issues.handle_note(args)
evidence = result["evidence"]
assert evidence["origin"] == "external-unverified"
assert evidence["source_state"] == "matches-current"
assert "argv" not in evidence["stages"][0]
(args.root / "dirty.txt").write_text("changed")
assert issues.handle_note(args)["evidence"]["source_state"] == "historical-source-differs"
@pytest.mark.parametrize("mutation", ["foreign", "bad-fingerprint", "false-pass", "duplicate-stage", "unknown-status", "noninteger-exit", "invalid-state-shape", "invalid-stage-state-shape", "bool-schema"])
def test_invalid_receipt_cannot_supply_evidence(args, mutation):
payload = receipt(args)
if mutation == "foreign":
payload["workspace_root"] = str(args.workspace_root.parent)
elif mutation == "bad-fingerprint":
payload["source_fingerprint"] = "claimed-green"
elif mutation == "false-pass":
payload["stages"][0]["exit_code"] = 1
elif mutation == "duplicate-stage":
payload["stages"] *= 2
elif mutation == "unknown-status":
payload["status"] = "complete-review"
elif mutation == "invalid-state-shape":
payload["status"] = {}
elif mutation == "invalid-stage-state-shape":
payload["stages"][0]["status"] = []
elif mutation == "bool-schema":
payload["schema_version"] = True
else:
payload["stages"][0]["exit_code"] = False
with pytest.raises(ValueError):
issues.validate_receipt(payload, args.workspace_root)
def test_local_receipt_integrity_is_checked_but_not_an_attestation(args):
payload = receipt(args)
payload["integrity_sha256"] = digest(payload)
path = state_root(args.workspace_root, args.state_dir) / "runs/fixture-run/receipt.json"
atomic_json(path, payload)
args.evidence = "fixture-run"
result = issues.handle_note(args)
assert result["evidence"]["origin"] == "local-integrity-checked"
assert "not an independent attestation" in result["evidence"]["attestation"]
payload["status"] = "failed"
atomic_json(path, payload)
with pytest.raises(ValueError, match="integrity"):
issues.handle_note(args)
def test_timed_out_stage_is_reportable_without_claiming_success(args):
payload = receipt(args)
payload["status"] = "failed"
payload["stages"][0].update(status="timed_out", exit_code=-15)
assert issues.validate_receipt(payload, args.workspace_root)["status"] == "failed"
def test_scoped_coverage_limits_are_preserved_in_evidence_and_rendered_note(args, monkeypatch):
payload = receipt(args)
monkeypatch.setenv("FIXTURE_SECRET", "fixture-secret-value")
payload["stages"][0]["coverage_notes"] = ["Compiler/chained suite not run.", "Manual <script>not executed</script> fixture-secret-value"]
path = args.workspace_root / "coverage.json"
path.write_text(json.dumps(payload))
args.evidence = str(path)
result = issues.handle_note(args)
evidence = result["evidence"]
assert evidence["coverage_notes"] == evidence["stages"][0]["coverage_notes"]
assert "Compiler/chained suite not run." in result["targets"][0]["body"]
assert "omitted checks ran" in result["targets"][0]["body"]
assert "<script>" not in result["targets"][0]["body"]
assert "fixture-secret-value" not in json.dumps(result)
assert any("coverage limitation" in line for line in result["summary"])
@pytest.mark.parametrize("notes", ["not-a-list", [None], [""], ["x" * 4097], ["limit"] * 2049, [issues.MARKER_PREFIX + "injection"]])
def test_malformed_coverage_metadata_is_rejected(args, notes):
payload = receipt(args)
payload["stages"][0]["coverage_notes"] = notes
with pytest.raises(ValueError):
issues.validate_receipt(payload, args.workspace_root)
@pytest.mark.parametrize("snapshot_verified", [None, False, "true", 1])
def test_external_pass_requires_verified_snapshot_flag(args, snapshot_verified):
payload = receipt(args)
payload["snapshot_verified"] = snapshot_verified
with pytest.raises(ValueError, match="verified source snapshot"):
issues.validate_receipt(payload, args.workspace_root)
+539
View File
@@ -0,0 +1,539 @@
"""Git maintenance runs only in disposable local repositories/bare fixtures."""
from __future__ import annotations
import argparse
from contextlib import contextmanager
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import time
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import maintenance
class MaintenanceTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-git-")
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.repo = self.root / "repo"
self.repo.mkdir()
self.git("init", "-b", "main")
self.git("config", "user.name", "Fixture")
self.git("config", "user.email", "fixture@example.invalid")
self.git("config", "commit.gpgsign", "false")
self.write("selected.txt", "base selected\n")
self.write("unrelated.txt", "base unrelated\n")
self.git("add", "--", "selected.txt", "unrelated.txt")
self.git("commit", "-m", "fixture base")
self.base = self.git("rev-parse", "HEAD").strip()
self.project = self.root / "project.json"
self.project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "fixture",
"repositories": [{"name": "repo", "path": "repo"}],
}
)
)
self.args = argparse.Namespace(
workspace_root=self.root,
state_dir=self.root / "state",
project=self.project,
repo="repo",
path=["selected.txt"],
message="selected change",
apply=False,
)
def git(self, *args):
return subprocess.run(
["git", "-C", str(self.repo), *args],
check=True,
capture_output=True,
text=True,
).stdout
def write(self, name, value):
(self.repo / name).write_text(value)
def save_plan(self):
self.args.apply = True
result = maintenance.plan(self.args)
self.args.plan_id = result["plan"]["plan_id"]
return result
def test_preview_does_not_write_a_plan_or_stage(self):
self.write("selected.txt", "changed\n")
before = self.git("ls-files", "--stage")
result = maintenance.plan(self.args)
self.assertEqual(result["state"]["status"], "preview")
self.assertFalse((self.root / "state").exists())
self.assertEqual(self.git("ls-files", "--stage"), before)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_selected_commit_preserves_unrelated_staged_and_dirty_changes(self):
self.write("selected.txt", "selected final\n")
self.write("unrelated.txt", "unrelated staged\n")
self.git("add", "--", "unrelated.txt")
self.write("unrelated.txt", "unrelated unstaged\n")
staged = self.git("show", ":unrelated.txt")
self.save_plan()
before_plan = (
maintenance._receipt_path(self.args, self.args.plan_id)
.joinpath("plan.json")
.read_bytes()
)
self.args.apply = False
self.assertEqual(maintenance.commit(self.args)["state"]["status"], "preview")
self.args.apply = True
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(
self.git(
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
).strip(),
"selected.txt",
)
self.assertEqual(self.git("show", ":unrelated.txt"), staged)
self.assertEqual(
(self.repo / "unrelated.txt").read_text(), "unrelated unstaged\n"
)
self.assertEqual(
maintenance.commit(self.args)["state"]["commit"], result["state"]["commit"]
)
self.assertEqual(
maintenance._receipt_path(self.args, self.args.plan_id)
.joinpath("plan.json")
.read_bytes(),
before_plan,
)
def test_new_selected_file_and_deletion_are_supported(self):
self.write("new file.txt", "new\n")
(self.repo / "selected.txt").unlink()
self.git("add", "--", "selected.txt")
self.args.path = ["new file.txt", "selected.txt"]
self.save_plan()
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(
set(
self.git(
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
).splitlines()
),
{"new file.txt", "selected.txt"},
)
def test_selected_partial_staging_is_not_overwritten(self):
self.write("selected.txt", "staged\n")
self.git("add", "--", "selected.txt")
self.write("selected.txt", "unstaged\n")
with self.assertRaisesRegex(ValueError, "different staged"):
self.save_plan()
self.assertEqual(self.git("show", ":selected.txt"), "staged\n")
def test_stale_content_index_and_origin_block_commit(self):
for change in ("content", "index", "origin"):
with self.subTest(change=change):
self.write("selected.txt", f"selected {change}\n")
self.save_plan()
if change == "content":
self.write("selected.txt", "different content\n")
elif change == "index":
self.write("unrelated.txt", "changed index\n")
self.git("add", "--", "unrelated.txt")
else:
self.git("remote", "add", "origin", str(self.root / "other.git"))
with self.assertRaisesRegex(ValueError, "changed; prepare a new plan"):
maintenance.commit(self.args)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_active_hooks_and_filters_are_refused_not_bypassed(self):
self.write("selected.txt", "changed\n")
hook = self.repo / ".git/hooks/pre-commit"
hook.write_text("#!/bin/sh\nexit 99\n")
hook.chmod(0o700)
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
self.save_plan()
hook.unlink()
self.git("config", "filter.example.clean", "some-external-command")
self.write(".gitattributes", "*.txt filter=example\n")
with self.assertRaisesRegex(ValueError, "Active Git filter"):
self.save_plan()
def test_directory_traversal_and_symlinks_are_refused(self):
self.write("selected.txt", "changed\n")
(self.repo / "linked.txt").symlink_to(self.repo / "selected.txt")
for name in (".", "../outside", ".git/config", "linked.txt"):
with self.subTest(name=name):
self.args.path = [name]
with self.assertRaises(ValueError):
self.save_plan()
def test_post_index_change_hook_is_refused_before_it_can_run(self):
self.write("selected.txt", "changed\n")
marker = self.root / "hook-executed"
hook = self.repo / ".git/hooks/post-index-change"
hook.write_text(f"#!/bin/sh\ntouch '{marker}'\n")
hook.chmod(0o700)
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
self.save_plan()
self.assertFalse(marker.exists())
def test_remote_helpers_and_recursive_operations_are_refused(self):
self.write("selected.txt", "changed\n")
for setting in (
"remote.origin.receivepack",
"remote.origin.uploadpack",
"remote.origin.vcs",
"core.gitProxy",
"core.alternateRefsCommand",
"push.recurseSubmodules",
"submodule.recurse",
"remote.origin.promisor",
):
with self.subTest(setting=setting):
self.git("config", setting, "true")
with self.assertRaises(ValueError):
self.save_plan()
self.git("config", "--unset", setting)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_git_namespace_identity_and_repository_overrides_are_refused(self):
self.write("selected.txt", "changed\n")
for key in (
"GIT_NAMESPACE",
"GIT_COMMON_DIR",
"GIT_QUARANTINE_PATH",
"GIT_SHALLOW_FILE",
"GIT_REPLACE_REF_BASE",
"GIT_ALTERNATE_OBJECT_DIRECTORIES",
"GIT_AUTHOR_NAME",
"GIT_CONFIG_COUNT",
"GIT_UNKNOWN_OVERRIDE",
):
with self.subTest(key=key), patch.dict(os.environ, {key: "unexpected"}):
with self.assertRaisesRegex(ValueError, "environment overrides"):
self.save_plan()
self.assertFalse((self.root / "state").exists())
def test_noncanonical_paths_are_rejected_before_commit(self):
(self.repo / "dir").mkdir()
self.write("dir/file.txt", "new\n")
self.args.path = ["dir//file.txt"]
with self.assertRaisesRegex(ValueError, "explicit relative"):
self.save_plan()
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_replacement_history_is_not_treated_as_the_real_frozen_tree(self):
self.write("selected.txt", "changed\n")
tree = self.git("rev-parse", "HEAD^{tree}").strip()
replacement = self.git("commit-tree", tree, "-m", "replacement fixture").strip()
self.git("replace", self.base, replacement)
with self.assertRaisesRegex(ValueError, "replacement/graft"):
self.save_plan()
def test_normal_push_is_explicit_and_verified_against_a_local_bare_fixture(self):
remote = self.root / "origin.git"
subprocess.run(
["git", "init", "--bare", str(remote)], check=True, capture_output=True
)
self.git("remote", "add", "origin", str(remote))
self.write("selected.txt", "selected final\n")
frozen = self.save_plan()
self.assertNotIn(str(remote), json.dumps(frozen))
result = maintenance.commit(self.args)
self.args.apply = False
self.assertEqual(maintenance.push(self.args)["state"]["status"], "preview")
self.assertEqual(
self.git("ls-remote", "--refs", "origin", "refs/heads/main"), ""
)
self.args.apply = True
pushed = maintenance.push(self.args)
self.assertEqual(pushed["state"]["status"], "pushed")
self.assertTrue(
self.git("ls-remote", "--refs", "origin", "refs/heads/main").startswith(
result["state"]["commit"]
)
)
def test_interrupted_commit_is_reconciled_without_creating_another_commit(self):
self.write("selected.txt", "selected final\n")
self.save_plan()
original = maintenance._write_state
def fail_receipt(directory, state, **updates):
if updates.get("status") == "committed":
raise OSError("fixture interruption after commit")
return original(directory, state, **updates)
with patch.object(maintenance, "_write_state", side_effect=fail_receipt):
with self.assertRaises(OSError):
maintenance.commit(self.args)
head = self.git("rev-parse", "HEAD").strip()
self.assertNotEqual(head, self.base)
with self.assertRaisesRegex(ValueError, "reconcile"):
maintenance.commit(self.args)
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
def test_lost_push_receipt_is_reconciled_without_repeating_push(self):
remote = self.root / "origin.git"
subprocess.run(
["git", "init", "--bare", str(remote)], check=True, capture_output=True
)
self.git("remote", "add", "origin", str(remote))
self.write("selected.txt", "selected final\n")
self.save_plan()
maintenance.commit(self.args)
original = maintenance._git
pushes = []
def lose_response(repo, *argv, **kwargs):
result = original(repo, *argv, **kwargs)
if argv[0] == "push":
pushes.append(argv)
raise ValueError("fixture response lost after remote update")
return result
with patch.object(maintenance, "_git", side_effect=lose_response):
with self.assertRaises(ValueError):
maintenance.push(self.args)
with self.assertRaisesRegex(ValueError, "reconcile"):
maintenance.push(self.args)
reconciled = maintenance.reconcile(self.args)
self.assertEqual(reconciled["state"]["status"], "pushed")
self.assertEqual(len(pushes), 1)
def test_reconciliation_rechecks_state_after_acquiring_lock(self):
self.write("selected.txt", "selected final\n")
self.save_plan()
maintenance.commit(self.args)
payload, state, directory = maintenance._read(self.args, self.args.plan_id)
maintenance._write_state(
directory, state, status="needs_reconcile", operation="commit"
)
@contextmanager
def another_reconciliation(_args, _repo):
maintenance._write_state(directory, state, status="committed")
yield
with patch.object(maintenance, "_locked", another_reconciliation):
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertTrue(
any("no replay or downgrade" in line for line in result["summary"])
)
def test_worktree_edit_racing_with_commit_remains_uncommitted(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._git
def race(repo, *argv, **kwargs):
if argv[0] == "commit-tree":
self.write("selected.txt", "new editor bytes\n")
return original(repo, *argv, **kwargs)
with patch.object(maintenance, "_git", side_effect=race):
result = maintenance.commit(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("show", "HEAD:selected.txt"), "approved bytes\n")
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
self.assertEqual((self.repo / "selected.txt").read_text(), "new editor bytes\n")
def test_branch_compare_and_swap_never_overwrites_a_racing_commit(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._git
raced = []
def race(repo, *argv, **kwargs):
if argv[0] == "update-ref" and not raced:
tree = self.git("rev-parse", "HEAD^{tree}").strip()
other = self.git(
"commit-tree", tree, "-p", self.base, "-m", "other writer"
).strip()
self.git("update-ref", "refs/heads/main", other, self.base)
raced.append(other)
return original(repo, *argv, **kwargs)
with patch.object(maintenance, "_git", side_effect=race):
with self.assertRaises(ValueError):
maintenance.commit(self.args)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), raced[0])
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
def test_interrupted_index_publication_reconciles_without_another_commit(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
with patch.object(
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
):
with self.assertRaises(OSError):
maintenance.commit(self.args)
head = self.git("rev-parse", "HEAD").strip()
self.assertNotEqual(head, self.base)
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
result = maintenance.reconcile(self.args)
self.assertEqual(result["state"]["status"], "committed")
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
def test_concurrent_index_changes_are_preserved_instead_of_overwritten(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._publish_index
def race(repo, locked, prepared, expected):
# Model an external writer that does not honor Git's index.lock.
outside = self.root / "outside.index"
outside.write_bytes((self.repo / ".git/index").read_bytes())
self.write("unrelated.txt", "new independent staging\n")
subprocess.run(
["git", "-C", str(self.repo), "add", "--", "unrelated.txt"],
env={**os.environ, "GIT_INDEX_FILE": str(outside)},
check=True,
capture_output=True,
)
(self.repo / ".git/index").write_bytes(outside.read_bytes())
return original(repo, locked, prepared, expected)
with patch.object(maintenance, "_publish_index", side_effect=race):
with self.assertRaisesRegex(ValueError, "index changed concurrently"):
maintenance.commit(self.args)
self.assertEqual(
self.git("show", ":unrelated.txt"), "new independent staging\n"
)
with self.assertRaises(ValueError):
maintenance.reconcile(self.args)
self.assertEqual(
self.git("show", ":unrelated.txt"), "new independent staging\n"
)
def test_selected_fifo_swap_cannot_block_capture(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._capture_blob
def race(repo, item):
(self.repo / "selected.txt").unlink()
os.mkfifo(self.repo / "selected.txt")
return original(repo, item)
started = time.monotonic()
with patch.object(maintenance, "_capture_blob", side_effect=race):
with self.assertRaises(ValueError):
maintenance.commit(self.args)
self.assertLess(time.monotonic() - started, 3)
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
def test_reconcile_never_adopts_a_head_that_races_with_index_preparation(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
with patch.object(
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
):
with self.assertRaises(OSError):
maintenance.commit(self.args)
candidate = self.git("rev-parse", "HEAD").strip()
original = maintenance._selected_index
raced = []
def race(repo, source, target, files):
result = original(repo, source, target, files)
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
other = self.git(
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
).strip()
self.git("update-ref", "refs/heads/main", other, candidate)
raced.append(other)
return result
with patch.object(maintenance, "_selected_index", side_effect=race):
with self.assertRaisesRegex(
ValueError, "changed during index reconciliation"
):
maintenance.reconcile(self.args)
recorded = maintenance.status(self.args)["state"]
self.assertEqual(recorded["status"], "needs_reconcile")
self.assertEqual(recorded["commit"], candidate)
self.assertNotEqual(recorded["commit"], raced[0])
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
def test_final_commit_receipt_cannot_bind_to_a_later_head(self):
self.write("selected.txt", "approved bytes\n")
self.save_plan()
original = maintenance._publish_index
raced = []
def race(repo, locked, prepared, expected):
original(repo, locked, prepared, expected)
candidate = self.git("rev-parse", "HEAD").strip()
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
other = self.git(
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
).strip()
self.git("update-ref", "refs/heads/main", other, candidate)
raced.append(other)
with patch.object(maintenance, "_publish_index", side_effect=race):
with self.assertRaisesRegex(
ValueError, "changed before the exact commit result"
):
maintenance.commit(self.args)
recorded = maintenance.status(self.args)["state"]
self.assertEqual(recorded["status"], "needs_reconcile")
self.assertNotEqual(recorded["commit"], raced[0])
def test_git_output_is_bounded_during_execution(self):
executable = self.root / "git"
executable.write_text(f"#!{sys.executable}\nprint('x' * 100000)\n")
executable.chmod(0o700)
with (
patch.dict(
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
),
patch.object(maintenance, "MAX_GIT_OUTPUT_BYTES", 256),
):
with self.assertRaisesRegex(ValueError, "output_limit"):
maintenance._git(self.repo, "fixture")
def test_timeout_terminates_git_helper_process_group(self):
executable = self.root / "git"
marker = self.root / "helper-ran"
child = (
"import signal,time,pathlib; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(1); pathlib.Path("
+ repr(str(marker))
+ ").write_text('unexpected')"
)
executable.write_text(
f"#!{sys.executable}\nimport subprocess,sys,time\nsubprocess.Popen([sys.executable, '-c', {child!r}])\ntime.sleep(20)\n"
)
executable.chmod(0o700)
with patch.dict(
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
):
with self.assertRaisesRegex(ValueError, "timed_out"):
maintenance._git(self.repo, "fixture", timeout=0.2)
time.sleep(1.1)
self.assertFalse(
marker.exists(),
"An orphaned helper must not finish the operation after timeout",
)
if __name__ == "__main__":
unittest.main()
+319
View File
@@ -0,0 +1,319 @@
"""Live/provisional monitoring and discovery never substitute for verified evidence."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
sys.path.insert(0, str(Path(__file__).resolve().parent))
from test_devkit_runner import example as example, run, stage
from govoplan_devkit import catalog, cli, monitoring, runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import atomic_json, read_json, state_root
from govoplan_devkit.process import OutputSnapshot
def command(args, *argv):
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers(dest="command", required=True))
selected = parser.parse_args(argv, namespace=Namespace(**vars(args)))
return selected.handler(selected)
def test_preparing_receipt_and_run_id_exist_before_source_fingerprinting(example):
args, repo = example
events = []
args.on_progress = events.append
original = Checkpoints.source
def fingerprint(self, *values, **kwargs):
assert events and events[0]["phase"] == "preparing"
first = events[0]
assert Path(first["receipt_path"]).is_file()
record = runner.read_receipt(
args.workspace_root, args.state_dir, first["run_id"]
)
if record["phase"] == "preparing":
assert record["status"] == "running"
assert record["snapshot_verified"] is False
assert record["source_fingerprint"] is None
return original(self, *values, **kwargs)
with patch.object(Checkpoints, "source", fingerprint):
result = run(args, [stage(repo)])
assert result["status"] == "passed"
assert events[-1]["phase"] == "finished"
assert events[-1]["run_id"] == events[0]["run_id"] == result["run_id"]
def test_preflight_failure_persists_discoverable_nonpassing_receipt(example):
args, repo = example
events = []
args.on_progress = events.append
with patch.object(
Checkpoints,
"source",
side_effect=ValueError("fixture fingerprint unavailable"),
):
with pytest.raises(ValueError, match="fixture fingerprint unavailable"):
run(args, [stage(repo)])
assert events[0]["phase"] == "preparing"
record = runner.read_receipt(
args.workspace_root, args.state_dir, events[0]["run_id"]
)
assert record["status"] == "failed"
assert record["phase"] == "finished"
assert record["snapshot_verified"] is False
assert all(item["status"] != "passed" for item in record["stages"])
assert "fixture fingerprint unavailable" in record["error"]
assert monitoring.latest_run(args)["status"] == "failed"
def test_run_history_is_read_only_and_cursor_pagination_has_no_duplicates(example):
args, repo = example
created = [run(args, [stage(repo)])["run_id"] for _ in range(3)]
base = state_root(args.workspace_root, args.state_dir)
before = {
path: (path.stat().st_mtime_ns, path.read_bytes())
for path in base.rglob("*")
if path.is_file()
}
page = monitoring.list_runs(Namespace(**vars(args), limit=2, before=None))
assert [item["run_id"] for item in page["runs"]] == sorted(created, reverse=True)[
:2
]
assert page["next_cursor"]
next_page = monitoring.list_runs(
Namespace(**vars(args), limit=2, before=page["next_cursor"])
)
assert [item["run_id"] for item in next_page["runs"]] == sorted(
created, reverse=True
)[2:]
assert next_page["next_cursor"] is None
assert monitoring.latest_run(args)["run_id"] == max(created)
assert before == {
path: (path.stat().st_mtime_ns, path.read_bytes())
for path in base.rglob("*")
if path.is_file()
}
def test_empty_run_history_does_not_create_a_state_directory(example):
args, _ = example
assert not args.state_dir.exists()
result = monitoring.list_runs(args)
assert result["runs"] == []
assert monitoring.latest_run(args)["status"] == "not_found"
assert not args.state_dir.exists()
def test_invalid_newest_run_is_visible_and_never_replaced_with_older_pass(example):
args, repo = example
older = run(args, [stage(repo)])["run_id"]
invalid = "zz-invalid-newest"
path = (
state_root(args.workspace_root, args.state_dir)
/ "runs"
/ invalid
/ "receipt.json"
)
atomic_json(path, {"malformed": True})
rows = monitoring.list_runs(args)
assert rows["runs"][0]["run_id"] == invalid
assert rows["runs"][0]["status"] == "invalid"
assert any(row["run_id"] == older for row in rows["runs"])
assert rows["_exit_code"] == 1
with pytest.raises(ValueError, match="Latest run is invalid"):
monitoring.latest_run(args)
@pytest.mark.parametrize(
"changes", [{"limit": 0}, {"limit": 101}, {"limit": True}, {"before": "../escape"}]
)
def test_history_rejects_invalid_bounds_and_cursor(example, changes):
args, _ = example
with pytest.raises(ValueError):
monitoring.list_runs(Namespace(**{**vars(args), **changes}))
def test_live_logs_are_available_before_completion_but_final_only_refuses_them(example):
args, repo = example
events, results, errors = [], [], []
args.on_progress = events.append
def execute():
try:
results.append(
run(
args,
[
stage(
repo,
code="import time; print('live ready',flush=True); time.sleep(1.5); print('finished')",
)
],
)
)
except BaseException as exc:
errors.append(exc)
worker = threading.Thread(target=execute)
worker.start()
try:
deadline = time.monotonic() + 4
live = None
while time.monotonic() < deadline:
if events:
live = command(args, "logs", events[0]["run_id"], "--stage", "one")
if "live ready" in live["excerpt"]:
break
time.sleep(0.02)
assert live is not None and "live ready" in live["excerpt"]
assert live["provisional"] is True
assert live["log_verified"] is False
assert live["snapshot_verified"] is False
assert live["run_status"] == "running"
with pytest.raises(ValueError, match="No finalized stage log"):
command(args, "logs", events[0]["run_id"], "--stage", "one", "--final-only")
finally:
worker.join(timeout=5)
assert not worker.is_alive() and not errors
final = command(
args, "logs", results[0]["run_id"], "--stage", "one", "--final-only"
)
assert final["provisional"] is False and final["log_verified"] is True
assert final["snapshot_verified"] is True
assert "finished" in final["excerpt"]
@pytest.mark.parametrize("quiet", [False, True])
def test_cli_keeps_json_stdout_clean_and_progress_on_stderr_or_quiet(
tmp_path, capsys, quiet
):
event = {
"event": "check_progress",
"run_id": "fixture",
"phase": "preparing",
"status": "running",
"counts": {"pending": 1},
"total_stages": 1,
"elapsed_seconds": 0,
"active_stages": [],
"receipt_path": str(tmp_path / "receipt.json"),
}
def check(args, _stages):
args.on_progress(event)
return {"status": "passed", "summary": ["fixture complete"]}
with (
patch.object(catalog, "build_stages", return_value=[]),
patch.object(runner, "run_checks", side_effect=check),
):
assert (
cli.main(
[
"check",
"--workspace-root",
str(tmp_path),
"--json",
*(["--quiet"] if quiet else []),
]
)
== 0
)
output = capsys.readouterr()
assert json.loads(output.out)["status"] == "passed"
if quiet:
assert output.err == ""
else:
assert json.loads(output.err)["event"] == "check_progress"
def snapshot(data, *, split=0, omitted=0, final=False):
return OutputSnapshot(data, b"", bool(omitted), omitted, 0, split, 0, final)
def test_provisional_output_withholds_incomplete_secret_line(monkeypatch):
monkeypatch.setenv("FIXTURE_SECRET", "credential-known-to-redaction")
data = b"public line\nAPI_KEY=credential-known-to-"
live = monitoring.capture_text(snapshot(data), provisional=True)
assert live == "public line\n"
final = monitoring.capture_text(
snapshot(b"public line\nAPI_KEY=credential-known-to-redaction", final=True)
)
assert "credential-known" not in final
assert "[redacted]" in final
def test_truncated_boundaries_cannot_expose_cut_authorization_or_secret_fragments():
head = b"safe head\nAuthorization: Bearer partial-secret-head"
tail = b"partial-secret-tail\nsafe final line\n"
for provisional in (False, True):
text = monitoring.capture_text(
snapshot(head + tail, split=len(head), omitted=90), provisional=provisional
)
assert "safe head" in text and "safe final line" in text
assert "partial-secret" not in text
assert "90 bytes omitted" in text
def test_final_failure_tail_survives_retention_and_stays_redacted(example, monkeypatch):
args, repo = example
monkeypatch.setenv("FIXTURE_SECRET", "sensitive-final-credential")
with patch.object(runner, "MAX_LOG_BYTES", 512):
result = run(
args,
[
stage(
repo,
code="import os; print('begin'); print('x'*20000); print(os.environ['FIXTURE_SECRET']); print('FINAL IMPORTANT ERROR'); raise SystemExit(9)",
)
],
)
final = command(
args, "logs", result["run_id"], "--stage", "one", "--final-only"
)
assert result["status"] == "failed"
assert final["log_verified"] is True
assert "FINAL IMPORTANT ERROR" in final["excerpt"]
assert "sensitive-final-credential" not in final["excerpt"]
assert "[redacted]" in final["excerpt"]
assert result["stages"][0]["omitted_output_bytes"] > 0
@pytest.mark.parametrize("maximum", [1, 2, 3, 7, 75, 76, 77, 80, 81, 82, 255])
@pytest.mark.parametrize("tail_only", [False, True])
def test_tiny_display_bounds_preserve_valid_utf8_and_byte_limit(maximum, tail_only):
value = "Ä😊 " * 200 + "FINAL"
text = monitoring.bounded_display(value, maximum, tail_only=tail_only)
assert len(text.encode("utf-8", errors="strict")) <= maximum
assert text.endswith("L")
if maximum >= 5 and (tail_only or maximum >= 255):
assert text.endswith("FINAL")
def test_live_record_is_bounded_provisional_and_rejects_mismatched_identity(example):
args, _ = example
run_id, stage_id = "fixture-run", "fixture-stage"
path = (
state_root(args.workspace_root, args.state_dir)
/ "runs"
/ run_id
/ (stage_id + ".log")
)
monitoring.write_live(path, stage_id, snapshot(b"line\n"), time.monotonic())
live = monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
assert live["provisional"] is True and live["excerpt"] == "line\n"
record = read_json(path.with_suffix(".live.json"))
record["run_id"] = "different-run"
atomic_json(path.with_suffix(".live.json"), record)
with pytest.raises(ValueError, match="Invalid provisional log"):
monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
+249
View File
@@ -0,0 +1,249 @@
"""Bounded fixture processes only; no project servers or external transports."""
from pathlib import Path
from dataclasses import FrozenInstanceError
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.process import _OutputBuffer, require_capture, run_captured
def python(code, **kwargs):
return run_captured([sys.executable, "-c", code], **kwargs)
def test_large_stdin_and_separate_outputs_are_multiplexed():
result = python(
"import sys; sys.stderr.write('diagnostic'); print(len(sys.stdin.buffer.read()))",
input_bytes=b"x" * 400000,
)
assert result.status == "passed"
assert result.stdout == b"400000\n"
assert result.stderr == b"diagnostic"
def test_live_output_limit_terminates_unbounded_producer():
result = python(
"import os;\nwhile True: os.write(1, b'x'*65536)", max_stdout=2048, timeout=3
)
assert result.status == "output_limit"
assert result.truncated
assert len(result.stdout) == 2048
def test_drain_mode_caps_memory_without_turning_success_into_failure():
result = python("print('x'*100000)", max_stdout=1024, terminate_on_limit=False)
assert result.status == "passed"
assert result.truncated and len(result.stdout) == 1024
def test_deadline_applies_after_command_closes_both_output_streams():
started = time.monotonic()
result = python(
"import os,time; os.close(1); os.close(2); time.sleep(30)", timeout=0.15
)
assert result.status == "timed_out"
assert time.monotonic() - started < 5
def test_cancellation_stops_owned_process():
cancelled = threading.Event()
timer = threading.Timer(0.1, cancelled.set)
timer.start()
try:
result = python("import time; time.sleep(30)", cancelled=cancelled)
finally:
timer.cancel()
timer.join()
assert result.status == "interrupted"
@pytest.mark.parametrize("redirected", [False, True])
def test_outliving_child_is_not_success_and_cannot_keep_running(redirected):
code = (
"import subprocess,sys; child=subprocess.Popen([sys.executable,'-c','import time; time.sleep(30)']"
+ (",stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL" if redirected else "")
+ "); print(child.pid,flush=True)"
)
result = python(code, timeout=4)
assert result.status == "leaked_process"
child_pid = int(result.stdout.strip())
# A terminated adopted child may remain a zombie until the host's init reaps it.
proc = Path(f"/proc/{child_pid}/stat")
if proc.exists():
assert proc.read_text().split(")", 1)[1].strip().split()[0] == "Z"
def test_required_capture_rejects_nonordinary_completion():
with pytest.raises(ValueError, match="output_limit"):
require_capture([sys.executable, "-c", "print('x'*10000)"], max_stdout=32)
def test_ordinary_nonzero_exit_retains_diagnostics():
result = python("import sys; print('problem',file=sys.stderr); sys.exit(7)")
assert (result.status, result.returncode, result.stderr) == (
"failed",
7,
b"problem\n",
)
def test_selector_setup_failure_still_terminates_spawned_process():
from govoplan_devkit import process
original = process.subprocess.Popen
spawned = []
def capture(*args, **kwargs):
child = original(*args, **kwargs)
spawned.append(child)
return child
with (
patch.object(process.subprocess, "Popen", side_effect=capture),
patch.object(
process.selectors,
"DefaultSelector",
side_effect=OSError("fixture selector unavailable"),
),
):
with pytest.raises(OSError, match="selector unavailable"):
python("import time; time.sleep(30)")
assert len(spawned) == 1 and spawned[0].poll() is not None
def test_head_tail_keeps_the_actual_failure_tail_within_original_bound():
output = b"BEGIN" + b"middle" * 100 + b"FINAL ERROR"
result = python(
f"import os; os.write(1, {output!r}); raise SystemExit(7)",
max_stdout=32,
capture_mode="head_tail",
terminate_on_limit=False,
)
assert (result.status, result.returncode) == ("failed", 7)
assert result.stdout == output[:16] + output[-16:]
assert result.stdout_head_bytes == 16
assert result.omitted_stdout_bytes == len(output) - 32
assert result.snapshot().final
assert "FINAL ERROR" in result.snapshot().text()
assert str(len(output) - 32) + " output bytes omitted" in result.snapshot().text()
@pytest.mark.parametrize("limit", [0, 1, 2, 3, 31, 1024])
def test_rolling_buffers_remain_bounded_for_every_chunk(limit):
buffer = _OutputBuffer(limit, "head_tail")
original = b""
for data in (b"a", b"bcdef", b"x" * 65536, b"last error"):
original += data
buffer.append(data)
assert len(buffer.head) + len(buffer.tail) <= limit
assert buffer.omitted == max(0, len(original) - limit)
if len(original) > limit:
head = limit // 2
tail = limit - head
expected = original[:head] + (original[-tail:] if tail else b"")
assert buffer.value() == expected
def test_prefix_capture_remains_exact_and_counts_omitted_bytes():
result = python(
"import os; os.write(1,b'0123456789')", max_stdout=4, terminate_on_limit=False
)
assert result.stdout == b"0123"
assert result.stdout_head_bytes == 4
assert result.omitted_stdout_bytes == 6
def test_callback_exposes_bounded_immutable_initial_and_final_snapshots():
snapshots = []
result = python(
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
"os.write(1,b'x'*10000+b'FINAL\\n'); time.sleep(.1)",
max_stdout=32,
capture_mode="head_tail",
terminate_on_limit=False,
on_output=snapshots.append,
)
assert not snapshots[0].final
assert snapshots[-1] == result.snapshot()
assert snapshots[0].stdout == b"first\n"
assert snapshots[-1].stdout.endswith(b"FINAL\n")
assert all(len(item.stdout) <= 32 for item in snapshots)
with pytest.raises(FrozenInstanceError):
snapshots[0].final = True
def test_callback_updates_dirty_output_while_child_becomes_quiet():
snapshots = []
python(
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
"os.write(1,b'second\\n'); time.sleep(1.15)",
on_output=lambda item: snapshots.append((time.monotonic(), item)),
)
assert len(snapshots) >= 3
assert snapshots[0][1].stdout == b"first\n"
assert not snapshots[1][1].final
assert snapshots[1][1].stdout.endswith(b"second\n")
assert snapshots[1][0] - snapshots[0][0] >= 0.95
assert snapshots[-1][1].final
def test_quiet_process_has_one_final_empty_snapshot():
snapshots = []
python("pass", on_output=snapshots.append)
assert len(snapshots) == 1
assert snapshots[0].final and snapshots[0].stdout == b""
def test_snapshot_text_handles_cut_and_invalid_utf8_without_malformed_strings():
result = python(
"import os; os.write(1, 'Ä😊Z'.encode()*20+b'\\xff\\xfe')",
max_stdout=9,
capture_mode="head_tail",
terminate_on_limit=False,
)
text = result.snapshot().text()
text.encode("utf-8", errors="strict")
assert "output bytes omitted" in text
assert len(result.stdout) == 9
with pytest.raises(ValueError, match="stream"):
result.snapshot().text("other")
def test_callback_failure_terminates_owned_process_and_propagates():
from govoplan_devkit import process
original = process.subprocess.Popen
spawned = []
def capture(*args, **kwargs):
child = original(*args, **kwargs)
spawned.append(child)
return child
def fail(_snapshot):
raise ValueError("fixture callback failed")
with patch.object(process.subprocess, "Popen", side_effect=capture):
with pytest.raises(ValueError, match="fixture callback failed"):
python(
"import time; print('ready',flush=True); time.sleep(30)", on_output=fail
)
assert len(spawned) == 1 and spawned[0].poll() is not None
@pytest.mark.parametrize(
"kwargs", [{"capture_mode": "other"}, {"max_stdout": -1}, {"max_stderr": True}]
)
def test_invalid_capture_configuration_fails_before_starting_a_process(kwargs):
from govoplan_devkit import process
with patch.object(process.subprocess, "Popen") as popen:
with pytest.raises(ValueError):
python("pass", **kwargs)
popen.assert_not_called()
+524
View File
@@ -0,0 +1,524 @@
"""Fixture-only devkit coverage of the real durable release API/store boundary."""
from __future__ import annotations
import argparse
from copy import deepcopy
import json
from pathlib import Path
import socket
import subprocess
import sys
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
ROOT = Path(__file__).resolve().parents[1]
for path in (ROOT / "tools/devkit", ROOT / "tools/release"):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
from govoplan_devkit import release # noqa: E402
from govoplan_release.release_execution import ReleaseExecutionAmbiguous, ReleaseExecutionBlocked # noqa: E402
from govoplan_release.release_run import ReleaseRunCorrupt, ReleaseRunStore # noqa: E402
from govoplan_release.candidate_artifact import ( # noqa: E402
candidate_output_path, harden_private_candidate_tree, issue_candidate_receipt,
)
from server import app as api # noqa: E402
def receipt(repo="govoplan-core", *, target_tag="v1.2.3"):
return {
"kind": "repository_state", "repo": repo, "head": "a" * 40,
"branch": "main", "remote": "origin", "remote_sha256": "b" * 64,
"worktree_clean": True, "target_tag": target_tag, "tag_object": None,
}
def plan(*, catalog=False):
steps = [
("core:preflight", "govoplan-core", False),
("core:tag", "govoplan-core", True),
] if not catalog else [
("catalog:selective-generator", None, True),
("catalog:validate-sign-publish", None, True),
]
return {
"generated_at": "2026-09-08T00:00:00Z", "target_channel": "stable",
"status": "attention", "units": [{"repo": "govoplan-core", "target_version": "1.2.3"}],
"compatibility": [], "gate_findings": [], "recommended_action": {},
"source_preflight_ready": True, "notes": [],
"dry_run_steps": [
{"id": identity, "repo": repo, "mutating": mutating, "title": identity,
"detail": "Fixture-only operation.", "command": "fixture never executed",
"cwd": "/fixture", "status": "planned"}
for identity, repo, mutating in steps
],
}
def namespace(workspace, state, *arguments):
parser = argparse.ArgumentParser()
parser.set_defaults(workspace_root=workspace, state_dir=state, format="json", project=None)
release.register(parser.add_subparsers(required=True))
return parser.parse_args(["release", *arguments])
@pytest.fixture
def environment(tmp_path, monkeypatch):
workspace = tmp_path / "workspace"
workspace.mkdir()
state = tmp_path / "private-state"
plans = [plan()]
dashboard = Mock(return_value={"summary": {"status": "ready"}, "repositories": []})
planner = Mock(side_effect=lambda *args, **kwargs: deepcopy(plans[0]))
execute = Mock(return_value=({"status": "inspected"}, receipt()))
def bind(**kwargs):
result = kwargs["plan"]
for step in result["dry_run_steps"]:
if step.get("repo"):
step["source_binding"] = receipt(step["repo"])
elif step["id"] == "catalog:validate-sign-publish":
step["source_binding"] = receipt("addideas-govoplan-website", target_tag="")
return result
monkeypatch.setattr(api, "build_dashboard", dashboard)
monkeypatch.setattr(api, "build_selective_release_plan", planner)
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock())
monkeypatch.setattr(api, "verify_release_runtime_binding", Mock())
monkeypatch.setattr(api, "bind_plan_source_states", bind)
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(return_value=receipt()))
monkeypatch.setattr(api, "verify_repository_step_precondition", Mock(return_value=receipt()))
monkeypatch.setattr(api, "execute_repository_step", execute)
monkeypatch.setattr(api, "default_signing_keys", lambda: ())
def forbidden(*args, **kwargs):
raise AssertionError("Fixture test attempted a real subprocess or network connection")
monkeypatch.setattr(subprocess, "run", forbidden)
monkeypatch.setattr(socket, "create_connection", forbidden)
def call(*args, state_dir=state, workspace_root=workspace):
values = namespace(workspace_root, state_dir, *args)
return values.handler(values)
def create(request="devkit-create-request-0001"):
result = call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", request, "--apply")
assert result["_exit_code"] == 0, result
return result
return SimpleNamespace(
workspace=workspace, state=state, plans=plans, dashboard=dashboard,
planner=planner, executor=execute, call=call, create=create,
)
def test_registration_does_not_import_heavy_dependencies():
script = """
import argparse, builtins, sys
sys.path.insert(0, sys.argv[1])
original = builtins.__import__
def guarded(name, *args, **kwargs):
if name.split('.')[0] in {'httpx', 'fastapi', 'govoplan_core', 'govoplan_release'}:
raise AssertionError('Heavy import during help registration: ' + name)
return original(name, *args, **kwargs)
builtins.__import__ = guarded
from govoplan_devkit.release import register
parser = argparse.ArgumentParser()
register(parser.add_subparsers())
assert 'release' in parser.format_help()
"""
result = subprocess.run([sys.executable, "-c", script, str(ROOT / "tools/devkit")], capture_output=True, text=True, check=False)
assert result.returncode == 0, result.stderr
def test_plan_is_selective_offline_and_does_not_create_run(environment):
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
assert result["_exit_code"] == 0
assert result["dry_run"] is True
assert environment.planner.call_args.kwargs["selected_repos"] == ("govoplan-core",)
arguments = environment.dashboard.call_args.kwargs
assert arguments["online"] is False
assert arguments["check_remote_tags"] is False
assert arguments["check_public_catalog"] is False
assert arguments["include_migrations"] is False
assert not list(environment.state.rglob("rr-*.json"))
assert "release-console/workspace-" in result["state_location"]
environment.executor.assert_not_called()
def test_status_reports_blocked_exit_and_explicit_check_flags(environment):
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
result = environment.call("status", "--online", "--include-migrations", "--include-website")
assert result["_exit_code"] == 1
assert environment.dashboard.call_args.kwargs["check_remote_tags"] is True
assert environment.dashboard.call_args.kwargs["check_public_catalog"] is True
assert environment.dashboard.call_args.kwargs["include_migrations"] is True
def test_plan_summary_exposes_attention_readiness_gates_and_next_action(environment):
fixture_plan = environment.plans[0]
fixture_plan["source_preflight_ready"] = False
fixture_plan["units"][0]["status"] = "attention"
fixture_plan["gate_findings"] = [{"code": "worktree_dirty", "repo": "govoplan-core", "message": "Uncommitted source changes need review."}]
fixture_plan["recommended_action"] = {"id": "prepare_changes", "title": "Prepare source changes", "remediation": "Review and commit the selected changes before preparing the release."}
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
summary = "\n".join(result["summary"])
assert "Release plan: attention." in summary
assert "Source preflight ready: false." in summary
assert "govoplan-core: attention; target 1.2.3." in summary
assert "Gate worktree_dirty (govoplan-core)" in summary
assert "Next: prepare_changes" in summary
assert "no run was created" in summary
assert "fixture never executed" not in summary
def test_summary_is_bounded_and_redacts_display_only_fields(monkeypatch):
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "fixture-value-not-for-display")
fixture_plan = plan()
fixture_plan["gate_findings"] = [{"code": "fixture", "message": "fixture-value-not-for-display " + "x" * 1000}] * 20
fixture_plan["units"] = [{"repo": f"repo-{index}", "target_version": "1.2.3", "status": "ready"} for index in range(50)]
summary = "\n".join(release._summary_lines("plan", fixture_plan))
assert "38 more selected repositories" in summary
assert "16 more gate findings" in summary
assert "fixture-value-not-for-display" not in summary
assert "[redacted]" in summary
assert len(summary) < 3000
def test_run_and_preview_summary_exposes_steps_and_next_action(environment):
run = environment.create()["result"]
shown = environment.call("show", run["run_id"])
summary = "\n".join(shown["summary"])
assert "Steps: 2 pending." in summary
assert "Step core:preflight: pending." in summary
assert "Next: execute_step [core:preflight]" in summary
preview = environment.call("preview", run["run_id"], "core:tag")
assert "Release preview: pending." in preview["summary"]
assert any("Complete core:preflight first." in line for line in preview["summary"])
def test_status_summary_exposes_repository_counts(environment):
environment.dashboard.return_value = {"summary": {"status": "attention", "repository_count": 77, "dirty_count": 42, "ahead_count": 1, "behind_count": 0, "error_count": 0}}
result = environment.call("status")
assert "Release status: attention." in result["summary"]
assert any("77 repositories, 42 dirty, 1 ahead, 0 behind, 0 errors" in line for line in result["summary"])
def test_display_redaction_cannot_change_semantic_failure_exit(environment, monkeypatch):
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "blocked")
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
result = environment.call("status")
assert result["_exit_code"] == 1
assert "Release status: [redacted]." in result["summary"]
def test_create_defaults_to_preview_without_run_record(environment):
result = environment.call("create", "--repo", "govoplan-core", "--target-version", "1.2.3", "--request-id", "preview-create-request-0001")
assert result["_exit_code"] == 0 and result["dry_run"]
assert not list(environment.state.rglob("rr-*.json"))
environment.executor.assert_not_called()
def test_create_show_and_same_id_replay_use_durable_service(environment):
created = environment.create()
run_id = created["result"]["run_id"]
repeated = environment.create()
assert repeated["result"]["run_id"] == run_id
environment.planner.assert_called_once()
shown = environment.call("show", run_id)
assert shown["result"]["immutable"] == created["result"]["immutable"]
assert shown["result"]["state"]["steps"][0]["executor"]["confirmation"] == ""
assert len(list(environment.state.rglob("rr-*.json"))) == 1
def test_create_replay_rejects_changed_inputs(environment):
environment.create()
result = environment.call("create", "--repo-version", "govoplan-core=1.2.4", "--request-id", "devkit-create-request-0001", "--apply")
assert result["http_status"] == 409
environment.planner.assert_called_once()
@pytest.mark.parametrize("arguments", [
("plan",),
("create", "--repo", "govoplan-core", "--request-id", "missing-version-request"),
("plan", "--repo-version", "govoplan-core=1.2.3", "--repo-version", "govoplan-core=1.2.4"),
("plan", "--repo-version", "govoplan-core=not-a-version"),
])
def test_invalid_selection_is_rejected_before_service_collection(environment, arguments):
result = environment.call(*arguments)
assert result["_exit_code"] == 2
environment.dashboard.assert_not_called()
environment.executor.assert_not_called()
def test_runtime_trust_guard_cannot_be_bypassed_by_cli(environment, monkeypatch):
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock(side_effect=ReleaseExecutionBlocked("Fixture untrusted runtime")))
result = environment.call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", "trust-create-request-0001", "--apply")
assert result["http_status"] == 409
assert not list(environment.state.rglob("rr-*.json"))
environment.planner.assert_not_called()
def test_dry_execute_and_generic_preview_never_claim_or_execute(environment):
run = environment.create()["result"]
dry = environment.call("execute", run["run_id"], "core:preflight", "--request-id", "dry-execute-request-0001")
preview = environment.call("preview", run["run_id"], "core:tag")
assert dry["dry_run"] and preview["dry_run"]
assert preview["result"]["state_step"]["executor"]["confirmation"] == "TAG"
assert preview["result"]["state_step"]["available"] is False
environment.executor.assert_not_called()
assert environment.call("show", run["run_id"])["result"]["state"]["steps"][0]["attempt_count"] == 0
def test_prerequisite_and_confirmation_guards_remain_enforced(environment):
run_id = environment.create()["result"]["run_id"]
blocked = environment.call("execute", run_id, "core:tag", "--request-id", "ordered-tag-request-0001", "--confirm", "TAG", "--apply")
assert blocked["http_status"] == 409
environment.executor.assert_not_called()
assert environment.call("execute", run_id, "core:preflight", "--request-id", "preflight-request-0001", "--apply")["_exit_code"] == 0
missing = environment.call("execute", run_id, "core:tag", "--request-id", "confirmed-tag-request-0001", "--apply")
assert missing["http_status"] == 409
assert environment.executor.call_count == 1
def test_execute_same_attempt_replays_without_repeating_effect(environment):
run_id = environment.create()["result"]["run_id"]
arguments = ("execute", run_id, "core:preflight", "--request-id", "exact-attempt-request-0001", "--apply")
first = environment.call(*arguments)
second = environment.call(*arguments)
assert first["_exit_code"] == second["_exit_code"] == 0
assert second["result"]["execution_result"]["status"] == "replayed"
environment.executor.assert_called_once()
assert environment.executor.call_args.kwargs["remote"] == "origin"
def test_lost_finish_write_is_interrupted_without_reexecuting_effect(environment, monkeypatch):
run_id = environment.create()["result"]["run_id"]
finish = ReleaseRunStore.finish_step
monkeypatch.setattr(ReleaseRunStore, "finish_step", Mock(side_effect=ReleaseRunCorrupt("Fixture durable write failure")))
arguments = ("execute", run_id, "core:preflight", "--request-id", "lost-finish-request-0001", "--apply")
interrupted = environment.call(*arguments)
assert interrupted["http_status"] == 409
monkeypatch.setattr(ReleaseRunStore, "finish_step", finish)
assert environment.call(*arguments)["http_status"] == 409
environment.executor.assert_called_once()
shown = environment.call("show", run_id)["result"]
assert shown["state"]["steps"][0]["state"] == "interrupted"
def test_source_guard_failure_does_not_call_executor(environment, monkeypatch):
run_id = environment.create()["result"]["run_id"]
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(side_effect=ReleaseExecutionBlocked("Frozen HEAD/remote changed")))
result = environment.call("execute", run_id, "core:preflight", "--request-id", "changed-source-request-0001", "--apply")
assert result["_exit_code"] == 1
environment.executor.assert_not_called()
def test_interrupted_write_requires_reconciliation_not_retry(environment, monkeypatch):
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture remote outcome uncertain")
run_id = environment.create()["result"]["run_id"]
execute = ("execute", run_id, "core:tag", "--request-id", "uncertain-tag-request-0001", "--confirm", "TAG", "--apply")
uncertain = environment.call(*execute)
assert uncertain["http_status"] == 409
assert environment.call(*execute)["http_status"] == 409
retry = environment.call("retry", run_id, "core:tag", "--request-id", "unsafe-retry-request-0001", "--apply")
assert retry["http_status"] == 409
environment.executor.assert_called_once()
invalid = environment.call("reconcile", run_id, "core:tag", "--request-id", "bad-reconcile-request-0001", "--outcome", "effect_absent", "--apply")
assert invalid["http_status"] == 409
reconciled = environment.call("reconcile", run_id, "core:tag", "--request-id", "reconcile-absent-request-0001", "--outcome", "effect_absent", "--confirm", "RECONCILE", "--apply")
assert reconciled["_exit_code"] == 0
assert reconciled["result"]["state"]["steps"][0]["state"] == "pending"
def test_effect_succeeded_reconciliation_keeps_independent_receipt_guard(environment, monkeypatch):
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture interruption")
run_id = environment.create()["result"]["run_id"]
environment.call("execute", run_id, "core:tag", "--request-id", "receipt-tag-request-0001", "--confirm", "TAG", "--apply")
guard = Mock(side_effect=ReleaseExecutionBlocked("Remote annotation mismatch"))
monkeypatch.setattr(api, "reconciled_repository_receipt", guard)
result = environment.call("reconcile", run_id, "core:tag", "--request-id", "receipt-success-request-0001", "--outcome", "effect_succeeded", "--confirm", "RECONCILE", "--apply")
assert result["http_status"] == 409
guard.assert_called_once()
environment.executor.assert_called_once()
def test_resume_and_retry_reuse_the_existing_running_attempt_rules(environment):
created = environment.create()
run_id = created["result"]["run_id"]
store = ReleaseRunStore(Path(created["state_location"]), expected_workspace_fingerprint=api.release_workspace_fingerprint(environment.workspace))
store.claim_step(run_id, "core:preflight", attempt_id="lost-process-request-0001")
dry = environment.call("resume", run_id, "--request-id", "resume-process-request-0001")
assert dry["dry_run"]
assert store.get(run_id)["state"]["steps"][0]["state"] == "running"
resumed = environment.call("resume", run_id, "--request-id", "resume-process-request-0001", "--apply")
assert resumed["result"]["state"]["steps"][0]["state"] == "interrupted"
retried = environment.call("retry", run_id, "core:preflight", "--request-id", "retry-readonly-request-0001", "--apply")
assert retried["result"]["state"]["steps"][0]["state"] == "pending"
environment.executor.assert_not_called()
def test_workspace_scoping_and_corrupt_records_fail_closed(environment, tmp_path):
created = environment.create()
run_id = created["result"]["run_id"]
another = tmp_path / "another-workspace"
another.mkdir()
foreign = environment.call("show", run_id, workspace_root=another)
assert foreign["http_status"] == 404
path = Path(created["state_location"]) / f"{run_id}.json"
record = json.loads(path.read_text())
record["immutable"]["input"]["repo_versions"]["govoplan-core"] = "9.9.9"
path.write_text(json.dumps(record))
assert environment.call("show", run_id)["http_status"] == 409
def test_run_list_keeps_cursor_pagination(environment):
environment.create("page-create-request-0001")
environment.create("page-create-request-0002")
first = environment.call("list", "--limit", "1")["result"]
second = environment.call("list", "--limit", "1", "--cursor", first["next_cursor"])["result"]
assert len(first["runs"]) == len(second["runs"]) == 1
assert first["runs"][0]["run_id"] != second["runs"][0]["run_id"]
assert second["next_cursor"] is None
def test_catalog_preview_calls_only_existing_receipt_bound_preview(environment, monkeypatch, tmp_path):
environment.plans[0] = plan(catalog=True)
run_id = environment.create()["result"]["run_id"]
candidate = tmp_path / "candidate"
verify = Mock(return_value=candidate)
publish = Mock(return_value={"status": "planned", "apply": False})
monkeypatch.setattr(api, "verified_run_candidate", verify)
monkeypatch.setattr(api, "publish_catalog_candidate", publish)
result = environment.call("preview", run_id, "catalog:validate-sign-publish")
assert result["_exit_code"] == 0
verify.assert_called_once()
assert publish.call_args.kwargs["apply"] is False
assert publish.call_args.kwargs["candidate_dir"] == candidate
assert publish.call_args.kwargs["remote"] == "origin"
environment.executor.assert_not_called()
def test_catalog_generation_and_publication_use_exact_durable_receipts(environment, monkeypatch):
environment.plans[0] = plan(catalog=True)
created = environment.create()
run_id = created["result"]["run_id"]
candidate_root = Path(created["candidate_location"])
def generated(**kwargs):
candidate_id = kwargs["candidate_id"]
candidate = candidate_output_path(candidate_root, candidate_id)
channels = candidate / "channels"
channels.mkdir(parents=True)
(channels / "stable.json").write_text(json.dumps({"channel": "stable", "signatures": [{}]}))
harden_private_candidate_tree(candidate)
return {"status": "ready"}, issue_candidate_receipt(root=candidate_root, candidate_id=candidate_id, channel="stable")
def published(**kwargs):
candidate = kwargs["candidate_receipt"]
website = kwargs["expected_website_receipt"]
return {"status": "published"}, {
"kind": "catalog_publication", "candidate_id": candidate["candidate_id"],
"catalog_sha256": candidate["catalog_sha256"], "keyring_sha256": "c" * 64,
"publication_commit_sha": "d" * 40, "publication_tag_object_sha": "e" * 40,
"publication_tag_commit_sha": "d" * 40, "branch": website["branch"],
"tag_name": "catalog-stable-1", "remote": "origin", "remote_sha256": website["remote_sha256"],
}
generator = Mock(side_effect=generated)
publisher = Mock(side_effect=published)
website = receipt("addideas-govoplan-website", target_tag="")
monkeypatch.setattr(api, "generate_catalog_candidate", generator)
monkeypatch.setattr(api, "publish_received_candidate", publisher)
monkeypatch.setattr(api, "verify_catalog_publication_precondition", Mock(return_value=website))
generate = ("execute", run_id, "catalog:selective-generator", "--request-id", "generate-candidate-request-0001", "--confirm", "GENERATE", "--apply")
first = environment.call(*generate, "--signing-key", "fixture-key=/private/fixture-key.pem")
assert first["_exit_code"] == 0, first
replayed = environment.call(*generate)
assert replayed["result"]["execution_result"]["status"] == "replayed"
generator.assert_called_once()
assert generator.call_args.kwargs["signing_keys"] == ("fixture-key=/private/fixture-key.pem",)
assert "/private/fixture-key.pem" not in json.dumps(first)
record_text = next(environment.state.rglob("rr-*.json")).read_text()
assert "/private/fixture-key.pem" not in record_text
missing_confirmation = environment.call("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--apply")
assert missing_confirmation["http_status"] == 409
publisher.assert_not_called()
publish = ("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--confirm", "PUSH", "--apply")
complete = environment.call(*publish)
assert complete["_exit_code"] == 0, complete
assert complete["result"]["state"]["status"] == "completed"
frozen = first["result"]["state"]["steps"][0]["result_receipt"]
assert publisher.call_args.kwargs["candidate_path"] == candidate_root / frozen["candidate_id"]
assert publisher.call_args.kwargs["candidate_receipt"] == frozen
assert publisher.call_args.kwargs["expected_website_receipt"] == website
assert publisher.call_args.kwargs["remote"] == "origin"
assert environment.call(*publish)["result"]["execution_result"]["status"] == "replayed"
publisher.assert_called_once()
def test_portable_project_cannot_override_release_catalog(environment):
args = namespace(environment.workspace, environment.state, "plan", "--repo-version", "govoplan-core=1.2.3")
args.project = environment.workspace / "custom-project.json"
result = release.handle(args)
assert result["_exit_code"] == 2
assert "does not accept --project" in result["summary"][0]
environment.dashboard.assert_not_called()
def test_foreign_cached_release_package_is_rejected_before_import(environment, monkeypatch, tmp_path):
monkeypatch.setitem(sys.modules, "server.app", SimpleNamespace(__file__=str(tmp_path / "foreign/app.py")))
importer = Mock(side_effect=AssertionError("Foreign module import must not occur"))
monkeypatch.setattr(release.importlib, "import_module", importer)
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
assert result["_exit_code"] == 2
assert "foreign module" in result["summary"][0]
importer.assert_not_called()
environment.dashboard.assert_not_called()
def test_no_arbitrary_remote_or_legacy_publication_flags():
run_id = "rr-request-" + "a" * 64
with pytest.raises(SystemExit):
namespace(Path("/fixture"), None, "execute", run_id, "core:tag", "--request-id", "arbitrary-remote-request", "--remote", "untrusted")
with pytest.raises(SystemExit):
namespace(Path("/fixture"), None, "publish-candidate", "--candidate-dir", "/untrusted")
def test_unknown_step_and_malformed_paths_do_not_become_other_routes(environment):
run_id = environment.create()["result"]["run_id"]
assert environment.call("preview", run_id, "unknown:step")["http_status"] == 404
with pytest.raises(SystemExit):
namespace(environment.workspace, environment.state, "preview", run_id, "../repositories/push")
def test_signing_material_is_not_echoed_by_validation(environment):
run_id = environment.create()["result"]["run_id"]
secret = "PRIVATE-KEY-MATERIAL\nDO-NOT-ECHO"
result = environment.call("execute", run_id, "core:preflight", "--request-id", "secret-input-request-0001", "--signing-key", secret, "--apply")
assert result["_exit_code"] == 2
assert secret not in json.dumps(result)
assert "SECRET" not in release._error_detail({"detail": [{"loc": ["body", "signing_keys"], "msg": "Invalid input", "input": "SECRET"}]})
environment.executor.assert_not_called()
def test_default_state_matches_console_and_token_never_appears_in_output(environment, monkeypatch, tmp_path):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
monkeypatch.setattr(release.secrets, "token_urlsafe", lambda size: "ephemeral-do-not-print-token")
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3", state_dir=None)
assert result["state_location"] == str(api.default_release_run_root(environment.workspace))
assert "ephemeral-do-not-print-token" not in json.dumps(result)
def test_documentation_keeps_disabled_generic_mutations_and_recovery_explicit():
console = (ROOT / "docs/operations/RELEASE_CONSOLE.md").read_text()
usage = (ROOT / "docs/operations/DEVKIT_RELEASE.md").read_text()
assert "generic push, sync and prepare\nmutation endpoints are disabled" in console
assert "ASGI application inside" in usage
assert "effect_absent" in usage and "effect_succeeded" in usage and "unresolved" in usage
assert "does **not** stage arbitrary source" in usage
+175
View File
@@ -0,0 +1,175 @@
import argparse
import json
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import review
from govoplan_devkit.cli import main
@pytest.fixture
def args(tmp_path):
repo = tmp_path / "optional-feature"
repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
paths = ["webui/src/pages/CampaignPage.tsx", "webui/src/dialogs/SettingsDialog.tsx",
"webui/src/settings/AdminSettings.tsx", "webui/src/widgets/SummaryWidget.tsx", "webui/src/module.ts"]
for relative in paths:
path = repo / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("// fixture only\n")
manifest = repo / "src/optional_feature/backend/manifest.py"
manifest.parent.mkdir(parents=True)
manifest.write_text("raise RuntimeError('MUST NEVER IMPORT MODULE CODE')\n")
(tmp_path / "principles.md").write_text("# Principles\nRevision **UI-2026-09-08**\n\n## UI-01 — Help beside text\n\n## UI-02 — Display first; edit deliberately\n")
inventory = {"schema_version": 1, "epic": {"repository": "meta", "number": 4, "url": "https://gitea.invalid/team/meta/issues/4"},
"issues": [{"scope_id": "campaigns", "name": "Campaign", "kind": "manifest", "repository": "optional-feature", "number": 8,
"url": "https://gitea.invalid/team/optional-feature/issues/8", "state_at_verification": "closed", "operation": "created"}]}
(tmp_path / "issues.json").write_text(json.dumps(inventory))
project = tmp_path / "project.json"
project.write_text(json.dumps({"schema_version": 1, "name": "Portable project", "repositories": [
{"name": "optional-feature", "path": "optional-feature", "aliases": ["campaign"]},
{"name": "absent-optional", "path": "absent-optional", "aliases": ["absent"]}],
"review": {"issue_inventory": "issues.json", "principles": "principles.md"},
"checks": [{"id": "ui-check", "argv": ["never-execute-this"], "repos": ["optional-feature"], "cwd": "optional-feature"}],
"profiles": {"ui": ["ui-check"]}}))
return argparse.Namespace(workspace_root=tmp_path, project=project, state_dir=tmp_path / "state",
module="campaign", bundle_module=None, profile="ui", evidence=None, output=None)
def test_bundle_contains_guidance_links_revision_and_unexecuted_check_plan(args):
result = review.handle_review(args)
assert result["module"] == "optional-feature"
assert result["inventory"]["ui_source_count"] == 5
assert not result["inventory"]["module_code_imported"]
assert result["inventory"]["manifest_paths"] == ["src/optional_feature/backend/manifest.py"]
assert result["principles"]["revision"] == "UI-2026-09-08"
assert [rule["id"] for rule in result["principles"]["rules"]] == ["UI-01", "UI-02"]
assert result["check_plan"]["stages"][0]["argv"] == ["never-execute-this"]
assert not result["check_plan"]["executed"]
assert result["review_completion"].startswith("Not assessed")
assert len(result["manual_checklist"]) >= 9
assert all("checked" not in item for item in result["manual_checklist"])
assert "state_at_verification" not in json.dumps(result)
assert "operation" not in result["issue_links"][0]
assert result["central_issue"]["number"] == 4
assert not args.state_dir.exists()
@pytest.mark.parametrize("name", ["campaign", "campaigns", "optional-feature"])
def test_alias_scope_id_and_repository_resolve_to_same_registered_module(args, name):
args.module = name
assert review.handle_review(args)["module"] == "optional-feature"
def test_bundle_alias_and_direct_cli_work_with_global_options_anywhere(args, capsys):
assert main(["--workspace-root", str(args.workspace_root), "review", "campaign", "--project", str(args.project), "--json"]) == 0
direct = json.loads(capsys.readouterr().out)
assert direct["module"] == "optional-feature"
assert main(["review", "bundle", "campaigns", "--workspace-root", str(args.workspace_root), "--project", str(args.project), "--format", "json"]) == 0
assert json.loads(capsys.readouterr().out)["module"] == direct["module"]
def test_absent_optional_repository_and_no_ui_do_not_complete_review(args):
args.module = "absent"
result = review.handle_review(args)
assert not result["repository"]["exists"]
assert result["repository"]["errors"]
assert result["inventory"]["ui_source_count"] == 0
assert result["check_plan"]["stages"] == []
assert result["review_completion"].startswith("Not assessed")
assert any("not automatic N/A" in warning for warning in result["warnings"])
assert any("not a passing" in warning for warning in result["warnings"])
def test_portable_project_does_not_inherit_govoplan_links_or_principles(args):
payload = json.loads(args.project.read_text())
payload.pop("review")
args.project.write_text(json.dumps(payload))
result = review.handle_review(args)
assert result["issue_links"] == [] and result["central_issue"] is None
assert not result["principles"]["available"]
assert "git.add-ideas.de" not in json.dumps(result)
@pytest.mark.parametrize("kind", ["symlink", "outside", "bad-url", "duplicate-scope", "foreign-repository"])
def test_review_inputs_cannot_escape_or_silently_misbind_scope(args, kind):
if kind in {"symlink", "outside"}:
payload = json.loads(args.project.read_text())
if kind == "symlink":
alias = args.workspace_root / "alias.json"
alias.symlink_to(args.workspace_root / "issues.json")
payload["review"]["issue_inventory"] = "alias.json"
else:
payload["review"]["principles"] = "../outside.md"
args.project.write_text(json.dumps(payload))
else:
path = args.workspace_root / "issues.json"
payload = json.loads(path.read_text())
if kind == "bad-url":
payload["issues"][0]["url"] = "https://gitea.invalid/team/wrong/issues/8"
elif kind == "duplicate-scope":
payload["issues"].append(payload["issues"][0])
else:
payload["issues"][0].update(repository="foreign", url="https://gitea.invalid/team/foreign/issues/8")
args.module = "campaigns"
path.write_text(json.dumps(payload))
with pytest.raises(ValueError):
review.handle_review(args)
def test_source_inventory_never_follows_symlinked_ancestors(args):
root = args.workspace_root / "indirect"
root.mkdir()
(root / "webui").symlink_to(args.workspace_root / "optional-feature/webui", target_is_directory=True)
result = review.source_inventory(root)
assert result["ui_source_count"] == 0
assert result["skipped_symlinks"] == ["webui/src"]
def test_explicit_artifact_is_only_local_and_preserves_issue_discovery_snapshot(args):
inventory = args.workspace_root / "issues.json"
original = inventory.read_bytes()
args.output = args.workspace_root / "artifacts/review.json"
result = review.handle_review(args)
assert json.loads(args.output.read_text()) == result
assert inventory.read_bytes() == original
assert args.output.stat().st_mode & 0o777 == 0o600
assert not args.state_dir.exists()
def test_attached_historical_receipt_stays_separate_from_current_review(args):
path = args.workspace_root / "historical.json"
path.write_text(json.dumps({"schema_version": 1, "run_id": "historical", "workspace_root": str(args.workspace_root),
"project_file": str(args.project), "source_fingerprint": "a" * 64, "status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08",
"finished_at": "2026-09-08", "stages": [{"id": "old-check", "status": "passed", "exit_code": 0, "duration_seconds": 1, "log_path": "/not-read"}]}))
args.evidence = str(path)
result = review.handle_review(args)
assert result["evidence"]["origin"] == "external-unverified"
assert result["evidence"]["source_state"] == "historical-source-differs"
assert result["review_completion"].startswith("Not assessed")
assert not result["check_plan"]["executed"]
def test_unknown_module_and_extra_positionals_are_errors(args):
args.module = "typo"
with pytest.raises(ValueError, match="Unknown repository"):
review.handle_review(args)
args.module, args.bundle_module = "campaign", "extra"
with pytest.raises(ValueError, match="one module"):
review.handle_review(args)
def test_compact_review_exposes_scoped_coverage_limits_without_hiding_full_list(args, monkeypatch):
notes = [f"Separate suite {number} was not included." for number in range(12)]
monkeypatch.setattr("govoplan_devkit.catalog.build_stages", lambda *_args, **_kwargs: [
{"id": "source-only", "argv": ["never-execute"], "coverage_notes": notes}])
result = review.handle_review(args)
assert result["coverage_notes"] == notes
assert sum(line.startswith("Coverage limitation:") for line in result["summary"]) == 8
assert any("4 additional coverage limitations" in line for line in result["summary"])
assert not result["check_plan"]["executed"]
+466
View File
@@ -0,0 +1,466 @@
"""Real fixture processes and Git worktrees; never invoke product/remote mutations."""
from argparse import Namespace
import argparse
import json
from pathlib import Path
import subprocess
import sys
import threading
import time
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit import runner
from govoplan_devkit.checkpoints import Checkpoints
from govoplan_devkit.common import (
atomic_json,
digest,
read_json,
resource_lock,
state_root,
)
from govoplan_devkit.workspace import load_project, source_fingerprint
@pytest.fixture
def example(tmp_path, monkeypatch):
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
workspace = tmp_path / "workspace"
repo = workspace / "example"
repo.mkdir(parents=True)
subprocess.run(["git", "init", "-q", str(repo)], check=True)
(repo / "source.txt").write_text("first\n")
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
subprocess.run(
[
"git",
"-C",
str(repo),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"-qm",
"fixture",
],
check=True,
)
project = tmp_path / "project.json"
project.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Example",
"repositories": [{"name": "example", "path": "example"}],
"checks": [],
"profiles": {},
}
)
)
args = Namespace(
workspace_root=workspace,
project=project,
state_dir=tmp_path / "state",
dry_run=False,
jobs=2,
profile="quick",
resume=None,
)
return args, repo
def stage(repo, name="one", code="print('ok')", **extra):
return {
"id": name,
"title": name,
"argv": [sys.executable, "-c", code],
"cwd": str(repo),
"timeout_seconds": 10,
**extra,
}
def run(args, stages):
# Environment inspection is tested separately; keep fixture executions cheap/deterministic.
with patch.object(runner, "environment_fingerprint", return_value="fixture-env"):
return runner.run_checks(args, stages)
def test_success_receipt_and_compact_summary(example):
args, repo = example
result = run(args, [stage(repo)])
assert result["status"] == "passed"
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
assert receipt["stages"][0]["exit_code"] == 0
assert Path(receipt["stages"][0]["log_path"]).read_text() == "ok\n"
assert runner.summarize(receipt)["counts"] == {"passed": 1}
assert Path(result["receipt_path"]).stat().st_mode & 0o777 == 0o600
def test_native_environment_binds_selected_checkout(tmp_path, monkeypatch):
from govoplan_devkit.environment import execution_environment
monkeypatch.setenv("GOVOPLAN_WORKSPACE_ROOT", "/another/workspace")
monkeypatch.setenv("GOVOPLAN_CORE_ROOT", "/another/core")
monkeypatch.setenv("GOVOPLAN_CORE_SOURCE_ROOT", "/another/source")
project = load_project(tmp_path)
env = execution_environment(
tmp_path, project, {"python": sys.executable, "node": "node", "npm": "npm"}
)
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(tmp_path)
assert env["GOVOPLAN_CORE_ROOT"] == str(tmp_path / "govoplan-core")
assert env["GOVOPLAN_CORE_SOURCE_ROOT"] == str(tmp_path / "govoplan-core")
def test_portable_environment_does_not_inject_govoplan_scope(example, monkeypatch):
from govoplan_devkit.environment import execution_environment
args, _ = example
for key in (
"GOVOPLAN_WORKSPACE_ROOT",
"GOVOPLAN_CORE_ROOT",
"GOVOPLAN_CORE_SOURCE_ROOT",
):
monkeypatch.delenv(key, raising=False)
env = execution_environment(
args.workspace_root,
load_project(args.workspace_root, args.project),
{"python": sys.executable, "node": "node", "npm": "npm"},
)
assert "GOVOPLAN_WORKSPACE_ROOT" not in env
def test_failure_skips_dependents_but_runs_independent_check(example):
args, repo = example
result = run(
args,
[
stage(repo, "bad", "raise SystemExit(4)"),
stage(repo, "dependent", deps=["bad"]),
stage(repo, "independent"),
],
)
states = {item["id"]: item["status"] for item in result["stages"]}
assert states == {"bad": "failed", "dependent": "skipped", "independent": "passed"}
assert result["_exit_code"] == 1
def test_timeout_is_not_a_pass(example):
args, repo = example
result = run(
args, [stage(repo, code="import time; time.sleep(10)", timeout_seconds=0.1)]
)
assert result["stages"][0]["status"] == "timed_out"
def test_dry_run_does_not_create_state(example):
args, repo = example
args.dry_run = True
result = run(args, [stage(repo, code="raise SystemExit(9)")])
assert result["status"] == "planned"
assert not args.state_dir.exists()
def test_empty_plan_is_not_claimed_as_verified(example):
args, _ = example
assert run(args, [])["status"] == "not_run"
def test_plan_validation_rejects_cycles_unknowns_duplicate_ids_escape(example):
args, repo = example
invalid = [
[stage(repo, deps=["unknown"])],
[stage(repo, "a", deps=["b"]), stage(repo, "b", deps=["a"])],
[stage(repo), stage(repo)],
[stage(repo, "../escape")],
[stage(repo.parent.parent)],
]
for plan in invalid:
with pytest.raises(ValueError):
runner.validate_stages(plan, args.workspace_root, {})
def test_source_identity_includes_staged_unstaged_and_untracked_bytes(example):
args, repo = example
project = load_project(args.workspace_root, args.project)
first = source_fingerprint(project)
(repo / "source.txt").write_text("second\n")
second = source_fingerprint(project)
assert second != first
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
third = source_fingerprint(project)
assert third != second
(repo / "extra.txt").write_text("extra")
assert source_fingerprint(project) != third
def test_source_mutation_during_run_invalidates_receipt(example):
args, repo = example
result = run(
args,
[
stage(
repo,
code="from pathlib import Path; Path('source.txt').write_text('changed')",
)
],
)
assert result["stages"][0]["status"] == "stale"
assert result["stages"][0]["checkpoint_verified"] is False
assert result["status"] == "stale"
def test_resume_reuses_only_identical_plan_and_inputs(example):
args, repo = example
plan = [stage(repo)]
first = run(args, plan)
args.resume = first["run_id"]
second = run(args, plan)
assert second["stages"][0]["reused_from"] == first["run_id"]
changed_plan = run(args, [stage(repo, code="print('different')")])
assert changed_plan["status"] == "passed"
assert "reused_from" not in changed_plan["stages"][0]
assert Path(changed_plan["stages"][0]["log_path"]).read_text() == "different\n"
(repo / "source.txt").write_text("changed")
changed_source = run(args, plan)
assert changed_source["status"] == "passed"
assert "reused_from" not in changed_source["stages"][0]
assert changed_source["stages"][0]["cache_key"] != first["stages"][0]["cache_key"]
def test_receipt_integrity_and_foreign_id_are_rejected(example):
args, repo = example
result = run(args, [stage(repo)])
path = Path(result["receipt_path"])
record = read_json(path)
record["status"] = "fabricated"
atomic_json(path, record)
with pytest.raises(ValueError, match="integrity"):
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
with pytest.raises(ValueError):
runner.read_receipt(args.workspace_root, args.state_dir, "../other")
def test_shared_resources_serialize_stages(example):
args, repo = example
active = 0
maximum = 0
lock = threading.Lock()
original = runner._execute_stage_command
def execute(*args, **kwargs):
nonlocal active, maximum
with lock:
active += 1
maximum = max(maximum, active)
try:
time.sleep(0.04)
return original(*args, **kwargs)
finally:
with lock:
active -= 1
with patch.object(runner, "_execute_stage_command", side_effect=execute):
result = run(
args,
[
stage(repo, "a", resources=["shared"]),
stage(repo, "b", resources=["shared"]),
],
)
assert result["status"] == "passed"
assert maximum == 1
def test_cross_process_resource_conflict_is_explicit(example):
args, repo = example
locks = state_root(args.workspace_root) / "resource-locks"
with resource_lock(locks, "shared"):
result = run(args, [stage(repo, resources=["shared"])])
assert result["stages"][0]["status"] == "blocked"
def test_output_is_bounded_and_known_secrets_redacted(example, monkeypatch):
args, repo = example
monkeypatch.setenv("FIXTURE_SECRET", "private-fixture-credential")
with patch.object(runner, "MAX_LOG_BYTES", 1024):
result = run(
args,
[
stage(
repo,
code="import os; print(os.environ['FIXTURE_SECRET']); print('x'*10000)",
)
],
)
record = result["stages"][0]
output = Path(record["log_path"]).read_text()
assert record["output_truncated"] is True
assert "private-fixture-credential" not in output
assert "[redacted]" in output
assert len(output) < 2000
def test_symlinked_state_is_rejected(example, tmp_path):
args, repo = example
target = tmp_path / "target"
target.mkdir()
args.state_dir.symlink_to(target, target_is_directory=True)
with pytest.raises(ValueError, match="symlink"):
run(args, [stage(repo)])
@pytest.mark.parametrize("flag", ["--assume-unchanged", "--skip-worktree"])
def test_hidden_tracked_edits_invalidate_source_identity(example, flag):
args, repo = example
subprocess.run(
["git", "-C", str(repo), "update-index", flag, "source.txt"], check=True
)
project = load_project(args.workspace_root, args.project)
initial = source_fingerprint(project)
(repo / "source.txt").write_text("hidden edit\n")
assert source_fingerprint(project) != initial
def test_failed_final_snapshot_never_persists_a_passing_receipt(example):
args, repo = example
events = []
args.on_progress = events.append
original = Checkpoints.source
def fingerprint(self, *values, **kwargs):
if events and events[-1]["phase"] == "finalizing":
raise ValueError("unreadable source")
return original(self, *values, **kwargs)
with patch.object(Checkpoints, "source", fingerprint):
result = run(args, [stage(repo)])
assert result["status"] == "stale"
assert result["snapshot_verified"] is False
assert result["_exit_code"] == 1
assert "unreadable source" in result["invalidated_reason"]
assert (
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])[
"status"
]
== "stale"
)
def test_resume_rejects_a_modified_cached_log(example):
args, repo = example
plan = [stage(repo)]
first = run(args, plan)
Path(first["stages"][0]["log_path"]).write_text("altered")
args.resume = first["run_id"]
result = run(args, plan)
assert result["status"] == "failed"
assert result["stages"][0]["status"] == "failed"
assert "log integrity" in result["stages"][0]["error"]
assert "reused_from" not in result["stages"][0]
def test_receipt_argument_redaction_applies_to_unknown_separate_values(example):
args, repo = example
plan = stage(repo)
plan["argv"].extend(["--password", "fixture-not-from-environment"])
result = run(args, [plan])
assert (
"fixture-not-from-environment" not in Path(result["receipt_path"]).read_text()
)
def recovery_parser():
parser = argparse.ArgumentParser()
runner.register(parser.add_subparsers())
return parser
def test_recovery_requires_manual_survivor_confirmation(example):
args, repo = example
result = run(args, [stage(repo)])
record = read_json(Path(result["receipt_path"]))
record["status"] = "running"
record["snapshot_verified"] = False
atomic_json(Path(result["receipt_path"]), runner._seal(record))
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply"], namespace=args
)
with pytest.raises(ValueError, match="manual verification"):
parsed.handler(parsed)
def test_recovery_does_not_overwrite_completion_between_reads(example):
args, repo = example
result = run(args, [stage(repo)])
completed = runner.read_receipt(
args.workspace_root, args.state_dir, result["run_id"]
)
initial = {**completed, "status": "running"}
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=args,
)
with (
patch.object(runner, "read_receipt", side_effect=[initial, completed]),
patch.object(runner, "atomic_json") as write,
):
assert parsed.handler(parsed)["status"] == "unchanged"
write.assert_not_called()
def test_active_owner_lock_blocks_recovery(example):
args, repo = example
result = run(args, [stage(repo)])
record = read_json(Path(result["receipt_path"]))
record.update(status="running", snapshot_verified=False)
atomic_json(Path(result["receipt_path"]), runner._seal(record))
parsed = recovery_parser().parse_args(
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
namespace=args,
)
with resource_lock(
state_root(args.workspace_root, args.state_dir) / "locks",
"run:" + result["run_id"],
):
with pytest.raises(RuntimeError, match="busy"):
parsed.handler(parsed)
@pytest.mark.parametrize(
"mutation",
[
"unknown_status",
"unverified_pass",
"duplicate_stage",
"nonboolean_verification",
"nonpassing_stage",
],
)
def test_even_correctly_hashed_receipts_must_have_consistent_states(example, mutation):
args, repo = example
result = run(args, [stage(repo)])
path = Path(result["receipt_path"])
record = read_json(path)
if mutation == "unknown_status":
record["status"] = "wonderful"
elif mutation == "unverified_pass":
record["snapshot_verified"] = False
elif mutation == "duplicate_stage":
record["stages"].append(record["stages"][0])
elif mutation == "nonboolean_verification":
record["snapshot_verified"] = "true"
else:
record["stages"][0]["status"] = "failed"
# Deliberately construct an externally corrupted but correctly hashed record;
# the runner's writer now rejects inconsistent checkpoint state before sealing.
record["integrity_sha256"] = digest(
{key: value for key, value in record.items() if key != "integrity_sha256"}
)
atomic_json(path, record)
with pytest.raises(ValueError):
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
+251
View File
@@ -0,0 +1,251 @@
"""Published-schema parity and all-declaration validation, without commands."""
from copy import deepcopy
import json
from pathlib import Path
import sys
import jsonschema
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/devkit"))
from govoplan_devkit.workspace import load_project # noqa: E402
from govoplan_devkit.validation import _schema_value # noqa: E402
def valid():
return {
"schema_version": 1,
"repositories": [{"name": "example", "path": "example"}],
"checks": [
{"id": "selected", "argv": ["true"]},
{"id": "unselected", "argv": ["true"]},
],
"profiles": {"quick": ["selected"], "full": ["unselected"]},
}
def check(tmp_path, value):
path = tmp_path / "project.json"
path.write_text(json.dumps(value))
return load_project(tmp_path, path)
@pytest.mark.parametrize(
"field,value",
[
("resource", ["shared"]),
("timout_seconds", 20),
("cwd", "../escape"),
("cwd", "/outside"),
("argv", [""]),
("argv", ["true", "x" * 8193]),
("argv", ["true"] * 257),
("argv", ["true", "bad\0value"]),
("resources", ["a", "a"]),
("resources", [""]),
("resources", ["x" * 257]),
("resources", ["bad\nname"]),
("resources", True),
("deps", ["selected", "selected"]),
("repos", ["example", "example"]),
("title", 10),
("title", "x" * 1025),
("timeout_seconds", True),
("timeout_seconds", 43201),
("timeout_seconds", 0),
("id", "invalid\n"),
],
)
def test_unselected_checks_obey_the_published_schema(tmp_path, field, value):
project = valid()
project["checks"][1][field] = value
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"mutation",
[
lambda value: value["profiles"].update(typo=[]),
lambda value: value["profiles"].update(full=["unselected", "unselected"]),
lambda value: value["tools"].update(pyton="python"),
lambda value: value["tools"].update(python="bad\0tool"),
lambda value: value["review"].update(principle="rules.md"),
lambda value: value["review"].update(principles="../rules.md"),
lambda value: value["repositories"][0].update(alias="other"),
lambda value: value["repositories"][0].update(aliases=["alias", "alias"]),
lambda value: value.update(checks=value["checks"] * 257),
lambda value: value.update(schema_version=True),
],
)
def test_all_nested_shape_constraints_match_schema(tmp_path, mutation):
project = valid()
project.update(tools={}, review={})
mutation(project)
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"mutation,reason",
[
(
lambda value: value["checks"][1].update(deps=["missing"]),
"Unknown check dependency",
),
(lambda value: value["checks"][1].update(deps=["unselected"]), "Cyclic"),
(
lambda value: value["checks"][1].update(repos=["missing"]),
"Unknown repository",
),
(lambda value: value["profiles"].update(full=["missing"]), "Unknown profile"),
(lambda value: value["checks"][1].update(id="selected"), "Duplicate"),
(
lambda value: value["repositories"].append(
{"name": "other", "path": "example"}
),
"Duplicate project repository path",
),
],
)
def test_all_semantic_references_are_checked_even_outside_selected_profile(
tmp_path, mutation, reason
):
project = valid()
mutation(project)
with pytest.raises(ValueError, match=reason):
check(tmp_path, project)
def test_valid_defaults_boundaries_and_dependency_order(tmp_path):
project = valid()
project["checks"][0].update(
deps=["unselected"], argv=["true", ""], timeout_seconds=0.1
)
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
jsonschema.validate(project, schema)
assert check(tmp_path, project).config == project
def test_runtime_shape_uses_only_implemented_schema_keywords():
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
permitted = {
"$schema",
"$defs",
"$ref",
"title",
"description",
"type",
"const",
"properties",
"additionalProperties",
"required",
"minItems",
"maxItems",
"uniqueItems",
"prefixItems",
"items",
"minLength",
"maxLength",
"pattern",
"maximum",
"exclusiveMinimum",
"enum",
}
def visit(node):
assert set(node) <= permitted
for group in ("properties", "$defs"):
for child in node.get(group, {}).values():
visit(child)
if "items" in node:
visit(node["items"])
for child in node.get("prefixItems", []):
visit(child)
visit(schema)
_schema_value(valid(), schema, schema["$defs"], "project")
def test_huge_numeric_input_is_a_controlled_bound_error(tmp_path):
project = deepcopy(valid())
project["checks"][1]["timeout_seconds"] = 10**1000
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"field,value",
[
("inputs", {}),
("inputs", None),
("inputs", {"repos": []}),
("inputs", {"paths": ["src/**"]}),
("inputs", {"repos": ["example", "example"]}),
("inputs", {"repos": [""]}),
("inputs", {"repos": ["example"], "glob": "*"}),
("after", ["selected", "selected"]),
("after", [""]),
("after", "selected"),
("reuse", True),
("reuse", "always"),
],
)
def test_input_order_and_reuse_shapes_apply_to_unselected_checks(
tmp_path, field, value
):
project = valid()
project["checks"][1][field] = value
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
with pytest.raises(jsonschema.ValidationError):
jsonschema.validate(project, schema)
with pytest.raises(ValueError):
check(tmp_path, project)
@pytest.mark.parametrize(
"change,reason",
[
({"inputs": {"repos": ["unknown"]}}, "Unknown input repository"),
({"after": ["unknown"]}, "Unknown check ordering"),
({"after": ["unselected"]}, "Cyclic"),
(
{"deps": ["selected"], "after": ["selected"]},
"Duplicate dependency/ordering",
),
],
)
def test_input_and_order_semantics_are_validated_before_selection(
tmp_path, change, reason
):
project = valid()
project["checks"][1].update(change)
with pytest.raises(ValueError, match=reason):
check(tmp_path, project)
def test_dependencies_and_order_only_edges_share_cycle_validation(tmp_path):
project = valid()
project["checks"][0]["after"] = ["unselected"]
project["checks"][1]["deps"] = ["selected"]
with pytest.raises(ValueError, match="Cyclic"):
check(tmp_path, project)
def test_valid_explicit_inputs_after_and_reuse_match_published_schema(tmp_path):
project = valid()
project["checks"][0].update(
inputs={"repos": ["example"]}, after=["unselected"], reuse="verified"
)
project["checks"][1]["reuse"] = "never"
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
jsonschema.validate(project, schema)
assert check(tmp_path, project).config == project
+151
View File
@@ -0,0 +1,151 @@
import json
import os
from pathlib import Path
import subprocess
import sys
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
from govoplan_devkit.common import atomic_json, read_json, redact, safe_output
from govoplan_devkit.context import build_context
from govoplan_devkit.workspace import (
load_project,
selected_repositories,
git_bytes,
inspect_repository,
Repository,
)
def test_portable_project_rejects_escape_alias_collisions_and_duplicate_json(tmp_path):
config = tmp_path / "project.json"
for repositories in (
[{"name": "repo", "path": "../outside"}],
[
{"name": "repo", "path": "repo", "aliases": ["other"]},
{"name": "other", "path": "other"},
],
):
config.write_text(
json.dumps({"schema_version": 1, "repositories": repositories})
)
with pytest.raises(ValueError):
load_project(tmp_path, config)
config.write_text('{"schema_version":1,"schema_version":2}')
with pytest.raises(ValueError, match="Duplicate"):
read_json(config)
def test_missing_repository_is_error_not_clean(tmp_path):
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{
"schema_version": 1,
"name": "Test",
"repositories": [{"name": "missing", "path": "missing"}],
}
)
)
result = build_context(tmp_path, config, [], True)
assert result["_exit_code"] == 1
assert result["repositories"][0]["errors"]
assert not result["remote_checked"]
def test_unknown_repo_filter_is_not_silently_ignored(tmp_path):
config = tmp_path / "project.json"
config.write_text(
json.dumps(
{"schema_version": 1, "repositories": [{"name": "repo", "path": "repo"}]}
)
)
with pytest.raises(ValueError, match="Unknown"):
selected_repositories(load_project(tmp_path, config), ["typo"])
def test_atomic_output_does_not_chmod_existing_parent(tmp_path):
folder = tmp_path / "public"
folder.mkdir(mode=0o755)
atomic_json(folder / "private.json", {"safe": True})
assert folder.stat().st_mode & 0o777 == 0o755
assert (folder / "private.json").stat().st_mode & 0o777 == 0o600
def test_read_rejects_fifo_and_symlink(tmp_path):
fifo = tmp_path / "fifo"
os.mkfifo(fifo)
with pytest.raises(ValueError, match="regular"):
read_json(fifo)
target = tmp_path / "target.json"
target.write_text("{}")
alias = tmp_path / "alias.json"
alias.symlink_to(target)
with pytest.raises(ValueError, match="symlink"):
read_json(alias)
def test_generic_secret_display_hygiene():
assert "topsecret" not in redact(
"Authorization: Bearer topsecret\nhttps://user:topsecret@example.invalid/"
)
def test_separate_secret_arguments_are_redacted_in_nested_json():
value = {
"stages": [{"argv": ["check", "--token", "hidden-value"]}],
"password": "hidden-password",
}
output = json.dumps(safe_output(value))
assert "hidden-value" not in output and "hidden-password" not in output
@pytest.mark.parametrize(
"name",
[
"GIT_DIR",
"GIT_NAMESPACE",
"GIT_COMMON_DIR",
"GIT_CONFIG_PARAMETERS",
"GIT_AUTHOR_NAME",
],
)
def test_inherited_git_redirects_are_rejected(tmp_path, monkeypatch, name):
monkeypatch.setenv(name, "fixture")
assert inspect_repository(Repository("example", tmp_path))["errors"]
with pytest.raises(ValueError, match="overrides"):
git_bytes(tmp_path, "status")
def test_deeply_nested_json_fails_as_controlled_input_error(tmp_path):
path = tmp_path / "deep.json"
path.write_text("[" * 10000 + "]" * 10000)
with pytest.raises(ValueError, match="nesting"):
read_json(path)
def test_clean_commit_without_upstream_is_still_changed(tmp_path):
subprocess.run(["git", "init", "-q", str(tmp_path)], check=True)
subprocess.run(
[
"git",
"-C",
str(tmp_path),
"-c",
"user.name=Fixture",
"-c",
"user.email=fixture@example.invalid",
"commit",
"--allow-empty",
"-qm",
"fixture",
],
check=True,
)
from govoplan_devkit.workspace import Project
repo = Repository("example", tmp_path)
assert selected_repositories(Project("Example", (repo,), {}), [], changed=True) == [
repo
]
+393
View File
@@ -0,0 +1,393 @@
"""Canonical phase dispatch tested with inert tools in disposable workspaces."""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import pytest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools/checks/check-focused.sh"
METADATA = META_ROOT / "tools/checks/focused-phases.json"
PHASE_IDS = [
"preflight",
"tooling",
"backend",
"core-ui",
"module-builds",
"browser",
"module-ui",
]
# These hashes bind the original working-copy check bodies at the phase split.
# Only two explicit Core/webui cd lines were added for independent invocation.
# Future deliberate changes to the canonical gate must update this contract.
LEGACY_BODY_SHA256 = {
"preflight": "b9c90fd3c84df788de5f2c001443672f683a9918459e1a7955ed4a225e6f20dd",
"tooling": "3d9a5bf32acbe97134f51327ed4b2457a69ad23b723c15a2b9bd0dce7f82396b",
"backend": "dd57c33919f06bd240516bbe47861be022e0b1c047334eedc7f6c596c2c49632",
"core-ui": "613f806a602970f1dbfb243f4c96ccc6ca4836849bb1f1a2c4cff97aea6f3aeb",
"module-builds": "1f2cd1e2c336f748fbf2972a2da87a0efec58ef9b084a9433511f794a456fa84",
"browser": "aa20d1e1d4ec9f00bc27c06cdee7ffa2456d3d3b8a7155da4e888a7e2209306d",
"module-ui": "c47794d82eb7bb47de114e86a95264e5898886f743c31eb7bdf82a079c827c9f",
}
EXTRA_TEST_COMMAND = '"$PYTHON" -m pytest -q tests/test_focused_phases.py\n'
def definitions():
source = SCRIPT.read_text()
return dict(
re.findall(
r"^# devkit-phase: ([a-z-]+) begin\n(.*?)^# devkit-phase: \1 end$",
source,
re.MULTILINE | re.DOTALL,
)
)
def test_original_commands_remain_exactly_once_in_original_order():
bodies = definitions()
metadata = json.loads(METADATA.read_text())
assert list(bodies) == PHASE_IDS
assert [phase["id"] for phase in metadata["phases"]] == PHASE_IDS
assert SCRIPT.read_text().count(EXTRA_TEST_COMMAND) == 1
for identity, body in bodies.items():
assert SCRIPT.read_text().count(f"# devkit-phase: {identity} begin") == 1
if identity == "tooling":
assert body.count(EXTRA_TEST_COMMAND) == 1
body = body.replace(EXTRA_TEST_COMMAND, "")
assert hashlib.sha256(body.encode()).hexdigest() == LEGACY_BODY_SHA256[identity]
def test_ordering_and_artifact_dependencies_are_distinct():
phases = json.loads(METADATA.read_text())["phases"]
assert [phase["order_after"] for phase in phases] == [
[],
*[[identity] for identity in PHASE_IDS[:-1]],
]
assert all(phase["depends_on"] == [] for phase in phases)
by_id = {phase["id"]: phase for phase in phases}
assert "{core}/webui/dist" in by_id["module-builds"]["outputs"]
assert any(
"does not serve or require module-builds dist" in note
for note in by_id["browser"]["notes"]
)
assert "port:4174" in by_id["browser"]["resources"]
cwd_lines = {
"core": 'cd "$ROOT"',
"meta": 'cd "$META_ROOT"',
"core-webui": 'cd "$ROOT/webui"',
"access-webui": 'cd "${WORKSPACE_ROOT}/govoplan-access/webui"',
}
for phase in phases:
assert definitions()[phase["id"]].splitlines()[0] == cwd_lines[phase["cwd"]]
FAKE_TOOL = r"""
import json
import os
from pathlib import Path
import sys
log = Path(os.environ["FOCUSED_FIXTURE_LOG"])
record = {
"tool": Path(sys.argv[0]).name,
"argv": sys.argv[1:],
"cwd": os.getcwd(),
"env": {key: os.environ.get(key) for key in (
"GOVOPLAN_WORKSPACE_ROOT", "NPM_CONFIG_USERCONFIG", "GOVOPLAN_NPM_USERCONFIG",
"NPM_CONFIG_TMP", "npm_config_tmp", "PYTHONPATH", "PATH",
)},
}
if "-" in sys.argv[1:]:
record["stdin"] = sys.stdin.read()
with log.open("a") as handle:
handle.write(json.dumps(record) + "\n")
if os.environ.get("FOCUSED_FIXTURE_FAIL_TOKEN") in sys.argv[1:]:
raise SystemExit(7)
"""
@pytest.fixture
def fixture_workspace(tmp_path):
workspace = tmp_path / "workspace"
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
copied = meta / "tools/checks/check-focused.sh"
copied.parent.mkdir(parents=True)
shutil.copyfile(SCRIPT, copied)
shutil.copyfile(METADATA, copied.with_name("focused-phases.json"))
for name in (
"govoplan",
"govoplan-core",
"govoplan-access",
"govoplan-payments",
"govoplan-dataflow",
"govoplan-datasources",
"govoplan-workflow",
"govoplan-dashboard",
"govoplan-approvals",
"govoplan-postbox",
"govoplan-mail",
"govoplan-files",
"govoplan-campaign",
"govoplan-policy",
"govoplan-wiki",
):
(workspace / name / "webui").mkdir(parents=True, exist_ok=True)
(workspace / name / "src").mkdir(exist_ok=True)
(meta / "tests").mkdir()
for name in ("test_devkit_alpha.py", "test_devkit_beta.py"):
(meta / "tests" / name).write_text("# inert glob fixture\n")
fake_bin = tmp_path / "bin"
fake_bin.mkdir()
for target in [
*(fake_bin / name for name in ("python-check", "node", "npm", "bash")),
core / "webui/node_modules/.bin/tsc",
meta / "tools/checks/check_dependency_boundaries.py",
]:
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(f"#!{sys.executable}\n" + FAKE_TOOL)
target.chmod(0o700)
temporary = tmp_path / "temporary"
temporary.mkdir()
log = tmp_path / "calls.jsonl"
env = {
**os.environ,
"PATH": str(fake_bin) + os.pathsep + os.environ["PATH"],
"GOVOPLAN_WORKSPACE_ROOT": str(workspace),
"GOVOPLAN_CORE_ROOT": str(core),
"PYTHON": str(fake_bin / "python-check"),
"NODE": str(fake_bin / "node"),
"NPM": str(fake_bin / "npm"),
"TMPDIR": str(temporary),
"FOCUSED_FIXTURE_LOG": str(log),
"PYTHONPATH": "inherited-fixture-tail",
"NPM_CONFIG_TMP": "must-be-unset",
"npm_config_tmp": "must-be-unset",
}
env.pop("FOCUSED_FIXTURE_FAIL_TOKEN", None)
return {
"workspace": workspace,
"meta": meta,
"core": core,
"script": copied,
"env": env,
"log": log,
"temporary": temporary,
}
def invoke(fixture, *arguments, env=None):
return subprocess.run(
["/bin/bash", str(fixture["script"]), *arguments],
cwd=fixture["workspace"].parent,
env=env or fixture["env"],
capture_output=True,
text=True,
timeout=30,
)
def records(fixture):
path = fixture["log"]
return (
[json.loads(line) for line in path.read_text().splitlines()]
if path.exists()
else []
)
def normalized(calls):
result = []
for call in calls:
call = json.loads(json.dumps(call))
call["env"].pop("NPM_CONFIG_USERCONFIG")
call["env"].pop("GOVOPLAN_NPM_USERCONFIG")
result.append(call)
return result
def test_default_gate_equals_sequential_independent_phases(fixture_workspace):
fixture = fixture_workspace
full = invoke(fixture)
assert full.returncode == 0, full.stderr
original = records(fixture)
offset = len(original)
for identity in PHASE_IDS:
isolated = invoke(fixture, "--phase", identity)
assert isolated.returncode == 0, isolated.stderr
assert normalized(original) == normalized(records(fixture)[offset:])
assert len({call["env"]["NPM_CONFIG_USERCONFIG"] for call in original}) == len(
PHASE_IDS
)
assert not list(fixture["temporary"].iterdir())
assert sum("test:conformance" in call["argv"] for call in original) == 1
assert sum("test:module-permutations" in call["argv"] for call in original) == 1
assert sum("tests/test_focused_phases.py" in call["argv"] for call in original) == 1
# The here-document remains one Python invocation, not executed shell text.
ast_scan = [call for call in original if "stdin" in call]
assert len(ast_scan) == 1
assert "AST syntax check passed for" in ast_scan[0]["stdin"]
@pytest.mark.parametrize("identity", PHASE_IDS)
def test_each_phase_initializes_its_own_cwd_and_environment(
fixture_workspace, identity
):
fixture = fixture_workspace
result = invoke(fixture, "--phase", identity)
assert result.returncode == 0, result.stderr
calls = records(fixture)
assert calls
phase = next(
item
for item in json.loads(METADATA.read_text())["phases"]
if item["id"] == identity
)
starts = {
"core": fixture["core"],
"meta": fixture["meta"],
"core-webui": fixture["core"] / "webui",
"access-webui": fixture["workspace"] / "govoplan-access/webui",
}
assert calls[0]["cwd"] == str(starts[phase["cwd"]])
for call in calls:
env = call["env"]
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(fixture["workspace"])
assert env["NPM_CONFIG_USERCONFIG"] == env["GOVOPLAN_NPM_USERCONFIG"]
assert Path(env["NPM_CONFIG_USERCONFIG"]).parent == fixture["temporary"]
assert not Path(env["NPM_CONFIG_USERCONFIG"]).exists()
assert env["NPM_CONFIG_TMP"] is None and env["npm_config_tmp"] is None
assert env["PATH"].split(os.pathsep)[0] == str(
fixture["core"] / "webui/node_modules/.bin"
)
assert env["PYTHONPATH"].endswith(os.pathsep + "inherited-fixture-tail")
assert str(fixture["core"] / "src") in env["PYTHONPATH"].split(os.pathsep)
assert not list(fixture["temporary"].iterdir())
def test_full_is_fail_fast_and_failure_cleans_setup(fixture_workspace):
fixture = fixture_workspace
result = invoke(
fixture,
env={
**fixture["env"],
"FOCUSED_FIXTURE_FAIL_TOKEN": "test:module-permutations",
},
)
assert result.returncode == 7
calls = records(fixture)
assert calls[-1]["argv"] == ["run", "test:module-permutations"]
assert not any("test:conformance" in call["argv"] for call in calls)
assert not any("test:passwords" in call["argv"] for call in calls)
assert not list(fixture["temporary"].iterdir())
@pytest.mark.parametrize(
"arguments",
[
["--unknown"],
["--phase"],
["--phase", ""],
["--phase", "not-a-phase"],
["--phase", "browser", "--phase", "tooling"],
["--list-phases", "--phase", "browser"],
["--phase", "browser", "--list-phases"],
["--list-phases", "--list-phases"],
["--json"],
["--list-phases", "--json", "--json"],
["--phase", "browser; touch unexpected"],
],
)
def test_invalid_selection_is_rejected_before_setup(fixture_workspace, arguments):
fixture = fixture_workspace
env = {
**fixture["env"],
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
"NODE": "absent-node",
"NPM": "absent-npm",
}
result = invoke(fixture, *arguments, env=env)
assert result.returncode == 2
assert "check-focused:" in result.stderr
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
@pytest.mark.parametrize(
"arguments",
[["--list-phases"], ["--list-phases", "--json"], ["--json", "--list-phases"]],
)
def test_listing_is_read_only_without_product_tools(fixture_workspace, arguments):
fixture = fixture_workspace
env = {
**fixture["env"],
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
"PYTHON": "/absent-python",
"NODE": "absent-node",
"NPM": "absent-npm",
}
result = invoke(fixture, *arguments, env=env)
assert result.returncode == 0, result.stderr
if "--json" in arguments:
assert json.loads(result.stdout) == json.loads(METADATA.read_text())
else:
assert [line.split("\t")[0] for line in result.stdout.splitlines()] == PHASE_IDS
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
def test_metadata_loading_never_imports_project_python_modules(fixture_workspace):
fixture = fixture_workspace
foreign = fixture["workspace"].parent / "json.py"
foreign.write_text(
"raise AssertionError('project module imported during listing')\n"
)
result = invoke(
fixture,
"--list-phases",
"--json",
env={**fixture["env"], "PYTHONPATH": str(foreign.parent)},
)
assert result.returncode == 0, result.stderr
assert json.loads(result.stdout)["schema_version"] == 1
assert not fixture["log"].exists()
@pytest.mark.parametrize(
"change",
[
"unknown-fields",
"duplicate-id",
"unknown-cwd",
"later-prerequisite",
"invalid-version",
],
)
def test_malformed_metadata_fails_before_any_check(fixture_workspace, change):
fixture = fixture_workspace
path = fixture["script"].with_name("focused-phases.json")
catalog = json.loads(path.read_text())
if change == "unknown-fields":
catalog["phases"][0]["shell"] = "touch should-not-run"
elif change == "duplicate-id":
catalog["phases"][1]["id"] = catalog["phases"][0]["id"]
elif change == "unknown-cwd":
catalog["phases"][0]["cwd"] = "elsewhere"
elif change == "later-prerequisite":
catalog["phases"][0]["depends_on"] = ["browser"]
else:
catalog["schema_version"] = True
path.write_text(json.dumps(catalog))
result = invoke(fixture, "--phase", "browser")
assert result.returncode == 2
assert not fixture["log"].exists()
assert not list(fixture["temporary"].iterdir())
+466
View File
@@ -0,0 +1,466 @@
from __future__ import annotations
import base64
from contextlib import ExitStack
import csv
from copy import deepcopy
import hashlib
import io
import json
import os
from pathlib import Path
import sys
import tarfile
import tempfile
import unittest
from unittest.mock import patch
from urllib.parse import quote
import zipfile
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import full_catalog # noqa: E402
from govoplan_release.artifact_identity import selected_artifact_identity_issues # noqa: E402
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.registry_reference import registry_entry_source # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
load_authenticated_catalog_base, public_key_base64, signature,
)
from govoplan_release.source_provenance import ( # noqa: E402
SourceTagProvenanceIssue, catalog_source_selection,
registered_source_origin_issues,
)
from govoplan_release.version_alignment import candidate_catalog_version_issues # noqa: E402
class FullRegistryCatalogTests(unittest.TestCase):
def setUp(self) -> None:
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.web = self.root / "addideas-govoplan-website"
self.wheels = self.root / "wheels"
self.npm = self.root / "npm"
self.wheels.mkdir(mode=0o700)
self.npm.mkdir(mode=0o700)
self.key = Ed25519PrivateKey.generate()
self.keypath = self.root / "key.pem"
self.keypath.write_bytes(self.key.private_bytes(
serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
))
self.keypath.chmod(0o600)
self.keyring = {
"keyring_version": "1", "keys": [{
"key_id": "known-key", "public_key": public_key_base64(self.key),
"status": "active",
}],
}
self.base = {
"catalog_version": "1", "channel": "stable", "sequence": 1,
"core_release": {"version": "1.0.0"}, "modules": [],
"release": {},
}
self.write_base()
self.package_set = {
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
"registries": {
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
},
"python": [self.package("govoplan-core"), self.package("govoplan-demo")],
"webui": [self.package("govoplan-core", webui=True)],
}
self.seal(self.package_set, "package_set_sha256")
self.lock = {
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
"registries": self.package_set["registries"],
"package_set_sha256": self.package_set["package_set_sha256"],
"python": [], "webui": [],
}
for row in self.package_set["python"]:
path = self.wheel(row["name"])
url = full_catalog._tool("resolve-package-artifacts")["_python_artifact_url"](
self.package_set["registries"]["python"], package=row, filename=path.name,
)
self.lock["python"].append(self.artifact(row, path, url))
npm_package = self.package_set["webui"][0]
npm_path = self.npm / "govoplan-core-webui-1.2.3.tgz"
self.tarball(npm_path, "@govoplan/core-webui", "1.2.3")
url = self.package_set["registries"]["npm"] + quote(npm_package["name"], safe="") + "/-/1.2.3/core-webui-1.2.3.tgz"
row = self.artifact(npm_package, npm_path, url)
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(npm_path.read_bytes()).digest()).decode()
self.lock["webui"].append(row)
self.seal(self.lock, "lock_sha256")
self.set_path = self.root / "package-set.json"
self.lock_path = self.root / "package-lock.json"
self.write_inputs()
self.output = self.root / "candidate"
@staticmethod
def package(repo: str, *, webui: bool = False) -> dict:
row = {
"name": "@govoplan/core-webui" if webui else repo, "version": "1.2.3",
"repository": repo, "tag": "v1.2.3",
"commit": ("a" if repo == "govoplan-core" else "b") * 40,
}
if not webui:
row["extras"] = ["server"] if repo == "govoplan-core" else []
return row
@staticmethod
def artifact(package: dict, path: Path, url: str) -> dict:
encoded = path.read_bytes()
return {**package, "filename": path.name, "url": url,
"sha256": hashlib.sha256(encoded).hexdigest(), "size": len(encoded)}
@staticmethod
def seal(payload: dict, field: str) -> None:
payload.pop(field, None)
payload[field] = hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
def write_inputs(self) -> None:
self.set_path.write_text(json.dumps(self.package_set))
self.lock_path.write_text(json.dumps(self.lock))
def write_base(self) -> None:
self.base.pop("signatures", None)
self.base["signatures"] = [signature(self.base, key_id="known-key", private_key=self.key)]
folder = self.web / "public/catalogs/v1"
(folder / "channels").mkdir(parents=True, exist_ok=True)
(folder / "channels/stable.json").write_text(json.dumps(self.base))
(folder / "keyring.json").write_text(json.dumps(self.keyring))
def wheel(self, package: str) -> Path:
stem = package.replace("-", "_")
info = f"{stem}-1.2.3.dist-info"
files = {
f"{stem}/__init__.py": b"VALUE = 1\n",
f"{info}/METADATA": f"Metadata-Version: 2.1\nName: {package}\nVersion: 1.2.3\n".encode(),
f"{info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
}
record = io.StringIO()
writer = csv.writer(record, lineterminator="\n")
for name, value in files.items():
writer.writerow((name, "", len(value)))
writer.writerow((f"{info}/RECORD", "", ""))
files[f"{info}/RECORD"] = record.getvalue().encode()
path = self.wheels / f"{stem}-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(path, "w") as archive:
for name, value in files.items():
archive.writestr(name, value)
return path
@staticmethod
def tarball(path: Path, name: str, version: str, *, duplicate: bool = False) -> None:
encoded = json.dumps({"name": name, "version": version}).encode()
with tarfile.open(path, "w:gz") as archive:
for _ in range(2 if duplicate else 1):
member = tarfile.TarInfo("package/package.json")
member.size = len(encoded)
archive.addfile(member, io.BytesIO(encoded))
def build(self, *, provenance_errors=(), origin_errors=()) -> dict:
registry_generator = full_catalog._tool("generate-release-catalog")
tools = {name: dict(full_catalog._tool(name)) for name in (
"generate-release-catalog", "generate-release-package-set", "resolve-package-artifacts",
)}
tools["generate-release-package-set"]["generate_package_set"] = lambda **kwargs: self.package_set
self.provenance = {
row["repository"]: {"commit_sha": row["commit"], "tag_object_sha": str(index + 1) * 40}
for index, row in enumerate(self.package_set["python"])
}
entry = {
"module_id": "demo", "name": "Demo", "version": "1.2.3",
"python_package": "govoplan-demo",
"python_ref": "govoplan-demo @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-demo.git@v1.2.3",
}
with ExitStack() as stack:
stack.enter_context(patch.object(full_catalog, "_tool", side_effect=lambda name: tools[name]))
stack.enter_context(patch.dict(registry_generator["_catalog_payload"].__globals__, {
"synthesize_repository_catalog_entries": lambda **kwargs: (dict(entry),),
}))
stack.enter_context(patch.object(full_catalog, "enforce_selected_version_alignment"))
stack.enter_context(patch.object(full_catalog, "registered_source_origin_issues", return_value=origin_errors))
self.provenance_check = stack.enter_context(patch.object(full_catalog, "source_tag_provenance_issues", return_value=provenance_errors))
stack.enter_context(patch.object(full_catalog, "selected_source_provenance", return_value=self.provenance))
return full_catalog.build_full_registry_candidate(
package_set_path=self.set_path, package_lock_path=self.lock_path,
wheelhouse=self.wheels, webui_packages=self.npm, output_dir=self.output,
selected_repositories=("govoplan-core",),
signing_keys=(f"known-key={self.keypath}",), workspace_root=self.root,
)
def candidate(self) -> dict:
return json.loads((self.output / "channels/stable.json").read_text())
def test_full_candidate_uses_registry_bytes_and_preserves_unchanged_tag_provenance(self) -> None:
result = self.build()
candidate = self.candidate()
self.assertEqual("ready", result["status"])
self.assertEqual(2, result["package_count"])
self.assertEqual(1, result["selected_count"])
self.assertEqual(self.keyring, json.loads((self.output / "keyring.json").read_text()))
self.assertEqual(canonical_hash(self.keyring), candidate["release"]["keyring_sha256"])
self.assertEqual(2, len(candidate["release"]["artifacts"]))
self.assertIn("/pypi/files/", candidate["core_release"]["python_ref"])
self.assertEqual((), candidate_catalog_version_issues(candidate))
self.assertEqual((), selected_artifact_identity_issues(candidate))
sources = catalog_source_selection(candidate)
self.assertEqual((), sources.issues)
self.assertEqual({"govoplan-core": "1.2.3"}, sources.selected_versions)
self.assertEqual({"govoplan-core": "1.2.3", "govoplan-demo": "1.2.3"}, sources.all_versions)
self.assertEqual("b" * 40, sources.selected_commits["govoplan-demo"])
self.assertEqual("2" * 40, sources.selected_tag_objects["govoplan-demo"])
self.assertEqual(2, self.provenance_check.call_count)
for call in self.provenance_check.call_args_list:
self.assertEqual({"govoplan-core"}, call.kwargs["require_head_repos"])
for path in [self.output, *self.output.rglob("*")]:
self.assertEqual(0o700 if path.is_dir() else 0o600, path.stat().st_mode & 0o777)
def test_legacy_base_is_authenticated_but_remains_rejected_by_selective(self) -> None:
self.build()
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=None, base_keyring=None, web_root=self.web,
channel="stable", public_base_url="https://unused.example",
signer_public_keys={"known-key": public_key_base64(self.key)},
)
def test_injected_key_or_mismatched_pinned_keyring_is_rejected(self) -> None:
for mutation in ("extra-key", "bad-hash"):
with self.subTest(mutation=mutation):
if mutation == "extra-key":
self.keyring["keys"].append({"key_id": "injected", "status": "active", "public_key": public_key_base64(Ed25519PrivateKey.generate())})
else:
self.keyring["keys"] = self.keyring["keys"][:1]
self.base["release"]["keyring_sha256"] = "f" * 64
self.write_base()
with self.assertRaises(ValueError):
self.build()
self.assertFalse(self.output.exists())
def test_tampered_base_signature_is_rejected(self) -> None:
path = self.web / "public/catalogs/v1/channels/stable.json"
payload = json.loads(path.read_text())
payload["sequence"] = 999
path.write_text(json.dumps(payload))
with self.assertRaisesRegex(ValueError, "signature verification"):
self.build()
def test_wrong_registry_bytes_and_reused_candidate_fail_closed(self) -> None:
self.build()
original = (self.output / "channels/stable.json").read_bytes()
with self.assertRaisesRegex(ValueError, "must not already exist"):
self.build()
self.assertEqual(original, (self.output / "channels/stable.json").read_bytes())
self.output = self.root / "candidate-2"
wheel = self.wheels / self.lock["python"][0]["filename"]
with wheel.open("ab") as stream:
stream.write(b"tampered")
with self.assertRaisesRegex(ValueError, "bytes differ"):
self.build()
self.assertFalse(self.output.exists())
def test_webui_identity_and_url_changes_are_rejected(self) -> None:
row = self.lock["webui"][0]
original = row["url"]
for url in (
original + "?alternate=true", original.replace("/-/1.2.3/", "/-/9.9.9/"),
original.replace("/npm/", "/npm/../other/"),
original.replace("/npm/", "/npm/%2e%2e/other/"),
original.replace("/npm/", "/npm/%252e%252e/other/"),
):
with self.subTest(url=url):
row["url"] = url
self.seal(self.lock, "lock_sha256")
self.write_inputs()
with self.assertRaisesRegex(ValueError, "URL differs"):
self.build()
def test_duplicate_missing_and_symlinked_artifacts_are_rejected(self) -> None:
row = self.lock["webui"][0]
self.lock["webui"].append(dict(row))
with self.assertRaisesRegex(ValueError, "duplicate/missing"):
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
self.lock["webui"].pop()
path = self.npm / row["filename"]
moved = self.root / "moved.tgz"
path.rename(moved)
path.symlink_to(moved)
with self.assertRaises(OSError):
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
def test_archive_metadata_is_bounded_and_not_ambiguous(self) -> None:
path = self.npm / "duplicate.tgz"
self.tarball(path, "@govoplan/core-webui", "1.2.3", duplicate=True)
with self.assertRaisesRegex(ValueError, "duplicate"):
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
def test_origin_or_tag_provenance_failure_prevents_output(self) -> None:
issue = SourceTagProvenanceIssue("govoplan-core", "v1.2.3", "wrong immutable identity")
for kwargs in ({"origin_errors": (issue,)}, {"provenance_errors": (issue,)}):
with self.subTest(kwargs=kwargs), self.assertRaisesRegex(ValueError, "gate failed"):
self.build(**kwargs)
self.assertFalse(self.output.exists())
def test_registered_source_origin_requires_exact_fetch_and_push_targets(self) -> None:
repo = self.root / "govoplan-core"
repo.mkdir()
spec = RepositorySpec("govoplan-core", "system", "kernel", "git@example.test:trusted/core.git", "govoplan-core")
with patch("govoplan_release.source_provenance.load_repository_specs", return_value=(spec,)):
for targets in ((spec.remote, spec.remote), ("git@evil.test:core.git", spec.remote), (spec.remote, "git@evil.test:core.git")):
with self.subTest(targets=targets), patch("govoplan_release.source_provenance.git_text", side_effect=targets):
issues = registered_source_origin_issues(repo_versions={"govoplan-core": "1.2.3"}, workspace=self.root, remote="origin")
self.assertEqual(targets != (spec.remote, spec.remote), bool(issues))
def test_registry_metadata_cannot_cross_wire_webui_or_archive_identity(self) -> None:
self.build()
candidate = self.candidate()
entry = candidate["core_release"]
entry["webui_package"] = "@govoplan/files-webui"
with self.assertRaisesRegex(ValueError, "another source repository"):
registry_entry_source(entry)
candidate = self.candidate()
candidate["release"]["artifacts"][0]["archive_sha256"] = "f" * 64
self.assertIn("matching inspected wheel", " ".join(selected_artifact_identity_issues(candidate)))
def test_registry_version_and_selected_source_identity_must_agree(self) -> None:
self.build()
for field in ("commit_sha", "tag_object_sha"):
candidate = self.candidate()
candidate["release"]["selected_units"][0][field] = "f" * 40
self.assertIn("differs", " ".join(issue.message for issue in catalog_source_selection(candidate).issues))
candidate = self.candidate()
candidate["modules"][0]["artifact_integrity"]["python"]["git_ref"] = "v9.9.9"
self.assertTrue(candidate_catalog_version_issues(candidate))
def test_repeated_module_projections_must_bind_identical_python_and_webui_bytes(self) -> None:
self.build()
for kind in ("python", "webui"):
for reversed_order in (False, True):
with self.subTest(kind=kind, reversed_order=reversed_order):
candidate = self.candidate()
entry = deepcopy(candidate["core_release"])
entry["module_id"] = "another-core-projection"
artifact = entry["artifact_integrity"][kind]
artifact["sha256"] = "f" * 64
if kind == "python":
entry["python_ref"] = artifact["ref"] = entry["python_ref"].split("#sha256=", 1)[0] + "#sha256=" + "f" * 64
if reversed_order:
original = candidate["core_release"]
candidate["core_release"] = entry
entry = original
candidate["modules"].append(entry)
self.assertIn(f"conflicting {kind}", " ".join(selected_artifact_identity_issues(candidate)))
self.assertTrue(candidate_catalog_version_issues(candidate))
candidate = self.candidate()
repeated = deepcopy(candidate["modules"][0])
repeated["module_id"] = "second-demo-projection"
candidate["modules"].append(repeated)
self.assertEqual((), selected_artifact_identity_issues(candidate))
self.assertEqual((), candidate_catalog_version_issues(candidate))
def test_metadata_inspection_uses_the_opened_archive_not_a_replaced_path(self) -> None:
path = self.npm / "original.tgz"
replacement = self.npm / "replacement.tgz"
saved = self.npm / "saved.tgz"
self.tarball(path, "@govoplan/incorrect-webui", "1.2.3")
self.tarball(replacement, "@govoplan/core-webui", "1.2.3")
real_open = tarfile.open
def replace_path(*args, **kwargs):
self.assertIn("fileobj", kwargs)
path.rename(saved)
replacement.rename(path)
return real_open(*args, **kwargs)
with patch("govoplan_release.full_catalog.tarfile.open", side_effect=replace_path):
with self.assertRaisesRegex(ValueError, "metadata differs"):
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
def test_registry_url_provenance_rejects_package_version_and_traversal_mismatch(self) -> None:
self.build()
for old, new in (
("/govoplan-core/1.2.3/", "/govoplan-files/1.2.3/"),
("/govoplan-core/1.2.3/", "/govoplan-core/9.9.9/"),
("/pypi/files/", "/pypi/files/%2e%2e/"),
("/pypi/files/", "/pypi/files/%252e%252e/"),
):
with self.subTest(new=new):
entry = self.candidate()["core_release"]
artifact = entry["artifact_integrity"]["python"]
artifact["url"] = artifact["url"].replace(old, new)
artifact["ref"] = entry["python_ref"] = entry["python_ref"].replace(old, new)
with self.assertRaises(ValueError):
registry_entry_source(entry)
def test_package_set_must_match_fixed_meta_pins_not_just_its_own_hash(self) -> None:
payload = deepcopy(self.package_set)
payload["python"][1]["version"] = "9.9.9"
self.seal(payload, "package_set_sha256")
self.set_path.write_text(json.dumps(payload))
with self.assertRaisesRegex(ValueError, "exact Meta full pins"):
self.build()
self.assertFalse(self.output.exists())
def test_package_set_git_reads_ignore_caller_redirection(self) -> None:
tool = full_catalog._tool("generate-release-package-set")
with patch.dict(os.environ, {"GIT_DIR": "/outside", "GIT_CONFIG_GLOBAL": "/outside/config", "PATH": "/outside/bin"}):
with patch("subprocess.check_output", return_value="a" * 40 + "\n") as execute:
self.assertEqual("a" * 40, tool["_git"](self.root, "rev-parse", "HEAD"))
self.assertEqual("/usr/bin/git", execute.call_args.args[0][0])
self.assertNotIn("GIT_DIR", execute.call_args.kwargs["env"])
self.assertEqual(os.devnull, execute.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"])
self.assertEqual("/usr/bin:/bin", execute.call_args.kwargs["env"]["PATH"])
def test_unchanged_real_annotated_ancestor_is_valid_but_selecting_it_requires_head(self) -> None:
from tests.test_release_source_provenance import git, git_text, make_repo
from govoplan_release.source_provenance import source_tag_provenance_issues
workspace = self.root / "source-workspace"
workspace.mkdir()
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
git(repo, "push", "origin", "refs/tags/v1.2.3")
tagged_commit = git_text(repo, "rev-parse", "HEAD")
(repo / "workflow-only.txt").write_text("post-tag workflow repair\n")
git(repo, "add", "workflow-only.txt")
git(repo, "commit", "-m", "repair workflow without replacing package")
git(repo, "push", "origin", "main")
common = {
"repo_versions": {"govoplan-access": "1.2.3"}, "workspace": workspace,
"expected_commits": {"govoplan-access": tagged_commit},
"expected_tag_objects": {"govoplan-access": git_text(repo, "rev-parse", "refs/tags/v1.2.3")},
}
self.assertEqual((), source_tag_provenance_issues(**common))
issues = source_tag_provenance_issues(**common, require_head_repos=("govoplan-access",))
self.assertIn("not selected HEAD", " ".join(issue.message for issue in issues))
def test_tagged_manifest_synthesis_ignores_local_git_replacement_objects(self) -> None:
from tests.test_release_source_provenance import git, git_text, make_repo
from govoplan_release.catalog_entry_synthesis import materialized_source_tree
workspace = self.root / "materialization-workspace"
workspace.mkdir()
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
original = (repo / "pyproject.toml").read_text()
tagged_commit = git_text(repo, "rev-parse", "HEAD")
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
(repo / "pyproject.toml").write_text(original.replace("1.2.3", "9.9.9"))
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", "unreviewed replacement tree")
git(repo, "replace", tagged_commit, git_text(repo, "rev-parse", "HEAD"))
with patch.dict(os.environ, {"GIT_DIR": str(self.root / "outside"), "PATH": "/outside/bin"}):
with materialized_source_tree(repo, source_ref="v1.2.3") as source:
self.assertEqual(original, (source / "pyproject.toml").read_text())
if __name__ == "__main__":
unittest.main()
+45 -2
View File
@@ -25,6 +25,11 @@ from govoplan_core.core.institutional import (
TemporalRevision,
service_launch_capability,
)
from govoplan_core.core.temporal import (
TemporalDataContext,
bind_temporal_data_context,
reset_temporal_data_context,
)
from govoplan_cases.backend.party_context import CasePartyContext
from govoplan_cases.backend.db.models import (
CaseAccessGrant,
@@ -134,6 +139,13 @@ class _Registry:
class InstitutionalGovernanceJourneyTests(unittest.TestCase):
def setUp(self) -> None:
# Portal's effective_at does not replace the SQL provider's request-local
# read clock. Keep both on the journey date, without bypassing validity
# filtering or extending the fixture's finite publication interval.
token = bind_temporal_data_context(TemporalDataContext(evaluated_at=NOW))
self.addCleanup(reset_temporal_data_context, token)
def test_service_to_formal_outcome_retains_governed_context(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
for table in (
@@ -220,13 +232,44 @@ class InstitutionalGovernanceJourneyTests(unittest.TestCase):
service_launch_capability("case"): object(),
}
)
entry = PortalServiceDirectory(service_registry).list_entries(
directory = PortalServiceDirectory(service_registry)
for outside_interval in (
service.temporal.valid_from - timedelta(microseconds=1),
service.temporal.valid_to,
):
with self.subTest(outside_interval=outside_interval):
token = bind_temporal_data_context(
TemporalDataContext(evaluated_at=outside_interval)
)
try:
# Keep Portal inside the valid interval: the real SQL
# provider must still exclude a service outside its own
# temporal read context, before Portal can project it.
self.assertEqual(
(),
directory.list_entries(
session,
principal,
tenant_id="tenant-1",
effective_at=NOW,
audiences=("resident",),
)[0]
),
)
finally:
reset_temporal_data_context(token)
entries = directory.list_entries(
session,
principal,
tenant_id="tenant-1",
effective_at=NOW,
audiences=("resident",),
)
self.assertEqual(
(service.reference,),
tuple(entry.definition.reference for entry in entries),
)
entry = entries[0]
self.assertTrue(entry.available)
intake = CaseServiceIntake().plan(
entry.definition,
+240
View File
@@ -0,0 +1,240 @@
"""Local mixed-owner admission/recovery evidence, not production capacity certification.
All inputs are synthetic and held in memory. The spawn observer temporarily
holds the admitted parent's handshake so the other owners encounter the same
occupied slot deterministically. Children perform real XLSX, template and
Dataflow work; there is no mocked process execution, database, or live service.
The non-queuing gate promises retryable rejection, not scheduler fairness.
"""
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor
from io import BytesIO
import json
import os
import threading
import time
from types import SimpleNamespace
import unittest
from unittest.mock import patch
try:
from openpyxl import Workbook
from govoplan_connectors.backend.tabular_adapters import (
parse_managed_tabular_content,
)
from govoplan_core.core.templates import TemplateRenderRequest
from govoplan_core.security import bounded_process
from govoplan_core.security.bounded_process import ProcessBudgetError
from govoplan_core.settings import settings
from govoplan_dataflow.backend.backends import execute_typed_graph
from govoplan_dataflow.backend.schemas import (
GraphEdge,
GraphNode,
GraphPosition,
PipelineGraph,
)
from govoplan_templates.backend.rendering import _render_payload
except ImportError as exc:
raise unittest.SkipTest(
"Mixed-owner isolation requires the optional module test environment."
) from exc
class IsolatedWorkCompositionTests(unittest.TestCase):
def setUp(self) -> None:
workbook = Workbook()
workbook.active.append(["name"])
workbook.active.append(["Ada"])
stream = BytesIO()
workbook.save(stream)
workbook.close()
self.workbook = stream.getvalue()
self.graph = PipelineGraph(
nodes=[
GraphNode(
id="source",
type="source.inline",
label="Source",
position=GraphPosition(x=0, y=0),
config={"source_name": "records", "rows": [{"name": "Ada"}]},
),
GraphNode(
id="output",
type="output",
label="Output",
position=GraphPosition(x=100, y=0),
config={},
),
],
edges=[GraphEdge(id="edge", source="source", target="output")],
)
def xlsx(self):
rows, sheet = parse_managed_tabular_content(
self.workbook,
filename="synthetic.xlsx",
content_type=None,
delimiter=",",
sheet_name=None,
)
self.assertEqual(rows, ({"name": "Ada"},))
self.assertEqual(sheet, "Sheet")
return "xlsx"
def templates(self):
payload, content_type, _pages = _render_payload(
SimpleNamespace(name="Synthetic template"),
SimpleNamespace(
content_text="Hello {{item.name}}",
content_html=None,
template_type="letter",
layout={},
output_profiles=[],
),
request=TemplateRenderRequest(
template_id="synthetic", output_format="text"
),
items=({"name": "Ada"},),
)
self.assertEqual(payload, b"Hello Ada")
self.assertEqual(content_type, "text/plain; charset=utf-8")
return "templates"
def dataflow(self):
result = execute_typed_graph(self.graph, backend="reference")
self.assertEqual(result.rows, [{"name": "Ada"}])
return "dataflow"
def test_one_shared_slot_rejects_other_owners_and_all_retries_recover(self):
owners = {
"xlsx": self.xlsx,
"templates": self.templates,
"dataflow": self.dataflow,
}
processes = []
modules = []
hold_next = False
entered = threading.Event()
release = threading.Event()
observer_lock = threading.Lock()
maximum_unreaped = 0
observer_timeouts = 0
original_popen = bounded_process.subprocess.Popen
def observe_spawn(*args, **kwargs):
nonlocal hold_next, maximum_unreaped, observer_timeouts
process = original_popen(*args, **kwargs)
with observer_lock:
processes.append(process)
modules.append(args[0][5])
maximum_unreaped = max(
maximum_unreaped, sum(item.returncode is None for item in processes)
)
should_hold = hold_next
hold_next = False
if should_hold:
entered.set()
if not release.wait(8):
# Return control so the real runner's normal timeout and
# process-group cleanup still own this child on test error.
observer_timeouts += 1
return process
def rejected(operation):
try:
operation()
except Exception as exc:
cause = exc
while cause is not None and not isinstance(cause, ProcessBudgetError):
cause = cause.__cause__
self.assertIsInstance(cause, ProcessBudgetError)
self.assertEqual(cause.code, "busy")
return "busy"
self.fail(
"A different module admitted work while the shared slot was occupied."
)
started = time.monotonic()
busy_count = 0
try:
with (
patch.object(settings, "isolated_process_concurrency", 1),
patch.object(bounded_process.subprocess, "Popen", observe_spawn),
ThreadPoolExecutor(max_workers=3) as executor,
):
for owner, operation in owners.items():
with self.subTest(admitted_owner=owner):
entered.clear()
release.clear()
hold_next = True
holder = executor.submit(operation)
try:
self.assertTrue(
entered.wait(5),
"The admitted operation never spawned its real child.",
)
children_before = len(processes)
others = [
work for label, work in owners.items() if label != owner
]
denied = [
executor.submit(rejected, work) for work in others
]
self.assertEqual(
[future.result(timeout=5) for future in denied],
["busy", "busy"],
)
busy_count += len(denied)
self.assertEqual(len(processes), children_before)
finally:
release.set()
self.assertEqual(holder.result(timeout=15), owner)
self.assertEqual(bounded_process._active, 0)
# Every rejected owner is retried through its real API.
# Each must complete after the previous holder releases.
for other in others:
other()
self.assertEqual(bounded_process._active, 0)
finally:
release.set()
for process in processes:
self.assertIsNotNone(process.returncode, "Worker was not reaped.")
self.assertTrue(
all(
stream.closed
for stream in (process.stdin, process.stdout, process.stderr)
)
)
with self.assertRaises(ChildProcessError):
os.waitpid(process.pid, os.WNOHANG)
self.assertEqual(maximum_unreaped, 1)
self.assertEqual(observer_timeouts, 0)
self.assertEqual(len(processes), 9)
self.assertEqual(busy_count, 6)
self.assertEqual(
set(modules),
{
"govoplan_connectors.backend.tabular_adapters",
"govoplan_templates.backend.rendering",
"govoplan_dataflow.backend.backends.reference",
},
)
print(
json.dumps(
{
"local_composition": {
"successful_children": len(processes),
"busy_rejections": busy_count,
"maximum_unreaped_children": maximum_unreaped,
"all_children_reaped": True,
"seconds": round(time.monotonic() - started, 3),
}
}
)
)
if __name__ == "__main__":
unittest.main()
+18 -3
View File
@@ -62,22 +62,37 @@ class PackageRegistryReleaseTests(unittest.TestCase):
encoding="utf-8"
)
)["project"]["version"]
meta_package = ROOT / "packages/govoplan-meta/pyproject.toml"
meta_project = tomllib.loads(
meta_package.read_text(encoding="utf-8")
)["project"]
expected_tasks_pin = next(
requirement
for requirement in (
*meta_project["dependencies"],
*meta_project["optional-dependencies"]["full"],
)
if requirement.startswith("govoplan-tasks==")
)
selected = PACKAGE_SET.parse_meta_package(
ROOT / "packages/govoplan-meta/pyproject.toml",
meta_package,
core_version=core_version,
)
by_name = {item["name"]: item for item in selected}
self.assertIn("govoplan-core", by_name)
self.assertIn("govoplan-records", by_name)
self.assertEqual("0.1.21", by_name["govoplan-tasks"]["version"])
self.assertEqual(
expected_tasks_pin,
f"govoplan-tasks=={by_name['govoplan-tasks']['version']}",
)
payload = PACKAGE_SET.generate_package_set(
core_version=core_version,
requirements=ROOT / "requirements-release.txt",
workspace=ROOT.parent,
profile="full",
meta_package=ROOT / "packages/govoplan-meta/pyproject.toml",
meta_package=meta_package,
)
self.assertEqual("full", payload["profile"])
self.assertEqual(len(selected), len(payload["python"]))
@@ -96,6 +96,18 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "tracking_issue"):
inventory._load_endpoint_declarations(path)
def test_bounded_workflow_read_apis_have_explicit_headless_declarations(self) -> None:
declarations = inventory._load_endpoint_declarations(inventory.DEFAULT_ENDPOINT_DECLARATIONS)
for path in (
"/workflow/instances/summaries", "/workflow/instances/{}/summary",
"/workflow/instances/{}/steps", "/workflow/instances/{}/events",
):
with self.subTest(path=path):
entry = declarations[("govoplan-workflow-engine", "GET", path)]
self.assertEqual("intentionally_headless", entry["category"])
self.assertIn("current-authorized", entry["rationale"])
self.assertIn("workflow.instance-history", entry["rationale"])
def test_inventory_reports_unclassified_and_stale_endpoint_declarations(
self,
) -> None:
+75
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
from pathlib import Path
import shlex
import subprocess
import sys
import unittest
@@ -17,6 +18,80 @@ from govoplan_release import git_state # noqa: E402
class ReleaseGitStateTests(unittest.TestCase):
def test_unset_ssh_address_family_preserves_original_command_and_operator_config(self) -> None:
environment = git_state.sanitized_git_environment({})
self.assertEqual(
[
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
],
shlex.split(environment["GIT_SSH_COMMAND"]),
)
self.assertNotIn("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY", environment)
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
def test_ssh_address_family_accepts_only_fixed_choices_and_survives_resanitizing(self) -> None:
for family in ("any", "inet", "inet6"):
with self.subTest(family=family):
environment = git_state.sanitized_git_environment({
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": family,
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
"GIT_SSH": "/attacker/ssh",
"PATH": "/attacker/bin",
})
self.assertEqual(
[
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
"-o", f"AddressFamily={family}",
],
shlex.split(environment["GIT_SSH_COMMAND"]),
)
self.assertNotIn("GIT_SSH", environment)
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
def test_invalid_ssh_address_family_is_rejected_before_git_runs(self) -> None:
for invalid in (
"", "INET", "ipv4", " inet", "inet ", "inet\n",
"inet; touch /not-executed", "inet -o StrictHostKeyChecking=no",
"$(not-executed)",
):
with (
self.subTest(value=invalid),
patch.dict("os.environ", {"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": invalid}),
patch.object(git_state.subprocess, "run") as run,
):
with self.assertRaisesRegex(
ValueError, "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6",
):
git_state.git(Path("/workspace/govoplan-core"), "status", "--porcelain")
run.assert_not_called()
def test_source_provenance_readback_keeps_family_but_discards_ssh_command_override(self) -> None:
from govoplan_release.source_provenance import inspect_remote_tag
completed = subprocess.CompletedProcess(
[], 0, f"{'a' * 40}\trefs/tags/v1.2.3\n{'b' * 40}\trefs/tags/v1.2.3^{{}}\n", "",
)
with (
patch.dict("os.environ", {
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": "inet",
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
}),
patch("govoplan_release.repository_tag.subprocess.run", return_value=completed) as run,
):
result = inspect_remote_tag(
path=Path("/workspace/govoplan-core"), remote="origin",
remote_url="git@git.add-ideas.de:GovOPlaN/govoplan-core.git", tag="v1.2.3",
)
self.assertEqual("b" * 40, result.commit)
self.assertEqual(
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8 -o AddressFamily=inet",
run.call_args.kwargs["env"]["GIT_SSH_COMMAND"],
)
def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
from tempfile import TemporaryDirectory
+330
View File
@@ -0,0 +1,330 @@
from __future__ import annotations
from dataclasses import replace
from contextlib import redirect_stdout
import io
import json
from pathlib import Path
import runpy
import shutil
import sys
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import meta_preparation # noqa: E402
from govoplan_release.git_state import collect_repository_snapshot # noqa: E402
from govoplan_release.meta_preparation import ( # noqa: E402
MetaPreparationError,
prepare_developer_meta_package,
)
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
from govoplan_release.version_metadata import ( # noqa: E402
VersionMetadataError,
apply_version_metadata_mutations,
version_metadata_mutations,
)
import test_release_meta_source_tag as meta_fixture # noqa: E402
from test_release_plan_guidance import dashboard # noqa: E402
from test_release_repository_tag import create_release_repo, git, git_text # noqa: E402
class MetaPreparationTests(unittest.TestCase):
synchronize = meta_fixture.MetaSourceTagTests.synchronize
def setUp(self):
meta_fixture.MetaSourceTagTests.setUp(self)
self.operator = self.root / "operator"
self.generator = (
self.operator / "tools/release/generate-developer-meta-package.py"
)
self.generator.parent.mkdir(parents=True)
shutil.copyfile(
ROOT / "tools/release/generate-developer-meta-package.py", self.generator
)
self.enterContext(patch.object(meta_preparation, "META_ROOT", self.operator))
def prepare_core(self):
apply_version_metadata_mutations(self.core, target_version="0.1.11")
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Prepared synthetic Core target")
def preview(self, **kwargs):
return prepare_developer_meta_package(
repo_path=self.meta, target_version="0.1.11", **kwargs
)
def apply(self, preview):
return self.preview(
apply=True, expected_receipt=preview["receipt"], confirm_out_of_run=True
)
def test_full_canonical_preview_apply_and_shared_mutation_discovery(self):
extra, remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-workflow-engine",
version="0.2.3",
)
self.specs.append(
{
"name": extra.name,
"path": extra.name,
"category": "module",
"subtype": "",
"remote": str(remote),
}
)
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.prepare_core()
before = self.package.read_bytes()
preview = self.preview()
self.assertEqual("planned", preview["status"])
self.assertEqual(before, self.package.read_bytes())
mutations = version_metadata_mutations(self.meta, target_version="0.1.11")
self.assertEqual([meta_preparation.PACKAGE], [item.path for item in mutations])
self.assertIn(b"govoplan-workflow-engine==0.2.3", mutations[0].after)
self.assertIn(b"govoplan-core==0.1.11", mutations[0].after)
result = self.apply(preview)
self.assertEqual("prepared", result["status"])
self.assertEqual(mutations[0].after, self.package.read_bytes())
self.assertEqual(
self.render(workspace=self.workspace, requirements=self.requirements),
self.package.read_text(),
)
self.assertEqual(
f"M {meta_preparation.PACKAGE}",
git_text(self.meta, "status", "--porcelain"),
)
self.assertFalse(git_text(self.meta, "tag", "--list"))
def test_core_target_must_already_be_prepared(self):
before = self.package.read_bytes()
with self.assertRaisesRegex(MetaPreparationError, "Prepare and commit Core"):
self.preview()
self.assertEqual(before, self.package.read_bytes())
def test_changed_requirements_receipt_blocks_before_any_output(self):
self.prepare_core()
preview = self.preview()
before = self.package.read_bytes()
self.requirements.write_text(
self.requirements.read_text() + "# reviewed different inputs\n"
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Changed synthetic requirements")
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_source_change_immediately_before_effect_is_rechecked(self):
self.prepare_core()
preview = self.preview()
before = self.package.read_bytes()
original = meta_preparation.preview_meta_mutation
def changed(**kwargs):
result = original(**kwargs)
self.requirements.write_text(
self.requirements.read_text() + "# concurrent change\n"
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Concurrent synthetic change")
return result
with patch.object(
meta_preparation, "preview_meta_mutation", side_effect=changed
):
with self.assertRaisesRegex(MetaPreparationError, "changed before"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_core_full_package_and_operator_generator_are_receipt_bound(self):
self.prepare_core()
for path, repository in (
(self.core / "pyproject.toml", self.core),
(self.access / "pyproject.toml", self.access),
(self.generator, None),
):
with self.subTest(input=path.name, repo=str(repository)):
preview = self.preview()
before = self.package.read_bytes()
path.write_text(path.read_text() + "\n# changed frozen input\n")
if repository is not None:
git(repository, "add", ".")
git(
repository,
"commit",
"-m",
"Changed synthetic composition input",
)
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_post_write_source_change_is_reported_without_retry_or_rollback(self):
from govoplan_release import version_metadata
self.prepare_core()
preview = self.preview()
original = version_metadata._atomic_write
def changed(path, payload):
original(path, payload)
self.requirements.write_text(
self.requirements.read_text() + "# concurrent after write\n"
)
with patch.object(
version_metadata, "_atomic_write", side_effect=changed
) as writer:
with self.assertRaisesRegex(
meta_preparation.MetaPreparationAmbiguous, "write/post-check failed"
):
self.apply(preview)
self.assertEqual(1, writer.call_count)
self.assertIn('version = "0.1.11"', self.package.read_text())
self.assertIn("# concurrent after write", self.requirements.read_text())
def test_write_failure_after_replace_requires_reconciliation(self):
from govoplan_release import version_metadata
self.prepare_core()
preview = self.preview()
original = version_metadata._atomic_write
def partial(path, payload):
original(path, payload)
raise OSError("Synthetic directory fsync failure after replacement")
with patch.object(version_metadata, "_atomic_write", side_effect=partial) as writer:
with self.assertRaisesRegex(meta_preparation.MetaPreparationAmbiguous, "may have been written"):
self.apply(preview)
self.assertEqual(1, writer.call_count)
self.assertIn('version = "0.1.11"', self.package.read_text())
def test_cli_requires_reviewed_receipt_and_explicit_out_of_run_confirmation(self):
self.prepare_core()
main = runpy.run_path(
str(ROOT / "tools/release/prepare-developer-meta-package.py")
)["main"]
arguments = [
"prepare-developer-meta-package.py",
"--workspace",
str(self.workspace),
"--target-version",
"0.1.11",
]
output = io.StringIO()
with patch.object(sys, "argv", arguments), redirect_stdout(output):
self.assertEqual(0, main())
preview = self.root / "meta-preview.json"
preview.write_text(output.getvalue())
with (
patch.object(sys, "argv", [*arguments, "--apply"]),
redirect_stdout(io.StringIO()),
):
self.assertEqual(1, main())
with (
patch.object(
sys,
"argv",
[
*arguments,
"--apply",
"--receipt",
str(preview),
"--confirm-out-of-run",
],
),
redirect_stdout(io.StringIO()),
):
self.assertEqual(0, main())
def test_unknown_full_input_and_unsafe_operator_tooling_fail_closed(self):
self.prepare_core()
unknown = self.workspace / "govoplan-unknown/pyproject.toml"
unknown.parent.mkdir()
unknown.write_text('[project]\nname="govoplan-unknown"\nversion="1.0.0"\n')
with self.assertRaisesRegex(MetaPreparationError, "unregistered"):
self.preview()
unknown.unlink()
self.generator.chmod(0o666)
with self.assertRaisesRegex(MetaPreparationError, "owned, bounded regular"):
self.preview()
def test_wrong_nested_identity_and_existing_immutable_tag_fail_closed(self):
self.prepare_core()
original = self.package.read_text()
self.package.write_text(
original.replace('name = "govoplan"', 'name = "not-govoplan"')
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Wrong synthetic package identity")
with self.assertRaisesRegex(MetaPreparationError, "identity"):
self.preview()
self.package.write_text(original)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Restore synthetic package identity")
git(self.meta, "tag", "-a", "v0.1.11", "-m", "Immutable target")
with self.assertRaisesRegex(MetaPreparationError, "target Meta tag"):
self.preview()
def test_no_generic_durable_self_mutation_or_running_tooling_target(self):
self.prepare_core()
preview = self.preview()
with self.assertRaisesRegex(VersionMetadataError, "outside durable runs"):
apply_version_metadata_mutations(self.meta, target_version="0.1.11")
with self.assertRaisesRegex(MetaPreparationError, "confirm"):
self.preview(apply=True, expected_receipt=preview["receipt"])
with patch.object(meta_preparation, "META_ROOT", self.meta):
with self.assertRaisesRegex(MetaPreparationError, "running operator"):
self.preview()
def test_next_version_plan_is_actionable_core_first_without_meta_executor(self):
snapshots = tuple(
collect_repository_snapshot(
RepositorySpec(**spec),
workspace_root=self.workspace,
target_tag="v0.1.11",
)
for spec in self.specs[:2]
)
source = replace(
dashboard(workspace=self.workspace, version=self.version),
repositories=snapshots,
)
plan = build_selective_release_plan(
source,
selected_repos=("govoplan", "govoplan-core"),
target_version="0.1.11",
)
self.assertEqual(
["govoplan-core", "govoplan"], [unit.repo for unit in plan.units]
)
findings = [
finding for finding in plan.gate_findings if finding.repo == "govoplan"
]
self.assertEqual(
["developer_meta_core_preparation_required"],
[finding.code for finding in findings],
)
self.assertIn("prepare-developer-meta-package.py", findings[0].remediation)
meta_steps = [step for step in plan.dry_run_steps if step.repo == "govoplan"]
self.assertEqual(
["govoplan:prepare-support", "govoplan:publish-support"],
[step.id for step in meta_steps],
)
self.assertTrue(all(step.status == "needs-executor" for step in meta_steps))
self.prepare_core()
prepared = build_selective_release_plan(
source, selected_repos=("govoplan",), target_version="0.1.11"
)
self.assertEqual("developer_meta_out_of_run", prepared.gate_findings[0].code)
if __name__ == "__main__":
unittest.main()
+569
View File
@@ -0,0 +1,569 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import runpy
import shutil
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import source_tag_batch as meta_source_tag, workspace # noqa: E402
from govoplan_release.git_state import collect_versions # noqa: E402
from govoplan_release.model import RepositorySnapshot, RepositorySpec, VersionSnapshot # noqa: E402
from govoplan_release.repository_tag import tag_repositories # noqa: E402
from govoplan_release.selective_planner import build_unit # noqa: E402
from govoplan_release.version_alignment import repository_version_issues # noqa: E402
from test_release_repository_tag import ( # noqa: E402
add_scoped_workflow_manifest,
create_release_repo,
git,
git_text,
ref_exists,
)
class MetaSourceTagTests(unittest.TestCase):
def setUp(self):
self.temporary = self.enterContext(
tempfile.TemporaryDirectory(prefix="meta-release-tests-")
)
self.root = Path(self.temporary)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
self.version = "0.1.10"
self.core, self.core_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-core",
version=self.version,
)
self.access, self.access_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-access",
version=self.version,
)
add_scoped_workflow_manifest(self.access)
self.meta = self.workspace / "govoplan"
self.meta_remote = self.root / "govoplan.git"
git(self.root, "init", "--bare", str(self.meta_remote))
git(self.workspace, "init", "-b", "main", str(self.meta))
git(self.meta, "config", "user.name", "Meta Release Fixture")
git(self.meta, "config", "user.email", "release@example.invalid")
self.package = self.meta / "packages/govoplan-meta/pyproject.toml"
self.package.parent.mkdir(parents=True)
self.requirements = self.meta / "requirements-release.txt"
self.requirements.write_text(
"../govoplan-core\ngovoplan-access @ git+ssh://git@example.invalid/GovOPlaN/govoplan-access.git@v0.1.10\n"
)
self.render = runpy.run_path(
str(ROOT / "tools/release/generate-developer-meta-package.py")
)["render"]
self.synchronize()
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Nested developer package")
git(self.meta, "remote", "add", "origin", str(self.meta_remote))
git(self.meta, "push", "-u", "origin", "main")
self.specs = [
{
"name": name,
"category": "system" if subtype else "module",
"subtype": subtype,
"path": name,
"remote": str(remote),
}
for name, subtype, remote in (
("govoplan", "meta", self.meta_remote),
("govoplan-core", "kernel", self.core_remote),
("govoplan-access", "", self.access_remote),
)
]
self.registry = self.root / "repositories.json"
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.enterContext(patch.object(workspace, "REPOSITORIES_FILE", self.registry))
def synchronize(self):
self.package.write_text(
self.render(workspace=self.workspace, requirements=self.requirements)
)
def commit_meta(self):
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Changed synthetic metadata")
def tag(
self,
*,
repos=("govoplan", "govoplan-core"),
apply=False,
push=False,
**overrides,
):
return tag_repositories(
repos=repos,
repo_versions={repo: self.version for repo in repos},
workspace_root=self.workspace,
apply=apply,
push=push,
**overrides,
)
def assert_no_tags(self):
for repo in (
self.meta,
self.meta_remote,
self.core,
self.core_remote,
self.access,
self.access_remote,
):
self.assertFalse(ref_exists(repo, "refs/tags/v0.1.10"), str(repo))
def test_explicit_nested_version_collection_and_alignment_without_root_package(
self,
):
versions = collect_versions(self.meta)
self.assertIsNone(versions.pyproject)
self.assertEqual(self.version, versions.developer_meta)
self.assertEqual(self.version, versions.primary)
self.assertFalse((self.meta / "pyproject.toml").exists())
self.assertEqual(
(), repository_version_issues(self.meta, expected_version=self.version)
)
mismatch = repository_version_issues(self.meta, expected_version="0.1.11")
self.assertTrue(
any(
issue.source == "packages/govoplan-meta/pyproject.toml"
for issue in mismatch
)
)
def test_planner_and_console_display_the_explicit_nested_version(self):
snapshot = RepositorySnapshot(
spec=RepositorySpec(**self.specs[0]),
absolute_path=str(self.meta),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(developer_meta=self.version),
)
unit = build_unit(snapshot, target_version=None, contracts=None)
self.assertEqual(self.version, unit.current_version)
self.assertEqual(self.version, unit.target_version)
html = (ROOT / "tools/release/webui/index.html").read_text()
self.assertIn(
"if (versions.developer_meta) return versions.developer_meta;", html
)
drift = RepositorySnapshot(
spec=snapshot.spec,
absolute_path=str(self.meta),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject="0.1.9", developer_meta=self.version),
)
self.assertTrue(
any(
"version metadata is not aligned" in item
for item in build_unit(
drift, target_version=self.version, contracts=None
).blockers
)
)
def test_unknown_nested_package_and_missing_or_wrong_meta_identity_fail_closed(
self,
):
unknown = self.workspace / "unknown"
nested = unknown / "packages/govoplan-meta/pyproject.toml"
nested.parent.mkdir(parents=True)
nested.write_text(self.package.read_text())
self.assertIsNone(collect_versions(unknown).primary)
self.assertIn(
"no version metadata",
repository_version_issues(unknown, expected_version=self.version)[
0
].message,
)
for value in ("", '[project]\nname="not-govoplan"\nversion="0.1.10"\n'):
with self.subTest(value=value):
self.package.write_text(value)
self.assertTrue(
repository_version_issues(self.meta, expected_version=self.version)
)
def test_preview_local_tag_and_publish_share_complete_nested_contract(self):
preview = self.tag(push=True)
self.assertEqual("planned", preview["status"], preview)
self.assertEqual(
["govoplan-core", "govoplan"],
[row["repo"] for row in preview["repositories"]],
)
self.assertEqual("registered-meta-batch-v1", preview["source_contract"])
self.assert_no_tags()
local = self.tag(apply=True)
self.assertEqual("tagged", local["status"], local)
for repo in (self.core, self.meta):
self.assertEqual(
"tag", git_text(repo, "cat-file", "-t", "refs/tags/v0.1.10")
)
self.assertFalse(ref_exists(self.meta_remote, "refs/tags/v0.1.10"))
published = self.tag(apply=True, push=True)
self.assertEqual("published", published["status"], published)
for repo, remote in (
(self.core, self.core_remote),
(self.meta, self.meta_remote),
):
self.assertEqual(
git_text(repo, "rev-parse", "HEAD"),
git_text(remote, "rev-parse", "refs/heads/main"),
)
self.assertEqual(
git_text(repo, "rev-parse", "refs/tags/v0.1.10"),
git_text(remote, "rev-parse", "refs/tags/v0.1.10"),
)
again = self.tag(apply=True, push=True)
self.assertEqual("published", again["status"], again)
def test_stale_composition_blocks_whole_batch_before_local_tag_or_push(self):
self.package.write_text(
self.package.read_text().replace(
"govoplan-access==0.1.10", "govoplan-access==0.1.9"
)
)
self.commit_meta()
for apply, push in ((False, False), (True, False), (True, True)):
result = self.tag(
repos=("govoplan-core", "govoplan-access", "govoplan"),
apply=apply,
push=push,
)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
def test_core_outside_batch_requires_matching_existing_and_published_tag(self):
self.assertEqual("blocked", self.tag(repos=("govoplan",))["status"])
git(self.core, "tag", "-a", "v0.1.10", "-m", "Core release")
self.assertEqual("planned", self.tag(repos=("govoplan",))["status"])
self.assertEqual("blocked", self.tag(repos=("govoplan",), push=True)["status"])
git(self.core, "push", "origin", "refs/tags/v0.1.10")
self.assertEqual("planned", self.tag(repos=("govoplan",), push=True)["status"])
def test_changed_core_version_and_explicit_selected_version_mismatch_block(self):
mismatch = tag_repositories(
repos=("govoplan", "govoplan-core"),
repo_versions={"govoplan": self.version, "govoplan-core": "0.1.11"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual("blocked", mismatch["status"], mismatch)
(self.core / "pyproject.toml").write_text(
'[project]\nname="govoplan-core"\nversion="0.1.11"\n'
)
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Core new version")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
def test_unsafe_origin_on_any_selected_repo_blocks_every_effect(self):
for repo in (self.meta, self.core, self.access):
with self.subTest(repo=repo.name):
git(
repo,
"config",
"remote.origin.pushurl",
str(self.root / "unregistered.git"),
)
result = self.tag(
repos=("govoplan-core", "govoplan-access", "govoplan"),
apply=True,
push=True,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("registered origin", result["repositories"][0]["detail"])
self.assert_no_tags()
git(repo, "config", "--unset", "remote.origin.pushurl")
def test_world_writable_nonsticky_parent_blocks_without_changing_permissions(self):
original = self.root.stat().st_mode & 0o7777
self.root.chmod(0o777)
try:
for apply in (False, True):
result = self.tag(apply=apply, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"group/world writable", result["repositories"][0]["detail"]
)
self.assertEqual(0o777, self.root.stat().st_mode & 0o7777)
self.assert_no_tags()
finally:
self.root.chmod(original)
def test_wrong_metadata_owner_blocks_before_remote_lookup(self):
config = self.meta / ".git/config"
original = Path.lstat
def wrong_owner(path, *args, **kwargs):
observed = original(path, *args, **kwargs)
if path == config:
fields = list(observed)
fields[4] = os.geteuid() + 1
return os.stat_result(fields)
return observed
with (
patch.object(Path, "lstat", new=wrong_owner),
patch.object(
meta_source_tag,
"registered_source_origin_issues",
side_effect=AssertionError("must validate ownership before Git"),
),
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("current operator", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_hidden_index_flags_cannot_disguise_modified_release_metadata(self):
for flag, undo in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
for repo, relative in (
(self.meta, "packages/govoplan-meta/pyproject.toml"),
(self.core, "pyproject.toml"),
):
with self.subTest(flag=flag, repo=repo.name):
target = repo / relative
original = target.read_text()
git(repo, "update-index", flag, relative)
target.write_text(
original
+ "\n# Hidden working-tree input differs from frozen HEAD\n"
)
try:
self.assertEqual("", git_text(repo, "status", "--porcelain"))
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"index entries", result["repositories"][0]["detail"]
)
self.assert_no_tags()
finally:
target.write_text(original)
git(repo, "update-index", undo, relative)
def test_read_only_git_target_is_not_repaired_or_tagged(self):
metadata = self.meta / ".git"
original = metadata.stat().st_mode & 0o7777
metadata.chmod(0o500)
try:
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertEqual(0o500, metadata.stat().st_mode & 0o7777)
self.assert_no_tags()
finally:
metadata.chmod(original)
def test_git_object_alternates_are_rejected_before_remote_lookup(self):
(self.meta / ".git/objects/info/alternates").write_text(
str(self.root / "outside-objects") + "\n"
)
with patch.object(
meta_source_tag,
"registered_source_origin_issues",
side_effect=AssertionError("must reject alternates before Git"),
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("alternates", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_owned_worktree_metadata_inside_private_workspace_is_supported(self):
main_checkout = self.workspace / "meta-main-storage"
self.meta.rename(main_checkout)
git(main_checkout, "worktree", "add", "--force", str(self.meta), "main")
self.assertTrue((self.meta / ".git").is_file())
result = self.tag(apply=True)
self.assertEqual("tagged", result["status"], result)
filesystem = result["source_receipts"]["govoplan"]["filesystem"]
self.assertEqual(
str(main_checkout / ".git"), filesystem["git_common_directory"][0]
)
def test_git_directory_replacement_with_same_head_changes_frozen_receipt(self):
preview = meta_source_tag._preview_repositories
def swapped_git_directory(**kwargs):
result = preview(**kwargs)
original = self.meta / ".git"
backup = self.root / "original-meta-git"
original.rename(backup)
shutil.copytree(backup, original)
return result
with patch.object(
meta_source_tag,
"_preview_repositories",
side_effect=swapped_git_directory,
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_non_main_and_divergent_live_main_fail_even_with_stale_tracking(self):
git(self.meta, "switch", "-c", "feature")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.meta, "switch", "main")
clone = self.root / "other-writer"
git(self.root, "clone", "--branch", "main", str(self.meta_remote), str(clone))
git(clone, "config", "user.name", "Other synthetic writer")
git(clone, "config", "user.email", "other@example.invalid")
(clone / "other.txt").write_text("remote divergence\n")
git(clone, "add", ".")
git(clone, "commit", "-m", "Remote main advanced")
git(clone, "push", "origin", "main")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("live origin/main", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_symlink_checkout_is_not_a_registered_source(self):
original = self.workspace / "moved-meta"
self.meta.rename(original)
self.meta.symlink_to(original, target_is_directory=True)
result = self.tag(apply=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("symlink", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_lightweight_and_conflicting_annotated_tags_block(self):
git(self.meta, "tag", "v0.1.10")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.meta, "tag", "-d", "v0.1.10")
git(self.meta, "tag", "-a", "v0.1.10", "-m", "First annotation")
git(self.meta, "push", "origin", "refs/tags/v0.1.10")
git(self.meta, "tag", "-d", "v0.1.10")
git(self.meta, "tag", "-a", "v0.1.10", "-m", "Different annotation")
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
def test_meta_source_receipt_changed_after_preflight_blocks_before_first_effect(
self,
):
preview = meta_source_tag._preview_repositories
def changed_after_preflight(**kwargs):
result = preview(**kwargs)
(self.meta / "new-review.txt").write_text("changed after preflight\n")
self.commit_meta()
return result
with patch.object(
meta_source_tag,
"_preview_repositories",
side_effect=changed_after_preflight,
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_fabricated_push_success_without_remote_receipt_fails_and_stops_batch(self):
original = meta_source_tag.run
def run(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
return subprocess.CompletedProcess(command, 0, "", "")
return original(command, **kwargs)
with patch.object(meta_source_tag, "run", side_effect=run):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertEqual("failed", result["repositories"][0]["status"])
self.assertEqual("skipped", result["repositories"][1]["status"])
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
def test_remote_tag_without_expected_main_receipt_is_not_success(self):
(self.core / "reviewed-change.txt").write_text("release source changes\n")
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Advance reviewed Core source")
original = meta_source_tag.run
pushes = []
def tag_only(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
pushes.append(command)
# Simulate a defective transport that claims atomic success,
# while publishing only the exact expected annotation object.
return original(("git", "push", "origin", command[-1]), **kwargs)
return original(command, **kwargs)
with patch.object(meta_source_tag, "run", side_effect=tag_only):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertEqual(1, len(pushes))
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assertEqual(
git_text(self.core, "rev-parse", "refs/tags/v0.1.10"),
git_text(self.core_remote, "rev-parse", "refs/tags/v0.1.10"),
)
self.assertNotEqual(
git_text(self.core, "rev-parse", "HEAD"),
git_text(self.core_remote, "rev-parse", "refs/heads/main"),
)
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
def test_meta_receipt_is_rechecked_after_an_earlier_successful_publication(self):
original = meta_source_tag.run
def changed_after_core(command, **kwargs):
result = original(command, **kwargs)
if (
command[:3] == ("git", "push", "--atomic")
and kwargs["cwd"] == self.core
):
(self.meta / "changed-review.txt").write_text(
"new Meta source after Core publication\n"
)
self.commit_meta()
return result
with patch.object(meta_source_tag, "run", side_effect=changed_after_core):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertTrue(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
self.assertEqual("skipped", result["repositories"][1]["status"])
def test_unknown_selected_repository_cannot_use_meta_support_exception(self):
result = self.tag(repos=("govoplan", "unknown"), apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("not registered", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_selected_checkout_generator_is_never_executed(self):
malicious = self.meta / "tools/release/generate-developer-meta-package.py"
malicious.parent.mkdir(parents=True)
malicious.write_text(
'raise RuntimeError("selected checkout must not execute")\n'
)
self.commit_meta()
self.assertEqual("planned", self.tag()["status"])
if __name__ == "__main__":
unittest.main()
+146
View File
@@ -65,6 +65,8 @@ branch_labels: Union[str, Sequence[str], None] = None
)
wrapper = development / release.name
wrapper.write_text(
"from importlib import import_module\n"
'_migration = import_module("govoplan_core.backend.migrations.versions.1234_example")\n'
"revision = _migration.revision\n"
"down_revision = _migration.down_revision\n"
"depends_on = _migration.depends_on\n"
@@ -79,6 +81,150 @@ branch_labels: Union[str, Sequence[str], None] = None
self.assertEqual(("base",), migration.down_revisions)
self.assertEqual(("core",), migration.depends_on)
def test_literal_wrapper_alias_and_different_filename_are_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_v019_example.py").write_text(
'revision = "1234"\ndown_revision = "base"\ndepends_on = "core"\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
for alias in ("_migration", "edit_revision", "message_actions"):
with self.subTest(alias=alias):
wrapper.write_text(
"from importlib import import_module as load_migration\n"
f'{alias} = load_migration("govoplan_campaign.backend.migrations.versions." "1234_v019_example")\n'
f"revision = {alias}.revision\ndown_revision = {alias}.down_revision\n"
f"depends_on = {alias}.depends_on\nbranch_labels = {alias}.branch_labels\n"
'raise AssertionError("Wrapper must not execute")\n',
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-campaign", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ("base",))
self.assertEqual(migration.depends_on, ("core",))
def test_core_literal_sibling_file_wrapper_is_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text(
'revision = "1234"\ndown_revision = None\ndepends_on = None\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
wrapper.write_text(
"from importlib.util import module_from_spec, spec_from_file_location\n"
"from pathlib import Path\n"
'_path = Path(__file__).resolve().parents[1] / "versions" / "1234_example.py"\n'
'_spec = spec_from_file_location("synthetic_migration", _path)\n'
"_module = module_from_spec(_spec)\n_spec.loader.exec_module(_module)\n"
"revision = _module.revision\ndown_revision = _module.down_revision\n"
"depends_on = _module.depends_on\nbranch_labels = _module.branch_labels\n",
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-core", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ())
def test_wrapper_rejects_dynamic_foreign_missing_or_rebound_targets(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text('revision = "1234"\n', encoding="utf-8")
wrapper = root / "dev_versions/1234_example.py"
valid = '_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
definitions = (
'_migration = import_module(module_name)\n',
'_migration = import_module("govoplan_mail.backend.migrations.versions.1234_example")\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions...other.1234_example")\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions.missing")\n',
valid + "_migration = another_module\n",
"import_module = another_loader\n" + valid,
"def import_module(value):\n return another_module\n" + valid,
"import another_loader as import_module\n" + valid,
valid + "class _migration:\n revision = 'another'\n",
"if condition:\n _migration = another_module\n" + valid,
valid + "del _migration\n",
)
for definition in definitions:
with self.subTest(definition=definition):
wrapper.write_text(
"from importlib import import_module\n" + definition + "revision = _migration.revision\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
audit.parse_migration_file("govoplan-campaign", wrapper)
def test_wrapper_rejects_symlink_and_mixed_release_metadata(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text('revision = "1234"\ndown_revision = None\n', encoding="utf-8")
(root / "versions/5678_example.py").write_text('revision = "5678"\ndown_revision = None\n', encoding="utf-8")
(root / "versions/linked.py").symlink_to(root / "versions/1234_example.py")
wrapper = root / "dev_versions/1234_example.py"
definitions = (
'_migration = import_module("govoplan_campaign.backend.migrations.versions.linked")\nrevision = _migration.revision\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
'_other = import_module("govoplan_campaign.backend.migrations.versions.5678_example")\n'
'revision = _migration.revision\ndown_revision = _other.down_revision\n',
)
for definition in definitions:
with self.subTest(definition=definition):
wrapper.write_text("from importlib import import_module\n" + definition, encoding="utf-8")
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
audit.parse_migration_file("govoplan-campaign", wrapper)
def test_unresolved_revision_expression_is_not_silently_omitted(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
path = Path(directory) / "1234_example.py"
path.write_text("revision = calculate_revision()\n", encoding="utf-8")
with self.assertRaisesRegex(ValueError, "Unsupported or ambiguous migration metadata"):
audit.parse_migration_file("govoplan-core", path)
def test_explicit_metadata_reexport_is_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/a234_example.py").write_text(
'revision = "1234"\ndown_revision = "base"\ndepends_on = None\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
source = "govoplan_organizations.backend.migrations.versions.a234_example"
wrapper.write_text(f"from {source} import revision, down_revision, depends_on, branch_labels\n", encoding="utf-8")
migration = audit.parse_migration_file("govoplan-organizations", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ("base",))
for declaration in (
f"from {source} import *\n",
f"from {source} import revision as down_revision\n",
f"from {source} import revision\nrevision = 'different'\n",
f"from {source} import revision\nimport another as revision\n",
f"from {source} import revision\ndef revision():\n pass\n",
f"from {source.replace('govoplan_organizations', 'govoplan_mail')} import revision\n",
):
with self.subTest(declaration=declaration):
wrapper.write_text(declaration, encoding="utf-8")
with self.assertRaises(ValueError):
audit.parse_migration_file("govoplan-organizations", wrapper)
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
audit = load_audit_module()
migrations = [
+239 -1
View File
@@ -1,10 +1,12 @@
from __future__ import annotations
import json
import runpy
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from pathlib import Path
from fastapi.testclient import TestClient
@@ -16,6 +18,7 @@ if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.repository_tag import tag_repositories # noqa: E402
from govoplan_release import workspace as release_workspace # noqa: E402
from server.app import create_app # noqa: E402
@@ -38,6 +41,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
version="0.1.10",
)
add_scoped_workflow_manifest(self.manifest_repo)
# The operator's test catalog explicitly registers known synthetic
# endpoints; production trust checks are not patched or bypassed.
self.registry = self.root / "registered-test-repositories.json"
self.registered = json.loads((META_ROOT / "repositories.json").read_text())
for spec in self.registered["repositories"]:
spec["remote"] = str(self.root / f"{spec['name']}.git")
self.registry.write_text(json.dumps(self.registered))
self.enterContext(patch.object(release_workspace, "REPOSITORIES_FILE", self.registry))
def tearDown(self) -> None:
self.temporary.cleanup()
@@ -127,6 +138,10 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
name="govoplan-core",
version="0.1.10",
)
for spec in self.registered["repositories"]:
if spec["name"] == "govoplan-core":
spec["remote"] = str(remote_root / "govoplan-core.git")
self.registry.write_text(json.dumps(self.registered))
result = tag_repositories(
repos=("govoplan-core",),
@@ -214,12 +229,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
git(self.repo, "commit", "-m", "Add release WebUI composition")
git(self.repo, "push", "origin", "main")
for push in (False, True):
with self.subTest(push=push):
result = tag_repositories(
repos=("govoplan-core", "govoplan-campaign"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
push=push,
)
self.assertEqual("blocked", result["status"])
@@ -300,6 +317,183 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
self.assertIn("Signed Website Catalog", ui.text)
self.assertIn("Apply + Website Tag", ui.text)
def test_local_module_candidate_precedes_core_lock_but_publication_does_not(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
remotes = (self.remote, self.manifest_remote, campaign_remote)
remote_refs = {path: git_text(path, "show-ref") for path in remotes}
arguments = {
"repos": ("govoplan-campaign",),
"repo_versions": {"govoplan-campaign": "0.1.10"},
"workspace_root": self.workspace,
}
preview = tag_repositories(**arguments, apply=False, push=False)
self.assertEqual("planned", preview["status"], preview)
self.assertFalse(ref_exists(campaign, "refs/tags/v0.1.10"))
candidate = tag_repositories(**arguments, apply=True, push=False)
self.assertEqual("tagged", candidate["status"], candidate)
self.assertEqual("tag", git_text(campaign, "cat-file", "-t", "v0.1.10"))
tag_object = git_text(campaign, "rev-parse", "v0.1.10")
head = git_text(campaign, "rev-parse", "HEAD")
self.assertEqual(head, git_text(campaign, "rev-parse", "v0.1.10^{commit}"))
for apply in (False, True):
with self.subTest(publish_apply=apply):
blocked = tag_repositories(**arguments, apply=apply, push=True)
self.assertEqual("blocked", blocked["status"], blocked)
self.assertIn(
"release WebUI composition gate failed",
blocked["repositories"][0]["detail"],
)
self.assertEqual(tag_object, git_text(campaign, "rev-parse", "v0.1.10"))
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
lock_path = self.repo / "webui" / "package-lock.release.json"
lock = json.loads(lock_path.read_text(encoding="utf-8"))
locked_campaign = lock["packages"]["node_modules/@govoplan/campaign-webui"]
locked_campaign["version"] = "0.1.10"
locked_campaign["resolved"] = f"git+ssh://git@example.test/acme/govoplan-campaign.git#{head}"
lock_path.write_text(json.dumps(lock) + "\n", encoding="utf-8")
git(self.repo, "add", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Resolve reviewed local Campaign candidate")
core_candidate = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("tagged", core_candidate["status"], core_candidate)
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
published = tag_repositories(**arguments, apply=True, push=True)
self.assertEqual("published", published["status"], published)
self.assertEqual(tag_object, git_text(campaign_remote, "rev-parse", "v0.1.10"))
self.assertEqual(head, git_text(campaign_remote, "rev-parse", "refs/heads/main"))
self.assertEqual(remote_refs[self.remote], git_text(self.remote, "show-ref"))
def test_local_core_candidate_still_requires_resolved_module_tags(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
for selected in (("govoplan-core",), ("govoplan-campaign", "govoplan-core")):
with self.subTest(selected=selected):
result = tag_repositories(
repos=selected,
repo_versions={repo: "0.1.10" for repo in selected},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
core_row = next(row for row in result["repositories"] if row["repo"] == "govoplan-core")
self.assertIn("version alignment gate failed", core_row["detail"])
self.assertIn("@govoplan/campaign-webui:resolved", core_row["detail"])
self.assertIn("expected 'local tag v0.1.10'", core_row["detail"])
for repository in (self.repo, self.remote, campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_local_module_candidate_preserves_version_and_worktree_gates(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
arguments = {
"repos": ("govoplan-campaign",),
"workspace_root": self.workspace,
"apply": True,
"push": False,
}
mismatch = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.11"})
self.assertEqual("blocked", mismatch["status"], mismatch)
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
(campaign / "unreviewed.txt").write_text("operator work\n", encoding="utf-8")
dirty = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.10"})
self.assertEqual("blocked", dirty["status"], dirty)
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
for repository in (campaign, campaign_remote):
for tag in ("v0.1.10", "v0.1.11"):
self.assertFalse(ref_exists(repository, f"refs/tags/{tag}"))
def test_local_module_candidate_preserves_manifest_gate(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
replace_with_unscoped_workflow_manifest(self.manifest_repo)
result = tag_repositories(
repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
for repository in (campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_local_module_candidate_preserves_remote_tag_immutability(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
git(campaign, "tag", "-a", "v0.1.10", "-m", "Existing immutable tag", "v0.1.9^{commit}")
git(campaign, "push", "origin", "refs/tags/v0.1.10")
remote_refs = git_text(campaign_remote, "show-ref")
for local_exists in (True, False):
with self.subTest(local_exists=local_exists):
if not local_exists:
git(campaign, "tag", "-d", "v0.1.10")
result = tag_repositories(
repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("immutable tag", result["repositories"][0]["detail"])
self.assertIn("not HEAD", result["repositories"][0]["detail"])
self.assertEqual(remote_refs, git_text(campaign_remote, "show-ref"))
self.assertEqual(local_exists, ref_exists(campaign, "refs/tags/v0.1.10"))
def _staged_campaign_bundle(self) -> tuple[Path, Path]:
campaign, remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-campaign",
version="0.1.9",
)
campaign_webui = campaign / "webui"
campaign_webui.mkdir()
package_path = campaign_webui / "package.json"
package_path.write_text(
'{"name":"@govoplan/campaign-webui","version":"0.1.9"}\n', encoding="utf-8"
)
git(campaign, "add", "webui/package.json")
git(campaign, "commit", "-m", "Prior Campaign WebUI package")
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
prior_commit = git_text(campaign, "rev-parse", "HEAD")
for path in (campaign / "pyproject.toml", package_path):
path.write_text(path.read_text(encoding="utf-8").replace("0.1.9", "0.1.10"), encoding="utf-8")
git(campaign, "add", "pyproject.toml", "webui/package.json")
git(campaign, "commit", "-m", "Reviewed Campaign candidate")
core_webui = self.repo / "webui"
core_webui.mkdir()
dependency_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.10"
package = {
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": dependency_ref},
}
(core_webui / "package.release.json").write_text(json.dumps(package) + "\n", encoding="utf-8")
(core_webui / "package-lock.release.json").write_text(
json.dumps({"packages": {
"": package,
"node_modules/@govoplan/campaign-webui": {
"version": "0.1.9",
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{prior_commit}",
},
}}) + "\n",
encoding="utf-8",
)
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Stage Core input before candidate lock resolution")
return campaign, remote
def git(cwd: Path, *args: str) -> None:
result = subprocess.run(
@@ -338,9 +532,50 @@ def add_scoped_workflow_manifest(repo: Path) -> None:
backend.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
(backend / "__init__.py").write_text("", encoding="utf-8")
# This small workspace still has to satisfy the real presentation contract.
# Keep that prerequisite shared by both valid and intentionally unscoped
# documentation fixtures, so each test reaches its intended release gate.
canonical_areas = runpy.run_path(
str(META_ROOT / "tools" / "checks" / "check-manifest-shapes.py")
)["CANONICAL_PRODUCT_AREAS"]
(backend / "release_fixture.py").write_text(
"""from govoplan_core.core.modules import FrontendModule, ProductAreaContribution
from govoplan_core.core.views import ViewSurface
def fixture_frontend():
return FrontendModule(
module_id="access",
view_surfaces=(
ViewSurface(
id="access.section.release-fixture",
module_id="access",
kind="section",
label="Release fixture",
),
),
product_areas=tuple(
ProductAreaContribution(
id=area_id,
module_id="access",
label=label,
icon=icon,
description=description,
order=order,
surface_ids=("access.section.release-fixture",),
)
for area_id, (label, icon, description, order) in CANONICAL_AREAS.items()
),
)
CANONICAL_AREAS = """ + repr(canonical_areas) + "\n",
encoding="utf-8",
)
(backend / "manifest.py").write_text(
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
from govoplan_core.core.provider_governance import declared_module_architecture
from .release_fixture import fixture_frontend
def get_manifest():
@@ -348,6 +583,7 @@ def get_manifest():
id="access",
name="Access",
version="0.1.10",
frontend=fixture_frontend(),
permissions=(
PermissionDefinition(
scope="access:item:read",
@@ -403,6 +639,7 @@ def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
manifest.write_text(
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
from govoplan_core.core.provider_governance import declared_module_architecture
from .release_fixture import fixture_frontend
def get_manifest():
@@ -410,6 +647,7 @@ def get_manifest():
id="access",
name="Access",
version="0.1.10",
frontend=fixture_frontend(),
documentation=(
DocumentationTopic(
id="access.workflow.unscoped",
+1 -1
View File
@@ -1891,7 +1891,7 @@ class ReleaseRunApiTests(unittest.TestCase):
def test_ui_and_runbook_state_tracking_boundary_are_explicit(self) -> None:
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
runbook = (META_ROOT / "docs" / "RELEASE_CONSOLE.md").read_text(
runbook = (META_ROOT / "docs" / "operations" / "RELEASE_CONSOLE.md").read_text(
encoding="utf-8"
)
+318
View File
@@ -0,0 +1,318 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import subprocess
import sys
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import source_tag_batch # noqa: E402
from govoplan_release.repository_tag import tag_repositories # noqa: E402
import test_release_meta_source_tag as meta_fixture # noqa: E402
import test_release_repository_tag as release_fixture # noqa: E402
from test_release_repository_tag import git, git_text, ref_exists # noqa: E402
class RegisteredSourceTagBatchTests(unittest.TestCase):
setUp = meta_fixture.MetaSourceTagTests.setUp
synchronize = meta_fixture.MetaSourceTagTests.synchronize
assert_no_tags = meta_fixture.MetaSourceTagTests.assert_no_tags
def tag(self, *, repos=("govoplan-access",), apply=False, push=False):
return tag_repositories(
repos=repos,
repo_versions={repo: self.version for repo in repos},
workspace_root=self.workspace,
apply=apply,
push=push,
)
def test_python_only_module_publishes_without_meta_or_core_checkout(self):
self.meta.rename(self.root / "unused-meta")
self.core.rename(self.root / "unused-core")
preview = self.tag(push=True)
self.assertEqual("planned", preview["status"], preview)
self.assertEqual("registered-source-batch-v1", preview["source_contract"])
self.assertEqual({}, preview["bundle_input_receipts"])
self.assertFalse(ref_exists(self.access, "refs/tags/v0.1.10"))
published = self.tag(apply=True, push=True)
self.assertEqual("published", published["status"], published)
self.assertEqual(["govoplan-access"], list(published["source_receipts"]))
self.assertEqual(
git_text(self.access, "rev-parse", "HEAD"),
git_text(self.access_remote, "rev-parse", "refs/heads/main"),
)
def test_core_only_batch_has_no_meta_composition_requirement(self):
self.meta.rename(self.root / "unused-meta")
result = self.tag(repos=("govoplan-core",), apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual(["govoplan-core"], list(result["source_receipts"]))
def test_backend_only_publication_never_reads_irrelevant_unsafe_core_json(self):
webui = self.core / "webui"
webui.mkdir()
(webui / "package.release.json").write_text("invalid unselected Core JSON")
(webui / "package-lock.release.json").symlink_to(
self.root / "not-a-core-release-lock"
)
from govoplan_release import version_alignment
original = version_alignment._json_object
def read(path):
if path.is_relative_to(webui):
raise AssertionError(
"backend-only publication must not read unrelated Core JSON"
)
return original(path)
with patch.object(version_alignment, "_json_object", side_effect=read):
result = self.tag(apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual({}, result["bundle_input_receipts"])
def test_source_origin_parent_and_read_only_target_guards_apply_without_meta(self):
for problem in ("origin", "parent", "read_only"):
with self.subTest(problem=problem):
parent_mode = self.root.stat().st_mode & 0o7777
git_directory = self.access / ".git"
git_mode = git_directory.stat().st_mode & 0o7777
if problem == "origin":
git(
self.access,
"config",
"remote.origin.pushurl",
str(self.root / "unknown.git"),
)
elif problem == "parent":
self.root.chmod(0o777)
else:
git_directory.chmod(0o500)
try:
for apply in (False, True):
result = self.tag(apply=apply, push=True)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
finally:
self.root.chmod(parent_mode)
git_directory.chmod(git_mode)
if problem == "origin":
git(self.access, "config", "--unset", "remote.origin.pushurl")
def test_wrong_owner_symlink_and_hidden_index_are_rejected_without_meta(self):
config = self.access / ".git/config"
original_stat = Path.lstat
def wrong_owner(path, *args, **kwargs):
observed = original_stat(path, *args, **kwargs)
if path == config:
fields = list(observed)
fields[4] = os.geteuid() + 1
return os.stat_result(fields)
return observed
with patch.object(Path, "lstat", new=wrong_owner):
self.assertEqual("blocked", self.tag(apply=True)["status"])
moved = self.root / "moved-access"
self.access.rename(moved)
self.access.symlink_to(moved, target_is_directory=True)
try:
self.assertEqual("blocked", self.tag(apply=True)["status"])
finally:
self.access.unlink()
moved.rename(self.access)
for flag, undo in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
with self.subTest(flag=flag):
git(self.access, "update-index", flag, "pyproject.toml")
try:
self.assertEqual("blocked", self.tag(apply=True)["status"])
self.assert_no_tags()
finally:
git(self.access, "update-index", undo, "pyproject.toml")
def test_live_remote_divergence_blocks_without_cached_tracking_update(self):
clone = self.root / "other-access-writer"
git(self.root, "clone", "--branch", "main", str(self.access_remote), str(clone))
git(clone, "config", "user.name", "Synthetic writer")
git(clone, "config", "user.email", "writer@example.invalid")
(clone / "advance.txt").write_text("new remote main\n")
git(clone, "add", ".")
git(clone, "commit", "-m", "Advance remote without updating original tracking")
git(clone, "push", "origin", "main")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("live origin/main", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_ignored_selected_version_metadata_cannot_supply_an_untagged_artifact(self):
project = self.access / "pyproject.toml"
original = project.read_text()
git(self.access, "rm", "pyproject.toml")
(self.access / ".gitignore").write_text("/pyproject.toml\n")
git(self.access, "add", ".gitignore")
git(self.access, "commit", "-m", "Ignored metadata absent from frozen tree")
project.write_text(original)
self.assertEqual("", git_text(self.access, "status", "--porcelain"))
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("must be tracked", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_lightweight_and_different_annotation_objects_remain_immutable(self):
git(self.access, "tag", "v0.1.10")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.access, "tag", "-d", "v0.1.10")
git(self.access, "tag", "-a", "v0.1.10", "-m", "Original immutable annotation")
git(self.access, "push", "origin", "refs/tags/v0.1.10")
original = git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
git(self.access, "tag", "-d", "v0.1.10")
git(
self.access,
"tag",
"-a",
"v0.1.10",
"-m",
"Conflicting immutable annotation",
)
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
self.assertEqual(
original, git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
)
def test_changed_source_after_preflight_stops_before_first_batch_effect(self):
original = source_tag_batch._preview_repositories
def changed(**kwargs):
result = original(**kwargs)
(self.access / "changed-source.txt").write_text(
"new source after preflight\n"
)
git(self.access, "add", ".")
git(self.access, "commit", "-m", "Source changed")
return result
with patch.object(
source_tag_batch, "_preview_repositories", side_effect=changed
):
result = self.tag(
repos=("govoplan-core", "govoplan-access"), apply=True, push=True
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_false_push_success_is_not_a_receipt_and_never_retries(self):
original = source_tag_batch.run
pushes = []
def run(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
pushes.append(command)
return subprocess.CompletedProcess(command, 0, "", "")
return original(command, **kwargs)
with patch.object(source_tag_batch, "run", side_effect=run):
result = self.tag(
repos=("govoplan-access", "govoplan-core"), apply=True, push=True
)
self.assertEqual("partial", result["status"], result)
self.assertEqual(1, len(pushes))
self.assertEqual("skipped", result["repositories"][1]["status"])
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.access_remote, "refs/tags/v0.1.10"))
def _ready_bundle(self):
self.repo = self.core # Existing fixture's Core name.
campaign, remote = (
release_fixture.ReleaseRepositoryTagTests._staged_campaign_bundle(self)
)
self.specs.append(
{
"name": "govoplan-campaign",
"category": "module",
"subtype": "domain",
"path": "govoplan-campaign",
"remote": str(remote),
}
)
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.assertEqual(
"tagged", self.tag(repos=("govoplan-campaign",), apply=True)["status"]
)
lock_path = self.core / "webui/package-lock.release.json"
payload = json.loads(lock_path.read_text())
package = payload["packages"]["node_modules/@govoplan/campaign-webui"]
package["version"] = self.version
package["resolved"] = (
"git+ssh://git@example.test/acme/govoplan-campaign.git#"
+ git_text(campaign, "rev-parse", "HEAD")
)
lock_path.write_text(json.dumps(payload))
# Core is a reviewed input only here: do not require an unrelated tag
# or silently impose a new clean-Core prerequisite for module release.
return campaign, remote, lock_path
def test_module_publication_freezes_core_inputs_without_requiring_core_tag(self):
_campaign, _remote, _lock = self._ready_bundle()
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual(["govoplan-campaign"], list(result["source_receipts"]))
self.assertEqual(
{"webui/package.release.json", "webui/package-lock.release.json"},
set(result["bundle_input_receipts"]),
)
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
def test_changed_or_group_writable_core_bundle_inputs_block_module_publication(
self,
):
campaign, remote, lock = self._ready_bundle()
original = source_tag_batch._preview_repositories
def changed(**kwargs):
result = original(**kwargs)
lock.write_text(lock.read_text() + "\n")
return result
with patch.object(
source_tag_batch, "_preview_repositories", side_effect=changed
):
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"bundle input receipt changed", result["repositories"][0]["detail"]
)
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
lock.chmod(0o666)
try:
self.assertEqual(
"blocked",
self.tag(repos=("govoplan-campaign",), apply=True, push=True)["status"],
)
finally:
lock.chmod(0o644)
self.assertTrue(ref_exists(campaign, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
def test_read_only_shared_preflight_has_no_apply_or_mutating_legacy_entry(self):
import inspect
from govoplan_release import repository_tag
self.assertNotIn(
"apply", inspect.signature(repository_tag._preview_repositories).parameters
)
self.assertFalse(hasattr(repository_tag, "_tag_repositories_legacy"))
if __name__ == "__main__":
unittest.main()
+35
View File
@@ -13,12 +13,47 @@ if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.version_metadata import ( # noqa: E402
VersionMetadataError,
apply_version_metadata_mutations,
version_metadata_mutations,
)
class ReleaseVersionMetadataTests(unittest.TestCase):
def test_updates_shared_module_version_without_rewriting_independent_interfaces(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
backend = root / "src" / "govoplan_example" / "backend"
backend.mkdir(parents=True)
manifest = backend / "manifest.py"
manifest.write_text(
'MODULE_VERSION: str = "1.2.3"\n'
'manifest = ModuleManifest(id="example", version=MODULE_VERSION,\n'
' provides_interfaces=(ModuleInterfaceProvider(name="api", version="2.0"),))\n',
encoding="utf-8",
)
changed = apply_version_metadata_mutations(root, target_version="1.2.4")
self.assertEqual(("src/govoplan_example/backend/manifest.py",), changed)
self.assertIn('MODULE_VERSION: str = "1.2.4"', manifest.read_text())
self.assertIn('version="2.0"', manifest.read_text())
self.assertEqual((), version_metadata_mutations(root, target_version="1.2.4"))
def test_dynamic_module_version_fails_before_any_metadata_is_written(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
backend = root / "src" / "govoplan_example" / "backend"
backend.mkdir(parents=True)
project = root / "pyproject.toml"
project.write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
before = project.read_bytes()
(backend / "manifest.py").write_text(
'MODULE_VERSION = compute_version()\n'
'manifest = ModuleManifest(id="example", version=MODULE_VERSION)\n',
)
with self.assertRaisesRegex(VersionMetadataError, "no literal MODULE_VERSION"):
apply_version_metadata_mutations(root, target_version="1.2.4")
self.assertEqual(before, project.read_bytes())
def test_updates_recognized_metadata_without_changing_interface_versions(
self,
) -> None:
+4
View File
@@ -98,6 +98,10 @@ class SecurityAuditWrapperTests(unittest.TestCase):
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
exit 0
fi
if [[ " $* " != *" --redact=100 "* ]]; then
echo 'secret scans must redact reports and logs' >&2
exit 3
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--report-path' ]]; then
+194
View File
@@ -0,0 +1,194 @@
"""Offline safety and completeness checks for the cross-product UI review program."""
import importlib.util
from pathlib import Path
import socket
import sys
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/gitea"))
SPEC = importlib.util.spec_from_file_location("ui_review_program", ROOT / "tools/gitea/gitea-ui-review-program.py")
assert SPEC and SPEC.loader
program = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = program
SPEC.loader.exec_module(program)
def scope(scope_id="campaigns", kind="manifest"):
return {
"scope_id": scope_id, "name": "Campaigns", "repository": "govoplan-campaign",
"kind": kind, "manifest_paths": ["src/govoplan_campaign/backend/manifest.py"],
"frontend": {
"routes": [{"path": "/campaigns/:campaignId/*", "component": "CampaignWorkspace"}],
"public_routes": [{"path": "/public/example", "component": "PublicExample"}],
"settings_routes": [{"path": "/settings/example", "component": "ExampleSettings"}],
"nav_items": [{"path": "/campaigns", "label": "Campaigns"}],
"view_surfaces": [{"id": "campaigns.widget.activity", "label": "Activity widget"}],
},
"source_groups": {"Dialogs and embedded editing surfaces": ["webui/src/ExampleDialog.tsx"]},
"ui_source_count": 1,
}
def test_catalog_includes_core_all_manifests_and_each_placeholder(tmp_path):
names = ["govoplan", "govoplan-core", "govoplan-campaign", "govoplan-ledger", "govoplan-xoev", "website"]
for name in names:
(tmp_path / name).mkdir()
catalog = {"repositories": [
{"name": name, "path": name, "category": "website" if name == "website" else "system" if name in {"govoplan", "govoplan-core"} else "connector" if name == "govoplan-xoev" else "module"}
for name in names
]}
manifests = [{"id": "campaigns", "name": "Campaigns", "repository": "govoplan-campaign", "frontend": None}]
scopes = program.build_scopes(catalog, tmp_path, manifests)
assert {item["scope_id"] for item in scopes} == {"core", "campaigns", "catalog:govoplan-ledger", "catalog:govoplan-xoev"}
assert sum(item["kind"] == "placeholder" for item in scopes) == 2
assert next(item for item in scopes if item["scope_id"] == "campaigns")["kind"] == "manifest"
def test_missing_checkout_or_implementation_without_manifest_is_not_called_placeholder(tmp_path):
catalog = {"repositories": [{"name": "govoplan-example", "path": "govoplan-example", "category": "module"}]}
with pytest.raises(program.GiteaError, match="Missing source checkout"):
program.build_scopes(catalog, tmp_path, [])
root = tmp_path / "govoplan-example"
root.mkdir()
(root / "pyproject.toml").touch()
with pytest.raises(program.GiteaError, match="implementation but no extracted manifest"):
program.build_scopes(catalog, tmp_path, [])
def test_duplicate_or_uncatalogued_manifest_is_rejected(tmp_path):
manifest = {"id": "example", "name": "Example", "repository": "govoplan-example", "frontend": None}
with pytest.raises(program.GiteaError, match="Duplicate source manifest"):
program.build_scopes({"repositories": []}, tmp_path, [manifest, manifest])
with pytest.raises(program.GiteaError, match="absent from the module review catalog"):
program.build_scopes({"repositories": []}, tmp_path, [manifest])
def test_existing_closed_issue_is_found_and_unchanged():
issue = {"number": 4, "title": "Reviewer renamed it", "state": "closed", "body": program.marker("campaigns") + "\nHuman findings\n- [x] Done"}
before = issue.copy()
assert program.find_existing([issue], "campaigns", program.issue_title(scope())) is issue
assert issue == before
def test_gitea_null_pull_request_field_is_an_ordinary_issue():
issue = {"number": 25, "title": "UI review", "state": "open", "body": program.marker("mail"), "pull_request": None}
assert program.find_existing([issue], "mail", "UI review") is issue
def test_unmanaged_title_and_ambiguous_marker_stop_without_overwrite():
title = program.issue_title(scope())
with pytest.raises(program.GiteaError, match="Unmanaged exact-title"):
program.find_existing([{"title": " " + title.upper() + " ", "body": "user content"}], "campaigns", title)
duplicate = {"title": title, "body": program.marker("campaigns")}
with pytest.raises(program.GiteaError, match="Ambiguous"):
program.find_existing([duplicate, duplicate.copy()], "campaigns", title)
def test_pr_not_used_as_matching_issue():
issue = {"title": program.issue_title(scope()), "body": program.marker("campaigns"), "pull_request": {}}
assert program.find_existing([issue], "campaigns", issue["title"]) is None
def test_child_inventory_and_all_principles_start_pending():
body = program.issue_body(scope(), "https://example.test/epic/56")
assert "**Pending / not reviewed.**" in body
assert "https://example.test/epic/56" in body
assert "/campaigns/:campaignId/*" in body
assert "/public/example" in body
assert "/settings/example" in body
assert "campaigns.widget.activity" in body
assert "webui/src/ExampleDialog.tsx" in body
assert "compact read-only campaign settings dashboard" in body
assert "Save/Cancel" in body and "dirty-state protection" in body
assert "- [x]" not in body
assert body.count("| Pending inventory | Pending review | Not yet recorded | None approved |") == 9
for identity, _ in program.PRINCIPLES:
assert identity in body
assert "Reopen this issue or link an owned follow-up" in body
def test_headless_and_placeholder_scopes_are_not_automatic_completions():
headless = scope("rest")
headless["frontend"] = None
assert "No standalone frontend is declared" in program.issue_body(headless, "epic")
assert "**The review is still pending:**" in program.issue_body(headless, "epic")
placeholder = scope("catalog:govoplan-ledger", "placeholder")
body = program.issue_body(placeholder, "epic")
assert "there is no runtime module ID, manifest or standalone WebUI" in body
assert "Keep the future interface review pending" in body
assert "- [x]" not in body
def test_source_grouping_uses_real_files_and_clearly_bounds_large_seeds(tmp_path):
source = tmp_path / "webui/src"
source.mkdir(parents=True)
for name in ["ExamplePage.tsx", "EditDialog.tsx", "TenantSettings.tsx", "ActivityWidget.tsx", "Button.tsx"]:
(source / name).touch()
groups = program.source_groups(tmp_path)
assert sum(map(len, groups.values())) == 5
assert groups["Dialogs and embedded editing surfaces"] == ["webui/src/EditDialog.tsx"]
large = scope()
large["source_groups"] = {"Pages": [f"webui/src/Page{index}.tsx" for index in range(45)]}
large["ui_source_count"] = 45
seed = program.source_seed(large)
assert "5 further files" in seed
assert "not a completed runtime audit" in seed
def test_epic_initialization_preserves_surrounding_text_and_keeps_all_unchecked():
body = program.EPIC_MARKER + "\nHuman introduction\n" + program.LIST_START + "\n" + program.INITIAL_LIST + "\n" + program.LIST_END + "\nHuman evidence"
records = [
{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"},
{"name": "Ledger", "scope_id": "catalog:govoplan-ledger", "repository": "govoplan-ledger", "kind": "placeholder", "url": "https://example.test/ledger/2"},
]
result = program.initialized_epic_body(body, records)
assert "Human introduction" in result and "Human evidence" in result
assert result.count("- [ ]") == 2
assert "Catalogued placeholders" in result
assert program.initialized_epic_body(result, records) == result
human_progress = result.replace("- [ ] [Core]", "- [x] [Core]")
assert program.initialized_epic_body(human_progress, records) == human_progress
def test_epic_edited_or_ambiguous_lists_are_never_overwritten():
records = [{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"}]
body = program.EPIC_MARKER + program.LIST_START + "User-managed content" + program.LIST_END
with pytest.raises(program.GiteaError, match="already edited"):
program.initialized_epic_body(body, records)
with pytest.raises(program.GiteaError, match="absent or ambiguous"):
program.initialized_epic_body(body + program.LIST_START, records)
with pytest.raises(program.GiteaError, match="incomplete issue link inventory"):
program.render_links([{**records[0], "url": None}])
def test_ipv4_override_is_host_scoped_and_restored(monkeypatch):
calls = []
def original(host, port, family=0, type=0, proto=0, flags=0):
calls.append((host, family))
return []
monkeypatch.setattr(socket, "getaddrinfo", original)
with program.ipv4_for_target(True):
socket.getaddrinfo("git.add-ideas.de", 443)
socket.getaddrinfo("unrelated.example", 443)
assert calls == [("git.add-ideas.de", socket.AF_INET), ("unrelated.example", 0)]
assert socket.getaddrinfo is original
def test_snapshot_covers_every_current_catalog_module():
import json
catalog = json.loads((ROOT / "repositories.json").read_text())
inventory_path = ROOT / "docs/project/ui-review-issue-inventory.json"
snapshot = json.loads(inventory_path.read_text())
expected = {repo["name"] for repo in catalog["repositories"] if repo["category"] in {"module", "connector"} or repo["name"] == "govoplan-core"}
assert {issue["repository"] for issue in snapshot["issues"]} == expected
assert len(snapshot["issues"]) == len(expected)
assert len({issue["url"] for issue in snapshot["issues"]}) == len(expected)
assert snapshot["scope_count"] == 77
assert snapshot["implemented_scopes"] == 73
assert snapshot["manifest_modules"] == 72
assert snapshot["catalogued_placeholders"] == 4
assert all(issue["number"] and issue["url"].startswith("https://git.add-ideas.de/GovOPlaN/") for issue in snapshot["issues"])
+78
View File
@@ -0,0 +1,78 @@
from __future__ import annotations
import json
from pathlib import Path
import runpy
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
CHECK = runpy.run_path(str(META_ROOT / "tools/checks/check-webui-package-facades.py"))
class WebuiPackageFacadeTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.repository = Path(self.temporary.name) / "govoplan-example"
source = self.repository / "webui/src"
source.mkdir(parents=True)
(source / "index.ts").write_text("export {};\n", encoding="utf-8")
(source / "styles.css").write_text(":root {}\n", encoding="utf-8")
self.webui = {
"name": "@govoplan/example-webui", "version": "0.1.2", "type": "module",
"main": "src/index.ts",
"exports": {".": {"import": "./src/index.ts"}, "./styles.css": "./src/styles.css"},
"peerDependencies": {"@govoplan/core-webui": "^0.1.45"},
"peerDependenciesMeta": {"@govoplan/core-webui": {"optional": True}},
}
self.root = {**self.webui, **{
field: CHECK["prefixed_entries"](self.webui[field])
for field in CHECK["ENTRY_FIELDS"] if field in self.webui
}}
self.write_manifests()
def write_manifests(self) -> None:
(self.repository / "webui/package.json").write_text(json.dumps(self.webui), encoding="utf-8")
(self.repository / "package.json").write_text(json.dumps(self.root), encoding="utf-8")
def issues(self) -> list[str]:
return CHECK["facade_issues"](self.repository, package_name="@govoplan/example-webui")
def test_matching_conditional_and_css_entries_are_accepted(self) -> None:
self.assertEqual([], self.issues())
def test_missing_root_or_generic_package_is_rejected(self) -> None:
(self.repository / "package.json").unlink()
self.assertIn("cannot read", " ".join(self.issues()))
self.root = {"name": "@govoplan/example", "version": "0.1.2"}
self.write_manifests()
self.assertIn("root name differs", " ".join(self.issues()))
self.assertIn("no WebUI entry point", " ".join(self.issues()))
def test_peer_drift_or_missing_entry_is_rejected(self) -> None:
self.root["peerDependencies"] = {"@govoplan/core-webui": "^9.0.0"}
self.write_manifests()
self.assertIn("peerDependencies differs", " ".join(self.issues()))
(self.repository / "webui/src/styles.css").unlink()
self.assertIn("missing exports entry", " ".join(self.issues()))
def test_entry_cannot_escape_webui_even_if_it_exists(self) -> None:
(self.repository / "outside.ts").write_text("export {};\n", encoding="utf-8")
self.root["main"] = "webui/../outside.ts"
self.write_manifests()
self.assertIn("escapes webui/", " ".join(self.issues()))
def test_release_composition_checks_only_declared_module_sources(self) -> None:
core = self.repository.parent / "govoplan-core/webui"
core.mkdir(parents=True)
(core / "package.release.json").write_text(json.dumps({"dependencies": {
"react": "19.2.7",
"@govoplan/example-webui": "git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-example.git#v0.1.2",
}}), encoding="utf-8")
self.assertEqual((1, []), CHECK["check_composition"](self.repository.parent))
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,154 @@
from __future__ import annotations
import os
from pathlib import Path
import subprocess
import tempfile
import textwrap
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
INSTALLER = META_ROOT / "tools" / "release" / "install-webui-release-dependencies.sh"
STAGES = ("base", "clone", "modules")
class WebUIReleaseDependencyRetryTests(unittest.TestCase):
def _run_installer(
self, stage: str, statuses: tuple[int, ...]
) -> tuple[subprocess.CompletedProcess[str], list[str]]:
with tempfile.TemporaryDirectory(prefix="govoplan-installer-retry-test-") as directory:
root = Path(directory)
stub_bin = root / "bin"
stub_bin.mkdir()
core_root = root / "core"
webui = core_root / "web ui"
webui.mkdir(parents=True)
work_root = root / "work"
work_root.mkdir()
log = root / "commands.log"
stub = "#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(
r"""
case "${0##*/}" in
node)
# Supply the shell's dependency list without requiring Node.
printf '%s\t%s\n' '@govoplan/example-webui' \
'git+https://example.invalid/module.git#v1.0.0' > "$GOVOPLAN_DEPS"
printf 'node\n' >> "$RETRY_TEST_LOG"
exit 0
;;
sleep)
printf 'sleep %s\n' "$*" >> "$RETRY_TEST_LOG"
exit 0
;;
npm)
case "${1:-}" in
cache)
printf 'cache\n' >> "$RETRY_TEST_LOG"
exit 0
;;
install)
stage=base
for argument in "$@"; do
if [[ "$argument" == --no-save ]]; then
stage=modules
fi
done
;;
*) exit 98 ;;
esac
;;
git)
[[ "${1:-}" == clone ]] || exit 98
stage=clone
;;
*) exit 98 ;;
esac
status=0
if [[ "$stage" == "$RETRY_TEST_STAGE" ]]; then
attempt=0
counter="$RETRY_TEST_ROOT/$stage.count"
if [[ -f "$counter" ]]; then
read -r attempt < "$counter"
fi
read -r -a statuses <<< "$RETRY_TEST_STATUSES"
status="${statuses[$attempt]:-99}"
printf '%s\n' "$((attempt + 1))" > "$counter"
fi
printf '%s %s\n' "$stage" "$status" >> "$RETRY_TEST_LOG"
exit "$status"
"""
)
for name in ("node", "npm", "git", "sleep"):
executable = stub_bin / name
executable.write_text(stub, encoding="utf-8")
executable.chmod(0o755)
env = os.environ.copy()
env.update(
{
"PATH": f"{stub_bin}:{os.defpath}",
"TMPDIR": str(work_root),
"GOVOPLAN_CORE_ROOT": str(core_root),
"GOVOPLAN_WEBUI_PACKAGE_LOCK": "",
"GOVOPLAN_WEBUI_PACKAGE_DIR": "",
"RETRY_TEST_ROOT": str(root),
"RETRY_TEST_LOG": str(log),
"RETRY_TEST_STAGE": stage,
"RETRY_TEST_STATUSES": " ".join(map(str, statuses)),
}
)
result = subprocess.run(
[
"bash",
"-c",
'set -euo pipefail; bash "$1" "$2"; '
'printf "caller-continued\\n" >> "$RETRY_TEST_LOG"',
"retry-test-caller",
str(INSTALLER),
str(webui),
],
cwd=root,
env=env,
text=True,
capture_output=True,
timeout=10,
check=False,
)
self.assertEqual(list(work_root.iterdir()), [], result.stderr)
return result, log.read_text(encoding="utf-8").splitlines()
def _assert_attempts(self, statuses: tuple[int, ...]) -> None:
for retried_stage in STAGES:
with self.subTest(stage=retried_stage, statuses=statuses):
result, commands = self._run_installer(retried_stage, statuses)
expected = ["node", "cache"]
for stage in STAGES:
attempts = statuses if stage == retried_stage else (0,)
for index, status in enumerate(attempts):
expected.append(f"{stage} {status}")
if status and index < 2:
expected.append(f"sleep {(index + 1) * 10}")
if attempts[-1]:
break
if statuses[-1] == 0:
expected.append("caller-continued")
self.assertEqual(result.returncode, statuses[-1], result.stderr)
self.assertEqual(commands, expected, result.stderr)
def test_success_on_first_attempt(self) -> None:
self._assert_attempts((0,))
def test_success_on_second_attempt(self) -> None:
self._assert_attempts((17, 0))
def test_success_on_third_attempt(self) -> None:
self._assert_attempts((17, 23, 0))
def test_exhaustion_preserves_final_status_and_stops_callers(self) -> None:
self._assert_attempts((17, 23, 47))
if __name__ == "__main__":
unittest.main()
+293 -43
View File
@@ -1,13 +1,151 @@
#!/usr/bin/env bash
set -euo pipefail
# The JSON catalog is the only phase order/metadata authority. Check commands
# remain in the marked functions below; devkit invokes this script, not copies.
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
FOCUSED_MODE=run
FOCUSED_PHASE=""
FOCUSED_JSON=0
focused_usage() {
echo "Usage: check-focused.sh [--phase ID] | --list-phases [--json]"
echo "Without --phase, run every canonical phase in catalog order (fail fast)."
}
while [[ $# -gt 0 ]]; do
case "$1" in
--phase)
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" || $# -lt 2 || -z "$2" || "$2" == --* ]]; then
echo "check-focused: --phase requires one ID and cannot be combined with listing." >&2
exit 2
fi
FOCUSED_PHASE="$2"
shift 2
;;
--list-phases)
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" ]]; then
echo "check-focused: duplicate or incompatible phase selection." >&2
exit 2
fi
FOCUSED_MODE=list
shift
;;
--json)
if [[ "$FOCUSED_JSON" == 1 ]]; then
echo "check-focused: duplicate --json." >&2
exit 2
fi
FOCUSED_JSON=1
shift
;;
--help|-h)
focused_usage
exit 0
;;
*)
echo "check-focused: unknown argument: $1" >&2
focused_usage >&2
exit 2
;;
esac
done
if [[ "$FOCUSED_JSON" == 1 && "$FOCUSED_MODE" != list ]]; then
echo "check-focused: --json is supported only with --list-phases." >&2
exit 2
fi
# Metadata-only operations need standard Python, not an installed product venv,
# Node, npm, a Core checkout, or any temporary/state directory.
FOCUSED_METADATA_PYTHON="$(command -v python3)" || {
echo "check-focused: Python 3 is required to read phase metadata." >&2
exit 127
}
FOCUSED_SELECTION="$(
PYTHONDONTWRITEBYTECODE=1 "$FOCUSED_METADATA_PYTHON" -I -S - \
"$META_ROOT/tools/checks/focused-phases.json" "$FOCUSED_MODE" "$FOCUSED_PHASE" "$FOCUSED_JSON" <<'PY'
import json
import os
from pathlib import Path
import re
import stat
import sys
try:
path = Path(sys.argv[1])
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
status = os.fstat(handle.fileno())
if not stat.S_ISREG(status.st_mode) or status.st_size > 1024 * 1024:
raise ValueError("phase metadata must be a bounded regular file")
encoded = handle.read(1024 * 1024 + 1)
if len(encoded) > 1024 * 1024:
raise ValueError("phase metadata exceeds its size bound")
def unique(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError("duplicate phase metadata key")
result[key] = value
return result
catalog = json.loads(encoded, object_pairs_hook=unique)
if not isinstance(catalog, dict) or set(catalog) != {"schema_version", "phases"} or type(catalog["schema_version"]) is not int or catalog["schema_version"] != 1:
raise ValueError("unsupported phase metadata schema")
phases = catalog["phases"]
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
raise ValueError("phase catalog requires 132 definitions")
seen = set()
fields = {"id", "title", "cwd", "order_after", "depends_on", "resources", "outputs", "notes"}
for phase in phases:
if not isinstance(phase, dict) or set(phase) != fields:
raise ValueError("invalid phase metadata fields")
identity = phase["id"]
if not isinstance(identity, str) or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity) or identity in seen:
raise ValueError("invalid or duplicate phase ID")
if not isinstance(phase["title"], str) or not phase["title"].strip() or len(phase["title"]) > 256 or any(ord(char) < 32 for char in phase["title"]):
raise ValueError("invalid phase title")
if phase["cwd"] not in {"core", "meta", "core-webui", "access-webui"}:
raise ValueError("unsupported phase working directory")
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
values = phase[field]
if not isinstance(values, list) or len(values) > 64 or any(not isinstance(value, str) or not value or len(value) > 4096 or "\0" in value for value in values):
raise ValueError("invalid phase list: " + field)
if len(set(values)) != len(values):
raise ValueError("duplicate phase list value: " + field)
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
raise ValueError("phase prerequisites must precede their consumer")
seen.add(identity)
mode, selected, as_json = sys.argv[2:]
if selected and selected not in seen:
raise ValueError("unknown phase: " + selected)
if mode == "list":
print(json.dumps(catalog, indent=2) if as_json == "1" else "\n".join(phase["id"] + "\t" + phase["title"] for phase in phases))
else:
print("\n".join(phase["id"] for phase in phases if not selected or phase["id"] == selected))
except (OSError, ValueError, TypeError, RecursionError) as exc:
print("check-focused: " + str(exc), file=sys.stderr)
raise SystemExit(2)
PY
)"
if [[ "$FOCUSED_MODE" == list ]]; then
printf '%s\n' "$FOCUSED_SELECTION"
exit 0
fi
focused_setup() {
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$META_ROOT")}"
export GOVOPLAN_WORKSPACE_ROOT="$WORKSPACE_ROOT"
ROOT="${GOVOPLAN_CORE_ROOT:-$WORKSPACE_ROOT/govoplan-core}"
ROOT="$(cd "$ROOT" && pwd)"
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
NODE="/home/zemion/.nvm/versions/node/v22.22.3/bin"
NPM="$NODE/npm"
NODE="$(command -v "${NODE:-node}")" || { echo "check-focused: Node is unavailable; run ./devkit doctor." >&2; exit 127; }
NPM="$(command -v "${NPM:-npm}")" || { echo "check-focused: npm is unavailable; run ./devkit doctor." >&2; exit 127; }
NODE_BIN="$(dirname "$NODE")"
WEBUI_BIN="$ROOT/webui/node_modules/.bin"
NPM_USERCONFIG="$(mktemp "${TMPDIR:-/tmp}/govoplan-npmrc.XXXXXXXX")"
@@ -19,7 +157,7 @@ trap 'rm -f "$NPM_USERCONFIG"' EXIT
exit 127
}
export PATH="$WEBUI_BIN:$NODE:$PATH"
export PATH="$WEBUI_BIN:$NODE_BIN:$PATH"
export NPM_CONFIG_USERCONFIG="$NPM_USERCONFIG"
export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG"
unset npm_config_tmp NPM_CONFIG_TMP
@@ -27,39 +165,65 @@ unset npm_config_tmp NPM_CONFIG_TMP
# Validate the current sibling checkouts even when a newly added module has not
# yet been installed into an existing development virtualenv.
SOURCE_PYTHONPATH=""
for source_dir in "$META_ROOT"/../govoplan*/src; do
for source_dir in "$WORKSPACE_ROOT"/govoplan*/src; do
[ -d "$source_dir" ] || continue
SOURCE_PYTHONPATH="${SOURCE_PYTHONPATH:+$SOURCE_PYTHONPATH:}$source_dir"
done
export PYTHONPATH="${SOURCE_PYTHONPATH}${PYTHONPATH:+:$PYTHONPATH}"
}
focused_phase_preflight() {
# devkit-phase: preflight begin
cd "$ROOT"
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
"$NODE" "$META_ROOT/tests/test-jsx-value-imports.mjs"
"$NODE" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
"$NODE" "$META_ROOT/tests/test-heading-help.mjs"
"$NODE" "$META_ROOT/tools/checks/check-heading-help.mjs"
"$NODE" --test "$META_ROOT/tests/test-devkit-display-labels.mjs"
"$NODE" --test "$ROOT/webui/tests/component-test-runner.test.mjs"
"$NODE" "$WORKSPACE_ROOT/govoplan-files/webui/scripts/test-archive-client.mjs"
# devkit-phase: preflight end
}
focused_phase_tooling() {
# devkit-phase: tooling begin
cd "$META_ROOT"
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
"$PYTHON" tools/inventory/platform-interface-inventory.py --workspace-root "$WORKSPACE_ROOT" --strict-declarations --strict-endpoints
"$PYTHON" tools/repo/sync-module-package-workflows.py --check
"$PYTHON" tools/release/generate-developer-meta-package.py --check
"$PYTHON" tools/checks/check-webui-package-facades.py
"$PYTHON" -m unittest tests.test_webui_package_facades
"$PYTHON" -m pytest -q tests/test_ui_review_program.py "$META_ROOT"/tests/test_devkit_*.py
"$PYTHON" -m pytest -q tests/test_focused_phases.py
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
"$PYTHON" -m unittest tests.test_deployment_installer
"$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries
"$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py
"$PYTHON" -m unittest tests.test_isolated_work_composition
"$PYTHON" -m unittest tests.test_capability_fit_evidence
"$PYTHON" -m unittest tests.test_capability_fit_generation tests.test_capability_fit_review
"$PYTHON" tools/assessments/generate-capability-fit-report.py --check
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
"$PYTHON" -m unittest tests.test_institutional_governance_journey
"$PYTHON" -m unittest tests.test_institutional_service_journey
# devkit-phase: tooling end
}
focused_phase_backend() {
# devkit-phase: backend begin
cd "$ROOT"
"$PYTHON" - <<'PY'
import ast
import pathlib
import os
import sys
repos_root = pathlib.Path("/mnt/DATA/git")
repos_root = pathlib.Path(os.environ["GOVOPLAN_WORKSPACE_ROOT"])
roots = [
repo / "src"
for repo in sorted(repos_root.glob("govoplan*"))
@@ -96,71 +260,157 @@ PY
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py"
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py"
"$PYTHON" -m unittest tests.test_module_system
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-quick-access/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-payments/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms-runtime/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-cases/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-committee/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-voting/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-approvals/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
"$PYTHON" -m unittest tests.test_bounded_process
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
"$PYTHON" -m pytest -q \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_managed_archives.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_work.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_staging.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_upload_response_batching.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_performance.py"
"$PYTHON" -m pytest -q \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_workers.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_inspection_bounds.py" \
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archives.py"
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests/test_external_function_mapping_migration.py"
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests"
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-templates/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-connectors/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-datasources/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dataflow/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow-engine/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-views/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-quick-access/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dashboard/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-postbox/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-portal/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-payments/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms-runtime/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-cases/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-committee/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-voting/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-approvals/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-identity-trust/tests"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-encryption/tests"
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-wiki/tests"
"$PYTHON" -m pytest -q \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_approval_gate.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_editor_state_security.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_mail_profile_boundary.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_independent_configuration_repairs.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_incremental_review_persistence.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_reviewed_build_mock.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_delivery_policy_settings.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_synchronous_delivery_policy.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_workerless_recovery.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_imap_batch_integration.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_testbed_claim_recovery.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py" \
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_archive_encryption_governance.py" \
"${WORKSPACE_ROOT}/govoplan-policy/tests/test_campaign_archive_encryption.py" \
"${WORKSPACE_ROOT}/govoplan-policy/tests/test_archive_encryption_api.py"
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_protocol_authorization.py" "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_imap_batch.py"
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-mail/tests"
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
"$PYTHON" -m unittest \
tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \
tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \
tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry
# devkit-phase: backend end
}
focused_phase_core_ui() {
# devkit-phase: core-ui begin
cd "$ROOT/webui"
"$NPM" run test:layout-primitives
"$NPM" run test:mail-components
"$NPM" run test:api-client-cache
"$NPM" run test:auth-action-state
"$NPM" run test:dependency-security
"$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components
"$NODE" --test tests/breadcrumb-bar.test.mjs
"$NPM" run test:module-capabilities
# devkit-phase: core-ui end
}
focused_phase_module_builds() {
# devkit-phase: module-builds begin
cd "$ROOT/webui"
"$NPM" run test:module-permutations
# devkit-phase: module-builds end
}
focused_phase_browser() {
# devkit-phase: browser begin
cd "$ROOT/webui"
"$NPM" run test:conformance
# devkit-phase: browser end
}
"$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json
focused_phase_module_ui() {
# devkit-phase: module-ui begin
cd "${WORKSPACE_ROOT}/govoplan-access/webui"
"$NPM" run test:passwords
cd /mnt/DATA/git/govoplan-payments/webui
"$WEBUI_BIN/tsc" -p "${WORKSPACE_ROOT}/govoplan-payments/webui/tsconfig.json"
cd "${WORKSPACE_ROOT}/govoplan-payments/webui"
"$NPM" run test:interface-pattern
cd /mnt/DATA/git/govoplan-dataflow/webui
cd "${WORKSPACE_ROOT}/govoplan-dataflow/webui"
"$NPM" run test:structure
cd /mnt/DATA/git/govoplan-datasources/webui
cd "${WORKSPACE_ROOT}/govoplan-datasources/webui"
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-workflow/webui
cd "${WORKSPACE_ROOT}/govoplan-workflow/webui"
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-dashboard/webui
cd "${WORKSPACE_ROOT}/govoplan-dashboard/webui"
"$NPM" run test:dashboard-layout
cd /mnt/DATA/git/govoplan-approvals/webui
cd "${WORKSPACE_ROOT}/govoplan-approvals/webui"
"$NPM" run test:workspace-layout
cd /mnt/DATA/git/govoplan-postbox/webui
cd "${WORKSPACE_ROOT}/govoplan-postbox/webui"
"$NPM" run test:ui-structure
cd /mnt/DATA/git/govoplan-mail/webui
cd "${WORKSPACE_ROOT}/govoplan-mail/webui"
"$NPM" run test:mail-ui
cd /mnt/DATA/git/govoplan-campaign/webui
cd "${WORKSPACE_ROOT}/govoplan-files/webui"
"$NPM" run test:managed-archive
cd "${WORKSPACE_ROOT}/govoplan-campaign/webui"
"$NPM" run test:policy-ui
"$NPM" run test:template-preview
"$NPM" run test:review-workflow
"$NPM" run test:accessibility-contract
"$NPM" run test:campaign-collaboration
"$NPM" run test:campaign-work
cd /mnt/DATA/git/govoplan-wiki/webui
cd "${WORKSPACE_ROOT}/govoplan-policy/webui"
"$NPM" run test:archive-encryption
cd "${WORKSPACE_ROOT}/govoplan-wiki/webui"
"$NPM" run test:interface-pattern
# devkit-phase: module-ui end
}
# Validate every selected implementation before setup can create a temp npmrc.
while IFS= read -r phase_id; do
if ! declare -F "focused_phase_${phase_id//-/_}" >/dev/null; then
echo "check-focused: missing phase implementation: $phase_id" >&2
exit 2
fi
done <<< "$FOCUSED_SELECTION"
while IFS= read -r phase_id; do
(
focused_setup
"focused_phase_${phase_id//-/_}"
)
done <<< "$FOCUSED_SELECTION"
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env node
/** UI-01: contextual documentation belongs beside text, never in action slots. */
import { existsSync, readFileSync, readdirSync } from "node:fs";
import { createRequire } from "node:module";
import { relative, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const workspaceRoot = resolve(import.meta.dirname, "../../..");
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
const ts = require("typescript");
const titleOwners = new Set(["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]);
const interactiveOwners = new Set(["a", "button", "Button", "IconButton"]);
// Existing domain dialog adapter; its owning structural tests must retain
// forwarding to Core Dialog.titleHelp (not a module-local heading definition).
const domainTitleOwners = new Map([["FileDialog", "/govoplan-files/webui/src/"]]);
export function findDetachedDocumentation(sources) {
const files = new Map(sources.map(({ path, source }) => [resolve(path),
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
const host = ts.createCompilerHost(options);
host.getSourceFile = (path) => files.get(resolve(path));
const checker = ts.createProgram([...files.keys()], options, host).getTypeChecker();
const findings = [];
let links = 0;
for (const [path, source] of files) {
const identifiers = [];
const helpNodes = [];
function importedName(node) {
if (ts.isPropertyAccessExpression(node) && ts.isIdentifier(node.expression)) {
const namespace = checker.getSymbolAtLocation(node.expression)?.declarations?.find(ts.isNamespaceImport);
if (namespace) return node.name.text;
}
const declarations = checker.getSymbolAtLocation(node)?.declarations ?? [];
const declaration = declarations.find(ts.isImportSpecifier);
if (declaration) return (declaration.propertyName ?? declaration.name).text;
const defaultImport = declarations.find(ts.isImportClause);
if (defaultImport && ts.isStringLiteral(defaultImport.parent.moduleSpecifier)) {
const component = defaultImport.parent.moduleSpecifier.text.split("/").at(-1).replace(/\.[cm]?[jt]sx?$/, "");
if (component === "DocumentationHelpLink" || titleOwners.has(component) || component === "TextWithHelp" || interactiveOwners.has(component)) return component;
}
return node.getText(source);
}
function tag(node) {
const opening = ts.isJsxElement(node) ? node.openingElement : ts.isJsxSelfClosingElement(node) ? node : null;
return opening ? importedName(opening.tagName) : null;
}
function collect(node) {
if (ts.isIdentifier(node)) identifiers.push(node);
if ((ts.isJsxSelfClosingElement(node) || ts.isJsxElement(node)) && tag(node) === "DocumentationHelpLink") helpNodes.push(node);
ts.forEachChild(node, collect);
}
collect(source);
function staticallyHidden(opening) {
const hidden = opening.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "hidden");
if (!hidden) return false;
if (!hidden.initializer || ts.isStringLiteral(hidden.initializer)) return true;
return ts.isJsxExpression(hidden.initializer) && hidden.initializer.expression?.kind === ts.SyntaxKind.TrueKeyword;
}
// Reject text that is definitely absent while preserving dynamic translated
// titles and components whose rendered text cannot be established statically.
function emptyText(node, seen = new Set()) {
if (!node) return true;
if (ts.isJsxText(node)) return !node.getText(source).trim();
if (ts.isJsxExpression(node)) return emptyText(node.expression, seen);
if (ts.isStringLiteralLike(node)) return !node.text.trim();
if ([ts.SyntaxKind.NullKeyword, ts.SyntaxKind.FalseKeyword, ts.SyntaxKind.TrueKeyword].includes(node.kind) || ts.isVoidExpression(node)) return true;
if (ts.isParenthesizedExpression(node) || ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isNonNullExpression(node)) return emptyText(node.expression, seen);
if (ts.isIdentifier(node)) {
const symbol = checker.getSymbolAtLocation(node);
const declaration = symbol?.declarations?.find(ts.isVariableDeclaration);
const immutable = declaration && ts.isVariableDeclarationList(declaration.parent) && Boolean(declaration.parent.flags & ts.NodeFlags.Const);
if (immutable && declaration.initializer && !seen.has(symbol)) return emptyText(declaration.initializer, new Set(seen).add(symbol));
return node.text === "undefined" && !symbol?.declarations?.length;
}
if (ts.isConditionalExpression(node)) return emptyText(node.whenTrue, seen) && emptyText(node.whenFalse, seen);
if (ts.isJsxFragment(node)) return node.children.every((child) => emptyText(child, seen));
if (ts.isArrayLiteralExpression(node)) return node.elements.every((child) => emptyText(child, seen));
if (ts.isJsxElement(node) || ts.isJsxSelfClosingElement(node)) {
const opening = ts.isJsxElement(node) ? node.openingElement : node;
if (staticallyHidden(opening)) return true;
if (/^[a-z]/.test(opening.tagName.getText(source))) return !ts.isJsxElement(node) || node.children.every((child) => emptyText(child, seen));
}
return false;
}
function isAnchored(node, seen = new Set()) {
// Check the complete rendered ancestry before returning at a recognized
// slot: the entire heading/text contract may itself be inside a button.
for (let parent = node.parent; parent; parent = parent.parent) {
if (interactiveOwners.has(tag(parent))) return false;
const opening = ts.isJsxElement(parent) ? parent.openingElement : ts.isJsxSelfClosingElement(parent) ? parent : null;
if (opening && staticallyHidden(opening)) return false;
}
for (let parent = node.parent; parent; parent = parent.parent) {
if (ts.isJsxAttribute(parent)) {
const owner = parent.parent.parent;
const name = importedName(owner.tagName);
const slot = parent.name.getText(source);
if (slot === "titleHelp" && (titleOwners.has(name) || (domainTitleOwners.has(name) && path.includes(domainTitleOwners.get(name))))) {
if (name === "PageTitle") {
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
return Boolean(element?.children.some((child) => !emptyText(child)));
}
const title = owner.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "title");
return Boolean(title && !emptyText(title.initializer));
}
if (slot === "help" && name === "TextWithHelp") {
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
return Boolean(element?.children.some((child) => !emptyText(child)));
}
return false;
}
if (ts.isVariableDeclaration(parent) && ts.isIdentifier(parent.name)) {
const symbol = checker.getSymbolAtLocation(parent.name);
if (!symbol || seen.has(symbol)) return false;
const next = new Set(seen).add(symbol);
const references = identifiers.filter((identifier) => identifier !== parent.name && checker.getSymbolAtLocation(identifier) === symbol);
return references.length > 0 && references.every((reference) => isAnchored(reference, next));
}
}
return false;
}
for (const node of helpNodes) {
links += 1;
// FieldLabel is the central label+book implementation; its browser/component
// contract verifies sibling text and prevents nested interactive controls.
if (path.endsWith("/govoplan-core/webui/src/components/help/FieldLabel.tsx")) continue;
if (!isAnchored(node)) {
const position = source.getLineAndCharacterOfPosition(node.getStart(source));
findings.push({ path, line: position.line + 1, column: position.character + 1 });
}
}
}
return { findings, links };
}
function sourceFiles(directory) {
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
const path = resolve(directory, entry.name);
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
});
}
export function checkWorkspace(root = workspaceRoot) {
const modules = readdirSync(root, { withFileTypes: true })
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
const paths = modules.flatMap(sourceFiles);
const { findings, links } = findDetachedDocumentation(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
for (const finding of findings) {
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: UI-01 documentation must use a heading's titleHelp or TextWithHelp beside visible text, not an action slot or detached row.`);
}
if (!findings.length) console.log(`Heading-help contract passed: ${links} documentation links in ${paths.length} TSX files across ${modules.length} WebUI modules.`);
return findings.length ? 1 : 0;
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) process.exitCode = checkWorkspace();
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env node
/** Reject erased type-only imports used as runtime JSX component tags. */
import { readFileSync, readdirSync, existsSync } from "node:fs";
import { createRequire } from "node:module";
import { resolve, relative } from "node:path";
import { fileURLToPath } from "node:url";
const workspaceRoot = resolve(import.meta.dirname, "../../..");
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
const ts = require("typescript");
function isTypeOnlyImport(declaration) {
if (ts.isImportSpecifier(declaration)) return declaration.isTypeOnly || declaration.parent.parent.isTypeOnly;
if (ts.isNamespaceImport(declaration)) return declaration.parent.isTypeOnly;
return (ts.isImportClause(declaration) || ts.isImportEqualsDeclaration(declaration)) && declaration.isTypeOnly;
}
/** Resolve lexical bindings, including shadowing; do not typecheck unrelated
* optional dependencies or report ordinary application diagnostics.
*/
export function findTypeOnlyJsxImports(sources) {
const files = new Map(sources.map(({ path, source }) => [resolve(path),
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
const host = ts.createCompilerHost(options);
host.getSourceFile = (path) => files.get(resolve(path));
const program = ts.createProgram([...files.keys()], options, host);
const checker = program.getTypeChecker();
const findings = [];
for (const [path, source] of files) {
function visit(node) {
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
let root = node.tagName;
// Lower-case direct tags are intrinsic HTML, not runtime bindings.
if (!(ts.isIdentifier(root) && /^[a-z]/.test(root.text))) {
while (ts.isPropertyAccessExpression(root)) root = root.expression;
const declarations = checker.getSymbolAtLocation(root)?.declarations ?? [];
if (declarations.some(isTypeOnlyImport)) {
const position = source.getLineAndCharacterOfPosition(node.tagName.getStart(source));
findings.push({ path, line: position.line + 1, column: position.character + 1, component: node.tagName.getText(source) });
}
}
}
ts.forEachChild(node, visit);
}
visit(source);
}
return findings;
}
function sourceFiles(directory) {
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
const path = resolve(directory, entry.name);
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
});
}
export function checkWorkspace(root = workspaceRoot) {
const modules = readdirSync(root, { withFileTypes: true })
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
const paths = modules.flatMap(sourceFiles);
const findings = findTypeOnlyJsxImports(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
for (const finding of findings) {
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: JSX component ${finding.component} is imported type-only and will be erased at runtime.`);
}
if (!findings.length) console.log(`JSX runtime-import contract passed: ${paths.length} TSX files across ${modules.length} WebUI modules.`);
return findings.length ? 1 : 0;
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
process.exitCode = checkWorkspace();
}
@@ -4,6 +4,7 @@ set -euo pipefail
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
ROOT="$(cd "$ROOT" && pwd)"
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$ROOT")}"
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
NPM="${NPM:-/home/zemion/.nvm/versions/node/v22.22.3/bin/npm}"
@@ -184,6 +185,7 @@ run_step "Validate installed module manifests and registry"
run_step "Validate platform interface and endpoint declarations"
"$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \
--workspace-root "$WORKSPACE_ROOT" \
--strict-declarations \
--strict-endpoints
+3
View File
@@ -677,6 +677,7 @@ run_gitleaks() {
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
gitleaks git \
--redact=100 \
--config "$ROOT/.gitleaks.toml" \
--report-format json \
--report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
@@ -687,6 +688,7 @@ run_gitleaks() {
# Scan the directory as well so pre-commit audits cover the exact code
# under review, while retaining the history scan above.
gitleaks dir \
--redact=100 \
--config "$ROOT/.gitleaks.toml" \
--report-format json \
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
@@ -696,6 +698,7 @@ run_gitleaks() {
else
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
gitleaks detect \
--redact=100 \
--source "$repo" \
--config "$ROOT/.gitleaks.toml" \
--report-format json \
@@ -86,6 +86,12 @@ CENTRAL_COMPONENTS = {
"CountBadge": pathlib.Path(
"govoplan-core/webui/src/components/CountBadge.tsx"
),
"MultiSelectFilter": pathlib.Path(
"govoplan-core/webui/src/components/MultiSelectFilter.tsx"
),
"ListSelectionFilter": pathlib.Path(
"govoplan-core/webui/src/components/ListSelectionFilter.tsx"
),
"SelectionList": pathlib.Path(
"govoplan-core/webui/src/components/SelectionList.tsx"
),
@@ -190,7 +196,21 @@ REQUIRED_CONSUMERS = {
"CountBadge": (
pathlib.Path("govoplan-core/webui/src/layout/Titlebar.tsx"),
pathlib.Path("govoplan-mail/webui/src/features/mail/MailboxPage.tsx"),
),
# Search's old count badge was part of a retired module-local filter menu.
# Both surfaces must now compose the owning facet adapter and Core dropdown.
"SearchFilters": (
pathlib.Path("govoplan-search/webui/src/features/search/SearchPage.tsx"),
pathlib.Path("govoplan-search/webui/src/components/GlobalSearch.tsx"),
),
"MultiSelectFilter": (
pathlib.Path("govoplan-search/webui/src/components/SearchFilters.tsx"),
pathlib.Path("govoplan-notifications/webui/src/features/notifications/NotificationCenterPage.tsx"),
pathlib.Path("govoplan-docs/webui/src/features/docs/DocsPage.tsx"),
),
"ListSelectionFilter": (
pathlib.Path("govoplan-core/webui/src/components/MultiSelectFilter.tsx"),
pathlib.Path("govoplan-core/webui/src/components/table/DataGrid.tsx"),
),
"SelectionListItemContent": (
pathlib.Path("govoplan-approvals/webui/src/features/approvals/ApprovalsPage.tsx"),
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env python3
"""Check that release Git dependencies expose their owning WebUI package."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
META_ROOT = Path(__file__).resolve().parents[2]
GIT_REPOSITORY = re.compile(r"/(govoplan-[a-z0-9-]+)\.git#v[^/]+$")
PARITY_FIELDS = (
"name", "version", "type", "dependencies", "optionalDependencies",
"peerDependencies", "peerDependenciesMeta",
)
ENTRY_FIELDS = ("main", "module", "types", "exports")
def prefixed_entries(value: object) -> object:
if isinstance(value, str):
return "./webui/" + value[2:] if value.startswith("./") else "webui/" + value
if isinstance(value, dict):
return {key: prefixed_entries(item) for key, item in value.items()}
if isinstance(value, list):
return [prefixed_entries(item) for item in value]
return value
def entry_paths(value: object) -> list[str]:
if isinstance(value, str):
return [value]
if isinstance(value, dict):
return [path for item in value.values() for path in entry_paths(item)]
if isinstance(value, list):
return [path for item in value for path in entry_paths(item)]
return []
def facade_issues(repository: Path, *, package_name: str) -> list[str]:
issues: list[str] = []
try:
root = json.loads((repository / "package.json").read_text(encoding="utf-8"))
webui = json.loads((repository / "webui/package.json").read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
return [f"{repository.name}: cannot read package facades: {exc}"]
if not isinstance(root, dict) or not isinstance(webui, dict):
return [f"{repository.name}: package manifests must be JSON objects"]
if webui.get("name") != package_name:
issues.append(f"{repository.name}: WebUI name does not match {package_name}")
for field in PARITY_FIELDS:
if root.get(field) != webui.get(field):
issues.append(f"{repository.name}: root {field} differs from owning WebUI package")
for field in ENTRY_FIELDS:
expected = prefixed_entries(webui.get(field))
if root.get(field) != expected:
issues.append(f"{repository.name}: root {field} must target the corresponding webui/ entry")
for entry in entry_paths(root.get(field)):
path = repository / entry
if not path.resolve().is_relative_to((repository / "webui").resolve()):
issues.append(f"{repository.name}: {field} entry escapes webui/: {entry}")
elif "*" not in entry and not path.is_file():
issues.append(f"{repository.name}: missing {field} entry: {entry}")
if not root.get("exports") and not root.get("main"):
issues.append(f"{repository.name}: root package has no WebUI entry point")
return issues
def check_composition(workspace: Path, *, core_root: Path | None = None) -> tuple[int, list[str]]:
core = core_root or workspace / "govoplan-core"
release = json.loads((core / "webui/package.release.json").read_text(encoding="utf-8"))
checked = 0
issues: list[str] = []
for name, reference in release.get("dependencies", {}).items():
if not name.startswith("@govoplan/"):
continue
match = GIT_REPOSITORY.search(reference) if isinstance(reference, str) else None
if match is None:
issues.append(f"{name}: release dependency is not a versioned GovOPlaN Git source")
continue
checked += 1
issues.extend(facade_issues(workspace / match.group(1), package_name=name))
return checked, issues
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace-root", type=Path, default=META_ROOT.parent)
parser.add_argument("--core-root", type=Path)
args = parser.parse_args()
checked, issues = check_composition(args.workspace_root, core_root=args.core_root)
if issues:
print("\n".join(issues))
return 1
print(f"Release WebUI package facade checks passed for {checked} Git dependencies")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -19,7 +19,6 @@
"resources": "Contract-only module: resource catalog and allocation persistence are not implemented; reassess before adding a migration-owned store.",
"rest": "Transport-only module: REST binds explicitly published functions and owns no domain or subject-data store.",
"soap": "Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store.",
"tenancy": "Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data.",
"transparency": "Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store.",
"workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage.",
"xrechnung": "Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store."
+142
View File
@@ -0,0 +1,142 @@
{
"schema_version": 1,
"phases": [
{
"id": "preflight",
"title": "Dependency and shared contract preflight",
"cwd": "core",
"order_after": [],
"depends_on": [],
"resources": [
"backend:test-state"
],
"outputs": [],
"notes": [
"Checks dependency hygiene, cross-module contracts, manifests and static WebUI conventions."
]
},
{
"id": "tooling",
"title": "Workspace inventories and development/release tooling tests",
"cwd": "meta",
"order_after": [
"preflight"
],
"depends_on": [],
"resources": [
"backend:test-state",
"artifact:platform-inventory"
],
"outputs": [
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.json",
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.md"
],
"notes": [
"Inventory reports are regenerated here; no later phase consumes them. Reused check evidence does not promise these reports still exist."
]
},
{
"id": "backend",
"title": "Workspace syntax, backend modules and integration checks",
"cwd": "core",
"order_after": [
"tooling"
],
"depends_on": [],
"resources": [
"backend:test-state"
],
"outputs": [],
"notes": [
"Tests may create their own temporary fixtures or test caches; no generated artifact is required by a later phase."
]
},
{
"id": "core-ui",
"title": "Core source tests and selected component contracts",
"cwd": "core-webui",
"order_after": [
"backend"
],
"depends_on": [],
"resources": [
"webui:govoplan-core"
],
"outputs": [
"{core}/webui/.module-test-build"
],
"notes": [
"The selected component batch creates and removes a private compilation directory. Module-capability tests rebuild .module-test-build themselves; later phases do not consume it."
]
},
{
"id": "module-builds",
"title": "Optional-module build permutations and bundle checks",
"cwd": "core-webui",
"order_after": [
"core-ui"
],
"depends_on": [],
"resources": [
"webui:govoplan-core"
],
"outputs": [
"{core}/webui/dist",
"{core}/webui/dist/module-permutation-bundle-metrics.json"
],
"notes": [
"Each permutation builds and reads its own fresh bundle metrics. Final dist and aggregate metrics persist, but browser/module-ui phases do not consume them. Reused check evidence is not artifact or deployment attestation."
]
},
{
"id": "browser",
"title": "Core browser conformance",
"cwd": "core-webui",
"order_after": [
"module-builds"
],
"depends_on": [],
"resources": [
"webui:govoplan-core",
"browser:chromium",
"port:4174"
],
"outputs": [
"{core}/webui/test-results",
"{core}/webui/node_modules/.vite/govoplan-conformance"
],
"notes": [
"Playwright starts its own Vite server from conformance sources with an isolated cache. It does not serve or require module-builds dist. The server is owned by the phase and stopped on completion."
]
},
{
"id": "module-ui",
"title": "Owning-module UI and type checks",
"cwd": "access-webui",
"order_after": [
"browser"
],
"depends_on": [],
"resources": [
"webui:govoplan-core",
"webui:govoplan-access",
"webui:govoplan-payments",
"webui:govoplan-dataflow",
"webui:govoplan-datasources",
"webui:govoplan-workflow",
"webui:govoplan-dashboard",
"webui:govoplan-approvals",
"webui:govoplan-postbox",
"webui:govoplan-mail",
"webui:govoplan-files",
"webui:govoplan-campaign",
"webui:govoplan-policy",
"webui:govoplan-wiki"
],
"outputs": [],
"notes": [
"The original 19-command tail retains each owning package's cwd. Its scripts prepare any private test output they need; no earlier phase artifact is a prerequisite."
]
}
]
}
+4 -1
View File
@@ -13,7 +13,10 @@ from urllib.parse import urlsplit
SCHEMA_VERSION = 1
DEFAULT_GARAGE_IMAGE = "dxflrs/garage:v2.3.0"
DEFAULT_LOAD_BALANCER_IMAGE = "haproxy:3.2.21-alpine"
DEFAULT_LOAD_BALANCER_IMAGE = (
"haproxy:3.2.23-alpine@sha256:"
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e"
)
DEFAULT_INGRESS_IMAGE = "caddy:2.10.2-alpine"
INSTALLATION_ID_PATTERN = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
ENV_NAME_PATTERN = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env node
// Complement the existing i18n-marker inventory with known plain-text display
// slots. Parse source only: never execute module registration/catalog code.
import { existsSync, readFileSync, readdirSync, realpathSync } from "node:fs";
import { createRequire } from "node:module";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const displayProps = new Map([
...["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"].map((name) => [name, new Set(["title", "subtitle"])]),
["FieldLabel", new Set(["label"])], ["MetricCard", new Set(["label"])],
]);
const childOwners = new Set(["PageTitle", "TextWithHelp", "h1", "h2", "h3", "h4", "h5", "h6"]);
const unknown = Symbol("dynamic");
export function createReader(ts, allowedRoot) {
const sources = new Map();
function source(file) {
file = resolve(file);
if (file !== allowedRoot && !file.startsWith(`${resolve(allowedRoot)}/`)) return null;
if (!existsSync(file)) return null;
if (!realpathSync(file).startsWith(`${resolve(allowedRoot)}/`)) return null;
if (!sources.has(file)) sources.set(file, ts.createSourceFile(file, readFileSync(file, "utf8"), ts.ScriptTarget.Latest, true, file.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS));
return sources.get(file);
}
function unwrap(node) {
while (node && (ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isParenthesizedExpression(node))) node = node.expression;
return node;
}
function imported(sf, identifier) {
for (const item of sf.statements) {
if (!ts.isImportDeclaration(item) || !item.importClause || !ts.isStringLiteral(item.moduleSpecifier)) continue;
const names = item.importClause.namedBindings;
if (names && ts.isNamedImports(names)) {
const match = names.elements.find((entry) => entry.name.text === identifier);
if (match) return { path: item.moduleSpecifier.text, name: match.propertyName?.text ?? match.name.text };
}
if (item.importClause.name?.text === identifier) return { path: item.moduleSpecifier.text, name: "default" };
}
return null;
}
function targetFile(sf, specifier) {
if (!specifier.startsWith(".")) return null;
const base = resolve(dirname(sf.fileName), specifier);
return [base, `${base}.ts`, `${base}.tsx`, join(base, "index.ts")].find((file) => existsSync(file) && /\.tsx?$/.test(file)) ?? null;
}
function declaration(sf, name) {
for (const statement of sf.statements) {
if (ts.isVariableStatement(statement) && statement.declarationList.flags & ts.NodeFlags.Const) {
const item = statement.declarationList.declarations.find((node) => ts.isIdentifier(node.name) && node.name.text === name);
if (item) return item.initializer;
}
if (name === "default" && ts.isExportAssignment(statement)) return statement.expression;
}
return null;
}
function value(node, sf, seen = new Set()) {
node = unwrap(node);
if (!node) return unknown;
if (ts.isStringLiteralLike(node)) return node.text;
if (ts.isIdentifier(node)) {
const key = `${sf.fileName}:${node.text}`;
if (seen.has(key)) return unknown;
const visited = new Set([...seen, key]);
const local = declaration(sf, node.text);
if (local) return value(local, sf, visited);
const external = imported(sf, node.text);
const target = external && targetFile(sf, external.path);
const loaded = target && source(target);
return loaded ? value(declaration(loaded, external.name), loaded, visited) : unknown;
}
if (ts.isObjectLiteralExpression(node)) {
const result = {};
for (const property of node.properties) {
if (ts.isSpreadAssignment(property)) {
const spread = value(property.expression, sf, seen);
if (spread !== unknown && spread && typeof spread === "object") Object.assign(result, spread);
else result.__dynamicSpread = true;
} else if (ts.isPropertyAssignment(property)) {
result[property.name.text ?? property.name.getText(sf)] = value(property.initializer, sf, seen);
} else if (ts.isShorthandPropertyAssignment(property)) result[property.name.text] = value(property.name, sf, seen);
}
return result;
}
return unknown;
}
return { source, value, declaration, imported, unwrap };
}
function componentName(ts, reader, sf, node) {
const written = node.getText(sf);
if (/^h[1-6]$/.test(written)) return written;
if (ts.isIdentifier(node)) {
const imported = reader.imported(sf, written);
if (imported?.name === "default") return imported.path.split("/").at(-1).replace(/\.[tj]sx?$/, "");
return imported?.name ?? written;
}
if (ts.isPropertyAccessExpression(node)) {
const owner = node.expression.getText(sf);
const namespace = sf.statements.find((item) => ts.isImportDeclaration(item) && item.importClause?.namedBindings && ts.isNamespaceImport(item.importClause.namedBindings) && item.importClause.namedBindings.name.text === owner);
if (namespace) return node.name.text;
}
return written;
}
function sourceFiles(directory) {
if (!existsSync(directory)) return [];
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
if (entry.name.startsWith(".") || entry.name === "node_modules") return [];
const file = join(directory, entry.name);
if (entry.isDirectory()) return sourceFiles(file);
return entry.isFile() && /\.[tj]sx?$/.test(file) ? [file] : [];
});
}
export function auditRepository(ts, repositoryRoot, coreRoot) {
const reader = createReader(ts, repositoryRoot);
const coreReader = createReader(ts, coreRoot);
const coreFile = coreReader.source(join(coreRoot, "webui/src/i18n/generatedTranslations.ts"));
const coreCatalog = coreFile ? coreReader.value(coreReader.declaration(coreFile, "generatedTranslations"), coreFile) : {};
const moduleFile = reader.source(join(repositoryRoot, "webui/src/module.ts"));
let moduleCatalog = {};
let registration = repositoryRoot === coreRoot ? "core-default" : "absent";
if (moduleFile) {
for (const statement of moduleFile.statements) {
if (!ts.isVariableStatement(statement) || !statement.modifiers?.some((item) => item.kind === ts.SyntaxKind.ExportKeyword)) continue;
for (const node of statement.declarationList.declarations) {
if (!node.initializer || !(/PlatformWebModule/.test(node.type?.getText(moduleFile) ?? "") || /Module$/.test(node.name.getText(moduleFile)))) continue;
const evaluated = reader.value(node.initializer, moduleFile);
if (evaluated && typeof evaluated === "object" && Object.hasOwn(evaluated, "translations")) {
moduleCatalog = evaluated.translations;
registration = moduleCatalog !== unknown && moduleCatalog && typeof moduleCatalog === "object" &&
!moduleCatalog.__dynamicSpread && !moduleCatalog.en?.__dynamicSpread && !moduleCatalog.de?.__dynamicSpread ? "registered" : "dynamic";
}
}
}
}
const findings = [], review = [], labels = [];
const add = (text, node, sf, slot) => {
const position = sf.getLineAndCharacterOfPosition(node.getStart(sf));
const location = { file: sf.fileName, line: position.line + 1, slot };
if (typeof text !== "string") {
review.push({ ...location, code: "dynamic-display-slot", message: "Runtime data or computed text: verify with the owning module/locale context." });
return;
}
text = text.replace(/\s+/g, " ").trim();
if (!text || text.startsWith("i18n:") || !/[\p{L}]/u.test(text)) return;
const missing = ["en", "de"].filter((locale) => {
const translated = moduleCatalog?.[locale]?.[text] ?? coreCatalog?.[locale]?.[text];
return typeof translated !== "string" || !translated.trim();
});
labels.push({ ...location, text, missing_locales: missing });
if (missing.length) {
const item = { ...location, code: "plain-label-missing-translation", text, missing_locales: missing, registration };
if (registration === "dynamic") review.push({ ...item, code: "dynamic-catalog-review" });
else findings.push(item);
}
};
for (const file of sourceFiles(join(repositoryRoot, "webui/src"))) {
if (file.includes("/i18n/")) continue;
const sf = reader.source(file);
function textChild(node, owner) {
if (ts.isJsxText(node)) add(node.text, node, sf, `${owner}.children`);
else if (ts.isJsxExpression(node)) { if (node.expression) add(reader.value(node.expression, sf), node, sf, `${owner}.children`); }
else if (ts.isJsxElement(node) || ts.isJsxFragment(node)) for (const child of node.children) textChild(child, owner);
}
function visit(node) {
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
const owner = componentName(ts, reader, sf, node.tagName);
for (const property of node.attributes.properties) {
if (!ts.isJsxAttribute(property) || !displayProps.get(owner)?.has(property.name.text) || !property.initializer) continue;
const expression = ts.isJsxExpression(property.initializer) ? property.initializer.expression : property.initializer;
add(reader.value(expression, sf), property, sf, `${owner}.${property.name.text}`);
}
}
if (ts.isJsxElement(node)) {
const owner = componentName(ts, reader, sf, node.openingElement.tagName);
if (childOwners.has(owner)) for (const child of node.children) textChild(child, owner);
}
ts.forEachChild(node, visit);
}
visit(sf);
}
const hasCatalog = sourceFiles(join(repositoryRoot, "webui/src/i18n")).some((file) => /Translations\.ts$/.test(file));
if (hasCatalog && registration === "absent") findings.push({ file: moduleFile?.fileName ?? join(repositoryRoot, "webui/src/module.ts"), line: 1, code: "catalog-not-registered", message: "Module-owned catalog exists but no static module translations registration was found." });
return { repository: repositoryRoot, registration, labels, findings, review };
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
const args = process.argv.slice(2);
let workspace = resolve(dirname(fileURLToPath(import.meta.url)), "../../../..");
const repos = [];
for (let index = 0; index < args.length; index++) {
if (args[index] === "--workspace-root") workspace = resolve(args[++index]);
else if (args[index] === "--repo") repos.push(args[++index]);
else throw new Error(`Unknown argument: ${args[index]}`);
}
const core = join(workspace, "govoplan-core");
const require = createRequire(join(core, "webui/package.json"));
const ts = require("typescript");
const catalog = JSON.parse(readFileSync(join(workspace, "govoplan/repositories.json"), "utf8"));
const selected = catalog.repositories.filter((repo) => !repos.length || repos.includes(repo.name));
if (repos.some((name) => !selected.some((repo) => repo.name === name))) throw new Error("Unknown repository selection");
for (const repo of selected) {
if (typeof repo.path !== "string" || !resolve(workspace, repo.path).startsWith(`${workspace}/`)) throw new Error("Repository path escapes the workspace");
}
const results = selected.filter((repo) => existsSync(join(workspace, repo.path, "webui/src"))).map((repo) => auditRepository(ts, join(workspace, repo.path), core));
const findings = results.reduce((total, item) => total + item.findings.length, 0);
process.stdout.write(JSON.stringify({ schema_version: 1, results, finding_count: findings,
limitations: ["Known display slots only; this is not a complete UI or linguistic review.", "Runtime data, computed labels and dynamic registrations require manual review.", "Core defaults plus each owning module are checked; optional sibling catalogs cannot mask missing registration.", "Explicit i18n markers are checked by the existing platform interface inventory."] }) + "\n");
process.exitCode = findings ? 1 : 0;
}
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env python3
"""Command-line entry point; use the repository-root ./devkit launcher."""
from govoplan_devkit.cli import main
if __name__ == "__main__":
raise SystemExit(main())
+32
View File
@@ -0,0 +1,32 @@
{
"schema_version": 1,
"name": "Example Python project",
"repositories": [{"name": "app", "path": "."}],
"checks": [
{
"id": "whitespace",
"title": "Git whitespace check",
"argv": ["git", "diff", "--check"],
"cwd": ".",
"repos": ["app"],
"inputs": {"repos": ["app"]},
"timeout_seconds": 30
},
{
"id": "unit-tests",
"title": "Python unit tests",
"argv": ["{python}", "-m", "unittest", "discover", "-s", "tests"],
"cwd": ".",
"repos": ["app"],
"inputs": {"repos": ["app"]},
"after": ["whitespace"],
"resources": ["test-database"],
"timeout_seconds": 300
}
],
"profiles": {
"quick": ["whitespace", "unit-tests"],
"backend": ["unit-tests"],
"full": ["whitespace", "unit-tests"]
}
}
+3
View File
@@ -0,0 +1,3 @@
"""Small, deterministic development commands for people, CI and coding agents."""
__version__ = "1.0.0"
+566
View File
@@ -0,0 +1,566 @@
"""Test planning over existing checks; planning never executes a test or server."""
from __future__ import annotations
from pathlib import Path
from copy import deepcopy
from itertools import islice
import re
from .common import read_json
from .package_tests import declared_tests, discovered_sources, source_stage_id
PROFILES = ("quick", "ui", "backend", "full")
def stage(
identity: str,
title: str,
argv: list[str],
cwd: Path,
*,
reason: str,
deps: list[str] | None = None,
after: list[str] | None = None,
resources: list[str] | None = None,
timeout_seconds: int = 300,
) -> dict:
return {
"id": identity,
"title": title,
"argv": argv,
"cwd": str(cwd),
"deps": deps or [],
"after": after or [],
"resources": resources or [],
"timeout_seconds": timeout_seconds,
"reason": reason,
}
def _custom_stages(
project, workspace_root: Path, profile: str, selected, filtered: bool
) -> list[dict]:
# load_project validates every nested declaration before selection.
records = project.config.get("checks", [])
checks = {item["id"]: item for item in records}
profiles = project.config.get("profiles", {})
if profile not in profiles:
raise ValueError(
f"Project {project.name!r} does not declare profile {profile!r}"
)
selected_names = {repo.name for repo in selected}
wanted = set()
def include(identity: str, visiting: frozenset[str] = frozenset()) -> None:
if identity not in checks:
raise ValueError(f"Unknown check dependency: {identity}")
if identity in visiting:
raise ValueError(f"Cyclic check dependency: {identity}")
if identity in wanted:
return
for dependency in [
*checks[identity].get("deps", []),
*checks[identity].get("after", []),
]:
include(dependency, visiting | {identity})
wanted.add(identity)
for identity in profiles[profile]:
if identity not in checks:
raise ValueError(f"Unknown profile check: {identity}")
owned = set(checks[identity].get("repos", []))
if not filtered or not owned or selected_names & owned:
include(identity)
result = []
# A dependency may precede/follow its consumer in the configuration: the
# execution engine owns scheduling, not JSON declaration order.
for identity, item in checks.items():
if identity in wanted:
result.append(
stage(
identity,
item.get("title", identity),
list(item["argv"]),
workspace_root / item.get("cwd", "."),
deps=list(item.get("deps", [])),
after=list(item.get("after", [])),
resources=list(item.get("resources", [])),
timeout_seconds=item.get("timeout_seconds", 300),
reason=f"Project profile {profile}",
)
)
for field in ("inputs", "reuse"):
if field in item:
result[-1][field] = deepcopy(item[field])
return result
def focused_phases(meta: Path) -> list[dict]:
"""Read the same bounded, ordered phase metadata as the standalone gate."""
value = read_json(meta / "tools/checks/focused-phases.json", max_bytes=1024 * 1024)
if (
not isinstance(value, dict)
or set(value) != {"schema_version", "phases"}
or type(value["schema_version"]) is not int
or value["schema_version"] != 1
):
raise ValueError("Unsupported focused phase metadata schema")
phases = value["phases"]
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
raise ValueError("Focused phase catalog requires 132 definitions")
seen = set()
fields = {
"id",
"title",
"cwd",
"order_after",
"depends_on",
"resources",
"outputs",
"notes",
}
for phase in phases:
if not isinstance(phase, dict) or set(phase) != fields:
raise ValueError("Invalid focused phase metadata fields")
identity = phase["id"]
if (
not isinstance(identity, str)
or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity)
or identity in seen
):
raise ValueError("Invalid or duplicate focused phase ID")
if (
not isinstance(phase["title"], str)
or not phase["title"].strip()
or len(phase["title"]) > 256
or any(ord(char) < 32 for char in phase["title"])
):
raise ValueError("Invalid focused phase title")
if not isinstance(phase["cwd"], str) or phase["cwd"] not in {
"core",
"meta",
"core-webui",
"access-webui",
}:
raise ValueError("Unsupported focused phase working directory")
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
values = phase[field]
if (
not isinstance(values, list)
or len(values) > 64
or any(
not isinstance(item, str)
or not item
or len(item) > 4096
or "\0" in item
for item in values
)
or len(set(values)) != len(values)
):
raise ValueError("Invalid focused phase list: " + field)
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
raise ValueError("Focused prerequisites must precede their consumer")
seen.add(identity)
return phases
def focused_phase_bodies(text: str, phases: list[dict]) -> dict[str, str]:
"""Only exact top-level registered wrappers are authoritative phase bodies.
Do not extract lookalike markers from heredocs, conditionals or unrelated
shell functions. This recognizes the maintained wrapper convention, not
arbitrary executable shell semantics.
"""
registered = {
"focused_phase_" + phase["id"].replace("-", "_"): phase["id"]
for phase in phases
}
lines, bodies, index, depth, heredoc = text.splitlines(), {}, 0, 0, None
while index < len(lines):
line = lines[index]
stripped = line.strip()
if heredoc is not None:
if stripped == heredoc:
heredoc = None
index += 1
continue
match = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
if match:
heredoc = match[1]
index += 1
continue
function = re.fullmatch(r"(focused_phase_[a-z0-9_]+)\(\) \{", line)
if depth == 0 and function and function[1] in registered:
identity = registered[function[1]]
if (
identity in bodies
or index + 1 >= len(lines)
or lines[index + 1] != f"# devkit-phase: {identity} begin"
):
raise ValueError("Invalid or duplicate focused phase wrapper")
end = index + 2
while end < len(lines) and lines[end] != f"# devkit-phase: {identity} end":
end += 1
if end + 1 >= len(lines) or lines[end + 1] != "}":
raise ValueError("Unclosed focused phase wrapper")
bodies[identity] = "\n".join(lines[index + 2 : end]) + "\n"
index = end + 2
continue
if re.match(
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
stripped,
):
depth += 1
elif re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
depth = max(0, depth - 1)
index += 1
if set(bodies) != {phase["id"] for phase in phases}:
raise ValueError("Focused phase metadata and marked implementations differ")
return {phase["id"]: bodies[phase["id"]] for phase in phases}
def _undeclared_source_note(project, workspace_root: Path) -> str | None:
"""Directory discovery is wider than registered Git input ownership."""
registered_paths = {repo.path.resolve() for repo in project.repositories}
children = list(islice(workspace_root.iterdir(), 4097))
if len(children) > 4096:
return "Workspace discovery exceeds its bounded ownership audit; native reuse is disabled."
for child in children:
if not child.name.startswith("govoplan") or child.resolve() in registered_paths:
continue
if any(
(child / name).exists() or (child / name).is_symlink()
for name in ("src", "webui")
):
return "Unregistered sibling src/WebUI inputs may be consumed by workspace discovery or PYTHONPATH; native reuse is disabled until their repository ownership is declared."
return None
def _apply_undeclared_source_note(checks: list[dict], note: str | None) -> None:
if note:
for check in checks:
check["reuse"] = "never"
check.setdefault("coverage_notes", []).append(note)
def _focused_ui_inputs(project) -> tuple[list[str] | None, str]:
names = {repo.name for repo in project.repositories}
if not {"govoplan", "govoplan-core"} <= names:
return None, "Missing Core/Meta ownership; input scope stays workspace-wide."
selected = {"govoplan", "govoplan-core"}
for repo in project.repositories:
root, webui = repo.path, repo.path / "webui"
if (
not root.is_dir()
or root.is_symlink()
or webui.is_symlink()
or webui.exists()
and not webui.is_dir()
):
return (
None,
"Missing or ambiguous repository/WebUI layout; input scope stays workspace-wide.",
)
if webui.is_dir():
selected.add(repo.name)
return (
sorted(selected),
"UI input scope includes whole Core/Meta and every registered WebUI repository, including helpers/configuration; it is not per-file dependency inference.",
)
def _apply_ui_inputs(check: dict, scope: tuple[list[str] | None, str]) -> None:
names, note = scope
if names is not None:
check["inputs"] = {"repos": list(names)}
check.setdefault("coverage_notes", []).append(note)
def _full_stages(project, workspace_root: Path, meta: Path, core: Path) -> list[dict]:
phases = focused_phases(meta)
ui_scope = _focused_ui_inputs(project)
directories = {
"core": core,
"meta": meta,
"core-webui": core / "webui",
"access-webui": workspace_root / "govoplan-access/webui",
}
checks = []
for phase in phases:
check = stage(
"focused." + phase["id"],
phase["title"],
[
"bash",
str(meta / "tools/checks/check-focused.sh"),
"--phase",
phase["id"],
],
directories[phase["cwd"]],
reason="Full retains every canonical phase in order; repository filters never narrow the required gate.",
deps=["focused." + value for value in phase["depends_on"]],
after=["focused." + value for value in phase["order_after"]],
resources=list(dict.fromkeys(["workspace:focused", *phase["resources"]])),
timeout_seconds=14400,
)
check["coverage_notes"] = list(phase["notes"])
check["phase_outputs"] = list(phase["outputs"])
if phase["id"] in {"core-ui", "module-builds", "browser", "module-ui"}:
check["resources"] = list(
dict.fromkeys(
[
*check["resources"],
*[f"webui:{repo.name}" for repo in project.repositories],
]
)
)
_apply_ui_inputs(check, ui_scope)
else:
check["coverage_notes"].append(
"Cross-module backend/tooling checks retain conservative whole-workspace inputs."
)
checks.append(check)
_apply_undeclared_source_note(
checks, _undeclared_source_note(project, workspace_root)
)
return checks
def _expanded_repositories(project, selected, *, changed: bool):
"""Conservative shared changes; declared interface consumers otherwise.
This is a selection aid, not a claim of exhaustive runtime dependency
analysis. The full profile always retains the canonical workspace gate.
"""
if not changed or not selected:
return selected, "explicit selection" if selected else "no changed repositories"
names = {repo.name for repo in selected}
if names & {"govoplan", "govoplan-core"}:
return list(
project.repositories
), "Core/Meta changed; all registered consumers conservatively selected"
# Reuse the release contract parser without importing module application
# code. If available declarations cannot be parsed, broaden selection.
import sys
meta = next(
(repo.path for repo in project.repositories if repo.name == "govoplan"), None
)
if meta is None:
return selected, "changed repositories; no GovOPlaN contract catalog"
release = meta / "tools" / "release"
if not (release / "govoplan_release" / "contracts.py").is_file():
return selected, "changed repositories; contract parser unavailable"
sys.path.insert(0, str(release))
try:
from govoplan_release.contracts import parse_manifest_contract
contracts = []
for repo in project.repositories:
for manifest in sorted((repo.path / "src").glob("*/backend/manifest.py")):
parsed = parse_manifest_contract(manifest, repo_name=repo.name)
if parsed is None:
return list(
project.repositories
), "unresolved manifest contract; conservative workspace selection"
contracts.append(parsed)
while True:
providers = {
item.name
for contract in contracts
if contract.repo in names
for item in contract.provides_interfaces
}
consumers = {
contract.repo
for contract in contracts
if any(item.name in providers for item in contract.requires_interfaces)
}
added = consumers - names
if not added:
break
names.update(added)
except (ImportError, AttributeError, OSError, SyntaxError, ValueError):
return list(
project.repositories
), "contract analysis unavailable; conservative workspace selection"
finally:
sys.path.remove(str(release))
return [
repo for repo in project.repositories if repo.name in names
], "changed repositories plus declared interface consumers"
def _module_ui_plan(repo, *, reason: str) -> tuple[list[dict], list[str]]:
"""Use package-owned direct Node test metadata, excluding shell/build chains.
Source structural scripts are also discoverable by the existing established
names. Unknown shell commands are deliberately not guessed or rewritten.
"""
webui = repo.path / "webui"
package_path = webui / "package.json"
if not package_path.is_file():
return [], []
scripts: dict[tuple[str, ...], list[str]] = {}
omitted = []
for item in [*declared_tests(repo, package_path), *discovered_sources(repo)]:
if item["component_suite"] is not None:
omitted.append(
f"{repo.name} {item['name']}: only covered by the explicit UI component batch; quick does not compile components"
)
elif item["name"] in {"test:module-permutations", "test:vite-cache-isolation"}:
omitted.append(
f"{repo.name} {item['name']}: separate environment/permutation verification, not run by this scoped profile"
)
elif item["_argv"]:
scripts[tuple(item["_argv"])] = item["_argv"]
else:
omitted.append(f"{repo.name} {item['name']}: {item['reason']}")
return [
stage(
source_stage_id(repo.name, argv),
f"{repo.name}: {Path(argv[-1]).stem}",
argv,
webui,
reason=reason,
resources=[f"webui:{repo.name}"],
)
for _, argv in sorted(scripts.items())
], omitted
def module_ui_stages(repo, *, reason: str) -> list[dict]:
return _module_ui_plan(repo, reason=reason)[0]
def build_stages(
workspace_root: Path,
profile: str,
repos: list[str],
changed: bool,
project: Path | None = None,
) -> list[dict]:
from .workspace import load_project, selected_repositories
if profile not in PROFILES:
raise ValueError(f"Unknown check profile: {profile}")
workspace_root = workspace_root.resolve()
loaded = load_project(workspace_root, project)
selected = selected_repositories(loaded, repos, changed=changed)
if project is not None:
return _custom_stages(
loaded, workspace_root, profile, selected, bool(repos or changed)
)
selected, reason = _expanded_repositories(loaded, selected, changed=changed)
mapping = {repo.name: repo.path for repo in loaded.repositories}
meta = mapping.get("govoplan", workspace_root / "govoplan")
core = mapping.get("govoplan-core", workspace_root / "govoplan-core")
if profile == "full":
return _full_stages(loaded, workspace_root, meta, core)
if changed and not selected:
return []
checks = []
for identity, command in (
(
"contracts",
[
"{python}",
str(meta / "tools/checks/check-contracts.py"),
"--workspace-root",
str(workspace_root),
"--no-impact",
],
),
(
"manifests",
[
"{python}",
str(meta / "tools/checks/check-manifest-shapes.py"),
"--workspace-root",
str(workspace_root),
"--require-architecture",
],
),
):
checks.append(
stage(
identity,
f"Workspace {identity}",
command,
meta,
reason="Shared manifest/interface invariants",
timeout_seconds=600,
)
)
if profile in {"quick", "ui"}:
ui_scope = _focused_ui_inputs(loaded)
coverage_notes = [
"Scoped source/component checks are not a complete module review or the full focused gate."
]
for identity in ("jsx-value-imports", "heading-help"):
checks.append(
stage(
identity,
f"Shared {identity} contract",
["{node}", str(meta / f"tools/checks/check-{identity}.mjs")],
meta,
reason="Existing source-only cross-module guard",
)
)
_apply_ui_inputs(checks[-1], ui_scope)
for repo in selected:
stages, omissions = _module_ui_plan(repo, reason=reason)
checks.extend(stages)
coverage_notes.extend(omissions)
checks[0]["coverage_notes"] = coverage_notes
if profile == "ui":
checks.append(
stage(
"core.component-batch",
"Core component suites (compile once)",
["{node}", str(core / "webui/scripts/run-component-tests.mjs")],
core / "webui",
reason="Shared components affect every UI consumer",
timeout_seconds=900,
)
)
_apply_ui_inputs(checks[-1], ui_scope)
if profile == "backend":
for repo in selected:
if (repo.path / "tests").is_dir():
checks.append(
stage(
f"{repo.name}.backend",
f"{repo.name} backend tests",
["{python}", "-m", "pytest", "-q", str(repo.path / "tests")],
repo.path,
reason=reason,
resources=["backend:test-state"],
timeout_seconds=1800,
)
)
_apply_undeclared_source_note(
checks, _undeclared_source_note(loaded, workspace_root)
)
return checks
def build_coverage(
workspace_root: Path,
profile: str,
repos: list[str],
changed: bool,
project: Path | None = None,
*,
stages: list[dict] | None = None,
) -> dict:
"""Read-only suite inventory; prebuilt stages avoid repeating selection queries."""
from .coverage import coverage_inventory
if profile not in PROFILES:
raise ValueError(f"Unknown check profile: {profile}")
if stages is None:
stages = build_stages(workspace_root, profile, repos, changed, project)
return coverage_inventory(workspace_root, profile, project, stages)
+244
View File
@@ -0,0 +1,244 @@
"""Independently verified check results; never an artifact/build-output cache."""
from __future__ import annotations
from copy import deepcopy
import threading
import re
from .common import digest, now
from .inputs import InputSnapshotter, FINGERPRINT_VERSION
CHECKPOINT_VERSION = 1
def validate_checkpoint_receipt(receipt):
"""New checkpoints are explicit; legacy status alone never certifies a phase."""
version = receipt.get("fingerprint_version")
if version is None:
return
if version != FINGERPRINT_VERSION:
raise ValueError("Unsupported check fingerprint version")
for stage in receipt["stages"]:
verified = stage.get("checkpoint_verified")
if type(verified) is not bool:
raise ValueError("Stage checkpoint verification must be boolean")
if stage["status"] == "passed" and not verified:
raise ValueError(
"Passing stage requires an independently verified checkpoint"
)
if not verified:
continue
if (
stage["status"] != "passed"
or stage.get("checkpoint_version") != CHECKPOINT_VERSION
):
raise ValueError("Invalid verified checkpoint state or version")
for field in (
"cache_key",
"input_fingerprint",
"stage_plan_fingerprint",
"log_sha256",
):
if not isinstance(stage.get(field), str) or not re.fullmatch(
r"[a-f0-9]{64}", stage[field]
):
raise ValueError(
"Verified checkpoint requires bounded content identities"
)
if (
not isinstance(stage.get("checkpoint_at"), str)
or not stage["checkpoint_at"]
):
raise ValueError("Verified checkpoint requires its recording time")
scope = stage.get("input_scope")
if (
not isinstance(scope, dict)
or scope.get("version") != 1
or scope.get("kind") not in {"workspace", "repositories"}
or type(scope.get("declared")) is not bool
):
raise ValueError("Verified checkpoint requires a versioned input scope")
names = scope.get("repos")
if (
not isinstance(names, list)
or not 1 <= len(names) <= 256
or any(not isinstance(name, str) for name in names)
or len(set(names)) != len(names)
):
raise ValueError(
"Verified checkpoint requires bounded repository identities"
)
class Checkpoints:
def __init__(self, project, workspace, plan, environment_probe, cancelled):
self.scanner = InputSnapshotter(project, workspace_root=workspace)
self.plan = {stage["id"]: stage for stage in plan}
self.environment_probe = environment_probe
self.cancelled = cancelled
self.lock = threading.RLock()
self.initial = None
self.environment = None
def source(self, stages=None):
with self.lock:
return self.scanner.snapshot(
list(self.plan.values()) if stages is None else stages
)
def probe_environment(self):
value = self.environment_probe()
self.check_cancelled()
return value
def check_cancelled(self):
if self.cancelled.is_set():
raise InterruptedError("Check cancelled during input verification")
def initialize(self):
self.initial = self.source()
self.check_cancelled()
self.environment = self.probe_environment()
return self.initial, self.environment
def closure(self, stage):
selected = {}
def include(item):
if item["id"] in selected:
return
selected[item["id"]] = item
for identity in item["deps"]:
include(self.plan[identity])
include(self.plan[stage["id"]])
return [selected[key] for key in sorted(selected)]
def identity(self, stage):
with self.lock:
closure = self.closure(stage)
snapshot = self.source(closure)
self.check_cancelled()
environment = self.probe_environment()
own = snapshot["stages"][stage["id"]]
key = digest(
{
"checkpoint_version": CHECKPOINT_VERSION,
"inputs": {
identity: value["fingerprint"]
for identity, value in snapshot["stages"].items()
},
"environment": environment,
}
)
return {
"checkpoint_version": CHECKPOINT_VERSION,
"cache_key": key,
"input_fingerprint": own["fingerprint"],
"input_scope": own["scope"],
"stage_plan_fingerprint": own["plan_fingerprint"],
"stage_environment_fingerprint": environment,
"dependency_input_fingerprints": {
item["id"]: snapshot["stages"][item["id"]]["fingerprint"]
for item in closure
if item["id"] != stage["id"]
},
}
def prepare(self, stage, prior, allow_reuse, verify_log):
before = self.identity(stage)
reason = "No previous verified checkpoint"
if stage.get("reuse", "verified") == "never":
reason = (
"This stage explicitly disables reuse (outputs/setup must be recreated)"
)
elif not allow_reuse:
reason = "A data dependency ran again; its consumers must run again"
elif (
prior
and prior.get("status") == "passed"
and prior.get("checkpoint_verified") is True
and prior.get("checkpoint_version") == CHECKPOINT_VERSION
):
if prior.get("cache_key") == before["cache_key"]:
# Receipt command text is never executed. Only the freshly planned
# stage runs; a cached log must independently match its content hash.
verify_log(prior)
result = {
**before,
**{
key: deepcopy(prior[key])
for key in (
"status",
"exit_code",
"duration_seconds",
"log_path",
"log_sha256",
"output_truncated",
"omitted_output_bytes",
"checkpoint_at",
)
if key in prior
},
}
result.update(
checkpoint_verified=True,
reuse_reason="Verified checkpoint matches current inputs, command, dependencies and environment",
)
return before, result
reason = (
"Declared inputs, command, dependency inputs or environment changed"
)
before["reuse_reason"] = reason
return before, None
def finish(self, stage, before, result):
result = {**result, **before, "checkpoint_verified": False}
if result["status"] != "passed" or result.get("exit_code") != 0:
return result
after = self.identity(stage)
if before["cache_key"] != after["cache_key"]:
result.update(
status="stale",
error="Stage inputs or environment changed during execution; no reusable checkpoint was recorded",
)
else:
result.update(checkpoint_verified=True, checkpoint_at=now())
return result
def finalize(self, stages):
final = self.source()
self.check_cancelled()
environment = self.probe_environment()
valid = (
final["observed_source_fingerprint"]
== self.initial["observed_source_fingerprint"]
and environment == self.environment
)
for stage in stages:
if stage["status"] != "passed":
valid = valid and stage["status"] != "stale"
continue
closure = self.closure(self.plan[stage["id"]])
key = digest(
{
"checkpoint_version": CHECKPOINT_VERSION,
"inputs": {
item["id"]: final["stages"][item["id"]]["fingerprint"]
for item in closure
},
"environment": environment,
}
)
if (
stage.get("checkpoint_verified") is not True
or stage.get("cache_key") != key
):
stage.update(
status="stale",
checkpoint_verified=False,
error="Final inputs no longer match this checkpoint",
)
valid = False
return valid, final, environment
+212
View File
@@ -0,0 +1,212 @@
"""A compact command catalog over maintained project tools."""
from __future__ import annotations
import argparse
import importlib
import json
from pathlib import Path
import sys
from . import __version__
from .common import META_ROOT, redact, safe_output
COMMAND_MODULES = (
"context",
"doctor",
"runner",
"review",
"docs",
"issues",
"release",
"maintenance",
)
def main(argv: list[str] | None = None) -> int:
argv = sys.argv[1:] if argv is None else argv
common = argparse.ArgumentParser(add_help=False, allow_abbrev=False)
common.add_argument(
"--workspace-root",
type=Path,
default=META_ROOT.parent,
help="Directory containing registered repositories",
)
common.add_argument(
"--project", type=Path, help="Explicit trusted portable-project JSON manifest"
)
common.add_argument(
"--state-dir",
type=Path,
help="Private evidence-state base (scoped again by workspace)",
)
common.add_argument("--format", choices=("summary", "json"), default="summary")
common.add_argument(
"--quiet",
action="store_true",
help="Suppress live progress on stderr; retain the final result",
)
common.add_argument(
"--json",
dest="format",
action="store_const",
const="json",
help="Alias for --format json",
)
parser = argparse.ArgumentParser(
prog="devkit",
description="Deterministic development workflows. Read-only previews by default for remote writes.",
parents=[common],
allow_abbrev=False,
)
parser.add_argument("--version", action="version", version=f"devkit {__version__}")
subparsers = parser.add_subparsers(dest="command", required=True)
for name in COMMAND_MODULES:
importlib.import_module("." + name, __package__).register(subparsers)
catalog = subparsers.add_parser("commands", help="Show the compact command catalog")
def commands(_):
entries = [
{
"command": "context",
"purpose": "Offline repository changes, ownership and instructions",
"effects": "read only",
},
{
"command": "doctor",
"purpose": "Local tool/dependency preflight and repair guidance",
"effects": "read only",
},
{
"command": "check",
"purpose": "Registered verification profiles with logs and source-bound receipts",
"effects": "tests/builds; --dry-run previews",
},
{
"command": "runs / latest / status / summary / logs",
"purpose": "Find runs and read progress, compact results and bounded live/final logs",
"effects": "read only",
},
{
"command": "coverage",
"purpose": "Explain declared suite coverage and exclusions for a check profile",
"effects": "read only",
},
{
"command": "resume / recover",
"purpose": "Resume verified identical work or recover an abandoned check run",
"effects": "local checks/state only",
},
{
"command": "review",
"purpose": "Module UI-review inventory and manual evidence checklist",
"effects": "local bundle only",
},
{
"command": "docs",
"purpose": "Existing documentation and translation audits",
"effects": "local checks/evidence",
},
{
"command": "issues",
"purpose": "Preview and explicitly publish deduplicated issue evidence",
"effects": "remote only with --apply",
},
{
"command": "release",
"purpose": "Existing durable release lifecycle, receipts and confirmations",
"effects": "explicit --apply and step confirmation",
},
{
"command": "git",
"purpose": "Frozen explicit-path commit and branch-push maintenance",
"effects": "explicit --apply; no bulk staging, force or tags",
},
]
return {
"commands": entries,
"summary": [
f"{item['command']}: {item['purpose']} ({item['effects']})"
for item in entries
],
}
catalog.set_defaults(handler=commands)
# Global flags work before or after the command, without copying defaults to every parser.
global_args, remaining = common.parse_known_args(argv)
args = parser.parse_args(remaining, namespace=global_args)
args.workspace_root = args.workspace_root.expanduser().resolve()
if args.project:
args.project = args.project.expanduser().absolute()
def progress(event):
if args.quiet:
return
if args.format == "json":
print(
json.dumps(safe_output(event), sort_keys=True, allow_nan=False),
file=sys.stderr,
flush=True,
)
else:
counts = event["counts"]
completed = sum(
count
for state, count in counts.items()
if state not in {"pending", "running"}
)
active = ", ".join(event["active_stages"])
print(
redact(
f"Run {event['run_id']}: {event['phase']} · {completed}/{event['total_stages']} stages · {event['elapsed_seconds']}s"
+ (f" · {active}" if active else "")
),
file=sys.stderr,
flush=True,
)
args.on_progress = progress
try:
result = args.handler(args)
if not isinstance(result, dict):
raise ValueError("Command did not return a result object")
code = int(result.pop("_exit_code", 0))
if args.format == "json":
print(
json.dumps(
safe_output(result),
sort_keys=True,
indent=2,
ensure_ascii=True,
allow_nan=False,
)
)
else:
summary = result.get("summary", [str(result.get("status", "Completed"))])
print(
"\n".join(redact(str(line)) for line in summary)
if isinstance(summary, list)
else redact(str(summary))
)
return code
except (ValueError, OSError, RuntimeError, ImportError) as exc:
message = redact(str(exc))
if args.format == "json":
print(
json.dumps(
{
"status": "error",
"error": message,
"error_type": type(exc).__name__,
}
)
)
else:
print(f"devkit: {message}", file=sys.stderr)
return 2
except KeyboardInterrupt:
print(
"devkit: interrupted; inspect the saved run before retrying",
file=sys.stderr,
)
return 130
+232
View File
@@ -0,0 +1,232 @@
"""Bounded local records and predictable output; no network or AI dependency."""
from __future__ import annotations
from contextlib import contextmanager
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import stat
import tempfile
import time
META_ROOT = Path(__file__).resolve().parents[3]
MAX_JSON_BYTES = 8 * 1024 * 1024
IDENTIFIER = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}\Z")
def now() -> str:
return datetime.now(timezone.utc).isoformat()
def canonical(value: object) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True, allow_nan=False
).encode()
def digest(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def identifier(value: str) -> str:
if (
not isinstance(value, str)
or not IDENTIFIER.fullmatch(value)
or value in {".", ".."}
):
raise ValueError("Invalid record identifier")
return value
def state_root(workspace_root: Path, state_dir: Path | None = None) -> Path:
base = (
state_dir
or Path(os.environ.get("XDG_STATE_HOME", str(Path.home() / ".local/state")))
/ "govoplan/devkit"
)
# Preserve spelling until symlink validation; resolving here would hide an unsafe alias.
base = Path(os.path.abspath(os.fspath(base.expanduser())))
return base / ("workspace-" + digest(str(workspace_root.resolve()))[:24])
def reject_symlinks(path: Path) -> None:
for item in (path, *path.parents):
if item.is_symlink():
raise ValueError("State and evidence paths must not contain symlinks")
def private_directory(path: Path) -> None:
reject_symlinks(path)
path.mkdir(parents=True, exist_ok=True, mode=0o700)
reject_symlinks(path)
metadata = path.stat()
if not stat.S_ISDIR(metadata.st_mode) or (
hasattr(os, "getuid") and metadata.st_uid != os.getuid()
):
raise ValueError("State directory must be owned by the current user")
path.chmod(0o700)
def read_bounded_bytes(path: Path, max_bytes: int = MAX_JSON_BYTES) -> bytes:
reject_symlinks(path)
descriptor = os.open(
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
)
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > max_bytes:
raise ValueError("Evidence must be a bounded regular file")
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes:
raise ValueError("Evidence file exceeds its size bound")
return encoded
def read_json(path: Path, max_bytes: int = MAX_JSON_BYTES) -> object:
encoded = read_bounded_bytes(path, max_bytes)
def unique(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError("Duplicate JSON keys are not accepted")
result[key] = value
return result
try:
return json.loads(
encoded,
object_pairs_hook=unique,
parse_constant=lambda _: (_ for _ in ()).throw(
ValueError("Non-finite JSON number")
),
)
except (RecursionError, UnicodeError) as exc:
raise ValueError("JSON nesting or encoding is unsupported") from exc
def atomic_text(path: Path, value: str, max_bytes: int = MAX_JSON_BYTES) -> None:
encoded = value.encode("utf-8")
if len(encoded) > max_bytes:
raise ValueError("Output exceeds its size bound")
reject_symlinks(path.parent)
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
reject_symlinks(path.parent)
reject_symlinks(path)
if path.exists() and not path.is_file():
raise ValueError("Output target must be a regular file")
descriptor, temporary = tempfile.mkstemp(prefix=".devkit-", dir=path.parent)
try:
with os.fdopen(descriptor, "wb") as handle:
os.fchmod(handle.fileno(), 0o600)
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory = os.open(path.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(directory)
finally:
os.close(directory)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def atomic_json(path: Path, payload: object) -> None:
atomic_text(
path,
json.dumps(
payload, indent=2, sort_keys=True, ensure_ascii=True, allow_nan=False
)
+ "\n",
)
def redact(text: str) -> str:
"""Best-effort display hygiene, not permission to include secrets in commands."""
for key, value in os.environ.items():
if (
re.search(r"TOKEN|SECRET|PASSWORD|API_KEY|PRIVATE_KEY", key, re.I)
and len(value) >= 4
):
text = text.replace(value, "[redacted]")
text = re.sub(r"(?im)(authorization\s*[:=]\s*)([^\r\n]+)", r"\1[redacted]", text)
text = re.sub(r"(?i)(https?://)[^/\s:@]+:[^/\s@]+@", r"\1[redacted]@", text)
text = re.sub(
r"(?i)((?:token|password|secret|api[_-]?key)\s*[=:]\s*)[^\s,;]+",
r"\1[redacted]",
text,
)
return text
def redact_argv(argv: list[str]) -> list[str]:
result, hide_next = [], False
for argument in argv:
if hide_next:
result.append("[redacted]")
hide_next = False
continue
if re.fullmatch(
r"--?(?:password|passwd|token|secret|api[-_]key|access[-_]token|authorization)",
argument,
re.I,
):
hide_next = True
result.append(redact(argument))
return result
def safe_output(value):
"""Redact presentation, not immutable identity hashes or execution inputs."""
if isinstance(value, dict):
return {
key: redact_argv(item)
if key == "argv"
and isinstance(item, list)
and all(isinstance(arg, str) for arg in item)
else "[redacted]"
if re.fullmatch(
r"password|passwd|token|secret|api[_-]?key|authorization",
str(key),
re.I,
)
else safe_output(item)
for key, item in value.items()
}
if isinstance(value, list):
return [safe_output(item) for item in value]
return redact(value) if isinstance(value, str) else value
@contextmanager
def resource_lock(directory: Path, name: str, timeout: float = 0):
"""Host-local advisory lock, released by the OS even after a process crash."""
import fcntl
private_directory(directory)
path = directory / (hashlib.sha256(name.encode()).hexdigest() + ".lock")
reject_symlinks(path)
descriptor = os.open(
path, os.O_CREAT | os.O_RDWR | getattr(os, "O_NOFOLLOW", 0), 0o600
)
try:
if not stat.S_ISREG(os.fstat(descriptor).st_mode):
raise ValueError("Lock target must be a regular file")
deadline = time.monotonic() + timeout
while True:
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
if time.monotonic() >= deadline:
raise RuntimeError(f"Resource is busy: {name}") from None
time.sleep(min(0.1, max(0, deadline - time.monotonic())))
yield
finally:
os.close(descriptor)
+90
View File
@@ -0,0 +1,90 @@
"""Small read-only context bundles; no automatic source/credential dumping."""
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from .common import META_ROOT, now, read_json
from .workspace import inspect_repository, load_project, selected_repositories
def build_context(
workspace_root: Path, project_file: Path | None, names: list[str], changed: bool
) -> dict:
project = load_project(workspace_root, project_file)
repos = selected_repositories(project, names)
with ThreadPoolExecutor(max_workers=8) as pool:
states = list(pool.map(inspect_repository, repos))
if changed:
states = [
state
for state in states
if state["errors"]
or state["dirty_entries"]
or state["ahead"]
or (state["head"] and not state["upstream"])
]
inventory = {}
if not project_file:
path = META_ROOT / "docs/project/ui-review-issue-inventory.json"
if path.is_file():
payload = read_json(path)
for item in payload.get("issues", []) if isinstance(payload, dict) else []:
if isinstance(item, dict):
inventory[item.get("repository")] = item
for state in states:
root = Path(state["path"])
state["instructions"] = [
str(path) for path in (root / "AGENTS.md",) if path.is_file()
]
state["documentation"] = [
str(path)
for path in (
root / "README.md",
root / "docs/README.md",
root / "docs/MODULE_ARCHITECTURE.md",
)
if path.is_file()
]
state["change_entry_count"] = len(state["dirty_entries"])
state["review_issue"] = inventory.get(state["name"], {}).get("url")
state["suggested_check"] = (
f"./devkit check --repo {state['name']} --profile quick --dry-run"
)
summary = [
f"{project.name}: {len(states)} repositories selected (offline; upstream counts may be stale)."
]
for state in states:
errors = f" ERROR: {'; '.join(state['errors'])}" if state["errors"] else ""
summary.append(
f"{state['name']}: {state['branch'] or '(detached/unborn)'}; {state['change_entry_count']} change entries; ahead={state['ahead']} behind={state['behind']}{errors}"
)
return {
"schema_version": 1,
"generated_at": now(),
"workspace_root": str(workspace_root),
"project": project.name,
"remote_checked": False,
"repositories": states,
"summary": summary,
"_exit_code": 1 if any(state["errors"] for state in states) else 0,
}
def register(subparsers):
parser = subparsers.add_parser(
"context",
help="Offline repository changes, ownership and relevant instruction paths",
)
parser.add_argument("--repo", action="append", default=[])
parser.add_argument(
"--changed",
action="store_true",
help="Show dirty or locally ahead repositories, retaining inspection errors",
)
parser.set_defaults(
handler=lambda args: build_context(
args.workspace_root, args.project, args.repo, args.changed
)
)
+453
View File
@@ -0,0 +1,453 @@
"""Explicit suite-plan coverage; never execute or guess nested shell/npm flows."""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import re
import shlex
import stat
from .common import digest, redact_argv, reject_symlinks
from .package_tests import CORE_COMPONENT_SUITES, declared_tests, discovered_sources
from .workspace import load_project
DISPOSITIONS = ("planned", "covered_elsewhere", "excluded", "unsupported")
MAX_SUITES = 4096
def _read_canonical(path: Path) -> str:
reject_symlinks(path)
descriptor = os.open(
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
)
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 1024 * 1024:
raise ValueError("Canonical focused gate must be a bounded regular file")
encoded = handle.read(1024 * 1024 + 1)
if len(encoded) > 1024 * 1024:
raise ValueError("Canonical focused gate exceeds its size bound")
return encoded.decode("utf-8")
def _node_argv(argv: list[str], cwd: Path) -> tuple[str, ...] | None:
if (
not argv
or argv[0] not in {"node", "{node}", "$NODE"}
and Path(argv[0]).name not in {"node", "nodejs"}
):
return None
offset = 2 if len(argv) > 1 and argv[1] == "--test" else 1
if len(argv) != offset + 1:
return None
target = Path(argv[offset])
if "$" in str(target):
return None
return ("node", *argv[1:offset], str((cwd / target).resolve()))
def canonical_invocations(workspace_root: Path, meta: Path, core: Path) -> dict:
"""Recognize direct commands in exact registered phase wrappers or legacy top level.
Shell is not evaluated. Bodies, conditionals, loops, functions, npm hooks and
recursive commands do not grant coverage. Unknown cwd invalidates matches.
The registered marked wrappers are the only function bodies admitted.
"""
path = meta / "tools/checks/check-focused.sh"
try:
text = _read_canonical(path)
except (OSError, ValueError):
return {
"path": str(path),
"sha256": None,
"npm": [],
"node": [],
"node_phases": [],
"notes": [
"Canonical focused script is unavailable/unreadable; no package coverage inferred."
],
}
values = {
"WORKSPACE_ROOT": str(workspace_root),
"META_ROOT": str(meta),
"ROOT": str(core),
}
def expand(value: str) -> str | None:
value = re.sub(
r"\$\{(WORKSPACE_ROOT|META_ROOT|ROOT)\}|\$(WORKSPACE_ROOT|META_ROOT|ROOT)(?![A-Za-z0-9_])",
lambda match: values[match[1] or match[2]],
value,
)
return None if "$" in value or "`" in value else value
result = {
"path": str(path),
"sha256": hashlib.sha256(text.encode()).hexdigest(),
"npm": [],
"node": [],
"node_phases": [],
"notes": [],
}
metadata = meta / "tools/checks/focused-phases.json"
blocks = {None: text}
if metadata.exists() or metadata.is_symlink() or "# devkit-phase:" in text:
from .catalog import focused_phases, focused_phase_bodies
try:
phases = focused_phases(meta)
blocks = focused_phase_bodies(text, phases)
result["phase_ids"] = [phase["id"] for phase in phases]
result["metadata_sha256"] = digest(phases)
except (OSError, ValueError):
result["notes"].append(
"Focused phase metadata/marked bodies are invalid or unavailable; no phase coverage inferred."
)
return result
cwd: Path | None = meta
body_end, nesting = None, 0
current_phase = object()
for phase_id, line in (
(phase_id, line)
for phase_id, block in blocks.items()
for line in block.replace("\\\n", " ").splitlines()
):
if current_phase != phase_id:
cwd, body_end, nesting = meta, None, 0
current_phase = phase_id
stripped = line.strip()
if body_end is not None:
if stripped == body_end:
body_end = None
continue
heredoc = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
if heredoc:
body_end = heredoc[1]
continue
if not stripped or stripped.startswith("#"):
continue
if re.match(
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
stripped,
):
nesting += 1
continue
if re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
nesting = max(0, nesting - 1)
continue
if nesting:
if re.search(r"\bcd\s", stripped):
cwd = None
continue
try:
parts = shlex.split(line, comments=True)
except ValueError:
result["notes"].append(
"An unparseable shell line grants no suite coverage."
)
continue
if not parts:
continue
if parts[0] == "cd":
destination = expand(parts[1]) if len(parts) == 2 else None
candidate = (
(cwd / destination).resolve()
if destination and cwd
else Path(destination).resolve()
if destination and Path(destination).is_absolute()
else None
)
cwd = (
candidate
if candidate and candidate.is_relative_to(workspace_root)
else None
)
continue
if cwd is None:
continue
if (
parts[0] in {"$NPM", "${NPM}", "npm"}
and len(parts) >= 3
and parts[1] == "run"
and re.fullmatch(r"test(?::[A-Za-z0-9_.:-]+)?", parts[2])
):
arguments = parts[3:]
if arguments and (
arguments[0] != "--"
or any(
not re.fullmatch(r"[A-Za-z0-9_.-]+", item) for item in arguments[1:]
)
):
continue
result["npm"].append(
{
"cwd": str(cwd),
"name": parts[2],
"args": arguments[1:] if arguments else [],
"phase": phase_id,
}
)
elif parts[0] in {"$NODE", "${NODE}", "node"}:
expanded = [expand(part) for part in parts[1:]]
if all(part is not None for part in expanded):
command = _node_argv(["node", *expanded], cwd)
if command:
result["node"].append(command)
result["node_phases"].append({"argv": command, "phase": phase_id})
result["notes"] = list(dict.fromkeys(result["notes"]))[:8]
return result
def coverage_inventory(
workspace_root: Path, profile: str, project_file: Path | None, stages: list[dict]
) -> dict:
workspace_root = workspace_root.resolve()
project = load_project(workspace_root, project_file)
stage_map = {item["id"]: item for item in stages}
direct = {}
explicit_npm = {}
for stage in stages:
argv, cwd = stage["argv"], Path(stage["cwd"])
normalized = _node_argv(argv, cwd)
if normalized:
direct[normalized] = stage["id"]
if (
len(argv) == 3
and (argv[0] in {"npm", "{npm}"} or Path(argv[0]).name == "npm")
and argv[1] == "run"
):
explicit_npm[(str(cwd.resolve()), argv[2])] = stage["id"]
paths = {repo.name: repo.path for repo in project.repositories}
canonical = (
canonical_invocations(
workspace_root,
paths.get("govoplan", workspace_root / "govoplan"),
paths.get("govoplan-core", workspace_root / "govoplan-core"),
)
if project_file is None
else None
)
canonical_stages = {}
if canonical is not None:
legacy = stage_map.get("focused-workspace")
if legacy and legacy["argv"] == ["bash", canonical["path"]]:
canonical_stages[None] = "focused-workspace"
for identity in canonical.get("phase_ids", []):
check = stage_map.get("focused." + identity)
if check and check["argv"] == [
"bash",
canonical["path"],
"--phase",
identity,
]:
canonical_stages[identity] = check["id"]
full = bool(canonical_stages)
invocations = (
[
{**item, "covering_stage": canonical_stages[item.get("phase")]}
for item in canonical["npm"]
if item.get("phase") in canonical_stages
]
if canonical
else []
)
canonical_nodes = (
{
tuple(item["argv"]): canonical_stages[item.get("phase")]
for item in canonical.get("node_phases", [])
if item.get("phase") in canonical_stages
}
if canonical
else {}
)
suites, notes = [], []
if canonical:
notes.extend(canonical["notes"])
core_package = (
paths.get("govoplan-core", workspace_root / "govoplan-core")
/ "webui/package.json"
)
core_requests = [
item
for item in invocations
if item["cwd"] == str(core_package.parent) and item["name"] == "test:components"
]
requested_components = set()
for item in core_requests:
requested_components.update(
CORE_COMPONENT_SUITES
if not item["args"] or item["args"] == ["all"]
else item["args"]
)
requested_components.intersection_update(CORE_COMPONENT_SUITES)
for repo in project.repositories:
for location in ("package.json", "webui/package.json"):
package = repo.path / location
if not package.exists() and not package.is_symlink():
continue
entries = declared_tests(repo, package)
if location == "webui/package.json":
declared_argv = {
tuple(item["_argv"]) for item in entries if item["_argv"]
}
entries.extend(
item
for item in discovered_sources(repo)
if not item["_argv"] or tuple(item["_argv"]) not in declared_argv
)
for item in entries:
private_argv = item.pop("_argv")
component = item["component_suite"]
identity = (str(package.parent), item["name"])
normalized = (
_node_argv(private_argv, package.parent) if private_argv else None
)
matching = [
value
for value in invocations
if (value["cwd"], value["name"]) == identity
]
item.update(disposition="excluded", covering_stage=None)
if component is not None:
all_or_selected = (
component == "all" or component in requested_components
)
if "core.component-batch" in stage_map:
item.update(
disposition="planned"
if component == "all"
else "covered_elsewhere",
covering_stage="core.component-batch",
reason="UI explicitly runs the shared component batch; quick never compiles it.",
)
elif full and core_requests and all_or_selected:
item.update(
disposition="planned"
if component == "all"
else "covered_elsewhere",
covering_stage=core_requests[0]["covering_stage"],
reason=f"Canonical focused gate explicitly selects {len(requested_components)}/{len(CORE_COMPONENT_SUITES)} component suites; this is not the complete component batch.",
)
else:
item["reason"] = (
"No component compilation in quick/backend; UI runs all components. Full runs only its explicitly named subset."
)
if component == "all":
selected = (
list(CORE_COMPONENT_SUITES)
if "core.component-batch" in stage_map
else sorted(requested_components)
)
item.update(
covered_components=selected,
excluded_components=[
name
for name in CORE_COMPONENT_SUITES
if name not in selected
],
)
elif identity in explicit_npm:
item.update(
disposition="planned",
covering_stage=explicit_npm[identity],
reason="Explicit project check invokes this exact package suite.",
)
elif matching:
if any(not value["args"] for value in matching):
item.update(
disposition="planned",
covering_stage=next(
value["covering_stage"]
for value in matching
if not value["args"]
),
reason="The canonical focused script directly invokes this exact package suite.",
)
else:
item["reason"] = (
"Canonical invocation supplies arguments; complete suite coverage cannot be inferred."
)
elif normalized in direct:
item.update(
disposition="planned",
covering_stage=direct[normalized],
reason="A selected stage runs this exact direct source-test command.",
)
elif normalized in canonical_nodes:
item.update(
disposition="covered_elsewhere",
covering_stage=canonical_nodes[normalized],
reason="Canonical focused script directly runs this suite's exact Node target.",
)
elif private_argv is None:
item["disposition"] = "unsupported"
elif item["name"] in {
"test:module-permutations",
"test:vite-cache-isolation",
}:
item["reason"] = (
"Separate environment/permutation suite; absent from the selected stage plan."
)
else:
item["reason"] = (
"Not directly present in this profile's stage plan; nested npm scripts/hooks are not inferred."
)
suites.append(item)
if len(suites) > MAX_SUITES:
raise ValueError("Coverage inventory exceeds its bounded suite count")
if project_file is not None:
for check in project.config.get("checks", []):
identity = check["id"]
suites.append(
{
"repo": ",".join(check.get("repos", [])) or "project",
"name": identity,
"kind": "project-check",
"package_path": None,
"argv": redact_argv(check["argv"]),
"command_sha256": digest(check["argv"]),
"disposition": "planned" if identity in stage_map else "excluded",
"covering_stage": identity if identity in stage_map else None,
"reason": "Selected declared check or dependency."
if identity in stage_map
else "Declared check is outside this profile/selection.",
}
)
if len(suites) > MAX_SUITES:
raise ValueError("Coverage inventory exceeds its bounded suite count")
counts = {
kind: sum(item["disposition"] == kind for item in suites)
for kind in DISPOSITIONS
}
notes.append(
"Coverage describes planned suite invocations, not passing tests, per-test coverage, or completed UI review."
)
if full:
notes.append(
"Full is the canonical focused gate, not every declared package test; recursive commands and npm hooks are deliberately not guessed."
)
return {
"schema_version": 1,
"profile": profile,
"scope": "Declared root/webui package test scripts, discovered UI structural checks, and custom project checks across registered repositories.",
"suite_count": len(suites),
"stages": list(stage_map),
"counts": counts,
"suites": suites,
"canonical_gate": {
key: canonical[key]
for key in ("path", "sha256", "metadata_sha256")
if key in canonical
}
if canonical
else None,
"notes": notes,
"summary": [
"Suite coverage: "
+ ", ".join(f"{value} {key}" for key, value in counts.items()),
*notes,
],
}
+133
View File
@@ -0,0 +1,133 @@
"""Recorded documentation checks built from existing owning-module contracts."""
from __future__ import annotations
from pathlib import Path
import uuid
from .catalog import stage
from .common import state_root
from .workspace import load_project, selected_repositories
LIMITATIONS = [
"Static baseline, marker coverage and known display slots do not prove complete workflow or linguistic coverage.",
"Computed labels, configured documentation providers and runtime module/permission combinations still require manual review.",
"Audit findings do not edit documentation, translate text, publish evidence or close a review issue.",
]
def build_doc_stages(args) -> list[dict]:
workspace = Path(args.workspace_root).resolve()
project = load_project(workspace, getattr(args, "project", None))
if getattr(args, "project", None):
raise ValueError(
"The docs audit currently targets GovOPlaN manifest/locale contracts. Register another project's documentation checks in its check profiles."
)
selected = selected_repositories(
project, getattr(args, "repo", []), changed=getattr(args, "changed", False)
)
if getattr(args, "changed", False) and not selected:
return []
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
core = next(
repo.path for repo in project.repositories if repo.name == "govoplan-core"
)
output = (
state_root(workspace, getattr(args, "state_dir", None))
/ "artifacts"
/ f"docs-{uuid.uuid4().hex}"
)
reason = "Reuse the existing owning-module documentation and translation checks"
labels = [
"{node}",
str(meta / "tools/devkit/audit-display-labels.mjs"),
"--workspace-root",
str(workspace),
]
for repo in selected:
labels.extend(["--repo", repo.name])
stages = [
stage(
"docs.manifests",
"Static user/admin documentation and manifest contracts",
[
"{python}",
str(meta / "tools/checks/check-manifest-shapes.py"),
"--workspace-root",
str(workspace),
"--require-architecture",
],
meta,
reason=reason,
timeout_seconds=600,
),
stage(
"docs.interface-inventory",
"Existing EN/DE markers, high-risk help and interface declarations",
[
"{python}",
str(meta / "tools/inventory/platform-interface-inventory.py"),
"--workspace-root",
str(workspace),
"--strict",
"--strict-declarations",
"--strict-endpoints",
"--output-dir",
str(output),
],
meta,
reason="Shared inventory remains workspace-wide; --repo narrows only the additional plain-label audit",
timeout_seconds=600,
),
stage(
"docs.translation-structure",
"Existing translation key/structural-value guard",
["{node}", str(core / "webui/scripts/audit-i18n-structural.mjs")],
core / "webui",
reason=reason,
),
stage(
"docs.plain-display-labels",
"Plain display labels and owning catalog registration",
labels,
meta,
reason="Supplement the marker inventory with known display slots; dynamic cases are review candidates",
timeout_seconds=600,
),
]
# These constraints belong to the saved evidence, not only the immediate
# CLI response. The publisher already validates bounded coverage_notes.
stages[0]["coverage_notes"] = list(LIMITATIONS)
return stages
def audit(args) -> dict:
from .runner import run_checks
result = run_checks(args, build_doc_stages(args))
result["limitations"] = list(LIMITATIONS)
result.setdefault("_exit_code", 0)
summary = result.setdefault("summary", [])
summary.extend(note for note in LIMITATIONS if note not in summary)
return result
def register(subparsers) -> None:
docs = subparsers.add_parser(
"docs", help="Audit existing documentation and translation contracts"
)
commands = docs.add_subparsers(dest="docs_command", required=True)
command = commands.add_parser(
"audit", help="Run recorded checks; never edit or generate translations"
)
command.add_argument(
"--repo",
action="append",
default=[],
help="Repository/alias for plain-label checks; shared checks stay workspace-wide",
)
command.add_argument("--changed", action="store_true")
command.add_argument("--jobs", type=int, default=2)
command.add_argument("--dry-run", action="store_true")
command.set_defaults(handler=audit, profile="docs")
+232
View File
@@ -0,0 +1,232 @@
"""Read-only environment preflight with actionable, never automatic repairs."""
from __future__ import annotations
from dataclasses import replace
import os
from pathlib import Path
import socket
from .common import now
from .environment import execution_environment, resolve_tools, tool_version
from .workspace import inspect_repository, load_project, selected_repositories
from .process import require_capture
def _required_tools(project, workspace_root, selected, *, filtered, profile=None):
"""Use the catalog's selected commands, including dependencies, without running them."""
from .catalog import _custom_stages
if profile is None:
# Doctor covers all declared profiles by default, or all declared checks
# when the project only uses the context/doctor commands so far.
profiles = project.config.get("profiles", {})
records = project.config.get("checks", [])
if not isinstance(profiles, dict) or not isinstance(records, list):
raise ValueError(
"Project profiles and checks must have their declared shapes"
)
identities = []
if profiles:
for values in profiles.values():
if not isinstance(values, list) or any(
not isinstance(value, str) for value in values
):
raise ValueError("A project profile must be a list of check IDs")
identities.extend(values)
else:
if any(not isinstance(record, dict) for record in records):
raise ValueError("Project checks must be objects")
identities = [record.get("id") for record in records]
if any(not isinstance(identity, str) for identity in identities):
raise ValueError("Project checks must have string IDs")
project = replace(
project,
config={
**project.config,
"profiles": {"quick": list(dict.fromkeys(identities))},
},
)
profile = "quick"
stages = _custom_stages(project, workspace_root, profile, selected, filtered)
# Python is required by the runner and its environment fingerprint even if
# all selected check commands use another interpreter. Explicitly configured
# tools also declare requirements for commands hidden inside project scripts.
required = {"python", *project.config.get("tools", {})}
for stage in stages:
argv = stage["argv"]
for name in ("node", "npm", "python"):
if any("{" + name + "}" in value for value in argv):
required.add(name)
executable = Path(argv[0]).name
if executable in {"node", "nodejs"}:
required.add("node")
elif executable in {"npm", "npx"}:
required.update({"npm", "node"})
if "npm" in required:
required.add("node")
return required
def diagnose(args) -> dict:
project = load_project(args.workspace_root, args.project)
tools = resolve_tools(args.workspace_root, project)
env = execution_environment(args.workspace_root, project, tools)
selected = selected_repositories(project, args.repo)
required = (
_required_tools(
project,
args.workspace_root,
selected,
filtered=bool(args.repo),
profile=getattr(args, "profile", None),
)
if args.project
else set(tools)
)
checks = []
for name, executable in tools.items():
if name not in required:
checks.append(
{
"id": name,
"status": "not_required",
"detail": "Not required by the selected declared project checks; not probed.",
"path": executable,
"repair": "For indirect dependencies inside scripts, declare the tool under project tools.",
}
)
continue
version = tool_version(executable, env)
checks.append(
{
"id": name,
"status": "passed" if version != "unavailable" else "blocked",
"detail": version,
"path": executable,
"repair": f"Install/configure {name}; set {name.upper()} or project tools.{name}. No installation was attempted.",
}
)
for repo in selected:
snapshot = inspect_repository(repo)
checks.append(
{
"id": repo.name,
"status": "blocked" if snapshot["errors"] else "passed",
"detail": "; ".join(snapshot["errors"])
or "Git checkout readable (dirty work is allowed).",
"repair": "Use tools/repo/bootstrap-repositories.py --check; review missing checkouts before cloning.",
}
)
for package_dir in (repo.path, repo.path / "webui"):
if (package_dir / "package.json").is_file():
present = (package_dir / "node_modules").is_dir()
checks.append(
{
"id": f"dependencies:{repo.name}:{package_dir.name}",
"status": "passed" if present else "warning",
"detail": "Dependency directory exists; availability is not a full dependency audit."
if present
else "No local node_modules directory; workspace-hoisted packages may still resolve.",
"repair": "Inspect the owning package lock and install instructions before running npm ci.",
}
)
if not args.project:
script = args.workspace_root / "govoplan/tools/repo/sync-python-environment.py"
if script.is_file():
result = require_capture(
[
tools["python"],
str(script),
"--check",
"--requirements",
str(args.workspace_root / "govoplan/requirements-dev.txt"),
"--python",
tools["python"],
],
timeout=60,
max_stdout=65536,
env=env,
)
checks.append(
{
"id": "python-environment-sync",
"status": "passed" if result.returncode == 0 else "warning",
"detail": "Environment synchronization fingerprint is current."
if result.returncode == 0
else "Environment sync check did not pass; inspect its --dry-run output.",
"repair": "./.venv/bin/python tools/repo/sync-python-environment.py --dry-run --requirements requirements-dev.txt --python ./.venv/bin/python",
}
)
browser_roots = [
Path.home() / ".cache/ms-playwright",
Path.home() / ".var/app/com.vscodium.codium/cache/ms-playwright",
]
explicit_browser = os.environ.get("PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH")
browser_available = (
(Path(explicit_browser).is_file() and os.access(explicit_browser, os.X_OK))
if explicit_browser
else any(
root.is_dir() and any(root.glob("chromium-*/chrome-linux*/chrome"))
for root in browser_roots
)
)
checks.append(
{
"id": "browser",
"status": "passed" if browser_available else "warning",
"detail": "Configured/cached Chromium found."
if browser_available
else "Chromium executable not found; check PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH or install the pinned Playwright browser.",
"repair": "Follow Core WebUI conformance setup; do not start a development server to repair this.",
}
)
with socket.socket() as probe:
probe.settimeout(0.25)
busy = probe.connect_ex(("127.0.0.1", 4174)) == 0
checks.append(
{
"id": "browser-test-port",
"status": "warning" if busy else "passed",
"detail": "Port 4174 is occupied; do not kill another run."
if busy
else "Port 4174 is free now (not a reservation).",
"repair": "Wait for the owning check run; use devkit resource coordination.",
}
)
summary = [
f"Environment preflight: {sum(item['status'] == 'blocked' for item in checks)} blockers, {sum(item['status'] == 'warning' for item in checks)} warnings."
]
summary.extend(
f"{item['id']}: {item['status']}{item['detail']}"
for item in checks
if item["status"] != "passed"
)
summary.append(
"Read-only: no installations, configuration repairs or servers started."
)
if args.project:
summary.append(
"Required tools follow declared checks and explicit project tools; indirect script dependencies must be declared explicitly."
)
return {
"schema_version": 1,
"generated_at": now(),
"checks": checks,
"required_tools": sorted(required),
"summary": summary,
"_exit_code": 1 if any(item["status"] == "blocked" for item in checks) else 0,
}
def register(subparsers):
parser = subparsers.add_parser(
"doctor", help="Read-only environment preflight and exact repair guidance"
)
parser.add_argument("--repo", action="append", default=[])
parser.add_argument(
"--profile",
choices=("quick", "ui", "backend", "full"),
help="For portable projects, limit tool requirements to this check profile (default: all declared profiles)",
)
parser.set_defaults(handler=diagnose)
+271
View File
@@ -0,0 +1,271 @@
"""Resolve local tools once, without installation, network access or server startup."""
from __future__ import annotations
import hashlib
from itertools import islice
import os
from pathlib import Path
import shutil
import stat
import subprocess
import sys
from .common import digest
from .workspace import Project
from .process import require_capture
MAX_DISCOVERY_CHILDREN = 4096
def resolve_tools(workspace_root: Path, project: Project) -> dict[str, str]:
configured = project.config.get("tools", {})
if not isinstance(configured, dict) or set(configured) - {"python", "node", "npm"}:
raise ValueError("Project tools must configure only python, node and npm")
tools = {}
python_env = (
Path(
os.environ.get("GOVOPLAN_VENV_ROOT", str(workspace_root / "govoplan/.venv"))
)
/ "bin/python"
)
for name in ("python", "node", "npm"):
default = (
str(python_env)
if name == "python" and python_env.is_file()
else sys.executable
if name == "python"
else name
)
value = configured.get(name) or os.environ.get(name.upper()) or default
if not isinstance(value, str) or not value or "\0" in value:
raise ValueError(f"Invalid executable configuration for {name}")
executable = shutil.which(value)
if executable:
# Keep venv executable symlinks: resolving them loses Python's environment.
tools[name] = os.path.abspath(executable)
else:
tools[name] = value
return tools
def execution_environment(
workspace_root: Path, project: Project, tools: dict[str, str]
) -> dict[str, str]:
env = dict(os.environ)
if (
project.config.get("organization") == "GovOPlaN"
and "schema_version" not in project.config
):
# Managed GovOPlaN checks must not inherit another checkout's scope.
env["GOVOPLAN_WORKSPACE_ROOT"] = str(workspace_root.resolve())
core = next(
repo.path for repo in project.repositories if repo.name == "govoplan-core"
)
env["GOVOPLAN_CORE_ROOT"] = str(core)
env["GOVOPLAN_CORE_SOURCE_ROOT"] = str(core)
directories = [
str(Path(tools[name]).parent)
for name in ("python", "node", "npm")
if Path(tools[name]).is_absolute()
]
core_bins = workspace_root / "govoplan-core/webui/node_modules/.bin"
if core_bins.is_dir():
directories.insert(0, str(core_bins))
env["PATH"] = os.pathsep.join([*directories, env.get("PATH", "")])
env.update({name.upper(): value for name, value in tools.items()})
sources = [
str(repo.path / "src")
for repo in project.repositories
if (repo.path / "src").is_dir()
]
if sources:
env["PYTHONPATH"] = os.pathsep.join(
sources + ([env["PYTHONPATH"]] if env.get("PYTHONPATH") else [])
)
env["GIT_OPTIONAL_LOCKS"] = "0"
env["PYTHONDONTWRITEBYTECODE"] = "1"
return env
def tool_version(executable: str, env: dict[str, str]) -> str:
try:
result = require_capture(
[executable, "--version"],
timeout=15,
max_stdout=4096,
max_stderr=4096,
env=env,
)
return (
(result.stdout or result.stderr).decode(errors="replace").strip()[:256]
if result.returncode == 0
else "unavailable"
)
except (OSError, subprocess.TimeoutExpired, ValueError):
return "unavailable"
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
return (
metadata.st_dev,
metadata.st_ino,
metadata.st_mode,
metadata.st_size,
metadata.st_ctime_ns,
metadata.st_mtime_ns,
)
def _environment_file_hash(path: Path, maximum: int) -> str | None:
"""Read a stable, bounded regular target without changing executable spelling.
Venv executables and package directories may legitimately be symlinks. Only
the read target is resolved; both names and the open descriptor are verified
again afterwards. Missing optional metadata stays absent, but a present
malformed or concurrently changing entry cannot certify an environment.
"""
try:
original = path.lstat()
except FileNotFoundError:
return None
try:
target = path.resolve(strict=True)
target_metadata = target.lstat()
descriptor = os.open(
target,
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
)
with os.fdopen(descriptor, "rb") as handle:
before = os.fstat(handle.fileno())
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
raise ValueError("Environment input must be a bounded regular file")
identity = _file_identity(before)
if identity != _file_identity(target_metadata):
raise ValueError("Environment input changed before fingerprinting")
hasher, size = hashlib.sha256(), 0
for chunk in iter(
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
):
size += len(chunk)
if size > maximum:
raise ValueError("Environment input grew beyond its size bound")
hasher.update(chunk)
if (
_file_identity(os.fstat(handle.fileno())) != identity
or _file_identity(target.lstat()) != identity
or _file_identity(path.lstat()) != _file_identity(original)
or path.resolve(strict=True) != target
):
raise ValueError("Environment input changed during fingerprinting")
return hasher.hexdigest()
except (OSError, RuntimeError) as exc:
raise ValueError("Environment input cannot be fingerprinted safely") from exc
def _discovery_path_shape(path: Path) -> dict:
"""Directory membership, not timestamps changed by ordinary build outputs."""
try:
metadata = path.lstat()
except (FileNotFoundError, NotADirectoryError):
return {"kind": "missing"}
try:
resolved = path.resolve()
try:
target_kind = stat.S_IFMT(resolved.lstat().st_mode)
except (FileNotFoundError, NotADirectoryError):
target_kind = "missing"
if target_kind == stat.S_IFLNK:
# Newer pathlib versions can retain a loop with strict=False.
raise ValueError("Native source discovery cannot be resolved safely")
return {
"kind": stat.S_IFMT(metadata.st_mode),
"resolved": str(resolved),
"target_kind": target_kind,
}
except (OSError, RuntimeError) as exc:
raise ValueError("Native source discovery cannot be resolved safely") from exc
def _native_discovery_fingerprint(workspace_root: Path, project: Project) -> str:
"""Bind glob discovery and registered ownership omitted by narrow Git scopes."""
children = list(islice(workspace_root.iterdir(), MAX_DISCOVERY_CHILDREN + 1))
if len(children) > MAX_DISCOVERY_CHILDREN:
raise ValueError("Native source discovery exceeds its bounded ownership audit")
def shape(path):
return {
name: _discovery_path_shape(path / name if name else path)
for name in ("", "src", "webui")
}
return digest(
{
"version": 1,
"siblings": [
{"name": child.name, "shape": shape(child)}
for child in sorted(children)
if child.name.startswith("govoplan")
],
"registered": [
{"name": repo.name, "path": str(repo.path), "shape": shape(repo.path)}
for repo in sorted(project.repositories, key=lambda item: item.name)
],
}
)
def environment_fingerprint(
workspace_root: Path, project: Project, tools: dict[str, str], env: dict[str, str]
) -> str:
identity = {
"environment": {
key: value
for key, value in env.items()
if key not in {"_", "SHLVL", "PWD", "OLDPWD"}
},
"tools": {},
}
if (
project.config.get("organization") == "GovOPlaN"
and "schema_version" not in project.config
):
identity["native_source_discovery"] = _native_discovery_fingerprint(
workspace_root, project
)
for name, executable in tools.items():
binary_hash = _environment_file_hash(Path(executable), 512 * 1024 * 1024)
identity["tools"][name] = {
"path": executable,
"version": tool_version(executable, env),
"sha256": binary_hash,
}
installed = {}
for repo in project.repositories:
for suffix in (
"node_modules/.package-lock.json",
"webui/node_modules/.package-lock.json",
".venv/pyvenv.cfg",
):
path = repo.path / suffix
value = _environment_file_hash(path, 32 * 1024 * 1024)
if value is not None:
installed[str(path)] = value
try:
result = require_capture(
[
tools["python"],
"-c",
"import importlib.metadata,json; print(json.dumps(sorted((d.metadata['Name'],d.version) for d in importlib.metadata.distributions())))",
],
timeout=30,
max_stdout=1024 * 1024,
env=env,
)
if result.returncode:
raise ValueError("Cannot fingerprint installed Python distributions")
installed["python_distributions"] = hashlib.sha256(result.stdout).hexdigest()
except (OSError, subprocess.TimeoutExpired) as exc:
raise ValueError("Cannot fingerprint Python environment") from exc
identity["installed"] = installed
return digest(identity)
+417
View File
@@ -0,0 +1,417 @@
"""Versioned repository-scoped input identities with run-local content memoization.
No stored receipt supplies commands to this engine. Every snapshot re-reads Git
HEAD/index/flags/membership and opens its inputs; only stable file-content hashes
are memoized, never repository snapshots or prior-run results.
"""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import re
import stat
from .common import META_ROOT, canonical, digest, identifier
from .workspace import Project, git_bytes
FINGERPRINT_VERSION = "repository-inputs-v1"
MAX_SOURCE_BYTES = 64 * 1024 * 1024
MAX_CACHE_FILES = 200_000
MAX_REPOSITORY_ENTRIES = 200_000
_INDEX_RECORD = re.compile(rb"[A-Za-z] [0-7]{6} [a-fA-F0-9]{40,64} [0-3]\t(.*)\Z", re.S)
_RUNTIME_FIELDS = {
"status",
"exit_code",
"duration_seconds",
"log_path",
"log_sha256",
"error",
"started_at",
"finished_at",
"reused_from",
"output_truncated",
"checkpoint_verified",
"checkpoint_at",
"cache_key",
"input_fingerprint",
"reuse_reason",
"omitted_output_bytes",
}
def validate_input_declaration(value: object, repository_names) -> dict:
"""Pure validation shared with planning; does not inspect files or run Git."""
if not isinstance(value, dict) or set(value) != {"repos"}:
raise ValueError(
"Stage inputs must contain only the required repos declaration"
)
names = value["repos"]
if not isinstance(names, list) or not 1 <= len(names) <= 256:
raise ValueError(
"Input repos must be a nonempty bounded list of canonical repository names"
)
for name in names:
identifier(name)
if len(set(names)) != len(names):
raise ValueError("Duplicate input repository reference")
if set(names) - set(repository_names):
raise ValueError(
"Input repos reference an unknown/noncanonical repository name"
)
return {"repos": sorted(names)}
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
return (
metadata.st_dev,
metadata.st_ino,
metadata.st_mode,
metadata.st_size,
metadata.st_ctime_ns,
metadata.st_mtime_ns,
)
def _stats() -> dict[str, int]:
return {
name: 0
for name in (
"repositories",
"git_calls",
"entries",
"files",
"bytes",
"cache_hits",
"tooling_files",
"tooling_bytes",
"tooling_cache_hits",
)
}
class InputSnapshotter:
"""One run's in-memory memo; create a fresh instance for each invocation."""
def __init__(self, project: Project, *, workspace_root: Path):
self.project = project
self.workspace_root = Path(workspace_root).resolve()
self.repositories = {}
for repo in project.repositories:
identifier(repo.name)
if repo.name in self.repositories:
raise ValueError("Duplicate input repository name")
if not repo.path.is_absolute() or not repo.path.resolve().is_relative_to(
self.workspace_root
):
raise ValueError("Input repository path escapes its workspace")
self.repositories[repo.name] = repo
if not self.repositories or len(self.repositories) > 256:
raise ValueError("Input project requires 1256 repositories")
# One most recent stable identity per path. This is deliberately not
# serializable/persisted, and hashes are never reused on mtime alone.
self._files: dict[Path, tuple[tuple[int, ...], bytes]] = {}
def _repo_names(self, names: object) -> list[str]:
return validate_input_declaration({"repos": names}, self.repositories)["repos"]
def scope(self, stage: dict) -> dict:
"""Validate one explicit scope; absent inputs means the whole workspace."""
if "inputs" not in stage:
return {
"version": 1,
"kind": "workspace",
"declared": False,
"repos": sorted(self.repositories),
}
value = validate_input_declaration(stage["inputs"], self.repositories)
return {
"version": 1,
"kind": "repositories",
"declared": True,
"repos": value["repos"],
}
def _file_hash(
self, path: Path, statistics: dict, *, expected=None, tooling: bool = False
) -> bytes:
prefix = "tooling_" if tooling else ""
maximum = 4 * 1024 * 1024 if tooling else MAX_SOURCE_BYTES
descriptor = os.open(
path,
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
)
with os.fdopen(descriptor, "rb") as handle:
before = os.fstat(handle.fileno())
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
raise ValueError("Input must be a bounded regular file")
identity = _file_identity(before)
if expected is not None and _file_identity(expected) != identity:
raise ValueError(
"Input changed before its source identity could be established"
)
statistics[prefix + "files"] += 1
cached = self._files.get(path)
if cached is not None and cached[0] == identity:
value = cached[1]
statistics[prefix + "cache_hits"] += 1
else:
hasher, size = hashlib.sha256(), 0
for chunk in iter(
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
):
size += len(chunk)
if size > maximum:
raise ValueError("Input grew beyond its source identity limit")
hasher.update(chunk)
statistics[prefix + "bytes"] += size
value = hasher.digest()
after = os.fstat(handle.fileno())
# Rechecking the pathname also rejects replacement after the open;
# a stable old FD is not evidence about a newly replaced source file.
current = path.lstat()
if _file_identity(after) != identity or _file_identity(current) != identity:
raise ValueError("Input changed while calculating its source identity")
if path not in self._files and len(self._files) >= MAX_CACHE_FILES:
self._files.clear() # Eviction only causes extra reads, never reuse.
self._files[path] = (identity, value)
return value
def _entry(self, repo, encoded_name: bytes, statistics: dict) -> dict:
relative = Path(os.fsdecode(encoded_name))
if relative.is_absolute() or ".." in relative.parts:
raise ValueError("Unsafe source membership path")
path = repo.path / relative
statistics["entries"] += 1
try:
metadata = path.lstat()
except FileNotFoundError:
return {"kind": "missing"}
if not path.parent.resolve().is_relative_to(repo.path.resolve()):
raise ValueError("Input path escapes its declared repository")
result = {"mode": stat.S_IMODE(metadata.st_mode)}
if stat.S_ISLNK(metadata.st_mode):
target_name = os.readlink(path)
try:
target = path.resolve()
except (OSError, RuntimeError) as exc:
raise ValueError("Input symlink cannot be resolved safely") from exc
if not target.is_relative_to(repo.path.resolve()):
raise ValueError(
"Input symlink target escapes its repository; undeclared target bytes cannot be reused"
)
result.update(
kind="symlink",
target_sha256=hashlib.sha256(os.fsencode(target_name)).hexdigest(),
)
try:
target_metadata = target.lstat()
except FileNotFoundError:
result["target_state"] = "missing"
else:
if not stat.S_ISREG(target_metadata.st_mode):
raise ValueError(
"Input symlink must target a regular file inside its repository"
)
result.update(
target_state="file",
target_mode=stat.S_IMODE(target_metadata.st_mode),
target_content=self._file_hash(
target, statistics, expected=target_metadata
).hex(),
)
if (
_file_identity(path.lstat()) != _file_identity(metadata)
or os.readlink(path) != target_name
):
raise ValueError(
"Input symlink changed while calculating source identity"
)
elif stat.S_ISREG(metadata.st_mode):
result.update(
kind="file",
content=self._file_hash(path, statistics, expected=metadata).hex(),
)
else:
raise ValueError(
"Unsupported source entry; scoped repository inputs require files or in-repository file symlinks"
)
return result
def _repository(self, name: str, statistics: dict) -> str:
repo = self.repositories[name]
statistics["repositories"] += 1
identity = {
"version": FINGERPRINT_VERSION,
"repo": name,
"path": str(repo.path),
}
if not repo.path.exists():
return digest({**identity, "state": "missing"})
if (
not repo.path.resolve().is_relative_to(self.workspace_root)
or not (repo.path / ".git").exists()
):
raise ValueError(
"Cannot establish scoped source identity for an escaped/non-Git repository"
)
statistics["git_calls"] += 1
head = git_bytes(
repo.path, "rev-parse", "--verify", "HEAD", allow_failure=True
).strip()
statistics["git_calls"] += 1
# This preserves stage numbers and assume-unchanged/skip-worktree flags,
# while also listing untracked membership in the same bounded process.
index = git_bytes(
repo.path,
"ls-files",
"--stage",
"-v",
"--cached",
"--others",
"--exclude-standard",
"-z",
)
members = set()
for record in index.split(b"\0"):
if not record:
continue
if record.startswith(b"? "):
members.add(record[2:])
else:
match = _INDEX_RECORD.fullmatch(record)
if not match:
raise ValueError(
"Unsupported Git membership record; cannot establish scoped source identity"
)
members.add(match[1])
if len(members) > MAX_REPOSITORY_ENTRIES:
raise ValueError(
"Repository source membership exceeds its bounded entry count"
)
hasher = hashlib.sha256(
canonical(
{
**identity,
"head_sha256": hashlib.sha256(head).hexdigest(),
"index_sha256": hashlib.sha256(index).hexdigest(),
}
)
)
for member in sorted(members):
hasher.update(
canonical(
{
"name_sha256": hashlib.sha256(member).hexdigest(),
"value": self._entry(repo, member, statistics),
}
)
)
hasher.update(b"\0")
return hasher.hexdigest()
def _source_identity(self, names: list[str], identities: dict[str, str]) -> str:
return digest(
{
"version": FINGERPRINT_VERSION,
"workspace_root": str(self.workspace_root),
"repositories": {name: identities[name] for name in names},
}
)
def _observe(self, names: list[str]) -> dict:
statistics = _stats()
identities = {name: self._repository(name, statistics) for name in names}
complete = set(names) == self.repositories.keys()
return {
"schema_version": 1,
"fingerprint_version": FINGERPRINT_VERSION,
"workspace_root": str(self.workspace_root),
"repository_fingerprints": identities,
"observed_source_fingerprint": self._source_identity(names, identities),
"observed_scope": {
"version": 1,
"kind": "workspace" if complete else "repositories",
"repos": names,
},
"complete_workspace": complete,
"scan_stats": statistics,
}
def source_snapshot(self, repos: list[str] | None = None) -> dict:
"""Compare observed source only, without interpreting any stored commands."""
return self._observe(
sorted(self.repositories) if repos is None else self._repo_names(repos)
)
def _tooling_identity(self, statistics: dict) -> str:
package = Path(__file__).resolve().parent
paths = set()
for path in package.rglob("*.py"):
paths.add(path)
if len(paths) > 256:
raise ValueError("Devkit tooling source exceeds its bounded inventory")
paths.update(
{
package.parent / "devkit.py",
package.parent / "project.schema.json",
META_ROOT / "devkit",
}
)
return digest(
{
str(path.relative_to(META_ROOT)): self._file_hash(
path, statistics, tooling=True
).hex()
for path in sorted(paths)
}
)
def snapshot(self, stages: list[dict], *, tooling_fingerprint: str = "") -> dict:
if not isinstance(stages, list) or len(stages) > 512:
raise ValueError("Input snapshot requires a bounded stage list")
if not isinstance(tooling_fingerprint, str) or len(tooling_fingerprint) > 4096:
raise ValueError("Invalid supplied tooling identity")
scopes = {}
for stage in stages:
if not isinstance(stage, dict):
raise ValueError("Input stages must be objects")
if set(stage) & _RUNTIME_FIELDS:
raise ValueError(
"Input identities require freshly planned stages, not mutable receipt/runtime fields"
)
identity = identifier(stage.get("id"))
if identity in scopes:
raise ValueError("Duplicate input stage identity")
scopes[identity] = self.scope(stage)
names = sorted({name for scope in scopes.values() for name in scope["repos"]})
observed = self._observe(names)
tools = digest(
{
"declared_tools": self.project.config.get("tools", {}),
"devkit_source": self._tooling_identity(observed["scan_stats"]),
"supplied_tooling": tooling_fingerprint,
}
)
entries = {}
for stage in stages:
scope = scopes[stage["id"]]
source = self._source_identity(
scope["repos"], observed["repository_fingerprints"]
)
plan = digest(stage)
entries[stage["id"]] = {
"source_fingerprint": source,
"plan_fingerprint": plan,
"scope": scope,
"fingerprint": digest(
{
"version": FINGERPRINT_VERSION,
"source": source,
"plan": plan,
"tooling": tools,
}
),
}
return {**observed, "tooling_fingerprint": tools, "stages": entries}
+547
View File
@@ -0,0 +1,547 @@
"""Append-only, idempotent Gitea evidence notes; preview is entirely offline."""
from __future__ import annotations
from contextlib import ExitStack
from dataclasses import dataclass
import hashlib
import html
import os
from pathlib import Path
import re
import subprocess
import sys
from typing import Any
from urllib.parse import urlsplit
from .common import META_ROOT, atomic_json, digest, identifier, read_json, redact, resource_lock, state_root
from .inputs import FINGERPRINT_VERSION
_GITEA_PATH = str(META_ROOT / "tools/gitea")
if _GITEA_PATH not in sys.path:
sys.path.insert(0, _GITEA_PATH)
from gitea_common import GiteaClient, RepoTarget, _parse_remote, repo_path # noqa: E402
MAX_NOTE_BYTES = 128 * 1024
MAX_TARGETS = 256
MAX_COVERAGE_NOTES = 2048
MARKER_PREFIX = "<!-- govoplan-devkit:evidence:v1:"
STATUSES = {"planned", "pending", "running", "passed", "failed", "timed_out", "skipped", "cancelled", "interrupted", "blocked", "partial", "stale"}
@dataclass(frozen=True)
class NoteTarget:
root: Path
base_url: str
owner: str
repository: str
issue: int
issue_id: int | None = None
@property
def url(self) -> str:
return f"{self.base_url}/{self.owner}/{self.repository}/issues/{self.issue}"
@property
def path(self) -> str:
return repo_path(self.owner, self.repository, f"/issues/{self.issue}")
def record(self) -> dict:
return {"root": str(self.root), "base_url": self.base_url, "owner": self.owner,
"repository": self.repository, "issue": self.issue, "url": self.url,
**({"issue_id": self.issue_id} if self.issue_id is not None else {})}
def register(subparsers) -> None:
parser = subparsers.add_parser("issues", help="Preview or append evidence to exact Gitea issues")
commands = parser.add_subparsers(dest="issues_command", required=True)
note = commands.add_parser("note", help="Append a deduplicated note; offline dry run by default")
note.add_argument("--root", type=Path, help="Target repository root or child directory")
note.add_argument("--issue", type=int)
note.add_argument("--target-plan", type=Path, help="JSON with schema_version=1 and exact root/issue/url targets")
note.add_argument("--remote", default="origin")
note.add_argument("--env-file", type=Path, help="Private GITEA_TOKEN dotenv file; target overrides are ignored")
note.add_argument("--evidence", help="Local run ID or explicit receipt JSON path")
note.add_argument("--key", default="verification", help="Stable note purpose within an evidence run")
note.add_argument("--summary", dest="note_summary", action="append", default=[])
note.add_argument("--next", dest="next_steps", action="append", default=[])
note.add_argument("--body-file", type=Path, help="Additional Markdown, never executed")
note.add_argument("--note-file", type=Path, help="Structured JSON: summary[], next[], body")
note.add_argument("--apply", action="store_true", help="Explicitly authorize serial comment creation")
note.add_argument("--retry-uncertain", action="store_true", help="After reconciliation, explicitly retry a still-unconfirmed earlier POST")
note.set_defaults(handler=handle_note)
def _text(value: Any, *, maximum: int = 16384) -> str:
if not isinstance(value, str) or len(value.encode("utf-8")) > maximum:
raise ValueError("Note/evidence text must be a bounded string")
if any(ord(character) < 32 and character not in "\n\t\r" for character in value):
raise ValueError("Note/evidence text contains control characters")
if MARKER_PREFIX in value:
raise ValueError("Evidence markers are reserved for the publisher")
return redact(value)
def _positive(value: Any) -> int:
if type(value) is not int or value <= 0:
raise ValueError("Issue and comment identities must be positive integers")
return value
def _base_url(value: str) -> str:
parsed = urlsplit(value)
if (parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username
or parsed.password or parsed.query or parsed.fragment or "\\" in value
or any(ord(char) < 33 for char in value)):
raise ValueError("Gitea target must be a credential-free HTTP(S) URL")
if any(part in {".", ".."} or "%" in part for part in parsed.path.split("/")):
raise ValueError("Gitea base URL contains an ambiguous path")
try:
parsed.port
except ValueError as exc:
raise ValueError("Invalid Gitea port") from exc
return value.rstrip("/")
def _name(value: str) -> str:
if not isinstance(value, str) or not re.fullmatch(r"[A-Za-z0-9_.-]+", value) or value in {".", ".."}:
raise ValueError("Invalid Gitea owner or repository name")
return value
def resolve_target(root: Path, issue: int, workspace_root: Path, *, remote: str = "origin",
expected_url: str | None = None, issue_id: int | None = None) -> NoteTarget:
requested = root if root.is_absolute() else workspace_root / root
resolved = requested.resolve()
if not resolved.is_relative_to(workspace_root.resolve()) or not resolved.is_dir():
raise ValueError("Issue target must be an existing repository inside the selected workspace")
if not re.fullmatch(r"[A-Za-z0-9_.-]+", remote):
raise ValueError("Invalid Git remote name")
command = subprocess.run(["git", "-C", str(resolved), "rev-parse", "--show-toplevel"],
capture_output=True, text=True, timeout=15)
if command.returncode:
raise ValueError("Issue target is not a Git checkout")
actual = Path(command.stdout.strip()).resolve()
if not actual.is_relative_to(workspace_root.resolve()):
raise ValueError("Resolved issue repository escapes the workspace")
result = subprocess.run(["git", "-C", str(actual), "remote", "get-url", remote],
capture_output=True, text=True, timeout=15)
if result.returncode or len(result.stdout) > 8192:
raise ValueError("Target repository has no usable Git remote")
remote_url = result.stdout.strip()
parsed = urlsplit(remote_url)
if (any(ord(char) < 33 for char in remote_url) or "\\" in remote_url
or parsed.query or parsed.fragment):
raise ValueError("Ambiguous Git remote URL cannot bind an issue target")
if parsed.scheme in {"http", "https"} and (parsed.username or parsed.password):
raise ValueError("Credential-bearing Git remotes cannot be used for issue publishing")
base, owner, repository = _parse_remote(remote_url)
target = NoteTarget(actual, _base_url(base), _name(owner), _name(repository), _positive(issue),
_positive(issue_id) if issue_id is not None else None)
if expected_url is not None and expected_url != target.url:
raise ValueError("Target plan issue URL does not match the exact repository remote and issue number")
return target
def _targets(args) -> list[NoteTarget]:
workspace_root = Path(args.workspace_root).resolve()
if args.target_plan:
if args.root is not None or args.issue is not None:
raise ValueError("Use either --root/--issue or --target-plan")
payload = read_json(args.target_plan, max_bytes=1024 * 1024)
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
raise ValueError("Target plan requires schema_version 1")
records = payload.get("targets")
if not isinstance(records, list) or not 1 <= len(records) <= MAX_TARGETS:
raise ValueError("Target plan must contain 1256 exact targets")
targets = []
for record in records:
if (not isinstance(record, dict) or not isinstance(record.get("root"), str)
or not isinstance(record.get("url"), str)):
raise ValueError("Each target plan entry requires root, issue and URL")
target = resolve_target(Path(record["root"]), record.get("issue"), workspace_root,
remote=args.remote, expected_url=record["url"], issue_id=record.get("issue_id"))
for key in ("base_url", "owner", "repository"):
if key in record and record[key] != getattr(target, key):
raise ValueError("Target plan repository identity is inconsistent")
targets.append(target)
else:
if args.root is None or args.issue is None:
raise ValueError("An exact --root and --issue are required")
targets = [resolve_target(args.root, args.issue, workspace_root, remote=args.remote)]
if len({target.url for target in targets}) != len(targets):
raise ValueError("Duplicate issue targets are not accepted")
if len({target.base_url for target in targets}) != 1:
raise ValueError("A credentialed target plan must be confined to one exact Gitea base URL")
return targets
def validate_receipt(payload: Any, workspace_root: Path) -> dict:
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
raise ValueError("Evidence requires receipt schema_version 1")
identifier(payload.get("run_id"))
recorded_workspace = payload.get("workspace_root")
if not isinstance(recorded_workspace, str) or Path(recorded_workspace).resolve() != workspace_root.resolve():
raise ValueError("Evidence belongs to another workspace")
fingerprint = payload.get("source_fingerprint")
if not isinstance(fingerprint, str) or not re.fullmatch(r"[a-f0-9]{64}", fingerprint):
raise ValueError("Evidence requires a recorded source fingerprint")
if not isinstance(payload.get("status"), str) or payload["status"] not in STATUSES:
raise ValueError("Unknown evidence status")
_text(payload.get("generated_at"), maximum=128)
if payload.get("finished_at") is not None:
_text(payload["finished_at"], maximum=128)
stages = payload.get("stages")
if not isinstance(stages, list) or len(stages) > 1000:
raise ValueError("Evidence requires a bounded stage list")
seen = set()
for stage in stages:
if not isinstance(stage, dict):
raise ValueError("Invalid evidence stage")
identity = _text(stage.get("id"), maximum=256)
if not identity or identity in seen or "\n" in identity:
raise ValueError("Evidence stage IDs must be unique")
seen.add(identity)
if not isinstance(stage.get("status"), str) or stage["status"] not in STATUSES:
raise ValueError("Unknown evidence stage status")
code = stage.get("exit_code")
if code is not None and type(code) is not int:
raise ValueError("Evidence exit code must be an integer or null")
duration = stage.get("duration_seconds")
if duration is not None and (type(duration) not in {int, float} or not 0 <= duration < 31536000):
raise ValueError("Invalid evidence stage duration")
if stage.get("log_path") is not None:
_text(stage["log_path"], maximum=4096)
if stage["status"] == "passed" and code != 0:
raise ValueError("Passed evidence stage must have exit code zero")
if payload["status"] == "passed" and (not stages or any(stage["status"] != "passed" for stage in stages)):
raise ValueError("Passed evidence is inconsistent with its stages")
if payload["status"] == "passed" and payload.get("snapshot_verified") is not True:
raise ValueError("Passed evidence requires a verified source snapshot")
coverage_notes(stages)
from .checkpoints import validate_checkpoint_receipt
validate_checkpoint_receipt(payload)
return payload
def coverage_notes(stages: list[dict]) -> list[str]:
"""Keep declared coverage limits visible without interpreting them as commands."""
notes, total, size = [], 0, 0
for stage in stages:
values = stage.get("coverage_notes", [])
if not isinstance(values, list):
raise ValueError("Evidence coverage_notes must be a bounded list of strings")
total += len(values)
if total > MAX_COVERAGE_NOTES:
raise ValueError("Evidence coverage_notes exceed their count bound")
for value in values:
text = _text(value, maximum=4096).strip()
if not text:
raise ValueError("Evidence coverage notes cannot be empty")
size += len(text.encode())
if size > MAX_NOTE_BYTES:
raise ValueError("Evidence coverage_notes exceed their size bound")
notes.append(text)
return list(dict.fromkeys(notes))
def evidence_record(value: str | None, args) -> dict | None:
if not value:
return None
workspace_root = Path(args.workspace_root).resolve()
external = value.endswith(".json") or "/" in value or "\\" in value
if external:
path = Path(value).expanduser().absolute()
payload = read_json(path)
origin = "external-unverified"
else:
from .runner import read_receipt
identity = identifier(value)
payload = read_receipt(workspace_root, args.state_dir, identity)
path = state_root(workspace_root, args.state_dir) / "runs" / identity / "receipt.json"
origin = "local-integrity-checked"
receipt = validate_receipt(payload, workspace_root)
from .workspace import load_project, source_fingerprint
expected_project = str(args.project.resolve()) if getattr(args, "project", None) else None
source_state = "not-compared"
current = None
if receipt.get("project_file") == expected_project:
try:
project = load_project(workspace_root, getattr(args, "project", None))
if receipt.get("fingerprint_version") == FINGERPRINT_VERSION:
from .inputs import InputSnapshotter
scope = receipt.get("source_scope", {})
if not isinstance(scope, dict) or not isinstance(scope.get("repos"), list):
raise ValueError("Scoped evidence requires an explicit recorded repository scope")
current = InputSnapshotter(project, workspace_root=workspace_root).source_snapshot(scope["repos"])["observed_source_fingerprint"]
else:
current = source_fingerprint(project)
source_state = "matches-current" if current == receipt["source_fingerprint"] else "historical-source-differs"
except (OSError, ValueError, subprocess.SubprocessError):
source_state = "current-source-unavailable"
else:
source_state = "different-project-not-compared"
return {"run_id": redact(receipt["run_id"]), "status": receipt["status"], "origin": origin,
"receipt_path": redact(str(path)), "source_fingerprint": receipt["source_fingerprint"],
"current_source_fingerprint": current, "source_state": source_state,
"snapshot_verified": receipt.get("snapshot_verified") is True,
"coverage_notes": coverage_notes(receipt["stages"]) + ([
"Source comparison is limited to the recorded repository input scope; this is not a whole-workspace or artifact verification."
] if receipt.get("fingerprint_version") == FINGERPRINT_VERSION and not receipt.get("source_scope", {}).get("complete_workspace") else []),
"fingerprint_version": receipt.get("fingerprint_version"),
"source_scope": receipt.get("source_scope"),
"generated_at": redact(receipt["generated_at"]), "finished_at": redact(receipt["finished_at"]) if receipt.get("finished_at") else None,
"stages": [{**{key: redact(stage[key]) if isinstance(stage.get(key), str) else stage.get(key)
for key in ("id", "status", "exit_code", "duration_seconds", "log_path")},
"coverage_notes": coverage_notes([stage])} for stage in receipt["stages"]],
"attestation": "Receipt metadata is not an independent attestation, a live verification, or a completed module review."}
def _structured_note(args) -> dict:
payload = read_json(args.note_file, max_bytes=MAX_NOTE_BYTES) if args.note_file else {}
if not isinstance(payload, dict) or set(payload) - {"summary", "next", "body"}:
raise ValueError("Structured note accepts only summary[], next[] and body")
result = {}
for key, values in (("summary", args.note_summary), ("next", args.next_steps)):
supplied = payload.get(key, [])
if not isinstance(supplied, list) or len(supplied) + len(values) > 100:
raise ValueError("Summary and next steps must be bounded lists")
result[key] = [_text(item).strip() for item in [*supplied, *values]]
body = _text(payload.get("body", ""), maximum=MAX_NOTE_BYTES)
if args.body_file:
# Reuse the bounded, no-symlink file reader by wrapping no content in code.
from .common import reject_symlinks
import stat
reject_symlinks(args.body_file)
descriptor = os.open(args.body_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > MAX_NOTE_BYTES:
raise ValueError("Additional body must be a bounded regular file")
text = handle.read(MAX_NOTE_BYTES + 1).decode("utf-8")
body += "\n\n" + _text(text, maximum=MAX_NOTE_BYTES)
result["body"] = body.strip()
return result
def _cell(value: Any) -> str:
return html.escape(str(value if value is not None else "")).replace("|", "&#124;").replace("`", "&#96;").replace("\n", " ")
def render_note(note: dict, evidence: dict | None, target: NoteTarget, key: str) -> tuple[str, str]:
identity = {"target": target.url, "key": identifier(key),
"evidence": evidence["run_id"] if evidence else digest(note)}
marker = MARKER_PREFIX + digest(identity) + " -->"
lines = [marker, "## Development evidence", "", f"Issue: {target.url}", ""]
for field, title in (("summary", "Summary"), ("next", "Next / remaining")):
if note[field]:
lines += [f"### {title}", "", *["- " + item for item in note[field]], ""]
if note["body"]:
lines += [note["body"], ""]
if evidence:
lines += ["### Recorded check evidence", "", f"Run: `{evidence['run_id']}`; reported result: **{evidence['status']}**.",
f"Receipt origin: `{evidence['origin']}`; source comparison: `{evidence['source_state']}`.",
f"Recorded source fingerprint: `{evidence['source_fingerprint']}`.",
f"Finished: {_cell(evidence['finished_at'])}; receipt: `{_cell(evidence['receipt_path'])}`.", "",
"| Stage | Reported status | Exit | Seconds | Local log reference |", "| --- | --- | --- | --- | --- |"]
for stage in evidence["stages"]:
lines.append("| " + " | ".join(_cell(stage.get(field)) for field in ("id", "status", "exit_code", "duration_seconds", "log_path")) + " |")
if evidence.get("coverage_notes"):
lines += ["", "### Coverage limitations / checks not included", "",
"A passing recorded stage does not mean these omitted checks ran.", "",
*["- " + _cell(value) for value in evidence["coverage_notes"]]]
lines += ["", evidence["attestation"], "Local logs are referenced only; their content has not been read or uploaded.", ""]
lines += ["This comment does not close the issue, complete its review, or change its checklist."]
body = redact("\n".join(lines).rstrip() + "\n")
if len(body.encode()) > MAX_NOTE_BYTES:
raise ValueError("Rendered note exceeds its size bound")
return marker, body
def _token(env_file: Path | None) -> str:
values = {}
if env_file is not None:
from .common import reject_symlinks
reject_symlinks(env_file)
import stat
descriptor = os.open(env_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 65536:
raise ValueError("Credential file must be a bounded regular file")
content = handle.read(65537)
if len(content) > 65536:
raise ValueError("Credential file exceeds its size limit")
for line in content.decode("utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if line.startswith("export "):
line = line[7:].strip()
key, separator, value = line.partition("=")
if not separator or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", key.strip()):
raise ValueError("Invalid credential file format")
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
value = value[1:-1]
if key.strip() in values:
raise ValueError("Duplicate credential file keys")
values[key.strip()] = value
token = os.environ.get("GITEA_TOKEN") or values.get("GITEA_TOKEN")
if not token or len(token) > 8192 or any(char.isspace() for char in token):
raise ValueError("GITEA_TOKEN is required for --apply; use the environment or --env-file")
return token
def make_client(target: NoteTarget, token: str):
return GiteaClient(RepoTarget(target.base_url, target.owner, target.repository), token)
def _issue(client, target: NoteTarget) -> dict:
issue = client.request_json("GET", target.path)
if (not isinstance(issue, dict) or issue.get("number") != target.issue or issue.get("html_url") != target.url
or issue.get("pull_request") is not None):
raise ValueError("Remote issue identity does not match the exact target")
_positive(issue.get("id"))
if target.issue_id is not None and issue["id"] != target.issue_id:
raise ValueError("Remote issue ID changed from the target plan")
return issue
def _comments(client, target: NoteTarget) -> list[dict]:
comments, seen = [], set()
for page in range(1, 10001):
values = client.request_json("GET", target.path + "/comments", query={"page": page, "limit": 50})
if not isinstance(values, list):
raise ValueError("Remote comment pagination did not return a list")
if not values:
return comments
for comment in values:
if not isinstance(comment, dict) or not isinstance(comment.get("body"), str):
raise ValueError("Remote comment has an invalid shape")
identity = _positive(comment.get("id"))
if identity in seen:
raise ValueError("Repeated comment pagination; cannot establish a complete duplicate check")
seen.add(identity)
comments.append(comment)
if len(comments) > 100000:
raise ValueError("Remote comments exceed the bounded duplicate-check limit")
raise ValueError("Remote comment pagination did not terminate")
def _existing(comments: list[dict], marker: str, body: str) -> dict | None:
matches = [comment for comment in comments if marker in comment["body"]]
if len(matches) > 1 or (matches and matches[0]["body"] != body):
raise ValueError("Evidence marker collision; existing comments are preserved")
return matches[0] if matches else None
def _readback(client, target: NoteTarget, comment: dict, body: str) -> dict:
identity = _positive(comment.get("id"))
fresh = client.request_json("GET", repo_path(target.owner, target.repository, f"/issues/comments/{identity}"))
issue_api_url = target.base_url + "/api/v1" + target.path
if not isinstance(fresh, dict) or fresh.get("id") != identity or fresh.get("body") != body:
raise ValueError("Posted comment read-back did not match")
if not fresh.get("html_url") and not fresh.get("issue_url"):
raise ValueError("Comment read-back has no issue binding")
if fresh.get("issue_url") and fresh["issue_url"] != issue_api_url:
raise ValueError("Comment read-back belongs to another issue")
if fresh.get("html_url") and fresh["html_url"].split("#", 1)[0] != target.url:
raise ValueError("Comment read-back URL belongs to another issue")
return fresh
def handle_note(args) -> dict:
targets = _targets(args)
note = _structured_note(args)
evidence = evidence_record(args.evidence, args)
if not evidence and not any((note["summary"], note["next"], note["body"])):
raise ValueError("Provide evidence or a nonempty structured note")
if args.retry_uncertain and not args.apply:
raise ValueError("--retry-uncertain requires --apply")
prepared = [(target, *render_note(note, evidence, target, args.key)) for target in targets]
result = {"schema_version": 1, "operation": "issues.note", "apply": bool(args.apply),
"evidence": evidence, "targets": [{**target.record(), "marker": marker, "body": body, "status": "would-post"} for target, marker, body in prepared],
"summary": [f"{'Apply' if args.apply else 'Offline dry run'}: {len(targets)} exact issue target(s); issue bodies and states are preserved."]}
if evidence and evidence["coverage_notes"]:
result["summary"].append(f"Evidence has {len(evidence['coverage_notes'])} coverage limitation(s), retained in the note; omitted checks are not claimed as passed.")
if not args.apply:
return result
token = _token(args.env_file)
# Tokens loaded from an explicit file are not inserted into the process environment.
for record in result["targets"]:
if token in record["body"]:
raise ValueError("A credential occurs in note content; refusing publication")
state = state_root(Path(args.workspace_root), args.state_dir)
with ExitStack() as stack:
for target, marker, _body in sorted(prepared, key=lambda item: item[0].url):
stack.enter_context(resource_lock(state / "locks", "issues.note:" + marker))
clients, bindings, journals = {}, {}, {}
try:
# Validate every target and every existing marker before the first write.
for target, marker, body in prepared:
client = make_client(target, token)
stack.callback(client.close)
clients[target.url] = client
bindings[target.url] = _issue(client, target)["id"]
_existing(_comments(client, target), marker, body)
journal_path = state / "issue-notes" / (hashlib.sha256(marker.encode()).hexdigest() + ".json")
prior = read_json(journal_path) if journal_path.exists() else None
if prior is not None and (not isinstance(prior, dict) or prior.get("schema_version") != 1
or prior.get("target") != target.url or prior.get("body_digest") != digest(body)
or prior.get("issue_id") != bindings[target.url]
or prior.get("status") not in {"posting", "uncertain", "verified"}):
raise ValueError("Local evidence journal identity collision; inspect before retrying")
journals[target.url] = (journal_path, prior)
for index, (target, marker, body) in enumerate(prepared):
client = clients[target.url]
output = result["targets"][index]
journal_path, prior = journals[target.url]
issue = _issue(client, target)
if issue["id"] != bindings[target.url]:
raise ValueError("Issue identity changed after preflight")
existing = _existing(_comments(client, target), marker, body)
if existing:
verified = _readback(client, target, existing, body)
output.update(status="existing-verified", comment_id=verified["id"])
atomic_json(journal_path, {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "verified", "comment_id": verified["id"]})
continue
if prior and prior.get("status") in {"uncertain", "posting", "verified"} and not args.retry_uncertain:
output["status"] = "uncertain-retry-required"
result["_exit_code"] = 2
result["summary"].append("An earlier POST is not visible after complete reconciliation; explicit --retry-uncertain is required. No further posts attempted.")
break
journal = {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "posting"}
atomic_json(journal_path, journal)
try:
posted = client.request_json("POST", target.path + "/comments", body={"body": body})
verified = _readback(client, target, posted, body)
unique = _existing(_comments(client, target), marker, body)
if unique is None or unique["id"] != verified["id"]:
raise ValueError("New comment is not uniquely visible in its issue")
output.update(status="posted-verified", comment_id=verified["id"])
except Exception:
# POST is never replayed automatically, even after a timeout or bad response.
atomic_json(journal_path, {**journal, "status": "uncertain"})
try:
observed = _existing(_comments(client, target), marker, body)
verified = _readback(client, target, observed, body) if observed else None
except Exception:
verified = None
if verified is None:
output["status"] = "uncertain"
result["_exit_code"] = 2
result["summary"].append("POST/read-back could not be confirmed. No automatic retry and no further posts; reconcile this target before continuing.")
break
output.update(status="reconciled-verified", comment_id=verified["id"])
atomic_json(journal_path, {**journal, "status": "verified", "comment_id": output["comment_id"]})
except Exception:
# HTTP response/error bodies may contain secrets; never return them to a report.
result["_exit_code"] = 2
result["summary"].append("Gitea validation or local journal checks failed; existing issues/comments were preserved. No further posts attempted.")
finally:
token = ""
for record in result["targets"]:
if record["status"] == "would-post":
record["status"] = "not-attempted"
result["summary"].append("; ".join(f"{record['url']}: {record['status']}" for record in result["targets"]))
return result
File diff suppressed because it is too large Load Diff
+251
View File
@@ -0,0 +1,251 @@
"""Read-only run discovery and bounded provisional output, never passing evidence."""
from __future__ import annotations
from datetime import datetime
import os
from pathlib import Path
import re
import time
from .common import (
atomic_json,
identifier,
now,
read_json,
redact,
reject_symlinks,
state_root,
)
MAX_LIVE_BYTES = 65536
MAX_HISTORY_ENTRIES = 10000
MAX_HISTORY_SCAN = 500
ANSI = re.compile(r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\))")
CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
def display_text(value: str) -> str:
return redact(CONTROLS.sub("", ANSI.sub("", value)))
def _closed_lines(value: bytes) -> bytes:
boundary = value.rfind(b"\n")
return value[: boundary + 1] if boundary >= 0 else b""
def capture_text(snapshot, *, provisional: bool = False) -> str:
"""Do not expose partial secret/context fragments at live or retention boundaries."""
data = snapshot.stdout
omitted = snapshot.omitted_stdout_bytes
if omitted:
head = _closed_lines(data[: snapshot.stdout_head_bytes])
tail = data[snapshot.stdout_head_bytes :]
# A rolling tail can start inside Authorization or a secret: drop that fragment.
tail = tail.partition(b"\n")[2]
if provisional:
tail = _closed_lines(tail)
return (
display_text(head.decode("utf-8", errors="replace"))
+ f"\n[Output truncated: {omitted} bytes omitted between retained head and tail; cut boundary lines are withheld.]\n"
+ display_text(tail.decode("utf-8", errors="replace"))
)
if provisional:
data = _closed_lines(data)
return display_text(data.decode("utf-8", errors="replace"))
def bounded_display(value: str, maximum: int, *, tail_only: bool = False) -> str:
if type(maximum) is not int or maximum < 1:
raise ValueError("Display bound must be a positive integer")
encoded = value.encode("utf-8")
if len(encoded) <= maximum:
return value
if tail_only:
return encoded[-maximum:].decode("utf-8", errors="ignore")
marker = b"\n[Display bound: middle omitted; retained beginning and final output follow.]\n"
if maximum <= len(marker):
return encoded[-maximum:].decode("utf-8", errors="ignore")
available = max(0, maximum - len(marker))
head = available // 2
return (
encoded[:head].decode("utf-8", errors="ignore")
+ marker.decode()
+ encoded[-(available - head) :].decode("utf-8", errors="ignore")
)
def write_live(log_path: Path, stage_id: str, snapshot, started: float) -> None:
# Complete lines only, even at final callback: the separately finalized log
# may include an unterminated final line after normal redaction.
excerpt = bounded_display(
capture_text(snapshot, provisional=True), MAX_LIVE_BYTES, tail_only=True
)
atomic_json(
log_path.with_suffix(".live.json"),
{
"schema_version": 1,
"run_id": log_path.parent.name,
"stage_id": stage_id,
"provisional": True,
"updated_at": now(),
"elapsed_seconds": round(time.monotonic() - started, 3),
"output_truncated": snapshot.truncated,
"omitted_bytes": snapshot.omitted_stdout_bytes,
"excerpt": excerpt,
},
)
def read_live(
workspace_root: Path, state_dir: Path | None, run_id: str, stage_id: str
) -> dict:
identifier(run_id)
identifier(stage_id)
path = (
state_root(workspace_root, state_dir)
/ "runs"
/ run_id
/ (stage_id + ".live.json")
)
if not path.exists() and not path.is_symlink():
return {"provisional": True, "excerpt": "", "live_available": False}
payload = read_json(path, max_bytes=MAX_LIVE_BYTES * 6 + 8192)
if (
not isinstance(payload, dict)
or payload.get("schema_version") != 1
or payload.get("run_id") != run_id
or payload.get("stage_id") != stage_id
or payload.get("provisional") is not True
or not isinstance(payload.get("excerpt"), str)
or len(payload["excerpt"].encode("utf-8")) > MAX_LIVE_BYTES
or not isinstance(payload.get("updated_at"), str)
or type(payload.get("elapsed_seconds")) not in {int, float}
or payload["elapsed_seconds"] < 0
or type(payload.get("output_truncated")) is not bool
):
raise ValueError("Invalid provisional log snapshot")
return {
"provisional": True,
"live_available": True,
"excerpt": display_text(payload["excerpt"]),
"updated_at": payload.get("updated_at"),
"elapsed_seconds": payload.get("elapsed_seconds"),
"output_truncated": payload.get("output_truncated"),
}
def elapsed_seconds(receipt: dict) -> float | None:
try:
started = datetime.fromisoformat(receipt["generated_at"])
finished = datetime.fromisoformat(receipt.get("finished_at") or now())
return max(0, round((finished - started).total_seconds(), 3))
except (KeyError, TypeError, ValueError):
return None
def list_runs(args) -> dict:
from .runner import read_receipt
limit = getattr(args, "limit", 10)
before = getattr(args, "before", None)
if type(limit) is not int or not 1 <= limit <= 100:
raise ValueError("Run history limit must be between 1 and 100")
if before:
identifier(before)
base = state_root(args.workspace_root, args.state_dir) / "runs"
reject_symlinks(base)
names = []
if base.exists():
with os.scandir(base) as entries:
for index, entry in enumerate(entries):
if index >= MAX_HISTORY_ENTRIES:
raise ValueError(
"Run history exceeds its directory bound; archive old evidence explicitly before listing"
)
if not entry.name.startswith("."):
identifier(entry.name)
names.append(entry.name)
names = sorted(
(name for name in names if before is None or name < before), reverse=True
)
rows, examined, cursor = [], 0, None
wanted_project = (
str(args.project.resolve()) if getattr(args, "project", None) else None
)
for identity in names:
examined += 1
cursor = identity
try:
record = read_receipt(args.workspace_root, args.state_dir, identity)
if wanted_project and record.get("project_file") != wanted_project:
if examined >= MAX_HISTORY_SCAN:
break
continue
rows.append(
{
"run_id": identity,
"status": record["status"],
"phase": record.get("phase"),
"profile": record.get("profile"),
"generated_at": record.get("generated_at"),
"elapsed_seconds": elapsed_seconds(record),
"snapshot_verified": record["snapshot_verified"],
"passed_stages": sum(
item["status"] == "passed" for item in record["stages"]
),
"total_stages": len(record["stages"]),
}
)
except (OSError, ValueError) as exc:
# Corrupt/newest evidence is visible, never silently replaced with an older pass.
rows.append(
{
"run_id": display_text(identity),
"status": "invalid",
"error": display_text(str(exc)),
}
)
if len(rows) >= limit or examined >= MAX_HISTORY_SCAN:
break
next_cursor = cursor if examined < len(names) else None
lines = [
f"{item['run_id']}: {item['status']}"
+ (
f" ({item.get('profile')}; {item.get('elapsed_seconds')}s)"
if item["status"] != "invalid"
else "" + item["error"]
)
for item in rows
]
if not lines:
lines = ["No matching check runs found; no verification is implied."]
if next_cursor:
lines.append("More history: use --before " + display_text(next_cursor))
return {
"runs": rows,
"next_cursor": next_cursor,
"examined": examined,
"summary": lines,
"_exit_code": 1 if any(item["status"] == "invalid" for item in rows) else 0,
}
def latest_run(args) -> dict:
from argparse import Namespace
from .runner import read_receipt, summarize
selected = list_runs(Namespace(**{**vars(args), "limit": 1}))
if not selected["runs"]:
return {
"status": "not_found",
"summary": selected["summary"],
"next_cursor": selected["next_cursor"],
"_exit_code": 1,
}
row = selected["runs"][0]
if row["status"] == "invalid":
raise ValueError(
"Latest run is invalid; inspect run history instead of assuming an older pass"
)
return summarize(read_receipt(args.workspace_root, args.state_dir, row["run_id"]))
+173
View File
@@ -0,0 +1,173 @@
"""Bounded package metadata and conservative direct-Node test discovery."""
from __future__ import annotations
from pathlib import Path
import re
import shlex
from .common import digest, read_json, redact_argv, reject_symlinks
MAX_PACKAGE_BYTES = 1024 * 1024
MAX_SCRIPTS = 512
CORE_COMPONENT_SUITES = (
"data-grid-actions",
"dialog-focus",
"explorer-tree",
"icon-button",
"layout-primitives",
"mail-components",
"metric-card",
"page-layout",
"workspace-layout",
"people-picker",
"password-field",
"resource-access",
"action-blocker",
"documentation-help",
"selection-list",
"wysiwyg-editor",
)
CORE_RUNNER = "scripts/run-component-tests.mjs"
def read_package(path: Path) -> dict:
try:
package = read_json(path, max_bytes=MAX_PACKAGE_BYTES)
except (OSError, ValueError) as exc:
raise ValueError(f"Cannot safely read package metadata: {path}") from exc
if not isinstance(package, dict):
raise ValueError(f"Package metadata must be an object: {path}")
scripts = package.get("scripts", {})
if not isinstance(scripts, dict) or len(scripts) > MAX_SCRIPTS:
raise ValueError(f"Package scripts must be a bounded object: {path}")
for name, command in scripts.items():
if (
not isinstance(name, str)
or not 1 <= len(name) <= 128
or any(char in name for char in "\0\r\n")
):
raise ValueError(f"Package script names must be bounded strings: {path}")
if (
not isinstance(command, str)
or not 1 <= len(command) <= 8192
or "\0" in command
):
raise ValueError(
f"Package script commands must be bounded nonempty strings: {path}"
)
return package
def core_component_alias(
repo_name: str, name: str, command: str, package_path: Path
) -> str | None:
if repo_name != "govoplan-core" or package_path.parent.name != "webui":
return None
if name == "test:components" and command == f"node {CORE_RUNNER}":
return "all"
for suite in CORE_COMPONENT_SUITES:
if name == "test:" + suite and command == f"node {CORE_RUNNER} {suite}":
return suite
return None
def direct_node(command: str, package_path: Path) -> tuple[list[str] | None, str]:
try:
parts = shlex.split(command)
except ValueError:
return None, "Malformed command quoting; no command was guessed or executed."
if not parts or parts[0] != "node":
return (
None,
"Not a direct Node source test; use its explicitly reviewed owning workflow.",
)
offset = 2 if len(parts) > 1 and parts[1] == "--test" else 1
if len(parts) != offset + 1 or not re.fullmatch(
r"(?:scripts|tests)/[A-Za-z0-9_.-]+\.mjs", parts[offset]
):
return (
None,
"Compound command, flags or arguments are unsupported by scoped source discovery.",
)
if parts[offset] == CORE_RUNNER:
return (
None,
"Only exact known Core component aliases belong to the shared UI batch.",
)
target = package_path.parent / parts[offset]
try:
reject_symlinks(target)
if not target.is_file() or not target.resolve().is_relative_to(
package_path.parent.resolve()
):
return None, "Declared test target is missing or escapes its package."
except (OSError, ValueError):
return None, "Declared test target is not a safe regular package file."
return [
"{node}",
*parts[1:offset],
str(target),
], "Direct package-owned source test."
def declared_tests(repo, package_path: Path) -> list[dict]:
package = read_package(package_path)
result = []
for name, command in sorted(package.get("scripts", {}).items()):
if name != "test" and not name.startswith("test:"):
continue
component = core_component_alias(repo.name, name, command, package_path)
argv, reason = (
direct_node(command, package_path)
if component is None
else (None, "Exact known Core component alias.")
)
result.append(
{
"repo": repo.name,
"package_path": str(package_path),
"name": name,
"command_sha256": digest(command),
"argv": redact_argv(argv) if argv else None,
"component_suite": component,
"reason": reason,
"_argv": argv,
}
)
return result
def discovered_sources(repo) -> list[dict]:
webui = repo.path / "webui"
found = {}
for folder in ("scripts", "tests"):
for pattern in ("test-interface-pattern*.mjs", "*structure*.mjs"):
for target in (webui / folder).glob(pattern):
command = (
"node "
+ ("--test " if target.name.endswith(".test.mjs") else "")
+ target.relative_to(webui).as_posix()
)
argv, reason = direct_node(command, webui / "package.json")
found[str(target)] = {
"repo": repo.name,
"package_path": str(webui / "package.json"),
"name": "file:" + target.relative_to(webui).as_posix(),
"discovered": True,
"command_sha256": digest(command),
"argv": redact_argv(argv) if argv else None,
"component_suite": None,
"reason": reason,
"_argv": argv,
}
if len(found) > MAX_SCRIPTS:
raise ValueError(
f"Too many discovered source checks in {repo.name}"
)
return [found[key] for key in sorted(found)]
def source_stage_id(repo_name: str, argv: list[str]) -> str:
stem = Path(argv[-1]).stem
return f"{repo_name}.{stem}"[:110] + "." + digest(argv)[:12]

Some files were not shown because too many files have changed in this diff Show More