Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 zemion
|
2ffdb23f69 | ||
|
zemion
|
14b19fbead | ||
|
zemion
|
845dcbafdb | ||
|
zemion
|
58d320d9b3 | ||
|
zemion
|
9554657bb5 |
@@ -23,6 +23,9 @@ jobs:
|
||||
- name: Test declarative deployment bundle
|
||||
working-directory: govoplan
|
||||
run: python -m unittest -v tests.test_deployment_installer
|
||||
- name: Test WebUI installer retry failures
|
||||
working-directory: govoplan
|
||||
run: python -m unittest -v tests.test_webui_release_dependency_retries
|
||||
- name: Build single-file deployer artifact
|
||||
working-directory: govoplan
|
||||
run: |
|
||||
|
||||
@@ -15,6 +15,13 @@ platform behavior remains owned by the corresponding module repository.
|
||||
|
||||
## Working Rules
|
||||
|
||||
- Start repeated workflows with `./devkit commands`; use `context --changed`,
|
||||
`check --profile quick --changed --dry-run`, and `review MODULE` instead of
|
||||
reconstructing repository/check inventories. See `docs/operations/DEVKIT.md`.
|
||||
- Prefer compact check receipts (`status`, `summary`, `logs`) to repeated full
|
||||
log dumps. `check --profile full` retains the required focused gate; targeted
|
||||
profiles or cached results do not waive required verification or approvals.
|
||||
|
||||
- Treat Gitea issues as the canonical backlog and state log.
|
||||
- Preserve optional module boundaries and use Core contracts or capabilities for integrations.
|
||||
- Prefer targeted checks before full workspace scans.
|
||||
|
||||
@@ -19,6 +19,17 @@ installed modules/connectors discovered by core.
|
||||
|
||||
## Common Commands
|
||||
|
||||
For repeated development and review workflows, start with the unified command
|
||||
suite. [Developer command guide](docs/operations/DEVKIT.md) documents profiles,
|
||||
evidence, safe Git/release operations and reuse in other projects.
|
||||
|
||||
```sh
|
||||
./devkit commands
|
||||
./devkit context --changed
|
||||
./devkit check --profile quick --changed --dry-run
|
||||
./devkit review campaign
|
||||
```
|
||||
|
||||
Create the whole-product development virtualenv in this meta repository:
|
||||
|
||||
```sh
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
#!/bin/sh
|
||||
# One stable entry point; no environment activation, installation or server startup.
|
||||
set -eu
|
||||
DEVKIT_ROOT=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
||||
if [ -n "${PYTHON:-}" ]; then
|
||||
DEVKIT_PYTHON=$(command -v "$PYTHON") || { echo "devkit: configured PYTHON is unavailable" >&2; exit 127; }
|
||||
elif [ -x "$DEVKIT_ROOT/.venv/bin/python" ]; then
|
||||
DEVKIT_PYTHON="$DEVKIT_ROOT/.venv/bin/python"
|
||||
else
|
||||
DEVKIT_PYTHON=$(command -v python3)
|
||||
fi
|
||||
exec "$DEVKIT_PYTHON" "$DEVKIT_ROOT/tools/devkit/devkit.py" "$@"
|
||||
@@ -64,6 +64,7 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
|
||||
| Evidence collection and promotion | [Target Maturity Evidence Runbook](operations/TARGET_MATURITY_EVIDENCE_RUNBOOK.md) |
|
||||
| Package publication and consumption | [Package Registry Releases](operations/PACKAGE_REGISTRY_RELEASES.md) |
|
||||
| Release-console operation | [Release Console](operations/RELEASE_CONSOLE.md) |
|
||||
| Repeated development, verification and evidence commands | [Developer Command Suite](operations/DEVKIT.md) |
|
||||
| Module compatibility and install behavior | [Module Contracts and Installs](operations/MODULE_CONTRACTS_AND_INSTALLS.md) |
|
||||
| Security-audit toolchain | [Security Audit](operations/SECURITY_AUDIT.md) |
|
||||
|
||||
@@ -75,6 +76,8 @@ in `govoplan-records/docs/EAKTE_ARCHITECTURE.md`.
|
||||
meta, module, deployment, and website content.
|
||||
- [Gitea Issues](project/GITEA_ISSUES.md) defines labels, templates, import, and
|
||||
state-update conventions.
|
||||
- [UI Review Program](project/UI_REVIEW_PROGRAM.md) defines the principle-led,
|
||||
per-module review process and links the canonical Gitea review inventory.
|
||||
|
||||
## Evidence And Archive
|
||||
|
||||
|
||||
Executable
+300
@@ -0,0 +1,300 @@
|
||||
# Developer command suite
|
||||
|
||||
`./devkit` is the maintained entry point for repeated development work. It uses
|
||||
ordinary local programs, not an AI service. People, CI and coding agents use the
|
||||
same commands and results. It composes existing GovOPlaN checks and release
|
||||
services rather than defining parallel product rules.
|
||||
|
||||
## Start here
|
||||
|
||||
From the Meta checkout:
|
||||
|
||||
```sh
|
||||
./devkit commands
|
||||
./devkit doctor --repo core
|
||||
./devkit context --changed
|
||||
./devkit check --profile quick --repo campaign --dry-run
|
||||
./devkit coverage --profile ui --repo campaign
|
||||
./devkit check --profile quick --repo campaign
|
||||
./devkit latest
|
||||
./devkit review campaign
|
||||
```
|
||||
|
||||
Use `--help` on each command. `--json` (or `--format json`) returns structured
|
||||
results; the default summary intentionally omits full logs and source contents.
|
||||
Global options work before or after the command. `--workspace-root` is the parent
|
||||
directory of the registered repositories, not the Core checkout. Unknown
|
||||
repository filters fail rather than silently widening/narrowing scope.
|
||||
|
||||
The launcher prefers Meta's `.venv/bin/python`, or the explicit `PYTHON`
|
||||
executable, with `python3` as the no-venv fallback. Checks resolve `NODE` and
|
||||
`NPM` from explicit environment settings or `PATH`; no username-specific Node
|
||||
installation path is required. Doctor reports repairs but never installs
|
||||
packages, changes configuration, kills an occupied port or starts a server.
|
||||
|
||||
## Command contracts
|
||||
|
||||
| Command | Purpose | Effects |
|
||||
| --- | --- | --- |
|
||||
| `context [--changed] [--repo NAME]` | Offline Git state, instruction/documentation paths and review links | Reads only; upstream counts are not freshly fetched |
|
||||
| `doctor [--repo NAME] [--profile PROFILE]` | Interpreter, dependency, browser and test-resource preflight | Reads only; repair suggestions are not executed |
|
||||
| `check --profile PROFILE` | Registered checks with explanations, bounded logs and a receipt | Runs trusted tests/builds; `--dry-run` only plans |
|
||||
| `coverage --profile PROFILE [--repo NAME]` | Declared suite dispositions and explicit coverage limits | Reads only; not execution evidence |
|
||||
| `runs [--limit 10] [--before RUN]`, `latest` | Discover recent runs without looking up state-directory paths | Reads only; malformed latest evidence is not replaced with an older pass |
|
||||
| `status RUN`, `summary RUN` | Stage counts and failure log locations | Reads only |
|
||||
| `logs RUN --stage STAGE --tail 40 [--final-only]` | Provisional live output or a hash-verified final stage log | Reads only; provisional output never proves a pass |
|
||||
| `resume RUN` | Replan the previous check selection and reuse eligible successful stages | Runs remaining checks, creating a new receipt |
|
||||
| `recover RUN [--apply]` | Recover an abandoned check-run record after acquiring its OS lock | No test replay; apply changes only the local record |
|
||||
| `review MODULE` | Source inventory, principle revision, issue links, check plan and walkthrough checklist | Does not perform or complete a module review |
|
||||
| `docs audit [--repo NAME] [--changed]` | Owning manifest/help/translation checks and visible-label gaps | Local checks only; never translates or edits content |
|
||||
| `issues note …` | Preview and explicitly append deduplicated evidence | Remote comment creation only with `--apply` |
|
||||
| `release …` | Existing durable release planning/execution/recovery | Explicit `--apply`, request IDs and existing step confirmations |
|
||||
| `git …` | Frozen, explicit-path maintenance commit and exact branch push | Preview first; mutations require `--apply`; no stage-all, force or tags |
|
||||
|
||||
Git maintenance is a separate, selected-path workflow; see
|
||||
[Git maintenance](DEVKIT_MAINTENANCE.md). Issue notes and module bundles are
|
||||
documented in [Evidence and review commands](DEVKIT_EVIDENCE.md). The complete
|
||||
release command syntax, limitations and examples are in
|
||||
[Headless release operations](DEVKIT_RELEASE.md).
|
||||
|
||||
## Check profiles and their limits
|
||||
|
||||
- `quick`: existing manifest/interface invariants, import/help guards and selected
|
||||
module source/structure tests. It does not compile component suites or run the
|
||||
full browser matrix.
|
||||
- `ui`: quick/source contracts plus the shared Core component batch, compiled
|
||||
once in an isolated directory. This is not a full visual/usability review.
|
||||
- `backend`: existing manifest/interface checks and selected repository backend
|
||||
test suites. Shared backend-test state is serialized.
|
||||
- `full`: the canonical `tools/checks/check-focused.sh`, including optional-module
|
||||
build permutations and integrated browser conformance. Repository filters do
|
||||
not reduce this required cross-module completion gate.
|
||||
|
||||
`--changed` includes uncommitted work, commits ahead of the locally cached
|
||||
upstream and repositories with commits but no configured upstream.
|
||||
Missing/unreadable repositories remain visible as errors. For the
|
||||
GovOPlaN profiles, Core or Meta changes conservatively select every registered
|
||||
consumer; other changes expand through declared interface consumers. This is
|
||||
not complete semantic dependency analysis. Use the full gate for cross-module
|
||||
completion as required by `AGENTS.md`.
|
||||
|
||||
The module catalog reuses declared direct-Node tests and established structural
|
||||
script names. Arbitrary shell chains are not guessed or rewritten. Unsupported
|
||||
declared scripts remain visible with a reason; only exact known Core component
|
||||
aliases are covered by the shared component runner. Invalid package metadata is
|
||||
an error, not an empty test inventory.
|
||||
|
||||
`coverage` inventories declared suites as `planned`, `covered_elsewhere`,
|
||||
`excluded` or `unsupported`, naming their covering stage where applicable. The
|
||||
same inventory accompanies check dry runs and saved receipts. **Full means the
|
||||
canonical focused gate, not every test in every package.** Its inventory follows
|
||||
explicit canonical commands conservatively; it does not infer arbitrary nested
|
||||
shell commands or npm hooks. Add new deterministic checks to the owning
|
||||
package/canonical gate and the catalog's tests as appropriate. Coverage planning
|
||||
does not execute the scripts it inventories or assert that they passed. See
|
||||
[Coverage and validation details](DEVKIT_COVERAGE.md) for examples and boundaries.
|
||||
|
||||
Independent checks can run with `--jobs 1..8` (default 2). Shared resources use
|
||||
OS locks for the current user, including across runs with different evidence
|
||||
directories. Browser/port locks are shared across that user's workspaces. Full runs
|
||||
reserve the shared WebUI/backend/browser resources. Standalone Core component
|
||||
commands use isolated temporary build directories; `npm run test:components --
|
||||
page-layout documentation-help` compiles once for that batch. Managed temporary
|
||||
output is cleaned; no user worktree is reset or restored.
|
||||
|
||||
## Evidence, interruption and safe reuse
|
||||
|
||||
A check announces its run ID and saves a `preparing` receipt before source and
|
||||
environment fingerprinting. Progress on stderr shows elapsed time, stage counts
|
||||
and active stages; it also distinguishes `checking` from final snapshot
|
||||
verification (`finalizing`). `--json` keeps stdout as one final JSON result and
|
||||
emits structured progress events on stderr; `--quiet` suppresses these events.
|
||||
Use `latest`, `status RUN` and `logs RUN --stage STAGE` from another terminal
|
||||
while a check runs. Counts are stages, not a guessed percentage of test effort.
|
||||
|
||||
Live snapshots are private, bounded, explicitly provisional and not accepted for
|
||||
reuse or issue evidence. A completed stage log can be hash-verified before the
|
||||
whole run finishes, but only the final run snapshot verifies the overall result.
|
||||
`--final-only` rejects unfinished logs. Run history supports cursor pagination
|
||||
(up to 100 rows per page), bounds directory scanning, and never deletes evidence.
|
||||
An explicitly selected `--project` also filters run discovery by project file.
|
||||
|
||||
Default records are private below
|
||||
`$XDG_STATE_HOME/govoplan/devkit/workspace-<identity>/runs/<run>/` (fallback
|
||||
`~/.local/state`). `--state-dir` selects another base; workspace scoping still
|
||||
applies. Status changes are atomically persisted with restrictive file modes;
|
||||
symlinked evidence paths are rejected. A record contains source/environment/plan
|
||||
fingerprints, stage reasons, status, exit code, timing and log identities.
|
||||
|
||||
Stages distinguish pending, running, passed, failed, timed out, blocked, skipped,
|
||||
stale and interrupted. A skipped or unexecuted stage is never a pass. A source or
|
||||
environment change during verification marks the overall result **stale**, even
|
||||
when individual subprocesses returned zero. Such a result is not passing evidence.
|
||||
|
||||
`resume RUN` replans the current selection and compares each phase independently.
|
||||
Matching **verified checkpoints** are reused; changed/new phases and phases
|
||||
without a checkpoint run again. A checkpoint binds the exact command and working
|
||||
directory, declared repository inputs, transitive data-dependency identities,
|
||||
devkit implementation/schema, tool/dependency metadata and environment. Logs must
|
||||
retain their recorded hashes; a missing or changed cached log fails closed.
|
||||
The summary reports how many checkpoints were reused; JSON records `input_scope`,
|
||||
`cache_key`, `checkpoint_verified`, `reused_from` and the reuse/rerun reason.
|
||||
|
||||
Each checkpoint is saved while its resource locks remain held, only after exit
|
||||
zero, a verified final log, and matching before/after inputs and environment.
|
||||
A phase that changes its own inputs is stale, even if another phase later restores
|
||||
those bytes. Final verification rechecks all passed/reused phases against current
|
||||
inputs before the new aggregate can pass. An interrupted, failed or stale run may
|
||||
donate an independently verified matching checkpoint, but never becomes passing
|
||||
evidence itself. Old receipts remain readable: legacy monolithic logs and receipts
|
||||
without these checkpoints cannot be retroactively split or reused as verified phases.
|
||||
|
||||
After a hard crash, `recover RUN` shows the abandoned owner and guidance. Inspect the
|
||||
recorded commands and manually stop any surviving test/build processes before
|
||||
using `recover RUN --apply --confirm-processes-stopped`. The command verifies the
|
||||
run lock is free before marking the record interrupted. A free parent lock alone
|
||||
does **not** prove children stopped after a hard kill. Recovery does not replay
|
||||
processes or infer that a partially completed operation succeeded.
|
||||
|
||||
### Canonical phases and narrower inputs
|
||||
|
||||
The full gate has seven explicit phases: `preflight`, `tooling`, `backend`,
|
||||
`core-ui`, `module-builds`, `browser`, and `module-ui`. Devkit stage IDs are
|
||||
`focused.<phase>`. The authoritative Bash bodies remain in
|
||||
`tools/checks/check-focused.sh`; the bounded metadata in
|
||||
`tools/checks/focused-phases.json` names their order and resources. The direct
|
||||
shell command still runs every phase in the original fail-fast order. For inspection:
|
||||
|
||||
```sh
|
||||
tools/checks/check-focused.sh --list-phases
|
||||
./devkit check --profile full --jobs 1
|
||||
./devkit resume RUN_ID --jobs 1
|
||||
```
|
||||
|
||||
For example, a browser failure no longer forces successful backend/build phases
|
||||
to run again if their inputs still match. Standalone `--phase browser` is available
|
||||
for diagnostics, but is not a substitute for the complete gate or its receipt.
|
||||
|
||||
Scopes are whole repositories, not inferred file globs. A trusted check may declare
|
||||
`"inputs": {"repos": ["canonical-repository-name"]}`; no declaration means the whole
|
||||
registered workspace. Dry runs show that boundary. Native source-discovery checks
|
||||
retain broad scopes where cross-module dependencies cannot safely be narrowed.
|
||||
Changes outside an explicit scope do not invalidate it, but tool/environment
|
||||
changes remain conservatively global. Scope declarations are a correctness
|
||||
contract: include every source repository that the check reads, not only its cwd.
|
||||
Native discovery ownership/presence is rechecked at phase boundaries, so a newly
|
||||
appearing module or WebUI directory invalidates the in-flight environment identity.
|
||||
Unregistered native sibling sources disable reuse with an explicit coverage note;
|
||||
register them before relying on cached results. Ordinary generated files inside
|
||||
existing source directories do not change this directory-shape identity.
|
||||
|
||||
Every snapshot rereads repository presence, HEAD, index/flags and tracked plus
|
||||
non-ignored untracked membership, including assume-unchanged files. File hashes
|
||||
use a bounded **in-process-only** memo with file identity/ctime/mtime/size checks
|
||||
before and after an open file descriptor; no persistent mtime cache is trusted.
|
||||
In-repository regular-file symlink targets are included; escaping or directory
|
||||
symlinks fail closed. Ignored files, undeclared external data and service state
|
||||
are not source evidence and need explicit verification.
|
||||
|
||||
Environment probes stream bounded regular-file contents and reject concurrent
|
||||
replacement, growth and symlink retargeting. Stable venv executable and dependency
|
||||
directory symlinks remain supported without changing the executable path; absent
|
||||
optional metadata remains optional. Environment identities are not persistently
|
||||
cached and remain deliberately global.
|
||||
|
||||
These checkpoints cache **verification results, not output artifacts**. The full
|
||||
gate's phases were audited so later phases do not require an earlier phase's
|
||||
retained build output. A custom setup/build that produces files consumed later
|
||||
must declare `"reuse": "never"`; its consumers must use `deps`. Until output
|
||||
manifests/restoration exist, do not assume a cached successful build recreates
|
||||
deleted ignored artifacts.
|
||||
|
||||
Check output is drained without unbounded memory growth. Stored output retains
|
||||
the beginning and actual final tail within an 8 MiB bound per stage, with a small
|
||||
truncation marker and an omitted-byte count; output truncation does not change
|
||||
the subprocess exit result. `logs` exposes at most 200 lines and 16 KiB. Live
|
||||
snapshots retain at most 64 KiB after redaction. Incomplete live lines and cut
|
||||
retention-boundary lines are withheld to avoid exposing fragments of secrets;
|
||||
very long single-line output may therefore be absent from live views.
|
||||
Known environment credentials and common authorization patterns are
|
||||
redacted as display hygiene. **Never pass credentials as check arguments or print
|
||||
them from tests.** Redaction is not a general secret-classification guarantee.
|
||||
JSON command output is a redacted presentation; the receipt file at the reported
|
||||
path is the canonical local record. Cancellation and deadlines terminate the
|
||||
owned process group; a parent leaving running descendants does not pass.
|
||||
During source/environment fingerprinting, cancellation is observed between
|
||||
probes; the current bounded probe may finish first. Cancellation during final
|
||||
verification is recorded as interrupted, never as a passing run.
|
||||
Deliberately detached processes are outside that group: this is not a sandbox.
|
||||
|
||||
These local hash-bound records detect accidental changes; they are not signed
|
||||
attestations, a security certification, proof of complete test coverage, or
|
||||
permission to publish. External service state and undeclared dependencies still
|
||||
need explicit verification. No automated result closes a Gitea issue or marks a
|
||||
module reviewed. Detailed logs remain local unless deliberately shared. Old run
|
||||
directories are not automatically deleted; review retention before removing any
|
||||
evidence referenced by an issue.
|
||||
|
||||
## Reuse in another project
|
||||
|
||||
The runner, repository context, doctor and evidence primitives are usable with an
|
||||
explicit local JSON project manifest. The example at
|
||||
`tools/devkit/examples/project.json` registers ordinary Python tests and a Git
|
||||
whitespace check; the format is described by
|
||||
`tools/devkit/project.schema.json`.
|
||||
|
||||
```sh
|
||||
/path/to/govoplan/devkit --workspace-root /path/to/project \
|
||||
--project /path/to/project/devkit-project.json check --profile quick --dry-run
|
||||
```
|
||||
|
||||
Repository paths and check working directories must remain inside the selected
|
||||
workspace. Check arguments are arrays, not shell-evaluated strings. The supported
|
||||
tool placeholders are `{python}`, `{node}`, `{npm}` and `{workspace}`. Check
|
||||
dependencies form an acyclic graph; resources serialize incompatible tasks.
|
||||
`deps` declares actual data dependencies: consumers rerun whenever a dependency
|
||||
runs again. `after` declares only fail-fast execution ordering: a failed predecessor
|
||||
skips the follower, but a successfully rerun independent predecessor does not
|
||||
invalidate the follower's inputs. Both edge types are validated together for
|
||||
cycles, missing IDs and overlap. `reuse` is `verified` by default or `never` for
|
||||
setup/output-producing checks. Repository `inputs` names are canonical, unique,
|
||||
nonempty and validated even for unselected checks.
|
||||
The complete custom manifest is validated, including unselected checks/profiles:
|
||||
unknown fields, misspellings, invalid bounds/types and unresolved dependencies
|
||||
fail early. This prevents silently ignored resource or timeout declarations.
|
||||
|
||||
Portable `doctor` always requires its Python runtime, plus tools needed by the
|
||||
selected checks and their dependencies and tools explicitly configured in the
|
||||
manifest. Without `--profile`, all declared profiles are considered. Unused
|
||||
Node/npm are marked `not_required` and do not block Python-only projects. Declare
|
||||
indirect tool dependencies explicitly: arbitrary script contents are not
|
||||
analyzed. Native GovOPlaN retains its Python/Node/npm requirements.
|
||||
Only use project manifests and scripts you trust: running a registered test is
|
||||
ordinary code execution, not a sandbox. Platform-specific release and Docs
|
||||
commands reject generic project manifests; put another project's checks in its
|
||||
own profiles rather than pretending its release policy is GovOPlaN's.
|
||||
|
||||
The initial runtime targets POSIX environments with Python 3.11+ and OS advisory
|
||||
locks; it does not claim Windows support. No globally installed service or Codex
|
||||
plugin is needed. Keep this implementation versioned and reuse it; avoid copying
|
||||
diverging helper implementations into each project.
|
||||
|
||||
Devkit is not a complete build system: it does not infer semantic dependencies,
|
||||
restore output artifacts, sandbox checks, or share signed remote caches. Repository
|
||||
scopes intentionally stop short of file/glob narrowing; environment identities
|
||||
remain global. These are explicit boundaries, not claims of exhaustive coverage.
|
||||
|
||||
## Development and conformance
|
||||
|
||||
```sh
|
||||
./.venv/bin/python -m pytest -q tests/test_devkit_*.py
|
||||
./.venv/bin/python -m pytest -q tests/test_focused_phases.py
|
||||
node --test tests/test-devkit-display-labels.mjs
|
||||
./.venv/bin/python -m unittest tests.test_documentation_structure
|
||||
```
|
||||
|
||||
The focused gate includes the Python command-suite regression tests. Fixture
|
||||
tests use temporary repositories/processes and mocked issue/release transports;
|
||||
they do not commit user work, send mail, publish artifacts or change live issues.
|
||||
Documentation is part of each command change: update `--help`, this guide and the
|
||||
corresponding targeted contract tests together.
|
||||
Executable
+72
@@ -0,0 +1,72 @@
|
||||
# Check configuration and suite coverage
|
||||
|
||||
`devkit coverage` explains what a selected plan intends to run. It does not run
|
||||
tests, start servers, or report successful verification:
|
||||
|
||||
```sh
|
||||
./devkit coverage --profile quick --repo campaign
|
||||
./devkit coverage --profile ui --repo portal --json
|
||||
./devkit coverage --profile full --json
|
||||
./devkit --project /path/to/project.json coverage --profile quick
|
||||
```
|
||||
|
||||
The inventory accounts for declared `test` and `test:*` scripts in each
|
||||
registered repository's root and `webui/package.json`, discovered UI structural
|
||||
checks, and explicitly configured project checks. It is a suite inventory, not
|
||||
an enumeration of every test function or recursive dependency. Unselected suites
|
||||
remain visible; every row has a disposition and reason:
|
||||
|
||||
| Disposition | Meaning |
|
||||
| --- | --- |
|
||||
| `planned` | A selected stage directly invokes the suite. |
|
||||
| `covered_elsewhere` | An exact equivalent invocation or an explicitly selected shared batch owns it; the covering stage is named. |
|
||||
| `excluded` | The suite is outside the actual selected plan. |
|
||||
| `unsupported` | Scoped discovery cannot safely interpret or locate the command; no command is guessed or executed. |
|
||||
|
||||
`full` means the existing canonical `check-focused.sh` gate. It does **not**
|
||||
mean every package script or every component suite. Coverage reads the current
|
||||
script's explicit npm/Node commands inside its seven registered marked phase
|
||||
bodies, without executing shell code or inferring arbitrary functions, nested
|
||||
scripts, npm hooks, branches or here-document contents. Each covered suite names
|
||||
the `focused.<phase>` that owns it. The
|
||||
current gate names four of the sixteen Core component suites: layout primitives,
|
||||
page layout, DataGrid actions/sizing, and Mail components. The other twelve stay
|
||||
explicitly excluded from that gate's component batch. `ui` runs the shared
|
||||
sixteen-suite batch once; `quick` does not compile component tests.
|
||||
|
||||
Only Core's exact known component aliases receive that shared-batch treatment.
|
||||
A different repository using the same filename, an unknown alias, or a compound
|
||||
command containing the runner is not silently credited or launched. Ordinary
|
||||
scoped source discovery accepts bounded direct `node`/`node --test` commands
|
||||
targeting regular package-owned `.mjs` files in `scripts` or `tests`; shell
|
||||
chains, extra flags, missing targets and symlinked targets are not rewritten.
|
||||
An explicit invocation already present in the canonical full gate remains part
|
||||
of that gate, even when the narrower discovery profile does not support it.
|
||||
|
||||
Coverage rows retain a command hash and, for supported commands, redacted argv.
|
||||
Unsupported shell bodies are not copied into the inventory. This is display
|
||||
hygiene, not permission to put secrets into project commands. Coverage is
|
||||
attached to check plans/receipts; it never turns an excluded suite into a pass.
|
||||
|
||||
## Portable-project validation
|
||||
|
||||
The runtime validates custom project metadata against the published
|
||||
[`project.schema.json`](../../tools/devkit/project.schema.json) using a small
|
||||
dependency-free validator. All checks and profiles are validated before
|
||||
selection, including checks the chosen profile does not execute. Unknown nested
|
||||
fields such as `resource` or `timout_seconds`, malformed types, oversized values,
|
||||
duplicate references, missing dependencies, unknown profiles and dependency
|
||||
cycles fail with controlled errors. Repository aliases and resolved paths must
|
||||
be unambiguous and confined to the workspace. Check `cwd` defaults to `.` when
|
||||
omitted; configured paths cannot escape the workspace.
|
||||
|
||||
Package JSON is bounded to 1 MiB and 512 scripts; duplicate JSON keys, excessive
|
||||
nesting and malformed script objects fail before planning. Command strings are
|
||||
bounded to 8,192 characters. This validates configuration, not the safety of its
|
||||
code: run only project manifests and test scripts you trust.
|
||||
|
||||
Checks may declare repository-scoped `inputs`, true data `deps`, order-only
|
||||
`after`, and `reuse: "never"` for setup/output-producing work. Unknown repositories,
|
||||
empty/duplicate scopes, overlapping edge types and cycles fail before execution.
|
||||
Missing inputs deliberately fall back to all registered repositories. See
|
||||
[checkpoint and input contracts](DEVKIT.md#canonical-phases-and-narrower-inputs).
|
||||
Executable
+171
@@ -0,0 +1,171 @@
|
||||
# Devkit review bundles and issue evidence
|
||||
|
||||
Gitea remains the canonical backlog and review state log. These commands assemble
|
||||
guidance and append evidence; they do not create a second progress tracker, close
|
||||
issues, rewrite issue bodies, tick checkboxes, or certify a module review.
|
||||
|
||||
## Prepare a module review
|
||||
|
||||
```sh
|
||||
./devkit review campaign
|
||||
./devkit review campaigns --json --output /tmp/campaign-review.json
|
||||
./devkit review campaign --evidence RUN_ID
|
||||
```
|
||||
|
||||
Repository names, repository aliases and scope IDs from the
|
||||
[issue discovery inventory](../project/ui-review-issue-inventory.json) are accepted.
|
||||
`review bundle campaign` is an equivalent spelling. A bundle contains the current
|
||||
source inventory and fingerprint, canonical module/central issue links, the
|
||||
Core principle revision and content hash, applicable **planned** check stages,
|
||||
and a manual checklist. No checks or application servers are started. The
|
||||
optional output file is a local snapshot, not an authoritative state record.
|
||||
|
||||
The filename inventory is a starting point, not proof that all runtime surfaces
|
||||
were found. It does not import application manifests or optional modules. Review
|
||||
routes, panes, dialogs, settings, widgets, public forms, contributed/headless
|
||||
interfaces, missing-module behavior, permissions, English/German and narrow/wide
|
||||
layouts manually. Missing source, no UI files or no planned checks is never an
|
||||
automatic pass or N/A. Record exceptions and remaining work in the module issue.
|
||||
|
||||
Apply [UI-01 and UI-02, and the other Core principles](../../../govoplan-core/docs/UI_DESIGN_PRINCIPLES.md):
|
||||
books belong beside meaningful visible text; normal screens display compact
|
||||
data, with intentional scoped dialog editing. A bulk editing mode needs a
|
||||
documented reason. Record new principles and their back-propagation to previously
|
||||
reviewed modules through the [central review program](../project/UI_REVIEW_PROGRAM.md).
|
||||
|
||||
An explicit portable `--project project.json` does not inherit GovOPlaN issue
|
||||
links or principles. Its optional configuration is:
|
||||
|
||||
```json
|
||||
{
|
||||
"review": {
|
||||
"issue_inventory": "meta/docs/review-issues.json",
|
||||
"principles": "core/docs/UI_DESIGN_PRINCIPLES.md"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Paths are relative to `--workspace-root` and cannot escape it or traverse
|
||||
symlinks. The inventory uses schema version 1, `issues[]` with `scope_id`,
|
||||
`repository`, `number`, `url`, and optional `name`/`kind`; an optional `epic`
|
||||
contains `repository`, `number`, `url`. Snapshot issue-state fields are not
|
||||
reported as current state. Custom checks come from the project's selected
|
||||
profile (`--profile ui` by default).
|
||||
|
||||
## Preview and append evidence
|
||||
|
||||
```sh
|
||||
./devkit issues note --root govoplan-campaign --issue 103 \
|
||||
--evidence RUN_ID --summary 'Targeted checks passed.' \
|
||||
--next 'Manual module review and remaining fixes are still required.' --json
|
||||
|
||||
./devkit issues note --root govoplan-campaign --issue 103 \
|
||||
--evidence RUN_ID --summary 'Targeted checks passed.' \
|
||||
--next 'Manual module review and remaining fixes are still required.' \
|
||||
--env-file /private/gitea.env --apply
|
||||
```
|
||||
|
||||
`--root` is resolved relative to the selected **workspace root**, not the shell's
|
||||
current directory. In the normal siblings workspace, use `--root govoplan-campaign`
|
||||
or an absolute checkout path; do not use a path escaping the workspace.
|
||||
|
||||
The default is a completely offline dry run; it does not load credentials or
|
||||
contact Gitea. `--apply` explicitly authorizes comment creation. Tokens are read
|
||||
from `GITEA_TOKEN` or the explicit `--env-file` (environment wins); there is no
|
||||
implicit `.env` search. The credential file is parsed, never sourced, and is not
|
||||
printed or uploaded. Git's selected remote (default `origin`) determines the
|
||||
exact target: ambient `GITEA_OWNER`, `GITEA_REPO` and `GITEA_URL` cannot retarget
|
||||
the request. Credential-bearing HTTP Git remote URLs are rejected.
|
||||
|
||||
Use repeatable `--summary`/`--next`, `--body-file` for Markdown, or `--note-file`
|
||||
with structured JSON `{ "summary": ["..."], "next": ["..."], "body": "..." }`.
|
||||
Do not put secrets in notes, paths, check arguments or logs. Display redaction is
|
||||
best-effort protection, not permission to include confidential data.
|
||||
|
||||
For a bounded multi-issue operation, replace `--root`/`--issue` with
|
||||
`--target-plan targets.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"targets": [
|
||||
{
|
||||
"root": "govoplan-campaign",
|
||||
"issue": 103,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Targets must be unique, inside the workspace and bound exactly to their Git
|
||||
remote/issue URLs, all on one exact Gitea base URL. Optional `issue_id` binds
|
||||
Gitea's immutable numeric ID as well. Every target, complete comment history,
|
||||
marker collision and local journal binding is checked before the first POST.
|
||||
Comments are then posted serially and read back against the exact issue. A
|
||||
closed issue stays closed; existing bodies, comments and checklists are untouched.
|
||||
|
||||
Each note has a stable hidden marker binding its issue, `--key` purpose
|
||||
(`verification` by default), and evidence run ID. Identical already-published
|
||||
notes are read back instead of reposted; changed content with the same marker
|
||||
is a collision, not an update. Use a distinct intentional `--key` for a separate
|
||||
follow-up. Without a receipt, the note's structured content determines identity.
|
||||
|
||||
A lost POST response is **not** retried automatically. The publisher scans all
|
||||
comment pages and reads back any matching comment. If it cannot confirm the
|
||||
result, it records `uncertain`, stops later posts, and exits nonzero. The next
|
||||
invocation reconciles again without replaying the POST. Only after inspecting
|
||||
the issue and local result, use `--apply --retry-uncertain` to permit one new
|
||||
attempt when the marker is still absent. Local journals and OS-released locks
|
||||
live in the workspace-scoped devkit state directory; do not delete them to bypass
|
||||
uncertainty. These are retry receipts, not issue state. Independent machines or
|
||||
different state directories cannot share a lock: coordinate publishers, because
|
||||
Gitea comment creation does not provide an atomic idempotency key.
|
||||
|
||||
## What a receipt does and does not prove
|
||||
|
||||
`--evidence RUN_ID` reads the workspace-scoped runner receipt and checks local
|
||||
integrity. `--evidence /path/to/receipt.json` accepts bounded validated external
|
||||
receipt metadata, explicitly labeled `external-unverified`. A self-consistent
|
||||
digest is not a signature or independent verification. Foreign-workspace,
|
||||
malformed and inconsistent successful receipts are rejected. Receipt-provided
|
||||
commands are never run; log files are not opened or uploaded, only referenced.
|
||||
Successful receipts must explicitly record a verified source snapshot; an
|
||||
external file's flag is still a reported claim, not an independent attestation.
|
||||
An aggregate containing a skipped stage cannot claim success. New versioned
|
||||
receipts additionally require independent verified checkpoints for passed phases.
|
||||
Failed/interrupted/stale aggregates remain labeled as such even when they contain
|
||||
reusable successful phases; reuse creates a new run and preserves the original.
|
||||
Stage coverage limitations are validated and retained in the note, including
|
||||
checks omitted by scoped profiles. Compact review output shows the first eight
|
||||
limitations and points to the complete list in its JSON bundle. A passing stage
|
||||
never implies that omitted compiler, browser, permutation or manual checks ran.
|
||||
|
||||
Documentation audit limitations are saved with their stage receipts, so later
|
||||
review bundles and issue notes retain the same scope as the original command.
|
||||
Managed inventory collectors use the explicit selected workspace throughout;
|
||||
an incomplete checkout never borrows sources or missing module imports from a
|
||||
fuller default workspace. Missing required Core/parser dependencies fail the
|
||||
audit. Legacy direct inventory commands without `--workspace-root` retain their
|
||||
discovery behavior; use the explicit option for reproducible evidence.
|
||||
|
||||
The note distinguishes `matches-current`, `historical-source-differs`,
|
||||
`different-project-not-compared` and unavailable comparisons. Matching source
|
||||
is a comparison at bundle/note preparation time, not a live test run or proof
|
||||
that a person completed review. If the source or environment has changed, run
|
||||
the appropriate checks again and keep historical evidence labeled honestly.
|
||||
|
||||
Versioned repository-input receipts compare the recorded repository scope, not a
|
||||
legacy whole-workspace hash. Partial scopes remain explicitly labeled in evidence
|
||||
notes. This comparison reads source only; receipt commands are never executed.
|
||||
Legacy receipts retain their original comparison format. A matching source scope
|
||||
does not prove that ignored build outputs, the environment or live services match.
|
||||
|
||||
Fixtures cover dry-run isolation, exact target bindings, capped/repeated
|
||||
pagination, collisions, append-only idempotency, uncertain POST reconciliation,
|
||||
secret-safe errors, receipt validation, portable projects and optional-module
|
||||
review boundaries:
|
||||
|
||||
```sh
|
||||
python3 -m pytest -q tests/test_devkit_issues.py tests/test_devkit_review.py
|
||||
```
|
||||
Executable
+97
@@ -0,0 +1,97 @@
|
||||
# Selected-path Git maintenance
|
||||
|
||||
The optional `git` namespace of `tools/devkit/devkit.py` is a deliberately
|
||||
narrow maintenance workflow. It is not a replacement for normal Git, the
|
||||
release executor, required human review, or Gitea's canonical issue state.
|
||||
|
||||
From the Meta repository, preview and then save a plan for explicit files:
|
||||
|
||||
```sh
|
||||
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example"
|
||||
python3 tools/devkit/devkit.py git plan --repo core --path webui/src/example.ts --message "Fix the reviewed example" --apply
|
||||
```
|
||||
|
||||
Use the saved `git-…` plan ID for the remaining commands. Each mutation needs
|
||||
its own `--apply`; an unqualified command only previews the operation:
|
||||
|
||||
```sh
|
||||
python3 tools/devkit/devkit.py git commit git-PLAN_ID
|
||||
python3 tools/devkit/devkit.py git commit git-PLAN_ID --apply
|
||||
python3 tools/devkit/devkit.py git push git-PLAN_ID
|
||||
python3 tools/devkit/devkit.py git push git-PLAN_ID --apply
|
||||
python3 tools/devkit/devkit.py git status git-PLAN_ID
|
||||
```
|
||||
|
||||
The immutable local plan binds the registered repository, HEAD, branch, origin
|
||||
fetch/push URL hashes, Git configuration, complete index identity, explicit
|
||||
selected file paths, working-tree hashes, expected Git blobs, and commit message.
|
||||
Raw remote URLs and file contents are not stored. A changed input blocks the
|
||||
operation; inspect it and create another plan instead of weakening the check.
|
||||
Receipts are private local records with integrity checks, not signed approval
|
||||
or a security attestation.
|
||||
|
||||
Commit captures each selected file once, verifies the captured bytes and Git
|
||||
blob against the plan, and builds an isolated index/tree from the frozen HEAD
|
||||
plus only those blobs. Git's normal identity rules create the commit object;
|
||||
an atomic branch compare-and-swap publishes it only if the planned parent is
|
||||
still current. Active hooks/signing remain refused rather than bypassed.
|
||||
An editor racing with this operation cannot substitute newer working-tree
|
||||
bytes: those edits remain uncommitted, and the helper never overwrites files.
|
||||
|
||||
The real index is protected by Git's standard index lock. Only selected entries
|
||||
are updated from a prepared private copy; unrelated staging and index flags are
|
||||
preserved. If the index changes independently, its new data is not overwritten
|
||||
and the receipt requires reconciliation. The resulting parent, message,
|
||||
changed paths, blobs/modes and unrelated index entries are verified.
|
||||
Git refs and index files are separately atomic, not one filesystem transaction.
|
||||
Independent writers that ignore the standard index lock can still change refs
|
||||
during publication. The helper rechecks the exact recorded candidate before
|
||||
index recovery and before recording success; it refuses an uncertain result
|
||||
instead of adopting a newer HEAD or claiming that competing work was its own.
|
||||
Directories, noncanonical paths, implicit globs, staging-all, amend,
|
||||
force-push and extra-tag publication are not supported. A selected file with
|
||||
different staged and working-tree changes is refused; decide explicitly which
|
||||
version to commit using normal Git first.
|
||||
|
||||
Push requires the recorded commit to remain current, an unchanged origin and
|
||||
one standard-transport push destination. It requests a normal, non-forced push
|
||||
of that commit to the recorded branch and checks the remote branch afterward.
|
||||
The push preview does not contact a remote. A normal push may run the remote's
|
||||
usual CI or server-side hooks; this is an external effect of the separately
|
||||
authorized push, not of planning or committing.
|
||||
|
||||
Local active hooks (including index-change hooks), filter/encoding attributes,
|
||||
signing, external fsmonitor/SSH configuration, Git environment overrides,
|
||||
in-progress merge/rebase/cherry-pick operations, submodules, symlinks and
|
||||
assume-unchanged/skip-worktree entries, split indexes and replacement/graft
|
||||
history are deliberately
|
||||
unsupported. The helper
|
||||
refuses these cases instead of disabling hooks, signatures or filters. An
|
||||
unused globally installed filter definition alone does not block maintenance;
|
||||
active attributes are checked across tracked files as well as selected files.
|
||||
Custom remote receive-pack/upload-pack/helper commands, recursive submodule
|
||||
pushes and partial-clone lazy fetch are also refused. Of inherited `GIT_*`
|
||||
variables, only `GIT_OPTIONAL_LOCKS`, `GIT_TERMINAL_PROMPT` and `GIT_PAGER` are
|
||||
allowed (every command explicitly passes `--no-pager`, so the pager is inert);
|
||||
namespace, identity, alternate-index/object-directory and unknown overrides
|
||||
are not silently removed. Git subprocesses have bounded input/output and
|
||||
deadlines; cancellation terminates their owned process group, including
|
||||
transport helpers. Network failures can still leave a remote effect uncertain,
|
||||
which is why receipts require explicit reconciliation rather than blind retry.
|
||||
|
||||
An interruption or uncertain failure is recorded before any retry. Use
|
||||
`git reconcile git-PLAN_ID` to preview, then add `--apply` to verify an
|
||||
already-existing result. Reconciliation never makes a commit or push. It may
|
||||
finish the selected real-index update after an interrupted commit publication,
|
||||
but only when the original index fingerprint still matches; independent staged
|
||||
work is preserved and requires normal Git resolution. For an
|
||||
uncertain push it may read the frozen remote, but only with `--apply`. If no
|
||||
planned commit is current, the helper reports that state and requires a fresh
|
||||
plan after inspection. A forcibly killed process may leave its private scratch
|
||||
index or an owned Git index lock; the helper never guesses that an existing
|
||||
lock is safe to delete. Blob/commit preparation can leave unreferenced Git
|
||||
objects for ordinary Git garbage collection. It never automatically resets,
|
||||
restores, deletes, or rolls back user work.
|
||||
|
||||
The maintenance tests use disposable local repositories and local bare remotes.
|
||||
They never commit or push the user's workspace repositories.
|
||||
Executable
+121
@@ -0,0 +1,121 @@
|
||||
# Headless durable releases
|
||||
|
||||
The devkit `release` commands use the **same release application, request
|
||||
validation, run store and bounded executors** as the
|
||||
[release console](RELEASE_CONSOLE.md). They invoke its ASGI application inside
|
||||
the current process, with an ephemeral internal token. No listening socket,
|
||||
background server, browser session or externally supplied console URL is used.
|
||||
The development environment's FastAPI/HTTPX dependencies load only when a
|
||||
release command runs; ordinary devkit help does not import them.
|
||||
|
||||
This is a GovOPlaN-specific provider. Repository and origin authority still comes
|
||||
from the release service's registered catalog. `--project` is explicitly rejected
|
||||
for release commands; a portable project configuration must not silently override
|
||||
that authority. A different workspace path does not authorize arbitrary release
|
||||
repositories, remotes, commands, signing policies or source bindings.
|
||||
|
||||
The default compact output includes the overall status, selected repository
|
||||
states and versions, source-preflight readiness, bounded gate findings and the
|
||||
recommended next action. Run inspection also shows step-state counts and the
|
||||
first outstanding steps. These are projections of the service response, not new
|
||||
readiness checks. Use `--json` for the complete existing plan or receipt; compact
|
||||
output deliberately omits executor commands, arguments and source bindings.
|
||||
|
||||
## Commands and boundaries
|
||||
|
||||
| Command | Purpose | Effect boundary |
|
||||
| --- | --- | --- |
|
||||
| `release status` | Current release dashboard | Offline by default; explicit remote/catalog flags enable their checks |
|
||||
| `release plan` | Selective repository/version plan | Inspection only; does not create a run |
|
||||
| `release list` | Bounded run history, with `--limit` and `--cursor` | Existing workspace-scoped store |
|
||||
| `release show RUN_ID` | Exact verified run, steps, receipts and required confirmations | No release execution |
|
||||
| `release create` | Freeze a selected repository/version plan | Preview by default; `--apply` persists a run |
|
||||
| `release preview RUN_ID STEP_ID` | Frozen step and its current constraints | No step claim; catalog publication additionally uses the existing validated-candidate preview |
|
||||
| `release execute RUN_ID STEP_ID` | One available durable step | Preview by default; `--apply`, explicit request ID and the step's exact confirmation are required |
|
||||
| `release resume RUN_ID` | Mark persisted running attempts interrupted | Requires `--apply`; never assumes an effect succeeded |
|
||||
| `release retry RUN_ID STEP_ID` | Prepare an eligible failed/read-only-interrupted step | Requires `--apply`; does not execute it |
|
||||
| `release reconcile RUN_ID STEP_ID` | Record a proven uncertain-write outcome | Requires `--apply`, `--confirm RECONCILE` and an explicit outcome |
|
||||
|
||||
Create and transition commands require caller-chosen `--request-id` values.
|
||||
Retain the same ID for an uncertain replay of the **same** command; changing
|
||||
inputs under an existing ID fails closed. No command automatically retries a
|
||||
mutation. These commands perform one explicit transition, not an implicit
|
||||
"release everything" loop.
|
||||
|
||||
Read commands do not commit, tag, push, publish or apply database migrations.
|
||||
The existing private run store can initialize its lock directory or upgrade a
|
||||
legacy record while inspecting it; this does not advance release steps.
|
||||
`--include-migrations` requests the existing audits, never migration application.
|
||||
`--online`, `--remote-tags` and `--public-catalog` are explicit network-check
|
||||
choices. A generic step preview describes frozen intent; it is not a claim that
|
||||
live preflight, remote identity or release acceptance has passed.
|
||||
|
||||
## Example: review, freeze and execute one step
|
||||
|
||||
Run the devkit through the workspace `./devkit` entrypoint. These examples put
|
||||
global options before `release`:
|
||||
|
||||
```sh
|
||||
./devkit --workspace-root /path/to/workspace --format json release plan \
|
||||
--repo-version govoplan-files=0.1.9
|
||||
|
||||
./devkit --workspace-root /path/to/workspace release create \
|
||||
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001
|
||||
|
||||
./devkit --workspace-root /path/to/workspace release create \
|
||||
--repo-version govoplan-files=0.1.9 --request-id files-release-create-0001 --apply
|
||||
|
||||
./devkit --workspace-root /path/to/workspace release show RUN_ID
|
||||
./devkit --workspace-root /path/to/workspace release preview RUN_ID STEP_ID
|
||||
./devkit --workspace-root /path/to/workspace release execute RUN_ID STEP_ID \
|
||||
--request-id files-release-step-0001 --confirm REQUIRED_CONFIRMATION --apply
|
||||
```
|
||||
|
||||
`RUN_ID`, `STEP_ID` and `REQUIRED_CONFIRMATION` stand for the actual values
|
||||
returned by the frozen run. For read-only preflight/alignment/install-verification
|
||||
steps, omit `--confirm` when the service reports an empty confirmation. Metadata,
|
||||
commit, release-lock, source-tag, source-push, candidate-generation and catalog
|
||||
publication steps keep the existing `UPDATE`, `COMMIT`, `LOCK`, `TAG`, `PUBLISH`,
|
||||
`GENERATE` and `PUSH` confirmations respectively. Prerequisite ordering is enforced
|
||||
by the same store; a later step cannot be forced through this adapter.
|
||||
|
||||
Version selection supports repeated `--repo-version REPO=VERSION`, or repeated
|
||||
`--repo REPO` with `--target-version VERSION`. Every created run needs explicit
|
||||
versions for all selected repositories. Conflicting version assignments are
|
||||
rejected. Planning is selective; repositories are never silently selected merely
|
||||
because their worktrees are dirty.
|
||||
|
||||
## Candidate publication and recovery
|
||||
|
||||
Use the frozen run's `catalog:selective-generator` and
|
||||
`catalog:validate-sign-publish` steps, not the disabled legacy mutation endpoints.
|
||||
Generation accepts repeated `--signing-key KEY_ID=PRIVATE_KEY_FILE`; these are
|
||||
operator-owned key-file references, never inline key material. Signing arguments
|
||||
are not emitted in devkit JSON or persisted by the adapter. Publication consumes
|
||||
the candidate receipt created by that run, not an arbitrary candidate directory.
|
||||
The existing source/runtime trust, immutable tag/remote identity, private
|
||||
candidate checks and exact commit-delta checks remain authoritative.
|
||||
|
||||
After an interrupted write, inspect the run and the actual local/remote effect.
|
||||
Use `resume --apply` if an attempt was left running. Then use `reconcile` with
|
||||
`effect_absent`, `effect_succeeded` or `unresolved`, `--confirm RECONCILE`, a new
|
||||
reconciliation request ID and `--apply`. Successful reconciliation still performs
|
||||
the service's independent receipt checks. An unresolved effect must not be
|
||||
retried or papered over by generating a new request ID. A failed/read-only
|
||||
attempt may use `retry --apply` and a new explicit execute attempt only when
|
||||
the existing lifecycle makes that safe.
|
||||
|
||||
## State and limitations
|
||||
|
||||
Every successful response exposes `state_location` and `candidate_location`.
|
||||
Without `--state-dir`, the adapter uses the console's existing default private,
|
||||
workspace-fingerprinted state directory. With `--state-dir PATH`, it uses
|
||||
`PATH/release-console/workspace-<fingerprint>/release-runs`. Workspaces cannot
|
||||
read or resume one another's runs. Existing ownership, symlink, permission,
|
||||
retention and record-integrity checks are unchanged.
|
||||
|
||||
Release preparation can commit only the recognized, receipt-bound metadata
|
||||
changes its executors produced. This command does **not** stage arbitrary source
|
||||
changes, use `git add -A`, force-push, retarget tags or enable the disabled generic
|
||||
prepare/sync/push endpoints. Normal selected-path maintenance requires its own
|
||||
explicit reviewed-change workflow; it is not silently folded into release.
|
||||
@@ -361,6 +361,15 @@ and WebUI API proxy traffic across API replicas. The WebUI and API services do
|
||||
not publish host ports. HAProxy has no Docker socket and discovers only the
|
||||
bounded replica slots rendered into `load-balancer.cfg`.
|
||||
|
||||
New installation specifications default to HAProxy `3.2.23-alpine`, pinned to
|
||||
registry index `sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
|
||||
This patch remains on HAProxy 3.2 LTS and Alpine 3.24.1. Loading an existing
|
||||
specification preserves its explicit image; it does not perform an upgrade.
|
||||
The [runtime remediation evidence](../security/RUNTIME_IMAGE_REMEDIATION_2026-09-08.md)
|
||||
records both architecture scans and the pending binary/configuration, runtime,
|
||||
inventory and final-image checks. The source default is not a release approval:
|
||||
runtime publication remains held in Meta #52.
|
||||
|
||||
Replica counts are desired state:
|
||||
|
||||
```sh
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# Integrity and performance source release — September 2026
|
||||
|
||||
Coordinated tracking: [Core #298](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298).
|
||||
This source publication advances only the affected packages; it is not a signed
|
||||
catalog publication, runtime-image release, or remote production rollout.
|
||||
|
||||
## Package set
|
||||
|
||||
| Package | Version |
|
||||
| --- | --- |
|
||||
| Core / developer meta-package | 0.1.46 |
|
||||
| Addresses | 0.1.23 |
|
||||
| Calendar | 0.1.24 |
|
||||
| Campaign | 0.1.29 |
|
||||
| Cases | 0.1.25 |
|
||||
| Committee | 0.1.22 |
|
||||
| Connectors | 0.1.27 |
|
||||
| Dataflow | 0.1.25 |
|
||||
| Datasources | 0.1.26 |
|
||||
| Files | 0.1.27 |
|
||||
| Forms Runtime | 0.1.22 |
|
||||
| IDM | 0.1.26 |
|
||||
| Mail | 0.1.28 |
|
||||
| Reporting | 0.1.22 |
|
||||
| Tickets | 0.1.23 |
|
||||
|
||||
Consumers of new Core helpers require Core 0.1.46 or later. Dataflow and
|
||||
Datasources also require the matching Core WebUI contract. The release manifests,
|
||||
immutable Git lock, and developer package describe this coordinated composition.
|
||||
Unchanged packages retain their independent versions.
|
||||
|
||||
## Database and data integrity
|
||||
|
||||
The reviewed additive heads are Connectors `d2a4c6e8f0b1`, Datasources
|
||||
`e2b8d4a0f6c3`, Files `a2b3c4d5e701`, and Mail `b5d6e7f8091a`.
|
||||
Back up the target deployment and rehearse its normal upgrade before rollout.
|
||||
Never downgrade away retained CSV originals without a separate recovery plan.
|
||||
Files preserves historical duplicate copies; Mail leaves legacy maildrop identity
|
||||
unset rather than guessing an account. A schema upgrade does not authorize POP3
|
||||
retrieval/reconciliation, provider deletion, or message resending.
|
||||
|
||||
Development startup may automatically apply pending migrations after a watched
|
||||
source change. Therefore, inspect actual Alembic heads and schema before assuming
|
||||
a live development database is still at its pre-change state. A backup taken
|
||||
after such an upgrade is a current-state recovery copy, not a pre-upgrade backup.
|
||||
|
||||
On 8 September, the local PostgreSQL development database already contained all
|
||||
four heads. Schema and constraint inspection passed; recomputing the Files
|
||||
identity backfill checked 7,385 rows with zero mismatches. A private current-state
|
||||
database backup was created. Per-instance backup paths and row contents are not
|
||||
published in this repository. The implementation's initial receipt claiming no
|
||||
live migration occurred was incorrect: watched development-server restarts had
|
||||
applied the migration files automatically. Restoring the backup into an isolated
|
||||
PostgreSQL 16 cluster and running the normal upgrade preserved all 281 public
|
||||
tables, 142,287 rows, and migration heads exactly. Both Datasources PostgreSQL
|
||||
concurrency regressions passed. The test-only cluster was then stopped.
|
||||
|
||||
Release preparation additionally corrected Alembic's ConfigParser handling of
|
||||
percent-escaped connection URLs, preserving the exact database URL. The static
|
||||
migration auditor now recognizes the existing reviewed development-wrapper
|
||||
aliases and peer filenames without executing migration code; historical
|
||||
migrations are unchanged.
|
||||
|
||||
## Verification and boundaries
|
||||
|
||||
The implementation passed the required focused workspace gate, including 63
|
||||
frontend build configurations and 230 browser conformance tests. Owning-module
|
||||
regressions cover exact import/rollback evidence, authorization-before-pagination,
|
||||
bounded recurrence and response processing, collision-safe attachment naming,
|
||||
and stale asynchronous UI completion. English/German feature documentation stays
|
||||
in each owning module; Core documents shared integrity contracts.
|
||||
|
||||
Mock-provider tests and local work-count measurements do not establish production
|
||||
throughput or provider race behavior. Real S3/SMB/Seafile and POP3 acceptance,
|
||||
representative load testing, and deployment authentication checks remain separate
|
||||
operational validation. Source tags trigger package workflows; a pushed source
|
||||
tag alone does not prove a registry artifact or signed catalog is published.
|
||||
|
||||
## Deutsch
|
||||
|
||||
Dieses koordinierte Quellrelease veröffentlicht nur die betroffenen Pakete.
|
||||
Produktions-Images, signierter Modulkatalog und entfernte Produktivinstanzen
|
||||
werden dadurch nicht ausgerollt. Die vier Migrationen bewahren bestehende Daten;
|
||||
historische POP3-Zuordnungen werden nicht geraten. POP3 ist ein eigener
|
||||
Import-Arbeitsablauf innerhalb von **Mail**, kein separat installierbares Modul.
|
||||
|
||||
Der Entwicklungsserver kann Migrationen beim automatischen Neustart nach einer
|
||||
Quelländerung bereits anwenden. Tatsächliche Schema-Stände prüfen; eine danach
|
||||
erstellte Sicherung enthält den aktuellen Stand und ist keine Sicherung vor dem
|
||||
Upgrade. Externe Transportaktionen, erneutes Versenden und Löschungen werden
|
||||
durch die Migration oder Quellveröffentlichung nicht ausgelöst.
|
||||
@@ -263,11 +263,198 @@ python tools/release/generate-developer-meta-package.py
|
||||
python tools/release/generate-developer-meta-package.py --check
|
||||
```
|
||||
|
||||
The direct generator is a development synchronization tool, not a receipt-gated
|
||||
release executor. For release preparation, use the guarded out-of-run stage below.
|
||||
|
||||
`push-release-tag.sh` performs this synchronization before release commits and
|
||||
tags. The meta-package is for editable/developer setup and composition tests. It
|
||||
does not enable modules, apply migrations, provision services, or establish
|
||||
backup and recovery evidence.
|
||||
|
||||
### Shared source-tag contract and Meta composition
|
||||
|
||||
The shared version collector names Meta's real
|
||||
`packages/govoplan-meta/pyproject.toml` separately from root `pyproject.toml`.
|
||||
Only the registered `govoplan` system/meta repository with nested project name
|
||||
`govoplan` receives this contract. Missing, unknown, or misidentified metadata
|
||||
does not become a versionless exception. Version alignment compares the complete
|
||||
nested file with the canonical operator-tool generator output: its version must
|
||||
match Core, and dependencies and `full` composition must match the reviewed
|
||||
requirements and workspace package versions. Validation never executes a
|
||||
generator from a selected checkout. The shared trusted manifest checker can
|
||||
load reviewed application manifests; these checks are not a code sandbox.
|
||||
|
||||
Meta's complete generated file is recognized by shared version-mutation discovery,
|
||||
but the generic durable version executor deliberately cannot write it. A durable
|
||||
run freezes the release console's own Meta checkout as trusted runtime code;
|
||||
changing it in place would invalidate that run. The planner therefore places Meta
|
||||
after Core and exposes only non-executable support preparation/publication steps,
|
||||
not misleading automatic Meta version, commit, tag, or push actions. A missing or
|
||||
different Core target produces an actionable preparation prerequisite.
|
||||
|
||||
Prepare Core and the intended module inputs first, commit their reviewed state,
|
||||
then stop active durable runs for the target workspace. Use trusted operator tools
|
||||
against a separate registered, private source checkout, never the running operator
|
||||
Meta directory. Preview outside any selected source checkout, for example:
|
||||
|
||||
```sh
|
||||
python tools/release/prepare-developer-meta-package.py \
|
||||
--workspace /private/release-workspace --target-version X.Y.Z \
|
||||
> /private/operator/meta-preview.json
|
||||
python tools/release/prepare-developer-meta-package.py \
|
||||
--workspace /private/release-workspace --target-version X.Y.Z \
|
||||
--receipt /private/operator/meta-preview.json --apply --confirm-out-of-run
|
||||
```
|
||||
|
||||
The explicit confirmation attests that no durable run is active for that target
|
||||
workspace; the helper does not discover or stop other processes. Preview/apply
|
||||
requires registered clean main sources, matching origins and live-main ancestry,
|
||||
Core already aligned at the target, the exact nested identity, and no existing or
|
||||
unverifiable target Meta tag. Frozen receipts cover source HEADs/filesystem
|
||||
identities, release requirements, every discovered registered full-composition
|
||||
pyproject, the trusted generator snapshot, and the resulting full-file hash.
|
||||
Inputs are limited to 128 selected files, 2 MiB per file and 16 MiB aggregate.
|
||||
The canonical generator renders copied bounded data in a temporary directory;
|
||||
no generator from the selected checkout executes. Changed receipts block before
|
||||
the file effect. Apply writes only `packages/govoplan-meta/pyproject.toml`, then
|
||||
rechecks the other sources and exact output. A write or post-check failure that
|
||||
may have changed the file reports `needs-reconciliation` and leaves that bounded
|
||||
delta for explicit review; it never retries, rolls back, commits or publishes.
|
||||
|
||||
Review the complete generated composition and manually commit the resulting file.
|
||||
Complete matching Core publication before guarded Meta source tagging/publication,
|
||||
then start a fresh durable run from reviewed, clean, published operator tooling.
|
||||
Hot self-updating durable Meta release execution remains explicitly unsupported;
|
||||
this out-of-run preparation is the existing developer-meta support contract.
|
||||
|
||||
For every `tag_repositories` batch, strict checks apply to every selected
|
||||
repository before any tag creation, fetch, or push: registered checkout and
|
||||
origin/push URL, a clean `main` tracking `origin/main`, live remote-main ancestry,
|
||||
exact frozen HEADs, and immutable annotated local/remote tag objects. Git metadata
|
||||
must remain inside the operator's trusted workspace. Missing local knowledge of
|
||||
live remote main is a blocker; fetch and review it separately. Unknown repositories
|
||||
and non-registered remote aliases fail closed. If selected, Meta runs last.
|
||||
For Meta only, the matching annotated Core release must exist before its effect; Core may be
|
||||
an earlier selected repository, or an already tagged dependency. Publication
|
||||
requires that Core's exact tag and main commit are already remote.
|
||||
|
||||
Non-Meta selections do not acquire Meta's composition or Core-tag prerequisite.
|
||||
Local module-candidate tags still work before Core's final release lock or tag.
|
||||
The existing Core WebUI bundle gate still applies to module publication and
|
||||
batches selecting Core: relevant Core release-package and release-lock inputs
|
||||
must be operator-owned regular files, at most 16 MiB each, and their identities
|
||||
and content hashes are frozen before preflight and rechecked before every effect.
|
||||
When Core is unselected, this does not require its checkout to be clean or tagged;
|
||||
reviewed pending composition inputs retain their previous meaning. Backend-only
|
||||
selections never read irrelevant Core WebUI files.
|
||||
|
||||
Before even read-only Git commands, source ancestry must be owned by root or the
|
||||
current operator and must not be group/world writable. A sticky shared ancestor
|
||||
such as `/tmp` is permitted only above an owned, protected child; the workspace
|
||||
and checkouts receive no writable-directory exception. The current operator must
|
||||
own source inputs and actual Git/worktree/common metadata, which must be regular
|
||||
files/directories, non-symlinked, and non-writable by other users. Metadata walks
|
||||
are bounded to 500,000 entries and 128 levels, and tracked inputs to 100,000 paths
|
||||
and 16 MiB of listing text. Read-only Git targets, object alternates/grafts and
|
||||
hidden/sparse/unmerged index entries are blocked. Frozen receipts include actual
|
||||
checkout/Git directory paths, devices, inodes, owners and modes, so replacing Git
|
||||
metadata with the same HEAD is still detected. All selected version/composition
|
||||
inputs must be tracked, including root and WebUI package/lock metadata, discovered
|
||||
module manifests and package initializers, and Meta's nested package and release
|
||||
requirements; ignored working files cannot supply declarations absent from a tag.
|
||||
No chmod, ownership repair or
|
||||
global Git trust change is performed. A shared writable workspace must first be
|
||||
recreated or reviewed in the operator's protected release area by an explicitly
|
||||
authorized preparation workflow.
|
||||
|
||||
Preview is read-only. Local-tag mode creates only pinned annotated tags (or
|
||||
retrieves an identical published annotation); it does not publish main or tags.
|
||||
Publish mode atomically pushes the frozen main commit and annotation object,
|
||||
without force, retagging, fallback, or automatic retry. The complete source
|
||||
receipt is rechecked before every effect and afterward; remote main and the
|
||||
exact annotated tag must both match, not merely the Git exit status. Changes
|
||||
after preflight stop the remaining batch. Atomicity is per repository, not
|
||||
across repositories: earlier successful publications and a newly created local
|
||||
tag can remain after a later failure. Inspect reported receipts and obtain a new
|
||||
review before retrying; do not move immutable tags.
|
||||
|
||||
Whole-batch revalidation deliberately repeats source and live-remote checks around
|
||||
each repository effect; the number of checks can grow quadratically with batch
|
||||
size. Plan release time accordingly rather than bypassing trust checks. The
|
||||
shared internal preflight is read-only and exposes no legacy mutation path.
|
||||
The fixture suite covers Meta and non-Meta preview/local-tag/
|
||||
publication using temporary local bare remotes, including stale compositions,
|
||||
unsafe origins, divergent branches, damaged tag identity, changed receipts and
|
||||
false publication success. This is local tooling evidence, not a real release
|
||||
publication or production permission check.
|
||||
|
||||
Deutsch: Die gemeinsamen Helfer erkennen ausschließlich das registrierte
|
||||
Meta-Repository mit dem echten Paket `packages/govoplan-meta/pyproject.toml`
|
||||
(Projektname `govoplan`). Version und vollständige Zusammensetzung müssen dem
|
||||
kanonischen Generator, Core und den geprüften Anforderungen entsprechen; der
|
||||
Generator stammt niemals aus dem ausgewählten Checkout. Der gemeinsame
|
||||
Manifestprüfer kann geprüften Anwendungscode laden und ist keine Sandbox.
|
||||
Die gemeinsame Änderungsplanung erkennt die vollständig generierte Paketdatei,
|
||||
aber der dauerhafte Versionsausführer darf Meta nicht selbst verändern: sein
|
||||
eingefrorener Lauf bindet den Meta-Checkout als vertrauenswürdigen Programmstand.
|
||||
Meta erscheint deshalb nach Core ausschließlich mit nicht automatisch ausführbaren
|
||||
Vorbereitungs-/Veröffentlichungsschritten. Zuerst Core und Modulquellen vorbereiten
|
||||
und geprüft committen; bei abweichender Core-Zielversion nennt der Plan diese
|
||||
Voraussetzung ausdrücklich. Aktive dauerhafte Läufe des Ziel-Workspaces beenden.
|
||||
Mit `prepare-developer-meta-package.py` zunächst eine Vorschau außerhalb der
|
||||
Quell-Checkouts speichern, dann deren JSON über `--receipt` zusammen mit `--apply`
|
||||
und `--confirm-out-of-run` bestätigen. Das Ziel muss ein separater registrierter
|
||||
privater Checkout sein, niemals das laufende Operator-Meta. Die Bestätigung ist
|
||||
eine Betreibererklärung; der Helfer sucht oder beendet keine fremden Prozesse.
|
||||
Quell-HEADs, Pfadidentitäten, Anforderungen, alle registrierten vollständigen
|
||||
Paket-Eingaben, der vertrauenswürdige Generator und der vollständige Ausgabehash
|
||||
werden eingefroren. Es gelten höchstens 128 Quelldateien, 2 MiB je Datei und
|
||||
16 MiB insgesamt. Core muss bereits vollständig zur Zielversion passen; vorhandene
|
||||
oder nicht verifizierbare Meta-Zieltags sperren die Vorbereitung. Geänderte
|
||||
Nachweise stoppen vor dem Schreiben. Ausschließlich die verschachtelte Paketdatei
|
||||
wird vollständig generiert und danach geprüft; ein Fehler nach dem Schreiben
|
||||
meldet `needs-reconciliation` und erfordert die manuelle Prüfung dieser begrenzten
|
||||
Änderung, ohne automatisches Zurücksetzen. Kein automatischer
|
||||
Commit, Push oder Wiederholungsversuch findet statt. Zusammensetzung prüfen,
|
||||
manuell committen, Core zuerst veröffentlichen, dann die geschützte Meta-Tag-Route
|
||||
verwenden und einen neuen dauerhaften Lauf starten. Eine Selbstaktualisierung
|
||||
des aktiven dauerhaften Meta-Laufs bleibt ausdrücklich nicht unterstützt.
|
||||
Für jeden Tag-Stapel, auch ohne Meta, gelten
|
||||
Vertrauens-, Origin-, saubere Main- und Live-Abstammungsprüfungen für die gesamte
|
||||
Auswahl vor jeder Änderung. Unbekannte Repositories und nicht registrierte
|
||||
Remote-Aliase sind gesperrt. Nur bei ausgewähltem Meta gelten zusätzlich dessen
|
||||
Zusammensetzungsprüfung und der passende annotierte Core-Tag als Voraussetzung;
|
||||
Meta folgt zuletzt. Für Metas Veröffentlichung müssen Core-Tag und Main-Commit
|
||||
bereits auf dem Remote vorliegen. Lokale Modul-Kandidatentags bleiben vor Cores
|
||||
abschließendem Release-Lock und Tag möglich. Die vorhandene Core-WebUI-Bundleprüfung
|
||||
bleibt bei Modulveröffentlichung und Core-Auswahl erhalten. Relevante Core-Paket-
|
||||
und Lockdateien müssen eigene reguläre Dateien mit höchstens je 16 MiB sein;
|
||||
Identität und Inhaltshash werden eingefroren und vor jeder Aktion erneut geprüft.
|
||||
Nicht ausgewähltes Core benötigt dafür weder einen sauberen Checkout noch einen
|
||||
Tag. Reine Backend-Auswahlen lesen keine irrelevanten Core-WebUI-Dateien.
|
||||
Vor Git-Aufrufen werden Eigentümer, Schreibrechte, sichere
|
||||
Pfadabstammung und echte Git-/Worktree-Metadaten geprüft; veränderbare gemeinsame
|
||||
Verzeichnisse, fremde Eigentümer, Alternates, Grafts und versteckte Indexeinträge
|
||||
sind gesperrt. Ein Sticky-Bit-Vorfahr wie `/tmp` ist nur oberhalb eines eigenen
|
||||
geschützten Unterverzeichnisses zulässig. Es erfolgen weder Rechtereparaturen
|
||||
noch globale Git-Vertrauensänderungen. Ausgewählte Versions- und Zusammensetzungs-
|
||||
dateien müssen versioniert sein: Paket-/Lockdateien, Modulmanifeste und
|
||||
Paketinitialisierer sowie Metas verschachteltes Paket und Release-Anforderungen.
|
||||
Ignorierte Arbeitsdateien dürfen keine vom Tag abweichenden Angaben liefern.
|
||||
Die Vorschau schreibt nichts, lokale Tags veröffentlichen nichts,
|
||||
und die Veröffentlichung überträgt Main und den exakten annotierten Tag atomar
|
||||
je Repository. Unmittelbar vor und nach den Aktionen werden die eingefrorenen
|
||||
Quellnachweise erneut geprüft, einschließlich entferntem Main und Tag-Objekt.
|
||||
Bei Änderungen oder Fehlern stoppt der Rest des Stapels ohne automatischen
|
||||
Wiederholungsversuch. Frühere Veröffentlichungen und neu erzeugte lokale Tags
|
||||
können bestehen bleiben: vor einem neuen Versuch Nachweise prüfen und erneut
|
||||
freigeben, niemals unveränderliche Tags verschieben. Die vollständigen Quell- und
|
||||
Live-Remote-Prüfungen werden um jede Aktion wiederholt; bei großen Stapeln kann
|
||||
deren Anzahl quadratisch wachsen. Diese konservativen Prüfkosten gehören zur
|
||||
Release-Planung. Der interne Vorprüfer ist ausschließlich lesend und besitzt
|
||||
keinen alten Änderungspfad. Tests für Auswahlen mit und ohne Meta verwenden
|
||||
nur temporäre lokale Remotes und ersetzen keine echte Veröffentlichungsprüfung.
|
||||
|
||||
If the tag-triggered developer meta-package job fails before publication, rerun
|
||||
`publish-developer-meta-package.yml` with the existing protected version. The
|
||||
manual path validates that tag against `main`, checks out its exact commit, and
|
||||
|
||||
@@ -354,8 +354,17 @@ such as `0.2.0` or `0.2.0-alpha1`, but requires the first three version numbers
|
||||
to move forward.
|
||||
|
||||
Plain repository pushes are separate from catalog publication. `Preview Push`
|
||||
shows the selected repository push commands. `Push Selected` requires `PUSH` in
|
||||
the repository push confirmation field.
|
||||
shows the selected repository push commands, but generic push, sync and prepare
|
||||
mutation endpoints are disabled: they require a separate durable, receipt-bound
|
||||
maintenance workflow and cannot be enabled by typing a confirmation. Source
|
||||
release branch/tag publication uses the durable release-run steps described
|
||||
above; do not route ordinary dirty worktrees through the legacy all-repository
|
||||
stage/commit/tag helper.
|
||||
|
||||
The [headless devkit release commands](DEVKIT_RELEASE.md) invoke this same
|
||||
application in-process without starting a server. They expose selective planning,
|
||||
bounded status/history, create/show/preview/execute and explicit recovery while
|
||||
retaining the same request IDs, confirmations, source bindings and receipts.
|
||||
|
||||
The source release panel retains `Preview Tag + Publish` as a non-mutating
|
||||
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# WebUI release dependency installer retries
|
||||
|
||||
## English
|
||||
|
||||
This operational note covers
|
||||
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
|
||||
and the exit-status repair tracked in
|
||||
[Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54).
|
||||
It applies to release administrators using the legacy runtime WebUI installer;
|
||||
there are no new application settings, permissions, or end-user workflows.
|
||||
|
||||
Each retried npm install or Git clone has at most three attempts. The installer
|
||||
waits 10 seconds after the first failure and 20 seconds after the second, and
|
||||
continues immediately after success. If all attempts fail, it exits with the
|
||||
last command's nonzero status. Its `set -e` execution stops before subsequent
|
||||
installation stages; callers using `set -e` also stop before subsequent work.
|
||||
Previously, the retry helper could report success after three failures because
|
||||
it captured the status of a completed `if` statement instead of the command.
|
||||
|
||||
On exhaustion, inspect the npm or Git error and correct the reported cause
|
||||
before rerunning the installation. The temporary dependency workspace is
|
||||
removed on exit. Earlier changes to `package.json`, removal of `package-lock.json`,
|
||||
cache cleaning, and completed dependency installations are not rolled back;
|
||||
prepare a fresh disposable release workspace when a clean retry is required.
|
||||
|
||||
The repair preserves the existing retry count, backoff, cache behavior, and
|
||||
peer-resolution flags. It does not lift the runtime publication hold tracked in
|
||||
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
Review the historical `--legacy-peer-deps` workaround separately before lifting
|
||||
that hold. Strict disposable Git-release and signed catalog verification do not
|
||||
use this installer; strict release verification must not bypass peer checks.
|
||||
See [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md) for release context.
|
||||
|
||||
Run the isolated regression suite from the meta repository:
|
||||
|
||||
```sh
|
||||
python3 -m unittest -v tests.test_webui_release_dependency_retries
|
||||
```
|
||||
|
||||
The suite executes the actual Bash installer and a caller using `set -e`, with
|
||||
local npm, Git, Node, and sleep stubs. It covers success on attempts one, two, and
|
||||
three, final failure status, backoff, and termination at each retry call site.
|
||||
It performs no network access, real waiting, or changes to the real npm cache.
|
||||
It checks shell control flow, not package resolution or runtime publication.
|
||||
|
||||
## Deutsch
|
||||
|
||||
Dieser Betriebshinweis beschreibt
|
||||
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
|
||||
und die unter [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54)
|
||||
erfasste Korrektur des Rückgabestatus. Er richtet sich an Release-Administratoren,
|
||||
die den bisherigen WebUI-Installer für Laufzeit-Releases verwenden. Neue
|
||||
Anwendungseinstellungen, Berechtigungen oder Endanwenderabläufe entstehen nicht.
|
||||
|
||||
Jede wiederholte npm-Installation und jeder Git-Klon erhält höchstens drei
|
||||
Versuche. Nach dem ersten Fehlschlag wartet der Installer 10 Sekunden, nach dem
|
||||
zweiten 20 Sekunden; nach einem Erfolg fährt er sofort fort. Scheitern alle
|
||||
Versuche, endet er mit dem letzten von null verschiedenen Rückgabestatus.
|
||||
Durch `set -e` werden nachfolgende Installationsschritte nicht ausgeführt;
|
||||
auch aufrufende Skripte mit `set -e` brechen vor ihren nächsten Schritten ab.
|
||||
Bisher konnte die Hilfsfunktion nach drei Fehlschlägen Erfolg melden, weil sie
|
||||
den Status der abgeschlossenen `if`-Anweisung statt des Befehls übernahm.
|
||||
|
||||
Prüfen Sie nach dem Abbruch die npm- oder Git-Fehlermeldung und beheben Sie deren
|
||||
Ursache vor einem erneuten Installationslauf. Das temporäre Verzeichnis für
|
||||
Abhängigkeiten wird beim Beenden entfernt. Vorherige Änderungen an `package.json`,
|
||||
das Entfernen von `package-lock.json`, die Cache-Bereinigung und abgeschlossene
|
||||
Installationen werden nicht zurückgerollt. Bereiten Sie bei Bedarf einen neuen
|
||||
temporären Release-Arbeitsbereich für einen sauberen Wiederholungslauf vor.
|
||||
|
||||
Die Korrektur erhält Anzahl und Wartezeiten der Versuche, Cache-Verhalten und
|
||||
Optionen zur Peer-Auflösung. Die Sperre für Laufzeitveröffentlichungen aus
|
||||
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52) bleibt bestehen.
|
||||
Der bisherige Einsatz von `--legacy-peer-deps` muss vor ihrer Aufhebung gesondert
|
||||
geprüft werden. Die strenge Git-Release-Prüfung in einem temporären Arbeitsbereich
|
||||
und die Prüfung signierter Kataloge verwenden diesen Installer nicht; die strenge
|
||||
Release-Prüfung darf Peer-Prüfungen nicht umgehen. Weitere Zusammenhänge erläutert
|
||||
[Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md).
|
||||
|
||||
Führen Sie die isolierten Regressionstests im Meta-Repository aus:
|
||||
|
||||
```sh
|
||||
python3 -m unittest -v tests.test_webui_release_dependency_retries
|
||||
```
|
||||
|
||||
Die Tests führen den tatsächlichen Bash-Installer und ein aufrufendes Skript mit
|
||||
`set -e` aus. Lokale Testprogramme ersetzen npm, Git, Node und sleep. Geprüft werden
|
||||
Erfolge im ersten, zweiten und dritten Versuch, der letzte Fehlerstatus,
|
||||
Warteintervalle und der Abbruch an jeder Aufrufstelle. Es gibt keine
|
||||
Netzwerkzugriffe, echten Wartezeiten oder Änderungen am tatsächlichen npm-Cache.
|
||||
Die Tests prüfen den Shell-Ablauf, nicht die Paketauflösung oder Veröffentlichung.
|
||||
Executable
+164
@@ -0,0 +1,164 @@
|
||||
# Product-wide UI review program
|
||||
|
||||
The [central Gitea epic](https://git.add-ideas.de/GovOPlaN/govoplan/issues/56)
|
||||
coordinates the review; its linked module issues are the canonical backlog,
|
||||
progress and evidence log. The [issue inventory](ui-review-issue-inventory.json)
|
||||
is a discovery/link snapshot, **not a second progress tracker**. All reviews were
|
||||
initialized pending. Implementing a shared component or moving a help icon does
|
||||
not by itself complete a module review.
|
||||
|
||||
The visual and interaction rules live in Core's
|
||||
[UI design principles](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/UI_DESIGN_PRINCIPLES.md).
|
||||
This document defines how to apply and verify those principles across repositories;
|
||||
it does not fork their definitions. The usual [Gitea issue contract](GITEA_ISSUES.md)
|
||||
still applies. At the user's request, this cross-product program is coordinated in
|
||||
Meta; shared primitives and their implementation remain owned by Core.
|
||||
|
||||
## Scope and initial inventory
|
||||
|
||||
The September 2026 source/catalog baseline contains **77 review tracks**:
|
||||
|
||||
- **73 implemented scopes:** Core/shared shell and 72 manifest-backed modules.
|
||||
- **4 registered placeholders:** Appointments, Ledger, XÖV and XTA/OSCI. They have
|
||||
README-only repositories at initialization. Their separate readiness tracks
|
||||
remain pending for future interfaces; they are not counted as implemented or
|
||||
reviewed modules.
|
||||
|
||||
Meta and the public website are not business-module review scopes. Module IDs
|
||||
come from actual manifests, not repository-name guesses: for example,
|
||||
`govoplan-campaign` owns `campaigns` and `govoplan-dist-lists` owns `dist_lists`.
|
||||
The inventory is derived from `repositories.json`, trusted module manifests and
|
||||
the owning repository's source files. Changes to the catalog must be reconciled
|
||||
with the central epic rather than silently dropping a module.
|
||||
|
||||
A module with no standalone WebUI is still in scope. Review its configuration
|
||||
and administrator experience, contributed operator/public surfaces, widgets,
|
||||
documentation, permissions, errors and interactions exposed through other
|
||||
modules. A missing standalone page is not evidence of conformance. A reviewer
|
||||
may mark a particular principle not applicable only with a concrete explanation
|
||||
and source/runtime evidence. A placeholder likewise needs an explicit future-UI
|
||||
gate, not a fabricated page inventory or an automatic green review.
|
||||
|
||||
## Work order and ownership
|
||||
|
||||
Prioritize user-visible defects, usability and consistent interactions before
|
||||
broader features. Start with Core/shared-shell contracts and Campaign's complete
|
||||
workflow, then Files, Mail, Templates, Notifications, Docs/Help, Dashboard, Quick
|
||||
Access and Views. The remaining queue is ordered by observed defects, shared
|
||||
dependencies and risk. This is a starting order, not a claim that those reviews
|
||||
have started or finished; the central issue records the current focus.
|
||||
|
||||
For Campaign, use a compact, read-only campaign settings overview with explicit,
|
||||
scoped edit dialogs. Large recipient or attachment grids may use the UI-02 bulk
|
||||
editing exception: clearly entered edit mode, explicit Save and Cancel, dirty
|
||||
navigation protection, and no silent loss or accidental save. This direction
|
||||
must be implemented and verified in the Campaign review, not marked complete by
|
||||
the program bootstrap.
|
||||
|
||||
Fix a repeated problem in the owning shared primitive or semantic contract first,
|
||||
then migrate consumers and verify representative actual-module fixtures. Keep
|
||||
module business behavior in its own repository and preserve optional module
|
||||
boundaries. Do not replace concrete user evidence with a generic fixture alone.
|
||||
Parallel reviewers may own different modules, but they must coordinate shared
|
||||
Core files and browser fixtures instead of racing the same resources.
|
||||
|
||||
## Per-module review workflow
|
||||
|
||||
1. **Inventory.** Verify and extend the issue's source-derived seed: navigation,
|
||||
pages, nested routes, dialogs, labels/forms, tables/trees, admin/system/tenant/
|
||||
user settings, public/operator interfaces, widgets and optional-module
|
||||
contributions. Follow actual module entrypoints and runtime contributions;
|
||||
filename heuristics and manifest routes are a starting point, not an exhaustive
|
||||
runtime audit. Note roles, permission boundaries and installed optional modules.
|
||||
2. **Observe.** Exercise realistic narrow and wide viewports, German and English,
|
||||
keyboard navigation and focus, empty/loading/error/success states, clean/dirty
|
||||
edit modes, reload, navigation away and return, and restoration of personal
|
||||
preferences. Include realistic row counts, long labels, horizontal overflow,
|
||||
pagination and two-way column resizing with fixed columns between text fields.
|
||||
3. **Record.** Keep a findings/TODO/done ledger in the module issue. Every finding
|
||||
names the surface, reproduction, expected principle, impact, owner and linked
|
||||
implementation/evidence. A source inventory, planned fix or green generic
|
||||
test is not a completed finding. State product decisions and manual checks
|
||||
explicitly with enough context to resolve them.
|
||||
4. **Implement.** Prefer reusable shared primitives and action/page contracts.
|
||||
Keep display mode readable and editing intentional. Preserve authorization,
|
||||
server validation, save/cancel behavior and data integrity. Never send mail,
|
||||
delete records, change live configuration or trigger other irreversible effects
|
||||
merely to obtain UI evidence; use authorized fixtures or a safe test context.
|
||||
5. **Verify.** Run proportionate unit, structure and actual-module browser tests.
|
||||
Record commands, results, source revision and safe runtime evidence. Update the
|
||||
owning module's manifest-driven EN/DE documentation for behavior changes; run
|
||||
`tools/checks/check-manifest-shapes.py` and relevant cross-module checks.
|
||||
6. **Conclude honestly.** Complete the principle matrix, evidence and manual
|
||||
checks. Link unresolved follow-ups and blockers; do not close a supposedly
|
||||
complete review while required work is still pending. Closing an issue requires
|
||||
explicit reviewed scope and the applicable design-principle revision.
|
||||
|
||||
## Principle ledger and back-propagation
|
||||
|
||||
Use the stable IDs from Core in each issue. The initial revision covers UI-01
|
||||
heading/label help placement; UI-02 display-first/scoped editing; UI-03 actions;
|
||||
UI-04 table/card geometry; UI-05 loading/error/progress; UI-06 tree interaction;
|
||||
UI-07 accessibility and German; UI-08 data integrity; and UI-09 propagation of
|
||||
revised principles. Core remains the authoritative definition.
|
||||
|
||||
Each module maintains this matrix, initially entirely pending:
|
||||
|
||||
| Principle/revision | Applicable surfaces / justified N/A | Applied / remaining work | Evidence | Exception / owner / follow-up |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| UI-01 … UI-09 | Pending inventory | Pending review | Not yet recorded | None approved |
|
||||
|
||||
Whenever a principle is added or changed:
|
||||
|
||||
1. Update Core's rule with its rationale, stable ID and revision/change reference.
|
||||
2. List **all already-reviewed modules**, not just the current implementation's
|
||||
consumers. Check whether the changed rule applies to each.
|
||||
3. Record unchanged conformance with evidence, or reopen the review / create a
|
||||
linked follow-up with owner, exact affected surfaces and required checks.
|
||||
4. Update the central epic's propagation ledger. Previously reviewed modules with
|
||||
outstanding applicable work are **follow-up required**, not silently green.
|
||||
5. Resolve the propagation sweep only after every affected module has current
|
||||
evidence or an explicit, owned and justified exception. Exceptions do not
|
||||
silently change the shared rule.
|
||||
|
||||
Suggested central ledger:
|
||||
|
||||
| Principle change | Already-reviewed modules checked | Conformant evidence | Reopened / follow-up required | Exceptions / owner | Sweep state |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Initial UI-01–UI-09 baseline | None reviewed at initialization | None | All module reviews pending | None approved | Pending |
|
||||
|
||||
## Safe inventory/bootstrap automation
|
||||
|
||||
`tools/gitea/gitea-ui-review-program.py` derives the current scopes from the
|
||||
catalog and source manifests, inventories routes and UI source entrypoints,
|
||||
then deduplicates issues by a stable hidden marker and normalized exact title.
|
||||
It includes closed issues in duplicate checks and never reopens, closes or
|
||||
overwrites existing module issues. Ambiguous or unmanaged duplicates stop the
|
||||
operation for review. Existing issue bodies, labels and review progress are
|
||||
preserved. A failed POST is not automatically retried; a subsequent run checks
|
||||
the marker again before deciding whether another create is needed.
|
||||
|
||||
Dry-run is the default. An authenticated dry-run performs only Gitea reads:
|
||||
|
||||
```sh
|
||||
.venv/bin/python tools/gitea/gitea-ui-review-program.py \
|
||||
--env-file /home/zemion/.config/gitea/gitea.env --epic 56
|
||||
```
|
||||
|
||||
Creating missing review issues requires explicit `--apply`. Initial population
|
||||
of the central epic's module-link block additionally requires
|
||||
`--initialize-links`; it is accepted only for the untouched bootstrap placeholder.
|
||||
Later runs must not rewrite checkboxes or human-maintained progress. Existing
|
||||
complete links are verified read-only. The token is loaded through the shared
|
||||
Gitea helpers and is never emitted in reports. A scoped `--ipv4` option works
|
||||
around host-specific IPv6 connectivity while retaining HTTPS certificate and
|
||||
hostname verification.
|
||||
|
||||
The tool emits a JSON result to stdout with source counts and issue links. The
|
||||
checked-in inventory is a reviewed initialization snapshot of that result;
|
||||
refreshing links does not authorize replacing live review status with the
|
||||
snapshot. Test the automation offline with:
|
||||
|
||||
```sh
|
||||
.venv/bin/python -m pytest tests/test_ui_review_program.py -q
|
||||
```
|
||||
Executable
+866
@@ -0,0 +1,866 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"snapshot_purpose": "Issue discovery links; live Gitea issues own review state and evidence.",
|
||||
"epic": {
|
||||
"repository": "govoplan",
|
||||
"number": 56,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan/issues/56"
|
||||
},
|
||||
"scope_count": 77,
|
||||
"manifest_modules": 72,
|
||||
"implemented_scopes": 73,
|
||||
"catalogued_placeholders": 4,
|
||||
"created": 77,
|
||||
"missing": 0,
|
||||
"issues": [
|
||||
{
|
||||
"scope_id": "core",
|
||||
"name": "Core / shared shell",
|
||||
"repository": "govoplan-core",
|
||||
"kind": "core",
|
||||
"ui_source_count": 111,
|
||||
"number": 301,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/301",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "access",
|
||||
"name": "Access",
|
||||
"repository": "govoplan-access",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 18,
|
||||
"number": 23,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/23",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "addresses",
|
||||
"name": "Addresses",
|
||||
"repository": "govoplan-addresses",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 26,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/26",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "admin",
|
||||
"name": "Admin",
|
||||
"repository": "govoplan-admin",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 9,
|
||||
"number": 11,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-admin/issues/11",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "approvals",
|
||||
"name": "Approvals",
|
||||
"repository": "govoplan-approvals",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "assets",
|
||||
"name": "Assets",
|
||||
"repository": "govoplan-assets",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-assets/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "audit",
|
||||
"name": "Audit",
|
||||
"repository": "govoplan-audit",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-audit/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "booking",
|
||||
"name": "Booking",
|
||||
"repository": "govoplan-booking",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-booking/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "calendar",
|
||||
"name": "Calendar",
|
||||
"repository": "govoplan-calendar",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 10,
|
||||
"number": 26,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-calendar/issues/26",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "campaigns",
|
||||
"name": "Campaigns",
|
||||
"repository": "govoplan-campaign",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 60,
|
||||
"number": 103,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/103",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "cases",
|
||||
"name": "Cases",
|
||||
"repository": "govoplan-cases",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 8,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-cases/issues/8",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "certificates",
|
||||
"name": "Certificates",
|
||||
"repository": "govoplan-certificates",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-certificates/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "committee",
|
||||
"name": "Committee",
|
||||
"repository": "govoplan-committee",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "connectors",
|
||||
"name": "Connectors",
|
||||
"repository": "govoplan-connectors",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 20,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/20",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "consultation",
|
||||
"name": "Consultation",
|
||||
"repository": "govoplan-consultation",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-consultation/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "contracts",
|
||||
"name": "Contracts",
|
||||
"repository": "govoplan-contracts",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-contracts/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "dashboard",
|
||||
"name": "Dashboard",
|
||||
"repository": "govoplan-dashboard",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 5,
|
||||
"number": 6,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dashboard/issues/6",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "dataflow",
|
||||
"name": "Dataflow",
|
||||
"repository": "govoplan-dataflow",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 5,
|
||||
"number": 24,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/24",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "datasources",
|
||||
"name": "Datasources",
|
||||
"repository": "govoplan-datasources",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "decisions",
|
||||
"name": "Decisions",
|
||||
"repository": "govoplan-decisions",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-decisions/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "dist_lists",
|
||||
"name": "Distribution Lists",
|
||||
"repository": "govoplan-dist-lists",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "dms",
|
||||
"name": "DMS",
|
||||
"repository": "govoplan-dms",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 3,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-dms/issues/3",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "docs",
|
||||
"name": "Docs",
|
||||
"repository": "govoplan-docs",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 23,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/23",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "encryption",
|
||||
"name": "Encryption",
|
||||
"repository": "govoplan-encryption",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 7,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/7",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "erp",
|
||||
"name": "ERP",
|
||||
"repository": "govoplan-erp",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-erp/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "evaluation",
|
||||
"name": "Evaluation",
|
||||
"repository": "govoplan-evaluation",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-evaluation/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "facilities",
|
||||
"name": "Facilities",
|
||||
"repository": "govoplan-facilities",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-facilities/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "files",
|
||||
"name": "Files",
|
||||
"repository": "govoplan-files",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 8,
|
||||
"number": 48,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/48",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "fit_connect",
|
||||
"name": "FIT-Connect",
|
||||
"repository": "govoplan-fit-connect",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-fit-connect/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "forms",
|
||||
"name": "Forms",
|
||||
"repository": "govoplan-forms",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 7,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/7",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "forms_runtime",
|
||||
"name": "Forms Runtime",
|
||||
"repository": "govoplan-forms-runtime",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 6,
|
||||
"number": 7,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/7",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "grants",
|
||||
"name": "Grants",
|
||||
"repository": "govoplan-grants",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-grants/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "helpdesk",
|
||||
"name": "Helpdesk",
|
||||
"repository": "govoplan-helpdesk",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-helpdesk/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "identity",
|
||||
"name": "Identity",
|
||||
"repository": "govoplan-identity",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "identity_trust",
|
||||
"name": "Identity Trust",
|
||||
"repository": "govoplan-identity-trust",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 4,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/4",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "idm",
|
||||
"name": "IDM",
|
||||
"repository": "govoplan-idm",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 14,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-idm/issues/14",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "inspections",
|
||||
"name": "Inspections",
|
||||
"repository": "govoplan-inspections",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-inspections/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "learning",
|
||||
"name": "Learning",
|
||||
"repository": "govoplan-learning",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-learning/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "mail",
|
||||
"name": "Mail",
|
||||
"repository": "govoplan-mail",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 6,
|
||||
"number": 25,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/25",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "mandates",
|
||||
"name": "Mandates",
|
||||
"repository": "govoplan-mandates",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-mandates/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "notifications",
|
||||
"name": "Notifications",
|
||||
"repository": "govoplan-notifications",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 7,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/7",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "ops",
|
||||
"name": "Ops",
|
||||
"repository": "govoplan-ops",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ops/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "organizations",
|
||||
"name": "Organizations",
|
||||
"repository": "govoplan-organizations",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 9,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-organizations/issues/9",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "parties",
|
||||
"name": "Parties",
|
||||
"repository": "govoplan-parties",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-parties/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "payments",
|
||||
"name": "Payments",
|
||||
"repository": "govoplan-payments",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 3,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-payments/issues/3",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "permits",
|
||||
"name": "Permits",
|
||||
"repository": "govoplan-permits",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-permits/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "policy",
|
||||
"name": "Policy",
|
||||
"repository": "govoplan-policy",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 14,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-policy/issues/14",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "poll",
|
||||
"name": "Poll",
|
||||
"repository": "govoplan-poll",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "portal",
|
||||
"name": "Portal",
|
||||
"repository": "govoplan-portal",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 4,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-portal/issues/4",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "postbox",
|
||||
"name": "Postbox",
|
||||
"repository": "govoplan-postbox",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 29,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-postbox/issues/29",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "procurement",
|
||||
"name": "Procurement",
|
||||
"repository": "govoplan-procurement",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-procurement/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "projects",
|
||||
"name": "Projects",
|
||||
"repository": "govoplan-projects",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 4,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-projects/issues/4",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "quick_access",
|
||||
"name": "Quick Access",
|
||||
"repository": "govoplan-quick-access",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 3,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-quick-access/issues/3",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "records",
|
||||
"name": "Records",
|
||||
"repository": "govoplan-records",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-records/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "reporting",
|
||||
"name": "Reporting",
|
||||
"repository": "govoplan-reporting",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 11,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/11",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "resources",
|
||||
"name": "Resources",
|
||||
"repository": "govoplan-resources",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-resources/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "rest",
|
||||
"name": "REST Connector",
|
||||
"repository": "govoplan-rest",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-rest/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "risk_compliance",
|
||||
"name": "Risk Compliance",
|
||||
"repository": "govoplan-risk-compliance",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "scheduling",
|
||||
"name": "Scheduling",
|
||||
"repository": "govoplan-scheduling",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 10,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-scheduling/issues/10",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "search",
|
||||
"name": "Search",
|
||||
"repository": "govoplan-search",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 4,
|
||||
"number": 6,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/6",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "services",
|
||||
"name": "Services",
|
||||
"repository": "govoplan-services",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-services/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "soap",
|
||||
"name": "SOAP Connector",
|
||||
"repository": "govoplan-soap",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-soap/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "tasks",
|
||||
"name": "Tasks",
|
||||
"repository": "govoplan-tasks",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 5,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tasks/issues/5",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "templates",
|
||||
"name": "Templates",
|
||||
"repository": "govoplan-templates",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 8,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/8",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "tenancy",
|
||||
"name": "Tenancy",
|
||||
"repository": "govoplan-tenancy",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 7,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tenancy/issues/7",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "tickets",
|
||||
"name": "Tickets",
|
||||
"repository": "govoplan-tickets",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 3,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-tickets/issues/3",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "transparency",
|
||||
"name": "Transparency",
|
||||
"repository": "govoplan-transparency",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-transparency/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "views",
|
||||
"name": "Views",
|
||||
"repository": "govoplan-views",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 3,
|
||||
"number": 6,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-views/issues/6",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "voting",
|
||||
"name": "Voting",
|
||||
"repository": "govoplan-voting",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 2,
|
||||
"number": 8,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-voting/issues/8",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "wiki",
|
||||
"name": "Wiki",
|
||||
"repository": "govoplan-wiki",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 1,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-wiki/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "workflow",
|
||||
"name": "Workflow",
|
||||
"repository": "govoplan-workflow",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 6,
|
||||
"number": 17,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow/issues/17",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "workflow_engine",
|
||||
"name": "Workflow Engine",
|
||||
"repository": "govoplan-workflow-engine",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 4,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/4",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "xrechnung",
|
||||
"name": "XRechnung",
|
||||
"repository": "govoplan-xrechnung",
|
||||
"kind": "manifest",
|
||||
"ui_source_count": 0,
|
||||
"number": 3,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/3",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "catalog:govoplan-appointments",
|
||||
"name": "Appointments",
|
||||
"repository": "govoplan-appointments",
|
||||
"kind": "placeholder",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-appointments/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "catalog:govoplan-ledger",
|
||||
"name": "Ledger",
|
||||
"repository": "govoplan-ledger",
|
||||
"kind": "placeholder",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-ledger/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "catalog:govoplan-xoev",
|
||||
"name": "Xoev",
|
||||
"repository": "govoplan-xoev",
|
||||
"kind": "placeholder",
|
||||
"ui_source_count": 0,
|
||||
"number": 2,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xoev/issues/2",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
},
|
||||
{
|
||||
"scope_id": "catalog:govoplan-xta-osci",
|
||||
"name": "Xta Osci",
|
||||
"repository": "govoplan-xta-osci",
|
||||
"kind": "placeholder",
|
||||
"ui_source_count": 0,
|
||||
"number": 1,
|
||||
"url": "https://git.add-ideas.de/GovOPlaN/govoplan-xta-osci/issues/1",
|
||||
"state_at_verification": "open",
|
||||
"operation": "created"
|
||||
}
|
||||
],
|
||||
"snapshot_date": "2026-09-08",
|
||||
"initial_review_status": "pending for every scope; this snapshot does not track subsequent issue progress"
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
# Runtime image remediation follow-up — 8 September 2026
|
||||
|
||||
Canonical tracking: [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
|
||||
and [website #9](https://git.add-ideas.de/add-ideas/addideas-govoplan-website/issues/9).
|
||||
This addendum supplements the [original audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md);
|
||||
it does not replace that historical baseline or lift either publication or
|
||||
deployment gate. The immutable 0.1.45 release and `catalog-v0.1.45` are unchanged.
|
||||
|
||||
## Source change and candidate decisions
|
||||
|
||||
New installer specifications now use
|
||||
`haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
|
||||
This is a patch update from 3.2.21 within the supported
|
||||
[3.2 LTS branch](https://www.haproxy.org/), keeping Alpine 3.24.1.
|
||||
The [publisher's exact build source](https://github.com/docker-library/haproxy/blob/7a5c202cde713867a737033dca56e7a211a8b8df/3.2/alpine/Dockerfile)
|
||||
and scanned image configuration retain the non-root `haproxy` user,
|
||||
`/usr/local/etc/haproxy/haproxy.cfg`, entrypoint and graceful-stop signal.
|
||||
The [upstream changelog](https://www.haproxy.org/download/3.2/src/CHANGELOG)
|
||||
includes HTTP parsing, TLS and memory-safety fixes in 3.2.22/3.2.23.
|
||||
Existing specifications retain their explicit image, including an older pin;
|
||||
this source change does not update a running installation.
|
||||
|
||||
| Candidate | OS | C / H / M / L / Unknown, per architecture | Disposition |
|
||||
| --- | --- | --- | --- |
|
||||
| HAProxy `3.2.23-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Installer source default updated; binary/runtime checks pending |
|
||||
| nginx-unprivileged `1.30.4-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Candidate only; compatibility and website OS upgrade review pending |
|
||||
| Caddy `2.11.4-alpine` | Alpine 3.23.5 | 1 / 38 / 41 / 12 / 23 | Not selected; all 39 C/H records have recorded fixes |
|
||||
| Node `24-alpine` (24.20.0) | Alpine 3.24.1 | 0 / 6 / 11 / 12 / 0 | Not selected; major change and six fixable HIGH records |
|
||||
|
||||
Each row was scanned separately for **linux/amd64 and linux/arm64**, with the
|
||||
same counts on both. Counts are package-vulnerability records, not distinct
|
||||
CVEs or proven exploits. The [machine-readable evidence](runtime-image-candidates-2026-09-08.json)
|
||||
contains exact index, platform-manifest, config and report digests, inventory
|
||||
counts, scanner bounds and decisions. It is audit data, not an accepted release
|
||||
manifest or an installer input.
|
||||
|
||||
nginx's candidate reference is
|
||||
`docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce`.
|
||||
It inventories 70 Alpine packages including nginx/NJS, uses UID 101 and exposes
|
||||
8080. The [upstream stable release](https://nginx.org/en/download.html) and
|
||||
[security advisories](https://nginx.org/en/security_advisories.html) include the
|
||||
1.30.4 fixes. The publisher retains its
|
||||
[unprivileged port and temporary-path contract](https://github.com/nginx/docker-nginx-unprivileged).
|
||||
For the website, this changes nginx 1.27.5 to 1.30.4, NJS 0.8.10 to 1.0.1 and
|
||||
Alpine 3.21.3 to 3.24.1. These changes are explicit review items; the website
|
||||
Dockerfile has not been changed. The GovOPlaN Web image still requires an
|
||||
explicit verified `NGINX_IMAGE` build argument.
|
||||
|
||||
Caddy's candidate reference is
|
||||
`docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648`.
|
||||
Although this is the current
|
||||
[official image line](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy),
|
||||
its inventory still includes Go 1.26.3, `x/crypto` 0.52.0, `x/net` 0.55.0,
|
||||
`x/text` 0.37.0 and gRPC 1.81.0. Recorded fixes include Go 1.26.6,
|
||||
`x/crypto` 0.55.0, `x/net` 0.56.0, `x/text` 0.39.0 and gRPC 1.83.1; Alpine
|
||||
findings also remain in c-ares, curl/libcurl and OpenSSL. The CRITICAL
|
||||
`CVE-2026-56854` concerns `x/crypto/ssh` source-address enforcement. A module
|
||||
record alone does not establish that this binary exposes that SSH path; exact
|
||||
binary symbol/reachability analysis is still required for a disposition.
|
||||
|
||||
The [official Node image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/node)
|
||||
still maps Node 22 Alpine to 22.23.2 and the previously scanned digest. Node 24's
|
||||
candidate is
|
||||
`docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf`.
|
||||
Its HIGH records remain in two OpenSSL packages and npm dependencies
|
||||
`brace-expansion`, `ip-address` and `tar`; fixing the earlier critical tar
|
||||
record alone is insufficient. The website builder stays on Node 22 pending
|
||||
a reviewed build-tool remedy and a final builder scan.
|
||||
|
||||
## Method, verification and retained evidence
|
||||
|
||||
The existing Trivy 0.74.0 executable was rehashed against the previously verified
|
||||
archive member: `d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791`.
|
||||
The same schema-2 vulnerability database was used, updated
|
||||
`2026-09-07T19:06:01.154199452Z`. No tool installation or database refresh occurred.
|
||||
Index bytes matched both the registry digest header and Docker Hub publisher
|
||||
metadata; both platform-manifest byte hashes matched the index. All eight
|
||||
registry-only scans completed successfully with validated JSON, `--list-all-pkgs`,
|
||||
`--scanners vuln`, an eight-minute/2GB image bound, an empty Docker configuration
|
||||
and no inherited credentials. Exit zero means execution succeeded. Private
|
||||
temporary paths and in-memory artifact cache isolated this follow-up from the
|
||||
earlier scanner's artifact cache; its vulnerability database was read only.
|
||||
|
||||
Raw reports, logs, manifests, publisher metadata and the scanner script are in
|
||||
`/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh/`:
|
||||
|
||||
- `summary.json` SHA-256: `0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56`.
|
||||
- `frozen-images.json` SHA-256: `d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467`.
|
||||
- `scan_successors.py` SHA-256: `f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07`.
|
||||
|
||||
Local retention is not permanent artifact hosting; preserve this evidence with
|
||||
the eventual reviewed release. The JSON evidence records compressed registry
|
||||
layer sizes; these are not expanded filesystem limits or final GovOPlaN sizes.
|
||||
|
||||
Installer regression checks cover the new generated image pin, legacy
|
||||
specification fallback, preserved explicit images, generated topology and
|
||||
configuration: `python -I -m unittest discover -s tests -p
|
||||
test_deployment_installer.py` ran 45 tests successfully with one skip because
|
||||
Core was not importable in that isolated test environment. The skipped Core
|
||||
startup-configuration integration was subsequently rerun in the shared development
|
||||
environment with Core available: all 45 installer tests passed with no skips,
|
||||
including generated-environment startup validation. This is configuration
|
||||
validation, not execution of the candidate image.
|
||||
Both repositories passed `git diff --check`; the audit JSON and all eight
|
||||
report hashes were checked against the retained evidence.
|
||||
**Docker, Podman and HAProxy executables are unavailable on
|
||||
this host**, so no image or HAProxy configuration was executed and no daemon was
|
||||
installed. Publisher metadata and installer tests support the scoped source
|
||||
patch; they do not establish binary or deployed compatibility.
|
||||
|
||||
## Gates that remain open
|
||||
|
||||
- Validate `haproxy -c` on generated local, existing-proxy and managed-ingress
|
||||
configurations using the exact pinned image and target architectures. Run
|
||||
bounded isolated checks without live mounts, secrets, privilege or external
|
||||
network access. Then verify DNS discovery, readiness, forwarded headers,
|
||||
replica routing and graceful termination in the intended runtime.
|
||||
- Test the nginx candidate with both the website configuration and GovOPlaN
|
||||
WebUI entrypoint/proxy configuration, including UID 101, writable temporary
|
||||
paths, health paths, cache headers and static catalog bytes. Approve the
|
||||
website nginx/NJS/Alpine version changes before changing its Dockerfile.
|
||||
- Resolve Caddy, Node build-tool and all unchanged baseline dependencies with
|
||||
updated publisher images or narrow reviewed applicability evidence. No
|
||||
severity-wide exceptions or custom third-party rebuilds were introduced.
|
||||
- Close the original source-built/static inventory gaps. HAProxy's 24-package
|
||||
OS inventory still omits the source-built HAProxy executable. Node's npm
|
||||
inventory still omits the Node executable/stdlib. Garage, CPython, Redis,
|
||||
PostgreSQL and QEMU gaps are unchanged. Alpine 3.24 EOL metadata is still
|
||||
missing from this scanner; zero findings is not complete coverage.
|
||||
- Scan **final built** API/Web/website layers and the selected managed
|
||||
dependencies on both architectures, then perform migration, worker,
|
||||
readiness and ingress smoke checks. Record failure and unknown states.
|
||||
Secrets, misconfiguration and image signature policy need separate checks.
|
||||
- Obtain the website deployment host/operator and rebuild/restart authority,
|
||||
preserving the exact immutable catalog/keyring/module-directory bytes and
|
||||
verifying fresh public responses after an authorized rollout.
|
||||
|
||||
No images were built, executed, published or deployed; no running service,
|
||||
release tag, signed manifest, CI image input or live infrastructure was changed.
|
||||
@@ -0,0 +1,219 @@
|
||||
# Security and performance follow-up — 8 September 2026
|
||||
|
||||
This follows the [original review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
and its post-release issue reconciliation. It describes new source work after
|
||||
the frozen 0.1.45 release; it does not change published tags, packages, signed
|
||||
catalogs or deployed images. Gitea remains the canonical state log.
|
||||
|
||||
## Implemented source slices
|
||||
|
||||
- [Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
|
||||
a shared disposable-process runner enforces wall/CPU/address-space/input/output
|
||||
limits, bounded stderr, process-group cleanup and non-queuing per-process
|
||||
admission. A private binary codec bounds decoding before allocating a full
|
||||
object graph and preserves explicitly supported data types without pickle.
|
||||
Read the owning Core `docs/BOUNDED_PROCESS_CONTRACT.md` before adding callers.
|
||||
- Connectors XLSX parsing, Templates rendering, Files ZIP/TAR inspection and
|
||||
extraction, and Dataflow reference previews/development execution now use
|
||||
that boundary. Existing authorization, sessions, provider credentials,
|
||||
idempotency and persistence remain in the parent. No unprotected inline
|
||||
fallback is used. Each module contributes static EN/DE user/admin limits and
|
||||
operational consequences through its manifest.
|
||||
- Files snapshots authorized sources inside shared admission, validates private
|
||||
staged members, and acknowledges each persisted member before decoding the
|
||||
next. Numeric progress remains available. The acknowledgement is event-driven,
|
||||
not a fixed sleep per member. Reads allocate by validated actual file size,
|
||||
not by the configured ceiling. Failures reap children, clear private staging
|
||||
and retain the existing transaction/blob cleanup and explicit retry behavior.
|
||||
- Dataflow's normal reference preview formerly bypassed the backend wrapper;
|
||||
it now enters the worker too. Nested source configurations cannot collide
|
||||
merely because subflows reuse node IDs. Combined reference-source data is
|
||||
checked before creating further columnar copies, while individual providers
|
||||
retain their own authorized-read bounds. Staging/production still require
|
||||
DuckDB; this change does not replace that separate backend.
|
||||
- [Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
|
||||
current-password change, session/CSRF rotation, cross-tenant session and human
|
||||
API-key revocation, and optional administrator-assisted recovery. Recovery
|
||||
codes are hashed, single-use, expire after 15 minutes, require a current local
|
||||
System owner and explicit identity verification, and recheck current account,
|
||||
membership, tenant and issuer authority at redemption. A password change also
|
||||
invalidates outstanding codes issued by that account for other people. Audit
|
||||
evidence and validation/error responses do not contain passwords or codes.
|
||||
External-provider and service-account rules remain separate.
|
||||
- The Access UI provides first-login/required change, self-service change,
|
||||
policy-aware sign-in help, public code redemption and eligible owner issuance.
|
||||
Core consumes an optional lazy auth-action capability rather than importing
|
||||
Access internals. The required-action gate fails closed if its UI is missing.
|
||||
- [Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
|
||||
full-history lists batch pinned revisions, while new summary and bounded
|
||||
step/event endpoints preserve authorization and explicit pagination. Existing
|
||||
full-history responses are not silently truncated. Exact inbox total semantics
|
||||
are retained and their counting cost is documented.
|
||||
- [Meta #55](https://git.add-ideas.de/GovOPlaN/govoplan/issues/55):
|
||||
shared version/planning helpers recognize the existing nested developer
|
||||
package, not invented root metadata. All tag batches enforce trusted private
|
||||
source ownership, registered origins and clean main/upstream state. Meta
|
||||
batches additionally require exact composition and matching Core evidence.
|
||||
Whole-batch preflight,
|
||||
frozen source receipts, annotated immutable tags, object-pinned atomic
|
||||
publication and post-effect remote checks are covered with temporary local
|
||||
repositories. Hidden Git index flags, unsafe ancestry, alternates and changed
|
||||
Git-directory identities are rejected. Selected version/composition metadata
|
||||
must be tracked, so ignored files cannot describe bytes absent from a tag.
|
||||
Applicable unselected Core WebUI inputs have bounded, frozen read receipts;
|
||||
backend-only releases do not read them. The existing local module-candidate
|
||||
exception remains intact. The weaker legacy mutation path was removed.
|
||||
Canonical whole-package preview and receipt-bound apply now cover Meta's
|
||||
version preparation too. Core must already match the target. Preparation
|
||||
requires a separate trusted checkout, explicit out-of-run confirmation and
|
||||
unchanged source/tooling receipts; it cannot rewrite the running operator.
|
||||
Plans place Meta after Core and explain the manual preparation/publication
|
||||
steps instead of promising a durable self-update. Ambiguous partial writes
|
||||
require reconciliation, without automatic rollback or retry.
|
||||
- [Runtime-image follow-up](RUNTIME_IMAGE_REMEDIATION_2026-09-08.md): eight new
|
||||
registry-only scans cover four exact candidates on amd64 and arm64. New
|
||||
installer specifications select the patched same-line HAProxy digest;
|
||||
existing specifications retain their explicit image. Other candidates and
|
||||
unresolved inventory/deployment gates remain visible, not blanket-approved.
|
||||
|
||||
## Verification record
|
||||
|
||||
Targeted checks include actual child execution, catastrophic regex CPU,
|
||||
aggregate memory exhaustion, TAR extension metadata, noisy output, malformed
|
||||
transport/staging data, Unicode allocation limits, cancellation/callback
|
||||
failures, descendant cleanup, rollback and explicit retries. Local mixed-owner
|
||||
composition tests completed nine real children, rejected six overlapping
|
||||
requests as busy, observed at most one unreaped child and recovered all slots.
|
||||
This is local admission evidence, not a target deployment load certification.
|
||||
|
||||
Workflow fixtures serialize 40 different pinned revisions with five SQL reads;
|
||||
summary lists use one query for 40 ordinary rows. Exact inbox totals for
|
||||
40/400/4,000 candidates used one query, with measured local costs approximately
|
||||
0.011/0.057/0.492 seconds. These are fixture measurements, not production SLOs.
|
||||
|
||||
The broader Core API smoke suite exposed three stale campaign assertions.
|
||||
All three failures were reproduced against the unchanged private frozen 0.1.45
|
||||
sources. Updated fixtures verify recipient-summary projection, detailed payload
|
||||
separation and explicit fenced recovery of a confirmed stopped runtime; observing
|
||||
SENDING alone must not make a claim recoverable. All 76 smoke tests then passed.
|
||||
No production Campaign behavior was changed to satisfy these tests.
|
||||
|
||||
The final release-tool suite passed 279 tests and 68 subtests, including
|
||||
temporary local remotes and adversarial source/tag/receipt changes. Rechecking
|
||||
the whole batch before effects is deliberately conservative: its repeated
|
||||
filesystem/Git/remote work grows quadratically with batch size. It is not a
|
||||
new unattended publication path or permission to execute unreviewed source.
|
||||
|
||||
Strict interface inventory now reports no unclassified endpoints and exact
|
||||
contextual help for all 133 high-risk controls. Seventeen password browser cases
|
||||
include actual F1 help from the restricted screen, empty workspace scopes,
|
||||
EN/DE layouts, Unicode boundaries and no credential values in help URLs.
|
||||
The initial production bundle remains within the unchanged limits (512,036
|
||||
raw bytes and 162,415 gzip bytes; 1,713 gzip bytes below its ceiling), with
|
||||
46 optional descriptors and no eager optional-module imports.
|
||||
|
||||
The focused checker now includes the new Core process, mixed-owner admission,
|
||||
Access password, Templates and Files worker tests, the repaired campaign smoke
|
||||
cases, and browser-side auth/password transport contracts. The full focused run
|
||||
passed, including 63 production module/build permutations and all 230 browser
|
||||
cases. Its two opt-in Datasources PostgreSQL cases were skipped in that run
|
||||
and subsequently passed against the isolated real database described below.
|
||||
The final Meta preparation gate was added after that full run and verified
|
||||
with the owning release-tool suite and the focused release-gate command.
|
||||
Manifest validation passed for all 72 modules. The full focused log is
|
||||
`/mnt/DATA/tmp/govoplan-security-followup-20260908-focused.log`.
|
||||
|
||||
The first follow-up quick audit captured an unchanged 79-repository snapshot
|
||||
in `/mnt/DATA/tmp/govoplan-security-followup-quick-20260908-7s8Sgh/`.
|
||||
All four required scanners completed, with zero missing/execution reports;
|
||||
all 168 report checksums and 163 machine-readable reports were validated.
|
||||
Gitleaks found no secrets in all 79 histories and 79 worktrees. Local Semgrep
|
||||
rules reported zero findings. Production Bandit reported 65 low and four medium
|
||||
warnings, and production Ruff retained 54 warnings. The two added Bandit
|
||||
warnings identify the new Core subprocess import and invocation: trusted
|
||||
server-owned arguments, no shell, and the documented resource/process boundary
|
||||
were reviewed; warnings remain visible. This is report-only evidence, not a
|
||||
warning-free audit or a penetration test. A final snapshot follows the
|
||||
cross-module declaration/contextual-help corrections and release-tool checks.
|
||||
|
||||
That final audit completed on 8 September, 05:59:46–06:02:18 UTC, in
|
||||
`/mnt/DATA/tmp/govoplan-security-final-quick-20260908-vAwQIh/`. All 79 start/end
|
||||
source fingerprints were identical; all four scanners completed, all 168
|
||||
registered report checksums matched, and all 163 JSON/SARIF reports parsed.
|
||||
There were no missing reports or scanner execution errors. Semgrep and both
|
||||
Gitleaks scopes again reported zero findings. Production counts were unchanged
|
||||
from the first follow-up: Bandit 65 low/four medium and Ruff 54. Test-only
|
||||
counts were Bandit 140 low/34 medium and Ruff 136. A separate frozen scan of
|
||||
all ten changed Meta release/deployment Python files reported seven low Bandit
|
||||
and four Ruff S603 warnings, with no execution errors. Its four argv-only
|
||||
subprocess sites were reviewed; the preparation additions introduced no new
|
||||
warnings. No findings were hidden or severity-wide exceptions added.
|
||||
The audit manifest SHA-256 is
|
||||
`a997b786239cd11443cb665d5f9041a968cc38f9d49171e68bb868bf2bd73310`;
|
||||
its report-checksum list SHA-256 is
|
||||
`dc590ca5b0a4e445019a05536d410226088d67b40d61cd7657bdef4a4eae56d8`.
|
||||
|
||||
The audit includes the eight committed feature/website source changes and the
|
||||
final uncommitted Meta source. Only this evidence document was updated after
|
||||
the source freeze ended; the final Meta commit and remote publication are
|
||||
recorded in the linked Gitea issues, not inferred from local audit completion.
|
||||
|
||||
Fresh dependency audits are retained in
|
||||
`/mnt/DATA/tmp/govoplan-dependency-final-20260908-d24LEK/`: all four full npm
|
||||
lockfile audits (Core WebUI, Mail root/WebUI and website) report zero known
|
||||
vulnerabilities. Installed Python auditing covers 137 distributions with zero
|
||||
known vulnerabilities; 51 local GovOPlaN distributions lack PyPI advisory
|
||||
coverage. Core's 46 linked packages are likewise not claimed covered by public
|
||||
registry advisories. All 12 dependency-file hashes and the installed inventory
|
||||
were unchanged. No packages were installed or automatically fixed.
|
||||
|
||||
Managed PostgreSQL 16.15 fixtures used private Unix sockets, synthetic roles
|
||||
and databases, no TCP listener, per-case schemas and bounded SQL/lock waits.
|
||||
Both previously skipped Datasources races passed. Twenty-one existing Access
|
||||
password HTTP tests and four additional races passed on PostgreSQL: single-use
|
||||
redemption, stale-session/password replacement, competing issuance, and issuer
|
||||
password revocation during redemption. Four release/development migration checks
|
||||
also passed for Access and Workflow, including credential preservation and
|
||||
idempotent indexes. The four races are now owning opt-in Access regressions;
|
||||
see `govoplan-access/docs/PASSWORD_RECOVERY_POSTGRES_TESTS.md`. These local
|
||||
database checks do not certify a deployment, fleet load or external recovery
|
||||
handover. With both explicit PostgreSQL test URLs enabled, the full Access suite
|
||||
passed 122 tests and 18 subtests, and the full Datasources suite passed 57 tests,
|
||||
without skips. Access retained 12 existing SQLite datetime-adapter warnings in
|
||||
its separate SQLite migration cases. Both temporary PostgreSQL fixtures were
|
||||
stopped and independently verified: no server process, private socket,
|
||||
generated schema or synthetic cluster remains. Scripts, logs and shutdown
|
||||
receipts are retained under `/home/zemion/.cache/govoplan-pg-security-20260908.RAUitg/`
|
||||
and `/home/zemion/.cache/govoplan-pg-promoted-20260908.JZcIKL/`.
|
||||
|
||||
## Adoption and remaining gates
|
||||
|
||||
1. `AUTH_LOCAL_PASSWORD_RECOVERY_ENABLED` remains **false** by default. The
|
||||
existing flag is still advisory until an operator explicitly adopts and
|
||||
enables the complete recovery policy. Confirm who verifies identity and how
|
||||
the one-time code is handed over; automated email recovery is not enabled.
|
||||
Test first-login, lost-password, code expiry and administrator availability
|
||||
in the target environment before enforcement.
|
||||
2. Access migration `e9a2c5f8b1d4` adds recovery evidence; Workflow migration
|
||||
`9e6b3f8a2c7d` adds summary-pagination indexes. Use normal backed-up upgrade
|
||||
procedures and account for index-build cost. No manual live migration or
|
||||
server restart was performed during this work. The user's existing devserver
|
||||
has automatic reload, so live schema state must not be assumed unchanged.
|
||||
3. Release preparation must assign new source/package versions and require a
|
||||
Core version containing the new worker/auth contracts in the affected module
|
||||
metadata, including matching WebUI assets. The old immutable release must
|
||||
not be relabelled or treated as containing these APIs.
|
||||
4. Resource limits are not an arbitrary-code, filesystem or network sandbox.
|
||||
Admission is per API/worker process, not fleet-wide. Validate Linux/cgroup
|
||||
memory, disk quotas, process counts, cancellation and legitimate large-file
|
||||
workloads on the intended runtime before increasing concurrency. Core #297
|
||||
retains this target-evidence follow-up.
|
||||
5. Meta #52 and website #9 retain runtime-image/publication/deployment holds.
|
||||
Docker/Podman/HAProxy executables are unavailable here. Final built images,
|
||||
binary/source inventories, ingress behavior, migration/readiness/worker
|
||||
smoke checks and the website's target/operator authority remain outstanding.
|
||||
Zero findings in a detected package inventory is not full image coverage.
|
||||
|
||||
No real messages, IMAP appends, password resets, provider operations or deployment
|
||||
actions were used as test fixtures. Development tests use temporary databases,
|
||||
private temporary files, mock transports and managed test-browser servers.
|
||||
@@ -174,3 +174,34 @@ permissions must be corrected rather than weakening the guard. Extreme sparse
|
||||
spreadsheets, overly deep/long archive paths and oversized padding intermediates
|
||||
can now fail early with diagnostics. No stored documents or configurations were
|
||||
deleted or silently migrated.
|
||||
|
||||
## Post-release follow-up — 2026-09-08
|
||||
|
||||
The findings and scanner counts above describe the original audit snapshot.
|
||||
The following source fixes are subsequent to the frozen `0.1.45` composition;
|
||||
they do not change its immutable tags or published package bytes.
|
||||
|
||||
- [Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2)
|
||||
now enforces the existing shared 2 MiB stdout/stderr limit during execution
|
||||
and kills/reaps the direct validator on overflow, timeout or cancellation.
|
||||
Report reads are bounded to 16 MiB plus one probe byte before interpretation.
|
||||
The 30-test module suite passes; noisy-child and report-read regressions were
|
||||
also demonstrated to fail against the previous source. Owning EN/DE static
|
||||
documentation is updated. POSIX pipe capture is required; disk quotas,
|
||||
descendant isolation and process-level CPU/memory limits remain separate work.
|
||||
- [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54) now preserves
|
||||
the last command's failure status after exhausted installer retries. Twelve
|
||||
isolated stage/scenario combinations cover every retry call site, success,
|
||||
backoff and caller termination under `set -e`. The test is included in the
|
||||
focused checks and installer CI. See the bilingual
|
||||
[installer retry note](../operations/WEBUI_RELEASE_DEPENDENCY_RETRIES.md).
|
||||
|
||||
These are unreleased follow-up source changes, not a new runtime release or
|
||||
deployment. The runtime-image hold under Meta #52 remains in force; the
|
||||
historical peer-dependency workaround still needs its separate review.
|
||||
|
||||
Further implementation and adoption gates are tracked in the
|
||||
[security follow-up](SECURITY_FOLLOWUP_2026-09-08.md), including disposable
|
||||
parsing/execution workers, opt-in password recovery, workflow read projections
|
||||
and the newer runtime-image evidence. The original scanner counts above remain
|
||||
historical and are not silently replaced by later test results.
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"purpose": "Audit evidence only; not a release manifest or active installer configuration.",
|
||||
"observed_at": "2026-09-08T03:56:47.666808+00:00",
|
||||
"runtime_publication_held": true,
|
||||
"website_deployment_held": true,
|
||||
"scan_execution_complete": true,
|
||||
"coverage_complete": false,
|
||||
"scanner": {
|
||||
"name": "Trivy",
|
||||
"version": "0.74.0",
|
||||
"binary_sha256": "d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791",
|
||||
"database_metadata": {
|
||||
"Version": 2,
|
||||
"NextUpdate": "2026-09-08T19:06:01.154199291Z",
|
||||
"UpdatedAt": "2026-09-07T19:06:01.154199452Z",
|
||||
"DownloadedAt": "2026-09-07T23:30:53.198039224Z"
|
||||
},
|
||||
"source": "remote",
|
||||
"scanners": [
|
||||
"vuln"
|
||||
],
|
||||
"list_all_packages": true,
|
||||
"images_executed": false,
|
||||
"existing_docker_credentials_used": false,
|
||||
"timeout": "8m",
|
||||
"maximum_image_size": "2GB"
|
||||
},
|
||||
"evidence": {
|
||||
"private_directory": "/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh",
|
||||
"summary_sha256": "0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56",
|
||||
"frozen_images_sha256": "d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467",
|
||||
"scanner_script_sha256": "f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07"
|
||||
},
|
||||
"candidates": [
|
||||
{
|
||||
"name": "haproxy",
|
||||
"image": "docker.io/library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
|
||||
"disposition": "source_default_updated_binary_runtime_verification_pending",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:0666a2c2f41d341084ed2da85392b48cdcd766adfa28231f31305724ed5c6ea5",
|
||||
"config_digest": "sha256:9621d75e50a8f26d3738ae3cdbf15e98c6aa5cd48e6baca0699492de502156f5",
|
||||
"compressed_layer_bytes": 20516844,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 24
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "40cfc3db74e29f09723f38698660ccdd50ac935c8d6e1e75c6e0555b3e9361fb",
|
||||
"log_sha256": "46881f695780f89c037d5b0b4dc0ded9ea4e459077c7658d80b786efc5754084"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:cd20b9dc6b4713956a2a043997001a1948167d345e7c8d5bd5ff2e667166651f",
|
||||
"config_digest": "sha256:19796bff8905d4a46c9463c576203b20cac8984d41b7b01ea9cbff55e8422c34",
|
||||
"compressed_layer_bytes": 20970772,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 24
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "0e1326c58abf358d592fa55c94333228ce1094edf4e489fcc4ece6e32d3320f6",
|
||||
"log_sha256": "397c2fbf1044cf50733d68b4b9cc4eb68211d96f1f302d5e9f8af0d11fb0de1c"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "nginx-stable",
|
||||
"image": "docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce",
|
||||
"disposition": "candidate_pending_compatibility",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:b8c179cd3c2ae222a873dd59fbae240fadc03836cae5198afc9e9c19919c3880",
|
||||
"config_digest": "sha256:8b5953dae38d27a76bca22373bb920fd6ce8d9d7da21578d2926e678002de8a0",
|
||||
"compressed_layer_bytes": 25526590,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 70
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "3ad164dae3cdf891b12e41451b8a8e65a5e1688824a7c01d848e1274363e6bf9",
|
||||
"log_sha256": "0f99ff3d9b4396de48655bf8299df30c14ba0c579f480d37baa7d2f6a4c11f1d"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:b6742a0cbd749add25346658991c3da06a8e38796949df120fed78db8c512576",
|
||||
"config_digest": "sha256:4d8b10f5d2ff99e7aa5f161930d8a4693a86a26f288ec8c0d4cd47f2ef5af179",
|
||||
"compressed_layer_bytes": 25892370,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 70
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "506fdeb97525ed8e4d9ae538c42bab7aa2217f662a7135f0f12d16d20410c7a6",
|
||||
"log_sha256": "c41ef9ea091d00f18c7a097404672591bee85e7477ae17d8f5ca771d8e42b2bb"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "caddy",
|
||||
"image": "docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648",
|
||||
"disposition": "not_selected_remaining_fixable_findings",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:98eb57d882ccd5213d1688764db10c1ca2c58a1ca3a6717a3411ad798f7a423a",
|
||||
"config_digest": "sha256:af555904a0961945f16bb323a501457b13a4f7e9bde969b145b97da80b38ecbe",
|
||||
"compressed_layer_bytes": 23907283,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.23.5"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 32
|
||||
},
|
||||
{
|
||||
"type": "gobinary",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 1,
|
||||
"HIGH": 38,
|
||||
"MEDIUM": 41,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 23
|
||||
},
|
||||
"fixable_high_critical": 39,
|
||||
"unique_cves": 68,
|
||||
"report_sha256": "e483352a1d5b9b97950dcf92e4dfcf4600f5b09306af3d0640b10ca229a07779",
|
||||
"log_sha256": "9cd599d6dd8c101b421fdeb57036cec1f69f815d765a8bf1f850ec60061036f4"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:1172d4213087d3fc30bafc7ff2c2896180eb0c41ff7f75f315568fb36cabdcba",
|
||||
"config_digest": "sha256:6b08c1b9858ca9a7d99c1da13c3695081e0e604c6cf214ca26a7ce0e2c4fd9b4",
|
||||
"compressed_layer_bytes": 22722712,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.23.5"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 32
|
||||
},
|
||||
{
|
||||
"type": "gobinary",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 1,
|
||||
"HIGH": 38,
|
||||
"MEDIUM": 41,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 23
|
||||
},
|
||||
"fixable_high_critical": 39,
|
||||
"unique_cves": 68,
|
||||
"report_sha256": "20e04d820e3b27d57575ea177e523e23109fd83344cdf57e184a4d2fad27e803",
|
||||
"log_sha256": "a1d9c37aa7948c137db64040d95e5930c5859a8acd71ac5bc41ff168e2b270c5"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "node-lts",
|
||||
"image": "docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf",
|
||||
"disposition": "not_selected_remaining_fixable_findings",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:4caaaf42195bcd6f6f3559a413b20cb8f8ad089e231ee874cf7701643966689f",
|
||||
"config_digest": "sha256:ee289c69ed1ac50a5a042112ea97f132800e2dd53e832da27784f00e45b3289c",
|
||||
"compressed_layer_bytes": 58486244,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 18
|
||||
},
|
||||
{
|
||||
"type": "node-pkg",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 6,
|
||||
"MEDIUM": 11,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 6,
|
||||
"unique_cves": 19,
|
||||
"report_sha256": "c927d995dde700c92027f6328dc6c273f0f1445cd8154301480757cb962e02b2",
|
||||
"log_sha256": "c7dc1900cbe39c9f91e228b2770bcbd394ec2914d2b3879478295fc7f8f77ab3"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:d3724e44ee368606d753e0027eb8d2a94fc1f275e5d9e4620178a12edb655f5f",
|
||||
"config_digest": "sha256:722cc1507731edf58a4c0bc3e29553c44ce5774d0849242c1b237abc79926a0a",
|
||||
"compressed_layer_bytes": 58935654,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 18
|
||||
},
|
||||
{
|
||||
"type": "node-pkg",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 6,
|
||||
"MEDIUM": 11,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 6,
|
||||
"unique_cves": 19,
|
||||
"report_sha256": "e5f7799761f23cefc36929381b4186eeb3afb46a2a559c789d12a7eea5dc30d0",
|
||||
"log_sha256": "fd24671f0da4d3b20dc8bcc2718531e6f6e19155d49bed15958965e21dd10813"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -4,44 +4,44 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "govoplan"
|
||||
version = "0.1.45"
|
||||
version = "0.1.46"
|
||||
description = "Developer convenience package for a versioned GovOPlaN composition"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "AGPL-3.0-or-later" }
|
||||
dependencies = [
|
||||
"govoplan-core[server]==0.1.45",
|
||||
"govoplan-core[server]==0.1.46",
|
||||
"govoplan-tenancy==0.1.22",
|
||||
"govoplan-organizations==0.1.21",
|
||||
"govoplan-identity==0.1.21",
|
||||
"govoplan-idm==0.1.25",
|
||||
"govoplan-idm==0.1.26",
|
||||
"govoplan-access==0.1.25",
|
||||
"govoplan-admin==0.1.23",
|
||||
"govoplan-policy==0.1.23",
|
||||
"govoplan-audit==0.1.20",
|
||||
"govoplan-dashboard==0.1.20",
|
||||
"govoplan-files==0.1.26",
|
||||
"govoplan-mail==0.1.27",
|
||||
"govoplan-campaign==0.1.28",
|
||||
"govoplan-calendar==0.1.23",
|
||||
"govoplan-files==0.1.27",
|
||||
"govoplan-mail==0.1.28",
|
||||
"govoplan-campaign==0.1.29",
|
||||
"govoplan-calendar==0.1.24",
|
||||
"govoplan-docs==0.1.23",
|
||||
"govoplan-ops==0.1.22",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
full = [
|
||||
"govoplan-addresses==0.1.22",
|
||||
"govoplan-addresses==0.1.23",
|
||||
"govoplan-approvals==0.1.21",
|
||||
"govoplan-assets==0.1.20",
|
||||
"govoplan-booking==0.1.20",
|
||||
"govoplan-cases==0.1.24",
|
||||
"govoplan-cases==0.1.25",
|
||||
"govoplan-certificates==0.1.20",
|
||||
"govoplan-committee==0.1.21",
|
||||
"govoplan-connectors==0.1.26",
|
||||
"govoplan-committee==0.1.22",
|
||||
"govoplan-connectors==0.1.27",
|
||||
"govoplan-consultation==0.1.20",
|
||||
"govoplan-contracts==0.1.20",
|
||||
"govoplan-dataflow==0.1.24",
|
||||
"govoplan-datasources==0.1.25",
|
||||
"govoplan-dataflow==0.1.25",
|
||||
"govoplan-datasources==0.1.26",
|
||||
"govoplan-decisions==0.1.19",
|
||||
"govoplan-dist-lists==0.1.21",
|
||||
"govoplan-dms==0.1.20",
|
||||
@@ -51,7 +51,7 @@ full = [
|
||||
"govoplan-facilities==0.1.20",
|
||||
"govoplan-fit-connect==0.1.20",
|
||||
"govoplan-forms==0.1.23",
|
||||
"govoplan-forms-runtime==0.1.21",
|
||||
"govoplan-forms-runtime==0.1.22",
|
||||
"govoplan-grants==0.1.20",
|
||||
"govoplan-helpdesk==0.1.21",
|
||||
"govoplan-identity-trust==0.1.21",
|
||||
@@ -69,7 +69,7 @@ full = [
|
||||
"govoplan-projects==0.1.20",
|
||||
"govoplan-quick-access==0.1.21",
|
||||
"govoplan-records==0.1.24",
|
||||
"govoplan-reporting==0.1.21",
|
||||
"govoplan-reporting==0.1.22",
|
||||
"govoplan-resources==0.1.20",
|
||||
"govoplan-rest==0.1.19",
|
||||
"govoplan-risk-compliance==0.1.21",
|
||||
@@ -79,7 +79,7 @@ full = [
|
||||
"govoplan-soap==0.1.19",
|
||||
"govoplan-tasks==0.1.23",
|
||||
"govoplan-templates==0.1.22",
|
||||
"govoplan-tickets==0.1.22",
|
||||
"govoplan-tickets==0.1.23",
|
||||
"govoplan-transparency==0.1.20",
|
||||
"govoplan-views==0.1.22",
|
||||
"govoplan-voting==0.1.21",
|
||||
|
||||
@@ -4,15 +4,15 @@
|
||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
|
||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
|
||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.25
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.26
|
||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
|
||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
|
||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
|
||||
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
|
||||
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.26
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.27
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.28
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.23
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.27
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.28
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.29
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.24
|
||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
|
||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
|
||||
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { test } from "node:test";
|
||||
import { auditRepository } from "../tools/devkit/audit-display-labels.mjs";
|
||||
|
||||
const require = createRequire(resolve(import.meta.dirname, "../../govoplan-core/webui/package.json"));
|
||||
const ts = require("typescript");
|
||||
function fixture(fn) {
|
||||
const root = mkdtempSync(join(tmpdir(), "govoplan-label-audit-"));
|
||||
const write = (relative, content) => { const target = join(root, relative); mkdirSync(dirname(target), { recursive: true }); writeFileSync(target, content); };
|
||||
try {
|
||||
write("core/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Shared: "Shared" }, de: { Shared: "Gemeinsam" } };');
|
||||
return fn(root, write);
|
||||
} finally { rmSync(root, { recursive: true, force: true }); }
|
||||
}
|
||||
|
||||
test("plain labels resolve owning registered catalogs, aliases, constants and Core defaults", () => fixture((root, write) => {
|
||||
write("module/webui/src/i18n/generatedTranslations.ts", 'const en = { Projects: "Projects" }; const de = { Projects: "Projekte" }; export const generatedTranslations = { en, de };');
|
||||
write("module/webui/src/module.ts", 'import { generatedTranslations as words } from "./i18n/generatedTranslations"; export const exampleModule: PlatformWebModule = { translations: words };');
|
||||
write("module/webui/src/Page.tsx", 'import { PageLayout as Layout, PageTitle } from "@govoplan/core-webui"; const title = "Projects"; export const page = <><Layout title={title} /><PageTitle>Shared</PageTitle></>;');
|
||||
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
|
||||
assert.equal(result.registration, "registered");
|
||||
assert.equal(result.labels.length, 2);
|
||||
assert.deepEqual(result.findings, []);
|
||||
}));
|
||||
|
||||
test("a catalog file without module registration cannot mask untranslated titles", () => fixture((root, write) => {
|
||||
write("module/webui/src/i18n/generatedTranslations.ts", 'export const generatedTranslations = { en: { Projects: "Projects" }, de: { Projects: "Projekte" } };');
|
||||
write("module/webui/src/module.ts", 'export const exampleModule = { id: "example" };');
|
||||
write("module/webui/src/Page.tsx", 'import * as Core from "@govoplan/core-webui"; export const page = <Core.PageLayout title="Projects" />;');
|
||||
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
|
||||
assert(result.findings.some((item) => item.code === "catalog-not-registered"));
|
||||
assert.deepEqual(result.labels[0].missing_locales, ["en", "de"]);
|
||||
}));
|
||||
|
||||
test("only known static display slots fail; markers and runtime data remain separate", () => fixture((root, write) => {
|
||||
write("module/webui/src/Page.tsx", 'import { PageLayout } from "@govoplan/core-webui"; export const page = <><PageLayout title="Untranslated" /><PageLayout title={campaign.name} /><PageLayout title="i18n:known.key" /><CustomThing title="Not a known slot" /></>;');
|
||||
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
|
||||
assert.equal(result.findings.length, 1);
|
||||
assert.equal(result.findings[0].text, "Untranslated");
|
||||
assert.equal(result.review.length, 1);
|
||||
assert.equal(result.review[0].code, "dynamic-display-slot");
|
||||
}));
|
||||
|
||||
test("dynamic translation registration is reported for review, not falsely missing", () => fixture((root, write) => {
|
||||
write("module/webui/src/module.ts", 'export const exampleModule: PlatformWebModule = { translations: configuredCatalog() };');
|
||||
write("module/webui/src/Page.tsx", 'export const page = <PageTitle>Example</PageTitle>;');
|
||||
const result = auditRepository(ts, join(root, "module"), join(root, "core"));
|
||||
assert.equal(result.registration, "dynamic");
|
||||
assert.deepEqual(result.findings, []);
|
||||
assert(result.review.some((item) => item.code === "dynamic-catalog-review"));
|
||||
}));
|
||||
Executable
+64
@@ -0,0 +1,64 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { findDetachedDocumentation } from "../tools/checks/check-heading-help.mjs";
|
||||
import "./test-heading-translations.mjs";
|
||||
|
||||
const inspect = (source) => findDetachedDocumentation([{ path: "/fixture/Page.tsx", source }]).findings;
|
||||
const book = '<DocumentationHelpLink reference={topic} />';
|
||||
|
||||
test("heading and text contracts accept contextual help", () => {
|
||||
for (const component of ["PageHeader", "PageLayout", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]) {
|
||||
assert.equal(inspect(`const Page=()=> <${component} title="Topic" titleHelp={${book}} />`).length, 0, component);
|
||||
}
|
||||
assert.equal(inspect(`const Page=()=> <PageTitle titleHelp={${book}}>Topic</PageTitle>`).length, 0);
|
||||
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}>Existing label</TextWithHelp>`).length, 0);
|
||||
});
|
||||
|
||||
test("detached action and body links are rejected", () => {
|
||||
for (const source of [`<PageActionBar helpAction={${book}} />`, `<Card title="Topic" actions={${book}} />`, `<div>${book}</div>`, `<Card title={${book}} />`]) {
|
||||
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
|
||||
}
|
||||
});
|
||||
|
||||
test("aliases, conditional help and simple local references remain checked", () => {
|
||||
assert.equal(inspect(`import {DocumentationHelpLink as Book, Card as Box} from '@govoplan/core-webui'; const Page=()=> <Box title="Topic" titleHelp={<Book reference={topic}/>} />`).length, 0);
|
||||
assert.equal(inspect(`import {DocumentationHelpLink as Book} from '@govoplan/core-webui'; const Page=()=> <div><Book reference={topic}/></div>`).length, 1);
|
||||
assert.equal(inspect(`const help=${book}; const Page=()=> <Card title="Topic" titleHelp={enabled ? help : null}/>`).length, 0);
|
||||
assert.equal(inspect(`const help=${book}; const Page=()=> <><Card title="Topic" titleHelp={help}/><PageActionBar helpAction={help}/></>`).length, 1);
|
||||
assert.equal(inspect(`const help=${book}; const again=help; const Page=()=> <Card title="Topic" titleHelp={again}/>`).length, 0);
|
||||
});
|
||||
|
||||
test("empty anchors, unknown contracts and nested interactive elements fail", () => {
|
||||
for (const source of [`<Card titleHelp={${book}}/>`, `<TextWithHelp help={${book}}/>`, `<TextWithHelp help={${book}}> </TextWithHelp>`, `<Unknown title="Topic" titleHelp={${book}}/>`, `<Card title="Topic" titleHelp={<button>${book}</button>}/>`]) {
|
||||
assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
|
||||
}
|
||||
});
|
||||
|
||||
test("namespace and explicit component default imports cannot bypass placement", () => {
|
||||
for (const source of [
|
||||
`import * as UI from '@govoplan/core-webui'; const Page=()=> <div><UI.DocumentationHelpLink reference={topic}/></div>`,
|
||||
`import Book from './components/help/DocumentationHelpLink'; const Page=()=> <div><Book reference={topic}/></div>`,
|
||||
]) assert.equal(inspect(source).length, 1, source);
|
||||
assert.equal(inspect(`import * as UI from '@govoplan/core-webui'; const Page=()=> <UI.Card title="Topic" titleHelp={<UI.DocumentationHelpLink reference={topic}/>}/>`).length, 0);
|
||||
assert.equal(inspect(`import Book from './components/help/DocumentationHelpLink'; import Label from './components/help/TextWithHelp'; const Page=()=> <Label help={<Book reference={topic}/>}>Topic</Label>`).length, 0);
|
||||
assert.equal(findDetachedDocumentation([{ path: "/fixture/Page.tsx", source: `import Book from './business/Book'; const Page=()=> <div><Book/></div>` }]).links, 0);
|
||||
});
|
||||
|
||||
test("outer interactive containers and statically absent text are rejected", () => {
|
||||
for (const source of [
|
||||
`<button><TextWithHelp help={${book}}>Topic</TextWithHelp></button>`,
|
||||
`<a href="/other"><Card title="Topic" titleHelp={${book}}/></a>`,
|
||||
`<Card title="" titleHelp={${book}}/>`,
|
||||
`<Card title={false} titleHelp={${book}}/>`,
|
||||
`<PageTitle titleHelp={${book}}>{null}</PageTitle>`,
|
||||
`<TextWithHelp help={${book}}>{/* Topic */}</TextWithHelp>`,
|
||||
`<TextWithHelp help={${book}}>{undefined}</TextWithHelp>`,
|
||||
`<TextWithHelp help={${book}}><span hidden>Topic</span></TextWithHelp>`,
|
||||
`<TextWithHelp hidden help={${book}}>Topic</TextWithHelp>`,
|
||||
]) assert.equal(inspect(`const Page=()=> ${source}`).length, 1, source);
|
||||
assert.equal(inspect(`const help=${book}; const Page=()=> <Button><TextWithHelp help={help}>Topic</TextWithHelp></Button>`).length, 1);
|
||||
assert.equal(inspect(`const title=null; const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 1);
|
||||
assert.equal(inspect(`const Page=()=> <Card title={translateText(title)} titleHelp={${book}}/>`).length, 0);
|
||||
assert.equal(inspect(`const Page=()=> <TextWithHelp help={${book}}><TranslatedTitle/></TextWithHelp>`).length, 0);
|
||||
assert.equal(inspect(`let title=""; title=translateText(key); const Page=()=> <Card title={title} titleHelp={${book}}/>`).length, 0);
|
||||
});
|
||||
Executable
+80
@@ -0,0 +1,80 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { resolve } from "node:path";
|
||||
import { test } from "node:test";
|
||||
|
||||
const root = resolve(import.meta.dirname, "../..");
|
||||
const require = createRequire(resolve(root, "govoplan-core/webui/package.json"));
|
||||
const { buildSync } = require("esbuild");
|
||||
const labels = {
|
||||
approvals: { "Approval requests": "Genehmigungsanträge" },
|
||||
forms: { "Form meaning": "Formularbedeutung" },
|
||||
"forms-runtime": { "Form instance": "Formularinstanz", "Forms runtime": "Formularlaufzeit" },
|
||||
helpdesk: { "Helpdesk profiles": "Helpdesk-Profile" },
|
||||
portal: { "Service directory": "Leistungsverzeichnis" },
|
||||
projects: { Projects: "Projekte" },
|
||||
reporting: { Reporting: "Reporting" },
|
||||
"risk-compliance": { "Risk Compliance": "Risiko und Compliance" },
|
||||
scheduling: { Scheduling: "Terminplanung" },
|
||||
tickets: { Tickets: "Tickets" },
|
||||
voting: { Ballots: "Abstimmungen" },
|
||||
wiki: { Wiki: "Wiki" },
|
||||
workflow: { Workflows: "Workflows", Workflow: "Workflow" },
|
||||
idm: {
|
||||
"Direct changes to this governed function are": "Direkte Änderungen an dieser gesteuerten Funktion sind",
|
||||
"emergency overrides": "Notfallübersteuerungen",
|
||||
". Use a request or grant above for the normal process.": ". Verwenden Sie für den regulären Prozess einen Antrag oder eine Vergabe.",
|
||||
},
|
||||
};
|
||||
|
||||
// Render the actual Core heading/locale contract with one owning module at a
|
||||
// time. The in-memory bundle creates no shared component-build artifacts.
|
||||
const names = Object.keys(labels);
|
||||
const imports = names.map((name, index) => `import { generatedTranslations as t${index} } from ${JSON.stringify(resolve(root, `govoplan-${name}/webui/src/i18n/generatedTranslations.ts`))};`).join("\n");
|
||||
const { outputFiles } = buildSync({
|
||||
stdin: {
|
||||
contents: `${imports}
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { PlatformLanguageProvider } from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/i18n/LanguageContext.tsx"))};
|
||||
import PageTitle from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/PageTitle.tsx"))};
|
||||
import DocumentationHelpLink from ${JSON.stringify(resolve(root, "govoplan-core/webui/src/components/help/DocumentationHelpLink.tsx"))};
|
||||
export const catalogs = [${names.map((_, index) => `t${index}`).join(",")}];
|
||||
export function heading(index, language, label) {
|
||||
return renderToStaticMarkup(<PlatformLanguageProvider preferredLanguageCode={language} moduleTranslations={[catalogs[index]]}>
|
||||
<PageTitle titleHelp={<DocumentationHelpLink reference={{ contextId: "heading-test" }} />}>{label}</PageTitle>
|
||||
</PlatformLanguageProvider>);
|
||||
}`,
|
||||
loader: "tsx",
|
||||
resolveDir: resolve(root, "govoplan-core/webui"),
|
||||
},
|
||||
bundle: true,
|
||||
write: false,
|
||||
platform: "node",
|
||||
format: "cjs",
|
||||
jsx: "automatic",
|
||||
external: ["react", "react-dom/server"],
|
||||
logLevel: "silent",
|
||||
});
|
||||
const compiled = { exports: {} };
|
||||
new Function("require", "module", "exports", outputFiles[0].text)(require, compiled, compiled.exports);
|
||||
const { catalogs, heading } = compiled.exports;
|
||||
|
||||
test("new contextual labels render from their owning EN/DE catalogues", async (context) => {
|
||||
for (const [index, name] of names.entries()) {
|
||||
await context.test(name, () => {
|
||||
const moduleSource = readFileSync(resolve(root, `govoplan-${name}/webui/src/module.ts`), "utf8");
|
||||
assert.match(moduleSource, /import\s*\{\s*generatedTranslations\s*\}\s*from\s*["']\.\/i18n\/generatedTranslations["']/);
|
||||
assert.match(moduleSource, /\btranslations(?:\s*:\s*generatedTranslations)?\s*,/);
|
||||
for (const [english, german] of Object.entries(labels[name])) {
|
||||
assert.equal(catalogs[index].en[english], english);
|
||||
assert.equal(catalogs[index].de[english], german);
|
||||
for (const [language, expected] of [["en", english], ["de", german]]) {
|
||||
const markup = heading(index, language, english);
|
||||
assert(markup.includes(`<h1>${expected}</h1>`), `${name}: ${language} contextual heading`);
|
||||
assert(markup.includes(language === "de" ? "Benutzerdokumentation öffnen" : "Open user documentation"));
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -780,6 +780,11 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertIn("redis", compose["services"])
|
||||
self.assertIn("worker", compose["services"])
|
||||
self.assertIn("load-balancer", compose["services"])
|
||||
self.assertEqual(
|
||||
"haproxy:3.2.23-alpine@sha256:"
|
||||
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
|
||||
compose["services"]["load-balancer"]["image"],
|
||||
)
|
||||
self.assertNotIn("test-mail", compose["services"])
|
||||
self.assertEqual(
|
||||
["127.0.0.1:8080:8080"],
|
||||
@@ -1145,8 +1150,26 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertEqual(1, parsed.replicas.web)
|
||||
self.assertEqual(1, parsed.replicas.worker)
|
||||
self.assertEqual("managed", parsed.components.load_balancer.mode)
|
||||
self.assertEqual(
|
||||
default_spec().components.load_balancer.image,
|
||||
parsed.components.load_balancer.image,
|
||||
)
|
||||
self.assertEqual("local", parsed.ingress.mode)
|
||||
|
||||
def test_load_balancer_patch_does_not_rewrite_an_existing_image(self) -> None:
|
||||
for image in (
|
||||
"haproxy:3.2.21-alpine",
|
||||
"registry.example.test/haproxy@sha256:" + "a" * 64,
|
||||
):
|
||||
with self.subTest(image=image):
|
||||
saved = default_spec(load_balancer_image=image).to_dict()
|
||||
|
||||
restored = parse_spec(json.loads(json.dumps(saved)))
|
||||
compose = render_compose(restored)
|
||||
|
||||
self.assertEqual(image, restored.components.load_balancer.image)
|
||||
self.assertEqual(image, compose["services"]["load-balancer"]["image"])
|
||||
|
||||
def test_compose_contains_no_secret_values(self) -> None:
|
||||
spec = default_spec()
|
||||
values = initial_secrets(spec)
|
||||
|
||||
Executable
+438
@@ -0,0 +1,438 @@
|
||||
"""Audit the selected fixture workspace, never a fuller neighboring checkout."""
|
||||
|
||||
from argparse import Namespace
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from types import ModuleType, SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = META_ROOT / "tools/inventory/platform-interface-inventory.py"
|
||||
WEBUI_SCRIPT = META_ROOT / "tools/inventory/extract-webui-structure.mjs"
|
||||
SPEC = importlib.util.spec_from_file_location("audit_scope_inventory", SCRIPT)
|
||||
assert SPEC is not None and SPEC.loader is not None
|
||||
inventory = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(inventory)
|
||||
|
||||
sys.path.insert(0, str(META_ROOT / "tools/devkit"))
|
||||
from govoplan_devkit import docs, issues, runner # noqa: E402
|
||||
from govoplan_devkit.workspace import Project, Repository # noqa: E402
|
||||
|
||||
|
||||
def write(path, content):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspaces(tmp_path):
|
||||
selected = tmp_path / "selected"
|
||||
default = tmp_path / "fuller-default"
|
||||
meta = tmp_path / "legacy-siblings/govoplan"
|
||||
selected.mkdir()
|
||||
meta.mkdir(parents=True)
|
||||
catalog = {
|
||||
"default_parent": str(default),
|
||||
"repositories": [
|
||||
{"name": "govoplan-core", "path": "govoplan-core"},
|
||||
{"name": "govoplan-example", "path": "nested/example"},
|
||||
{"name": "govoplan-optional", "path": "govoplan-optional"},
|
||||
],
|
||||
}
|
||||
write(meta / "repositories.json", json.dumps(catalog))
|
||||
write(
|
||||
selected / "nested/example/webui/src/Page.tsx",
|
||||
"export const page = <h1>SELECTED_ONLY</h1>;\n",
|
||||
)
|
||||
write(
|
||||
default / "nested/example/webui/src/Page.tsx",
|
||||
"export const page = <h1>DEFAULT_ONLY</h1>;\n",
|
||||
)
|
||||
write(
|
||||
default / "govoplan-optional/webui/src/Page.tsx",
|
||||
"export const page = <h1>DEFAULT_OPTIONAL</h1>;\n",
|
||||
)
|
||||
for item in catalog["repositories"]:
|
||||
(default / item["path"] / "src").mkdir(parents=True)
|
||||
return selected, default, meta, catalog
|
||||
|
||||
|
||||
def test_explicit_partial_root_beats_fuller_legacy_discovery(workspaces, monkeypatch):
|
||||
selected, default, meta, catalog = workspaces
|
||||
monkeypatch.setattr(inventory, "META_ROOT", meta)
|
||||
assert inventory._resolve_workspace_root(catalog) == default
|
||||
assert inventory._resolve_workspace_root(catalog, selected) == selected
|
||||
with pytest.raises(ValueError, match="existing directory"):
|
||||
inventory._resolve_workspace_root(catalog, selected / "missing")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("relative", ["../fuller-default", "/absolute/path"])
|
||||
def test_repository_paths_cannot_escape_explicit_root(workspaces, relative):
|
||||
selected, _, _, _ = workspaces
|
||||
with pytest.raises(ValueError, match="inside the selected workspace"):
|
||||
inventory._validate_repository_roots(
|
||||
{"repositories": [{"path": relative}]}, selected
|
||||
)
|
||||
|
||||
|
||||
def test_linked_repository_cannot_borrow_default_sources(workspaces):
|
||||
selected, default, _, catalog = workspaces
|
||||
(selected / "govoplan-core").symlink_to(
|
||||
default / "govoplan-core", target_is_directory=True
|
||||
)
|
||||
with pytest.raises(ValueError, match="escapes the selected workspace"):
|
||||
inventory._validate_repository_roots(catalog, selected)
|
||||
|
||||
|
||||
def test_python_forwards_selected_root_and_configured_node(workspaces, monkeypatch):
|
||||
selected, _, meta, _ = workspaces
|
||||
monkeypatch.setattr(inventory, "META_ROOT", meta)
|
||||
monkeypatch.setenv("NODE", "/selected/toolchain/node")
|
||||
calls = []
|
||||
|
||||
def run(argv, **kwargs):
|
||||
calls.append((argv, kwargs))
|
||||
return SimpleNamespace(stdout=json.dumps({"workspaceRoot": str(selected)}))
|
||||
|
||||
monkeypatch.setattr(inventory.subprocess, "run", run)
|
||||
assert inventory._extract_webui(selected)["workspaceRoot"] == str(selected)
|
||||
argv, options = calls[0]
|
||||
assert argv == [
|
||||
"/selected/toolchain/node",
|
||||
str(meta / "tools/inventory/extract-webui-structure.mjs"),
|
||||
str(meta),
|
||||
"--workspace-root",
|
||||
str(selected),
|
||||
]
|
||||
assert options == {"check": True, "capture_output": True, "text": True}
|
||||
|
||||
|
||||
def test_python_rejects_webui_result_from_another_root(workspaces, monkeypatch):
|
||||
selected, default, _, _ = workspaces
|
||||
monkeypatch.setattr(
|
||||
inventory.subprocess,
|
||||
"run",
|
||||
lambda *a, **k: SimpleNamespace(
|
||||
stdout=json.dumps({"workspaceRoot": str(default)})
|
||||
),
|
||||
)
|
||||
with pytest.raises(ValueError, match="did not confirm"):
|
||||
inventory._extract_webui(selected)
|
||||
|
||||
|
||||
def test_main_forwards_same_root_to_every_collector(workspaces, monkeypatch):
|
||||
selected, _, meta, catalog = workspaces
|
||||
monkeypatch.setattr(inventory, "META_ROOT", meta)
|
||||
roots = []
|
||||
monkeypatch.setattr(
|
||||
inventory, "_extract_webui", lambda root: roots.append(root) or {}
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
inventory,
|
||||
"_extract_backend_endpoints",
|
||||
lambda data, root: roots.append(root) or [],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
inventory, "_extract_manifests", lambda data, root: roots.append(root) or []
|
||||
)
|
||||
monkeypatch.setattr(inventory, "_load_endpoint_declarations", lambda _: {})
|
||||
monkeypatch.setattr(inventory, "_load_high_risk_help_baseline", lambda _: {})
|
||||
monkeypatch.setattr(inventory, "_assemble_inventory", lambda **kwargs: {})
|
||||
monkeypatch.setattr(inventory, "_render_markdown", lambda data: "fixture\n")
|
||||
output = selected / "output"
|
||||
monkeypatch.setattr(
|
||||
sys,
|
||||
"argv",
|
||||
[str(SCRIPT), "--workspace-root", str(selected), "--output-dir", str(output)],
|
||||
)
|
||||
assert inventory.main() == 0
|
||||
assert roots == [selected, selected, selected]
|
||||
report = json.loads((output / "platform-interface-inventory.json").read_text())
|
||||
assert report["workspace_root"] == str(selected)
|
||||
assert report["workspace_selection"] == "explicit"
|
||||
|
||||
|
||||
def install_fixture_compiler(root):
|
||||
compiler = META_ROOT.parent / "govoplan-core/webui/node_modules/typescript"
|
||||
if not compiler.is_dir() or not shutil.which("node"):
|
||||
pytest.skip("Node and the installed Core TypeScript parser are required")
|
||||
target = root / "govoplan-core/webui/node_modules/typescript"
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Parser dependencies may be shared; audited source checkouts may not.
|
||||
target.symlink_to(compiler, target_is_directory=True)
|
||||
|
||||
|
||||
def collect_webui(meta, selected=None):
|
||||
argv = [shutil.which("node") or "node", str(WEBUI_SCRIPT), str(meta)]
|
||||
if selected is not None:
|
||||
argv += ["--workspace-root", str(selected)]
|
||||
return subprocess.run(argv, capture_output=True, text=True, timeout=30)
|
||||
|
||||
|
||||
def test_javascript_explicit_root_is_authoritative_and_legacy_cli_still_works(
|
||||
workspaces,
|
||||
):
|
||||
selected, default, meta, _ = workspaces
|
||||
install_fixture_compiler(selected)
|
||||
install_fixture_compiler(default)
|
||||
explicit = collect_webui(meta, selected)
|
||||
assert explicit.returncode == 0, explicit.stderr
|
||||
report = json.loads(explicit.stdout)
|
||||
assert report["workspaceRoot"] == str(selected)
|
||||
assert [item["value"] for item in report["visibleText"]] == ["SELECTED_ONLY"]
|
||||
assert report["visibleText"][0]["file"] == "webui/src/Page.tsx"
|
||||
legacy = collect_webui(meta)
|
||||
assert legacy.returncode == 0, legacy.stderr
|
||||
assert json.loads(legacy.stdout)["workspaceRoot"] == str(default)
|
||||
assert {item["value"] for item in json.loads(legacy.stdout)["visibleText"]} == {
|
||||
"DEFAULT_ONLY",
|
||||
"DEFAULT_OPTIONAL",
|
||||
}
|
||||
|
||||
|
||||
def test_javascript_does_not_borrow_missing_core_dependencies(workspaces):
|
||||
selected, default, meta, _ = workspaces
|
||||
install_fixture_compiler(default)
|
||||
result = collect_webui(meta, selected)
|
||||
assert result.returncode != 0
|
||||
assert (
|
||||
str(selected / "govoplan-core/webui/node_modules/typescript") in result.stderr
|
||||
)
|
||||
assert not result.stdout
|
||||
|
||||
|
||||
def test_javascript_rejects_source_root_linked_outside_workspace(workspaces):
|
||||
selected, default, meta, _ = workspaces
|
||||
install_fixture_compiler(selected)
|
||||
linked = selected / "govoplan-optional/webui/src"
|
||||
linked.parent.mkdir(parents=True)
|
||||
linked.symlink_to(default / "govoplan-optional/webui/src", target_is_directory=True)
|
||||
result = collect_webui(meta, selected)
|
||||
assert result.returncode != 0
|
||||
assert "source root escapes the selected workspace" in result.stderr
|
||||
|
||||
|
||||
def test_backend_does_not_borrow_optional_endpoints(workspaces):
|
||||
selected, default, _, catalog = workspaces
|
||||
source = "from fastapi import APIRouter\nrouter = APIRouter()\n@router.get('/selected')\ndef endpoint(): pass\n"
|
||||
write(selected / "nested/example/src/routes.py", source)
|
||||
write(
|
||||
default / "govoplan-optional/src/routes.py", source.replace("selected", "other")
|
||||
)
|
||||
endpoints = inventory._extract_backend_endpoints(catalog, selected)
|
||||
assert [item["path"] for item in endpoints] == ["/selected"]
|
||||
(selected / "nested/example/src/foreign.py").symlink_to(
|
||||
default / "govoplan-optional/src/routes.py"
|
||||
)
|
||||
with pytest.raises(ValueError, match="Backend source path escapes"):
|
||||
inventory._extract_backend_endpoints(catalog, selected)
|
||||
|
||||
|
||||
def test_manifests_require_selected_core_sources(workspaces, monkeypatch):
|
||||
selected, default, _, catalog = workspaces
|
||||
write(
|
||||
default / "govoplan-core/src/govoplan_core/core/platform_interfaces.py",
|
||||
"raise AssertionError('foreign source imported')",
|
||||
)
|
||||
monkeypatch.syspath_prepend(str(default / "govoplan-core/src"))
|
||||
with pytest.raises(ValueError, match="requires Core interface sources"):
|
||||
inventory._extract_manifests(catalog, selected)
|
||||
|
||||
|
||||
def test_cached_application_import_cannot_replace_missing_checkout(
|
||||
workspaces, monkeypatch
|
||||
):
|
||||
selected, default, _, _ = workspaces
|
||||
# Meta's own devkit/release packages may audit another workspace.
|
||||
tooling = ModuleType("govoplan_release")
|
||||
tooling.__file__ = str(default / "tools/release/govoplan_release/__init__.py")
|
||||
modules = {"govoplan_release": tooling, "govoplan_devkit.docs": docs}
|
||||
# Isolate the fixture from application packages collected by unrelated
|
||||
# suites, without removing or replacing those real cached imports.
|
||||
monkeypatch.setattr(inventory, "sys", SimpleNamespace(modules=modules))
|
||||
inventory._assert_workspace_imports(selected)
|
||||
application = ModuleType("govoplan_audit_fixture")
|
||||
application.__file__ = str(
|
||||
default / "govoplan-optional/src/govoplan_audit_fixture/__init__.py"
|
||||
)
|
||||
modules["govoplan_audit_fixture"] = application
|
||||
with pytest.raises(ValueError, match="outside the selected inventory workspace"):
|
||||
inventory._assert_workspace_imports(selected)
|
||||
|
||||
|
||||
def test_partial_manifest_collection_uses_selected_sources_in_fresh_process(workspaces):
|
||||
selected, default, _, catalog = workspaces
|
||||
core = selected / "govoplan-core/src/govoplan_core"
|
||||
write(core / "__init__.py", "")
|
||||
write(core / "core/__init__.py", "")
|
||||
write(
|
||||
core / "core/platform_interfaces.py",
|
||||
"def manifest_interface_catalog(manifest): return {'selected': True}\n",
|
||||
)
|
||||
write(
|
||||
default / "govoplan-core/src/govoplan_core/__init__.py",
|
||||
"raise AssertionError('foreign Core loaded')\n",
|
||||
)
|
||||
module = selected / "nested/example/src/govoplan_example"
|
||||
write(module / "__init__.py", "")
|
||||
write(module / "backend/__init__.py", "")
|
||||
manifest = (
|
||||
"from types import SimpleNamespace as S\n"
|
||||
"def get_manifest():\n"
|
||||
" return S(id='selected', name='Selected', version='1', dependencies=(), "
|
||||
"optional_dependencies=(), required_capabilities=(), provides_interfaces=(), "
|
||||
"capability_factories={}, permissions=(), documentation=(), architecture=None, "
|
||||
"information_governance=S(to_dict=lambda: {}), frontend=None)\n"
|
||||
)
|
||||
write(module / "backend/manifest.py", manifest)
|
||||
write(
|
||||
default / "govoplan-optional/src/govoplan_optional/backend/manifest.py",
|
||||
manifest.replace("selected", "foreign"),
|
||||
)
|
||||
code = (
|
||||
"import importlib.util, json; from pathlib import Path; "
|
||||
f"spec=importlib.util.spec_from_file_location('fixture', {str(SCRIPT)!r}); "
|
||||
"module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module); "
|
||||
f"print(json.dumps(module._extract_manifests({catalog!r}, Path({str(selected)!r}))))"
|
||||
)
|
||||
env = {
|
||||
**os.environ,
|
||||
"PYTHONPATH": os.pathsep.join(
|
||||
str(default / item["path"] / "src") for item in catalog["repositories"]
|
||||
),
|
||||
}
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", code],
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
manifests = json.loads(result.stdout)
|
||||
assert [manifest["id"] for manifest in manifests] == ["selected"]
|
||||
assert manifests[0]["interface_catalog"] == {"selected": True}
|
||||
|
||||
|
||||
def doc_plan(workspace, monkeypatch):
|
||||
repos = tuple(
|
||||
Repository(name, workspace / name)
|
||||
for name in ("govoplan", "govoplan-core", "govoplan-example")
|
||||
)
|
||||
project = Project("fixture", repos, {})
|
||||
monkeypatch.setattr(docs, "load_project", lambda *a: project)
|
||||
args = Namespace(
|
||||
workspace_root=workspace,
|
||||
project=None,
|
||||
state_dir=workspace.parent / "state",
|
||||
repo=["govoplan-example"],
|
||||
changed=False,
|
||||
)
|
||||
return docs.build_doc_stages(args)
|
||||
|
||||
|
||||
def test_docs_plan_forwards_root_and_persists_all_limitations(workspaces, monkeypatch):
|
||||
selected, _, _, _ = workspaces
|
||||
stages = doc_plan(selected, monkeypatch)
|
||||
by_id = {stage["id"]: stage for stage in stages}
|
||||
for stage_id in (
|
||||
"docs.manifests",
|
||||
"docs.interface-inventory",
|
||||
"docs.plain-display-labels",
|
||||
):
|
||||
argv = by_id[stage_id]["argv"]
|
||||
assert argv[argv.index("--workspace-root") + 1] == str(selected)
|
||||
assert by_id["docs.translation-structure"]["argv"][1] == str(
|
||||
selected / "govoplan-core/webui/scripts/audit-i18n-structural.mjs"
|
||||
)
|
||||
assert issues.coverage_notes(stages) == docs.LIMITATIONS
|
||||
assert by_id["docs.plain-display-labels"]["argv"][-2:] == [
|
||||
"--repo",
|
||||
"govoplan-example",
|
||||
]
|
||||
|
||||
|
||||
def test_docs_limitations_survive_real_receipt_and_issue_evidence(
|
||||
workspaces, monkeypatch, tmp_path
|
||||
):
|
||||
selected, _, _, _ = workspaces
|
||||
stages = doc_plan(selected, monkeypatch)
|
||||
repo = selected / "example"
|
||||
repo.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(repo)], check=True, timeout=10)
|
||||
write(repo / "source.txt", "fixture\n")
|
||||
subprocess.run(
|
||||
["git", "-C", str(repo), "add", "source.txt"], check=True, timeout=10
|
||||
)
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"fixture",
|
||||
],
|
||||
check=True,
|
||||
timeout=10,
|
||||
)
|
||||
project = tmp_path / "project.json"
|
||||
write(
|
||||
project,
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "fixture",
|
||||
"repositories": [{"name": "example", "path": "example"}],
|
||||
"checks": [],
|
||||
"profiles": {},
|
||||
}
|
||||
),
|
||||
)
|
||||
args = Namespace(
|
||||
workspace_root=selected,
|
||||
project=project,
|
||||
state_dir=tmp_path / "state",
|
||||
dry_run=False,
|
||||
jobs=2,
|
||||
profile="docs",
|
||||
resume=None,
|
||||
)
|
||||
for stage in stages:
|
||||
stage.update(
|
||||
argv=[sys.executable, "-c", "print('fixture audit')"],
|
||||
cwd=str(repo),
|
||||
deps=[],
|
||||
resources=[],
|
||||
timeout_seconds=10,
|
||||
)
|
||||
monkeypatch.setattr(docs, "build_doc_stages", lambda _: stages)
|
||||
monkeypatch.setattr(
|
||||
runner, "environment_fingerprint", lambda *a, **k: "fixture-env"
|
||||
)
|
||||
result = docs.audit(args)
|
||||
assert result["status"] == "passed", result
|
||||
assert all(result["summary"].count(note) == 1 for note in docs.LIMITATIONS)
|
||||
receipt = runner.read_receipt(selected, args.state_dir, result["run_id"])
|
||||
assert issues.coverage_notes(receipt["stages"]) == docs.LIMITATIONS
|
||||
evidence = issues.evidence_record(result["run_id"], args)
|
||||
assert evidence["coverage_notes"] == docs.LIMITATIONS
|
||||
assert evidence["source_state"] == "matches-current"
|
||||
target = issues.NoteTarget(
|
||||
repo, "https://gitea.example.invalid", "fixture", "example", 1
|
||||
)
|
||||
_, body = issues.render_note(
|
||||
{"summary": [], "next": [], "body": ""}, evidence, target, "fixture"
|
||||
)
|
||||
assert all(note in body for note in docs.LIMITATIONS)
|
||||
Executable
+88
@@ -0,0 +1,88 @@
|
||||
"""Signals during snapshot probes must not become passing run evidence."""
|
||||
|
||||
import os
|
||||
import signal
|
||||
|
||||
import pytest
|
||||
|
||||
import test_devkit_runner as fixtures
|
||||
from govoplan_devkit import runner
|
||||
from govoplan_devkit.checkpoints import Checkpoints
|
||||
|
||||
example = fixtures.example
|
||||
|
||||
|
||||
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
|
||||
@pytest.mark.parametrize("probe", ["source", "environment"])
|
||||
def test_interrupt_during_final_snapshot_is_not_a_passing_run(
|
||||
example, monkeypatch, interruption, probe
|
||||
):
|
||||
args, repo = example
|
||||
calls = {"source": 0, "environment": 0}
|
||||
events, interruptions = [], []
|
||||
args.on_progress = events.append
|
||||
original = Checkpoints.source
|
||||
|
||||
def reached(kind):
|
||||
calls[kind] += 1
|
||||
if kind == probe and events and events[-1]["phase"] == "finalizing":
|
||||
# Intermediate checkpoint probes are also real verification. Target
|
||||
# finalization explicitly, while the runner owns signal handlers.
|
||||
interruptions.append(kind)
|
||||
os.kill(os.getpid(), interruption)
|
||||
|
||||
def source(self, *values, **kwargs):
|
||||
reached("source")
|
||||
return original(self, *values, **kwargs)
|
||||
|
||||
def environment(*_args, **_kwargs):
|
||||
reached("environment")
|
||||
return "fixture-environment"
|
||||
|
||||
monkeypatch.setattr(Checkpoints, "source", source)
|
||||
monkeypatch.setattr(runner, "environment_fingerprint", environment)
|
||||
result = runner.run_checks(args, [fixtures.stage(repo)])
|
||||
assert interruptions == [probe]
|
||||
assert calls[probe] >= 2
|
||||
assert result["stages"][0]["status"] == "passed"
|
||||
assert result["status"] == "interrupted"
|
||||
assert result["_exit_code"] != 0
|
||||
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
|
||||
assert receipt["status"] == "interrupted"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("interruption", [signal.SIGINT, signal.SIGTERM])
|
||||
@pytest.mark.parametrize("probe", ["source", "environment"])
|
||||
def test_preparation_interrupt_stops_at_probe_boundary_without_running_checks(
|
||||
example, monkeypatch, interruption, probe
|
||||
):
|
||||
args, repo = example
|
||||
calls = []
|
||||
original = Checkpoints.source
|
||||
|
||||
def reached(kind):
|
||||
calls.append(kind)
|
||||
if kind == probe:
|
||||
os.kill(os.getpid(), interruption)
|
||||
|
||||
def source(self, *values, **kwargs):
|
||||
reached("source")
|
||||
return original(self, *values, **kwargs)
|
||||
|
||||
def environment(*_args, **_kwargs):
|
||||
reached("environment")
|
||||
return "fixture-environment"
|
||||
|
||||
def must_not_execute(*args, **kwargs):
|
||||
pytest.fail("A cancelled preparation must not start a check")
|
||||
|
||||
monkeypatch.setattr(Checkpoints, "source", source)
|
||||
monkeypatch.setattr(runner, "environment_fingerprint", environment)
|
||||
monkeypatch.setattr(runner, "execute_stage", must_not_execute)
|
||||
result = runner.run_checks(args, [fixtures.stage(repo)])
|
||||
assert calls == (["source"] if probe == "source" else ["source", "environment"])
|
||||
assert result["status"] == "interrupted"
|
||||
assert result["snapshot_verified"] is False
|
||||
assert result["_exit_code"] != 0
|
||||
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
|
||||
assert receipt["status"] == "interrupted"
|
||||
Executable
+451
@@ -0,0 +1,451 @@
|
||||
"""Bounded planning fixtures: no application imports, compilers or servers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit.catalog import (
|
||||
_expanded_repositories,
|
||||
build_stages,
|
||||
module_ui_stages,
|
||||
)
|
||||
from govoplan_devkit.docs import build_doc_stages
|
||||
from govoplan_devkit.workspace import Project, Repository
|
||||
|
||||
|
||||
class CatalogTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-catalog-")
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.core = Repository("govoplan-core", self.root / "govoplan-core", ("core",))
|
||||
self.meta = Repository("govoplan", self.root / "govoplan", ("meta",))
|
||||
self.module = Repository(
|
||||
"govoplan-example", self.root / "govoplan-example", ("example",)
|
||||
)
|
||||
self.project = Project("fixture", (self.meta, self.core, self.module), {})
|
||||
for repo in self.project.repositories:
|
||||
repo.path.mkdir()
|
||||
(self.meta.path / "tools/checks").mkdir(parents=True)
|
||||
shutil.copyfile(
|
||||
Path(__file__).resolve().parents[1] / "tools/checks/focused-phases.json",
|
||||
self.meta.path / "tools/checks/focused-phases.json",
|
||||
)
|
||||
|
||||
def write(self, relative, content):
|
||||
path = self.root / relative
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
def plan(self, profile, repos=None, changed=False):
|
||||
with patch("govoplan_devkit.workspace.load_project", return_value=self.project):
|
||||
return build_stages(self.root, profile, repos or [], changed)
|
||||
|
||||
def test_ui_compiles_once_and_quick_does_not_compile(self):
|
||||
self.write(
|
||||
"govoplan-core/webui/package.json",
|
||||
json.dumps(
|
||||
{"scripts": {"test:components": "node scripts/run-component-tests.mjs"}}
|
||||
),
|
||||
)
|
||||
self.write("govoplan-core/webui/scripts/run-component-tests.mjs", "// fixture")
|
||||
self.assertFalse(
|
||||
any(
|
||||
"run-component-tests.mjs" in " ".join(item["argv"])
|
||||
for item in self.plan("quick")
|
||||
)
|
||||
)
|
||||
matches = [
|
||||
item
|
||||
for item in self.plan("ui")
|
||||
if "run-component-tests.mjs" in " ".join(item["argv"])
|
||||
]
|
||||
self.assertEqual(len(matches), 1)
|
||||
self.assertEqual(matches[0]["id"], "core.component-batch")
|
||||
|
||||
def test_module_script_metadata_is_bounded_and_deduplicated(self):
|
||||
self.write(
|
||||
"govoplan-example/webui/package.json",
|
||||
json.dumps(
|
||||
{
|
||||
"scripts": {
|
||||
"test:interface-pattern": "node scripts/test-interface-pattern-language.mjs",
|
||||
"test:source": "node --test tests/source.test.mjs",
|
||||
"test:dangerous-chain": "node tests/source.test.mjs && npm run dev",
|
||||
"test:flags": "node --eval 'startServer()'",
|
||||
}
|
||||
}
|
||||
),
|
||||
)
|
||||
self.write(
|
||||
"govoplan-example/webui/scripts/test-interface-pattern-language.mjs",
|
||||
"// source-only fixture",
|
||||
)
|
||||
self.write(
|
||||
"govoplan-example/webui/tests/source.test.mjs", "// source-only fixture"
|
||||
)
|
||||
stages = module_ui_stages(self.module, reason="fixture")
|
||||
self.assertEqual(len(stages), 2)
|
||||
self.assertEqual(len({item["id"] for item in stages}), 2)
|
||||
self.assertTrue(all(item["argv"][0] == "{node}" for item in stages))
|
||||
self.assertFalse(any("&&" in item["argv"] for item in stages))
|
||||
|
||||
def test_full_is_never_narrowed_by_repo_filter(self):
|
||||
result = self.plan("full", ["example"])
|
||||
self.assertEqual(
|
||||
[item["id"] for item in result],
|
||||
[
|
||||
"focused." + identity
|
||||
for identity in (
|
||||
"preflight",
|
||||
"tooling",
|
||||
"backend",
|
||||
"core-ui",
|
||||
"module-builds",
|
||||
"browser",
|
||||
"module-ui",
|
||||
)
|
||||
],
|
||||
)
|
||||
self.assertEqual(
|
||||
[item["after"] for item in result],
|
||||
[[], *[[item["id"]] for item in result[:-1]]],
|
||||
)
|
||||
self.assertTrue(all(item["deps"] == [] for item in result))
|
||||
self.assertTrue(
|
||||
all(
|
||||
item["argv"]
|
||||
== [
|
||||
"bash",
|
||||
str(self.meta.path / "tools/checks/check-focused.sh"),
|
||||
"--phase",
|
||||
item["id"].removeprefix("focused."),
|
||||
]
|
||||
for item in result
|
||||
)
|
||||
)
|
||||
self.assertIn("webui:govoplan-example", result[-1]["resources"])
|
||||
self.assertIn("backend:test-state", result[0]["resources"])
|
||||
|
||||
def test_changed_empty_is_not_full_verification(self):
|
||||
with (
|
||||
patch("govoplan_devkit.workspace.load_project", return_value=self.project),
|
||||
patch("govoplan_devkit.workspace.selected_repositories", return_value=[]),
|
||||
):
|
||||
self.assertEqual(build_stages(self.root, "quick", [], True), [])
|
||||
self.assertEqual(len(build_stages(self.root, "full", [], True)), 7)
|
||||
|
||||
def test_full_ui_scope_excludes_backend_only_but_includes_all_ui_owners(self):
|
||||
backend = Repository(
|
||||
"govoplan-backend-only", self.root / "govoplan-backend-only"
|
||||
)
|
||||
backend.path.mkdir()
|
||||
(self.module.path / "webui").mkdir()
|
||||
(self.core.path / "webui").mkdir()
|
||||
project = Project("fixture", (*self.project.repositories, backend), {})
|
||||
with patch("govoplan_devkit.workspace.load_project", return_value=project):
|
||||
checks = build_stages(self.root, "full", ["example"], False)
|
||||
for check in checks[:3]:
|
||||
self.assertNotIn("inputs", check)
|
||||
for check in checks[3:]:
|
||||
self.assertEqual(
|
||||
check["inputs"]["repos"],
|
||||
["govoplan", "govoplan-core", "govoplan-example"],
|
||||
)
|
||||
(backend.path / "webui").mkdir()
|
||||
with patch("govoplan_devkit.workspace.load_project", return_value=project):
|
||||
replanned = build_stages(self.root, "full", [], False)
|
||||
self.assertIn(backend.name, replanned[3]["inputs"]["repos"])
|
||||
|
||||
def test_full_missing_checkout_explicitly_falls_back_to_workspace_inputs(self):
|
||||
missing = Repository("govoplan-missing", self.root / "govoplan-missing")
|
||||
project = Project("fixture", (*self.project.repositories, missing), {})
|
||||
with patch("govoplan_devkit.workspace.load_project", return_value=project):
|
||||
checks = build_stages(self.root, "full", [], False)
|
||||
self.assertTrue(all("inputs" not in check for check in checks))
|
||||
self.assertTrue(
|
||||
all(
|
||||
any("workspace-wide" in note for note in check["coverage_notes"])
|
||||
for check in checks[3:]
|
||||
)
|
||||
)
|
||||
|
||||
def test_full_requires_authoritative_phase_metadata(self):
|
||||
(self.meta.path / "tools/checks/focused-phases.json").write_text("{}")
|
||||
with self.assertRaisesRegex(ValueError, "phase metadata"):
|
||||
self.plan("full")
|
||||
|
||||
def test_native_ui_guards_and_component_batch_share_safe_ui_owner_scope(self):
|
||||
backend = Repository(
|
||||
"govoplan-backend-only", self.root / "govoplan-backend-only"
|
||||
)
|
||||
backend.path.mkdir()
|
||||
(self.module.path / "webui").mkdir()
|
||||
(self.core.path / "webui").mkdir()
|
||||
project = Project("fixture", (*self.project.repositories, backend), {})
|
||||
with patch("govoplan_devkit.workspace.load_project", return_value=project):
|
||||
checks = build_stages(self.root, "ui", ["example"], False)
|
||||
for check in checks:
|
||||
if check["id"] in {
|
||||
"jsx-value-imports",
|
||||
"heading-help",
|
||||
"core.component-batch",
|
||||
}:
|
||||
self.assertEqual(
|
||||
check["inputs"]["repos"],
|
||||
["govoplan", "govoplan-core", "govoplan-example"],
|
||||
)
|
||||
else:
|
||||
self.assertNotIn("inputs", check)
|
||||
|
||||
def test_unregistered_src_or_webui_disables_all_native_reuse(self):
|
||||
for directory in ("src", "webui"):
|
||||
with self.subTest(directory=directory):
|
||||
unknown = self.root / "govoplan-unregistered" / directory
|
||||
unknown.mkdir(parents=True)
|
||||
try:
|
||||
for profile in ("quick", "ui", "backend", "full"):
|
||||
checks = self.plan(profile)
|
||||
self.assertTrue(checks)
|
||||
self.assertTrue(
|
||||
all(check["reuse"] == "never" for check in checks)
|
||||
)
|
||||
self.assertTrue(
|
||||
all(
|
||||
any(
|
||||
"Unregistered sibling" in note
|
||||
for note in check["coverage_notes"]
|
||||
)
|
||||
for check in checks
|
||||
)
|
||||
)
|
||||
finally:
|
||||
unknown.rmdir()
|
||||
unknown.parent.rmdir()
|
||||
(self.root / "govoplan-unused-empty").mkdir()
|
||||
self.assertTrue(all("reuse" not in check for check in self.plan("full")))
|
||||
|
||||
def test_unregistered_broken_source_symlink_does_not_allow_reuse(self):
|
||||
unknown = self.root / "govoplan-unregistered"
|
||||
unknown.mkdir()
|
||||
(unknown / "src").symlink_to(self.root / "missing")
|
||||
self.assertTrue(all(check["reuse"] == "never" for check in self.plan("full")))
|
||||
|
||||
def test_unknown_repo_is_not_silently_ignored(self):
|
||||
with self.assertRaisesRegex(ValueError, "Unknown repository"):
|
||||
self.plan("ui", ["not-a-repo"])
|
||||
|
||||
def test_changed_provider_selects_transitive_declared_consumers(self):
|
||||
other = Repository("govoplan-other", self.root / "govoplan-other")
|
||||
final = Repository("govoplan-final", self.root / "govoplan-final")
|
||||
project = Project("fixture", (self.meta, self.module, other, final), {})
|
||||
self.write("govoplan/tools/release/govoplan_release/contracts.py", "# fixture")
|
||||
for name in ("example", "other", "final"):
|
||||
self.write(f"govoplan-{name}/src/fixture/backend/manifest.py", "# fixture")
|
||||
|
||||
def contract(_path, repo_name):
|
||||
gives = {
|
||||
self.module.name: ["first"],
|
||||
other.name: ["second"],
|
||||
final.name: [],
|
||||
}[repo_name]
|
||||
needs = {
|
||||
self.module.name: [],
|
||||
other.name: ["first"],
|
||||
final.name: ["second"],
|
||||
}[repo_name]
|
||||
return SimpleNamespace(
|
||||
repo=repo_name,
|
||||
provides_interfaces=[SimpleNamespace(name=value) for value in gives],
|
||||
requires_interfaces=[SimpleNamespace(name=value) for value in needs],
|
||||
)
|
||||
|
||||
with patch.dict(
|
||||
sys.modules,
|
||||
{
|
||||
"govoplan_release.contracts": SimpleNamespace(
|
||||
parse_manifest_contract=contract
|
||||
)
|
||||
},
|
||||
):
|
||||
selected, reason = _expanded_repositories(
|
||||
project, [self.module], changed=True
|
||||
)
|
||||
self.assertEqual(
|
||||
[repo.name for repo in selected], [self.module.name, other.name, final.name]
|
||||
)
|
||||
self.assertIn("declared interface consumers", reason)
|
||||
|
||||
def test_generic_dependency_closure_includes_filtered_prerequisites(self):
|
||||
config = {
|
||||
"schema_version": 1,
|
||||
"name": "generic",
|
||||
"repositories": [
|
||||
{"name": "one", "path": "one"},
|
||||
{"name": "two", "path": "two"},
|
||||
],
|
||||
"checks": [
|
||||
{
|
||||
"id": "compile",
|
||||
"argv": ["{node}", "compile.mjs"],
|
||||
"cwd": "one",
|
||||
"repos": ["two"],
|
||||
},
|
||||
{
|
||||
"id": "test",
|
||||
"argv": ["{node}", "test.mjs"],
|
||||
"cwd": "one",
|
||||
"deps": ["compile"],
|
||||
"repos": ["one"],
|
||||
},
|
||||
{
|
||||
"id": "other",
|
||||
"argv": ["{python}", "test.py"],
|
||||
"cwd": "two",
|
||||
"repos": ["two"],
|
||||
},
|
||||
],
|
||||
"profiles": {"quick": ["test", "other"]},
|
||||
}
|
||||
self.write("project.json", json.dumps(config))
|
||||
result = build_stages(
|
||||
self.root, "quick", ["one"], False, self.root / "project.json"
|
||||
)
|
||||
self.assertEqual([item["id"] for item in result], ["compile", "test"])
|
||||
self.assertEqual(result[1]["deps"], ["compile"])
|
||||
self.assertEqual(result[1]["cwd"], str(self.root / "one"))
|
||||
|
||||
def test_generic_cycles_and_duplicate_ids_fail(self):
|
||||
config = {
|
||||
"schema_version": 1,
|
||||
"name": "generic",
|
||||
"repositories": [{"name": "one", "path": "one"}],
|
||||
"checks": [
|
||||
{"id": "one", "argv": ["test"], "deps": ["two"]},
|
||||
{"id": "two", "argv": ["test"], "deps": ["one"]},
|
||||
],
|
||||
"profiles": {"quick": ["one"]},
|
||||
}
|
||||
self.write("project.json", json.dumps(config))
|
||||
with self.assertRaisesRegex(ValueError, "Cyclic"):
|
||||
build_stages(self.root, "quick", [], False, self.root / "project.json")
|
||||
config["checks"][1]["id"] = "one"
|
||||
self.write("project.json", json.dumps(config))
|
||||
with self.assertRaisesRegex(ValueError, "Duplicate"):
|
||||
build_stages(self.root, "quick", [], False, self.root / "project.json")
|
||||
|
||||
def test_generic_order_prerequisites_inputs_and_reuse_survive_planning(self):
|
||||
config = {
|
||||
"schema_version": 1,
|
||||
"name": "generic",
|
||||
"repositories": [
|
||||
{"name": "one", "path": "one"},
|
||||
{"name": "two", "path": "two"},
|
||||
],
|
||||
"checks": [
|
||||
{
|
||||
"id": "prepare",
|
||||
"argv": ["prepare"],
|
||||
"repos": ["two"],
|
||||
"reuse": "never",
|
||||
"inputs": {"repos": ["two"]},
|
||||
},
|
||||
{
|
||||
"id": "test",
|
||||
"argv": ["test"],
|
||||
"repos": ["one"],
|
||||
"after": ["prepare"],
|
||||
"reuse": "verified",
|
||||
"inputs": {"repos": ["one"]},
|
||||
},
|
||||
{"id": "broad", "argv": ["check"], "repos": ["one"]},
|
||||
],
|
||||
"profiles": {"quick": ["test", "broad"]},
|
||||
}
|
||||
self.write("project.json", json.dumps(config))
|
||||
result = build_stages(
|
||||
self.root, "quick", ["one"], False, self.root / "project.json"
|
||||
)
|
||||
self.assertEqual([item["id"] for item in result], ["prepare", "test", "broad"])
|
||||
self.assertEqual(result[1]["after"], ["prepare"])
|
||||
self.assertEqual(result[1]["deps"], [])
|
||||
self.assertEqual(result[0]["reuse"], "never")
|
||||
self.assertEqual(result[1]["reuse"], "verified")
|
||||
self.assertEqual(result[0]["inputs"], {"repos": ["two"]})
|
||||
self.assertEqual(result[1]["inputs"], {"repos": ["one"]})
|
||||
self.assertNotIn("inputs", result[2])
|
||||
|
||||
def test_generic_original_types_are_validated_before_coercion(self):
|
||||
for field, invalid in (
|
||||
("argv", "false"),
|
||||
("argv", None),
|
||||
("resources", "shared"),
|
||||
("cwd", None),
|
||||
("cwd", "../outside"),
|
||||
("title", 5),
|
||||
("timeout_seconds", True),
|
||||
):
|
||||
with self.subTest(field=field, invalid=invalid):
|
||||
config = {
|
||||
"schema_version": 1,
|
||||
"name": "generic",
|
||||
"repositories": [{"name": "one", "path": "one"}],
|
||||
"checks": [{"id": "test", "argv": ["test"], field: invalid}],
|
||||
"profiles": {"quick": ["test"]},
|
||||
}
|
||||
self.write("project.json", json.dumps(config))
|
||||
with self.assertRaises(ValueError):
|
||||
build_stages(
|
||||
self.root, "quick", [], False, self.root / "project.json"
|
||||
)
|
||||
|
||||
def test_ui_coverage_notes_make_excluded_suites_explicit_and_discover_tests_folder(
|
||||
self,
|
||||
):
|
||||
self.write(
|
||||
"govoplan-example/webui/package.json",
|
||||
json.dumps({"scripts": {"test:full-ui": "tsc && node tests/full-ui.js"}}),
|
||||
)
|
||||
self.write(
|
||||
"govoplan-example/webui/tests/aggregate-report-structure.test.mjs",
|
||||
"// fixture",
|
||||
)
|
||||
stages = self.plan("ui", ["example"])
|
||||
self.assertTrue(
|
||||
any("aggregate-report-structure" in item["id"] for item in stages)
|
||||
)
|
||||
self.assertTrue(
|
||||
any("test:full-ui" in note for note in stages[0]["coverage_notes"])
|
||||
)
|
||||
|
||||
def test_docs_reuses_existing_guards_and_only_narrows_plain_labels(self):
|
||||
args = argparse.Namespace(
|
||||
workspace_root=self.root,
|
||||
state_dir=self.root / "state",
|
||||
project=None,
|
||||
repo=["example"],
|
||||
changed=False,
|
||||
)
|
||||
with patch("govoplan_devkit.docs.load_project", return_value=self.project):
|
||||
stages = build_doc_stages(args)
|
||||
self.assertEqual(len(stages), 4)
|
||||
self.assertIn("check-manifest-shapes.py", " ".join(stages[0]["argv"]))
|
||||
self.assertIn("platform-interface-inventory.py", " ".join(stages[1]["argv"]))
|
||||
self.assertIn("--strict-declarations", stages[1]["argv"])
|
||||
self.assertEqual(stages[3]["argv"][-2:], ["--repo", "govoplan-example"])
|
||||
self.assertFalse(
|
||||
(self.root / "state").exists(), "Planning must not create artifacts"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Executable
+158
@@ -0,0 +1,158 @@
|
||||
"""Public CLI contracts and an executable portable-project smoke fixture."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import cli, doctor, runner
|
||||
from govoplan_devkit.common import META_ROOT
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"arguments",
|
||||
[["--json", "commands"], ["commands", "--json"], ["commands", "--format", "json"]],
|
||||
)
|
||||
def test_global_output_flags_work_on_either_side_of_command(arguments, capsys):
|
||||
assert cli.main(arguments) == 0
|
||||
output = json.loads(capsys.readouterr().out)
|
||||
names = {item["command"] for item in output["commands"]}
|
||||
assert {"context", "check", "doctor", "docs", "issues", "release", "git"} <= names
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"command",
|
||||
[
|
||||
"context",
|
||||
"doctor",
|
||||
"check",
|
||||
"resume",
|
||||
"recover",
|
||||
"docs",
|
||||
"review",
|
||||
"issues",
|
||||
"release",
|
||||
"git",
|
||||
],
|
||||
)
|
||||
def test_command_help_does_not_require_live_services(command, capsys):
|
||||
with pytest.raises(SystemExit) as stopped:
|
||||
cli.main([command, "--help"])
|
||||
assert stopped.value.code == 0
|
||||
assert "usage:" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_portable_project_executes_registered_commands_and_reads_receipt(
|
||||
tmp_path, capsys, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "state"))
|
||||
repo = tmp_path / "example"
|
||||
repo.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"--allow-empty",
|
||||
"-qm",
|
||||
"fixture",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "Portable",
|
||||
"repositories": [{"name": "example", "path": "example"}],
|
||||
"checks": [
|
||||
{
|
||||
"id": "test",
|
||||
"argv": ["{python}", "-c", "print('portable ok')"],
|
||||
"cwd": "example",
|
||||
}
|
||||
],
|
||||
"profiles": {"quick": ["test"]},
|
||||
}
|
||||
)
|
||||
)
|
||||
common = ["--workspace-root", str(tmp_path), "--project", str(project), "--json"]
|
||||
with patch.object(runner, "environment_fingerprint", return_value="fixture"):
|
||||
assert cli.main(common + ["check", "--profile", "quick"]) == 0
|
||||
result = json.loads(capsys.readouterr().out)
|
||||
assert result["status"] == "passed"
|
||||
assert cli.main(["status", result["run_id"], *common]) == 0
|
||||
status = json.loads(capsys.readouterr().out)
|
||||
assert status["snapshot_verified"] is True
|
||||
assert cli.main(["logs", result["run_id"], "--stage", "test", *common]) == 0
|
||||
assert "portable ok" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_portable_example_matches_published_schema():
|
||||
import jsonschema
|
||||
|
||||
jsonschema.validate(
|
||||
json.loads((META_ROOT / "tools/devkit/examples/project.json").read_text()),
|
||||
json.loads((META_ROOT / "tools/devkit/project.schema.json").read_text()),
|
||||
)
|
||||
|
||||
|
||||
def test_malformed_project_is_a_controlled_json_error(tmp_path, capsys):
|
||||
project = tmp_path / "bad.json"
|
||||
project.write_text('{"schema_version":true,"repositories":[]}')
|
||||
assert (
|
||||
cli.main(
|
||||
[
|
||||
"context",
|
||||
"--workspace-root",
|
||||
str(tmp_path),
|
||||
"--project",
|
||||
str(project),
|
||||
"--json",
|
||||
]
|
||||
)
|
||||
== 2
|
||||
)
|
||||
assert json.loads(capsys.readouterr().out)["status"] == "error"
|
||||
|
||||
|
||||
def test_doctor_is_read_only_and_preserves_dependency_warnings(tmp_path, capsys):
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(
|
||||
json.dumps(
|
||||
{"schema_version": 1, "repositories": [{"name": "example", "path": "."}]}
|
||||
)
|
||||
)
|
||||
(tmp_path / "package.json").write_text("{}")
|
||||
before = set(tmp_path.iterdir())
|
||||
with (
|
||||
patch.object(doctor, "tool_version", return_value="fixture"),
|
||||
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
|
||||
):
|
||||
assert (
|
||||
cli.main(
|
||||
[
|
||||
"doctor",
|
||||
"--workspace-root",
|
||||
str(tmp_path),
|
||||
"--project",
|
||||
str(project),
|
||||
"--json",
|
||||
]
|
||||
)
|
||||
== 0
|
||||
)
|
||||
output = json.loads(capsys.readouterr().out)
|
||||
assert any(item["status"] == "warning" for item in output["checks"])
|
||||
assert set(tmp_path.iterdir()) == before
|
||||
Executable
+339
@@ -0,0 +1,339 @@
|
||||
"""Coverage inventory is explicit intent, never execution or guessed completion."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/devkit"))
|
||||
from govoplan_devkit.catalog import build_coverage, build_stages, module_ui_stages # noqa: E402
|
||||
from govoplan_devkit.coverage import canonical_invocations # noqa: E402
|
||||
from govoplan_devkit.package_tests import ( # noqa: E402
|
||||
CORE_COMPONENT_SUITES,
|
||||
declared_tests,
|
||||
read_package,
|
||||
)
|
||||
from govoplan_devkit.workspace import Project, Repository # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fixture(tmp_path):
|
||||
core = Repository("govoplan-core", tmp_path / "govoplan-core", ("core",))
|
||||
meta = Repository("govoplan", tmp_path / "govoplan", ("meta",))
|
||||
module = Repository("govoplan-example", tmp_path / "govoplan-example", ("example",))
|
||||
for repo in (core, meta, module):
|
||||
(repo.path / "webui/scripts").mkdir(parents=True)
|
||||
(repo.path / "webui/tests").mkdir()
|
||||
scripts = {
|
||||
"test:components": "node scripts/run-component-tests.mjs",
|
||||
**{
|
||||
f"test:{name}": f"node scripts/run-component-tests.mjs {name}"
|
||||
for name in CORE_COMPONENT_SUITES
|
||||
},
|
||||
}
|
||||
(core.path / "webui/package.json").write_text(json.dumps({"scripts": scripts}))
|
||||
(core.path / "webui/scripts/run-component-tests.mjs").write_text("// fixture")
|
||||
(module.path / "webui/package.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"scripts": {
|
||||
"test:safe": "node --test tests/source.test.mjs",
|
||||
"test:compound": "tsc && node test.js",
|
||||
"test:bad-quote": "node 'broken",
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
(module.path / "webui/tests/source.test.mjs").write_text("// fixture")
|
||||
script = meta.path / "tools/checks/check-focused.sh"
|
||||
script.parent.mkdir(parents=True)
|
||||
phase_metadata = ROOT / "tools/checks/focused-phases.json"
|
||||
shutil.copyfile(phase_metadata, script.with_name("focused-phases.json"))
|
||||
bodies = {
|
||||
"core-ui": 'cd "$ROOT/webui"\n"$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components\n',
|
||||
"module-ui": 'cd "$WORKSPACE_ROOT/govoplan-example/webui"\n"$NPM" run test:compound\n',
|
||||
}
|
||||
script.write_text(
|
||||
"\n".join(
|
||||
f"focused_phase_{phase['id'].replace('-', '_')}() {{\n# devkit-phase: {phase['id']} begin\n"
|
||||
+ bodies.get(phase["id"], 'cd "$ROOT"\n')
|
||||
+ f"# devkit-phase: {phase['id']} end\n}}\n"
|
||||
for phase in json.loads(phase_metadata.read_text())["phases"]
|
||||
)
|
||||
)
|
||||
project = Project("Fixture", (meta, core, module), {})
|
||||
with (
|
||||
patch("govoplan_devkit.workspace.load_project", return_value=project),
|
||||
patch("govoplan_devkit.coverage.load_project", return_value=project),
|
||||
):
|
||||
yield tmp_path, core, module, project
|
||||
|
||||
|
||||
def rows(coverage, repo="govoplan-core"):
|
||||
return {row["name"]: row for row in coverage["suites"] if row["repo"] == repo}
|
||||
|
||||
|
||||
def test_core_aliases_are_explicitly_excluded_in_quick_and_covered_by_one_ui_stage(
|
||||
fixture,
|
||||
):
|
||||
root, _, _, _ = fixture
|
||||
quick = rows(build_coverage(root, "quick", [], False))
|
||||
ui = rows(build_coverage(root, "ui", [], False))
|
||||
for suite in CORE_COMPONENT_SUITES:
|
||||
assert quick["test:" + suite]["disposition"] == "excluded"
|
||||
assert "quick" in quick["test:" + suite]["reason"]
|
||||
assert ui["test:" + suite]["disposition"] == "covered_elsewhere"
|
||||
assert ui["test:" + suite]["covering_stage"] == "core.component-batch"
|
||||
assert ui["test:components"]["covered_components"] == list(CORE_COMPONENT_SUITES)
|
||||
|
||||
|
||||
def test_full_reports_only_four_of_sixteen_components_and_exact_shell_suite(fixture):
|
||||
root, _, _, _ = fixture
|
||||
result = build_coverage(root, "full", ["example"], False)
|
||||
core = rows(result)
|
||||
aliases = [core["test:" + suite] for suite in CORE_COMPONENT_SUITES]
|
||||
assert sum(item["disposition"] == "covered_elsewhere" for item in aliases) == 4
|
||||
assert sum(item["disposition"] == "excluded" for item in aliases) == 12
|
||||
assert len(core["test:components"]["covered_components"]) == 4
|
||||
assert "4/16" in core["test:components"]["reason"]
|
||||
module = rows(result, "govoplan-example")
|
||||
assert module["test:compound"]["disposition"] == "planned"
|
||||
assert module["test:safe"]["disposition"] == "excluded"
|
||||
assert result["stages"] == [
|
||||
"focused." + phase["id"]
|
||||
for phase in json.loads(
|
||||
(ROOT / "tools/checks/focused-phases.json").read_text()
|
||||
)["phases"]
|
||||
]
|
||||
assert core["test:components"]["covering_stage"] == "focused.core-ui"
|
||||
assert module["test:compound"]["covering_stage"] == "focused.module-ui"
|
||||
|
||||
|
||||
def test_prebuilt_plan_avoids_replanning(fixture):
|
||||
root, _, _, _ = fixture
|
||||
stages = build_stages(root, "quick", [], False)
|
||||
with patch(
|
||||
"govoplan_devkit.catalog.build_stages",
|
||||
side_effect=AssertionError("must not replan"),
|
||||
):
|
||||
result = build_coverage(root, "quick", [], False, stages=stages)
|
||||
assert rows(result, "govoplan-example")["test:safe"]["disposition"] == "planned"
|
||||
assert sum(result["counts"].values()) == result["suite_count"]
|
||||
assert all(item["reason"] for item in result["suites"])
|
||||
|
||||
|
||||
def test_missing_or_spoofed_phase_stage_does_not_grant_component_coverage(fixture):
|
||||
root, _, _, _ = fixture
|
||||
stages = build_stages(root, "full", [], False)
|
||||
without_core = [item for item in stages if item["id"] != "focused.core-ui"]
|
||||
result = build_coverage(root, "full", [], False, stages=without_core)
|
||||
assert rows(result)["test:components"]["covered_components"] == []
|
||||
assert (
|
||||
rows(result, "govoplan-example")["test:compound"]["covering_stage"]
|
||||
== "focused.module-ui"
|
||||
)
|
||||
next(item for item in stages if item["id"] == "focused.core-ui")["argv"] = ["true"]
|
||||
spoofed = build_coverage(root, "full", [], False, stages=stages)
|
||||
assert rows(spoofed)["test:components"]["covered_components"] == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wrapper", ["heredoc", "function", "conditional"])
|
||||
def test_lookalike_phase_wrappers_outside_top_level_do_not_grant_coverage(
|
||||
fixture, wrapper
|
||||
):
|
||||
root, core, _, project = fixture
|
||||
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
|
||||
path = meta / "tools/checks/check-focused.sh"
|
||||
fake = 'focused_phase_core_ui() {\n# devkit-phase: core-ui begin\ncd "$ROOT/webui"\n"$NPM" run test:spoof\n# devkit-phase: core-ui end\n}\n'
|
||||
prefix = {
|
||||
"heredoc": "cat <<'BODY'\n" + fake + "BODY\n",
|
||||
"function": "unused() {\n" + fake + "}\n",
|
||||
"conditional": "if false; then\n" + fake + "fi\n",
|
||||
}[wrapper]
|
||||
path.write_text(prefix + path.read_text())
|
||||
result = canonical_invocations(root, meta, core.path)
|
||||
assert result["notes"] == []
|
||||
assert [item["name"] for item in result["npm"]] == [
|
||||
"test:components",
|
||||
"test:compound",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("damage", ["missing", "duplicate", "bad-end"])
|
||||
def test_invalid_marked_phase_bodies_do_not_infer_coverage(fixture, damage):
|
||||
root, core, _, project = fixture
|
||||
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
|
||||
path = meta / "tools/checks/check-focused.sh"
|
||||
text = path.read_text()
|
||||
if damage == "missing":
|
||||
text = text.replace("focused_phase_core_ui()", "unregistered_core_ui()")
|
||||
elif damage == "duplicate":
|
||||
text += text
|
||||
else:
|
||||
text = text.replace(
|
||||
"# devkit-phase: core-ui end", "# devkit-phase: another end"
|
||||
)
|
||||
path.write_text(text)
|
||||
result = canonical_invocations(root, meta, core.path)
|
||||
assert result["npm"] == [] and result["node"] == []
|
||||
assert "no phase coverage inferred" in result["notes"][0]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"repo_name,name,command",
|
||||
[
|
||||
("govoplan-example", "test:components", "node scripts/run-component-tests.mjs"),
|
||||
(
|
||||
"govoplan-core",
|
||||
"test:dialog-focus",
|
||||
"node scripts/run-component-tests.mjs dialog-focus && npm run dev",
|
||||
),
|
||||
(
|
||||
"govoplan-core",
|
||||
"test:unrecognized",
|
||||
"node scripts/run-component-tests.mjs dialog-focus",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_component_alias_exemption_is_exact_and_core_only(
|
||||
tmp_path, repo_name, name, command
|
||||
):
|
||||
repo = Repository(repo_name, tmp_path / repo_name)
|
||||
package = repo.path / "webui/package.json"
|
||||
package.parent.mkdir(parents=True)
|
||||
package.write_text(json.dumps({"scripts": {name: command}}))
|
||||
declared = declared_tests(repo, package)
|
||||
assert declared[0]["component_suite"] is None
|
||||
assert declared[0]["_argv"] is None
|
||||
assert module_ui_stages(repo, reason="fixture") == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"body",
|
||||
[
|
||||
"[]",
|
||||
"null",
|
||||
'{"scripts":[]}',
|
||||
'{"scripts":{"test:bad":42}}',
|
||||
'{"scripts":{"test:one":"node a","test:one":"node b"}}',
|
||||
"[" * 2000 + "]" * 2000,
|
||||
],
|
||||
)
|
||||
def test_malformed_package_metadata_is_a_controlled_error(tmp_path, body):
|
||||
package = tmp_path / "package.json"
|
||||
package.write_text(body)
|
||||
with pytest.raises(ValueError):
|
||||
read_package(package)
|
||||
|
||||
|
||||
def test_oversize_package_and_symlinked_test_are_not_discovered(tmp_path):
|
||||
repo = Repository("example", tmp_path)
|
||||
webui = tmp_path / "webui"
|
||||
(webui / "tests").mkdir(parents=True)
|
||||
package = webui / "package.json"
|
||||
package.write_text(" " * (1024 * 1024 + 1))
|
||||
with pytest.raises(ValueError):
|
||||
read_package(package)
|
||||
package.write_text(
|
||||
json.dumps({"scripts": {"test:escape": "node tests/escape.mjs"}})
|
||||
)
|
||||
(webui / "tests/escape.mjs").symlink_to(tmp_path / "outside.mjs")
|
||||
(tmp_path / "outside.mjs").write_text("// fixture")
|
||||
assert module_ui_stages(repo, reason="fixture") == []
|
||||
|
||||
|
||||
def test_unparseable_and_sensitive_commands_do_not_leak_into_coverage(fixture):
|
||||
root, _, module, _ = fixture
|
||||
package = module.path / "webui/package.json"
|
||||
package.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"scripts": {
|
||||
"test:secret": "node tests/source.test.mjs --token unknown-private-value",
|
||||
"test:broken": "node 'unknown-other-secret",
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
result = build_coverage(root, "quick", [], False)
|
||||
encoded = json.dumps(result)
|
||||
assert (
|
||||
"unknown-private-value" not in encoded and "unknown-other-secret" not in encoded
|
||||
)
|
||||
assert "_argv" not in encoded and '"command"' not in encoded
|
||||
assert all(
|
||||
item["disposition"] == "unsupported"
|
||||
for item in rows(result, "govoplan-example").values()
|
||||
)
|
||||
|
||||
|
||||
def test_custom_check_coverage_redacts_separate_token_and_includes_unselected(tmp_path):
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"repositories": [{"name": "app", "path": "."}],
|
||||
"checks": [
|
||||
{
|
||||
"id": "one",
|
||||
"argv": ["check", "--token", "unknown-private-value"],
|
||||
},
|
||||
{"id": "two", "argv": ["true"]},
|
||||
],
|
||||
"profiles": {"quick": ["one"]},
|
||||
}
|
||||
)
|
||||
)
|
||||
result = build_coverage(tmp_path, "quick", [], False, project)
|
||||
assert "unknown-private-value" not in json.dumps(result)
|
||||
assert result["counts"]["planned"] == 1 and result["counts"]["excluded"] == 1
|
||||
|
||||
|
||||
def test_canonical_parser_does_not_credit_comments_heredocs_conditionals_or_chains(
|
||||
tmp_path,
|
||||
):
|
||||
meta, core = tmp_path / "govoplan", tmp_path / "govoplan-core"
|
||||
path = meta / "tools/checks/check-focused.sh"
|
||||
path.parent.mkdir(parents=True)
|
||||
path.write_text(
|
||||
'cd "$ROOT/webui"\n# "$NPM" run test:comment\n"$PYTHON" - <<\'PY\'\n"$NPM" run test:heredoc\nPY\nif false; then\n"$NPM" run test:conditional\nfi\n"$NPM" run test:compound && true\n"$NPM" run test:real\n'
|
||||
)
|
||||
result = canonical_invocations(tmp_path, meta, core)
|
||||
assert [item["name"] for item in result["npm"]] == ["test:real"]
|
||||
|
||||
|
||||
def test_real_canonical_gate_has_four_explicit_component_suites():
|
||||
result = canonical_invocations(ROOT.parent, ROOT, ROOT.parent / "govoplan-core")
|
||||
calls = [item for item in result["npm"] if item["name"] == "test:components"]
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["args"] == [
|
||||
"layout-primitives",
|
||||
"page-layout",
|
||||
"data-grid-actions",
|
||||
"mail-components",
|
||||
]
|
||||
assert calls[0]["phase"] == "core-ui"
|
||||
for name, phase in (
|
||||
("test:module-permutations", "module-builds"),
|
||||
("test:conformance", "browser"),
|
||||
):
|
||||
matching = [item for item in result["npm"] if item["name"] == name]
|
||||
assert len(matching) == 1 and matching[0]["phase"] == phase
|
||||
|
||||
|
||||
def test_known_core_component_aliases_match_the_owned_runner_registry():
|
||||
runner = (
|
||||
ROOT.parent / "govoplan-core/webui/scripts/run-component-tests.mjs"
|
||||
).read_text()
|
||||
block = runner.split("export const componentSuites = Object.freeze({", 1)[1].split(
|
||||
"});", 1
|
||||
)[0]
|
||||
assert set(re.findall(r'^ "([a-z0-9-]+)":', block, re.M)) == set(
|
||||
CORE_COMPONENT_SUITES
|
||||
)
|
||||
Executable
+164
@@ -0,0 +1,164 @@
|
||||
"""Portable preflight inference uses fixtures only and never installs or starts tools."""
|
||||
|
||||
from argparse import Namespace
|
||||
from pathlib import Path
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import doctor
|
||||
from govoplan_devkit.workspace import Project, Repository
|
||||
|
||||
|
||||
def project(tmp_path, checks, *, tools=None, profiles=None):
|
||||
return Project(
|
||||
"Fixture",
|
||||
(Repository("app", tmp_path / "app"), Repository("other", tmp_path / "other")),
|
||||
{
|
||||
"checks": checks,
|
||||
"profiles": profiles
|
||||
if profiles is not None
|
||||
else {"quick": [item["id"] for item in checks]},
|
||||
"tools": tools or {},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def check(identity, executable, *, repos=None, deps=None):
|
||||
return {
|
||||
"id": identity,
|
||||
"argv": [executable, "--version"],
|
||||
"cwd": ".",
|
||||
"repos": repos or [],
|
||||
"deps": deps or [],
|
||||
}
|
||||
|
||||
|
||||
def diagnose(tmp_path, configured, *, profile=None, repos=None, versions=None):
|
||||
args = Namespace(
|
||||
workspace_root=tmp_path,
|
||||
project=tmp_path / "not-read.json",
|
||||
repo=repos or [],
|
||||
profile=profile,
|
||||
)
|
||||
tools = {"python": sys.executable, "node": "fixture-node", "npm": "fixture-npm"}
|
||||
version_map = versions or {
|
||||
sys.executable: "Python 3",
|
||||
"fixture-node": "unavailable",
|
||||
"fixture-npm": "unavailable",
|
||||
}
|
||||
with (
|
||||
patch.object(doctor, "load_project", return_value=configured),
|
||||
patch.object(doctor, "resolve_tools", return_value=tools),
|
||||
patch.object(
|
||||
doctor,
|
||||
"tool_version",
|
||||
side_effect=lambda executable, _env: version_map[executable],
|
||||
) as probe,
|
||||
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
|
||||
):
|
||||
result = doctor.diagnose(args)
|
||||
return result, probe
|
||||
|
||||
|
||||
def test_python_only_portable_project_does_not_probe_or_block_on_unused_node(tmp_path):
|
||||
configured = project(tmp_path, [check("python-test", "{python}")])
|
||||
result, probe = diagnose(tmp_path, configured)
|
||||
assert result["_exit_code"] == 0
|
||||
assert result["required_tools"] == ["python"]
|
||||
assert probe.call_count == 1
|
||||
assert {
|
||||
item["id"] for item in result["checks"] if item["status"] == "not_required"
|
||||
} == {"node", "npm"}
|
||||
|
||||
|
||||
def test_selected_profile_excludes_other_profile_tool_requirements(tmp_path):
|
||||
configured = project(
|
||||
tmp_path,
|
||||
[check("py", "{python}"), check("js", "{npm}")],
|
||||
profiles={"quick": ["py"], "ui": ["js"]},
|
||||
)
|
||||
result, _ = diagnose(tmp_path, configured, profile="quick")
|
||||
assert result["required_tools"] == ["python"]
|
||||
result, _ = diagnose(tmp_path, configured)
|
||||
assert result["required_tools"] == ["node", "npm", "python"]
|
||||
assert result["_exit_code"] == 1
|
||||
|
||||
|
||||
def test_selected_repository_includes_dependency_tool_requirements(tmp_path):
|
||||
configured = project(
|
||||
tmp_path,
|
||||
[
|
||||
check("build", "{npm}", repos=["other"]),
|
||||
check("test", "{python}", repos=["app"], deps=["build"]),
|
||||
],
|
||||
)
|
||||
result, _ = diagnose(tmp_path, configured, repos=["app"])
|
||||
assert result["required_tools"] == ["node", "npm", "python"]
|
||||
|
||||
|
||||
def test_unselected_repository_does_not_require_its_tool(tmp_path):
|
||||
configured = project(
|
||||
tmp_path,
|
||||
[check("js", "{npm}", repos=["other"]), check("py", "{python}", repos=["app"])],
|
||||
)
|
||||
result, _ = diagnose(tmp_path, configured, repos=["app"])
|
||||
assert result["required_tools"] == ["python"]
|
||||
|
||||
|
||||
def test_explicit_tool_configuration_declares_indirect_script_dependency(tmp_path):
|
||||
configured = project(tmp_path, [check("shell", "sh")], tools={"npm": "fixture-npm"})
|
||||
result, _ = diagnose(tmp_path, configured)
|
||||
assert result["required_tools"] == ["node", "npm", "python"]
|
||||
assert result["_exit_code"] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"executable,expected",
|
||||
[
|
||||
("/opt/node/bin/node", ["node", "python"]),
|
||||
("npm", ["node", "npm", "python"]),
|
||||
("npx", ["node", "npm", "python"]),
|
||||
],
|
||||
)
|
||||
def test_direct_tool_names_are_inferred(tmp_path, executable, expected):
|
||||
result, _ = diagnose(tmp_path, project(tmp_path, [check("test", executable)]))
|
||||
assert result["required_tools"] == expected
|
||||
|
||||
|
||||
def test_context_only_project_needs_no_node_and_makes_no_files(tmp_path):
|
||||
before = set(tmp_path.iterdir())
|
||||
result, _ = diagnose(tmp_path, project(tmp_path, [], profiles={}))
|
||||
assert result["required_tools"] == ["python"]
|
||||
assert set(tmp_path.iterdir()) == before
|
||||
|
||||
|
||||
def test_missing_profile_is_not_silently_widened(tmp_path):
|
||||
with pytest.raises(ValueError, match="does not declare profile"):
|
||||
diagnose(tmp_path, project(tmp_path, []), profile="ui")
|
||||
|
||||
|
||||
def test_native_govoplan_still_requires_all_three_tools(tmp_path):
|
||||
configured = project(tmp_path, [])
|
||||
args = Namespace(workspace_root=tmp_path, project=None, repo=[], profile=None)
|
||||
with (
|
||||
patch.object(doctor, "load_project", return_value=configured),
|
||||
patch.object(
|
||||
doctor,
|
||||
"resolve_tools",
|
||||
return_value={"python": sys.executable, "node": "node", "npm": "npm"},
|
||||
),
|
||||
patch.object(
|
||||
doctor,
|
||||
"tool_version",
|
||||
side_effect=lambda executable, _env: (
|
||||
"Python" if executable == sys.executable else "unavailable"
|
||||
),
|
||||
),
|
||||
patch.object(doctor, "inspect_repository", return_value={"errors": []}),
|
||||
):
|
||||
result = doctor.diagnose(args)
|
||||
assert result["required_tools"] == ["node", "npm", "python"]
|
||||
assert result["_exit_code"] == 1
|
||||
Executable
+378
@@ -0,0 +1,378 @@
|
||||
"""Environment probes read bounded stable files; all fixtures are local."""
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import environment
|
||||
from govoplan_devkit.common import digest
|
||||
from govoplan_devkit.workspace import Project, Repository
|
||||
|
||||
|
||||
def mutate_during_read(monkeypatch, callback):
|
||||
original = hashlib.sha256
|
||||
mutated = False
|
||||
|
||||
class MutatingHasher:
|
||||
def __init__(self):
|
||||
self.hasher = original()
|
||||
|
||||
def update(self, chunk):
|
||||
nonlocal mutated
|
||||
self.hasher.update(chunk)
|
||||
if not mutated:
|
||||
mutated = True
|
||||
callback()
|
||||
|
||||
def hexdigest(self):
|
||||
return self.hasher.hexdigest()
|
||||
|
||||
monkeypatch.setattr(environment.hashlib, "sha256", MutatingHasher)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("content", [b"", b"stable", b"x" * (1024 * 1024 + 3)])
|
||||
def test_file_hash_keeps_existing_digest_and_exact_size_boundary(tmp_path, content):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(content)
|
||||
assert (
|
||||
environment._environment_file_hash(path, len(content))
|
||||
== hashlib.sha256(content).hexdigest()
|
||||
)
|
||||
|
||||
|
||||
def test_missing_optional_file_remains_optional_without_a_persistent_cache(tmp_path):
|
||||
path = tmp_path / "input"
|
||||
assert environment._environment_file_hash(path, 10) is None
|
||||
path.write_bytes(b"first")
|
||||
first = environment._environment_file_hash(path, 10)
|
||||
path.write_bytes(b"other")
|
||||
assert environment._environment_file_hash(path, 10) != first
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["fifo", "directory", "oversized", "dangling"])
|
||||
def test_present_unsafe_input_is_rejected_without_blocking(tmp_path, kind):
|
||||
path = tmp_path / "input"
|
||||
if kind == "fifo":
|
||||
os.mkfifo(path)
|
||||
elif kind == "directory":
|
||||
path.mkdir()
|
||||
elif kind == "oversized":
|
||||
path.write_bytes(b"too large")
|
||||
else:
|
||||
path.symlink_to(tmp_path / "missing")
|
||||
with pytest.raises(ValueError, match="Environment input"):
|
||||
environment._environment_file_hash(path, 2)
|
||||
|
||||
|
||||
def test_stable_venv_executable_symlink_keeps_original_path_and_identity(tmp_path):
|
||||
target = tmp_path / "real-python"
|
||||
target.write_bytes(b"fixture binary")
|
||||
executable = tmp_path / "venv" / "bin" / "python"
|
||||
executable.parent.mkdir(parents=True)
|
||||
executable.symlink_to(target)
|
||||
repo = tmp_path / "repo"
|
||||
metadata = repo / "node_modules" / ".package-lock.json"
|
||||
metadata.parent.mkdir(parents=True)
|
||||
metadata.write_bytes(b'{"fixture":true}')
|
||||
project = Project("Fixture", (Repository("repo", repo),), {})
|
||||
tools = {"python": str(executable)}
|
||||
env = {"PATH": "fixture", "PWD": "ignored"}
|
||||
distributions = b'[["fixture", "1"]]\n'
|
||||
with (
|
||||
patch.object(environment, "tool_version", return_value="fixture-version"),
|
||||
patch.object(
|
||||
environment,
|
||||
"require_capture",
|
||||
return_value=SimpleNamespace(returncode=0, stdout=distributions),
|
||||
) as capture,
|
||||
):
|
||||
actual = environment.environment_fingerprint(tmp_path, project, tools, env)
|
||||
assert actual == digest(
|
||||
{
|
||||
"environment": {"PATH": "fixture"},
|
||||
"tools": {
|
||||
"python": {
|
||||
"path": str(executable),
|
||||
"version": "fixture-version",
|
||||
"sha256": hashlib.sha256(target.read_bytes()).hexdigest(),
|
||||
}
|
||||
},
|
||||
"installed": {
|
||||
str(metadata): hashlib.sha256(metadata.read_bytes()).hexdigest(),
|
||||
"python_distributions": hashlib.sha256(distributions).hexdigest(),
|
||||
},
|
||||
}
|
||||
)
|
||||
assert capture.call_args.args[0][0] == str(executable)
|
||||
|
||||
|
||||
def test_symlinked_package_directory_is_allowed_when_stable(tmp_path):
|
||||
actual = tmp_path / "packages"
|
||||
actual.mkdir()
|
||||
(actual / ".package-lock.json").write_bytes(b"fixture")
|
||||
link = tmp_path / "node_modules"
|
||||
link.symlink_to(actual, target_is_directory=True)
|
||||
assert environment._environment_file_hash(link / ".package-lock.json", 10)
|
||||
|
||||
|
||||
def test_fifo_replacement_between_inspection_and_open_does_not_block(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"fixture")
|
||||
real_open = os.open
|
||||
|
||||
def replace_before_open(target, flags):
|
||||
path.unlink()
|
||||
os.mkfifo(path)
|
||||
assert flags & os.O_NONBLOCK
|
||||
return real_open(target, flags)
|
||||
|
||||
monkeypatch.setattr(environment.os, "open", replace_before_open)
|
||||
with pytest.raises(ValueError, match="bounded regular"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_growth_between_inspection_and_open_cannot_bypass_size_bound(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"a")
|
||||
real_open = os.open
|
||||
|
||||
def grow_before_open(target, flags):
|
||||
path.write_bytes(b"x" * 11)
|
||||
return real_open(target, flags)
|
||||
|
||||
monkeypatch.setattr(environment.os, "open", grow_before_open)
|
||||
with pytest.raises(ValueError, match="bounded regular"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_growth_during_read_cannot_bypass_size_bound(tmp_path, monkeypatch):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"a")
|
||||
mutate_during_read(monkeypatch, lambda: path.write_bytes(b"x" * 11))
|
||||
with pytest.raises(ValueError, match="grew beyond"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_same_size_edit_with_restored_mtime_is_not_a_stable_identity(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"first")
|
||||
metadata = path.stat()
|
||||
|
||||
def mutate():
|
||||
path.write_bytes(b"other")
|
||||
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
|
||||
|
||||
mutate_during_read(monkeypatch, mutate)
|
||||
with pytest.raises(ValueError, match="changed during"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_replacement_with_identical_bytes_is_not_a_stable_identity(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"fixture")
|
||||
replacement = tmp_path / "replacement"
|
||||
replacement.write_bytes(b"fixture")
|
||||
mutate_during_read(monkeypatch, lambda: replacement.replace(path))
|
||||
with pytest.raises(ValueError, match="changed during"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_symlink_retarget_with_identical_bytes_is_not_a_stable_identity(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
first, second, link = (tmp_path / name for name in ("first", "second", "python"))
|
||||
first.write_bytes(b"fixture")
|
||||
second.write_bytes(b"fixture")
|
||||
link.symlink_to(first)
|
||||
|
||||
def retarget():
|
||||
link.unlink()
|
||||
link.symlink_to(second)
|
||||
|
||||
mutate_during_read(monkeypatch, retarget)
|
||||
with pytest.raises(ValueError, match="changed during"):
|
||||
environment._environment_file_hash(link, 10)
|
||||
|
||||
|
||||
def test_source_disappearing_during_read_fails_closed(tmp_path, monkeypatch):
|
||||
path = tmp_path / "input"
|
||||
path.write_bytes(b"fixture")
|
||||
mutate_during_read(monkeypatch, path.unlink)
|
||||
with pytest.raises(ValueError, match="Environment input"):
|
||||
environment._environment_file_hash(path, 10)
|
||||
|
||||
|
||||
def test_parent_symlink_retarget_is_rejected_even_for_the_same_target_inode(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
first, second, link = (tmp_path / name for name in ("first", "second", "bin"))
|
||||
first.mkdir()
|
||||
second.mkdir()
|
||||
(first / "python").write_bytes(b"fixture")
|
||||
os.link(first / "python", second / "python")
|
||||
link.symlink_to(first, target_is_directory=True)
|
||||
|
||||
def retarget():
|
||||
link.unlink()
|
||||
link.symlink_to(second, target_is_directory=True)
|
||||
|
||||
mutate_during_read(monkeypatch, retarget)
|
||||
with pytest.raises(ValueError, match="changed during"):
|
||||
environment._environment_file_hash(link / "python", 10)
|
||||
|
||||
|
||||
def test_invalid_executable_is_rejected_before_any_version_process(tmp_path):
|
||||
executable = tmp_path / "python"
|
||||
os.mkfifo(executable)
|
||||
project = Project("Fixture", (), {})
|
||||
with patch.object(environment, "tool_version") as version:
|
||||
with pytest.raises(ValueError, match="bounded regular"):
|
||||
environment.environment_fingerprint(
|
||||
tmp_path, project, {"python": str(executable)}, {}
|
||||
)
|
||||
version.assert_not_called()
|
||||
|
||||
|
||||
def discovery_project(root, *, path="govoplan-backend"):
|
||||
repo = root / path
|
||||
(repo / "src").mkdir(parents=True)
|
||||
return Project(
|
||||
"GovOPlaN",
|
||||
(Repository("govoplan-backend", repo),),
|
||||
{"organization": "GovOPlaN"},
|
||||
)
|
||||
|
||||
|
||||
def test_native_shape_ignores_build_files_and_directory_timestamps(tmp_path):
|
||||
project = discovery_project(tmp_path)
|
||||
repo = project.repositories[0].path
|
||||
(repo / "webui").mkdir()
|
||||
before = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
for directory in (repo, repo / "src", repo / "webui"):
|
||||
(directory / "temporary-build-output").write_text("changed")
|
||||
(directory / "temporary-build-directory").mkdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) == before
|
||||
(repo / "webui" / "temporary-build-output").unlink()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) == before
|
||||
|
||||
|
||||
def test_native_shape_detects_unknown_sibling_and_source_addition_removal(tmp_path):
|
||||
project = discovery_project(tmp_path)
|
||||
baseline = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
sibling = tmp_path / "govoplan-unregistered"
|
||||
sibling.mkdir()
|
||||
empty = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
assert empty != baseline
|
||||
for name in ("src", "webui"):
|
||||
(sibling / name).mkdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) != empty
|
||||
(sibling / name).rmdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) == empty
|
||||
sibling.rmdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
|
||||
|
||||
|
||||
@pytest.mark.parametrize("repo_path", ["govoplan-backend", "nonstandard-layout"])
|
||||
def test_native_shape_detects_registered_backend_gaining_webui(tmp_path, repo_path):
|
||||
project = discovery_project(tmp_path, path=repo_path)
|
||||
baseline = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
webui = project.repositories[0].path / "webui"
|
||||
webui.mkdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) != baseline
|
||||
webui.rmdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) == baseline
|
||||
|
||||
|
||||
def test_native_shape_binds_registered_ownership_names(tmp_path):
|
||||
project = discovery_project(tmp_path, path="nonstandard-layout")
|
||||
renamed = Project(
|
||||
project.name,
|
||||
(Repository("govoplan-renamed", project.repositories[0].path),),
|
||||
project.config,
|
||||
)
|
||||
assert environment._native_discovery_fingerprint(
|
||||
tmp_path, project
|
||||
) != environment._native_discovery_fingerprint(tmp_path, renamed)
|
||||
|
||||
|
||||
def test_native_shape_detects_source_link_retarget_and_dangling_target(tmp_path):
|
||||
project = discovery_project(tmp_path)
|
||||
first, second = tmp_path / "first", tmp_path / "second"
|
||||
first.mkdir()
|
||||
second.mkdir()
|
||||
link = project.repositories[0].path / "webui"
|
||||
link.symlink_to(first, target_is_directory=True)
|
||||
baseline = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
link.unlink()
|
||||
link.symlink_to(second, target_is_directory=True)
|
||||
changed = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
assert changed != baseline
|
||||
second.rmdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) != changed
|
||||
|
||||
|
||||
def test_native_shape_encodes_non_directory_source_entries(tmp_path):
|
||||
project = discovery_project(tmp_path)
|
||||
webui = project.repositories[0].path / "webui"
|
||||
baseline = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
webui.write_text("not a directory")
|
||||
file_shape = environment._native_discovery_fingerprint(tmp_path, project)
|
||||
assert file_shape != baseline
|
||||
webui.unlink()
|
||||
webui.mkdir()
|
||||
assert environment._native_discovery_fingerprint(tmp_path, project) != file_shape
|
||||
|
||||
|
||||
def test_native_shape_audit_is_bounded_including_nonmatching_children(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
project = discovery_project(tmp_path)
|
||||
(tmp_path / "unrelated-one").mkdir()
|
||||
(tmp_path / "unrelated-two").mkdir()
|
||||
monkeypatch.setattr(environment, "MAX_DISCOVERY_CHILDREN", 2)
|
||||
with pytest.raises(ValueError, match="bounded ownership"):
|
||||
environment._native_discovery_fingerprint(tmp_path, project)
|
||||
|
||||
|
||||
def test_native_shape_resolution_loop_fails_closed(tmp_path):
|
||||
project = discovery_project(tmp_path)
|
||||
link = project.repositories[0].path / "webui"
|
||||
link.symlink_to(link)
|
||||
with pytest.raises(ValueError, match="discovery cannot be resolved"):
|
||||
environment._native_discovery_fingerprint(tmp_path, project)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("portable", [False, True])
|
||||
def test_only_native_environment_identity_binds_discovery_shape(tmp_path, portable):
|
||||
project = discovery_project(tmp_path)
|
||||
if portable:
|
||||
project.config["schema_version"] = 1
|
||||
executable = tmp_path / "fixture-python"
|
||||
executable.write_text("not executed")
|
||||
tools = {"python": str(executable)}
|
||||
with (
|
||||
patch.object(environment, "tool_version", return_value="fixture"),
|
||||
patch.object(
|
||||
environment,
|
||||
"require_capture",
|
||||
return_value=SimpleNamespace(returncode=0, stdout=b"[]"),
|
||||
),
|
||||
):
|
||||
baseline = environment.environment_fingerprint(tmp_path, project, tools, {})
|
||||
(tmp_path / "govoplan-new" / "src").mkdir(parents=True)
|
||||
changed = environment.environment_fingerprint(tmp_path, project, tools, {})
|
||||
assert (baseline == changed) is portable
|
||||
Executable
+601
@@ -0,0 +1,601 @@
|
||||
"""Incremental checkpoints use isolated local repositories, never product or remote state."""
|
||||
|
||||
from argparse import Namespace
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import runner
|
||||
from govoplan_devkit.checkpoints import Checkpoints
|
||||
from govoplan_devkit.common import atomic_json, read_json, resource_lock, state_root
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg"))
|
||||
root = tmp_path / "workspace"
|
||||
repos = {}
|
||||
for name in ("alpha", "beta"):
|
||||
repo = root / name
|
||||
repo.mkdir(parents=True)
|
||||
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
||||
(repo / "source.txt").write_text(name + " original\n")
|
||||
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"fixture",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
repos[name] = repo
|
||||
config = tmp_path / "project.json"
|
||||
config.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "Incremental fixture",
|
||||
"repositories": [{"name": name, "path": name} for name in repos],
|
||||
"checks": [],
|
||||
"profiles": {},
|
||||
}
|
||||
)
|
||||
)
|
||||
traces = tmp_path / "traces"
|
||||
traces.mkdir()
|
||||
args = Namespace(
|
||||
workspace_root=root,
|
||||
project=config,
|
||||
state_dir=tmp_path / "state",
|
||||
dry_run=False,
|
||||
jobs=2,
|
||||
profile="quick",
|
||||
resume=None,
|
||||
repo=[],
|
||||
changed=False,
|
||||
)
|
||||
return args, repos, traces
|
||||
|
||||
|
||||
def stage(workspace, identity, repo="alpha", *, inputs=True, body="", **extra):
|
||||
_, repos, traces = workspace
|
||||
counter = traces / identity
|
||||
code = (
|
||||
"from pathlib import Path; "
|
||||
f"counter=Path({str(counter)!r}); "
|
||||
"counter.write_text(str(int(counter.read_text())+1 if counter.exists() else 1)); "
|
||||
f"print({identity!r},flush=True); " + body
|
||||
)
|
||||
result = {
|
||||
"id": identity,
|
||||
"title": identity,
|
||||
"argv": [sys.executable, "-c", code],
|
||||
"cwd": str(repos[repo]),
|
||||
"timeout_seconds": 5,
|
||||
**extra,
|
||||
}
|
||||
if inputs:
|
||||
result["inputs"] = {"repos": [repo]}
|
||||
return result
|
||||
|
||||
|
||||
def run(workspace, stages, *, environment="e" * 64):
|
||||
args, _, _ = workspace
|
||||
with patch.object(runner, "environment_fingerprint", return_value=environment):
|
||||
return runner.run_checks(args, stages)
|
||||
|
||||
|
||||
def count(workspace, identity):
|
||||
path = workspace[2] / identity
|
||||
return int(path.read_text()) if path.exists() else 0
|
||||
|
||||
|
||||
def stages_by_id(result):
|
||||
return {item["id"]: item for item in result["stages"]}
|
||||
|
||||
|
||||
def test_checkpoint_probes_and_durable_persistence_hold_stage_resource_lock(workspace):
|
||||
args, _, _ = workspace
|
||||
locks = state_root(args.workspace_root) / "resource-locks"
|
||||
original_identity, original_write = Checkpoints.identity, runner.atomic_json
|
||||
probes, persisted = [], []
|
||||
|
||||
def assert_held():
|
||||
with pytest.raises(RuntimeError, match="busy"):
|
||||
with resource_lock(locks, "checkpoint-fixture"):
|
||||
pass
|
||||
|
||||
def identity(self, selected):
|
||||
assert_held()
|
||||
probes.append(selected["id"])
|
||||
return original_identity(self, selected)
|
||||
|
||||
def write(path, payload):
|
||||
original_write(path, payload)
|
||||
if (
|
||||
payload.get("phase") == "checking"
|
||||
and payload["stages"][0].get("checkpoint_verified") is True
|
||||
and not persisted
|
||||
):
|
||||
assert_held()
|
||||
assert read_json(path)["stages"][0]["checkpoint_verified"] is True
|
||||
persisted.append(path)
|
||||
|
||||
with (
|
||||
patch.object(Checkpoints, "identity", identity),
|
||||
patch.object(runner, "atomic_json", side_effect=write),
|
||||
):
|
||||
result = run(
|
||||
workspace, [stage(workspace, "a", resources=["checkpoint-fixture"])]
|
||||
)
|
||||
assert result["status"] == "passed"
|
||||
assert len(probes) >= 2 and len(persisted) == 1
|
||||
|
||||
|
||||
def test_checkpoint_save_failure_never_releases_dependent_execution(workspace):
|
||||
args, _, _ = workspace
|
||||
original_write, original_command, original_wait = (
|
||||
runner.atomic_json,
|
||||
runner._execute_stage_command,
|
||||
runner.wait,
|
||||
)
|
||||
scheduler_waiting = threading.Event()
|
||||
checkpoint_stalled = threading.Event()
|
||||
scheduler_rechecked = threading.Event()
|
||||
release_failure = threading.Event()
|
||||
failed_write_paths, results, errors = [], [], []
|
||||
|
||||
def command(selected, *values, **kwargs):
|
||||
if selected["id"] == "producer":
|
||||
# Let the scheduler finish its initial running-state save and enter
|
||||
# its wait loop before the command publishes a passing checkpoint.
|
||||
assert scheduler_waiting.wait(timeout=5)
|
||||
return original_command(selected, *values, **kwargs)
|
||||
|
||||
def wait(*values, **kwargs):
|
||||
scheduler_waiting.set()
|
||||
result = original_wait(*values, **kwargs)
|
||||
if checkpoint_stalled.is_set():
|
||||
scheduler_rechecked.set()
|
||||
return result
|
||||
|
||||
def write(path, payload):
|
||||
producer = next(
|
||||
(item for item in payload.get("stages", []) if item["id"] == "producer"),
|
||||
None,
|
||||
)
|
||||
if (
|
||||
producer
|
||||
and producer.get("checkpoint_verified") is True
|
||||
and not failed_write_paths
|
||||
):
|
||||
failed_write_paths.append(path)
|
||||
checkpoint_stalled.set()
|
||||
assert release_failure.wait(timeout=5)
|
||||
raise OSError("fixture checkpoint persistence failed")
|
||||
return original_write(path, payload)
|
||||
|
||||
def execute():
|
||||
try:
|
||||
results.append(
|
||||
run(
|
||||
workspace,
|
||||
[
|
||||
stage(workspace, "producer"),
|
||||
stage(workspace, "dependent", "beta", deps=["producer"]),
|
||||
],
|
||||
)
|
||||
)
|
||||
except BaseException as exc:
|
||||
errors.append(exc)
|
||||
|
||||
worker = threading.Thread(target=execute)
|
||||
with (
|
||||
patch.object(runner, "atomic_json", side_effect=write),
|
||||
patch.object(runner, "_execute_stage_command", side_effect=command),
|
||||
patch.object(runner, "wait", side_effect=wait),
|
||||
):
|
||||
worker.start()
|
||||
try:
|
||||
assert checkpoint_stalled.wait(timeout=5)
|
||||
assert scheduler_rechecked.wait(timeout=5)
|
||||
# Keep persistence blocked across a scheduling turn: the in-memory
|
||||
# producer status must not grant authority to start a consumer.
|
||||
time.sleep(0.15)
|
||||
assert count(workspace, "dependent") == 0
|
||||
durable = read_json(failed_write_paths[0])
|
||||
assert (
|
||||
stages_by_id(durable)["producer"].get("checkpoint_verified") is not True
|
||||
)
|
||||
finally:
|
||||
release_failure.set()
|
||||
worker.join(timeout=8)
|
||||
assert not worker.is_alive() and not errors
|
||||
assert len(results) == 1 and results[0]["status"] == "failed"
|
||||
assert count(workspace, "producer") == 1 and count(workspace, "dependent") == 0
|
||||
by_id = stages_by_id(results[0])
|
||||
assert by_id["producer"]["status"] == "failed"
|
||||
assert by_id["producer"]["checkpoint_verified"] is False
|
||||
assert "fixture checkpoint persistence failed" in by_id["producer"]["error"]
|
||||
assert by_id["dependent"]["status"] == "skipped"
|
||||
persisted = runner.read_receipt(
|
||||
args.workspace_root, args.state_dir, results[0]["run_id"]
|
||||
)
|
||||
assert stages_by_id(persisted)["producer"]["checkpoint_verified"] is False
|
||||
|
||||
|
||||
def test_verified_checkpoint_reuses_unchanged_stage_after_other_repo_changes(workspace):
|
||||
args, repos, _ = workspace
|
||||
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
|
||||
first = run(workspace, plan)
|
||||
old_bytes = Path(first["receipt_path"]).read_bytes()
|
||||
(repos["beta"] / "source.txt").write_text("beta changed\n")
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
by_id = stages_by_id(second)
|
||||
assert second["status"] == "passed" and second["snapshot_verified"] is True
|
||||
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
|
||||
assert by_id["a"]["reused_from"] == first["run_id"]
|
||||
assert "reused_from" not in by_id["b"]
|
||||
assert by_id["a"]["checkpoint_verified"] is True
|
||||
assert by_id["a"]["checkpoint_version"] == 1
|
||||
assert isinstance(by_id["a"]["cache_key"], str) and by_id["a"]["cache_key"]
|
||||
assert second["source_fingerprint"] != first["source_fingerprint"]
|
||||
assert Path(first["receipt_path"]).read_bytes() == old_bytes
|
||||
|
||||
|
||||
def test_failed_post_execution_probe_preserves_actual_log_without_certifying_it(
|
||||
workspace,
|
||||
):
|
||||
from govoplan_devkit.checkpoints import Checkpoints
|
||||
|
||||
identity = Checkpoints.identity
|
||||
calls = 0
|
||||
|
||||
def probe(self, selected):
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
if calls == 2:
|
||||
raise ValueError("fixture input became unreadable")
|
||||
return identity(self, selected)
|
||||
|
||||
with patch.object(Checkpoints, "identity", probe):
|
||||
result = run(workspace, [stage(workspace, "a")])
|
||||
selected = result["stages"][0]
|
||||
assert result["status"] == "stale"
|
||||
assert selected["status"] == "stale" and selected["exit_code"] == 0
|
||||
assert selected["checkpoint_verified"] is False
|
||||
assert Path(selected["log_path"]).read_text() == "a\n"
|
||||
assert "fixture input became unreadable" in selected["error"]
|
||||
|
||||
|
||||
def test_changed_and_new_commands_run_without_discarding_unrelated_checkpoint(
|
||||
workspace,
|
||||
):
|
||||
args, _, _ = workspace
|
||||
first = run(workspace, [stage(workspace, "a"), stage(workspace, "b", "beta")])
|
||||
args.resume = first["run_id"]
|
||||
second = run(
|
||||
workspace,
|
||||
[
|
||||
stage(workspace, "a"),
|
||||
stage(workspace, "b", "beta", body="print('changed command')"),
|
||||
stage(workspace, "new", "beta"),
|
||||
],
|
||||
)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 1
|
||||
assert count(workspace, "b") == 2
|
||||
assert count(workspace, "new") == 1
|
||||
assert first["plan_fingerprint"] != second["plan_fingerprint"]
|
||||
|
||||
|
||||
def test_unrelated_profile_edit_does_not_change_existing_stage_execution_identity(
|
||||
workspace,
|
||||
):
|
||||
args, _, _ = workspace
|
||||
plan = [stage(workspace, "a")]
|
||||
first = run(workspace, plan)
|
||||
config = json.loads(args.project.read_text())
|
||||
config["checks"] = [
|
||||
{
|
||||
"id": "extra",
|
||||
"argv": [sys.executable, "-c", "print('extra')"],
|
||||
"cwd": "beta",
|
||||
"repos": ["beta"],
|
||||
}
|
||||
]
|
||||
config["profiles"] = {"backend": ["extra"]}
|
||||
args.project.write_text(json.dumps(config))
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 1
|
||||
|
||||
|
||||
def test_added_dependency_edge_invalidates_consumer_even_when_both_repo_bytes_match(
|
||||
workspace,
|
||||
):
|
||||
args, _, _ = workspace
|
||||
a, b = stage(workspace, "a"), stage(workspace, "b", "beta")
|
||||
first = run(workspace, [a, b])
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, [a, {**b, "deps": ["a"]}])
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
|
||||
|
||||
|
||||
def test_failed_run_reuses_successful_phase_but_retries_failed_phase(workspace):
|
||||
args, _, traces = workspace
|
||||
ready = traces / "ready"
|
||||
plan = [
|
||||
stage(workspace, "a"),
|
||||
stage(
|
||||
workspace,
|
||||
"b",
|
||||
"beta",
|
||||
after=["a"],
|
||||
body=f"raise SystemExit(0 if Path({str(ready)!r}).exists() else 3)",
|
||||
),
|
||||
]
|
||||
first = run(workspace, plan)
|
||||
assert first["status"] == "failed"
|
||||
assert stages_by_id(first)["a"]["checkpoint_verified"] is True
|
||||
ready.touch()
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
|
||||
|
||||
|
||||
def test_changed_dependency_invalidates_transitive_consumers_but_not_independent_stage(
|
||||
workspace,
|
||||
):
|
||||
args, repos, _ = workspace
|
||||
plan = [
|
||||
stage(workspace, "a"),
|
||||
stage(workspace, "b", "beta", deps=["a"]),
|
||||
stage(workspace, "c", "beta", deps=["b"]),
|
||||
stage(workspace, "independent", "beta"),
|
||||
]
|
||||
first = run(workspace, plan)
|
||||
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert {
|
||||
identity: count(workspace, identity)
|
||||
for identity in ("a", "b", "c", "independent")
|
||||
} == {"a": 2, "b": 2, "c": 2, "independent": 1}
|
||||
|
||||
|
||||
def test_order_only_predecessor_change_does_not_invalidate_independent_stage(workspace):
|
||||
args, repos, _ = workspace
|
||||
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
|
||||
first = run(workspace, plan)
|
||||
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 2 and count(workspace, "b") == 1
|
||||
assert stages_by_id(second)["b"]["reused_from"] == first["run_id"]
|
||||
|
||||
|
||||
def test_order_only_failure_prevents_new_downstream_execution(workspace):
|
||||
plan = [
|
||||
stage(workspace, "a", body="raise SystemExit(8)"),
|
||||
stage(workspace, "b", "beta", after=["a"]),
|
||||
]
|
||||
result = run(workspace, plan)
|
||||
assert result["status"] == "failed"
|
||||
assert count(workspace, "a") == 1 and count(workspace, "b") == 0
|
||||
assert stages_by_id(result)["b"]["status"] == "skipped"
|
||||
|
||||
|
||||
def test_never_reused_stage_and_actual_consumers_rerun_but_order_only_stage_can_reuse(
|
||||
workspace,
|
||||
):
|
||||
args, _, _ = workspace
|
||||
plan = [
|
||||
stage(workspace, "producer", reuse="never"),
|
||||
stage(workspace, "consumer", "beta", deps=["producer"]),
|
||||
stage(workspace, "independent", "beta", after=["producer"]),
|
||||
]
|
||||
first = run(workspace, plan)
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "producer") == 2
|
||||
assert count(workspace, "consumer") == 2
|
||||
assert count(workspace, "independent") == 1
|
||||
|
||||
|
||||
def test_unspecified_input_scope_stays_conservatively_workspace_wide(workspace):
|
||||
args, repos, _ = workspace
|
||||
plan = [stage(workspace, "broad", "beta", inputs=False)]
|
||||
first = run(workspace, plan)
|
||||
(repos["alpha"] / "source.txt").write_text("alpha changed\n")
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "broad") == 2
|
||||
|
||||
|
||||
def test_global_environment_change_invalidates_all_scoped_checkpoints(workspace):
|
||||
args, _, _ = workspace
|
||||
plan = [stage(workspace, "a"), stage(workspace, "b", "beta")]
|
||||
first = run(workspace, plan)
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan, environment="f" * 64)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 2 and count(workspace, "b") == 2
|
||||
assert second["environment_fingerprint"] != first["environment_fingerprint"]
|
||||
|
||||
|
||||
def test_changed_declared_scope_prevents_same_id_reuse(workspace):
|
||||
args, _, _ = workspace
|
||||
selected = stage(workspace, "a")
|
||||
first = run(workspace, [selected])
|
||||
args.resume = first["run_id"]
|
||||
selected = {**selected, "inputs": {"repos": ["alpha", "beta"]}}
|
||||
second = run(workspace, [selected])
|
||||
assert second["status"] == "passed" and count(workspace, "a") == 2
|
||||
|
||||
|
||||
def test_input_mutation_during_passing_command_never_creates_reusable_checkpoint(
|
||||
workspace,
|
||||
):
|
||||
args, repos, _ = workspace
|
||||
plan = [
|
||||
stage(
|
||||
workspace,
|
||||
"mutates",
|
||||
body="Path('source.txt').write_text('mutated during stage\\n')",
|
||||
)
|
||||
]
|
||||
first = run(workspace, plan)
|
||||
assert first["status"] != "passed"
|
||||
assert stages_by_id(first)["mutates"].get("checkpoint_verified") is not True
|
||||
args.resume = first["run_id"]
|
||||
# The same command is now stable against the already-mutated current bytes;
|
||||
# its unverified first result still cannot be skipped.
|
||||
second = run(workspace, plan)
|
||||
assert count(workspace, "mutates") == 2
|
||||
assert second["status"] == "passed"
|
||||
assert (repos["alpha"] / "source.txt").read_text() == "mutated during stage\n"
|
||||
|
||||
|
||||
def test_later_restoration_of_bytes_cannot_turn_invalid_phase_into_overall_pass(
|
||||
workspace,
|
||||
):
|
||||
_, repos, _ = workspace
|
||||
source = repos["alpha"] / "source.txt"
|
||||
original = source.read_text()
|
||||
plan = [
|
||||
stage(
|
||||
workspace,
|
||||
"mutates",
|
||||
body="Path('source.txt').write_text('temporary change\\n')",
|
||||
),
|
||||
stage(
|
||||
workspace,
|
||||
"restores",
|
||||
"beta",
|
||||
after=["mutates"],
|
||||
body=f"Path({str(source)!r}).write_text({original!r})",
|
||||
),
|
||||
]
|
||||
result = run(workspace, plan)
|
||||
assert result["status"] != "passed"
|
||||
assert stages_by_id(result)["mutates"].get("checkpoint_verified") is not True
|
||||
|
||||
|
||||
def test_recovered_interruption_reuses_only_verified_durable_checkpoint(workspace):
|
||||
args, _, _ = workspace
|
||||
plan = [stage(workspace, "a"), stage(workspace, "b", "beta", after=["a"])]
|
||||
first = run(workspace, plan)
|
||||
receipt = read_json(Path(first["receipt_path"]))
|
||||
receipt.update(
|
||||
status="running", phase="checking", snapshot_verified=False, finished_at=None
|
||||
)
|
||||
unfinished = stages_by_id(receipt)["b"]
|
||||
unfinished.update(status="running", exit_code=None, checkpoint_verified=False)
|
||||
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
|
||||
parser = argparse.ArgumentParser()
|
||||
runner.register(parser.add_subparsers(dest="command", required=True))
|
||||
recovered = parser.parse_args(
|
||||
["recover", first["run_id"], "--apply", "--confirm-processes-stopped"],
|
||||
namespace=Namespace(**vars(args)),
|
||||
)
|
||||
recovery = recovered.handler(recovered)
|
||||
assert recovery["status"] == "interrupted"
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 1 and count(workspace, "b") == 2
|
||||
assert stages_by_id(second)["a"]["reused_from"] == first["run_id"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["tampered", "missing"])
|
||||
def test_cached_log_tamper_or_loss_is_not_accepted_as_verified_evidence(
|
||||
workspace, change
|
||||
):
|
||||
args, _, _ = workspace
|
||||
plan = [stage(workspace, "a")]
|
||||
first = run(workspace, plan)
|
||||
path = Path(first["stages"][0]["log_path"])
|
||||
if change == "missing":
|
||||
path.unlink()
|
||||
else:
|
||||
path.write_text("tampered evidence\n")
|
||||
args.resume = first["run_id"]
|
||||
result = run(workspace, plan)
|
||||
assert result["status"] == "failed"
|
||||
assert (
|
||||
result["snapshot_verified"] is not True
|
||||
or result["stages"][0]["status"] == "failed"
|
||||
)
|
||||
assert result["stages"][0].get("checkpoint_verified") is not True
|
||||
assert "reused_from" not in result["stages"][0]
|
||||
assert "error" in result["stages"][0]
|
||||
assert count(workspace, "a") == 1
|
||||
|
||||
|
||||
def test_stale_run_donates_only_checkpoints_matching_final_current_inputs(workspace):
|
||||
args, repos, _ = workspace
|
||||
source = repos["alpha"] / "source.txt"
|
||||
plan = [
|
||||
stage(workspace, "a"),
|
||||
stage(
|
||||
workspace,
|
||||
"changes-alpha",
|
||||
"beta",
|
||||
after=["a"],
|
||||
body=f"Path({str(source)!r}).write_text('new alpha bytes\\n')",
|
||||
),
|
||||
]
|
||||
first = run(workspace, plan)
|
||||
assert first["status"] != "passed"
|
||||
args.resume = first["run_id"]
|
||||
second = run(workspace, plan)
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 2
|
||||
# This order-only phase did not consume alpha and its own scoped inputs are
|
||||
# still identical; it may retain its independently verified checkpoint.
|
||||
assert count(workspace, "changes-alpha") == 1
|
||||
|
||||
|
||||
def test_legacy_checkpointless_receipt_does_not_gain_incremental_authority(workspace):
|
||||
args, repos, _ = workspace
|
||||
plan = [stage(workspace, "a")]
|
||||
first = run(workspace, plan)
|
||||
receipt = read_json(Path(first["receipt_path"]))
|
||||
receipt.pop("fingerprint_version", None)
|
||||
for item in receipt["stages"]:
|
||||
for field in ("checkpoint_version", "checkpoint_verified", "cache_key"):
|
||||
item.pop(field, None)
|
||||
atomic_json(Path(first["receipt_path"]), runner._seal(receipt))
|
||||
(repos["beta"] / "source.txt").write_text("unrelated changed source\n")
|
||||
args.resume = first["run_id"]
|
||||
try:
|
||||
second = run(workspace, plan)
|
||||
except ValueError:
|
||||
return # Rejecting legacy incremental reuse is also safely fail-closed.
|
||||
assert second["status"] == "passed"
|
||||
assert count(workspace, "a") == 2
|
||||
Executable
+393
@@ -0,0 +1,393 @@
|
||||
"""Repository input identities use disposable Git fixtures, never remote effects."""
|
||||
|
||||
from copy import deepcopy
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import inputs
|
||||
from govoplan_devkit.inputs import InputSnapshotter, validate_input_declaration
|
||||
from govoplan_devkit.workspace import Project, Repository, source_fingerprint
|
||||
|
||||
|
||||
def git(repo, *arguments):
|
||||
return subprocess.run(
|
||||
["git", "-C", str(repo), *arguments], capture_output=True, check=True
|
||||
).stdout
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fixture(tmp_path, monkeypatch):
|
||||
repositories = []
|
||||
for name in ("alpha", "beta"):
|
||||
path = tmp_path / name
|
||||
path.mkdir()
|
||||
git(path, "init", "-q")
|
||||
(path / "source.txt").write_text("initial\n")
|
||||
git(path, "add", "source.txt")
|
||||
git(
|
||||
path,
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"fixture",
|
||||
)
|
||||
repositories.append(Repository(name, path, ("alias-" + name,)))
|
||||
project = Project(
|
||||
"Fixture",
|
||||
tuple(repositories),
|
||||
{"tools": {}, "profiles": {"quick": ["one"]}, "checks": []},
|
||||
)
|
||||
# Avoid incidental edits by other agents affecting these source-input tests.
|
||||
# The actual tool-source hashing implementation is checked separately.
|
||||
monkeypatch.setattr(
|
||||
InputSnapshotter, "_tooling_identity", lambda *_: "fixture-devkit-source"
|
||||
)
|
||||
return tmp_path, project, InputSnapshotter(project, workspace_root=tmp_path)
|
||||
|
||||
|
||||
def stage(identity="one", repos=None, **extra):
|
||||
return {
|
||||
"id": identity,
|
||||
"argv": ["true"],
|
||||
**({"inputs": {"repos": repos}} if repos is not None else {}),
|
||||
**extra,
|
||||
}
|
||||
|
||||
|
||||
def fingerprint(snapshot, name="one"):
|
||||
return snapshot["stages"][name]["fingerprint"]
|
||||
|
||||
|
||||
def test_undeclared_inputs_remain_whole_workspace(fixture):
|
||||
root, _, engine = fixture
|
||||
result = engine.snapshot([stage()])
|
||||
assert result["complete_workspace"] is True
|
||||
assert result["observed_scope"]["repos"] == ["alpha", "beta"]
|
||||
assert result["stages"]["one"]["scope"] == {
|
||||
"version": 1,
|
||||
"kind": "workspace",
|
||||
"declared": False,
|
||||
"repos": ["alpha", "beta"],
|
||||
}
|
||||
(root / "beta/source.txt").write_text("changed\n")
|
||||
assert fingerprint(engine.snapshot([stage()])) != fingerprint(result)
|
||||
|
||||
|
||||
def test_scoped_input_does_not_read_unrelated_repository_bytes(fixture, monkeypatch):
|
||||
root, _, engine = fixture
|
||||
original = inputs.os.open
|
||||
|
||||
def guarded(path, *args, **kwargs):
|
||||
if Path(path).is_relative_to(root / "beta"):
|
||||
raise AssertionError("Unrelated repository input was opened")
|
||||
return original(path, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(inputs.os, "open", guarded)
|
||||
first = engine.snapshot([stage(repos=["alpha"])])
|
||||
(root / "beta/source.txt").write_text("unrelated change")
|
||||
second = engine.snapshot([stage(repos=["alpha"])])
|
||||
assert fingerprint(first) == fingerprint(second)
|
||||
assert second["scan_stats"]["repositories"] == 1
|
||||
assert second["scan_stats"]["git_calls"] == 2
|
||||
assert second["complete_workspace"] is False
|
||||
|
||||
|
||||
def test_repository_union_scans_once_per_snapshot(fixture):
|
||||
_, _, engine = fixture
|
||||
result = engine.snapshot(
|
||||
[
|
||||
stage("one", ["alpha"]),
|
||||
stage("two", ["alpha"]),
|
||||
stage("three", ["beta", "alpha"]),
|
||||
]
|
||||
)
|
||||
assert result["scan_stats"]["repositories"] == 2
|
||||
assert result["scan_stats"]["git_calls"] == 4
|
||||
assert result["stages"]["three"]["scope"]["repos"] == ["alpha", "beta"]
|
||||
|
||||
|
||||
def test_session_rechecks_metadata_but_reuses_stable_file_content(fixture):
|
||||
_, _, engine = fixture
|
||||
plan = [stage(repos=["alpha"])]
|
||||
before = engine.snapshot(plan)
|
||||
after = engine.snapshot(plan)
|
||||
assert fingerprint(before) == fingerprint(after)
|
||||
assert before["scan_stats"]["bytes"] == len("initial\n")
|
||||
assert after["scan_stats"]["bytes"] == 0
|
||||
assert after["scan_stats"]["cache_hits"] == 1
|
||||
assert after["scan_stats"]["git_calls"] == 2
|
||||
|
||||
|
||||
def test_changed_bytes_with_restored_mtime_are_not_reused(fixture):
|
||||
root, _, engine = fixture
|
||||
plan = [stage(repos=["alpha"])]
|
||||
before = engine.snapshot(plan)
|
||||
path = root / "alpha/source.txt"
|
||||
metadata = path.stat()
|
||||
path.write_text("changed\n")
|
||||
os.utime(path, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
|
||||
after = engine.snapshot(plan)
|
||||
assert fingerprint(after) != fingerprint(before)
|
||||
assert after["scan_stats"]["bytes"] == len("changed\n")
|
||||
|
||||
|
||||
def test_inode_replacement_with_same_size_and_mtime_is_not_reused(fixture):
|
||||
root, _, engine = fixture
|
||||
plan = [stage(repos=["alpha"])]
|
||||
before = engine.snapshot(plan)
|
||||
path = root / "alpha/source.txt"
|
||||
metadata = path.stat()
|
||||
replacement = root / "replacement.txt"
|
||||
replacement.write_text("changed\n")
|
||||
os.utime(replacement, ns=(metadata.st_atime_ns, metadata.st_mtime_ns))
|
||||
replacement.replace(path)
|
||||
after = engine.snapshot(plan)
|
||||
assert fingerprint(before) != fingerprint(after)
|
||||
assert after["scan_stats"]["cache_hits"] == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize("flag", ["assume-unchanged", "skip-worktree"])
|
||||
def test_hidden_index_flags_do_not_hide_worktree_changes(fixture, flag):
|
||||
root, _, engine = fixture
|
||||
git(root / "alpha", "update-index", "--" + flag, "source.txt")
|
||||
before = engine.source_snapshot(["alpha"])
|
||||
(root / "alpha/source.txt").write_text("hidden change\n")
|
||||
after = engine.source_snapshot(["alpha"])
|
||||
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
|
||||
|
||||
|
||||
def test_index_head_and_new_deleted_files_are_bound(fixture):
|
||||
root, _, engine = fixture
|
||||
repo = root / "alpha"
|
||||
identities = []
|
||||
|
||||
def record():
|
||||
identities.append(
|
||||
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
|
||||
)
|
||||
|
||||
record()
|
||||
(repo / "source.txt").write_text("changed\n")
|
||||
record()
|
||||
git(repo, "add", "source.txt")
|
||||
record()
|
||||
git(
|
||||
repo,
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"change",
|
||||
)
|
||||
record()
|
||||
(repo / "new.txt").write_text("new")
|
||||
record()
|
||||
(repo / "source.txt").unlink()
|
||||
record()
|
||||
assert len(set(identities)) == len(identities)
|
||||
|
||||
|
||||
def test_missing_registered_repository_becoming_present_invalidates(fixture):
|
||||
root, project, _ = fixture
|
||||
extra = Repository("missing", root / "missing")
|
||||
extended = Project("Fixture", (*project.repositories, extra), project.config)
|
||||
engine = InputSnapshotter(extended, workspace_root=root)
|
||||
before = engine.source_snapshot(["missing"])
|
||||
extra.path.mkdir()
|
||||
git(extra.path, "init", "-q")
|
||||
after = engine.source_snapshot(["missing"])
|
||||
assert before["observed_source_fingerprint"] != after["observed_source_fingerprint"]
|
||||
|
||||
|
||||
def test_unrelated_plan_config_edits_do_not_invalidate_scoped_stage(fixture):
|
||||
root, project, engine = fixture
|
||||
plan = [stage(repos=["alpha"])]
|
||||
before = engine.snapshot(plan)
|
||||
updated = deepcopy(project.config)
|
||||
updated["profiles"]["full"] = ["unrelated"]
|
||||
updated["checks"].append({"id": "unrelated", "argv": ["false"]})
|
||||
other = InputSnapshotter(
|
||||
Project("Changed label", project.repositories, updated), workspace_root=root
|
||||
)
|
||||
assert fingerprint(other.snapshot(plan)) == fingerprint(before)
|
||||
|
||||
|
||||
def test_scope_command_tools_and_tooling_change_invalidate(fixture):
|
||||
root, project, engine = fixture
|
||||
first = engine.snapshot([stage(repos=["alpha"])])
|
||||
assert fingerprint(engine.snapshot([stage(repos=["beta"])])) != fingerprint(first)
|
||||
assert fingerprint(
|
||||
engine.snapshot([stage(repos=["alpha"], argv=["false"])])
|
||||
) != fingerprint(first)
|
||||
assert fingerprint(
|
||||
engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="new-env")
|
||||
) != fingerprint(first)
|
||||
changed = deepcopy(project.config)
|
||||
changed["tools"] = {"node": "/another/node"}
|
||||
other = InputSnapshotter(
|
||||
Project("Fixture", project.repositories, changed), workspace_root=root
|
||||
)
|
||||
assert fingerprint(other.snapshot([stage(repos=["alpha"])])) != fingerprint(first)
|
||||
|
||||
|
||||
def test_source_only_attestation_is_independent_of_plan_and_tooling(fixture):
|
||||
_, _, engine = fixture
|
||||
one = engine.snapshot([stage(repos=["alpha"])], tooling_fingerprint="env-one")
|
||||
two = engine.snapshot(
|
||||
[stage("different", ["alpha"], argv=["false"])], tooling_fingerprint="env-two"
|
||||
)
|
||||
source = engine.source_snapshot(["alpha"])
|
||||
assert (
|
||||
one["observed_source_fingerprint"]
|
||||
== two["observed_source_fingerprint"]
|
||||
== source["observed_source_fingerprint"]
|
||||
)
|
||||
assert (
|
||||
one["stages"]["one"]["source_fingerprint"]
|
||||
== source["observed_source_fingerprint"]
|
||||
)
|
||||
assert source["fingerprint_version"] == inputs.FINGERPRINT_VERSION
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
[
|
||||
None,
|
||||
{},
|
||||
{"paths": ["src/**"]},
|
||||
{"repos": []},
|
||||
{"repos": "alpha"},
|
||||
{"repos": ["unknown"]},
|
||||
{"repos": ["alias-alpha"]},
|
||||
{"repos": ["alpha", "alpha"]},
|
||||
{"repos": [""]},
|
||||
{"repos": ["alpha"], "extra": True},
|
||||
],
|
||||
)
|
||||
def test_synthetic_stage_scopes_fail_closed(fixture, value):
|
||||
_, _, engine = fixture
|
||||
with pytest.raises(ValueError):
|
||||
engine.snapshot([stage(inputs=value)])
|
||||
|
||||
|
||||
def test_duplicate_stage_ids_and_escaped_repository_paths_fail(fixture):
|
||||
root, project, engine = fixture
|
||||
with pytest.raises(ValueError, match="Duplicate"):
|
||||
engine.snapshot([stage(), stage()])
|
||||
bad = Project("Bad", (Repository("outside", root.parent),), project.config)
|
||||
with pytest.raises(ValueError, match="escapes"):
|
||||
InputSnapshotter(bad, workspace_root=root)
|
||||
|
||||
|
||||
def test_declaration_validation_is_pure_before_dry_run(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
inputs,
|
||||
"git_bytes",
|
||||
lambda *_a, **_k: pytest.fail("Planning validation ran Git"),
|
||||
)
|
||||
assert validate_input_declaration(
|
||||
{"repos": ["beta", "alpha"]}, {"alpha", "beta"}
|
||||
) == {"repos": ["alpha", "beta"]}
|
||||
with pytest.raises(ValueError):
|
||||
validate_input_declaration({"repos": ["unknown"]}, {"alpha", "beta"})
|
||||
|
||||
|
||||
def test_runtime_records_are_not_hashed_as_execution_plans(fixture):
|
||||
_, _, engine = fixture
|
||||
with pytest.raises(ValueError, match="freshly planned"):
|
||||
engine.snapshot([stage(repos=["alpha"], status="passed")])
|
||||
|
||||
|
||||
def test_membership_count_is_bounded_before_entry_hashing(fixture, monkeypatch):
|
||||
root, _, engine = fixture
|
||||
(root / "alpha/new.txt").write_text("extra")
|
||||
monkeypatch.setattr(inputs, "MAX_REPOSITORY_ENTRIES", 1)
|
||||
with pytest.raises(ValueError, match="entry count"):
|
||||
engine.source_snapshot(["alpha"])
|
||||
|
||||
|
||||
def test_symlink_to_ignored_file_inside_repository_binds_target(fixture):
|
||||
root, _, engine = fixture
|
||||
repo = root / "alpha"
|
||||
(repo / ".gitignore").write_text("ignored.txt\n")
|
||||
(repo / "ignored.txt").write_text("first")
|
||||
(repo / "linked.txt").symlink_to("ignored.txt")
|
||||
before = engine.source_snapshot(["alpha"])
|
||||
(repo / "ignored.txt").write_text("other")
|
||||
assert (
|
||||
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
|
||||
!= before["observed_source_fingerprint"]
|
||||
)
|
||||
|
||||
|
||||
def test_dangling_symlink_target_creation_is_observed(fixture):
|
||||
root, _, engine = fixture
|
||||
repo = root / "alpha"
|
||||
(repo / "linked.txt").symlink_to("future.txt")
|
||||
before = engine.source_snapshot(["alpha"])
|
||||
(repo / "future.txt").write_text("created")
|
||||
assert (
|
||||
engine.source_snapshot(["alpha"])["observed_source_fingerprint"]
|
||||
!= before["observed_source_fingerprint"]
|
||||
)
|
||||
|
||||
|
||||
def test_cross_repository_and_directory_symlinks_are_not_silently_reused(fixture):
|
||||
root, _, engine = fixture
|
||||
link = root / "alpha/linked.txt"
|
||||
link.symlink_to(root / "beta/source.txt")
|
||||
with pytest.raises(ValueError, match="escapes"):
|
||||
engine.source_snapshot(["alpha"])
|
||||
link.unlink()
|
||||
link.symlink_to(".")
|
||||
with pytest.raises(ValueError, match="regular file"):
|
||||
engine.source_snapshot(["alpha"])
|
||||
|
||||
|
||||
def test_file_change_during_hash_is_rejected(fixture, monkeypatch):
|
||||
root, _, engine = fixture
|
||||
path = root / "alpha/source.txt"
|
||||
original = inputs.os.fstat
|
||||
changed = False
|
||||
|
||||
def mutate(descriptor):
|
||||
nonlocal changed
|
||||
metadata = original(descriptor)
|
||||
if not changed:
|
||||
changed = True
|
||||
path.write_text("changed during read")
|
||||
return metadata
|
||||
|
||||
monkeypatch.setattr(inputs.os, "fstat", mutate)
|
||||
with pytest.raises(ValueError, match="changed"):
|
||||
engine._file_hash(path, inputs._stats())
|
||||
|
||||
|
||||
def test_legacy_whole_project_api_remains_independent(fixture):
|
||||
_, project, engine = fixture
|
||||
legacy = source_fingerprint(project)
|
||||
engine.snapshot([stage(repos=["alpha"])])
|
||||
assert source_fingerprint(project) == legacy
|
||||
|
||||
|
||||
def test_real_tooling_inventory_is_bound_and_memoized(tmp_path):
|
||||
repo = Repository("example", tmp_path)
|
||||
project = Project("Fixture", (repo,), {"tools": {}})
|
||||
engine = InputSnapshotter(project, workspace_root=tmp_path)
|
||||
before_stats, after_stats = inputs._stats(), inputs._stats()
|
||||
before = engine._tooling_identity(before_stats)
|
||||
after = engine._tooling_identity(after_stats)
|
||||
assert before == after
|
||||
assert before_stats["tooling_files"] > 15 and before_stats["tooling_bytes"] > 0
|
||||
assert after_stats["tooling_files"] == after_stats["tooling_cache_hits"]
|
||||
assert after_stats["tooling_bytes"] == 0
|
||||
Executable
+387
@@ -0,0 +1,387 @@
|
||||
import argparse
|
||||
from copy import deepcopy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import issues
|
||||
from govoplan_devkit.common import atomic_json, digest, state_root
|
||||
from govoplan_devkit.workspace import load_project, source_fingerprint
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def args(tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("GITEA_TOKEN", raising=False)
|
||||
root = tmp_path / "repo"
|
||||
root.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(root)], check=True)
|
||||
subprocess.run(["git", "-C", str(root), "remote", "add", "origin", "https://gitea.invalid/team/repo.git"], check=True)
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(json.dumps({"schema_version": 1, "name": "Fixture", "repositories": [{"name": "repo", "path": "repo"}]}))
|
||||
env_file = tmp_path / "private.env"
|
||||
env_file.write_text("GITEA_TOKEN=fixture-private-token\nGITEA_OWNER=wrong-owner\n")
|
||||
return argparse.Namespace(workspace_root=tmp_path, state_dir=tmp_path / "state", project=project,
|
||||
root=root, issue=7, target_plan=None, remote="origin", env_file=env_file, evidence=None,
|
||||
key="verification", note_summary=["Verified scoped changes."], next_steps=["Manual review remains."],
|
||||
body_file=None, note_file=None, apply=False, retry_uncertain=False)
|
||||
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, target):
|
||||
self.target = target
|
||||
self.calls = []
|
||||
self.comments = []
|
||||
self.issue_id = 700
|
||||
self.post_mode = "normal"
|
||||
self.bad_binding = False
|
||||
self.repeat_pages = False
|
||||
self.bad_issue = False
|
||||
self.fail = False
|
||||
self.page_size = 2 # A server may enforce a cap smaller than the requested limit.
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
def request_json(self, method, path, body=None, query=None):
|
||||
self.calls.append((method, path, deepcopy(body), deepcopy(query)))
|
||||
if self.fail:
|
||||
raise RuntimeError("Remote echoed fixture-private-token")
|
||||
if method == "GET" and path == self.target.path:
|
||||
return {"id": self.issue_id, "number": self.target.issue, "html_url": self.target.url + ("wrong" if self.bad_issue else ""),
|
||||
"state": "closed", "body": "- [ ] Preserve this checklist"}
|
||||
if method == "GET" and path == self.target.path + "/comments":
|
||||
page = 1 if self.repeat_pages else query["page"]
|
||||
return deepcopy(self.comments[(page - 1) * self.page_size:page * self.page_size])
|
||||
if method == "GET" and "/issues/comments/" in path:
|
||||
comment = deepcopy(next(item for item in self.comments if str(item["id"]) == path.rsplit("/", 1)[1]))
|
||||
comment["html_url"] = self.target.url + ("-other" if self.bad_binding else "") + "#issuecomment-" + str(comment["id"])
|
||||
return comment
|
||||
if method == "POST" and path == self.target.path + "/comments":
|
||||
if self.post_mode == "timeout-before-commit":
|
||||
raise TimeoutError("fixture-private-token")
|
||||
comment = {"id": 1000 + len(self.comments), "body": body["body"]}
|
||||
self.comments.append(comment)
|
||||
if self.post_mode == "timeout-after-commit":
|
||||
raise TimeoutError("fixture-private-token")
|
||||
return deepcopy(comment)
|
||||
raise AssertionError((method, path))
|
||||
|
||||
@property
|
||||
def posts(self):
|
||||
return [call for call in self.calls if call[0] == "POST"]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(args, monkeypatch):
|
||||
client = FakeClient(issues.resolve_target(args.root, args.issue, args.workspace_root))
|
||||
monkeypatch.setattr(issues, "make_client", lambda target, token: client)
|
||||
return client
|
||||
|
||||
|
||||
def test_default_dry_run_is_offline_and_ignores_ambient_target_overrides(args, monkeypatch):
|
||||
monkeypatch.setenv("GITEA_OWNER", "wrong")
|
||||
monkeypatch.setenv("GITEA_REPO", "wrong")
|
||||
monkeypatch.setenv("GITEA_URL", "https://wrong.invalid")
|
||||
monkeypatch.setattr(issues, "make_client", lambda *_: pytest.fail("No network in preview"))
|
||||
args.env_file = args.workspace_root / "does-not-exist.env"
|
||||
result = issues.handle_note(args)
|
||||
assert result["targets"][0]["url"] == "https://gitea.invalid/team/repo/issues/7"
|
||||
assert result["targets"][0]["status"] == "would-post"
|
||||
assert not args.state_dir.exists()
|
||||
|
||||
|
||||
def test_scoped_checkpoint_evidence_compares_only_recorded_sources(args, monkeypatch):
|
||||
from govoplan_devkit import runner
|
||||
|
||||
other = args.workspace_root / "other"
|
||||
other.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(other)], check=True)
|
||||
declaration = json.loads(args.project.read_text())
|
||||
declaration["repositories"].append({"name": "other", "path": "other"})
|
||||
args.project.write_text(json.dumps(declaration))
|
||||
monkeypatch.setattr(runner, "environment_fingerprint", lambda *_: "fixture-env")
|
||||
run_args = argparse.Namespace(**{**vars(args), "jobs": 1, "profile": "quick", "dry_run": False})
|
||||
result = runner.run_checks(run_args, [{"id": "scoped", "argv": [sys.executable, "-c", "print('ok')"],
|
||||
"cwd": str(args.root), "inputs": {"repos": ["repo"]}}])
|
||||
assert result["status"] == "passed"
|
||||
evidence = issues.evidence_record(result["run_id"], args)
|
||||
assert evidence["source_state"] == "matches-current"
|
||||
assert evidence["source_scope"]["repos"] == ["repo"]
|
||||
assert any("recorded repository input scope" in note for note in evidence["coverage_notes"])
|
||||
(other / "unrelated.txt").write_text("Outside the recorded scope")
|
||||
assert issues.evidence_record(result["run_id"], args)["source_state"] == "matches-current"
|
||||
(args.root / "changed.txt").write_text("Inside the recorded scope")
|
||||
assert issues.evidence_record(result["run_id"], args)["source_state"] == "historical-source-differs"
|
||||
|
||||
|
||||
def test_passing_aggregate_cannot_hide_skipped_checks(args):
|
||||
payload = receipt(args)
|
||||
payload["stages"].append({"id": "skipped", "status": "skipped", "exit_code": None})
|
||||
with pytest.raises(ValueError, match="inconsistent"):
|
||||
issues.validate_receipt(payload, args.workspace_root)
|
||||
|
||||
|
||||
def test_apply_is_append_only_read_back_and_idempotent(args, client):
|
||||
args.apply = True
|
||||
result = issues.handle_note(args)
|
||||
assert result["targets"][0]["status"] == "posted-verified"
|
||||
assert len(client.posts) == 1
|
||||
again = issues.handle_note(args)
|
||||
assert again["targets"][0]["status"] == "existing-verified"
|
||||
assert len(client.posts) == 1
|
||||
assert {call[0] for call in client.calls} == {"GET", "POST"}
|
||||
assert all(call[1].endswith("/comments") for call in client.posts)
|
||||
|
||||
|
||||
def test_complete_pagination_continues_past_short_pages(args, client):
|
||||
body = issues.handle_note(args)["targets"][0]["body"]
|
||||
client.comments = [{"id": n, "body": "unrelated"} for n in range(1, 6)] + [{"id": 6, "body": body}]
|
||||
args.apply = True
|
||||
result = issues.handle_note(args)
|
||||
assert result["targets"][0]["status"] == "existing-verified"
|
||||
assert not client.posts
|
||||
assert max(call[3]["page"] for call in client.calls if call[3]) == 4
|
||||
|
||||
|
||||
@pytest.mark.parametrize("collision", ["different-body", "duplicate", "repeating-pagination"])
|
||||
def test_collisions_and_incomplete_pagination_refuse_post(args, client, collision):
|
||||
body = issues.handle_note(args)["targets"][0]["body"]
|
||||
client.comments = [{"id": 1, "body": body}]
|
||||
if collision == "different-body":
|
||||
client.comments[0]["body"] += "changed"
|
||||
elif collision == "duplicate":
|
||||
client.comments.append({"id": 2, "body": body})
|
||||
else:
|
||||
client.repeat_pages = True
|
||||
args.apply = True
|
||||
assert issues.handle_note(args)["_exit_code"] == 2
|
||||
assert not client.posts
|
||||
|
||||
|
||||
def test_timeout_after_commit_reconciles_without_replaying_post(args, client):
|
||||
args.apply = True
|
||||
client.post_mode = "timeout-after-commit"
|
||||
result = issues.handle_note(args)
|
||||
assert result["targets"][0]["status"] == "reconciled-verified"
|
||||
assert len(client.posts) == 1
|
||||
assert "fixture-private-token" not in json.dumps(result)
|
||||
|
||||
|
||||
def test_uncertain_post_requires_explicit_retry_after_reconciliation(args, client):
|
||||
args.apply = True
|
||||
client.post_mode = "timeout-before-commit"
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
|
||||
client.post_mode = "normal"
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain-retry-required"
|
||||
assert len(client.posts) == 1
|
||||
args.retry_uncertain = True
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
|
||||
assert len(client.posts) == 2
|
||||
journals = list(state_root(args.workspace_root, args.state_dir).glob("issue-notes/*.json"))
|
||||
assert journals and all("fixture-private-token" not in path.read_text() for path in journals)
|
||||
|
||||
|
||||
def test_uncertain_readback_is_not_reported_verified_and_later_reconciles(args, client):
|
||||
args.apply = True
|
||||
client.bad_binding = True
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "uncertain"
|
||||
client.bad_binding = False
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "existing-verified"
|
||||
assert len(client.posts) == 1
|
||||
|
||||
|
||||
def test_journal_binds_immutable_issue_id(args, client):
|
||||
args.apply = True
|
||||
assert issues.handle_note(args)["targets"][0]["status"] == "posted-verified"
|
||||
client.issue_id += 1
|
||||
client.comments.clear()
|
||||
args.retry_uncertain = True
|
||||
assert issues.handle_note(args)["_exit_code"] == 2
|
||||
assert len(client.posts) == 1
|
||||
|
||||
|
||||
def _plan(args, rows):
|
||||
args.root = args.issue = None
|
||||
args.target_plan = args.workspace_root / "targets.json"
|
||||
args.target_plan.write_text(json.dumps({"schema_version": 1, "targets": rows}))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["wrong-url", "duplicate", "outside", "mixed-origin", "credentials"])
|
||||
def test_target_plans_require_exact_unique_workspace_bindings(args, change):
|
||||
row = {"root": "repo", "issue": 7, "url": "https://gitea.invalid/team/repo/issues/7"}
|
||||
if change == "wrong-url":
|
||||
row["url"] = "https://gitea.invalid/team/other/issues/7"
|
||||
elif change == "outside":
|
||||
row["root"] = "../other"
|
||||
elif change == "credentials":
|
||||
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", "https://user:private@gitea.invalid/team/repo.git"], check=True)
|
||||
rows = [row, deepcopy(row)] if change == "duplicate" else [row]
|
||||
if change == "mixed-origin":
|
||||
other = args.workspace_root / "other"
|
||||
other.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(other)], check=True)
|
||||
subprocess.run(["git", "-C", str(other), "remote", "add", "origin", "http://gitea.invalid/team/other.git"], check=True)
|
||||
rows.append({"root": "other", "issue": 8, "url": "http://gitea.invalid/team/other/issues/8"})
|
||||
_plan(args, rows)
|
||||
with pytest.raises(ValueError):
|
||||
issues.handle_note(args)
|
||||
|
||||
|
||||
def test_all_targets_preflight_before_first_post_and_posts_are_serial(args, client, monkeypatch):
|
||||
first = client.target.record()
|
||||
second = {**first, "issue": 8, "url": first["url"].rsplit("/", 1)[0] + "/8"}
|
||||
_plan(args, [first, second])
|
||||
second_client = FakeClient(issues.resolve_target(Path(first["root"]), 8, args.workspace_root))
|
||||
second_client.bad_issue = True
|
||||
monkeypatch.setattr(issues, "make_client", lambda target, token: client if target.issue == 7 else second_client)
|
||||
args.apply = True
|
||||
assert issues.handle_note(args)["_exit_code"] == 2
|
||||
assert not client.posts and not second_client.posts
|
||||
second_client.bad_issue = False
|
||||
result = issues.handle_note(args)
|
||||
assert [row["status"] for row in result["targets"]] == ["posted-verified", "posted-verified"]
|
||||
assert len(client.posts) == len(second_client.posts) == 1
|
||||
|
||||
|
||||
def test_credentials_and_response_errors_are_not_returned(args, client):
|
||||
args.apply = True
|
||||
client.fail = True
|
||||
result = issues.handle_note(args)
|
||||
assert result["_exit_code"] == 2
|
||||
assert "fixture-private-token" not in json.dumps(result)
|
||||
args.note_summary = ["fixture-private-token"]
|
||||
with pytest.raises(ValueError, match="credential"):
|
||||
issues.handle_note(args)
|
||||
|
||||
|
||||
def test_marker_injection_and_symlink_inputs_are_rejected(args):
|
||||
args.note_summary = [issues.MARKER_PREFIX + "fake -->"]
|
||||
with pytest.raises(ValueError, match="reserved"):
|
||||
issues.handle_note(args)
|
||||
args.note_summary = ["safe"]
|
||||
args.body_file = args.workspace_root / "alias.md"
|
||||
args.body_file.symlink_to(args.env_file)
|
||||
with pytest.raises(ValueError, match="symlink"):
|
||||
issues.handle_note(args)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("remote", ["https://gitea.invalid/team/repo.git?token=secret", "https://gitea.invalid/team/repo.git#other", "https://gitea.invalid:bad/team/repo.git"])
|
||||
def test_ambiguous_git_remote_is_not_silently_reinterpreted(args, remote):
|
||||
subprocess.run(["git", "-C", str(args.root), "remote", "set-url", "origin", remote], check=True)
|
||||
with pytest.raises(ValueError):
|
||||
issues.handle_note(args)
|
||||
|
||||
|
||||
def test_structured_inputs_are_merged_as_data_not_executed(args):
|
||||
args.note_file = args.workspace_root / "note.json"
|
||||
args.note_file.write_text(json.dumps({"summary": ["Recorded earlier"], "next": ["Still pending"], "body": "$(never-execute)"}))
|
||||
args.body_file = args.workspace_root / "body.md"
|
||||
args.body_file.write_text("`never-run-this-either`")
|
||||
body = issues.handle_note(args)["targets"][0]["body"]
|
||||
assert "Recorded earlier" in body and "Verified scoped changes" in body
|
||||
assert "$(never-execute)" in body and "`never-run-this-either`" in body
|
||||
|
||||
|
||||
def receipt(args):
|
||||
return {"schema_version": 1, "run_id": "fixture-run", "workspace_root": str(args.workspace_root),
|
||||
"project_file": str(args.project), "source_fingerprint": source_fingerprint(load_project(args.workspace_root, args.project)),
|
||||
"status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08T12:00:00Z", "finished_at": "2026-09-08T12:00:01Z",
|
||||
"stages": [{"id": "fixture", "status": "passed", "exit_code": 0, "duration_seconds": 1,
|
||||
"log_path": "/private/log-not-opened", "argv": ["never-execute-this"]}]}
|
||||
|
||||
|
||||
def test_external_receipts_are_unverified_metadata_with_source_comparison(args):
|
||||
path = args.workspace_root / "external.json"
|
||||
path.write_text(json.dumps(receipt(args)))
|
||||
args.evidence = str(path)
|
||||
result = issues.handle_note(args)
|
||||
evidence = result["evidence"]
|
||||
assert evidence["origin"] == "external-unverified"
|
||||
assert evidence["source_state"] == "matches-current"
|
||||
assert "argv" not in evidence["stages"][0]
|
||||
(args.root / "dirty.txt").write_text("changed")
|
||||
assert issues.handle_note(args)["evidence"]["source_state"] == "historical-source-differs"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mutation", ["foreign", "bad-fingerprint", "false-pass", "duplicate-stage", "unknown-status", "noninteger-exit", "invalid-state-shape", "invalid-stage-state-shape", "bool-schema"])
|
||||
def test_invalid_receipt_cannot_supply_evidence(args, mutation):
|
||||
payload = receipt(args)
|
||||
if mutation == "foreign":
|
||||
payload["workspace_root"] = str(args.workspace_root.parent)
|
||||
elif mutation == "bad-fingerprint":
|
||||
payload["source_fingerprint"] = "claimed-green"
|
||||
elif mutation == "false-pass":
|
||||
payload["stages"][0]["exit_code"] = 1
|
||||
elif mutation == "duplicate-stage":
|
||||
payload["stages"] *= 2
|
||||
elif mutation == "unknown-status":
|
||||
payload["status"] = "complete-review"
|
||||
elif mutation == "invalid-state-shape":
|
||||
payload["status"] = {}
|
||||
elif mutation == "invalid-stage-state-shape":
|
||||
payload["stages"][0]["status"] = []
|
||||
elif mutation == "bool-schema":
|
||||
payload["schema_version"] = True
|
||||
else:
|
||||
payload["stages"][0]["exit_code"] = False
|
||||
with pytest.raises(ValueError):
|
||||
issues.validate_receipt(payload, args.workspace_root)
|
||||
|
||||
|
||||
def test_local_receipt_integrity_is_checked_but_not_an_attestation(args):
|
||||
payload = receipt(args)
|
||||
payload["integrity_sha256"] = digest(payload)
|
||||
path = state_root(args.workspace_root, args.state_dir) / "runs/fixture-run/receipt.json"
|
||||
atomic_json(path, payload)
|
||||
args.evidence = "fixture-run"
|
||||
result = issues.handle_note(args)
|
||||
assert result["evidence"]["origin"] == "local-integrity-checked"
|
||||
assert "not an independent attestation" in result["evidence"]["attestation"]
|
||||
payload["status"] = "failed"
|
||||
atomic_json(path, payload)
|
||||
with pytest.raises(ValueError, match="integrity"):
|
||||
issues.handle_note(args)
|
||||
|
||||
|
||||
def test_timed_out_stage_is_reportable_without_claiming_success(args):
|
||||
payload = receipt(args)
|
||||
payload["status"] = "failed"
|
||||
payload["stages"][0].update(status="timed_out", exit_code=-15)
|
||||
assert issues.validate_receipt(payload, args.workspace_root)["status"] == "failed"
|
||||
|
||||
|
||||
def test_scoped_coverage_limits_are_preserved_in_evidence_and_rendered_note(args, monkeypatch):
|
||||
payload = receipt(args)
|
||||
monkeypatch.setenv("FIXTURE_SECRET", "fixture-secret-value")
|
||||
payload["stages"][0]["coverage_notes"] = ["Compiler/chained suite not run.", "Manual <script>not executed</script> fixture-secret-value"]
|
||||
path = args.workspace_root / "coverage.json"
|
||||
path.write_text(json.dumps(payload))
|
||||
args.evidence = str(path)
|
||||
result = issues.handle_note(args)
|
||||
evidence = result["evidence"]
|
||||
assert evidence["coverage_notes"] == evidence["stages"][0]["coverage_notes"]
|
||||
assert "Compiler/chained suite not run." in result["targets"][0]["body"]
|
||||
assert "omitted checks ran" in result["targets"][0]["body"]
|
||||
assert "<script>" not in result["targets"][0]["body"]
|
||||
assert "fixture-secret-value" not in json.dumps(result)
|
||||
assert any("coverage limitation" in line for line in result["summary"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("notes", ["not-a-list", [None], [""], ["x" * 4097], ["limit"] * 2049, [issues.MARKER_PREFIX + "injection"]])
|
||||
def test_malformed_coverage_metadata_is_rejected(args, notes):
|
||||
payload = receipt(args)
|
||||
payload["stages"][0]["coverage_notes"] = notes
|
||||
with pytest.raises(ValueError):
|
||||
issues.validate_receipt(payload, args.workspace_root)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("snapshot_verified", [None, False, "true", 1])
|
||||
def test_external_pass_requires_verified_snapshot_flag(args, snapshot_verified):
|
||||
payload = receipt(args)
|
||||
payload["snapshot_verified"] = snapshot_verified
|
||||
with pytest.raises(ValueError, match="verified source snapshot"):
|
||||
issues.validate_receipt(payload, args.workspace_root)
|
||||
Executable
+539
@@ -0,0 +1,539 @@
|
||||
"""Git maintenance runs only in disposable local repositories/bare fixtures."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from contextlib import contextmanager
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import maintenance
|
||||
|
||||
|
||||
class MaintenanceTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix="govoplan-devkit-git-")
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.repo = self.root / "repo"
|
||||
self.repo.mkdir()
|
||||
self.git("init", "-b", "main")
|
||||
self.git("config", "user.name", "Fixture")
|
||||
self.git("config", "user.email", "fixture@example.invalid")
|
||||
self.git("config", "commit.gpgsign", "false")
|
||||
self.write("selected.txt", "base selected\n")
|
||||
self.write("unrelated.txt", "base unrelated\n")
|
||||
self.git("add", "--", "selected.txt", "unrelated.txt")
|
||||
self.git("commit", "-m", "fixture base")
|
||||
self.base = self.git("rev-parse", "HEAD").strip()
|
||||
self.project = self.root / "project.json"
|
||||
self.project.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "fixture",
|
||||
"repositories": [{"name": "repo", "path": "repo"}],
|
||||
}
|
||||
)
|
||||
)
|
||||
self.args = argparse.Namespace(
|
||||
workspace_root=self.root,
|
||||
state_dir=self.root / "state",
|
||||
project=self.project,
|
||||
repo="repo",
|
||||
path=["selected.txt"],
|
||||
message="selected change",
|
||||
apply=False,
|
||||
)
|
||||
|
||||
def git(self, *args):
|
||||
return subprocess.run(
|
||||
["git", "-C", str(self.repo), *args],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout
|
||||
|
||||
def write(self, name, value):
|
||||
(self.repo / name).write_text(value)
|
||||
|
||||
def save_plan(self):
|
||||
self.args.apply = True
|
||||
result = maintenance.plan(self.args)
|
||||
self.args.plan_id = result["plan"]["plan_id"]
|
||||
return result
|
||||
|
||||
def test_preview_does_not_write_a_plan_or_stage(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
before = self.git("ls-files", "--stage")
|
||||
result = maintenance.plan(self.args)
|
||||
self.assertEqual(result["state"]["status"], "preview")
|
||||
self.assertFalse((self.root / "state").exists())
|
||||
self.assertEqual(self.git("ls-files", "--stage"), before)
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
|
||||
|
||||
def test_selected_commit_preserves_unrelated_staged_and_dirty_changes(self):
|
||||
self.write("selected.txt", "selected final\n")
|
||||
self.write("unrelated.txt", "unrelated staged\n")
|
||||
self.git("add", "--", "unrelated.txt")
|
||||
self.write("unrelated.txt", "unrelated unstaged\n")
|
||||
staged = self.git("show", ":unrelated.txt")
|
||||
self.save_plan()
|
||||
before_plan = (
|
||||
maintenance._receipt_path(self.args, self.args.plan_id)
|
||||
.joinpath("plan.json")
|
||||
.read_bytes()
|
||||
)
|
||||
self.args.apply = False
|
||||
self.assertEqual(maintenance.commit(self.args)["state"]["status"], "preview")
|
||||
self.args.apply = True
|
||||
result = maintenance.commit(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertEqual(
|
||||
self.git(
|
||||
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
|
||||
).strip(),
|
||||
"selected.txt",
|
||||
)
|
||||
self.assertEqual(self.git("show", ":unrelated.txt"), staged)
|
||||
self.assertEqual(
|
||||
(self.repo / "unrelated.txt").read_text(), "unrelated unstaged\n"
|
||||
)
|
||||
self.assertEqual(
|
||||
maintenance.commit(self.args)["state"]["commit"], result["state"]["commit"]
|
||||
)
|
||||
self.assertEqual(
|
||||
maintenance._receipt_path(self.args, self.args.plan_id)
|
||||
.joinpath("plan.json")
|
||||
.read_bytes(),
|
||||
before_plan,
|
||||
)
|
||||
|
||||
def test_new_selected_file_and_deletion_are_supported(self):
|
||||
self.write("new file.txt", "new\n")
|
||||
(self.repo / "selected.txt").unlink()
|
||||
self.git("add", "--", "selected.txt")
|
||||
self.args.path = ["new file.txt", "selected.txt"]
|
||||
self.save_plan()
|
||||
result = maintenance.commit(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertEqual(
|
||||
set(
|
||||
self.git(
|
||||
"diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"
|
||||
).splitlines()
|
||||
),
|
||||
{"new file.txt", "selected.txt"},
|
||||
)
|
||||
|
||||
def test_selected_partial_staging_is_not_overwritten(self):
|
||||
self.write("selected.txt", "staged\n")
|
||||
self.git("add", "--", "selected.txt")
|
||||
self.write("selected.txt", "unstaged\n")
|
||||
with self.assertRaisesRegex(ValueError, "different staged"):
|
||||
self.save_plan()
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "staged\n")
|
||||
|
||||
def test_stale_content_index_and_origin_block_commit(self):
|
||||
for change in ("content", "index", "origin"):
|
||||
with self.subTest(change=change):
|
||||
self.write("selected.txt", f"selected {change}\n")
|
||||
self.save_plan()
|
||||
if change == "content":
|
||||
self.write("selected.txt", "different content\n")
|
||||
elif change == "index":
|
||||
self.write("unrelated.txt", "changed index\n")
|
||||
self.git("add", "--", "unrelated.txt")
|
||||
else:
|
||||
self.git("remote", "add", "origin", str(self.root / "other.git"))
|
||||
with self.assertRaisesRegex(ValueError, "changed; prepare a new plan"):
|
||||
maintenance.commit(self.args)
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
|
||||
|
||||
def test_active_hooks_and_filters_are_refused_not_bypassed(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
hook = self.repo / ".git/hooks/pre-commit"
|
||||
hook.write_text("#!/bin/sh\nexit 99\n")
|
||||
hook.chmod(0o700)
|
||||
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
|
||||
self.save_plan()
|
||||
hook.unlink()
|
||||
self.git("config", "filter.example.clean", "some-external-command")
|
||||
self.write(".gitattributes", "*.txt filter=example\n")
|
||||
with self.assertRaisesRegex(ValueError, "Active Git filter"):
|
||||
self.save_plan()
|
||||
|
||||
def test_directory_traversal_and_symlinks_are_refused(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
(self.repo / "linked.txt").symlink_to(self.repo / "selected.txt")
|
||||
for name in (".", "../outside", ".git/config", "linked.txt"):
|
||||
with self.subTest(name=name):
|
||||
self.args.path = [name]
|
||||
with self.assertRaises(ValueError):
|
||||
self.save_plan()
|
||||
|
||||
def test_post_index_change_hook_is_refused_before_it_can_run(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
marker = self.root / "hook-executed"
|
||||
hook = self.repo / ".git/hooks/post-index-change"
|
||||
hook.write_text(f"#!/bin/sh\ntouch '{marker}'\n")
|
||||
hook.chmod(0o700)
|
||||
with self.assertRaisesRegex(ValueError, "Active Git hooks"):
|
||||
self.save_plan()
|
||||
self.assertFalse(marker.exists())
|
||||
|
||||
def test_remote_helpers_and_recursive_operations_are_refused(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
for setting in (
|
||||
"remote.origin.receivepack",
|
||||
"remote.origin.uploadpack",
|
||||
"remote.origin.vcs",
|
||||
"core.gitProxy",
|
||||
"core.alternateRefsCommand",
|
||||
"push.recurseSubmodules",
|
||||
"submodule.recurse",
|
||||
"remote.origin.promisor",
|
||||
):
|
||||
with self.subTest(setting=setting):
|
||||
self.git("config", setting, "true")
|
||||
with self.assertRaises(ValueError):
|
||||
self.save_plan()
|
||||
self.git("config", "--unset", setting)
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
|
||||
|
||||
def test_git_namespace_identity_and_repository_overrides_are_refused(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
for key in (
|
||||
"GIT_NAMESPACE",
|
||||
"GIT_COMMON_DIR",
|
||||
"GIT_QUARANTINE_PATH",
|
||||
"GIT_SHALLOW_FILE",
|
||||
"GIT_REPLACE_REF_BASE",
|
||||
"GIT_ALTERNATE_OBJECT_DIRECTORIES",
|
||||
"GIT_AUTHOR_NAME",
|
||||
"GIT_CONFIG_COUNT",
|
||||
"GIT_UNKNOWN_OVERRIDE",
|
||||
):
|
||||
with self.subTest(key=key), patch.dict(os.environ, {key: "unexpected"}):
|
||||
with self.assertRaisesRegex(ValueError, "environment overrides"):
|
||||
self.save_plan()
|
||||
self.assertFalse((self.root / "state").exists())
|
||||
|
||||
def test_noncanonical_paths_are_rejected_before_commit(self):
|
||||
(self.repo / "dir").mkdir()
|
||||
self.write("dir/file.txt", "new\n")
|
||||
self.args.path = ["dir//file.txt"]
|
||||
with self.assertRaisesRegex(ValueError, "explicit relative"):
|
||||
self.save_plan()
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
|
||||
|
||||
def test_replacement_history_is_not_treated_as_the_real_frozen_tree(self):
|
||||
self.write("selected.txt", "changed\n")
|
||||
tree = self.git("rev-parse", "HEAD^{tree}").strip()
|
||||
replacement = self.git("commit-tree", tree, "-m", "replacement fixture").strip()
|
||||
self.git("replace", self.base, replacement)
|
||||
with self.assertRaisesRegex(ValueError, "replacement/graft"):
|
||||
self.save_plan()
|
||||
|
||||
def test_normal_push_is_explicit_and_verified_against_a_local_bare_fixture(self):
|
||||
remote = self.root / "origin.git"
|
||||
subprocess.run(
|
||||
["git", "init", "--bare", str(remote)], check=True, capture_output=True
|
||||
)
|
||||
self.git("remote", "add", "origin", str(remote))
|
||||
self.write("selected.txt", "selected final\n")
|
||||
frozen = self.save_plan()
|
||||
self.assertNotIn(str(remote), json.dumps(frozen))
|
||||
result = maintenance.commit(self.args)
|
||||
self.args.apply = False
|
||||
self.assertEqual(maintenance.push(self.args)["state"]["status"], "preview")
|
||||
self.assertEqual(
|
||||
self.git("ls-remote", "--refs", "origin", "refs/heads/main"), ""
|
||||
)
|
||||
self.args.apply = True
|
||||
pushed = maintenance.push(self.args)
|
||||
self.assertEqual(pushed["state"]["status"], "pushed")
|
||||
self.assertTrue(
|
||||
self.git("ls-remote", "--refs", "origin", "refs/heads/main").startswith(
|
||||
result["state"]["commit"]
|
||||
)
|
||||
)
|
||||
|
||||
def test_interrupted_commit_is_reconciled_without_creating_another_commit(self):
|
||||
self.write("selected.txt", "selected final\n")
|
||||
self.save_plan()
|
||||
original = maintenance._write_state
|
||||
|
||||
def fail_receipt(directory, state, **updates):
|
||||
if updates.get("status") == "committed":
|
||||
raise OSError("fixture interruption after commit")
|
||||
return original(directory, state, **updates)
|
||||
|
||||
with patch.object(maintenance, "_write_state", side_effect=fail_receipt):
|
||||
with self.assertRaises(OSError):
|
||||
maintenance.commit(self.args)
|
||||
head = self.git("rev-parse", "HEAD").strip()
|
||||
self.assertNotEqual(head, self.base)
|
||||
with self.assertRaisesRegex(ValueError, "reconcile"):
|
||||
maintenance.commit(self.args)
|
||||
result = maintenance.reconcile(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
|
||||
|
||||
def test_lost_push_receipt_is_reconciled_without_repeating_push(self):
|
||||
remote = self.root / "origin.git"
|
||||
subprocess.run(
|
||||
["git", "init", "--bare", str(remote)], check=True, capture_output=True
|
||||
)
|
||||
self.git("remote", "add", "origin", str(remote))
|
||||
self.write("selected.txt", "selected final\n")
|
||||
self.save_plan()
|
||||
maintenance.commit(self.args)
|
||||
original = maintenance._git
|
||||
pushes = []
|
||||
|
||||
def lose_response(repo, *argv, **kwargs):
|
||||
result = original(repo, *argv, **kwargs)
|
||||
if argv[0] == "push":
|
||||
pushes.append(argv)
|
||||
raise ValueError("fixture response lost after remote update")
|
||||
return result
|
||||
|
||||
with patch.object(maintenance, "_git", side_effect=lose_response):
|
||||
with self.assertRaises(ValueError):
|
||||
maintenance.push(self.args)
|
||||
with self.assertRaisesRegex(ValueError, "reconcile"):
|
||||
maintenance.push(self.args)
|
||||
reconciled = maintenance.reconcile(self.args)
|
||||
self.assertEqual(reconciled["state"]["status"], "pushed")
|
||||
self.assertEqual(len(pushes), 1)
|
||||
|
||||
def test_reconciliation_rechecks_state_after_acquiring_lock(self):
|
||||
self.write("selected.txt", "selected final\n")
|
||||
self.save_plan()
|
||||
maintenance.commit(self.args)
|
||||
payload, state, directory = maintenance._read(self.args, self.args.plan_id)
|
||||
maintenance._write_state(
|
||||
directory, state, status="needs_reconcile", operation="commit"
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def another_reconciliation(_args, _repo):
|
||||
maintenance._write_state(directory, state, status="committed")
|
||||
yield
|
||||
|
||||
with patch.object(maintenance, "_locked", another_reconciliation):
|
||||
result = maintenance.reconcile(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertTrue(
|
||||
any("no replay or downgrade" in line for line in result["summary"])
|
||||
)
|
||||
|
||||
def test_worktree_edit_racing_with_commit_remains_uncommitted(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
original = maintenance._git
|
||||
|
||||
def race(repo, *argv, **kwargs):
|
||||
if argv[0] == "commit-tree":
|
||||
self.write("selected.txt", "new editor bytes\n")
|
||||
return original(repo, *argv, **kwargs)
|
||||
|
||||
with patch.object(maintenance, "_git", side_effect=race):
|
||||
result = maintenance.commit(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertEqual(self.git("show", "HEAD:selected.txt"), "approved bytes\n")
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
|
||||
self.assertEqual((self.repo / "selected.txt").read_text(), "new editor bytes\n")
|
||||
|
||||
def test_branch_compare_and_swap_never_overwrites_a_racing_commit(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
original = maintenance._git
|
||||
raced = []
|
||||
|
||||
def race(repo, *argv, **kwargs):
|
||||
if argv[0] == "update-ref" and not raced:
|
||||
tree = self.git("rev-parse", "HEAD^{tree}").strip()
|
||||
other = self.git(
|
||||
"commit-tree", tree, "-p", self.base, "-m", "other writer"
|
||||
).strip()
|
||||
self.git("update-ref", "refs/heads/main", other, self.base)
|
||||
raced.append(other)
|
||||
return original(repo, *argv, **kwargs)
|
||||
|
||||
with patch.object(maintenance, "_git", side_effect=race):
|
||||
with self.assertRaises(ValueError):
|
||||
maintenance.commit(self.args)
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), raced[0])
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
|
||||
|
||||
def test_interrupted_index_publication_reconciles_without_another_commit(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
with patch.object(
|
||||
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
|
||||
):
|
||||
with self.assertRaises(OSError):
|
||||
maintenance.commit(self.args)
|
||||
head = self.git("rev-parse", "HEAD").strip()
|
||||
self.assertNotEqual(head, self.base)
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
|
||||
result = maintenance.reconcile(self.args)
|
||||
self.assertEqual(result["state"]["status"], "committed")
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "approved bytes\n")
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), head)
|
||||
|
||||
def test_concurrent_index_changes_are_preserved_instead_of_overwritten(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
original = maintenance._publish_index
|
||||
|
||||
def race(repo, locked, prepared, expected):
|
||||
# Model an external writer that does not honor Git's index.lock.
|
||||
outside = self.root / "outside.index"
|
||||
outside.write_bytes((self.repo / ".git/index").read_bytes())
|
||||
self.write("unrelated.txt", "new independent staging\n")
|
||||
subprocess.run(
|
||||
["git", "-C", str(self.repo), "add", "--", "unrelated.txt"],
|
||||
env={**os.environ, "GIT_INDEX_FILE": str(outside)},
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
(self.repo / ".git/index").write_bytes(outside.read_bytes())
|
||||
return original(repo, locked, prepared, expected)
|
||||
|
||||
with patch.object(maintenance, "_publish_index", side_effect=race):
|
||||
with self.assertRaisesRegex(ValueError, "index changed concurrently"):
|
||||
maintenance.commit(self.args)
|
||||
self.assertEqual(
|
||||
self.git("show", ":unrelated.txt"), "new independent staging\n"
|
||||
)
|
||||
with self.assertRaises(ValueError):
|
||||
maintenance.reconcile(self.args)
|
||||
self.assertEqual(
|
||||
self.git("show", ":unrelated.txt"), "new independent staging\n"
|
||||
)
|
||||
|
||||
def test_selected_fifo_swap_cannot_block_capture(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
original = maintenance._capture_blob
|
||||
|
||||
def race(repo, item):
|
||||
(self.repo / "selected.txt").unlink()
|
||||
os.mkfifo(self.repo / "selected.txt")
|
||||
return original(repo, item)
|
||||
|
||||
started = time.monotonic()
|
||||
with patch.object(maintenance, "_capture_blob", side_effect=race):
|
||||
with self.assertRaises(ValueError):
|
||||
maintenance.commit(self.args)
|
||||
self.assertLess(time.monotonic() - started, 3)
|
||||
self.assertEqual(self.git("rev-parse", "HEAD").strip(), self.base)
|
||||
|
||||
def test_reconcile_never_adopts_a_head_that_races_with_index_preparation(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
with patch.object(
|
||||
maintenance, "_publish_index", side_effect=OSError("fixture interruption")
|
||||
):
|
||||
with self.assertRaises(OSError):
|
||||
maintenance.commit(self.args)
|
||||
candidate = self.git("rev-parse", "HEAD").strip()
|
||||
original = maintenance._selected_index
|
||||
raced = []
|
||||
|
||||
def race(repo, source, target, files):
|
||||
result = original(repo, source, target, files)
|
||||
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
|
||||
other = self.git(
|
||||
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
|
||||
).strip()
|
||||
self.git("update-ref", "refs/heads/main", other, candidate)
|
||||
raced.append(other)
|
||||
return result
|
||||
|
||||
with patch.object(maintenance, "_selected_index", side_effect=race):
|
||||
with self.assertRaisesRegex(
|
||||
ValueError, "changed during index reconciliation"
|
||||
):
|
||||
maintenance.reconcile(self.args)
|
||||
recorded = maintenance.status(self.args)["state"]
|
||||
self.assertEqual(recorded["status"], "needs_reconcile")
|
||||
self.assertEqual(recorded["commit"], candidate)
|
||||
self.assertNotEqual(recorded["commit"], raced[0])
|
||||
self.assertEqual(self.git("show", ":selected.txt"), "base selected\n")
|
||||
|
||||
def test_final_commit_receipt_cannot_bind_to_a_later_head(self):
|
||||
self.write("selected.txt", "approved bytes\n")
|
||||
self.save_plan()
|
||||
original = maintenance._publish_index
|
||||
raced = []
|
||||
|
||||
def race(repo, locked, prepared, expected):
|
||||
original(repo, locked, prepared, expected)
|
||||
candidate = self.git("rev-parse", "HEAD").strip()
|
||||
tree = self.git("rev-parse", f"{self.base}^{{tree}}").strip()
|
||||
other = self.git(
|
||||
"commit-tree", tree, "-p", candidate, "-m", "independent writer"
|
||||
).strip()
|
||||
self.git("update-ref", "refs/heads/main", other, candidate)
|
||||
raced.append(other)
|
||||
|
||||
with patch.object(maintenance, "_publish_index", side_effect=race):
|
||||
with self.assertRaisesRegex(
|
||||
ValueError, "changed before the exact commit result"
|
||||
):
|
||||
maintenance.commit(self.args)
|
||||
recorded = maintenance.status(self.args)["state"]
|
||||
self.assertEqual(recorded["status"], "needs_reconcile")
|
||||
self.assertNotEqual(recorded["commit"], raced[0])
|
||||
|
||||
def test_git_output_is_bounded_during_execution(self):
|
||||
executable = self.root / "git"
|
||||
executable.write_text(f"#!{sys.executable}\nprint('x' * 100000)\n")
|
||||
executable.chmod(0o700)
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
|
||||
),
|
||||
patch.object(maintenance, "MAX_GIT_OUTPUT_BYTES", 256),
|
||||
):
|
||||
with self.assertRaisesRegex(ValueError, "output_limit"):
|
||||
maintenance._git(self.repo, "fixture")
|
||||
|
||||
def test_timeout_terminates_git_helper_process_group(self):
|
||||
executable = self.root / "git"
|
||||
marker = self.root / "helper-ran"
|
||||
child = (
|
||||
"import signal,time,pathlib; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(1); pathlib.Path("
|
||||
+ repr(str(marker))
|
||||
+ ").write_text('unexpected')"
|
||||
)
|
||||
executable.write_text(
|
||||
f"#!{sys.executable}\nimport subprocess,sys,time\nsubprocess.Popen([sys.executable, '-c', {child!r}])\ntime.sleep(20)\n"
|
||||
)
|
||||
executable.chmod(0o700)
|
||||
with patch.dict(
|
||||
os.environ, {"PATH": str(self.root) + os.pathsep + os.environ["PATH"]}
|
||||
):
|
||||
with self.assertRaisesRegex(ValueError, "timed_out"):
|
||||
maintenance._git(self.repo, "fixture", timeout=0.2)
|
||||
time.sleep(1.1)
|
||||
self.assertFalse(
|
||||
marker.exists(),
|
||||
"An orphaned helper must not finish the operation after timeout",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Executable
+319
@@ -0,0 +1,319 @@
|
||||
"""Live/provisional monitoring and discovery never substitute for verified evidence."""
|
||||
|
||||
from argparse import Namespace
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from test_devkit_runner import example as example, run, stage
|
||||
from govoplan_devkit import catalog, cli, monitoring, runner
|
||||
from govoplan_devkit.checkpoints import Checkpoints
|
||||
from govoplan_devkit.common import atomic_json, read_json, state_root
|
||||
from govoplan_devkit.process import OutputSnapshot
|
||||
|
||||
|
||||
def command(args, *argv):
|
||||
parser = argparse.ArgumentParser()
|
||||
runner.register(parser.add_subparsers(dest="command", required=True))
|
||||
selected = parser.parse_args(argv, namespace=Namespace(**vars(args)))
|
||||
return selected.handler(selected)
|
||||
|
||||
|
||||
def test_preparing_receipt_and_run_id_exist_before_source_fingerprinting(example):
|
||||
args, repo = example
|
||||
events = []
|
||||
args.on_progress = events.append
|
||||
original = Checkpoints.source
|
||||
|
||||
def fingerprint(self, *values, **kwargs):
|
||||
assert events and events[0]["phase"] == "preparing"
|
||||
first = events[0]
|
||||
assert Path(first["receipt_path"]).is_file()
|
||||
record = runner.read_receipt(
|
||||
args.workspace_root, args.state_dir, first["run_id"]
|
||||
)
|
||||
if record["phase"] == "preparing":
|
||||
assert record["status"] == "running"
|
||||
assert record["snapshot_verified"] is False
|
||||
assert record["source_fingerprint"] is None
|
||||
return original(self, *values, **kwargs)
|
||||
|
||||
with patch.object(Checkpoints, "source", fingerprint):
|
||||
result = run(args, [stage(repo)])
|
||||
assert result["status"] == "passed"
|
||||
assert events[-1]["phase"] == "finished"
|
||||
assert events[-1]["run_id"] == events[0]["run_id"] == result["run_id"]
|
||||
|
||||
|
||||
def test_preflight_failure_persists_discoverable_nonpassing_receipt(example):
|
||||
args, repo = example
|
||||
events = []
|
||||
args.on_progress = events.append
|
||||
with patch.object(
|
||||
Checkpoints,
|
||||
"source",
|
||||
side_effect=ValueError("fixture fingerprint unavailable"),
|
||||
):
|
||||
with pytest.raises(ValueError, match="fixture fingerprint unavailable"):
|
||||
run(args, [stage(repo)])
|
||||
assert events[0]["phase"] == "preparing"
|
||||
record = runner.read_receipt(
|
||||
args.workspace_root, args.state_dir, events[0]["run_id"]
|
||||
)
|
||||
assert record["status"] == "failed"
|
||||
assert record["phase"] == "finished"
|
||||
assert record["snapshot_verified"] is False
|
||||
assert all(item["status"] != "passed" for item in record["stages"])
|
||||
assert "fixture fingerprint unavailable" in record["error"]
|
||||
assert monitoring.latest_run(args)["status"] == "failed"
|
||||
|
||||
|
||||
def test_run_history_is_read_only_and_cursor_pagination_has_no_duplicates(example):
|
||||
args, repo = example
|
||||
created = [run(args, [stage(repo)])["run_id"] for _ in range(3)]
|
||||
base = state_root(args.workspace_root, args.state_dir)
|
||||
before = {
|
||||
path: (path.stat().st_mtime_ns, path.read_bytes())
|
||||
for path in base.rglob("*")
|
||||
if path.is_file()
|
||||
}
|
||||
page = monitoring.list_runs(Namespace(**vars(args), limit=2, before=None))
|
||||
assert [item["run_id"] for item in page["runs"]] == sorted(created, reverse=True)[
|
||||
:2
|
||||
]
|
||||
assert page["next_cursor"]
|
||||
next_page = monitoring.list_runs(
|
||||
Namespace(**vars(args), limit=2, before=page["next_cursor"])
|
||||
)
|
||||
assert [item["run_id"] for item in next_page["runs"]] == sorted(
|
||||
created, reverse=True
|
||||
)[2:]
|
||||
assert next_page["next_cursor"] is None
|
||||
assert monitoring.latest_run(args)["run_id"] == max(created)
|
||||
assert before == {
|
||||
path: (path.stat().st_mtime_ns, path.read_bytes())
|
||||
for path in base.rglob("*")
|
||||
if path.is_file()
|
||||
}
|
||||
|
||||
|
||||
def test_empty_run_history_does_not_create_a_state_directory(example):
|
||||
args, _ = example
|
||||
assert not args.state_dir.exists()
|
||||
result = monitoring.list_runs(args)
|
||||
assert result["runs"] == []
|
||||
assert monitoring.latest_run(args)["status"] == "not_found"
|
||||
assert not args.state_dir.exists()
|
||||
|
||||
|
||||
def test_invalid_newest_run_is_visible_and_never_replaced_with_older_pass(example):
|
||||
args, repo = example
|
||||
older = run(args, [stage(repo)])["run_id"]
|
||||
invalid = "zz-invalid-newest"
|
||||
path = (
|
||||
state_root(args.workspace_root, args.state_dir)
|
||||
/ "runs"
|
||||
/ invalid
|
||||
/ "receipt.json"
|
||||
)
|
||||
atomic_json(path, {"malformed": True})
|
||||
rows = monitoring.list_runs(args)
|
||||
assert rows["runs"][0]["run_id"] == invalid
|
||||
assert rows["runs"][0]["status"] == "invalid"
|
||||
assert any(row["run_id"] == older for row in rows["runs"])
|
||||
assert rows["_exit_code"] == 1
|
||||
with pytest.raises(ValueError, match="Latest run is invalid"):
|
||||
monitoring.latest_run(args)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"changes", [{"limit": 0}, {"limit": 101}, {"limit": True}, {"before": "../escape"}]
|
||||
)
|
||||
def test_history_rejects_invalid_bounds_and_cursor(example, changes):
|
||||
args, _ = example
|
||||
with pytest.raises(ValueError):
|
||||
monitoring.list_runs(Namespace(**{**vars(args), **changes}))
|
||||
|
||||
|
||||
def test_live_logs_are_available_before_completion_but_final_only_refuses_them(example):
|
||||
args, repo = example
|
||||
events, results, errors = [], [], []
|
||||
args.on_progress = events.append
|
||||
|
||||
def execute():
|
||||
try:
|
||||
results.append(
|
||||
run(
|
||||
args,
|
||||
[
|
||||
stage(
|
||||
repo,
|
||||
code="import time; print('live ready',flush=True); time.sleep(1.5); print('finished')",
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
except BaseException as exc:
|
||||
errors.append(exc)
|
||||
|
||||
worker = threading.Thread(target=execute)
|
||||
worker.start()
|
||||
try:
|
||||
deadline = time.monotonic() + 4
|
||||
live = None
|
||||
while time.monotonic() < deadline:
|
||||
if events:
|
||||
live = command(args, "logs", events[0]["run_id"], "--stage", "one")
|
||||
if "live ready" in live["excerpt"]:
|
||||
break
|
||||
time.sleep(0.02)
|
||||
assert live is not None and "live ready" in live["excerpt"]
|
||||
assert live["provisional"] is True
|
||||
assert live["log_verified"] is False
|
||||
assert live["snapshot_verified"] is False
|
||||
assert live["run_status"] == "running"
|
||||
with pytest.raises(ValueError, match="No finalized stage log"):
|
||||
command(args, "logs", events[0]["run_id"], "--stage", "one", "--final-only")
|
||||
finally:
|
||||
worker.join(timeout=5)
|
||||
assert not worker.is_alive() and not errors
|
||||
final = command(
|
||||
args, "logs", results[0]["run_id"], "--stage", "one", "--final-only"
|
||||
)
|
||||
assert final["provisional"] is False and final["log_verified"] is True
|
||||
assert final["snapshot_verified"] is True
|
||||
assert "finished" in final["excerpt"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("quiet", [False, True])
|
||||
def test_cli_keeps_json_stdout_clean_and_progress_on_stderr_or_quiet(
|
||||
tmp_path, capsys, quiet
|
||||
):
|
||||
event = {
|
||||
"event": "check_progress",
|
||||
"run_id": "fixture",
|
||||
"phase": "preparing",
|
||||
"status": "running",
|
||||
"counts": {"pending": 1},
|
||||
"total_stages": 1,
|
||||
"elapsed_seconds": 0,
|
||||
"active_stages": [],
|
||||
"receipt_path": str(tmp_path / "receipt.json"),
|
||||
}
|
||||
|
||||
def check(args, _stages):
|
||||
args.on_progress(event)
|
||||
return {"status": "passed", "summary": ["fixture complete"]}
|
||||
|
||||
with (
|
||||
patch.object(catalog, "build_stages", return_value=[]),
|
||||
patch.object(runner, "run_checks", side_effect=check),
|
||||
):
|
||||
assert (
|
||||
cli.main(
|
||||
[
|
||||
"check",
|
||||
"--workspace-root",
|
||||
str(tmp_path),
|
||||
"--json",
|
||||
*(["--quiet"] if quiet else []),
|
||||
]
|
||||
)
|
||||
== 0
|
||||
)
|
||||
output = capsys.readouterr()
|
||||
assert json.loads(output.out)["status"] == "passed"
|
||||
if quiet:
|
||||
assert output.err == ""
|
||||
else:
|
||||
assert json.loads(output.err)["event"] == "check_progress"
|
||||
|
||||
|
||||
def snapshot(data, *, split=0, omitted=0, final=False):
|
||||
return OutputSnapshot(data, b"", bool(omitted), omitted, 0, split, 0, final)
|
||||
|
||||
|
||||
def test_provisional_output_withholds_incomplete_secret_line(monkeypatch):
|
||||
monkeypatch.setenv("FIXTURE_SECRET", "credential-known-to-redaction")
|
||||
data = b"public line\nAPI_KEY=credential-known-to-"
|
||||
live = monitoring.capture_text(snapshot(data), provisional=True)
|
||||
assert live == "public line\n"
|
||||
final = monitoring.capture_text(
|
||||
snapshot(b"public line\nAPI_KEY=credential-known-to-redaction", final=True)
|
||||
)
|
||||
assert "credential-known" not in final
|
||||
assert "[redacted]" in final
|
||||
|
||||
|
||||
def test_truncated_boundaries_cannot_expose_cut_authorization_or_secret_fragments():
|
||||
head = b"safe head\nAuthorization: Bearer partial-secret-head"
|
||||
tail = b"partial-secret-tail\nsafe final line\n"
|
||||
for provisional in (False, True):
|
||||
text = monitoring.capture_text(
|
||||
snapshot(head + tail, split=len(head), omitted=90), provisional=provisional
|
||||
)
|
||||
assert "safe head" in text and "safe final line" in text
|
||||
assert "partial-secret" not in text
|
||||
assert "90 bytes omitted" in text
|
||||
|
||||
|
||||
def test_final_failure_tail_survives_retention_and_stays_redacted(example, monkeypatch):
|
||||
args, repo = example
|
||||
monkeypatch.setenv("FIXTURE_SECRET", "sensitive-final-credential")
|
||||
with patch.object(runner, "MAX_LOG_BYTES", 512):
|
||||
result = run(
|
||||
args,
|
||||
[
|
||||
stage(
|
||||
repo,
|
||||
code="import os; print('begin'); print('x'*20000); print(os.environ['FIXTURE_SECRET']); print('FINAL IMPORTANT ERROR'); raise SystemExit(9)",
|
||||
)
|
||||
],
|
||||
)
|
||||
final = command(
|
||||
args, "logs", result["run_id"], "--stage", "one", "--final-only"
|
||||
)
|
||||
assert result["status"] == "failed"
|
||||
assert final["log_verified"] is True
|
||||
assert "FINAL IMPORTANT ERROR" in final["excerpt"]
|
||||
assert "sensitive-final-credential" not in final["excerpt"]
|
||||
assert "[redacted]" in final["excerpt"]
|
||||
assert result["stages"][0]["omitted_output_bytes"] > 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize("maximum", [1, 2, 3, 7, 75, 76, 77, 80, 81, 82, 255])
|
||||
@pytest.mark.parametrize("tail_only", [False, True])
|
||||
def test_tiny_display_bounds_preserve_valid_utf8_and_byte_limit(maximum, tail_only):
|
||||
value = "Ä😊 " * 200 + "FINAL"
|
||||
text = monitoring.bounded_display(value, maximum, tail_only=tail_only)
|
||||
assert len(text.encode("utf-8", errors="strict")) <= maximum
|
||||
assert text.endswith("L")
|
||||
if maximum >= 5 and (tail_only or maximum >= 255):
|
||||
assert text.endswith("FINAL")
|
||||
|
||||
|
||||
def test_live_record_is_bounded_provisional_and_rejects_mismatched_identity(example):
|
||||
args, _ = example
|
||||
run_id, stage_id = "fixture-run", "fixture-stage"
|
||||
path = (
|
||||
state_root(args.workspace_root, args.state_dir)
|
||||
/ "runs"
|
||||
/ run_id
|
||||
/ (stage_id + ".log")
|
||||
)
|
||||
monitoring.write_live(path, stage_id, snapshot(b"line\n"), time.monotonic())
|
||||
live = monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
|
||||
assert live["provisional"] is True and live["excerpt"] == "line\n"
|
||||
record = read_json(path.with_suffix(".live.json"))
|
||||
record["run_id"] = "different-run"
|
||||
atomic_json(path.with_suffix(".live.json"), record)
|
||||
with pytest.raises(ValueError, match="Invalid provisional log"):
|
||||
monitoring.read_live(args.workspace_root, args.state_dir, run_id, stage_id)
|
||||
Executable
+249
@@ -0,0 +1,249 @@
|
||||
"""Bounded fixture processes only; no project servers or external transports."""
|
||||
|
||||
from pathlib import Path
|
||||
from dataclasses import FrozenInstanceError
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit.process import _OutputBuffer, require_capture, run_captured
|
||||
|
||||
|
||||
def python(code, **kwargs):
|
||||
return run_captured([sys.executable, "-c", code], **kwargs)
|
||||
|
||||
|
||||
def test_large_stdin_and_separate_outputs_are_multiplexed():
|
||||
result = python(
|
||||
"import sys; sys.stderr.write('diagnostic'); print(len(sys.stdin.buffer.read()))",
|
||||
input_bytes=b"x" * 400000,
|
||||
)
|
||||
assert result.status == "passed"
|
||||
assert result.stdout == b"400000\n"
|
||||
assert result.stderr == b"diagnostic"
|
||||
|
||||
|
||||
def test_live_output_limit_terminates_unbounded_producer():
|
||||
result = python(
|
||||
"import os;\nwhile True: os.write(1, b'x'*65536)", max_stdout=2048, timeout=3
|
||||
)
|
||||
assert result.status == "output_limit"
|
||||
assert result.truncated
|
||||
assert len(result.stdout) == 2048
|
||||
|
||||
|
||||
def test_drain_mode_caps_memory_without_turning_success_into_failure():
|
||||
result = python("print('x'*100000)", max_stdout=1024, terminate_on_limit=False)
|
||||
assert result.status == "passed"
|
||||
assert result.truncated and len(result.stdout) == 1024
|
||||
|
||||
|
||||
def test_deadline_applies_after_command_closes_both_output_streams():
|
||||
started = time.monotonic()
|
||||
result = python(
|
||||
"import os,time; os.close(1); os.close(2); time.sleep(30)", timeout=0.15
|
||||
)
|
||||
assert result.status == "timed_out"
|
||||
assert time.monotonic() - started < 5
|
||||
|
||||
|
||||
def test_cancellation_stops_owned_process():
|
||||
cancelled = threading.Event()
|
||||
timer = threading.Timer(0.1, cancelled.set)
|
||||
timer.start()
|
||||
try:
|
||||
result = python("import time; time.sleep(30)", cancelled=cancelled)
|
||||
finally:
|
||||
timer.cancel()
|
||||
timer.join()
|
||||
assert result.status == "interrupted"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("redirected", [False, True])
|
||||
def test_outliving_child_is_not_success_and_cannot_keep_running(redirected):
|
||||
code = (
|
||||
"import subprocess,sys; child=subprocess.Popen([sys.executable,'-c','import time; time.sleep(30)']"
|
||||
+ (",stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL" if redirected else "")
|
||||
+ "); print(child.pid,flush=True)"
|
||||
)
|
||||
result = python(code, timeout=4)
|
||||
assert result.status == "leaked_process"
|
||||
child_pid = int(result.stdout.strip())
|
||||
# A terminated adopted child may remain a zombie until the host's init reaps it.
|
||||
proc = Path(f"/proc/{child_pid}/stat")
|
||||
if proc.exists():
|
||||
assert proc.read_text().split(")", 1)[1].strip().split()[0] == "Z"
|
||||
|
||||
|
||||
def test_required_capture_rejects_nonordinary_completion():
|
||||
with pytest.raises(ValueError, match="output_limit"):
|
||||
require_capture([sys.executable, "-c", "print('x'*10000)"], max_stdout=32)
|
||||
|
||||
|
||||
def test_ordinary_nonzero_exit_retains_diagnostics():
|
||||
result = python("import sys; print('problem',file=sys.stderr); sys.exit(7)")
|
||||
assert (result.status, result.returncode, result.stderr) == (
|
||||
"failed",
|
||||
7,
|
||||
b"problem\n",
|
||||
)
|
||||
|
||||
|
||||
def test_selector_setup_failure_still_terminates_spawned_process():
|
||||
from govoplan_devkit import process
|
||||
|
||||
original = process.subprocess.Popen
|
||||
spawned = []
|
||||
|
||||
def capture(*args, **kwargs):
|
||||
child = original(*args, **kwargs)
|
||||
spawned.append(child)
|
||||
return child
|
||||
|
||||
with (
|
||||
patch.object(process.subprocess, "Popen", side_effect=capture),
|
||||
patch.object(
|
||||
process.selectors,
|
||||
"DefaultSelector",
|
||||
side_effect=OSError("fixture selector unavailable"),
|
||||
),
|
||||
):
|
||||
with pytest.raises(OSError, match="selector unavailable"):
|
||||
python("import time; time.sleep(30)")
|
||||
assert len(spawned) == 1 and spawned[0].poll() is not None
|
||||
|
||||
|
||||
def test_head_tail_keeps_the_actual_failure_tail_within_original_bound():
|
||||
output = b"BEGIN" + b"middle" * 100 + b"FINAL ERROR"
|
||||
result = python(
|
||||
f"import os; os.write(1, {output!r}); raise SystemExit(7)",
|
||||
max_stdout=32,
|
||||
capture_mode="head_tail",
|
||||
terminate_on_limit=False,
|
||||
)
|
||||
assert (result.status, result.returncode) == ("failed", 7)
|
||||
assert result.stdout == output[:16] + output[-16:]
|
||||
assert result.stdout_head_bytes == 16
|
||||
assert result.omitted_stdout_bytes == len(output) - 32
|
||||
assert result.snapshot().final
|
||||
assert "FINAL ERROR" in result.snapshot().text()
|
||||
assert str(len(output) - 32) + " output bytes omitted" in result.snapshot().text()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("limit", [0, 1, 2, 3, 31, 1024])
|
||||
def test_rolling_buffers_remain_bounded_for_every_chunk(limit):
|
||||
buffer = _OutputBuffer(limit, "head_tail")
|
||||
original = b""
|
||||
for data in (b"a", b"bcdef", b"x" * 65536, b"last error"):
|
||||
original += data
|
||||
buffer.append(data)
|
||||
assert len(buffer.head) + len(buffer.tail) <= limit
|
||||
assert buffer.omitted == max(0, len(original) - limit)
|
||||
if len(original) > limit:
|
||||
head = limit // 2
|
||||
tail = limit - head
|
||||
expected = original[:head] + (original[-tail:] if tail else b"")
|
||||
assert buffer.value() == expected
|
||||
|
||||
|
||||
def test_prefix_capture_remains_exact_and_counts_omitted_bytes():
|
||||
result = python(
|
||||
"import os; os.write(1,b'0123456789')", max_stdout=4, terminate_on_limit=False
|
||||
)
|
||||
assert result.stdout == b"0123"
|
||||
assert result.stdout_head_bytes == 4
|
||||
assert result.omitted_stdout_bytes == 6
|
||||
|
||||
|
||||
def test_callback_exposes_bounded_immutable_initial_and_final_snapshots():
|
||||
snapshots = []
|
||||
result = python(
|
||||
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
|
||||
"os.write(1,b'x'*10000+b'FINAL\\n'); time.sleep(.1)",
|
||||
max_stdout=32,
|
||||
capture_mode="head_tail",
|
||||
terminate_on_limit=False,
|
||||
on_output=snapshots.append,
|
||||
)
|
||||
assert not snapshots[0].final
|
||||
assert snapshots[-1] == result.snapshot()
|
||||
assert snapshots[0].stdout == b"first\n"
|
||||
assert snapshots[-1].stdout.endswith(b"FINAL\n")
|
||||
assert all(len(item.stdout) <= 32 for item in snapshots)
|
||||
with pytest.raises(FrozenInstanceError):
|
||||
snapshots[0].final = True
|
||||
|
||||
|
||||
def test_callback_updates_dirty_output_while_child_becomes_quiet():
|
||||
snapshots = []
|
||||
python(
|
||||
"import os,time; os.write(1,b'first\\n'); time.sleep(.15); "
|
||||
"os.write(1,b'second\\n'); time.sleep(1.15)",
|
||||
on_output=lambda item: snapshots.append((time.monotonic(), item)),
|
||||
)
|
||||
assert len(snapshots) >= 3
|
||||
assert snapshots[0][1].stdout == b"first\n"
|
||||
assert not snapshots[1][1].final
|
||||
assert snapshots[1][1].stdout.endswith(b"second\n")
|
||||
assert snapshots[1][0] - snapshots[0][0] >= 0.95
|
||||
assert snapshots[-1][1].final
|
||||
|
||||
|
||||
def test_quiet_process_has_one_final_empty_snapshot():
|
||||
snapshots = []
|
||||
python("pass", on_output=snapshots.append)
|
||||
assert len(snapshots) == 1
|
||||
assert snapshots[0].final and snapshots[0].stdout == b""
|
||||
|
||||
|
||||
def test_snapshot_text_handles_cut_and_invalid_utf8_without_malformed_strings():
|
||||
result = python(
|
||||
"import os; os.write(1, 'Ä😊Z'.encode()*20+b'\\xff\\xfe')",
|
||||
max_stdout=9,
|
||||
capture_mode="head_tail",
|
||||
terminate_on_limit=False,
|
||||
)
|
||||
text = result.snapshot().text()
|
||||
text.encode("utf-8", errors="strict")
|
||||
assert "output bytes omitted" in text
|
||||
assert len(result.stdout) == 9
|
||||
with pytest.raises(ValueError, match="stream"):
|
||||
result.snapshot().text("other")
|
||||
|
||||
|
||||
def test_callback_failure_terminates_owned_process_and_propagates():
|
||||
from govoplan_devkit import process
|
||||
|
||||
original = process.subprocess.Popen
|
||||
spawned = []
|
||||
|
||||
def capture(*args, **kwargs):
|
||||
child = original(*args, **kwargs)
|
||||
spawned.append(child)
|
||||
return child
|
||||
|
||||
def fail(_snapshot):
|
||||
raise ValueError("fixture callback failed")
|
||||
|
||||
with patch.object(process.subprocess, "Popen", side_effect=capture):
|
||||
with pytest.raises(ValueError, match="fixture callback failed"):
|
||||
python(
|
||||
"import time; print('ready',flush=True); time.sleep(30)", on_output=fail
|
||||
)
|
||||
assert len(spawned) == 1 and spawned[0].poll() is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs", [{"capture_mode": "other"}, {"max_stdout": -1}, {"max_stderr": True}]
|
||||
)
|
||||
def test_invalid_capture_configuration_fails_before_starting_a_process(kwargs):
|
||||
from govoplan_devkit import process
|
||||
|
||||
with patch.object(process.subprocess, "Popen") as popen:
|
||||
with pytest.raises(ValueError):
|
||||
python("pass", **kwargs)
|
||||
popen.assert_not_called()
|
||||
Executable
+524
@@ -0,0 +1,524 @@
|
||||
"""Fixture-only devkit coverage of the real durable release API/store boundary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from copy import deepcopy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
for path in (ROOT / "tools/devkit", ROOT / "tools/release"):
|
||||
if str(path) not in sys.path:
|
||||
sys.path.insert(0, str(path))
|
||||
|
||||
from govoplan_devkit import release # noqa: E402
|
||||
from govoplan_release.release_execution import ReleaseExecutionAmbiguous, ReleaseExecutionBlocked # noqa: E402
|
||||
from govoplan_release.release_run import ReleaseRunCorrupt, ReleaseRunStore # noqa: E402
|
||||
from govoplan_release.candidate_artifact import ( # noqa: E402
|
||||
candidate_output_path, harden_private_candidate_tree, issue_candidate_receipt,
|
||||
)
|
||||
from server import app as api # noqa: E402
|
||||
|
||||
|
||||
def receipt(repo="govoplan-core", *, target_tag="v1.2.3"):
|
||||
return {
|
||||
"kind": "repository_state", "repo": repo, "head": "a" * 40,
|
||||
"branch": "main", "remote": "origin", "remote_sha256": "b" * 64,
|
||||
"worktree_clean": True, "target_tag": target_tag, "tag_object": None,
|
||||
}
|
||||
|
||||
|
||||
def plan(*, catalog=False):
|
||||
steps = [
|
||||
("core:preflight", "govoplan-core", False),
|
||||
("core:tag", "govoplan-core", True),
|
||||
] if not catalog else [
|
||||
("catalog:selective-generator", None, True),
|
||||
("catalog:validate-sign-publish", None, True),
|
||||
]
|
||||
return {
|
||||
"generated_at": "2026-09-08T00:00:00Z", "target_channel": "stable",
|
||||
"status": "attention", "units": [{"repo": "govoplan-core", "target_version": "1.2.3"}],
|
||||
"compatibility": [], "gate_findings": [], "recommended_action": {},
|
||||
"source_preflight_ready": True, "notes": [],
|
||||
"dry_run_steps": [
|
||||
{"id": identity, "repo": repo, "mutating": mutating, "title": identity,
|
||||
"detail": "Fixture-only operation.", "command": "fixture never executed",
|
||||
"cwd": "/fixture", "status": "planned"}
|
||||
for identity, repo, mutating in steps
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def namespace(workspace, state, *arguments):
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.set_defaults(workspace_root=workspace, state_dir=state, format="json", project=None)
|
||||
release.register(parser.add_subparsers(required=True))
|
||||
return parser.parse_args(["release", *arguments])
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def environment(tmp_path, monkeypatch):
|
||||
workspace = tmp_path / "workspace"
|
||||
workspace.mkdir()
|
||||
state = tmp_path / "private-state"
|
||||
plans = [plan()]
|
||||
dashboard = Mock(return_value={"summary": {"status": "ready"}, "repositories": []})
|
||||
planner = Mock(side_effect=lambda *args, **kwargs: deepcopy(plans[0]))
|
||||
execute = Mock(return_value=({"status": "inspected"}, receipt()))
|
||||
|
||||
def bind(**kwargs):
|
||||
result = kwargs["plan"]
|
||||
for step in result["dry_run_steps"]:
|
||||
if step.get("repo"):
|
||||
step["source_binding"] = receipt(step["repo"])
|
||||
elif step["id"] == "catalog:validate-sign-publish":
|
||||
step["source_binding"] = receipt("addideas-govoplan-website", target_tag="")
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(api, "build_dashboard", dashboard)
|
||||
monkeypatch.setattr(api, "build_selective_release_plan", planner)
|
||||
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock())
|
||||
monkeypatch.setattr(api, "verify_release_runtime_binding", Mock())
|
||||
monkeypatch.setattr(api, "bind_plan_source_states", bind)
|
||||
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(return_value=receipt()))
|
||||
monkeypatch.setattr(api, "verify_repository_step_precondition", Mock(return_value=receipt()))
|
||||
monkeypatch.setattr(api, "execute_repository_step", execute)
|
||||
monkeypatch.setattr(api, "default_signing_keys", lambda: ())
|
||||
|
||||
def forbidden(*args, **kwargs):
|
||||
raise AssertionError("Fixture test attempted a real subprocess or network connection")
|
||||
monkeypatch.setattr(subprocess, "run", forbidden)
|
||||
monkeypatch.setattr(socket, "create_connection", forbidden)
|
||||
|
||||
def call(*args, state_dir=state, workspace_root=workspace):
|
||||
values = namespace(workspace_root, state_dir, *args)
|
||||
return values.handler(values)
|
||||
|
||||
def create(request="devkit-create-request-0001"):
|
||||
result = call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", request, "--apply")
|
||||
assert result["_exit_code"] == 0, result
|
||||
return result
|
||||
|
||||
return SimpleNamespace(
|
||||
workspace=workspace, state=state, plans=plans, dashboard=dashboard,
|
||||
planner=planner, executor=execute, call=call, create=create,
|
||||
)
|
||||
|
||||
|
||||
def test_registration_does_not_import_heavy_dependencies():
|
||||
script = """
|
||||
import argparse, builtins, sys
|
||||
sys.path.insert(0, sys.argv[1])
|
||||
original = builtins.__import__
|
||||
def guarded(name, *args, **kwargs):
|
||||
if name.split('.')[0] in {'httpx', 'fastapi', 'govoplan_core', 'govoplan_release'}:
|
||||
raise AssertionError('Heavy import during help registration: ' + name)
|
||||
return original(name, *args, **kwargs)
|
||||
builtins.__import__ = guarded
|
||||
from govoplan_devkit.release import register
|
||||
parser = argparse.ArgumentParser()
|
||||
register(parser.add_subparsers())
|
||||
assert 'release' in parser.format_help()
|
||||
"""
|
||||
result = subprocess.run([sys.executable, "-c", script, str(ROOT / "tools/devkit")], capture_output=True, text=True, check=False)
|
||||
assert result.returncode == 0, result.stderr
|
||||
|
||||
|
||||
def test_plan_is_selective_offline_and_does_not_create_run(environment):
|
||||
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
|
||||
assert result["_exit_code"] == 0
|
||||
assert result["dry_run"] is True
|
||||
assert environment.planner.call_args.kwargs["selected_repos"] == ("govoplan-core",)
|
||||
arguments = environment.dashboard.call_args.kwargs
|
||||
assert arguments["online"] is False
|
||||
assert arguments["check_remote_tags"] is False
|
||||
assert arguments["check_public_catalog"] is False
|
||||
assert arguments["include_migrations"] is False
|
||||
assert not list(environment.state.rglob("rr-*.json"))
|
||||
assert "release-console/workspace-" in result["state_location"]
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_status_reports_blocked_exit_and_explicit_check_flags(environment):
|
||||
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
|
||||
result = environment.call("status", "--online", "--include-migrations", "--include-website")
|
||||
assert result["_exit_code"] == 1
|
||||
assert environment.dashboard.call_args.kwargs["check_remote_tags"] is True
|
||||
assert environment.dashboard.call_args.kwargs["check_public_catalog"] is True
|
||||
assert environment.dashboard.call_args.kwargs["include_migrations"] is True
|
||||
|
||||
|
||||
def test_plan_summary_exposes_attention_readiness_gates_and_next_action(environment):
|
||||
fixture_plan = environment.plans[0]
|
||||
fixture_plan["source_preflight_ready"] = False
|
||||
fixture_plan["units"][0]["status"] = "attention"
|
||||
fixture_plan["gate_findings"] = [{"code": "worktree_dirty", "repo": "govoplan-core", "message": "Uncommitted source changes need review."}]
|
||||
fixture_plan["recommended_action"] = {"id": "prepare_changes", "title": "Prepare source changes", "remediation": "Review and commit the selected changes before preparing the release."}
|
||||
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
|
||||
summary = "\n".join(result["summary"])
|
||||
assert "Release plan: attention." in summary
|
||||
assert "Source preflight ready: false." in summary
|
||||
assert "govoplan-core: attention; target 1.2.3." in summary
|
||||
assert "Gate worktree_dirty (govoplan-core)" in summary
|
||||
assert "Next: prepare_changes" in summary
|
||||
assert "no run was created" in summary
|
||||
assert "fixture never executed" not in summary
|
||||
|
||||
|
||||
def test_summary_is_bounded_and_redacts_display_only_fields(monkeypatch):
|
||||
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "fixture-value-not-for-display")
|
||||
fixture_plan = plan()
|
||||
fixture_plan["gate_findings"] = [{"code": "fixture", "message": "fixture-value-not-for-display " + "x" * 1000}] * 20
|
||||
fixture_plan["units"] = [{"repo": f"repo-{index}", "target_version": "1.2.3", "status": "ready"} for index in range(50)]
|
||||
summary = "\n".join(release._summary_lines("plan", fixture_plan))
|
||||
assert "38 more selected repositories" in summary
|
||||
assert "16 more gate findings" in summary
|
||||
assert "fixture-value-not-for-display" not in summary
|
||||
assert "[redacted]" in summary
|
||||
assert len(summary) < 3000
|
||||
|
||||
|
||||
def test_run_and_preview_summary_exposes_steps_and_next_action(environment):
|
||||
run = environment.create()["result"]
|
||||
shown = environment.call("show", run["run_id"])
|
||||
summary = "\n".join(shown["summary"])
|
||||
assert "Steps: 2 pending." in summary
|
||||
assert "Step core:preflight: pending." in summary
|
||||
assert "Next: execute_step [core:preflight]" in summary
|
||||
preview = environment.call("preview", run["run_id"], "core:tag")
|
||||
assert "Release preview: pending." in preview["summary"]
|
||||
assert any("Complete core:preflight first." in line for line in preview["summary"])
|
||||
|
||||
|
||||
def test_status_summary_exposes_repository_counts(environment):
|
||||
environment.dashboard.return_value = {"summary": {"status": "attention", "repository_count": 77, "dirty_count": 42, "ahead_count": 1, "behind_count": 0, "error_count": 0}}
|
||||
result = environment.call("status")
|
||||
assert "Release status: attention." in result["summary"]
|
||||
assert any("77 repositories, 42 dirty, 1 ahead, 0 behind, 0 errors" in line for line in result["summary"])
|
||||
|
||||
|
||||
def test_display_redaction_cannot_change_semantic_failure_exit(environment, monkeypatch):
|
||||
monkeypatch.setenv("DEVKIT_FIXTURE_SECRET", "blocked")
|
||||
environment.dashboard.return_value = {"summary": {"status": "blocked"}}
|
||||
result = environment.call("status")
|
||||
assert result["_exit_code"] == 1
|
||||
assert "Release status: [redacted]." in result["summary"]
|
||||
|
||||
|
||||
def test_create_defaults_to_preview_without_run_record(environment):
|
||||
result = environment.call("create", "--repo", "govoplan-core", "--target-version", "1.2.3", "--request-id", "preview-create-request-0001")
|
||||
assert result["_exit_code"] == 0 and result["dry_run"]
|
||||
assert not list(environment.state.rglob("rr-*.json"))
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_create_show_and_same_id_replay_use_durable_service(environment):
|
||||
created = environment.create()
|
||||
run_id = created["result"]["run_id"]
|
||||
repeated = environment.create()
|
||||
assert repeated["result"]["run_id"] == run_id
|
||||
environment.planner.assert_called_once()
|
||||
shown = environment.call("show", run_id)
|
||||
assert shown["result"]["immutable"] == created["result"]["immutable"]
|
||||
assert shown["result"]["state"]["steps"][0]["executor"]["confirmation"] == ""
|
||||
assert len(list(environment.state.rglob("rr-*.json"))) == 1
|
||||
|
||||
|
||||
def test_create_replay_rejects_changed_inputs(environment):
|
||||
environment.create()
|
||||
result = environment.call("create", "--repo-version", "govoplan-core=1.2.4", "--request-id", "devkit-create-request-0001", "--apply")
|
||||
assert result["http_status"] == 409
|
||||
environment.planner.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("arguments", [
|
||||
("plan",),
|
||||
("create", "--repo", "govoplan-core", "--request-id", "missing-version-request"),
|
||||
("plan", "--repo-version", "govoplan-core=1.2.3", "--repo-version", "govoplan-core=1.2.4"),
|
||||
("plan", "--repo-version", "govoplan-core=not-a-version"),
|
||||
])
|
||||
def test_invalid_selection_is_rejected_before_service_collection(environment, arguments):
|
||||
result = environment.call(*arguments)
|
||||
assert result["_exit_code"] == 2
|
||||
environment.dashboard.assert_not_called()
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_runtime_trust_guard_cannot_be_bypassed_by_cli(environment, monkeypatch):
|
||||
monkeypatch.setattr(api, "require_trusted_release_runtime", Mock(side_effect=ReleaseExecutionBlocked("Fixture untrusted runtime")))
|
||||
result = environment.call("create", "--repo-version", "govoplan-core=1.2.3", "--request-id", "trust-create-request-0001", "--apply")
|
||||
assert result["http_status"] == 409
|
||||
assert not list(environment.state.rglob("rr-*.json"))
|
||||
environment.planner.assert_not_called()
|
||||
|
||||
|
||||
def test_dry_execute_and_generic_preview_never_claim_or_execute(environment):
|
||||
run = environment.create()["result"]
|
||||
dry = environment.call("execute", run["run_id"], "core:preflight", "--request-id", "dry-execute-request-0001")
|
||||
preview = environment.call("preview", run["run_id"], "core:tag")
|
||||
assert dry["dry_run"] and preview["dry_run"]
|
||||
assert preview["result"]["state_step"]["executor"]["confirmation"] == "TAG"
|
||||
assert preview["result"]["state_step"]["available"] is False
|
||||
environment.executor.assert_not_called()
|
||||
assert environment.call("show", run["run_id"])["result"]["state"]["steps"][0]["attempt_count"] == 0
|
||||
|
||||
|
||||
def test_prerequisite_and_confirmation_guards_remain_enforced(environment):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
blocked = environment.call("execute", run_id, "core:tag", "--request-id", "ordered-tag-request-0001", "--confirm", "TAG", "--apply")
|
||||
assert blocked["http_status"] == 409
|
||||
environment.executor.assert_not_called()
|
||||
assert environment.call("execute", run_id, "core:preflight", "--request-id", "preflight-request-0001", "--apply")["_exit_code"] == 0
|
||||
missing = environment.call("execute", run_id, "core:tag", "--request-id", "confirmed-tag-request-0001", "--apply")
|
||||
assert missing["http_status"] == 409
|
||||
assert environment.executor.call_count == 1
|
||||
|
||||
|
||||
def test_execute_same_attempt_replays_without_repeating_effect(environment):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
arguments = ("execute", run_id, "core:preflight", "--request-id", "exact-attempt-request-0001", "--apply")
|
||||
first = environment.call(*arguments)
|
||||
second = environment.call(*arguments)
|
||||
assert first["_exit_code"] == second["_exit_code"] == 0
|
||||
assert second["result"]["execution_result"]["status"] == "replayed"
|
||||
environment.executor.assert_called_once()
|
||||
assert environment.executor.call_args.kwargs["remote"] == "origin"
|
||||
|
||||
|
||||
def test_lost_finish_write_is_interrupted_without_reexecuting_effect(environment, monkeypatch):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
finish = ReleaseRunStore.finish_step
|
||||
monkeypatch.setattr(ReleaseRunStore, "finish_step", Mock(side_effect=ReleaseRunCorrupt("Fixture durable write failure")))
|
||||
arguments = ("execute", run_id, "core:preflight", "--request-id", "lost-finish-request-0001", "--apply")
|
||||
interrupted = environment.call(*arguments)
|
||||
assert interrupted["http_status"] == 409
|
||||
monkeypatch.setattr(ReleaseRunStore, "finish_step", finish)
|
||||
assert environment.call(*arguments)["http_status"] == 409
|
||||
environment.executor.assert_called_once()
|
||||
shown = environment.call("show", run_id)["result"]
|
||||
assert shown["state"]["steps"][0]["state"] == "interrupted"
|
||||
|
||||
|
||||
def test_source_guard_failure_does_not_call_executor(environment, monkeypatch):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
monkeypatch.setattr(api, "verify_repository_preflight_binding", Mock(side_effect=ReleaseExecutionBlocked("Frozen HEAD/remote changed")))
|
||||
result = environment.call("execute", run_id, "core:preflight", "--request-id", "changed-source-request-0001", "--apply")
|
||||
assert result["_exit_code"] == 1
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_interrupted_write_requires_reconciliation_not_retry(environment, monkeypatch):
|
||||
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
|
||||
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture remote outcome uncertain")
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
execute = ("execute", run_id, "core:tag", "--request-id", "uncertain-tag-request-0001", "--confirm", "TAG", "--apply")
|
||||
uncertain = environment.call(*execute)
|
||||
assert uncertain["http_status"] == 409
|
||||
assert environment.call(*execute)["http_status"] == 409
|
||||
retry = environment.call("retry", run_id, "core:tag", "--request-id", "unsafe-retry-request-0001", "--apply")
|
||||
assert retry["http_status"] == 409
|
||||
environment.executor.assert_called_once()
|
||||
invalid = environment.call("reconcile", run_id, "core:tag", "--request-id", "bad-reconcile-request-0001", "--outcome", "effect_absent", "--apply")
|
||||
assert invalid["http_status"] == 409
|
||||
reconciled = environment.call("reconcile", run_id, "core:tag", "--request-id", "reconcile-absent-request-0001", "--outcome", "effect_absent", "--confirm", "RECONCILE", "--apply")
|
||||
assert reconciled["_exit_code"] == 0
|
||||
assert reconciled["result"]["state"]["steps"][0]["state"] == "pending"
|
||||
|
||||
|
||||
def test_effect_succeeded_reconciliation_keeps_independent_receipt_guard(environment, monkeypatch):
|
||||
environment.plans[0]["dry_run_steps"] = [environment.plans[0]["dry_run_steps"][1]]
|
||||
environment.executor.side_effect = ReleaseExecutionAmbiguous("Fixture interruption")
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
environment.call("execute", run_id, "core:tag", "--request-id", "receipt-tag-request-0001", "--confirm", "TAG", "--apply")
|
||||
guard = Mock(side_effect=ReleaseExecutionBlocked("Remote annotation mismatch"))
|
||||
monkeypatch.setattr(api, "reconciled_repository_receipt", guard)
|
||||
result = environment.call("reconcile", run_id, "core:tag", "--request-id", "receipt-success-request-0001", "--outcome", "effect_succeeded", "--confirm", "RECONCILE", "--apply")
|
||||
assert result["http_status"] == 409
|
||||
guard.assert_called_once()
|
||||
environment.executor.assert_called_once()
|
||||
|
||||
|
||||
def test_resume_and_retry_reuse_the_existing_running_attempt_rules(environment):
|
||||
created = environment.create()
|
||||
run_id = created["result"]["run_id"]
|
||||
store = ReleaseRunStore(Path(created["state_location"]), expected_workspace_fingerprint=api.release_workspace_fingerprint(environment.workspace))
|
||||
store.claim_step(run_id, "core:preflight", attempt_id="lost-process-request-0001")
|
||||
dry = environment.call("resume", run_id, "--request-id", "resume-process-request-0001")
|
||||
assert dry["dry_run"]
|
||||
assert store.get(run_id)["state"]["steps"][0]["state"] == "running"
|
||||
resumed = environment.call("resume", run_id, "--request-id", "resume-process-request-0001", "--apply")
|
||||
assert resumed["result"]["state"]["steps"][0]["state"] == "interrupted"
|
||||
retried = environment.call("retry", run_id, "core:preflight", "--request-id", "retry-readonly-request-0001", "--apply")
|
||||
assert retried["result"]["state"]["steps"][0]["state"] == "pending"
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_workspace_scoping_and_corrupt_records_fail_closed(environment, tmp_path):
|
||||
created = environment.create()
|
||||
run_id = created["result"]["run_id"]
|
||||
another = tmp_path / "another-workspace"
|
||||
another.mkdir()
|
||||
foreign = environment.call("show", run_id, workspace_root=another)
|
||||
assert foreign["http_status"] == 404
|
||||
path = Path(created["state_location"]) / f"{run_id}.json"
|
||||
record = json.loads(path.read_text())
|
||||
record["immutable"]["input"]["repo_versions"]["govoplan-core"] = "9.9.9"
|
||||
path.write_text(json.dumps(record))
|
||||
assert environment.call("show", run_id)["http_status"] == 409
|
||||
|
||||
|
||||
def test_run_list_keeps_cursor_pagination(environment):
|
||||
environment.create("page-create-request-0001")
|
||||
environment.create("page-create-request-0002")
|
||||
first = environment.call("list", "--limit", "1")["result"]
|
||||
second = environment.call("list", "--limit", "1", "--cursor", first["next_cursor"])["result"]
|
||||
assert len(first["runs"]) == len(second["runs"]) == 1
|
||||
assert first["runs"][0]["run_id"] != second["runs"][0]["run_id"]
|
||||
assert second["next_cursor"] is None
|
||||
|
||||
|
||||
def test_catalog_preview_calls_only_existing_receipt_bound_preview(environment, monkeypatch, tmp_path):
|
||||
environment.plans[0] = plan(catalog=True)
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
candidate = tmp_path / "candidate"
|
||||
verify = Mock(return_value=candidate)
|
||||
publish = Mock(return_value={"status": "planned", "apply": False})
|
||||
monkeypatch.setattr(api, "verified_run_candidate", verify)
|
||||
monkeypatch.setattr(api, "publish_catalog_candidate", publish)
|
||||
result = environment.call("preview", run_id, "catalog:validate-sign-publish")
|
||||
assert result["_exit_code"] == 0
|
||||
verify.assert_called_once()
|
||||
assert publish.call_args.kwargs["apply"] is False
|
||||
assert publish.call_args.kwargs["candidate_dir"] == candidate
|
||||
assert publish.call_args.kwargs["remote"] == "origin"
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_catalog_generation_and_publication_use_exact_durable_receipts(environment, monkeypatch):
|
||||
environment.plans[0] = plan(catalog=True)
|
||||
created = environment.create()
|
||||
run_id = created["result"]["run_id"]
|
||||
candidate_root = Path(created["candidate_location"])
|
||||
|
||||
def generated(**kwargs):
|
||||
candidate_id = kwargs["candidate_id"]
|
||||
candidate = candidate_output_path(candidate_root, candidate_id)
|
||||
channels = candidate / "channels"
|
||||
channels.mkdir(parents=True)
|
||||
(channels / "stable.json").write_text(json.dumps({"channel": "stable", "signatures": [{}]}))
|
||||
harden_private_candidate_tree(candidate)
|
||||
return {"status": "ready"}, issue_candidate_receipt(root=candidate_root, candidate_id=candidate_id, channel="stable")
|
||||
|
||||
def published(**kwargs):
|
||||
candidate = kwargs["candidate_receipt"]
|
||||
website = kwargs["expected_website_receipt"]
|
||||
return {"status": "published"}, {
|
||||
"kind": "catalog_publication", "candidate_id": candidate["candidate_id"],
|
||||
"catalog_sha256": candidate["catalog_sha256"], "keyring_sha256": "c" * 64,
|
||||
"publication_commit_sha": "d" * 40, "publication_tag_object_sha": "e" * 40,
|
||||
"publication_tag_commit_sha": "d" * 40, "branch": website["branch"],
|
||||
"tag_name": "catalog-stable-1", "remote": "origin", "remote_sha256": website["remote_sha256"],
|
||||
}
|
||||
|
||||
generator = Mock(side_effect=generated)
|
||||
publisher = Mock(side_effect=published)
|
||||
website = receipt("addideas-govoplan-website", target_tag="")
|
||||
monkeypatch.setattr(api, "generate_catalog_candidate", generator)
|
||||
monkeypatch.setattr(api, "publish_received_candidate", publisher)
|
||||
monkeypatch.setattr(api, "verify_catalog_publication_precondition", Mock(return_value=website))
|
||||
generate = ("execute", run_id, "catalog:selective-generator", "--request-id", "generate-candidate-request-0001", "--confirm", "GENERATE", "--apply")
|
||||
first = environment.call(*generate, "--signing-key", "fixture-key=/private/fixture-key.pem")
|
||||
assert first["_exit_code"] == 0, first
|
||||
replayed = environment.call(*generate)
|
||||
assert replayed["result"]["execution_result"]["status"] == "replayed"
|
||||
generator.assert_called_once()
|
||||
assert generator.call_args.kwargs["signing_keys"] == ("fixture-key=/private/fixture-key.pem",)
|
||||
assert "/private/fixture-key.pem" not in json.dumps(first)
|
||||
record_text = next(environment.state.rglob("rr-*.json")).read_text()
|
||||
assert "/private/fixture-key.pem" not in record_text
|
||||
missing_confirmation = environment.call("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--apply")
|
||||
assert missing_confirmation["http_status"] == 409
|
||||
publisher.assert_not_called()
|
||||
publish = ("execute", run_id, "catalog:validate-sign-publish", "--request-id", "publish-candidate-request-0001", "--confirm", "PUSH", "--apply")
|
||||
complete = environment.call(*publish)
|
||||
assert complete["_exit_code"] == 0, complete
|
||||
assert complete["result"]["state"]["status"] == "completed"
|
||||
frozen = first["result"]["state"]["steps"][0]["result_receipt"]
|
||||
assert publisher.call_args.kwargs["candidate_path"] == candidate_root / frozen["candidate_id"]
|
||||
assert publisher.call_args.kwargs["candidate_receipt"] == frozen
|
||||
assert publisher.call_args.kwargs["expected_website_receipt"] == website
|
||||
assert publisher.call_args.kwargs["remote"] == "origin"
|
||||
assert environment.call(*publish)["result"]["execution_result"]["status"] == "replayed"
|
||||
publisher.assert_called_once()
|
||||
|
||||
|
||||
def test_portable_project_cannot_override_release_catalog(environment):
|
||||
args = namespace(environment.workspace, environment.state, "plan", "--repo-version", "govoplan-core=1.2.3")
|
||||
args.project = environment.workspace / "custom-project.json"
|
||||
result = release.handle(args)
|
||||
assert result["_exit_code"] == 2
|
||||
assert "does not accept --project" in result["summary"][0]
|
||||
environment.dashboard.assert_not_called()
|
||||
|
||||
|
||||
def test_foreign_cached_release_package_is_rejected_before_import(environment, monkeypatch, tmp_path):
|
||||
monkeypatch.setitem(sys.modules, "server.app", SimpleNamespace(__file__=str(tmp_path / "foreign/app.py")))
|
||||
importer = Mock(side_effect=AssertionError("Foreign module import must not occur"))
|
||||
monkeypatch.setattr(release.importlib, "import_module", importer)
|
||||
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3")
|
||||
assert result["_exit_code"] == 2
|
||||
assert "foreign module" in result["summary"][0]
|
||||
importer.assert_not_called()
|
||||
environment.dashboard.assert_not_called()
|
||||
|
||||
|
||||
def test_no_arbitrary_remote_or_legacy_publication_flags():
|
||||
run_id = "rr-request-" + "a" * 64
|
||||
with pytest.raises(SystemExit):
|
||||
namespace(Path("/fixture"), None, "execute", run_id, "core:tag", "--request-id", "arbitrary-remote-request", "--remote", "untrusted")
|
||||
with pytest.raises(SystemExit):
|
||||
namespace(Path("/fixture"), None, "publish-candidate", "--candidate-dir", "/untrusted")
|
||||
|
||||
|
||||
def test_unknown_step_and_malformed_paths_do_not_become_other_routes(environment):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
assert environment.call("preview", run_id, "unknown:step")["http_status"] == 404
|
||||
with pytest.raises(SystemExit):
|
||||
namespace(environment.workspace, environment.state, "preview", run_id, "../repositories/push")
|
||||
|
||||
|
||||
def test_signing_material_is_not_echoed_by_validation(environment):
|
||||
run_id = environment.create()["result"]["run_id"]
|
||||
secret = "PRIVATE-KEY-MATERIAL\nDO-NOT-ECHO"
|
||||
result = environment.call("execute", run_id, "core:preflight", "--request-id", "secret-input-request-0001", "--signing-key", secret, "--apply")
|
||||
assert result["_exit_code"] == 2
|
||||
assert secret not in json.dumps(result)
|
||||
assert "SECRET" not in release._error_detail({"detail": [{"loc": ["body", "signing_keys"], "msg": "Invalid input", "input": "SECRET"}]})
|
||||
environment.executor.assert_not_called()
|
||||
|
||||
|
||||
def test_default_state_matches_console_and_token_never_appears_in_output(environment, monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
|
||||
monkeypatch.setattr(release.secrets, "token_urlsafe", lambda size: "ephemeral-do-not-print-token")
|
||||
result = environment.call("plan", "--repo-version", "govoplan-core=1.2.3", state_dir=None)
|
||||
assert result["state_location"] == str(api.default_release_run_root(environment.workspace))
|
||||
assert "ephemeral-do-not-print-token" not in json.dumps(result)
|
||||
|
||||
|
||||
def test_documentation_keeps_disabled_generic_mutations_and_recovery_explicit():
|
||||
console = (ROOT / "docs/operations/RELEASE_CONSOLE.md").read_text()
|
||||
usage = (ROOT / "docs/operations/DEVKIT_RELEASE.md").read_text()
|
||||
assert "generic push, sync and prepare\nmutation endpoints are disabled" in console
|
||||
assert "ASGI application inside" in usage
|
||||
assert "effect_absent" in usage and "effect_succeeded" in usage and "unresolved" in usage
|
||||
assert "does **not** stage arbitrary source" in usage
|
||||
Executable
+175
@@ -0,0 +1,175 @@
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import review
|
||||
from govoplan_devkit.cli import main
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def args(tmp_path):
|
||||
repo = tmp_path / "optional-feature"
|
||||
repo.mkdir()
|
||||
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
||||
paths = ["webui/src/pages/CampaignPage.tsx", "webui/src/dialogs/SettingsDialog.tsx",
|
||||
"webui/src/settings/AdminSettings.tsx", "webui/src/widgets/SummaryWidget.tsx", "webui/src/module.ts"]
|
||||
for relative in paths:
|
||||
path = repo / relative
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("// fixture only\n")
|
||||
manifest = repo / "src/optional_feature/backend/manifest.py"
|
||||
manifest.parent.mkdir(parents=True)
|
||||
manifest.write_text("raise RuntimeError('MUST NEVER IMPORT MODULE CODE')\n")
|
||||
(tmp_path / "principles.md").write_text("# Principles\nRevision **UI-2026-09-08**\n\n## UI-01 — Help beside text\n\n## UI-02 — Display first; edit deliberately\n")
|
||||
inventory = {"schema_version": 1, "epic": {"repository": "meta", "number": 4, "url": "https://gitea.invalid/team/meta/issues/4"},
|
||||
"issues": [{"scope_id": "campaigns", "name": "Campaign", "kind": "manifest", "repository": "optional-feature", "number": 8,
|
||||
"url": "https://gitea.invalid/team/optional-feature/issues/8", "state_at_verification": "closed", "operation": "created"}]}
|
||||
(tmp_path / "issues.json").write_text(json.dumps(inventory))
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(json.dumps({"schema_version": 1, "name": "Portable project", "repositories": [
|
||||
{"name": "optional-feature", "path": "optional-feature", "aliases": ["campaign"]},
|
||||
{"name": "absent-optional", "path": "absent-optional", "aliases": ["absent"]}],
|
||||
"review": {"issue_inventory": "issues.json", "principles": "principles.md"},
|
||||
"checks": [{"id": "ui-check", "argv": ["never-execute-this"], "repos": ["optional-feature"], "cwd": "optional-feature"}],
|
||||
"profiles": {"ui": ["ui-check"]}}))
|
||||
return argparse.Namespace(workspace_root=tmp_path, project=project, state_dir=tmp_path / "state",
|
||||
module="campaign", bundle_module=None, profile="ui", evidence=None, output=None)
|
||||
|
||||
|
||||
def test_bundle_contains_guidance_links_revision_and_unexecuted_check_plan(args):
|
||||
result = review.handle_review(args)
|
||||
assert result["module"] == "optional-feature"
|
||||
assert result["inventory"]["ui_source_count"] == 5
|
||||
assert not result["inventory"]["module_code_imported"]
|
||||
assert result["inventory"]["manifest_paths"] == ["src/optional_feature/backend/manifest.py"]
|
||||
assert result["principles"]["revision"] == "UI-2026-09-08"
|
||||
assert [rule["id"] for rule in result["principles"]["rules"]] == ["UI-01", "UI-02"]
|
||||
assert result["check_plan"]["stages"][0]["argv"] == ["never-execute-this"]
|
||||
assert not result["check_plan"]["executed"]
|
||||
assert result["review_completion"].startswith("Not assessed")
|
||||
assert len(result["manual_checklist"]) >= 9
|
||||
assert all("checked" not in item for item in result["manual_checklist"])
|
||||
assert "state_at_verification" not in json.dumps(result)
|
||||
assert "operation" not in result["issue_links"][0]
|
||||
assert result["central_issue"]["number"] == 4
|
||||
assert not args.state_dir.exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["campaign", "campaigns", "optional-feature"])
|
||||
def test_alias_scope_id_and_repository_resolve_to_same_registered_module(args, name):
|
||||
args.module = name
|
||||
assert review.handle_review(args)["module"] == "optional-feature"
|
||||
|
||||
|
||||
def test_bundle_alias_and_direct_cli_work_with_global_options_anywhere(args, capsys):
|
||||
assert main(["--workspace-root", str(args.workspace_root), "review", "campaign", "--project", str(args.project), "--json"]) == 0
|
||||
direct = json.loads(capsys.readouterr().out)
|
||||
assert direct["module"] == "optional-feature"
|
||||
assert main(["review", "bundle", "campaigns", "--workspace-root", str(args.workspace_root), "--project", str(args.project), "--format", "json"]) == 0
|
||||
assert json.loads(capsys.readouterr().out)["module"] == direct["module"]
|
||||
|
||||
|
||||
def test_absent_optional_repository_and_no_ui_do_not_complete_review(args):
|
||||
args.module = "absent"
|
||||
result = review.handle_review(args)
|
||||
assert not result["repository"]["exists"]
|
||||
assert result["repository"]["errors"]
|
||||
assert result["inventory"]["ui_source_count"] == 0
|
||||
assert result["check_plan"]["stages"] == []
|
||||
assert result["review_completion"].startswith("Not assessed")
|
||||
assert any("not automatic N/A" in warning for warning in result["warnings"])
|
||||
assert any("not a passing" in warning for warning in result["warnings"])
|
||||
|
||||
|
||||
def test_portable_project_does_not_inherit_govoplan_links_or_principles(args):
|
||||
payload = json.loads(args.project.read_text())
|
||||
payload.pop("review")
|
||||
args.project.write_text(json.dumps(payload))
|
||||
result = review.handle_review(args)
|
||||
assert result["issue_links"] == [] and result["central_issue"] is None
|
||||
assert not result["principles"]["available"]
|
||||
assert "git.add-ideas.de" not in json.dumps(result)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["symlink", "outside", "bad-url", "duplicate-scope", "foreign-repository"])
|
||||
def test_review_inputs_cannot_escape_or_silently_misbind_scope(args, kind):
|
||||
if kind in {"symlink", "outside"}:
|
||||
payload = json.loads(args.project.read_text())
|
||||
if kind == "symlink":
|
||||
alias = args.workspace_root / "alias.json"
|
||||
alias.symlink_to(args.workspace_root / "issues.json")
|
||||
payload["review"]["issue_inventory"] = "alias.json"
|
||||
else:
|
||||
payload["review"]["principles"] = "../outside.md"
|
||||
args.project.write_text(json.dumps(payload))
|
||||
else:
|
||||
path = args.workspace_root / "issues.json"
|
||||
payload = json.loads(path.read_text())
|
||||
if kind == "bad-url":
|
||||
payload["issues"][0]["url"] = "https://gitea.invalid/team/wrong/issues/8"
|
||||
elif kind == "duplicate-scope":
|
||||
payload["issues"].append(payload["issues"][0])
|
||||
else:
|
||||
payload["issues"][0].update(repository="foreign", url="https://gitea.invalid/team/foreign/issues/8")
|
||||
args.module = "campaigns"
|
||||
path.write_text(json.dumps(payload))
|
||||
with pytest.raises(ValueError):
|
||||
review.handle_review(args)
|
||||
|
||||
|
||||
def test_source_inventory_never_follows_symlinked_ancestors(args):
|
||||
root = args.workspace_root / "indirect"
|
||||
root.mkdir()
|
||||
(root / "webui").symlink_to(args.workspace_root / "optional-feature/webui", target_is_directory=True)
|
||||
result = review.source_inventory(root)
|
||||
assert result["ui_source_count"] == 0
|
||||
assert result["skipped_symlinks"] == ["webui/src"]
|
||||
|
||||
|
||||
def test_explicit_artifact_is_only_local_and_preserves_issue_discovery_snapshot(args):
|
||||
inventory = args.workspace_root / "issues.json"
|
||||
original = inventory.read_bytes()
|
||||
args.output = args.workspace_root / "artifacts/review.json"
|
||||
result = review.handle_review(args)
|
||||
assert json.loads(args.output.read_text()) == result
|
||||
assert inventory.read_bytes() == original
|
||||
assert args.output.stat().st_mode & 0o777 == 0o600
|
||||
assert not args.state_dir.exists()
|
||||
|
||||
|
||||
def test_attached_historical_receipt_stays_separate_from_current_review(args):
|
||||
path = args.workspace_root / "historical.json"
|
||||
path.write_text(json.dumps({"schema_version": 1, "run_id": "historical", "workspace_root": str(args.workspace_root),
|
||||
"project_file": str(args.project), "source_fingerprint": "a" * 64, "status": "passed", "snapshot_verified": True, "generated_at": "2026-09-08",
|
||||
"finished_at": "2026-09-08", "stages": [{"id": "old-check", "status": "passed", "exit_code": 0, "duration_seconds": 1, "log_path": "/not-read"}]}))
|
||||
args.evidence = str(path)
|
||||
result = review.handle_review(args)
|
||||
assert result["evidence"]["origin"] == "external-unverified"
|
||||
assert result["evidence"]["source_state"] == "historical-source-differs"
|
||||
assert result["review_completion"].startswith("Not assessed")
|
||||
assert not result["check_plan"]["executed"]
|
||||
|
||||
|
||||
def test_unknown_module_and_extra_positionals_are_errors(args):
|
||||
args.module = "typo"
|
||||
with pytest.raises(ValueError, match="Unknown repository"):
|
||||
review.handle_review(args)
|
||||
args.module, args.bundle_module = "campaign", "extra"
|
||||
with pytest.raises(ValueError, match="one module"):
|
||||
review.handle_review(args)
|
||||
|
||||
|
||||
def test_compact_review_exposes_scoped_coverage_limits_without_hiding_full_list(args, monkeypatch):
|
||||
notes = [f"Separate suite {number} was not included." for number in range(12)]
|
||||
monkeypatch.setattr("govoplan_devkit.catalog.build_stages", lambda *_args, **_kwargs: [
|
||||
{"id": "source-only", "argv": ["never-execute"], "coverage_notes": notes}])
|
||||
result = review.handle_review(args)
|
||||
assert result["coverage_notes"] == notes
|
||||
assert sum(line.startswith("Coverage limitation:") for line in result["summary"]) == 8
|
||||
assert any("4 additional coverage limitations" in line for line in result["summary"])
|
||||
assert not result["check_plan"]["executed"]
|
||||
Executable
+466
@@ -0,0 +1,466 @@
|
||||
"""Real fixture processes and Git worktrees; never invoke product/remote mutations."""
|
||||
|
||||
from argparse import Namespace
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit import runner
|
||||
from govoplan_devkit.checkpoints import Checkpoints
|
||||
from govoplan_devkit.common import (
|
||||
atomic_json,
|
||||
digest,
|
||||
read_json,
|
||||
resource_lock,
|
||||
state_root,
|
||||
)
|
||||
from govoplan_devkit.workspace import load_project, source_fingerprint
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def example(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path / "xdg-state"))
|
||||
workspace = tmp_path / "workspace"
|
||||
repo = workspace / "example"
|
||||
repo.mkdir(parents=True)
|
||||
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
||||
(repo / "source.txt").write_text("first\n")
|
||||
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"-qm",
|
||||
"fixture",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
project = tmp_path / "project.json"
|
||||
project.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "Example",
|
||||
"repositories": [{"name": "example", "path": "example"}],
|
||||
"checks": [],
|
||||
"profiles": {},
|
||||
}
|
||||
)
|
||||
)
|
||||
args = Namespace(
|
||||
workspace_root=workspace,
|
||||
project=project,
|
||||
state_dir=tmp_path / "state",
|
||||
dry_run=False,
|
||||
jobs=2,
|
||||
profile="quick",
|
||||
resume=None,
|
||||
)
|
||||
return args, repo
|
||||
|
||||
|
||||
def stage(repo, name="one", code="print('ok')", **extra):
|
||||
return {
|
||||
"id": name,
|
||||
"title": name,
|
||||
"argv": [sys.executable, "-c", code],
|
||||
"cwd": str(repo),
|
||||
"timeout_seconds": 10,
|
||||
**extra,
|
||||
}
|
||||
|
||||
|
||||
def run(args, stages):
|
||||
# Environment inspection is tested separately; keep fixture executions cheap/deterministic.
|
||||
with patch.object(runner, "environment_fingerprint", return_value="fixture-env"):
|
||||
return runner.run_checks(args, stages)
|
||||
|
||||
|
||||
def test_success_receipt_and_compact_summary(example):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
assert result["status"] == "passed"
|
||||
receipt = runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
|
||||
assert receipt["stages"][0]["exit_code"] == 0
|
||||
assert Path(receipt["stages"][0]["log_path"]).read_text() == "ok\n"
|
||||
assert runner.summarize(receipt)["counts"] == {"passed": 1}
|
||||
assert Path(result["receipt_path"]).stat().st_mode & 0o777 == 0o600
|
||||
|
||||
|
||||
def test_native_environment_binds_selected_checkout(tmp_path, monkeypatch):
|
||||
from govoplan_devkit.environment import execution_environment
|
||||
|
||||
monkeypatch.setenv("GOVOPLAN_WORKSPACE_ROOT", "/another/workspace")
|
||||
monkeypatch.setenv("GOVOPLAN_CORE_ROOT", "/another/core")
|
||||
monkeypatch.setenv("GOVOPLAN_CORE_SOURCE_ROOT", "/another/source")
|
||||
project = load_project(tmp_path)
|
||||
env = execution_environment(
|
||||
tmp_path, project, {"python": sys.executable, "node": "node", "npm": "npm"}
|
||||
)
|
||||
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(tmp_path)
|
||||
assert env["GOVOPLAN_CORE_ROOT"] == str(tmp_path / "govoplan-core")
|
||||
assert env["GOVOPLAN_CORE_SOURCE_ROOT"] == str(tmp_path / "govoplan-core")
|
||||
|
||||
|
||||
def test_portable_environment_does_not_inject_govoplan_scope(example, monkeypatch):
|
||||
from govoplan_devkit.environment import execution_environment
|
||||
|
||||
args, _ = example
|
||||
for key in (
|
||||
"GOVOPLAN_WORKSPACE_ROOT",
|
||||
"GOVOPLAN_CORE_ROOT",
|
||||
"GOVOPLAN_CORE_SOURCE_ROOT",
|
||||
):
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
env = execution_environment(
|
||||
args.workspace_root,
|
||||
load_project(args.workspace_root, args.project),
|
||||
{"python": sys.executable, "node": "node", "npm": "npm"},
|
||||
)
|
||||
assert "GOVOPLAN_WORKSPACE_ROOT" not in env
|
||||
|
||||
|
||||
def test_failure_skips_dependents_but_runs_independent_check(example):
|
||||
args, repo = example
|
||||
result = run(
|
||||
args,
|
||||
[
|
||||
stage(repo, "bad", "raise SystemExit(4)"),
|
||||
stage(repo, "dependent", deps=["bad"]),
|
||||
stage(repo, "independent"),
|
||||
],
|
||||
)
|
||||
states = {item["id"]: item["status"] for item in result["stages"]}
|
||||
assert states == {"bad": "failed", "dependent": "skipped", "independent": "passed"}
|
||||
assert result["_exit_code"] == 1
|
||||
|
||||
|
||||
def test_timeout_is_not_a_pass(example):
|
||||
args, repo = example
|
||||
result = run(
|
||||
args, [stage(repo, code="import time; time.sleep(10)", timeout_seconds=0.1)]
|
||||
)
|
||||
assert result["stages"][0]["status"] == "timed_out"
|
||||
|
||||
|
||||
def test_dry_run_does_not_create_state(example):
|
||||
args, repo = example
|
||||
args.dry_run = True
|
||||
result = run(args, [stage(repo, code="raise SystemExit(9)")])
|
||||
assert result["status"] == "planned"
|
||||
assert not args.state_dir.exists()
|
||||
|
||||
|
||||
def test_empty_plan_is_not_claimed_as_verified(example):
|
||||
args, _ = example
|
||||
assert run(args, [])["status"] == "not_run"
|
||||
|
||||
|
||||
def test_plan_validation_rejects_cycles_unknowns_duplicate_ids_escape(example):
|
||||
args, repo = example
|
||||
invalid = [
|
||||
[stage(repo, deps=["unknown"])],
|
||||
[stage(repo, "a", deps=["b"]), stage(repo, "b", deps=["a"])],
|
||||
[stage(repo), stage(repo)],
|
||||
[stage(repo, "../escape")],
|
||||
[stage(repo.parent.parent)],
|
||||
]
|
||||
for plan in invalid:
|
||||
with pytest.raises(ValueError):
|
||||
runner.validate_stages(plan, args.workspace_root, {})
|
||||
|
||||
|
||||
def test_source_identity_includes_staged_unstaged_and_untracked_bytes(example):
|
||||
args, repo = example
|
||||
project = load_project(args.workspace_root, args.project)
|
||||
first = source_fingerprint(project)
|
||||
(repo / "source.txt").write_text("second\n")
|
||||
second = source_fingerprint(project)
|
||||
assert second != first
|
||||
subprocess.run(["git", "-C", str(repo), "add", "source.txt"], check=True)
|
||||
third = source_fingerprint(project)
|
||||
assert third != second
|
||||
(repo / "extra.txt").write_text("extra")
|
||||
assert source_fingerprint(project) != third
|
||||
|
||||
|
||||
def test_source_mutation_during_run_invalidates_receipt(example):
|
||||
args, repo = example
|
||||
result = run(
|
||||
args,
|
||||
[
|
||||
stage(
|
||||
repo,
|
||||
code="from pathlib import Path; Path('source.txt').write_text('changed')",
|
||||
)
|
||||
],
|
||||
)
|
||||
assert result["stages"][0]["status"] == "stale"
|
||||
assert result["stages"][0]["checkpoint_verified"] is False
|
||||
assert result["status"] == "stale"
|
||||
|
||||
|
||||
def test_resume_reuses_only_identical_plan_and_inputs(example):
|
||||
args, repo = example
|
||||
plan = [stage(repo)]
|
||||
first = run(args, plan)
|
||||
args.resume = first["run_id"]
|
||||
second = run(args, plan)
|
||||
assert second["stages"][0]["reused_from"] == first["run_id"]
|
||||
changed_plan = run(args, [stage(repo, code="print('different')")])
|
||||
assert changed_plan["status"] == "passed"
|
||||
assert "reused_from" not in changed_plan["stages"][0]
|
||||
assert Path(changed_plan["stages"][0]["log_path"]).read_text() == "different\n"
|
||||
(repo / "source.txt").write_text("changed")
|
||||
changed_source = run(args, plan)
|
||||
assert changed_source["status"] == "passed"
|
||||
assert "reused_from" not in changed_source["stages"][0]
|
||||
assert changed_source["stages"][0]["cache_key"] != first["stages"][0]["cache_key"]
|
||||
|
||||
|
||||
def test_receipt_integrity_and_foreign_id_are_rejected(example):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
path = Path(result["receipt_path"])
|
||||
record = read_json(path)
|
||||
record["status"] = "fabricated"
|
||||
atomic_json(path, record)
|
||||
with pytest.raises(ValueError, match="integrity"):
|
||||
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
|
||||
with pytest.raises(ValueError):
|
||||
runner.read_receipt(args.workspace_root, args.state_dir, "../other")
|
||||
|
||||
|
||||
def test_shared_resources_serialize_stages(example):
|
||||
args, repo = example
|
||||
active = 0
|
||||
maximum = 0
|
||||
lock = threading.Lock()
|
||||
original = runner._execute_stage_command
|
||||
|
||||
def execute(*args, **kwargs):
|
||||
nonlocal active, maximum
|
||||
with lock:
|
||||
active += 1
|
||||
maximum = max(maximum, active)
|
||||
try:
|
||||
time.sleep(0.04)
|
||||
return original(*args, **kwargs)
|
||||
finally:
|
||||
with lock:
|
||||
active -= 1
|
||||
|
||||
with patch.object(runner, "_execute_stage_command", side_effect=execute):
|
||||
result = run(
|
||||
args,
|
||||
[
|
||||
stage(repo, "a", resources=["shared"]),
|
||||
stage(repo, "b", resources=["shared"]),
|
||||
],
|
||||
)
|
||||
assert result["status"] == "passed"
|
||||
assert maximum == 1
|
||||
|
||||
|
||||
def test_cross_process_resource_conflict_is_explicit(example):
|
||||
args, repo = example
|
||||
locks = state_root(args.workspace_root) / "resource-locks"
|
||||
with resource_lock(locks, "shared"):
|
||||
result = run(args, [stage(repo, resources=["shared"])])
|
||||
assert result["stages"][0]["status"] == "blocked"
|
||||
|
||||
|
||||
def test_output_is_bounded_and_known_secrets_redacted(example, monkeypatch):
|
||||
args, repo = example
|
||||
monkeypatch.setenv("FIXTURE_SECRET", "private-fixture-credential")
|
||||
with patch.object(runner, "MAX_LOG_BYTES", 1024):
|
||||
result = run(
|
||||
args,
|
||||
[
|
||||
stage(
|
||||
repo,
|
||||
code="import os; print(os.environ['FIXTURE_SECRET']); print('x'*10000)",
|
||||
)
|
||||
],
|
||||
)
|
||||
record = result["stages"][0]
|
||||
output = Path(record["log_path"]).read_text()
|
||||
assert record["output_truncated"] is True
|
||||
assert "private-fixture-credential" not in output
|
||||
assert "[redacted]" in output
|
||||
assert len(output) < 2000
|
||||
|
||||
|
||||
def test_symlinked_state_is_rejected(example, tmp_path):
|
||||
args, repo = example
|
||||
target = tmp_path / "target"
|
||||
target.mkdir()
|
||||
args.state_dir.symlink_to(target, target_is_directory=True)
|
||||
with pytest.raises(ValueError, match="symlink"):
|
||||
run(args, [stage(repo)])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("flag", ["--assume-unchanged", "--skip-worktree"])
|
||||
def test_hidden_tracked_edits_invalidate_source_identity(example, flag):
|
||||
args, repo = example
|
||||
subprocess.run(
|
||||
["git", "-C", str(repo), "update-index", flag, "source.txt"], check=True
|
||||
)
|
||||
project = load_project(args.workspace_root, args.project)
|
||||
initial = source_fingerprint(project)
|
||||
(repo / "source.txt").write_text("hidden edit\n")
|
||||
assert source_fingerprint(project) != initial
|
||||
|
||||
|
||||
def test_failed_final_snapshot_never_persists_a_passing_receipt(example):
|
||||
args, repo = example
|
||||
events = []
|
||||
args.on_progress = events.append
|
||||
original = Checkpoints.source
|
||||
|
||||
def fingerprint(self, *values, **kwargs):
|
||||
if events and events[-1]["phase"] == "finalizing":
|
||||
raise ValueError("unreadable source")
|
||||
return original(self, *values, **kwargs)
|
||||
|
||||
with patch.object(Checkpoints, "source", fingerprint):
|
||||
result = run(args, [stage(repo)])
|
||||
assert result["status"] == "stale"
|
||||
assert result["snapshot_verified"] is False
|
||||
assert result["_exit_code"] == 1
|
||||
assert "unreadable source" in result["invalidated_reason"]
|
||||
assert (
|
||||
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])[
|
||||
"status"
|
||||
]
|
||||
== "stale"
|
||||
)
|
||||
|
||||
|
||||
def test_resume_rejects_a_modified_cached_log(example):
|
||||
args, repo = example
|
||||
plan = [stage(repo)]
|
||||
first = run(args, plan)
|
||||
Path(first["stages"][0]["log_path"]).write_text("altered")
|
||||
args.resume = first["run_id"]
|
||||
result = run(args, plan)
|
||||
assert result["status"] == "failed"
|
||||
assert result["stages"][0]["status"] == "failed"
|
||||
assert "log integrity" in result["stages"][0]["error"]
|
||||
assert "reused_from" not in result["stages"][0]
|
||||
|
||||
|
||||
def test_receipt_argument_redaction_applies_to_unknown_separate_values(example):
|
||||
args, repo = example
|
||||
plan = stage(repo)
|
||||
plan["argv"].extend(["--password", "fixture-not-from-environment"])
|
||||
result = run(args, [plan])
|
||||
assert (
|
||||
"fixture-not-from-environment" not in Path(result["receipt_path"]).read_text()
|
||||
)
|
||||
|
||||
|
||||
def recovery_parser():
|
||||
parser = argparse.ArgumentParser()
|
||||
runner.register(parser.add_subparsers())
|
||||
return parser
|
||||
|
||||
|
||||
def test_recovery_requires_manual_survivor_confirmation(example):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
record = read_json(Path(result["receipt_path"]))
|
||||
record["status"] = "running"
|
||||
record["snapshot_verified"] = False
|
||||
atomic_json(Path(result["receipt_path"]), runner._seal(record))
|
||||
parsed = recovery_parser().parse_args(
|
||||
["recover", result["run_id"], "--apply"], namespace=args
|
||||
)
|
||||
with pytest.raises(ValueError, match="manual verification"):
|
||||
parsed.handler(parsed)
|
||||
|
||||
|
||||
def test_recovery_does_not_overwrite_completion_between_reads(example):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
completed = runner.read_receipt(
|
||||
args.workspace_root, args.state_dir, result["run_id"]
|
||||
)
|
||||
initial = {**completed, "status": "running"}
|
||||
parsed = recovery_parser().parse_args(
|
||||
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
|
||||
namespace=args,
|
||||
)
|
||||
with (
|
||||
patch.object(runner, "read_receipt", side_effect=[initial, completed]),
|
||||
patch.object(runner, "atomic_json") as write,
|
||||
):
|
||||
assert parsed.handler(parsed)["status"] == "unchanged"
|
||||
write.assert_not_called()
|
||||
|
||||
|
||||
def test_active_owner_lock_blocks_recovery(example):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
record = read_json(Path(result["receipt_path"]))
|
||||
record.update(status="running", snapshot_verified=False)
|
||||
atomic_json(Path(result["receipt_path"]), runner._seal(record))
|
||||
parsed = recovery_parser().parse_args(
|
||||
["recover", result["run_id"], "--apply", "--confirm-processes-stopped"],
|
||||
namespace=args,
|
||||
)
|
||||
with resource_lock(
|
||||
state_root(args.workspace_root, args.state_dir) / "locks",
|
||||
"run:" + result["run_id"],
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="busy"):
|
||||
parsed.handler(parsed)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"mutation",
|
||||
[
|
||||
"unknown_status",
|
||||
"unverified_pass",
|
||||
"duplicate_stage",
|
||||
"nonboolean_verification",
|
||||
"nonpassing_stage",
|
||||
],
|
||||
)
|
||||
def test_even_correctly_hashed_receipts_must_have_consistent_states(example, mutation):
|
||||
args, repo = example
|
||||
result = run(args, [stage(repo)])
|
||||
path = Path(result["receipt_path"])
|
||||
record = read_json(path)
|
||||
if mutation == "unknown_status":
|
||||
record["status"] = "wonderful"
|
||||
elif mutation == "unverified_pass":
|
||||
record["snapshot_verified"] = False
|
||||
elif mutation == "duplicate_stage":
|
||||
record["stages"].append(record["stages"][0])
|
||||
elif mutation == "nonboolean_verification":
|
||||
record["snapshot_verified"] = "true"
|
||||
else:
|
||||
record["stages"][0]["status"] = "failed"
|
||||
# Deliberately construct an externally corrupted but correctly hashed record;
|
||||
# the runner's writer now rejects inconsistent checkpoint state before sealing.
|
||||
record["integrity_sha256"] = digest(
|
||||
{key: value for key, value in record.items() if key != "integrity_sha256"}
|
||||
)
|
||||
atomic_json(path, record)
|
||||
with pytest.raises(ValueError):
|
||||
runner.read_receipt(args.workspace_root, args.state_dir, result["run_id"])
|
||||
Executable
+251
@@ -0,0 +1,251 @@
|
||||
"""Published-schema parity and all-declaration validation, without commands."""
|
||||
|
||||
from copy import deepcopy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import jsonschema
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/devkit"))
|
||||
from govoplan_devkit.workspace import load_project # noqa: E402
|
||||
from govoplan_devkit.validation import _schema_value # noqa: E402
|
||||
|
||||
|
||||
def valid():
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"repositories": [{"name": "example", "path": "example"}],
|
||||
"checks": [
|
||||
{"id": "selected", "argv": ["true"]},
|
||||
{"id": "unselected", "argv": ["true"]},
|
||||
],
|
||||
"profiles": {"quick": ["selected"], "full": ["unselected"]},
|
||||
}
|
||||
|
||||
|
||||
def check(tmp_path, value):
|
||||
path = tmp_path / "project.json"
|
||||
path.write_text(json.dumps(value))
|
||||
return load_project(tmp_path, path)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field,value",
|
||||
[
|
||||
("resource", ["shared"]),
|
||||
("timout_seconds", 20),
|
||||
("cwd", "../escape"),
|
||||
("cwd", "/outside"),
|
||||
("argv", [""]),
|
||||
("argv", ["true", "x" * 8193]),
|
||||
("argv", ["true"] * 257),
|
||||
("argv", ["true", "bad\0value"]),
|
||||
("resources", ["a", "a"]),
|
||||
("resources", [""]),
|
||||
("resources", ["x" * 257]),
|
||||
("resources", ["bad\nname"]),
|
||||
("resources", True),
|
||||
("deps", ["selected", "selected"]),
|
||||
("repos", ["example", "example"]),
|
||||
("title", 10),
|
||||
("title", "x" * 1025),
|
||||
("timeout_seconds", True),
|
||||
("timeout_seconds", 43201),
|
||||
("timeout_seconds", 0),
|
||||
("id", "invalid\n"),
|
||||
],
|
||||
)
|
||||
def test_unselected_checks_obey_the_published_schema(tmp_path, field, value):
|
||||
project = valid()
|
||||
project["checks"][1][field] = value
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
with pytest.raises(jsonschema.ValidationError):
|
||||
jsonschema.validate(project, schema)
|
||||
with pytest.raises(ValueError):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"mutation",
|
||||
[
|
||||
lambda value: value["profiles"].update(typo=[]),
|
||||
lambda value: value["profiles"].update(full=["unselected", "unselected"]),
|
||||
lambda value: value["tools"].update(pyton="python"),
|
||||
lambda value: value["tools"].update(python="bad\0tool"),
|
||||
lambda value: value["review"].update(principle="rules.md"),
|
||||
lambda value: value["review"].update(principles="../rules.md"),
|
||||
lambda value: value["repositories"][0].update(alias="other"),
|
||||
lambda value: value["repositories"][0].update(aliases=["alias", "alias"]),
|
||||
lambda value: value.update(checks=value["checks"] * 257),
|
||||
lambda value: value.update(schema_version=True),
|
||||
],
|
||||
)
|
||||
def test_all_nested_shape_constraints_match_schema(tmp_path, mutation):
|
||||
project = valid()
|
||||
project.update(tools={}, review={})
|
||||
mutation(project)
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
with pytest.raises(jsonschema.ValidationError):
|
||||
jsonschema.validate(project, schema)
|
||||
with pytest.raises(ValueError):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"mutation,reason",
|
||||
[
|
||||
(
|
||||
lambda value: value["checks"][1].update(deps=["missing"]),
|
||||
"Unknown check dependency",
|
||||
),
|
||||
(lambda value: value["checks"][1].update(deps=["unselected"]), "Cyclic"),
|
||||
(
|
||||
lambda value: value["checks"][1].update(repos=["missing"]),
|
||||
"Unknown repository",
|
||||
),
|
||||
(lambda value: value["profiles"].update(full=["missing"]), "Unknown profile"),
|
||||
(lambda value: value["checks"][1].update(id="selected"), "Duplicate"),
|
||||
(
|
||||
lambda value: value["repositories"].append(
|
||||
{"name": "other", "path": "example"}
|
||||
),
|
||||
"Duplicate project repository path",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_all_semantic_references_are_checked_even_outside_selected_profile(
|
||||
tmp_path, mutation, reason
|
||||
):
|
||||
project = valid()
|
||||
mutation(project)
|
||||
with pytest.raises(ValueError, match=reason):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
def test_valid_defaults_boundaries_and_dependency_order(tmp_path):
|
||||
project = valid()
|
||||
project["checks"][0].update(
|
||||
deps=["unselected"], argv=["true", ""], timeout_seconds=0.1
|
||||
)
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
jsonschema.validate(project, schema)
|
||||
assert check(tmp_path, project).config == project
|
||||
|
||||
|
||||
def test_runtime_shape_uses_only_implemented_schema_keywords():
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
permitted = {
|
||||
"$schema",
|
||||
"$defs",
|
||||
"$ref",
|
||||
"title",
|
||||
"description",
|
||||
"type",
|
||||
"const",
|
||||
"properties",
|
||||
"additionalProperties",
|
||||
"required",
|
||||
"minItems",
|
||||
"maxItems",
|
||||
"uniqueItems",
|
||||
"prefixItems",
|
||||
"items",
|
||||
"minLength",
|
||||
"maxLength",
|
||||
"pattern",
|
||||
"maximum",
|
||||
"exclusiveMinimum",
|
||||
"enum",
|
||||
}
|
||||
|
||||
def visit(node):
|
||||
assert set(node) <= permitted
|
||||
for group in ("properties", "$defs"):
|
||||
for child in node.get(group, {}).values():
|
||||
visit(child)
|
||||
if "items" in node:
|
||||
visit(node["items"])
|
||||
for child in node.get("prefixItems", []):
|
||||
visit(child)
|
||||
|
||||
visit(schema)
|
||||
_schema_value(valid(), schema, schema["$defs"], "project")
|
||||
|
||||
|
||||
def test_huge_numeric_input_is_a_controlled_bound_error(tmp_path):
|
||||
project = deepcopy(valid())
|
||||
project["checks"][1]["timeout_seconds"] = 10**1000
|
||||
with pytest.raises(ValueError):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field,value",
|
||||
[
|
||||
("inputs", {}),
|
||||
("inputs", None),
|
||||
("inputs", {"repos": []}),
|
||||
("inputs", {"paths": ["src/**"]}),
|
||||
("inputs", {"repos": ["example", "example"]}),
|
||||
("inputs", {"repos": [""]}),
|
||||
("inputs", {"repos": ["example"], "glob": "*"}),
|
||||
("after", ["selected", "selected"]),
|
||||
("after", [""]),
|
||||
("after", "selected"),
|
||||
("reuse", True),
|
||||
("reuse", "always"),
|
||||
],
|
||||
)
|
||||
def test_input_order_and_reuse_shapes_apply_to_unselected_checks(
|
||||
tmp_path, field, value
|
||||
):
|
||||
project = valid()
|
||||
project["checks"][1][field] = value
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
with pytest.raises(jsonschema.ValidationError):
|
||||
jsonschema.validate(project, schema)
|
||||
with pytest.raises(ValueError):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"change,reason",
|
||||
[
|
||||
({"inputs": {"repos": ["unknown"]}}, "Unknown input repository"),
|
||||
({"after": ["unknown"]}, "Unknown check ordering"),
|
||||
({"after": ["unselected"]}, "Cyclic"),
|
||||
(
|
||||
{"deps": ["selected"], "after": ["selected"]},
|
||||
"Duplicate dependency/ordering",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_input_and_order_semantics_are_validated_before_selection(
|
||||
tmp_path, change, reason
|
||||
):
|
||||
project = valid()
|
||||
project["checks"][1].update(change)
|
||||
with pytest.raises(ValueError, match=reason):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
def test_dependencies_and_order_only_edges_share_cycle_validation(tmp_path):
|
||||
project = valid()
|
||||
project["checks"][0]["after"] = ["unselected"]
|
||||
project["checks"][1]["deps"] = ["selected"]
|
||||
with pytest.raises(ValueError, match="Cyclic"):
|
||||
check(tmp_path, project)
|
||||
|
||||
|
||||
def test_valid_explicit_inputs_after_and_reuse_match_published_schema(tmp_path):
|
||||
project = valid()
|
||||
project["checks"][0].update(
|
||||
inputs={"repos": ["example"]}, after=["unselected"], reuse="verified"
|
||||
)
|
||||
project["checks"][1]["reuse"] = "never"
|
||||
schema = json.loads((ROOT / "tools/devkit/project.schema.json").read_text())
|
||||
jsonschema.validate(project, schema)
|
||||
assert check(tmp_path, project).config == project
|
||||
Executable
+151
@@ -0,0 +1,151 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools/devkit"))
|
||||
from govoplan_devkit.common import atomic_json, read_json, redact, safe_output
|
||||
from govoplan_devkit.context import build_context
|
||||
from govoplan_devkit.workspace import (
|
||||
load_project,
|
||||
selected_repositories,
|
||||
git_bytes,
|
||||
inspect_repository,
|
||||
Repository,
|
||||
)
|
||||
|
||||
|
||||
def test_portable_project_rejects_escape_alias_collisions_and_duplicate_json(tmp_path):
|
||||
config = tmp_path / "project.json"
|
||||
for repositories in (
|
||||
[{"name": "repo", "path": "../outside"}],
|
||||
[
|
||||
{"name": "repo", "path": "repo", "aliases": ["other"]},
|
||||
{"name": "other", "path": "other"},
|
||||
],
|
||||
):
|
||||
config.write_text(
|
||||
json.dumps({"schema_version": 1, "repositories": repositories})
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
load_project(tmp_path, config)
|
||||
config.write_text('{"schema_version":1,"schema_version":2}')
|
||||
with pytest.raises(ValueError, match="Duplicate"):
|
||||
read_json(config)
|
||||
|
||||
|
||||
def test_missing_repository_is_error_not_clean(tmp_path):
|
||||
config = tmp_path / "project.json"
|
||||
config.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "Test",
|
||||
"repositories": [{"name": "missing", "path": "missing"}],
|
||||
}
|
||||
)
|
||||
)
|
||||
result = build_context(tmp_path, config, [], True)
|
||||
assert result["_exit_code"] == 1
|
||||
assert result["repositories"][0]["errors"]
|
||||
assert not result["remote_checked"]
|
||||
|
||||
|
||||
def test_unknown_repo_filter_is_not_silently_ignored(tmp_path):
|
||||
config = tmp_path / "project.json"
|
||||
config.write_text(
|
||||
json.dumps(
|
||||
{"schema_version": 1, "repositories": [{"name": "repo", "path": "repo"}]}
|
||||
)
|
||||
)
|
||||
with pytest.raises(ValueError, match="Unknown"):
|
||||
selected_repositories(load_project(tmp_path, config), ["typo"])
|
||||
|
||||
|
||||
def test_atomic_output_does_not_chmod_existing_parent(tmp_path):
|
||||
folder = tmp_path / "public"
|
||||
folder.mkdir(mode=0o755)
|
||||
atomic_json(folder / "private.json", {"safe": True})
|
||||
assert folder.stat().st_mode & 0o777 == 0o755
|
||||
assert (folder / "private.json").stat().st_mode & 0o777 == 0o600
|
||||
|
||||
|
||||
def test_read_rejects_fifo_and_symlink(tmp_path):
|
||||
fifo = tmp_path / "fifo"
|
||||
os.mkfifo(fifo)
|
||||
with pytest.raises(ValueError, match="regular"):
|
||||
read_json(fifo)
|
||||
target = tmp_path / "target.json"
|
||||
target.write_text("{}")
|
||||
alias = tmp_path / "alias.json"
|
||||
alias.symlink_to(target)
|
||||
with pytest.raises(ValueError, match="symlink"):
|
||||
read_json(alias)
|
||||
|
||||
|
||||
def test_generic_secret_display_hygiene():
|
||||
assert "topsecret" not in redact(
|
||||
"Authorization: Bearer topsecret\nhttps://user:topsecret@example.invalid/"
|
||||
)
|
||||
|
||||
|
||||
def test_separate_secret_arguments_are_redacted_in_nested_json():
|
||||
value = {
|
||||
"stages": [{"argv": ["check", "--token", "hidden-value"]}],
|
||||
"password": "hidden-password",
|
||||
}
|
||||
output = json.dumps(safe_output(value))
|
||||
assert "hidden-value" not in output and "hidden-password" not in output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[
|
||||
"GIT_DIR",
|
||||
"GIT_NAMESPACE",
|
||||
"GIT_COMMON_DIR",
|
||||
"GIT_CONFIG_PARAMETERS",
|
||||
"GIT_AUTHOR_NAME",
|
||||
],
|
||||
)
|
||||
def test_inherited_git_redirects_are_rejected(tmp_path, monkeypatch, name):
|
||||
monkeypatch.setenv(name, "fixture")
|
||||
assert inspect_repository(Repository("example", tmp_path))["errors"]
|
||||
with pytest.raises(ValueError, match="overrides"):
|
||||
git_bytes(tmp_path, "status")
|
||||
|
||||
|
||||
def test_deeply_nested_json_fails_as_controlled_input_error(tmp_path):
|
||||
path = tmp_path / "deep.json"
|
||||
path.write_text("[" * 10000 + "]" * 10000)
|
||||
with pytest.raises(ValueError, match="nesting"):
|
||||
read_json(path)
|
||||
|
||||
|
||||
def test_clean_commit_without_upstream_is_still_changed(tmp_path):
|
||||
subprocess.run(["git", "init", "-q", str(tmp_path)], check=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(tmp_path),
|
||||
"-c",
|
||||
"user.name=Fixture",
|
||||
"-c",
|
||||
"user.email=fixture@example.invalid",
|
||||
"commit",
|
||||
"--allow-empty",
|
||||
"-qm",
|
||||
"fixture",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
from govoplan_devkit.workspace import Project
|
||||
|
||||
repo = Repository("example", tmp_path)
|
||||
assert selected_repositories(Project("Example", (repo,), {}), [], changed=True) == [
|
||||
repo
|
||||
]
|
||||
Executable
+393
@@ -0,0 +1,393 @@
|
||||
"""Canonical phase dispatch tested with inert tools in disposable workspaces."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = META_ROOT / "tools/checks/check-focused.sh"
|
||||
METADATA = META_ROOT / "tools/checks/focused-phases.json"
|
||||
PHASE_IDS = [
|
||||
"preflight",
|
||||
"tooling",
|
||||
"backend",
|
||||
"core-ui",
|
||||
"module-builds",
|
||||
"browser",
|
||||
"module-ui",
|
||||
]
|
||||
# These hashes bind the original working-copy check bodies at the phase split.
|
||||
# Only two explicit Core/webui cd lines were added for independent invocation.
|
||||
# Future deliberate changes to the canonical gate must update this contract.
|
||||
LEGACY_BODY_SHA256 = {
|
||||
"preflight": "b9c90fd3c84df788de5f2c001443672f683a9918459e1a7955ed4a225e6f20dd",
|
||||
"tooling": "3d9a5bf32acbe97134f51327ed4b2457a69ad23b723c15a2b9bd0dce7f82396b",
|
||||
"backend": "dd57c33919f06bd240516bbe47861be022e0b1c047334eedc7f6c596c2c49632",
|
||||
"core-ui": "613f806a602970f1dbfb243f4c96ccc6ca4836849bb1f1a2c4cff97aea6f3aeb",
|
||||
"module-builds": "1f2cd1e2c336f748fbf2972a2da87a0efec58ef9b084a9433511f794a456fa84",
|
||||
"browser": "aa20d1e1d4ec9f00bc27c06cdee7ffa2456d3d3b8a7155da4e888a7e2209306d",
|
||||
"module-ui": "c47794d82eb7bb47de114e86a95264e5898886f743c31eb7bdf82a079c827c9f",
|
||||
}
|
||||
EXTRA_TEST_COMMAND = '"$PYTHON" -m pytest -q tests/test_focused_phases.py\n'
|
||||
|
||||
|
||||
def definitions():
|
||||
source = SCRIPT.read_text()
|
||||
return dict(
|
||||
re.findall(
|
||||
r"^# devkit-phase: ([a-z-]+) begin\n(.*?)^# devkit-phase: \1 end$",
|
||||
source,
|
||||
re.MULTILINE | re.DOTALL,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_original_commands_remain_exactly_once_in_original_order():
|
||||
bodies = definitions()
|
||||
metadata = json.loads(METADATA.read_text())
|
||||
assert list(bodies) == PHASE_IDS
|
||||
assert [phase["id"] for phase in metadata["phases"]] == PHASE_IDS
|
||||
assert SCRIPT.read_text().count(EXTRA_TEST_COMMAND) == 1
|
||||
for identity, body in bodies.items():
|
||||
assert SCRIPT.read_text().count(f"# devkit-phase: {identity} begin") == 1
|
||||
if identity == "tooling":
|
||||
assert body.count(EXTRA_TEST_COMMAND) == 1
|
||||
body = body.replace(EXTRA_TEST_COMMAND, "")
|
||||
assert hashlib.sha256(body.encode()).hexdigest() == LEGACY_BODY_SHA256[identity]
|
||||
|
||||
|
||||
def test_ordering_and_artifact_dependencies_are_distinct():
|
||||
phases = json.loads(METADATA.read_text())["phases"]
|
||||
assert [phase["order_after"] for phase in phases] == [
|
||||
[],
|
||||
*[[identity] for identity in PHASE_IDS[:-1]],
|
||||
]
|
||||
assert all(phase["depends_on"] == [] for phase in phases)
|
||||
by_id = {phase["id"]: phase for phase in phases}
|
||||
assert "{core}/webui/dist" in by_id["module-builds"]["outputs"]
|
||||
assert any(
|
||||
"does not serve or require module-builds dist" in note
|
||||
for note in by_id["browser"]["notes"]
|
||||
)
|
||||
assert "port:4174" in by_id["browser"]["resources"]
|
||||
cwd_lines = {
|
||||
"core": 'cd "$ROOT"',
|
||||
"meta": 'cd "$META_ROOT"',
|
||||
"core-webui": 'cd "$ROOT/webui"',
|
||||
"access-webui": 'cd "${WORKSPACE_ROOT}/govoplan-access/webui"',
|
||||
}
|
||||
for phase in phases:
|
||||
assert definitions()[phase["id"]].splitlines()[0] == cwd_lines[phase["cwd"]]
|
||||
|
||||
|
||||
FAKE_TOOL = r"""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
log = Path(os.environ["FOCUSED_FIXTURE_LOG"])
|
||||
record = {
|
||||
"tool": Path(sys.argv[0]).name,
|
||||
"argv": sys.argv[1:],
|
||||
"cwd": os.getcwd(),
|
||||
"env": {key: os.environ.get(key) for key in (
|
||||
"GOVOPLAN_WORKSPACE_ROOT", "NPM_CONFIG_USERCONFIG", "GOVOPLAN_NPM_USERCONFIG",
|
||||
"NPM_CONFIG_TMP", "npm_config_tmp", "PYTHONPATH", "PATH",
|
||||
)},
|
||||
}
|
||||
if "-" in sys.argv[1:]:
|
||||
record["stdin"] = sys.stdin.read()
|
||||
with log.open("a") as handle:
|
||||
handle.write(json.dumps(record) + "\n")
|
||||
if os.environ.get("FOCUSED_FIXTURE_FAIL_TOKEN") in sys.argv[1:]:
|
||||
raise SystemExit(7)
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fixture_workspace(tmp_path):
|
||||
workspace = tmp_path / "workspace"
|
||||
meta = workspace / "govoplan"
|
||||
core = workspace / "govoplan-core"
|
||||
copied = meta / "tools/checks/check-focused.sh"
|
||||
copied.parent.mkdir(parents=True)
|
||||
shutil.copyfile(SCRIPT, copied)
|
||||
shutil.copyfile(METADATA, copied.with_name("focused-phases.json"))
|
||||
for name in (
|
||||
"govoplan",
|
||||
"govoplan-core",
|
||||
"govoplan-access",
|
||||
"govoplan-payments",
|
||||
"govoplan-dataflow",
|
||||
"govoplan-datasources",
|
||||
"govoplan-workflow",
|
||||
"govoplan-dashboard",
|
||||
"govoplan-approvals",
|
||||
"govoplan-postbox",
|
||||
"govoplan-mail",
|
||||
"govoplan-files",
|
||||
"govoplan-campaign",
|
||||
"govoplan-policy",
|
||||
"govoplan-wiki",
|
||||
):
|
||||
(workspace / name / "webui").mkdir(parents=True, exist_ok=True)
|
||||
(workspace / name / "src").mkdir(exist_ok=True)
|
||||
(meta / "tests").mkdir()
|
||||
for name in ("test_devkit_alpha.py", "test_devkit_beta.py"):
|
||||
(meta / "tests" / name).write_text("# inert glob fixture\n")
|
||||
fake_bin = tmp_path / "bin"
|
||||
fake_bin.mkdir()
|
||||
for target in [
|
||||
*(fake_bin / name for name in ("python-check", "node", "npm", "bash")),
|
||||
core / "webui/node_modules/.bin/tsc",
|
||||
meta / "tools/checks/check_dependency_boundaries.py",
|
||||
]:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(f"#!{sys.executable}\n" + FAKE_TOOL)
|
||||
target.chmod(0o700)
|
||||
temporary = tmp_path / "temporary"
|
||||
temporary.mkdir()
|
||||
log = tmp_path / "calls.jsonl"
|
||||
env = {
|
||||
**os.environ,
|
||||
"PATH": str(fake_bin) + os.pathsep + os.environ["PATH"],
|
||||
"GOVOPLAN_WORKSPACE_ROOT": str(workspace),
|
||||
"GOVOPLAN_CORE_ROOT": str(core),
|
||||
"PYTHON": str(fake_bin / "python-check"),
|
||||
"NODE": str(fake_bin / "node"),
|
||||
"NPM": str(fake_bin / "npm"),
|
||||
"TMPDIR": str(temporary),
|
||||
"FOCUSED_FIXTURE_LOG": str(log),
|
||||
"PYTHONPATH": "inherited-fixture-tail",
|
||||
"NPM_CONFIG_TMP": "must-be-unset",
|
||||
"npm_config_tmp": "must-be-unset",
|
||||
}
|
||||
env.pop("FOCUSED_FIXTURE_FAIL_TOKEN", None)
|
||||
return {
|
||||
"workspace": workspace,
|
||||
"meta": meta,
|
||||
"core": core,
|
||||
"script": copied,
|
||||
"env": env,
|
||||
"log": log,
|
||||
"temporary": temporary,
|
||||
}
|
||||
|
||||
|
||||
def invoke(fixture, *arguments, env=None):
|
||||
return subprocess.run(
|
||||
["/bin/bash", str(fixture["script"]), *arguments],
|
||||
cwd=fixture["workspace"].parent,
|
||||
env=env or fixture["env"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
|
||||
def records(fixture):
|
||||
path = fixture["log"]
|
||||
return (
|
||||
[json.loads(line) for line in path.read_text().splitlines()]
|
||||
if path.exists()
|
||||
else []
|
||||
)
|
||||
|
||||
|
||||
def normalized(calls):
|
||||
result = []
|
||||
for call in calls:
|
||||
call = json.loads(json.dumps(call))
|
||||
call["env"].pop("NPM_CONFIG_USERCONFIG")
|
||||
call["env"].pop("GOVOPLAN_NPM_USERCONFIG")
|
||||
result.append(call)
|
||||
return result
|
||||
|
||||
|
||||
def test_default_gate_equals_sequential_independent_phases(fixture_workspace):
|
||||
fixture = fixture_workspace
|
||||
full = invoke(fixture)
|
||||
assert full.returncode == 0, full.stderr
|
||||
original = records(fixture)
|
||||
offset = len(original)
|
||||
for identity in PHASE_IDS:
|
||||
isolated = invoke(fixture, "--phase", identity)
|
||||
assert isolated.returncode == 0, isolated.stderr
|
||||
assert normalized(original) == normalized(records(fixture)[offset:])
|
||||
assert len({call["env"]["NPM_CONFIG_USERCONFIG"] for call in original}) == len(
|
||||
PHASE_IDS
|
||||
)
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
assert sum("test:conformance" in call["argv"] for call in original) == 1
|
||||
assert sum("test:module-permutations" in call["argv"] for call in original) == 1
|
||||
assert sum("tests/test_focused_phases.py" in call["argv"] for call in original) == 1
|
||||
# The here-document remains one Python invocation, not executed shell text.
|
||||
ast_scan = [call for call in original if "stdin" in call]
|
||||
assert len(ast_scan) == 1
|
||||
assert "AST syntax check passed for" in ast_scan[0]["stdin"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("identity", PHASE_IDS)
|
||||
def test_each_phase_initializes_its_own_cwd_and_environment(
|
||||
fixture_workspace, identity
|
||||
):
|
||||
fixture = fixture_workspace
|
||||
result = invoke(fixture, "--phase", identity)
|
||||
assert result.returncode == 0, result.stderr
|
||||
calls = records(fixture)
|
||||
assert calls
|
||||
phase = next(
|
||||
item
|
||||
for item in json.loads(METADATA.read_text())["phases"]
|
||||
if item["id"] == identity
|
||||
)
|
||||
starts = {
|
||||
"core": fixture["core"],
|
||||
"meta": fixture["meta"],
|
||||
"core-webui": fixture["core"] / "webui",
|
||||
"access-webui": fixture["workspace"] / "govoplan-access/webui",
|
||||
}
|
||||
assert calls[0]["cwd"] == str(starts[phase["cwd"]])
|
||||
for call in calls:
|
||||
env = call["env"]
|
||||
assert env["GOVOPLAN_WORKSPACE_ROOT"] == str(fixture["workspace"])
|
||||
assert env["NPM_CONFIG_USERCONFIG"] == env["GOVOPLAN_NPM_USERCONFIG"]
|
||||
assert Path(env["NPM_CONFIG_USERCONFIG"]).parent == fixture["temporary"]
|
||||
assert not Path(env["NPM_CONFIG_USERCONFIG"]).exists()
|
||||
assert env["NPM_CONFIG_TMP"] is None and env["npm_config_tmp"] is None
|
||||
assert env["PATH"].split(os.pathsep)[0] == str(
|
||||
fixture["core"] / "webui/node_modules/.bin"
|
||||
)
|
||||
assert env["PYTHONPATH"].endswith(os.pathsep + "inherited-fixture-tail")
|
||||
assert str(fixture["core"] / "src") in env["PYTHONPATH"].split(os.pathsep)
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
|
||||
|
||||
def test_full_is_fail_fast_and_failure_cleans_setup(fixture_workspace):
|
||||
fixture = fixture_workspace
|
||||
result = invoke(
|
||||
fixture,
|
||||
env={
|
||||
**fixture["env"],
|
||||
"FOCUSED_FIXTURE_FAIL_TOKEN": "test:module-permutations",
|
||||
},
|
||||
)
|
||||
assert result.returncode == 7
|
||||
calls = records(fixture)
|
||||
assert calls[-1]["argv"] == ["run", "test:module-permutations"]
|
||||
assert not any("test:conformance" in call["argv"] for call in calls)
|
||||
assert not any("test:passwords" in call["argv"] for call in calls)
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"arguments",
|
||||
[
|
||||
["--unknown"],
|
||||
["--phase"],
|
||||
["--phase", ""],
|
||||
["--phase", "not-a-phase"],
|
||||
["--phase", "browser", "--phase", "tooling"],
|
||||
["--list-phases", "--phase", "browser"],
|
||||
["--phase", "browser", "--list-phases"],
|
||||
["--list-phases", "--list-phases"],
|
||||
["--json"],
|
||||
["--list-phases", "--json", "--json"],
|
||||
["--phase", "browser; touch unexpected"],
|
||||
],
|
||||
)
|
||||
def test_invalid_selection_is_rejected_before_setup(fixture_workspace, arguments):
|
||||
fixture = fixture_workspace
|
||||
env = {
|
||||
**fixture["env"],
|
||||
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
|
||||
"NODE": "absent-node",
|
||||
"NPM": "absent-npm",
|
||||
}
|
||||
result = invoke(fixture, *arguments, env=env)
|
||||
assert result.returncode == 2
|
||||
assert "check-focused:" in result.stderr
|
||||
assert not fixture["log"].exists()
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"arguments",
|
||||
[["--list-phases"], ["--list-phases", "--json"], ["--json", "--list-phases"]],
|
||||
)
|
||||
def test_listing_is_read_only_without_product_tools(fixture_workspace, arguments):
|
||||
fixture = fixture_workspace
|
||||
env = {
|
||||
**fixture["env"],
|
||||
"GOVOPLAN_CORE_ROOT": str(fixture["workspace"] / "absent-core"),
|
||||
"PYTHON": "/absent-python",
|
||||
"NODE": "absent-node",
|
||||
"NPM": "absent-npm",
|
||||
}
|
||||
result = invoke(fixture, *arguments, env=env)
|
||||
assert result.returncode == 0, result.stderr
|
||||
if "--json" in arguments:
|
||||
assert json.loads(result.stdout) == json.loads(METADATA.read_text())
|
||||
else:
|
||||
assert [line.split("\t")[0] for line in result.stdout.splitlines()] == PHASE_IDS
|
||||
assert not fixture["log"].exists()
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
|
||||
|
||||
def test_metadata_loading_never_imports_project_python_modules(fixture_workspace):
|
||||
fixture = fixture_workspace
|
||||
foreign = fixture["workspace"].parent / "json.py"
|
||||
foreign.write_text(
|
||||
"raise AssertionError('project module imported during listing')\n"
|
||||
)
|
||||
result = invoke(
|
||||
fixture,
|
||||
"--list-phases",
|
||||
"--json",
|
||||
env={**fixture["env"], "PYTHONPATH": str(foreign.parent)},
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert json.loads(result.stdout)["schema_version"] == 1
|
||||
assert not fixture["log"].exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"change",
|
||||
[
|
||||
"unknown-fields",
|
||||
"duplicate-id",
|
||||
"unknown-cwd",
|
||||
"later-prerequisite",
|
||||
"invalid-version",
|
||||
],
|
||||
)
|
||||
def test_malformed_metadata_fails_before_any_check(fixture_workspace, change):
|
||||
fixture = fixture_workspace
|
||||
path = fixture["script"].with_name("focused-phases.json")
|
||||
catalog = json.loads(path.read_text())
|
||||
if change == "unknown-fields":
|
||||
catalog["phases"][0]["shell"] = "touch should-not-run"
|
||||
elif change == "duplicate-id":
|
||||
catalog["phases"][1]["id"] = catalog["phases"][0]["id"]
|
||||
elif change == "unknown-cwd":
|
||||
catalog["phases"][0]["cwd"] = "elsewhere"
|
||||
elif change == "later-prerequisite":
|
||||
catalog["phases"][0]["depends_on"] = ["browser"]
|
||||
else:
|
||||
catalog["schema_version"] = True
|
||||
path.write_text(json.dumps(catalog))
|
||||
result = invoke(fixture, "--phase", "browser")
|
||||
assert result.returncode == 2
|
||||
assert not fixture["log"].exists()
|
||||
assert not list(fixture["temporary"].iterdir())
|
||||
@@ -0,0 +1,240 @@
|
||||
"""Local mixed-owner admission/recovery evidence, not production capacity certification.
|
||||
|
||||
All inputs are synthetic and held in memory. The spawn observer temporarily
|
||||
holds the admitted parent's handshake so the other owners encounter the same
|
||||
occupied slot deterministically. Children perform real XLSX, template and
|
||||
Dataflow work; there is no mocked process execution, database, or live service.
|
||||
The non-queuing gate promises retryable rejection, not scheduler fairness.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from io import BytesIO
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from openpyxl import Workbook
|
||||
from govoplan_connectors.backend.tabular_adapters import (
|
||||
parse_managed_tabular_content,
|
||||
)
|
||||
from govoplan_core.core.templates import TemplateRenderRequest
|
||||
from govoplan_core.security import bounded_process
|
||||
from govoplan_core.security.bounded_process import ProcessBudgetError
|
||||
from govoplan_core.settings import settings
|
||||
from govoplan_dataflow.backend.backends import execute_typed_graph
|
||||
from govoplan_dataflow.backend.schemas import (
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
GraphPosition,
|
||||
PipelineGraph,
|
||||
)
|
||||
from govoplan_templates.backend.rendering import _render_payload
|
||||
except ImportError as exc:
|
||||
raise unittest.SkipTest(
|
||||
"Mixed-owner isolation requires the optional module test environment."
|
||||
) from exc
|
||||
|
||||
|
||||
class IsolatedWorkCompositionTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
workbook = Workbook()
|
||||
workbook.active.append(["name"])
|
||||
workbook.active.append(["Ada"])
|
||||
stream = BytesIO()
|
||||
workbook.save(stream)
|
||||
workbook.close()
|
||||
self.workbook = stream.getvalue()
|
||||
self.graph = PipelineGraph(
|
||||
nodes=[
|
||||
GraphNode(
|
||||
id="source",
|
||||
type="source.inline",
|
||||
label="Source",
|
||||
position=GraphPosition(x=0, y=0),
|
||||
config={"source_name": "records", "rows": [{"name": "Ada"}]},
|
||||
),
|
||||
GraphNode(
|
||||
id="output",
|
||||
type="output",
|
||||
label="Output",
|
||||
position=GraphPosition(x=100, y=0),
|
||||
config={},
|
||||
),
|
||||
],
|
||||
edges=[GraphEdge(id="edge", source="source", target="output")],
|
||||
)
|
||||
|
||||
def xlsx(self):
|
||||
rows, sheet = parse_managed_tabular_content(
|
||||
self.workbook,
|
||||
filename="synthetic.xlsx",
|
||||
content_type=None,
|
||||
delimiter=",",
|
||||
sheet_name=None,
|
||||
)
|
||||
self.assertEqual(rows, ({"name": "Ada"},))
|
||||
self.assertEqual(sheet, "Sheet")
|
||||
return "xlsx"
|
||||
|
||||
def templates(self):
|
||||
payload, content_type, _pages = _render_payload(
|
||||
SimpleNamespace(name="Synthetic template"),
|
||||
SimpleNamespace(
|
||||
content_text="Hello {{item.name}}",
|
||||
content_html=None,
|
||||
template_type="letter",
|
||||
layout={},
|
||||
output_profiles=[],
|
||||
),
|
||||
request=TemplateRenderRequest(
|
||||
template_id="synthetic", output_format="text"
|
||||
),
|
||||
items=({"name": "Ada"},),
|
||||
)
|
||||
self.assertEqual(payload, b"Hello Ada")
|
||||
self.assertEqual(content_type, "text/plain; charset=utf-8")
|
||||
return "templates"
|
||||
|
||||
def dataflow(self):
|
||||
result = execute_typed_graph(self.graph, backend="reference")
|
||||
self.assertEqual(result.rows, [{"name": "Ada"}])
|
||||
return "dataflow"
|
||||
|
||||
def test_one_shared_slot_rejects_other_owners_and_all_retries_recover(self):
|
||||
owners = {
|
||||
"xlsx": self.xlsx,
|
||||
"templates": self.templates,
|
||||
"dataflow": self.dataflow,
|
||||
}
|
||||
processes = []
|
||||
modules = []
|
||||
hold_next = False
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
observer_lock = threading.Lock()
|
||||
maximum_unreaped = 0
|
||||
observer_timeouts = 0
|
||||
original_popen = bounded_process.subprocess.Popen
|
||||
|
||||
def observe_spawn(*args, **kwargs):
|
||||
nonlocal hold_next, maximum_unreaped, observer_timeouts
|
||||
process = original_popen(*args, **kwargs)
|
||||
with observer_lock:
|
||||
processes.append(process)
|
||||
modules.append(args[0][5])
|
||||
maximum_unreaped = max(
|
||||
maximum_unreaped, sum(item.returncode is None for item in processes)
|
||||
)
|
||||
should_hold = hold_next
|
||||
hold_next = False
|
||||
if should_hold:
|
||||
entered.set()
|
||||
if not release.wait(8):
|
||||
# Return control so the real runner's normal timeout and
|
||||
# process-group cleanup still own this child on test error.
|
||||
observer_timeouts += 1
|
||||
return process
|
||||
|
||||
def rejected(operation):
|
||||
try:
|
||||
operation()
|
||||
except Exception as exc:
|
||||
cause = exc
|
||||
while cause is not None and not isinstance(cause, ProcessBudgetError):
|
||||
cause = cause.__cause__
|
||||
self.assertIsInstance(cause, ProcessBudgetError)
|
||||
self.assertEqual(cause.code, "busy")
|
||||
return "busy"
|
||||
self.fail(
|
||||
"A different module admitted work while the shared slot was occupied."
|
||||
)
|
||||
|
||||
started = time.monotonic()
|
||||
busy_count = 0
|
||||
try:
|
||||
with (
|
||||
patch.object(settings, "isolated_process_concurrency", 1),
|
||||
patch.object(bounded_process.subprocess, "Popen", observe_spawn),
|
||||
ThreadPoolExecutor(max_workers=3) as executor,
|
||||
):
|
||||
for owner, operation in owners.items():
|
||||
with self.subTest(admitted_owner=owner):
|
||||
entered.clear()
|
||||
release.clear()
|
||||
hold_next = True
|
||||
holder = executor.submit(operation)
|
||||
try:
|
||||
self.assertTrue(
|
||||
entered.wait(5),
|
||||
"The admitted operation never spawned its real child.",
|
||||
)
|
||||
children_before = len(processes)
|
||||
others = [
|
||||
work for label, work in owners.items() if label != owner
|
||||
]
|
||||
denied = [
|
||||
executor.submit(rejected, work) for work in others
|
||||
]
|
||||
self.assertEqual(
|
||||
[future.result(timeout=5) for future in denied],
|
||||
["busy", "busy"],
|
||||
)
|
||||
busy_count += len(denied)
|
||||
self.assertEqual(len(processes), children_before)
|
||||
finally:
|
||||
release.set()
|
||||
self.assertEqual(holder.result(timeout=15), owner)
|
||||
self.assertEqual(bounded_process._active, 0)
|
||||
# Every rejected owner is retried through its real API.
|
||||
# Each must complete after the previous holder releases.
|
||||
for other in others:
|
||||
other()
|
||||
self.assertEqual(bounded_process._active, 0)
|
||||
finally:
|
||||
release.set()
|
||||
for process in processes:
|
||||
self.assertIsNotNone(process.returncode, "Worker was not reaped.")
|
||||
self.assertTrue(
|
||||
all(
|
||||
stream.closed
|
||||
for stream in (process.stdin, process.stdout, process.stderr)
|
||||
)
|
||||
)
|
||||
with self.assertRaises(ChildProcessError):
|
||||
os.waitpid(process.pid, os.WNOHANG)
|
||||
self.assertEqual(maximum_unreaped, 1)
|
||||
self.assertEqual(observer_timeouts, 0)
|
||||
self.assertEqual(len(processes), 9)
|
||||
self.assertEqual(busy_count, 6)
|
||||
self.assertEqual(
|
||||
set(modules),
|
||||
{
|
||||
"govoplan_connectors.backend.tabular_adapters",
|
||||
"govoplan_templates.backend.rendering",
|
||||
"govoplan_dataflow.backend.backends.reference",
|
||||
},
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"local_composition": {
|
||||
"successful_children": len(processes),
|
||||
"busy_rejections": busy_count,
|
||||
"maximum_unreaped_children": maximum_unreaped,
|
||||
"all_children_reaped": True,
|
||||
"seconds": round(time.monotonic() - started, 3),
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -96,6 +96,18 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
|
||||
with self.assertRaisesRegex(ValueError, "tracking_issue"):
|
||||
inventory._load_endpoint_declarations(path)
|
||||
|
||||
def test_bounded_workflow_read_apis_have_explicit_headless_declarations(self) -> None:
|
||||
declarations = inventory._load_endpoint_declarations(inventory.DEFAULT_ENDPOINT_DECLARATIONS)
|
||||
for path in (
|
||||
"/workflow/instances/summaries", "/workflow/instances/{}/summary",
|
||||
"/workflow/instances/{}/steps", "/workflow/instances/{}/events",
|
||||
):
|
||||
with self.subTest(path=path):
|
||||
entry = declarations[("govoplan-workflow-engine", "GET", path)]
|
||||
self.assertEqual("intentionally_headless", entry["category"])
|
||||
self.assertIn("current-authorized", entry["rationale"])
|
||||
self.assertIn("workflow.instance-history", entry["rationale"])
|
||||
|
||||
def test_inventory_reports_unclassified_and_stale_endpoint_declarations(
|
||||
self,
|
||||
) -> None:
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from contextlib import redirect_stdout
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import shutil
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import meta_preparation # noqa: E402
|
||||
from govoplan_release.git_state import collect_repository_snapshot # noqa: E402
|
||||
from govoplan_release.meta_preparation import ( # noqa: E402
|
||||
MetaPreparationError,
|
||||
prepare_developer_meta_package,
|
||||
)
|
||||
from govoplan_release.model import RepositorySpec # noqa: E402
|
||||
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
|
||||
from govoplan_release.version_metadata import ( # noqa: E402
|
||||
VersionMetadataError,
|
||||
apply_version_metadata_mutations,
|
||||
version_metadata_mutations,
|
||||
)
|
||||
import test_release_meta_source_tag as meta_fixture # noqa: E402
|
||||
from test_release_plan_guidance import dashboard # noqa: E402
|
||||
from test_release_repository_tag import create_release_repo, git, git_text # noqa: E402
|
||||
|
||||
|
||||
class MetaPreparationTests(unittest.TestCase):
|
||||
synchronize = meta_fixture.MetaSourceTagTests.synchronize
|
||||
|
||||
def setUp(self):
|
||||
meta_fixture.MetaSourceTagTests.setUp(self)
|
||||
self.operator = self.root / "operator"
|
||||
self.generator = (
|
||||
self.operator / "tools/release/generate-developer-meta-package.py"
|
||||
)
|
||||
self.generator.parent.mkdir(parents=True)
|
||||
shutil.copyfile(
|
||||
ROOT / "tools/release/generate-developer-meta-package.py", self.generator
|
||||
)
|
||||
self.enterContext(patch.object(meta_preparation, "META_ROOT", self.operator))
|
||||
|
||||
def prepare_core(self):
|
||||
apply_version_metadata_mutations(self.core, target_version="0.1.11")
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Prepared synthetic Core target")
|
||||
|
||||
def preview(self, **kwargs):
|
||||
return prepare_developer_meta_package(
|
||||
repo_path=self.meta, target_version="0.1.11", **kwargs
|
||||
)
|
||||
|
||||
def apply(self, preview):
|
||||
return self.preview(
|
||||
apply=True, expected_receipt=preview["receipt"], confirm_out_of_run=True
|
||||
)
|
||||
|
||||
def test_full_canonical_preview_apply_and_shared_mutation_discovery(self):
|
||||
extra, remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-workflow-engine",
|
||||
version="0.2.3",
|
||||
)
|
||||
self.specs.append(
|
||||
{
|
||||
"name": extra.name,
|
||||
"path": extra.name,
|
||||
"category": "module",
|
||||
"subtype": "",
|
||||
"remote": str(remote),
|
||||
}
|
||||
)
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.prepare_core()
|
||||
before = self.package.read_bytes()
|
||||
preview = self.preview()
|
||||
self.assertEqual("planned", preview["status"])
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
mutations = version_metadata_mutations(self.meta, target_version="0.1.11")
|
||||
self.assertEqual([meta_preparation.PACKAGE], [item.path for item in mutations])
|
||||
self.assertIn(b"govoplan-workflow-engine==0.2.3", mutations[0].after)
|
||||
self.assertIn(b"govoplan-core==0.1.11", mutations[0].after)
|
||||
result = self.apply(preview)
|
||||
self.assertEqual("prepared", result["status"])
|
||||
self.assertEqual(mutations[0].after, self.package.read_bytes())
|
||||
self.assertEqual(
|
||||
self.render(workspace=self.workspace, requirements=self.requirements),
|
||||
self.package.read_text(),
|
||||
)
|
||||
self.assertEqual(
|
||||
f"M {meta_preparation.PACKAGE}",
|
||||
git_text(self.meta, "status", "--porcelain"),
|
||||
)
|
||||
self.assertFalse(git_text(self.meta, "tag", "--list"))
|
||||
|
||||
def test_core_target_must_already_be_prepared(self):
|
||||
before = self.package.read_bytes()
|
||||
with self.assertRaisesRegex(MetaPreparationError, "Prepare and commit Core"):
|
||||
self.preview()
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_changed_requirements_receipt_blocks_before_any_output(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# reviewed different inputs\n"
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Changed synthetic requirements")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_source_change_immediately_before_effect_is_rechecked(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
original = meta_preparation.preview_meta_mutation
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# concurrent change\n"
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Concurrent synthetic change")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_preparation, "preview_meta_mutation", side_effect=changed
|
||||
):
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed before"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_core_full_package_and_operator_generator_are_receipt_bound(self):
|
||||
self.prepare_core()
|
||||
for path, repository in (
|
||||
(self.core / "pyproject.toml", self.core),
|
||||
(self.access / "pyproject.toml", self.access),
|
||||
(self.generator, None),
|
||||
):
|
||||
with self.subTest(input=path.name, repo=str(repository)):
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
path.write_text(path.read_text() + "\n# changed frozen input\n")
|
||||
if repository is not None:
|
||||
git(repository, "add", ".")
|
||||
git(
|
||||
repository,
|
||||
"commit",
|
||||
"-m",
|
||||
"Changed synthetic composition input",
|
||||
)
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_post_write_source_change_is_reported_without_retry_or_rollback(self):
|
||||
from govoplan_release import version_metadata
|
||||
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
original = version_metadata._atomic_write
|
||||
|
||||
def changed(path, payload):
|
||||
original(path, payload)
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# concurrent after write\n"
|
||||
)
|
||||
|
||||
with patch.object(
|
||||
version_metadata, "_atomic_write", side_effect=changed
|
||||
) as writer:
|
||||
with self.assertRaisesRegex(
|
||||
meta_preparation.MetaPreparationAmbiguous, "write/post-check failed"
|
||||
):
|
||||
self.apply(preview)
|
||||
self.assertEqual(1, writer.call_count)
|
||||
self.assertIn('version = "0.1.11"', self.package.read_text())
|
||||
self.assertIn("# concurrent after write", self.requirements.read_text())
|
||||
|
||||
def test_write_failure_after_replace_requires_reconciliation(self):
|
||||
from govoplan_release import version_metadata
|
||||
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
original = version_metadata._atomic_write
|
||||
|
||||
def partial(path, payload):
|
||||
original(path, payload)
|
||||
raise OSError("Synthetic directory fsync failure after replacement")
|
||||
|
||||
with patch.object(version_metadata, "_atomic_write", side_effect=partial) as writer:
|
||||
with self.assertRaisesRegex(meta_preparation.MetaPreparationAmbiguous, "may have been written"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(1, writer.call_count)
|
||||
self.assertIn('version = "0.1.11"', self.package.read_text())
|
||||
|
||||
def test_cli_requires_reviewed_receipt_and_explicit_out_of_run_confirmation(self):
|
||||
self.prepare_core()
|
||||
main = runpy.run_path(
|
||||
str(ROOT / "tools/release/prepare-developer-meta-package.py")
|
||||
)["main"]
|
||||
arguments = [
|
||||
"prepare-developer-meta-package.py",
|
||||
"--workspace",
|
||||
str(self.workspace),
|
||||
"--target-version",
|
||||
"0.1.11",
|
||||
]
|
||||
output = io.StringIO()
|
||||
with patch.object(sys, "argv", arguments), redirect_stdout(output):
|
||||
self.assertEqual(0, main())
|
||||
preview = self.root / "meta-preview.json"
|
||||
preview.write_text(output.getvalue())
|
||||
with (
|
||||
patch.object(sys, "argv", [*arguments, "--apply"]),
|
||||
redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(1, main())
|
||||
with (
|
||||
patch.object(
|
||||
sys,
|
||||
"argv",
|
||||
[
|
||||
*arguments,
|
||||
"--apply",
|
||||
"--receipt",
|
||||
str(preview),
|
||||
"--confirm-out-of-run",
|
||||
],
|
||||
),
|
||||
redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(0, main())
|
||||
|
||||
def test_unknown_full_input_and_unsafe_operator_tooling_fail_closed(self):
|
||||
self.prepare_core()
|
||||
unknown = self.workspace / "govoplan-unknown/pyproject.toml"
|
||||
unknown.parent.mkdir()
|
||||
unknown.write_text('[project]\nname="govoplan-unknown"\nversion="1.0.0"\n')
|
||||
with self.assertRaisesRegex(MetaPreparationError, "unregistered"):
|
||||
self.preview()
|
||||
unknown.unlink()
|
||||
self.generator.chmod(0o666)
|
||||
with self.assertRaisesRegex(MetaPreparationError, "owned, bounded regular"):
|
||||
self.preview()
|
||||
|
||||
def test_wrong_nested_identity_and_existing_immutable_tag_fail_closed(self):
|
||||
self.prepare_core()
|
||||
original = self.package.read_text()
|
||||
self.package.write_text(
|
||||
original.replace('name = "govoplan"', 'name = "not-govoplan"')
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Wrong synthetic package identity")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "identity"):
|
||||
self.preview()
|
||||
self.package.write_text(original)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Restore synthetic package identity")
|
||||
git(self.meta, "tag", "-a", "v0.1.11", "-m", "Immutable target")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "target Meta tag"):
|
||||
self.preview()
|
||||
|
||||
def test_no_generic_durable_self_mutation_or_running_tooling_target(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
with self.assertRaisesRegex(VersionMetadataError, "outside durable runs"):
|
||||
apply_version_metadata_mutations(self.meta, target_version="0.1.11")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "confirm"):
|
||||
self.preview(apply=True, expected_receipt=preview["receipt"])
|
||||
with patch.object(meta_preparation, "META_ROOT", self.meta):
|
||||
with self.assertRaisesRegex(MetaPreparationError, "running operator"):
|
||||
self.preview()
|
||||
|
||||
def test_next_version_plan_is_actionable_core_first_without_meta_executor(self):
|
||||
snapshots = tuple(
|
||||
collect_repository_snapshot(
|
||||
RepositorySpec(**spec),
|
||||
workspace_root=self.workspace,
|
||||
target_tag="v0.1.11",
|
||||
)
|
||||
for spec in self.specs[:2]
|
||||
)
|
||||
source = replace(
|
||||
dashboard(workspace=self.workspace, version=self.version),
|
||||
repositories=snapshots,
|
||||
)
|
||||
plan = build_selective_release_plan(
|
||||
source,
|
||||
selected_repos=("govoplan", "govoplan-core"),
|
||||
target_version="0.1.11",
|
||||
)
|
||||
self.assertEqual(
|
||||
["govoplan-core", "govoplan"], [unit.repo for unit in plan.units]
|
||||
)
|
||||
findings = [
|
||||
finding for finding in plan.gate_findings if finding.repo == "govoplan"
|
||||
]
|
||||
self.assertEqual(
|
||||
["developer_meta_core_preparation_required"],
|
||||
[finding.code for finding in findings],
|
||||
)
|
||||
self.assertIn("prepare-developer-meta-package.py", findings[0].remediation)
|
||||
meta_steps = [step for step in plan.dry_run_steps if step.repo == "govoplan"]
|
||||
self.assertEqual(
|
||||
["govoplan:prepare-support", "govoplan:publish-support"],
|
||||
[step.id for step in meta_steps],
|
||||
)
|
||||
self.assertTrue(all(step.status == "needs-executor" for step in meta_steps))
|
||||
self.prepare_core()
|
||||
prepared = build_selective_release_plan(
|
||||
source, selected_repos=("govoplan",), target_version="0.1.11"
|
||||
)
|
||||
self.assertEqual("developer_meta_out_of_run", prepared.gate_findings[0].code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,569 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import source_tag_batch as meta_source_tag, workspace # noqa: E402
|
||||
from govoplan_release.git_state import collect_versions # noqa: E402
|
||||
from govoplan_release.model import RepositorySnapshot, RepositorySpec, VersionSnapshot # noqa: E402
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
from govoplan_release.selective_planner import build_unit # noqa: E402
|
||||
from govoplan_release.version_alignment import repository_version_issues # noqa: E402
|
||||
from test_release_repository_tag import ( # noqa: E402
|
||||
add_scoped_workflow_manifest,
|
||||
create_release_repo,
|
||||
git,
|
||||
git_text,
|
||||
ref_exists,
|
||||
)
|
||||
|
||||
|
||||
class MetaSourceTagTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temporary = self.enterContext(
|
||||
tempfile.TemporaryDirectory(prefix="meta-release-tests-")
|
||||
)
|
||||
self.root = Path(self.temporary)
|
||||
self.workspace = self.root / "workspace"
|
||||
self.workspace.mkdir()
|
||||
self.version = "0.1.10"
|
||||
self.core, self.core_remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-core",
|
||||
version=self.version,
|
||||
)
|
||||
self.access, self.access_remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-access",
|
||||
version=self.version,
|
||||
)
|
||||
add_scoped_workflow_manifest(self.access)
|
||||
self.meta = self.workspace / "govoplan"
|
||||
self.meta_remote = self.root / "govoplan.git"
|
||||
git(self.root, "init", "--bare", str(self.meta_remote))
|
||||
git(self.workspace, "init", "-b", "main", str(self.meta))
|
||||
git(self.meta, "config", "user.name", "Meta Release Fixture")
|
||||
git(self.meta, "config", "user.email", "release@example.invalid")
|
||||
self.package = self.meta / "packages/govoplan-meta/pyproject.toml"
|
||||
self.package.parent.mkdir(parents=True)
|
||||
self.requirements = self.meta / "requirements-release.txt"
|
||||
self.requirements.write_text(
|
||||
"../govoplan-core\ngovoplan-access @ git+ssh://git@example.invalid/GovOPlaN/govoplan-access.git@v0.1.10\n"
|
||||
)
|
||||
self.render = runpy.run_path(
|
||||
str(ROOT / "tools/release/generate-developer-meta-package.py")
|
||||
)["render"]
|
||||
self.synchronize()
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Nested developer package")
|
||||
git(self.meta, "remote", "add", "origin", str(self.meta_remote))
|
||||
git(self.meta, "push", "-u", "origin", "main")
|
||||
self.specs = [
|
||||
{
|
||||
"name": name,
|
||||
"category": "system" if subtype else "module",
|
||||
"subtype": subtype,
|
||||
"path": name,
|
||||
"remote": str(remote),
|
||||
}
|
||||
for name, subtype, remote in (
|
||||
("govoplan", "meta", self.meta_remote),
|
||||
("govoplan-core", "kernel", self.core_remote),
|
||||
("govoplan-access", "", self.access_remote),
|
||||
)
|
||||
]
|
||||
self.registry = self.root / "repositories.json"
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.enterContext(patch.object(workspace, "REPOSITORIES_FILE", self.registry))
|
||||
|
||||
def synchronize(self):
|
||||
self.package.write_text(
|
||||
self.render(workspace=self.workspace, requirements=self.requirements)
|
||||
)
|
||||
|
||||
def commit_meta(self):
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Changed synthetic metadata")
|
||||
|
||||
def tag(
|
||||
self,
|
||||
*,
|
||||
repos=("govoplan", "govoplan-core"),
|
||||
apply=False,
|
||||
push=False,
|
||||
**overrides,
|
||||
):
|
||||
return tag_repositories(
|
||||
repos=repos,
|
||||
repo_versions={repo: self.version for repo in repos},
|
||||
workspace_root=self.workspace,
|
||||
apply=apply,
|
||||
push=push,
|
||||
**overrides,
|
||||
)
|
||||
|
||||
def assert_no_tags(self):
|
||||
for repo in (
|
||||
self.meta,
|
||||
self.meta_remote,
|
||||
self.core,
|
||||
self.core_remote,
|
||||
self.access,
|
||||
self.access_remote,
|
||||
):
|
||||
self.assertFalse(ref_exists(repo, "refs/tags/v0.1.10"), str(repo))
|
||||
|
||||
def test_explicit_nested_version_collection_and_alignment_without_root_package(
|
||||
self,
|
||||
):
|
||||
versions = collect_versions(self.meta)
|
||||
self.assertIsNone(versions.pyproject)
|
||||
self.assertEqual(self.version, versions.developer_meta)
|
||||
self.assertEqual(self.version, versions.primary)
|
||||
self.assertFalse((self.meta / "pyproject.toml").exists())
|
||||
self.assertEqual(
|
||||
(), repository_version_issues(self.meta, expected_version=self.version)
|
||||
)
|
||||
mismatch = repository_version_issues(self.meta, expected_version="0.1.11")
|
||||
self.assertTrue(
|
||||
any(
|
||||
issue.source == "packages/govoplan-meta/pyproject.toml"
|
||||
for issue in mismatch
|
||||
)
|
||||
)
|
||||
|
||||
def test_planner_and_console_display_the_explicit_nested_version(self):
|
||||
snapshot = RepositorySnapshot(
|
||||
spec=RepositorySpec(**self.specs[0]),
|
||||
absolute_path=str(self.meta),
|
||||
exists=True,
|
||||
is_git=True,
|
||||
has_head=True,
|
||||
branch="main",
|
||||
versions=VersionSnapshot(developer_meta=self.version),
|
||||
)
|
||||
unit = build_unit(snapshot, target_version=None, contracts=None)
|
||||
self.assertEqual(self.version, unit.current_version)
|
||||
self.assertEqual(self.version, unit.target_version)
|
||||
html = (ROOT / "tools/release/webui/index.html").read_text()
|
||||
self.assertIn(
|
||||
"if (versions.developer_meta) return versions.developer_meta;", html
|
||||
)
|
||||
drift = RepositorySnapshot(
|
||||
spec=snapshot.spec,
|
||||
absolute_path=str(self.meta),
|
||||
exists=True,
|
||||
is_git=True,
|
||||
has_head=True,
|
||||
branch="main",
|
||||
versions=VersionSnapshot(pyproject="0.1.9", developer_meta=self.version),
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
"version metadata is not aligned" in item
|
||||
for item in build_unit(
|
||||
drift, target_version=self.version, contracts=None
|
||||
).blockers
|
||||
)
|
||||
)
|
||||
|
||||
def test_unknown_nested_package_and_missing_or_wrong_meta_identity_fail_closed(
|
||||
self,
|
||||
):
|
||||
unknown = self.workspace / "unknown"
|
||||
nested = unknown / "packages/govoplan-meta/pyproject.toml"
|
||||
nested.parent.mkdir(parents=True)
|
||||
nested.write_text(self.package.read_text())
|
||||
self.assertIsNone(collect_versions(unknown).primary)
|
||||
self.assertIn(
|
||||
"no version metadata",
|
||||
repository_version_issues(unknown, expected_version=self.version)[
|
||||
0
|
||||
].message,
|
||||
)
|
||||
for value in ("", '[project]\nname="not-govoplan"\nversion="0.1.10"\n'):
|
||||
with self.subTest(value=value):
|
||||
self.package.write_text(value)
|
||||
self.assertTrue(
|
||||
repository_version_issues(self.meta, expected_version=self.version)
|
||||
)
|
||||
|
||||
def test_preview_local_tag_and_publish_share_complete_nested_contract(self):
|
||||
preview = self.tag(push=True)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertEqual(
|
||||
["govoplan-core", "govoplan"],
|
||||
[row["repo"] for row in preview["repositories"]],
|
||||
)
|
||||
self.assertEqual("registered-meta-batch-v1", preview["source_contract"])
|
||||
self.assert_no_tags()
|
||||
local = self.tag(apply=True)
|
||||
self.assertEqual("tagged", local["status"], local)
|
||||
for repo in (self.core, self.meta):
|
||||
self.assertEqual(
|
||||
"tag", git_text(repo, "cat-file", "-t", "refs/tags/v0.1.10")
|
||||
)
|
||||
self.assertFalse(ref_exists(self.meta_remote, "refs/tags/v0.1.10"))
|
||||
published = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
for repo, remote in (
|
||||
(self.core, self.core_remote),
|
||||
(self.meta, self.meta_remote),
|
||||
):
|
||||
self.assertEqual(
|
||||
git_text(repo, "rev-parse", "HEAD"),
|
||||
git_text(remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
self.assertEqual(
|
||||
git_text(repo, "rev-parse", "refs/tags/v0.1.10"),
|
||||
git_text(remote, "rev-parse", "refs/tags/v0.1.10"),
|
||||
)
|
||||
again = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", again["status"], again)
|
||||
|
||||
def test_stale_composition_blocks_whole_batch_before_local_tag_or_push(self):
|
||||
self.package.write_text(
|
||||
self.package.read_text().replace(
|
||||
"govoplan-access==0.1.10", "govoplan-access==0.1.9"
|
||||
)
|
||||
)
|
||||
self.commit_meta()
|
||||
for apply, push in ((False, False), (True, False), (True, True)):
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access", "govoplan"),
|
||||
apply=apply,
|
||||
push=push,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_core_outside_batch_requires_matching_existing_and_published_tag(self):
|
||||
self.assertEqual("blocked", self.tag(repos=("govoplan",))["status"])
|
||||
git(self.core, "tag", "-a", "v0.1.10", "-m", "Core release")
|
||||
self.assertEqual("planned", self.tag(repos=("govoplan",))["status"])
|
||||
self.assertEqual("blocked", self.tag(repos=("govoplan",), push=True)["status"])
|
||||
git(self.core, "push", "origin", "refs/tags/v0.1.10")
|
||||
self.assertEqual("planned", self.tag(repos=("govoplan",), push=True)["status"])
|
||||
|
||||
def test_changed_core_version_and_explicit_selected_version_mismatch_block(self):
|
||||
mismatch = tag_repositories(
|
||||
repos=("govoplan", "govoplan-core"),
|
||||
repo_versions={"govoplan": self.version, "govoplan-core": "0.1.11"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
self.assertEqual("blocked", mismatch["status"], mismatch)
|
||||
(self.core / "pyproject.toml").write_text(
|
||||
'[project]\nname="govoplan-core"\nversion="0.1.11"\n'
|
||||
)
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Core new version")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_unsafe_origin_on_any_selected_repo_blocks_every_effect(self):
|
||||
for repo in (self.meta, self.core, self.access):
|
||||
with self.subTest(repo=repo.name):
|
||||
git(
|
||||
repo,
|
||||
"config",
|
||||
"remote.origin.pushurl",
|
||||
str(self.root / "unregistered.git"),
|
||||
)
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access", "govoplan"),
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("registered origin", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
git(repo, "config", "--unset", "remote.origin.pushurl")
|
||||
|
||||
def test_world_writable_nonsticky_parent_blocks_without_changing_permissions(self):
|
||||
original = self.root.stat().st_mode & 0o7777
|
||||
self.root.chmod(0o777)
|
||||
try:
|
||||
for apply in (False, True):
|
||||
result = self.tag(apply=apply, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"group/world writable", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assertEqual(0o777, self.root.stat().st_mode & 0o7777)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
self.root.chmod(original)
|
||||
|
||||
def test_wrong_metadata_owner_blocks_before_remote_lookup(self):
|
||||
config = self.meta / ".git/config"
|
||||
original = Path.lstat
|
||||
|
||||
def wrong_owner(path, *args, **kwargs):
|
||||
observed = original(path, *args, **kwargs)
|
||||
if path == config:
|
||||
fields = list(observed)
|
||||
fields[4] = os.geteuid() + 1
|
||||
return os.stat_result(fields)
|
||||
return observed
|
||||
|
||||
with (
|
||||
patch.object(Path, "lstat", new=wrong_owner),
|
||||
patch.object(
|
||||
meta_source_tag,
|
||||
"registered_source_origin_issues",
|
||||
side_effect=AssertionError("must validate ownership before Git"),
|
||||
),
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("current operator", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_hidden_index_flags_cannot_disguise_modified_release_metadata(self):
|
||||
for flag, undo in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
for repo, relative in (
|
||||
(self.meta, "packages/govoplan-meta/pyproject.toml"),
|
||||
(self.core, "pyproject.toml"),
|
||||
):
|
||||
with self.subTest(flag=flag, repo=repo.name):
|
||||
target = repo / relative
|
||||
original = target.read_text()
|
||||
git(repo, "update-index", flag, relative)
|
||||
target.write_text(
|
||||
original
|
||||
+ "\n# Hidden working-tree input differs from frozen HEAD\n"
|
||||
)
|
||||
try:
|
||||
self.assertEqual("", git_text(repo, "status", "--porcelain"))
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"index entries", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
target.write_text(original)
|
||||
git(repo, "update-index", undo, relative)
|
||||
|
||||
def test_read_only_git_target_is_not_repaired_or_tagged(self):
|
||||
metadata = self.meta / ".git"
|
||||
original = metadata.stat().st_mode & 0o7777
|
||||
metadata.chmod(0o500)
|
||||
try:
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertEqual(0o500, metadata.stat().st_mode & 0o7777)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
metadata.chmod(original)
|
||||
|
||||
def test_git_object_alternates_are_rejected_before_remote_lookup(self):
|
||||
(self.meta / ".git/objects/info/alternates").write_text(
|
||||
str(self.root / "outside-objects") + "\n"
|
||||
)
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"registered_source_origin_issues",
|
||||
side_effect=AssertionError("must reject alternates before Git"),
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("alternates", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_owned_worktree_metadata_inside_private_workspace_is_supported(self):
|
||||
main_checkout = self.workspace / "meta-main-storage"
|
||||
self.meta.rename(main_checkout)
|
||||
git(main_checkout, "worktree", "add", "--force", str(self.meta), "main")
|
||||
self.assertTrue((self.meta / ".git").is_file())
|
||||
result = self.tag(apply=True)
|
||||
self.assertEqual("tagged", result["status"], result)
|
||||
filesystem = result["source_receipts"]["govoplan"]["filesystem"]
|
||||
self.assertEqual(
|
||||
str(main_checkout / ".git"), filesystem["git_common_directory"][0]
|
||||
)
|
||||
|
||||
def test_git_directory_replacement_with_same_head_changes_frozen_receipt(self):
|
||||
preview = meta_source_tag._preview_repositories
|
||||
|
||||
def swapped_git_directory(**kwargs):
|
||||
result = preview(**kwargs)
|
||||
original = self.meta / ".git"
|
||||
backup = self.root / "original-meta-git"
|
||||
original.rename(backup)
|
||||
shutil.copytree(backup, original)
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"_preview_repositories",
|
||||
side_effect=swapped_git_directory,
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_non_main_and_divergent_live_main_fail_even_with_stale_tracking(self):
|
||||
git(self.meta, "switch", "-c", "feature")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.meta, "switch", "main")
|
||||
clone = self.root / "other-writer"
|
||||
git(self.root, "clone", "--branch", "main", str(self.meta_remote), str(clone))
|
||||
git(clone, "config", "user.name", "Other synthetic writer")
|
||||
git(clone, "config", "user.email", "other@example.invalid")
|
||||
(clone / "other.txt").write_text("remote divergence\n")
|
||||
git(clone, "add", ".")
|
||||
git(clone, "commit", "-m", "Remote main advanced")
|
||||
git(clone, "push", "origin", "main")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("live origin/main", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_symlink_checkout_is_not_a_registered_source(self):
|
||||
original = self.workspace / "moved-meta"
|
||||
self.meta.rename(original)
|
||||
self.meta.symlink_to(original, target_is_directory=True)
|
||||
result = self.tag(apply=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("symlink", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_lightweight_and_conflicting_annotated_tags_block(self):
|
||||
git(self.meta, "tag", "v0.1.10")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.meta, "tag", "-d", "v0.1.10")
|
||||
git(self.meta, "tag", "-a", "v0.1.10", "-m", "First annotation")
|
||||
git(self.meta, "push", "origin", "refs/tags/v0.1.10")
|
||||
git(self.meta, "tag", "-d", "v0.1.10")
|
||||
git(self.meta, "tag", "-a", "v0.1.10", "-m", "Different annotation")
|
||||
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_meta_source_receipt_changed_after_preflight_blocks_before_first_effect(
|
||||
self,
|
||||
):
|
||||
preview = meta_source_tag._preview_repositories
|
||||
|
||||
def changed_after_preflight(**kwargs):
|
||||
result = preview(**kwargs)
|
||||
(self.meta / "new-review.txt").write_text("changed after preflight\n")
|
||||
self.commit_meta()
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"_preview_repositories",
|
||||
side_effect=changed_after_preflight,
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_fabricated_push_success_without_remote_receipt_fails_and_stops_batch(self):
|
||||
original = meta_source_tag.run
|
||||
|
||||
def run(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
return subprocess.CompletedProcess(command, 0, "", "")
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=run):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual("failed", result["repositories"][0]["status"])
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_remote_tag_without_expected_main_receipt_is_not_success(self):
|
||||
(self.core / "reviewed-change.txt").write_text("release source changes\n")
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Advance reviewed Core source")
|
||||
original = meta_source_tag.run
|
||||
pushes = []
|
||||
|
||||
def tag_only(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
pushes.append(command)
|
||||
# Simulate a defective transport that claims atomic success,
|
||||
# while publishing only the exact expected annotation object.
|
||||
return original(("git", "push", "origin", command[-1]), **kwargs)
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=tag_only):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual(1, len(pushes))
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assertEqual(
|
||||
git_text(self.core, "rev-parse", "refs/tags/v0.1.10"),
|
||||
git_text(self.core_remote, "rev-parse", "refs/tags/v0.1.10"),
|
||||
)
|
||||
self.assertNotEqual(
|
||||
git_text(self.core, "rev-parse", "HEAD"),
|
||||
git_text(self.core_remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_meta_receipt_is_rechecked_after_an_earlier_successful_publication(self):
|
||||
original = meta_source_tag.run
|
||||
|
||||
def changed_after_core(command, **kwargs):
|
||||
result = original(command, **kwargs)
|
||||
if (
|
||||
command[:3] == ("git", "push", "--atomic")
|
||||
and kwargs["cwd"] == self.core
|
||||
):
|
||||
(self.meta / "changed-review.txt").write_text(
|
||||
"new Meta source after Core publication\n"
|
||||
)
|
||||
self.commit_meta()
|
||||
return result
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=changed_after_core):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertTrue(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
|
||||
def test_unknown_selected_repository_cannot_use_meta_support_exception(self):
|
||||
result = self.tag(repos=("govoplan", "unknown"), apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("not registered", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_selected_checkout_generator_is_never_executed(self):
|
||||
malicious = self.meta / "tools/release/generate-developer-meta-package.py"
|
||||
malicious.parent.mkdir(parents=True)
|
||||
malicious.write_text(
|
||||
'raise RuntimeError("selected checkout must not execute")\n'
|
||||
)
|
||||
self.commit_meta()
|
||||
self.assertEqual("planned", self.tag()["status"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -65,6 +65,8 @@ branch_labels: Union[str, Sequence[str], None] = None
|
||||
)
|
||||
wrapper = development / release.name
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module\n"
|
||||
'_migration = import_module("govoplan_core.backend.migrations.versions.1234_example")\n'
|
||||
"revision = _migration.revision\n"
|
||||
"down_revision = _migration.down_revision\n"
|
||||
"depends_on = _migration.depends_on\n"
|
||||
@@ -79,6 +81,150 @@ branch_labels: Union[str, Sequence[str], None] = None
|
||||
self.assertEqual(("base",), migration.down_revisions)
|
||||
self.assertEqual(("core",), migration.depends_on)
|
||||
|
||||
def test_literal_wrapper_alias_and_different_filename_are_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_v019_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = "base"\ndepends_on = "core"\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
for alias in ("_migration", "edit_revision", "message_actions"):
|
||||
with self.subTest(alias=alias):
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module as load_migration\n"
|
||||
f'{alias} = load_migration("govoplan_campaign.backend.migrations.versions." "1234_v019_example")\n'
|
||||
f"revision = {alias}.revision\ndown_revision = {alias}.down_revision\n"
|
||||
f"depends_on = {alias}.depends_on\nbranch_labels = {alias}.branch_labels\n"
|
||||
'raise AssertionError("Wrapper must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
migration = audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ("base",))
|
||||
self.assertEqual(migration.depends_on, ("core",))
|
||||
|
||||
def test_core_literal_sibling_file_wrapper_is_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = None\ndepends_on = None\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
wrapper.write_text(
|
||||
"from importlib.util import module_from_spec, spec_from_file_location\n"
|
||||
"from pathlib import Path\n"
|
||||
'_path = Path(__file__).resolve().parents[1] / "versions" / "1234_example.py"\n'
|
||||
'_spec = spec_from_file_location("synthetic_migration", _path)\n'
|
||||
"_module = module_from_spec(_spec)\n_spec.loader.exec_module(_module)\n"
|
||||
"revision = _module.revision\ndown_revision = _module.down_revision\n"
|
||||
"depends_on = _module.depends_on\nbranch_labels = _module.branch_labels\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
migration = audit.parse_migration_file("govoplan-core", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ())
|
||||
|
||||
def test_wrapper_rejects_dynamic_foreign_missing_or_rebound_targets(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text('revision = "1234"\n', encoding="utf-8")
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
valid = '_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||
definitions = (
|
||||
'_migration = import_module(module_name)\n',
|
||||
'_migration = import_module("govoplan_mail.backend.migrations.versions.1234_example")\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions...other.1234_example")\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.missing")\n',
|
||||
valid + "_migration = another_module\n",
|
||||
"import_module = another_loader\n" + valid,
|
||||
"def import_module(value):\n return another_module\n" + valid,
|
||||
"import another_loader as import_module\n" + valid,
|
||||
valid + "class _migration:\n revision = 'another'\n",
|
||||
"if condition:\n _migration = another_module\n" + valid,
|
||||
valid + "del _migration\n",
|
||||
)
|
||||
for definition in definitions:
|
||||
with self.subTest(definition=definition):
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module\n" + definition + "revision = _migration.revision\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
|
||||
def test_wrapper_rejects_symlink_and_mixed_release_metadata(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text('revision = "1234"\ndown_revision = None\n', encoding="utf-8")
|
||||
(root / "versions/5678_example.py").write_text('revision = "5678"\ndown_revision = None\n', encoding="utf-8")
|
||||
(root / "versions/linked.py").symlink_to(root / "versions/1234_example.py")
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
definitions = (
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.linked")\nrevision = _migration.revision\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||
'_other = import_module("govoplan_campaign.backend.migrations.versions.5678_example")\n'
|
||||
'revision = _migration.revision\ndown_revision = _other.down_revision\n',
|
||||
)
|
||||
for definition in definitions:
|
||||
with self.subTest(definition=definition):
|
||||
wrapper.write_text("from importlib import import_module\n" + definition, encoding="utf-8")
|
||||
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
|
||||
def test_unresolved_revision_expression_is_not_silently_omitted(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
path = Path(directory) / "1234_example.py"
|
||||
path.write_text("revision = calculate_revision()\n", encoding="utf-8")
|
||||
with self.assertRaisesRegex(ValueError, "Unsupported or ambiguous migration metadata"):
|
||||
audit.parse_migration_file("govoplan-core", path)
|
||||
|
||||
def test_explicit_metadata_reexport_is_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/a234_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = "base"\ndepends_on = None\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
source = "govoplan_organizations.backend.migrations.versions.a234_example"
|
||||
wrapper.write_text(f"from {source} import revision, down_revision, depends_on, branch_labels\n", encoding="utf-8")
|
||||
migration = audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ("base",))
|
||||
for declaration in (
|
||||
f"from {source} import *\n",
|
||||
f"from {source} import revision as down_revision\n",
|
||||
f"from {source} import revision\nrevision = 'different'\n",
|
||||
f"from {source} import revision\nimport another as revision\n",
|
||||
f"from {source} import revision\ndef revision():\n pass\n",
|
||||
f"from {source.replace('govoplan_organizations', 'govoplan_mail')} import revision\n",
|
||||
):
|
||||
with self.subTest(declaration=declaration):
|
||||
wrapper.write_text(declaration, encoding="utf-8")
|
||||
with self.assertRaises(ValueError):
|
||||
audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||
|
||||
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
|
||||
audit = load_audit_module()
|
||||
migrations = [
|
||||
|
||||
@@ -6,6 +6,7 @@ import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
@@ -17,6 +18,7 @@ if str(RELEASE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(RELEASE_ROOT))
|
||||
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
from govoplan_release import workspace as release_workspace # noqa: E402
|
||||
from server.app import create_app # noqa: E402
|
||||
|
||||
|
||||
@@ -39,6 +41,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
version="0.1.10",
|
||||
)
|
||||
add_scoped_workflow_manifest(self.manifest_repo)
|
||||
# The operator's test catalog explicitly registers known synthetic
|
||||
# endpoints; production trust checks are not patched or bypassed.
|
||||
self.registry = self.root / "registered-test-repositories.json"
|
||||
self.registered = json.loads((META_ROOT / "repositories.json").read_text())
|
||||
for spec in self.registered["repositories"]:
|
||||
spec["remote"] = str(self.root / f"{spec['name']}.git")
|
||||
self.registry.write_text(json.dumps(self.registered))
|
||||
self.enterContext(patch.object(release_workspace, "REPOSITORIES_FILE", self.registry))
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.temporary.cleanup()
|
||||
@@ -128,6 +138,10 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
name="govoplan-core",
|
||||
version="0.1.10",
|
||||
)
|
||||
for spec in self.registered["repositories"]:
|
||||
if spec["name"] == "govoplan-core":
|
||||
spec["remote"] = str(remote_root / "govoplan-core.git")
|
||||
self.registry.write_text(json.dumps(self.registered))
|
||||
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core",),
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import source_tag_batch # noqa: E402
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
import test_release_meta_source_tag as meta_fixture # noqa: E402
|
||||
import test_release_repository_tag as release_fixture # noqa: E402
|
||||
from test_release_repository_tag import git, git_text, ref_exists # noqa: E402
|
||||
|
||||
|
||||
class RegisteredSourceTagBatchTests(unittest.TestCase):
|
||||
setUp = meta_fixture.MetaSourceTagTests.setUp
|
||||
synchronize = meta_fixture.MetaSourceTagTests.synchronize
|
||||
assert_no_tags = meta_fixture.MetaSourceTagTests.assert_no_tags
|
||||
|
||||
def tag(self, *, repos=("govoplan-access",), apply=False, push=False):
|
||||
return tag_repositories(
|
||||
repos=repos,
|
||||
repo_versions={repo: self.version for repo in repos},
|
||||
workspace_root=self.workspace,
|
||||
apply=apply,
|
||||
push=push,
|
||||
)
|
||||
|
||||
def test_python_only_module_publishes_without_meta_or_core_checkout(self):
|
||||
self.meta.rename(self.root / "unused-meta")
|
||||
self.core.rename(self.root / "unused-core")
|
||||
preview = self.tag(push=True)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertEqual("registered-source-batch-v1", preview["source_contract"])
|
||||
self.assertEqual({}, preview["bundle_input_receipts"])
|
||||
self.assertFalse(ref_exists(self.access, "refs/tags/v0.1.10"))
|
||||
published = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
self.assertEqual(["govoplan-access"], list(published["source_receipts"]))
|
||||
self.assertEqual(
|
||||
git_text(self.access, "rev-parse", "HEAD"),
|
||||
git_text(self.access_remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
|
||||
def test_core_only_batch_has_no_meta_composition_requirement(self):
|
||||
self.meta.rename(self.root / "unused-meta")
|
||||
result = self.tag(repos=("govoplan-core",), apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual(["govoplan-core"], list(result["source_receipts"]))
|
||||
|
||||
def test_backend_only_publication_never_reads_irrelevant_unsafe_core_json(self):
|
||||
webui = self.core / "webui"
|
||||
webui.mkdir()
|
||||
(webui / "package.release.json").write_text("invalid unselected Core JSON")
|
||||
(webui / "package-lock.release.json").symlink_to(
|
||||
self.root / "not-a-core-release-lock"
|
||||
)
|
||||
from govoplan_release import version_alignment
|
||||
|
||||
original = version_alignment._json_object
|
||||
|
||||
def read(path):
|
||||
if path.is_relative_to(webui):
|
||||
raise AssertionError(
|
||||
"backend-only publication must not read unrelated Core JSON"
|
||||
)
|
||||
return original(path)
|
||||
|
||||
with patch.object(version_alignment, "_json_object", side_effect=read):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual({}, result["bundle_input_receipts"])
|
||||
|
||||
def test_source_origin_parent_and_read_only_target_guards_apply_without_meta(self):
|
||||
for problem in ("origin", "parent", "read_only"):
|
||||
with self.subTest(problem=problem):
|
||||
parent_mode = self.root.stat().st_mode & 0o7777
|
||||
git_directory = self.access / ".git"
|
||||
git_mode = git_directory.stat().st_mode & 0o7777
|
||||
if problem == "origin":
|
||||
git(
|
||||
self.access,
|
||||
"config",
|
||||
"remote.origin.pushurl",
|
||||
str(self.root / "unknown.git"),
|
||||
)
|
||||
elif problem == "parent":
|
||||
self.root.chmod(0o777)
|
||||
else:
|
||||
git_directory.chmod(0o500)
|
||||
try:
|
||||
for apply in (False, True):
|
||||
result = self.tag(apply=apply, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
self.root.chmod(parent_mode)
|
||||
git_directory.chmod(git_mode)
|
||||
if problem == "origin":
|
||||
git(self.access, "config", "--unset", "remote.origin.pushurl")
|
||||
|
||||
def test_wrong_owner_symlink_and_hidden_index_are_rejected_without_meta(self):
|
||||
config = self.access / ".git/config"
|
||||
original_stat = Path.lstat
|
||||
|
||||
def wrong_owner(path, *args, **kwargs):
|
||||
observed = original_stat(path, *args, **kwargs)
|
||||
if path == config:
|
||||
fields = list(observed)
|
||||
fields[4] = os.geteuid() + 1
|
||||
return os.stat_result(fields)
|
||||
return observed
|
||||
|
||||
with patch.object(Path, "lstat", new=wrong_owner):
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
moved = self.root / "moved-access"
|
||||
self.access.rename(moved)
|
||||
self.access.symlink_to(moved, target_is_directory=True)
|
||||
try:
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
finally:
|
||||
self.access.unlink()
|
||||
moved.rename(self.access)
|
||||
for flag, undo in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
with self.subTest(flag=flag):
|
||||
git(self.access, "update-index", flag, "pyproject.toml")
|
||||
try:
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
git(self.access, "update-index", undo, "pyproject.toml")
|
||||
|
||||
def test_live_remote_divergence_blocks_without_cached_tracking_update(self):
|
||||
clone = self.root / "other-access-writer"
|
||||
git(self.root, "clone", "--branch", "main", str(self.access_remote), str(clone))
|
||||
git(clone, "config", "user.name", "Synthetic writer")
|
||||
git(clone, "config", "user.email", "writer@example.invalid")
|
||||
(clone / "advance.txt").write_text("new remote main\n")
|
||||
git(clone, "add", ".")
|
||||
git(clone, "commit", "-m", "Advance remote without updating original tracking")
|
||||
git(clone, "push", "origin", "main")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("live origin/main", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_ignored_selected_version_metadata_cannot_supply_an_untagged_artifact(self):
|
||||
project = self.access / "pyproject.toml"
|
||||
original = project.read_text()
|
||||
git(self.access, "rm", "pyproject.toml")
|
||||
(self.access / ".gitignore").write_text("/pyproject.toml\n")
|
||||
git(self.access, "add", ".gitignore")
|
||||
git(self.access, "commit", "-m", "Ignored metadata absent from frozen tree")
|
||||
project.write_text(original)
|
||||
self.assertEqual("", git_text(self.access, "status", "--porcelain"))
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("must be tracked", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_lightweight_and_different_annotation_objects_remain_immutable(self):
|
||||
git(self.access, "tag", "v0.1.10")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.access, "tag", "-d", "v0.1.10")
|
||||
git(self.access, "tag", "-a", "v0.1.10", "-m", "Original immutable annotation")
|
||||
git(self.access, "push", "origin", "refs/tags/v0.1.10")
|
||||
original = git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
|
||||
git(self.access, "tag", "-d", "v0.1.10")
|
||||
git(
|
||||
self.access,
|
||||
"tag",
|
||||
"-a",
|
||||
"v0.1.10",
|
||||
"-m",
|
||||
"Conflicting immutable annotation",
|
||||
)
|
||||
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
|
||||
self.assertEqual(
|
||||
original, git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
|
||||
)
|
||||
|
||||
def test_changed_source_after_preflight_stops_before_first_batch_effect(self):
|
||||
original = source_tag_batch._preview_repositories
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
(self.access / "changed-source.txt").write_text(
|
||||
"new source after preflight\n"
|
||||
)
|
||||
git(self.access, "add", ".")
|
||||
git(self.access, "commit", "-m", "Source changed")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
source_tag_batch, "_preview_repositories", side_effect=changed
|
||||
):
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access"), apply=True, push=True
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_false_push_success_is_not_a_receipt_and_never_retries(self):
|
||||
original = source_tag_batch.run
|
||||
pushes = []
|
||||
|
||||
def run(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
pushes.append(command)
|
||||
return subprocess.CompletedProcess(command, 0, "", "")
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(source_tag_batch, "run", side_effect=run):
|
||||
result = self.tag(
|
||||
repos=("govoplan-access", "govoplan-core"), apply=True, push=True
|
||||
)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual(1, len(pushes))
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.access_remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def _ready_bundle(self):
|
||||
self.repo = self.core # Existing fixture's Core name.
|
||||
campaign, remote = (
|
||||
release_fixture.ReleaseRepositoryTagTests._staged_campaign_bundle(self)
|
||||
)
|
||||
self.specs.append(
|
||||
{
|
||||
"name": "govoplan-campaign",
|
||||
"category": "module",
|
||||
"subtype": "domain",
|
||||
"path": "govoplan-campaign",
|
||||
"remote": str(remote),
|
||||
}
|
||||
)
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.assertEqual(
|
||||
"tagged", self.tag(repos=("govoplan-campaign",), apply=True)["status"]
|
||||
)
|
||||
lock_path = self.core / "webui/package-lock.release.json"
|
||||
payload = json.loads(lock_path.read_text())
|
||||
package = payload["packages"]["node_modules/@govoplan/campaign-webui"]
|
||||
package["version"] = self.version
|
||||
package["resolved"] = (
|
||||
"git+ssh://git@example.test/acme/govoplan-campaign.git#"
|
||||
+ git_text(campaign, "rev-parse", "HEAD")
|
||||
)
|
||||
lock_path.write_text(json.dumps(payload))
|
||||
# Core is a reviewed input only here: do not require an unrelated tag
|
||||
# or silently impose a new clean-Core prerequisite for module release.
|
||||
return campaign, remote, lock_path
|
||||
|
||||
def test_module_publication_freezes_core_inputs_without_requiring_core_tag(self):
|
||||
_campaign, _remote, _lock = self._ready_bundle()
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual(["govoplan-campaign"], list(result["source_receipts"]))
|
||||
self.assertEqual(
|
||||
{"webui/package.release.json", "webui/package-lock.release.json"},
|
||||
set(result["bundle_input_receipts"]),
|
||||
)
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_changed_or_group_writable_core_bundle_inputs_block_module_publication(
|
||||
self,
|
||||
):
|
||||
campaign, remote, lock = self._ready_bundle()
|
||||
original = source_tag_batch._preview_repositories
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
lock.write_text(lock.read_text() + "\n")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
source_tag_batch, "_preview_repositories", side_effect=changed
|
||||
):
|
||||
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"bundle input receipt changed", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
|
||||
lock.chmod(0o666)
|
||||
try:
|
||||
self.assertEqual(
|
||||
"blocked",
|
||||
self.tag(repos=("govoplan-campaign",), apply=True, push=True)["status"],
|
||||
)
|
||||
finally:
|
||||
lock.chmod(0o644)
|
||||
self.assertTrue(ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_read_only_shared_preflight_has_no_apply_or_mutating_legacy_entry(self):
|
||||
import inspect
|
||||
from govoplan_release import repository_tag
|
||||
|
||||
self.assertNotIn(
|
||||
"apply", inspect.signature(repository_tag._preview_repositories).parameters
|
||||
)
|
||||
self.assertFalse(hasattr(repository_tag, "_tag_repositories_legacy"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Executable
+194
@@ -0,0 +1,194 @@
|
||||
"""Offline safety and completeness checks for the cross-product UI review program."""
|
||||
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/gitea"))
|
||||
SPEC = importlib.util.spec_from_file_location("ui_review_program", ROOT / "tools/gitea/gitea-ui-review-program.py")
|
||||
assert SPEC and SPEC.loader
|
||||
program = importlib.util.module_from_spec(SPEC)
|
||||
sys.modules[SPEC.name] = program
|
||||
SPEC.loader.exec_module(program)
|
||||
|
||||
|
||||
def scope(scope_id="campaigns", kind="manifest"):
|
||||
return {
|
||||
"scope_id": scope_id, "name": "Campaigns", "repository": "govoplan-campaign",
|
||||
"kind": kind, "manifest_paths": ["src/govoplan_campaign/backend/manifest.py"],
|
||||
"frontend": {
|
||||
"routes": [{"path": "/campaigns/:campaignId/*", "component": "CampaignWorkspace"}],
|
||||
"public_routes": [{"path": "/public/example", "component": "PublicExample"}],
|
||||
"settings_routes": [{"path": "/settings/example", "component": "ExampleSettings"}],
|
||||
"nav_items": [{"path": "/campaigns", "label": "Campaigns"}],
|
||||
"view_surfaces": [{"id": "campaigns.widget.activity", "label": "Activity widget"}],
|
||||
},
|
||||
"source_groups": {"Dialogs and embedded editing surfaces": ["webui/src/ExampleDialog.tsx"]},
|
||||
"ui_source_count": 1,
|
||||
}
|
||||
|
||||
|
||||
def test_catalog_includes_core_all_manifests_and_each_placeholder(tmp_path):
|
||||
names = ["govoplan", "govoplan-core", "govoplan-campaign", "govoplan-ledger", "govoplan-xoev", "website"]
|
||||
for name in names:
|
||||
(tmp_path / name).mkdir()
|
||||
catalog = {"repositories": [
|
||||
{"name": name, "path": name, "category": "website" if name == "website" else "system" if name in {"govoplan", "govoplan-core"} else "connector" if name == "govoplan-xoev" else "module"}
|
||||
for name in names
|
||||
]}
|
||||
manifests = [{"id": "campaigns", "name": "Campaigns", "repository": "govoplan-campaign", "frontend": None}]
|
||||
scopes = program.build_scopes(catalog, tmp_path, manifests)
|
||||
assert {item["scope_id"] for item in scopes} == {"core", "campaigns", "catalog:govoplan-ledger", "catalog:govoplan-xoev"}
|
||||
assert sum(item["kind"] == "placeholder" for item in scopes) == 2
|
||||
assert next(item for item in scopes if item["scope_id"] == "campaigns")["kind"] == "manifest"
|
||||
|
||||
|
||||
def test_missing_checkout_or_implementation_without_manifest_is_not_called_placeholder(tmp_path):
|
||||
catalog = {"repositories": [{"name": "govoplan-example", "path": "govoplan-example", "category": "module"}]}
|
||||
with pytest.raises(program.GiteaError, match="Missing source checkout"):
|
||||
program.build_scopes(catalog, tmp_path, [])
|
||||
root = tmp_path / "govoplan-example"
|
||||
root.mkdir()
|
||||
(root / "pyproject.toml").touch()
|
||||
with pytest.raises(program.GiteaError, match="implementation but no extracted manifest"):
|
||||
program.build_scopes(catalog, tmp_path, [])
|
||||
|
||||
|
||||
def test_duplicate_or_uncatalogued_manifest_is_rejected(tmp_path):
|
||||
manifest = {"id": "example", "name": "Example", "repository": "govoplan-example", "frontend": None}
|
||||
with pytest.raises(program.GiteaError, match="Duplicate source manifest"):
|
||||
program.build_scopes({"repositories": []}, tmp_path, [manifest, manifest])
|
||||
with pytest.raises(program.GiteaError, match="absent from the module review catalog"):
|
||||
program.build_scopes({"repositories": []}, tmp_path, [manifest])
|
||||
|
||||
|
||||
def test_existing_closed_issue_is_found_and_unchanged():
|
||||
issue = {"number": 4, "title": "Reviewer renamed it", "state": "closed", "body": program.marker("campaigns") + "\nHuman findings\n- [x] Done"}
|
||||
before = issue.copy()
|
||||
assert program.find_existing([issue], "campaigns", program.issue_title(scope())) is issue
|
||||
assert issue == before
|
||||
|
||||
|
||||
def test_gitea_null_pull_request_field_is_an_ordinary_issue():
|
||||
issue = {"number": 25, "title": "UI review", "state": "open", "body": program.marker("mail"), "pull_request": None}
|
||||
assert program.find_existing([issue], "mail", "UI review") is issue
|
||||
|
||||
|
||||
def test_unmanaged_title_and_ambiguous_marker_stop_without_overwrite():
|
||||
title = program.issue_title(scope())
|
||||
with pytest.raises(program.GiteaError, match="Unmanaged exact-title"):
|
||||
program.find_existing([{"title": " " + title.upper() + " ", "body": "user content"}], "campaigns", title)
|
||||
duplicate = {"title": title, "body": program.marker("campaigns")}
|
||||
with pytest.raises(program.GiteaError, match="Ambiguous"):
|
||||
program.find_existing([duplicate, duplicate.copy()], "campaigns", title)
|
||||
|
||||
|
||||
def test_pr_not_used_as_matching_issue():
|
||||
issue = {"title": program.issue_title(scope()), "body": program.marker("campaigns"), "pull_request": {}}
|
||||
assert program.find_existing([issue], "campaigns", issue["title"]) is None
|
||||
|
||||
|
||||
def test_child_inventory_and_all_principles_start_pending():
|
||||
body = program.issue_body(scope(), "https://example.test/epic/56")
|
||||
assert "**Pending / not reviewed.**" in body
|
||||
assert "https://example.test/epic/56" in body
|
||||
assert "/campaigns/:campaignId/*" in body
|
||||
assert "/public/example" in body
|
||||
assert "/settings/example" in body
|
||||
assert "campaigns.widget.activity" in body
|
||||
assert "webui/src/ExampleDialog.tsx" in body
|
||||
assert "compact read-only campaign settings dashboard" in body
|
||||
assert "Save/Cancel" in body and "dirty-state protection" in body
|
||||
assert "- [x]" not in body
|
||||
assert body.count("| Pending inventory | Pending review | Not yet recorded | None approved |") == 9
|
||||
for identity, _ in program.PRINCIPLES:
|
||||
assert identity in body
|
||||
assert "Reopen this issue or link an owned follow-up" in body
|
||||
|
||||
|
||||
def test_headless_and_placeholder_scopes_are_not_automatic_completions():
|
||||
headless = scope("rest")
|
||||
headless["frontend"] = None
|
||||
assert "No standalone frontend is declared" in program.issue_body(headless, "epic")
|
||||
assert "**The review is still pending:**" in program.issue_body(headless, "epic")
|
||||
placeholder = scope("catalog:govoplan-ledger", "placeholder")
|
||||
body = program.issue_body(placeholder, "epic")
|
||||
assert "there is no runtime module ID, manifest or standalone WebUI" in body
|
||||
assert "Keep the future interface review pending" in body
|
||||
assert "- [x]" not in body
|
||||
|
||||
|
||||
def test_source_grouping_uses_real_files_and_clearly_bounds_large_seeds(tmp_path):
|
||||
source = tmp_path / "webui/src"
|
||||
source.mkdir(parents=True)
|
||||
for name in ["ExamplePage.tsx", "EditDialog.tsx", "TenantSettings.tsx", "ActivityWidget.tsx", "Button.tsx"]:
|
||||
(source / name).touch()
|
||||
groups = program.source_groups(tmp_path)
|
||||
assert sum(map(len, groups.values())) == 5
|
||||
assert groups["Dialogs and embedded editing surfaces"] == ["webui/src/EditDialog.tsx"]
|
||||
large = scope()
|
||||
large["source_groups"] = {"Pages": [f"webui/src/Page{index}.tsx" for index in range(45)]}
|
||||
large["ui_source_count"] = 45
|
||||
seed = program.source_seed(large)
|
||||
assert "5 further files" in seed
|
||||
assert "not a completed runtime audit" in seed
|
||||
|
||||
|
||||
def test_epic_initialization_preserves_surrounding_text_and_keeps_all_unchecked():
|
||||
body = program.EPIC_MARKER + "\nHuman introduction\n" + program.LIST_START + "\n" + program.INITIAL_LIST + "\n" + program.LIST_END + "\nHuman evidence"
|
||||
records = [
|
||||
{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"},
|
||||
{"name": "Ledger", "scope_id": "catalog:govoplan-ledger", "repository": "govoplan-ledger", "kind": "placeholder", "url": "https://example.test/ledger/2"},
|
||||
]
|
||||
result = program.initialized_epic_body(body, records)
|
||||
assert "Human introduction" in result and "Human evidence" in result
|
||||
assert result.count("- [ ]") == 2
|
||||
assert "Catalogued placeholders" in result
|
||||
assert program.initialized_epic_body(result, records) == result
|
||||
human_progress = result.replace("- [ ] [Core]", "- [x] [Core]")
|
||||
assert program.initialized_epic_body(human_progress, records) == human_progress
|
||||
|
||||
|
||||
def test_epic_edited_or_ambiguous_lists_are_never_overwritten():
|
||||
records = [{"name": "Core", "scope_id": "core", "repository": "govoplan-core", "kind": "core", "url": "https://example.test/core/1"}]
|
||||
body = program.EPIC_MARKER + program.LIST_START + "User-managed content" + program.LIST_END
|
||||
with pytest.raises(program.GiteaError, match="already edited"):
|
||||
program.initialized_epic_body(body, records)
|
||||
with pytest.raises(program.GiteaError, match="absent or ambiguous"):
|
||||
program.initialized_epic_body(body + program.LIST_START, records)
|
||||
with pytest.raises(program.GiteaError, match="incomplete issue link inventory"):
|
||||
program.render_links([{**records[0], "url": None}])
|
||||
|
||||
|
||||
def test_ipv4_override_is_host_scoped_and_restored(monkeypatch):
|
||||
calls = []
|
||||
def original(host, port, family=0, type=0, proto=0, flags=0):
|
||||
calls.append((host, family))
|
||||
return []
|
||||
monkeypatch.setattr(socket, "getaddrinfo", original)
|
||||
with program.ipv4_for_target(True):
|
||||
socket.getaddrinfo("git.add-ideas.de", 443)
|
||||
socket.getaddrinfo("unrelated.example", 443)
|
||||
assert calls == [("git.add-ideas.de", socket.AF_INET), ("unrelated.example", 0)]
|
||||
assert socket.getaddrinfo is original
|
||||
|
||||
|
||||
def test_snapshot_covers_every_current_catalog_module():
|
||||
import json
|
||||
catalog = json.loads((ROOT / "repositories.json").read_text())
|
||||
inventory_path = ROOT / "docs/project/ui-review-issue-inventory.json"
|
||||
snapshot = json.loads(inventory_path.read_text())
|
||||
expected = {repo["name"] for repo in catalog["repositories"] if repo["category"] in {"module", "connector"} or repo["name"] == "govoplan-core"}
|
||||
assert {issue["repository"] for issue in snapshot["issues"]} == expected
|
||||
assert len(snapshot["issues"]) == len(expected)
|
||||
assert len({issue["url"] for issue in snapshot["issues"]}) == len(expected)
|
||||
assert snapshot["scope_count"] == 77
|
||||
assert snapshot["implemented_scopes"] == 73
|
||||
assert snapshot["manifest_modules"] == 72
|
||||
assert snapshot["catalogued_placeholders"] == 4
|
||||
assert all(issue["number"] and issue["url"].startswith("https://git.add-ideas.de/GovOPlaN/") for issue in snapshot["issues"])
|
||||
@@ -0,0 +1,154 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import textwrap
|
||||
import unittest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
INSTALLER = META_ROOT / "tools" / "release" / "install-webui-release-dependencies.sh"
|
||||
STAGES = ("base", "clone", "modules")
|
||||
|
||||
|
||||
class WebUIReleaseDependencyRetryTests(unittest.TestCase):
|
||||
def _run_installer(
|
||||
self, stage: str, statuses: tuple[int, ...]
|
||||
) -> tuple[subprocess.CompletedProcess[str], list[str]]:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-installer-retry-test-") as directory:
|
||||
root = Path(directory)
|
||||
stub_bin = root / "bin"
|
||||
stub_bin.mkdir()
|
||||
core_root = root / "core"
|
||||
webui = core_root / "web ui"
|
||||
webui.mkdir(parents=True)
|
||||
work_root = root / "work"
|
||||
work_root.mkdir()
|
||||
log = root / "commands.log"
|
||||
stub = "#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(
|
||||
r"""
|
||||
case "${0##*/}" in
|
||||
node)
|
||||
# Supply the shell's dependency list without requiring Node.
|
||||
printf '%s\t%s\n' '@govoplan/example-webui' \
|
||||
'git+https://example.invalid/module.git#v1.0.0' > "$GOVOPLAN_DEPS"
|
||||
printf 'node\n' >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
sleep)
|
||||
printf 'sleep %s\n' "$*" >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
npm)
|
||||
case "${1:-}" in
|
||||
cache)
|
||||
printf 'cache\n' >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
install)
|
||||
stage=base
|
||||
for argument in "$@"; do
|
||||
if [[ "$argument" == --no-save ]]; then
|
||||
stage=modules
|
||||
fi
|
||||
done
|
||||
;;
|
||||
*) exit 98 ;;
|
||||
esac
|
||||
;;
|
||||
git)
|
||||
[[ "${1:-}" == clone ]] || exit 98
|
||||
stage=clone
|
||||
;;
|
||||
*) exit 98 ;;
|
||||
esac
|
||||
|
||||
status=0
|
||||
if [[ "$stage" == "$RETRY_TEST_STAGE" ]]; then
|
||||
attempt=0
|
||||
counter="$RETRY_TEST_ROOT/$stage.count"
|
||||
if [[ -f "$counter" ]]; then
|
||||
read -r attempt < "$counter"
|
||||
fi
|
||||
read -r -a statuses <<< "$RETRY_TEST_STATUSES"
|
||||
status="${statuses[$attempt]:-99}"
|
||||
printf '%s\n' "$((attempt + 1))" > "$counter"
|
||||
fi
|
||||
printf '%s %s\n' "$stage" "$status" >> "$RETRY_TEST_LOG"
|
||||
exit "$status"
|
||||
"""
|
||||
)
|
||||
for name in ("node", "npm", "git", "sleep"):
|
||||
executable = stub_bin / name
|
||||
executable.write_text(stub, encoding="utf-8")
|
||||
executable.chmod(0o755)
|
||||
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"PATH": f"{stub_bin}:{os.defpath}",
|
||||
"TMPDIR": str(work_root),
|
||||
"GOVOPLAN_CORE_ROOT": str(core_root),
|
||||
"GOVOPLAN_WEBUI_PACKAGE_LOCK": "",
|
||||
"GOVOPLAN_WEBUI_PACKAGE_DIR": "",
|
||||
"RETRY_TEST_ROOT": str(root),
|
||||
"RETRY_TEST_LOG": str(log),
|
||||
"RETRY_TEST_STAGE": stage,
|
||||
"RETRY_TEST_STATUSES": " ".join(map(str, statuses)),
|
||||
}
|
||||
)
|
||||
result = subprocess.run(
|
||||
[
|
||||
"bash",
|
||||
"-c",
|
||||
'set -euo pipefail; bash "$1" "$2"; '
|
||||
'printf "caller-continued\\n" >> "$RETRY_TEST_LOG"',
|
||||
"retry-test-caller",
|
||||
str(INSTALLER),
|
||||
str(webui),
|
||||
],
|
||||
cwd=root,
|
||||
env=env,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=10,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(list(work_root.iterdir()), [], result.stderr)
|
||||
return result, log.read_text(encoding="utf-8").splitlines()
|
||||
|
||||
def _assert_attempts(self, statuses: tuple[int, ...]) -> None:
|
||||
for retried_stage in STAGES:
|
||||
with self.subTest(stage=retried_stage, statuses=statuses):
|
||||
result, commands = self._run_installer(retried_stage, statuses)
|
||||
expected = ["node", "cache"]
|
||||
for stage in STAGES:
|
||||
attempts = statuses if stage == retried_stage else (0,)
|
||||
for index, status in enumerate(attempts):
|
||||
expected.append(f"{stage} {status}")
|
||||
if status and index < 2:
|
||||
expected.append(f"sleep {(index + 1) * 10}")
|
||||
if attempts[-1]:
|
||||
break
|
||||
if statuses[-1] == 0:
|
||||
expected.append("caller-continued")
|
||||
self.assertEqual(result.returncode, statuses[-1], result.stderr)
|
||||
self.assertEqual(commands, expected, result.stderr)
|
||||
|
||||
def test_success_on_first_attempt(self) -> None:
|
||||
self._assert_attempts((0,))
|
||||
|
||||
def test_success_on_second_attempt(self) -> None:
|
||||
self._assert_attempts((17, 0))
|
||||
|
||||
def test_success_on_third_attempt(self) -> None:
|
||||
self._assert_attempts((17, 23, 0))
|
||||
|
||||
def test_exhaustion_preserves_final_status_and_stops_callers(self) -> None:
|
||||
self._assert_attempts((17, 23, 47))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+280
-69
@@ -1,13 +1,151 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# The JSON catalog is the only phase order/metadata authority. Check commands
|
||||
# remain in the marked functions below; devkit invokes this script, not copies.
|
||||
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
|
||||
|
||||
FOCUSED_MODE=run
|
||||
FOCUSED_PHASE=""
|
||||
FOCUSED_JSON=0
|
||||
|
||||
focused_usage() {
|
||||
echo "Usage: check-focused.sh [--phase ID] | --list-phases [--json]"
|
||||
echo "Without --phase, run every canonical phase in catalog order (fail fast)."
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--phase)
|
||||
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" || $# -lt 2 || -z "$2" || "$2" == --* ]]; then
|
||||
echo "check-focused: --phase requires one ID and cannot be combined with listing." >&2
|
||||
exit 2
|
||||
fi
|
||||
FOCUSED_PHASE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--list-phases)
|
||||
if [[ "$FOCUSED_MODE" != run || -n "$FOCUSED_PHASE" ]]; then
|
||||
echo "check-focused: duplicate or incompatible phase selection." >&2
|
||||
exit 2
|
||||
fi
|
||||
FOCUSED_MODE=list
|
||||
shift
|
||||
;;
|
||||
--json)
|
||||
if [[ "$FOCUSED_JSON" == 1 ]]; then
|
||||
echo "check-focused: duplicate --json." >&2
|
||||
exit 2
|
||||
fi
|
||||
FOCUSED_JSON=1
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
focused_usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "check-focused: unknown argument: $1" >&2
|
||||
focused_usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if [[ "$FOCUSED_JSON" == 1 && "$FOCUSED_MODE" != list ]]; then
|
||||
echo "check-focused: --json is supported only with --list-phases." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Metadata-only operations need standard Python, not an installed product venv,
|
||||
# Node, npm, a Core checkout, or any temporary/state directory.
|
||||
FOCUSED_METADATA_PYTHON="$(command -v python3)" || {
|
||||
echo "check-focused: Python 3 is required to read phase metadata." >&2
|
||||
exit 127
|
||||
}
|
||||
FOCUSED_SELECTION="$(
|
||||
PYTHONDONTWRITEBYTECODE=1 "$FOCUSED_METADATA_PYTHON" -I -S - \
|
||||
"$META_ROOT/tools/checks/focused-phases.json" "$FOCUSED_MODE" "$FOCUSED_PHASE" "$FOCUSED_JSON" <<'PY'
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
import sys
|
||||
|
||||
try:
|
||||
path = Path(sys.argv[1])
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
status = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(status.st_mode) or status.st_size > 1024 * 1024:
|
||||
raise ValueError("phase metadata must be a bounded regular file")
|
||||
encoded = handle.read(1024 * 1024 + 1)
|
||||
if len(encoded) > 1024 * 1024:
|
||||
raise ValueError("phase metadata exceeds its size bound")
|
||||
|
||||
def unique(pairs):
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
if key in result:
|
||||
raise ValueError("duplicate phase metadata key")
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
catalog = json.loads(encoded, object_pairs_hook=unique)
|
||||
if not isinstance(catalog, dict) or set(catalog) != {"schema_version", "phases"} or type(catalog["schema_version"]) is not int or catalog["schema_version"] != 1:
|
||||
raise ValueError("unsupported phase metadata schema")
|
||||
phases = catalog["phases"]
|
||||
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
|
||||
raise ValueError("phase catalog requires 1–32 definitions")
|
||||
seen = set()
|
||||
fields = {"id", "title", "cwd", "order_after", "depends_on", "resources", "outputs", "notes"}
|
||||
for phase in phases:
|
||||
if not isinstance(phase, dict) or set(phase) != fields:
|
||||
raise ValueError("invalid phase metadata fields")
|
||||
identity = phase["id"]
|
||||
if not isinstance(identity, str) or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity) or identity in seen:
|
||||
raise ValueError("invalid or duplicate phase ID")
|
||||
if not isinstance(phase["title"], str) or not phase["title"].strip() or len(phase["title"]) > 256 or any(ord(char) < 32 for char in phase["title"]):
|
||||
raise ValueError("invalid phase title")
|
||||
if phase["cwd"] not in {"core", "meta", "core-webui", "access-webui"}:
|
||||
raise ValueError("unsupported phase working directory")
|
||||
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
|
||||
values = phase[field]
|
||||
if not isinstance(values, list) or len(values) > 64 or any(not isinstance(value, str) or not value or len(value) > 4096 or "\0" in value for value in values):
|
||||
raise ValueError("invalid phase list: " + field)
|
||||
if len(set(values)) != len(values):
|
||||
raise ValueError("duplicate phase list value: " + field)
|
||||
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
|
||||
raise ValueError("phase prerequisites must precede their consumer")
|
||||
seen.add(identity)
|
||||
mode, selected, as_json = sys.argv[2:]
|
||||
if selected and selected not in seen:
|
||||
raise ValueError("unknown phase: " + selected)
|
||||
if mode == "list":
|
||||
print(json.dumps(catalog, indent=2) if as_json == "1" else "\n".join(phase["id"] + "\t" + phase["title"] for phase in phases))
|
||||
else:
|
||||
print("\n".join(phase["id"] for phase in phases if not selected or phase["id"] == selected))
|
||||
except (OSError, ValueError, TypeError, RecursionError) as exc:
|
||||
print("check-focused: " + str(exc), file=sys.stderr)
|
||||
raise SystemExit(2)
|
||||
PY
|
||||
)"
|
||||
|
||||
if [[ "$FOCUSED_MODE" == list ]]; then
|
||||
printf '%s\n' "$FOCUSED_SELECTION"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
focused_setup() {
|
||||
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$META_ROOT")}"
|
||||
export GOVOPLAN_WORKSPACE_ROOT="$WORKSPACE_ROOT"
|
||||
ROOT="${GOVOPLAN_CORE_ROOT:-$WORKSPACE_ROOT/govoplan-core}"
|
||||
ROOT="$(cd "$ROOT" && pwd)"
|
||||
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
|
||||
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
|
||||
NODE="/home/zemion/.nvm/versions/node/v22.22.3/bin"
|
||||
NPM="$NODE/npm"
|
||||
NODE="$(command -v "${NODE:-node}")" || { echo "check-focused: Node is unavailable; run ./devkit doctor." >&2; exit 127; }
|
||||
NPM="$(command -v "${NPM:-npm}")" || { echo "check-focused: npm is unavailable; run ./devkit doctor." >&2; exit 127; }
|
||||
NODE_BIN="$(dirname "$NODE")"
|
||||
WEBUI_BIN="$ROOT/webui/node_modules/.bin"
|
||||
NPM_USERCONFIG="$(mktemp "${TMPDIR:-/tmp}/govoplan-npmrc.XXXXXXXX")"
|
||||
|
||||
@@ -19,7 +157,7 @@ trap 'rm -f "$NPM_USERCONFIG"' EXIT
|
||||
exit 127
|
||||
}
|
||||
|
||||
export PATH="$WEBUI_BIN:$NODE:$PATH"
|
||||
export PATH="$WEBUI_BIN:$NODE_BIN:$PATH"
|
||||
export NPM_CONFIG_USERCONFIG="$NPM_USERCONFIG"
|
||||
export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG"
|
||||
unset npm_config_tmp NPM_CONFIG_TMP
|
||||
@@ -27,44 +165,65 @@ unset npm_config_tmp NPM_CONFIG_TMP
|
||||
# Validate the current sibling checkouts even when a newly added module has not
|
||||
# yet been installed into an existing development virtualenv.
|
||||
SOURCE_PYTHONPATH=""
|
||||
for source_dir in "$META_ROOT"/../govoplan*/src; do
|
||||
for source_dir in "$WORKSPACE_ROOT"/govoplan*/src; do
|
||||
[ -d "$source_dir" ] || continue
|
||||
SOURCE_PYTHONPATH="${SOURCE_PYTHONPATH:+$SOURCE_PYTHONPATH:}$source_dir"
|
||||
done
|
||||
export PYTHONPATH="${SOURCE_PYTHONPATH}${PYTHONPATH:+:$PYTHONPATH}"
|
||||
}
|
||||
|
||||
focused_phase_preflight() {
|
||||
# devkit-phase: preflight begin
|
||||
cd "$ROOT"
|
||||
|
||||
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
|
||||
"$NODE/node" "$META_ROOT/tests/test-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/../govoplan-files/webui/scripts/test-archive-client.mjs"
|
||||
"$NODE" "$META_ROOT/tests/test-jsx-value-imports.mjs"
|
||||
"$NODE" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
|
||||
"$NODE" "$META_ROOT/tests/test-heading-help.mjs"
|
||||
"$NODE" "$META_ROOT/tools/checks/check-heading-help.mjs"
|
||||
"$NODE" --test "$META_ROOT/tests/test-devkit-display-labels.mjs"
|
||||
"$NODE" --test "$ROOT/webui/tests/component-test-runner.test.mjs"
|
||||
"$NODE" "$WORKSPACE_ROOT/govoplan-files/webui/scripts/test-archive-client.mjs"
|
||||
# devkit-phase: preflight end
|
||||
}
|
||||
|
||||
focused_phase_tooling() {
|
||||
# devkit-phase: tooling begin
|
||||
cd "$META_ROOT"
|
||||
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
|
||||
"$PYTHON" tools/inventory/platform-interface-inventory.py --workspace-root "$WORKSPACE_ROOT" --strict-declarations --strict-endpoints
|
||||
"$PYTHON" tools/repo/sync-module-package-workflows.py --check
|
||||
"$PYTHON" tools/release/generate-developer-meta-package.py --check
|
||||
"$PYTHON" tools/checks/check-webui-package-facades.py
|
||||
"$PYTHON" -m unittest tests.test_webui_package_facades
|
||||
"$PYTHON" -m pytest -q tests/test_ui_review_program.py "$META_ROOT"/tests/test_devkit_*.py
|
||||
"$PYTHON" -m pytest -q tests/test_focused_phases.py
|
||||
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries
|
||||
"$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py
|
||||
"$PYTHON" -m unittest tests.test_isolated_work_composition
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_evidence
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_generation tests.test_capability_fit_review
|
||||
"$PYTHON" tools/assessments/generate-capability-fit-report.py --check
|
||||
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
|
||||
"$PYTHON" -m unittest tests.test_institutional_governance_journey
|
||||
"$PYTHON" -m unittest tests.test_institutional_service_journey
|
||||
# devkit-phase: tooling end
|
||||
}
|
||||
|
||||
focused_phase_backend() {
|
||||
# devkit-phase: backend begin
|
||||
cd "$ROOT"
|
||||
|
||||
"$PYTHON" - <<'PY'
|
||||
import ast
|
||||
import pathlib
|
||||
import os
|
||||
import sys
|
||||
|
||||
repos_root = pathlib.Path("/mnt/DATA/git")
|
||||
repos_root = pathlib.Path(os.environ["GOVOPLAN_WORKSPACE_ROOT"])
|
||||
roots = [
|
||||
repo / "src"
|
||||
for repo in sorted(repos_root.glob("govoplan*"))
|
||||
@@ -101,96 +260,131 @@ PY
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py"
|
||||
"$PYTHON" -m unittest tests.test_module_system
|
||||
"$PYTHON" -m unittest tests.test_bounded_process
|
||||
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
|
||||
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_managed_archives.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_work.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_staging.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_upload_response_batching.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_performance.py
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests/test_external_function_mapping_migration.py
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-quick-access/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-payments/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms-runtime/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-cases/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-committee/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-voting/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-approvals/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_managed_archives.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_work.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_staging.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_upload_response_batching.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_performance.py"
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_editor_state_security.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_mail_profile_boundary.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_independent_configuration_repairs.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_incremental_review_persistence.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_reviewed_build_mock.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_delivery_policy_settings.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_synchronous_delivery_policy.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_workerless_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_imap_batch_integration.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_testbed_claim_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_archive_encryption_governance.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_campaign_archive_encryption.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_archive_encryption_api.py
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_workers.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archive_inspection_bounds.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-files/tests/test_archives.py"
|
||||
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests/test_external_function_mapping_migration.py"
|
||||
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-access/tests"
|
||||
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-templates/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-connectors/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-datasources/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dataflow/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow-engine/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-workflow/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-views/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-quick-access/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-dashboard/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-postbox/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-portal/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-payments/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-forms-runtime/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-cases/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-committee/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-voting/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-approvals/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-identity-trust/tests"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-encryption/tests"
|
||||
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-wiki/tests"
|
||||
"$PYTHON" -m pytest -q \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_approval_gate.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_editor_state_security.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_mail_profile_boundary.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_independent_configuration_repairs.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_incremental_review_persistence.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_reviewed_build_mock.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_delivery_policy_settings.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_synchronous_delivery_policy.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_workerless_recovery.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_imap_batch_integration.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_testbed_claim_recovery.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-campaign/tests/test_archive_encryption_governance.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-policy/tests/test_campaign_archive_encryption.py" \
|
||||
"${WORKSPACE_ROOT}/govoplan-policy/tests/test_archive_encryption_api.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-mail/tests/test_campaign_protocol_authorization.py /mnt/DATA/git/govoplan-mail/tests/test_campaign_imap_batch.py
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
|
||||
"$PYTHON" -m pytest -q "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_protocol_authorization.py" "${WORKSPACE_ROOT}/govoplan-mail/tests/test_campaign_imap_batch.py"
|
||||
"$PYTHON" -m unittest discover -s "${WORKSPACE_ROOT}/govoplan-mail/tests"
|
||||
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
|
||||
"$PYTHON" -m unittest \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry
|
||||
# devkit-phase: backend end
|
||||
}
|
||||
|
||||
focused_phase_core_ui() {
|
||||
# devkit-phase: core-ui begin
|
||||
cd "$ROOT/webui"
|
||||
"$NPM" run test:api-client-cache
|
||||
"$NPM" run test:auth-action-state
|
||||
"$NPM" run test:dependency-security
|
||||
"$NPM" run test:layout-primitives
|
||||
"$NPM" run test:mail-components
|
||||
"$NPM" run test:components -- layout-primitives page-layout data-grid-actions mail-components
|
||||
"$NODE" --test tests/breadcrumb-bar.test.mjs
|
||||
"$NPM" run test:module-capabilities
|
||||
# devkit-phase: core-ui end
|
||||
}
|
||||
|
||||
focused_phase_module_builds() {
|
||||
# devkit-phase: module-builds begin
|
||||
cd "$ROOT/webui"
|
||||
"$NPM" run test:module-permutations
|
||||
# devkit-phase: module-builds end
|
||||
}
|
||||
|
||||
focused_phase_browser() {
|
||||
# devkit-phase: browser begin
|
||||
cd "$ROOT/webui"
|
||||
"$NPM" run test:conformance
|
||||
# devkit-phase: browser end
|
||||
}
|
||||
|
||||
"$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json
|
||||
focused_phase_module_ui() {
|
||||
# devkit-phase: module-ui begin
|
||||
cd "${WORKSPACE_ROOT}/govoplan-access/webui"
|
||||
"$NPM" run test:passwords
|
||||
|
||||
cd /mnt/DATA/git/govoplan-payments/webui
|
||||
"$WEBUI_BIN/tsc" -p "${WORKSPACE_ROOT}/govoplan-payments/webui/tsconfig.json"
|
||||
|
||||
cd "${WORKSPACE_ROOT}/govoplan-payments/webui"
|
||||
"$NPM" run test:interface-pattern
|
||||
|
||||
cd /mnt/DATA/git/govoplan-dataflow/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-dataflow/webui"
|
||||
"$NPM" run test:structure
|
||||
|
||||
cd /mnt/DATA/git/govoplan-datasources/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-datasources/webui"
|
||||
"$NPM" run typecheck
|
||||
|
||||
cd /mnt/DATA/git/govoplan-workflow/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-workflow/webui"
|
||||
"$NPM" run typecheck
|
||||
|
||||
cd /mnt/DATA/git/govoplan-dashboard/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-dashboard/webui"
|
||||
"$NPM" run test:dashboard-layout
|
||||
|
||||
cd /mnt/DATA/git/govoplan-approvals/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-approvals/webui"
|
||||
"$NPM" run test:workspace-layout
|
||||
|
||||
cd /mnt/DATA/git/govoplan-postbox/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-postbox/webui"
|
||||
"$NPM" run test:ui-structure
|
||||
|
||||
cd /mnt/DATA/git/govoplan-mail/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-mail/webui"
|
||||
"$NPM" run test:mail-ui
|
||||
|
||||
cd /mnt/DATA/git/govoplan-files/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-files/webui"
|
||||
"$NPM" run test:managed-archive
|
||||
|
||||
cd /mnt/DATA/git/govoplan-campaign/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-campaign/webui"
|
||||
"$NPM" run test:policy-ui
|
||||
"$NPM" run test:template-preview
|
||||
"$NPM" run test:review-workflow
|
||||
@@ -198,8 +392,25 @@ cd /mnt/DATA/git/govoplan-campaign/webui
|
||||
"$NPM" run test:campaign-collaboration
|
||||
"$NPM" run test:campaign-work
|
||||
|
||||
cd /mnt/DATA/git/govoplan-policy/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-policy/webui"
|
||||
"$NPM" run test:archive-encryption
|
||||
|
||||
cd /mnt/DATA/git/govoplan-wiki/webui
|
||||
cd "${WORKSPACE_ROOT}/govoplan-wiki/webui"
|
||||
"$NPM" run test:interface-pattern
|
||||
# devkit-phase: module-ui end
|
||||
}
|
||||
|
||||
# Validate every selected implementation before setup can create a temp npmrc.
|
||||
while IFS= read -r phase_id; do
|
||||
if ! declare -F "focused_phase_${phase_id//-/_}" >/dev/null; then
|
||||
echo "check-focused: missing phase implementation: $phase_id" >&2
|
||||
exit 2
|
||||
fi
|
||||
done <<< "$FOCUSED_SELECTION"
|
||||
|
||||
while IFS= read -r phase_id; do
|
||||
(
|
||||
focused_setup
|
||||
"focused_phase_${phase_id//-/_}"
|
||||
)
|
||||
done <<< "$FOCUSED_SELECTION"
|
||||
|
||||
Executable
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env node
|
||||
/** UI-01: contextual documentation belongs beside text, never in action slots. */
|
||||
import { existsSync, readFileSync, readdirSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { relative, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const workspaceRoot = resolve(import.meta.dirname, "../../..");
|
||||
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
|
||||
const ts = require("typescript");
|
||||
const titleOwners = new Set(["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"]);
|
||||
const interactiveOwners = new Set(["a", "button", "Button", "IconButton"]);
|
||||
// Existing domain dialog adapter; its owning structural tests must retain
|
||||
// forwarding to Core Dialog.titleHelp (not a module-local heading definition).
|
||||
const domainTitleOwners = new Map([["FileDialog", "/govoplan-files/webui/src/"]]);
|
||||
|
||||
export function findDetachedDocumentation(sources) {
|
||||
const files = new Map(sources.map(({ path, source }) => [resolve(path),
|
||||
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
|
||||
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
|
||||
const host = ts.createCompilerHost(options);
|
||||
host.getSourceFile = (path) => files.get(resolve(path));
|
||||
const checker = ts.createProgram([...files.keys()], options, host).getTypeChecker();
|
||||
const findings = [];
|
||||
let links = 0;
|
||||
for (const [path, source] of files) {
|
||||
const identifiers = [];
|
||||
const helpNodes = [];
|
||||
function importedName(node) {
|
||||
if (ts.isPropertyAccessExpression(node) && ts.isIdentifier(node.expression)) {
|
||||
const namespace = checker.getSymbolAtLocation(node.expression)?.declarations?.find(ts.isNamespaceImport);
|
||||
if (namespace) return node.name.text;
|
||||
}
|
||||
const declarations = checker.getSymbolAtLocation(node)?.declarations ?? [];
|
||||
const declaration = declarations.find(ts.isImportSpecifier);
|
||||
if (declaration) return (declaration.propertyName ?? declaration.name).text;
|
||||
const defaultImport = declarations.find(ts.isImportClause);
|
||||
if (defaultImport && ts.isStringLiteral(defaultImport.parent.moduleSpecifier)) {
|
||||
const component = defaultImport.parent.moduleSpecifier.text.split("/").at(-1).replace(/\.[cm]?[jt]sx?$/, "");
|
||||
if (component === "DocumentationHelpLink" || titleOwners.has(component) || component === "TextWithHelp" || interactiveOwners.has(component)) return component;
|
||||
}
|
||||
return node.getText(source);
|
||||
}
|
||||
function tag(node) {
|
||||
const opening = ts.isJsxElement(node) ? node.openingElement : ts.isJsxSelfClosingElement(node) ? node : null;
|
||||
return opening ? importedName(opening.tagName) : null;
|
||||
}
|
||||
function collect(node) {
|
||||
if (ts.isIdentifier(node)) identifiers.push(node);
|
||||
if ((ts.isJsxSelfClosingElement(node) || ts.isJsxElement(node)) && tag(node) === "DocumentationHelpLink") helpNodes.push(node);
|
||||
ts.forEachChild(node, collect);
|
||||
}
|
||||
collect(source);
|
||||
|
||||
function staticallyHidden(opening) {
|
||||
const hidden = opening.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "hidden");
|
||||
if (!hidden) return false;
|
||||
if (!hidden.initializer || ts.isStringLiteral(hidden.initializer)) return true;
|
||||
return ts.isJsxExpression(hidden.initializer) && hidden.initializer.expression?.kind === ts.SyntaxKind.TrueKeyword;
|
||||
}
|
||||
|
||||
// Reject text that is definitely absent while preserving dynamic translated
|
||||
// titles and components whose rendered text cannot be established statically.
|
||||
function emptyText(node, seen = new Set()) {
|
||||
if (!node) return true;
|
||||
if (ts.isJsxText(node)) return !node.getText(source).trim();
|
||||
if (ts.isJsxExpression(node)) return emptyText(node.expression, seen);
|
||||
if (ts.isStringLiteralLike(node)) return !node.text.trim();
|
||||
if ([ts.SyntaxKind.NullKeyword, ts.SyntaxKind.FalseKeyword, ts.SyntaxKind.TrueKeyword].includes(node.kind) || ts.isVoidExpression(node)) return true;
|
||||
if (ts.isParenthesizedExpression(node) || ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isNonNullExpression(node)) return emptyText(node.expression, seen);
|
||||
if (ts.isIdentifier(node)) {
|
||||
const symbol = checker.getSymbolAtLocation(node);
|
||||
const declaration = symbol?.declarations?.find(ts.isVariableDeclaration);
|
||||
const immutable = declaration && ts.isVariableDeclarationList(declaration.parent) && Boolean(declaration.parent.flags & ts.NodeFlags.Const);
|
||||
if (immutable && declaration.initializer && !seen.has(symbol)) return emptyText(declaration.initializer, new Set(seen).add(symbol));
|
||||
return node.text === "undefined" && !symbol?.declarations?.length;
|
||||
}
|
||||
if (ts.isConditionalExpression(node)) return emptyText(node.whenTrue, seen) && emptyText(node.whenFalse, seen);
|
||||
if (ts.isJsxFragment(node)) return node.children.every((child) => emptyText(child, seen));
|
||||
if (ts.isArrayLiteralExpression(node)) return node.elements.every((child) => emptyText(child, seen));
|
||||
if (ts.isJsxElement(node) || ts.isJsxSelfClosingElement(node)) {
|
||||
const opening = ts.isJsxElement(node) ? node.openingElement : node;
|
||||
if (staticallyHidden(opening)) return true;
|
||||
if (/^[a-z]/.test(opening.tagName.getText(source))) return !ts.isJsxElement(node) || node.children.every((child) => emptyText(child, seen));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function isAnchored(node, seen = new Set()) {
|
||||
// Check the complete rendered ancestry before returning at a recognized
|
||||
// slot: the entire heading/text contract may itself be inside a button.
|
||||
for (let parent = node.parent; parent; parent = parent.parent) {
|
||||
if (interactiveOwners.has(tag(parent))) return false;
|
||||
const opening = ts.isJsxElement(parent) ? parent.openingElement : ts.isJsxSelfClosingElement(parent) ? parent : null;
|
||||
if (opening && staticallyHidden(opening)) return false;
|
||||
}
|
||||
for (let parent = node.parent; parent; parent = parent.parent) {
|
||||
if (ts.isJsxAttribute(parent)) {
|
||||
const owner = parent.parent.parent;
|
||||
const name = importedName(owner.tagName);
|
||||
const slot = parent.name.getText(source);
|
||||
if (slot === "titleHelp" && (titleOwners.has(name) || (domainTitleOwners.has(name) && path.includes(domainTitleOwners.get(name))))) {
|
||||
if (name === "PageTitle") {
|
||||
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
|
||||
return Boolean(element?.children.some((child) => !emptyText(child)));
|
||||
}
|
||||
const title = owner.attributes.properties.find((attribute) => ts.isJsxAttribute(attribute) && attribute.name.getText(source) === "title");
|
||||
return Boolean(title && !emptyText(title.initializer));
|
||||
}
|
||||
if (slot === "help" && name === "TextWithHelp") {
|
||||
const element = ts.isJsxOpeningElement(owner) ? owner.parent : null;
|
||||
return Boolean(element?.children.some((child) => !emptyText(child)));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (ts.isVariableDeclaration(parent) && ts.isIdentifier(parent.name)) {
|
||||
const symbol = checker.getSymbolAtLocation(parent.name);
|
||||
if (!symbol || seen.has(symbol)) return false;
|
||||
const next = new Set(seen).add(symbol);
|
||||
const references = identifiers.filter((identifier) => identifier !== parent.name && checker.getSymbolAtLocation(identifier) === symbol);
|
||||
return references.length > 0 && references.every((reference) => isAnchored(reference, next));
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
for (const node of helpNodes) {
|
||||
links += 1;
|
||||
// FieldLabel is the central label+book implementation; its browser/component
|
||||
// contract verifies sibling text and prevents nested interactive controls.
|
||||
if (path.endsWith("/govoplan-core/webui/src/components/help/FieldLabel.tsx")) continue;
|
||||
if (!isAnchored(node)) {
|
||||
const position = source.getLineAndCharacterOfPosition(node.getStart(source));
|
||||
findings.push({ path, line: position.line + 1, column: position.character + 1 });
|
||||
}
|
||||
}
|
||||
}
|
||||
return { findings, links };
|
||||
}
|
||||
|
||||
function sourceFiles(directory) {
|
||||
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
|
||||
const path = resolve(directory, entry.name);
|
||||
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
|
||||
});
|
||||
}
|
||||
|
||||
export function checkWorkspace(root = workspaceRoot) {
|
||||
const modules = readdirSync(root, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
|
||||
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
|
||||
const paths = modules.flatMap(sourceFiles);
|
||||
const { findings, links } = findDetachedDocumentation(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
|
||||
for (const finding of findings) {
|
||||
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: UI-01 documentation must use a heading's titleHelp or TextWithHelp beside visible text, not an action slot or detached row.`);
|
||||
}
|
||||
if (!findings.length) console.log(`Heading-help contract passed: ${links} documentation links in ${paths.length} TSX files across ${modules.length} WebUI modules.`);
|
||||
return findings.length ? 1 : 0;
|
||||
}
|
||||
|
||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) process.exitCode = checkWorkspace();
|
||||
@@ -4,6 +4,7 @@ set -euo pipefail
|
||||
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
ROOT="${GOVOPLAN_CORE_ROOT:-$META_ROOT/../govoplan-core}"
|
||||
ROOT="$(cd "$ROOT" && pwd)"
|
||||
WORKSPACE_ROOT="${GOVOPLAN_WORKSPACE_ROOT:-$(dirname "$ROOT")}"
|
||||
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
|
||||
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
|
||||
NPM="${NPM:-/home/zemion/.nvm/versions/node/v22.22.3/bin/npm}"
|
||||
@@ -184,6 +185,7 @@ run_step "Validate installed module manifests and registry"
|
||||
|
||||
run_step "Validate platform interface and endpoint declarations"
|
||||
"$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \
|
||||
--workspace-root "$WORKSPACE_ROOT" \
|
||||
--strict-declarations \
|
||||
--strict-endpoints
|
||||
|
||||
|
||||
Executable
+142
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"phases": [
|
||||
{
|
||||
"id": "preflight",
|
||||
"title": "Dependency and shared contract preflight",
|
||||
"cwd": "core",
|
||||
"order_after": [],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"backend:test-state"
|
||||
],
|
||||
"outputs": [],
|
||||
"notes": [
|
||||
"Checks dependency hygiene, cross-module contracts, manifests and static WebUI conventions."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "tooling",
|
||||
"title": "Workspace inventories and development/release tooling tests",
|
||||
"cwd": "meta",
|
||||
"order_after": [
|
||||
"preflight"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"backend:test-state",
|
||||
"artifact:platform-inventory"
|
||||
],
|
||||
"outputs": [
|
||||
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.json",
|
||||
"{meta}/audit-reports/platform-inventory/platform-interface-inventory.md"
|
||||
],
|
||||
"notes": [
|
||||
"Inventory reports are regenerated here; no later phase consumes them. Reused check evidence does not promise these reports still exist."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "backend",
|
||||
"title": "Workspace syntax, backend modules and integration checks",
|
||||
"cwd": "core",
|
||||
"order_after": [
|
||||
"tooling"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"backend:test-state"
|
||||
],
|
||||
"outputs": [],
|
||||
"notes": [
|
||||
"Tests may create their own temporary fixtures or test caches; no generated artifact is required by a later phase."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "core-ui",
|
||||
"title": "Core source tests and selected component contracts",
|
||||
"cwd": "core-webui",
|
||||
"order_after": [
|
||||
"backend"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"webui:govoplan-core"
|
||||
],
|
||||
"outputs": [
|
||||
"{core}/webui/.module-test-build"
|
||||
],
|
||||
"notes": [
|
||||
"The selected component batch creates and removes a private compilation directory. Module-capability tests rebuild .module-test-build themselves; later phases do not consume it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "module-builds",
|
||||
"title": "Optional-module build permutations and bundle checks",
|
||||
"cwd": "core-webui",
|
||||
"order_after": [
|
||||
"core-ui"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"webui:govoplan-core"
|
||||
],
|
||||
"outputs": [
|
||||
"{core}/webui/dist",
|
||||
"{core}/webui/dist/module-permutation-bundle-metrics.json"
|
||||
],
|
||||
"notes": [
|
||||
"Each permutation builds and reads its own fresh bundle metrics. Final dist and aggregate metrics persist, but browser/module-ui phases do not consume them. Reused check evidence is not artifact or deployment attestation."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "browser",
|
||||
"title": "Core browser conformance",
|
||||
"cwd": "core-webui",
|
||||
"order_after": [
|
||||
"module-builds"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"webui:govoplan-core",
|
||||
"browser:chromium",
|
||||
"port:4174"
|
||||
],
|
||||
"outputs": [
|
||||
"{core}/webui/test-results",
|
||||
"{core}/webui/node_modules/.vite/govoplan-conformance"
|
||||
],
|
||||
"notes": [
|
||||
"Playwright starts its own Vite server from conformance sources with an isolated cache. It does not serve or require module-builds dist. The server is owned by the phase and stopped on completion."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "module-ui",
|
||||
"title": "Owning-module UI and type checks",
|
||||
"cwd": "access-webui",
|
||||
"order_after": [
|
||||
"browser"
|
||||
],
|
||||
"depends_on": [],
|
||||
"resources": [
|
||||
"webui:govoplan-core",
|
||||
"webui:govoplan-access",
|
||||
"webui:govoplan-payments",
|
||||
"webui:govoplan-dataflow",
|
||||
"webui:govoplan-datasources",
|
||||
"webui:govoplan-workflow",
|
||||
"webui:govoplan-dashboard",
|
||||
"webui:govoplan-approvals",
|
||||
"webui:govoplan-postbox",
|
||||
"webui:govoplan-mail",
|
||||
"webui:govoplan-files",
|
||||
"webui:govoplan-campaign",
|
||||
"webui:govoplan-policy",
|
||||
"webui:govoplan-wiki"
|
||||
],
|
||||
"outputs": [],
|
||||
"notes": [
|
||||
"The original 19-command tail retains each owning package's cwd. Its scripts prepare any private test output they need; no earlier phase artifact is a prerequisite."
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -13,7 +13,10 @@ from urllib.parse import urlsplit
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
DEFAULT_GARAGE_IMAGE = "dxflrs/garage:v2.3.0"
|
||||
DEFAULT_LOAD_BALANCER_IMAGE = "haproxy:3.2.21-alpine"
|
||||
DEFAULT_LOAD_BALANCER_IMAGE = (
|
||||
"haproxy:3.2.23-alpine@sha256:"
|
||||
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e"
|
||||
)
|
||||
DEFAULT_INGRESS_IMAGE = "caddy:2.10.2-alpine"
|
||||
INSTALLATION_ID_PATTERN = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
|
||||
ENV_NAME_PATTERN = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
|
||||
|
||||
Executable
+212
@@ -0,0 +1,212 @@
|
||||
#!/usr/bin/env node
|
||||
// Complement the existing i18n-marker inventory with known plain-text display
|
||||
// slots. Parse source only: never execute module registration/catalog code.
|
||||
import { existsSync, readFileSync, readdirSync, realpathSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const displayProps = new Map([
|
||||
...["PageLayout", "PageHeader", "PageTitle", "AdminPageLayout", "Card", "Dialog", "PageActionBar", "WorkspaceActionBar"].map((name) => [name, new Set(["title", "subtitle"])]),
|
||||
["FieldLabel", new Set(["label"])], ["MetricCard", new Set(["label"])],
|
||||
]);
|
||||
const childOwners = new Set(["PageTitle", "TextWithHelp", "h1", "h2", "h3", "h4", "h5", "h6"]);
|
||||
const unknown = Symbol("dynamic");
|
||||
|
||||
export function createReader(ts, allowedRoot) {
|
||||
const sources = new Map();
|
||||
function source(file) {
|
||||
file = resolve(file);
|
||||
if (file !== allowedRoot && !file.startsWith(`${resolve(allowedRoot)}/`)) return null;
|
||||
if (!existsSync(file)) return null;
|
||||
if (!realpathSync(file).startsWith(`${resolve(allowedRoot)}/`)) return null;
|
||||
if (!sources.has(file)) sources.set(file, ts.createSourceFile(file, readFileSync(file, "utf8"), ts.ScriptTarget.Latest, true, file.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS));
|
||||
return sources.get(file);
|
||||
}
|
||||
function unwrap(node) {
|
||||
while (node && (ts.isAsExpression(node) || ts.isSatisfiesExpression(node) || ts.isParenthesizedExpression(node))) node = node.expression;
|
||||
return node;
|
||||
}
|
||||
function imported(sf, identifier) {
|
||||
for (const item of sf.statements) {
|
||||
if (!ts.isImportDeclaration(item) || !item.importClause || !ts.isStringLiteral(item.moduleSpecifier)) continue;
|
||||
const names = item.importClause.namedBindings;
|
||||
if (names && ts.isNamedImports(names)) {
|
||||
const match = names.elements.find((entry) => entry.name.text === identifier);
|
||||
if (match) return { path: item.moduleSpecifier.text, name: match.propertyName?.text ?? match.name.text };
|
||||
}
|
||||
if (item.importClause.name?.text === identifier) return { path: item.moduleSpecifier.text, name: "default" };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
function targetFile(sf, specifier) {
|
||||
if (!specifier.startsWith(".")) return null;
|
||||
const base = resolve(dirname(sf.fileName), specifier);
|
||||
return [base, `${base}.ts`, `${base}.tsx`, join(base, "index.ts")].find((file) => existsSync(file) && /\.tsx?$/.test(file)) ?? null;
|
||||
}
|
||||
function declaration(sf, name) {
|
||||
for (const statement of sf.statements) {
|
||||
if (ts.isVariableStatement(statement) && statement.declarationList.flags & ts.NodeFlags.Const) {
|
||||
const item = statement.declarationList.declarations.find((node) => ts.isIdentifier(node.name) && node.name.text === name);
|
||||
if (item) return item.initializer;
|
||||
}
|
||||
if (name === "default" && ts.isExportAssignment(statement)) return statement.expression;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
function value(node, sf, seen = new Set()) {
|
||||
node = unwrap(node);
|
||||
if (!node) return unknown;
|
||||
if (ts.isStringLiteralLike(node)) return node.text;
|
||||
if (ts.isIdentifier(node)) {
|
||||
const key = `${sf.fileName}:${node.text}`;
|
||||
if (seen.has(key)) return unknown;
|
||||
const visited = new Set([...seen, key]);
|
||||
const local = declaration(sf, node.text);
|
||||
if (local) return value(local, sf, visited);
|
||||
const external = imported(sf, node.text);
|
||||
const target = external && targetFile(sf, external.path);
|
||||
const loaded = target && source(target);
|
||||
return loaded ? value(declaration(loaded, external.name), loaded, visited) : unknown;
|
||||
}
|
||||
if (ts.isObjectLiteralExpression(node)) {
|
||||
const result = {};
|
||||
for (const property of node.properties) {
|
||||
if (ts.isSpreadAssignment(property)) {
|
||||
const spread = value(property.expression, sf, seen);
|
||||
if (spread !== unknown && spread && typeof spread === "object") Object.assign(result, spread);
|
||||
else result.__dynamicSpread = true;
|
||||
} else if (ts.isPropertyAssignment(property)) {
|
||||
result[property.name.text ?? property.name.getText(sf)] = value(property.initializer, sf, seen);
|
||||
} else if (ts.isShorthandPropertyAssignment(property)) result[property.name.text] = value(property.name, sf, seen);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
return unknown;
|
||||
}
|
||||
return { source, value, declaration, imported, unwrap };
|
||||
}
|
||||
|
||||
function componentName(ts, reader, sf, node) {
|
||||
const written = node.getText(sf);
|
||||
if (/^h[1-6]$/.test(written)) return written;
|
||||
if (ts.isIdentifier(node)) {
|
||||
const imported = reader.imported(sf, written);
|
||||
if (imported?.name === "default") return imported.path.split("/").at(-1).replace(/\.[tj]sx?$/, "");
|
||||
return imported?.name ?? written;
|
||||
}
|
||||
if (ts.isPropertyAccessExpression(node)) {
|
||||
const owner = node.expression.getText(sf);
|
||||
const namespace = sf.statements.find((item) => ts.isImportDeclaration(item) && item.importClause?.namedBindings && ts.isNamespaceImport(item.importClause.namedBindings) && item.importClause.namedBindings.name.text === owner);
|
||||
if (namespace) return node.name.text;
|
||||
}
|
||||
return written;
|
||||
}
|
||||
|
||||
function sourceFiles(directory) {
|
||||
if (!existsSync(directory)) return [];
|
||||
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
|
||||
if (entry.name.startsWith(".") || entry.name === "node_modules") return [];
|
||||
const file = join(directory, entry.name);
|
||||
if (entry.isDirectory()) return sourceFiles(file);
|
||||
return entry.isFile() && /\.[tj]sx?$/.test(file) ? [file] : [];
|
||||
});
|
||||
}
|
||||
|
||||
export function auditRepository(ts, repositoryRoot, coreRoot) {
|
||||
const reader = createReader(ts, repositoryRoot);
|
||||
const coreReader = createReader(ts, coreRoot);
|
||||
const coreFile = coreReader.source(join(coreRoot, "webui/src/i18n/generatedTranslations.ts"));
|
||||
const coreCatalog = coreFile ? coreReader.value(coreReader.declaration(coreFile, "generatedTranslations"), coreFile) : {};
|
||||
const moduleFile = reader.source(join(repositoryRoot, "webui/src/module.ts"));
|
||||
let moduleCatalog = {};
|
||||
let registration = repositoryRoot === coreRoot ? "core-default" : "absent";
|
||||
if (moduleFile) {
|
||||
for (const statement of moduleFile.statements) {
|
||||
if (!ts.isVariableStatement(statement) || !statement.modifiers?.some((item) => item.kind === ts.SyntaxKind.ExportKeyword)) continue;
|
||||
for (const node of statement.declarationList.declarations) {
|
||||
if (!node.initializer || !(/PlatformWebModule/.test(node.type?.getText(moduleFile) ?? "") || /Module$/.test(node.name.getText(moduleFile)))) continue;
|
||||
const evaluated = reader.value(node.initializer, moduleFile);
|
||||
if (evaluated && typeof evaluated === "object" && Object.hasOwn(evaluated, "translations")) {
|
||||
moduleCatalog = evaluated.translations;
|
||||
registration = moduleCatalog !== unknown && moduleCatalog && typeof moduleCatalog === "object" &&
|
||||
!moduleCatalog.__dynamicSpread && !moduleCatalog.en?.__dynamicSpread && !moduleCatalog.de?.__dynamicSpread ? "registered" : "dynamic";
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
const findings = [], review = [], labels = [];
|
||||
const add = (text, node, sf, slot) => {
|
||||
const position = sf.getLineAndCharacterOfPosition(node.getStart(sf));
|
||||
const location = { file: sf.fileName, line: position.line + 1, slot };
|
||||
if (typeof text !== "string") {
|
||||
review.push({ ...location, code: "dynamic-display-slot", message: "Runtime data or computed text: verify with the owning module/locale context." });
|
||||
return;
|
||||
}
|
||||
text = text.replace(/\s+/g, " ").trim();
|
||||
if (!text || text.startsWith("i18n:") || !/[\p{L}]/u.test(text)) return;
|
||||
const missing = ["en", "de"].filter((locale) => {
|
||||
const translated = moduleCatalog?.[locale]?.[text] ?? coreCatalog?.[locale]?.[text];
|
||||
return typeof translated !== "string" || !translated.trim();
|
||||
});
|
||||
labels.push({ ...location, text, missing_locales: missing });
|
||||
if (missing.length) {
|
||||
const item = { ...location, code: "plain-label-missing-translation", text, missing_locales: missing, registration };
|
||||
if (registration === "dynamic") review.push({ ...item, code: "dynamic-catalog-review" });
|
||||
else findings.push(item);
|
||||
}
|
||||
};
|
||||
for (const file of sourceFiles(join(repositoryRoot, "webui/src"))) {
|
||||
if (file.includes("/i18n/")) continue;
|
||||
const sf = reader.source(file);
|
||||
function textChild(node, owner) {
|
||||
if (ts.isJsxText(node)) add(node.text, node, sf, `${owner}.children`);
|
||||
else if (ts.isJsxExpression(node)) { if (node.expression) add(reader.value(node.expression, sf), node, sf, `${owner}.children`); }
|
||||
else if (ts.isJsxElement(node) || ts.isJsxFragment(node)) for (const child of node.children) textChild(child, owner);
|
||||
}
|
||||
function visit(node) {
|
||||
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
|
||||
const owner = componentName(ts, reader, sf, node.tagName);
|
||||
for (const property of node.attributes.properties) {
|
||||
if (!ts.isJsxAttribute(property) || !displayProps.get(owner)?.has(property.name.text) || !property.initializer) continue;
|
||||
const expression = ts.isJsxExpression(property.initializer) ? property.initializer.expression : property.initializer;
|
||||
add(reader.value(expression, sf), property, sf, `${owner}.${property.name.text}`);
|
||||
}
|
||||
}
|
||||
if (ts.isJsxElement(node)) {
|
||||
const owner = componentName(ts, reader, sf, node.openingElement.tagName);
|
||||
if (childOwners.has(owner)) for (const child of node.children) textChild(child, owner);
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
visit(sf);
|
||||
}
|
||||
const hasCatalog = sourceFiles(join(repositoryRoot, "webui/src/i18n")).some((file) => /Translations\.ts$/.test(file));
|
||||
if (hasCatalog && registration === "absent") findings.push({ file: moduleFile?.fileName ?? join(repositoryRoot, "webui/src/module.ts"), line: 1, code: "catalog-not-registered", message: "Module-owned catalog exists but no static module translations registration was found." });
|
||||
return { repository: repositoryRoot, registration, labels, findings, review };
|
||||
}
|
||||
|
||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
const args = process.argv.slice(2);
|
||||
let workspace = resolve(dirname(fileURLToPath(import.meta.url)), "../../../..");
|
||||
const repos = [];
|
||||
for (let index = 0; index < args.length; index++) {
|
||||
if (args[index] === "--workspace-root") workspace = resolve(args[++index]);
|
||||
else if (args[index] === "--repo") repos.push(args[++index]);
|
||||
else throw new Error(`Unknown argument: ${args[index]}`);
|
||||
}
|
||||
const core = join(workspace, "govoplan-core");
|
||||
const require = createRequire(join(core, "webui/package.json"));
|
||||
const ts = require("typescript");
|
||||
const catalog = JSON.parse(readFileSync(join(workspace, "govoplan/repositories.json"), "utf8"));
|
||||
const selected = catalog.repositories.filter((repo) => !repos.length || repos.includes(repo.name));
|
||||
if (repos.some((name) => !selected.some((repo) => repo.name === name))) throw new Error("Unknown repository selection");
|
||||
for (const repo of selected) {
|
||||
if (typeof repo.path !== "string" || !resolve(workspace, repo.path).startsWith(`${workspace}/`)) throw new Error("Repository path escapes the workspace");
|
||||
}
|
||||
const results = selected.filter((repo) => existsSync(join(workspace, repo.path, "webui/src"))).map((repo) => auditRepository(ts, join(workspace, repo.path), core));
|
||||
const findings = results.reduce((total, item) => total + item.findings.length, 0);
|
||||
process.stdout.write(JSON.stringify({ schema_version: 1, results, finding_count: findings,
|
||||
limitations: ["Known display slots only; this is not a complete UI or linguistic review.", "Runtime data, computed labels and dynamic registrations require manual review.", "Core defaults plus each owning module are checked; optional sibling catalogs cannot mask missing registration.", "Explicit i18n markers are checked by the existing platform interface inventory."] }) + "\n");
|
||||
process.exitCode = findings ? 1 : 0;
|
||||
}
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Command-line entry point; use the repository-root ./devkit launcher."""
|
||||
from govoplan_devkit.cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Executable
+32
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "Example Python project",
|
||||
"repositories": [{"name": "app", "path": "."}],
|
||||
"checks": [
|
||||
{
|
||||
"id": "whitespace",
|
||||
"title": "Git whitespace check",
|
||||
"argv": ["git", "diff", "--check"],
|
||||
"cwd": ".",
|
||||
"repos": ["app"],
|
||||
"inputs": {"repos": ["app"]},
|
||||
"timeout_seconds": 30
|
||||
},
|
||||
{
|
||||
"id": "unit-tests",
|
||||
"title": "Python unit tests",
|
||||
"argv": ["{python}", "-m", "unittest", "discover", "-s", "tests"],
|
||||
"cwd": ".",
|
||||
"repos": ["app"],
|
||||
"inputs": {"repos": ["app"]},
|
||||
"after": ["whitespace"],
|
||||
"resources": ["test-database"],
|
||||
"timeout_seconds": 300
|
||||
}
|
||||
],
|
||||
"profiles": {
|
||||
"quick": ["whitespace", "unit-tests"],
|
||||
"backend": ["unit-tests"],
|
||||
"full": ["whitespace", "unit-tests"]
|
||||
}
|
||||
}
|
||||
Executable
+3
@@ -0,0 +1,3 @@
|
||||
"""Small, deterministic development commands for people, CI and coding agents."""
|
||||
|
||||
__version__ = "1.0.0"
|
||||
Executable
+566
@@ -0,0 +1,566 @@
|
||||
"""Test planning over existing checks; planning never executes a test or server."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from copy import deepcopy
|
||||
from itertools import islice
|
||||
import re
|
||||
|
||||
from .common import read_json
|
||||
from .package_tests import declared_tests, discovered_sources, source_stage_id
|
||||
|
||||
|
||||
PROFILES = ("quick", "ui", "backend", "full")
|
||||
|
||||
|
||||
def stage(
|
||||
identity: str,
|
||||
title: str,
|
||||
argv: list[str],
|
||||
cwd: Path,
|
||||
*,
|
||||
reason: str,
|
||||
deps: list[str] | None = None,
|
||||
after: list[str] | None = None,
|
||||
resources: list[str] | None = None,
|
||||
timeout_seconds: int = 300,
|
||||
) -> dict:
|
||||
return {
|
||||
"id": identity,
|
||||
"title": title,
|
||||
"argv": argv,
|
||||
"cwd": str(cwd),
|
||||
"deps": deps or [],
|
||||
"after": after or [],
|
||||
"resources": resources or [],
|
||||
"timeout_seconds": timeout_seconds,
|
||||
"reason": reason,
|
||||
}
|
||||
|
||||
|
||||
def _custom_stages(
|
||||
project, workspace_root: Path, profile: str, selected, filtered: bool
|
||||
) -> list[dict]:
|
||||
# load_project validates every nested declaration before selection.
|
||||
records = project.config.get("checks", [])
|
||||
checks = {item["id"]: item for item in records}
|
||||
profiles = project.config.get("profiles", {})
|
||||
if profile not in profiles:
|
||||
raise ValueError(
|
||||
f"Project {project.name!r} does not declare profile {profile!r}"
|
||||
)
|
||||
selected_names = {repo.name for repo in selected}
|
||||
wanted = set()
|
||||
|
||||
def include(identity: str, visiting: frozenset[str] = frozenset()) -> None:
|
||||
if identity not in checks:
|
||||
raise ValueError(f"Unknown check dependency: {identity}")
|
||||
if identity in visiting:
|
||||
raise ValueError(f"Cyclic check dependency: {identity}")
|
||||
if identity in wanted:
|
||||
return
|
||||
for dependency in [
|
||||
*checks[identity].get("deps", []),
|
||||
*checks[identity].get("after", []),
|
||||
]:
|
||||
include(dependency, visiting | {identity})
|
||||
wanted.add(identity)
|
||||
|
||||
for identity in profiles[profile]:
|
||||
if identity not in checks:
|
||||
raise ValueError(f"Unknown profile check: {identity}")
|
||||
owned = set(checks[identity].get("repos", []))
|
||||
if not filtered or not owned or selected_names & owned:
|
||||
include(identity)
|
||||
result = []
|
||||
# A dependency may precede/follow its consumer in the configuration: the
|
||||
# execution engine owns scheduling, not JSON declaration order.
|
||||
for identity, item in checks.items():
|
||||
if identity in wanted:
|
||||
result.append(
|
||||
stage(
|
||||
identity,
|
||||
item.get("title", identity),
|
||||
list(item["argv"]),
|
||||
workspace_root / item.get("cwd", "."),
|
||||
deps=list(item.get("deps", [])),
|
||||
after=list(item.get("after", [])),
|
||||
resources=list(item.get("resources", [])),
|
||||
timeout_seconds=item.get("timeout_seconds", 300),
|
||||
reason=f"Project profile {profile}",
|
||||
)
|
||||
)
|
||||
for field in ("inputs", "reuse"):
|
||||
if field in item:
|
||||
result[-1][field] = deepcopy(item[field])
|
||||
return result
|
||||
|
||||
|
||||
def focused_phases(meta: Path) -> list[dict]:
|
||||
"""Read the same bounded, ordered phase metadata as the standalone gate."""
|
||||
value = read_json(meta / "tools/checks/focused-phases.json", max_bytes=1024 * 1024)
|
||||
if (
|
||||
not isinstance(value, dict)
|
||||
or set(value) != {"schema_version", "phases"}
|
||||
or type(value["schema_version"]) is not int
|
||||
or value["schema_version"] != 1
|
||||
):
|
||||
raise ValueError("Unsupported focused phase metadata schema")
|
||||
phases = value["phases"]
|
||||
if not isinstance(phases, list) or not 1 <= len(phases) <= 32:
|
||||
raise ValueError("Focused phase catalog requires 1–32 definitions")
|
||||
seen = set()
|
||||
fields = {
|
||||
"id",
|
||||
"title",
|
||||
"cwd",
|
||||
"order_after",
|
||||
"depends_on",
|
||||
"resources",
|
||||
"outputs",
|
||||
"notes",
|
||||
}
|
||||
for phase in phases:
|
||||
if not isinstance(phase, dict) or set(phase) != fields:
|
||||
raise ValueError("Invalid focused phase metadata fields")
|
||||
identity = phase["id"]
|
||||
if (
|
||||
not isinstance(identity, str)
|
||||
or not re.fullmatch(r"[a-z][a-z0-9-]{0,63}", identity)
|
||||
or identity in seen
|
||||
):
|
||||
raise ValueError("Invalid or duplicate focused phase ID")
|
||||
if (
|
||||
not isinstance(phase["title"], str)
|
||||
or not phase["title"].strip()
|
||||
or len(phase["title"]) > 256
|
||||
or any(ord(char) < 32 for char in phase["title"])
|
||||
):
|
||||
raise ValueError("Invalid focused phase title")
|
||||
if not isinstance(phase["cwd"], str) or phase["cwd"] not in {
|
||||
"core",
|
||||
"meta",
|
||||
"core-webui",
|
||||
"access-webui",
|
||||
}:
|
||||
raise ValueError("Unsupported focused phase working directory")
|
||||
for field in ("order_after", "depends_on", "resources", "outputs", "notes"):
|
||||
values = phase[field]
|
||||
if (
|
||||
not isinstance(values, list)
|
||||
or len(values) > 64
|
||||
or any(
|
||||
not isinstance(item, str)
|
||||
or not item
|
||||
or len(item) > 4096
|
||||
or "\0" in item
|
||||
for item in values
|
||||
)
|
||||
or len(set(values)) != len(values)
|
||||
):
|
||||
raise ValueError("Invalid focused phase list: " + field)
|
||||
if (set(phase["order_after"]) | set(phase["depends_on"])) - seen:
|
||||
raise ValueError("Focused prerequisites must precede their consumer")
|
||||
seen.add(identity)
|
||||
return phases
|
||||
|
||||
|
||||
def focused_phase_bodies(text: str, phases: list[dict]) -> dict[str, str]:
|
||||
"""Only exact top-level registered wrappers are authoritative phase bodies.
|
||||
|
||||
Do not extract lookalike markers from heredocs, conditionals or unrelated
|
||||
shell functions. This recognizes the maintained wrapper convention, not
|
||||
arbitrary executable shell semantics.
|
||||
"""
|
||||
registered = {
|
||||
"focused_phase_" + phase["id"].replace("-", "_"): phase["id"]
|
||||
for phase in phases
|
||||
}
|
||||
lines, bodies, index, depth, heredoc = text.splitlines(), {}, 0, 0, None
|
||||
while index < len(lines):
|
||||
line = lines[index]
|
||||
stripped = line.strip()
|
||||
if heredoc is not None:
|
||||
if stripped == heredoc:
|
||||
heredoc = None
|
||||
index += 1
|
||||
continue
|
||||
match = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
|
||||
if match:
|
||||
heredoc = match[1]
|
||||
index += 1
|
||||
continue
|
||||
function = re.fullmatch(r"(focused_phase_[a-z0-9_]+)\(\) \{", line)
|
||||
if depth == 0 and function and function[1] in registered:
|
||||
identity = registered[function[1]]
|
||||
if (
|
||||
identity in bodies
|
||||
or index + 1 >= len(lines)
|
||||
or lines[index + 1] != f"# devkit-phase: {identity} begin"
|
||||
):
|
||||
raise ValueError("Invalid or duplicate focused phase wrapper")
|
||||
end = index + 2
|
||||
while end < len(lines) and lines[end] != f"# devkit-phase: {identity} end":
|
||||
end += 1
|
||||
if end + 1 >= len(lines) or lines[end + 1] != "}":
|
||||
raise ValueError("Unclosed focused phase wrapper")
|
||||
bodies[identity] = "\n".join(lines[index + 2 : end]) + "\n"
|
||||
index = end + 2
|
||||
continue
|
||||
if re.match(
|
||||
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
|
||||
stripped,
|
||||
):
|
||||
depth += 1
|
||||
elif re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
|
||||
depth = max(0, depth - 1)
|
||||
index += 1
|
||||
if set(bodies) != {phase["id"] for phase in phases}:
|
||||
raise ValueError("Focused phase metadata and marked implementations differ")
|
||||
return {phase["id"]: bodies[phase["id"]] for phase in phases}
|
||||
|
||||
|
||||
def _undeclared_source_note(project, workspace_root: Path) -> str | None:
|
||||
"""Directory discovery is wider than registered Git input ownership."""
|
||||
registered_paths = {repo.path.resolve() for repo in project.repositories}
|
||||
children = list(islice(workspace_root.iterdir(), 4097))
|
||||
if len(children) > 4096:
|
||||
return "Workspace discovery exceeds its bounded ownership audit; native reuse is disabled."
|
||||
for child in children:
|
||||
if not child.name.startswith("govoplan") or child.resolve() in registered_paths:
|
||||
continue
|
||||
if any(
|
||||
(child / name).exists() or (child / name).is_symlink()
|
||||
for name in ("src", "webui")
|
||||
):
|
||||
return "Unregistered sibling src/WebUI inputs may be consumed by workspace discovery or PYTHONPATH; native reuse is disabled until their repository ownership is declared."
|
||||
return None
|
||||
|
||||
|
||||
def _apply_undeclared_source_note(checks: list[dict], note: str | None) -> None:
|
||||
if note:
|
||||
for check in checks:
|
||||
check["reuse"] = "never"
|
||||
check.setdefault("coverage_notes", []).append(note)
|
||||
|
||||
|
||||
def _focused_ui_inputs(project) -> tuple[list[str] | None, str]:
|
||||
names = {repo.name for repo in project.repositories}
|
||||
if not {"govoplan", "govoplan-core"} <= names:
|
||||
return None, "Missing Core/Meta ownership; input scope stays workspace-wide."
|
||||
selected = {"govoplan", "govoplan-core"}
|
||||
for repo in project.repositories:
|
||||
root, webui = repo.path, repo.path / "webui"
|
||||
if (
|
||||
not root.is_dir()
|
||||
or root.is_symlink()
|
||||
or webui.is_symlink()
|
||||
or webui.exists()
|
||||
and not webui.is_dir()
|
||||
):
|
||||
return (
|
||||
None,
|
||||
"Missing or ambiguous repository/WebUI layout; input scope stays workspace-wide.",
|
||||
)
|
||||
if webui.is_dir():
|
||||
selected.add(repo.name)
|
||||
return (
|
||||
sorted(selected),
|
||||
"UI input scope includes whole Core/Meta and every registered WebUI repository, including helpers/configuration; it is not per-file dependency inference.",
|
||||
)
|
||||
|
||||
|
||||
def _apply_ui_inputs(check: dict, scope: tuple[list[str] | None, str]) -> None:
|
||||
names, note = scope
|
||||
if names is not None:
|
||||
check["inputs"] = {"repos": list(names)}
|
||||
check.setdefault("coverage_notes", []).append(note)
|
||||
|
||||
|
||||
def _full_stages(project, workspace_root: Path, meta: Path, core: Path) -> list[dict]:
|
||||
phases = focused_phases(meta)
|
||||
ui_scope = _focused_ui_inputs(project)
|
||||
directories = {
|
||||
"core": core,
|
||||
"meta": meta,
|
||||
"core-webui": core / "webui",
|
||||
"access-webui": workspace_root / "govoplan-access/webui",
|
||||
}
|
||||
checks = []
|
||||
for phase in phases:
|
||||
check = stage(
|
||||
"focused." + phase["id"],
|
||||
phase["title"],
|
||||
[
|
||||
"bash",
|
||||
str(meta / "tools/checks/check-focused.sh"),
|
||||
"--phase",
|
||||
phase["id"],
|
||||
],
|
||||
directories[phase["cwd"]],
|
||||
reason="Full retains every canonical phase in order; repository filters never narrow the required gate.",
|
||||
deps=["focused." + value for value in phase["depends_on"]],
|
||||
after=["focused." + value for value in phase["order_after"]],
|
||||
resources=list(dict.fromkeys(["workspace:focused", *phase["resources"]])),
|
||||
timeout_seconds=14400,
|
||||
)
|
||||
check["coverage_notes"] = list(phase["notes"])
|
||||
check["phase_outputs"] = list(phase["outputs"])
|
||||
if phase["id"] in {"core-ui", "module-builds", "browser", "module-ui"}:
|
||||
check["resources"] = list(
|
||||
dict.fromkeys(
|
||||
[
|
||||
*check["resources"],
|
||||
*[f"webui:{repo.name}" for repo in project.repositories],
|
||||
]
|
||||
)
|
||||
)
|
||||
_apply_ui_inputs(check, ui_scope)
|
||||
else:
|
||||
check["coverage_notes"].append(
|
||||
"Cross-module backend/tooling checks retain conservative whole-workspace inputs."
|
||||
)
|
||||
checks.append(check)
|
||||
_apply_undeclared_source_note(
|
||||
checks, _undeclared_source_note(project, workspace_root)
|
||||
)
|
||||
return checks
|
||||
|
||||
|
||||
def _expanded_repositories(project, selected, *, changed: bool):
|
||||
"""Conservative shared changes; declared interface consumers otherwise.
|
||||
|
||||
This is a selection aid, not a claim of exhaustive runtime dependency
|
||||
analysis. The full profile always retains the canonical workspace gate.
|
||||
"""
|
||||
if not changed or not selected:
|
||||
return selected, "explicit selection" if selected else "no changed repositories"
|
||||
names = {repo.name for repo in selected}
|
||||
if names & {"govoplan", "govoplan-core"}:
|
||||
return list(
|
||||
project.repositories
|
||||
), "Core/Meta changed; all registered consumers conservatively selected"
|
||||
# Reuse the release contract parser without importing module application
|
||||
# code. If available declarations cannot be parsed, broaden selection.
|
||||
import sys
|
||||
|
||||
meta = next(
|
||||
(repo.path for repo in project.repositories if repo.name == "govoplan"), None
|
||||
)
|
||||
if meta is None:
|
||||
return selected, "changed repositories; no GovOPlaN contract catalog"
|
||||
release = meta / "tools" / "release"
|
||||
if not (release / "govoplan_release" / "contracts.py").is_file():
|
||||
return selected, "changed repositories; contract parser unavailable"
|
||||
sys.path.insert(0, str(release))
|
||||
try:
|
||||
from govoplan_release.contracts import parse_manifest_contract
|
||||
|
||||
contracts = []
|
||||
for repo in project.repositories:
|
||||
for manifest in sorted((repo.path / "src").glob("*/backend/manifest.py")):
|
||||
parsed = parse_manifest_contract(manifest, repo_name=repo.name)
|
||||
if parsed is None:
|
||||
return list(
|
||||
project.repositories
|
||||
), "unresolved manifest contract; conservative workspace selection"
|
||||
contracts.append(parsed)
|
||||
while True:
|
||||
providers = {
|
||||
item.name
|
||||
for contract in contracts
|
||||
if contract.repo in names
|
||||
for item in contract.provides_interfaces
|
||||
}
|
||||
consumers = {
|
||||
contract.repo
|
||||
for contract in contracts
|
||||
if any(item.name in providers for item in contract.requires_interfaces)
|
||||
}
|
||||
added = consumers - names
|
||||
if not added:
|
||||
break
|
||||
names.update(added)
|
||||
except (ImportError, AttributeError, OSError, SyntaxError, ValueError):
|
||||
return list(
|
||||
project.repositories
|
||||
), "contract analysis unavailable; conservative workspace selection"
|
||||
finally:
|
||||
sys.path.remove(str(release))
|
||||
return [
|
||||
repo for repo in project.repositories if repo.name in names
|
||||
], "changed repositories plus declared interface consumers"
|
||||
|
||||
|
||||
def _module_ui_plan(repo, *, reason: str) -> tuple[list[dict], list[str]]:
|
||||
"""Use package-owned direct Node test metadata, excluding shell/build chains.
|
||||
|
||||
Source structural scripts are also discoverable by the existing established
|
||||
names. Unknown shell commands are deliberately not guessed or rewritten.
|
||||
"""
|
||||
webui = repo.path / "webui"
|
||||
package_path = webui / "package.json"
|
||||
if not package_path.is_file():
|
||||
return [], []
|
||||
scripts: dict[tuple[str, ...], list[str]] = {}
|
||||
omitted = []
|
||||
for item in [*declared_tests(repo, package_path), *discovered_sources(repo)]:
|
||||
if item["component_suite"] is not None:
|
||||
omitted.append(
|
||||
f"{repo.name} {item['name']}: only covered by the explicit UI component batch; quick does not compile components"
|
||||
)
|
||||
elif item["name"] in {"test:module-permutations", "test:vite-cache-isolation"}:
|
||||
omitted.append(
|
||||
f"{repo.name} {item['name']}: separate environment/permutation verification, not run by this scoped profile"
|
||||
)
|
||||
elif item["_argv"]:
|
||||
scripts[tuple(item["_argv"])] = item["_argv"]
|
||||
else:
|
||||
omitted.append(f"{repo.name} {item['name']}: {item['reason']}")
|
||||
return [
|
||||
stage(
|
||||
source_stage_id(repo.name, argv),
|
||||
f"{repo.name}: {Path(argv[-1]).stem}",
|
||||
argv,
|
||||
webui,
|
||||
reason=reason,
|
||||
resources=[f"webui:{repo.name}"],
|
||||
)
|
||||
for _, argv in sorted(scripts.items())
|
||||
], omitted
|
||||
|
||||
|
||||
def module_ui_stages(repo, *, reason: str) -> list[dict]:
|
||||
return _module_ui_plan(repo, reason=reason)[0]
|
||||
|
||||
|
||||
def build_stages(
|
||||
workspace_root: Path,
|
||||
profile: str,
|
||||
repos: list[str],
|
||||
changed: bool,
|
||||
project: Path | None = None,
|
||||
) -> list[dict]:
|
||||
from .workspace import load_project, selected_repositories
|
||||
|
||||
if profile not in PROFILES:
|
||||
raise ValueError(f"Unknown check profile: {profile}")
|
||||
workspace_root = workspace_root.resolve()
|
||||
loaded = load_project(workspace_root, project)
|
||||
selected = selected_repositories(loaded, repos, changed=changed)
|
||||
if project is not None:
|
||||
return _custom_stages(
|
||||
loaded, workspace_root, profile, selected, bool(repos or changed)
|
||||
)
|
||||
selected, reason = _expanded_repositories(loaded, selected, changed=changed)
|
||||
mapping = {repo.name: repo.path for repo in loaded.repositories}
|
||||
meta = mapping.get("govoplan", workspace_root / "govoplan")
|
||||
core = mapping.get("govoplan-core", workspace_root / "govoplan-core")
|
||||
if profile == "full":
|
||||
return _full_stages(loaded, workspace_root, meta, core)
|
||||
if changed and not selected:
|
||||
return []
|
||||
checks = []
|
||||
for identity, command in (
|
||||
(
|
||||
"contracts",
|
||||
[
|
||||
"{python}",
|
||||
str(meta / "tools/checks/check-contracts.py"),
|
||||
"--workspace-root",
|
||||
str(workspace_root),
|
||||
"--no-impact",
|
||||
],
|
||||
),
|
||||
(
|
||||
"manifests",
|
||||
[
|
||||
"{python}",
|
||||
str(meta / "tools/checks/check-manifest-shapes.py"),
|
||||
"--workspace-root",
|
||||
str(workspace_root),
|
||||
"--require-architecture",
|
||||
],
|
||||
),
|
||||
):
|
||||
checks.append(
|
||||
stage(
|
||||
identity,
|
||||
f"Workspace {identity}",
|
||||
command,
|
||||
meta,
|
||||
reason="Shared manifest/interface invariants",
|
||||
timeout_seconds=600,
|
||||
)
|
||||
)
|
||||
if profile in {"quick", "ui"}:
|
||||
ui_scope = _focused_ui_inputs(loaded)
|
||||
coverage_notes = [
|
||||
"Scoped source/component checks are not a complete module review or the full focused gate."
|
||||
]
|
||||
for identity in ("jsx-value-imports", "heading-help"):
|
||||
checks.append(
|
||||
stage(
|
||||
identity,
|
||||
f"Shared {identity} contract",
|
||||
["{node}", str(meta / f"tools/checks/check-{identity}.mjs")],
|
||||
meta,
|
||||
reason="Existing source-only cross-module guard",
|
||||
)
|
||||
)
|
||||
_apply_ui_inputs(checks[-1], ui_scope)
|
||||
for repo in selected:
|
||||
stages, omissions = _module_ui_plan(repo, reason=reason)
|
||||
checks.extend(stages)
|
||||
coverage_notes.extend(omissions)
|
||||
checks[0]["coverage_notes"] = coverage_notes
|
||||
if profile == "ui":
|
||||
checks.append(
|
||||
stage(
|
||||
"core.component-batch",
|
||||
"Core component suites (compile once)",
|
||||
["{node}", str(core / "webui/scripts/run-component-tests.mjs")],
|
||||
core / "webui",
|
||||
reason="Shared components affect every UI consumer",
|
||||
timeout_seconds=900,
|
||||
)
|
||||
)
|
||||
_apply_ui_inputs(checks[-1], ui_scope)
|
||||
if profile == "backend":
|
||||
for repo in selected:
|
||||
if (repo.path / "tests").is_dir():
|
||||
checks.append(
|
||||
stage(
|
||||
f"{repo.name}.backend",
|
||||
f"{repo.name} backend tests",
|
||||
["{python}", "-m", "pytest", "-q", str(repo.path / "tests")],
|
||||
repo.path,
|
||||
reason=reason,
|
||||
resources=["backend:test-state"],
|
||||
timeout_seconds=1800,
|
||||
)
|
||||
)
|
||||
_apply_undeclared_source_note(
|
||||
checks, _undeclared_source_note(loaded, workspace_root)
|
||||
)
|
||||
return checks
|
||||
|
||||
|
||||
def build_coverage(
|
||||
workspace_root: Path,
|
||||
profile: str,
|
||||
repos: list[str],
|
||||
changed: bool,
|
||||
project: Path | None = None,
|
||||
*,
|
||||
stages: list[dict] | None = None,
|
||||
) -> dict:
|
||||
"""Read-only suite inventory; prebuilt stages avoid repeating selection queries."""
|
||||
from .coverage import coverage_inventory
|
||||
|
||||
if profile not in PROFILES:
|
||||
raise ValueError(f"Unknown check profile: {profile}")
|
||||
if stages is None:
|
||||
stages = build_stages(workspace_root, profile, repos, changed, project)
|
||||
return coverage_inventory(workspace_root, profile, project, stages)
|
||||
Executable
+244
@@ -0,0 +1,244 @@
|
||||
"""Independently verified check results; never an artifact/build-output cache."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
import threading
|
||||
import re
|
||||
|
||||
from .common import digest, now
|
||||
from .inputs import InputSnapshotter, FINGERPRINT_VERSION
|
||||
|
||||
CHECKPOINT_VERSION = 1
|
||||
|
||||
|
||||
def validate_checkpoint_receipt(receipt):
|
||||
"""New checkpoints are explicit; legacy status alone never certifies a phase."""
|
||||
version = receipt.get("fingerprint_version")
|
||||
if version is None:
|
||||
return
|
||||
if version != FINGERPRINT_VERSION:
|
||||
raise ValueError("Unsupported check fingerprint version")
|
||||
for stage in receipt["stages"]:
|
||||
verified = stage.get("checkpoint_verified")
|
||||
if type(verified) is not bool:
|
||||
raise ValueError("Stage checkpoint verification must be boolean")
|
||||
if stage["status"] == "passed" and not verified:
|
||||
raise ValueError(
|
||||
"Passing stage requires an independently verified checkpoint"
|
||||
)
|
||||
if not verified:
|
||||
continue
|
||||
if (
|
||||
stage["status"] != "passed"
|
||||
or stage.get("checkpoint_version") != CHECKPOINT_VERSION
|
||||
):
|
||||
raise ValueError("Invalid verified checkpoint state or version")
|
||||
for field in (
|
||||
"cache_key",
|
||||
"input_fingerprint",
|
||||
"stage_plan_fingerprint",
|
||||
"log_sha256",
|
||||
):
|
||||
if not isinstance(stage.get(field), str) or not re.fullmatch(
|
||||
r"[a-f0-9]{64}", stage[field]
|
||||
):
|
||||
raise ValueError(
|
||||
"Verified checkpoint requires bounded content identities"
|
||||
)
|
||||
if (
|
||||
not isinstance(stage.get("checkpoint_at"), str)
|
||||
or not stage["checkpoint_at"]
|
||||
):
|
||||
raise ValueError("Verified checkpoint requires its recording time")
|
||||
scope = stage.get("input_scope")
|
||||
if (
|
||||
not isinstance(scope, dict)
|
||||
or scope.get("version") != 1
|
||||
or scope.get("kind") not in {"workspace", "repositories"}
|
||||
or type(scope.get("declared")) is not bool
|
||||
):
|
||||
raise ValueError("Verified checkpoint requires a versioned input scope")
|
||||
names = scope.get("repos")
|
||||
if (
|
||||
not isinstance(names, list)
|
||||
or not 1 <= len(names) <= 256
|
||||
or any(not isinstance(name, str) for name in names)
|
||||
or len(set(names)) != len(names)
|
||||
):
|
||||
raise ValueError(
|
||||
"Verified checkpoint requires bounded repository identities"
|
||||
)
|
||||
|
||||
|
||||
class Checkpoints:
|
||||
def __init__(self, project, workspace, plan, environment_probe, cancelled):
|
||||
self.scanner = InputSnapshotter(project, workspace_root=workspace)
|
||||
self.plan = {stage["id"]: stage for stage in plan}
|
||||
self.environment_probe = environment_probe
|
||||
self.cancelled = cancelled
|
||||
self.lock = threading.RLock()
|
||||
self.initial = None
|
||||
self.environment = None
|
||||
|
||||
def source(self, stages=None):
|
||||
with self.lock:
|
||||
return self.scanner.snapshot(
|
||||
list(self.plan.values()) if stages is None else stages
|
||||
)
|
||||
|
||||
def probe_environment(self):
|
||||
value = self.environment_probe()
|
||||
self.check_cancelled()
|
||||
return value
|
||||
|
||||
def check_cancelled(self):
|
||||
if self.cancelled.is_set():
|
||||
raise InterruptedError("Check cancelled during input verification")
|
||||
|
||||
def initialize(self):
|
||||
self.initial = self.source()
|
||||
self.check_cancelled()
|
||||
self.environment = self.probe_environment()
|
||||
return self.initial, self.environment
|
||||
|
||||
def closure(self, stage):
|
||||
selected = {}
|
||||
|
||||
def include(item):
|
||||
if item["id"] in selected:
|
||||
return
|
||||
selected[item["id"]] = item
|
||||
for identity in item["deps"]:
|
||||
include(self.plan[identity])
|
||||
|
||||
include(self.plan[stage["id"]])
|
||||
return [selected[key] for key in sorted(selected)]
|
||||
|
||||
def identity(self, stage):
|
||||
with self.lock:
|
||||
closure = self.closure(stage)
|
||||
snapshot = self.source(closure)
|
||||
self.check_cancelled()
|
||||
environment = self.probe_environment()
|
||||
own = snapshot["stages"][stage["id"]]
|
||||
key = digest(
|
||||
{
|
||||
"checkpoint_version": CHECKPOINT_VERSION,
|
||||
"inputs": {
|
||||
identity: value["fingerprint"]
|
||||
for identity, value in snapshot["stages"].items()
|
||||
},
|
||||
"environment": environment,
|
||||
}
|
||||
)
|
||||
return {
|
||||
"checkpoint_version": CHECKPOINT_VERSION,
|
||||
"cache_key": key,
|
||||
"input_fingerprint": own["fingerprint"],
|
||||
"input_scope": own["scope"],
|
||||
"stage_plan_fingerprint": own["plan_fingerprint"],
|
||||
"stage_environment_fingerprint": environment,
|
||||
"dependency_input_fingerprints": {
|
||||
item["id"]: snapshot["stages"][item["id"]]["fingerprint"]
|
||||
for item in closure
|
||||
if item["id"] != stage["id"]
|
||||
},
|
||||
}
|
||||
|
||||
def prepare(self, stage, prior, allow_reuse, verify_log):
|
||||
before = self.identity(stage)
|
||||
reason = "No previous verified checkpoint"
|
||||
if stage.get("reuse", "verified") == "never":
|
||||
reason = (
|
||||
"This stage explicitly disables reuse (outputs/setup must be recreated)"
|
||||
)
|
||||
elif not allow_reuse:
|
||||
reason = "A data dependency ran again; its consumers must run again"
|
||||
elif (
|
||||
prior
|
||||
and prior.get("status") == "passed"
|
||||
and prior.get("checkpoint_verified") is True
|
||||
and prior.get("checkpoint_version") == CHECKPOINT_VERSION
|
||||
):
|
||||
if prior.get("cache_key") == before["cache_key"]:
|
||||
# Receipt command text is never executed. Only the freshly planned
|
||||
# stage runs; a cached log must independently match its content hash.
|
||||
verify_log(prior)
|
||||
result = {
|
||||
**before,
|
||||
**{
|
||||
key: deepcopy(prior[key])
|
||||
for key in (
|
||||
"status",
|
||||
"exit_code",
|
||||
"duration_seconds",
|
||||
"log_path",
|
||||
"log_sha256",
|
||||
"output_truncated",
|
||||
"omitted_output_bytes",
|
||||
"checkpoint_at",
|
||||
)
|
||||
if key in prior
|
||||
},
|
||||
}
|
||||
result.update(
|
||||
checkpoint_verified=True,
|
||||
reuse_reason="Verified checkpoint matches current inputs, command, dependencies and environment",
|
||||
)
|
||||
return before, result
|
||||
reason = (
|
||||
"Declared inputs, command, dependency inputs or environment changed"
|
||||
)
|
||||
before["reuse_reason"] = reason
|
||||
return before, None
|
||||
|
||||
def finish(self, stage, before, result):
|
||||
result = {**result, **before, "checkpoint_verified": False}
|
||||
if result["status"] != "passed" or result.get("exit_code") != 0:
|
||||
return result
|
||||
after = self.identity(stage)
|
||||
if before["cache_key"] != after["cache_key"]:
|
||||
result.update(
|
||||
status="stale",
|
||||
error="Stage inputs or environment changed during execution; no reusable checkpoint was recorded",
|
||||
)
|
||||
else:
|
||||
result.update(checkpoint_verified=True, checkpoint_at=now())
|
||||
return result
|
||||
|
||||
def finalize(self, stages):
|
||||
final = self.source()
|
||||
self.check_cancelled()
|
||||
environment = self.probe_environment()
|
||||
valid = (
|
||||
final["observed_source_fingerprint"]
|
||||
== self.initial["observed_source_fingerprint"]
|
||||
and environment == self.environment
|
||||
)
|
||||
for stage in stages:
|
||||
if stage["status"] != "passed":
|
||||
valid = valid and stage["status"] != "stale"
|
||||
continue
|
||||
closure = self.closure(self.plan[stage["id"]])
|
||||
key = digest(
|
||||
{
|
||||
"checkpoint_version": CHECKPOINT_VERSION,
|
||||
"inputs": {
|
||||
item["id"]: final["stages"][item["id"]]["fingerprint"]
|
||||
for item in closure
|
||||
},
|
||||
"environment": environment,
|
||||
}
|
||||
)
|
||||
if (
|
||||
stage.get("checkpoint_verified") is not True
|
||||
or stage.get("cache_key") != key
|
||||
):
|
||||
stage.update(
|
||||
status="stale",
|
||||
checkpoint_verified=False,
|
||||
error="Final inputs no longer match this checkpoint",
|
||||
)
|
||||
valid = False
|
||||
return valid, final, environment
|
||||
Executable
+212
@@ -0,0 +1,212 @@
|
||||
"""A compact command catalog over maintained project tools."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import importlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
from . import __version__
|
||||
from .common import META_ROOT, redact, safe_output
|
||||
|
||||
COMMAND_MODULES = (
|
||||
"context",
|
||||
"doctor",
|
||||
"runner",
|
||||
"review",
|
||||
"docs",
|
||||
"issues",
|
||||
"release",
|
||||
"maintenance",
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
argv = sys.argv[1:] if argv is None else argv
|
||||
common = argparse.ArgumentParser(add_help=False, allow_abbrev=False)
|
||||
common.add_argument(
|
||||
"--workspace-root",
|
||||
type=Path,
|
||||
default=META_ROOT.parent,
|
||||
help="Directory containing registered repositories",
|
||||
)
|
||||
common.add_argument(
|
||||
"--project", type=Path, help="Explicit trusted portable-project JSON manifest"
|
||||
)
|
||||
common.add_argument(
|
||||
"--state-dir",
|
||||
type=Path,
|
||||
help="Private evidence-state base (scoped again by workspace)",
|
||||
)
|
||||
common.add_argument("--format", choices=("summary", "json"), default="summary")
|
||||
common.add_argument(
|
||||
"--quiet",
|
||||
action="store_true",
|
||||
help="Suppress live progress on stderr; retain the final result",
|
||||
)
|
||||
common.add_argument(
|
||||
"--json",
|
||||
dest="format",
|
||||
action="store_const",
|
||||
const="json",
|
||||
help="Alias for --format json",
|
||||
)
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="devkit",
|
||||
description="Deterministic development workflows. Read-only previews by default for remote writes.",
|
||||
parents=[common],
|
||||
allow_abbrev=False,
|
||||
)
|
||||
parser.add_argument("--version", action="version", version=f"devkit {__version__}")
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
for name in COMMAND_MODULES:
|
||||
importlib.import_module("." + name, __package__).register(subparsers)
|
||||
catalog = subparsers.add_parser("commands", help="Show the compact command catalog")
|
||||
|
||||
def commands(_):
|
||||
entries = [
|
||||
{
|
||||
"command": "context",
|
||||
"purpose": "Offline repository changes, ownership and instructions",
|
||||
"effects": "read only",
|
||||
},
|
||||
{
|
||||
"command": "doctor",
|
||||
"purpose": "Local tool/dependency preflight and repair guidance",
|
||||
"effects": "read only",
|
||||
},
|
||||
{
|
||||
"command": "check",
|
||||
"purpose": "Registered verification profiles with logs and source-bound receipts",
|
||||
"effects": "tests/builds; --dry-run previews",
|
||||
},
|
||||
{
|
||||
"command": "runs / latest / status / summary / logs",
|
||||
"purpose": "Find runs and read progress, compact results and bounded live/final logs",
|
||||
"effects": "read only",
|
||||
},
|
||||
{
|
||||
"command": "coverage",
|
||||
"purpose": "Explain declared suite coverage and exclusions for a check profile",
|
||||
"effects": "read only",
|
||||
},
|
||||
{
|
||||
"command": "resume / recover",
|
||||
"purpose": "Resume verified identical work or recover an abandoned check run",
|
||||
"effects": "local checks/state only",
|
||||
},
|
||||
{
|
||||
"command": "review",
|
||||
"purpose": "Module UI-review inventory and manual evidence checklist",
|
||||
"effects": "local bundle only",
|
||||
},
|
||||
{
|
||||
"command": "docs",
|
||||
"purpose": "Existing documentation and translation audits",
|
||||
"effects": "local checks/evidence",
|
||||
},
|
||||
{
|
||||
"command": "issues",
|
||||
"purpose": "Preview and explicitly publish deduplicated issue evidence",
|
||||
"effects": "remote only with --apply",
|
||||
},
|
||||
{
|
||||
"command": "release",
|
||||
"purpose": "Existing durable release lifecycle, receipts and confirmations",
|
||||
"effects": "explicit --apply and step confirmation",
|
||||
},
|
||||
{
|
||||
"command": "git",
|
||||
"purpose": "Frozen explicit-path commit and branch-push maintenance",
|
||||
"effects": "explicit --apply; no bulk staging, force or tags",
|
||||
},
|
||||
]
|
||||
return {
|
||||
"commands": entries,
|
||||
"summary": [
|
||||
f"{item['command']}: {item['purpose']} ({item['effects']})"
|
||||
for item in entries
|
||||
],
|
||||
}
|
||||
|
||||
catalog.set_defaults(handler=commands)
|
||||
# Global flags work before or after the command, without copying defaults to every parser.
|
||||
global_args, remaining = common.parse_known_args(argv)
|
||||
args = parser.parse_args(remaining, namespace=global_args)
|
||||
args.workspace_root = args.workspace_root.expanduser().resolve()
|
||||
if args.project:
|
||||
args.project = args.project.expanduser().absolute()
|
||||
|
||||
def progress(event):
|
||||
if args.quiet:
|
||||
return
|
||||
if args.format == "json":
|
||||
print(
|
||||
json.dumps(safe_output(event), sort_keys=True, allow_nan=False),
|
||||
file=sys.stderr,
|
||||
flush=True,
|
||||
)
|
||||
else:
|
||||
counts = event["counts"]
|
||||
completed = sum(
|
||||
count
|
||||
for state, count in counts.items()
|
||||
if state not in {"pending", "running"}
|
||||
)
|
||||
active = ", ".join(event["active_stages"])
|
||||
print(
|
||||
redact(
|
||||
f"Run {event['run_id']}: {event['phase']} · {completed}/{event['total_stages']} stages · {event['elapsed_seconds']}s"
|
||||
+ (f" · {active}" if active else "")
|
||||
),
|
||||
file=sys.stderr,
|
||||
flush=True,
|
||||
)
|
||||
|
||||
args.on_progress = progress
|
||||
try:
|
||||
result = args.handler(args)
|
||||
if not isinstance(result, dict):
|
||||
raise ValueError("Command did not return a result object")
|
||||
code = int(result.pop("_exit_code", 0))
|
||||
if args.format == "json":
|
||||
print(
|
||||
json.dumps(
|
||||
safe_output(result),
|
||||
sort_keys=True,
|
||||
indent=2,
|
||||
ensure_ascii=True,
|
||||
allow_nan=False,
|
||||
)
|
||||
)
|
||||
else:
|
||||
summary = result.get("summary", [str(result.get("status", "Completed"))])
|
||||
print(
|
||||
"\n".join(redact(str(line)) for line in summary)
|
||||
if isinstance(summary, list)
|
||||
else redact(str(summary))
|
||||
)
|
||||
return code
|
||||
except (ValueError, OSError, RuntimeError, ImportError) as exc:
|
||||
message = redact(str(exc))
|
||||
if args.format == "json":
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"status": "error",
|
||||
"error": message,
|
||||
"error_type": type(exc).__name__,
|
||||
}
|
||||
)
|
||||
)
|
||||
else:
|
||||
print(f"devkit: {message}", file=sys.stderr)
|
||||
return 2
|
||||
except KeyboardInterrupt:
|
||||
print(
|
||||
"devkit: interrupted; inspect the saved run before retrying",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 130
|
||||
Executable
+232
@@ -0,0 +1,232 @@
|
||||
"""Bounded local records and predictable output; no network or AI dependency."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime, timezone
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[3]
|
||||
MAX_JSON_BYTES = 8 * 1024 * 1024
|
||||
IDENTIFIER = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}\Z")
|
||||
|
||||
|
||||
def now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def canonical(value: object) -> bytes:
|
||||
return json.dumps(
|
||||
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True, allow_nan=False
|
||||
).encode()
|
||||
|
||||
|
||||
def digest(value: object) -> str:
|
||||
return hashlib.sha256(canonical(value)).hexdigest()
|
||||
|
||||
|
||||
def identifier(value: str) -> str:
|
||||
if (
|
||||
not isinstance(value, str)
|
||||
or not IDENTIFIER.fullmatch(value)
|
||||
or value in {".", ".."}
|
||||
):
|
||||
raise ValueError("Invalid record identifier")
|
||||
return value
|
||||
|
||||
|
||||
def state_root(workspace_root: Path, state_dir: Path | None = None) -> Path:
|
||||
base = (
|
||||
state_dir
|
||||
or Path(os.environ.get("XDG_STATE_HOME", str(Path.home() / ".local/state")))
|
||||
/ "govoplan/devkit"
|
||||
)
|
||||
# Preserve spelling until symlink validation; resolving here would hide an unsafe alias.
|
||||
base = Path(os.path.abspath(os.fspath(base.expanduser())))
|
||||
return base / ("workspace-" + digest(str(workspace_root.resolve()))[:24])
|
||||
|
||||
|
||||
def reject_symlinks(path: Path) -> None:
|
||||
for item in (path, *path.parents):
|
||||
if item.is_symlink():
|
||||
raise ValueError("State and evidence paths must not contain symlinks")
|
||||
|
||||
|
||||
def private_directory(path: Path) -> None:
|
||||
reject_symlinks(path)
|
||||
path.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
reject_symlinks(path)
|
||||
metadata = path.stat()
|
||||
if not stat.S_ISDIR(metadata.st_mode) or (
|
||||
hasattr(os, "getuid") and metadata.st_uid != os.getuid()
|
||||
):
|
||||
raise ValueError("State directory must be owned by the current user")
|
||||
path.chmod(0o700)
|
||||
|
||||
|
||||
def read_bounded_bytes(path: Path, max_bytes: int = MAX_JSON_BYTES) -> bytes:
|
||||
reject_symlinks(path)
|
||||
descriptor = os.open(
|
||||
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
metadata = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > max_bytes:
|
||||
raise ValueError("Evidence must be a bounded regular file")
|
||||
encoded = handle.read(max_bytes + 1)
|
||||
if len(encoded) > max_bytes:
|
||||
raise ValueError("Evidence file exceeds its size bound")
|
||||
return encoded
|
||||
|
||||
|
||||
def read_json(path: Path, max_bytes: int = MAX_JSON_BYTES) -> object:
|
||||
encoded = read_bounded_bytes(path, max_bytes)
|
||||
|
||||
def unique(pairs):
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
if key in result:
|
||||
raise ValueError("Duplicate JSON keys are not accepted")
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
try:
|
||||
return json.loads(
|
||||
encoded,
|
||||
object_pairs_hook=unique,
|
||||
parse_constant=lambda _: (_ for _ in ()).throw(
|
||||
ValueError("Non-finite JSON number")
|
||||
),
|
||||
)
|
||||
except (RecursionError, UnicodeError) as exc:
|
||||
raise ValueError("JSON nesting or encoding is unsupported") from exc
|
||||
|
||||
|
||||
def atomic_text(path: Path, value: str, max_bytes: int = MAX_JSON_BYTES) -> None:
|
||||
encoded = value.encode("utf-8")
|
||||
if len(encoded) > max_bytes:
|
||||
raise ValueError("Output exceeds its size bound")
|
||||
reject_symlinks(path.parent)
|
||||
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
reject_symlinks(path.parent)
|
||||
reject_symlinks(path)
|
||||
if path.exists() and not path.is_file():
|
||||
raise ValueError("Output target must be a regular file")
|
||||
descriptor, temporary = tempfile.mkstemp(prefix=".devkit-", dir=path.parent)
|
||||
try:
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
os.fchmod(handle.fileno(), 0o600)
|
||||
handle.write(encoded)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(temporary, path)
|
||||
directory = os.open(path.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
|
||||
try:
|
||||
os.fsync(directory)
|
||||
finally:
|
||||
os.close(directory)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
|
||||
def atomic_json(path: Path, payload: object) -> None:
|
||||
atomic_text(
|
||||
path,
|
||||
json.dumps(
|
||||
payload, indent=2, sort_keys=True, ensure_ascii=True, allow_nan=False
|
||||
)
|
||||
+ "\n",
|
||||
)
|
||||
|
||||
|
||||
def redact(text: str) -> str:
|
||||
"""Best-effort display hygiene, not permission to include secrets in commands."""
|
||||
for key, value in os.environ.items():
|
||||
if (
|
||||
re.search(r"TOKEN|SECRET|PASSWORD|API_KEY|PRIVATE_KEY", key, re.I)
|
||||
and len(value) >= 4
|
||||
):
|
||||
text = text.replace(value, "[redacted]")
|
||||
text = re.sub(r"(?im)(authorization\s*[:=]\s*)([^\r\n]+)", r"\1[redacted]", text)
|
||||
text = re.sub(r"(?i)(https?://)[^/\s:@]+:[^/\s@]+@", r"\1[redacted]@", text)
|
||||
text = re.sub(
|
||||
r"(?i)((?:token|password|secret|api[_-]?key)\s*[=:]\s*)[^\s,;]+",
|
||||
r"\1[redacted]",
|
||||
text,
|
||||
)
|
||||
return text
|
||||
|
||||
|
||||
def redact_argv(argv: list[str]) -> list[str]:
|
||||
result, hide_next = [], False
|
||||
for argument in argv:
|
||||
if hide_next:
|
||||
result.append("[redacted]")
|
||||
hide_next = False
|
||||
continue
|
||||
if re.fullmatch(
|
||||
r"--?(?:password|passwd|token|secret|api[-_]key|access[-_]token|authorization)",
|
||||
argument,
|
||||
re.I,
|
||||
):
|
||||
hide_next = True
|
||||
result.append(redact(argument))
|
||||
return result
|
||||
|
||||
|
||||
def safe_output(value):
|
||||
"""Redact presentation, not immutable identity hashes or execution inputs."""
|
||||
if isinstance(value, dict):
|
||||
return {
|
||||
key: redact_argv(item)
|
||||
if key == "argv"
|
||||
and isinstance(item, list)
|
||||
and all(isinstance(arg, str) for arg in item)
|
||||
else "[redacted]"
|
||||
if re.fullmatch(
|
||||
r"password|passwd|token|secret|api[_-]?key|authorization",
|
||||
str(key),
|
||||
re.I,
|
||||
)
|
||||
else safe_output(item)
|
||||
for key, item in value.items()
|
||||
}
|
||||
if isinstance(value, list):
|
||||
return [safe_output(item) for item in value]
|
||||
return redact(value) if isinstance(value, str) else value
|
||||
|
||||
|
||||
@contextmanager
|
||||
def resource_lock(directory: Path, name: str, timeout: float = 0):
|
||||
"""Host-local advisory lock, released by the OS even after a process crash."""
|
||||
import fcntl
|
||||
|
||||
private_directory(directory)
|
||||
path = directory / (hashlib.sha256(name.encode()).hexdigest() + ".lock")
|
||||
reject_symlinks(path)
|
||||
descriptor = os.open(
|
||||
path, os.O_CREAT | os.O_RDWR | getattr(os, "O_NOFOLLOW", 0), 0o600
|
||||
)
|
||||
try:
|
||||
if not stat.S_ISREG(os.fstat(descriptor).st_mode):
|
||||
raise ValueError("Lock target must be a regular file")
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
try:
|
||||
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
break
|
||||
except BlockingIOError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise RuntimeError(f"Resource is busy: {name}") from None
|
||||
time.sleep(min(0.1, max(0, deadline - time.monotonic())))
|
||||
yield
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
Executable
+90
@@ -0,0 +1,90 @@
|
||||
"""Small read-only context bundles; no automatic source/credential dumping."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
|
||||
from .common import META_ROOT, now, read_json
|
||||
from .workspace import inspect_repository, load_project, selected_repositories
|
||||
|
||||
|
||||
def build_context(
|
||||
workspace_root: Path, project_file: Path | None, names: list[str], changed: bool
|
||||
) -> dict:
|
||||
project = load_project(workspace_root, project_file)
|
||||
repos = selected_repositories(project, names)
|
||||
with ThreadPoolExecutor(max_workers=8) as pool:
|
||||
states = list(pool.map(inspect_repository, repos))
|
||||
if changed:
|
||||
states = [
|
||||
state
|
||||
for state in states
|
||||
if state["errors"]
|
||||
or state["dirty_entries"]
|
||||
or state["ahead"]
|
||||
or (state["head"] and not state["upstream"])
|
||||
]
|
||||
inventory = {}
|
||||
if not project_file:
|
||||
path = META_ROOT / "docs/project/ui-review-issue-inventory.json"
|
||||
if path.is_file():
|
||||
payload = read_json(path)
|
||||
for item in payload.get("issues", []) if isinstance(payload, dict) else []:
|
||||
if isinstance(item, dict):
|
||||
inventory[item.get("repository")] = item
|
||||
for state in states:
|
||||
root = Path(state["path"])
|
||||
state["instructions"] = [
|
||||
str(path) for path in (root / "AGENTS.md",) if path.is_file()
|
||||
]
|
||||
state["documentation"] = [
|
||||
str(path)
|
||||
for path in (
|
||||
root / "README.md",
|
||||
root / "docs/README.md",
|
||||
root / "docs/MODULE_ARCHITECTURE.md",
|
||||
)
|
||||
if path.is_file()
|
||||
]
|
||||
state["change_entry_count"] = len(state["dirty_entries"])
|
||||
state["review_issue"] = inventory.get(state["name"], {}).get("url")
|
||||
state["suggested_check"] = (
|
||||
f"./devkit check --repo {state['name']} --profile quick --dry-run"
|
||||
)
|
||||
summary = [
|
||||
f"{project.name}: {len(states)} repositories selected (offline; upstream counts may be stale)."
|
||||
]
|
||||
for state in states:
|
||||
errors = f" ERROR: {'; '.join(state['errors'])}" if state["errors"] else ""
|
||||
summary.append(
|
||||
f"{state['name']}: {state['branch'] or '(detached/unborn)'}; {state['change_entry_count']} change entries; ahead={state['ahead']} behind={state['behind']}{errors}"
|
||||
)
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"generated_at": now(),
|
||||
"workspace_root": str(workspace_root),
|
||||
"project": project.name,
|
||||
"remote_checked": False,
|
||||
"repositories": states,
|
||||
"summary": summary,
|
||||
"_exit_code": 1 if any(state["errors"] for state in states) else 0,
|
||||
}
|
||||
|
||||
|
||||
def register(subparsers):
|
||||
parser = subparsers.add_parser(
|
||||
"context",
|
||||
help="Offline repository changes, ownership and relevant instruction paths",
|
||||
)
|
||||
parser.add_argument("--repo", action="append", default=[])
|
||||
parser.add_argument(
|
||||
"--changed",
|
||||
action="store_true",
|
||||
help="Show dirty or locally ahead repositories, retaining inspection errors",
|
||||
)
|
||||
parser.set_defaults(
|
||||
handler=lambda args: build_context(
|
||||
args.workspace_root, args.project, args.repo, args.changed
|
||||
)
|
||||
)
|
||||
Executable
+453
@@ -0,0 +1,453 @@
|
||||
"""Explicit suite-plan coverage; never execute or guess nested shell/npm flows."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shlex
|
||||
import stat
|
||||
|
||||
from .common import digest, redact_argv, reject_symlinks
|
||||
from .package_tests import CORE_COMPONENT_SUITES, declared_tests, discovered_sources
|
||||
from .workspace import load_project
|
||||
|
||||
DISPOSITIONS = ("planned", "covered_elsewhere", "excluded", "unsupported")
|
||||
MAX_SUITES = 4096
|
||||
|
||||
|
||||
def _read_canonical(path: Path) -> str:
|
||||
reject_symlinks(path)
|
||||
descriptor = os.open(
|
||||
path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0)
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
metadata = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 1024 * 1024:
|
||||
raise ValueError("Canonical focused gate must be a bounded regular file")
|
||||
encoded = handle.read(1024 * 1024 + 1)
|
||||
if len(encoded) > 1024 * 1024:
|
||||
raise ValueError("Canonical focused gate exceeds its size bound")
|
||||
return encoded.decode("utf-8")
|
||||
|
||||
|
||||
def _node_argv(argv: list[str], cwd: Path) -> tuple[str, ...] | None:
|
||||
if (
|
||||
not argv
|
||||
or argv[0] not in {"node", "{node}", "$NODE"}
|
||||
and Path(argv[0]).name not in {"node", "nodejs"}
|
||||
):
|
||||
return None
|
||||
offset = 2 if len(argv) > 1 and argv[1] == "--test" else 1
|
||||
if len(argv) != offset + 1:
|
||||
return None
|
||||
target = Path(argv[offset])
|
||||
if "$" in str(target):
|
||||
return None
|
||||
return ("node", *argv[1:offset], str((cwd / target).resolve()))
|
||||
|
||||
|
||||
def canonical_invocations(workspace_root: Path, meta: Path, core: Path) -> dict:
|
||||
"""Recognize direct commands in exact registered phase wrappers or legacy top level.
|
||||
|
||||
Shell is not evaluated. Bodies, conditionals, loops, functions, npm hooks and
|
||||
recursive commands do not grant coverage. Unknown cwd invalidates matches.
|
||||
The registered marked wrappers are the only function bodies admitted.
|
||||
"""
|
||||
path = meta / "tools/checks/check-focused.sh"
|
||||
try:
|
||||
text = _read_canonical(path)
|
||||
except (OSError, ValueError):
|
||||
return {
|
||||
"path": str(path),
|
||||
"sha256": None,
|
||||
"npm": [],
|
||||
"node": [],
|
||||
"node_phases": [],
|
||||
"notes": [
|
||||
"Canonical focused script is unavailable/unreadable; no package coverage inferred."
|
||||
],
|
||||
}
|
||||
values = {
|
||||
"WORKSPACE_ROOT": str(workspace_root),
|
||||
"META_ROOT": str(meta),
|
||||
"ROOT": str(core),
|
||||
}
|
||||
|
||||
def expand(value: str) -> str | None:
|
||||
value = re.sub(
|
||||
r"\$\{(WORKSPACE_ROOT|META_ROOT|ROOT)\}|\$(WORKSPACE_ROOT|META_ROOT|ROOT)(?![A-Za-z0-9_])",
|
||||
lambda match: values[match[1] or match[2]],
|
||||
value,
|
||||
)
|
||||
return None if "$" in value or "`" in value else value
|
||||
|
||||
result = {
|
||||
"path": str(path),
|
||||
"sha256": hashlib.sha256(text.encode()).hexdigest(),
|
||||
"npm": [],
|
||||
"node": [],
|
||||
"node_phases": [],
|
||||
"notes": [],
|
||||
}
|
||||
metadata = meta / "tools/checks/focused-phases.json"
|
||||
blocks = {None: text}
|
||||
if metadata.exists() or metadata.is_symlink() or "# devkit-phase:" in text:
|
||||
from .catalog import focused_phases, focused_phase_bodies
|
||||
|
||||
try:
|
||||
phases = focused_phases(meta)
|
||||
blocks = focused_phase_bodies(text, phases)
|
||||
result["phase_ids"] = [phase["id"] for phase in phases]
|
||||
result["metadata_sha256"] = digest(phases)
|
||||
except (OSError, ValueError):
|
||||
result["notes"].append(
|
||||
"Focused phase metadata/marked bodies are invalid or unavailable; no phase coverage inferred."
|
||||
)
|
||||
return result
|
||||
cwd: Path | None = meta
|
||||
body_end, nesting = None, 0
|
||||
current_phase = object()
|
||||
for phase_id, line in (
|
||||
(phase_id, line)
|
||||
for phase_id, block in blocks.items()
|
||||
for line in block.replace("\\\n", " ").splitlines()
|
||||
):
|
||||
if current_phase != phase_id:
|
||||
cwd, body_end, nesting = meta, None, 0
|
||||
current_phase = phase_id
|
||||
stripped = line.strip()
|
||||
if body_end is not None:
|
||||
if stripped == body_end:
|
||||
body_end = None
|
||||
continue
|
||||
heredoc = re.search(r"<<-?\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", line)
|
||||
if heredoc:
|
||||
body_end = heredoc[1]
|
||||
continue
|
||||
if not stripped or stripped.startswith("#"):
|
||||
continue
|
||||
if re.match(
|
||||
r"(?:if|for|while|until|case|select)\b|(?:function\s+\w+|\w+\s*\(\s*\))",
|
||||
stripped,
|
||||
):
|
||||
nesting += 1
|
||||
continue
|
||||
if re.match(r"(?:fi|done|esac)\b|^}\s*;?$", stripped):
|
||||
nesting = max(0, nesting - 1)
|
||||
continue
|
||||
if nesting:
|
||||
if re.search(r"\bcd\s", stripped):
|
||||
cwd = None
|
||||
continue
|
||||
try:
|
||||
parts = shlex.split(line, comments=True)
|
||||
except ValueError:
|
||||
result["notes"].append(
|
||||
"An unparseable shell line grants no suite coverage."
|
||||
)
|
||||
continue
|
||||
if not parts:
|
||||
continue
|
||||
if parts[0] == "cd":
|
||||
destination = expand(parts[1]) if len(parts) == 2 else None
|
||||
candidate = (
|
||||
(cwd / destination).resolve()
|
||||
if destination and cwd
|
||||
else Path(destination).resolve()
|
||||
if destination and Path(destination).is_absolute()
|
||||
else None
|
||||
)
|
||||
cwd = (
|
||||
candidate
|
||||
if candidate and candidate.is_relative_to(workspace_root)
|
||||
else None
|
||||
)
|
||||
continue
|
||||
if cwd is None:
|
||||
continue
|
||||
if (
|
||||
parts[0] in {"$NPM", "${NPM}", "npm"}
|
||||
and len(parts) >= 3
|
||||
and parts[1] == "run"
|
||||
and re.fullmatch(r"test(?::[A-Za-z0-9_.:-]+)?", parts[2])
|
||||
):
|
||||
arguments = parts[3:]
|
||||
if arguments and (
|
||||
arguments[0] != "--"
|
||||
or any(
|
||||
not re.fullmatch(r"[A-Za-z0-9_.-]+", item) for item in arguments[1:]
|
||||
)
|
||||
):
|
||||
continue
|
||||
result["npm"].append(
|
||||
{
|
||||
"cwd": str(cwd),
|
||||
"name": parts[2],
|
||||
"args": arguments[1:] if arguments else [],
|
||||
"phase": phase_id,
|
||||
}
|
||||
)
|
||||
elif parts[0] in {"$NODE", "${NODE}", "node"}:
|
||||
expanded = [expand(part) for part in parts[1:]]
|
||||
if all(part is not None for part in expanded):
|
||||
command = _node_argv(["node", *expanded], cwd)
|
||||
if command:
|
||||
result["node"].append(command)
|
||||
result["node_phases"].append({"argv": command, "phase": phase_id})
|
||||
result["notes"] = list(dict.fromkeys(result["notes"]))[:8]
|
||||
return result
|
||||
|
||||
|
||||
def coverage_inventory(
|
||||
workspace_root: Path, profile: str, project_file: Path | None, stages: list[dict]
|
||||
) -> dict:
|
||||
workspace_root = workspace_root.resolve()
|
||||
project = load_project(workspace_root, project_file)
|
||||
stage_map = {item["id"]: item for item in stages}
|
||||
direct = {}
|
||||
explicit_npm = {}
|
||||
for stage in stages:
|
||||
argv, cwd = stage["argv"], Path(stage["cwd"])
|
||||
normalized = _node_argv(argv, cwd)
|
||||
if normalized:
|
||||
direct[normalized] = stage["id"]
|
||||
if (
|
||||
len(argv) == 3
|
||||
and (argv[0] in {"npm", "{npm}"} or Path(argv[0]).name == "npm")
|
||||
and argv[1] == "run"
|
||||
):
|
||||
explicit_npm[(str(cwd.resolve()), argv[2])] = stage["id"]
|
||||
paths = {repo.name: repo.path for repo in project.repositories}
|
||||
canonical = (
|
||||
canonical_invocations(
|
||||
workspace_root,
|
||||
paths.get("govoplan", workspace_root / "govoplan"),
|
||||
paths.get("govoplan-core", workspace_root / "govoplan-core"),
|
||||
)
|
||||
if project_file is None
|
||||
else None
|
||||
)
|
||||
canonical_stages = {}
|
||||
if canonical is not None:
|
||||
legacy = stage_map.get("focused-workspace")
|
||||
if legacy and legacy["argv"] == ["bash", canonical["path"]]:
|
||||
canonical_stages[None] = "focused-workspace"
|
||||
for identity in canonical.get("phase_ids", []):
|
||||
check = stage_map.get("focused." + identity)
|
||||
if check and check["argv"] == [
|
||||
"bash",
|
||||
canonical["path"],
|
||||
"--phase",
|
||||
identity,
|
||||
]:
|
||||
canonical_stages[identity] = check["id"]
|
||||
full = bool(canonical_stages)
|
||||
invocations = (
|
||||
[
|
||||
{**item, "covering_stage": canonical_stages[item.get("phase")]}
|
||||
for item in canonical["npm"]
|
||||
if item.get("phase") in canonical_stages
|
||||
]
|
||||
if canonical
|
||||
else []
|
||||
)
|
||||
canonical_nodes = (
|
||||
{
|
||||
tuple(item["argv"]): canonical_stages[item.get("phase")]
|
||||
for item in canonical.get("node_phases", [])
|
||||
if item.get("phase") in canonical_stages
|
||||
}
|
||||
if canonical
|
||||
else {}
|
||||
)
|
||||
suites, notes = [], []
|
||||
if canonical:
|
||||
notes.extend(canonical["notes"])
|
||||
core_package = (
|
||||
paths.get("govoplan-core", workspace_root / "govoplan-core")
|
||||
/ "webui/package.json"
|
||||
)
|
||||
core_requests = [
|
||||
item
|
||||
for item in invocations
|
||||
if item["cwd"] == str(core_package.parent) and item["name"] == "test:components"
|
||||
]
|
||||
requested_components = set()
|
||||
for item in core_requests:
|
||||
requested_components.update(
|
||||
CORE_COMPONENT_SUITES
|
||||
if not item["args"] or item["args"] == ["all"]
|
||||
else item["args"]
|
||||
)
|
||||
requested_components.intersection_update(CORE_COMPONENT_SUITES)
|
||||
|
||||
for repo in project.repositories:
|
||||
for location in ("package.json", "webui/package.json"):
|
||||
package = repo.path / location
|
||||
if not package.exists() and not package.is_symlink():
|
||||
continue
|
||||
entries = declared_tests(repo, package)
|
||||
if location == "webui/package.json":
|
||||
declared_argv = {
|
||||
tuple(item["_argv"]) for item in entries if item["_argv"]
|
||||
}
|
||||
entries.extend(
|
||||
item
|
||||
for item in discovered_sources(repo)
|
||||
if not item["_argv"] or tuple(item["_argv"]) not in declared_argv
|
||||
)
|
||||
for item in entries:
|
||||
private_argv = item.pop("_argv")
|
||||
component = item["component_suite"]
|
||||
identity = (str(package.parent), item["name"])
|
||||
normalized = (
|
||||
_node_argv(private_argv, package.parent) if private_argv else None
|
||||
)
|
||||
matching = [
|
||||
value
|
||||
for value in invocations
|
||||
if (value["cwd"], value["name"]) == identity
|
||||
]
|
||||
item.update(disposition="excluded", covering_stage=None)
|
||||
if component is not None:
|
||||
all_or_selected = (
|
||||
component == "all" or component in requested_components
|
||||
)
|
||||
if "core.component-batch" in stage_map:
|
||||
item.update(
|
||||
disposition="planned"
|
||||
if component == "all"
|
||||
else "covered_elsewhere",
|
||||
covering_stage="core.component-batch",
|
||||
reason="UI explicitly runs the shared component batch; quick never compiles it.",
|
||||
)
|
||||
elif full and core_requests and all_or_selected:
|
||||
item.update(
|
||||
disposition="planned"
|
||||
if component == "all"
|
||||
else "covered_elsewhere",
|
||||
covering_stage=core_requests[0]["covering_stage"],
|
||||
reason=f"Canonical focused gate explicitly selects {len(requested_components)}/{len(CORE_COMPONENT_SUITES)} component suites; this is not the complete component batch.",
|
||||
)
|
||||
else:
|
||||
item["reason"] = (
|
||||
"No component compilation in quick/backend; UI runs all components. Full runs only its explicitly named subset."
|
||||
)
|
||||
if component == "all":
|
||||
selected = (
|
||||
list(CORE_COMPONENT_SUITES)
|
||||
if "core.component-batch" in stage_map
|
||||
else sorted(requested_components)
|
||||
)
|
||||
item.update(
|
||||
covered_components=selected,
|
||||
excluded_components=[
|
||||
name
|
||||
for name in CORE_COMPONENT_SUITES
|
||||
if name not in selected
|
||||
],
|
||||
)
|
||||
elif identity in explicit_npm:
|
||||
item.update(
|
||||
disposition="planned",
|
||||
covering_stage=explicit_npm[identity],
|
||||
reason="Explicit project check invokes this exact package suite.",
|
||||
)
|
||||
elif matching:
|
||||
if any(not value["args"] for value in matching):
|
||||
item.update(
|
||||
disposition="planned",
|
||||
covering_stage=next(
|
||||
value["covering_stage"]
|
||||
for value in matching
|
||||
if not value["args"]
|
||||
),
|
||||
reason="The canonical focused script directly invokes this exact package suite.",
|
||||
)
|
||||
else:
|
||||
item["reason"] = (
|
||||
"Canonical invocation supplies arguments; complete suite coverage cannot be inferred."
|
||||
)
|
||||
elif normalized in direct:
|
||||
item.update(
|
||||
disposition="planned",
|
||||
covering_stage=direct[normalized],
|
||||
reason="A selected stage runs this exact direct source-test command.",
|
||||
)
|
||||
elif normalized in canonical_nodes:
|
||||
item.update(
|
||||
disposition="covered_elsewhere",
|
||||
covering_stage=canonical_nodes[normalized],
|
||||
reason="Canonical focused script directly runs this suite's exact Node target.",
|
||||
)
|
||||
elif private_argv is None:
|
||||
item["disposition"] = "unsupported"
|
||||
elif item["name"] in {
|
||||
"test:module-permutations",
|
||||
"test:vite-cache-isolation",
|
||||
}:
|
||||
item["reason"] = (
|
||||
"Separate environment/permutation suite; absent from the selected stage plan."
|
||||
)
|
||||
else:
|
||||
item["reason"] = (
|
||||
"Not directly present in this profile's stage plan; nested npm scripts/hooks are not inferred."
|
||||
)
|
||||
suites.append(item)
|
||||
if len(suites) > MAX_SUITES:
|
||||
raise ValueError("Coverage inventory exceeds its bounded suite count")
|
||||
if project_file is not None:
|
||||
for check in project.config.get("checks", []):
|
||||
identity = check["id"]
|
||||
suites.append(
|
||||
{
|
||||
"repo": ",".join(check.get("repos", [])) or "project",
|
||||
"name": identity,
|
||||
"kind": "project-check",
|
||||
"package_path": None,
|
||||
"argv": redact_argv(check["argv"]),
|
||||
"command_sha256": digest(check["argv"]),
|
||||
"disposition": "planned" if identity in stage_map else "excluded",
|
||||
"covering_stage": identity if identity in stage_map else None,
|
||||
"reason": "Selected declared check or dependency."
|
||||
if identity in stage_map
|
||||
else "Declared check is outside this profile/selection.",
|
||||
}
|
||||
)
|
||||
if len(suites) > MAX_SUITES:
|
||||
raise ValueError("Coverage inventory exceeds its bounded suite count")
|
||||
counts = {
|
||||
kind: sum(item["disposition"] == kind for item in suites)
|
||||
for kind in DISPOSITIONS
|
||||
}
|
||||
notes.append(
|
||||
"Coverage describes planned suite invocations, not passing tests, per-test coverage, or completed UI review."
|
||||
)
|
||||
if full:
|
||||
notes.append(
|
||||
"Full is the canonical focused gate, not every declared package test; recursive commands and npm hooks are deliberately not guessed."
|
||||
)
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"profile": profile,
|
||||
"scope": "Declared root/webui package test scripts, discovered UI structural checks, and custom project checks across registered repositories.",
|
||||
"suite_count": len(suites),
|
||||
"stages": list(stage_map),
|
||||
"counts": counts,
|
||||
"suites": suites,
|
||||
"canonical_gate": {
|
||||
key: canonical[key]
|
||||
for key in ("path", "sha256", "metadata_sha256")
|
||||
if key in canonical
|
||||
}
|
||||
if canonical
|
||||
else None,
|
||||
"notes": notes,
|
||||
"summary": [
|
||||
"Suite coverage: "
|
||||
+ ", ".join(f"{value} {key}" for key, value in counts.items()),
|
||||
*notes,
|
||||
],
|
||||
}
|
||||
Executable
+133
@@ -0,0 +1,133 @@
|
||||
"""Recorded documentation checks built from existing owning-module contracts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import uuid
|
||||
|
||||
from .catalog import stage
|
||||
from .common import state_root
|
||||
from .workspace import load_project, selected_repositories
|
||||
|
||||
|
||||
LIMITATIONS = [
|
||||
"Static baseline, marker coverage and known display slots do not prove complete workflow or linguistic coverage.",
|
||||
"Computed labels, configured documentation providers and runtime module/permission combinations still require manual review.",
|
||||
"Audit findings do not edit documentation, translate text, publish evidence or close a review issue.",
|
||||
]
|
||||
|
||||
|
||||
def build_doc_stages(args) -> list[dict]:
|
||||
workspace = Path(args.workspace_root).resolve()
|
||||
project = load_project(workspace, getattr(args, "project", None))
|
||||
if getattr(args, "project", None):
|
||||
raise ValueError(
|
||||
"The docs audit currently targets GovOPlaN manifest/locale contracts. Register another project's documentation checks in its check profiles."
|
||||
)
|
||||
selected = selected_repositories(
|
||||
project, getattr(args, "repo", []), changed=getattr(args, "changed", False)
|
||||
)
|
||||
if getattr(args, "changed", False) and not selected:
|
||||
return []
|
||||
meta = next(repo.path for repo in project.repositories if repo.name == "govoplan")
|
||||
core = next(
|
||||
repo.path for repo in project.repositories if repo.name == "govoplan-core"
|
||||
)
|
||||
output = (
|
||||
state_root(workspace, getattr(args, "state_dir", None))
|
||||
/ "artifacts"
|
||||
/ f"docs-{uuid.uuid4().hex}"
|
||||
)
|
||||
reason = "Reuse the existing owning-module documentation and translation checks"
|
||||
labels = [
|
||||
"{node}",
|
||||
str(meta / "tools/devkit/audit-display-labels.mjs"),
|
||||
"--workspace-root",
|
||||
str(workspace),
|
||||
]
|
||||
for repo in selected:
|
||||
labels.extend(["--repo", repo.name])
|
||||
stages = [
|
||||
stage(
|
||||
"docs.manifests",
|
||||
"Static user/admin documentation and manifest contracts",
|
||||
[
|
||||
"{python}",
|
||||
str(meta / "tools/checks/check-manifest-shapes.py"),
|
||||
"--workspace-root",
|
||||
str(workspace),
|
||||
"--require-architecture",
|
||||
],
|
||||
meta,
|
||||
reason=reason,
|
||||
timeout_seconds=600,
|
||||
),
|
||||
stage(
|
||||
"docs.interface-inventory",
|
||||
"Existing EN/DE markers, high-risk help and interface declarations",
|
||||
[
|
||||
"{python}",
|
||||
str(meta / "tools/inventory/platform-interface-inventory.py"),
|
||||
"--workspace-root",
|
||||
str(workspace),
|
||||
"--strict",
|
||||
"--strict-declarations",
|
||||
"--strict-endpoints",
|
||||
"--output-dir",
|
||||
str(output),
|
||||
],
|
||||
meta,
|
||||
reason="Shared inventory remains workspace-wide; --repo narrows only the additional plain-label audit",
|
||||
timeout_seconds=600,
|
||||
),
|
||||
stage(
|
||||
"docs.translation-structure",
|
||||
"Existing translation key/structural-value guard",
|
||||
["{node}", str(core / "webui/scripts/audit-i18n-structural.mjs")],
|
||||
core / "webui",
|
||||
reason=reason,
|
||||
),
|
||||
stage(
|
||||
"docs.plain-display-labels",
|
||||
"Plain display labels and owning catalog registration",
|
||||
labels,
|
||||
meta,
|
||||
reason="Supplement the marker inventory with known display slots; dynamic cases are review candidates",
|
||||
timeout_seconds=600,
|
||||
),
|
||||
]
|
||||
# These constraints belong to the saved evidence, not only the immediate
|
||||
# CLI response. The publisher already validates bounded coverage_notes.
|
||||
stages[0]["coverage_notes"] = list(LIMITATIONS)
|
||||
return stages
|
||||
|
||||
|
||||
def audit(args) -> dict:
|
||||
from .runner import run_checks
|
||||
|
||||
result = run_checks(args, build_doc_stages(args))
|
||||
result["limitations"] = list(LIMITATIONS)
|
||||
result.setdefault("_exit_code", 0)
|
||||
summary = result.setdefault("summary", [])
|
||||
summary.extend(note for note in LIMITATIONS if note not in summary)
|
||||
return result
|
||||
|
||||
|
||||
def register(subparsers) -> None:
|
||||
docs = subparsers.add_parser(
|
||||
"docs", help="Audit existing documentation and translation contracts"
|
||||
)
|
||||
commands = docs.add_subparsers(dest="docs_command", required=True)
|
||||
command = commands.add_parser(
|
||||
"audit", help="Run recorded checks; never edit or generate translations"
|
||||
)
|
||||
command.add_argument(
|
||||
"--repo",
|
||||
action="append",
|
||||
default=[],
|
||||
help="Repository/alias for plain-label checks; shared checks stay workspace-wide",
|
||||
)
|
||||
command.add_argument("--changed", action="store_true")
|
||||
command.add_argument("--jobs", type=int, default=2)
|
||||
command.add_argument("--dry-run", action="store_true")
|
||||
command.set_defaults(handler=audit, profile="docs")
|
||||
Executable
+232
@@ -0,0 +1,232 @@
|
||||
"""Read-only environment preflight with actionable, never automatic repairs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
|
||||
from .common import now
|
||||
from .environment import execution_environment, resolve_tools, tool_version
|
||||
from .workspace import inspect_repository, load_project, selected_repositories
|
||||
from .process import require_capture
|
||||
|
||||
|
||||
def _required_tools(project, workspace_root, selected, *, filtered, profile=None):
|
||||
"""Use the catalog's selected commands, including dependencies, without running them."""
|
||||
from .catalog import _custom_stages
|
||||
|
||||
if profile is None:
|
||||
# Doctor covers all declared profiles by default, or all declared checks
|
||||
# when the project only uses the context/doctor commands so far.
|
||||
profiles = project.config.get("profiles", {})
|
||||
records = project.config.get("checks", [])
|
||||
if not isinstance(profiles, dict) or not isinstance(records, list):
|
||||
raise ValueError(
|
||||
"Project profiles and checks must have their declared shapes"
|
||||
)
|
||||
identities = []
|
||||
if profiles:
|
||||
for values in profiles.values():
|
||||
if not isinstance(values, list) or any(
|
||||
not isinstance(value, str) for value in values
|
||||
):
|
||||
raise ValueError("A project profile must be a list of check IDs")
|
||||
identities.extend(values)
|
||||
else:
|
||||
if any(not isinstance(record, dict) for record in records):
|
||||
raise ValueError("Project checks must be objects")
|
||||
identities = [record.get("id") for record in records]
|
||||
if any(not isinstance(identity, str) for identity in identities):
|
||||
raise ValueError("Project checks must have string IDs")
|
||||
project = replace(
|
||||
project,
|
||||
config={
|
||||
**project.config,
|
||||
"profiles": {"quick": list(dict.fromkeys(identities))},
|
||||
},
|
||||
)
|
||||
profile = "quick"
|
||||
stages = _custom_stages(project, workspace_root, profile, selected, filtered)
|
||||
# Python is required by the runner and its environment fingerprint even if
|
||||
# all selected check commands use another interpreter. Explicitly configured
|
||||
# tools also declare requirements for commands hidden inside project scripts.
|
||||
required = {"python", *project.config.get("tools", {})}
|
||||
for stage in stages:
|
||||
argv = stage["argv"]
|
||||
for name in ("node", "npm", "python"):
|
||||
if any("{" + name + "}" in value for value in argv):
|
||||
required.add(name)
|
||||
executable = Path(argv[0]).name
|
||||
if executable in {"node", "nodejs"}:
|
||||
required.add("node")
|
||||
elif executable in {"npm", "npx"}:
|
||||
required.update({"npm", "node"})
|
||||
if "npm" in required:
|
||||
required.add("node")
|
||||
return required
|
||||
|
||||
|
||||
def diagnose(args) -> dict:
|
||||
project = load_project(args.workspace_root, args.project)
|
||||
tools = resolve_tools(args.workspace_root, project)
|
||||
env = execution_environment(args.workspace_root, project, tools)
|
||||
selected = selected_repositories(project, args.repo)
|
||||
required = (
|
||||
_required_tools(
|
||||
project,
|
||||
args.workspace_root,
|
||||
selected,
|
||||
filtered=bool(args.repo),
|
||||
profile=getattr(args, "profile", None),
|
||||
)
|
||||
if args.project
|
||||
else set(tools)
|
||||
)
|
||||
checks = []
|
||||
for name, executable in tools.items():
|
||||
if name not in required:
|
||||
checks.append(
|
||||
{
|
||||
"id": name,
|
||||
"status": "not_required",
|
||||
"detail": "Not required by the selected declared project checks; not probed.",
|
||||
"path": executable,
|
||||
"repair": "For indirect dependencies inside scripts, declare the tool under project tools.",
|
||||
}
|
||||
)
|
||||
continue
|
||||
version = tool_version(executable, env)
|
||||
checks.append(
|
||||
{
|
||||
"id": name,
|
||||
"status": "passed" if version != "unavailable" else "blocked",
|
||||
"detail": version,
|
||||
"path": executable,
|
||||
"repair": f"Install/configure {name}; set {name.upper()} or project tools.{name}. No installation was attempted.",
|
||||
}
|
||||
)
|
||||
for repo in selected:
|
||||
snapshot = inspect_repository(repo)
|
||||
checks.append(
|
||||
{
|
||||
"id": repo.name,
|
||||
"status": "blocked" if snapshot["errors"] else "passed",
|
||||
"detail": "; ".join(snapshot["errors"])
|
||||
or "Git checkout readable (dirty work is allowed).",
|
||||
"repair": "Use tools/repo/bootstrap-repositories.py --check; review missing checkouts before cloning.",
|
||||
}
|
||||
)
|
||||
for package_dir in (repo.path, repo.path / "webui"):
|
||||
if (package_dir / "package.json").is_file():
|
||||
present = (package_dir / "node_modules").is_dir()
|
||||
checks.append(
|
||||
{
|
||||
"id": f"dependencies:{repo.name}:{package_dir.name}",
|
||||
"status": "passed" if present else "warning",
|
||||
"detail": "Dependency directory exists; availability is not a full dependency audit."
|
||||
if present
|
||||
else "No local node_modules directory; workspace-hoisted packages may still resolve.",
|
||||
"repair": "Inspect the owning package lock and install instructions before running npm ci.",
|
||||
}
|
||||
)
|
||||
if not args.project:
|
||||
script = args.workspace_root / "govoplan/tools/repo/sync-python-environment.py"
|
||||
if script.is_file():
|
||||
result = require_capture(
|
||||
[
|
||||
tools["python"],
|
||||
str(script),
|
||||
"--check",
|
||||
"--requirements",
|
||||
str(args.workspace_root / "govoplan/requirements-dev.txt"),
|
||||
"--python",
|
||||
tools["python"],
|
||||
],
|
||||
timeout=60,
|
||||
max_stdout=65536,
|
||||
env=env,
|
||||
)
|
||||
checks.append(
|
||||
{
|
||||
"id": "python-environment-sync",
|
||||
"status": "passed" if result.returncode == 0 else "warning",
|
||||
"detail": "Environment synchronization fingerprint is current."
|
||||
if result.returncode == 0
|
||||
else "Environment sync check did not pass; inspect its --dry-run output.",
|
||||
"repair": "./.venv/bin/python tools/repo/sync-python-environment.py --dry-run --requirements requirements-dev.txt --python ./.venv/bin/python",
|
||||
}
|
||||
)
|
||||
browser_roots = [
|
||||
Path.home() / ".cache/ms-playwright",
|
||||
Path.home() / ".var/app/com.vscodium.codium/cache/ms-playwright",
|
||||
]
|
||||
explicit_browser = os.environ.get("PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH")
|
||||
browser_available = (
|
||||
(Path(explicit_browser).is_file() and os.access(explicit_browser, os.X_OK))
|
||||
if explicit_browser
|
||||
else any(
|
||||
root.is_dir() and any(root.glob("chromium-*/chrome-linux*/chrome"))
|
||||
for root in browser_roots
|
||||
)
|
||||
)
|
||||
checks.append(
|
||||
{
|
||||
"id": "browser",
|
||||
"status": "passed" if browser_available else "warning",
|
||||
"detail": "Configured/cached Chromium found."
|
||||
if browser_available
|
||||
else "Chromium executable not found; check PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH or install the pinned Playwright browser.",
|
||||
"repair": "Follow Core WebUI conformance setup; do not start a development server to repair this.",
|
||||
}
|
||||
)
|
||||
with socket.socket() as probe:
|
||||
probe.settimeout(0.25)
|
||||
busy = probe.connect_ex(("127.0.0.1", 4174)) == 0
|
||||
checks.append(
|
||||
{
|
||||
"id": "browser-test-port",
|
||||
"status": "warning" if busy else "passed",
|
||||
"detail": "Port 4174 is occupied; do not kill another run."
|
||||
if busy
|
||||
else "Port 4174 is free now (not a reservation).",
|
||||
"repair": "Wait for the owning check run; use devkit resource coordination.",
|
||||
}
|
||||
)
|
||||
summary = [
|
||||
f"Environment preflight: {sum(item['status'] == 'blocked' for item in checks)} blockers, {sum(item['status'] == 'warning' for item in checks)} warnings."
|
||||
]
|
||||
summary.extend(
|
||||
f"{item['id']}: {item['status']} — {item['detail']}"
|
||||
for item in checks
|
||||
if item["status"] != "passed"
|
||||
)
|
||||
summary.append(
|
||||
"Read-only: no installations, configuration repairs or servers started."
|
||||
)
|
||||
if args.project:
|
||||
summary.append(
|
||||
"Required tools follow declared checks and explicit project tools; indirect script dependencies must be declared explicitly."
|
||||
)
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"generated_at": now(),
|
||||
"checks": checks,
|
||||
"required_tools": sorted(required),
|
||||
"summary": summary,
|
||||
"_exit_code": 1 if any(item["status"] == "blocked" for item in checks) else 0,
|
||||
}
|
||||
|
||||
|
||||
def register(subparsers):
|
||||
parser = subparsers.add_parser(
|
||||
"doctor", help="Read-only environment preflight and exact repair guidance"
|
||||
)
|
||||
parser.add_argument("--repo", action="append", default=[])
|
||||
parser.add_argument(
|
||||
"--profile",
|
||||
choices=("quick", "ui", "backend", "full"),
|
||||
help="For portable projects, limit tool requirements to this check profile (default: all declared profiles)",
|
||||
)
|
||||
parser.set_defaults(handler=diagnose)
|
||||
Executable
+271
@@ -0,0 +1,271 @@
|
||||
"""Resolve local tools once, without installation, network access or server startup."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from itertools import islice
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from .common import digest
|
||||
from .workspace import Project
|
||||
from .process import require_capture
|
||||
|
||||
MAX_DISCOVERY_CHILDREN = 4096
|
||||
|
||||
|
||||
def resolve_tools(workspace_root: Path, project: Project) -> dict[str, str]:
|
||||
configured = project.config.get("tools", {})
|
||||
if not isinstance(configured, dict) or set(configured) - {"python", "node", "npm"}:
|
||||
raise ValueError("Project tools must configure only python, node and npm")
|
||||
tools = {}
|
||||
python_env = (
|
||||
Path(
|
||||
os.environ.get("GOVOPLAN_VENV_ROOT", str(workspace_root / "govoplan/.venv"))
|
||||
)
|
||||
/ "bin/python"
|
||||
)
|
||||
for name in ("python", "node", "npm"):
|
||||
default = (
|
||||
str(python_env)
|
||||
if name == "python" and python_env.is_file()
|
||||
else sys.executable
|
||||
if name == "python"
|
||||
else name
|
||||
)
|
||||
value = configured.get(name) or os.environ.get(name.upper()) or default
|
||||
if not isinstance(value, str) or not value or "\0" in value:
|
||||
raise ValueError(f"Invalid executable configuration for {name}")
|
||||
executable = shutil.which(value)
|
||||
if executable:
|
||||
# Keep venv executable symlinks: resolving them loses Python's environment.
|
||||
tools[name] = os.path.abspath(executable)
|
||||
else:
|
||||
tools[name] = value
|
||||
return tools
|
||||
|
||||
|
||||
def execution_environment(
|
||||
workspace_root: Path, project: Project, tools: dict[str, str]
|
||||
) -> dict[str, str]:
|
||||
env = dict(os.environ)
|
||||
if (
|
||||
project.config.get("organization") == "GovOPlaN"
|
||||
and "schema_version" not in project.config
|
||||
):
|
||||
# Managed GovOPlaN checks must not inherit another checkout's scope.
|
||||
env["GOVOPLAN_WORKSPACE_ROOT"] = str(workspace_root.resolve())
|
||||
core = next(
|
||||
repo.path for repo in project.repositories if repo.name == "govoplan-core"
|
||||
)
|
||||
env["GOVOPLAN_CORE_ROOT"] = str(core)
|
||||
env["GOVOPLAN_CORE_SOURCE_ROOT"] = str(core)
|
||||
directories = [
|
||||
str(Path(tools[name]).parent)
|
||||
for name in ("python", "node", "npm")
|
||||
if Path(tools[name]).is_absolute()
|
||||
]
|
||||
core_bins = workspace_root / "govoplan-core/webui/node_modules/.bin"
|
||||
if core_bins.is_dir():
|
||||
directories.insert(0, str(core_bins))
|
||||
env["PATH"] = os.pathsep.join([*directories, env.get("PATH", "")])
|
||||
env.update({name.upper(): value for name, value in tools.items()})
|
||||
sources = [
|
||||
str(repo.path / "src")
|
||||
for repo in project.repositories
|
||||
if (repo.path / "src").is_dir()
|
||||
]
|
||||
if sources:
|
||||
env["PYTHONPATH"] = os.pathsep.join(
|
||||
sources + ([env["PYTHONPATH"]] if env.get("PYTHONPATH") else [])
|
||||
)
|
||||
env["GIT_OPTIONAL_LOCKS"] = "0"
|
||||
env["PYTHONDONTWRITEBYTECODE"] = "1"
|
||||
return env
|
||||
|
||||
|
||||
def tool_version(executable: str, env: dict[str, str]) -> str:
|
||||
try:
|
||||
result = require_capture(
|
||||
[executable, "--version"],
|
||||
timeout=15,
|
||||
max_stdout=4096,
|
||||
max_stderr=4096,
|
||||
env=env,
|
||||
)
|
||||
return (
|
||||
(result.stdout or result.stderr).decode(errors="replace").strip()[:256]
|
||||
if result.returncode == 0
|
||||
else "unavailable"
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired, ValueError):
|
||||
return "unavailable"
|
||||
|
||||
|
||||
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
|
||||
return (
|
||||
metadata.st_dev,
|
||||
metadata.st_ino,
|
||||
metadata.st_mode,
|
||||
metadata.st_size,
|
||||
metadata.st_ctime_ns,
|
||||
metadata.st_mtime_ns,
|
||||
)
|
||||
|
||||
|
||||
def _environment_file_hash(path: Path, maximum: int) -> str | None:
|
||||
"""Read a stable, bounded regular target without changing executable spelling.
|
||||
|
||||
Venv executables and package directories may legitimately be symlinks. Only
|
||||
the read target is resolved; both names and the open descriptor are verified
|
||||
again afterwards. Missing optional metadata stays absent, but a present
|
||||
malformed or concurrently changing entry cannot certify an environment.
|
||||
"""
|
||||
try:
|
||||
original = path.lstat()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
try:
|
||||
target = path.resolve(strict=True)
|
||||
target_metadata = target.lstat()
|
||||
descriptor = os.open(
|
||||
target,
|
||||
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
before = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
|
||||
raise ValueError("Environment input must be a bounded regular file")
|
||||
identity = _file_identity(before)
|
||||
if identity != _file_identity(target_metadata):
|
||||
raise ValueError("Environment input changed before fingerprinting")
|
||||
hasher, size = hashlib.sha256(), 0
|
||||
for chunk in iter(
|
||||
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
|
||||
):
|
||||
size += len(chunk)
|
||||
if size > maximum:
|
||||
raise ValueError("Environment input grew beyond its size bound")
|
||||
hasher.update(chunk)
|
||||
if (
|
||||
_file_identity(os.fstat(handle.fileno())) != identity
|
||||
or _file_identity(target.lstat()) != identity
|
||||
or _file_identity(path.lstat()) != _file_identity(original)
|
||||
or path.resolve(strict=True) != target
|
||||
):
|
||||
raise ValueError("Environment input changed during fingerprinting")
|
||||
return hasher.hexdigest()
|
||||
except (OSError, RuntimeError) as exc:
|
||||
raise ValueError("Environment input cannot be fingerprinted safely") from exc
|
||||
|
||||
|
||||
def _discovery_path_shape(path: Path) -> dict:
|
||||
"""Directory membership, not timestamps changed by ordinary build outputs."""
|
||||
try:
|
||||
metadata = path.lstat()
|
||||
except (FileNotFoundError, NotADirectoryError):
|
||||
return {"kind": "missing"}
|
||||
try:
|
||||
resolved = path.resolve()
|
||||
try:
|
||||
target_kind = stat.S_IFMT(resolved.lstat().st_mode)
|
||||
except (FileNotFoundError, NotADirectoryError):
|
||||
target_kind = "missing"
|
||||
if target_kind == stat.S_IFLNK:
|
||||
# Newer pathlib versions can retain a loop with strict=False.
|
||||
raise ValueError("Native source discovery cannot be resolved safely")
|
||||
return {
|
||||
"kind": stat.S_IFMT(metadata.st_mode),
|
||||
"resolved": str(resolved),
|
||||
"target_kind": target_kind,
|
||||
}
|
||||
except (OSError, RuntimeError) as exc:
|
||||
raise ValueError("Native source discovery cannot be resolved safely") from exc
|
||||
|
||||
|
||||
def _native_discovery_fingerprint(workspace_root: Path, project: Project) -> str:
|
||||
"""Bind glob discovery and registered ownership omitted by narrow Git scopes."""
|
||||
children = list(islice(workspace_root.iterdir(), MAX_DISCOVERY_CHILDREN + 1))
|
||||
if len(children) > MAX_DISCOVERY_CHILDREN:
|
||||
raise ValueError("Native source discovery exceeds its bounded ownership audit")
|
||||
|
||||
def shape(path):
|
||||
return {
|
||||
name: _discovery_path_shape(path / name if name else path)
|
||||
for name in ("", "src", "webui")
|
||||
}
|
||||
|
||||
return digest(
|
||||
{
|
||||
"version": 1,
|
||||
"siblings": [
|
||||
{"name": child.name, "shape": shape(child)}
|
||||
for child in sorted(children)
|
||||
if child.name.startswith("govoplan")
|
||||
],
|
||||
"registered": [
|
||||
{"name": repo.name, "path": str(repo.path), "shape": shape(repo.path)}
|
||||
for repo in sorted(project.repositories, key=lambda item: item.name)
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def environment_fingerprint(
|
||||
workspace_root: Path, project: Project, tools: dict[str, str], env: dict[str, str]
|
||||
) -> str:
|
||||
identity = {
|
||||
"environment": {
|
||||
key: value
|
||||
for key, value in env.items()
|
||||
if key not in {"_", "SHLVL", "PWD", "OLDPWD"}
|
||||
},
|
||||
"tools": {},
|
||||
}
|
||||
if (
|
||||
project.config.get("organization") == "GovOPlaN"
|
||||
and "schema_version" not in project.config
|
||||
):
|
||||
identity["native_source_discovery"] = _native_discovery_fingerprint(
|
||||
workspace_root, project
|
||||
)
|
||||
for name, executable in tools.items():
|
||||
binary_hash = _environment_file_hash(Path(executable), 512 * 1024 * 1024)
|
||||
identity["tools"][name] = {
|
||||
"path": executable,
|
||||
"version": tool_version(executable, env),
|
||||
"sha256": binary_hash,
|
||||
}
|
||||
installed = {}
|
||||
for repo in project.repositories:
|
||||
for suffix in (
|
||||
"node_modules/.package-lock.json",
|
||||
"webui/node_modules/.package-lock.json",
|
||||
".venv/pyvenv.cfg",
|
||||
):
|
||||
path = repo.path / suffix
|
||||
value = _environment_file_hash(path, 32 * 1024 * 1024)
|
||||
if value is not None:
|
||||
installed[str(path)] = value
|
||||
try:
|
||||
result = require_capture(
|
||||
[
|
||||
tools["python"],
|
||||
"-c",
|
||||
"import importlib.metadata,json; print(json.dumps(sorted((d.metadata['Name'],d.version) for d in importlib.metadata.distributions())))",
|
||||
],
|
||||
timeout=30,
|
||||
max_stdout=1024 * 1024,
|
||||
env=env,
|
||||
)
|
||||
if result.returncode:
|
||||
raise ValueError("Cannot fingerprint installed Python distributions")
|
||||
installed["python_distributions"] = hashlib.sha256(result.stdout).hexdigest()
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise ValueError("Cannot fingerprint Python environment") from exc
|
||||
identity["installed"] = installed
|
||||
return digest(identity)
|
||||
Executable
+417
@@ -0,0 +1,417 @@
|
||||
"""Versioned repository-scoped input identities with run-local content memoization.
|
||||
|
||||
No stored receipt supplies commands to this engine. Every snapshot re-reads Git
|
||||
HEAD/index/flags/membership and opens its inputs; only stable file-content hashes
|
||||
are memoized, never repository snapshots or prior-run results.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
|
||||
from .common import META_ROOT, canonical, digest, identifier
|
||||
from .workspace import Project, git_bytes
|
||||
|
||||
FINGERPRINT_VERSION = "repository-inputs-v1"
|
||||
MAX_SOURCE_BYTES = 64 * 1024 * 1024
|
||||
MAX_CACHE_FILES = 200_000
|
||||
MAX_REPOSITORY_ENTRIES = 200_000
|
||||
_INDEX_RECORD = re.compile(rb"[A-Za-z] [0-7]{6} [a-fA-F0-9]{40,64} [0-3]\t(.*)\Z", re.S)
|
||||
_RUNTIME_FIELDS = {
|
||||
"status",
|
||||
"exit_code",
|
||||
"duration_seconds",
|
||||
"log_path",
|
||||
"log_sha256",
|
||||
"error",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"reused_from",
|
||||
"output_truncated",
|
||||
"checkpoint_verified",
|
||||
"checkpoint_at",
|
||||
"cache_key",
|
||||
"input_fingerprint",
|
||||
"reuse_reason",
|
||||
"omitted_output_bytes",
|
||||
}
|
||||
|
||||
|
||||
def validate_input_declaration(value: object, repository_names) -> dict:
|
||||
"""Pure validation shared with planning; does not inspect files or run Git."""
|
||||
if not isinstance(value, dict) or set(value) != {"repos"}:
|
||||
raise ValueError(
|
||||
"Stage inputs must contain only the required repos declaration"
|
||||
)
|
||||
names = value["repos"]
|
||||
if not isinstance(names, list) or not 1 <= len(names) <= 256:
|
||||
raise ValueError(
|
||||
"Input repos must be a nonempty bounded list of canonical repository names"
|
||||
)
|
||||
for name in names:
|
||||
identifier(name)
|
||||
if len(set(names)) != len(names):
|
||||
raise ValueError("Duplicate input repository reference")
|
||||
if set(names) - set(repository_names):
|
||||
raise ValueError(
|
||||
"Input repos reference an unknown/noncanonical repository name"
|
||||
)
|
||||
return {"repos": sorted(names)}
|
||||
|
||||
|
||||
def _file_identity(metadata: os.stat_result) -> tuple[int, ...]:
|
||||
return (
|
||||
metadata.st_dev,
|
||||
metadata.st_ino,
|
||||
metadata.st_mode,
|
||||
metadata.st_size,
|
||||
metadata.st_ctime_ns,
|
||||
metadata.st_mtime_ns,
|
||||
)
|
||||
|
||||
|
||||
def _stats() -> dict[str, int]:
|
||||
return {
|
||||
name: 0
|
||||
for name in (
|
||||
"repositories",
|
||||
"git_calls",
|
||||
"entries",
|
||||
"files",
|
||||
"bytes",
|
||||
"cache_hits",
|
||||
"tooling_files",
|
||||
"tooling_bytes",
|
||||
"tooling_cache_hits",
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
class InputSnapshotter:
|
||||
"""One run's in-memory memo; create a fresh instance for each invocation."""
|
||||
|
||||
def __init__(self, project: Project, *, workspace_root: Path):
|
||||
self.project = project
|
||||
self.workspace_root = Path(workspace_root).resolve()
|
||||
self.repositories = {}
|
||||
for repo in project.repositories:
|
||||
identifier(repo.name)
|
||||
if repo.name in self.repositories:
|
||||
raise ValueError("Duplicate input repository name")
|
||||
if not repo.path.is_absolute() or not repo.path.resolve().is_relative_to(
|
||||
self.workspace_root
|
||||
):
|
||||
raise ValueError("Input repository path escapes its workspace")
|
||||
self.repositories[repo.name] = repo
|
||||
if not self.repositories or len(self.repositories) > 256:
|
||||
raise ValueError("Input project requires 1–256 repositories")
|
||||
# One most recent stable identity per path. This is deliberately not
|
||||
# serializable/persisted, and hashes are never reused on mtime alone.
|
||||
self._files: dict[Path, tuple[tuple[int, ...], bytes]] = {}
|
||||
|
||||
def _repo_names(self, names: object) -> list[str]:
|
||||
return validate_input_declaration({"repos": names}, self.repositories)["repos"]
|
||||
|
||||
def scope(self, stage: dict) -> dict:
|
||||
"""Validate one explicit scope; absent inputs means the whole workspace."""
|
||||
if "inputs" not in stage:
|
||||
return {
|
||||
"version": 1,
|
||||
"kind": "workspace",
|
||||
"declared": False,
|
||||
"repos": sorted(self.repositories),
|
||||
}
|
||||
value = validate_input_declaration(stage["inputs"], self.repositories)
|
||||
return {
|
||||
"version": 1,
|
||||
"kind": "repositories",
|
||||
"declared": True,
|
||||
"repos": value["repos"],
|
||||
}
|
||||
|
||||
def _file_hash(
|
||||
self, path: Path, statistics: dict, *, expected=None, tooling: bool = False
|
||||
) -> bytes:
|
||||
prefix = "tooling_" if tooling else ""
|
||||
maximum = 4 * 1024 * 1024 if tooling else MAX_SOURCE_BYTES
|
||||
descriptor = os.open(
|
||||
path,
|
||||
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0),
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
before = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
|
||||
raise ValueError("Input must be a bounded regular file")
|
||||
identity = _file_identity(before)
|
||||
if expected is not None and _file_identity(expected) != identity:
|
||||
raise ValueError(
|
||||
"Input changed before its source identity could be established"
|
||||
)
|
||||
statistics[prefix + "files"] += 1
|
||||
cached = self._files.get(path)
|
||||
if cached is not None and cached[0] == identity:
|
||||
value = cached[1]
|
||||
statistics[prefix + "cache_hits"] += 1
|
||||
else:
|
||||
hasher, size = hashlib.sha256(), 0
|
||||
for chunk in iter(
|
||||
lambda: handle.read(min(1024 * 1024, maximum - size + 1)), b""
|
||||
):
|
||||
size += len(chunk)
|
||||
if size > maximum:
|
||||
raise ValueError("Input grew beyond its source identity limit")
|
||||
hasher.update(chunk)
|
||||
statistics[prefix + "bytes"] += size
|
||||
value = hasher.digest()
|
||||
after = os.fstat(handle.fileno())
|
||||
# Rechecking the pathname also rejects replacement after the open;
|
||||
# a stable old FD is not evidence about a newly replaced source file.
|
||||
current = path.lstat()
|
||||
if _file_identity(after) != identity or _file_identity(current) != identity:
|
||||
raise ValueError("Input changed while calculating its source identity")
|
||||
if path not in self._files and len(self._files) >= MAX_CACHE_FILES:
|
||||
self._files.clear() # Eviction only causes extra reads, never reuse.
|
||||
self._files[path] = (identity, value)
|
||||
return value
|
||||
|
||||
def _entry(self, repo, encoded_name: bytes, statistics: dict) -> dict:
|
||||
relative = Path(os.fsdecode(encoded_name))
|
||||
if relative.is_absolute() or ".." in relative.parts:
|
||||
raise ValueError("Unsafe source membership path")
|
||||
path = repo.path / relative
|
||||
statistics["entries"] += 1
|
||||
try:
|
||||
metadata = path.lstat()
|
||||
except FileNotFoundError:
|
||||
return {"kind": "missing"}
|
||||
if not path.parent.resolve().is_relative_to(repo.path.resolve()):
|
||||
raise ValueError("Input path escapes its declared repository")
|
||||
result = {"mode": stat.S_IMODE(metadata.st_mode)}
|
||||
if stat.S_ISLNK(metadata.st_mode):
|
||||
target_name = os.readlink(path)
|
||||
try:
|
||||
target = path.resolve()
|
||||
except (OSError, RuntimeError) as exc:
|
||||
raise ValueError("Input symlink cannot be resolved safely") from exc
|
||||
if not target.is_relative_to(repo.path.resolve()):
|
||||
raise ValueError(
|
||||
"Input symlink target escapes its repository; undeclared target bytes cannot be reused"
|
||||
)
|
||||
result.update(
|
||||
kind="symlink",
|
||||
target_sha256=hashlib.sha256(os.fsencode(target_name)).hexdigest(),
|
||||
)
|
||||
try:
|
||||
target_metadata = target.lstat()
|
||||
except FileNotFoundError:
|
||||
result["target_state"] = "missing"
|
||||
else:
|
||||
if not stat.S_ISREG(target_metadata.st_mode):
|
||||
raise ValueError(
|
||||
"Input symlink must target a regular file inside its repository"
|
||||
)
|
||||
result.update(
|
||||
target_state="file",
|
||||
target_mode=stat.S_IMODE(target_metadata.st_mode),
|
||||
target_content=self._file_hash(
|
||||
target, statistics, expected=target_metadata
|
||||
).hex(),
|
||||
)
|
||||
if (
|
||||
_file_identity(path.lstat()) != _file_identity(metadata)
|
||||
or os.readlink(path) != target_name
|
||||
):
|
||||
raise ValueError(
|
||||
"Input symlink changed while calculating source identity"
|
||||
)
|
||||
elif stat.S_ISREG(metadata.st_mode):
|
||||
result.update(
|
||||
kind="file",
|
||||
content=self._file_hash(path, statistics, expected=metadata).hex(),
|
||||
)
|
||||
else:
|
||||
raise ValueError(
|
||||
"Unsupported source entry; scoped repository inputs require files or in-repository file symlinks"
|
||||
)
|
||||
return result
|
||||
|
||||
def _repository(self, name: str, statistics: dict) -> str:
|
||||
repo = self.repositories[name]
|
||||
statistics["repositories"] += 1
|
||||
identity = {
|
||||
"version": FINGERPRINT_VERSION,
|
||||
"repo": name,
|
||||
"path": str(repo.path),
|
||||
}
|
||||
if not repo.path.exists():
|
||||
return digest({**identity, "state": "missing"})
|
||||
if (
|
||||
not repo.path.resolve().is_relative_to(self.workspace_root)
|
||||
or not (repo.path / ".git").exists()
|
||||
):
|
||||
raise ValueError(
|
||||
"Cannot establish scoped source identity for an escaped/non-Git repository"
|
||||
)
|
||||
statistics["git_calls"] += 1
|
||||
head = git_bytes(
|
||||
repo.path, "rev-parse", "--verify", "HEAD", allow_failure=True
|
||||
).strip()
|
||||
statistics["git_calls"] += 1
|
||||
# This preserves stage numbers and assume-unchanged/skip-worktree flags,
|
||||
# while also listing untracked membership in the same bounded process.
|
||||
index = git_bytes(
|
||||
repo.path,
|
||||
"ls-files",
|
||||
"--stage",
|
||||
"-v",
|
||||
"--cached",
|
||||
"--others",
|
||||
"--exclude-standard",
|
||||
"-z",
|
||||
)
|
||||
members = set()
|
||||
for record in index.split(b"\0"):
|
||||
if not record:
|
||||
continue
|
||||
if record.startswith(b"? "):
|
||||
members.add(record[2:])
|
||||
else:
|
||||
match = _INDEX_RECORD.fullmatch(record)
|
||||
if not match:
|
||||
raise ValueError(
|
||||
"Unsupported Git membership record; cannot establish scoped source identity"
|
||||
)
|
||||
members.add(match[1])
|
||||
if len(members) > MAX_REPOSITORY_ENTRIES:
|
||||
raise ValueError(
|
||||
"Repository source membership exceeds its bounded entry count"
|
||||
)
|
||||
hasher = hashlib.sha256(
|
||||
canonical(
|
||||
{
|
||||
**identity,
|
||||
"head_sha256": hashlib.sha256(head).hexdigest(),
|
||||
"index_sha256": hashlib.sha256(index).hexdigest(),
|
||||
}
|
||||
)
|
||||
)
|
||||
for member in sorted(members):
|
||||
hasher.update(
|
||||
canonical(
|
||||
{
|
||||
"name_sha256": hashlib.sha256(member).hexdigest(),
|
||||
"value": self._entry(repo, member, statistics),
|
||||
}
|
||||
)
|
||||
)
|
||||
hasher.update(b"\0")
|
||||
return hasher.hexdigest()
|
||||
|
||||
def _source_identity(self, names: list[str], identities: dict[str, str]) -> str:
|
||||
return digest(
|
||||
{
|
||||
"version": FINGERPRINT_VERSION,
|
||||
"workspace_root": str(self.workspace_root),
|
||||
"repositories": {name: identities[name] for name in names},
|
||||
}
|
||||
)
|
||||
|
||||
def _observe(self, names: list[str]) -> dict:
|
||||
statistics = _stats()
|
||||
identities = {name: self._repository(name, statistics) for name in names}
|
||||
complete = set(names) == self.repositories.keys()
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"fingerprint_version": FINGERPRINT_VERSION,
|
||||
"workspace_root": str(self.workspace_root),
|
||||
"repository_fingerprints": identities,
|
||||
"observed_source_fingerprint": self._source_identity(names, identities),
|
||||
"observed_scope": {
|
||||
"version": 1,
|
||||
"kind": "workspace" if complete else "repositories",
|
||||
"repos": names,
|
||||
},
|
||||
"complete_workspace": complete,
|
||||
"scan_stats": statistics,
|
||||
}
|
||||
|
||||
def source_snapshot(self, repos: list[str] | None = None) -> dict:
|
||||
"""Compare observed source only, without interpreting any stored commands."""
|
||||
return self._observe(
|
||||
sorted(self.repositories) if repos is None else self._repo_names(repos)
|
||||
)
|
||||
|
||||
def _tooling_identity(self, statistics: dict) -> str:
|
||||
package = Path(__file__).resolve().parent
|
||||
paths = set()
|
||||
for path in package.rglob("*.py"):
|
||||
paths.add(path)
|
||||
if len(paths) > 256:
|
||||
raise ValueError("Devkit tooling source exceeds its bounded inventory")
|
||||
paths.update(
|
||||
{
|
||||
package.parent / "devkit.py",
|
||||
package.parent / "project.schema.json",
|
||||
META_ROOT / "devkit",
|
||||
}
|
||||
)
|
||||
return digest(
|
||||
{
|
||||
str(path.relative_to(META_ROOT)): self._file_hash(
|
||||
path, statistics, tooling=True
|
||||
).hex()
|
||||
for path in sorted(paths)
|
||||
}
|
||||
)
|
||||
|
||||
def snapshot(self, stages: list[dict], *, tooling_fingerprint: str = "") -> dict:
|
||||
if not isinstance(stages, list) or len(stages) > 512:
|
||||
raise ValueError("Input snapshot requires a bounded stage list")
|
||||
if not isinstance(tooling_fingerprint, str) or len(tooling_fingerprint) > 4096:
|
||||
raise ValueError("Invalid supplied tooling identity")
|
||||
scopes = {}
|
||||
for stage in stages:
|
||||
if not isinstance(stage, dict):
|
||||
raise ValueError("Input stages must be objects")
|
||||
if set(stage) & _RUNTIME_FIELDS:
|
||||
raise ValueError(
|
||||
"Input identities require freshly planned stages, not mutable receipt/runtime fields"
|
||||
)
|
||||
identity = identifier(stage.get("id"))
|
||||
if identity in scopes:
|
||||
raise ValueError("Duplicate input stage identity")
|
||||
scopes[identity] = self.scope(stage)
|
||||
names = sorted({name for scope in scopes.values() for name in scope["repos"]})
|
||||
observed = self._observe(names)
|
||||
tools = digest(
|
||||
{
|
||||
"declared_tools": self.project.config.get("tools", {}),
|
||||
"devkit_source": self._tooling_identity(observed["scan_stats"]),
|
||||
"supplied_tooling": tooling_fingerprint,
|
||||
}
|
||||
)
|
||||
entries = {}
|
||||
for stage in stages:
|
||||
scope = scopes[stage["id"]]
|
||||
source = self._source_identity(
|
||||
scope["repos"], observed["repository_fingerprints"]
|
||||
)
|
||||
plan = digest(stage)
|
||||
entries[stage["id"]] = {
|
||||
"source_fingerprint": source,
|
||||
"plan_fingerprint": plan,
|
||||
"scope": scope,
|
||||
"fingerprint": digest(
|
||||
{
|
||||
"version": FINGERPRINT_VERSION,
|
||||
"source": source,
|
||||
"plan": plan,
|
||||
"tooling": tools,
|
||||
}
|
||||
),
|
||||
}
|
||||
return {**observed, "tooling_fingerprint": tools, "stages": entries}
|
||||
Executable
+547
@@ -0,0 +1,547 @@
|
||||
"""Append-only, idempotent Gitea evidence notes; preview is entirely offline."""
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import ExitStack
|
||||
from dataclasses import dataclass
|
||||
import hashlib
|
||||
import html
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Any
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .common import META_ROOT, atomic_json, digest, identifier, read_json, redact, resource_lock, state_root
|
||||
from .inputs import FINGERPRINT_VERSION
|
||||
|
||||
_GITEA_PATH = str(META_ROOT / "tools/gitea")
|
||||
if _GITEA_PATH not in sys.path:
|
||||
sys.path.insert(0, _GITEA_PATH)
|
||||
from gitea_common import GiteaClient, RepoTarget, _parse_remote, repo_path # noqa: E402
|
||||
|
||||
MAX_NOTE_BYTES = 128 * 1024
|
||||
MAX_TARGETS = 256
|
||||
MAX_COVERAGE_NOTES = 2048
|
||||
MARKER_PREFIX = "<!-- govoplan-devkit:evidence:v1:"
|
||||
STATUSES = {"planned", "pending", "running", "passed", "failed", "timed_out", "skipped", "cancelled", "interrupted", "blocked", "partial", "stale"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NoteTarget:
|
||||
root: Path
|
||||
base_url: str
|
||||
owner: str
|
||||
repository: str
|
||||
issue: int
|
||||
issue_id: int | None = None
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return f"{self.base_url}/{self.owner}/{self.repository}/issues/{self.issue}"
|
||||
|
||||
@property
|
||||
def path(self) -> str:
|
||||
return repo_path(self.owner, self.repository, f"/issues/{self.issue}")
|
||||
|
||||
def record(self) -> dict:
|
||||
return {"root": str(self.root), "base_url": self.base_url, "owner": self.owner,
|
||||
"repository": self.repository, "issue": self.issue, "url": self.url,
|
||||
**({"issue_id": self.issue_id} if self.issue_id is not None else {})}
|
||||
|
||||
|
||||
def register(subparsers) -> None:
|
||||
parser = subparsers.add_parser("issues", help="Preview or append evidence to exact Gitea issues")
|
||||
commands = parser.add_subparsers(dest="issues_command", required=True)
|
||||
note = commands.add_parser("note", help="Append a deduplicated note; offline dry run by default")
|
||||
note.add_argument("--root", type=Path, help="Target repository root or child directory")
|
||||
note.add_argument("--issue", type=int)
|
||||
note.add_argument("--target-plan", type=Path, help="JSON with schema_version=1 and exact root/issue/url targets")
|
||||
note.add_argument("--remote", default="origin")
|
||||
note.add_argument("--env-file", type=Path, help="Private GITEA_TOKEN dotenv file; target overrides are ignored")
|
||||
note.add_argument("--evidence", help="Local run ID or explicit receipt JSON path")
|
||||
note.add_argument("--key", default="verification", help="Stable note purpose within an evidence run")
|
||||
note.add_argument("--summary", dest="note_summary", action="append", default=[])
|
||||
note.add_argument("--next", dest="next_steps", action="append", default=[])
|
||||
note.add_argument("--body-file", type=Path, help="Additional Markdown, never executed")
|
||||
note.add_argument("--note-file", type=Path, help="Structured JSON: summary[], next[], body")
|
||||
note.add_argument("--apply", action="store_true", help="Explicitly authorize serial comment creation")
|
||||
note.add_argument("--retry-uncertain", action="store_true", help="After reconciliation, explicitly retry a still-unconfirmed earlier POST")
|
||||
note.set_defaults(handler=handle_note)
|
||||
|
||||
|
||||
def _text(value: Any, *, maximum: int = 16384) -> str:
|
||||
if not isinstance(value, str) or len(value.encode("utf-8")) > maximum:
|
||||
raise ValueError("Note/evidence text must be a bounded string")
|
||||
if any(ord(character) < 32 and character not in "\n\t\r" for character in value):
|
||||
raise ValueError("Note/evidence text contains control characters")
|
||||
if MARKER_PREFIX in value:
|
||||
raise ValueError("Evidence markers are reserved for the publisher")
|
||||
return redact(value)
|
||||
|
||||
|
||||
def _positive(value: Any) -> int:
|
||||
if type(value) is not int or value <= 0:
|
||||
raise ValueError("Issue and comment identities must be positive integers")
|
||||
return value
|
||||
|
||||
|
||||
def _base_url(value: str) -> str:
|
||||
parsed = urlsplit(value)
|
||||
if (parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username
|
||||
or parsed.password or parsed.query or parsed.fragment or "\\" in value
|
||||
or any(ord(char) < 33 for char in value)):
|
||||
raise ValueError("Gitea target must be a credential-free HTTP(S) URL")
|
||||
if any(part in {".", ".."} or "%" in part for part in parsed.path.split("/")):
|
||||
raise ValueError("Gitea base URL contains an ambiguous path")
|
||||
try:
|
||||
parsed.port
|
||||
except ValueError as exc:
|
||||
raise ValueError("Invalid Gitea port") from exc
|
||||
return value.rstrip("/")
|
||||
|
||||
|
||||
def _name(value: str) -> str:
|
||||
if not isinstance(value, str) or not re.fullmatch(r"[A-Za-z0-9_.-]+", value) or value in {".", ".."}:
|
||||
raise ValueError("Invalid Gitea owner or repository name")
|
||||
return value
|
||||
|
||||
|
||||
def resolve_target(root: Path, issue: int, workspace_root: Path, *, remote: str = "origin",
|
||||
expected_url: str | None = None, issue_id: int | None = None) -> NoteTarget:
|
||||
requested = root if root.is_absolute() else workspace_root / root
|
||||
resolved = requested.resolve()
|
||||
if not resolved.is_relative_to(workspace_root.resolve()) or not resolved.is_dir():
|
||||
raise ValueError("Issue target must be an existing repository inside the selected workspace")
|
||||
if not re.fullmatch(r"[A-Za-z0-9_.-]+", remote):
|
||||
raise ValueError("Invalid Git remote name")
|
||||
command = subprocess.run(["git", "-C", str(resolved), "rev-parse", "--show-toplevel"],
|
||||
capture_output=True, text=True, timeout=15)
|
||||
if command.returncode:
|
||||
raise ValueError("Issue target is not a Git checkout")
|
||||
actual = Path(command.stdout.strip()).resolve()
|
||||
if not actual.is_relative_to(workspace_root.resolve()):
|
||||
raise ValueError("Resolved issue repository escapes the workspace")
|
||||
result = subprocess.run(["git", "-C", str(actual), "remote", "get-url", remote],
|
||||
capture_output=True, text=True, timeout=15)
|
||||
if result.returncode or len(result.stdout) > 8192:
|
||||
raise ValueError("Target repository has no usable Git remote")
|
||||
remote_url = result.stdout.strip()
|
||||
parsed = urlsplit(remote_url)
|
||||
if (any(ord(char) < 33 for char in remote_url) or "\\" in remote_url
|
||||
or parsed.query or parsed.fragment):
|
||||
raise ValueError("Ambiguous Git remote URL cannot bind an issue target")
|
||||
if parsed.scheme in {"http", "https"} and (parsed.username or parsed.password):
|
||||
raise ValueError("Credential-bearing Git remotes cannot be used for issue publishing")
|
||||
base, owner, repository = _parse_remote(remote_url)
|
||||
target = NoteTarget(actual, _base_url(base), _name(owner), _name(repository), _positive(issue),
|
||||
_positive(issue_id) if issue_id is not None else None)
|
||||
if expected_url is not None and expected_url != target.url:
|
||||
raise ValueError("Target plan issue URL does not match the exact repository remote and issue number")
|
||||
return target
|
||||
|
||||
|
||||
def _targets(args) -> list[NoteTarget]:
|
||||
workspace_root = Path(args.workspace_root).resolve()
|
||||
if args.target_plan:
|
||||
if args.root is not None or args.issue is not None:
|
||||
raise ValueError("Use either --root/--issue or --target-plan")
|
||||
payload = read_json(args.target_plan, max_bytes=1024 * 1024)
|
||||
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
|
||||
raise ValueError("Target plan requires schema_version 1")
|
||||
records = payload.get("targets")
|
||||
if not isinstance(records, list) or not 1 <= len(records) <= MAX_TARGETS:
|
||||
raise ValueError("Target plan must contain 1–256 exact targets")
|
||||
targets = []
|
||||
for record in records:
|
||||
if (not isinstance(record, dict) or not isinstance(record.get("root"), str)
|
||||
or not isinstance(record.get("url"), str)):
|
||||
raise ValueError("Each target plan entry requires root, issue and URL")
|
||||
target = resolve_target(Path(record["root"]), record.get("issue"), workspace_root,
|
||||
remote=args.remote, expected_url=record["url"], issue_id=record.get("issue_id"))
|
||||
for key in ("base_url", "owner", "repository"):
|
||||
if key in record and record[key] != getattr(target, key):
|
||||
raise ValueError("Target plan repository identity is inconsistent")
|
||||
targets.append(target)
|
||||
else:
|
||||
if args.root is None or args.issue is None:
|
||||
raise ValueError("An exact --root and --issue are required")
|
||||
targets = [resolve_target(args.root, args.issue, workspace_root, remote=args.remote)]
|
||||
if len({target.url for target in targets}) != len(targets):
|
||||
raise ValueError("Duplicate issue targets are not accepted")
|
||||
if len({target.base_url for target in targets}) != 1:
|
||||
raise ValueError("A credentialed target plan must be confined to one exact Gitea base URL")
|
||||
return targets
|
||||
|
||||
|
||||
def validate_receipt(payload: Any, workspace_root: Path) -> dict:
|
||||
if not isinstance(payload, dict) or type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
|
||||
raise ValueError("Evidence requires receipt schema_version 1")
|
||||
identifier(payload.get("run_id"))
|
||||
recorded_workspace = payload.get("workspace_root")
|
||||
if not isinstance(recorded_workspace, str) or Path(recorded_workspace).resolve() != workspace_root.resolve():
|
||||
raise ValueError("Evidence belongs to another workspace")
|
||||
fingerprint = payload.get("source_fingerprint")
|
||||
if not isinstance(fingerprint, str) or not re.fullmatch(r"[a-f0-9]{64}", fingerprint):
|
||||
raise ValueError("Evidence requires a recorded source fingerprint")
|
||||
if not isinstance(payload.get("status"), str) or payload["status"] not in STATUSES:
|
||||
raise ValueError("Unknown evidence status")
|
||||
_text(payload.get("generated_at"), maximum=128)
|
||||
if payload.get("finished_at") is not None:
|
||||
_text(payload["finished_at"], maximum=128)
|
||||
stages = payload.get("stages")
|
||||
if not isinstance(stages, list) or len(stages) > 1000:
|
||||
raise ValueError("Evidence requires a bounded stage list")
|
||||
seen = set()
|
||||
for stage in stages:
|
||||
if not isinstance(stage, dict):
|
||||
raise ValueError("Invalid evidence stage")
|
||||
identity = _text(stage.get("id"), maximum=256)
|
||||
if not identity or identity in seen or "\n" in identity:
|
||||
raise ValueError("Evidence stage IDs must be unique")
|
||||
seen.add(identity)
|
||||
if not isinstance(stage.get("status"), str) or stage["status"] not in STATUSES:
|
||||
raise ValueError("Unknown evidence stage status")
|
||||
code = stage.get("exit_code")
|
||||
if code is not None and type(code) is not int:
|
||||
raise ValueError("Evidence exit code must be an integer or null")
|
||||
duration = stage.get("duration_seconds")
|
||||
if duration is not None and (type(duration) not in {int, float} or not 0 <= duration < 31536000):
|
||||
raise ValueError("Invalid evidence stage duration")
|
||||
if stage.get("log_path") is not None:
|
||||
_text(stage["log_path"], maximum=4096)
|
||||
if stage["status"] == "passed" and code != 0:
|
||||
raise ValueError("Passed evidence stage must have exit code zero")
|
||||
if payload["status"] == "passed" and (not stages or any(stage["status"] != "passed" for stage in stages)):
|
||||
raise ValueError("Passed evidence is inconsistent with its stages")
|
||||
if payload["status"] == "passed" and payload.get("snapshot_verified") is not True:
|
||||
raise ValueError("Passed evidence requires a verified source snapshot")
|
||||
coverage_notes(stages)
|
||||
from .checkpoints import validate_checkpoint_receipt
|
||||
validate_checkpoint_receipt(payload)
|
||||
return payload
|
||||
|
||||
|
||||
def coverage_notes(stages: list[dict]) -> list[str]:
|
||||
"""Keep declared coverage limits visible without interpreting them as commands."""
|
||||
notes, total, size = [], 0, 0
|
||||
for stage in stages:
|
||||
values = stage.get("coverage_notes", [])
|
||||
if not isinstance(values, list):
|
||||
raise ValueError("Evidence coverage_notes must be a bounded list of strings")
|
||||
total += len(values)
|
||||
if total > MAX_COVERAGE_NOTES:
|
||||
raise ValueError("Evidence coverage_notes exceed their count bound")
|
||||
for value in values:
|
||||
text = _text(value, maximum=4096).strip()
|
||||
if not text:
|
||||
raise ValueError("Evidence coverage notes cannot be empty")
|
||||
size += len(text.encode())
|
||||
if size > MAX_NOTE_BYTES:
|
||||
raise ValueError("Evidence coverage_notes exceed their size bound")
|
||||
notes.append(text)
|
||||
return list(dict.fromkeys(notes))
|
||||
|
||||
|
||||
def evidence_record(value: str | None, args) -> dict | None:
|
||||
if not value:
|
||||
return None
|
||||
workspace_root = Path(args.workspace_root).resolve()
|
||||
external = value.endswith(".json") or "/" in value or "\\" in value
|
||||
if external:
|
||||
path = Path(value).expanduser().absolute()
|
||||
payload = read_json(path)
|
||||
origin = "external-unverified"
|
||||
else:
|
||||
from .runner import read_receipt
|
||||
identity = identifier(value)
|
||||
payload = read_receipt(workspace_root, args.state_dir, identity)
|
||||
path = state_root(workspace_root, args.state_dir) / "runs" / identity / "receipt.json"
|
||||
origin = "local-integrity-checked"
|
||||
receipt = validate_receipt(payload, workspace_root)
|
||||
from .workspace import load_project, source_fingerprint
|
||||
expected_project = str(args.project.resolve()) if getattr(args, "project", None) else None
|
||||
source_state = "not-compared"
|
||||
current = None
|
||||
if receipt.get("project_file") == expected_project:
|
||||
try:
|
||||
project = load_project(workspace_root, getattr(args, "project", None))
|
||||
if receipt.get("fingerprint_version") == FINGERPRINT_VERSION:
|
||||
from .inputs import InputSnapshotter
|
||||
scope = receipt.get("source_scope", {})
|
||||
if not isinstance(scope, dict) or not isinstance(scope.get("repos"), list):
|
||||
raise ValueError("Scoped evidence requires an explicit recorded repository scope")
|
||||
current = InputSnapshotter(project, workspace_root=workspace_root).source_snapshot(scope["repos"])["observed_source_fingerprint"]
|
||||
else:
|
||||
current = source_fingerprint(project)
|
||||
source_state = "matches-current" if current == receipt["source_fingerprint"] else "historical-source-differs"
|
||||
except (OSError, ValueError, subprocess.SubprocessError):
|
||||
source_state = "current-source-unavailable"
|
||||
else:
|
||||
source_state = "different-project-not-compared"
|
||||
return {"run_id": redact(receipt["run_id"]), "status": receipt["status"], "origin": origin,
|
||||
"receipt_path": redact(str(path)), "source_fingerprint": receipt["source_fingerprint"],
|
||||
"current_source_fingerprint": current, "source_state": source_state,
|
||||
"snapshot_verified": receipt.get("snapshot_verified") is True,
|
||||
"coverage_notes": coverage_notes(receipt["stages"]) + ([
|
||||
"Source comparison is limited to the recorded repository input scope; this is not a whole-workspace or artifact verification."
|
||||
] if receipt.get("fingerprint_version") == FINGERPRINT_VERSION and not receipt.get("source_scope", {}).get("complete_workspace") else []),
|
||||
"fingerprint_version": receipt.get("fingerprint_version"),
|
||||
"source_scope": receipt.get("source_scope"),
|
||||
"generated_at": redact(receipt["generated_at"]), "finished_at": redact(receipt["finished_at"]) if receipt.get("finished_at") else None,
|
||||
"stages": [{**{key: redact(stage[key]) if isinstance(stage.get(key), str) else stage.get(key)
|
||||
for key in ("id", "status", "exit_code", "duration_seconds", "log_path")},
|
||||
"coverage_notes": coverage_notes([stage])} for stage in receipt["stages"]],
|
||||
"attestation": "Receipt metadata is not an independent attestation, a live verification, or a completed module review."}
|
||||
|
||||
|
||||
def _structured_note(args) -> dict:
|
||||
payload = read_json(args.note_file, max_bytes=MAX_NOTE_BYTES) if args.note_file else {}
|
||||
if not isinstance(payload, dict) or set(payload) - {"summary", "next", "body"}:
|
||||
raise ValueError("Structured note accepts only summary[], next[] and body")
|
||||
result = {}
|
||||
for key, values in (("summary", args.note_summary), ("next", args.next_steps)):
|
||||
supplied = payload.get(key, [])
|
||||
if not isinstance(supplied, list) or len(supplied) + len(values) > 100:
|
||||
raise ValueError("Summary and next steps must be bounded lists")
|
||||
result[key] = [_text(item).strip() for item in [*supplied, *values]]
|
||||
body = _text(payload.get("body", ""), maximum=MAX_NOTE_BYTES)
|
||||
if args.body_file:
|
||||
# Reuse the bounded, no-symlink file reader by wrapping no content in code.
|
||||
from .common import reject_symlinks
|
||||
import stat
|
||||
reject_symlinks(args.body_file)
|
||||
descriptor = os.open(args.body_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
metadata = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > MAX_NOTE_BYTES:
|
||||
raise ValueError("Additional body must be a bounded regular file")
|
||||
text = handle.read(MAX_NOTE_BYTES + 1).decode("utf-8")
|
||||
body += "\n\n" + _text(text, maximum=MAX_NOTE_BYTES)
|
||||
result["body"] = body.strip()
|
||||
return result
|
||||
|
||||
|
||||
def _cell(value: Any) -> str:
|
||||
return html.escape(str(value if value is not None else "—")).replace("|", "|").replace("`", "`").replace("\n", " ")
|
||||
|
||||
|
||||
def render_note(note: dict, evidence: dict | None, target: NoteTarget, key: str) -> tuple[str, str]:
|
||||
identity = {"target": target.url, "key": identifier(key),
|
||||
"evidence": evidence["run_id"] if evidence else digest(note)}
|
||||
marker = MARKER_PREFIX + digest(identity) + " -->"
|
||||
lines = [marker, "## Development evidence", "", f"Issue: {target.url}", ""]
|
||||
for field, title in (("summary", "Summary"), ("next", "Next / remaining")):
|
||||
if note[field]:
|
||||
lines += [f"### {title}", "", *["- " + item for item in note[field]], ""]
|
||||
if note["body"]:
|
||||
lines += [note["body"], ""]
|
||||
if evidence:
|
||||
lines += ["### Recorded check evidence", "", f"Run: `{evidence['run_id']}`; reported result: **{evidence['status']}**.",
|
||||
f"Receipt origin: `{evidence['origin']}`; source comparison: `{evidence['source_state']}`.",
|
||||
f"Recorded source fingerprint: `{evidence['source_fingerprint']}`.",
|
||||
f"Finished: {_cell(evidence['finished_at'])}; receipt: `{_cell(evidence['receipt_path'])}`.", "",
|
||||
"| Stage | Reported status | Exit | Seconds | Local log reference |", "| --- | --- | --- | --- | --- |"]
|
||||
for stage in evidence["stages"]:
|
||||
lines.append("| " + " | ".join(_cell(stage.get(field)) for field in ("id", "status", "exit_code", "duration_seconds", "log_path")) + " |")
|
||||
if evidence.get("coverage_notes"):
|
||||
lines += ["", "### Coverage limitations / checks not included", "",
|
||||
"A passing recorded stage does not mean these omitted checks ran.", "",
|
||||
*["- " + _cell(value) for value in evidence["coverage_notes"]]]
|
||||
lines += ["", evidence["attestation"], "Local logs are referenced only; their content has not been read or uploaded.", ""]
|
||||
lines += ["This comment does not close the issue, complete its review, or change its checklist."]
|
||||
body = redact("\n".join(lines).rstrip() + "\n")
|
||||
if len(body.encode()) > MAX_NOTE_BYTES:
|
||||
raise ValueError("Rendered note exceeds its size bound")
|
||||
return marker, body
|
||||
|
||||
|
||||
def _token(env_file: Path | None) -> str:
|
||||
values = {}
|
||||
if env_file is not None:
|
||||
from .common import reject_symlinks
|
||||
reject_symlinks(env_file)
|
||||
import stat
|
||||
descriptor = os.open(env_file, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
metadata = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 65536:
|
||||
raise ValueError("Credential file must be a bounded regular file")
|
||||
content = handle.read(65537)
|
||||
if len(content) > 65536:
|
||||
raise ValueError("Credential file exceeds its size limit")
|
||||
for line in content.decode("utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
if line.startswith("export "):
|
||||
line = line[7:].strip()
|
||||
key, separator, value = line.partition("=")
|
||||
if not separator or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", key.strip()):
|
||||
raise ValueError("Invalid credential file format")
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}:
|
||||
value = value[1:-1]
|
||||
if key.strip() in values:
|
||||
raise ValueError("Duplicate credential file keys")
|
||||
values[key.strip()] = value
|
||||
token = os.environ.get("GITEA_TOKEN") or values.get("GITEA_TOKEN")
|
||||
if not token or len(token) > 8192 or any(char.isspace() for char in token):
|
||||
raise ValueError("GITEA_TOKEN is required for --apply; use the environment or --env-file")
|
||||
return token
|
||||
|
||||
|
||||
def make_client(target: NoteTarget, token: str):
|
||||
return GiteaClient(RepoTarget(target.base_url, target.owner, target.repository), token)
|
||||
|
||||
|
||||
def _issue(client, target: NoteTarget) -> dict:
|
||||
issue = client.request_json("GET", target.path)
|
||||
if (not isinstance(issue, dict) or issue.get("number") != target.issue or issue.get("html_url") != target.url
|
||||
or issue.get("pull_request") is not None):
|
||||
raise ValueError("Remote issue identity does not match the exact target")
|
||||
_positive(issue.get("id"))
|
||||
if target.issue_id is not None and issue["id"] != target.issue_id:
|
||||
raise ValueError("Remote issue ID changed from the target plan")
|
||||
return issue
|
||||
|
||||
|
||||
def _comments(client, target: NoteTarget) -> list[dict]:
|
||||
comments, seen = [], set()
|
||||
for page in range(1, 10001):
|
||||
values = client.request_json("GET", target.path + "/comments", query={"page": page, "limit": 50})
|
||||
if not isinstance(values, list):
|
||||
raise ValueError("Remote comment pagination did not return a list")
|
||||
if not values:
|
||||
return comments
|
||||
for comment in values:
|
||||
if not isinstance(comment, dict) or not isinstance(comment.get("body"), str):
|
||||
raise ValueError("Remote comment has an invalid shape")
|
||||
identity = _positive(comment.get("id"))
|
||||
if identity in seen:
|
||||
raise ValueError("Repeated comment pagination; cannot establish a complete duplicate check")
|
||||
seen.add(identity)
|
||||
comments.append(comment)
|
||||
if len(comments) > 100000:
|
||||
raise ValueError("Remote comments exceed the bounded duplicate-check limit")
|
||||
raise ValueError("Remote comment pagination did not terminate")
|
||||
|
||||
|
||||
def _existing(comments: list[dict], marker: str, body: str) -> dict | None:
|
||||
matches = [comment for comment in comments if marker in comment["body"]]
|
||||
if len(matches) > 1 or (matches and matches[0]["body"] != body):
|
||||
raise ValueError("Evidence marker collision; existing comments are preserved")
|
||||
return matches[0] if matches else None
|
||||
|
||||
|
||||
def _readback(client, target: NoteTarget, comment: dict, body: str) -> dict:
|
||||
identity = _positive(comment.get("id"))
|
||||
fresh = client.request_json("GET", repo_path(target.owner, target.repository, f"/issues/comments/{identity}"))
|
||||
issue_api_url = target.base_url + "/api/v1" + target.path
|
||||
if not isinstance(fresh, dict) or fresh.get("id") != identity or fresh.get("body") != body:
|
||||
raise ValueError("Posted comment read-back did not match")
|
||||
if not fresh.get("html_url") and not fresh.get("issue_url"):
|
||||
raise ValueError("Comment read-back has no issue binding")
|
||||
if fresh.get("issue_url") and fresh["issue_url"] != issue_api_url:
|
||||
raise ValueError("Comment read-back belongs to another issue")
|
||||
if fresh.get("html_url") and fresh["html_url"].split("#", 1)[0] != target.url:
|
||||
raise ValueError("Comment read-back URL belongs to another issue")
|
||||
return fresh
|
||||
|
||||
|
||||
def handle_note(args) -> dict:
|
||||
targets = _targets(args)
|
||||
note = _structured_note(args)
|
||||
evidence = evidence_record(args.evidence, args)
|
||||
if not evidence and not any((note["summary"], note["next"], note["body"])):
|
||||
raise ValueError("Provide evidence or a nonempty structured note")
|
||||
if args.retry_uncertain and not args.apply:
|
||||
raise ValueError("--retry-uncertain requires --apply")
|
||||
prepared = [(target, *render_note(note, evidence, target, args.key)) for target in targets]
|
||||
result = {"schema_version": 1, "operation": "issues.note", "apply": bool(args.apply),
|
||||
"evidence": evidence, "targets": [{**target.record(), "marker": marker, "body": body, "status": "would-post"} for target, marker, body in prepared],
|
||||
"summary": [f"{'Apply' if args.apply else 'Offline dry run'}: {len(targets)} exact issue target(s); issue bodies and states are preserved."]}
|
||||
if evidence and evidence["coverage_notes"]:
|
||||
result["summary"].append(f"Evidence has {len(evidence['coverage_notes'])} coverage limitation(s), retained in the note; omitted checks are not claimed as passed.")
|
||||
if not args.apply:
|
||||
return result
|
||||
token = _token(args.env_file)
|
||||
# Tokens loaded from an explicit file are not inserted into the process environment.
|
||||
for record in result["targets"]:
|
||||
if token in record["body"]:
|
||||
raise ValueError("A credential occurs in note content; refusing publication")
|
||||
state = state_root(Path(args.workspace_root), args.state_dir)
|
||||
with ExitStack() as stack:
|
||||
for target, marker, _body in sorted(prepared, key=lambda item: item[0].url):
|
||||
stack.enter_context(resource_lock(state / "locks", "issues.note:" + marker))
|
||||
clients, bindings, journals = {}, {}, {}
|
||||
try:
|
||||
# Validate every target and every existing marker before the first write.
|
||||
for target, marker, body in prepared:
|
||||
client = make_client(target, token)
|
||||
stack.callback(client.close)
|
||||
clients[target.url] = client
|
||||
bindings[target.url] = _issue(client, target)["id"]
|
||||
_existing(_comments(client, target), marker, body)
|
||||
journal_path = state / "issue-notes" / (hashlib.sha256(marker.encode()).hexdigest() + ".json")
|
||||
prior = read_json(journal_path) if journal_path.exists() else None
|
||||
if prior is not None and (not isinstance(prior, dict) or prior.get("schema_version") != 1
|
||||
or prior.get("target") != target.url or prior.get("body_digest") != digest(body)
|
||||
or prior.get("issue_id") != bindings[target.url]
|
||||
or prior.get("status") not in {"posting", "uncertain", "verified"}):
|
||||
raise ValueError("Local evidence journal identity collision; inspect before retrying")
|
||||
journals[target.url] = (journal_path, prior)
|
||||
for index, (target, marker, body) in enumerate(prepared):
|
||||
client = clients[target.url]
|
||||
output = result["targets"][index]
|
||||
journal_path, prior = journals[target.url]
|
||||
issue = _issue(client, target)
|
||||
if issue["id"] != bindings[target.url]:
|
||||
raise ValueError("Issue identity changed after preflight")
|
||||
existing = _existing(_comments(client, target), marker, body)
|
||||
if existing:
|
||||
verified = _readback(client, target, existing, body)
|
||||
output.update(status="existing-verified", comment_id=verified["id"])
|
||||
atomic_json(journal_path, {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "verified", "comment_id": verified["id"]})
|
||||
continue
|
||||
if prior and prior.get("status") in {"uncertain", "posting", "verified"} and not args.retry_uncertain:
|
||||
output["status"] = "uncertain-retry-required"
|
||||
result["_exit_code"] = 2
|
||||
result["summary"].append("An earlier POST is not visible after complete reconciliation; explicit --retry-uncertain is required. No further posts attempted.")
|
||||
break
|
||||
journal = {"schema_version": 1, "target": target.url, "issue_id": issue["id"], "body_digest": digest(body), "status": "posting"}
|
||||
atomic_json(journal_path, journal)
|
||||
try:
|
||||
posted = client.request_json("POST", target.path + "/comments", body={"body": body})
|
||||
verified = _readback(client, target, posted, body)
|
||||
unique = _existing(_comments(client, target), marker, body)
|
||||
if unique is None or unique["id"] != verified["id"]:
|
||||
raise ValueError("New comment is not uniquely visible in its issue")
|
||||
output.update(status="posted-verified", comment_id=verified["id"])
|
||||
except Exception:
|
||||
# POST is never replayed automatically, even after a timeout or bad response.
|
||||
atomic_json(journal_path, {**journal, "status": "uncertain"})
|
||||
try:
|
||||
observed = _existing(_comments(client, target), marker, body)
|
||||
verified = _readback(client, target, observed, body) if observed else None
|
||||
except Exception:
|
||||
verified = None
|
||||
if verified is None:
|
||||
output["status"] = "uncertain"
|
||||
result["_exit_code"] = 2
|
||||
result["summary"].append("POST/read-back could not be confirmed. No automatic retry and no further posts; reconcile this target before continuing.")
|
||||
break
|
||||
output.update(status="reconciled-verified", comment_id=verified["id"])
|
||||
atomic_json(journal_path, {**journal, "status": "verified", "comment_id": output["comment_id"]})
|
||||
except Exception:
|
||||
# HTTP response/error bodies may contain secrets; never return them to a report.
|
||||
result["_exit_code"] = 2
|
||||
result["summary"].append("Gitea validation or local journal checks failed; existing issues/comments were preserved. No further posts attempted.")
|
||||
finally:
|
||||
token = ""
|
||||
for record in result["targets"]:
|
||||
if record["status"] == "would-post":
|
||||
record["status"] = "not-attempted"
|
||||
result["summary"].append("; ".join(f"{record['url']}: {record['status']}" for record in result["targets"]))
|
||||
return result
|
||||
Executable
+1032
File diff suppressed because it is too large
Load Diff
Executable
+251
@@ -0,0 +1,251 @@
|
||||
"""Read-only run discovery and bounded provisional output, never passing evidence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import time
|
||||
|
||||
from .common import (
|
||||
atomic_json,
|
||||
identifier,
|
||||
now,
|
||||
read_json,
|
||||
redact,
|
||||
reject_symlinks,
|
||||
state_root,
|
||||
)
|
||||
|
||||
MAX_LIVE_BYTES = 65536
|
||||
MAX_HISTORY_ENTRIES = 10000
|
||||
MAX_HISTORY_SCAN = 500
|
||||
ANSI = re.compile(r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\))")
|
||||
CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
|
||||
|
||||
|
||||
def display_text(value: str) -> str:
|
||||
return redact(CONTROLS.sub("", ANSI.sub("", value)))
|
||||
|
||||
|
||||
def _closed_lines(value: bytes) -> bytes:
|
||||
boundary = value.rfind(b"\n")
|
||||
return value[: boundary + 1] if boundary >= 0 else b""
|
||||
|
||||
|
||||
def capture_text(snapshot, *, provisional: bool = False) -> str:
|
||||
"""Do not expose partial secret/context fragments at live or retention boundaries."""
|
||||
data = snapshot.stdout
|
||||
omitted = snapshot.omitted_stdout_bytes
|
||||
if omitted:
|
||||
head = _closed_lines(data[: snapshot.stdout_head_bytes])
|
||||
tail = data[snapshot.stdout_head_bytes :]
|
||||
# A rolling tail can start inside Authorization or a secret: drop that fragment.
|
||||
tail = tail.partition(b"\n")[2]
|
||||
if provisional:
|
||||
tail = _closed_lines(tail)
|
||||
return (
|
||||
display_text(head.decode("utf-8", errors="replace"))
|
||||
+ f"\n[Output truncated: {omitted} bytes omitted between retained head and tail; cut boundary lines are withheld.]\n"
|
||||
+ display_text(tail.decode("utf-8", errors="replace"))
|
||||
)
|
||||
if provisional:
|
||||
data = _closed_lines(data)
|
||||
return display_text(data.decode("utf-8", errors="replace"))
|
||||
|
||||
|
||||
def bounded_display(value: str, maximum: int, *, tail_only: bool = False) -> str:
|
||||
if type(maximum) is not int or maximum < 1:
|
||||
raise ValueError("Display bound must be a positive integer")
|
||||
encoded = value.encode("utf-8")
|
||||
if len(encoded) <= maximum:
|
||||
return value
|
||||
if tail_only:
|
||||
return encoded[-maximum:].decode("utf-8", errors="ignore")
|
||||
marker = b"\n[Display bound: middle omitted; retained beginning and final output follow.]\n"
|
||||
if maximum <= len(marker):
|
||||
return encoded[-maximum:].decode("utf-8", errors="ignore")
|
||||
available = max(0, maximum - len(marker))
|
||||
head = available // 2
|
||||
return (
|
||||
encoded[:head].decode("utf-8", errors="ignore")
|
||||
+ marker.decode()
|
||||
+ encoded[-(available - head) :].decode("utf-8", errors="ignore")
|
||||
)
|
||||
|
||||
|
||||
def write_live(log_path: Path, stage_id: str, snapshot, started: float) -> None:
|
||||
# Complete lines only, even at final callback: the separately finalized log
|
||||
# may include an unterminated final line after normal redaction.
|
||||
excerpt = bounded_display(
|
||||
capture_text(snapshot, provisional=True), MAX_LIVE_BYTES, tail_only=True
|
||||
)
|
||||
atomic_json(
|
||||
log_path.with_suffix(".live.json"),
|
||||
{
|
||||
"schema_version": 1,
|
||||
"run_id": log_path.parent.name,
|
||||
"stage_id": stage_id,
|
||||
"provisional": True,
|
||||
"updated_at": now(),
|
||||
"elapsed_seconds": round(time.monotonic() - started, 3),
|
||||
"output_truncated": snapshot.truncated,
|
||||
"omitted_bytes": snapshot.omitted_stdout_bytes,
|
||||
"excerpt": excerpt,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def read_live(
|
||||
workspace_root: Path, state_dir: Path | None, run_id: str, stage_id: str
|
||||
) -> dict:
|
||||
identifier(run_id)
|
||||
identifier(stage_id)
|
||||
path = (
|
||||
state_root(workspace_root, state_dir)
|
||||
/ "runs"
|
||||
/ run_id
|
||||
/ (stage_id + ".live.json")
|
||||
)
|
||||
if not path.exists() and not path.is_symlink():
|
||||
return {"provisional": True, "excerpt": "", "live_available": False}
|
||||
payload = read_json(path, max_bytes=MAX_LIVE_BYTES * 6 + 8192)
|
||||
if (
|
||||
not isinstance(payload, dict)
|
||||
or payload.get("schema_version") != 1
|
||||
or payload.get("run_id") != run_id
|
||||
or payload.get("stage_id") != stage_id
|
||||
or payload.get("provisional") is not True
|
||||
or not isinstance(payload.get("excerpt"), str)
|
||||
or len(payload["excerpt"].encode("utf-8")) > MAX_LIVE_BYTES
|
||||
or not isinstance(payload.get("updated_at"), str)
|
||||
or type(payload.get("elapsed_seconds")) not in {int, float}
|
||||
or payload["elapsed_seconds"] < 0
|
||||
or type(payload.get("output_truncated")) is not bool
|
||||
):
|
||||
raise ValueError("Invalid provisional log snapshot")
|
||||
return {
|
||||
"provisional": True,
|
||||
"live_available": True,
|
||||
"excerpt": display_text(payload["excerpt"]),
|
||||
"updated_at": payload.get("updated_at"),
|
||||
"elapsed_seconds": payload.get("elapsed_seconds"),
|
||||
"output_truncated": payload.get("output_truncated"),
|
||||
}
|
||||
|
||||
|
||||
def elapsed_seconds(receipt: dict) -> float | None:
|
||||
try:
|
||||
started = datetime.fromisoformat(receipt["generated_at"])
|
||||
finished = datetime.fromisoformat(receipt.get("finished_at") or now())
|
||||
return max(0, round((finished - started).total_seconds(), 3))
|
||||
except (KeyError, TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def list_runs(args) -> dict:
|
||||
from .runner import read_receipt
|
||||
|
||||
limit = getattr(args, "limit", 10)
|
||||
before = getattr(args, "before", None)
|
||||
if type(limit) is not int or not 1 <= limit <= 100:
|
||||
raise ValueError("Run history limit must be between 1 and 100")
|
||||
if before:
|
||||
identifier(before)
|
||||
base = state_root(args.workspace_root, args.state_dir) / "runs"
|
||||
reject_symlinks(base)
|
||||
names = []
|
||||
if base.exists():
|
||||
with os.scandir(base) as entries:
|
||||
for index, entry in enumerate(entries):
|
||||
if index >= MAX_HISTORY_ENTRIES:
|
||||
raise ValueError(
|
||||
"Run history exceeds its directory bound; archive old evidence explicitly before listing"
|
||||
)
|
||||
if not entry.name.startswith("."):
|
||||
identifier(entry.name)
|
||||
names.append(entry.name)
|
||||
names = sorted(
|
||||
(name for name in names if before is None or name < before), reverse=True
|
||||
)
|
||||
rows, examined, cursor = [], 0, None
|
||||
wanted_project = (
|
||||
str(args.project.resolve()) if getattr(args, "project", None) else None
|
||||
)
|
||||
for identity in names:
|
||||
examined += 1
|
||||
cursor = identity
|
||||
try:
|
||||
record = read_receipt(args.workspace_root, args.state_dir, identity)
|
||||
if wanted_project and record.get("project_file") != wanted_project:
|
||||
if examined >= MAX_HISTORY_SCAN:
|
||||
break
|
||||
continue
|
||||
rows.append(
|
||||
{
|
||||
"run_id": identity,
|
||||
"status": record["status"],
|
||||
"phase": record.get("phase"),
|
||||
"profile": record.get("profile"),
|
||||
"generated_at": record.get("generated_at"),
|
||||
"elapsed_seconds": elapsed_seconds(record),
|
||||
"snapshot_verified": record["snapshot_verified"],
|
||||
"passed_stages": sum(
|
||||
item["status"] == "passed" for item in record["stages"]
|
||||
),
|
||||
"total_stages": len(record["stages"]),
|
||||
}
|
||||
)
|
||||
except (OSError, ValueError) as exc:
|
||||
# Corrupt/newest evidence is visible, never silently replaced with an older pass.
|
||||
rows.append(
|
||||
{
|
||||
"run_id": display_text(identity),
|
||||
"status": "invalid",
|
||||
"error": display_text(str(exc)),
|
||||
}
|
||||
)
|
||||
if len(rows) >= limit or examined >= MAX_HISTORY_SCAN:
|
||||
break
|
||||
next_cursor = cursor if examined < len(names) else None
|
||||
lines = [
|
||||
f"{item['run_id']}: {item['status']}"
|
||||
+ (
|
||||
f" ({item.get('profile')}; {item.get('elapsed_seconds')}s)"
|
||||
if item["status"] != "invalid"
|
||||
else " — " + item["error"]
|
||||
)
|
||||
for item in rows
|
||||
]
|
||||
if not lines:
|
||||
lines = ["No matching check runs found; no verification is implied."]
|
||||
if next_cursor:
|
||||
lines.append("More history: use --before " + display_text(next_cursor))
|
||||
return {
|
||||
"runs": rows,
|
||||
"next_cursor": next_cursor,
|
||||
"examined": examined,
|
||||
"summary": lines,
|
||||
"_exit_code": 1 if any(item["status"] == "invalid" for item in rows) else 0,
|
||||
}
|
||||
|
||||
|
||||
def latest_run(args) -> dict:
|
||||
from argparse import Namespace
|
||||
from .runner import read_receipt, summarize
|
||||
|
||||
selected = list_runs(Namespace(**{**vars(args), "limit": 1}))
|
||||
if not selected["runs"]:
|
||||
return {
|
||||
"status": "not_found",
|
||||
"summary": selected["summary"],
|
||||
"next_cursor": selected["next_cursor"],
|
||||
"_exit_code": 1,
|
||||
}
|
||||
row = selected["runs"][0]
|
||||
if row["status"] == "invalid":
|
||||
raise ValueError(
|
||||
"Latest run is invalid; inspect run history instead of assuming an older pass"
|
||||
)
|
||||
return summarize(read_receipt(args.workspace_root, args.state_dir, row["run_id"]))
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
"""Bounded package metadata and conservative direct-Node test discovery."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shlex
|
||||
|
||||
from .common import digest, read_json, redact_argv, reject_symlinks
|
||||
|
||||
MAX_PACKAGE_BYTES = 1024 * 1024
|
||||
MAX_SCRIPTS = 512
|
||||
CORE_COMPONENT_SUITES = (
|
||||
"data-grid-actions",
|
||||
"dialog-focus",
|
||||
"explorer-tree",
|
||||
"icon-button",
|
||||
"layout-primitives",
|
||||
"mail-components",
|
||||
"metric-card",
|
||||
"page-layout",
|
||||
"workspace-layout",
|
||||
"people-picker",
|
||||
"password-field",
|
||||
"resource-access",
|
||||
"action-blocker",
|
||||
"documentation-help",
|
||||
"selection-list",
|
||||
"wysiwyg-editor",
|
||||
)
|
||||
CORE_RUNNER = "scripts/run-component-tests.mjs"
|
||||
|
||||
|
||||
def read_package(path: Path) -> dict:
|
||||
try:
|
||||
package = read_json(path, max_bytes=MAX_PACKAGE_BYTES)
|
||||
except (OSError, ValueError) as exc:
|
||||
raise ValueError(f"Cannot safely read package metadata: {path}") from exc
|
||||
if not isinstance(package, dict):
|
||||
raise ValueError(f"Package metadata must be an object: {path}")
|
||||
scripts = package.get("scripts", {})
|
||||
if not isinstance(scripts, dict) or len(scripts) > MAX_SCRIPTS:
|
||||
raise ValueError(f"Package scripts must be a bounded object: {path}")
|
||||
for name, command in scripts.items():
|
||||
if (
|
||||
not isinstance(name, str)
|
||||
or not 1 <= len(name) <= 128
|
||||
or any(char in name for char in "\0\r\n")
|
||||
):
|
||||
raise ValueError(f"Package script names must be bounded strings: {path}")
|
||||
if (
|
||||
not isinstance(command, str)
|
||||
or not 1 <= len(command) <= 8192
|
||||
or "\0" in command
|
||||
):
|
||||
raise ValueError(
|
||||
f"Package script commands must be bounded nonempty strings: {path}"
|
||||
)
|
||||
return package
|
||||
|
||||
|
||||
def core_component_alias(
|
||||
repo_name: str, name: str, command: str, package_path: Path
|
||||
) -> str | None:
|
||||
if repo_name != "govoplan-core" or package_path.parent.name != "webui":
|
||||
return None
|
||||
if name == "test:components" and command == f"node {CORE_RUNNER}":
|
||||
return "all"
|
||||
for suite in CORE_COMPONENT_SUITES:
|
||||
if name == "test:" + suite and command == f"node {CORE_RUNNER} {suite}":
|
||||
return suite
|
||||
return None
|
||||
|
||||
|
||||
def direct_node(command: str, package_path: Path) -> tuple[list[str] | None, str]:
|
||||
try:
|
||||
parts = shlex.split(command)
|
||||
except ValueError:
|
||||
return None, "Malformed command quoting; no command was guessed or executed."
|
||||
if not parts or parts[0] != "node":
|
||||
return (
|
||||
None,
|
||||
"Not a direct Node source test; use its explicitly reviewed owning workflow.",
|
||||
)
|
||||
offset = 2 if len(parts) > 1 and parts[1] == "--test" else 1
|
||||
if len(parts) != offset + 1 or not re.fullmatch(
|
||||
r"(?:scripts|tests)/[A-Za-z0-9_.-]+\.mjs", parts[offset]
|
||||
):
|
||||
return (
|
||||
None,
|
||||
"Compound command, flags or arguments are unsupported by scoped source discovery.",
|
||||
)
|
||||
if parts[offset] == CORE_RUNNER:
|
||||
return (
|
||||
None,
|
||||
"Only exact known Core component aliases belong to the shared UI batch.",
|
||||
)
|
||||
target = package_path.parent / parts[offset]
|
||||
try:
|
||||
reject_symlinks(target)
|
||||
if not target.is_file() or not target.resolve().is_relative_to(
|
||||
package_path.parent.resolve()
|
||||
):
|
||||
return None, "Declared test target is missing or escapes its package."
|
||||
except (OSError, ValueError):
|
||||
return None, "Declared test target is not a safe regular package file."
|
||||
return [
|
||||
"{node}",
|
||||
*parts[1:offset],
|
||||
str(target),
|
||||
], "Direct package-owned source test."
|
||||
|
||||
|
||||
def declared_tests(repo, package_path: Path) -> list[dict]:
|
||||
package = read_package(package_path)
|
||||
result = []
|
||||
for name, command in sorted(package.get("scripts", {}).items()):
|
||||
if name != "test" and not name.startswith("test:"):
|
||||
continue
|
||||
component = core_component_alias(repo.name, name, command, package_path)
|
||||
argv, reason = (
|
||||
direct_node(command, package_path)
|
||||
if component is None
|
||||
else (None, "Exact known Core component alias.")
|
||||
)
|
||||
result.append(
|
||||
{
|
||||
"repo": repo.name,
|
||||
"package_path": str(package_path),
|
||||
"name": name,
|
||||
"command_sha256": digest(command),
|
||||
"argv": redact_argv(argv) if argv else None,
|
||||
"component_suite": component,
|
||||
"reason": reason,
|
||||
"_argv": argv,
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def discovered_sources(repo) -> list[dict]:
|
||||
webui = repo.path / "webui"
|
||||
found = {}
|
||||
for folder in ("scripts", "tests"):
|
||||
for pattern in ("test-interface-pattern*.mjs", "*structure*.mjs"):
|
||||
for target in (webui / folder).glob(pattern):
|
||||
command = (
|
||||
"node "
|
||||
+ ("--test " if target.name.endswith(".test.mjs") else "")
|
||||
+ target.relative_to(webui).as_posix()
|
||||
)
|
||||
argv, reason = direct_node(command, webui / "package.json")
|
||||
found[str(target)] = {
|
||||
"repo": repo.name,
|
||||
"package_path": str(webui / "package.json"),
|
||||
"name": "file:" + target.relative_to(webui).as_posix(),
|
||||
"discovered": True,
|
||||
"command_sha256": digest(command),
|
||||
"argv": redact_argv(argv) if argv else None,
|
||||
"component_suite": None,
|
||||
"reason": reason,
|
||||
"_argv": argv,
|
||||
}
|
||||
if len(found) > MAX_SCRIPTS:
|
||||
raise ValueError(
|
||||
f"Too many discovered source checks in {repo.name}"
|
||||
)
|
||||
return [found[key] for key in sorted(found)]
|
||||
|
||||
|
||||
def source_stage_id(repo_name: str, argv: list[str]) -> str:
|
||||
stem = Path(argv[-1]).stem
|
||||
return f"{repo_name}.{stem}"[:110] + "." + digest(argv)[:12]
|
||||
Executable
+300
@@ -0,0 +1,300 @@
|
||||
"""Bounded subprocess capture shared by execution, discovery and environment probes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import os
|
||||
from pathlib import Path
|
||||
import selectors
|
||||
import signal
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
from typing import Callable
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OutputSnapshot:
|
||||
"""Immutable bounded output; raw bytes are decoded only at the display boundary."""
|
||||
|
||||
stdout: bytes
|
||||
stderr: bytes
|
||||
truncated: bool
|
||||
omitted_stdout_bytes: int
|
||||
omitted_stderr_bytes: int
|
||||
stdout_head_bytes: int
|
||||
stderr_head_bytes: int
|
||||
final: bool
|
||||
|
||||
def text(self, stream: str = "stdout") -> str:
|
||||
"""Render each retained segment separately, never joining cut UTF-8 sequences.
|
||||
|
||||
This is not secret redaction. Callers must redact before publishing or
|
||||
persisting a snapshot, including withholding partial live lines as needed.
|
||||
"""
|
||||
if stream not in {"stdout", "stderr"}:
|
||||
raise ValueError("Output stream must be stdout or stderr")
|
||||
data = getattr(self, stream)
|
||||
omitted = getattr(self, "omitted_" + stream + "_bytes")
|
||||
if not omitted:
|
||||
return data.decode("utf-8", errors="replace")
|
||||
split = getattr(self, stream + "_head_bytes")
|
||||
return (
|
||||
data[:split].decode("utf-8", errors="replace")
|
||||
+ f"\n[{omitted} output bytes omitted between retained segments]\n"
|
||||
+ data[split:].decode("utf-8", errors="replace")
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Capture:
|
||||
stdout: bytes
|
||||
stderr: bytes
|
||||
returncode: int
|
||||
status: str
|
||||
truncated: bool
|
||||
omitted_stdout_bytes: int = 0
|
||||
omitted_stderr_bytes: int = 0
|
||||
stdout_head_bytes: int = 0
|
||||
stderr_head_bytes: int = 0
|
||||
|
||||
def snapshot(self) -> OutputSnapshot:
|
||||
return OutputSnapshot(
|
||||
self.stdout,
|
||||
self.stderr,
|
||||
self.truncated,
|
||||
self.omitted_stdout_bytes,
|
||||
self.omitted_stderr_bytes,
|
||||
self.stdout_head_bytes,
|
||||
self.stderr_head_bytes,
|
||||
True,
|
||||
)
|
||||
|
||||
|
||||
class _OutputBuffer:
|
||||
def __init__(self, limit: int, mode: str):
|
||||
self.limit, self.mode, self.total = limit, mode, 0
|
||||
self.head, self.tail = bytearray(), bytearray()
|
||||
self.head_limit = limit if mode == "prefix" else limit // 2
|
||||
self.tail_limit = limit - self.head_limit
|
||||
|
||||
def append(self, data: bytes) -> None:
|
||||
self.total += len(data)
|
||||
count = min(len(data), self.head_limit - len(self.head))
|
||||
self.head.extend(data[:count])
|
||||
remaining = data[count:]
|
||||
if self.tail_limit and remaining:
|
||||
if len(remaining) >= self.tail_limit:
|
||||
self.tail[:] = remaining[-self.tail_limit :]
|
||||
else:
|
||||
excess = max(0, len(self.tail) + len(remaining) - self.tail_limit)
|
||||
del self.tail[:excess]
|
||||
self.tail.extend(remaining)
|
||||
|
||||
@property
|
||||
def omitted(self) -> int:
|
||||
return self.total - len(self.head) - len(self.tail)
|
||||
|
||||
def value(self) -> bytes:
|
||||
return bytes(self.head) + bytes(self.tail)
|
||||
|
||||
|
||||
def group_exists(pid: int) -> bool:
|
||||
try:
|
||||
os.killpg(pid, 0)
|
||||
return True
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
|
||||
|
||||
def stop_group(process: subprocess.Popen) -> None:
|
||||
try:
|
||||
os.killpg(process.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
return
|
||||
deadline = time.monotonic() + 0.5
|
||||
while time.monotonic() < deadline and group_exists(process.pid):
|
||||
process.poll()
|
||||
time.sleep(0.02)
|
||||
try:
|
||||
os.killpg(process.pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
process.wait(timeout=3)
|
||||
|
||||
|
||||
def run_captured(
|
||||
argv: list[str],
|
||||
*,
|
||||
cwd: Path | str | None = None,
|
||||
env: dict[str, str] | None = None,
|
||||
timeout: float = 30,
|
||||
max_stdout: int = 1024 * 1024,
|
||||
max_stderr: int = 65536,
|
||||
input_bytes: bytes | None = None,
|
||||
cancelled: threading.Event | None = None,
|
||||
merge_stderr: bool = False,
|
||||
terminate_on_limit: bool = True,
|
||||
capture_mode: str = "prefix",
|
||||
on_output: Callable[[OutputSnapshot], None] | None = None,
|
||||
) -> Capture:
|
||||
"""No shell, capped memory, finite deadline, owned process-group cleanup.
|
||||
|
||||
Prefix capture preserves probe semantics. Head/tail capture retains the
|
||||
beginning and actual latest output within the same byte bound. Optional
|
||||
callbacks receive a first-data snapshot, at most one dirty update per second,
|
||||
and a final snapshot. Callback failures propagate after owned-process cleanup.
|
||||
|
||||
This is not a sandbox: a deliberately detached new process session is outside
|
||||
the original process group. Only run trusted project commands.
|
||||
"""
|
||||
if capture_mode not in {"prefix", "head_tail"}:
|
||||
raise ValueError("Capture mode must be prefix or head_tail")
|
||||
if any(type(value) is not int or value < 0 for value in (max_stdout, max_stderr)):
|
||||
raise ValueError("Output bounds must be nonnegative integers")
|
||||
buffers = {
|
||||
"stdout": _OutputBuffer(max_stdout, capture_mode),
|
||||
"stderr": _OutputBuffer(max_stderr, capture_mode),
|
||||
}
|
||||
last_notified, notified_total = None, -1
|
||||
|
||||
def snapshot(final: bool) -> OutputSnapshot:
|
||||
out, err = buffers["stdout"], buffers["stderr"]
|
||||
return OutputSnapshot(
|
||||
out.value(),
|
||||
err.value(),
|
||||
bool(out.omitted or err.omitted),
|
||||
out.omitted,
|
||||
err.omitted,
|
||||
len(out.head),
|
||||
len(err.head),
|
||||
final,
|
||||
)
|
||||
|
||||
def notify(final: bool = False) -> None:
|
||||
nonlocal last_notified, notified_total
|
||||
if on_output is None:
|
||||
return
|
||||
total = sum(buffer.total for buffer in buffers.values())
|
||||
instant = time.monotonic()
|
||||
if final or (
|
||||
total > 0
|
||||
and total != notified_total
|
||||
and (last_notified is None or instant - last_notified >= 1)
|
||||
):
|
||||
on_output(snapshot(final))
|
||||
last_notified, notified_total = instant, total
|
||||
|
||||
process = subprocess.Popen(
|
||||
argv,
|
||||
cwd=cwd,
|
||||
env=env,
|
||||
stdin=subprocess.PIPE if input_bytes is not None else subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT if merge_stderr else subprocess.PIPE,
|
||||
start_new_session=True,
|
||||
)
|
||||
pending_input = memoryview(input_bytes or b"")
|
||||
selector = None
|
||||
try:
|
||||
selector = selectors.DefaultSelector()
|
||||
assert process.stdout is not None
|
||||
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
|
||||
if process.stderr:
|
||||
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
|
||||
if process.stdin:
|
||||
if pending_input:
|
||||
selector.register(process.stdin, selectors.EVENT_WRITE, "stdin")
|
||||
else:
|
||||
process.stdin.close()
|
||||
except BaseException:
|
||||
stop_group(process)
|
||||
if selector is not None:
|
||||
selector.close()
|
||||
for handle in (process.stdin, process.stdout, process.stderr):
|
||||
if handle and not handle.closed:
|
||||
handle.close()
|
||||
raise
|
||||
deadline = time.monotonic() + timeout
|
||||
exited_at = None
|
||||
state = None
|
||||
try:
|
||||
while selector.get_map() or process.poll() is None:
|
||||
if cancelled and cancelled.is_set():
|
||||
state = "interrupted"
|
||||
break
|
||||
if time.monotonic() >= deadline:
|
||||
state = "timed_out"
|
||||
break
|
||||
for key, _ in selector.select(
|
||||
timeout=min(0.1, max(0, deadline - time.monotonic()))
|
||||
):
|
||||
if key.data == "stdin":
|
||||
try:
|
||||
written = os.write(key.fd, pending_input[:4096])
|
||||
pending_input = pending_input[written:]
|
||||
except BrokenPipeError:
|
||||
pending_input = memoryview(b"")
|
||||
if not pending_input:
|
||||
selector.unregister(key.fileobj)
|
||||
key.fileobj.close()
|
||||
continue
|
||||
data = os.read(key.fd, 65536)
|
||||
if not data:
|
||||
selector.unregister(key.fileobj)
|
||||
continue
|
||||
target = buffers[key.data]
|
||||
target.append(data)
|
||||
if target.omitted and terminate_on_limit:
|
||||
state = "output_limit"
|
||||
break
|
||||
notify()
|
||||
if state:
|
||||
break
|
||||
if process.poll() is not None:
|
||||
exited_at = exited_at or time.monotonic()
|
||||
if selector.get_map() and time.monotonic() - exited_at >= 0.5:
|
||||
state = "leaked_process"
|
||||
break
|
||||
if state:
|
||||
stop_group(process)
|
||||
else:
|
||||
process.wait(timeout=3)
|
||||
# A child can redirect all output then outlive an otherwise successful parent.
|
||||
grace = time.monotonic() + 0.1
|
||||
while group_exists(process.pid) and time.monotonic() < grace:
|
||||
time.sleep(0.01)
|
||||
if group_exists(process.pid):
|
||||
state = "leaked_process"
|
||||
stop_group(process)
|
||||
else:
|
||||
state = "passed" if process.returncode == 0 else "failed"
|
||||
notify(final=True)
|
||||
finally:
|
||||
if process.poll() is None or group_exists(process.pid):
|
||||
stop_group(process)
|
||||
selector.close()
|
||||
for handle in (process.stdin, process.stdout, process.stderr):
|
||||
if handle and not handle.closed:
|
||||
handle.close()
|
||||
output = snapshot(True)
|
||||
return Capture(
|
||||
output.stdout,
|
||||
output.stderr,
|
||||
process.returncode,
|
||||
state,
|
||||
output.truncated,
|
||||
output.omitted_stdout_bytes,
|
||||
output.omitted_stderr_bytes,
|
||||
output.stdout_head_bytes,
|
||||
output.stderr_head_bytes,
|
||||
)
|
||||
|
||||
|
||||
def require_capture(argv: list[str], **kwargs) -> Capture:
|
||||
result = run_captured(argv, **kwargs)
|
||||
if result.status in {"timed_out", "interrupted", "output_limit", "leaked_process"}:
|
||||
raise ValueError(
|
||||
f"Bounded subprocess did not complete normally: {result.status}"
|
||||
)
|
||||
return result
|
||||
Executable
+366
@@ -0,0 +1,366 @@
|
||||
"""Headless adapter to the existing, receipt-bound release console lifecycle.
|
||||
|
||||
The ASGI application runs in this process; no HTTP socket or background server
|
||||
is started. Critical release orchestration remains in the existing service.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import importlib
|
||||
from pathlib import Path
|
||||
import re
|
||||
import secrets
|
||||
import sys
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
from .common import redact
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[3]
|
||||
RELEASE_ROOT = META_ROOT / "tools/release"
|
||||
_REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}\Z")
|
||||
_VERSION = re.compile(r"[0-9]+\.[0-9]+\.[0-9]+(?:[-+][0-9A-Za-z][0-9A-Za-z.-]*)?\Z")
|
||||
_RUN = re.compile(r"rr-(?:[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}|request-[0-9a-f]{64})\Z")
|
||||
_STEP = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:@-]{0,159}\Z")
|
||||
_REQUEST = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:@/-]{7,127}\Z")
|
||||
|
||||
|
||||
def _typed(pattern: re.Pattern[str], label: str):
|
||||
def parse(value: str) -> str:
|
||||
if not pattern.fullmatch(value):
|
||||
raise argparse.ArgumentTypeError(f"Invalid {label}.")
|
||||
return value
|
||||
return parse
|
||||
|
||||
|
||||
def _limit(value: str) -> int:
|
||||
try:
|
||||
result = int(value)
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError("Limit must be an integer from 1 to 100.") from exc
|
||||
if not 1 <= result <= 100:
|
||||
raise argparse.ArgumentTypeError("Limit must be an integer from 1 to 100.")
|
||||
return result
|
||||
|
||||
|
||||
def _planning_options(parser: argparse.ArgumentParser, *, selection: bool) -> None:
|
||||
if selection:
|
||||
parser.add_argument("--repo", action="append", default=[], type=_typed(_REPOSITORY, "repository name"))
|
||||
parser.add_argument("--repo-version", action="append", default=[], metavar="REPO=VERSION")
|
||||
parser.add_argument("--target-version", type=_typed(_VERSION, "target version"))
|
||||
parser.add_argument("--channel", default="stable")
|
||||
parser.add_argument("--online", action="store_true", help="Allow the existing remote/catalog checks.")
|
||||
parser.add_argument("--remote-tags", action="store_true", help="Explicitly inspect remote Git tags.")
|
||||
parser.add_argument("--public-catalog", action="store_true", help="Explicitly inspect the public catalog.")
|
||||
parser.add_argument("--include-migrations", action="store_true", help="Run migration audits; never applies migrations.")
|
||||
|
||||
|
||||
def register(subparsers: Any) -> None:
|
||||
"""Register release commands without importing FastAPI, HTTPX or Core."""
|
||||
parser = subparsers.add_parser("release", help="Plan and operate durable GovOPlaN release runs.")
|
||||
commands = parser.add_subparsers(dest="release_command", required=True)
|
||||
for name in ("plan", "status", "create"):
|
||||
command = commands.add_parser(name)
|
||||
_planning_options(command, selection=name != "status")
|
||||
if name == "status":
|
||||
command.add_argument("--include-website", action="store_true")
|
||||
if name == "create":
|
||||
command.add_argument("--request-id", required=True, type=_typed(_REQUEST, "request ID"))
|
||||
command.add_argument("--apply", action="store_true", help="Persist the frozen run; otherwise preview only.")
|
||||
command.set_defaults(handler=handle)
|
||||
|
||||
listing = commands.add_parser("list", help="List bounded, workspace-scoped durable run history.")
|
||||
listing.add_argument("--limit", type=_limit, default=20)
|
||||
listing.add_argument("--cursor")
|
||||
listing.set_defaults(handler=handle)
|
||||
for name in ("show", "preview", "execute", "resume", "retry", "reconcile"):
|
||||
command = commands.add_parser(name)
|
||||
command.add_argument("run_id", type=_typed(_RUN, "run ID"))
|
||||
if name in {"preview", "execute", "retry", "reconcile"}:
|
||||
command.add_argument("step_id", type=_typed(_STEP, "step ID"))
|
||||
if name in {"execute", "resume", "retry", "reconcile"}:
|
||||
command.add_argument("--request-id", required=True, type=_typed(_REQUEST, "request ID"))
|
||||
command.add_argument("--apply", action="store_true", help="Apply this explicit durable transition; otherwise preview only.")
|
||||
if name in {"execute", "reconcile"}:
|
||||
command.add_argument("--confirm", default="", help="Exact confirmation required by the existing release service.")
|
||||
if name == "execute":
|
||||
command.add_argument("--signing-key", action="append", default=[], metavar="KEY_ID=PRIVATE_KEY_FILE")
|
||||
if name == "reconcile":
|
||||
command.add_argument("--outcome", required=True, choices=("effect_absent", "effect_succeeded", "unresolved"))
|
||||
command.set_defaults(handler=handle)
|
||||
|
||||
|
||||
def _selection(args: argparse.Namespace) -> tuple[list[str], dict[str, str]]:
|
||||
versions: dict[str, str] = {}
|
||||
for item in args.repo_version:
|
||||
repo, separator, version = item.partition("=")
|
||||
if not separator or not _REPOSITORY.fullmatch(repo) or not _VERSION.fullmatch(version):
|
||||
raise ValueError("--repo-version must be REPO=VERSION with a valid repository and version.")
|
||||
if repo in versions and versions[repo] != version:
|
||||
raise ValueError(f"Conflicting target versions for {repo}.")
|
||||
versions[repo] = version
|
||||
repos = list(dict.fromkeys([*args.repo, *versions]))
|
||||
if not repos:
|
||||
raise ValueError("Select at least one --repo or --repo-version explicitly.")
|
||||
for repo in repos:
|
||||
if repo not in versions and args.target_version:
|
||||
versions[repo] = args.target_version
|
||||
if args.release_command == "create" and any(repo not in versions for repo in repos):
|
||||
raise ValueError("Creating a run requires an explicit version for every selected repository.")
|
||||
return repos, versions
|
||||
|
||||
|
||||
def _planning_query(args: argparse.Namespace) -> dict[str, Any]:
|
||||
return {
|
||||
"channel": args.channel,
|
||||
"online": args.online,
|
||||
"remote_tags": args.remote_tags,
|
||||
"public_catalog": args.public_catalog or args.online,
|
||||
"include_migrations": args.include_migrations,
|
||||
**({"target_version": args.target_version} if args.target_version else {}),
|
||||
}
|
||||
|
||||
|
||||
def _load_application(args: argparse.Namespace) -> tuple[Any, str]:
|
||||
# The generic CLI/help path stays dependency-light. The backend still checks
|
||||
# its operator-controlled runtime and registered source origins on apply.
|
||||
for name, loaded in list(sys.modules.items()):
|
||||
if name in {"server", "govoplan_release"} or name.startswith(("server.", "govoplan_release.")):
|
||||
expected = RELEASE_ROOT / ("server" if name.startswith("server") else "govoplan_release")
|
||||
source = getattr(loaded, "__file__", None)
|
||||
if not isinstance(source, str) or not Path(source).resolve().is_relative_to(expected.resolve()):
|
||||
raise ValueError("A foreign module shadows the trusted GovOPlaN release service.")
|
||||
# Reprioritize even when the path was added previously below an unrelated
|
||||
# working directory. Validate cached packages before importing any submodule.
|
||||
sys.path[:] = [str(RELEASE_ROOT), *(path for path in sys.path if path != str(RELEASE_ROOT))]
|
||||
module = importlib.import_module("server.app")
|
||||
if Path(module.__file__).resolve() != (RELEASE_ROOT / "server/app.py").resolve():
|
||||
raise ValueError("A foreign server.app module shadows the GovOPlaN release service.")
|
||||
token = secrets.token_urlsafe(32)
|
||||
state_dir = getattr(args, "state_dir", None)
|
||||
app = module.create_app(
|
||||
workspace_root=Path(args.workspace_root).expanduser().resolve(),
|
||||
token=token,
|
||||
run_state_root=Path(state_dir) / "release-console" if state_dir is not None else None,
|
||||
)
|
||||
return app, token
|
||||
|
||||
|
||||
def _error_detail(payload: Any) -> str:
|
||||
detail = payload.get("detail") if isinstance(payload, dict) else None
|
||||
if isinstance(detail, str):
|
||||
return detail
|
||||
if isinstance(detail, list):
|
||||
# Validation input may contain signing-key arguments. Never echo it.
|
||||
return "; ".join(
|
||||
".".join(str(part) for part in item.get("loc", [])) + ": " + str(item.get("msg", "Invalid request"))
|
||||
for item in detail if isinstance(item, dict)
|
||||
)
|
||||
return "The release service did not return a valid successful response."
|
||||
|
||||
|
||||
class _ServiceError(Exception):
|
||||
def __init__(self, status: int, payload: Any):
|
||||
super().__init__(_error_detail(payload))
|
||||
self.status = status
|
||||
|
||||
|
||||
def _brief(value: Any, *, limit: int = 240) -> str:
|
||||
"""Bound display-only fields; never serialize an executor or its arguments."""
|
||||
if not isinstance(value, (str, int, float, bool)):
|
||||
return "unknown"
|
||||
text = " ".join(redact(str(value)).split())
|
||||
return text if len(text) <= limit else text[:limit - 1] + "…"
|
||||
|
||||
|
||||
def _status(payload: dict[str, Any]) -> str:
|
||||
status = payload.get("status")
|
||||
for key in ("summary", "state"):
|
||||
if isinstance(payload.get(key), dict):
|
||||
status = payload[key].get("status", status)
|
||||
if isinstance(payload.get("state_step"), dict):
|
||||
status = payload["state_step"].get("state", status)
|
||||
# Return the semantic value unchanged: display redaction must never affect
|
||||
# failure exit codes, even when an environment secret happens to equal it.
|
||||
return status if isinstance(status, str) else "ok"
|
||||
|
||||
|
||||
def _summary_lines(name: str, payload: dict[str, Any], note: str | None = None) -> list[str]:
|
||||
"""Compact, bounded projection; the unchanged JSON result retains details."""
|
||||
lines = [f"Release {name}: {_brief(_status(payload))}."]
|
||||
if note:
|
||||
lines.append(note)
|
||||
plan = payload.get("immutable", {}).get("plan", {}) if isinstance(payload.get("immutable"), dict) else payload
|
||||
if not isinstance(plan, dict):
|
||||
plan = {}
|
||||
if isinstance(plan.get("source_preflight_ready"), bool):
|
||||
prefix = "Frozen plan source" if "immutable" in payload else "Source"
|
||||
lines.append(f"{prefix} preflight ready: {str(plan['source_preflight_ready']).lower()}.")
|
||||
units = plan.get("units", [])
|
||||
if isinstance(units, list):
|
||||
for unit in units[:12]:
|
||||
if isinstance(unit, dict):
|
||||
lines.append(f"{_brief(unit.get('repo'))}: {_brief(unit.get('status', 'planned'))}; target {_brief(unit.get('target_version'))}.")
|
||||
if len(units) > 12:
|
||||
lines.append(f"{len(units) - 12} more selected repositories; use --json for every repository.")
|
||||
dashboard = payload.get("summary")
|
||||
if isinstance(dashboard, dict):
|
||||
counts = [f"{dashboard[key]} {label}" for key, label in (
|
||||
("repository_count", "repositories"), ("missing_count", "missing"),
|
||||
("dirty_count", "dirty"), ("ahead_count", "ahead"),
|
||||
("behind_count", "behind"), ("error_count", "errors"),
|
||||
) if isinstance(dashboard.get(key), int)]
|
||||
if counts:
|
||||
lines.append("Repository status: " + ", ".join(counts) + ".")
|
||||
findings = plan.get("gate_findings", payload.get("collection_errors", []))
|
||||
if isinstance(findings, list):
|
||||
for finding in findings[:4]:
|
||||
if isinstance(finding, dict):
|
||||
scope = f" ({_brief(finding['repo'])})" if finding.get("repo") else ""
|
||||
lines.append(f"Gate {_brief(finding.get('code'))}{scope}: {_brief(finding.get('message'))}")
|
||||
if len(findings) > 4:
|
||||
lines.append(f"{len(findings) - 4} more gate findings; use --json for details.")
|
||||
state = payload.get("state", {})
|
||||
steps = state.get("steps", []) if isinstance(state, dict) else []
|
||||
if isinstance(payload.get("state_step"), dict):
|
||||
steps = [payload["state_step"]]
|
||||
if isinstance(steps, list) and steps:
|
||||
counts: dict[str, int] = {}
|
||||
for step in steps:
|
||||
if isinstance(step, dict):
|
||||
status = _brief(step.get("state", "unknown"))
|
||||
counts[status] = counts.get(status, 0) + 1
|
||||
lines.append("Steps: " + ", ".join(f"{count} {state}" for state, count in sorted(counts.items())) + ".")
|
||||
relevant = [step for step in steps if isinstance(step, dict) and step.get("state") != "succeeded"]
|
||||
for step in relevant[:3]:
|
||||
lines.append(f"Step {_brief(step.get('id'))}: {_brief(step.get('state'))}." +
|
||||
(f" {_brief(step['disabled_reason'])}" if step.get("disabled_reason") else ""))
|
||||
execution = payload.get("execution_result")
|
||||
if isinstance(execution, dict):
|
||||
lines.append(f"Executor result: {_brief(execution.get('status', 'recorded'))}.")
|
||||
recommendation = payload.get("recommended_next", plan.get("recommended_action"))
|
||||
if isinstance(recommendation, dict) and recommendation:
|
||||
step = f" [{_brief(recommendation['step_id'])}]" if recommendation.get("step_id") else ""
|
||||
lines.append(f"Next: {_brief(recommendation.get('id'))}{step} — {_brief(recommendation.get('title'))}.")
|
||||
if recommendation.get("remediation"):
|
||||
lines.append(_brief(recommendation["remediation"]))
|
||||
runs = payload.get("runs")
|
||||
if isinstance(runs, list):
|
||||
lines.append(f"{len(runs)} release runs in this page.")
|
||||
for run in runs[:12]:
|
||||
if isinstance(run, dict):
|
||||
lines.append(f"{_brief(run.get('run_id'))}: {_brief(_status(run))}.")
|
||||
if payload.get("next_cursor"):
|
||||
lines.append("More history available; use --json for the next cursor.")
|
||||
return lines
|
||||
|
||||
|
||||
async def _run(args: argparse.Namespace) -> dict[str, Any]:
|
||||
# Resolve CLI-only validation before loading the service or creating state.
|
||||
if getattr(args, "project", None) is not None:
|
||||
raise ValueError("Release uses the authoritative GovOPlaN catalog and does not accept --project overrides; select the registered --workspace-root instead.")
|
||||
name = args.release_command
|
||||
selection = _selection(args) if name in {"plan", "create"} else None
|
||||
keys = getattr(args, "signing_key", [])
|
||||
if len(keys) > 8 or any(not re.fullmatch(r"[A-Za-z0-9._-]{1,128}=.+", key) or "-----BEGIN" in key or "\n" in key or "\r" in key or len(key) > 4096 for key in keys):
|
||||
raise ValueError("Provide at most eight --signing-key KEY_ID=PRIVATE_KEY_FILE arguments, never key material.")
|
||||
import httpx
|
||||
app, token = _load_application(args)
|
||||
metadata = {
|
||||
"workspace_root": str(app.state.workspace_root),
|
||||
"state_location": str(app.state.release_runs.root),
|
||||
"candidate_location": str(app.state.release_candidate_root),
|
||||
}
|
||||
transport = httpx.ASGITransport(app=app, raise_app_exceptions=False)
|
||||
async with httpx.AsyncClient(
|
||||
transport=transport, base_url="http://govoplan-devkit.invalid",
|
||||
headers={"X-Release-Console-Token": token}, timeout=None,
|
||||
follow_redirects=False,
|
||||
) as client:
|
||||
async def request(method: str, path: str, **kwargs: Any) -> dict[str, Any]:
|
||||
response = await client.request(method, path, **kwargs)
|
||||
try:
|
||||
payload = response.json()
|
||||
except ValueError:
|
||||
payload = None
|
||||
if response.status_code >= 400 or not isinstance(payload, dict):
|
||||
raise _ServiceError(response.status_code, payload)
|
||||
return payload
|
||||
|
||||
def result(payload: dict[str, Any], *, dry_run: bool = False, summary: str | None = None) -> dict[str, Any]:
|
||||
status = _status(payload)
|
||||
execution_failed = isinstance(payload.get("execution_result"), dict) and payload["execution_result"].get("status") == "failed"
|
||||
return {
|
||||
**metadata, "operation": name, "dry_run": dry_run,
|
||||
"result": payload,
|
||||
"_exit_code": 1 if execution_failed or status in {"blocked", "failed", "interrupted"} else 0,
|
||||
"summary": [*_summary_lines(name, payload, summary), f"Durable state: {metadata['state_location']}"],
|
||||
}
|
||||
|
||||
async def preview(run: dict[str, Any], step_id: str) -> dict[str, Any]:
|
||||
plan = run.get("immutable", {}).get("plan", {})
|
||||
plan_step = next((step for step in plan.get("dry_run_steps", []) if step.get("id") == step_id), None)
|
||||
state_step = next((step for step in run.get("state", {}).get("steps", []) if step.get("id") == step_id), None)
|
||||
if plan_step is None or state_step is None:
|
||||
raise _ServiceError(404, {"detail": "Release run step was not found."})
|
||||
if state_step.get("executor", {}).get("kind") == "catalog_publish":
|
||||
return await request("POST", f"{run_path}/steps/{quote(step_id, safe='')}/preview", json={"remote": "origin"})
|
||||
return {
|
||||
"run_id": run["run_id"], "plan_step": plan_step, "state_step": state_step,
|
||||
"note": "Frozen-plan inspection only; no executor was called and no live preflight is claimed.",
|
||||
}
|
||||
|
||||
if name == "status":
|
||||
payload = await request("GET", "/api/dashboard", params={**_planning_query(args), "include_website": args.include_website})
|
||||
return result(payload)
|
||||
if name == "list":
|
||||
params = {"limit": args.limit, **({"cursor": args.cursor} if args.cursor else {})}
|
||||
return result(await request("GET", "/api/release-runs", params=params))
|
||||
if name in {"plan", "create"}:
|
||||
assert selection is not None
|
||||
repos, versions = selection
|
||||
query = {
|
||||
**_planning_query(args), "repos": ",".join(repos),
|
||||
"repo_versions": ",".join(f"{repo}={version}" for repo, version in versions.items()),
|
||||
}
|
||||
if name == "plan" or not args.apply:
|
||||
payload = await request("GET", "/api/selective-plan", params=query)
|
||||
return result(payload, dry_run=True, summary="Release plan inspected; no run was created and no release step executed.")
|
||||
body = {key: value for key, value in _planning_query(args).items() if key != "target_version"}
|
||||
payload = await request("POST", "/api/release-runs", json={**body, "request_id": args.request_id, "repo_versions": versions})
|
||||
return result(payload)
|
||||
|
||||
run_path = "/api/release-runs/" + quote(args.run_id, safe="")
|
||||
if name == "show":
|
||||
return result(await request("GET", run_path))
|
||||
if name == "preview" or not args.apply:
|
||||
run = await request("GET", run_path)
|
||||
payload = await preview(run, args.step_id) if name in {"preview", "execute"} else run
|
||||
return result(payload, dry_run=True, summary=f"Release {name} inspected; no durable transition or executor was invoked.")
|
||||
body = {"request_id": args.request_id}
|
||||
if name in {"execute", "reconcile"}:
|
||||
body["confirm"] = args.confirm
|
||||
if name == "execute":
|
||||
body.update({"remote": "origin", "signing_keys": keys})
|
||||
if name == "reconcile":
|
||||
body["outcome"] = args.outcome
|
||||
path = f"{run_path}/resume" if name == "resume" else f"{run_path}/steps/{quote(args.step_id, safe='')}/{name}"
|
||||
return result(await request("POST", path, json=body))
|
||||
|
||||
|
||||
def handle(args: argparse.Namespace) -> dict[str, Any]:
|
||||
"""Return the common devkit JSON/summary envelope; never retry mutations."""
|
||||
try:
|
||||
return asyncio.run(_run(args))
|
||||
except _ServiceError as exc:
|
||||
summary = [f"Release {args.release_command} failed (HTTP {exc.status}): {exc}"]
|
||||
if args.release_command in {"create", "execute", "resume", "retry", "reconcile"}:
|
||||
summary.append("No automatic retry occurred. Inspect the run; reuse the same request ID for a known replay, or resume/reconcile an uncertain effect before a new attempt.")
|
||||
return {"_exit_code": 1, "status": "error", "http_status": exc.status, "summary": summary}
|
||||
except ModuleNotFoundError as exc:
|
||||
return {"_exit_code": 2, "status": "unavailable", "summary": [f"Release commands need the GovOPlaN development dependencies ({exc.name} is unavailable)."]}
|
||||
except ValueError as exc:
|
||||
return {"_exit_code": 2, "status": "invalid", "summary": [str(exc)]}
|
||||
Executable
+244
@@ -0,0 +1,244 @@
|
||||
"""Local, source-derived review guidance. Gitea remains the only review state log."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .common import atomic_json, now, read_json, redact, reject_symlinks
|
||||
from .issues import coverage_notes, evidence_record, _base_url, _name, _positive
|
||||
from .workspace import inspect_repository, load_project, selected_repositories, source_fingerprint
|
||||
|
||||
MAX_SOURCES = 10000
|
||||
MANUAL_CHECKS = (
|
||||
("surfaces", "Confirm every route, pane, dialog, settings surface, widget, public form and contributed interface; source discovery is only a starting list."),
|
||||
("display-edit", "Show compact readable data first; edit coherent groups in scoped dialogs. Record a reason for a deliberate bulk editing mode or other exception."),
|
||||
("help", "Check documentation books beside meaningful visible headings/labels, including widgets, loading and configuration states; preserve optional Docs fallback."),
|
||||
("actions", "Check predictable action order, Save/Cancel, unsaved drafts, destructive consequences and truthful unavailable-action explanations."),
|
||||
("geometry", "Check shared cards/tables/dialogs, column resizing, pagination, long data, narrow layouts, zoom and the reported wide-window configuration."),
|
||||
("states", "Exercise loading, empty, partial, error, stale, conflict and permission-denied states; verify no accidental mutation on read or cancel."),
|
||||
("accessibility", "Exercise keyboard order, visible focus, accessible names, dialog focus restoration and non-color-only feedback."),
|
||||
("language-docs", "Review English and German, long headings and module-owned user/admin documentation for every changed workflow and limitation."),
|
||||
("boundaries", "Exercise tenant/authorization boundaries and optional-module absence; inspect headless modules' contributed interfaces rather than marking them complete automatically."),
|
||||
("propagation", "Record applied principle revision, exceptions, fixes, evidence and remaining work in the module issue; propagate new rules to already-reviewed modules through the central issue."),
|
||||
)
|
||||
|
||||
|
||||
def register(subparsers) -> None:
|
||||
parser = subparsers.add_parser("review", help="Assemble a module's local UI-review guidance (does not complete a review)")
|
||||
parser.add_argument("module", help="Repository name, alias, or review inventory scope ID")
|
||||
parser.add_argument("bundle_module", nargs="?", help="Also accepts: review bundle MODULE")
|
||||
parser.add_argument("--profile", default="ui", help="Plan this registered check profile, without running it")
|
||||
parser.add_argument("--evidence", help="Existing local run ID or explicit receipt JSON path")
|
||||
parser.add_argument("--output", type=Path, help="Optional local JSON artifact; not a progress tracker")
|
||||
parser.set_defaults(handler=handle_review)
|
||||
|
||||
|
||||
def _project_path(root: Path, value: str) -> Path:
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("Review inventory/principles paths must be strings")
|
||||
raw = Path(value)
|
||||
if raw.is_absolute() or ".." in raw.parts:
|
||||
raise ValueError("Review configuration paths must be relative within the workspace")
|
||||
path = root / raw
|
||||
reject_symlinks(path)
|
||||
if not path.resolve().is_relative_to(root.resolve()):
|
||||
raise ValueError("Review input escapes the workspace")
|
||||
return path
|
||||
|
||||
|
||||
def _link(record: dict) -> dict:
|
||||
if not isinstance(record, dict):
|
||||
raise ValueError("Review issue link must be an object")
|
||||
repo = _name(record.get("repository"))
|
||||
number = _positive(record.get("number"))
|
||||
url = record.get("url")
|
||||
if not isinstance(url, str):
|
||||
raise ValueError("Review issue URL is missing")
|
||||
parsed = urlsplit(_base_url(url))
|
||||
parts = parsed.path.rstrip("/").split("/")
|
||||
if len(parts) < 5 or parts[-3:] != [repo, "issues", str(number)]:
|
||||
raise ValueError("Review issue URL does not match its repository and number")
|
||||
_name(parts[-4])
|
||||
# Snapshot status/operation fields are deliberately not projected as live state.
|
||||
return {"repository": repo, "number": number, "url": url}
|
||||
|
||||
|
||||
def _issue_inventory(path: Path | None) -> tuple[list[dict], dict | None]:
|
||||
if path is None or not path.exists():
|
||||
return [], None
|
||||
payload = read_json(path, max_bytes=2 * 1024 * 1024)
|
||||
if not isinstance(payload, dict) or payload.get("schema_version") != 1:
|
||||
raise ValueError("Review issue inventory requires schema_version 1")
|
||||
rows = payload.get("issues")
|
||||
if not isinstance(rows, list) or len(rows) > 1024:
|
||||
raise ValueError("Review issue inventory must contain a bounded issues list")
|
||||
issues, scopes, urls = [], set(), set()
|
||||
for row in rows:
|
||||
link = _link(row)
|
||||
scope = row.get("scope_id")
|
||||
if not isinstance(scope, str) or not re.fullmatch(r"[A-Za-z0-9_:.-]{1,128}", scope) or scope in scopes or link["url"] in urls:
|
||||
raise ValueError("Review issue inventory has an invalid or duplicate scope/issue")
|
||||
scopes.add(scope)
|
||||
urls.add(link["url"])
|
||||
issues.append({**link, "scope_id": scope, "name": str(row.get("name", scope)), "kind": str(row.get("kind", "unspecified"))})
|
||||
epic = _link(payload["epic"]) if payload.get("epic") else None
|
||||
return issues, epic
|
||||
|
||||
|
||||
def _read_text(path: Path) -> str:
|
||||
reject_symlinks(path)
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
metadata = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > 1024 * 1024:
|
||||
raise ValueError("Principles must be a bounded regular text file")
|
||||
encoded = handle.read(1024 * 1024 + 1)
|
||||
if len(encoded) > 1024 * 1024:
|
||||
raise ValueError("Principles exceed the file-size bound")
|
||||
return encoded.decode("utf-8")
|
||||
|
||||
|
||||
def _principles(path: Path | None) -> dict:
|
||||
if path is None or not path.exists():
|
||||
return {"path": str(path) if path else None, "available": False, "revision": None, "rules": []}
|
||||
text = _read_text(path)
|
||||
revision = re.search(r"\bUI-\d{4}-\d{2}-\d{2}\b", text)
|
||||
rules = [{"id": match.group(1), "title": match.group(2).strip()}
|
||||
for match in re.finditer(r"^##\s+(UI-\d{2})\s*[—–:-]\s*(.+)$", text, re.MULTILINE)]
|
||||
return {"path": str(path), "available": True, "revision": revision.group(0) if revision else None,
|
||||
"content_sha256": hashlib.sha256(text.encode()).hexdigest(), "rules": rules}
|
||||
|
||||
|
||||
def source_inventory(root: Path) -> dict:
|
||||
"""Enumerate filenames only; never execute module manifests or import optional modules."""
|
||||
groups = {name: [] for name in ("pages-and-navigation", "dialogs-and-embedded-editors", "settings-and-administration", "widgets-and-public-surfaces", "other-ui-sources")}
|
||||
manifests, skipped = [], []
|
||||
count = 0
|
||||
for base, mode in ((root / "webui/src", "ui"), (root / "src", "backend")):
|
||||
try:
|
||||
reject_symlinks(base)
|
||||
except ValueError:
|
||||
skipped.append(str(base.relative_to(root)))
|
||||
continue
|
||||
for directory, dirs, names in os.walk(base, followlinks=False):
|
||||
folder = Path(directory)
|
||||
safe_dirs = []
|
||||
for name in sorted(dirs):
|
||||
child = folder / name
|
||||
if child.is_symlink():
|
||||
skipped.append(str(child.relative_to(root)))
|
||||
elif name not in {"node_modules", ".git", "__pycache__", ".venv", "dist"}:
|
||||
safe_dirs.append(name)
|
||||
dirs[:] = safe_dirs
|
||||
for name in sorted(names):
|
||||
path = folder / name
|
||||
if path.is_symlink():
|
||||
skipped.append(str(path.relative_to(root)))
|
||||
continue
|
||||
if mode == "backend":
|
||||
if name == "manifest.py":
|
||||
manifests.append(str(path.relative_to(root)))
|
||||
continue
|
||||
if path.suffix not in {".tsx", ".jsx", ".vue", ".svelte"} and name not in {"index.ts", "module.ts", "routes.ts"}:
|
||||
continue
|
||||
count += 1
|
||||
if count > MAX_SOURCES:
|
||||
raise ValueError("Module UI source inventory exceeds its size bound")
|
||||
relative = str(path.relative_to(root))
|
||||
lower = relative.lower()
|
||||
group = ("dialogs-and-embedded-editors" if any(part in lower for part in ("dialog", "modal", "editor")) else
|
||||
"settings-and-administration" if any(part in lower for part in ("setting", "admin")) else
|
||||
"widgets-and-public-surfaces" if any(part in lower for part in ("widget", "public")) else
|
||||
"pages-and-navigation" if any(part in lower for part in ("/pages/", "page.", "navigation", "routes.ts")) else "other-ui-sources")
|
||||
groups[group].append(relative)
|
||||
return {"basis": "Static filename discovery, not a complete runtime surface inventory or a review result.",
|
||||
"ui_source_count": count, "groups": groups, "manifest_paths": manifests,
|
||||
"skipped_symlinks": skipped, "module_code_imported": False}
|
||||
|
||||
|
||||
def _redact_tree(value):
|
||||
if isinstance(value, str):
|
||||
return redact(value)
|
||||
if isinstance(value, list):
|
||||
return [_redact_tree(item) for item in value]
|
||||
if isinstance(value, dict):
|
||||
return {key: _redact_tree(item) for key, item in value.items()}
|
||||
return value
|
||||
|
||||
|
||||
def handle_review(args) -> dict:
|
||||
name = args.module
|
||||
if name == "bundle":
|
||||
if not args.bundle_module:
|
||||
raise ValueError("review bundle requires a module")
|
||||
name = args.bundle_module
|
||||
elif args.bundle_module:
|
||||
raise ValueError("Review accepts one module; use review MODULE")
|
||||
workspace_root = Path(args.workspace_root).resolve()
|
||||
project = load_project(workspace_root, args.project)
|
||||
configured = project.config.get("review", {}) if args.project else {}
|
||||
if not isinstance(configured, dict) or set(configured) - {"issue_inventory", "principles"}:
|
||||
raise ValueError("Project review configuration accepts issue_inventory and principles paths")
|
||||
meta = next((repo.path for repo in project.repositories if repo.name == "govoplan"), None)
|
||||
core = next((repo.path for repo in project.repositories if repo.name == "govoplan-core"), None)
|
||||
inventory_path = (_project_path(workspace_root, configured["issue_inventory"]) if "issue_inventory" in configured else
|
||||
meta / "docs/project/ui-review-issue-inventory.json" if meta and not args.project else None)
|
||||
principles_path = (_project_path(workspace_root, configured["principles"]) if "principles" in configured else
|
||||
core / "docs/UI_DESIGN_PRINCIPLES.md" if core and not args.project else None)
|
||||
issues, epic = _issue_inventory(inventory_path)
|
||||
scope_matches = [item for item in issues if item["scope_id"] == name]
|
||||
selected = selected_repositories(project, [scope_matches[0]["repository"] if scope_matches else name])
|
||||
if len(selected) != 1:
|
||||
raise ValueError("Review must resolve to exactly one registered repository")
|
||||
repo = selected[0]
|
||||
links = [item for item in issues if item["repository"] == repo.name]
|
||||
inventory = source_inventory(repo.path)
|
||||
principles = _principles(principles_path)
|
||||
state = inspect_repository(repo)
|
||||
warnings = []
|
||||
if not links:
|
||||
warnings.append("No module issue discovery link is configured; locate/create the canonical Gitea review issue before recording review work.")
|
||||
if not principles["available"] or not principles["revision"]:
|
||||
warnings.append("The principle document or dated UI revision is unavailable; confirm the governing rules before reviewing.")
|
||||
if not inventory["ui_source_count"]:
|
||||
warnings.append("No UI source filenames were discovered. Check contributed/headless/placeholder scope manually; this is not automatic N/A or completion.")
|
||||
if inventory["skipped_symlinks"]:
|
||||
warnings.append("Symlinked source paths were not traversed; the source starting inventory is incomplete.")
|
||||
if state["errors"]:
|
||||
warnings.append("Repository inspection reported errors; missing or unreadable source is not a clean review.")
|
||||
try:
|
||||
fingerprint = source_fingerprint(project)
|
||||
except (OSError, ValueError):
|
||||
fingerprint = None
|
||||
warnings.append("Current workspace source identity could not be established; do not present attached historical evidence as current verification.")
|
||||
from .catalog import build_stages
|
||||
checks = build_stages(workspace_root, args.profile, [repo.name], False, project=args.project)
|
||||
if not checks:
|
||||
warnings.append("No automated stages are planned for this selection; this is not a passing verification result.")
|
||||
evidence = evidence_record(args.evidence, args)
|
||||
limits = list(dict.fromkeys([*coverage_notes(checks), *(evidence["coverage_notes"] if evidence else [])]))
|
||||
warnings.extend("Coverage limitation: " + note for note in limits[:8])
|
||||
if len(limits) > 8:
|
||||
warnings.append(f"{len(limits) - 8} additional coverage limitations are retained in the JSON bundle.")
|
||||
result = {"schema_version": 1, "operation": "review.bundle", "generated_at": now(), "workspace_root": str(workspace_root),
|
||||
"project": project.name, "module": repo.name, "source_fingerprint": fingerprint, "repository": state,
|
||||
"issue_links": links, "central_issue": epic, "issue_inventory_path": str(inventory_path) if inventory_path else None,
|
||||
"state_authority": "Live Gitea issues are the canonical backlog and review state log. Discovery links do not report current issue state.",
|
||||
"inventory": inventory, "principles": principles, "check_plan": {"profile": args.profile, "executed": False, "stages": checks},
|
||||
"manual_checklist": [{"id": identity, "prompt": prompt} for identity, prompt in MANUAL_CHECKS],
|
||||
"review_completion": "Not assessed. Automated checks and this bundle never complete a module review or update issue checklists.",
|
||||
"evidence": evidence, "coverage_notes": limits, "warnings": warnings,
|
||||
"summary": [f"Review bundle: {repo.name}; {inventory['ui_source_count']} UI source files, {len(checks)} planned checks (not executed).",
|
||||
"Principle revision: " + (principles["revision"] or "unavailable"),
|
||||
*["Module issue: " + item["url"] for item in links],
|
||||
*(["Central issue: " + epic["url"]] if epic else []),
|
||||
"Manual review remains unassessed; record outcomes and remaining work in Gitea.", *warnings]}
|
||||
result = _redact_tree(result)
|
||||
if args.output:
|
||||
result["summary"].append("Local bundle artifact: " + redact(str(args.output)))
|
||||
atomic_json(args.output, result)
|
||||
return result
|
||||
Executable
+1034
File diff suppressed because it is too large
Load Diff
Executable
+142
@@ -0,0 +1,142 @@
|
||||
"""Dependency-free validation against the published portable-project schema."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
from .common import META_ROOT, canonical, read_json
|
||||
|
||||
|
||||
def _schema_value(value, schema: dict, definitions: dict, path: str) -> None:
|
||||
if "$ref" in schema:
|
||||
return _schema_value(
|
||||
value,
|
||||
definitions[schema["$ref"].removeprefix("#/$defs/")],
|
||||
definitions,
|
||||
path,
|
||||
)
|
||||
kind = schema.get("type")
|
||||
numeric = type(value) is int or type(value) is float and math.isfinite(value)
|
||||
matches = {
|
||||
"object": isinstance(value, dict),
|
||||
"array": isinstance(value, list),
|
||||
"string": isinstance(value, str),
|
||||
"number": numeric,
|
||||
"integer": numeric and int(value) == value,
|
||||
}
|
||||
if kind and not matches[kind]:
|
||||
raise ValueError(f"{path} must be a {kind}")
|
||||
if "const" in schema and value != schema["const"]:
|
||||
raise ValueError(f"{path} has an unsupported value")
|
||||
if "enum" in schema and value not in schema["enum"]:
|
||||
raise ValueError(f"{path} has an unsupported value")
|
||||
if isinstance(value, dict):
|
||||
properties = schema.get("properties", {})
|
||||
unknown = set(value) - set(properties)
|
||||
if schema.get("additionalProperties") is False and unknown:
|
||||
names = ", ".join(key[:80] for key in sorted(unknown)[:4])
|
||||
raise ValueError(f"Unknown field in {path}: {names}")
|
||||
if set(schema.get("required", [])) - set(value):
|
||||
raise ValueError(f"{path} is missing a required field")
|
||||
for key, item in value.items():
|
||||
if key in properties:
|
||||
_schema_value(item, properties[key], definitions, f"{path}.{key}")
|
||||
if isinstance(value, list):
|
||||
if (
|
||||
not schema.get("minItems", 0)
|
||||
<= len(value)
|
||||
<= schema.get("maxItems", len(value))
|
||||
):
|
||||
raise ValueError(f"{path} exceeds its item bounds")
|
||||
if schema.get("uniqueItems") and len(
|
||||
{canonical(item) for item in value}
|
||||
) != len(value):
|
||||
raise ValueError(f"Duplicate value in {path}")
|
||||
prefix = schema.get("prefixItems", [])
|
||||
for index, item in enumerate(value):
|
||||
_schema_value(
|
||||
item,
|
||||
prefix[index] if index < len(prefix) else schema.get("items", {}),
|
||||
definitions,
|
||||
f"{path}[{index}]",
|
||||
)
|
||||
if isinstance(value, str):
|
||||
if (
|
||||
not schema.get("minLength", 0)
|
||||
<= len(value)
|
||||
<= schema.get("maxLength", len(value))
|
||||
):
|
||||
raise ValueError(f"{path} exceeds its string bounds")
|
||||
if "pattern" in schema and re.search(schema["pattern"], value) is None:
|
||||
raise ValueError(
|
||||
f"{path} contains an invalid identifier, path or character"
|
||||
)
|
||||
if type(value) in {int, float}:
|
||||
if (
|
||||
"maximum" in schema
|
||||
and value > schema["maximum"]
|
||||
or "exclusiveMinimum" in schema
|
||||
and value <= schema["exclusiveMinimum"]
|
||||
):
|
||||
raise ValueError(f"{path} exceeds its numeric bounds")
|
||||
|
||||
|
||||
def validate_project(payload: dict, workspace_root: Path) -> None:
|
||||
"""Validate every declaration, not just the selected profile/dependencies."""
|
||||
schema = read_json(META_ROOT / "tools/devkit/project.schema.json")
|
||||
_schema_value(payload, schema, schema["$defs"], "project")
|
||||
root = workspace_root.resolve()
|
||||
records = payload["repositories"]
|
||||
names = {record["name"] for record in records}
|
||||
if len(names) != len(records):
|
||||
raise ValueError("Duplicate project repository name")
|
||||
paths, aliases = set(), set()
|
||||
for record in records:
|
||||
path = (root / record["path"]).resolve()
|
||||
if not path.is_relative_to(root):
|
||||
raise ValueError("Repository path escapes the workspace")
|
||||
if path in paths:
|
||||
raise ValueError("Duplicate project repository path")
|
||||
paths.add(path)
|
||||
keys = {record["name"], *record.get("aliases", [])}
|
||||
if aliases & keys:
|
||||
raise ValueError("Repository names and aliases must be unambiguous")
|
||||
aliases.update(keys)
|
||||
checks = payload.get("checks", [])
|
||||
identities = {item["id"] for item in checks}
|
||||
if len(identities) != len(checks):
|
||||
raise ValueError("Duplicate project check ID")
|
||||
for item in checks:
|
||||
if set(item.get("repos", [])) - names:
|
||||
raise ValueError(f"Unknown repository reference in check {item['id']}")
|
||||
if set(item.get("deps", [])) - identities:
|
||||
raise ValueError(f"Unknown check dependency in {item['id']}")
|
||||
if set(item.get("after", [])) - identities:
|
||||
raise ValueError(f"Unknown check ordering reference in {item['id']}")
|
||||
if set(item.get("deps", [])) & set(item.get("after", [])):
|
||||
raise ValueError(f"Duplicate dependency/ordering reference in {item['id']}")
|
||||
if set(item.get("inputs", {}).get("repos", [])) - names:
|
||||
raise ValueError(f"Unknown input repository reference in {item['id']}")
|
||||
if not (root / item.get("cwd", ".")).resolve().is_relative_to(root):
|
||||
raise ValueError(f"Check {item['id']} cwd escapes the workspace")
|
||||
for name, selected in payload.get("profiles", {}).items():
|
||||
if set(selected) - identities:
|
||||
raise ValueError(f"Unknown profile check in {name}")
|
||||
remaining = {
|
||||
item["id"]: set(item.get("deps", [])) | set(item.get("after", []))
|
||||
for item in checks
|
||||
}
|
||||
while remaining:
|
||||
ready = {identity for identity, deps in remaining.items() if not deps}
|
||||
if not ready:
|
||||
raise ValueError("Cyclic check dependency in project declarations")
|
||||
remaining = {
|
||||
identity: deps - ready
|
||||
for identity, deps in remaining.items()
|
||||
if identity not in ready
|
||||
}
|
||||
for value in payload.get("review", {}).values():
|
||||
if not (root / value).resolve().is_relative_to(root):
|
||||
raise ValueError("Project review path escapes the workspace")
|
||||
Executable
+273
@@ -0,0 +1,273 @@
|
||||
"""Repository discovery, offline snapshots and exact source fingerprints."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import sys
|
||||
|
||||
from .common import META_ROOT, digest, identifier, read_json
|
||||
from .process import require_capture
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Repository:
|
||||
name: str
|
||||
path: Path
|
||||
aliases: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Project:
|
||||
name: str
|
||||
repositories: tuple[Repository, ...]
|
||||
config: dict
|
||||
|
||||
|
||||
def load_project(workspace_root: Path, project: Path | None = None) -> Project:
|
||||
root = workspace_root.resolve()
|
||||
payload = read_json(project or META_ROOT / "repositories.json")
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("Project manifest must be an object")
|
||||
if project:
|
||||
from .validation import validate_project
|
||||
|
||||
validate_project(payload, root)
|
||||
records = payload.get("repositories")
|
||||
if not isinstance(records, list) or not records or len(records) > 256:
|
||||
raise ValueError("Project manifest requires 1–256 repositories")
|
||||
repos, names = [], set()
|
||||
for record in records:
|
||||
if not isinstance(record, dict) or not isinstance(record.get("path"), str):
|
||||
raise ValueError("Invalid repository record")
|
||||
if not record["path"] or (
|
||||
project and set(record) - {"name", "path", "aliases"}
|
||||
):
|
||||
raise ValueError("Repository requires a nonempty path and known fields")
|
||||
name = identifier(record.get("name"))
|
||||
raw = Path(record["path"])
|
||||
if raw.is_absolute() or ".." in raw.parts:
|
||||
raise ValueError("Repository paths must be relative within the workspace")
|
||||
path = (root / raw).resolve()
|
||||
if not path.is_relative_to(root):
|
||||
raise ValueError("Repository path escapes the workspace")
|
||||
aliases = (
|
||||
record.get("aliases", []) if project else [name.removeprefix("govoplan-")]
|
||||
)
|
||||
if not isinstance(aliases, list) or any(
|
||||
not isinstance(alias, str) for alias in aliases
|
||||
):
|
||||
raise ValueError("Repository aliases must be strings")
|
||||
if len(aliases) != len(set(aliases)):
|
||||
raise ValueError("Duplicate repository alias")
|
||||
keys = {name, *[identifier(alias) for alias in aliases]}
|
||||
if names.intersection(keys):
|
||||
raise ValueError("Repository names and aliases must be unambiguous")
|
||||
names.update(keys)
|
||||
repos.append(Repository(name, path, tuple(aliases)))
|
||||
return Project(
|
||||
payload.get("name", "Project" if project else "GovOPlaN"), tuple(repos), payload
|
||||
)
|
||||
|
||||
|
||||
def inspect_repository(repo: Repository) -> dict:
|
||||
# Reuse the established read-only model, including distinct Git errors.
|
||||
sys.path.insert(0, str(META_ROOT / "tools/release")) if str(
|
||||
META_ROOT / "tools/release"
|
||||
) not in sys.path else None
|
||||
from govoplan_release.git_state import collect_repository_snapshot
|
||||
from govoplan_release.model import RepositorySpec
|
||||
|
||||
unsafe = unsafe_git_environment()
|
||||
if unsafe:
|
||||
return {
|
||||
"name": repo.name,
|
||||
"path": str(repo.path),
|
||||
"exists": repo.path.exists(),
|
||||
"is_git": (repo.path / ".git").exists(),
|
||||
"branch": None,
|
||||
"head": None,
|
||||
"upstream": None,
|
||||
"ahead": None,
|
||||
"behind": None,
|
||||
"remote_checked": False,
|
||||
"dirty_entries": [],
|
||||
"errors": ["Git environment overrides prevent scoped inspection"],
|
||||
"safe_directory_required": False,
|
||||
}
|
||||
snapshot = collect_repository_snapshot(
|
||||
RepositorySpec(
|
||||
name=repo.name,
|
||||
category="module",
|
||||
subtype="",
|
||||
remote="",
|
||||
path=str(repo.path),
|
||||
),
|
||||
workspace_root=repo.path.parent,
|
||||
target_tag=None,
|
||||
online=False,
|
||||
)
|
||||
return {
|
||||
"name": repo.name,
|
||||
"path": str(repo.path),
|
||||
"exists": snapshot.exists,
|
||||
"is_git": snapshot.is_git,
|
||||
"branch": snapshot.branch,
|
||||
"head": snapshot.head,
|
||||
"upstream": snapshot.upstream,
|
||||
"ahead": snapshot.ahead,
|
||||
"behind": snapshot.behind,
|
||||
"remote_checked": False,
|
||||
"dirty_entries": list(snapshot.dirty_entries),
|
||||
"errors": list(snapshot.errors),
|
||||
"safe_directory_required": snapshot.safe_directory_required,
|
||||
}
|
||||
|
||||
|
||||
def selected_repositories(
|
||||
project: Project, names: list[str], changed: bool = False
|
||||
) -> list[Repository]:
|
||||
selected = list(project.repositories)
|
||||
if names:
|
||||
wanted = set(names)
|
||||
known = {key for repo in selected for key in (repo.name, *repo.aliases)}
|
||||
if wanted - known:
|
||||
raise ValueError(
|
||||
"Unknown repository filter: " + ", ".join(sorted(wanted - known))
|
||||
)
|
||||
selected = [
|
||||
repo for repo in selected if wanted.intersection((repo.name, *repo.aliases))
|
||||
]
|
||||
if changed:
|
||||
result = []
|
||||
for repo in selected:
|
||||
state = inspect_repository(repo)
|
||||
if (
|
||||
state["errors"]
|
||||
or state["dirty_entries"]
|
||||
or state["ahead"]
|
||||
or (state["head"] and not state["upstream"])
|
||||
):
|
||||
result.append(repo)
|
||||
selected = result
|
||||
return selected
|
||||
|
||||
|
||||
def unsafe_git_environment() -> set[str]:
|
||||
return {
|
||||
key
|
||||
for key in os.environ
|
||||
if key.startswith("GIT_")
|
||||
and key not in {"GIT_OPTIONAL_LOCKS", "GIT_TERMINAL_PROMPT", "GIT_PAGER"}
|
||||
}
|
||||
|
||||
|
||||
def git_bytes(path: Path, *argv: str, allow_failure: bool = False) -> bytes:
|
||||
if unsafe_git_environment():
|
||||
raise ValueError(
|
||||
"Git environment overrides prevent a scoped source fingerprint"
|
||||
)
|
||||
result = require_capture(
|
||||
["git", "--no-pager", "-C", str(path), *argv],
|
||||
timeout=30,
|
||||
max_stdout=32 * 1024 * 1024,
|
||||
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0", "GIT_TERMINAL_PROMPT": "0"},
|
||||
)
|
||||
if result.returncode:
|
||||
if allow_failure:
|
||||
return b"unborn"
|
||||
raise ValueError(
|
||||
f"Could not fingerprint Git state in {path.name}; inspect context first"
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
def source_fingerprint(project: Project) -> str:
|
||||
"""Bind HEAD, index and every tracked/untracked working file, including hidden changes."""
|
||||
overall = hashlib.sha256()
|
||||
overall.update(digest(project.config).encode())
|
||||
for repo in sorted(project.repositories, key=lambda item: item.name):
|
||||
overall.update(repo.name.encode() + b"\0" + str(repo.path).encode() + b"\0")
|
||||
if not repo.path.exists():
|
||||
overall.update(b"missing\0")
|
||||
continue
|
||||
if not (repo.path / ".git").exists():
|
||||
raise ValueError(
|
||||
f"Cannot establish source identity for non-Git repository {repo.name}"
|
||||
)
|
||||
overall.update(
|
||||
git_bytes(
|
||||
repo.path, "rev-parse", "--verify", "HEAD", allow_failure=True
|
||||
).strip()
|
||||
)
|
||||
overall.update(git_bytes(repo.path, "ls-files", "--stage", "-z"))
|
||||
overall.update(git_bytes(repo.path, "ls-files", "-v", "-z"))
|
||||
tracked = git_bytes(repo.path, "ls-files", "--cached", "-z")
|
||||
untracked = git_bytes(
|
||||
repo.path, "ls-files", "--others", "--exclude-standard", "-z"
|
||||
)
|
||||
for encoded_name in sorted(
|
||||
set(tracked.split(b"\0") + untracked.split(b"\0")) - {b""}
|
||||
):
|
||||
name = os.fsdecode(encoded_name)
|
||||
relative = Path(name)
|
||||
if relative.is_absolute() or ".." in relative.parts:
|
||||
raise ValueError("Unsafe repository file name")
|
||||
path = repo.path / relative
|
||||
overall.update(encoded_name + b"\0")
|
||||
if not path.exists() and not path.is_symlink():
|
||||
overall.update(b"deleted\0")
|
||||
continue
|
||||
metadata = path.lstat()
|
||||
overall.update(str(stat.S_IMODE(metadata.st_mode)).encode() + b"\0")
|
||||
if path.is_symlink():
|
||||
overall.update(os.fsencode(os.readlink(path)))
|
||||
elif path.is_file():
|
||||
if not path.resolve().is_relative_to(repo.path):
|
||||
raise ValueError("Fingerprint input escapes its repository")
|
||||
maximum = 64 * 1024 * 1024
|
||||
descriptor = os.open(
|
||||
path,
|
||||
os.O_RDONLY
|
||||
| getattr(os, "O_NOFOLLOW", 0)
|
||||
| getattr(os, "O_NONBLOCK", 0),
|
||||
)
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
before = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(before.st_mode) or before.st_size > maximum:
|
||||
raise ValueError(
|
||||
f"Fingerprint input must be a bounded regular file in {repo.name}"
|
||||
)
|
||||
file_hash, count = hashlib.sha256(), 0
|
||||
for chunk in iter(
|
||||
lambda: handle.read(min(1024 * 1024, maximum - count + 1)), b""
|
||||
):
|
||||
count += len(chunk)
|
||||
if count > maximum:
|
||||
raise ValueError("File grew beyond fingerprint limit")
|
||||
file_hash.update(chunk)
|
||||
after = os.fstat(handle.fileno())
|
||||
if (
|
||||
before.st_ino,
|
||||
before.st_size,
|
||||
before.st_mtime_ns,
|
||||
before.st_ctime_ns,
|
||||
) != (
|
||||
after.st_ino,
|
||||
after.st_size,
|
||||
after.st_mtime_ns,
|
||||
after.st_ctime_ns,
|
||||
):
|
||||
raise ValueError(
|
||||
"File changed while calculating source identity"
|
||||
)
|
||||
overall.update(file_hash.digest())
|
||||
else:
|
||||
raise ValueError(
|
||||
"Unsupported changed-file type; cannot establish source identity"
|
||||
)
|
||||
overall.update(b"\0")
|
||||
return overall.hexdigest()
|
||||
Executable
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "Devkit portable project configuration",
|
||||
"description": "The runtime additionally checks reference existence, cycles, resolved path confinement and cross-repository alias/path uniqueness. cwd defaults to the workspace when omitted.",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["schema_version", "repositories"],
|
||||
"properties": {
|
||||
"schema_version": {"type": "integer", "const": 1},
|
||||
"name": {"type": "string", "minLength": 1, "maxLength": 128, "pattern": "^[^\\u0000\\r\\n]+$"},
|
||||
"repositories": {
|
||||
"type": "array", "minItems": 1, "maxItems": 256,
|
||||
"items": {
|
||||
"type": "object", "additionalProperties": false, "required": ["name", "path"],
|
||||
"properties": {
|
||||
"name": {"$ref": "#/$defs/identifier"},
|
||||
"path": {"$ref": "#/$defs/relativePath"},
|
||||
"aliases": {"type": "array", "maxItems": 128, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}
|
||||
}
|
||||
}
|
||||
},
|
||||
"tools": {
|
||||
"type": "object", "additionalProperties": false,
|
||||
"properties": {"python": {"$ref": "#/$defs/executable"}, "node": {"$ref": "#/$defs/executable"}, "npm": {"$ref": "#/$defs/executable"}}
|
||||
},
|
||||
"review": {
|
||||
"type": "object", "additionalProperties": false,
|
||||
"properties": {
|
||||
"issue_inventory": {"$ref": "#/$defs/relativePath"},
|
||||
"principles": {"$ref": "#/$defs/relativePath"}
|
||||
}
|
||||
},
|
||||
"checks": {
|
||||
"type": "array", "maxItems": 512,
|
||||
"items": {
|
||||
"type": "object", "additionalProperties": false, "required": ["id", "argv"],
|
||||
"properties": {
|
||||
"id": {"$ref": "#/$defs/identifier"},
|
||||
"title": {"type": "string", "maxLength": 1024, "pattern": "^[^\\u0000]*$"},
|
||||
"argv": {"type": "array", "minItems": 1, "maxItems": 256, "prefixItems": [{"type": "string", "minLength": 1, "maxLength": 8192, "pattern": "^[^\\u0000]+$"}], "items": {"type": "string", "maxLength": 8192, "pattern": "^[^\\u0000]*$"}},
|
||||
"cwd": {"$ref": "#/$defs/relativePath"},
|
||||
"deps": {"$ref": "#/$defs/profile"},
|
||||
"after": {"$ref": "#/$defs/profile"},
|
||||
"reuse": {"type": "string", "enum": ["verified", "never"]},
|
||||
"inputs": {"$ref": "#/$defs/inputs"},
|
||||
"resources": {"type": "array", "maxItems": 256, "uniqueItems": true, "items": {"type": "string", "minLength": 1, "maxLength": 256, "pattern": "^[^\\u0000\\r\\n]+$"}},
|
||||
"repos": {"type": "array", "maxItems": 256, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}},
|
||||
"timeout_seconds": {"type": "number", "exclusiveMinimum": 0, "maximum": 43200}
|
||||
}
|
||||
}
|
||||
},
|
||||
"profiles": {
|
||||
"type": "object", "additionalProperties": false,
|
||||
"properties": {
|
||||
"quick": {"$ref": "#/$defs/profile"}, "ui": {"$ref": "#/$defs/profile"},
|
||||
"backend": {"$ref": "#/$defs/profile"}, "full": {"$ref": "#/$defs/profile"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"inputs": {"type": "object", "additionalProperties": false, "required": ["repos"], "properties": {"repos": {"type": "array", "minItems": 1, "maxItems": 256, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}}},
|
||||
"identifier": {"type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$(?![\\s\\S])"},
|
||||
"executable": {"type": "string", "minLength": 1, "maxLength": 4096, "pattern": "^[^\\u0000\\r\\n]+$"},
|
||||
"relativePath": {"type": "string", "minLength": 1, "maxLength": 4096, "pattern": "^(?!/)(?!\\.\\.(?:/|$))(?!.*?/\\.\\.(?:/|$))[^\\u0000\\r\\n]+$"},
|
||||
"profile": {"type": "array", "maxItems": 512, "uniqueItems": true, "items": {"$ref": "#/$defs/identifier"}}
|
||||
}
|
||||
}
|
||||
Executable
+386
@@ -0,0 +1,386 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Inventory all module UI review scopes and safely create missing Gitea tracks.
|
||||
|
||||
Dry-run by default. Existing issues are never rewritten or closed. The optional
|
||||
one-time epic link initialization refuses to replace an edited or completed list.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from contextlib import contextmanager
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import socket
|
||||
import sys
|
||||
from typing import Any, Iterator
|
||||
|
||||
from gitea_common import (
|
||||
GiteaClient, GiteaError, RepoTarget, load_dotenv,
|
||||
org_path, repo_path, require_token,
|
||||
)
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
BASE_URL = "https://git.add-ideas.de"
|
||||
OWNER = "GovOPlaN"
|
||||
EPIC_MARKER = "<!-- govoplan-ui-review:v1:epic -->"
|
||||
LIST_START = "<!-- govoplan-ui-review:module-list:start -->"
|
||||
LIST_END = "<!-- govoplan-ui-review:module-list:end -->"
|
||||
INITIAL_LIST = "The linked inventory is being initialized. All 77 tracks are pending; no checkboxes are complete."
|
||||
PRINCIPLES_URL = f"{BASE_URL}/{OWNER}/govoplan-core/src/branch/main/docs/UI_DESIGN_PRINCIPLES.md"
|
||||
PROCESS_URL = f"{BASE_URL}/{OWNER}/govoplan/src/branch/main/docs/project/UI_REVIEW_PROGRAM.md"
|
||||
PRINCIPLES = (
|
||||
("UI-01", "Help icons beside the relevant heading/label, not action-button rows"),
|
||||
("UI-02", "Display-first with scoped edit dialogs; explicit bulk-grid editing exception"),
|
||||
("UI-03", "Shared page actions, Reload/New ordering, Save/Cancel and destructive separation"),
|
||||
("UI-04", "Full-width table/card geometry, visible actions, pagination and two-way resizing"),
|
||||
("UI-05", "Scoped loading/error feedback, useful progress and retained state"),
|
||||
("UI-06", "Predictable tree selection, expansion, grouping and reordering"),
|
||||
("UI-07", "Keyboard/focus/accessibility, responsive layouts and understandable German"),
|
||||
("UI-08", "Authorization, optional-module boundaries and save/cancel/retry data integrity"),
|
||||
("UI-09", "Revision evidence and retroactive checks for changed design principles"),
|
||||
)
|
||||
|
||||
|
||||
def source_url(repository: str, path: str) -> str:
|
||||
return f"{BASE_URL}/{OWNER}/{repository}/src/branch/main/{path}"
|
||||
|
||||
|
||||
def marker(scope_id: str) -> str:
|
||||
return f"<!-- govoplan-ui-review:v1:module:{scope_id} -->"
|
||||
|
||||
|
||||
def normalized_title(value: str) -> str:
|
||||
return " ".join(value.casefold().split())
|
||||
|
||||
|
||||
def find_existing(issues: list[dict[str, Any]], scope_id: str, title: str) -> dict[str, Any] | None:
|
||||
matches = [
|
||||
issue for issue in issues if issue.get("pull_request") is None and (
|
||||
marker(scope_id) in (issue.get("body") or "")
|
||||
or normalized_title(issue.get("title") or "") == normalized_title(title)
|
||||
)
|
||||
]
|
||||
if len(matches) > 1:
|
||||
raise GiteaError(f"Ambiguous review issue matches for {scope_id}; inspect before making changes.")
|
||||
if matches and marker(scope_id) not in (matches[0].get("body") or ""):
|
||||
raise GiteaError(f"Unmanaged exact-title issue for {scope_id}; preserve it and resolve the duplicate manually.")
|
||||
return matches[0] if matches else None
|
||||
|
||||
|
||||
def extract_manifests(catalog: dict[str, Any], workspace_root: Path) -> list[dict[str, Any]]:
|
||||
path = META_ROOT / "tools/inventory/platform-interface-inventory.py"
|
||||
spec = importlib.util.spec_from_file_location("ui_review_source_inventory", path)
|
||||
assert spec and spec.loader
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module._extract_manifests(catalog, workspace_root)
|
||||
|
||||
|
||||
def source_groups(root: Path) -> dict[str, list[str]]:
|
||||
groups: dict[str, list[str]] = {
|
||||
"Pages and navigation entrypoints": [],
|
||||
"Dialogs and embedded editing surfaces": [],
|
||||
"Settings and administrator surfaces": [],
|
||||
"Widgets, public/operator and contributed surfaces": [],
|
||||
"Shared components and other UI entrypoints": [],
|
||||
}
|
||||
source_root = root / "webui/src"
|
||||
for path in sorted(source_root.rglob("*.tsx")):
|
||||
relative = path.relative_to(root).as_posix()
|
||||
lower = relative.casefold()
|
||||
name = path.stem.casefold()
|
||||
if "page" in name or "navigation" in name or name in {"app", "routes", "index"}:
|
||||
groups["Pages and navigation entrypoints"].append(relative)
|
||||
elif re.search(r"dialog|modal|drawer|chooser|overlay", name):
|
||||
groups["Dialogs and embedded editing surfaces"].append(relative)
|
||||
elif re.search(r"settings|configur|admin", lower):
|
||||
groups["Settings and administrator surfaces"].append(relative)
|
||||
elif re.search(r"widget|public|operator|contribution", lower):
|
||||
groups["Widgets, public/operator and contributed surfaces"].append(relative)
|
||||
else:
|
||||
groups["Shared components and other UI entrypoints"].append(relative)
|
||||
return groups
|
||||
|
||||
|
||||
def build_scopes(
|
||||
catalog: dict[str, Any], workspace_root: Path, manifests: list[dict[str, Any]],
|
||||
) -> list[dict[str, Any]]:
|
||||
selected = [
|
||||
repo for repo in catalog["repositories"]
|
||||
if repo["category"] in {"module", "connector"} or repo["name"] == "govoplan-core"
|
||||
]
|
||||
by_repository: dict[str, dict[str, Any]] = {}
|
||||
ids: set[str] = set()
|
||||
for manifest in manifests:
|
||||
if manifest["repository"] in by_repository or manifest["id"] in ids:
|
||||
raise GiteaError("Duplicate source manifest repository or module ID.")
|
||||
by_repository[manifest["repository"]] = manifest
|
||||
ids.add(manifest["id"])
|
||||
selected_names = {repo["name"] for repo in selected}
|
||||
if set(by_repository) - selected_names:
|
||||
raise GiteaError("Source manifests contain repositories absent from the module review catalog.")
|
||||
scopes: list[dict[str, Any]] = []
|
||||
for repo in selected:
|
||||
root = workspace_root / repo["path"]
|
||||
if not root.is_dir():
|
||||
raise GiteaError(f"Missing source checkout for {repo['name']}; cannot infer review scope safely.")
|
||||
manifest = by_repository.get(repo["name"])
|
||||
paths = sorted(path.relative_to(root).as_posix() for path in root.glob("src/*/backend/manifest.py"))
|
||||
if repo["name"] == "govoplan-core":
|
||||
scope_id, name, kind = "core", "Core / shared shell", "core"
|
||||
elif manifest:
|
||||
scope_id, name, kind = manifest["id"], manifest["name"], "manifest"
|
||||
else:
|
||||
if paths or (root / "pyproject.toml").exists() or (root / "webui/package.json").exists():
|
||||
raise GiteaError(f"{repo['name']} has implementation but no extracted manifest; inspect instead of calling it a placeholder.")
|
||||
scope_id = "catalog:" + repo["name"]
|
||||
name = repo["name"].removeprefix("govoplan-").replace("-", " ").title()
|
||||
kind = "placeholder"
|
||||
groups = source_groups(root)
|
||||
scopes.append({
|
||||
"scope_id": scope_id, "name": name, "repository": repo["name"],
|
||||
"kind": kind, "manifest_paths": paths,
|
||||
"frontend": manifest.get("frontend") if manifest else None,
|
||||
"source_groups": groups,
|
||||
"ui_source_count": sum(len(paths) for paths in groups.values()),
|
||||
})
|
||||
if len({scope["scope_id"] for scope in scopes}) != len(scopes):
|
||||
raise GiteaError("Duplicate review scope IDs.")
|
||||
return sorted(scopes, key=lambda scope: (scope["kind"] == "placeholder", scope["scope_id"] != "core", scope["name"].casefold()))
|
||||
|
||||
|
||||
def issue_title(scope: dict[str, Any]) -> str:
|
||||
suffix = "readiness and future UI review" if scope["kind"] == "placeholder" else "visual and interaction conformance"
|
||||
return f"[UI review] {scope['name']}: {suffix}"
|
||||
|
||||
|
||||
def source_seed(scope: dict[str, Any]) -> str:
|
||||
repo = scope["repository"]
|
||||
lines = [
|
||||
"This is a **source-derived starting inventory, not a completed runtime audit**. Verify nested routes, embedded dialogs and contributions in the installed module context; add missing surfaces to this issue.",
|
||||
"", f"Repository: [{repo}]({BASE_URL}/{OWNER}/{repo}).",
|
||||
]
|
||||
if scope["kind"] == "placeholder":
|
||||
lines += [
|
||||
"", f"Catalog entry `{repo}` is currently README-only; there is no runtime module ID, manifest or standalone WebUI to claim as reviewed. Source: [README]({source_url(repo, 'README.md')}).",
|
||||
"", "- [ ] Confirm the catalog/readiness scope and record prerequisites for the first implementation.",
|
||||
"- [ ] Keep the future interface review pending until actual configuration, public/operator or UI surfaces exist; do not manufacture N/A evidence to close this track.",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
for path in scope["manifest_paths"]:
|
||||
lines.append(f"Manifest source: [{path}]({source_url(repo, path)}).")
|
||||
if scope["kind"] == "core":
|
||||
lines += [
|
||||
"", "Core/shared shell additionally owns navigation/rail/breadcrumbs, module routing, page/action archetypes, cards/tables/forms/dialogs, loading and error surfaces, help affordances, authentication and user settings. Review optional-module and permission contexts, not just standalone primitives.",
|
||||
]
|
||||
frontend = scope["frontend"]
|
||||
if frontend:
|
||||
for key, label in (
|
||||
("routes", "Declared routes"), ("public_routes", "Declared public routes"),
|
||||
("settings_routes", "Declared settings routes"), ("nav_items", "Declared navigation"),
|
||||
("view_surfaces", "Declared view, settings and contributed surfaces"),
|
||||
):
|
||||
entries = frontend.get(key) or []
|
||||
lines += ["", f"**{label} ({len(entries)}):**"]
|
||||
if not entries:
|
||||
lines.append("None declared in this manifest; verify indirect/contributed surfaces before marking anything not applicable.")
|
||||
for entry in entries:
|
||||
identity = entry.get("path") or entry.get("id") or entry.get("component") or "unnamed declaration"
|
||||
detail = entry.get("component") or entry.get("label") or entry.get("kind") or ""
|
||||
lines.append(f"- `{identity}`" + (f" — {detail}" if detail else ""))
|
||||
elif scope["kind"] != "core":
|
||||
lines += [
|
||||
"", "No standalone frontend is declared. **The review is still pending:** inspect owned configuration/admin workflows, manifest documentation, errors and any interfaces contributed through host modules, public routes or operator tools. Record concrete evidence before claiming a principle does not apply.",
|
||||
]
|
||||
lines += ["", f"<details><summary>Source entrypoint seed ({scope['ui_source_count']} TSX files; classification is heuristic)</summary>", ""]
|
||||
for label, paths in scope["source_groups"].items():
|
||||
if not paths:
|
||||
continue
|
||||
lines += [f"**{label} ({len(paths)}):**", ""]
|
||||
for path in paths[:40]:
|
||||
lines.append(f"- [{path}]({source_url(repo, path)})")
|
||||
if len(paths) > 40:
|
||||
lines.append(f"- {len(paths) - 40} further files: inspect [the source tree]({source_url(repo, 'webui/src')}); expand the issue inventory during review.")
|
||||
lines.append("")
|
||||
lines += ["</details>"]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def issue_body(scope: dict[str, Any], epic_url: str) -> str:
|
||||
lines = [
|
||||
marker(scope["scope_id"]), "## Status and objective", "",
|
||||
f"**Pending / not reviewed.** This module track belongs to the [product-wide UI review epic]({epic_url}).",
|
||||
f"Apply the [Core design principles]({PRINCIPLES_URL}) using the [shared review process]({PROCESS_URL}). The current heading-help icon pass and any existing isolated fixes are preparation, not evidence that this whole module is complete. Source documents are being prepared in the current working tree; this issue does not claim they are released.",
|
||||
"", "Prioritize usability defects, consistent interaction and shared-component fixes before broader features. Preserve permissions, security, optional-module boundaries and data integrity.",
|
||||
"", "## Source inventory to verify", "", source_seed(scope),
|
||||
"", "## Review and implementation TODO", "",
|
||||
"- [ ] Confirm every actual page, nested route, dialog, field/form, table/tree, settings level, public/operator surface, widget and cross-module contribution; document role/module prerequisites.",
|
||||
"- [ ] Exercise EN/DE, keyboard/focus, narrow and wide layouts, long content, empty/loading/error states and realistic datasets.",
|
||||
"- [ ] Check UI-01 heading/label help placement and UI-02 display-first/scoped editing; document any justified large-grid bulk-edit exception with explicit mode, Save/Cancel and dirty-navigation guard.",
|
||||
"- [ ] Verify consistent top-right actions (Reload left of New), clean/dirty Save/Cancel behavior, destructive separation and safe navigation/reload.",
|
||||
"- [ ] Verify full-width cards/tables, visible last-column actions, pagination, initial sizing, two-way pointer/keyboard resizing and preference reload with fixed columns and horizontal overflow.",
|
||||
"- [ ] Check scoped progress/error feedback and predictable tree selection versus expansion; no unnecessary global blocking or repeated background reload.",
|
||||
"- [ ] Record findings and implement shared-contract corrections plus all affected consumers, not local CSS/action-row exceptions without justification.",
|
||||
"- [ ] Verify save/cancel/retry and partial-failure behavior without unintended writes, sends, deletes or loss of persisted/unsaved data.",
|
||||
"- [ ] Update owning manifest-driven EN/DE documentation; record targeted automated checks and manual evidence against actual module surfaces.",
|
||||
"- [ ] Complete the principle matrix, list unresolved decisions/manual checks and link follow-ups before proposing closure.",
|
||||
]
|
||||
if scope["scope_id"] == "campaigns":
|
||||
lines += [
|
||||
"", "### Campaign-specific starting direction", "",
|
||||
"- [ ] Present a compact read-only campaign settings dashboard/overview and use explicit scoped edit dialogs for settings instead of a permanently editable form wall.",
|
||||
"- [ ] Keep large recipient/attachment tables practical through an explicit bulk-edit mode where appropriate, with Save/Cancel, dirty-state protection and reload persistence; this is the documented UI-02 exception, not silent autosave.",
|
||||
"- [ ] Review the complete compose → attachments → validation/review → delivery/report/operator workflow, including mail-profile migration, ZIP policy, multiple recipients and SMTP/IMAP progress, without sending live messages just to collect UI evidence.",
|
||||
]
|
||||
lines += [
|
||||
"", "## Principle applicability / application / evidence / exceptions", "",
|
||||
"Reviewed Core principle revision: **not yet recorded**. No exceptions approved.", "",
|
||||
"| Principle | Applicable surfaces / justified N/A | Applied / remaining work | Evidence | Exception / owner / follow-up |",
|
||||
"| --- | --- | --- | --- | --- |",
|
||||
]
|
||||
for identity, description in PRINCIPLES:
|
||||
lines.append(f"| {identity}: {description} | Pending inventory | Pending review | Not yet recorded | None approved |")
|
||||
lines += [
|
||||
"", "When a principle changes after this review, re-check applicability and record current evidence. Reopen this issue or link an owned follow-up for outstanding work; notify the central epic. A past review must not silently remain green against an obsolete rule.",
|
||||
"", "## Findings / TODO / done ledger", "",
|
||||
"| Finding / surface / reproduction | Principle and expected behavior | TODO / implementation or follow-up | Verified done evidence |",
|
||||
"| --- | --- | --- | --- |",
|
||||
"| Full review not started | UI-01–UI-09 | Inventory and review pending | None; no completed review claimed |",
|
||||
"", "## Manual checks, decisions and closure evidence", "",
|
||||
"- Manual work pending: safe actual-module walkthrough in both languages and realistic viewports, keyboard/focus, permissions/optional-module contexts, loading/error/empty states, edits/Save/Cancel/navigation/reload, tables/trees and progress.",
|
||||
"- Decisions: none invented by this bootstrap. Record any product or policy choice with context and a recommendation; isolate independent implementation work from blocked decisions.",
|
||||
"- Automated evidence: not yet recorded for this complete module review. Existing targeted fixes/tests may be linked as partial evidence only.",
|
||||
"- Closure gate: verified inventory, complete current principle matrix, resolved required findings, owning EN/DE documentation and automated/manual evidence. Unimplemented placeholders remain pending until real surfaces can be reviewed or an explicit catalog/product decision changes scope.",
|
||||
]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def result_record(scope: dict[str, Any], issue: dict[str, Any] | None, action: str) -> dict[str, Any]:
|
||||
return {
|
||||
"scope_id": scope["scope_id"], "name": scope["name"], "repository": scope["repository"],
|
||||
"kind": scope["kind"], "ui_source_count": scope["ui_source_count"],
|
||||
"number": issue.get("number") if issue else None,
|
||||
"url": issue.get("html_url") if issue else None,
|
||||
"state_at_verification": issue.get("state") if issue else None,
|
||||
"operation": action,
|
||||
}
|
||||
|
||||
|
||||
def render_links(records: list[dict[str, Any]]) -> str:
|
||||
lines = ["### Implemented scopes — pending review", ""]
|
||||
for placeholder in (False, True):
|
||||
if placeholder:
|
||||
lines += ["", "### Catalogued placeholders — pending readiness / future UI review", ""]
|
||||
for item in records:
|
||||
if (item["kind"] == "placeholder") == placeholder:
|
||||
if not item["url"]:
|
||||
raise GiteaError("Cannot initialize an incomplete issue link inventory.")
|
||||
lines.append(f"- [ ] [{item['name']}]({item['url']}) — `{item['scope_id']}` / `{item['repository']}`; pending.")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def initialized_epic_body(body: str, records: list[dict[str, Any]]) -> str:
|
||||
if body.count(LIST_START) != 1 or body.count(LIST_END) != 1 or EPIC_MARKER not in body:
|
||||
raise GiteaError("Epic managed-list markers are absent or ambiguous; preserve its body.")
|
||||
before, tail = body.split(LIST_START, 1)
|
||||
current, after = tail.split(LIST_END, 1)
|
||||
if current.strip() != INITIAL_LIST:
|
||||
if all(item["url"] and f"]({item['url']})" in current for item in records):
|
||||
return body # Never reset human checkboxes, evidence or subsequent edits.
|
||||
raise GiteaError("Epic list was already edited; update missing links manually without replacing progress.")
|
||||
return before + LIST_START + "\n" + render_links(records) + "\n" + LIST_END + after
|
||||
|
||||
|
||||
@contextmanager
|
||||
def ipv4_for_target(enabled: bool) -> Iterator[None]:
|
||||
original = socket.getaddrinfo
|
||||
def scoped(host: Any, port: Any, family: int = 0, type: int = 0, proto: int = 0, flags: int = 0) -> Any:
|
||||
return original(host, port, socket.AF_INET if host == "git.add-ideas.de" else family, type, proto, flags)
|
||||
if enabled:
|
||||
socket.getaddrinfo = scoped
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
socket.getaddrinfo = original
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--env-file", type=Path)
|
||||
parser.add_argument("--epic", type=int, required=True, help="Existing managed UI-review epic number in GovOPlaN/govoplan")
|
||||
parser.add_argument("--apply", action="store_true", help="Create missing review issues; existing issues remain untouched")
|
||||
parser.add_argument("--initialize-links", action="store_true", help="One-time initialization of the untouched epic module-list placeholder")
|
||||
parser.add_argument("--ipv4", action="store_true", help="Prefer IPv4 only for git.add-ideas.de; keep HTTPS verification")
|
||||
args = parser.parse_args()
|
||||
if args.epic <= 0 or (args.initialize_links and not args.apply):
|
||||
parser.error("A positive --epic is required; --initialize-links requires --apply.")
|
||||
try:
|
||||
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
|
||||
workspace_root = Path(catalog["default_parent"])
|
||||
scopes = build_scopes(catalog, workspace_root, extract_manifests(catalog, workspace_root))
|
||||
load_dotenv(args.env_file)
|
||||
token = require_token()
|
||||
target = RepoTarget(BASE_URL, OWNER, "govoplan")
|
||||
with ipv4_for_target(args.ipv4), GiteaClient(target, token) as central:
|
||||
epic = central.request_json("GET", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"))
|
||||
if EPIC_MARKER not in (epic.get("body") or "") or epic.get("state") != "open":
|
||||
raise GiteaError("Expected an open, managed UI-review epic; no child issues created.")
|
||||
epic_url = f"{BASE_URL}/{OWNER}/govoplan/issues/{args.epic}"
|
||||
org_labels = {item["name"]: item["id"] for item in central.paginate(org_path(OWNER, "/labels"))}
|
||||
|
||||
def reconcile(scope: dict[str, Any]) -> dict[str, Any]:
|
||||
repo = scope["repository"]
|
||||
with GiteaClient(RepoTarget(BASE_URL, OWNER, repo), token) as client:
|
||||
issues = client.paginate(repo_path(OWNER, repo, "/issues"), query={"state": "all", "type": "issues"})
|
||||
existing = find_existing(issues, scope["scope_id"], issue_title(scope))
|
||||
if existing:
|
||||
return result_record(scope, existing, "existing")
|
||||
if not args.apply:
|
||||
return result_record(scope, None, "would-create")
|
||||
labels = dict(org_labels)
|
||||
labels.update({item["name"]: item["id"] for item in client.paginate(repo_path(OWNER, repo, "/labels"))})
|
||||
desired = ["type/task", "area/webui", "area/docs", "priority/p2", f"module/{repo.removeprefix('govoplan-')}"]
|
||||
desired += ["status/triage"] if scope["kind"] == "placeholder" else ["status/ready", "codex/ready"]
|
||||
issue = client.request_json("POST", repo_path(OWNER, repo, "/issues"), body={
|
||||
"title": issue_title(scope), "body": issue_body(scope, epic_url),
|
||||
"labels": [labels[name] for name in desired if name in labels],
|
||||
})
|
||||
verified = client.request_json("GET", repo_path(OWNER, repo, f"/issues/{issue['number']}"))
|
||||
if marker(scope["scope_id"]) not in (verified.get("body") or "") or verified.get("state") != "open":
|
||||
raise GiteaError(f"New review issue verification failed for {scope['scope_id']}.")
|
||||
print(f"Created {repo}#{verified['number']} (pending)", file=sys.stderr, flush=True)
|
||||
return result_record(scope, verified, "created")
|
||||
|
||||
with ThreadPoolExecutor(max_workers=4) as executor:
|
||||
records = list(executor.map(reconcile, scopes))
|
||||
if args.initialize_links:
|
||||
fresh = central.request_json("GET", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"))
|
||||
body = initialized_epic_body(fresh["body"], records)
|
||||
if body != fresh["body"]:
|
||||
central.request_json("PATCH", repo_path(OWNER, "govoplan", f"/issues/{args.epic}"), body={"body": body})
|
||||
summary = {
|
||||
"schema_version": 1, "snapshot_purpose": "Issue discovery links; live Gitea issues own review state and evidence.",
|
||||
"epic": {"repository": "govoplan", "number": args.epic, "url": epic_url},
|
||||
"scope_count": len(scopes),
|
||||
"manifest_modules": sum(scope["kind"] == "manifest" for scope in scopes),
|
||||
"implemented_scopes": sum(scope["kind"] != "placeholder" for scope in scopes),
|
||||
"catalogued_placeholders": sum(scope["kind"] == "placeholder" for scope in scopes),
|
||||
"created": sum(item["operation"] == "created" for item in records),
|
||||
"missing": sum(item["operation"] == "would-create" for item in records),
|
||||
"issues": records,
|
||||
}
|
||||
print(json.dumps(summary, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
except (GiteaError, OSError, ValueError) as exc:
|
||||
print(f"UI review program: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -2133,6 +2133,34 @@
|
||||
"rationale": "This capability-first module intentionally exposes a headless API for other modules and integrations.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/summaries",
|
||||
"rationale": "Workflow Engine publishes bounded, current-authorized summary discovery for module and API consumers; the existing Workflow UI retains its compatible full-history contract. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/events",
|
||||
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded event history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/steps",
|
||||
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded step history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/summary",
|
||||
"rationale": "Workflow Engine publishes a current-authorized, history-free instance summary for module and API consumers; the existing Workflow UI retains its compatible full-history detail contract. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
@@ -2259,6 +2287,27 @@
|
||||
"path": "/tasks/{}",
|
||||
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
|
||||
"repository": "govoplan-tasks"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/connectors/tabular-sources/{}/original-csv",
|
||||
"rationale": "Authorized connector clients explicitly export retained original CSV after tenant, lifecycle, read-scope and source-integrity checks; ordinary catalogue responses never include source text.",
|
||||
"repository": "govoplan-connectors"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/datasources/{}/materializations/{}/original-csv",
|
||||
"rationale": "Administrators explicitly export retained original CSV through an audited API; current and historical field, row and access policies must permit the entire original and source-integrity checks must pass.",
|
||||
"repository": "govoplan-datasources"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/mail/profiles/{}/pop3/imports/{}/bind-maildrop",
|
||||
"rationale": "Authorized mail operators explicitly reconcile a legacy POP3 import to the current maildrop using confirmed binding, a current transport revision token and an exact retained/downloaded-byte match; the audited API never guesses historical account identity.",
|
||||
"repository": "govoplan-mail"
|
||||
}
|
||||
],
|
||||
"schema_version": 1
|
||||
|
||||
@@ -5,9 +5,16 @@ import path from "node:path";
|
||||
import { createHash } from "node:crypto";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const [metaRootArgument] = process.argv.slice(2);
|
||||
const [metaRootArgument, ...argumentsRest] = process.argv.slice(2);
|
||||
if (!metaRootArgument) {
|
||||
throw new Error("Usage: extract-webui-structure.mjs META_ROOT");
|
||||
throw new Error("Usage: extract-webui-structure.mjs META_ROOT [--workspace-root ROOT]");
|
||||
}
|
||||
let explicitWorkspaceRoot;
|
||||
for (let index = 0; index < argumentsRest.length; index++) {
|
||||
if (argumentsRest[index] !== "--workspace-root" || explicitWorkspaceRoot !== undefined || !argumentsRest[index + 1]) {
|
||||
throw new Error("Expected one --workspace-root ROOT option");
|
||||
}
|
||||
explicitWorkspaceRoot = path.resolve(argumentsRest[++index]);
|
||||
}
|
||||
|
||||
const metaRoot = path.resolve(metaRootArgument);
|
||||
@@ -15,12 +22,18 @@ const repositoryCatalog = JSON.parse(
|
||||
fs.readFileSync(path.join(metaRoot, "repositories.json"), "utf8")
|
||||
);
|
||||
const siblingWorkspaceRoot = path.dirname(metaRoot);
|
||||
const configuredWorkspaceRoot = path.resolve(repositoryCatalog.default_parent);
|
||||
const workspaceRoot = fs.existsSync(
|
||||
// Discovery is retained only for old direct callers. An explicit root wins
|
||||
// even when another configured checkout contains more optional modules.
|
||||
const workspaceRoot = explicitWorkspaceRoot ?? (fs.existsSync(
|
||||
path.join(siblingWorkspaceRoot, "govoplan-core", "webui")
|
||||
)
|
||||
? siblingWorkspaceRoot
|
||||
: configuredWorkspaceRoot;
|
||||
) ? siblingWorkspaceRoot : path.resolve(repositoryCatalog.default_parent));
|
||||
if (!fs.statSync(workspaceRoot).isDirectory()) throw new Error("Inventory workspace root must be an existing directory");
|
||||
const actualWorkspaceRoot = fs.realpathSync(workspaceRoot);
|
||||
function withinWorkspace(candidate) {
|
||||
const relative = path.relative(actualWorkspaceRoot, fs.realpathSync(candidate));
|
||||
return relative === "" || (!relative.startsWith(`..${path.sep}`) && relative !== ".." && !path.isAbsolute(relative));
|
||||
}
|
||||
if (!Array.isArray(repositoryCatalog.repositories)) throw new Error("Repository catalog requires a repositories array");
|
||||
const typescriptPath = path.join(
|
||||
workspaceRoot,
|
||||
"govoplan-core",
|
||||
@@ -96,6 +109,7 @@ const contributionTypes = new Map([
|
||||
]);
|
||||
|
||||
const result = {
|
||||
workspaceRoot: actualWorkspaceRoot,
|
||||
fields: [],
|
||||
actions: [],
|
||||
labels: [],
|
||||
@@ -111,8 +125,12 @@ const result = {
|
||||
};
|
||||
|
||||
for (const repository of repositoryCatalog.repositories) {
|
||||
const sourceRoot = path.join(workspaceRoot, repository.path, "webui", "src");
|
||||
if (!repository || typeof repository.path !== "string" || !repository.path || path.isAbsolute(repository.path) || repository.path.split(/[\\/]/).includes("..")) {
|
||||
throw new Error("Inventory repository paths must remain inside the selected workspace");
|
||||
}
|
||||
const sourceRoot = path.join(actualWorkspaceRoot, repository.path, "webui", "src");
|
||||
if (!fs.existsSync(sourceRoot)) continue;
|
||||
if (!withinWorkspace(sourceRoot)) throw new Error("Inventory source root escapes the selected workspace");
|
||||
for (const sourcePath of sourceFiles(sourceRoot)) {
|
||||
inspectSource(repository.name, sourceRoot, sourcePath);
|
||||
}
|
||||
@@ -135,7 +153,7 @@ function sourceFiles(root) {
|
||||
}
|
||||
const candidate = path.join(current, entry.name);
|
||||
if (entry.isDirectory()) pending.push(candidate);
|
||||
else if (/\.(?:ts|tsx)$/.test(entry.name)) files.push(candidate);
|
||||
else if (entry.isFile() && /\.(?:ts|tsx)$/.test(entry.name)) files.push(candidate);
|
||||
}
|
||||
}
|
||||
return files.sort();
|
||||
@@ -150,7 +168,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
true,
|
||||
sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS
|
||||
);
|
||||
const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath);
|
||||
const relativeFile = path.relative(path.resolve(sourceRoot, "..", ".."), sourcePath);
|
||||
const identityCounters = new Map();
|
||||
|
||||
function location(node) {
|
||||
|
||||
@@ -9,6 +9,7 @@ from collections import Counter
|
||||
from dataclasses import asdict, is_dataclass
|
||||
import importlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
@@ -40,6 +41,11 @@ REFERENCE_LOCALE = "de"
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--workspace-root",
|
||||
type=Path,
|
||||
help="Authoritative directory containing registered checkouts; never falls back to another workspace.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
@@ -87,8 +93,9 @@ def main() -> int:
|
||||
args = parser.parse_args()
|
||||
|
||||
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
|
||||
workspace_root = _resolve_workspace_root(catalog)
|
||||
webui = _extract_webui()
|
||||
workspace_root = _resolve_workspace_root(catalog, args.workspace_root)
|
||||
_validate_repository_roots(catalog, workspace_root)
|
||||
webui = _extract_webui(workspace_root)
|
||||
backend_endpoints = _extract_backend_endpoints(catalog, workspace_root)
|
||||
manifests = _extract_manifests(catalog, workspace_root)
|
||||
endpoint_declarations = _load_endpoint_declarations(
|
||||
@@ -109,6 +116,10 @@ def main() -> int:
|
||||
else None
|
||||
),
|
||||
)
|
||||
inventory["workspace_root"] = str(workspace_root)
|
||||
inventory["workspace_selection"] = (
|
||||
"explicit" if args.workspace_root is not None else "legacy-discovery"
|
||||
)
|
||||
|
||||
output_dir = args.output_dir.resolve()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -212,7 +223,16 @@ def _strict_failures(
|
||||
return failures
|
||||
|
||||
|
||||
def _resolve_workspace_root(catalog: dict[str, Any]) -> Path:
|
||||
def _resolve_workspace_root(
|
||||
catalog: dict[str, Any], explicit_root: Path | None = None
|
||||
) -> Path:
|
||||
if explicit_root is not None:
|
||||
root = explicit_root.expanduser().resolve()
|
||||
if not root.is_dir():
|
||||
raise ValueError("The explicit inventory workspace root must be an existing directory")
|
||||
return root
|
||||
# Compatibility for direct legacy callers only. Managed callers always
|
||||
# supply their selected root; a partial checkout must not borrow sources.
|
||||
sibling_root = META_ROOT.parent.resolve()
|
||||
configured_root = Path(str(catalog["default_parent"])).expanduser().resolve()
|
||||
repositories = catalog.get("repositories")
|
||||
@@ -233,15 +253,44 @@ def _resolve_workspace_root(catalog: dict[str, Any]) -> Path:
|
||||
return sibling_root if sibling_count >= configured_count else configured_root
|
||||
|
||||
|
||||
def _extract_webui() -> dict[str, Any]:
|
||||
def _validate_repository_roots(catalog: dict[str, Any], workspace_root: Path) -> None:
|
||||
repositories = catalog.get("repositories")
|
||||
if not isinstance(repositories, list):
|
||||
raise ValueError("repository catalog has no repositories array")
|
||||
for repository in repositories:
|
||||
if not isinstance(repository, dict) or not isinstance(repository.get("path"), str):
|
||||
raise ValueError("Invalid inventory repository path")
|
||||
relative = Path(repository["path"])
|
||||
if not repository["path"] or relative.is_absolute() or ".." in relative.parts:
|
||||
raise ValueError("Inventory repository paths must remain inside the selected workspace")
|
||||
root = workspace_root / relative
|
||||
# Missing optional checkouts are allowed; links to another checkout are
|
||||
# not evidence for the selected workspace.
|
||||
if not root.resolve().is_relative_to(workspace_root):
|
||||
raise ValueError("Inventory repository path escapes the selected workspace")
|
||||
for source in (root / "src", root / "webui/src"):
|
||||
if not source.resolve().is_relative_to(workspace_root):
|
||||
raise ValueError("Inventory source root escapes the selected workspace")
|
||||
|
||||
|
||||
def _extract_webui(workspace_root: Path | None = None) -> dict[str, Any]:
|
||||
helper = META_ROOT / "tools" / "inventory" / "extract-webui-structure.mjs"
|
||||
argv = [os.environ.get("NODE", "node"), str(helper), str(META_ROOT)]
|
||||
if workspace_root is not None:
|
||||
argv.extend(["--workspace-root", str(workspace_root)])
|
||||
completed = subprocess.run(
|
||||
["node", str(helper), str(META_ROOT)],
|
||||
argv,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return json.loads(completed.stdout)
|
||||
result = json.loads(completed.stdout)
|
||||
if workspace_root is not None and (
|
||||
not isinstance(result, dict)
|
||||
or result.get("workspaceRoot") != str(workspace_root.resolve())
|
||||
):
|
||||
raise ValueError("WebUI collector did not confirm the selected inventory workspace")
|
||||
return result
|
||||
|
||||
|
||||
def _extract_backend_endpoints(
|
||||
@@ -255,6 +304,8 @@ def _extract_backend_endpoints(
|
||||
if not source_root.is_dir():
|
||||
continue
|
||||
for source_path in sorted(source_root.rglob("*.py")):
|
||||
if not source_path.resolve().is_relative_to(workspace_root):
|
||||
raise ValueError("Backend source path escapes the selected workspace")
|
||||
try:
|
||||
tree = ast.parse(
|
||||
source_path.read_text(encoding="utf-8"),
|
||||
@@ -346,15 +397,26 @@ def _extract_manifests(
|
||||
catalog: dict[str, Any],
|
||||
workspace_root: Path,
|
||||
) -> list[dict[str, Any]]:
|
||||
_validate_repository_roots(catalog, workspace_root)
|
||||
source_roots = [
|
||||
workspace_root / repository["path"] / "src"
|
||||
for repository in catalog["repositories"]
|
||||
if (workspace_root / repository["path"] / "src").is_dir()
|
||||
]
|
||||
core_root = next(
|
||||
(workspace_root / repository["path"] / "src"
|
||||
for repository in catalog["repositories"]
|
||||
if repository.get("name") == "govoplan-core"),
|
||||
workspace_root / "govoplan-core/src",
|
||||
)
|
||||
if not (core_root / "govoplan_core/core/platform_interfaces.py").is_file():
|
||||
raise ValueError("Inventory requires Core interface sources in the selected workspace")
|
||||
_assert_workspace_imports(workspace_root)
|
||||
sys.path[:0] = [str(path) for path in source_roots]
|
||||
from govoplan_core.core.platform_interfaces import ( # noqa: PLC0415
|
||||
manifest_interface_catalog,
|
||||
)
|
||||
_assert_workspace_imports(workspace_root)
|
||||
|
||||
manifests: list[dict[str, Any]] = []
|
||||
for repository in catalog["repositories"]:
|
||||
@@ -366,7 +428,12 @@ def _extract_manifests(
|
||||
manifest_path.relative_to(source_root).with_suffix("").parts
|
||||
)
|
||||
loaded = importlib.import_module(module_name)
|
||||
source = getattr(loaded, "__file__", None)
|
||||
if not isinstance(source, str) or Path(source).resolve() != manifest_path.resolve():
|
||||
raise ValueError("Manifest import did not resolve to its selected workspace source")
|
||||
_assert_workspace_imports(workspace_root)
|
||||
manifest = loaded.get_manifest()
|
||||
_assert_workspace_imports(workspace_root)
|
||||
frontend = manifest.frontend
|
||||
manifests.append(
|
||||
{
|
||||
@@ -452,6 +519,26 @@ def _extract_manifests(
|
||||
return sorted(manifests, key=lambda item: item["id"])
|
||||
|
||||
|
||||
def _assert_workspace_imports(workspace_root: Path) -> None:
|
||||
# An editable installation or cached import must not stand in for a missing
|
||||
# optional checkout. Direct callers with another workspace use a fresh
|
||||
# process instead of replacing already-loaded application packages.
|
||||
for name, module in list(sys.modules.items()):
|
||||
# The Meta tools may audit a separate checkout; they are not module
|
||||
# contributions and must not be confused with application packages.
|
||||
package = name.partition(".")[0]
|
||||
if not package.startswith("govoplan_") or package in {
|
||||
"govoplan_devkit", "govoplan_release"
|
||||
}:
|
||||
continue
|
||||
filename = getattr(module, "__file__", None)
|
||||
locations = list(getattr(module, "__path__", ()))
|
||||
if isinstance(filename, str):
|
||||
locations.append(filename)
|
||||
if any(not Path(location).resolve().is_relative_to(workspace_root) for location in locations):
|
||||
raise ValueError("A GovOPlaN import originates outside the selected inventory workspace")
|
||||
|
||||
|
||||
def _assemble_inventory(
|
||||
*,
|
||||
webui: dict[str, Any],
|
||||
|
||||
@@ -11,7 +11,7 @@ import tomllib
|
||||
|
||||
from .contracts import parse_manifest_contract
|
||||
from .model import RepositorySnapshot, RepositorySpec, VersionSnapshot
|
||||
from .workspace import resolve_repo_path
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
def collect_repository_snapshot(
|
||||
@@ -98,6 +98,7 @@ def collect_repository_snapshot(
|
||||
def collect_versions(path: Path) -> VersionSnapshot:
|
||||
return VersionSnapshot(
|
||||
pyproject=read_pyproject_version(path),
|
||||
developer_meta=read_developer_meta_version(path),
|
||||
package=read_json_version(path / "package.json"),
|
||||
webui_package=read_json_version(path / "webui" / "package.json"),
|
||||
manifests=read_manifest_versions(path),
|
||||
@@ -105,6 +106,35 @@ def collect_versions(path: Path) -> VersionSnapshot:
|
||||
)
|
||||
|
||||
|
||||
def registered_developer_meta_path(path: Path) -> Path | None:
|
||||
"""Recognize only the catalog's explicit Meta support-repository identity.
|
||||
|
||||
This is metadata discovery, not authorization to access a remote or mutate
|
||||
a checkout. Tagging applies its separate registered source trust contract.
|
||||
"""
|
||||
for spec in load_repository_specs(include_website=False):
|
||||
if (
|
||||
spec.name == "govoplan"
|
||||
and spec.category == "system"
|
||||
and spec.subtype == "meta"
|
||||
and path.absolute() == resolve_repo_path(spec, path.parent).absolute()
|
||||
):
|
||||
return path / "packages" / "govoplan-meta" / "pyproject.toml"
|
||||
return None
|
||||
|
||||
|
||||
def read_developer_meta_version(path: Path) -> str | None:
|
||||
package = registered_developer_meta_path(path)
|
||||
if package is None or not package.is_file():
|
||||
return None
|
||||
with package.open("rb") as handle:
|
||||
project = tomllib.load(handle).get("project")
|
||||
if isinstance(project, dict) and project.get("name") == "govoplan":
|
||||
version = project.get("version")
|
||||
return version if isinstance(version, str) else None
|
||||
return None
|
||||
|
||||
|
||||
def read_pyproject_version(path: Path) -> str | None:
|
||||
pyproject = path / "pyproject.toml"
|
||||
if not pyproject.exists():
|
||||
|
||||
@@ -0,0 +1,380 @@
|
||||
"""Receipt-bound, out-of-run preparation of the real developer meta-package.
|
||||
|
||||
This deliberately does not commit, tag, publish, or update a running release
|
||||
console. The complete generated file is reviewed in a separate source checkout.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import stat
|
||||
import tempfile
|
||||
import tomllib
|
||||
|
||||
from .git_state import (
|
||||
collect_repository_snapshot,
|
||||
git,
|
||||
git_text,
|
||||
registered_developer_meta_path,
|
||||
)
|
||||
from .repository_tag import normalize_version, remote_tag_commit, run
|
||||
from .source_provenance import registered_source_origin_issues
|
||||
from .source_tag_batch import (
|
||||
_OBJECT,
|
||||
_owned_path,
|
||||
_source_filesystem,
|
||||
_tracked_worktree,
|
||||
_trusted_ancestry,
|
||||
)
|
||||
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
|
||||
|
||||
PACKAGE = "packages/govoplan-meta/pyproject.toml"
|
||||
MAX_INPUT_FILES = 128
|
||||
MAX_INPUT_BYTES = 2 * 1024 * 1024
|
||||
MAX_TOTAL_BYTES = 16 * 1024 * 1024
|
||||
GENERATOR = ".operator/generate-developer-meta-package.py"
|
||||
|
||||
|
||||
class MetaPreparationError(ValueError):
|
||||
"""Preparation is blocked, or an applied file needs explicit reconciliation."""
|
||||
|
||||
|
||||
class MetaPreparationAmbiguous(MetaPreparationError):
|
||||
"""The file effect may have happened and requires explicit reconciliation."""
|
||||
|
||||
|
||||
def preparation_command(*, workspace: Path, target_version: str) -> str:
|
||||
import shlex
|
||||
|
||||
return " ".join(
|
||||
shlex.quote(value)
|
||||
for value in (
|
||||
"python",
|
||||
str(META_ROOT / "tools/release/prepare-developer-meta-package.py"),
|
||||
"--workspace",
|
||||
str(workspace),
|
||||
"--target-version",
|
||||
target_version,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _read_input(path: Path) -> tuple[bytes, dict]:
|
||||
def identity(value):
|
||||
return (
|
||||
value.st_dev,
|
||||
value.st_ino,
|
||||
value.st_uid,
|
||||
value.st_gid,
|
||||
value.st_mode,
|
||||
value.st_size,
|
||||
value.st_mtime_ns,
|
||||
value.st_ctime_ns,
|
||||
)
|
||||
|
||||
before = path.lstat()
|
||||
if (
|
||||
not stat.S_ISREG(before.st_mode)
|
||||
or before.st_uid != os.geteuid()
|
||||
or before.st_mode & 0o022
|
||||
or not 0 < before.st_size <= MAX_INPUT_BYTES
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs must be owned, bounded regular files."
|
||||
)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
opened = os.fstat(source.fileno())
|
||||
if identity(opened) != identity(before):
|
||||
raise MetaPreparationError("Preparation input changed before reading.")
|
||||
payload = source.read(before.st_size + 1)
|
||||
if (
|
||||
identity(os.fstat(source.fileno())) != identity(opened)
|
||||
or len(payload) != before.st_size
|
||||
):
|
||||
raise MetaPreparationError("Preparation input changed while reading.")
|
||||
return payload, {
|
||||
"sha256": hashlib.sha256(payload).hexdigest(),
|
||||
"identity": [
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_uid,
|
||||
before.st_gid,
|
||||
stat.S_IMODE(before.st_mode),
|
||||
before.st_size,
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _snapshot(*, repo_path: Path, target_version: str, output_dirty: bool = False):
|
||||
workspace = repo_path.parent
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
if registered_developer_meta_path(repo_path) != repo_path / PACKAGE:
|
||||
raise MetaPreparationError(
|
||||
"Only the registered Meta nested-package identity can be prepared."
|
||||
)
|
||||
if repo_path.resolve() == META_ROOT.resolve():
|
||||
raise MetaPreparationError(
|
||||
"Prepare a separate registered source checkout, never the running operator tooling."
|
||||
)
|
||||
paths = [repo_path / PACKAGE, repo_path / "requirements-release.txt"]
|
||||
for path in workspace.glob("govoplan-*/pyproject.toml"):
|
||||
paths.append(path)
|
||||
if len(paths) > MAX_INPUT_FILES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its input-file bound."
|
||||
)
|
||||
if workspace / "govoplan-core/pyproject.toml" not in paths:
|
||||
raise MetaPreparationError(
|
||||
"Prepare and commit matching Core metadata before Meta preparation."
|
||||
)
|
||||
repositories = {"govoplan": repo_path}
|
||||
for path in paths[2:]:
|
||||
name = path.parent.name
|
||||
if (
|
||||
name not in specs
|
||||
or resolve_repo_path(specs[name], workspace) != path.parent
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Developer composition contains an unregistered package checkout."
|
||||
)
|
||||
repositories[name] = path.parent
|
||||
filesystems = {}
|
||||
for name, path in repositories.items():
|
||||
filesystems[name] = _source_filesystem(path=path, workspace=workspace)
|
||||
_tracked_worktree(path)
|
||||
issues = registered_source_origin_issues(
|
||||
repo_versions={name: target_version for name in repositories},
|
||||
workspace=workspace,
|
||||
remote="origin",
|
||||
)
|
||||
if issues:
|
||||
raise MetaPreparationError(
|
||||
"Preparation source origins do not match the registered repositories."
|
||||
)
|
||||
sources = {}
|
||||
for name, path in sorted(repositories.items()):
|
||||
snapshot = collect_repository_snapshot(
|
||||
specs[name],
|
||||
workspace_root=workspace,
|
||||
target_tag=None,
|
||||
online=False,
|
||||
)
|
||||
dirty_allowed = (
|
||||
output_dirty
|
||||
and name == "govoplan"
|
||||
and snapshot.dirty_entries == (f" M {PACKAGE}",)
|
||||
)
|
||||
if (
|
||||
snapshot.errors
|
||||
or not snapshot.has_head
|
||||
or snapshot.branch != "main"
|
||||
or snapshot.upstream != "origin/main"
|
||||
or snapshot.behind
|
||||
or (snapshot.dirty and not dirty_allowed)
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation requires reviewed clean main sources tracking origin/main."
|
||||
)
|
||||
head = git_text(path, "rev-parse", "--verify", "HEAD")
|
||||
live = run(
|
||||
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
|
||||
cwd=path,
|
||||
)
|
||||
lines = live.stdout.strip().splitlines()
|
||||
if live.returncode or len(lines) != 1:
|
||||
raise MetaPreparationError("Could not verify live preparation source main.")
|
||||
remote_main, separator, reference = lines[0].partition("\t")
|
||||
if (
|
||||
not _OBJECT.fullmatch(head)
|
||||
or not _OBJECT.fullmatch(remote_main)
|
||||
or not separator
|
||||
or reference != "refs/heads/main"
|
||||
or git(path, "merge-base", "--is-ancestor", remote_main, head).returncode
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation source main diverged; fetch and review separately."
|
||||
)
|
||||
sources[name] = {
|
||||
"head": head,
|
||||
"remote_main": remote_main,
|
||||
"filesystem": filesystems[name],
|
||||
}
|
||||
tag = f"v{target_version}"
|
||||
published = remote_tag_commit(repo_path, remote="origin", tag=tag)
|
||||
if (
|
||||
published.error
|
||||
or published.tag_object
|
||||
or git_text(repo_path, "rev-parse", "--verify", f"refs/tags/{tag}")
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"An existing or unverifiable target Meta tag blocks source preparation."
|
||||
)
|
||||
inputs, payloads = {}, {}
|
||||
total = 0
|
||||
for path in sorted(paths):
|
||||
payload, identity = _read_input(path)
|
||||
total += len(payload)
|
||||
if total > MAX_TOTAL_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its aggregate input bound."
|
||||
)
|
||||
relative = path.relative_to(workspace).as_posix()
|
||||
inputs[relative], payloads[relative] = identity, payload
|
||||
generator_path = META_ROOT / "tools/release/generate-developer-meta-package.py"
|
||||
_trusted_ancestry(generator_path.parent)
|
||||
_owned_path(META_ROOT, directory=True)
|
||||
generator, generator_identity = _read_input(generator_path)
|
||||
if total + len(generator) > MAX_TOTAL_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its aggregate input bound."
|
||||
)
|
||||
payloads[GENERATOR] = generator
|
||||
current = tomllib.loads(payloads[f"govoplan/{PACKAGE}"].decode("utf-8")).get(
|
||||
"project", {}
|
||||
)
|
||||
core = tomllib.loads(payloads["govoplan-core/pyproject.toml"].decode("utf-8")).get(
|
||||
"project", {}
|
||||
)
|
||||
if (
|
||||
not isinstance(current, dict)
|
||||
or current.get("name") != "govoplan"
|
||||
or not isinstance(current.get("version"), str)
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Nested developer-package identity and version must be exact."
|
||||
)
|
||||
if (
|
||||
not isinstance(core, dict)
|
||||
or core.get("name") != "govoplan-core"
|
||||
or core.get("version") != target_version
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Prepare and commit Core at the requested target version before Meta preparation."
|
||||
)
|
||||
from .version_alignment import repository_version_issues
|
||||
|
||||
if repository_version_issues(
|
||||
workspace / "govoplan-core", expected_version=target_version
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Prepare aligned Core version metadata before Meta preparation."
|
||||
)
|
||||
receipt = {
|
||||
"kind": "developer_meta_preparation_v1",
|
||||
"workspace": str(workspace),
|
||||
"target_version": target_version,
|
||||
"sources": sources,
|
||||
"inputs": inputs,
|
||||
"operator_generator": generator_identity,
|
||||
}
|
||||
return receipt, payloads
|
||||
|
||||
|
||||
def _render(payloads: dict[str, bytes]) -> bytes:
|
||||
# The trusted operator generator sees only the frozen bounded data snapshot.
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-meta-render-") as temporary:
|
||||
workspace = Path(temporary)
|
||||
for relative, payload in payloads.items():
|
||||
path = workspace / relative
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(payload)
|
||||
render = runpy.run_path(str(workspace / GENERATOR))["render"]
|
||||
result = render(
|
||||
workspace=workspace,
|
||||
requirements=workspace / "govoplan/requirements-release.txt",
|
||||
).encode("utf-8")
|
||||
if len(result) > MAX_INPUT_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Generated developer package exceeds its output bound."
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def preview_meta_mutation(*, repo_path: Path, target_version: str):
|
||||
version = normalize_version(target_version)
|
||||
if not version:
|
||||
raise MetaPreparationError("A valid target release version is required.")
|
||||
receipt, payloads = _snapshot(repo_path=repo_path, target_version=version)
|
||||
after = _render(payloads)
|
||||
observed, _ = _snapshot(repo_path=repo_path, target_version=version)
|
||||
if observed != receipt:
|
||||
raise MetaPreparationError(
|
||||
"Developer preparation inputs changed during preview."
|
||||
)
|
||||
receipt["output_sha256"] = hashlib.sha256(after).hexdigest()
|
||||
return receipt, payloads[f"govoplan/{PACKAGE}"], after
|
||||
|
||||
|
||||
def prepare_developer_meta_package(
|
||||
*,
|
||||
repo_path: Path,
|
||||
target_version: str,
|
||||
apply: bool = False,
|
||||
expected_receipt=None,
|
||||
confirm_out_of_run: bool = False,
|
||||
) -> dict:
|
||||
"""Preview or explicitly apply one full generated file; never commit/publish."""
|
||||
try:
|
||||
receipt, before, after = preview_meta_mutation(
|
||||
repo_path=repo_path, target_version=target_version
|
||||
)
|
||||
result = {
|
||||
"status": "planned" if before != after else "noop",
|
||||
"path": PACKAGE,
|
||||
"receipt": receipt,
|
||||
"after_sha256": hashlib.sha256(after).hexdigest(),
|
||||
"changed": before != after,
|
||||
}
|
||||
if not apply:
|
||||
return result
|
||||
if not confirm_out_of_run:
|
||||
raise MetaPreparationError(
|
||||
"Explicitly confirm that no durable run is active for this source workspace."
|
||||
)
|
||||
if receipt != expected_receipt:
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs changed since the reviewed preview; create a fresh preview."
|
||||
)
|
||||
if before == after:
|
||||
return result
|
||||
from .version_metadata import _atomic_write
|
||||
|
||||
source_receipt = {
|
||||
key: value for key, value in receipt.items() if key != "output_sha256"
|
||||
}
|
||||
observed, _ = _snapshot(
|
||||
repo_path=repo_path, target_version=receipt["target_version"]
|
||||
)
|
||||
if observed != source_receipt:
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs changed before the file effect; create a fresh preview."
|
||||
)
|
||||
try:
|
||||
_atomic_write(repo_path / PACKAGE, after)
|
||||
observed, payloads = _snapshot(
|
||||
repo_path=repo_path,
|
||||
target_version=receipt["target_version"],
|
||||
output_dirty=True,
|
||||
)
|
||||
comparison = copy.deepcopy(observed)
|
||||
output = f"govoplan/{PACKAGE}"
|
||||
comparison["inputs"][output] = receipt["inputs"][output]
|
||||
if comparison != source_receipt or payloads[output] != after:
|
||||
raise MetaPreparationError("Preparation inputs changed after writing.")
|
||||
except Exception as exc:
|
||||
raise MetaPreparationAmbiguous(
|
||||
"Generated file may have been written but its write/post-check failed; "
|
||||
"review the delta and reconcile manually."
|
||||
) from exc
|
||||
return {**result, "status": "prepared", "after_receipt": observed}
|
||||
except MetaPreparationError:
|
||||
raise
|
||||
except (OSError, UnicodeError, ValueError, KeyError, TypeError) as exc:
|
||||
raise MetaPreparationError(
|
||||
f"Developer preparation failed closed ({type(exc).__name__})."
|
||||
) from exc
|
||||
@@ -23,6 +23,7 @@ class RepositorySpec:
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VersionSnapshot:
|
||||
pyproject: str | None = None
|
||||
developer_meta: str | None = None
|
||||
package: str | None = None
|
||||
webui_package: str | None = None
|
||||
manifests: tuple[str, ...] = ()
|
||||
@@ -32,6 +33,7 @@ class VersionSnapshot:
|
||||
def primary(self) -> str | None:
|
||||
return (
|
||||
self.pyproject
|
||||
or self.developer_meta
|
||||
or self.package
|
||||
or self.webui_package
|
||||
or (self.manifests[0] if self.manifests else None)
|
||||
|
||||
@@ -34,7 +34,24 @@ def tag_repositories(
|
||||
apply: bool = False, # noqa: A002 - mirrors API field.
|
||||
push: bool = False,
|
||||
) -> dict[str, object]:
|
||||
"""Create annotated tags and optionally publish branch and tag atomically.
|
||||
from .source_tag_batch import tag_source_batch
|
||||
|
||||
return tag_source_batch(
|
||||
repos=repos, repo_versions=repo_versions, workspace_root=workspace_root,
|
||||
remote=remote, message=message, apply=apply, push=push,
|
||||
)
|
||||
|
||||
|
||||
def _preview_repositories(
|
||||
*,
|
||||
repos: tuple[str, ...],
|
||||
repo_versions: dict[str, str],
|
||||
workspace_root: Path | str | None = None,
|
||||
remote: str = "origin",
|
||||
message: str | None = None,
|
||||
push: bool = False,
|
||||
) -> dict[str, object]:
|
||||
"""Read-only shared manifest/version/composition/tag preflight.
|
||||
|
||||
A release tag is only created for a clean, aligned, non-behind worktree.
|
||||
Both local and remote tags are resolved to commits before mutation so an
|
||||
@@ -64,42 +81,7 @@ def tag_repositories(
|
||||
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
|
||||
)
|
||||
|
||||
if apply:
|
||||
preflight = tag_repositories(
|
||||
repos=selected,
|
||||
repo_versions=repo_versions,
|
||||
workspace_root=workspace,
|
||||
remote=remote,
|
||||
message=message,
|
||||
apply=False,
|
||||
push=push,
|
||||
)
|
||||
preflight_rows = preflight.get("repositories")
|
||||
if isinstance(preflight_rows, list) and any(
|
||||
isinstance(item, dict) and item.get("status") in {"blocked", "failed"}
|
||||
for item in preflight_rows
|
||||
):
|
||||
blocked_rows = []
|
||||
for item in preflight_rows:
|
||||
if not isinstance(item, dict) or item.get("status") in {"blocked", "failed"}:
|
||||
blocked_rows.append(item)
|
||||
continue
|
||||
blocked_rows.append(
|
||||
{
|
||||
**item,
|
||||
"status": "skipped",
|
||||
"detail": "preflight passed, but no release tag was changed because another selected repository is blocked",
|
||||
}
|
||||
)
|
||||
return {
|
||||
"status": "blocked",
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": remote,
|
||||
"detail": "batch preflight failed; no selected repository was mutated",
|
||||
"repositories": blocked_rows,
|
||||
}
|
||||
elif selected:
|
||||
if selected:
|
||||
manifest_gate_issue = manifest_shape_gate_issue(workspace)
|
||||
if manifest_gate_issue:
|
||||
return {
|
||||
@@ -255,130 +237,17 @@ def tag_repositories(
|
||||
"remote_tag_object": remote_result.tag_object,
|
||||
}
|
||||
)
|
||||
if not apply:
|
||||
detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push)
|
||||
status = "noop" if remote_result.commit or (local_commit and not push) else "planned"
|
||||
results.append({**row, "status": status, "detail": detail})
|
||||
continue
|
||||
|
||||
if remote_result.commit:
|
||||
if not local_commit:
|
||||
fetch_result = run(("git", "fetch", remote, f"refs/tags/{tag}:refs/tags/{tag}"), cwd=path)
|
||||
if fetch_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"remote tag {tag} exists at HEAD but could not be fetched locally",
|
||||
"returncode": fetch_result.returncode,
|
||||
"stdout": compact_output(fetch_result.stdout),
|
||||
"stderr": compact_output(fetch_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published",
|
||||
"detail": f"immutable tag {tag} is already published at HEAD",
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_remote_tag_commit": head_commit,
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
created = False
|
||||
if not local_commit:
|
||||
create_result = run(create_command, cwd=path)
|
||||
if create_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"could not create annotated tag {tag}",
|
||||
"returncode": create_result.returncode,
|
||||
"stdout": compact_output(create_result.stdout),
|
||||
"stderr": compact_output(create_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
created = True
|
||||
|
||||
if not push:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "tagged" if created else "noop",
|
||||
"detail": f"created annotated tag {tag} at HEAD" if created else f"annotated tag {tag} already exists at HEAD",
|
||||
"after_local_tag_commit": head_commit,
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
publish_result = run(publish_command, cwd=path)
|
||||
if publish_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"created local tag {tag}, but atomic branch and tag publication failed" if created else f"atomic branch and tag publication failed for {tag}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
after_local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}")
|
||||
after_remote = remote_tag_commit(path, remote=remote, tag=tag)
|
||||
if (
|
||||
after_remote.error
|
||||
or not after_remote.annotated
|
||||
or after_remote.commit != head_commit
|
||||
or after_remote.tag_object != after_local_object
|
||||
):
|
||||
verification_detail = after_remote.error or "remote tag did not resolve to the published annotated tag object at HEAD"
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"Git push returned success, but the remote release-tag postcondition failed: {verification_detail}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_local_tag_object": after_local_object,
|
||||
"after_remote_tag_commit": after_remote.commit,
|
||||
"after_remote_tag_object": after_remote.tag_object,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published",
|
||||
"detail": f"published branch {snapshot.branch} and immutable tag {tag} atomically to {remote}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_remote_tag_commit": head_commit,
|
||||
"after_local_tag_object": after_local_object,
|
||||
"after_remote_tag_object": after_remote.tag_object,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push)
|
||||
row_status = "noop" if remote_result.commit or (local_commit and not push) else "planned"
|
||||
results.append({**row, "status": row_status, "detail": detail})
|
||||
|
||||
if any(item["status"] in {"blocked", "failed"} for item in results):
|
||||
status = "blocked" if not apply else "partial"
|
||||
elif any(item["status"] == "published" for item in results):
|
||||
status = "published"
|
||||
elif any(item["status"] == "tagged" for item in results):
|
||||
status = "tagged"
|
||||
result_status = "blocked"
|
||||
elif any(item["status"] == "planned" for item in results):
|
||||
status = "planned"
|
||||
result_status = "planned"
|
||||
else:
|
||||
status = "noop"
|
||||
return {"status": status, "apply": apply, "push": push, "remote": remote, "repositories": results}
|
||||
result_status = "noop"
|
||||
return {"status": result_status, "apply": False, "push": push, "remote": remote, "repositories": results}
|
||||
|
||||
|
||||
def normalize_version(value: str | None) -> str:
|
||||
|
||||
@@ -8,6 +8,7 @@ from pathlib import Path
|
||||
import shlex
|
||||
|
||||
from .contracts import validate_contracts
|
||||
from .git_state import registered_developer_meta_path, read_pyproject_version
|
||||
from .model import (
|
||||
CompatibilityIssue,
|
||||
InterfaceProviderSnapshot,
|
||||
@@ -111,6 +112,39 @@ def apply_repository_version_gate(
|
||||
version_update_supported_by_repo: dict[str, bool] = {}
|
||||
deferred_core_lock_repos: set[str] = set()
|
||||
for unit in units:
|
||||
if registered_developer_meta_path(workspace / unit.repo) is not None:
|
||||
from .meta_preparation import preparation_command
|
||||
|
||||
try:
|
||||
core_version = read_pyproject_version(workspace / "govoplan-core")
|
||||
except (OSError, ValueError, TypeError):
|
||||
core_version = None
|
||||
core_ready = core_version == unit.target_version
|
||||
issues_by_repo.setdefault(unit.repo, []).append(
|
||||
ReleaseGateFinding(
|
||||
code="developer_meta_out_of_run" if core_ready else "developer_meta_core_preparation_required",
|
||||
severity="blocker",
|
||||
message=(
|
||||
"Meta is an out-of-run support release, not a self-updating durable executor."
|
||||
if core_ready else
|
||||
"Prepare and commit Core at the requested target before regenerating Meta."
|
||||
),
|
||||
remediation=(
|
||||
"Complete Core and module preparation first. Stop active durable runs for this workspace. "
|
||||
"In a separate trusted source checkout, preview "
|
||||
+ preparation_command(workspace=workspace, target_version=unit.target_version)
|
||||
+ ". Review its receipt; apply with --receipt <preview.json> --apply --confirm-out-of-run. "
|
||||
"Review and commit the whole generated package, publish the matching Core release first, "
|
||||
"then use guarded Meta source tagging/publication and create a fresh durable run."
|
||||
),
|
||||
repo=unit.repo, source="developer meta-package preparation",
|
||||
expected=unit.target_version, actual=core_version or "missing Core version",
|
||||
)
|
||||
)
|
||||
version_update_supported_by_repo[unit.repo] = False
|
||||
# Its complete canonical composition remains a publication gate;
|
||||
# this plan must not claim a generic in-run mutation/commit path.
|
||||
continue
|
||||
version_update_supported = unit.current_version == unit.target_version
|
||||
if unit.current_version and unit.current_version != unit.target_version:
|
||||
try:
|
||||
@@ -437,6 +471,7 @@ def build_unit(
|
||||
value
|
||||
for value in (
|
||||
repo.versions.pyproject,
|
||||
repo.versions.developer_meta,
|
||||
repo.versions.package,
|
||||
repo.versions.webui_package,
|
||||
*repo.versions.manifests,
|
||||
@@ -572,7 +607,7 @@ def repository_capabilities(
|
||||
def dependency_ordered_units(
|
||||
units: tuple[ReleasePlanUnit, ...],
|
||||
) -> tuple[ReleasePlanUnit, ...]:
|
||||
"""Order module providers before consumers while keeping Core last."""
|
||||
"""Order modules before Core, followed by the out-of-run Meta support unit."""
|
||||
|
||||
by_repo = {unit.repo: unit for unit in units}
|
||||
providers: dict[str, set[str]] = {}
|
||||
@@ -592,8 +627,10 @@ def dependency_ordered_units(
|
||||
)
|
||||
if "govoplan-core" in dependencies:
|
||||
dependencies["govoplan-core"].update(
|
||||
repo for repo in by_repo if repo != "govoplan-core"
|
||||
repo for repo in by_repo if repo not in {"govoplan-core", "govoplan"}
|
||||
)
|
||||
if "govoplan" in dependencies:
|
||||
dependencies["govoplan"].update(repo for repo in by_repo if repo != "govoplan")
|
||||
|
||||
ordered: list[ReleasePlanUnit] = []
|
||||
remaining = set(by_repo)
|
||||
@@ -690,6 +727,8 @@ def dry_run_steps(
|
||||
*, units: tuple[ReleasePlanUnit, ...], dashboard: ReleaseDashboard, channel: str
|
||||
) -> tuple[ReleasePlanStep, ...]:
|
||||
steps: list[ReleasePlanStep] = []
|
||||
meta_units = tuple(unit for unit in units if unit.repo == "govoplan")
|
||||
units = tuple(unit for unit in units if unit.repo != "govoplan")
|
||||
snapshots = {repo.spec.name: repo for repo in dashboard.repositories}
|
||||
core_unit = next((unit for unit in units if unit.repo == "govoplan-core"), None)
|
||||
non_core_units = tuple(unit for unit in units if unit.repo != "govoplan-core")
|
||||
@@ -991,6 +1030,33 @@ def dry_run_steps(
|
||||
status="planned",
|
||||
)
|
||||
)
|
||||
for unit in meta_units:
|
||||
from .meta_preparation import preparation_command
|
||||
|
||||
steps.extend((
|
||||
ReleasePlanStep(
|
||||
id="govoplan:prepare-support",
|
||||
title="Prepare the complete developer meta-package outside this run",
|
||||
detail=(
|
||||
"First prepare and commit Core at the target and review module/requirements inputs. "
|
||||
"Stop active runs, preview and explicitly apply the frozen composition in a separate "
|
||||
"source checkout; review and commit manually. No durable self-update is supported."
|
||||
),
|
||||
command=preparation_command(workspace=Path(dashboard.workspace_root), target_version=unit.target_version),
|
||||
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
|
||||
),
|
||||
ReleasePlanStep(
|
||||
id="govoplan:publish-support",
|
||||
title="Publish the prepared Meta support source after Core",
|
||||
detail=(
|
||||
"After the matching Core annotated tag and exact main are published, use the shared "
|
||||
"guarded Meta tag preview/local-tag/publish route. Commit/push reviewed preparation "
|
||||
"and create a fresh durable run; do not update the current runtime binding."
|
||||
),
|
||||
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
|
||||
mutating=True,
|
||||
),
|
||||
))
|
||||
return tuple(steps)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,792 @@
|
||||
"""Strict registered-source release contract for every source-tag batch.
|
||||
|
||||
Meta is not a root Python package. Its nested developer package is released only
|
||||
after a whole-batch source preflight and the matching immutable Core release.
|
||||
No selected checkout supplies the developer-package generator or release
|
||||
validation tooling. The trusted shared checker may load reviewed application
|
||||
manifests; this is not an untrusted-code sandbox.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
|
||||
from .git_state import collect_repository_snapshot, git, git_text
|
||||
from .repository_tag import (
|
||||
_preview_repositories,
|
||||
basic_blocker,
|
||||
normalize_version,
|
||||
ref_commit,
|
||||
remote_tag_commit,
|
||||
run,
|
||||
)
|
||||
from .source_provenance import registered_source_origin_issues
|
||||
from .version_alignment import (
|
||||
repository_version_issues,
|
||||
selected_release_webui_bundle_issues,
|
||||
selected_webui_repository_names,
|
||||
)
|
||||
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
|
||||
|
||||
_OBJECT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
|
||||
|
||||
|
||||
class SourceReceiptError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def _owned_path(path, *, directory):
|
||||
observed = path.lstat()
|
||||
expected = stat.S_ISDIR if directory else stat.S_ISREG
|
||||
if stat.S_ISLNK(observed.st_mode) or not expected(observed.st_mode):
|
||||
raise SourceReceiptError(
|
||||
"source authority must use real paths, without symlinks or special files"
|
||||
)
|
||||
if observed.st_uid != os.geteuid():
|
||||
raise SourceReceiptError(
|
||||
"source authority is not owned by the current operator"
|
||||
)
|
||||
if observed.st_mode & 0o022:
|
||||
raise SourceReceiptError("source authority is group/world writable")
|
||||
return observed
|
||||
|
||||
|
||||
def _trusted_ancestry(path):
|
||||
# Same ownership/mode policy as the publisher's trust-path guard. A sticky
|
||||
# shared ancestor such as /tmp may contain an owned, non-writable child;
|
||||
# the workspace/repository themselves are never given that exception.
|
||||
for ancestor in (path, *path.parents):
|
||||
observed = ancestor.lstat()
|
||||
if stat.S_ISLNK(observed.st_mode) or not stat.S_ISDIR(observed.st_mode):
|
||||
raise SourceReceiptError(
|
||||
"source ancestry must contain real directories, not symlinks"
|
||||
)
|
||||
if observed.st_uid not in {0, os.geteuid()}:
|
||||
raise SourceReceiptError("source ancestry has an untrusted owner")
|
||||
if observed.st_mode & 0o022 and not observed.st_mode & stat.S_ISVTX:
|
||||
raise SourceReceiptError("source ancestry is group/world writable")
|
||||
|
||||
|
||||
def _git_pointer(path):
|
||||
_owned_path(path, directory=False)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
observed = os.fstat(source.fileno())
|
||||
if not stat.S_ISREG(observed.st_mode) or not 0 < observed.st_size <= 4096:
|
||||
raise SourceReceiptError("Git metadata pointer is invalid or oversized")
|
||||
value = source.read(observed.st_size + 1)
|
||||
if len(value) != observed.st_size:
|
||||
raise SourceReceiptError("Git metadata pointer changed during validation")
|
||||
return value.decode("utf-8").strip()
|
||||
|
||||
|
||||
def _git_tree(root):
|
||||
pending = [(root, 0)]
|
||||
count = 0
|
||||
while pending:
|
||||
directory, depth = pending.pop()
|
||||
_owned_path(directory, directory=True)
|
||||
if depth > 128:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata exceeds its trust-validation depth limit"
|
||||
)
|
||||
with os.scandir(directory) as entries:
|
||||
for entry in entries:
|
||||
count += 1
|
||||
if count > 500_000:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata exceeds its trust-validation entry limit"
|
||||
)
|
||||
candidate = Path(entry.path)
|
||||
observed = candidate.lstat()
|
||||
if stat.S_ISDIR(observed.st_mode):
|
||||
pending.append((candidate, depth + 1))
|
||||
else:
|
||||
_owned_path(candidate, directory=False)
|
||||
|
||||
|
||||
def _filesystem_identity(path, observed):
|
||||
return [
|
||||
str(path),
|
||||
observed.st_dev,
|
||||
observed.st_ino,
|
||||
observed.st_uid,
|
||||
observed.st_gid,
|
||||
stat.S_IMODE(observed.st_mode),
|
||||
]
|
||||
|
||||
|
||||
def _source_filesystem(*, path, workspace):
|
||||
"""Validate source/Git ownership before invoking even read-only Git."""
|
||||
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(
|
||||
workspace.resolve()
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"source checkout leaves the private workspace or traverses a symlink"
|
||||
)
|
||||
_trusted_ancestry(path)
|
||||
workspace_info = _owned_path(workspace, directory=True)
|
||||
repo_info = _owned_path(path, directory=True)
|
||||
marker = path / ".git"
|
||||
marker_info = marker.lstat()
|
||||
if stat.S_ISDIR(marker_info.st_mode):
|
||||
git_dir = marker
|
||||
else:
|
||||
value = _git_pointer(marker)
|
||||
if not value.startswith("gitdir: "):
|
||||
raise SourceReceiptError("Git worktree pointer is invalid")
|
||||
git_dir = Path(os.path.abspath(path / value.removeprefix("gitdir: ")))
|
||||
if git_dir.absolute() != git_dir.resolve() or not git_dir.resolve().is_relative_to(
|
||||
workspace.resolve()
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
_trusted_ancestry(git_dir)
|
||||
_owned_path(git_dir, directory=True)
|
||||
common_dir = git_dir
|
||||
common_pointer = git_dir / "commondir"
|
||||
if common_pointer.exists() or common_pointer.is_symlink():
|
||||
common_dir = Path(os.path.abspath(git_dir / _git_pointer(common_pointer)))
|
||||
identities = {
|
||||
"workspace": _filesystem_identity(workspace, workspace_info),
|
||||
"checkout": _filesystem_identity(path, repo_info),
|
||||
}
|
||||
scanned = set()
|
||||
for label, directory in (
|
||||
("git_directory", git_dir),
|
||||
("git_common_directory", common_dir),
|
||||
):
|
||||
if (
|
||||
directory.absolute() != directory.resolve()
|
||||
or not directory.resolve().is_relative_to(workspace.resolve())
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
_trusted_ancestry(directory)
|
||||
observed = _owned_path(directory, directory=True)
|
||||
if observed.st_mode & 0o700 != 0o700:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata target must be readable and writable by its operator"
|
||||
)
|
||||
identities[label] = _filesystem_identity(directory, observed)
|
||||
if directory not in scanned:
|
||||
_git_tree(directory)
|
||||
scanned.add(directory)
|
||||
identities["git_marker"] = _filesystem_identity(
|
||||
marker, _owned_path(marker, directory=stat.S_ISDIR(marker_info.st_mode))
|
||||
)
|
||||
for candidate in (
|
||||
common_dir / "objects/info/alternates",
|
||||
common_dir / "objects/info/http-alternates",
|
||||
common_dir / "info/grafts",
|
||||
):
|
||||
if candidate.exists() or candidate.is_symlink():
|
||||
raise SourceReceiptError(
|
||||
"Git object alternates and grafts are not permitted"
|
||||
)
|
||||
return identities
|
||||
|
||||
|
||||
def _tracked_worktree(path):
|
||||
tracked = git(path, "ls-files", "-v", "-z", timeout=30)
|
||||
if tracked.returncode or len(tracked.stdout) > 16 * 1024 * 1024:
|
||||
raise SourceReceiptError(
|
||||
"tracked release inputs exceed their trust-validation limit"
|
||||
)
|
||||
checked = {path}
|
||||
tracked_paths = set()
|
||||
names = tracked.stdout.split("\0")
|
||||
if len(names) > 100_001:
|
||||
raise SourceReceiptError(
|
||||
"tracked release inputs exceed their trust-validation count limit"
|
||||
)
|
||||
for entry in filter(None, names):
|
||||
# git status deliberately hides assume-unchanged and skip-worktree
|
||||
# paths. Never validate mutable working metadata and then tag different
|
||||
# committed bytes because an index flag suppressed the dirty evidence.
|
||||
if not entry.startswith("H "):
|
||||
raise SourceReceiptError(
|
||||
"hidden, sparse or unmerged tracked index entries are not permitted"
|
||||
)
|
||||
name = entry[2:]
|
||||
tracked_paths.add(name)
|
||||
relative = Path(name)
|
||||
if relative.is_absolute() or ".." in relative.parts:
|
||||
raise SourceReceiptError("tracked release input has an unsafe path")
|
||||
candidate = path / relative
|
||||
for parent in candidate.parents:
|
||||
if parent == path:
|
||||
break
|
||||
if parent not in checked:
|
||||
_owned_path(parent, directory=True)
|
||||
checked.add(parent)
|
||||
_owned_path(candidate, directory=False)
|
||||
# Version/composition checks inspect existing files, including ignored
|
||||
# paths. Their declarations must come from the selected committed source,
|
||||
# not ignored working bytes absent from the tag's tree.
|
||||
metadata = [
|
||||
path / name
|
||||
for name in (
|
||||
"pyproject.toml",
|
||||
"package.json",
|
||||
"package-lock.json",
|
||||
"webui/package.json",
|
||||
"webui/package.release.json",
|
||||
"webui/package-lock.json",
|
||||
"webui/package-lock.release.json",
|
||||
)
|
||||
]
|
||||
if path.name == "govoplan":
|
||||
metadata.extend(
|
||||
path / name
|
||||
for name in (
|
||||
"packages/govoplan-meta/pyproject.toml",
|
||||
"requirements-release.txt",
|
||||
)
|
||||
)
|
||||
metadata.extend((path / "src").glob("**/backend/manifest.py"))
|
||||
metadata.extend((path / "src").glob("*/__init__.py"))
|
||||
for candidate in metadata:
|
||||
if (candidate.exists() or candidate.is_symlink()) and candidate.relative_to(
|
||||
path
|
||||
).as_posix() not in tracked_paths:
|
||||
raise SourceReceiptError(
|
||||
"selected release version/composition metadata must be tracked in the frozen source"
|
||||
)
|
||||
|
||||
|
||||
def _receipt(*, spec, workspace, version, filesystem):
|
||||
path = resolve_repo_path(spec, workspace)
|
||||
_tracked_worktree(path)
|
||||
snapshot = collect_repository_snapshot(
|
||||
spec, workspace_root=workspace, target_tag=f"v{version}", online=False
|
||||
)
|
||||
blocker = basic_blocker(snapshot=snapshot, version=version)
|
||||
if blocker:
|
||||
raise SourceReceiptError(blocker)
|
||||
if (
|
||||
not snapshot.exists
|
||||
or not snapshot.is_git
|
||||
or not snapshot.has_head
|
||||
or snapshot.errors
|
||||
or snapshot.safe_directory_required
|
||||
or snapshot.dirty
|
||||
or snapshot.branch != "main"
|
||||
or snapshot.upstream != "origin/main"
|
||||
or snapshot.behind
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"requires a clean registered main checkout tracking origin/main, not behind"
|
||||
)
|
||||
common = git_text(path, "rev-parse", "--path-format=absolute", "--git-common-dir")
|
||||
if (
|
||||
not common
|
||||
or Path(common).absolute() != Path(common).resolve()
|
||||
or not Path(common).resolve().is_relative_to(workspace.resolve())
|
||||
or git_text(path, "rev-parse", "--show-toplevel") != str(path.resolve())
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
head = git_text(path, "rev-parse", "--verify", "HEAD")
|
||||
if not _OBJECT.fullmatch(head):
|
||||
raise SourceReceiptError("source HEAD is not an exact commit")
|
||||
live = run(
|
||||
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
|
||||
cwd=path,
|
||||
)
|
||||
lines = live.stdout.strip().splitlines()
|
||||
if live.returncode or len(lines) != 1:
|
||||
raise SourceReceiptError("could not verify live origin/main")
|
||||
remote_main, separator, ref = lines[0].partition("\t")
|
||||
if not separator or ref != "refs/heads/main" or not _OBJECT.fullmatch(remote_main):
|
||||
raise SourceReceiptError("live origin/main returned an invalid source receipt")
|
||||
if git(path, "merge-base", "--is-ancestor", remote_main, head).returncode != 0:
|
||||
raise SourceReceiptError(
|
||||
"live origin/main is unavailable locally or diverges; fetch and review before retrying"
|
||||
)
|
||||
tag = f"v{version}"
|
||||
local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}") or None
|
||||
if local_object:
|
||||
if git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag":
|
||||
raise SourceReceiptError("local immutable tag must be annotated")
|
||||
if ref_commit(path, f"refs/tags/{tag}") != head:
|
||||
raise SourceReceiptError(
|
||||
"local immutable tag points to another commit, not HEAD"
|
||||
)
|
||||
remote_tag = remote_tag_commit(path, remote="origin", tag=tag)
|
||||
if remote_tag.error:
|
||||
raise SourceReceiptError("could not verify the remote release tag")
|
||||
if remote_tag.tag_object:
|
||||
if not remote_tag.annotated:
|
||||
raise SourceReceiptError("remote immutable tag must be annotated")
|
||||
if remote_tag.commit != head:
|
||||
raise SourceReceiptError(
|
||||
"remote immutable tag points to another commit, not HEAD"
|
||||
)
|
||||
if local_object and remote_tag.tag_object != local_object:
|
||||
raise SourceReceiptError(
|
||||
"local and remote immutable tag annotation objects differ"
|
||||
)
|
||||
return {
|
||||
"head": head,
|
||||
"branch": "main",
|
||||
"upstream": "origin/main",
|
||||
"origin": spec.remote,
|
||||
"remote_main": remote_main,
|
||||
"tag": tag,
|
||||
"local_tag_object": local_object,
|
||||
"remote_tag_object": remote_tag.tag_object,
|
||||
"filesystem": filesystem,
|
||||
}
|
||||
|
||||
|
||||
def _collect_receipts(*, versions, specs, workspace):
|
||||
filesystems = {}
|
||||
for repo in versions:
|
||||
if repo not in specs:
|
||||
raise SourceReceiptError(f"{repo}: source repository is not registered")
|
||||
filesystems[repo] = _source_filesystem(
|
||||
path=resolve_repo_path(specs[repo], workspace), workspace=workspace
|
||||
)
|
||||
issues = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote="origin"
|
||||
)
|
||||
if issues:
|
||||
raise SourceReceiptError(
|
||||
"; ".join(f"{issue.repo}: {issue.message}" for issue in issues)
|
||||
)
|
||||
receipts = {}
|
||||
for repo, version in versions.items():
|
||||
if repo not in specs:
|
||||
raise SourceReceiptError(f"{repo}: source repository is not registered")
|
||||
try:
|
||||
receipts[repo] = _receipt(
|
||||
spec=specs[repo],
|
||||
workspace=workspace,
|
||||
version=version,
|
||||
filesystem=filesystems[repo],
|
||||
)
|
||||
except SourceReceiptError as exc:
|
||||
raise SourceReceiptError(f"{repo}: {exc}") from exc
|
||||
return receipts
|
||||
|
||||
|
||||
def _bundle_input_receipt(*, selected, workspace, push):
|
||||
"""Freeze only Core files used by the already-applicable WebUI gate.
|
||||
|
||||
These are read-only composition inputs, not a new Core-tag/version or
|
||||
clean-Core prerequisite. Local module candidates intentionally need none.
|
||||
"""
|
||||
if not push and "govoplan-core" not in selected:
|
||||
return {}
|
||||
if not selected_webui_repository_names(
|
||||
repo_versions=dict.fromkeys(selected, ""), workspace=workspace
|
||||
):
|
||||
return {}
|
||||
result = {}
|
||||
core = workspace / "govoplan-core"
|
||||
for relative in ("webui/package.release.json", "webui/package-lock.release.json"):
|
||||
path = core / relative
|
||||
if not path.exists() and not path.is_symlink():
|
||||
result[relative] = None # The unchanged shared gate explains missing input.
|
||||
continue
|
||||
_trusted_ancestry(path.parent)
|
||||
_owned_path(core, directory=True)
|
||||
_owned_path(path.parent, directory=True)
|
||||
observed = _owned_path(path, directory=False)
|
||||
if not 0 < observed.st_size <= 16 * 1024 * 1024:
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input exceeds its 16 MiB limit"
|
||||
)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
before = os.fstat(source.fileno())
|
||||
if (
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_size,
|
||||
before.st_mtime_ns,
|
||||
before.st_ctime_ns,
|
||||
) != (
|
||||
observed.st_dev,
|
||||
observed.st_ino,
|
||||
observed.st_size,
|
||||
observed.st_mtime_ns,
|
||||
observed.st_ctime_ns,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input changed during inspection"
|
||||
)
|
||||
content = source.read(before.st_size + 1)
|
||||
after = os.fstat(source.fileno())
|
||||
if len(content) != before.st_size or (
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_size,
|
||||
before.st_mtime_ns,
|
||||
before.st_ctime_ns,
|
||||
) != (
|
||||
after.st_dev,
|
||||
after.st_ino,
|
||||
after.st_size,
|
||||
after.st_mtime_ns,
|
||||
after.st_ctime_ns,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input changed during inspection"
|
||||
)
|
||||
result[relative] = {
|
||||
"file": _filesystem_identity(path, observed),
|
||||
"sha256": hashlib.sha256(content).hexdigest(),
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def _frozen_receipts(
|
||||
*, expected, versions, specs, workspace, selected, push, bundle_inputs
|
||||
):
|
||||
actual = _collect_receipts(versions=versions, specs=specs, workspace=workspace)
|
||||
for repo in expected:
|
||||
if actual[repo] != expected[repo]:
|
||||
raise SourceReceiptError(
|
||||
f"{repo}: source receipt changed after whole-batch preflight"
|
||||
)
|
||||
for repo in versions:
|
||||
issues = repository_version_issues(
|
||||
resolve_repo_path(specs[repo], workspace), expected_version=versions[repo]
|
||||
)
|
||||
if issues:
|
||||
raise SourceReceiptError(
|
||||
f"{repo}: version/composition changed after whole-batch preflight"
|
||||
)
|
||||
if (
|
||||
_bundle_input_receipt(selected=selected, workspace=workspace, push=push)
|
||||
!= bundle_inputs
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input receipt changed after whole-batch preflight"
|
||||
)
|
||||
if push or "govoplan-core" in selected:
|
||||
if selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: versions[repo] for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"release WebUI composition changed after whole-batch preflight"
|
||||
)
|
||||
|
||||
|
||||
def _require_core(receipts, *, push):
|
||||
core = receipts["govoplan-core"]
|
||||
if not core["local_tag_object"] or (
|
||||
push and (not core["remote_tag_object"] or core["remote_main"] != core["head"])
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Meta requires the matching annotated Core tag locally and, for publication, remotely"
|
||||
)
|
||||
|
||||
|
||||
def _blocked(*, selected, apply, push, detail, rows=()): # noqa: A002
|
||||
known = {row["repo"]: row for row in rows}
|
||||
identified = next(
|
||||
(repo for repo in selected if detail.startswith(repo + ":")), None
|
||||
)
|
||||
return {
|
||||
"status": "blocked",
|
||||
"apply": apply,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"detail": "whole-batch source preflight failed; no selected repository was mutated",
|
||||
"repositories": [
|
||||
{
|
||||
**known.get(repo, {"repo": repo}),
|
||||
"status": "blocked"
|
||||
if (
|
||||
known.get(repo, {}).get("status") == "blocked"
|
||||
or (not rows and (identified is None or repo == identified))
|
||||
)
|
||||
else "skipped",
|
||||
"detail": known[repo]["detail"]
|
||||
if known.get(repo, {}).get("status") == "blocked"
|
||||
else detail,
|
||||
}
|
||||
for repo in selected
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def tag_source_batch(
|
||||
*, repos, repo_versions, workspace_root, remote, message, apply, push
|
||||
): # noqa: A002
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
has_meta = "govoplan" in repos
|
||||
selected = tuple(dict.fromkeys(repos))
|
||||
if has_meta:
|
||||
selected = tuple(repo for repo in selected if repo != "govoplan") + (
|
||||
"govoplan",
|
||||
)
|
||||
if not selected:
|
||||
return {
|
||||
"status": "noop",
|
||||
"apply": apply,
|
||||
"push": push,
|
||||
"remote": remote.strip() or "origin",
|
||||
"repositories": [],
|
||||
}
|
||||
versions = {repo: normalize_version(repo_versions.get(repo)) for repo in selected}
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
meta_version = versions.get("govoplan")
|
||||
try:
|
||||
if remote.strip() not in {"", "origin"}:
|
||||
raise SourceReceiptError(
|
||||
"Source-tag batches require the registered origin remote"
|
||||
)
|
||||
if any(not version for version in versions.values()):
|
||||
raise SourceReceiptError(
|
||||
"every selected repository requires an explicit valid version"
|
||||
)
|
||||
if has_meta and versions.get("govoplan-core", meta_version) != meta_version:
|
||||
raise SourceReceiptError(
|
||||
"Meta developer-package version must match the selected Core release"
|
||||
)
|
||||
# Only Meta requires a frozen version-matched Core source/tag dependency.
|
||||
if has_meta:
|
||||
versions.setdefault("govoplan-core", meta_version)
|
||||
receipts = _collect_receipts(
|
||||
versions=versions, specs=specs, workspace=workspace
|
||||
)
|
||||
if has_meta and "govoplan-core" not in selected:
|
||||
_require_core(receipts, push=push)
|
||||
core_issues = (
|
||||
repository_version_issues(
|
||||
resolve_repo_path(specs["govoplan-core"], workspace),
|
||||
expected_version=meta_version,
|
||||
)
|
||||
if has_meta
|
||||
else ()
|
||||
)
|
||||
if core_issues:
|
||||
raise SourceReceiptError(
|
||||
"Core source metadata must match the selected Meta version"
|
||||
)
|
||||
bundle_inputs = _bundle_input_receipt(
|
||||
selected=selected, workspace=workspace, push=push
|
||||
)
|
||||
except (SourceReceiptError, OSError, ValueError) as exc:
|
||||
return _blocked(selected=selected, apply=apply, push=push, detail=str(exc))
|
||||
|
||||
# Preserve the shared complete manifest, package/lock and immutable tag
|
||||
# preflight. This invocation is always read-only; strict effects stay below.
|
||||
try:
|
||||
preview = _preview_repositories(
|
||||
repos=selected,
|
||||
repo_versions=repo_versions,
|
||||
workspace_root=workspace,
|
||||
remote="origin",
|
||||
message=message,
|
||||
push=push,
|
||||
)
|
||||
except (OSError, ValueError) as exc:
|
||||
return _blocked(
|
||||
selected=selected,
|
||||
apply=apply,
|
||||
push=push,
|
||||
detail=f"shared release preflight could not validate its inputs ({type(exc).__name__})",
|
||||
)
|
||||
rows = preview["repositories"]
|
||||
if preview["status"] in {"blocked", "partial"}:
|
||||
if not apply:
|
||||
return preview
|
||||
return _blocked(
|
||||
selected=selected,
|
||||
apply=True,
|
||||
push=push,
|
||||
detail="shared release preflight failed",
|
||||
rows=rows,
|
||||
)
|
||||
if not apply:
|
||||
rows = [
|
||||
{
|
||||
**row,
|
||||
"status": "planned",
|
||||
"detail": "existing annotated release tag requires atomic main publication",
|
||||
}
|
||||
if push
|
||||
and receipts[row["repo"]]["remote_main"] != receipts[row["repo"]]["head"]
|
||||
else row
|
||||
for row in rows
|
||||
]
|
||||
preview = {**preview, "repositories": rows}
|
||||
if any(row["status"] == "planned" for row in rows):
|
||||
preview["status"] = "planned"
|
||||
return {
|
||||
**preview,
|
||||
"source_receipts": receipts,
|
||||
"source_contract": "registered-meta-batch-v1"
|
||||
if has_meta
|
||||
else "registered-source-batch-v1",
|
||||
"bundle_input_receipts": bundle_inputs,
|
||||
}
|
||||
|
||||
results = []
|
||||
effected = False
|
||||
for row in rows:
|
||||
repo = row["repo"]
|
||||
receipt = receipts[repo]
|
||||
path = resolve_repo_path(specs[repo], workspace)
|
||||
tag = receipt["tag"]
|
||||
head = receipt["head"]
|
||||
try:
|
||||
# Recheck the entire frozen batch, including Meta and Core, before
|
||||
# every effect. Earlier successful effects update only their exact
|
||||
# anticipated tag/branch receipt fields below.
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
if repo == "govoplan":
|
||||
_require_core(receipts, push=push)
|
||||
local_object = receipt["local_tag_object"]
|
||||
created = False
|
||||
if not local_object:
|
||||
if receipt["remote_tag_object"]:
|
||||
command = (
|
||||
"git",
|
||||
"fetch",
|
||||
"--no-tags",
|
||||
"origin",
|
||||
f"refs/tags/{tag}:refs/tags/{tag}",
|
||||
)
|
||||
else:
|
||||
command = ("git", "tag", "-a", tag, head, "-m", row["message"])
|
||||
created = True
|
||||
effected = True
|
||||
if run(command, cwd=path).returncode:
|
||||
raise SourceReceiptError(
|
||||
"annotated local release tag could not be created or retrieved"
|
||||
)
|
||||
local_object = git_text(
|
||||
path, "rev-parse", "--verify", f"refs/tags/{tag}"
|
||||
)
|
||||
if (
|
||||
not local_object
|
||||
or ref_commit(path, f"refs/tags/{tag}") != head
|
||||
or git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag"
|
||||
or (
|
||||
receipt["remote_tag_object"]
|
||||
and local_object != receipt["remote_tag_object"]
|
||||
)
|
||||
):
|
||||
raise SourceReceiptError("local release tag postcondition failed")
|
||||
receipt["local_tag_object"] = local_object
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
if push and (
|
||||
receipt["remote_tag_object"] != local_object
|
||||
or receipt["remote_main"] != head
|
||||
):
|
||||
# Pin both effects to verified objects, not mutable HEAD/tag
|
||||
# names. No force, retagging, fallback or non-atomic retry.
|
||||
command = (
|
||||
"git",
|
||||
"push",
|
||||
"--atomic",
|
||||
"origin",
|
||||
f"{head}:refs/heads/main",
|
||||
f"{local_object}:refs/tags/{tag}",
|
||||
)
|
||||
effected = True
|
||||
if run(command, cwd=path).returncode:
|
||||
raise SourceReceiptError(
|
||||
"atomic main and annotated tag publication failed; inspect receipts before retrying"
|
||||
)
|
||||
receipt["remote_main"] = head
|
||||
receipt["remote_tag_object"] = local_object
|
||||
# Verify remote main AND exact annotated object, as well as local
|
||||
# source state. A successful Git exit alone is never a receipt.
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published" if push else "tagged" if created else "noop",
|
||||
"detail": "verified strict registered-source release"
|
||||
if not receipt["remote_tag_object"] or created
|
||||
else "verified strict registered-source release; immutable annotation already published or present",
|
||||
"after_local_tag_commit": head,
|
||||
"after_local_tag_object": local_object,
|
||||
"after_remote_tag_commit": head
|
||||
if receipt["remote_tag_object"]
|
||||
else None,
|
||||
"after_remote_tag_object": receipt["remote_tag_object"],
|
||||
"after_remote_main_commit": receipt["remote_main"],
|
||||
}
|
||||
)
|
||||
except (SourceReceiptError, OSError, ValueError) as exc:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed" if effected else "blocked",
|
||||
"detail": str(exc),
|
||||
}
|
||||
)
|
||||
results.extend(
|
||||
{
|
||||
**later,
|
||||
"status": "skipped",
|
||||
"detail": "earlier strict source effect or receipt failed",
|
||||
}
|
||||
for later in rows[len(results) :]
|
||||
)
|
||||
return {
|
||||
"status": "partial" if effected else "blocked",
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"repositories": results,
|
||||
}
|
||||
status = (
|
||||
"published"
|
||||
if push
|
||||
else "tagged"
|
||||
if any(row["status"] == "tagged" for row in results)
|
||||
else "noop"
|
||||
)
|
||||
return {
|
||||
"status": status,
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"repositories": results,
|
||||
"source_receipts": receipts,
|
||||
"source_contract": "registered-meta-batch-v1"
|
||||
if has_meta
|
||||
else "registered-source-batch-v1",
|
||||
"bundle_input_receipts": bundle_inputs,
|
||||
}
|
||||
@@ -6,12 +6,13 @@ from dataclasses import dataclass
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import runpy
|
||||
import subprocess
|
||||
import tomllib
|
||||
|
||||
from .git_state import collect_versions, sanitized_git_environment
|
||||
from .git_state import collect_versions, registered_developer_meta_path, sanitized_git_environment
|
||||
from .registry_reference import registry_artifact_conflicts, registry_entry_source
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
_PYTHON_RELEASE_REF = re.compile(
|
||||
@@ -43,6 +44,7 @@ def repository_version_issues(
|
||||
versions = collect_versions(repo_path)
|
||||
declared = {
|
||||
"pyproject.toml": versions.pyproject,
|
||||
"packages/govoplan-meta/pyproject.toml": versions.developer_meta,
|
||||
"package.json": versions.package,
|
||||
"webui/package.json": versions.webui_package,
|
||||
"webui/package.release.json": _json_version(repo_path / "webui" / "package.release.json")
|
||||
@@ -83,6 +85,8 @@ def repository_version_issues(
|
||||
for source, version in declared.items()
|
||||
if version is not None and version != canonical_version
|
||||
]
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
issues.extend(developer_meta_composition_issues(repo_path))
|
||||
|
||||
if expected_version is not None and canonical_version.removeprefix("v") != expected_version.removeprefix("v"):
|
||||
issues.append(
|
||||
@@ -120,6 +124,34 @@ def repository_version_issues(
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def developer_meta_composition_issues(repo_path: Path) -> tuple[VersionAlignmentIssue, ...]:
|
||||
"""Compare the real nested package with the trusted generator's exact output.
|
||||
|
||||
Never execute a generator from a selected checkout. Only the installed
|
||||
operator tooling provides code; selected TOML/requirements are data inputs.
|
||||
"""
|
||||
package_path = registered_developer_meta_path(repo_path)
|
||||
if package_path is None:
|
||||
return (VersionAlignmentIssue(repo_path.name, "repository", "registered Meta support repository", "", "nested developer-package identity is not registered"),)
|
||||
source = "packages/govoplan-meta/pyproject.toml"
|
||||
try:
|
||||
current = package_path.read_text(encoding="utf-8")
|
||||
project = tomllib.loads(current).get("project")
|
||||
if not isinstance(project, dict) or project.get("name") != "govoplan":
|
||||
return (VersionAlignmentIssue("govoplan", source + ":project.name", "govoplan", str(project.get("name") if isinstance(project, dict) else ""), "developer meta-package identity must be exact"),)
|
||||
# META_ROOT belongs to the running operator tools, not repo_path.
|
||||
generator = runpy.run_path(str(META_ROOT / "tools/release/generate-developer-meta-package.py"))
|
||||
expected = generator["render"](
|
||||
workspace=repo_path.parent,
|
||||
requirements=repo_path / "requirements-release.txt",
|
||||
)
|
||||
except (OSError, UnicodeError, KeyError, ValueError, TypeError) as exc:
|
||||
return (VersionAlignmentIssue("govoplan", source, "readable exact developer composition and Core version", type(exc).__name__, "developer meta-package composition could not be validated"),)
|
||||
if current != expected:
|
||||
return (VersionAlignmentIssue("govoplan", source, "trusted generator output matching Core and release requirements", "stale composition", "developer meta-package must exactly match the generator --check contract"),)
|
||||
return ()
|
||||
|
||||
|
||||
def selected_repository_version_issues(
|
||||
*,
|
||||
repo_versions: dict[str, str],
|
||||
@@ -165,6 +197,11 @@ def selected_repository_version_issues(
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def selected_webui_repository_names(*, repo_versions: dict[str, str], workspace: Path) -> tuple[str, ...]:
|
||||
"""Identify the exact selections for which Core's WebUI inputs are relevant."""
|
||||
return tuple(repo for repo in sorted(repo_versions) if repo != "govoplan-core" and (workspace / repo / "webui/package.json").exists())
|
||||
|
||||
|
||||
def selected_release_webui_bundle_issues(
|
||||
*,
|
||||
repo_versions: dict[str, str],
|
||||
@@ -177,6 +214,8 @@ def selected_release_webui_bundle_issues(
|
||||
immutable release package input and lockfile will actually install.
|
||||
"""
|
||||
|
||||
if not selected_webui_repository_names(repo_versions=repo_versions, workspace=workspace):
|
||||
return ()
|
||||
core_webui = workspace / "govoplan-core" / "webui"
|
||||
release_package_path = core_webui / "package.release.json"
|
||||
release_lock_path = core_webui / "package-lock.release.json"
|
||||
|
||||
@@ -34,6 +34,19 @@ def version_metadata_mutations(
|
||||
) -> tuple[VersionFileMutation, ...]:
|
||||
"""Render all recognized repository version files without writing them."""
|
||||
|
||||
from .git_state import registered_developer_meta_path
|
||||
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
from .meta_preparation import MetaPreparationError, PACKAGE, preview_meta_mutation
|
||||
|
||||
try:
|
||||
_receipt, before, after = preview_meta_mutation(
|
||||
repo_path=repo_path, target_version=target_version,
|
||||
)
|
||||
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
|
||||
raise VersionMetadataError(str(exc)) from exc
|
||||
return (VersionFileMutation(PACKAGE, before, after),) if before != after else ()
|
||||
|
||||
version = target_version.removeprefix("v")
|
||||
candidates: list[tuple[Path, str]] = []
|
||||
if (repo_path / "pyproject.toml").is_file():
|
||||
@@ -115,6 +128,14 @@ def apply_version_metadata_mutations(
|
||||
) -> tuple[str, ...]:
|
||||
"""Apply one deterministic version update, rolling back on write failure."""
|
||||
|
||||
from .git_state import registered_developer_meta_path
|
||||
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
raise VersionMetadataError(
|
||||
"Meta is prepared outside durable runs with prepare-developer-meta-package.py; "
|
||||
"review its complete generated composition, commit, and create a fresh run."
|
||||
)
|
||||
|
||||
mutations = version_metadata_mutations(
|
||||
repo_path,
|
||||
target_version=target_version,
|
||||
|
||||
@@ -19,8 +19,9 @@ retry() {
|
||||
for attempt in 1 2 3; do
|
||||
if "$@"; then
|
||||
return 0
|
||||
else
|
||||
status=$?
|
||||
fi
|
||||
status=$?
|
||||
if [[ "$attempt" == 3 ]]; then
|
||||
return "$status"
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Preview or explicitly prepare Meta outside a durable release run."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from govoplan_release.meta_preparation import (
|
||||
MetaPreparationAmbiguous,
|
||||
MetaPreparationError,
|
||||
_read_input,
|
||||
prepare_developer_meta_package,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--workspace", type=Path, required=True)
|
||||
parser.add_argument("--target-version", required=True)
|
||||
parser.add_argument("--apply", action="store_true")
|
||||
parser.add_argument("--receipt", type=Path)
|
||||
parser.add_argument("--confirm-out-of-run", action="store_true")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
expected = None
|
||||
if args.apply:
|
||||
if args.receipt is None:
|
||||
raise MetaPreparationError(
|
||||
"Apply requires the reviewed preview JSON via --receipt."
|
||||
)
|
||||
payload, _ = _read_input(args.receipt)
|
||||
expected = json.loads(payload)["receipt"]
|
||||
result = prepare_developer_meta_package(
|
||||
repo_path=args.workspace.absolute() / "govoplan",
|
||||
target_version=args.target_version,
|
||||
apply=args.apply,
|
||||
expected_receipt=expected,
|
||||
confirm_out_of_run=args.confirm_out_of_run,
|
||||
)
|
||||
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
|
||||
status = "needs-reconciliation" if isinstance(exc, MetaPreparationAmbiguous) else "blocked"
|
||||
print(json.dumps({"status": status, "detail": str(exc)}))
|
||||
return 1
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -8,7 +8,7 @@ from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import Any, NoReturn
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
@@ -168,12 +168,13 @@ def owner_for_versions_dir(versions_dir: Path) -> str:
|
||||
|
||||
|
||||
def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
if path.stat().st_size > 2 * 1024 * 1024:
|
||||
raise ValueError(f"{path}: migration source exceeds the 2 MiB audit bound")
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
values: dict[str, Any] = {}
|
||||
wrapped: Migration | None = None
|
||||
release_peer = path.parent.parent / "versions" / path.name
|
||||
if path.parent.name == "dev_versions" and release_peer.is_file():
|
||||
wrapped = parse_migration_file(owner, release_peer)
|
||||
values = _imported_release_metadata(owner, path, tree)
|
||||
wrapped = _wrapped_release_metadata(owner, path, tree)
|
||||
if values and wrapped:
|
||||
raise ValueError(f"{path}: mixed migration metadata wrapper styles are ambiguous")
|
||||
for statement in tree.body:
|
||||
if isinstance(statement, ast.Assign):
|
||||
for target in statement.targets:
|
||||
@@ -196,6 +197,8 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
)
|
||||
revision = values.get("revision")
|
||||
if not isinstance(revision, str):
|
||||
if "revision" in values:
|
||||
raise ValueError(f"{path}: migration revision must be an explicit string")
|
||||
return None
|
||||
return Migration(
|
||||
owner=owner,
|
||||
@@ -207,28 +210,200 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
)
|
||||
|
||||
|
||||
def _ast_binding_count(tree: ast.Module, name: str) -> int:
|
||||
count = 0
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Name) and node.id == name and isinstance(node.ctx, (ast.Store, ast.Del)):
|
||||
count += 1
|
||||
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and node.name == name:
|
||||
count += 1
|
||||
elif isinstance(node, (ast.Import, ast.ImportFrom)):
|
||||
count += sum(
|
||||
(alias.asname or (alias.name.split(".")[0] if isinstance(node, ast.Import) else alias.name)) == name
|
||||
for alias in node.names
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def _release_wrapper_peer(owner: str, path: Path, filename: str) -> Migration:
|
||||
versions = path.parent.parent / "versions"
|
||||
peer = versions / filename
|
||||
if (
|
||||
path.parent.name != "dev_versions"
|
||||
or Path(filename).name != filename or not filename.endswith(".py")
|
||||
or versions.is_symlink() or peer.is_symlink() or not peer.is_file()
|
||||
or peer.resolve().parent != versions.resolve()
|
||||
or peer.stat().st_size > 2 * 1024 * 1024
|
||||
):
|
||||
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||
migration = parse_migration_file(owner, peer)
|
||||
if migration is None:
|
||||
raise ValueError(f"{path}: release wrapper target has no migration metadata")
|
||||
return migration
|
||||
|
||||
|
||||
def _imported_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Any]:
|
||||
"""Recognize explicit metadata re-exports, never star/dynamic imports."""
|
||||
names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||
values: dict[str, Any] = {}
|
||||
targets: set[str] = set()
|
||||
for statement in tree.body:
|
||||
if not isinstance(statement, ast.ImportFrom):
|
||||
continue
|
||||
selected = [alias for alias in statement.names if alias.name in names or (alias.asname or alias.name) in names]
|
||||
if not selected:
|
||||
if (statement.module or "").startswith(prefix) and any(alias.name == "*" for alias in statement.names):
|
||||
raise ValueError(f"{path}: unsupported wildcard migration metadata")
|
||||
continue
|
||||
stem = (statement.module or "").removeprefix(prefix)
|
||||
if (
|
||||
statement.level or not (statement.module or "").startswith(prefix)
|
||||
or not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem)
|
||||
):
|
||||
raise ValueError(f"{path}: unsupported or ambiguous imported migration metadata")
|
||||
targets.add(stem)
|
||||
if len(targets) != 1:
|
||||
raise ValueError(f"{path}: ambiguous imported migration metadata sources")
|
||||
migration = _release_wrapper_peer(owner, path, stem + ".py")
|
||||
projection = {
|
||||
"revision": migration.revision, "down_revision": migration.down_revisions,
|
||||
"depends_on": migration.depends_on, "branch_labels": migration.branch_labels,
|
||||
}
|
||||
for alias in selected:
|
||||
name = alias.asname or alias.name
|
||||
if name != alias.name or name in values or _ast_binding_count(tree, name) != 1:
|
||||
raise ValueError(f"{path}: ambiguous imported migration metadata assignment")
|
||||
values[name] = projection[name]
|
||||
return values
|
||||
|
||||
|
||||
def _wrapped_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Migration]:
|
||||
"""Resolve only known literal sibling wrappers, without importing any code."""
|
||||
metadata_names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||
aliases: set[str] = set()
|
||||
bindings: dict[str, list[ast.expr]] = {}
|
||||
imported: dict[str, list[str]] = {}
|
||||
for statement in tree.body:
|
||||
if isinstance(statement, ast.ImportFrom) and not statement.level:
|
||||
for alias in statement.names:
|
||||
imported.setdefault(alias.asname or alias.name, []).append(f"{statement.module}.{alias.name}")
|
||||
targets: list[ast.expr] = []
|
||||
value: ast.expr | None = None
|
||||
if isinstance(statement, ast.Assign):
|
||||
targets, value = statement.targets, statement.value
|
||||
elif isinstance(statement, ast.AnnAssign) and statement.value is not None:
|
||||
targets, value = [statement.target], statement.value
|
||||
for target in targets:
|
||||
if isinstance(target, ast.Name) and value is not None:
|
||||
bindings.setdefault(target.id, []).append(value)
|
||||
if target.id in metadata_names and isinstance(value, ast.Attribute) and isinstance(value.value, ast.Name):
|
||||
aliases.add(value.value.id)
|
||||
if not aliases:
|
||||
return {}
|
||||
if path.parent.name != "dev_versions":
|
||||
raise ValueError(f"{path}: non-literal release migration metadata is unsupported")
|
||||
|
||||
def reject() -> NoReturn:
|
||||
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||
|
||||
def binding_count(name: str) -> int:
|
||||
return _ast_binding_count(tree, name)
|
||||
|
||||
def assigned(name: str) -> ast.expr:
|
||||
values = bindings.get(name, ())
|
||||
if len(values) != 1 or binding_count(name) != 1 or name in imported:
|
||||
reject()
|
||||
return values[0]
|
||||
|
||||
def imported_as(node: ast.expr, qualified: str) -> bool:
|
||||
return (
|
||||
isinstance(node, ast.Name)
|
||||
and imported.get(node.id) == [qualified]
|
||||
and binding_count(node.id) == 1
|
||||
)
|
||||
|
||||
def call(node: ast.expr, qualified: str, arguments: int) -> bool:
|
||||
return isinstance(node, ast.Call) and imported_as(node.func, qualified) and len(node.args) == arguments and not node.keywords
|
||||
|
||||
def file_wrapper_target(node: ast.expr) -> str:
|
||||
if binding_count("__file__"):
|
||||
reject()
|
||||
if isinstance(node, ast.Name):
|
||||
node = assigned(node.id)
|
||||
if not (
|
||||
isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div)
|
||||
and isinstance(node.right, ast.Constant) and isinstance(node.right.value, str)
|
||||
and isinstance(node.left, ast.BinOp) and isinstance(node.left.op, ast.Div)
|
||||
and isinstance(node.left.right, ast.Constant) and node.left.right.value == "versions"
|
||||
):
|
||||
reject()
|
||||
root = node.left.left
|
||||
for name in imported:
|
||||
if imported_as(ast.Name(id=name), "pathlib.Path"):
|
||||
expected = ast.parse(f"{name}(__file__).resolve().parents[1]", mode="eval").body
|
||||
if ast.dump(root) == ast.dump(expected):
|
||||
return node.right.value
|
||||
reject()
|
||||
|
||||
result: dict[str, Migration] = {}
|
||||
resolved: set[Path] = set()
|
||||
for alias in sorted(aliases):
|
||||
value = assigned(alias)
|
||||
if call(value, "importlib.import_module", 1):
|
||||
argument = value.args[0]
|
||||
if not isinstance(argument, ast.Constant) or not isinstance(argument.value, str):
|
||||
reject()
|
||||
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||
if not argument.value.startswith(prefix):
|
||||
reject()
|
||||
stem = argument.value.removeprefix(prefix)
|
||||
if not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem):
|
||||
reject()
|
||||
filename = stem + ".py"
|
||||
elif call(value, "importlib.util.module_from_spec", 1):
|
||||
argument = value.args[0]
|
||||
if not isinstance(argument, ast.Name):
|
||||
reject()
|
||||
specification = assigned(argument.id)
|
||||
if not call(specification, "importlib.util.spec_from_file_location", 2):
|
||||
reject()
|
||||
if not isinstance(specification.args[0], ast.Constant) or not isinstance(specification.args[0].value, str):
|
||||
reject()
|
||||
filename = file_wrapper_target(specification.args[1])
|
||||
else:
|
||||
reject()
|
||||
migration = _release_wrapper_peer(owner, path, filename)
|
||||
peer = migration.path
|
||||
resolved.add(peer.resolve())
|
||||
if len(resolved) > 1:
|
||||
reject()
|
||||
result[alias] = migration
|
||||
return result
|
||||
|
||||
|
||||
def _migration_assignment_value(
|
||||
name: str,
|
||||
value: ast.expr,
|
||||
*,
|
||||
wrapped: Migration | None,
|
||||
wrapped: dict[str, Migration],
|
||||
) -> Any:
|
||||
try:
|
||||
return ast.literal_eval(value)
|
||||
except (ValueError, TypeError):
|
||||
if (
|
||||
wrapped is None
|
||||
or not isinstance(value, ast.Attribute)
|
||||
not isinstance(value, ast.Attribute)
|
||||
or not isinstance(value.value, ast.Name)
|
||||
or value.value.id != "_migration"
|
||||
or value.value.id not in wrapped
|
||||
or value.attr != name
|
||||
):
|
||||
return None
|
||||
raise ValueError(f"Unsupported or ambiguous migration metadata: {name}")
|
||||
migration = wrapped[value.value.id]
|
||||
return {
|
||||
"revision": wrapped.revision,
|
||||
"down_revision": wrapped.down_revisions,
|
||||
"depends_on": wrapped.depends_on,
|
||||
"branch_labels": wrapped.branch_labels,
|
||||
"revision": migration.revision,
|
||||
"down_revision": migration.down_revisions,
|
||||
"depends_on": migration.depends_on,
|
||||
"branch_labels": migration.branch_labels,
|
||||
}[name]
|
||||
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user