8 Commits
Author SHA1 Message Date
zemion a24c94435e Release v0.1.15
Dependency Audit / dependency-audit (push) Failing after 1m49s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m50s
Developer Meta-package Release / publish-package (push) Failing after 4s
2026-08-04 15:20:50 +02:00
zemion 774793976c Support legacy module version declarations
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m42s
Security Audit / security-audit (push) Successful in 10m43s
2026-08-04 15:08:46 +02:00
zemion 492449a4e2 Align all release version declarations
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 10m42s
2026-08-04 15:06:36 +02:00
zemion 8e890b37ed Validate candidate migration baseline during release
Dependency Audit / dependency-audit (push) Failing after 1m42s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m38s
2026-08-04 15:01:50 +02:00
zemion 077735bc24 Bind migration heads to coordinated releases
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m44s
Security Audit / security-audit (push) Successful in 10m36s
2026-08-04 14:55:58 +02:00
zemion d9522d3cc4 Publish release tags in dependency order
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m53s
2026-08-04 14:54:54 +02:00
zemion bad0ea37a7 Automate exact package-set publication
Dependency Audit / dependency-audit (push) Failing after 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m46s
2026-08-04 14:32:06 +02:00
zemion 9ffd46fe22 Make package publication retries hash-safe
Dependency Audit / dependency-audit (push) Failing after 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m35s
2026-08-04 14:18:49 +02:00
12 changed files with 833 additions and 99 deletions
+56
View File
@@ -37,12 +37,68 @@ changes:
python tools/gitea/gitea-configure-package-releases.py
```
Preview and dispatch the exact wheel/WebUI versions selected by the developer
meta-package with:
```bash
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env \
--apply
```
The dispatcher reads exact versions from `packages/govoplan-meta/pyproject.toml`,
inspects the selected tag to determine whether a WebUI package is expected,
skips complete registry pairs and does not duplicate an active workflow. Use
`--repository govoplan-core` for a bounded dispatch or `--verify-existing` to
rebuild and hash-verify versions already present in both registries.
For coordinated lockstep tags, `push-release-tag.sh` pushes module tags first,
Core next, and the meta tag last. This is a dependency guarantee for a
single-capacity Actions runner: the developer package cannot run before its
exact Core and module versions have entered the queue.
The same release entry point first validates the migration graph, then records
the reviewed current Alembic heads under the target release version and reruns
the strict migration audit before it changes package versions, commits, or
tags. The default preflight intentionally does not require those heads to exist
in the previous release baseline. A failed candidate-baseline check therefore
cannot produce a protected package release.
The source gate validates `pyproject.toml`, the module version declaration
(`MODULE_VERSION` or the top-level `ModuleManifest.version`), public package
`__version__`, and WebUI metadata before creating tags. Release-tag artifact
checks run only after the candidate tags and immutable WebUI lock have been
created locally.
Release-lock regeneration resolves a fresh immutable lock from the reviewed
candidate manifests; it does not seed resolution from the previous release
lock. This prevents removed transitive packages and stale peer metadata from
blocking or contaminating the new release. Candidate resolution also uses an
isolated temporary npm cache, so a locally replaced tag cannot reuse metadata
from a failed, unpushed release attempt.
Modules that retain the same WebUI package identity in both a root publish
manifest and `webui/package.json` use the WebUI manifest as the canonical peer
contract. The coordinated release synchronizes `peerDependencies` and
`peerDependenciesMeta` into the publish manifest before creating the module
tag, then synchronizes each lockfile root from the final package metadata. A
distinct root package remains independent.
It builds one wheel and, where applicable, one npm tarball. The workflow records
the source tag, source commit, filename, size, and SHA-256 in
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
same package version, so correction requires a new version rather than artifact
replacement.
A retry after partial publication is safe. Before upload, the workflow reads the
native package registry file record and compares its SHA-256 with the artifact
rebuilt from the protected tag. An exact existing artifact is skipped; a
same-version artifact with another digest or an unexpected file set fails
closed. This permits a failed npm publication to resume without weakening
package immutability or accepting `--skip-existing` blindly.
The npm tarball is always published through an explicit local `./dist/...`
path. Without that prefix, npm may interpret a relative tarball name as a Git
package shorthand before it ever contacts the configured registry.
+67 -67
View File
@@ -4,82 +4,82 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan"
version = "0.1.14"
version = "0.1.15"
description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" }
dependencies = [
"govoplan-core[server]==0.1.14",
"govoplan-tenancy==0.1.8",
"govoplan-organizations==0.1.8",
"govoplan-identity==0.1.8",
"govoplan-idm==0.1.8",
"govoplan-access==0.1.8",
"govoplan-admin==0.1.8",
"govoplan-policy==0.1.8",
"govoplan-audit==0.1.8",
"govoplan-dashboard==0.1.8",
"govoplan-files==0.1.8",
"govoplan-mail==0.1.10",
"govoplan-campaign==0.1.11",
"govoplan-calendar==0.1.8",
"govoplan-docs==0.1.8",
"govoplan-ops==0.1.8",
"govoplan-core[server]==0.1.15",
"govoplan-tenancy==0.1.15",
"govoplan-organizations==0.1.15",
"govoplan-identity==0.1.15",
"govoplan-idm==0.1.15",
"govoplan-access==0.1.15",
"govoplan-admin==0.1.15",
"govoplan-policy==0.1.15",
"govoplan-audit==0.1.15",
"govoplan-dashboard==0.1.15",
"govoplan-files==0.1.15",
"govoplan-mail==0.1.15",
"govoplan-campaign==0.1.15",
"govoplan-calendar==0.1.15",
"govoplan-docs==0.1.15",
"govoplan-ops==0.1.15",
]
[project.optional-dependencies]
full = [
"govoplan-addresses==0.1.9",
"govoplan-approvals==0.1.14",
"govoplan-assets==0.1.8",
"govoplan-booking==0.1.8",
"govoplan-cases==0.1.8",
"govoplan-certificates==0.1.8",
"govoplan-committee==0.1.8",
"govoplan-connectors==0.1.14",
"govoplan-consultation==0.1.8",
"govoplan-contracts==0.1.8",
"govoplan-dataflow==0.1.14",
"govoplan-datasources==0.1.14",
"govoplan-decisions==0.1.14",
"govoplan-dist-lists==0.1.14",
"govoplan-encryption==0.1.14",
"govoplan-evaluation==0.1.8",
"govoplan-facilities==0.1.8",
"govoplan-forms==0.1.14",
"govoplan-forms-runtime==0.1.14",
"govoplan-grants==0.1.8",
"govoplan-helpdesk==0.1.8",
"govoplan-identity-trust==0.1.14",
"govoplan-inspections==0.1.8",
"govoplan-learning==0.1.8",
"govoplan-mandates==0.1.14",
"govoplan-notifications==0.1.8",
"govoplan-parties==0.1.14",
"govoplan-permits==0.1.8",
"govoplan-poll==0.1.11",
"govoplan-portal==0.1.8",
"govoplan-postbox==0.1.2",
"govoplan-procurement==0.1.8",
"govoplan-projects==0.1.14",
"govoplan-records==0.1.8",
"govoplan-reporting==0.1.14",
"govoplan-resources==0.1.8",
"govoplan-rest==0.1.7",
"govoplan-risk-compliance==0.1.8",
"govoplan-scheduling==0.1.11",
"govoplan-search==0.1.14",
"govoplan-services==0.1.14",
"govoplan-soap==0.1.7",
"govoplan-templates==0.1.14",
"govoplan-tickets==0.1.8",
"govoplan-transparency==0.1.8",
"govoplan-views==0.1.0",
"govoplan-voting==0.1.14",
"govoplan-wiki==0.1.14",
"govoplan-workflow==0.1.14",
"govoplan-workflow-engine==0.1.14",
"govoplan-addresses==0.1.15",
"govoplan-approvals==0.1.15",
"govoplan-assets==0.1.15",
"govoplan-booking==0.1.15",
"govoplan-cases==0.1.15",
"govoplan-certificates==0.1.15",
"govoplan-committee==0.1.15",
"govoplan-connectors==0.1.15",
"govoplan-consultation==0.1.15",
"govoplan-contracts==0.1.15",
"govoplan-dataflow==0.1.15",
"govoplan-datasources==0.1.15",
"govoplan-decisions==0.1.15",
"govoplan-dist-lists==0.1.15",
"govoplan-encryption==0.1.15",
"govoplan-evaluation==0.1.15",
"govoplan-facilities==0.1.15",
"govoplan-forms==0.1.15",
"govoplan-forms-runtime==0.1.15",
"govoplan-grants==0.1.15",
"govoplan-helpdesk==0.1.15",
"govoplan-identity-trust==0.1.15",
"govoplan-inspections==0.1.15",
"govoplan-learning==0.1.15",
"govoplan-mandates==0.1.15",
"govoplan-notifications==0.1.15",
"govoplan-parties==0.1.15",
"govoplan-permits==0.1.15",
"govoplan-poll==0.1.15",
"govoplan-portal==0.1.15",
"govoplan-postbox==0.1.15",
"govoplan-procurement==0.1.15",
"govoplan-projects==0.1.15",
"govoplan-records==0.1.15",
"govoplan-reporting==0.1.15",
"govoplan-resources==0.1.15",
"govoplan-rest==0.1.15",
"govoplan-risk-compliance==0.1.15",
"govoplan-scheduling==0.1.15",
"govoplan-search==0.1.15",
"govoplan-services==0.1.15",
"govoplan-soap==0.1.15",
"govoplan-templates==0.1.15",
"govoplan-tickets==0.1.15",
"govoplan-transparency==0.1.15",
"govoplan-views==0.1.15",
"govoplan-voting==0.1.15",
"govoplan-wiki==0.1.15",
"govoplan-workflow==0.1.15",
"govoplan-workflow-engine==0.1.15",
]
[project.urls]
+15 -15
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published.
../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.10
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.11
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.8
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.15
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.15
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.15
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.15
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.15
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.15
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.15
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.15
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.15
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.15
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.15
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.15
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.15
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.15
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.15
+4
View File
@@ -26,6 +26,10 @@ class ModulePackageWorkflowTests(unittest.TestCase):
self.assertIn("api/packages/GovOPlaN/pypi", workflow)
self.assertIn("api/packages/GovOPlaN/npm", workflow)
self.assertIn('npm publish "./${webui_packages[0]}"', workflow)
self.assertIn("Check immutable registry state", workflow)
self.assertIn('files[0].get("sha256") != expected_sha256', workflow)
self.assertIn('if [[ "$PUBLISH_PYPI" == 1 ]]', workflow)
self.assertIn('[[ "$PUBLISH_NPM" == 1 ]]', workflow)
self.assertIn("GOVOPLAN_PACKAGE_TOKEN", workflow)
self.assertIn("must resolve to an exact registry version", workflow)
self.assertIn("git\\\\.add-ideas\\\\.de/(?:GovOPlaN|add-ideas)", workflow)
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
TOOLS_ROOT = META_ROOT / "tools" / "gitea"
if str(TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(TOOLS_ROOT))
SCRIPT = TOOLS_ROOT / "gitea-dispatch-package-set.py"
SPEC = importlib.util.spec_from_file_location("gitea_dispatch_package_set", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
MODULE = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = MODULE
SPEC.loader.exec_module(MODULE)
class PackageSetDispatchTests(unittest.TestCase):
def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None:
targets = MODULE.package_targets()
self.assertEqual(66, len(targets))
self.assertEqual(66, len({target.distribution for target in targets}))
by_name = {target.distribution: target for target in targets}
self.assertEqual("v0.1.14", by_name["govoplan-core"].tag)
self.assertEqual("v0.1.8", by_name["govoplan-access"].tag)
self.assertTrue(by_name["govoplan-core"].tag_exists)
self.assertTrue(by_name["govoplan-access"].has_webui)
self.assertEqual(
"@govoplan/access-webui",
by_name["govoplan-access"].webui_package,
)
if __name__ == "__main__":
unittest.main()
+55
View File
@@ -29,17 +29,72 @@ class ReleaseEntrypointGateTests(unittest.TestCase):
workflow = script[confirm:]
source_gate = workflow.index("run_version_alignment_gate source")
baseline = workflow.index("record_migration_release_baseline")
first_commit = workflow.index('run git -C "$repo" commit')
lock_generation = workflow.index("generate_release_lock")
full_gate = workflow.index("run_version_alignment_gate", source_gate + 1)
first_push = workflow.index('run git -C "$repo" push')
self.assertLess(baseline, source_gate)
self.assertLess(source_gate, first_commit)
self.assertLess(first_commit, lock_generation)
self.assertLess(lock_generation, full_gate)
self.assertLess(full_gate, first_push)
self.assertLess(manifest_gate, confirm)
def test_lockstep_release_pushes_meta_package_after_core(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
module_push = script.index('for repo in "${MODULE_REPOS[@]}"; do\n run git -C "$repo" push')
core_push = script.index('run git -C "$ROOT" push', module_push)
support_push = script.index('for repo in "${SUPPORT_REPOS[@]}"; do\n run git -C "$repo" push', core_push)
self.assertLess(module_push, core_push)
self.assertLess(core_push, support_push)
def test_default_migration_preflight_accepts_new_release_heads(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
audit_function = script[
script.index("run_migration_release_audit()") :
script.index("record_migration_release_baseline()")
]
self.assertNotIn("--strict-if-baseline", audit_function)
self.assertIn('command+=("--strict")', audit_function)
def test_source_gate_does_not_require_tags_before_they_are_created(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
gate = script[
script.index("run_version_alignment_gate()") :
script.index("run_manifest_shape_gate()")
]
self.assertIn('command+=(--source-metadata-only)', gate)
self.assertIn('else\n command+=(--release-composition)', gate)
def test_version_updater_targets_canonical_runtime_declarations(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
self.assertIn("^manifest\\s*=\\s*ModuleManifest", script)
self.assertIn("could not update module version declaration", script)
self.assertIn("update_package_init_versions", script)
self.assertIn("synchronize-webui-package-metadata.py", script)
self.assertIn('"peerDependenciesMeta",', script)
self.assertLess(
script.index('synchronize-webui-package-metadata.py" --repo "$repo"'),
script.index('synchronize_lockfile_root "$package_path"', script.index('synchronize-webui-package-metadata.py" --repo "$repo"')),
)
self.assertNotIn("could not update ModuleManifest.version", script)
def test_release_lock_refreshes_candidate_govoplan_metadata(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-lock.sh").read_text()
self.assertEqual(2, script.count('"npm_config_cache=$TMP_DIR/npm-cache"'))
self.assertNotIn(
'cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"',
script,
)
self.assertIn('cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"', script)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "release" / "synchronize-webui-package-metadata.py"
class SynchronizeWebuiPackageMetadataTests(unittest.TestCase):
def test_copies_peer_contract_without_changing_publish_paths(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"exports": {".": "./webui/src/index.ts"},
"peerDependencies": {"vite": "^6"},
}
)
)
(repo / "webui" / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"peerDependencies": {"vite": "^7"},
"peerDependenciesMeta": {"vite": {"optional": True}},
}
)
)
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
package = json.loads((repo / "package.json").read_text())
self.assertEqual({"vite": "^7"}, package["peerDependencies"])
self.assertEqual({"vite": {"optional": True}}, package["peerDependenciesMeta"])
self.assertEqual({".": "./webui/src/index.ts"}, package["exports"])
def test_leaves_distinct_root_and_webui_packages_separate(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(json.dumps({"name": "@govoplan/one"}))
(repo / "webui" / "package.json").write_text(json.dumps({"name": "@govoplan/two"}))
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
root = json.loads((repo / "package.json").read_text())
self.assertEqual("@govoplan/one", root["name"])
self.assertNotIn("peerDependencies", root)
if __name__ == "__main__":
unittest.main()
+302
View File
@@ -0,0 +1,302 @@
#!/usr/bin/env python3
"""Dispatch protected package releases required by the govoplan meta-package."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import json
from pathlib import Path
import re
import subprocess
import sys
import tomllib
from gitea_common import (
GiteaClient,
GiteaError,
RepoTarget,
load_dotenv,
org_path,
quote_path,
repo_path,
require_token,
)
META_ROOT = Path(__file__).resolve().parents[2]
META_PROJECT = META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml"
WORKFLOW_ID = "module-package-release.yml"
EXACT_REQUIREMENT = re.compile(
r"^(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])?==(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$"
)
ACTIVE_STATES = {"queued", "waiting", "in_progress", "running"}
@dataclass(frozen=True, slots=True)
class PackageTarget:
distribution: str
version: str
repository: str
tag_exists: bool
has_webui: bool
@property
def tag(self) -> str:
return f"v{self.version}"
@property
def webui_package(self) -> str | None:
if not self.has_webui:
return None
return f"@govoplan/{self.distribution.removeprefix('govoplan-')}-webui"
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--url", default="https://git.add-ideas.de")
parser.add_argument("--owner", default="GovOPlaN")
parser.add_argument("--env-file", type=Path)
parser.add_argument(
"--repository",
action="append",
default=[],
help="Limit dispatch to one repository; repeat as needed.",
)
parser.add_argument(
"--verify-existing",
action="store_true",
help="Also rerun exact versions already present in both registries.",
)
parser.add_argument("--apply", action="store_true")
return parser
def package_targets(project_path: Path = META_PROJECT) -> tuple[PackageTarget, ...]:
project = tomllib.loads(project_path.read_text(encoding="utf-8"))["project"]
requirements = list(project.get("dependencies") or [])
requirements.extend(project.get("optional-dependencies", {}).get("full") or [])
parsed: dict[str, str] = {}
for requirement in requirements:
match = EXACT_REQUIREMENT.fullmatch(str(requirement))
if match is None:
raise ValueError(
f"Meta-package requirement is not an exact GovOPlaN version: {requirement!r}"
)
name = match.group("name")
version = match.group("version")
previous = parsed.setdefault(name, version)
if previous != version:
raise ValueError(f"Meta-package selects conflicting versions for {name}")
targets: list[PackageTarget] = []
for distribution, version in sorted(parsed.items()):
repository = distribution
repository_root = META_ROOT.parent / repository
if not (repository_root / ".git").is_dir():
raise ValueError(f"Package repository is not checked out: {repository}")
tag = f"v{version}"
tag_exists = _tag_exists(repository_root, tag)
has_webui = (
_tag_has_path(repository_root, tag, "webui/package.json")
if tag_exists
else False
)
targets.append(
PackageTarget(
distribution=distribution,
version=version,
repository=repository,
tag_exists=tag_exists,
has_webui=has_webui,
)
)
return tuple(targets)
def _tag_exists(repository: Path, tag: str) -> bool:
result = subprocess.run(
(
"git",
"-C",
str(repository),
"rev-parse",
"--verify",
"--quiet",
f"refs/tags/{tag}",
),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 1}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}: {result.stderr.strip()}"
)
return result.returncode == 0
def _tag_has_path(repository: Path, tag: str, path: str) -> bool:
result = subprocess.run(
("git", "-C", str(repository), "cat-file", "-e", f"{tag}:{path}"),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 128}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}:{path}: {result.stderr.strip()}"
)
return result.returncode == 0
def _published_packages(
client: GiteaClient, *, owner: str, package_type: str
) -> set[tuple[str, str]]:
values = client.paginate(
f"/packages/{quote_path(owner)}",
query={"type": package_type, "q": "govoplan"},
)
return {
(str(item.get("name") or ""), str(item.get("version") or ""))
for item in values
if item.get("type") == package_type
}
def _has_active_run(
client: GiteaClient, *, owner: str, repository: str
) -> bool:
payload = client.request_json(
"GET",
repo_path(
owner,
repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/runs",
),
query={"limit": 10},
)
runs = payload.get("workflow_runs") if isinstance(payload, dict) else None
return isinstance(runs, list) and any(
isinstance(run, dict) and str(run.get("status") or "") in ACTIVE_STATES
for run in runs
)
def dispatch(
client: GiteaClient,
*,
owner: str,
targets: tuple[PackageTarget, ...],
published_pypi: set[tuple[str, str]],
published_npm: set[tuple[str, str]],
verify_existing: bool,
apply: bool,
) -> tuple[int, int, int]:
dispatched = 0
active = 0
complete = 0
for target in targets:
wheel_exists = (target.distribution, target.version) in published_pypi
npm_exists = target.webui_package is None or (
target.webui_package,
target.version,
) in published_npm
if wheel_exists and npm_exists and not verify_existing:
complete += 1
print(f"complete {target.repository}:{target.tag}")
continue
if _has_active_run(client, owner=owner, repository=target.repository):
active += 1
print(f"active {target.repository}:{target.tag}")
continue
action = "dispatching" if apply else "would dispatch"
print(
f"{action} {target.repository}:{target.tag} "
f"(wheel={'present' if wheel_exists else 'missing'}, "
f"webui={'present' if npm_exists else 'missing'})"
)
if apply:
client.request_json(
"POST",
repo_path(
owner,
target.repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/dispatches",
),
body={"ref": "main", "inputs": {"release_tag": target.tag}},
)
dispatched += 1
return dispatched, active, complete
def main() -> int:
args = build_parser().parse_args()
try:
load_dotenv(args.env_file)
token = require_token()
targets = package_targets()
selected = set(args.repository)
if selected:
known = {target.repository for target in targets}
unknown = sorted(selected - known)
if unknown:
raise ValueError(
"Unknown meta-package repositories: " + ", ".join(unknown)
)
targets = tuple(
target for target in targets if target.repository in selected
)
missing_tags = [
f"{target.repository}:{target.tag}"
for target in targets
if not target.tag_exists
]
if missing_tags:
raise ValueError(
"Meta-package release tags are missing: " + ", ".join(missing_tags)
)
target = RepoTarget(base_url=args.url, owner=args.owner, repo="govoplan")
with GiteaClient(target, token) as client:
secrets = client.request_json(
"GET", org_path(args.owner, "/actions/secrets"), query={"limit": 50}
)
secret_names = {
str(item.get("name") or "")
for item in secrets
if isinstance(item, dict)
}
required = {"GOVOPLAN_PACKAGE_USERNAME", "GOVOPLAN_PACKAGE_TOKEN"}
if not required <= secret_names:
raise ValueError(
"Organization package publisher secrets are not configured"
)
published_pypi = _published_packages(
client, owner=args.owner, package_type="pypi"
)
published_npm = _published_packages(
client, owner=args.owner, package_type="npm"
)
counts = dispatch(
client,
owner=args.owner,
targets=targets,
published_pypi=published_pypi,
published_npm=published_npm,
verify_existing=args.verify_existing,
apply=args.apply,
)
action = "dispatched" if args.apply else "planned"
print(
f"Package set {action}: {counts[0]}; active: {counts[1]}; "
f"already complete: {counts[2]}."
)
return 0
except (GiteaError, OSError, ValueError, json.JSONDecodeError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+8 -5
View File
@@ -94,9 +94,6 @@ cleanup() {
trap cleanup EXIT
cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"
if [[ -f "$WEBUI/package-lock.release.json" ]]; then
cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"
fi
echo "Generating release lockfile from $WEBUI/package.release.json"
echo "Temporary workspace: $TMP_DIR"
@@ -120,7 +117,10 @@ GIT_ENV+=("GIT_CONFIG_COUNT=$git_config_count")
(
cd "$TMP_DIR"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" install --package-lock-only --ignore-scripts
mapfile -t GIT_PACKAGES < <(
PATH="$(dirname "$NODE_BIN"):$PATH" "$NODE_BIN" <<'NODE'
const fs = require("fs");
@@ -136,7 +136,10 @@ NODE
)
if [[ "${#GIT_PACKAGES[@]}" -gt 0 ]]; then
echo "Refreshing git package lock entries: ${GIT_PACKAGES[*]}"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
fi
)
+77 -7
View File
@@ -333,20 +333,53 @@ path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(\s*version=)["\'][^"\']+["\'](,?\s*)$',
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count == 0:
text, count = re.subn(
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
r'(?ms)(^manifest\s*=\s*ModuleManifest\(.*?^\s*version\s*=\s*)["\'][^"\']+["\'](,?\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update ModuleManifest.version in {path}")
raise SystemExit(f"could not update module version declaration in {path}")
path.write_text(text)
PYCODE
done
}
update_package_init_versions() {
local repo="$1"
local version="$2"
local package_init=""
for package_init in "$repo"/src/*/__init__.py; do
[[ -f "$package_init" ]] || continue
if ! grep -q '^__version__\s*=' "$package_init"; then
continue
fi
"$PYTHON" - "$package_init" "$version" <<'PYCODE'
from __future__ import annotations
import pathlib
import re
import sys
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(__version__\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update __version__ in {path}")
path.write_text(text)
PYCODE
done
@@ -380,6 +413,11 @@ if project_name != "govoplan-core":
peers["@govoplan/core-webui"] = f"^{new_version}"
path.write_text(json.dumps(data, indent=2) + "\n")
PYCODE
done
"$PYTHON" "$META_ROOT/tools/release/synchronize-webui-package-metadata.py" --repo "$repo"
for package_path in "$repo/package.json" "$repo/webui/package.json"; do
[[ -f "$package_path" ]] || continue
synchronize_lockfile_root "$package_path" "${package_path%package.json}package-lock.json"
done
@@ -428,7 +466,19 @@ if not isinstance(version, str) or not version:
lock["version"] = version
packages = lock.get("packages")
if isinstance(packages, dict) and isinstance(packages.get(""), dict):
packages[""]["version"] = version
root = packages[""]
root["version"] = version
for group in (
"dependencies",
"devDependencies",
"optionalDependencies",
"peerDependencies",
"peerDependenciesMeta",
):
if group in package:
root[group] = package[group]
else:
root.pop(group, None)
lock_path.write_text(json.dumps(lock, indent=2) + "\n")
PYCODE
}
@@ -470,6 +520,7 @@ update_version_files() {
update_pyproject "$repo" "$version"
update_manifest_version "$repo" "$project_name" "$version"
update_package_init_versions "$repo" "$version"
update_webui_package "$repo" "$project_name" "$version"
}
@@ -500,10 +551,11 @@ run_version_alignment_gate() {
"$PYTHON"
"$META_ROOT/tools/checks/check-version-alignment.py"
--workspace-root "$PARENT"
--release-composition
)
if [[ "$mode" == "source" ]]; then
command+=(--source-metadata-only)
else
command+=(--release-composition)
fi
local repo
for repo in "${PACKAGE_REPOS[@]}"; do
@@ -533,7 +585,8 @@ run_migration_release_audit() {
command+=("--strict")
;;
auto)
command+=("--strict-if-baseline")
# A coordinated release creates a new baseline after confirmation. The
# preflight validates the graph; strictness applies to that new baseline.
;;
warn)
;;
@@ -546,6 +599,18 @@ run_migration_release_audit() {
run "${command[@]}"
}
record_migration_release_baseline() {
local audit_script="$META_ROOT/tools/release/release-migration-audit.py"
[[ -f "$audit_script" ]] || fail "missing migration audit helper: $audit_script"
run "$PYTHON" "$audit_script" \
--track release \
--record-release "$TARGET_VERSION"
if [[ "$DRY_RUN" -eq 0 ]]; then
"$PYTHON" "$audit_script" --track release --strict
fi
}
print_command() {
printf '+'
printf ' %q' "$@"
@@ -872,6 +937,8 @@ run_manifest_shape_gate
confirm_release
record_migration_release_baseline
for repo in "${PACKAGE_REPOS[@]}"; do
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Would update version files in $repo to $TARGET_VERSION"
@@ -938,10 +1005,13 @@ fi
run git -C "$ROOT" commit -m "$COMMIT_MESSAGE"
run git -C "$ROOT" tag -a "$TAG" -m "$TAG_MESSAGE"
for repo in "${PRE_CORE_REPOS[@]}"; do
for repo in "${MODULE_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
run git -C "$ROOT" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$ROOT]}" "refs/tags/$TAG"
for repo in "${SUPPORT_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
if [[ "$PUBLISH_WEB_CATALOG" -eq 1 ]]; then
CATALOG_ARGS=(
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Synchronize duplicated publish and development WebUI package contracts."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
SYNCHRONIZED_KEYS = ("peerDependencies", "peerDependenciesMeta")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--repo", type=Path, required=True)
args = parser.parse_args()
root_path = args.repo / "package.json"
webui_path = args.repo / "webui" / "package.json"
if not root_path.exists() or not webui_path.exists():
return 0
root = _load(root_path)
webui = _load(webui_path)
root_name = root.get("name")
webui_name = webui.get("name")
if not isinstance(root_name, str) or root_name != webui_name:
return 0
changed = False
for key in SYNCHRONIZED_KEYS:
if key in webui:
value = webui[key]
if root.get(key) != value:
root[key] = value
changed = True
elif key in root:
del root[key]
changed = True
if changed:
root_path.write_text(json.dumps(root, indent=2) + "\n")
print(f"Synchronized WebUI peer metadata in {root_path}")
return 0
def _load(path: Path) -> dict[str, object]:
payload = json.loads(path.read_text())
if not isinstance(payload, dict):
raise SystemExit(f"package metadata must be an object: {path}")
return payload
if __name__ == "__main__":
raise SystemExit(main())
@@ -163,6 +163,78 @@ jobs:
with:
name: module-packages-${{ gitea.ref_name }}
path: dist/package-artifacts.json
- name: Check immutable registry state
shell: bash
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
api_root = "https://git.add-ideas.de/api/v1/packages/GovOPlaN"
token = os.environ["PACKAGE_TOKEN"]
def should_publish(kind, name, version, path):
package_url = "/".join(
(api_root, kind, quote(name, safe=""), quote(version, safe=""), "files")
)
request = Request(
package_url,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code == 404:
print(f"{kind} package {name}=={version} is not published yet")
return True
raise
if not isinstance(files, list) or len(files) != 1:
raise SystemExit(
f"immutable {kind} package {name}=={version} has an unexpected file set"
)
expected_sha256 = hashlib.sha256(path.read_bytes()).hexdigest()
if files[0].get("sha256") != expected_sha256:
raise SystemExit(
f"immutable {kind} package {name}=={version} already exists with a different SHA-256"
)
print(f"verified existing {kind} package {name}=={version} ({expected_sha256})")
return False
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("release build must contain exactly one wheel")
publish_pypi = should_publish(
"pypi", str(project["name"]), str(project["version"]), wheels[0]
)
tarballs = tuple(Path("dist").glob("*.tgz"))
if len(tarballs) > 1:
raise SystemExit("release build must contain at most one npm package")
publish_npm = False
if tarballs:
webui = json.loads(
Path(".package-webui/package.json").read_text(encoding="utf-8")
)
publish_npm = should_publish(
"npm", str(webui["name"]), str(webui["version"]), tarballs[0]
)
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish_pypi)}\n")
env_file.write(f"PUBLISH_NPM={int(publish_npm)}\n")
PY
- name: Publish wheel and WebUI package
shell: bash
env:
@@ -172,13 +244,17 @@ jobs:
set -euo pipefail
test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN"
if [[ "$PUBLISH_PYPI" == 1 ]]; then
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl
else
echo "Exact wheel is already present; skipping immutable retry."
fi
shopt -s nullglob
webui_packages=(dist/*.tgz)
if (( ${#webui_packages[@]} )); then
if (( ${#webui_packages[@]} )) && [[ "$PUBLISH_NPM" == 1 ]]; then
npmrc="$(mktemp)"
trap 'rm -f "$npmrc"' EXIT
chmod 600 "$npmrc"
@@ -189,4 +265,6 @@ jobs:
NPM_CONFIG_USERCONFIG="$npmrc" npm publish "./${webui_packages[0]}" \
--ignore-scripts --access public \
--registry https://git.add-ideas.de/api/packages/GovOPlaN/npm/
elif (( ${#webui_packages[@]} )); then
echo "Exact WebUI package is already present; skipping immutable retry."
fi