chore(release): coordinate integrity and performance source updates
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
This commit is contained in:
@@ -0,0 +1,91 @@
|
|||||||
|
# Integrity and performance source release — September 2026
|
||||||
|
|
||||||
|
Coordinated tracking: [Core #298](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298).
|
||||||
|
This source publication advances only the affected packages; it is not a signed
|
||||||
|
catalog publication, runtime-image release, or remote production rollout.
|
||||||
|
|
||||||
|
## Package set
|
||||||
|
|
||||||
|
| Package | Version |
|
||||||
|
| --- | --- |
|
||||||
|
| Core / developer meta-package | 0.1.46 |
|
||||||
|
| Addresses | 0.1.23 |
|
||||||
|
| Calendar | 0.1.24 |
|
||||||
|
| Campaign | 0.1.29 |
|
||||||
|
| Cases | 0.1.25 |
|
||||||
|
| Committee | 0.1.22 |
|
||||||
|
| Connectors | 0.1.27 |
|
||||||
|
| Dataflow | 0.1.25 |
|
||||||
|
| Datasources | 0.1.26 |
|
||||||
|
| Files | 0.1.27 |
|
||||||
|
| Forms Runtime | 0.1.22 |
|
||||||
|
| IDM | 0.1.26 |
|
||||||
|
| Mail | 0.1.28 |
|
||||||
|
| Reporting | 0.1.22 |
|
||||||
|
| Tickets | 0.1.23 |
|
||||||
|
|
||||||
|
Consumers of new Core helpers require Core 0.1.46 or later. Dataflow and
|
||||||
|
Datasources also require the matching Core WebUI contract. The release manifests,
|
||||||
|
immutable Git lock, and developer package describe this coordinated composition.
|
||||||
|
Unchanged packages retain their independent versions.
|
||||||
|
|
||||||
|
## Database and data integrity
|
||||||
|
|
||||||
|
The reviewed additive heads are Connectors `d2a4c6e8f0b1`, Datasources
|
||||||
|
`e2b8d4a0f6c3`, Files `a2b3c4d5e701`, and Mail `b5d6e7f8091a`.
|
||||||
|
Back up the target deployment and rehearse its normal upgrade before rollout.
|
||||||
|
Never downgrade away retained CSV originals without a separate recovery plan.
|
||||||
|
Files preserves historical duplicate copies; Mail leaves legacy maildrop identity
|
||||||
|
unset rather than guessing an account. A schema upgrade does not authorize POP3
|
||||||
|
retrieval/reconciliation, provider deletion, or message resending.
|
||||||
|
|
||||||
|
Development startup may automatically apply pending migrations after a watched
|
||||||
|
source change. Therefore, inspect actual Alembic heads and schema before assuming
|
||||||
|
a live development database is still at its pre-change state. A backup taken
|
||||||
|
after such an upgrade is a current-state recovery copy, not a pre-upgrade backup.
|
||||||
|
|
||||||
|
On 8 September, the local PostgreSQL development database already contained all
|
||||||
|
four heads. Schema and constraint inspection passed; recomputing the Files
|
||||||
|
identity backfill checked 7,385 rows with zero mismatches. A private current-state
|
||||||
|
database backup was created. Per-instance backup paths and row contents are not
|
||||||
|
published in this repository. The implementation's initial receipt claiming no
|
||||||
|
live migration occurred was incorrect: watched development-server restarts had
|
||||||
|
applied the migration files automatically. Restoring the backup into an isolated
|
||||||
|
PostgreSQL 16 cluster and running the normal upgrade preserved all 281 public
|
||||||
|
tables, 142,287 rows, and migration heads exactly. Both Datasources PostgreSQL
|
||||||
|
concurrency regressions passed. The test-only cluster was then stopped.
|
||||||
|
|
||||||
|
Release preparation additionally corrected Alembic's ConfigParser handling of
|
||||||
|
percent-escaped connection URLs, preserving the exact database URL. The static
|
||||||
|
migration auditor now recognizes the existing reviewed development-wrapper
|
||||||
|
aliases and peer filenames without executing migration code; historical
|
||||||
|
migrations are unchanged.
|
||||||
|
|
||||||
|
## Verification and boundaries
|
||||||
|
|
||||||
|
The implementation passed the required focused workspace gate, including 63
|
||||||
|
frontend build configurations and 230 browser conformance tests. Owning-module
|
||||||
|
regressions cover exact import/rollback evidence, authorization-before-pagination,
|
||||||
|
bounded recurrence and response processing, collision-safe attachment naming,
|
||||||
|
and stale asynchronous UI completion. English/German feature documentation stays
|
||||||
|
in each owning module; Core documents shared integrity contracts.
|
||||||
|
|
||||||
|
Mock-provider tests and local work-count measurements do not establish production
|
||||||
|
throughput or provider race behavior. Real S3/SMB/Seafile and POP3 acceptance,
|
||||||
|
representative load testing, and deployment authentication checks remain separate
|
||||||
|
operational validation. Source tags trigger package workflows; a pushed source
|
||||||
|
tag alone does not prove a registry artifact or signed catalog is published.
|
||||||
|
|
||||||
|
## Deutsch
|
||||||
|
|
||||||
|
Dieses koordinierte Quellrelease veröffentlicht nur die betroffenen Pakete.
|
||||||
|
Produktions-Images, signierter Modulkatalog und entfernte Produktivinstanzen
|
||||||
|
werden dadurch nicht ausgerollt. Die vier Migrationen bewahren bestehende Daten;
|
||||||
|
historische POP3-Zuordnungen werden nicht geraten. POP3 ist ein eigener
|
||||||
|
Import-Arbeitsablauf innerhalb von **Mail**, kein separat installierbares Modul.
|
||||||
|
|
||||||
|
Der Entwicklungsserver kann Migrationen beim automatischen Neustart nach einer
|
||||||
|
Quelländerung bereits anwenden. Tatsächliche Schema-Stände prüfen; eine danach
|
||||||
|
erstellte Sicherung enthält den aktuellen Stand und ist keine Sicherung vor dem
|
||||||
|
Upgrade. Externe Transportaktionen, erneutes Versenden und Löschungen werden
|
||||||
|
durch die Migration oder Quellveröffentlichung nicht ausgelöst.
|
||||||
@@ -4,15 +4,15 @@
|
|||||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
|
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
|
||||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
|
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
|
||||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
|
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
|
||||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.25
|
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.26
|
||||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
|
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
|
||||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
|
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
|
||||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
|
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
|
||||||
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
|
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
|
||||||
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
|
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
|
||||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.26
|
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.27
|
||||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.27
|
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.28
|
||||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.28
|
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.29
|
||||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.23
|
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.24
|
||||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
|
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
|
||||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
|
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
|
||||||
|
|||||||
@@ -65,6 +65,8 @@ branch_labels: Union[str, Sequence[str], None] = None
|
|||||||
)
|
)
|
||||||
wrapper = development / release.name
|
wrapper = development / release.name
|
||||||
wrapper.write_text(
|
wrapper.write_text(
|
||||||
|
"from importlib import import_module\n"
|
||||||
|
'_migration = import_module("govoplan_core.backend.migrations.versions.1234_example")\n'
|
||||||
"revision = _migration.revision\n"
|
"revision = _migration.revision\n"
|
||||||
"down_revision = _migration.down_revision\n"
|
"down_revision = _migration.down_revision\n"
|
||||||
"depends_on = _migration.depends_on\n"
|
"depends_on = _migration.depends_on\n"
|
||||||
@@ -79,6 +81,150 @@ branch_labels: Union[str, Sequence[str], None] = None
|
|||||||
self.assertEqual(("base",), migration.down_revisions)
|
self.assertEqual(("base",), migration.down_revisions)
|
||||||
self.assertEqual(("core",), migration.depends_on)
|
self.assertEqual(("core",), migration.depends_on)
|
||||||
|
|
||||||
|
def test_literal_wrapper_alias_and_different_filename_are_resolved_without_execution(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "versions").mkdir()
|
||||||
|
(root / "dev_versions").mkdir()
|
||||||
|
(root / "versions/1234_v019_example.py").write_text(
|
||||||
|
'revision = "1234"\ndown_revision = "base"\ndepends_on = "core"\nbranch_labels = None\n'
|
||||||
|
'raise AssertionError("Migration implementation must not execute")\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
wrapper = root / "dev_versions/1234_example.py"
|
||||||
|
for alias in ("_migration", "edit_revision", "message_actions"):
|
||||||
|
with self.subTest(alias=alias):
|
||||||
|
wrapper.write_text(
|
||||||
|
"from importlib import import_module as load_migration\n"
|
||||||
|
f'{alias} = load_migration("govoplan_campaign.backend.migrations.versions." "1234_v019_example")\n'
|
||||||
|
f"revision = {alias}.revision\ndown_revision = {alias}.down_revision\n"
|
||||||
|
f"depends_on = {alias}.depends_on\nbranch_labels = {alias}.branch_labels\n"
|
||||||
|
'raise AssertionError("Wrapper must not execute")\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
migration = audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||||
|
self.assertEqual(migration.revision, "1234")
|
||||||
|
self.assertEqual(migration.down_revisions, ("base",))
|
||||||
|
self.assertEqual(migration.depends_on, ("core",))
|
||||||
|
|
||||||
|
def test_core_literal_sibling_file_wrapper_is_resolved_without_execution(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "versions").mkdir()
|
||||||
|
(root / "dev_versions").mkdir()
|
||||||
|
(root / "versions/1234_example.py").write_text(
|
||||||
|
'revision = "1234"\ndown_revision = None\ndepends_on = None\nbranch_labels = None\n'
|
||||||
|
'raise AssertionError("Migration implementation must not execute")\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
wrapper = root / "dev_versions/1234_example.py"
|
||||||
|
wrapper.write_text(
|
||||||
|
"from importlib.util import module_from_spec, spec_from_file_location\n"
|
||||||
|
"from pathlib import Path\n"
|
||||||
|
'_path = Path(__file__).resolve().parents[1] / "versions" / "1234_example.py"\n'
|
||||||
|
'_spec = spec_from_file_location("synthetic_migration", _path)\n'
|
||||||
|
"_module = module_from_spec(_spec)\n_spec.loader.exec_module(_module)\n"
|
||||||
|
"revision = _module.revision\ndown_revision = _module.down_revision\n"
|
||||||
|
"depends_on = _module.depends_on\nbranch_labels = _module.branch_labels\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
migration = audit.parse_migration_file("govoplan-core", wrapper)
|
||||||
|
self.assertEqual(migration.revision, "1234")
|
||||||
|
self.assertEqual(migration.down_revisions, ())
|
||||||
|
|
||||||
|
def test_wrapper_rejects_dynamic_foreign_missing_or_rebound_targets(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "versions").mkdir()
|
||||||
|
(root / "dev_versions").mkdir()
|
||||||
|
(root / "versions/1234_example.py").write_text('revision = "1234"\n', encoding="utf-8")
|
||||||
|
wrapper = root / "dev_versions/1234_example.py"
|
||||||
|
valid = '_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||||
|
definitions = (
|
||||||
|
'_migration = import_module(module_name)\n',
|
||||||
|
'_migration = import_module("govoplan_mail.backend.migrations.versions.1234_example")\n',
|
||||||
|
'_migration = import_module("govoplan_campaign.backend.migrations.versions...other.1234_example")\n',
|
||||||
|
'_migration = import_module("govoplan_campaign.backend.migrations.versions.missing")\n',
|
||||||
|
valid + "_migration = another_module\n",
|
||||||
|
"import_module = another_loader\n" + valid,
|
||||||
|
"def import_module(value):\n return another_module\n" + valid,
|
||||||
|
"import another_loader as import_module\n" + valid,
|
||||||
|
valid + "class _migration:\n revision = 'another'\n",
|
||||||
|
"if condition:\n _migration = another_module\n" + valid,
|
||||||
|
valid + "del _migration\n",
|
||||||
|
)
|
||||||
|
for definition in definitions:
|
||||||
|
with self.subTest(definition=definition):
|
||||||
|
wrapper.write_text(
|
||||||
|
"from importlib import import_module\n" + definition + "revision = _migration.revision\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||||
|
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||||
|
|
||||||
|
def test_wrapper_rejects_symlink_and_mixed_release_metadata(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "versions").mkdir()
|
||||||
|
(root / "dev_versions").mkdir()
|
||||||
|
(root / "versions/1234_example.py").write_text('revision = "1234"\ndown_revision = None\n', encoding="utf-8")
|
||||||
|
(root / "versions/5678_example.py").write_text('revision = "5678"\ndown_revision = None\n', encoding="utf-8")
|
||||||
|
(root / "versions/linked.py").symlink_to(root / "versions/1234_example.py")
|
||||||
|
wrapper = root / "dev_versions/1234_example.py"
|
||||||
|
definitions = (
|
||||||
|
'_migration = import_module("govoplan_campaign.backend.migrations.versions.linked")\nrevision = _migration.revision\n',
|
||||||
|
'_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||||
|
'_other = import_module("govoplan_campaign.backend.migrations.versions.5678_example")\n'
|
||||||
|
'revision = _migration.revision\ndown_revision = _other.down_revision\n',
|
||||||
|
)
|
||||||
|
for definition in definitions:
|
||||||
|
with self.subTest(definition=definition):
|
||||||
|
wrapper.write_text("from importlib import import_module\n" + definition, encoding="utf-8")
|
||||||
|
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||||
|
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||||
|
|
||||||
|
def test_unresolved_revision_expression_is_not_silently_omitted(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
path = Path(directory) / "1234_example.py"
|
||||||
|
path.write_text("revision = calculate_revision()\n", encoding="utf-8")
|
||||||
|
with self.assertRaisesRegex(ValueError, "Unsupported or ambiguous migration metadata"):
|
||||||
|
audit.parse_migration_file("govoplan-core", path)
|
||||||
|
|
||||||
|
def test_explicit_metadata_reexport_is_resolved_without_execution(self) -> None:
|
||||||
|
audit = load_audit_module()
|
||||||
|
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "versions").mkdir()
|
||||||
|
(root / "dev_versions").mkdir()
|
||||||
|
(root / "versions/a234_example.py").write_text(
|
||||||
|
'revision = "1234"\ndown_revision = "base"\ndepends_on = None\nbranch_labels = None\n'
|
||||||
|
'raise AssertionError("Migration implementation must not execute")\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
wrapper = root / "dev_versions/1234_example.py"
|
||||||
|
source = "govoplan_organizations.backend.migrations.versions.a234_example"
|
||||||
|
wrapper.write_text(f"from {source} import revision, down_revision, depends_on, branch_labels\n", encoding="utf-8")
|
||||||
|
migration = audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||||
|
self.assertEqual(migration.revision, "1234")
|
||||||
|
self.assertEqual(migration.down_revisions, ("base",))
|
||||||
|
for declaration in (
|
||||||
|
f"from {source} import *\n",
|
||||||
|
f"from {source} import revision as down_revision\n",
|
||||||
|
f"from {source} import revision\nrevision = 'different'\n",
|
||||||
|
f"from {source} import revision\nimport another as revision\n",
|
||||||
|
f"from {source} import revision\ndef revision():\n pass\n",
|
||||||
|
f"from {source.replace('govoplan_organizations', 'govoplan_mail')} import revision\n",
|
||||||
|
):
|
||||||
|
with self.subTest(declaration=declaration):
|
||||||
|
wrapper.write_text(declaration, encoding="utf-8")
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||||
|
|
||||||
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
|
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
|
||||||
audit = load_audit_module()
|
audit = load_audit_module()
|
||||||
migrations = [
|
migrations = [
|
||||||
|
|||||||
@@ -2287,6 +2287,27 @@
|
|||||||
"path": "/tasks/{}",
|
"path": "/tasks/{}",
|
||||||
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
|
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
|
||||||
"repository": "govoplan-tasks"
|
"repository": "govoplan-tasks"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"category": "intentionally_headless",
|
||||||
|
"method": "GET",
|
||||||
|
"path": "/connectors/tabular-sources/{}/original-csv",
|
||||||
|
"rationale": "Authorized connector clients explicitly export retained original CSV after tenant, lifecycle, read-scope and source-integrity checks; ordinary catalogue responses never include source text.",
|
||||||
|
"repository": "govoplan-connectors"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"category": "intentionally_headless",
|
||||||
|
"method": "GET",
|
||||||
|
"path": "/datasources/{}/materializations/{}/original-csv",
|
||||||
|
"rationale": "Administrators explicitly export retained original CSV through an audited API; current and historical field, row and access policies must permit the entire original and source-integrity checks must pass.",
|
||||||
|
"repository": "govoplan-datasources"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"category": "intentionally_headless",
|
||||||
|
"method": "POST",
|
||||||
|
"path": "/mail/profiles/{}/pop3/imports/{}/bind-maildrop",
|
||||||
|
"rationale": "Authorized mail operators explicitly reconcile a legacy POP3 import to the current maildrop using confirmed binding, a current transport revision token and an exact retained/downloaded-byte match; the audited API never guesses historical account identity.",
|
||||||
|
"repository": "govoplan-mail"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"schema_version": 1
|
"schema_version": 1
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from datetime import datetime, timezone
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any, NoReturn
|
||||||
|
|
||||||
|
|
||||||
META_ROOT = Path(__file__).resolve().parents[2]
|
META_ROOT = Path(__file__).resolve().parents[2]
|
||||||
@@ -168,12 +168,13 @@ def owner_for_versions_dir(versions_dir: Path) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||||
|
if path.stat().st_size > 2 * 1024 * 1024:
|
||||||
|
raise ValueError(f"{path}: migration source exceeds the 2 MiB audit bound")
|
||||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||||
values: dict[str, Any] = {}
|
values = _imported_release_metadata(owner, path, tree)
|
||||||
wrapped: Migration | None = None
|
wrapped = _wrapped_release_metadata(owner, path, tree)
|
||||||
release_peer = path.parent.parent / "versions" / path.name
|
if values and wrapped:
|
||||||
if path.parent.name == "dev_versions" and release_peer.is_file():
|
raise ValueError(f"{path}: mixed migration metadata wrapper styles are ambiguous")
|
||||||
wrapped = parse_migration_file(owner, release_peer)
|
|
||||||
for statement in tree.body:
|
for statement in tree.body:
|
||||||
if isinstance(statement, ast.Assign):
|
if isinstance(statement, ast.Assign):
|
||||||
for target in statement.targets:
|
for target in statement.targets:
|
||||||
@@ -196,6 +197,8 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
|||||||
)
|
)
|
||||||
revision = values.get("revision")
|
revision = values.get("revision")
|
||||||
if not isinstance(revision, str):
|
if not isinstance(revision, str):
|
||||||
|
if "revision" in values:
|
||||||
|
raise ValueError(f"{path}: migration revision must be an explicit string")
|
||||||
return None
|
return None
|
||||||
return Migration(
|
return Migration(
|
||||||
owner=owner,
|
owner=owner,
|
||||||
@@ -207,28 +210,200 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ast_binding_count(tree: ast.Module, name: str) -> int:
|
||||||
|
count = 0
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, ast.Name) and node.id == name and isinstance(node.ctx, (ast.Store, ast.Del)):
|
||||||
|
count += 1
|
||||||
|
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and node.name == name:
|
||||||
|
count += 1
|
||||||
|
elif isinstance(node, (ast.Import, ast.ImportFrom)):
|
||||||
|
count += sum(
|
||||||
|
(alias.asname or (alias.name.split(".")[0] if isinstance(node, ast.Import) else alias.name)) == name
|
||||||
|
for alias in node.names
|
||||||
|
)
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def _release_wrapper_peer(owner: str, path: Path, filename: str) -> Migration:
|
||||||
|
versions = path.parent.parent / "versions"
|
||||||
|
peer = versions / filename
|
||||||
|
if (
|
||||||
|
path.parent.name != "dev_versions"
|
||||||
|
or Path(filename).name != filename or not filename.endswith(".py")
|
||||||
|
or versions.is_symlink() or peer.is_symlink() or not peer.is_file()
|
||||||
|
or peer.resolve().parent != versions.resolve()
|
||||||
|
or peer.stat().st_size > 2 * 1024 * 1024
|
||||||
|
):
|
||||||
|
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||||
|
migration = parse_migration_file(owner, peer)
|
||||||
|
if migration is None:
|
||||||
|
raise ValueError(f"{path}: release wrapper target has no migration metadata")
|
||||||
|
return migration
|
||||||
|
|
||||||
|
|
||||||
|
def _imported_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Any]:
|
||||||
|
"""Recognize explicit metadata re-exports, never star/dynamic imports."""
|
||||||
|
names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||||
|
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||||
|
values: dict[str, Any] = {}
|
||||||
|
targets: set[str] = set()
|
||||||
|
for statement in tree.body:
|
||||||
|
if not isinstance(statement, ast.ImportFrom):
|
||||||
|
continue
|
||||||
|
selected = [alias for alias in statement.names if alias.name in names or (alias.asname or alias.name) in names]
|
||||||
|
if not selected:
|
||||||
|
if (statement.module or "").startswith(prefix) and any(alias.name == "*" for alias in statement.names):
|
||||||
|
raise ValueError(f"{path}: unsupported wildcard migration metadata")
|
||||||
|
continue
|
||||||
|
stem = (statement.module or "").removeprefix(prefix)
|
||||||
|
if (
|
||||||
|
statement.level or not (statement.module or "").startswith(prefix)
|
||||||
|
or not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem)
|
||||||
|
):
|
||||||
|
raise ValueError(f"{path}: unsupported or ambiguous imported migration metadata")
|
||||||
|
targets.add(stem)
|
||||||
|
if len(targets) != 1:
|
||||||
|
raise ValueError(f"{path}: ambiguous imported migration metadata sources")
|
||||||
|
migration = _release_wrapper_peer(owner, path, stem + ".py")
|
||||||
|
projection = {
|
||||||
|
"revision": migration.revision, "down_revision": migration.down_revisions,
|
||||||
|
"depends_on": migration.depends_on, "branch_labels": migration.branch_labels,
|
||||||
|
}
|
||||||
|
for alias in selected:
|
||||||
|
name = alias.asname or alias.name
|
||||||
|
if name != alias.name or name in values or _ast_binding_count(tree, name) != 1:
|
||||||
|
raise ValueError(f"{path}: ambiguous imported migration metadata assignment")
|
||||||
|
values[name] = projection[name]
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def _wrapped_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Migration]:
|
||||||
|
"""Resolve only known literal sibling wrappers, without importing any code."""
|
||||||
|
metadata_names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||||
|
aliases: set[str] = set()
|
||||||
|
bindings: dict[str, list[ast.expr]] = {}
|
||||||
|
imported: dict[str, list[str]] = {}
|
||||||
|
for statement in tree.body:
|
||||||
|
if isinstance(statement, ast.ImportFrom) and not statement.level:
|
||||||
|
for alias in statement.names:
|
||||||
|
imported.setdefault(alias.asname or alias.name, []).append(f"{statement.module}.{alias.name}")
|
||||||
|
targets: list[ast.expr] = []
|
||||||
|
value: ast.expr | None = None
|
||||||
|
if isinstance(statement, ast.Assign):
|
||||||
|
targets, value = statement.targets, statement.value
|
||||||
|
elif isinstance(statement, ast.AnnAssign) and statement.value is not None:
|
||||||
|
targets, value = [statement.target], statement.value
|
||||||
|
for target in targets:
|
||||||
|
if isinstance(target, ast.Name) and value is not None:
|
||||||
|
bindings.setdefault(target.id, []).append(value)
|
||||||
|
if target.id in metadata_names and isinstance(value, ast.Attribute) and isinstance(value.value, ast.Name):
|
||||||
|
aliases.add(value.value.id)
|
||||||
|
if not aliases:
|
||||||
|
return {}
|
||||||
|
if path.parent.name != "dev_versions":
|
||||||
|
raise ValueError(f"{path}: non-literal release migration metadata is unsupported")
|
||||||
|
|
||||||
|
def reject() -> NoReturn:
|
||||||
|
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||||
|
|
||||||
|
def binding_count(name: str) -> int:
|
||||||
|
return _ast_binding_count(tree, name)
|
||||||
|
|
||||||
|
def assigned(name: str) -> ast.expr:
|
||||||
|
values = bindings.get(name, ())
|
||||||
|
if len(values) != 1 or binding_count(name) != 1 or name in imported:
|
||||||
|
reject()
|
||||||
|
return values[0]
|
||||||
|
|
||||||
|
def imported_as(node: ast.expr, qualified: str) -> bool:
|
||||||
|
return (
|
||||||
|
isinstance(node, ast.Name)
|
||||||
|
and imported.get(node.id) == [qualified]
|
||||||
|
and binding_count(node.id) == 1
|
||||||
|
)
|
||||||
|
|
||||||
|
def call(node: ast.expr, qualified: str, arguments: int) -> bool:
|
||||||
|
return isinstance(node, ast.Call) and imported_as(node.func, qualified) and len(node.args) == arguments and not node.keywords
|
||||||
|
|
||||||
|
def file_wrapper_target(node: ast.expr) -> str:
|
||||||
|
if binding_count("__file__"):
|
||||||
|
reject()
|
||||||
|
if isinstance(node, ast.Name):
|
||||||
|
node = assigned(node.id)
|
||||||
|
if not (
|
||||||
|
isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div)
|
||||||
|
and isinstance(node.right, ast.Constant) and isinstance(node.right.value, str)
|
||||||
|
and isinstance(node.left, ast.BinOp) and isinstance(node.left.op, ast.Div)
|
||||||
|
and isinstance(node.left.right, ast.Constant) and node.left.right.value == "versions"
|
||||||
|
):
|
||||||
|
reject()
|
||||||
|
root = node.left.left
|
||||||
|
for name in imported:
|
||||||
|
if imported_as(ast.Name(id=name), "pathlib.Path"):
|
||||||
|
expected = ast.parse(f"{name}(__file__).resolve().parents[1]", mode="eval").body
|
||||||
|
if ast.dump(root) == ast.dump(expected):
|
||||||
|
return node.right.value
|
||||||
|
reject()
|
||||||
|
|
||||||
|
result: dict[str, Migration] = {}
|
||||||
|
resolved: set[Path] = set()
|
||||||
|
for alias in sorted(aliases):
|
||||||
|
value = assigned(alias)
|
||||||
|
if call(value, "importlib.import_module", 1):
|
||||||
|
argument = value.args[0]
|
||||||
|
if not isinstance(argument, ast.Constant) or not isinstance(argument.value, str):
|
||||||
|
reject()
|
||||||
|
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||||
|
if not argument.value.startswith(prefix):
|
||||||
|
reject()
|
||||||
|
stem = argument.value.removeprefix(prefix)
|
||||||
|
if not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem):
|
||||||
|
reject()
|
||||||
|
filename = stem + ".py"
|
||||||
|
elif call(value, "importlib.util.module_from_spec", 1):
|
||||||
|
argument = value.args[0]
|
||||||
|
if not isinstance(argument, ast.Name):
|
||||||
|
reject()
|
||||||
|
specification = assigned(argument.id)
|
||||||
|
if not call(specification, "importlib.util.spec_from_file_location", 2):
|
||||||
|
reject()
|
||||||
|
if not isinstance(specification.args[0], ast.Constant) or not isinstance(specification.args[0].value, str):
|
||||||
|
reject()
|
||||||
|
filename = file_wrapper_target(specification.args[1])
|
||||||
|
else:
|
||||||
|
reject()
|
||||||
|
migration = _release_wrapper_peer(owner, path, filename)
|
||||||
|
peer = migration.path
|
||||||
|
resolved.add(peer.resolve())
|
||||||
|
if len(resolved) > 1:
|
||||||
|
reject()
|
||||||
|
result[alias] = migration
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
def _migration_assignment_value(
|
def _migration_assignment_value(
|
||||||
name: str,
|
name: str,
|
||||||
value: ast.expr,
|
value: ast.expr,
|
||||||
*,
|
*,
|
||||||
wrapped: Migration | None,
|
wrapped: dict[str, Migration],
|
||||||
) -> Any:
|
) -> Any:
|
||||||
try:
|
try:
|
||||||
return ast.literal_eval(value)
|
return ast.literal_eval(value)
|
||||||
except (ValueError, TypeError):
|
except (ValueError, TypeError):
|
||||||
if (
|
if (
|
||||||
wrapped is None
|
not isinstance(value, ast.Attribute)
|
||||||
or not isinstance(value, ast.Attribute)
|
|
||||||
or not isinstance(value.value, ast.Name)
|
or not isinstance(value.value, ast.Name)
|
||||||
or value.value.id != "_migration"
|
or value.value.id not in wrapped
|
||||||
or value.attr != name
|
or value.attr != name
|
||||||
):
|
):
|
||||||
return None
|
raise ValueError(f"Unsupported or ambiguous migration metadata: {name}")
|
||||||
|
migration = wrapped[value.value.id]
|
||||||
return {
|
return {
|
||||||
"revision": wrapped.revision,
|
"revision": migration.revision,
|
||||||
"down_revision": wrapped.down_revisions,
|
"down_revision": migration.down_revisions,
|
||||||
"depends_on": wrapped.depends_on,
|
"depends_on": migration.depends_on,
|
||||||
"branch_labels": wrapped.branch_labels,
|
"branch_labels": migration.branch_labels,
|
||||||
}[name]
|
}[name]
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user