[Task] Define release migration squashing and baseline workflow #223

New Issue

2026-07-09T11:37:01+02:00

zemion commented

2026-07-09 11:37:01 +02:00

Objective

Define and implement the GovOPlaN release policy for Alembic migration squashing/baselines so fresh installations do not pay for unreleased development churn, while released installations keep immutable upgrade paths.

Scope

Owning repository: govoplan-core.
In scope: release migration policy, release workflow integration, migration audit helper, baseline bookkeeping convention, and release verification gates.
Out of scope: deleting already released migration IDs after a stable release, and automatically generating complex data migrations without review.

Desired policy

Unreleased development migrations may be rewritten or squashed before a stable release.
Released migration IDs are immutable once a release tag has shipped.
Each stable release records the public migration head(s) that installation databases may contain.
Fresh installations should create the release schema directly through release-level baselines/upgrades, not through create-then-rename dev churn.
Release-to-release upgrade migrations should be folded into one reviewed migration per migration owner where practical.

Checklist

Document the migration baseline/squash policy in release docs.
Add the policy to the module architecture/database migration contract.
Add a release audit helper that reports current migration heads and whether baselines are recorded.
Wire the audit into release preparation docs and the release tag workflow.
Decide and implement the baseline metadata format for released heads.
Add tests for the audit helper and migration graph validation.
Add a reviewed/manual squash plan mode instead of an automatic schema-history rewriter.
Add a baseline recorder for reviewed release heads.
Decide and implement push-release-tag default behavior: warn before first baseline, strict after a baseline exists.
Before the first stable release, manually squash current unreleased dev migrations into the first public baseline, run release checks, and record the real release heads.

Verification Target

scripts/release-migration-audit.py reports the current migration graph and baseline state.
scripts/release-migration-audit.py --squash-plan prints the manual release squash workflow.
scripts/release-migration-audit.py --record-release <x.y.z> records reviewed heads.
scripts/push-release-tag.sh defaults to automatic migration audit behavior.
PostgreSQL release check still passes after any actual squash.

## Objective Define and implement the GovOPlaN release policy for Alembic migration squashing/baselines so fresh installations do not pay for unreleased development churn, while released installations keep immutable upgrade paths. ## Scope - Owning repository: govoplan-core. - In scope: release migration policy, release workflow integration, migration audit helper, baseline bookkeeping convention, and release verification gates. - Out of scope: deleting already released migration IDs after a stable release, and automatically generating complex data migrations without review. ## Desired policy - Unreleased development migrations may be rewritten or squashed before a stable release. - Released migration IDs are immutable once a release tag has shipped. - Each stable release records the public migration head(s) that installation databases may contain. - Fresh installations should create the release schema directly through release-level baselines/upgrades, not through create-then-rename dev churn. - Release-to-release upgrade migrations should be folded into one reviewed migration per migration owner where practical. ## Checklist - [x] Document the migration baseline/squash policy in release docs. - [x] Add the policy to the module architecture/database migration contract. - [x] Add a release audit helper that reports current migration heads and whether baselines are recorded. - [x] Wire the audit into release preparation docs and the release tag workflow. - [x] Decide and implement the baseline metadata format for released heads. - [x] Add tests for the audit helper and migration graph validation. - [x] Add a reviewed/manual squash plan mode instead of an automatic schema-history rewriter. - [x] Add a baseline recorder for reviewed release heads. - [x] Decide and implement push-release-tag default behavior: warn before first baseline, strict after a baseline exists. - [ ] Before the first stable release, manually squash current unreleased dev migrations into the first public baseline, run release checks, and record the real release heads. ## Verification Target - scripts/release-migration-audit.py reports the current migration graph and baseline state. - scripts/release-migration-audit.py --squash-plan prints the manual release squash workflow. - scripts/release-migration-audit.py --record-release <x.y.z> records reviewed heads. - scripts/push-release-tag.sh defaults to automatic migration audit behavior. - PostgreSQL release check still passes after any actual squash.

zemion commented

2026-07-09 11:41:42 +02:00

Implemented the first workflow slice.

Done:

documented the migration squash/baseline policy in RELEASE_DEPENDENCIES.md,
added the release-artifact rule to MODULE_ARCHITECTURE.md,
added RELEASE_CATALOG_WORKFLOW.md release steps for strict migration audits,
added docs/migration-release-baselines.json as the v1 baseline manifest stub,
added scripts/release-migration-audit.py,
wired scripts/push-release-tag.sh to run the audit in warn mode by default, with --strict-migration-audit and --skip-migration-audit,
added tests/test_release_migration_audit.py.

Current audit result: no migration graph errors. There is intentionally no recorded release baseline yet, so strict mode fails until the first stable release baseline is created and recorded.

Verification:

python -m unittest tests.test_release_migration_audit
python scripts/release-migration-audit.py
bash -n scripts/push-release-tag.sh

Remaining work before first stable release: manually squash/review the unreleased development migrations into first public release baselines, populate docs/migration-release-baselines.json, then run the PostgreSQL release check.

Implemented the first workflow slice. Done: - documented the migration squash/baseline policy in RELEASE_DEPENDENCIES.md, - added the release-artifact rule to MODULE_ARCHITECTURE.md, - added RELEASE_CATALOG_WORKFLOW.md release steps for strict migration audits, - added docs/migration-release-baselines.json as the v1 baseline manifest stub, - added scripts/release-migration-audit.py, - wired scripts/push-release-tag.sh to run the audit in warn mode by default, with --strict-migration-audit and --skip-migration-audit, - added tests/test_release_migration_audit.py. Current audit result: no migration graph errors. There is intentionally no recorded release baseline yet, so strict mode fails until the first stable release baseline is created and recorded. Verification: - python -m unittest tests.test_release_migration_audit - python scripts/release-migration-audit.py - bash -n scripts/push-release-tag.sh Remaining work before first stable release: manually squash/review the unreleased development migrations into first public release baselines, populate docs/migration-release-baselines.json, then run the PostgreSQL release check.

zemion commented

2026-07-09 11:52:09 +02:00

Finished the helper/workflow gap items.

Implemented:

scripts/release-migration-audit.py --squash-plan for the reviewed/manual squash checklist;
scripts/release-migration-audit.py --record-release plus --replace-release for baseline recording;
--strict-if-baseline so release automation can warn before the first baseline and become strict afterwards;
push-release-tag.sh default migration audit mode now uses that automatic behavior;
--warn-migration-audit, --strict-migration-audit, and --skip-migration-audit remain explicit overrides;
tests for typed Alembic parsing, owner-head subset installs, baseline recording, and graph errors.

Decision: do not implement an automatic migration squash generator right now. Squashing remains a reviewed release step because generating destructive/rewrite migration history automatically is too risky for the current stage. The helper now prints the squash plan and records the reviewed outcome.

I intentionally did not record docs/migration-release-baselines.json for the current working tree. HEAD is tagged v0.1.6 but the workspace has unreleased migration changes, so stamping those heads as v0.1.6 would be misleading.

Verification:

python -m unittest tests.test_release_migration_audit
python scripts/release-migration-audit.py --squash-plan
python scripts/release-migration-audit.py --strict-if-baseline
temp-file exercise of --record-release followed by --strict
bash -n scripts/push-release-tag.sh

Finished the helper/workflow gap items. Implemented: - scripts/release-migration-audit.py --squash-plan for the reviewed/manual squash checklist; - scripts/release-migration-audit.py --record-release <version> plus --replace-release for baseline recording; - --strict-if-baseline so release automation can warn before the first baseline and become strict afterwards; - push-release-tag.sh default migration audit mode now uses that automatic behavior; - --warn-migration-audit, --strict-migration-audit, and --skip-migration-audit remain explicit overrides; - tests for typed Alembic parsing, owner-head subset installs, baseline recording, and graph errors. Decision: do not implement an automatic migration squash generator right now. Squashing remains a reviewed release step because generating destructive/rewrite migration history automatically is too risky for the current stage. The helper now prints the squash plan and records the reviewed outcome. I intentionally did not record docs/migration-release-baselines.json for the current working tree. HEAD is tagged v0.1.6 but the workspace has unreleased migration changes, so stamping those heads as v0.1.6 would be misleading. Verification: - python -m unittest tests.test_release_migration_audit - python scripts/release-migration-audit.py --squash-plan - python scripts/release-migration-audit.py --strict-if-baseline - temp-file exercise of --record-release followed by --strict - bash -n scripts/push-release-tag.sh

zemion commented

2026-07-10 11:03:30 +02:00

Codex State: progress

Summary

Reviewed before push. The release migration audit tooling and docs are in place, but the first stable release baseline has not been recorded yet.
scripts/release-migration-audit.py --squash-plan prints the manual workflow and current owner heads; --strict correctly fails before the first recorded baseline.
Current unrecorded heads: govoplan-calendar 9e0f1a2b3c4d, govoplan-campaign 2c3d4e5f7081, govoplan-core 3f4a5b6c7d8e, govoplan-files a7b8c9d0e1f3, govoplan-mail 3d4e5f708192.

Verification

./.venv/bin/python scripts/release-migration-audit.py --squash-plan ran
./.venv/bin/python scripts/release-migration-audit.py --strict fails as expected: no released migration baseline recorded

Next / Blocked

Do not cut a stable release tag until the unreleased migrations are manually reviewed/squashed as needed and scripts/release-migration-audit.py --record-release records the reviewed heads.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - Reviewed before push. The release migration audit tooling and docs are in place, but the first stable release baseline has not been recorded yet. - scripts/release-migration-audit.py --squash-plan prints the manual workflow and current owner heads; --strict correctly fails before the first recorded baseline. - Current unrecorded heads: govoplan-calendar 9e0f1a2b3c4d, govoplan-campaign 2c3d4e5f7081, govoplan-core 3f4a5b6c7d8e, govoplan-files a7b8c9d0e1f3, govoplan-mail 3d4e5f708192. ### Verification - `./.venv/bin/python scripts/release-migration-audit.py --squash-plan ran` - `./.venv/bin/python scripts/release-migration-audit.py --strict fails as expected: no released migration baseline recorded` ### Next / Blocked - Do not cut a stable release tag until the unreleased migrations are manually reviewed/squashed as needed and scripts/release-migration-audit.py --record-release <version> records the reviewed heads. Suggested status label: `status/in-progress`

zemion referenced this issue

2026-07-10 23:39:46 +02:00

[Debt] Remove legacy compatibility code only after migration/export compatibility requirements are explicit #40

zemion commented

2026-07-10 23:39:51 +02:00

Linked compatibility-cleanup umbrella add-ideas/govoplan-core#40: add-ideas/govoplan-core#40.

Migration baselines/squashing define when historical compatibility code can be retired safely.

Linked compatibility-cleanup umbrella `add-ideas/govoplan-core#40`: https://git.add-ideas.de/add-ideas/govoplan-core/issues/40. Migration baselines/squashing define when historical compatibility code can be retired safely.

zemion commented

2026-07-11 01:33:50 +02:00

Current check after the target-state planner work:

The audit/helper/workflow parts are implemented and working.
scripts/release-migration-audit.py reports a clean graph but no recorded release baseline.
Current heads are: campaign 2c3d4e5f7081, mail 3d4e5f708192, access 4a5b6c7d8e9f, core 4f2a9c8e7b6d, organizations 6d7e8f9a0b1c, idm 8f9a0b1c2d3e, calendar 9e0f1a2b3c4d, files a7b8c9d0e1f3.
v0.1.6 is not the current code state, and the workspace has unreleased planner/task-contract changes, so recording these heads as 0.1.6 would be wrong.

Decision: the remaining checkbox is not a normal implementation slice. It is a release-freeze action: choose the first stable baseline version, decide whether to manually squash the unreleased migration chains or accept the current chains as the first public baseline, run the release checks, then record that baseline. Do not close this until that release decision is made and the real release heads are recorded.

Current check after the target-state planner work: - The audit/helper/workflow parts are implemented and working. - `scripts/release-migration-audit.py` reports a clean graph but no recorded release baseline. - Current heads are: campaign `2c3d4e5f7081`, mail `3d4e5f708192`, access `4a5b6c7d8e9f`, core `4f2a9c8e7b6d`, organizations `6d7e8f9a0b1c`, idm `8f9a0b1c2d3e`, calendar `9e0f1a2b3c4d`, files `a7b8c9d0e1f3`. - `v0.1.6` is not the current code state, and the workspace has unreleased planner/task-contract changes, so recording these heads as `0.1.6` would be wrong. Decision: the remaining checkbox is not a normal implementation slice. It is a release-freeze action: choose the first stable baseline version, decide whether to manually squash the unreleased migration chains or accept the current chains as the first public baseline, run the release checks, then record that baseline. Do not close this until that release decision is made and the real release heads are recorded.

zemion commented

2026-07-11 02:59:14 +02:00

Released v0.1.7 and recorded the migration baseline. Core was amended once before final verification so the published core v0.1.7 tag includes the catalog test expectation update for 0.1.7.

Verification:

pushed module/core release tags v0.1.7
regenerated webui/package-lock.release.json
recorded docs/migration-release-baselines.json for 0.1.7
scripts/release-migration-audit.py --strict passes
python -m unittest tests.test_module_system tests.test_database_migrations tests.test_release_migration_audit passes

Core remote tag refs/tags/v0.1.7 dereferences to bfc882f.

Released v0.1.7 and recorded the migration baseline. Core was amended once before final verification so the published core v0.1.7 tag includes the catalog test expectation update for 0.1.7. Verification: - pushed module/core release tags v0.1.7 - regenerated webui/package-lock.release.json - recorded docs/migration-release-baselines.json for 0.1.7 - scripts/release-migration-audit.py --strict passes - python -m unittest tests.test_module_system tests.test_database_migrations tests.test_release_migration_audit passes Core remote tag refs/tags/v0.1.7 dereferences to bfc882f.

zemion closed this issue

2026-07-11 02:59:14 +02:00

zemion commented

2026-07-11 03:12:01 +02:00

Final release refs after including the late IDM/organizations UI commits and regenerating the core release lock:

govoplan-core v0.1.7 -> a00ef54821
govoplan-idm v0.1.7 -> b9b371a70470e30f470f8a9a3b03ee53e8657f84
govoplan-organizations v0.1.7 -> 213157aca1079ecf9576bad77366be9601dfabb9

webui/package-lock.release.json now resolves those module commits. Final checks passed:

scripts/release-migration-audit.py --strict
python -m unittest tests.test_module_system tests.test_database_migrations tests.test_release_migration_audit
npm run build in govoplan-core/webui
all govoplan-* working trees clean / not ahead / not behind.

Final release refs after including the late IDM/organizations UI commits and regenerating the core release lock: - govoplan-core v0.1.7 -> a00ef54821ee1c7be62c8ec199f970c4e0500ae8 - govoplan-idm v0.1.7 -> b9b371a70470e30f470f8a9a3b03ee53e8657f84 - govoplan-organizations v0.1.7 -> 213157aca1079ecf9576bad77366be9601dfabb9 webui/package-lock.release.json now resolves those module commits. Final checks passed: - scripts/release-migration-audit.py --strict - python -m unittest tests.test_module_system tests.test_database_migrations tests.test_release_migration_audit - npm run build in govoplan-core/webui - all govoplan-* working trees clean / not ahead / not behind.

zemion commented

2026-07-11 03:13:39 +02:00

Addendum: govoplan-web was also tagged as tag-only for v0.1.7.

govoplan-web v0.1.7 -> a506532084b289bed021d4d95c855547362af3d0

Final all-repo check: every /mnt/DATA/git/govoplan-* repository has local v0.1.7, and no repo is dirty/ahead/behind.

Addendum: govoplan-web was also tagged as tag-only for v0.1.7. - govoplan-web v0.1.7 -> a506532084b289bed021d4d95c855547362af3d0 Final all-repo check: every /mnt/DATA/git/govoplan-* repository has local v0.1.7, and no repo is dirty/ahead/behind.

zemion commented

2026-07-11 03:41:59 +02:00

Implemented the v0.1.7 first-baseline migration squash. The old unreleased v0.0.0 -> v0.1.7 development chains were replaced with one reviewed baseline migration per migration owner (core, access, calendar, campaign, files, identity, idm, mail, organizations), preserving the public revision IDs. The release audit now understands Alembic depends_on dependencies, and docs/tests were updated for dependency-leaf graph heads vs per-owner heads. Verified with strict migration audit, fresh SQLite baseline migration tests for core/default/full+idm, and module-system tests.

zemion commented

2026-07-11 11:49:38 +02:00

Implemented the two-track migration policy requested after the initial v0.1.7 squash: release/default loads reviewed shortcut migrations from versions, while explicit GOVOPLAN_MIGRATION_TRACK=dev loads restored detailed chains from dev_versions. Old development migrations were restored into dev_versions across core/access/calendar/campaign/files/identity/idm/mail/organizations; release baseline files remain in versions. The release audit now supports --track release|dev, records release baselines only on the release track, and release scripts force release-track audit. Added docs and tests for both tracks, including a dev-track catch-up migration for audit_outbox_events so dev and release fresh schemas converge. Verified with migration unit tests, release/dev audits, module-system tests, and backend smoke on the release track.

Implemented the two-track migration policy requested after the initial v0.1.7 squash: release/default loads reviewed shortcut migrations from `versions`, while explicit `GOVOPLAN_MIGRATION_TRACK=dev` loads restored detailed chains from `dev_versions`. Old development migrations were restored into dev_versions across core/access/calendar/campaign/files/identity/idm/mail/organizations; release baseline files remain in versions. The release audit now supports `--track release|dev`, records release baselines only on the release track, and release scripts force release-track audit. Added docs and tests for both tracks, including a dev-track catch-up migration for audit_outbox_events so dev and release fresh schemas converge. Verified with migration unit tests, release/dev audits, module-system tests, and backend smoke on the release track.

zemion referenced this issue

2026-07-29 20:44:54 +02:00

[Debt] Remove legacy compatibility code only after migration/export compatibility requirements are explicit #40

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-core#223