Repository navigation
Conversation
Merging this PR will not alter performance1 warning
Comparing |
|
A few small things worth polishing before merge — none of them affect the core fix, just documentation and test coverage: 1. 2. 3. Consider an explicit no-op snapshot test. 4. 5. CHANGELOG could mention the internal change, not just the speedup. Everything else looks good — happy to approve once these are addressed (or if you'd rather handle them in a follow-up, that's fine too). |
|
@waketzheng sorry for the late reply and fixes. All 5 points are addressed in separate commits. |
Description
This change improves migration performance by avoiding repeated reconstruction of the complete historical model state.
It introduces lightweight migration state snapshots that:
ModelStatedata, including field dictionaries.old_state.old_stateandnew_stateoperation API for forwards migrations, rollbacks, custom operations, and SQL collection.It also changes model reloads so initial querysets are rebuilt only for models that were actually re-rendered. Previously, every operation rebuilt querysets for every model accumulated in the migration state.
Motivation and Context
Migration execution scaled approximately with the number of operations multiplied by the size of the accumulated schema.
Two repeated operations caused most of the overhead:
This was particularly visible in a large project containing over 2,500 migration operations and approximately 900 models.
Running SQL generation for a late migration previously took approximately three minutes:
With lightweight snapshots alone, the command still took approximately three minutes because state replay, not snapshot creation was the dominant cost for this migration.
After limiting queryset rebuilding to changed models, the same command completed in approximately five seconds:
How Has This Been Tested?
Testing was performed on macOS arm64 with Python 3.13.14.
The complete migration test directory was run:
results
304 passed, 6 skippedRegression tests were added to verify that:
The sqlmigrate command above and the same migrate command was run using an editable installation of this branch to measure the complete CLI path.
Ruff formatting and targeted lint checks also passed.
Checklist: