I have a huge Django project with 5000 database models spread over 30 or so apps. (an engineering CAD tool). Each model is fully populated with tons of fields and indexes and constraints. The initial migrations takes weeks. Is there a way to speed up the migration? Each individual operation in the migrations files takes minutes, and there are tens of thousands of these operations across all the apps. Is there a way to speed up each individual operation? It seems like each operation - including AddIndex and AddConstraint - reloads all the models, which slows down everything.
How many migration files are there?
Since you’re talking about initial migrations, have you squashed or simply removed and recreated all migrations?
(That’s what I would suggest - make a copy of your project, delete all existing migrations and rerun makemigrations. Then test that against a fresh database to see how long it takes.)
1 Like
This is on a fresh initial database. Each app might have 1-2 large migration files. 0001_initial and 0002_iniital. (a few 0003_initial).
Squashing doesn’t do anything. It just produces the same migration file out. It seems each individual operation is what takes time, and that after each operation it reloads all models, taking longer and longer as more database models are loaded. So currently trying to figure out a way to not have it reload models after each individual operation.
The other option I am thinking of is having the migrations write out SQL and run SQL migrations directly on the database, instead of the Python migrations, in order to avoid reloading models after each operation.
That sounds like a remarkably useful idea.
I wonder if there’s any benefit to be gained by running the migrations one app at a time.