Very slow migrations with large numbers of tables

Using Django Using the ORM
7

@charettes Thank you for the thoughtful, detailed response. I had run across at least one of those before, but had hoped there had been more recent traction. I understand it is a difficult problem.

In the short term, I have implemented the following change in my Django, which allows the migrations to run much more quickly and does not seem to cause any problems. I’m curious if you see any glaring issues with this patch.

I would eventually like to help solve this problem for Django, but I need to immerse myself in past efforts first. I will see about plugging in on one of the existing PRs, unless you think it would be better to start fresh, or if you would like me not to do any further work on this for now.

diff --git a/django/db/migrations/migration.py b/django/db/migrations/migration.py
index 3c7713c5ea..e0bc7cc619 100644
--- a/django/db/migrations/migration.py
+++ b/django/db/migrations/migration.py
@@ -101,6 +101,8 @@ class Migration:
         Migrations.
         """
         for operation in self.operations:
+            if isinstance(operation, RunPython):
+                project_state.reload_models(models=None)
             # If this operation cannot be represented as SQL, place a comment
             # there instead
             if collect_sql:
diff --git a/django/db/migrations/state.py b/django/db/migrations/state.py
index ae55967383..780096dca0 100644
--- a/django/db/migrations/state.py
+++ b/django/db/migrations/state.py
@@ -396,16 +396,20 @@ class ProjectState:
 
     def reload_model(self, app_label, model_name, delay=False):
         if "apps" in self.__dict__:  # hasattr would cache the property
-            related_models = self._find_reload_model(app_label, model_name, delay)
-            self._reload(related_models)
+            # related_models = self._find_reload_model(app_label, model_name, delay)
+            # self._reload(related_models)
+            self._reload(set([(app_label, model_name)]))
 
     def reload_models(self, models, delay=True):
         if "apps" in self.__dict__:  # hasattr would cache the property
-            related_models = set()
-            for app_label, model_name in models:
-                related_models.update(
-                    self._find_reload_model(app_label, model_name, delay)
-                )
+            if models is None:
+                related_models = set(self.models.keys())
+            else:
+                related_models = set()
+                for app_label, model_name in models:
+                    related_models.update(
+                        self._find_reload_model(app_label, model_name, delay)
+                    )
             self._reload(related_models)
 
     def _reload(self, related_models):
1 Like