Doc update: Describe steps for manually squashing migrations

Django Internals
1

Hi folks,

I’m looking to pick up the ticket on documenting the process for manually squashing migrations: as it looks like Mike doesn’t have the time to work on this. (Thanks Mike for putting in a valiant effort btw!) Also see existing thread on the mailing list.

I think we should push through something at least so as to let folks know that manually squashing is an easy option.

There are a few things that’d be good to clear up and get decisions on before submitting a PR:

  1. I think it’s worth continuing to document the squashmigrations command as the recommended “preferred” option. Carlton has mentioned they’ve used it regularly without any issue.

  2. The squashmigrations command can become problematic for larger projects. There are a few reports on issues with circular dependencies but I also had issues trying to get libs like django-pghistory/django-pgtrigger to work nicely with it. I think this is where we document “Here’s how you can manually squash migrations for when the squashmigrations command no longer works for you”.

  3. It seems there are 2 types of manual migration that people are doing:

a. “Resetting”: Deleting migrations, deleting database and simply rerunning makemigrations then applying fake migrations on the deployments as necessary.

b. “Replacing”: Moving migrations “out of the way”; running makemigrations with a distinct name; then moving the old migrations back and listing them in the “replaces” attribute similar to squashmigrations, while also copying any non-elidable operations (eg runpython/runsql) and their dependencies into the squashed migration. The process moving forward from here is the same for squashmigrations (wait til migrations are out of the “squash zone” then remove, update references, remove replaces, run migrate --prune, etc).

I’ve had success doing b. and found it quite easy to run through this particular project didn’t have circular cross-app dependencies. I’d recommend b over a as it makes it easier to manage deployments or other dev’s setup.

Is there anything anyone else would like to add? Would anyone strongly oppose b in favour of a? I heard some reports that running makemigrations from scratch with circular dependencies still won’t work? (I tested a small sample setup with very basic circular dependency and it seemed to run fine)

PS: There was a forum post a couple of years ago for an idea to add a --replace option which sounds similar to b albeit without the non-elidable detection. This would be useful for helping people run b.

Cheers,
David

2

Hey David.

That all sounds very sensible to me.

IIRC my objection to the previous PR was that it wasn’t very clear in the end — it introduced an extra notion of Trimming, which seemed to muddy the (already somewhat silty) water. I’d guess explaining clearly the options as they are today would be most of it (with maybe extra flags etc if needed from there).

Thanks for picking this up!

3

I took a first stab at this, however…

I was initially aiming to make the smallest update possible but realised that this section would be missing 2 useful new additions that have been introduced since this section was written:

  • makemigrations --update
  • optimizemigration

I really think we ought to include these under a general section on squashing/optimising (heading tbd?)

I was thinking:

h1 <heading tbd>

    h2 Optimising migrations
    
     - introduce the concept of optimising (will be referred to in sections below)
     - introduce the optimizemigration command
    
    h2 Updating migrations
     - introduce the makemigrations --update command and how it optimises
    
    h2 Squashing migrations
     - Explain the concept of squashing and how to coordinate deployments & other developers
    
        h3 Using squashmigrations
         - introduce the squashmigrations command (and possibly how it optimises)
            
        h3 Manually squashing
         - explain the steps involved if squashmigrations fails
1 Like
4

Draft PR up for feedback: Draft update to docs on squashing migrations by shangxiao · Pull Request #16843 · django/django · GitHub