Migration optimizer does not optimize choices

Django Internals ORM
1

We have a lot of tables in our db which have a lot of category-type fields, and those charfields have a fixed set of choices that have been growing in the history of the project.

First migration:

        migrations.CreateModel(
            name="...",
            fields=[...
                (
                    "transaction_type",
                    models.CharField(
                        blank=True,
                        choices=[
                            ("DD_FIRST_COLLECTION", "Direct debit first collection"),
                            ("DD_REGULAR_COLLECTION", "Direct debit regular collection"),
                        ],
                        max_length=512,
                    ),
                ),

And lots of these:

        migrations.AlterField(
            model_name="accountpayment",
            name="transaction_type",
            field=models.CharField(
                blank=True,
                choices=[
                    ("DD_FIRST_COLLECTION", "Direct debit first collection"),
                    ("DD_REGULAR_COLLECTION", "Direct debit regular collection"),
                    ("CREDIT_CARD", "Credit card collection"),
....

I think the right behaviour, during squashmigrations, should be that the migrations optimizer merges these operations back into one. Especially if the second set is a superset of the first one.

Could this be a suggested improvement to the optimizer’s code? or is there a edge case which i am not seeing?

2

The problem - and the reason choices are there in the first place - is that the optimiser doesn’t know if choices is relevant or not (it’s the same with any field argument), and so it can’t do special handling just for it.

If we could trust that choices didn’t affect the database - which it doesn’t in almost all situations - then it would be fine. In your case, it probably is - but I don’t think we could turn that setting turned on by default.

The right approach here might be pluggable optimiser strategies that allow people to opt-in to certain fields not affecting the DB or being last-value-wins, like this.

3

Hey Peeps, reviving this ancient topic in the context of Django5’s new callable choices.

Django5 allows us to now wrap our choices enums in callables and pass these to modelfields instead:

def get_taxcode_choices() -> list[tuple[str, str]]:
    return TaxCode.choices

With this, migrations no longer need to change every time we change our enums! :partying_face: However, we’re still forced to create all these callables around what is let’s face it the majority of the time just a TextChoices class.

I put to you that we introduce a check that says

When writing modelfield choices out to a migration, if the choices are of the form <TextChoicesClass>.choices (or a similar django-recognised enum), we pass this expression directly to the migration without evaluating it out to a list.

This would be a beautiful QoL change, building upon improvements to the Choices class and choices modelfield attrib that have come through in more recent django versions.

Thoughts (on this approach, or some equivalent)?
It’s just super frustrating presently defining this beautiful TextChoices class only to not be able to pass it directly to our model fields..

4

I believe that another option that is present for a long time, is to inherit from the field and add a deconstruct method to remove the choices from the kwargs.
One also can monkey patch the django Field class deconstruct method to apply this to all of the fields on that project, I remember doing this for a project and it worked really well, but it’s real “magic” that you need to keep track of.

1 Like
5

Clever. I guess the tradeoff against “magic” improves with the number of times choices is used in the project. A hack like this could also be done as a bet that a future Django version might support what I’ve defined above, in which case there’d be close to no refactor needed to upgrade, beyond just removing the monkeypatch.

6

What if we avoided magic and had an explicit callable offered by TextChoices?

class Choices(enum.Enum, metaclass=ChoicesType):

    @classmethod
    def model_choices(cls):
        return cls.choices

    ...

Then our models could just do

class Customer(models.Model):
    class TaxCode(models.TextChoices):
        TAX = 'TAX', 'Tax'
        EXEMPT = 'EXEMPT', 'Exempt'

    taxcode = models.CharField(max_length=6, choices=TaxCode.model_choices)

Raised a django FR here: Improve Choices for migrations · Issue #51 · django/new-features · GitHub