Makemigrations bug or not?

Django Internals ORM
1

Hey guys!

I found some interesting bug today.

I created two models: Document model and Language model.

This is a Language model:

class Language(models.Model):
    """
    Document language model
    """

    code = models.CharField(blank=False, max_length=2)
    label = models.CharField(blank=False, max_length=40)

And this is a Document model:

class Document(AbstractModel):
    """
    Document model
    """

    title = models.CharField(max_length=300, blank=False)
    author = models.CharField(max_length=300, blank=False)
    ...

    created_by = models.ForeignKey(User, related_name='documents',
                                   on_delete=models.PROTECT)
    languages = models.ManyToManyField(Language, blank=True)

And then I ran makemigrations. This is migration file’s content:

class Migration(migrations.Migration):

    initial = True

    dependencies = [
        ...
    ]

    operations = [
        ...
        migrations.CreateModel(
            name='Document',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('title', models.CharField(max_length=300)),
                ('author', models.CharField(max_length=300)),
                ...
                ('created_by', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='documents', to=settings.AUTH_USER_MODEL)),
            ],
            options={
                'verbose_name': 'Document',
                'verbose_name_plural': 'Documents',
            },
        ),
        migrations.CreateModel(
            name='Language',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('code', models.CharField(max_length=2)),
                ('label', models.CharField(max_length=40)),
            ],
            options={
                'verbose_name': 'Language',
                'verbose_name_plural': 'Languages',
            },
        ),
        ...
        migrations.AddField(
            model_name='document',
            name='languages',
            field=models.ManyToManyField(blank=True, to='documents.Language'),
        ),
    ]

After that I decided to make the code field a primary key in a Language model:

class Language(models.Model):
    """
    Document language model
    """

    code = models.CharField(blank=False, max_length=2, primary_key=True)
    label = models.CharField(blank=False, max_length=40)

And I ran makemigrations again. This is second migration file’s content:

class Migration(migrations.Migration):

    dependencies = [
        ...
    ]

    operations = [
        migrations.RemoveField(
            model_name='language',
            name='id',
        ),
        migrations.AlterField(
            model_name='language',
            name='code',
            field=models.CharField(max_length=2, primary_key=True, serialize=False),
        ),
    ]

Now we have code field as primary key for Language model. And it’s type is a char. But let’s look at the model document_document_language (It created automatically):

document_document_languages:
    id              integer
    document_id     integer
    language_id     integer

You can see that the relationship between models is maintained through an identifier that is no longer in the Language model (language_id is integer).

As result we have SQL error when we trying to create m2m relationship between this models:

django.db.utils.ProgrammingError: operator does not exist: character varying = integer
LINE 1: ...cument_languages" ON ("documents_language"."code" = "documen...

If we remove the last two migration files and rerun makemigrations that problem will be solved:

document_document_languages:
    id              integer
    document_id     integer
    language_id     char

So help me figure it out, please. What is it?

It’s a really bug in Django or it’s my wrong action’s result?

If it’s the result of my wrong actions, tell me how to avoid similar problems in the future?

Sometimes you have to change the model after a long time, and then it is impossible to delete all these migrations.

I’m using Django 3.1, thanks.

1 Like
2

Changing the primary key of a table under any circumstances is potentially problematic.
(I’m extremely conservative when it comes to modifying primary keys.)

The most appropriate way that I can think of to manage something like this through the Django ORM would be a multi-step process:

  1. Add the unique clause to code - don’t make it the primary key yet.
  2. Add another foreign key field to Document, using the to_field clause to create the index to the code column
  3. Add a manual migration step that creates new entries mapping the existing values in the many-to-many table
  4. Then you should be able to remove the original fields and keys, and then rename your fields as desired.

(I’m extremely conservative when it comes to modifying primary keys.)

You’re not going to want to try something like this in one large step - it’s a whole lot easier when you take multiple steps that are all individually reversable.

(I’m extremely conservative when it comes to modifying primary keys.)

Or, another / potentially simpler option would be to just create a new Language table with the proper structure and create a quick script to move data and references from the old to the new.

Having said that, when I’ve had to perform major surgery on a table structure, I either perform the steps manually and “rebase” my migrations on the new structure, or I create a complete self-written migration to do everything needing to be done. (Much more often the former than the latter, but that’s mostly because that type of surgery occurs before the site has been deployed anywhere and so there’s usually only one or two people affected by that change.)

Ken

1 Like
3

Thank you for detailed explanation! Now I’ll be really more careful with modifying primary keys!