makemigrations nightmare

Using Django Mystery Errors
1

Django works well, except when you try to do migrations. Then you are in hell. I just added 2 fields to a model and bingo. Since I had already bad experience with migrations I already started afresh and removed all previous migrations. No advance. So the final error on which I’m completely stuck because Django gives no clue what is wrong in which of my files. So apparently I’m not experienced enough to understand what is happening and where. From my previous experience I already expected problems with a model of which the fk’s are replaced by something else, so I wisely decided to rename that model. So that shouldn’t cause any trouble. But… the error I get hints to something wrong with a fk. So here is the error:

Traceback (most recent call last):
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 22, in <module>
    main()
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 457, in execute
    self.check(**check_kwargs)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 492, in check
    all_issues = checks.run_checks(
                 ^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/checks/registry.py", line 89, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 52, in check_admin_app
    errors.extend(site.check(app_configs))
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/sites.py", line 90, in check
    errors.extend(modeladmin.check())
                  ^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 151, in check
    return self.checks_class().check(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 809, in check
    *self._check_inlines(admin_obj),
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 846, in _check_inlines
    return list(
           ^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 848, in <genexpr>
    self._check_inlines_item(obj, item, "inlines[%d]" % index)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 889, in _check_inlines_item
    return inline(obj.model, obj.admin_site).check()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 151, in check
    return self.checks_class().check(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 1245, in check
    *self._check_relation(inline_obj, parent_model),
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/contrib/admin/checks.py", line 1286, in _check_relation
    _get_foreign_key(parent_model, obj.model, fk_name=obj.fk_name)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/forms/models.py", line 1245, in _get_foreign_key
    fks_to_parent = [
                    ^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/forms/models.py", line 1253, in <listcomp>
    f.remote_field.model._meta.proxy
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute '_meta'

I reviewed all my fk’s but I don’t see anything wrong.

I appreciate some light in this darkness.

2

Migrations work fine.

You’ll need to share the code for your models if you want someone to help you find where the error is in your models.

3

I’ll add to the previous response to say that doing this on a project accessing a database where some migrations have already been applied is a particularly bad idea.

If you are getting this error in a situation where you are connecting to a database where you have previously run migrations, you’re likely going to have a number of problems getting things back in sync. One thing to try would be to create a new database, set your project up to reference that database, and try the makemigrations again.

4

Sorry to disagree. But I found a missing “ in a fk statement which I must have deleted by accident and without realising it.

But this brings me back where I started this morning where django looses it. When I perform a makemigrations django cannot find the fields I’m trying to add. Of courrse not.

Ken suggests to recreate the database but I’m not ready (yet) to go there. It is a flaw in the makemigrations as far as I’m concerned. I’m new in Django but not with databases.

5

Ok. I created the fields in postgres and now Django is happy. I still have some issues in admin, but that is easily fixed.

after that the makemigrations should run, but that ‘s for tomorrow

6

Nothing wrong with makemigrations?

 warnings.warn(
Was customer.courier renamed to customer.courier_id (a ForeignKey)? [y/N] y^M^?^[[3~^[[3~^[[3~^?^?^?^CTraceback (most recent call last):
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 22, in <module>
    main()
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/commands/makemigrations.py", line 236, in handle
    changes = autodetector.changes(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 67, in changes
    changes = self._detect_changes(convert_apps, graph)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 200, in _detect_changes
    self.create_renamed_fields()
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 1044, in create_renamed_fields
    if self.questioner.ask_rename(
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/questioner.py", line 226, in ask_rename
    return self._boolean_input(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/questioner.py", line 101, in _boolean_input
    result = input()
             ^^^^^^^
KeyboardInterrupt

nothing wrong with my keyboard. Pressing enter after Y resulted in gibberish. Not once but every time. Had to change my field back to the original name to get past this.

7

That looks like a likely python environmental issue. (A munged installation / conflicting packages / incomplete upgrade / corrupted files in __pycache__ / ???)

The input function used by makemigrations is a standard python built-in function. If it’s not working, you have more fundamental problems with your Python installation.

8

Creating columns manually in the database will only cause more problems. You have to use the tools as they are designed.

I understand that you are frustrated, but blaming the tool when you are using it wrong is just a way to keep being frustrated and confused, it’s not a way out of confusion.

There are indeed many ways to use migrations incorrectly (deleting migrations files, editing the db outside migrations are the biggest two) that, when you do it, migrations will become very unhappy. Sure it could be more fool proof, but it’s not that migrations is broken.

Since I had already bad experience with migrations I already started afresh and removed all previous migrations.

This was the wrong approach. If you delete migrations, you should also recreate the database. If migrations isn’t picking up models? That’s probably because you deleted the migrations folder (instead of all the migrations in that folder).

When getting into trouble, it’s best to stop and figure things out directly, not doing other things in anger and then becoming more angry when those things lead to more problems, and then blaming the tools.

9

This is the continuing story of makemigrations.

I removed all migrations and started all over again, because it worked last time I struggled with makemigrations. This time however it didn’t work. So I put all migrations back but got a InconsistentMigrationsHistory. This shouldn’t be an error but a warning. Let me decide whether it is a problem or not. So I removed the instruction that raised the error in the loader.py. Then makemigrations finally gave a positive result. But then migrate started to make trouble.

It asked whether I was sure to change the name of a FK. I said Y and then this happend:

 warnings.warn(
Was customer.courier renamed to customer.courier_id (a ForeignKey)? [y/N] y^M^?^[[3~^[[3~^[[3~^?^?^?^CTraceback (most recent call last):
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 22, in <module>
    main()
  File "/mnt/nvme_ssd/Aussie_projects/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/core/management/commands/makemigrations.py", line 236, in handle
    changes = autodetector.changes(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 67, in changes
    changes = self._detect_changes(convert_apps, graph)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 200, in _detect_changes
    self.create_renamed_fields()
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/autodetector.py", line 1044, in create_renamed_fields
    if self.questioner.ask_rename(
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/questioner.py", line 226, in ask_rename
    return self._boolean_input(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/eddys/.local/lib/python3.11/site-packages/django/db/migrations/questioner.py", line 101, in _boolean_input
    result = input()
             ^^^^^^^
KeyboardInterrupt

I had to reverse the renaming and then migrate made all the changes. It gave me an error on a class I’m not even using and has been there since I started doing tutorials. I guess it has something to to with the inconsistent migration history.

So I’m good again, until the next makemigrations

10

It is a problem regardless of whether you think it is, because at this point, the migrate command will be unable to function correctly.

The key to having the migration subsystem work is to let it do its job. Don’t try to interfere with it.

This means:

  • Do not manually edit system-generated migration files, unless you really know what you are doing.
  • Do not manually edit migrations after they have been applied.
  • Do not delete migration files that have been applied.
  • Do not delete unapplied migration files without verifying that they’re not a dependency of a subsequent migration.
  • Do not manually change the database schema for managed entities.

Whenever you think that performing any of the above actions is the solution to an issue, don’t do it.

Make the decision that either you will manage the entities within a schema that Django can manage or that Django will - don’t try to mix the two.

If you absolutely follow this, migrations can generally be assumed to “do the right thing” under all but the most unusual circumstances. (There are many people who do rely upon it - I’m just one of them.)

If you don’t, any sort of strange and unusual situations can - and will - occur, and recovering from them can be quite painful.

(Once you understand how the migration subsystem works, you can relax these rules a bit, because you will have the knowledge of how to work with it instead of trying to work around it. But it’s still usually the easiest case to do the right thing and stick with it.)

11

I totally agree, but explain why after 12 successful migrations makemigrations refues to add 2 fields to a table.

12

Given the amount of “manual intervention” you have performed on your system, I’m not even going to try to hazard a guess. (I’ve actually explained it above - when you try to circumvent migration’s normal path of operations, virtually anything can go wrong.)

If you are able to recreate the error in a clean environment (new database, new virtual environment, consistent use of makemigration / migrate) then it could be worth investigating.

13

I started programming in early 80-ies. I once had a user that said my code was wrong. Checked it, and found nothing wrong with my code, but since he insisted I went to see him. My code gave a wrong result, but my code was right. So I printed the assembler code and found a compiler error.

What I’m saying is that the problem with makemigrations started BEFORE I did any manual intervention.

But I agree, a bug is a bug when it can be reproduced.

Maybe next time.

14

As someone who starting working in the late 70s in a systems-shop with a mixed IBM (MVS) / Honeywell (GCOS) environment, I’m well-aware of what needed to be done at times for errors that would regularly occur throughout the environment.

One of the hardest lessons for me to unlearn when moving to more modern environments was the tendency to want to intervene below the level at which errors would occur. I struggled a lot in the late 90s early 00 working with web frameworks that didn’t quite work the way that I either wanted them to, or expected them to. I realized that a lot of the thought patterns and techniques that I used to use were no longer valid.

It’s what lead me into the direction of the philosophy that I now phrase as “Don’t fight the framework”.

I no longer try to make frameworks fit my mental models - I work to adjust my mental model to the intent designed by the developers. Since then, things have been a whole lot smoother for me.

5 Likes
15

It took a while but with a lot of patience and putting breakpoints here and there I was able to restart django makemigrations in a clean state.

What I learned is that Django is using my current models.py and one it saved with the makemigrations somewhere to make a delta, but doesnot look at the actual tables in the database.

I expected that a delta would be made from my models and the tables in the database.

16

As I understand it, there’s no need for makemigrations to look at the database. It only needs to compare the current state of your models with the sum result of all previous migrations. Looking at the state of the database wouldn’t tell it anything that it can’t glean from the previous migrations.

17

Personally I’m not interested in what changes I have made in the past. I understand that people who are not familiar with databases it is a clean way of doing things but for me it is not flexible enough. I normally work on the tables in the database and reflect the changes afterwards in the model But that doesn’t work in Django. So I have to change my mode of operandi. I’ll do it but I’m not happy with it.

Verzonden vanaf Outlook voor Android

18

Yes, it does work. You just can’t mix this method of working with migrations.

You can identify individual models with managed=False, which tells Django to ignore that model. So you can build and manage those models manually, as you are used to doing. Other models that you aren’t managing manually can be managed by Django. This all works fine.

(We do a lot of this as we regularly integrate Django with other systems where those other systems are responsible for managing their tables.)

19

I tried that but it didn’t work. I guess one needs to to set managed to false at the creation of the model when no previous makemigration ran on that model.
But it is very well possible I did something wrong.

Verzonden vanaf Outlook voor Android

20

Yea, that’s how it works. I re-read Migrations | Django documentation | Django and I see that it’s mostly explained indirectly/between the lines, so if one comes into it assuming that it will read the database structure, I can see that one would not be disabused of that incorrect assumption. Maybe the docs could be improved with a note like:

NOTE: The migrations system does not inspect your current database structure, it only looks at the model definitions when producing migrations

I think that might avoid this misunderstanding