Why command “python manage.py migrate” after success work lock console?

I try to solve following task. I have Django app and I want to start it using Docker. As ‘db’ I add a container with PostgreSQL or MySQL. In my case I need run “python manage.py migrate” before “python manage.py runserver”, so I wrote in docker-compose.yml file in fiels “command” folloving string:sh -c “sleep 20 && python manage.py migrate --noinput && python manage.py runserver 0.0.0.0:8000 --noreload”. But after applying all migrations command “python manage.py migrate” doesn’t stop and doesn’t give to run “python manage.py runserver”.

some lines....
web_1  |   Applying datapage.0001_initial... OK
web_1  |   Applying datapage.0002_auto_20190709_1955... OK
web_1  |   Applying datapage.0003_auto_20190713_1358... OK
web_1  |   Applying datapage.0004_auto_20190720_1107... OK
web_1  |   Applying sessions.0001_initial... OK

If I use:

docker-compose exec web python manage.py migrate

we have same result:

some lines....
web_1  |   Applying datapage.0001_initial... OK
web_1  |   Applying datapage.0002_auto_20190709_1955... OK
web_1  |   Applying datapage.0003_auto_20190713_1358... OK
web_1  |   Applying datapage.0004_auto_20190720_1107... OK
web_1  |   Applying sessions.0001_initial... OK

After pressing Ctrl + C, we get the following error:

^CException ignored in: <module 'threading' from '/usr/local/lib/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 1307, in _shutdown
    lock.acquire()
KeyboardInterrupt 

Settings of docker-compose.yml:

version: "3.7"

services:
  db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
  web:
    build: .
    command: >
      sh -c "sleep 20 &&
              python manage.py migrate --noinput &&
              python manage.py runserver 0.0.0.0:8000 --noreload"

    ports:
      - "8000:8000"
    depends_on:
      - db

In this case we don’t have to run “python manage.py makemigrations”, because it is first initialization of ‘db’, but if try to do this we will get:

No changes detected
(pressing Ctrl + C)
^CException ignored in: <module 'threading' from '/usr/local/lib/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 1307, in _shutdown
    lock.acquire()
KeyboardInterrupt

Such behavior I get in case working with PostgreSQL и MySQL (by Docker and local server), but with SQLite “python manage.py migrate” work correct and come back to console. Also I checked that problem is not in long work of command “python manage.py migrate”, I stayed it to works at night, but it didn’t help.

I tried to change versions in range python 3.5-3.9 и Django 2.0-2.2, but I got same result. Also I checked it in clear project that was created with “django-admin startproject”, eventually result same.

Google didn’t help me with finding answers for my case. But it is strange that I use same way for using “python manage.py migrate” in Docker as here and in similar sources, so I’m very confused, where I have made mistake.

So I want to know, why “python manage.py migrate” gives such behavior, might it is Django ORM bug? Maybe you met this misstake and know how solve it, in order to this command works from docker-compose.yml?

Also give link for project code on GitHub

You could add the -v 3 as a parameter to get the most detail from the logs to see where it’s getting hung up.

I don’t have a specific answer for you, but I can tell you that running migrate from within a docker container does work - we do something similar to what you’re doing for our docker builds.

We don’t put multiple commands on the command line in the docker-compose file, we run a bash script inside the container. Among a number of other things, our bash script contains the following:

  python manage.py migrate --no-input -v 0
  python manage.py collectstatic --no-input --clear --no-color -v 0
  exec /usr/local/bin/uwsgi /pmcs/uwsgi/uwsgi.ini

and this works great for us.

I have run python manage.py migrate --no-input -v 3 and have got following result:

web_1  | Operations to perform:
web_1  |   Apply all migrations: admin, auth, contenttypes, datapage, sessions
web_1  | Running pre-migrate handlers for application admin
web_1  | Running pre-migrate handlers for application auth
web_1  | Running pre-migrate handlers for application contenttypes
web_1  | Running pre-migrate handlers for application sessions
web_1  | Running pre-migrate handlers for application datapage
web_1  | Running pre-migrate handlers for application frontpage
web_1  | Running migrations:
web_1  |   Applying contenttypes.0001_initial... OK (0.034s)
web_1  |   Applying auth.0001_initial... OK (0.201s)
web_1  |   Applying admin.0001_initial... OK (0.042s)
web_1  |   Applying admin.0002_logentry_remove_auto_add... OK (0.026s)
web_1  |   Applying admin.0003_logentry_add_action_flag_choices... OK (0.017s)
web_1  |   Applying contenttypes.0002_remove_content_type_name... OK (0.040s)
web_1  |   Applying auth.0002_alter_permission_name_max_length... OK (0.012s)
web_1  |   Applying auth.0003_alter_user_email_max_length... OK (0.017s)
web_1  |   Applying auth.0004_alter_user_username_opts... OK (0.016s)
web_1  |   Applying auth.0005_alter_user_last_login_null... OK (0.018s)
web_1  |   Applying auth.0006_require_contenttypes_0002... OK (0.004s)
web_1  |   Applying auth.0007_alter_validators_add_error_messages... OK (0.016s)
web_1  |   Applying auth.0008_alter_user_username_max_length... OK (0.030s)
web_1  |   Applying auth.0009_alter_user_last_name_max_length... OK (0.018s)
web_1  |   Applying auth.0010_alter_group_name_max_length... OK (0.017s)
web_1  |   Applying auth.0011_update_proxy_permissions... OK (0.017s)
web_1  |   Applying datapage.0001_initial... OK (0.218s)
web_1  |   Applying datapage.0002_auto_20190709_1955... OK (0.158s)
web_1  |   Applying datapage.0003_auto_20190713_1358... OK (0.030s)
web_1  |   Applying datapage.0004_auto_20190720_1107... OK (0.056s)
web_1  |   Applying sessions.0001_initial... OK (0.053s)
web_1  | Running post-migrate handlers for application admin
web_1  | Adding content type 'admin | logentry'
web_1  | Adding permission 'Permission object (1)'
web_1  | Adding permission 'Permission object (2)'
web_1  | Adding permission 'Permission object (3)'
web_1  | Adding permission 'Permission object (4)'
web_1  | Running post-migrate handlers for application auth
web_1  | Adding content type 'auth | permission'
web_1  | Adding content type 'auth | group'
web_1  | Adding content type 'auth | user'
web_1  | Adding permission 'Permission object (5)'
web_1  | Adding permission 'Permission object (6)'
web_1  | Adding permission 'Permission object (7)'
web_1  | Adding permission 'Permission object (8)'
web_1  | Adding permission 'Permission object (9)'
web_1  | Adding permission 'Permission object (10)'
web_1  | Adding permission 'Permission object (11)'
web_1  | Adding permission 'Permission object (12)'
web_1  | Adding permission 'Permission object (13)'
web_1  | Adding permission 'Permission object (14)'
web_1  | Adding permission 'Permission object (15)'
web_1  | Adding permission 'Permission object (16)'
web_1  | Running post-migrate handlers for application contenttypes
web_1  | Adding content type 'contenttypes | contenttype'
web_1  | Adding permission 'Permission object (17)'
web_1  | Adding permission 'Permission object (18)'
web_1  | Adding permission 'Permission object (19)'
web_1  | Adding permission 'Permission object (20)'
web_1  | Running post-migrate handlers for application sessions
web_1  | Adding content type 'sessions | session'
web_1  | Adding permission 'Permission object (21)'
web_1  | Adding permission 'Permission object (22)'
web_1  | Adding permission 'Permission object (23)'
web_1  | Adding permission 'Permission object (24)'
web_1  | Running post-migrate handlers for application datapage
web_1  | Adding content type 'datapage | energyobjects'
web_1  | Adding content type 'datapage | log_strikes'
web_1  | Adding content type 'datapage | object_type'
web_1  | Adding content type 'datapage | region'
web_1  | Adding content type 'datapage | rup'
web_1  | Adding content type 'datapage | strikes'
web_1  | Adding content type 'datapage | links'
web_1  | Adding content type 'datapage | district'
web_1  | Adding content type 'datapage | coordinates_vl'
web_1  | Adding content type 'datapage | coordinates_ps'
web_1  | Adding content type 'datapage | branch'
web_1  | Adding permission 'Permission object (25)'
web_1  | Adding permission 'Permission object (26)'
web_1  | Adding permission 'Permission object (27)'
web_1  | Adding permission 'Permission object (28)'
web_1  | Adding permission 'Permission object (29)'
web_1  | Adding permission 'Permission object (30)'
web_1  | Adding permission 'Permission object (31)'
web_1  | Adding permission 'Permission object (32)'
web_1  | Adding permission 'Permission object (33)'
web_1  | Adding permission 'Permission object (34)'
web_1  | Adding permission 'Permission object (35)'
web_1  | Adding permission 'Permission object (36)'
web_1  | Adding permission 'Permission object (37)'
web_1  | Adding permission 'Permission object (38)'
web_1  | Adding permission 'Permission object (39)'
web_1  | Adding permission 'Permission object (40)'
web_1  | Adding permission 'Permission object (41)'
web_1  | Adding permission 'Permission object (42)'
web_1  | Adding permission 'Permission object (43)'
web_1  | Adding permission 'Permission object (44)'
web_1  | Adding permission 'Permission object (45)'
web_1  | Adding permission 'Permission object (46)'
web_1  | Adding permission 'Permission object (47)'
web_1  | Adding permission 'Permission object (48)'
web_1  | Adding permission 'Permission object (49)'
web_1  | Adding permission 'Permission object (50)'
web_1  | Adding permission 'Permission object (51)'
web_1  | Adding permission 'Permission object (52)'
web_1  | Adding permission 'Permission object (53)'
web_1  | Adding permission 'Permission object (54)'
web_1  | Adding permission 'Permission object (55)'
web_1  | Adding permission 'Permission object (56)'
web_1  | Adding permission 'Permission object (57)'
web_1  | Adding permission 'Permission object (58)'
web_1  | Adding permission 'Permission object (59)'
web_1  | Adding permission 'Permission object (60)'
web_1  | Adding permission 'Permission object (61)'
web_1  | Adding permission 'Permission object (62)'
web_1  | Adding permission 'Permission object (63)'
web_1  | Adding permission 'Permission object (64)'
web_1  | Adding permission 'Permission object (65)'
web_1  | Adding permission 'Permission object (66)'
web_1  | Adding permission 'Permission object (67)'
web_1  | Adding permission 'Permission object (68)'
web_1  | Running post-migrate handlers for application frontpage

If I remove frontpage application from settings.py I will get same result without last line. It doesn’t mater how and where I run python manage.py migrate, I get same result anywhere. Maybe do you have links to simple projects that you sure about them correct work so that I will be able to check them?

I don’t think I’ve ever seen behavior like this from a docker-compose. I guess the next idea that I would have is to do a docker-exec and run a shell in that container to investigate what the current status is. You could also check the database to ensure that all the migration steps have completed.

Sorry to bother you , because that mistake was caused by using threading inside project for endless background task, which wasn’t daemon, so created thread wasn’t giving app to stop. Since background task was simple, I didn’t want to use Celery, RabbitMQ, Redis. I have solved this problem by creating BaseCommand class for background task and run in inside manage.py by subprocess.Popen and after line execute_from_command_line(sys.argv) I kill subprocess.
Thank you so much for your advices!

If that’s the way you want to handle that situation, that’s your decision. But I would never want to try and do it that way - that just seems like it’s going to be ripe for all sorts of problems in a production environment when you have multiple instances of Django running. (But then you’re running Django under runserver, which really is a poor choice anyway.)

If you’re ever going to have more than a small handful of users, you really want to work on getting your deployment environment right. (On the other hand, if you’re creating a site like most of the ones that I create, only having 2 or 3 users, you may be ok for the most part.)

I absolutely agree with you. In my case it is just training project, so I can use this way for creating background task. In serious projects I’m sure I should use better ways than that.

1 Like