'ProgrammingError: column does not exist' in Docker

I’ve been moving development of my website over to using Docker. I replaced sqlite as my database with postgresql then ran the command docker-compose exec web python manage.py migrate in my Docker environment. I also updated MEDIA ROOT and MEDIA settings in my settings.py file and updated mysite/urls.py. When I go to 127.0.0.1:8000/admin to the look at stored/uploaded files I get an error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above exception (column human_requirementschat.alias does not exist
LINE 1: SELECT "human_requirementschat"."id", "human_requirementscha...
                                              ^
) was the direct cause of the following exception:
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 688, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 134, in _wrapper_view
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/decorators/cache.py", line 62, in _wrapper_view_func
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/sites.py", line 242, in inner
    return view(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 134, in _wrapper_view
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/options.py", line 2065, in changelist_view
    "selection_note": _("0 of %(cnt)s selected") % {"cnt": len(cl.result_list)},
                                                           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
    ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1560, in execute_sql
    cursor.execute(sql, params)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exception Type: ProgrammingError at /admin/human/requirementschat/
Exception Value: column human_requirementschat.alias does not exist
LINE 1: SELECT "human_requirementschat"."id", "human_requirementscha...
                                              ^

Here are the relevant files:

urls.py:

from django.conf import settings # new
from django.conf.urls.static import static # new
from django.contrib import admin
from django.urls import include, path
#from translator import views

urlpatterns = [
    path('', include('homepage.urls')),#redirects to transcribe/,
    path('transcribe/', include('transcribe.urls')),
    path('human/', include('human.urls')),
    path('admin/', admin.site.urls),
]+ static(
settings.MEDIA_URL, document_root=settings.MEDIA_ROOT
) # new

settings.py:

DATABASES = {
"default": env.dj_db_url("DATABASE_URL",
default="postgres://postgres@db/postgres")
}#new

MEDIA_URL='/media/' #new
MEDIA_ROOT = BASE_DIR/'media' #new

models.py:

class RequirementsChat(models.Model):
    id = models.CharField(primary_key=True, max_length=38)
    #id = models.UUIDField(default=uuid.uuid4, unique=True, primary_key=True, max_length=37)
    #id_temp = models.UUIDField(default=uuid.uuid4, unique=True)
    alias = models.CharField(max_length=20, blank=True, null=True)
    email = models.CharField(max_length=80, blank=True, null=True)
    language = models.CharField(max_length=10, blank=True, null=True)
    due_date = models.CharField(max_length=10, blank=True, null=True)
    subtitle_type = models.CharField(max_length=10, blank=True, null=True)
    transcript_file_type = models.CharField(max_length=10, blank=True, null=True)
    additional_requirements = models.TextField(max_length=500, blank=True, null=True)
    date = models.DateTimeField(auto_now_add=True, blank=True, null=True)
    url = models.CharField(max_length=250, blank=True, null=True)
    task_completed = models.BooleanField(default=False)
    
class UploadedFile(models.Model):
    input_file = models.FileField(upload_to='human_upload/')#new
    chat_id = models.CharField(max_length=33, null= True)
    requirements_chat = models.ForeignKey(RequirementsChat, on_delete=models.CASCADE, related_name='human_upload', null=True)

I deleted all my migration files and ran makemigrations, makemigrations human and migrate several times but the same error always occurs. Grateful for any help.

Are you running PostgreSQL in a container? If so, is the data being stored in a volume?

What version of PostgreSQL are you using?

What do the docker logs show after you run migrate?

I"m running my entire project in a Docker container. I’m still pretty new to this. How can I ensure postgre is running in a container?

Here is my docker-compose.yml file:

version: "3.9"
services:
  web:
    build: .
    command: python /code/manage.py runserver  0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      -  8000:8000
    depends_on:
      - db
    environment:
      -  "DJANGO_SECRET_KEY=)...my_secret_key..."
      -  "DJANGO_DEBUG=True"
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"
volumes:
  postgres_data:

Here are the docker logs

Operations to perform:
  Apply all migrations: accounts, admin, auth, contenttypes, human, sessions, transcribe
Running migrations:
  Applying human.0002_remove_requirementschat_alias...Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column "alias" of relation "human_requirementschat" does not exist


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/code/manage.py", line 22, in <module>
    main()
  File "/code/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/operations/fields.py", line 170, in database_forwards
    schema_editor.remove_field(
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 766, in remove_field
    self.execute(sql)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 48, in execute
    return super().execute(sql, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 201, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: column "alias" of relation "human_requirementschat" does not exist

Where are you running migrate in this process?

Also, it’s a really bad idea to be using runserver to run your production Django issue. Quoting directly from the docs for runserver:

DO NOT USE THIS SERVER IN A PRODUCTION SETTING.

(Emphasis added)

You should be using something like uwsgi or gunicorn to run your project. You also should be running this behind a real web server such as nginx or apache.

But more importantly, you shouldn’t be running the code directly. In most cases, you want your docker command to run migrate and collectstatic before starting the server. If you’re running that exec command after the container is up, you’re too late.

You may want to review the deployment docs as well, starting with How to deploy Django | Django documentation | Django. You’ll definitely want to work your way through the Deployment checklist | Django documentation | Django.

I’d also suggest you do a native deployment before trying to deploy in a docker environment. That’s just adding one more potential problem to try and diagnose if you’re not familiar with doing deployments, and if you’re not all that comfortable with docker, it just makes the situation worse.

I’m still in development actually. I ran docker-compose exec web python manage.py migrate in docker. I know that during production I shouldn’t use the development server. I’m not yet ready to deploy. I’ve also already ran my project natively before moving over to Docker