Deployment with Docker, Nginx, Gunicorn - No Permissions to Collect Static

josidridolfo · April 8, 2024, 11:40am

Unsure if this should be in Deployment or Mystery Errors, but given it’s an error and I have no idea how to resolve it, I’m putting it in ‘Mystery Errors’.

I’ve deployed my Django web app, and when my entrypoint.sh file attempts to collectstatic, it throws a PermissionError [Errno 13] Permission Denied: '/home/app/web/staticfiles/css/bootstrap.css.map'.

The error:

code-web-1    |   Applying sites.0002_alter_domain_unique...
code-web-1    |  OK  ## Here, all migrations have been applied. On to collect static....
code-web-1    | Traceback (most recent call last):
code-web-1    |   File "/home/app/web/manage.py", line 22, in <module>
code-web-1    |     main()
code-web-1    |   File "/home/app/web/manage.py", line 18, in main
code-web-1    |     execute_from_command_line(sys.argv)
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
code-web-1    |     utility.execute()
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
code-web-1    |     self.fetch_command(subcommand).run_from_argv(self.argv)
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
code-web-1    |     self.execute(*args, **cmd_options)
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
code-web-1    |     output = self.handle(*args, **options)
code-web-1    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 209, in handle
code-web-1    |     collected = self.collect()
code-web-1    |                 ^^^^^^^^^^^^^^
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 135, in collect
code-web-1    |     handler(path, prefixed_path, storage)
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 368, in copy_file
code-web-1    |     if not self.delete_file(path, prefixed_path, source_storage):
code-web-1    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 322, in delete_file
code-web-1    |     self.storage.delete(prefixed_path)
code-web-1    |   File "/usr/local/lib/python3.12/site-packages/django/core/files/storage/filesystem.py", line 158, in delete
code-web-1    |     os.remove(name)
code-web-1    | PermissionError: [Errno 13] Permission denied: '/home/app/web/staticfiles/css/bootstrap.css.map'
code-web-1    | [2024-04-08 06:30:09 +0000] [1] [INFO] Starting gunicorn 21.2.0

The Dockerfile:

###########
# BUILDER #
###########

# pull official base image
FROM python:3.12.2-bookworm as builder

# set work directory
WORKDIR /usr/src/app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1 \
    PYTHONUNBUFFERED 1

# install system dependencies
RUN apt-get update && \
    apt-get install -y \
    netcat-openbsd \
    graphviz \
    libgraphviz-dev \
    python3-dev \
    gcc \
    libc-dev && \
    pip install pipenv && \
    pip install --upgrade pip

COPY Pipfile Pipfile.lock ./
RUN pipenv install --system --deploy

COPY . .


#########
# FINAL #
#########

# pull official base image
FROM python:3.12.2-bookworm

# create directory for the app user, plus make app user and app group
RUN mkdir -p /home/app && \
    addgroup --system app && \
    adduser --system --group app

# create the appropriate directories
ENV HOME=/home/app \
    APP_HOME=/home/app/web

RUN mkdir -p $APP_HOME/staticfiles $APP_HOME/mediafiles && \
    apt-get update && apt-get install -y \
    netcat-openbsd \
    graphviz \
    libgraphviz-dev \
    python3-dev \
    gcc \
    libc-dev && \
    rm -rf /var/lib/apt/lists/* && \
    chown -R app:app $APP_HOME

RUN chown -R app:app $APP_HOME/staticfiles $APP_HOME/mediafiles

WORKDIR $APP_HOME

# Copy Pipfile and install dependencies
COPY --from=builder /usr/src/app/Pipfile /usr/src/app/Pipfile.lock ./
# Copy installed dependencies from builder
COPY --from=builder /usr/src/app $APP_HOME

RUN pip install pipenv && \
    pipenv install --system --deploy

# copy AND PREPARE entrypoint.prod.sh
COPY ./entrypoint.prod.sh .
RUN sed -i 's/\r$//g' $APP_HOME/entrypoint.prod.sh && \
    chmod +x  $APP_HOME/entrypoint.prod.sh

# 
RUN chown -R app:app $APP_HOME/staticfiles $APP_HOME/mediafiles
# change to the app user
USER app

# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.prod.sh"]

The docker-compose.yml:

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile.prod
    command: gunicorn project.wsgi:application --bind 0.0.0.0:8000
    volumes:
      - static_volume:/home/app/web/staticfiles
      - media_volume:/home/app/web/mediafiles
    expose:
      - 8000
    env_file:
      - .env-lolz-name-changed
    depends_on:
      - db
  db:
    image: postgres:16
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    env_file:
      - .env-lolz-name-changed-lollerskates
  nginx:
      build: ./nginx
      volumes:
        - static_volume:/home/app/web/staticfiles
        - media_volume:/home/app/web/mediafiles
      ports:
        - 1337:80 # For http
        - 443:443 # HTTPS
      depends_on:
        - web
volumes:
  postgres_data:
  static_volume:
  media_volume:

The entrypoint.sh

#!/bin/sh

APPS="app1 app2 app3"

SCRIPTS="script1 script2 script3"


if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $DB_HOST $DB_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi
# Flush the DB
python manage.py flush --no-input
# Make app-specific migrations
for app in $APPS;
do
  echo "Making migrations for $app"
  python manage.py makemigrations "$app"
  sleep 10
done
# Make universal migrations
python manage.py makemigrations
# Migrate apps
for app in $APPS;
do
  echo "Migrating $app"
  python manage.py migrate "$app"
sleep 10
done
# Migrate everything
  python manage.py migrate --no-input
sleep 10
# Collect static ########### THIS IS WHERE THE ERROR OCCURS
python manage.py collectstatic --no-input
# Populate db with scripts
for script in $SCRIPTS;
do
  echo "Executing $script"
  python manage.py "$script"
done

exec "$@"

For some reason, it seems like the Docker build does not execute the chown -R app:app... command, as shown when I enter docker exec -it code-web-1 ls -l /home/app/web/:

-rw-r--r--  1 root root   1170 Mar 29 02:35 Dockerfile
-rw-r--r--  1 root root    889 Apr  8 06:12 Pipfile
-rw-r--r--  1 root root 127830 Apr  8 06:12 Pipfile.lock
-rw-r--r--  1 root root      0 Mar 25 08:18 __init__.py
drwxr-xr-x  3 root root   4096 Mar 26 07:16 audits
drwxr-xr-x  3 root root   4096 Mar 25 08:51 data
-rw-r--r--  1 root root    358 Mar 25 08:24 docker-compose.yml
-rwxr-xr-x  1 root root   1210 Apr  8 07:08 entrypoint.sh
-rwxr-xr-x  1 root root    668 Mar 25 08:24 manage.py
drwxr-xr-x  2 root root   4096 Mar 25 08:51 media
drwxr-xr-x  2 app  app    4096 Apr  8 06:58 mediafiles
-rwxr-xr-x  1 root root   1259 Apr  8 06:12 reset_server.sh
drwxr-xr-x  2 root root   4096 Mar 25 08:23 scripts
drwxr-xr-x  2 root root   4096 Apr  8 06:12 project
drwxr-xr-x  5 root root   4096 Apr  8 06:12 static
drwxr-xr-x 14 root root   4096 Apr  8 07:01 staticfiles
drwxr-xr-x  4 root root   4096 Apr  8 06:12 app1
drwxr-xr-x 12 root root   4096 Apr  8 06:12 app2
drwxr-xr-x  4 root root   4096 Mar 30 03:52 app3

FYI my app names are not just ‘app1 app2 app3’ and the script names are also not ‘script1 script2 script3’; they have meaningful names. But the names here are not the issue; the issue is the permissions.

From the above, mediafiles is the only directory for which user/group app has permissions, which is unexpected. Shouldn’t the recursive chown in the Dockerfile pass permissions down to every file/folder in the target directory?

Any advice on how to remedy this issue? Has anyone else experienced something similar?

I’ve tried the following:

Dumping the containers with the below script, and then rebuilding:

#!/bin/sh

cd code
echo "Stopping the production server"
docker-compose stop
echo "Taking down the production server"
docker-compose down -v
echo "Removing all Docker containers, images, volumes, and networks"
docker system prune -a --volumes

Changing the entrypoint.sh file so that it executes chown on the target directory (it doesn’t have permission because it is being executed by user/group app).
Explicitly adding the target directories to the chown command (earlier it was just chown -R app:app $APP_HOME at the end of the RUN mkdir -p $APP_HOME/staticfiles $APP_HOME/mediafiles && \ in the Final section of the Dockerfile.

Topic		Replies	Views
Django and Nginx permission issue on Ubuntu Deployment	3	1649	January 10, 2024
collectstatic deploy problem Mystery Errors	15	3209	December 17, 2022
Troubleshooting a Django collectstatic Error After Upgrading to Version 4.2. Mystery Errors	9	2374	November 12, 2023
Django: how to fix collectstatic in django heroku? Mystery Errors	1	2752	August 9, 2022
Unauthorized access problem on django Using Django	6	2345	July 16, 2020

Deployment with Docker, Nginx, Gunicorn - No Permissions to Collect Static

Related topics