One project (Django 3 with mod_wsgi installed within py 3.8 venv) works but another (Django 5 with mod_wsgi installed within py 3.12) fails

Issue: Internal Server Error with Django and mod_wsgi on Ubuntu Server

Setup:

  • Ubuntu 20.04
  • Python 3.8 for existing project
  • Python 3.12.4 for new project
  • Apache
  • mod_wsgi installed in respective virtual environments. The system level one has been removed.

Projects:

  1. Existing Project (Working :white_check_mark:)

    • Directory: /var/www/existing-project/
    • Python Version: 3.8
    • Django Version: 3.x
    • Virtual Environment Path: /var/www/existing-project/venv
    • WSGI File Path: /var/www/existing-project/project/project/wsgi.py
    • Apache Configuration:
      Protocols h2 http/1.1
      WSGIApplicationGroup %{GLOBAL}
      LoadModule wsgi_module "/var/www/existing-project/venv/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi.cpython-38-x86_64-linux-gnu.so"
      
      <VirtualHost *:80>
          ServerAdmin admin@example.com
          ServerName existing-project.com
          DocumentRoot /var/www/existing-project/project/
          RemoteIPHeader CF-Connecting-IP
      
          ErrorLog ${APACHE_LOG_DIR}/existing-project_error.log
          CustomLog ${APACHE_LOG_DIR}/existing-project_access.log combined
      
          <Directory /var/www/existing-project/project/project>
              <Files wsgi.py>
                  Require all granted
              </Files>
          </Directory>
      
          Alias /favicon.ico /var/www/existing-project/project/static/images/favicon/favicon.ico
          Alias /static /var/www/existing-project/project/static
      
          <Directory /var/www/existing-project/project/static>
              Require all granted
          </Directory>
      
          <IfModule mod_expires.c>
              <FilesMatch "\.(png|jp?g|gif|ico|mp4|wmv|mov|mpeg|css|map|woff?|eot|svg|ttf|js|json|pdf|csv)">
                  ExpiresActive on
                  ExpiresDefault "access plus 1 year"
              </FilesMatch>
          </IfModule>
      
          WSGIDaemonProcess existing-project python-home=/var/www/existing-project/venv python-path=/var/www/existing-project/project
          WSGIProcessGroup existing-project
          WSGIScriptAlias / /var/www/existing-project/project/project/wsgi.py
      
          RewriteEngine on
          RewriteCond %{SERVER_NAME} =existing-project.com
          RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
      </VirtualHost>
      
  2. New Project (Not working :x:)

    • Directory: /var/www/new-project/
    • Python Version: 3.12.4
    • Django Version: 5
    • Virtual Environment Path: /var/www/new-project/venv
    • WSGI File Path: /var/www/new-project/project/project/wsgi.py
    • Apache Configuration:
      WSGIApplicationGroup %{GLOBAL}
      LoadModule wsgi_module "/var/www/new-project/venv/lib/python3.12/site-packages/mod_wsgi/server/mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so"
      
      <VirtualHost *:80>
          ServerAdmin admin@example.com
          ServerName new-project.com
          DocumentRoot /var/www/new-project/project/
          RemoteIPHeader CF-Connecting-IP
      
          # Basic HTTP Authentication
          <Directory /var/www/new-project/project>
              AuthType Basic
              AuthName "Restricted Area"
              AuthUserFile /etc/apache2/.htpasswd
              Require valid-user
          </Directory>
      
          ErrorLog ${APACHE_LOG_DIR}/new-project_error.log
          CustomLog ${APACHE_LOG_DIR}/new-project_access.log combined
      
          <Directory /var/www/new-project/project/project>
              <Files wsgi.py>
                  Require all granted
              </Files>
          </Directory>
      
          Alias /favicon.ico /var/www/new-project/project/static/images/favicon/favicon.ico
          Alias /static /var/www/new-project/project/static
      
          <Directory /var/www/new-project/project/static>
              Require all granted
          </Directory>
      
          <IfModule mod_expires.c>
              <FilesMatch "\.(png|jp?g|gif|ico|mp4|wmv|mov|mpeg|css|map|woff?|eot|svg|ttf|js|json|pdf|csv)">
                  ExpiresActive on
                  ExpiresDefault "access plus 1 year"
              </FilesMatch>
          </IfModule>
      
          WSGIDaemonProcess new-project python-home=/var/www/new-project/venv python-path=/var/www/new-project/project
          WSGIProcessGroup new-project
          WSGIScriptAlias / /var/www/new-project/project/project/wsgi.py
      
          # RewriteEngine on
          # RewriteCond %{SERVER_NAME} =new-project.com
          # RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
      </VirtualHost>
      

Issue:
When accessing new-project.com, I receive an “Internal Server Error” with the following error logs:

[wsgi:error] [pid 1198771] mod_wsgi (pid=1198771): Exception occurred processing WSGI script '/var/www/new-project/project/project/wsgi.py'.
[wsgi:error] [pid 1198771] Traceback (most recent call last):
File "/var/www/new-project/project/project/wsgi.py", line 12, in <module>
    from django.core.wsgi import get_wsgi_application
ModuleNotFoundError: No module named 'django'

Troubleshooting Steps Taken:

  1. Confirmed Django installation:
    (venv) $ pip show django
    (venv) $ python -m django --version
             5.0.6
    
  2. Verified permissions:
    sudo chown -R www-data:www-data /var/www/new-project/
    sudo chmod -R 755 /var/www/new-project/
    
  3. Checked mod_wsgi installation:
    (venv) /usr/bin$ mod_wsgi-express module-config
    LoadModule wsgi_module "/var/www/new-project/venv/lib/python3.12/site-packages/mod_wsgi/server/mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so"
    WSGIPythonHome "/var/www/new-project/venv"
    ldd /var/www/new-project/venv/lib/python3.12/site-packages/mod_wsgi/server/mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so
    
  4. Verified Python version in WSGI script:
    import sys
    import os
    print("Python executable:", sys.executable)
    print("Python version:", sys.version)
    print("Python path:", sys.path)
    print("Current working directory:", os.getcwd())
    
    [wsgi:error] [pid 1201444] Python executable: /var/www/new-project/venv/bin/python
    [wsgi:error] [pid 1201444] Python version: 3.8.10 (default, Mar 25 2024, 10:42:49)
    [wsgi:error] [pid 1201444] [GCC 9.4.0]
    [wsgi:error] [pid 1201444] Python path: ['/var/www/new-project/project', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload']
    [wsgi:error] [pid 1201444] Current working directory: /
    

Question:

I’ve been stuck on this for three days and I am out of options to troubleshoot. How can I resolve the “ModuleNotFoundError”? What is happening here?

ModuleNotFoundError happens when a package can’t be imported.

Assuming the venv in the “Confirmed Django installation” is /var/www/new-project/venv, that should work. pip show django will show the version and the install location, which can help avoid other weird path issues.

If I create and activate a virtualenv locally, my sys.path contains the path to the virtual environment’s site-packages. You’re clearly using the right Python executable, but something about it isn’t picking up the rest of its modules.

As crazy as it sounds, have tried deleting the virtual environment, and recreating it from scratch?

rm -rf venv/
python3.8 -m venv venv
venv/bin/pip install django
venv/bin/pip show django

As crazy as it sounds, have tried deleting the virtual environment, and recreating it from scratch?

Creating venv with Python 3.8 will cause a different problem as Django 5 does not support Python 3.8. So I have to make it work with Python 3.12. :confused:

Django 4.2 still supports Python 3.8. If you can survive on 4.2 until you can upgrade to a newer version of Python (5.0 and 5.1 are 3.10+), then that might be a good stable path forward.

That’d be my last resort. I’d rather update my old projects to Django 5 and compile mod_wsgi system wide with python 3.12. But that will take time though.

And I feel so dumb not to figure out how to fix this. I’d love to see a resolution to this problem at hand.

user:/usr/bin$ ls -lrth python*
lrwxrwxrwx 1 root root   16 Mar 13  2020 python3-config -> python3.8-config
lrwxrwxrwx 1 root root    9 Mar 13  2020 python3 -> python3.8
-rwxr-xr-x 1 root root  152 Apr  9  2020 python3-pbr
-rwxr-xr-x 1 root root  388 Jan 25  2023 python3-pasteurize
-rwxr-xr-x 1 root root  384 Jan 25  2023 python3-futurize
lrwxrwxrwx 1 root root   33 Mar 25 10:42 python3.8-config -> x86_64-linux-gnu-python3.8-config
-rwxr-xr-x 1 root root 5.3M Mar 25 10:42 python3.8
lrwxrwxrwx 1 root root   34 Jun  8 18:29 python3.12-config -> x86_64-linux-gnu-python3.12-config
-rwxr-xr-x 1 root root 8.1M Jun  8 18:29 python3.12

I am still investigating this. Could this be due to my python3 being a symbolic link to python3.8? As shown by sys.version?