After upgrading Django/python, django is still trying to use python 2.7

My apologies in advance, I really only know enough to be dangerous, only do this part time, and am only allowed to upgrades maybe once a year, so…

I am finally upgrading from Django 1.11 and python 2.7. I was expecting to need some code tweaks. So far,when I ran “manage.py --version”, it told me I needed to fix some tabs and add on_delete to some models. Nifty - that should mean the upgrade was successful, right?

Nope. Restarted my app, with “sudo service myapp restart”,

 `[Unit]
 Description=uWSGI Server for myapp website
 After=network.target
 
 [Service]
 ExecStart=/usr/bin/uwsgi --plugins python -x /var/www/myapp/project/uwsgi.xml
 ExecReload=/usr/bin/uwsgi --plugins python -x /var/www/myapp/project/uwsgi.xml
 #User=www-data
 #Group=www-data
 #ExecReload=/bin/kill -HUP $MAINPID
 KillMode=process
 Restart=on-failure
 #RestartPreventExitStatus=255
 Type=notify
 #RuntimeDirectory=sshd
 #RuntimeDirectoryMode=0755
 
 [Install]
 WantedBy=multi-user.target

I got an internal server error. Syslog shows:

 *** Operational MODE: single process ***
 Dec 29 04:35:37 localhost uwsgi[12986]: added /var/www/etap/ to pythonpath.
 Dec 29 04:35:37 localhost uwsgi[12986]: added /usr/bin/python to pythonpath.
 Dec 29 04:35:37 localhost uwsgi[12986]: added /var/www/etap/project/apps/ to pythonpath.
 Dec 29 04:35:40 localhost uwsgi[12986]: Traceback (most recent call last):
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "./wsgi.py", line 7, in <module>
 Dec 29 04:35:40 localhost uwsgi[12986]:     application = get_wsgi_application()
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 13, in get_wsgi_application
 Dec 29 04:35:40 localhost uwsgi[12986]:     django.setup(set_prefix=False)
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
 Dec 29 04:35:40 localhost uwsgi[12986]:     apps.populate(settings.INSTALLED_APPS)
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
 Dec 29 04:35:40 localhost uwsgi[12986]:     app_config.import_models()
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/config.py", line 202, in import_models
 Dec 29 04:35:40 localhost uwsgi[12986]:     self.models_module = import_module(models_module_name)
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
 Dec 29 04:35:40 localhost uwsgi[12986]:     __import__(name)
 Dec 29 04:35:40 localhost uwsgi[12986]:   File "/var/www/etap/project/apps/imports/models.py", line 5, in <module>
 Dec 29 04:35:40 localhost uwsgi[12986]:     from bs4 import BeautifulSoup
 Dec 29 04:35:40 localhost uwsgi[12986]: ImportError: No module named bs4
 Dec 29 04:35:40 localhost uwsgi[12986]: unable to load app 0 (mountpoint='') (callable not found or import error)
 Dec 29 04:35:40 localhost uwsgi[12986]: *** no app loaded. going in full dynamic mode ***
 Dec 29 04:35:40 localhost uwsgi[12986]: dropping root privileges after application loading
 Dec 29 04:35:40 localhost uwsgi[12986]: uWSGI running as root, you can use --uid/--gid/--chroot options
 Dec 29 04:35:40 localhost uwsgi[12986]: *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
 Dec 29 04:35:40 localhost uwsgi[12986]: *** uWSGI is running in multiple interpreter mode ***
 Dec 29 04:35:40 localhost uwsgi[12986]: gracefully (RE)spawned uWSGI master process (pid: 12986)
 Dec 29 04:35:40 localhost uwsgi[12986]: sendmsg(): Transport endpoint is already connected [core/notify.c line 39]
 Dec 29 04:35:40 localhost uwsgi[12986]: spawned uWSGI worker 1 (pid: 13196, cores: 1)

So it’s looking for the wrong version of python. python --version give 3.6.9, and manage.py from the command line give:

Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/config.py", line 565, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/lib/python3.6/logging/config.py", line 738, in configure_handler
    result = factory(**kwargs)
  File "/usr/lib/python3.6/logging/handlers.py", line 150, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
  File "/usr/lib/python3.6/logging/handlers.py", line 57, in __init__
    logging.FileHandler.__init__(self, filename, mode, encoding, delay)
  File "/usr/lib/python3.6/logging/__init__.py", line 1032, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/lib/python3.6/logging/__init__.py", line 1061, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/var/www/django/log/mylog.log'

During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "./manage.py", line 30, in <module>
     execute_from_command_line(sys.argv)
   File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
     utility.execute()
   File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 395, in execute
     django.setup()
   File "/usr/local/lib/python3.6/dist-packages/django/__init__.py", line 19, in setup
     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
   File "/usr/local/lib/python3.6/dist-packages/django/utils/log.py", line 75, in configure_logging
     logging_config_func(logging_settings)
   File "/usr/lib/python3.6/logging/config.py", line 802, in dictConfig
     dictConfigClass(config).configure()
   File "/usr/lib/python3.6/logging/config.py", line 573, in configure
     '%r: %s' % (name, e))
 ValueError: Unable to configure handler 'default': [Errno 13] Permission denied: '/var/www/django/log/mylog.log'

So it’s finding python 3.6, and somewhere I’ve got something telling it to use Python 2.7, but I can’t for the life of me figure out where.

Some of the files I have checked:
wsgy.py:

 import sys
 import os
 from django.core.wsgi import get_wsgi_application
 
 sys.path.append('/var/www/myapp/project/')
 os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
 application = get_wsgi_application()

uwsgi.xml:

 <uwsgi>
   <socket>127.0.0.1:3032</socket>
   <master/>
   <processes>1</processes>
   <chdir>/var/www/etap/project</chdir>
   <pythonpath>/var/www/etap/</pythonpath>
   <pythonpath>/usr/bin/python</pythonpath>
 
   <pythonpath>/var/www/etap/project/apps/</pythonpath>
   <module>wsgi</module>
 </uwsgi>

uwsgi.conf: (Sort of think I may not need both, but they precede my involvement…)

 <uwsgi>
     <pythonpath>/usr/bin/python</pythonpath>
     <pythonpath>/var/www/etap/</pythonpath>
     <pythonpath>/var/www/etap/project/</pythonpath>
     <pythonpath>/var/www/etap/project/apps/</pythonpath>
     <app mountpoint="/">
         <script>project.wsgi</script>
     </app>
 </uwsgi>

uwsgi.ini

 module=myapp.uwsgi:application
 logto = /var/log/uwsgi/%n.log
 uid = uwsgi
 gid = uwsgi
 pidfile = /run/uwsgi/uwsgi.pid
 socket = :3032

myapp.service:
 [Unit]
 Description=uWSGI Server for  website
 After=network.target
 
 [Service]
 ExecStart=/usr/bin/uwsgi --plugins python -x /var/www/myapp/project/uwsgi.xml
 ExecReload=/usr/bin/uwsgi --plugins python -x /var/www/myapp/project/uwsgi.xml
 #User=www-data
 #Group=www-data
 #ExecReload=/bin/kill -HUP $MAINPID
 KillMode=process
 Restart=on-failure
 #RestartPreventExitStatus=255
 Type=notify
 #RuntimeDirectory=sshd
 #RuntimeDirectoryMode=0755
 
 [Install]
 WantedBy=multi-user.target

uwsgi_params for nginx:

 uwsgi_param  QUERY_STRING       $query_string;
 uwsgi_param  REQUEST_METHOD     $request_method;
 uwsgi_param  CONTENT_TYPE       $content_type;
 uwsgi_param  CONTENT_LENGTH     $content_length;
 
 uwsgi_param  REQUEST_URI        $request_uri;
 uwsgi_param  PATH_INFO          $document_uri;
 uwsgi_param  DOCUMENT_ROOT      $document_root;
 uwsgi_param  SERVER_PROTOCOL    $server_protocol;
 uwsgi_param  HTTPS              $https if_not_empty;
 
 uwsgi_param  REMOTE_ADDR        $remote_addr;
 uwsgi_param  REMOTE_PORT        $remote_port;
 uwsgi_param  SERVER_PORT        $server_port;
 uwsgi_param  SERVER_NAME        $server_name;

There is nothing in the uwsgi log, or my application log.

I am on a shared server, but not an explicit venv. Nothing I can see says “hey, ignore what the system thinks, use this version!”, so I’m just really stumped now.

Any tips/hints/suggestions or outright solutions are very much appreciated, otherwise, thanks for reading.

Please describe the deployment environment in more detail.

What distro / version are you using? Are you using a pre-packaged uwsgi or compiled from source?

Which executable is /usr/bin/python linked to? (You might be able to just change the references to /usr/bin/python to /usr/bin/python3.6 - but that’s only going to work if you have a Python3.6-compatible version of uwsgi installed.)

Also, when you’re posting code / configuration files, you can surround the block with lines of three backtick - ` characters to maintain proper formatting. You’ll have a line of ```, then the code, then another line of ```. It works better than the > notation, which is what I think you’re using here. (Notice how your xml doesn’t display properly.)
If you’re inclined to do so, you can edit your original post without needing to repost anything.

Thanks - I edited with the backticks, so I hope that helps it display properly now.

It is Ubuntu 18.04.6, with uwsgi installed from apt-get, version 2.0.20. /usr/bin/python is pointed to 3.6.9

me@dev:~$ python --version
Python 3.6.9
me@dev:~$ /usr/bin/python --version
Python 3.6.9
me@dev:~$  python -m django --version
3.2.10
me@dev:~$ nginx -v
nginx version: nginx/1.14.0 (Ubuntu)

This is something I almost forgot about, which may or may not be relevant.

For some reason, when the app was initially developed, django, beautiful soup, and another package were put into a directly /var/www/cortex - it’s something I found on my own, and my best guess was those packages were in a repository somewhere that I was never told about. I’d removed everything but BeautifulSoup - do not remember why that is still there. Out of curiosity today, I copied the bs4 package into that directory today and got:

 Traceback (most recent call last):
Dec 30 04:40:04 localhost uwsgi[7300]:   File "./wsgi.py", line 7, in <module>
Dec 30 04:40:04 localhost uwsgi[7300]:     application = get_wsgi_application()
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 13, in get_wsgi_application
Dec 30 04:40:04 localhost uwsgi[7300]:     django.setup(set_prefix=False)
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
Dec 30 04:40:04 localhost uwsgi[7300]:     apps.populate(settings.INSTALLED_APPS)
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
Dec 30 04:40:04 localhost uwsgi[7300]:     app_config.import_models()
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/config.py", line 202, in import_models
Dec 30 04:40:04 localhost uwsgi[7300]:     self.models_module = import_module(models_module_name)
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
Dec 30 04:40:04 localhost uwsgi[7300]:     __import__(name)
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/var/www/etap/project/apps/imports/models.py", line 5, in <module>
Dec 30 04:40:04 localhost uwsgi[7300]:     from bs4 import BeautifulSoup
Dec 30 04:40:04 localhost uwsgi[7300]:   File "/var/www/cortex/bs4/__init__.py", line 36, in <module>
Dec 30 04:40:04 localhost uwsgi[7300]:     raise ImportError('You are trying to use a Python 3-specific version of Beautiful Soup under Python 2. This will not work. The final version of Beautiful Soup to support Python 2 was 4.9.3.')
Dec 30 04:40:04 localhost uwsgi[7300]: ImportError: You are trying to use a Python 3-specific version of Beautiful Soup under Python 2. This will not work. The final version of Beautiful Soup to support Python 2 was 4.9.3.
Dec 30 04:40:04 localhost uwsgi[7300]: unable to load app 0 (mountpoint='') (callable not found or import error)
Dec 30 04:40:04 localhost uwsgi[7300]: *** no app loaded. going in full dynamic mode ***

So it definitely picked up the packages from that directory. I moved it and the old package out of that directory and got:

Traceback (most recent call last):
Dec 30 04:41:40 localhost uwsgi[7300]:   File "./wsgi.py", line 7, in <module>
Dec 30 04:41:40 localhost uwsgi[7300]:     application = get_wsgi_application()
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 13, in get_wsgi_application
Dec 30 04:41:40 localhost uwsgi[7300]:     django.setup(set_prefix=False)
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
Dec 30 04:41:40 localhost uwsgi[7300]:     apps.populate(settings.INSTALLED_APPS)
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
Dec 30 04:41:40 localhost uwsgi[7300]:     app_config.import_models()
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/usr/local/lib/python2.7/dist-packages/django/apps/config.py", line 202, in import_models
Dec 30 04:41:40 localhost uwsgi[7300]:     self.models_module = import_module(models_module_name)
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
Dec 30 04:41:40 localhost uwsgi[7300]:     __import__(name)
Dec 30 04:41:40 localhost uwsgi[7300]:   File "/var/www/etap/project/apps/imports/models.py", line 5, in <module>
Dec 30 04:41:40 localhost uwsgi[7300]:     from bs4 import BeautifulSoup
Dec 30 04:41:40 localhost uwsgi[7300]: ImportError: No module named bs4
Dec 30 04:41:40 localhost uwsgi[7300]: unable to load app 0 (mountpoint='') (callable not found or import error)
Dec 30 04:41:40 localhost uwsgi[7300]: *** no app loaded. going in full dynamic mode ***

So that’s not what’s telling it to look for 2.7. (And the cortex dir is now empty.)

I think I fixed it!

I was poking around to see if I had any more details to give you. I did a “whereis uwsgi” and noticed it was in both /usr/bin and /usr/local/bin. An ll the first pointed at /etc/alternatives, with an August (2020?) install date; the second listed Dec 2021 install date, so I updated the service file to point to /usr/local/bin.

It then updated the error to reflect 3.6, but still said it couldn’t find bs4. I looked at the details again, and noticed the /etc/alternatives had a listing for “uwsgi-python”… so I poked Google and then installed the uwsgi-plugin-python3.

Then it complained there was no haystack! Installed that, and I can now at least get to an error page on my website!! New error is haystack views related, so I will go see what I can figure out about that.

Thanks!!!

1 Like