So i have been struggling for the past week to deploy this small project on my VPS server(domains.co.za). After all this time i have managed to figure a few things out on my own, working on AlmaLinux 9. I connected to the IP locally. Then started to configure my Gunicorn and Nginx. I did gunicorn service and socket file and the nginx. Also i did not buy a domain and want to connect via IP. For some reason i created the hostname as mydjangoserver.co.za dont know if that causes problems. Anyway everything is active and running but i get a 502 bad gateway when i try to acces it via IP. Please help ! Here are my configurations:
Gunicorn:
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=marco
Group=www-data
WorkingDirectory=/home/marco/domestic_voicelogging
ExecStart=/home/marco/domestic_voicelogging/venv/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
domestic_voicelogging.wsgi:application
[Install]
WantedBy=multi-user.target
Socket:
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
SocketUser=marco
SocketGroup=www-data
SocketMode=0660
[Install]
WantedBy=sockets.target
Nginx:
server {
listen 80;
server_name 41.76.110.165;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/marco/domestic_voicelogging;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
I get this error:
2024/07/09 10:11:40 [crit] 21042#21042: *8 connect() to unix:/opt/domestic_voicelogging.sock failed (2: No such file or directory) while connecting to upstream, client: 41.13.200.12, server: mydjangoserver.co.za, request: "GET / HTTP/1.1", upstream: "http://unix:/opt/domestic_voicelogging.sock:/", host: "41.76.110.165"
here is my run sock:
(venv) [marco@mydjangoserver domestic_voicelogging]$ ls -l /run/gunicorn.sock
srw-rw-rw-. 1 marco www-data 0 Jul 9 09:37 /run/gunicorn.sock
(venv) [marco@mydjangoserver domestic_voicelogging]$
see why is it like this [marco@mydjangoserver] and nit like this? = [marco@41.76.110.165]
Side Note: When posting code here, enclose the code between lines of three backtick - ` characters. This means you’ll have a line of ```, then your code, then another line of ```. This forces the forum software to keep your code properly formatted. (I’ve taken the liberty of fixing your original post for this.)
To answer your last question first:
That’s your shell prompt. Depending upon which distro of Linux you are using, and what shell, you’ll find this setting in something like your .bashrc or .profile file, or possibly the system defaults in /etc such as /etc/bash.bashrc or /etc/profile. It all depends upon how your system and user are configured.
You’ve shown the nginx error, check your syslog and gunicorn logs (and whatever logs you have configured in your project) for error messages. (It may be easiest just to restart the gunicorn service and then look at syslog, rather than trying to find the messages in the existing log.)
Also, are you sure you’re referring to the right IP addresses here?
From your error message:
... while connecting to upstream, client: 41.13.200.12, server: ...
Where’s that address coming from?
Firstly thanks for the fast reply!
Yes, sorry about the format will correct that from now on.
This is the IP address of the client that made the HTTP request to the server. In this case, it’s trying to access this i believe http://41.76.110.165/. Checked now this (41.13.200.12) is my public IP.
I really think the problem is at Nginx config since the nginx error log does not match up with the nginx config file info at all. Can be wrong im not sure.
Here are the error logs after restart.
Gunicorn logs:
(venv) [marco@mydjangoserver domestic_voicelogging]$ sudo journalctl -u gunicorn -n 25
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 13:21:49 +0200] [24109] [INFO] Starting gunicorn 22.0.0
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 13:21:49 +0200] [24109] [INFO] Listening at: unix:/run/gunicorn.sock (24109)
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 13:21:49 +0200] [24109] [INFO] Using worker: sync
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24110]: [2024-07-09 13:21:49 +0200] [24110] [INFO] Booting worker with pid: 24110
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24111]: [2024-07-09 13:21:49 +0200] [24111] [INFO] Booting worker with pid: 24111
Jul 09 13:21:49 mydjangoserver.co.za gunicorn[24112]: [2024-07-09 13:21:49 +0200] [24112] [INFO] Booting worker with pid: 24112
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 14:12:57 +0200] [24109] [INFO] Handling signal: term
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24110]: [2024-07-09 12:12:57 +0000] [24110] [INFO] Worker exiting (pid: 24110)
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24112]: [2024-07-09 12:12:57 +0000] [24112] [INFO] Worker exiting (pid: 24112)
Jul 09 14:12:57 mydjangoserver.co.za systemd[1]: Stopping gunicorn daemon...
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24111]: [2024-07-09 12:12:57 +0000] [24111] [INFO] Worker exiting (pid: 24111)
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 14:12:57 +0200] [24109] [ERROR] Worker (pid:24110) was sent SIGTERM!
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 14:12:57 +0200] [24109] [ERROR] Worker (pid:24111) was sent SIGTERM!
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 14:12:57 +0200] [24109] [ERROR] Worker (pid:24112) was sent SIGTERM!
Jul 09 14:12:57 mydjangoserver.co.za gunicorn[24109]: [2024-07-09 14:12:57 +0200] [24109] [INFO] Shutting down: Master
Jul 09 14:12:57 mydjangoserver.co.za systemd[1]: gunicorn.service: Deactivated successfully.
Jul 09 14:12:57 mydjangoserver.co.za systemd[1]: Stopped gunicorn daemon.
Jul 09 14:12:57 mydjangoserver.co.za systemd[1]: gunicorn.service: Consumed 1.445s CPU time.
Jul 09 14:12:58 mydjangoserver.co.za systemd[1]: Started gunicorn daemon.
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24611]: [2024-07-09 14:12:58 +0200] [24611] [INFO] Starting gunicorn 22.0.0
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24611]: [2024-07-09 14:12:58 +0200] [24611] [INFO] Listening at: unix:/run/gunicorn.sock (24611)
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24611]: [2024-07-09 14:12:58 +0200] [24611] [INFO] Using worker: sync
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24612]: [2024-07-09 14:12:58 +0200] [24612] [INFO] Booting worker with pid: 24612
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24613]: [2024-07-09 14:12:58 +0200] [24613] [INFO] Booting worker with pid: 24613
Jul 09 14:12:58 mydjangoserver.co.za gunicorn[24614]: [2024-07-09 14:12:58 +0200] [24614] [INFO] Booting worker with pid: 24614
(venv) [marco@mydjangoserver domestic_voicelogging]$
System Logs:
(venv) [marco@mydjangoserver domestic_voicelogging]$ sudo journalctl -n 40
Jul 09 14:16:34 mydjangoserver.co.za unix_chkpwd[24656]: password check failed for user (root)
Jul 09 14:16:36 mydjangoserver.co.za sshd[24652]: Failed password for root from 218.92.0.27 port 27669 ssh2
Jul 09 14:16:36 mydjangoserver.co.za sshd[24652]: Received disconnect from 218.92.0.27 port 27669:11: [preauth]
Jul 09 14:16:36 mydjangoserver.co.za sshd[24652]: Disconnected from authenticating user root 218.92.0.27 port 27669 [preauth]
Jul 09 14:16:36 mydjangoserver.co.za sshd[24652]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.27 user=root
Jul 09 14:16:38 mydjangoserver.co.za unix_chkpwd[24659]: password check failed for user (root)
Jul 09 14:16:38 mydjangoserver.co.za sshd[24657]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.27 user=root
Jul 09 14:16:40 mydjangoserver.co.za sshd[24657]: Failed password for root from 218.92.0.27 port 53410 ssh2
Jul 09 14:16:42 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:16:3e:03:a9:a2:48:a9:8a:85:6d:9a:08:00 SRC=124.223.22.248 DST=41.76.110.165 LEN=60 TOS=0x00 PREC=0x00 TTL=50 ID>
Jul 09 14:16:42 mydjangoserver.co.za unix_chkpwd[24660]: password check failed for user (root)
Jul 09 14:16:44 mydjangoserver.co.za sshd[24657]: Failed password for root from 218.92.0.27 port 53410 ssh2
Jul 09 14:16:46 mydjangoserver.co.za unix_chkpwd[24662]: password check failed for user (root)
Jul 09 14:16:48 mydjangoserver.co.za sshd[24657]: Failed password for root from 218.92.0.27 port 53410 ssh2
Jul 09 14:16:51 mydjangoserver.co.za sshd[24657]: Received disconnect from 218.92.0.27 port 53410:11: [preauth]
Jul 09 14:16:51 mydjangoserver.co.za sshd[24657]: Disconnected from authenticating user root 218.92.0.27 port 53410 [preauth]
Jul 09 14:16:51 mydjangoserver.co.za sshd[24657]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.27 user=root
Jul 09 14:16:52 mydjangoserver.co.za systemd[24534]: Created slice User Background Tasks Slice.
Jul 09 14:16:52 mydjangoserver.co.za systemd[24534]: Starting Cleanup of User's Temporary Files and Directories...
Jul 09 14:16:52 mydjangoserver.co.za systemd[24534]: Finished Cleanup of User's Temporary Files and Directories.
Jul 09 14:16:52 mydjangoserver.co.za unix_chkpwd[24666]: password check failed for user (root)
Jul 09 14:16:52 mydjangoserver.co.za sshd[24663]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.27 user=root
Jul 09 14:16:54 mydjangoserver.co.za sshd[24663]: Failed password for root from 218.92.0.27 port 42004 ssh2
Jul 09 14:16:54 mydjangoserver.co.za unix_chkpwd[24667]: password check failed for user (root)
Jul 09 14:16:57 mydjangoserver.co.za sshd[24663]: Failed password for root from 218.92.0.27 port 42004 ssh2
Jul 09 14:16:59 mydjangoserver.co.za unix_chkpwd[24668]: password check failed for user (root)
Jul 09 14:17:00 mydjangoserver.co.za sshd[24663]: Failed password for root from 218.92.0.27 port 42004 ssh2
Jul 09 14:17:01 mydjangoserver.co.za sshd[24663]: Received disconnect from 218.92.0.27 port 42004:11: [preauth]
Jul 09 14:17:01 mydjangoserver.co.za sshd[24663]: Disconnected from authenticating user root 218.92.0.27 port 42004 [preauth]
Jul 09 14:17:01 mydjangoserver.co.za sshd[24663]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.27 user=root
Jul 09 14:17:01 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:16:3e:03:a9:a2:48:a9:8a:85:6d:9a:08:00 SRC=80.94.95.209 DST=41.76.110.165 LEN=60 TOS=0x00 PREC=0x00 TTL=49 ID=5>
Jul 09 14:17:19 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:00:16:3e:38:1a:f6:86:dd SRC=fe80:0000:0000:0000:0216:3eff:fe38:1af6 DST=ff02:0000:0000:0000:0000>
Jul 09 14:17:19 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth1 OUT= MAC=33:33:00:00:00:01:00:16:3e:62:49:ee:86:dd SRC=fe80:0000:0000:0000:0216:3eff:fe62:49ee DST=ff02:0000:0000:0000:0000>
Jul 09 14:17:22 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:16:3e:03:a9:a2:48:a9:8a:85:6d:9a:08:00 SRC=41.207.248.204 DST=41.76.110.165 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID>
Jul 09 14:17:38 mydjangoserver.co.za sudo[24670]: marco : TTY=pts/0 ; PWD=/home/marco/domestic_voicelogging ; USER=root ; COMMAND=/bin/journalctl -n 25
Jul 09 14:17:38 mydjangoserver.co.za sudo[24670]: pam_unix(sudo:session): session opened for user root(uid=0) by marco(uid=1001)
Jul 09 14:17:42 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:16:3e:03:a9:a2:48:a9:8a:85:6d:9a:08:00 SRC=124.223.22.248 DST=41.76.110.165 LEN=60 TOS=0x00 PREC=0x00 TTL=50 ID>
Jul 09 14:18:00 mydjangoserver.co.za sudo[24670]: pam_unix(sudo:session): session closed for user root
Jul 09 14:18:02 mydjangoserver.co.za kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:16:3e:03:a9:a2:48:a9:8a:85:6d:9a:08:00 SRC=80.94.95.242 DST=41.76.110.165 LEN=60 TOS=0x00 PREC=0x00 TTL=49 ID=1>
Jul 09 14:18:07 mydjangoserver.co.za sudo[24675]: marco : TTY=pts/0 ; PWD=/home/marco/domestic_voicelogging ; USER=root ; COMMAND=/bin/journalctl -n 40
Jul 09 14:18:07 mydjangoserver.co.za sudo[24675]: pam_unix(sudo:session): session opened for user root(uid=0) by marco(uid=1001)
lines 1-40/40 (END)
Hope we can figure this out.
For what it’s worth, I think you’re on the right track here.
Please show full directory listings of /etc/nginx, /etc/nginx/sites-available and /etc/nginx/sites-enabled. We’ll probably need to dig further after that.
Also, the snippet of nginx configuration that you showed in your original post - what is the filename with those contents, and what directory is it in? Is that the complete file, or is there more in it than that?
Here is the directory listing:
(venv) [marco@mydjangoserver nginx]$ ls -l
total 88
drwxr-xr-x. 2 root root 4096 Jul 2 15:01 conf.d
drwxr-xr-x. 2 root root 4096 Oct 16 2023 default.d
-rw-r--r--. 1 root root 1077 Oct 16 2023 fastcgi.conf
-rw-r--r--. 1 root root 1077 Oct 16 2023 fastcgi.conf.default
-rw-r--r--. 1 root root 1007 Oct 16 2023 fastcgi_params
-rw-r--r--. 1 root root 1007 Oct 16 2023 fastcgi_params.default
-rw-r--r--. 1 root root 2837 Oct 16 2023 koi-utf
-rw-r--r--. 1 root root 2223 Oct 16 2023 koi-win
-rw-r--r--. 1 root root 5231 Oct 16 2023 mime.types
-rw-r--r--. 1 root root 5231 Oct 16 2023 mime.types.default
-rw-r--r--. 1 root root 2372 Jul 5 09:38 nginx.conf
-rw-r--r--. 1 root root 2656 Oct 16 2023 nginx.conf.default
-rw-r--r--. 1 root root 175 Jul 2 15:07 proxy_params
-rw-r--r--. 1 root root 636 Oct 16 2023 scgi_params
-rw-r--r--. 1 root root 636 Oct 16 2023 scgi_params.default
drwxr-xr-x. 2 root root 4096 Jul 2 14:53 sites-available
drwxr-xr-x. 2 root root 4096 Jul 3 09:07 sites-enabled
-rw-r--r--. 1 root root 664 Oct 16 2023 uwsgi_params
-rw-r--r--. 1 root root 664 Oct 16 2023 uwsgi_params.default
-rw-r--r--. 1 root root 3610 Oct 16 2023 win-utf
(venv) [marco@mydjangoserver nginx]$ cd sites-enabled/
(venv) [marco@mydjangoserver sites-enabled]$ ls -l
total 0
lrwxrwxrwx. 1 root root 48 Jul 3 09:07 domestic_voicelogging -> /etc/nginx/sites-available/domestic_voicelogging
(venv) [marco@mydjangoserver sites-enabled]$ cd ..
(venv) [marco@mydjangoserver nginx]$ cd sites-available/
(venv) [marco@mydjangoserver sites-available]$ ls -l
total 4
-rw-r--r--. 1 root root 313 Jul 9 10:07 domestic_voicelogging
The original snippet of nginx configuration path:
This is what i use to edit that file = sudo nano /etc/nginx/sites-available/domestic_voicelogging
Something already seems off about the sites-enabled and sites-available. Why in the sites-enabled dir does it use the date 3 July? I have been working on this from last week. Is it using an old config file and not the new one, do i need to enable something first or recreate the symbolic link. Let me know what you think!
Because it shows the date the symlink was created / modified, not the timestamp of the underlying file. As long as you can do something like cat /etc/nginx/sites-enabled/domestic_voicelogging and see the expected contents, it’s ok.
What have you edited / changed in /etc/nginx/nginx.conf? I see a July 5 date on it. Also, your proxy_params file has been edited.
Yes i am able to!
Regarding the nginx.conf file i added this line:
server_names_hash_bucket_size 64;
Here is the file:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
server_names_hash_bucket_size 64;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
Regarding the proxy_params i don’t remember adding anything to the file but here is the contents:
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
Working under the assumption that you are (or have) restarted the nginx process after creating or updating either of these files, please post the output of the nginx -T command. It’ll produce a complete output of all the active settings. It can help identify if something has been missed or unintentionally altered.
Note: The output is going to be about 400 lines logs. You might want to capture it to a file before copying the text and posting it here.
I think we found the problem. That command showed me this nginx confg file:
# configuration file /etc/nginx/conf.d/mydjangoserver.conf:
server {
listen 80;
server_name mydjangoserver.co.za;
location / {
include proxy_params;
proxy_pass http://unix:/opt/domestic_voicelogging.sock;
}
location /static/ {
alias /opt/domestic_voicelogging/static/;
}
location /media/ {
alias /opt/domestic_voicelogging/media/;
}
}
This is the one we saw in the error log. I changed it to match my other nginx server config:
server {
listen 80;
server_name 41.76.110.165;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/marco/domestic_voicelogging;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
Now when i access my server IP i see this:
So this means it works now right?!
Now just to upload my project through git. Sure that wont be so hard?
Where was that file? I didn’t see it in any of the directory listings.
Yea, that works - but is, or would be, confusing to other people to have it there. That file belongs in /etc/nginx/sites-available and linked by /etc/nginx/sites-enabled.
The more standard solution would have been to keep your configuration where it was, and delete this file.
Would you recommend to delete that file or just leave it as it is now?
I would - but it’s really more a question of what you are comfortable with. (It is your system, after all.) There’s nothing fundamentally “wrong” with having it there, it’s just not the typical location. (It’s not a technical decision, it’s an administrative decision.)
Keep in mind that all these directories and files are configurable items. There’s nothing special about the names themselves. The default configuration file name is nginx.conf, but even that can be changed.
If you look at your nginx.conf file you’ll see:
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
so even in the context on how these files are used, they end up being included in your configuration in the same overall location.
(You could also change these if you wanted to - give them different names or locations.)
I will definitely consider it but for today I’m going to take the win.
I understand and now that i understand better i think i am able to handle it on my own for now. THANK YOU SO MUCH FOR EVERYTHING KEN !!!
1 Like
