Django Websockets deployment issue

theyali · September 27, 2023, 5:23am

Hello, I’ve been struggling with an error for several days when deploying django application with websockets.(using Django==4.2.5, channels == 3.0.5, daphne == 3.0.2)
Server os :Ubuntu 22.04.3 LTS (GNU/Linux 5.2.0 x86_64)
Server: nginx
Also configured on the server: Redis, Gunicorn, Daphne

All these services are working fine and starting successfully but I am getting an error while connecting to websocket
(index):346 WebSocket connection to ‘wss://happy-moment.live:8001/search_game/’ failed:

Here is my services status and journalctl

ngix

root@851fcb08b962:~# sudo systemctl status  nginx
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-27 07:04:17 MSK; 48min ago
       Docs: man:nginx(8)
    Process: 76329 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 76330 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 76331 (nginx)
     CGroup: /system.slice/nginx.service
             ├─76331 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
             ├─76332 "nginx: worker process" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""
             └─76333 "nginx: worker process" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""

Sep 27 07:04:17 851fcb08b962.vps.myjino.ru systemd[1]: Starting A high performance web server and a reverse proxy server...
Sep 27 07:04:17 851fcb08b962.vps.myjino.ru systemd[1]: Started A high performance web server and a reverse proxy server.

redis.service

root@851fcb08b962:~# sudo systemctl status redis.service
● redis-server.service - Advanced key-value store
     Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-27 03:37:01 MSK; 4h 17min ago
       Docs: http://redis.io/documentation,
             man:redis-server(1)
   Main PID: 233 (redis-server)
     Status: "Ready to accept connections"
     CGroup: /system.slice/redis-server.service
             └─233 "/usr/bin/redis-server 127.0.0.1:6379" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""
 "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""

Sep 26 08:18:33 851fcb08b962.vps.myjino.ru systemd[1]: Stopping Advanced key-value store...
Sep 26 08:18:33 851fcb08b962.vps.myjino.ru systemd[1]: redis-server.service: Deactivated successfully.
Sep 26 08:18:33 851fcb08b962.vps.myjino.ru systemd[1]: Stopped Advanced key-value store.
Sep 27 03:36:55 851fcb08b962.vps.myjino.ru systemd[1]: Stopping Advanced key-value store...
Sep 27 03:36:55 851fcb08b962.vps.myjino.ru systemd[1]: redis-server.service: Deactivated successfully.
Sep 27 03:36:55 851fcb08b962.vps.myjino.ru systemd[1]: Stopped Advanced key-value store.
Sep 27 03:37:01 851fcb08b962.vps.myjino.ru systemd[1]: Starting Advanced key-value store...
Sep 27 03:37:01 851fcb08b962.vps.myjino.ru systemd[1]: Started Advanced key-value store.
Notice: journal has been rotated since unit was started, output may be incomplete.

daphne service

root@851fcb08b962:~# sudo systemctl status daphne.service
● daphne.service - WebSocket Daphne Service
     Loaded: loaded (/etc/systemd/system/daphne.service; disabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-27 06:59:54 MSK; 55min ago
   Main PID: 74669 (python)
     CGroup: /system.slice/daphne.service
             └─74669 /home/ftpuser/venv/bin/python /home/ftpuser/venv/bin/daphne -e ssl:8001:privateKey=/etc/ssl/private.key:certKey
=/etc/ssl/certificate.crt main.asgi:application

Sep 27 06:59:54 851fcb08b962.vps.myjino.ru systemd[1]: Started WebSocket Daphne Service.
Sep 27 06:59:54 851fcb08b962.vps.myjino.ru python[74669]: 2023-09-27 03:59:54,710 INFO     Starting server at ssl:8001:privateKey=
/etc/ssl/private.key:certKey=/etc/ssl/certificate.crt
Sep 27 06:59:54 851fcb08b962.vps.myjino.ru python[74669]: 2023-09-27 03:59:54,710 INFO     HTTP/2 support enabled
Sep 27 06:59:54 851fcb08b962.vps.myjino.ru python[74669]: 2023-09-27 03:59:54,710 INFO     Configuring endpoint ssl:8001:privateKe
y=/etc/ssl/private.key:certKey=/etc/ssl/certificate.crt
Sep 27 06:59:54 851fcb08b962.vps.myjino.ru python[74669]: 2023-09-27 03:59:54,715 INFO     Listening on TCP address 0.0.0.0:8001

gunicorn

root@851fcb08b962:~# sudo systemctl status gunicorn.service
● gunicorn.service - gunicorn daemon
     Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-27 04:41:05 MSK; 3h 14min ago
TriggeredBy: ● gunicorn.socket
   Main PID: 23606 (gunicorn)
     CGroup: /system.slice/gunicorn.service
             ├─23606 /home/ftpuser/venv/bin/python3 /home/ftpuser/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/
gunicorn.sock main.wsgi:application
             ├─23607 /home/ftpuser/venv/bin/python3 /home/ftpuser/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/
gunicorn.sock main.wsgi:application
             ├─23608 /home/ftpuser/venv/bin/python3 /home/ftpuser/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/
gunicorn.sock main.wsgi:application
             └─23609 /home/ftpuser/venv/bin/python3 /home/ftpuser/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/
gunicorn.sock main.wsgi:application

Here is my services config files

gunicorn.socket

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

gunicorn.service

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=ftpuser
Group=www-data
WorkingDirectory=/home/ftpuser/main
ExecStart=/home/ftpuser/venv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          main.wsgi:application

[Install]
WantedBy=multi-user.target

Daphne service

[Unit]
Description=WebSocket Daphne Service
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/home/ftpuser/main
ExecStart=/home/ftpuser/venv/bin/python /home/ftpuser/venv/bin/daphne -e ssl:8001:privateKey=/etc/ssl/private.key:certKey=/etc/ssl/certificate.crt main.asgi:application

Restart=on-failure

[Install]
WantedBy=multi-user.target

Note:Also i tried without specifying privateKey and certKey

Nginx conf

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
 	client_max_body_size 10M; 

}

My website conf file

server {

    server_name happy-moment.live;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/ftpuser/main;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }

    location /ws/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_redirect off;
        proxy_pass http://127.0.0.1:8001;
    }
}

Note:Also tried redirect all http request to the port 443 using this lines of code:

server {
    if ($host = happy-moment.live) {
        return 301 https://$host$request_uri;
    } 


    server_name happy-moment.live;
    listen 80;
    return 404; 


}

But in this case i get ERR_TOO_MANY_REDIRECTS
that’s why i duplicated the code in my conf file for each request(for 80 and 443)

Here is my settings.py

ALLOWED_HOSTS = ['*']
# CORS_ALLOWED_ORIGINS = ['*']
CSRF_TRUSTED_ORIGINS =[
    'https://happy-moment.live',
]
ASGI_APPLICATION = 'chat.routing.application'
WSGI_APPLICATION = 'main.wsgi.application'

SECURE_CROSS_ORIGIN_OPENER_POLICY='same-origin-allow-popups'
CHANNEL_LAYERS = {
   # 'default': {
    #    'BACKEND': 'channels.layers.InMemoryChannelLayer',
   # },
      "default": {
         "BACKEND": "channels_redis.core.RedisChannelLayer",
         "CONFIG": {
             "hosts": [("127.0.0.1", 6379)],
         },
     },
}

main/asgi.py

import os
import django
from channels.routing import get_default_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'main.settings')
django.setup()
application = get_default_application()

chat/routing.py

from channels.auth import AuthMiddlewareStack
from django.core.asgi import get_asgi_application
from channels.routing import ProtocolTypeRouter, URLRouter
from django.urls import path
from channels.security.websocket import AllowedHostsOriginValidator
from django.urls import re_path
from . import consumers

application = ProtocolTypeRouter({
    'http': get_asgi_application(),
    'websocket':AllowedHostsOriginValidator(
      AuthMiddlewareStack(
            URLRouter([
            re_path(r'/chat/(?P<pk>\d+)/$', consumers.GameConsumer.as_asgi()),
            path('search_game/', consumers.SearchGameConsumer.as_asgi()),
            ])
      )
    )
})

I tried a lot of things to solve this problem, I also installed soft tshark to analyze traffic and saw that even on the server I was not receiving a request for port 8001
I think main problem with ssl sertificate but I no longer understand how to determine this and how to fix it because the site works correctly with both https and http
Did anyone have this problem
Can anyone help please?

KenWhitesell · September 28, 2023, 2:05am

First, I would check to ensure that port 8001 is open through any system firewalls that may exist.

Then, check the browser to see that it is trying to open the socket on port 8001. You should at least see the connection attempt from the browser side.

As a side note, I perform all ssl termination in nginx. What gets proxied through to Daphne is not ssl encrypted.

My nginx location directive for one of my Channels-based projects looks like this:

    location /ws/mpge/ {
        proxy_pass http://unix:/run/daphne.sock:/ws/mpge/;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }

My Daphne command makes no references to ssl protocol-related settings. The command looks like this:

python3 bin/daphne -u /run/daphne.sock mpge_channels.asgi:application

theyali · September 28, 2023, 2:17pm

Hello, I apologize that I forgot to mention the point about ufw ports
ufw status
Screenshot (469)

Also in Dev tools I noticed that my browser sends request on deployment without any headers

Also add to my config file to the ws location logs,
but even after several attempts I realized that my logs are empty and this suggests that either ngix does not send ws requests to daphne or simply does not receive these requests
Screenshot (471)

KenWhitesell · September 28, 2023, 2:45pm

What does your JavaScript code look like for establishing the connection?

theyali · September 28, 2023, 2:49pm

Here is the js code

  var ws_scheme = window.location.protocol == "https:" ? "wss" : "ws";
  var ws_path = ws_scheme + '://' + window.location.host + ":8001/search_game/";

  var searchGameSocket = new WebSocket(ws_path);
  
  searchGameSocket.onmessage = function(e) 
  {
      let data = JSON.parse(e.data);
      console.log(data)
      if(data.game_id){
        window.location.href = data.url + data.game_id;
    } else {
        window.location.href = data.url;
    }
  };

theyali · September 28, 2023, 7:39pm

I found the problem
The problem was connecting to the websocket

instead of this code

 var ws_scheme = window.location.protocol == "https:" ? "wss" : "ws";
  var ws_path = ws_scheme + '://' + window.location.host + ":8001/search_game/";

i wrote this

  var ws_scheme = window.location.protocol == "https:" ? "wss" : "ws";
  var ws_path = ws_scheme + '://' + window.location.host + "/ws/search_game/";

Also changed my routing.py
the fact is that nginx redirects requests containing ws and sent to port 80 to Daphne

Topic		Replies	Views
App with websockets deployment error Async/Channels	7	855	December 12, 2023
Websocket connection failed(WSREJECT) in production. (Nginx, Gunicorn, Daphne) Async/Channels	73	958	January 11, 2024
Websocket + Django on production (https) Async/Channels	23	9931	April 14, 2023
Daphne: websocket works locally with ws protocol, but it connects and quickly disconnects in production Async/Channels	26	646	February 17, 2024
Websocket connection failed in production. (Nginx, Gunicorn, Daphne) (Django/React) Async/Channels	15	6282	March 15, 2024

Django Websockets deployment issue

Related Topics