Dear guys,
I am struggling with deployment of my web app that contains real time chat as a feature.
The web app works fine on localhost.
When it comes to deployment to production + https, the websocket client (browser) couldn’t connect to our consumer. We have set the configuration of ngnix and location /ws/ + proxy things + port 5000…
The problem is not with our daphne, becasue we have tested as well with another asgi server uvicorn but same error/warning is showing in server log:
WARNING NOT FOUND, GET /ws/path_consumer/ not found 404
(same message with uvicorn)
Does it require to pass ssl certificate as well to ngnix ? The web app is hosted on EC2 and all ports and load balancer is set it up with SSL. but nginx is listening to 80.
We don’t from where is the problem is from our consumer.py ? is it from our settings.py ? The paths are fine because they are working fine on localhost.
We need to see the specifics here:
- Your JavaScript code that is trying to open the websocket.
- Your nginx configuration that is proxying the connection
- Your asgi.py file
- The Daphne command you’re using to run Daphne
Hi @KenWhitesell:
Thanks for your quick reply. Sure, here you can find:
- JavaScript client:
var ws_scheme = window.location.protocol === "https:" ? "wss" : "ws";
const ws_port = '';
var roomName = JSON.parse(document.getElementById('room-name').textContent);
const chatRoomName = JSON.parse(document.getElementById('chat_notif_room').textContent);
const chatsSocket = new WebSocket(
ws_scheme
+ '://'
+ window.location.host
+ ws_port
+ '/ws/chats/'
+ chatRoomName
+ '/'
);
- Nginx configuration:
#Elastic Beanstalk Nginx Configuration File
user nginx;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 32780;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
include conf.d/*.conf;
map $http_upgrade $connection_upgrade {
default "upgrade";
}
server {
listen 80 default_server;
access_log /var/log/nginx/access.log main;
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
gzip off;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/*.conf;
}
}
- Webapp.conf file:
#File conf.d/elasticbeanstalk/webapp.conf
location /ws/ {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
- asgi.py file
import os
import django
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application
from channels.security.websocket import AllowedHostsOriginValidator
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
django.setup()
from channels.auth import AuthMiddleware, AuthMiddlewareStack
from NotificationsApp.routing import websocket_urlpatterns
application = ProtocolTypeRouter(
{
"http": get_asgi_application(),
"websocket": AllowedHostsOriginValidator(
AuthMiddlewareStack(URLRouter(websocket_urlpatterns))
),
}
)
# NotificationsApp.routing file:
from ChatApp import consumers as chat_consumers
from . import consumers
websocket_urlpatterns = [
re_path(
r"ws/notification/(?P<room_name>\w+)/$",
consumers.NotificationConsumer.as_asgi(),
),
re_path(r"ws/chats/(?P<room_name>\w+)/$", chat_consumers.ChatConsumer.as_asgi()),
]
- Daphne command:
daphne -b :: -p 5000 myapp.asgi:application
PS: The host of web app is on AWS EC2 via AWS EB where I configured my load balance to add my AWS ACM ssl certificate to accept https.
First, a disclaimer - I’ve deployed fewer than a half-dozen projects that use websockets. I know what works for us - but that’s not to say that I know every possible way to deploy this in a production environment.
Having said that, I’ve never tried to use django.setup in my asgi.py file. I’ve always used the method as shown in part 2 of the channels tutorial.
e.g.
django_asgi_app = get_asgi_application()
...
"http": django_asgi_app,
I believe (with absolutely no firm knowledge as to why I believe this), that there’s an issue regarding the scope of the objects being created that it works better this way.
So yes, my first recommendation is to change your asgi.py file to more closely resemble the structure of what the tutorial shows.
The other main difference between what you have and what I use is that in the nginx configuration, we also have the following setting in the /ws/ location:
proxy_set_header X-Forwarded-Host $server_name
(Again, I don’t remember off-hand when/why we added that, but it has just become part of our standard configuration.)
Side note: (Not a specific recommendation) We also still deploy the “Django” app under uwsgi, and only serve the websockets connections under Daphne. In other words, only the /ws/ url goes to Daphne, everything else (standard Django) gets proxied to uwsgi. I mention it as a difference in deployment, but not that I consider it to be a relevent factor here.
Yes exactly for asgi.py file, django.setup() was just a try I did to check but indeed I followed exactly that tutorial since I started developing my django web app.
So I removed that line and put back as it is in tutorial + I updated the nginx conf to proxy_set_header X-Forwarded-Host $server_name But unfortually still does not work 
and daphne could not find path for my websocket consumer class…
What does your ChatConsumer look like?
Your Daphne log shows that nginx is definitely routing the request through to Daphne, otherwise it wouldn’t be Daphne returning a 404.
What does your root urls.py look like? I’m wondering if there’s a conflict with the name resolution between your Django app and the websocket urls.
Yes indeed the nginx is working fine because it routes the requests to Daphne but it doesnt find it:
here is my root url
from django.conf import settings
from django.conf.urls import url
from django.conf.urls.static import static
from django.contrib import admin
from django.urls import include, path
from django.views.static import serve
urlpatterns = [
path("admin/", admin.site.urls),
path("chat/", include("ChatApp.urls")),
path("notif/", include("NotificationsApp.urls")),
path("api/auth/oauth/", include("rest_framework_social_oauth2.urls")),
url(r"^media/(?P<path>.*)$", serve, {"document_root": settings.MEDIA_ROOT}),
url(r"^static/(?P<path>.*)$", serve, {"document_root": settings.STATIC_ROOT}),
]
My chat rooting.py file :
from django.urls import re_path
from . import consumers
chat_websocket_urlpatterns = [
re_path(r"ws/chats/(?P<room_name>\w+)/$", consumers.ChatConsumer.as_asgi()),
]
This is my consumer:
import json
from asgiref.sync import async_to_sync, sync_to_async
from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from channels.layers import get_channel_layer
from django.contrib.auth.models import AnonymousUser
from AccountApp.models import User
from .models import Message
class ChatConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.room_name = self.scope["url_route"]["kwargs"]["room_name"]
self.room_group_name = "chats_%s" % self.room_name
# Join room group
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
async def disconnect(self, close_code):
# Leave room group
await self.channel_layer.group_discard(self.room_group_name, self.channel_name)
# Receive message from web socket
async def receive(self, text_data):
data = json.loads(text_data)
chat_content = data["message"]
username = data["username"]
userid = data["userid"]
chat_room = data["room"]
profile_pic = data["profile_pic"]
sent_date = data["sent_date"]
chat_notif_room = data["chat_notif_room"]
type_socket = data["type_socket"]
# get chat_notif_room
user = await self.get_user(username)
if type_socket == "chat":
await self.save_message(user, chat_room, chat_content)
# Send message to room group
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "chat_message",
"message": chat_content,
"username": username,
"userid": userid,
"profile_pic": profile_pic,
"sent_date": sent_date,
"chat_notif_room": chat_notif_room,
"room": chat_room,
"type_socket": type_socket,
},
)
@database_sync_to_async
def get_user(self, username):
try:
return User.objects.get(username=username)
except:
return AnonymousUser()
# Receive message from room group
async def chat_message(self, event):
message = event["message"]
username = event["username"]
userid = event["userid"]
profile_pic = event["profile_pic"]
sent_date = event["sent_date"]
chat_notif_room = event["chat_notif_room"]
chat_room = event["room"]
type_socket = event["type_socket"]
# Send message to WebSocket
await self.send(
text_data=json.dumps(
{
"message": message,
"username": username,
"userid": userid,
"profile_pic": profile_pic,
"sent_date": sent_date,
"chat_notif_room": chat_notif_room,
"room": chat_room,
"type_socket": type_socket
# 'chat_room':chat_room
}
)
)
@sync_to_async
def save_message(self, user, chat_room, chat_content):
print("========== save_message =======")
Message.objects.create(
user=user, chat_room=chat_room, chat_content=chat_content
)
Do we need to pass ssl context/certificate or something to Daphne to be able to find the paths ?
Hi mtn, i saw that you answer my previous topic related to a similar problem.
I found the solution for this related problem, all is explained there : ssl - Is there a way to enable secure websockets on Django? - Stack Overflow
I hope this will help you.
All the best, Mallory.
No, since you’re using nginx as a proxy, and you have defined:
Your application doesn’t even see the ssl-layer communication. As far as your Django app is concerned, ssl isn’t involved. (You’re forwarding the requests through an http connection, not https.)
No, what’s bugging me about this is that you’re seeing GET requests on that URL. In my system, Daphne doesn’t see a GET for the WS connection, only the WSCONNECT.
Do you have anything else that might be trying to do a GET on that URL before opening up the websocket?
Note: The other difference I see between what you have and what I have - and what I see in the nginx docs for websockets is that I have:
proxy_set_header Connection "Upgrade";
I see where you’re setting those values in your configuration, but it’s not clear to me what the scope is of that definition relative to your location directive. I’d try setting the constants to see what happens.
Yes @KenWhitesell indeed you are right: I also checked that in my localhost and I dont see GET /ws/…
This is what I got which I guess aligned with you have
Well I don’t have because I use the same code on localhost and it works fine, the only GETs is about redirecting the views like this one:
HTTP GET /home/ 200 [0.10, 127.0.0.1:50989]
HTTP GET /chat/ 200 [0.20, 127.0.0.1:50989]
I have also tested that before but it doesnt work.
So I guess there is a problem of routing or something normally we should see WebSocket requests not GET requests
Side note: I’ll point out here that running runserver successfully alone isn’t necessarily a good parallel to getting things working in a production environment - as you’re beginning to see. Adding nginx to the mix changes things on a couple of fundamental levels.
Are you saying that you are seeing these GET requests in your nginx logs? Or are these from your runserver logs?
Have you checked the nginx logs to see what requests have been logged?
What does your nginx configuration look like for your non-channels requests?
Right now, the thought running through my head is that your nginx configuration might be matching the “ws” url with the location for your non-ws handler, and passing it along as a GET instead of a CONNECT.
Exactly, I only use the runserver command on my local machine for development but not on production server. For production, I am making my Procfile file that is normally responsible for running the code on gunicorn server for django app and daphne server for my websocket when I deploy my new version of through aws eb deploy command.
Here is my Procfile:
web: gunicorn --bind :8000 --workers 3 --threads 2 trsap.wsgi:application
websocket: daphne -b :: -p 5000 trsap.asgi:application
To answer your question regarding the GET requests, yes I am seeing them from my runserver logs (localhost)
The GET requests I can see them in my nginx logs are a bit similar. Here is an example:
IP.IP.IP.IP - - [12/Mar/2023:15:44:20 +0000] "GET /home/ HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.5481.177 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "66.249.73.188"
Yeah makes sense what you have said, I am going to investigate that deeply in the mean time, this what I found my nginx error logs about websocket:
----------------------------------------
/var/log/nginx/error.log
----------------------------------------
2023/03/11 22:01:58 [warn] 14578#14578: *30 an upstream response is buffered to a temporary file /var/lib/nginx/tmp/proxy/2/00/0000000002 while reading upstream, client: IP.IP.IP.IP , server: , request: "GET /media/user_picture/user__naziha123__naz.jpg HTTP/1.1", upstream: "http://127.0.0.1:8000/media/user_picture/user__naziha123__naz.jpg", host: "mydomain.com", referrer: "https://mydomain.com/ad/list/"
2023/03/11 22:15:27 [emerg] 14720#14720: unknown "proxy_add_forwarded" variable
2023/03/11 22:15:41 [emerg] 14736#14736: unknown "proxy_add_forwarded" variable
2023/03/11 22:19:06 [notice] 14827#14827: signal process started
2023/03/11 22:19:16 [warn] 14828#14828: *331 an upstream response is buffered to a temporary file /var/lib/nginx/tmp/proxy/3/00/0000000003 while reading upstream, client: IP.IP.IP.IP , server: , request: "GET /media/user_picture/user__naziha123__naz.jpg HTTP/1.1", upstream: "http://127.0.0.1:8000/media/user_picture/user__naziha123__naz.jpg", host: "mydomain.com", referrer: "https://mydomain.com/ad/list/"
2023/03/11 22:26:54 [notice] 14924#14924: signal process started
2023/03/11 22:29:02 [warn] 14949#14949: *79 an upstream response is buffered to a temporary file /var/lib/nginx/tmp/proxy/1/00/0000000001 while reading upstream, client: IP.IP.IP.IP , server: , request: "GET /media/user_picture/user__naziha123__naz.jpg HTTP/1.1", upstream: "http://127.0.0.1:8000/media/user_picture/user__naziha123__naz.jpg", host: "mydomain.com", referrer: "https://mydomain.com/ad/list/"
2023/03/11 22:29:02 [error] 14949#14949: *64 connect() failed (111: Connection refused) while connecting to upstream, client: IP.IP.IP.IP , server: , request: "GET /ws/notification/broadcast_3/ HTTP/1.1", upstream: "http://127.0.0.1:5000/ws/notification/broadcast_3/", host: "mydomain.com"
2023/03/11 22:29:02 [error] 14949#14949: *64 connect() failed (111: Connection refused) while connecting to upstream, client: IP.IP.IP.IP , server: , request: "GET /ws/chats/notif_broadcast_naziha123/ HTTP/1.1", upstream: "http://127.0.0.1:5000/ws/chats/notif_broadcast_naziha123/", host: "mydomain.com"
If you’re looking for more from me, I’m waiting for you.
I asked:
You replied:
Ah I thought that I shared my ngnix configuration + non-channel conf, my bad, here is :
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Ok, your locations look good to me. (I usually see problems crop up when there’s a mix of fixed paths and regex specifications - that’s not the case here.)
Clearly /ws/ is going to be a longer prefix than / so I don’t see a conflict there. You don’t mention the file name of your / location file, but I don’t think this should be sequence-sensitive. (That wouldn’t explain the symptoms anyway.)
I think what I need to do here is replicate this on one of my test servers.
Just as a wild off-the-wall question, can you verify that your nginx instance is compiled with websocket support?
Also, could you please re-post your current asgi.py file? I know you posted it earlier, but there have been some messages back & forth about making changes to it, so I’d like to verify its current contents.
I’ve replicated your configuration - at least to the degree that you’ve shown here - and it works.
What this means to me is that one of the following is likely to be true:
-
There’s something at the system layer preventing websockets from working, such as firewall, traffic filter, nginx settings, whatever.
-
There’s something in your configuration that you’re not showing here that is affecting this. No idea what that might be, unless you’ve edited stuff out of the files you’ve posted.
-
Something in your JavaScript code for handling the socket that is failing.
Unfortunately at this point, I don’t have much in the way of advice. If I can’t recreate the issue, I’m not going to be able to identify the source of the problem.