django websocket not working properly using daphne and nginx

I have setup django websocket on aws with daphne and I am using nginx to serve it, the setup works well, it gets connected when I test it with postman or even on vuejs. But the issue I am having now is that it disconnects when I send a message. I have tried all I could to make it work to no avail.

nginx error:
1159#1159: *1 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 102.xx.xx.xx, server: 98.xx.xx.xx, request: "GET /ws/chat/chikjib_chikejibunoh523/?token=eyJhbxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxXQiOjE3MjY1MTYxMzMsImp0aSI6ImMzYjcxNzM4YjQ4NjRjNTU5NDFhYzAzYzMzYmUwNzdmIiwidXNlcl9pZCI6NH0.8OknTXb8IfP9oPOOV_ppGWU9exCLdBIqSXx6T1JHoV8 HTTP/1.1", upstream: "http://127.0.0.1:8001/ws/chat/chikjib_chikejibunoh523/?token=eyJhbxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxmMzYjcxNzM4YjQ4NjRjNTU5NDFhYzAzYzMzYmUwNzdmIiwidXNlcl9pZCI6NH0.8OknTXb8IfP9oPOOV_ppGWU9exCLdBIqSXx6T1JHoV8", host: "98.xx.xx.xxx"

nginx conf


server {
    server_name 98.xx.xx.xxx;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        autoindex on;
        alias /var/www/staticfiles/;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }

    location /ws/ {
        proxy_pass http://127.0.0.1:8001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }
}

gunicorn.service

[Unit]
Description=WebSocket Daphne Service
After=network.target

[Service]
Type=simple
User=xxxxxx
WorkingDirectory=/home/xxx/src
ExecStart=/home/succour_root/succour_venv/bin/python /home/xxx/succour_venv/bin/daphne -b 127.0.0.1 -p 8001 succourBackend.asgi:application
Restart=on-failure

[Install]
WantedBy=multi-user.target

asgi.py

"""
ASGI config for succourBackend project.

It exposes the ASGI callable as a module-level variable named ``application``.
"""

import os
import django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "succourBackend.settings")
django.setup()

# from django.core.asgi import get_asgi_application
from channels.auth import AuthMiddlewareStack
from channels.security.websocket import AllowedHostsOriginValidator
from channels.routing import ProtocolTypeRouter, URLRouter
from succourBackend.routing import websocket_urlpatterns
from succourBackend.middleware import JWTAuthMiddleware


application = ProtocolTypeRouter(
    {
        "websocket": 
            AllowedHostsOriginValidator(
              AuthMiddlewareStack(
                JWTAuthMiddleware(
                  URLRouter(websocket_urlpatterns)
                )
              )
            )
        ,
    }
)

routing.py

from django.urls import re_path
from api import consumers
import logging

logger = logging.getLogger(__name__)

logger.info("Got to the route")

websocket_urlpatterns = [
    re_path(r'ws/chat/(?P<room_name>[^/]+)/$', consumers.ChatConsumer.as_asgi()),
]

consumers.py

from channels.generic.websocket import AsyncWebsocketConsumer, WebsocketConsumer
import json
from channels.db import database_sync_to_async
from urllib.parse import urljoin

from api.models import ChatMessage
from api.serializers import UserProfileSerializer
from django.contrib.auth import get_user_model

User = get_user_model()

class ChatConsumer(AsyncWebsocketConsumer):

    async def connect(self):
        self.user = self.scope.get("user")
        if not self.user.is_authenticated:
            await self.close()
            print("User not authenticated")
            return

        self.room_name = self.scope["url_route"]["kwargs"]["room_name"]
        self.room_group_name = f"private_chat_{self.room_name}"

        # Add user to the room group
        await self.channel_layer.group_add(self.room_group_name, self.channel_name)

        # Retrieve and send unread messages for this user
        unread_messages = await self.get_unread_messages(self.room_group_name)

        # Determine the host to build absolute URLs
        protocol = "https" if self.scope.get("scheme") == "https" else "http"
        host = f"{protocol}://{self.scope['server'][0]}:{self.scope['server'][1]}"
        
        for message in unread_messages:
            profile_picture_url = None
            if message["profile_picture"]:
                profile_picture_url = urljoin(host, message["profile_picture"].url)

            await self.channel_layer.group_send(
                self.room_group_name,
                {
                    "type": "private_chat_message",
                    "message": message["message"],
                    "username": message["sender_username"],
                    "timestamp": message["timestamp"].isoformat(),
                    "sender": {
                        "full_name": message["full_name"],
                        "type": message["type"],
                        "profile_picture": profile_picture_url,
                    },
                },
            )

        await self.accept()
        print("User connected to the room:", self.room_name)

    async def disconnect(self, close_code):
        # Remove user from the room group
        await self.channel_layer.group_discard(self.room_group_name, self.channel_name)
        print("User disconnected from the room:", self.room_name)

    async def receive(self, text_data):
        text_data_json = json.loads(text_data)
        message = text_data_json.get("message")
        recipient_username = text_data_json.get("recipient")

        if recipient_username:
            recipient_user = await self.get_user_by_username(recipient_username)

            if recipient_user:
                # Check if the user is part of this chat
                # if not self.is_user_in_chat(recipient_user):
                #     print("User not authorized to join this chat")
                #     return

                await self.save_private_message(
                    self.user.id, recipient_user.id, message, self.room_group_name
                )
                await self.channel_layer.group_send(
                    self.room_group_name,
                    {
                        "type": "private_chat_message",
                        "message": message,
                        "username": self.user.username,
                    },
                )

    async def private_chat_message(self, event):
        message = event["message"]
        username = event["username"]
        timestamp = event["timestamp"]

        await self.send(
            text_data=json.dumps(
                {
                    "message": message,
                    "username": username,
                    "timestamp": timestamp,
                    "sender": event["sender"],
                    "type": "private_chat",
                }
            )
        )

    @database_sync_to_async
    def save_private_message(self, sender_id, recipient_id, message, group_name):
        sender = User.objects.get(id=sender_id)
        recipient = User.objects.get(id=recipient_id)
        return ChatMessage.objects.create(
            sender=sender, recipient=recipient, message=message, group_name=group_name
        )

    @database_sync_to_async
    def get_user_by_username(self, username):
        try:
            return User.objects.get(username=username)
        except User.DoesNotExist:
            return None

    @database_sync_to_async
    def get_unread_messages(self, group_name):
        # Get all messages for the specified group
        messages = ChatMessage.objects.filter(group_name=group_name)
        # print(UserProfileSerializer(messages[0].sender).data)
        return [
            {
                "message": message.message,
                "sender_username": message.sender.username,
                "timestamp": message.timestamp,
                "full_name": message.sender.get_full_name(),
                "type": message.sender.type,
                "profile_picture": message.sender.profile_picture,
            }
            for message in messages
        ]

    def is_user_in_chat(self, recipient_user):
        # Check if the user and recipient are part of this chat room
        users_in_chat = self.room_name.split("__")
        return (
            self.user.username in users_in_chat
            and recipient_user.username in users_in_chat
        )

    async def chat_message(self, event):
        message = event["message"]
        username = event["username"]

        await self.send(
            text_data=json.dumps(
                {
                    "message": message,
                    "username": username,
                }
            )
        )

        self.send(
            text_data=json.dumps(
                {
                    "message": "message",
                }
            )
        )

Please what could be the cause. I need help.

Are you getting any messages logged by Daphne? (Check both the Daphne logs and syslog.)

Please clarify, when you say:

Are you talking about sending a message from the browser through the websocket to the server? Or are you talking about the consumer sending a message out to the browser?

Side note: When first learning how to work with channels in this type of environment, I added a lot of print statements throughout my consumer to help me understand exactly was was being called, and in what order, and what data was being passed. It also helped me find sources of errors that were happening.

Also, I believe:

needs to specify proxy_pass http://127.0.0.1:8001/ws/;

Thanks for your response.

I am testing the websocket endpoint on Postman and I am sending a message from user to user, I connected both websocket on a group, from the nginx error you can see chikjib_chikejibunoh523, meaning I am sending a message in group using through channels.

Again in my consumers.py I have printed some of the lines as you can see. I have a table in my database where I save the messages. The websocket connects, It allows me to send a message, but as soon as I hit the send message it disconnects. When I reconnect back the message I sent is already recorded the sender side and on the receiver side. So my major issue is why does it disconnect as soon as I hit send.

Even when I change it restarted my services and server, the same disconnecting experience is recorded.

All through the logs I can’t see anything.

I have used
sudo tail -F /var/log/nginx/error.log
sudo journalctl -u nginx
sudo journalctl -u daphne

No logs recorded

This is my channel layers setting

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            "hosts": [('127.0.0.1', 6379)],
            'capacity': 1500,
            'expiry': 10,
        },
    },
}

Not knowing anything at all about Postman and how it manages websockets, that would be my first suspect since it’s such an unknown. How do you know that Postman isn’t disconnecting you? What information does the Postman Console provide regarding this?

Also, you have not answered the previous questions regarding error messages associated with Daphne. What are you seeing in the Daphne logs or console output about this? (You can try adding the -v 3 parameter to your Daphne command to make the output more verbose.)

Not good enough. Go snooping through /var/log looking for information. You also should directly check syslog. Read the files directly, don’t expect any of the standard “summary” tools to show you everything you may need to see.

Okay, I have done everything you recommended. I have added verbose to the command


[Unit]
Description=WebSocket Daphne Service
After=network.target

[Service]
Type=simple
User=xxxx
WorkingDirectory=/home/xxxxx/src
ExecStart=/home/succour_root/succour_venv/bin/python /home/succour_root/succour_venv/bin/daphne --verbosity 3  -b 127.0.0.1 -p 8001 succourBackend.asgi:application
Restart=on-failure

[Install]
WantedBy=multi-user.target

I have the following logs you asked for

Postman Console

Daphne Error log in both /var/log/syslog and journalctl is same as below part 1:

Daphne Error log in both /var/log/syslog and journalctl is same as below part 2:

So after i added verbose to the daphne service

The daphne error log after i restarted daphne i got this error:

couldn’t import psycopg ‘c’ implementation: No module named ‘psycopg_c’

I don’t know if that could make it to disconnect as soon as i hit send

Note: This error now shows only when i restart daphne. But it does not affect my connecting to the websocket.

Please do not post images of textual information here. Copy/paste the text of that console output into the body of your post.

Beyond that, you commented:

Yes, it could.

Simply connecting to the websocket does not necessarily cause any requests to be issued to the database. But if your Daphne instance is having problems connecting to the database, then anything that the consumer does that needs the database could fail.

Please show your DATABASES settings from your settings.py file, and a pip list of the virtual environment in which this is being run.

Okay your comment is noted on the issue of images.

This is my settings.py as it relates to DATABASE settings

DATABASES = {
    "default": {
        # 'ENGINE': 'django.db.backends.mysql',
        "ENGINE": "django.db.backends.postgresql",
        "STORAGE_ENGINE": "InnoDB",
        "NAME": config("DATABASE_NAME"),
        "USER": config("DATABASE_USER"),
        "PASSWORD": config("DATABASE_PASSWORD"),
        "HOST": config("DATABASE_HOST"),
        "PORT": config("PORT"),
    }
}

Result of pip list


Package                       Version
----------------------------- ---------
asgiref                       3.8.1
astroid                       3.2.2
async-timeout                 4.0.3
attrs                         23.2.0
autobahn                      23.6.2
Automat                       22.10.0
certifi                       2024.6.2
cffi                          1.16.0
channels                      4.1.0
channels-redis                4.2.0
charset-normalizer            3.3.2
constantly                    23.10.4
coreapi                       2.3.3
coreschema                    0.0.4
cryptography                  42.0.8
daphne                        4.1.2
dill                          0.3.8
Django                        5.0.6
django-anymail                10.3
django-cors-headers           4.3.1
django-crontab                0.7.1
django-rest-swagger           2.2.0
djangorestframework           3.15.1
djangorestframework-simplejwt 5.3.1
drf-yasg                      1.21.7
gevent                        24.2.1
greenlet                      3.0.3
gunicorn                      23.0.0
hyperlink                     21.0.0
idna                          3.7
incremental                   22.10.0
inflection                    0.5.1
isort                         5.13.2
itypes                        1.2.0
Jinja2                        3.1.4
MarkupSafe                    2.1.5
mccabe                        0.7.0
msgpack                       1.0.8
oauthlib                      3.2.2
openapi-codec                 1.3.2
packaging                     24.0
pillow                        10.3.0
pip                           24.0
platformdirs                  4.2.2
psycopg                       3.1.19
psycopg-binary                3.1.19
pyasn1                        0.6.0
pyasn1_modules                0.4.0
pycparser                     2.22
PyJWT                         2.8.0
pylint                        3.2.3
pylint-django                 2.5.5
pylint-plugin-utils           0.8.2
pyOpenSSL                     24.1.0
python-decouple               3.8
pytz                          2024.1
PyYAML                        6.0.1
redis                         5.0.7
requests                      2.31.0
requests-oauthlib             2.0.0
service-identity              24.1.0
setuptools                    74.1.1
simplejson                    3.19.2
six                           1.16.0
sqlparse                      0.5.0
tomlkit                       0.12.5
Twisted                       24.3.0
txaio                         23.1.1
typing_extensions             4.11.0
tzdata                        2024.1
uritemplate                   4.1.1
urllib3                       2.2.1
websocket                     0.2.1
whitenoise                    6.6.0
zope.event                    5.0
zope.interface                6.4.post2

Based on this error:
couldn’t import psycopg ‘c’ implementation: No module named ‘psycopg_c’,
I have been able to resolve it by
pip install “psycopg[c]”,

I restarted daphne and the error was gone. But still yet the disconnecting issue of the websocket still persist.

Please note that on my local system, using python manage.py runserver, the websocket works perfectly. But when I run it on my aws ec2 (ubuntu os), I have not been able to make it work using daphne. I just feel the issue is from daphne or django channels but I have not been able to locate the issue.

The only error i get on my POSTMAN CONSOLE is

1011 Server Error:Internal server error while operating.

after it disconnects.

Side note while I’m looking at this: If you look at your console output when you’re using runserver in development, you’ll see that it is using Daphne.
Example:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
September 17, 2024 - 16:14:32
Django version 5.0, using settings 'chat_server.settings'
Starting ASGI/Daphne version 4.0.0 development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Yes very correct, that’s what i get locally.

So the point here is that you’re not changing the asgi container for your application. It’s not like you’re changing something in that part of the stack when you’re shifting from local development to your deployed environment. There’s something else that’s the root issue here.

Exactly what I have been looking for, for weeks now. Please any help, because I have passed my delivery date. Thanks

If this is something truly time-critical for you, then I suggest you hire a consultant with expertise in this area.

Please remember that we are all volunteers here, answering questions as time, knowledge, and energy allows.