Django Channels with Redis slow intialization

Using Django Async/Channels
1

Hi,

I noticed a certain delay in the initialization state of a Channel. I measured the connection time on a websocket handshake of a AsyncJsonWebsocketConsumer:

async def connect(self) -> None:
        current_user: User = self.scope['user']
        if current_user.is_anonymous:
            raise DenyConnection('Unauthorized user')
        else:
            now = datetime.datetime.now()
            await self.channel_layer.group_add(
                current_user.get_channel_name(),
                self.channel_name
            )
            exec_time = (datetime.datetime.now() - now)
            print(f'CONNECT: {exec_time.total_seconds()} s')
            await self.accept()

The result was round about 2 seconds:

grafik
grafik1061×303 54.8 KB

This delay only appears for the very first user that opens the website, if no other user is currently on the website. I gues the connection to the DB is closed, if the last user disconnected from the website. This behavior makes perfectly sense.

Problem now: I use the websocket open event to retrieve some information, when the connection was established. To be specific: if the user has some unread notifications/messages.

This information has now this 2 second delay. Therefore now the question: Is there any way, to accelerate this in-memory DB connection?

P.S.: You may ask why I know it is the in memory DB? Because I have a development & production implementation of the consumer. The development one works instantaneously.

2

I’m a little confused by what you’ve written here.

You’re using the InMemoryChannelLayer in production, and that is causing you to see a 2-second delay, while you’re using a redis channel layer in development, and it is not showing that same delay?

Also, your log is showing a reference to a development_asgi settings file being served on localhost. I can’t reconcile this with the rest of your description.

Or are you talking about some other database connection? (If so, what?)

If it’s just an issue of making a first connection, you could write a script that opens a websocket, where that script gets run after Daphne is started. That moves the delay up to the start time.

3

No, I use the Redis backend in production. I just call it “in memroy DB” (!= “in memory”) because it’s not a Redis, but a Redict. So technically calling it: “Redis” is wrong. But I see now, that this is confusing. Therefore, let’s call it Redis.

OK yes, but i triggered the production configurations for the “Django Channels” framewwork in this settings file. Simply put: Ignore this.

No, this does not work. Problem here: I may start may own Redis client wherever I want, but this client has nothing to do with the client from the “Django Channels” framework. You would observe the same behaviour.

4

Calling it Redict would also work. I’m fine either way.

I’m not saying to run a redis client. I’m saying to open a websocket connection to your Daphne server, to make it open its redis connection.

Actually I don’t. As part of my deployment, I run a “smoke test” script that effectively does just what I described above. It logs into the Django site, gets the home page, and opens the websocket, all to verify that the deployment worked and that I’m left with an operational system. So if there’s a 2 second delay anywhere in that process, I don’t see it.

5

Hmm…, this is interesting. But this connection would need to stay open forever. Well…, as long as Django runs.

Are you using the entire “Django Channels” framework for your test or do you just open a Redis client & pushing some value? Somthing like this:

def redis_test():
    r = Redis(host='127.0.0.1', port=6379, decode_responses=True)

    success = r.set('foo', 'bar')
    value = r.get('foo')
    print(value)

    r.delete('foo')
    value = r.get('foo')
    print(value)

    r.close()

if __name__ == '__main__':
    print('LETS GO...')
    redis_test()
    print('DONE!')

This works fast. The “Django Channels” one, does not.

6

Why? I thought you mentioned that this delay was “first connection only”?

Yes. My client makes no direct connection to redis. There is no redis client involved at all.

I use requests and BeautifulSoup to check the basic Django app, and the websockets library for opening the websocket with nginx. (I serve basic Django with uwsgi, allowing Daphne to only be responsible for the websocket support. That’s why I have two separate tests.)

Also, my development environment is identical to my production environment. Development is also done with redis and PostgreSQL, and I know I don’t see any delays in development. (If there were a two-second delay in a connection, I would see it there.)

7

Sorry but I cannot confirm this. I don’t know what your test are measuring, but this 2 second delay is there.

I built a minimalistic project: Owner avatar DjangoChannelsTest

This project does barely more than the Django Channels tutorial.

If I run this minimal example, I still get the 2 seconds.

grafik
grafik1219×460 59.9 KB

8

In my test lab:

(d50) tskww@Ubuntu2004:~/git/DjangoChannelsTest$ ./manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
August 01, 2024 - 16:37:47
Django version 5.0.6, using settings 'core.settings'
Starting ASGI/Daphne version 4.1.2 development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
HTTP GET / 200 [0.03, 127.0.0.1:35470]
HTTP GET /static/bootstrap/css/bootstrap.min.css 304 [0.00, 127.0.0.1:35470]
HTTP GET /static/image/anonymous.png 200 [0.01, 127.0.0.1:35472]
HTTP GET /static/icons/bootstrap/font/bootstrap-icons.min.css 304 [0.01, 127.0.0.1:35470]
HTTP GET /static/bootstrap/js/bootstrap.bundle.min.js 304 [0.00, 127.0.0.1:35470]
HTTP GET /static/messenger/js/message_types.js 304 [0.00, 127.0.0.1:35472]
WebSocket HANDSHAKING /ws/notify/ [127.0.0.1:35508]
CONNECT: 0.006113s
WebSocket CONNECT /ws/notify/ [127.0.0.1:35508]

First, the most significant difference that I can see is that you’re running Daphne under Windows. I know that historically, you had to do something special when running Twisted under Windows to use the most efficient reactor. But since I gave up doing any web development in Windows about 20 years ago, I haven’t really followed its current status. (I do see in the Twisted docs that there is an installation option for Windows. See Installing Twisted — Twisted 24.3.0.post0 documentation. I’m not sure how that relates to setting it up to run with Daphne though.)

Second, I am running redis natively - not within a Docker container. I have no idea if there’s any interaction there or not.

But no, I am unable to recreate the issues you are seeing.

9

Hmmm…, ok. I guess i have to dig into it.

Thanks for the testing.

10

So…, I installed & started this on WSL2. The Result: The 2 seconds are gone. Therefore, this seems to be a problem with Windows indeed.

grafik
grafik1234×809 95.8 KB