Open a socket from channels

Using Django
#1

How would you go about opening a socket (or websocket) to a service running outside of django channels.
Quick drawing:

client(browser) <---- ws ----> django/channels <----- socket -----> external/third party "service"

Where the third party could be considered as a “real-time” backend (based on asyncio.start_server), pushing data to channels, which would bubble that data up to the client.

So far the following works well:
. client<->channels communication
. opening a socket (via the streams api) to the backend
. sending from client to channels
. having channels “forward” the client message to the backend

What does not work however is the StreamReader “listening” in channels. This blocks.

import asyncio
from channels.generic.websocket import AsyncWebsocketConsumer

class CustomConsumer(AsyncWebsocketConsumer):
    """
    extends AsyncWebsocketConsumer
    """

    backend_writer = None
    backend_reader = None

    async def init_backend_connection(self):
        self.backend_reader, self.backend_writer = await asyncio.open_connection('127.0.0.1', 8001)
        print(1)
        await self.backend_receive() # kaboom here?
        print(2)

    async def backend_receive(self):
        while True:
            try:
                data = (await self.backend_reader.read(128)).decode()
                print(f'message from backend: "{data}"')
            except (
                BrokenPipeError,
                ConnectionRefusedError,
                ConnectionResetError,
                OSError,
                asyncio.TimeoutError,
            ):
                break

    async def backend_send(self, message=None):
        if message:
            self.backend_writer.write(message)
            await self.backend_writer.drain()

class Consumer(CustomConsumer):
        async def connect(self):
            await self.accept()
            await self.init_backend_connection()

        async def receive(self, text_data=None, bytes_data=None):
            if bytes_data:
                # forward to backend
                await self.backend_send(message=bytes_data)

In init_backend_connection, print(1) is output. print(2) is never output. So it is as if self.backend_reader does not work. Does it block? If so why?

#2

If the connection to the back end is specifically associated with a connection, then I’d create an asyncio task within the consumer to handle the connection and let it fly. (Briefly, you don’t await a task. You start it and let the event loop dispatch to it as necessary. (It’s then up to you to close that task when the client disconnects.)
You can use queues to communicate between those tasks.

If the backend connection is supposed to be used by multiple connections, then I’d set it up as a channels worker and have it run in a separate process. If you do it this way, you can then use the channel layer to communicate between the two processes. (This is what we do - that way, the consumers share a single connection to the back end, and the messages are used to manage the traffic routing. Also in this case, we have control over the external service, making that easier to do - YMMV.)

Do you have an async working version of this outside the channels context? That might make a easier base to start from.