Can Django channel execute multiple consumers run in parallel per request to perform backend calculation

Using Django Async/Channels
1

Hi,
Want to know Django channel capability to perform backend complex calculation by running multiple consumers for one request.
I am about to build one backend calculation system under which for one client request there will be 1 job request and for that 1 job request can have upto 1k candidates (sometimes upto 10k). Doing calculation for 1k candidates at a time is very time consuming and load on database.

So in this case, can i create 10 consumers ( it can be many more or less based on no. of candidates under job request ) per 100 candidates and each consumer to process calculation for those 100 candidates.

Will this be possible with Django channel, if yes, then which protocol should i use WebSocket or http and why? do i need to create multiple channels? how much consumers can split max upto? how will i send response to after every consumer processing finished. i was referring this Worker and Background Tasks — Channels 4.0.0 documentation but its not giving full confidence whether it is possible with this or not.
How will my Django channel system basic system architecture or is there any best way to handle this using Django?
Any help will really be really appreciated.

2

From what I think I’m understanding of your description, I’m more inclined to believe that Celery would be your preferred solution. It scales up far more than individual worker processes would.

In the channels environment, you don’t create worker processes on-the-fly. They’re started by the runworker command and are either synchronous or async. If your processes are CPU bound, it wouldn’t matter which one you’re using - either way, throughput is going to be limited by what that process can handle.

Note - you can still architect this using Channels for the asynchronous notification of completion - I just wouldn’t try to handle that degree of processing within a worker - at least not if there’s any degree of concurrency required.