SynchronousOnlyOperation from celery task using gevent execution pool

Using Django Async/Channels
1

I opened an issue about a library I’m using for my django project, here’s the link: Async DataBaseHandler (Sink?) Django · Issue #879 · Delgan/loguru · GitHub
Searching the internet I found a question on stackoverflow which describes exactly what happens: django - SynchronousOnlyOperation from celery task using gevent execution pool - Stack Overflow

commands:

celery==5.2.7
django==3.2.19

celery -A scheduler.celery beat -l INFO --uid=nobody --gid=nogroup --detach
celery -A scheduler.celery worker -l INFO -E --uid=nobody --gid=nogroup -O fair --pool=gevent --hostname=celery-workerAnalyser --autoscale=30,10

The main question is: But why isn’t this exception raised in 100% of cases but only in a minority of cases?

2

The accepted answer on the SO thread has this:

It turned out my assumptions were wrong and I was using code which was not greenlet safe with gevent.

I’d guess you’d have to ask on gevent what that means. (I have to admit to having no idea myself. :sweat_smile:)

This isn’t really a Django question I think…

3

I was wondering if queries made via Models could be greenlet safe with gevent

4

Stumbled across this discussion while trying to debug the same problem.

If you look at the Django source, this exception is raised when there is a running event loop in the current OS thread. I was seeing it raised in Celery tasks that had nothing to do with an async event loop. In fact I went to great lengths to make sure that the event loop was exclusively in a thread created just for that purpose.

I believe the problem is the gevent “monkey patch” process. It wants to patch threading to make the Thread class … not a thread, but a greenlet that acts like a thread.

https://www.gevent.org/api/gevent.monkey.html#gevent.monkey.patch_all

Celery calls monkey_patch_all: