call django model in post_worker_init gunicorn function

Using Django Deployment
1

I want to load all books in memory per worker (store in django cache)

# gunicorn_config.py
bind = "0.0.0.0:8000"
workers = 6
worker_class = "gthread"
threads = 16
keepalive = 300

max_requests = 1000
max_requests_jitter = 500

worker_tmp_dir = "/dev/shm"
forwarded_allow_ips = "*"

errorlog = "-"
loglevel = "warning"

accesslog = "-"
access_log_format = '%({x-forwarded-for}i)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" %(M)s'

preload_app = True

from your_django_app.models import Book  # Import your Book model

def post_worker_init(worker):
    # Load data from the Book model after forking
    books = Book.objects.all()

    for book in books:
        print(book.title)

Sometimes my project is not executed and stop workers(SIGTERM):

psycopg2.OperationalError: lost synchronization with server: got message type "
or

django.db.utils.OperationalError: unexpected field count in "D" message
lost synchronization with server: got message type "4", length 942818304

or
psycopg2.OperationalError: lost synchronization with server: got message type "1", length 1413762115
or
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Sometimes my project runs without any errors

2

This sort of behavior would indicate some type of race condition, perhaps between the Django process instances and Gunicorn.

<opinion>
From the brief documentation of post_worker_init, it’s tough to identify exactly when that signal will be fired for each worker, and whether Django has completed its internal initialization process before the signal executes.
</opinion>

<conjecture>
Gunicorn is going to start the worker process, and then Django will begin its initialization process. My working assumption would be that this signal fires when the process has been forked off and started and not when Django returns some kind of “ready” status.
</conjecture>

But my first reaction to this is “why bother”? Unless you’re dealing with millions of books, I have a hard time seeing where you’re going to gain anything by this, especially since you don’t really have control over Django’s internal query cache. It would be interesting to see what the effects on the “first request” is in this situation.

3

thanks.

I have local bloom filter. I want to add book id to local bloom filter.
I deployed my project by gunicorn with --preload option. when we used --preload option we cant create local bloomfilter in ready app function becouse just one instance created so I want to create local bloomfilter per worker.

note (I do not want to use redis-bloom-filter)

when we used --preload option load django application code before the worker processes are forked.