I have a django app which is using the integration with celery described here, most notably all the celery configurations are specified the django settings. I have a need to hot refresh some settings, specifically the celery broker password, in both the server and celery workers. Our RMQ broker passwords are rotated somewhat frequently. I can write some logic which refreshes that password, but I can’t seem to find a way to inject it without restarting the workers and servers.
Anyone have ideas for me?
If that’s the case, you might need to use a more native Celery, rather than integrating Django with Celery.
- In Django, you might need to write some logic to monitor changes to the Celery configuration file, and restart the worker when changes occur (this way, the servers do not need to be restarted).
- Of course, there’s another way: create a mapping between the Celery broker password and the rotation password of your RMQ management (this specific implementation can refer to dynamic passwords). In this way, you can achieve your requirements without restarting the worker and servers. Theoretically, with this method, neither the workers nor the servers need to be restarted.
If you have any other questions for above things, I’d like to talk something with you, expect your reply.
I proposed this issue in celery to add a hook for this behavior. I was able to achieve something similar without that feature, simply decorating the relevant connect_XXX methods. It works, but I think would look cleaner with a hook in the library itself.
Your workaround sounds interesting! Does it handle password updates smoothly without causing any downtime for workers?