Our team has been seeing occasional (2006, 'MySQL server has gone away’) errors when trying to make queries during sparse usage.
Normally, it looks like each time a request is started and finished, Django goes through and closes old connections. However, because we are using Django in a standalone script outside of the request context, this signal is not called.
Currently, there are some workarounds, such as wrapping queries in a decorator to run close_old_connections beforehand. Are there any other suggestions for how to fix this issue?
It seems like this might be a useful functionality built in to Django itself. What do you think people think about automatically calling close_old_connections lower in the stack, say at the query set level?
1 Like
Connections are stateful - they can contain open transactions/savepoints, temporary variable, settings, CTE’s, and tables. Because of this state, especially open transactions/savepoints, I don’t think it would make sense, across all db backends and usecases, to call close_old_connections when a QuerySet is evaluated.
SQLAlchemy handles this problem by forcing users to be explicit about connection lifetime. There, you can use a session to manage access through a connection, which comes from a connection pool. When you call session.close(), the connection is returned to the pool which gives it a chance to decide around closing it or not.
I’m not sure it would be possible to transition Django to such an explicit API, since the connection objects are global thread singletons. We could perhaps provide the decorator/context manager that calls close_old_connections in its finally clause, as per that linked post. But I’m afraid it wouldn’t be much of a shortcut, and would still need explicit placement in worker contexts like yours.