Hi, all!
We’re using Django with a cloud backend, where query parameters number is limited to 950. At some point we started to get errors of hitting this limit on a test session run, specifically, at the point of creating permissions:
File "/home/runner/work/python-spanner-django/python-spanner-django/django_tests_dir/django/django/contrib/auth/management/__init__.py", line 83, in create_permissions
Permission.objects.using(using).bulk_create(perms)
We have a max_query_params set to 900 at the DatabaseFeatures class, but something tells me Django doesn’t see it. I didn’t notice any use of the attribute in the bulk_create() method. Is it really used (in docs it’s said that only SQLite uses a limit, in all other cases the method is trying to insert all the rows in a single batch)? If not, it’d be great if it would start to use, 'cause we have the attribute set, but we’re still getting an error of hitting the limit.
If I’m reading the source code correctly - and you can verify this by looking at the code in places such as django.db.backends.*.operations.DatabaseOperations - that setting (max_query_params) is only checked in the oracle and sqlite3 backends in the bulk_batch_size method.
Outside of editing the Django package directly, I don’t see where there’s an easy way to change that method other than monkey-patching the engine. (Well, you could copy the entire engine into your project and edit that.)
Yes, looks like only Django package change can solve this. Is there a way to create a “feature request” to the Django team?
We’d prefer not to sustain our own version of Django, as it’s a big project, keeping it actual can be challenging.
My initial suggestion would be to join the Developer’s mailing list and open this up as a topic for discussion. That’s usually the best place to start talking about a feature request. (Someone there may have an even easier workaround as well.)
Until then, I don’t believe you would need to maintain a complete version of Django, just a couple of classes from the DB engine. (Or, monkey-patch the DatabaseOperations class.) Either way, you might still want to do this even if your feature request is approved, because it may take a few months before the code for it is written along with the necessary tests and documentation and then to be merged into the released version.
1 Like
Got it. I’ve written in the mailing list, thanks.
Oh, don’t know, monkey patching seems to be a bit complicated at this case (we have some patches for other problems, but they are way more simple). While patching one method, another one is also required to be patched, and then the third… Trying to figure out now, if there is a way to make it simple.