Adding database backend methods to get hardcoded or nonexistent primary key values for tests

Django Internals
1

At Natalia’s request, I’m surfacing ticket #37175 “to see what our creative community can suggest.”

The problem:

While hardcoding primary key values in the Django test suite is avoided as much as possible, sometimes there are legitimate uses. The existing hardcoded integer values, however, don’t work on databases that don’t use integer primary key values. For example, MongoDB uses DEFAULT_AUTO_FIELD = "django_mongodb_backend.fields.ObjectIdAutoField".

My proposal:

Add new methods DatabaseOperations.get_hardcoded_pk() and get_nonexistent_pk() to allow a database backend to provide suitable values, minimizing the size of the Django fork needed for running the test suite on such databases.

Natalia’s reply:

I understand your concerns and the issue that this raises for other backends. I agree we could improve this situation by providing, potentially, some hooks at the database backend layer, but I also think that many of the tests currently hardcoding the PKs could be re-worked to not use a PK at all, or at least not assume is an int.

I’m accepting it right now since the problem statement is real, but I’d like to evaluate other options for the solution, ideally a solution that would not require a get_hardcoded_pk().

I provided PR 21497 which implements my proposal and has been tested with the CockroachDB and MongoDB backends. (CockroachDB does not need to use the new APIs, but it uses monotonically increasing, non-sequential integer primary keys which is different from the sequential IDs generated by the databases of Django’s built-in backends.)

Incidentally, after 2 1/4 years of contracting with MongoDB to build Django MongoDB Backend, my contract won’t be renewed when it ends next week (partly a casualty of the AI revolution, perhaps). So, in the event this proposal isn’t accepted in its current form, it will be up to the members of the MongoDB Python Drivers team to pursue this further. A big thank you to MongoDB for supporting this project, as well as to the members and staff of the Django community for reviewing my contributions to Django related to this project.

4 Likes