Breaking a lance for "get_or_none" feature

Hi folks!

TLDR;

We should add a helper similar to get_or_create to fetch one object and return None if none exist.

User Story

As a Django developer I want to efficiently fetch an object where I’m not 100% sure that this object exists. An example could be an async task which does some chores on an object. If this object is gone for some reason, I just do nothing and that’s perfectly fine.

Motivation

Recently I came across this article in Django News. This hit me quite hard since I always used to go with the qs.first() and then check for None approach.

Since I find the try/except MyModel.DoesNotExist quite verbose, I’ve created a wrapper called get_or_none() which does what I want without having all that LOC.

Solution example

CodeTestsDocs

Background

I know that there we already some requets on adding this feature (Ticket 17546, Ticket 2659, Ticket 11352) which were rejected. Thx @sarahboyce for the research!

Breaking a lance

  • I feel that since Django provides a get_or_create helper, having this one is an obvious addition.
  • I didn’t know about the performance issues so I guess 98% of other Django developers won’t, too. Let’s help these people be green and save some ressources
  • The linked tickets argue about a slightly different approach as far as I understood it
  • Maintenance overhead for the Django maintainer of this helper is minimal

My colleage suggested to have a .any() queryset method instead of get_or_none() to be able to chain it to a queryset.


So dear fellow Django enthusiats, what do you think about that?

I’m not religious about the implementation details. If you think we should add something like that but in a different way, let’s talk about it! :heart:

1 Like

Actually, it already exists - it’s called first()

Quoting directly from the docs:

Returns the first object matched by the queryset, or None if there is no matching object.

If you dig through the discussions on the Developer’s mailing list, such as:

You’ll see discussions about this going back to 2006, with clear indications that first was the desired implementation of the idea of get_or_none. (https://groups.google.com/g/django-developers/c/h50RLblVDU8/m/u24rvRt7G5IJ , https://groups.google.com/g/django-developers/c/3RwDxWKPZ_A/m/lCViRoYQ3QYJ )

Please see Database optimization isn’t always obvious | Whitesell on whatever… - The article you reference is inaccurate, it’s based on a number of incorrect assumptions.

4 Likes

To me, the main benefit of a get_or_none method seems to be clarity of intent.

filter(...).first() actually expresses pretty well what it does: Return the first result of a queryset. The notion of “first” implies some kind of ordering, so ordering by the primary key if no other ordering is specified, seems reasonable.

But, as @GitRon wrote, there’s often the case where I’m expecting at most one object.
More often than I’d like to admin, I opt for filter(...).first() in those cases, because

  • I find it just more readable than a clunky try...except,
  • it fits better into the data flow
  • and I’m lazy.

And while filter(...).first() might be correct at the time of implementation (for example a DB constraint might make sure that there can only be one result), a few weeks from now, I might introduce some changes in my data model and thereby introduce a subtle bug that can go unnoticed for quite a while because filter(...) would actually return more than one result, but filter(...).first() hides it.

If get_or_none existed, I’d definitely use it.

Welcome @fbinz !

The functionality of get_or_none does exist - it’s just that Django calls it first.

Again, I suggest reading the Google groups developer threads listed above.

If you’re going to limit get_or_none to one possible result, then you’re back to having to handle the exception case if the query returns more than one result - in which case, get_or_none doesn’t really save you anything, and in which case it’s even more clear to catch the two exceptions separately.

The developers decided that the desired functionality of a get_or_none function was better named first, because it more accurately expresses how they’re handling the three different query cases.

This is all much better explained and in more detail in those threads.

Again, I suggest reading the Google groups developer threads listed above.

I did.

But just because something has been discussed (+ the usual bike-shedding) and decided over ten years ago, does not mean we should not ever bring the topic up again, does it?

If you’re going to limit get_or_none to one possible result, then you’re back to having to handle the exception case if the query returns more than one result - in which case, get_or_none doesn’t really save you anything, and in which case it’s even more clear to catch the two exceptions separately.

I don’t agree.

get_or_none would never throw a DoesNotExist error, so I don’t need to check for that.
And any MultipleObjectsReturned is clearly a violation of some invariant - i.e. a bug - that should fail loudly, so I would not check for that either.
Instead, I’d hope my tests (or in the worst case Sentry…) would inform me quickly, that somewhere I broke an invariant that would just go unnoticed with the filter(...).first() construct.

So, with get_or_none I could clearly express the intent, that I expect one object (like in get) or None.

If you read the complete threads - and found the others threads associated with that topic, you would see that it has been discussed multiple times since then, and reaffirmed in the group mailing list at least as recently as 2022. (https://groups.google.com/g/django-developers/c/SIgsYl_Mib4/m/fG4NwKxCAgAJ, https://groups.google.com/g/django-developers/c/SIgsYl_Mib4/m/eh6yZ5zMAwAJ, https://groups.google.com/g/django-developers/c/SIgsYl_Mib4/m/CWMSbkxfAgAJ)

The continuing and ongoing response remains that adding another function to perform the same thing as first with the exception that it exposes the multiple objects exception is considered an unnecessary expansion of the ORM API, and that if that behavior is desired, it is easily implemented by the developer.
(This is not an exact quote, it is my paraphrasing of the salient points made during the discussion and an aggregate summary of the responses.)

1 Like

Thank you for pointing me to the more recent threads. I did indeed not find those.

I think I’m with the others in the “this isn’t necessary” camp. Data issues shouldn’t lead to the server throwing a 500, so if you have to wrap every call to get_or_none in a try/catch for MultipleObjectsReturned, then what benefit does it really bring? You might as well do the plain get and catch both DoesNotExist and MultipleObjectsReturned anyway, or use first.

I’m against adding another shortcut to do exact the same thing that is currently possible (as I’ve stated many times in the past).

Nothing has changed in the few last years that would justify opening this discussion, TBH. We have exactly the same situation, all arguments against it are still valid, etc.

Yes, this. I don’t see any harm in folks adding their own project level helpers, but we already have the (proverbial) one way to do this, and expanding the API here would just be more surface area for no real gain.

1 Like