Why aren't Managers and QuerySets one thing?

Using Django Using the ORM
1

Yes it’s another iteration of that question, but maybe I can settle the underlying conceptual issue instead of just re-asking “when to use a custom Manager vs QuerySet?”

Why didn’t the Django designers just make object a QuerySet?

In general nobody can answer[^1] when not to use QuerySet.as_manager(), or at least not beyond personal preference or convention[^2]. It’s been speculated[^3] that the distinction is just a hangover which has yet to be deprecated.

The closest I’ve found to functionality implied to exist for Managers but not QuerySets is that multiple models can have the same custom manager[^4]. But the article then contradicts this with an example custom queryset which, of course, is equally agnostic as to the model! Everything else in the article is pure convention: “Ideal for encapsulating methods affecting the entire queryset” vs “Designed for fine-grained, query-specific methods”? How? Why can’t querysets do the former? They have the same functionality under the same method names!

So much so in fact, that we have the infamous QuerySet.as_manager() to preserve DRY[^5]…

EXCEPT reading that documentation we see

Not every QuerySet method makes sense at the Manager level; for instance we intentionally prevent the QuerySet.delete() method from being copied onto the Manager class.

Methods are copied according to the following rules:

  • Public methods are copied by default.
  • Private methods (starting with an underscore) are not copied by default.
  • Methods with a queryset_only attribute set to False are always copied.
  • Methods with a queryset_only attribute set to True are never copied.

But those rules reveal nothing about the conceptual distinction that would cause a queryset method not to “make sense at the Manager level”.

A quick search of the Django codebase reveals queryset_only is only used to prevent QuerySet.delete() and QuerySet.as_manager() itself from copying to the generated manager. But why doesn’t .delete() “make sense at the Manager level”? Sure it’s unlikely you’d want to delete all records, but no less likely than wanting to do so on the queryset from Manager.all()…

If this too never gets answered, then at least I’ve collated the confusion into one place, so future searchers needn’t feel they’re missing something obvious while scrolling through millions of “QuerySet vs Manager?” posts on here, Stack Overflow, Medium etc.

(Apologies for the footnotes, new forum users are only allowed 2 URLs per post)

[^1]https://forum.djangoproject.com/t/when-to-use-a-manager-instead-of-a-queryset-with-as-manager/29496
[^2]https://forum.djangoproject.com/t/model-methods-custom-managers-queryset-when-to-use-them/7028
[^3]https://forum.djangoproject.com/t/managers-adding-extra-manager-method-vs-creating-a-manager-with-queryset-methods/8876/4
[^4]https://medium.com/django-unleashed/when-and-how-to-override-managers-and-querysets-in-django-f9f8f228fcb4
[^5]https://docs.djangoproject.com/en/5.1/topics/db/managers/#creating-a-manager-with-queryset-methods

3 Likes
2

Managers can’t be sliced, or iterated either, so that’s another difference.

I guess the biggest feature of this copy-methods approach is that you can write

Foo.objects.filter(...)

instead of the hypothetical

Foo.objects.qs.filter()

or whatever the syntax would be.

In any case, you can declare your own managers with custom methods, which would be a bit weird/awkward if it was a QuerySet directly I think.

3

Welcome @benxyzzy !

Side note: When wanting to post multiple reference links in a post, surround the link with single backticks to make it literal text. It’s no longer going to be a link, but it can be copy/pasted into a new tab.
e.g.:

becomes:
https://forum.djangoproject.com/t/when-to-use-a-manager-instead-of-a-queryset-with-as-manager/29496

1 Like
4

Note: There are many reasons why a question may go unanswered, and it is a mistake to assume that the lack of an answer means that no one can answer that question. Please keep in mind that no one is being paid to answer questions. All questions get answered as time, energy, and knowledge permits. (And yes, the reason can be as simple as the message being overlooked for whatever reason.)

Now, the key difference in my mind between a QuerySet and a Manager is that a QuerySet must return a queryset, and usually one that is related to the model to which that queryset is assigned, while a Manager method has no such restriction.
I can create manager methods returning JSON, CSV, or text strings in addition to queryset results. (E.g., see the values, values_list, and other manager methods that don’t return querysets.)
This was the point I was trying to make in the post you referenced above (^2). I do have a need, and use for, custom manager methods not returning querysets. I do not have a currently-identified need for factoring any of that out into a specific QuerySet object.

5

Hey boxed, thank you for responding -

From the Medium article I linked, it doesn’t look like custom querysets (with a custom method) are any more difficult than custom managers. In fact, the latter has to start a chain with self.get_queryset() whereas the custom queryset just has to start with self...

And if objects were a queryset to begin with (i.e. no need for managers), you could just write Foo.objects.filter(...)

apologies if I’m misunderstanding something

6

Hey Ken, thank you for the welcome (and for all the responses you’ve given on this forum which have helped me more times than I can remember) -

I didn’t consider methods which don’t return querysets at all, a light bulb went off there! I’ll wait a couple of days for other responses and then probably accept the answer.

Although just to say, it looks like both the default manager and queryset have values() and values_list(), and they return the same thing in either case: querysets containing dicts or tuples.

Perhaps it’s possible to add methods on custom querysets which return non-querysets, but that is certainly breaking the expected interface from what I’ve read. So a bit more than “just a convention”

7

Yea, rereading my response - I worded that really poorly.

Yes, values and values_list return objects that act like a queryset.

In addition to that, there are other manager methods returning other things. For example, there is get, first, last, etc, returning an instance of an object. Then there’s exists and contains that returns a boolean. The explain method returns a string.

The key point here is that a manager method can appropriately return anything, which cannot be said for a queryset.

8

That seems like a weak rationale though. A QuerySet subclass could do that too.

9

Except it’s not a rationale. It’s a definition of what is.

Could it be different? Sure. Got a time machine handy to go back and change a decision made 20 years ago?

10

aren’t those all queryset methods too? QuerySet API reference/QuerySet API/Methods that do not return QuerySets

It’s quite confusing because a) all the examples in that section use objects, a manager not a queryset, and b) I agree, I understood it to be the case that “querysets should return querysets” (for chaining presumably)

although I’ve checked locally and of course the docs are correct

Am I misunderstanding something basic here? I guess all of those non-queryset-returning methods are nonetheless returning something directly related to the query or records, rather than a JSON or CSV export

11

Yes, you are correct.

I’m going to need to dig back through the project where we did this and refresh my memory in more detail. (That was about 2016 or so…)

12

get, first, last, contains, and exists are all methods on QuerySet, not Manager.

I used to say what Ken said. Now I pragmatically just make one QuerySet per model, and use Queryset.as_manager(). Model manager methods aren’t chainable and what I really like about the ORM is the fluent api. I usually don’t think about it again, until I have to mentor a new hire, and they ask why the docs/internet mostly talk about custom managers but all they see in our code are QuerySets.

13

Yeah, it seems QuerySet functionality really is just a superset of Manager functionality. The example here kind of convinces me that your approach of putting custom methods in a QuerySet and using .as_manager() avoids chaining difficulty without downsides

14

Huh? All the model manager methods like filter() etc return a QuerySet so then after that they chain cleanly. I don’t understand what you mean by this.

15

@boxed see the example in my last link

class PersonManager(models.Manager):
    def with_extra_fields(self):
        return self.annotate(
            full_name=Concat("first_name", models.Value(" "), "last_name"),
        )

    def experienced(self):
        return self.filter(
            join_date__lt=date.today() - timedelta(days=1000)
        )


"""Unfortunately, there are some inconveniences to using managers.
The biggest one is that manager methods cannot be chained 😞,
which is really powerful when working with QuerySets."""


>>> Person.objects.experienced().with_extra_fields()
...
AttributeError: 'QuerySet' object has no attribute 'with_extra_fields'
16

Aaah, the custom manager methods aren’t chainable. Yea ok, that makes sense.

17

I think the reality is that managers are used to manage the data. So you have for example Model.objects.create() to create a new instance. If you want to add custom methods that manipulate data, they should be in managers. QuerySets are only for queries (fetching data).

18

.create() is also a QuerySet method, same as those which @massover listed.

Furthermore I would say that in SQL parlance, creating records is indeed achieved via a “query” (INSERT)

19

Hah, that’s very weird!

In [6]: Role.objects.filter(name='nothingmatches').create(name='foo')
Out[6]: <Role pk=10 name=foo>

That to me seems like it shouldn’t work. It makes no semantic sense.

20

Oops so yeah it is a queryset method but it doesn’t make any sense. The docstring for Queryset says “Represent a lazy database lookup for a set of objects.” which is what it should be. Now I am confused too :slight_smile:

But I disagree here that because it is a “query” it should be defined in the queryset.