How to access annotated information from custom model manager in admin list display

Using Django Using the ORM
1

Hello everyone,

please find below a reduced example of the original problem that I am currently facing. Essentially, I am trying to access an annotated property n_references through some property number_of_references for the string-representation of a related ModelB instance in the ModelC admin list view (sorry in case the terminology might not be fully correct).

# Models

from django.db import models as mo


class ModelA(mo.Model):
    some_field = mo.CharField()


class ModelBManager(mo.Manager):
    def get_queryset(self):
        return super().annotate(n_references=mo.Count("references"))


class ModelB(mo.Model):
    class Meta:
        base_manager_name = "objects"

    objects = ModelBManager()
    references = mo.ManyToManyField("ModelA")

    @property
    def number_of_references(self):
        return self.n_references

    def __str__(self):
        return f"Number of references: {self.number_of_references}"


class ModelC(mo.Model):
    relation = mo.ForeignKey("ModelB", on_delete=mo.PROTECT)


# Admin

from django.contrib import admin


@admin.register(ModelC)
class ModelCAdmin(admin.ModelAdmin):
    list_display = ("relation",)

If I try to access the list view for ModelC, I receive an error that the instance of ModelB does not have an attribute n_references. Therefore, it seems to me that for retrieving the data for the list view, I am currently not using the manager ModelBManager.

Can you tell me how to specify the correct manager in this context? Please note that in the dropdown fields of the admin creation forms of ModelC, the available instances of ModelB for the relation field are show correctly, so it seems that the correct manager is used at least in this place already.

Thank you very much.

2

Welcome @adbuerger !

In your model methods, self is a reference to the current instance, and not a reference to a queryset or manager method.

That’s why you get an error at:

there is no attribute n_references in the model.

However, you can get the count directly from the related object manager:
e.g. self.referemces.count()

3

Thank you @KenWhitesell for the welcome and for your fast reply!

I understand that self is a reference to the current object. However, when retrieving a ModelB instance from the database like ModelB.objects.first(), I see that I can actually access the annotated information via self.n_references.

To me, this seems like the ModelBManager is applied in such context, but not when the model instances are retrieved for display in the admin list view of ModelC. I would assume that this is connected the circumstance that ModelB is not retrieved, well, “directly”, but via a foreign key relation for ModelC. And while this also seems to work in other contexts in my code, there appears to be a problem specifically in the Django admin.

Would that make sense? And how could I achieve to use the manager also in this context? Also, thank you for the suggestion to just evaluate the count()-method directly in the property, however, the actual use case is more elaborate that the example shown above.

4

What is the “this” that you are referring to here? (What exactly is working in other contexts within your code?)

Note from the docs for Model._base_manager:

Base managers aren’t used when querying on related models, or when accessing a one-to-many or many-to-many relationship.

(I’m not entirely sure how this statement applies here, but it does seem relevent.)

5

@KenWhitesell thank you for your reply and sorry for not being completely clear about what I mean whit “this”. Let me narrow the problem down by systematically going through other use cases from my code. When calling


b = ModelB.objects.first()

then I can access b.n_references. When calling


c = ModelC.objects.first()

or


c = ModelC.objects.prefetch_related("relation").first()

then I can access c.relation.n_references, but only if base_manager_name = "objects" is set in the Meta class of ModelB. Otherwise the attribute n_references does not exist.

This is consistent I think with the information given in the section of the reference you provided (while the part from the documentation saying

Base managers aren’t used when querying on related models, or when accessing a one-to-many or many-to-many relationship.

relates to another use case I think, e.g., if I would try to filter as in

ModelC.objects.filter(relation__n_references=5)

then this would not work as the base manager would not applied here).

So, the problem seems to be that when Django collects the information about the related ModelB instances for display in the admin list view for ModelC, it does not use ModelBManager but the generic manager of ModelB which leads to the missing attribute n_references for the ModelB instances. However, when I slightly adjust the get_queryset method of the ModelCAdmin as in

class ModelC(mo.Model):
    relation = mo.ForeignKey("ModelB", on_delete=mo.PROTECT)

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        return qs

(no functional changes but I can inspect the queryset in qs) the ModelC instances in qs do actually allow to access relation.n_references. So at the moment, I am a bit stuck as I cannot understand what is going wrong between collecting the queryset and displaying the entries in the list, or at which place the information would be retrieved again from the database in a different query.

Any ideas on how to continue to investigate are highly appreciated, thank you very much.

6

Couple different thoughts:
What happens if instead of:

you change this to:

class ModelBManager(mo.Manager):
    def get_queryset(self):
        return super().get_queryset().annotate(n_references=mo.Count("references"))

(I think you’re calling the wrong super method in the manager. I believe you’re supposed to call get_queryset and apply the annotate to it - but I’m far from certain that this is an issue, I’m just going by the example in the docs.)

And what happens in your list display if instead of:
list_display = ("relation",)

you tried either:
list_display = ("relation__n_references",)
or
list_display = ("relation__number_of_references",)

(I don’t know where in the “list view” processing it’s going to call the __str__ method on the model instance as compared to directly specifying the field or property name in the list_display attribute.)

EDIT: I just tried this and none of these suggestions have any effect. Still looking …

7

Quick note:

The “prefetch_related” doesn’t do anything for you, because it’s ModelC that has the FK to ModelB. (It’s creating a second, unnecessary query, and it’s that second query that is using the ModelBManager.) The appropriate function here is select_related - and in which case this does not work to populate the annotated field.

8

You are right of course, it should be super().get_queryset().annotate(...). Thanks for pointing it out, however, the mistake was in the forum post only, in my local testing code I had it correct. It seems though that I cannot update the original post anymore otherwise I would have liked to correct it.

9

True, thank you for pointing this out. And I think I found the place where the annotations go missing: django/django/contrib/admin/views/main.py at ea4a1fb61e0bc6a4294a0123b82183da947e5efb · django/django · GitHub At that place, we have some select_related-call, after that the annotations are gone.

What I understand so far is that in my case, since there is the related field relation in list_display, select_related is issued for the queryset qs. I don’t know whether this happens for performance or other reasons.

@KenWhitesell do you happen to know why the custom manager is not used when an object is retrieved using select_related? Maybe I am overlooking an obvious reason but I did not find a good explanation for this yet.

10

Do I know for sure? Not a chance.

My thinking on this is that it’s fundamentally the responsibility of the manager to build the query. In this case, the manager being used is the ModelC manager. Using select_related isn’t creating a query, it’s modifying the query that this current manager is creating. I see no need or reason to involve a different manager in this process.

I would think that what you would need to do here is (somehow) add a manager method in the ModelC manager to apply that annotation to the related model. (No idea precisely what that would look like, but superficially seems to be the appropriate approach here.)

11

Thank you once more for your help and hints @KenWhitesell

Now that I know the reason for the missing attribute and the possibilities to work around it, what I finally ended up with is to check at the relevant places whether the required attributes exist for an object, and if not, to retrieve the required information from the database, which looks like a good tradeoff between effort and result to me, though it introduced a bit of redundancy in the code.