N+1 problem in the list of generic relations

Using Django Using the ORM
1

Hi everyone! I have N+1 problem inside Django Admin when use Generic Relations. Here is a sample:

class Category(models.Model):
    title = models.CharField()
    notes = GenericRelation('Notes')

class Author(models.Model):
    name = models.CharField()
    notes = GenericRelation('Notes')

class Book(models.Model):
    title = models.CharField()
    author = models.ForeignKey(Author)
    category = models.ForeignKey(Category)
    notes = GenericRelation('Notes')

    def __str__(self):
        return f'{self.author.name} - {self.title} - {self.category.title}'

class Notes(models.Model):
    text = models.TextField()

    content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
    object_id = models.PositiveIntegerField()
    note_to = GenericForeignKey('content_type', 'object_id')

    def __str__(self):
        return f"Note for {self.note_to}: {self.text[:50]}"

So you can write note to any entity (except note to note). When I generate a list of notes in Django Admin, I receive N+1 problem due to Book string representation.

I know that I should use select_related/prefetch_related. And in the list of Books I can easily create custom query set and use it, for example:

class BookQuerySet(models.QuerySet):
    def with_related(self):
        return self.select_related('author', 'category')

class Book(models.Model):
....
    objects = BookQuerySet.as_manager()
....

And then in book/admin.py:

def get_queryset(self, request):
    return super().get_queryset(request).with_related()

But I don’t understand how to apply the same functionality to Generic Relations. Also, maybe in future Category string representation will have more complex structure with foreignkey (like Book now). And how to resolve this issue in common way?

2

If the issue is a real performance issue based upon measurements of real problems and not just the identification of an N+1 situation, then the solution is to restructure your data to avoid using GFKs. (And given that the Admin should not be your primary interface from a user-facing perspective, the N+1 issue in the Admin really shouldn’t be a huge problem.)

See Avoid Django's GenericForeignKey - lukeplant.me.uk and Generic relation vs Many to Many for a couple of perspectives on this. (The first article in particular identifies some alternatives.)

3

Thank you for your reply and related links.

Yes, this is real performance issue. Code I wrote above is just a sample to demonstrate the problem, real code is much more complicated. I know that GFK is not very good decision, but this decision was made by the team, and I have to work with this.

I made some research and found that there is GenericPrefetch method. It’s like Prefetch, but it can accept more than one querysets. So, for every GFK I need to rewrite get_queryset in admin, something like that:

class NotesAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        prefetch_query = GenericPrefetch(
            'note_to',
            querysets=(
                Book.objects.select_related('author', 'category'),
            )
        )
        return super().get_queryset(request).prefetch_related(prefetch_query)

Just one thing - it was introduced in Django 5.0.