How to prefetch a GenericForeignKey with ManyToMany field?

I have some Models:


class User(models.Model):
    name = models.CharField()


class Project(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='projects')


class Task(models.Model):
    name = models.CharField()
    link_entity_type = models.ForeignKey(ContentType, on_delete=models.DO_NOTHING, null=True, related_name='link_tasks')
    link_entity_id = models.IntegerField(null=True)
    link = GenericForeignKey('link_entity_type', 'link_entity_id')


class Shot(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='shots')

    link_tasks = GenericRelation(
        'Task',
        object_id_field='link_entity_id',
        content_type_field='link_entity_type'.format(field.name),
        related_query_name='link_Shot'
    )


class Asset(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='assets')

    link_tasks = GenericRelation(
        'Task',
        object_id_field='link_entity_id',
        content_type_field='link_entity_type'.format(field.name),
        related_query_name='link_Asset'
    )

If I query Task and prefetch project and users, I will use:

query = Task.objects.all().prefetch_related('project__users')
for i in query:
    print(i.id)

This is ok, I can get the result.

But if I want to query Task and prefetch link_Shot and users

query = Task.objects.all().prefetch_related('link_Shot__users')
for i in query:
    print(i.id)

I got error:


Traceback (most recent call last):
  File "/ela/workspace/xinghuan/issues/sins/sins2server/tmp/tst_query.py", line 35, in <module>
    for i in query:
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 1883, in _fetch_all
    self._prefetch_related_objects()
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 1273, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 2321, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 2463, in prefetch_one_level
    ) = prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level))
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related_descriptors.py", line 746, in get_prefetch_queryset
    instance = instances_dict[rel_obj_attr(rel_obj)]
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related.py", line 750, in get_local_related_value
    return self.get_instance_value_for_fields(instance, self.local_related_fields)
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related.py", line 772, in get_instance_value_for_fields
    ret.append(getattr(instance, field.attname))
AttributeError: 'Shot' object has no attribute 'link_entity_id'

Is there any other way I can use or is this not supported?

See the (relatively new) GenericPrefetch object.

Could you provide the exact Django version you are using?

I assume you don’t get a crash on prefetch_related('link_Shot') but do get one when adding __users?

If that’s the case it might be a bug when chaining generic relation prefetch with many-to-many ones.

I don’t think GenericPrefetch will help here as it only allows to customize how each content type of a GenericForeignKey should be fetched but in the case of a generic relation it’s already narrowed to a single type.

I could be missing something but I think that while the docs claim that prefetch_related supports GenericRelation there’s absolutely no tests that covers the reverse case and it appears to have been broken since at least Django 4.2

diff --git a/django/db/models/fields/related_descriptors.py b/django/db/models/fields/related_descriptors.py
index 422b08e6ca..e139357e91 100644
--- a/django/db/models/fields/related_descriptors.py
+++ b/django/db/models/fields/related_descriptors.py
@@ -728,6 +728,9 @@ def get_queryset(self):
                 return self._apply_rel_filters(queryset)

         def get_prefetch_queryset(self, instances, queryset=None):
+            from django.contrib.contenttypes.fields import GenericRelation
+
+            assert not isinstance(self.field, GenericRelation)
             if queryset is None:
                 queryset = super().get_queryset()

No tests in the suite triggers an assertion error for this code path which means it’s untested.

I was able to reproduce your issue even without chaining to a many-to-many

diff --git a/tests/prefetch_related/tests.py b/tests/prefetch_related/tests.py
index c153c0d9ec..3a2f5b4c05 100644
--- a/tests/prefetch_related/tests.py
+++ b/tests/prefetch_related/tests.py
@@ -1243,6 +1243,12 @@ def test_deleted_GFK(self):
                 ],
             )

+    def test_reverse_gfk(self):
+        bookmark = Bookmark.objects.create()
+        TaggedItem.objects.create(content_object=bookmark, favorite=bookmark)
+        with self.assertNumQueries(1):
+            list(TaggedItem.objects.prefetch_related("favorite_bookmarks"))
+

 class MultiTableInheritanceTest(TestCase):
     @classmethod

I’d suggest filing a bug report demonstrating that while forward GenericRelation prefetching work reverse is broken and crashes.

Ok, I will create a new ticket about this.