How to wrap a generator for a model property

Using Django Using the ORM
1

In my models, I have different types of relationships between Events and Profiles. I want to work with the profiles that are an event’s promoters in a templae, so I’ve created a promoters property.

Right now, the @property returns a generator:

class ProfileEventRelationship(models.Model):

    class RelationshipType(models.TextChoices):

        PROMOTER = "P"

        OTHERS = "*"

    profile = models.ForeignKey(
        Profile,
        on_delete=models.CASCADE,
        related_name="event_relationships",
    )

    event = models.ForeignKey(
        "Event", on_delete=models.CASCADE, related_name="profile_relationships"
    )

    relationship_type = models.CharField(max_length=1, choices=RelationshipType)


class Event(models.Model):

    @property
    def promoters(self):
        return (
            pr.profile
            for pr in self.profile_relationships.filter(
                relationship_type=ProfileEventRelationship.RelationshipType.PROMOTER
            )
        )

… but filters like first don’t work with generators: ‘generator’ object is not subscriptable.

So what’s the right way to do this in Django? Is there a way to wrap a QuerySet around a generator, that also caches the data, so it’s not expensive to first check whether there are any promoters?

Actually, in this case, I only really expect one or two promoters, so just returning a list would be fine. But is a more Djangoistic way, that would also work for large numbers of items, say for pagination?

Also, is there a less verbose way to write that promoters filter, where I don’t have to specify the whole relationship class name?

2

Create a query for this. Identify what the conditions are for the set of objects to be retrieved, and build the appropriate query for it.

Unfortunately, I’m having a difficult time understanding what you’re trying to do here, so at the moment, I can’t be more specific.

Are you trying to query for a set of Event, or a set of ProfileEventRelationship?

What is the condition that needs to be satisfied for that set?

3

Sorry I wasn’t clear.

I want a property in my Event model, that returns all the related Profiles, where relationship_type=ProfileEventRelationship.RelationshipType.PROMOTER.

In SQL, it would be this:

select p.* from profile p join profileeventrelationship r on r.profile_id == p.id join event e on r.event_id = e.id where r.relationship_type = "P" and e.id = "{ self.id }";

I’ve actually already created a query; it’s in the promoters property function in the code above. But the query returns ProfileEventRelationships, and I want the property to return the related Profiles, so I wrote the generator. But I doubt that’s the right way.

4

From your response, it looks like you actually want some of the facilities that the models.ManyToManyField already does for you.
I suggest that you review the documentation for it, and the one for Extra fields on many-to-many relationships.

This would achieve the result you’re expecting and also behaving like a QuerySet, where you can call filter or other methods, like order_by, etc.

1 Like
5

Thanks. TBH, I removed my code that used through, because I kept running into related_name conflicts, and I found the property names that automatically got added to the model confusing—the Event model actually has relationships to a lot of other tables, including a direct relationship to a Profile. The way I coded it sidesteps all that confusion, but maybe it also prevents me from querying profiles through ProfileEventRelationships the way I want.

But I’m not sure it actually does. More or less the equivalent of what I want, in the Person/Group/Membership example you referred to, would be to add this to the Group model:

    @property
    def names_of_recent_members(self):
        return (m.name for m in self.members.filter(date_joined__gt=date(2025, 1, 1)))

Again, this returns a generator, which is problematic.

Of course I wouldn’t do it that way for this example; I’d just query through members and do what I want with the name property. But to make my templates less messy, I’m trying to write a property that returns models, not strings.

To make the example more relevant, let’s say the Person model also has this:

proficiency = models.ForeignKey(
    Instrument,
    related_name="people_who_play_this",
)

How would I write a proficiencies_of_recent_members(self) function for the Group model, that would return Instruments?

Anyhow, I’ve moved on by passing the generator to a list() constructor and returning that. As I said, shouldn’t cause any issues with performance or memory in this case, but I was wondering whether there’s a better way.

Thanks.

6

Yes, there is.

Use the ORM as it’s designed. Create your ManyToMany relationships and leverage the related object managers that are provided for you.

The related managers do that.

Again, unnecessary. The related managers with filters do that for you without the need of an additional, manually-created property.

And to continue along this line of thought…

You don’t. You access the members through the related objects.

Assuming, Person is a model, related to Group through a many-to-many field on Person named groups.

For an instance of Group named group, then group.person_set.all() is an expression yielding all instances of Person related to group. If you then iterate over that set, e.g. for person in group.person_set.all(), then person.proficiency gives you the instance of Instrument related to that person.

All this is handled “automatically” by the Django ORM. As a performance enhancement, you can use the select_related and prefetch_related functions as appropriate for the queries to be executed all at once. Without those functions, multiple queries may be generated to retrieve the data as needed - leading to potential N+1 query situations and the corresponding performance implications.

7

select_related seems to help a lot. Thanks.

But:

Doesn’t that put a lot of business logic in the templates? And since I need this in multiple views, lead to code repetition?

8

If you find yourself repeating it too much, you can write a custom manager, for example.
Or handle the duplicate by creating a method that returns a QuerySet.

Honestly, for me, the lesser the template size, the better. I tend to do most of the jobs on the view. And providing a “ready” context for the view, so the template only consume it

9

To answer your other question and to add to @leandrodesouzadev 's answer:

Keep in mind that this is a potential performance enhancement only. It’s never required.

There’s no “business logic” involved in the templates with this. You’ve defined the query in the view. The template is only iterating over the data structures.

10

… which gets us back to my original question, whether there’s a way to wrap a QuerySet around a generator. There doesn’t seem to be an easy way, so I’ll just stick to returning a sequence.

For me, too, hence the @property.

Noted—definitely overkill for this case, but good to know. Thanks.

In this case, I’m actually putting the business logic in the model, since I need it in multiple views. It seems the way I’ve done it is fine.

Thanks, both of you, for your help.

11

You started out by effectively asking what the “most Djangoish” way of handling this would be. (More specifically, you asked “what’s the right way to do this in Django.”)

The root answer to that is to actually use Django, and not write code that replicates built-in Django functionality - and in a less efficient manner as well.

There’s no easy way, because it’s unnecessary. You’re intentionally making things harder for yourself by trying to do it this way.

12

Fair, but I’m still not seeing how to do that in this case. Hopefully, if you can tell me more explicitly, I’ll see whatever I’m missing.

Is there a way to improve on iterating through filter(relationship_type=ProfileEventRelationship.RelationshipType.PROMOTER) and building a sequence, for the views to pass on to templates?

I could write a service function for views to pass Event instances to, and have that call something like ProfileEventRelationship.objects.filter(event=event, relationship_type=ProfileEventRelationship.RelationshipType.PROMOTER). But then I’d still end up with ProfileEventRelationships, instead of the Profiles I’m actually interested in. So either the views have to build sequences of Profiles for the templates, just like my model code currently does, or the templates need to iterate through the ProfileEventRelationships, instead of the Profiles themselves, making them bigger and fuss about details that seem beyond their scope.

13

A proper solution for this requires a more “holistic” approach - looking at the view as a whole, and not focusing on one specific aspect of it.

I would need to see the complete models involved, the current view, and the template you’re trying to render to understand what all is specifically needed.

14

It’s pretty simple, actually: I have multiple views that show events, and in most of them, I’ll need to display the names of the people who promote them, and for the names to link to their profile pages. Those data are in the Profile model, so all the templates need are sequences of Profile instances. Everything else that’s relevant is in the code I posted.

But while going for a walk to clear my head, I realised that I’ve been going about this backwards. It’s Profiles I want, so it’s Profiles that I should query. D’oh!

So, off the top of my head, maybe something like:

event_promoter_relationships = ProfileEventRelationship.objects.filter(event=event, relationship_type=ProfileEventRelationship.RelationshipType.PROMOTER)
promoters = Profile.objects.filter(event_relationships__in=event_promoter_relationships)

… although this is probably better done with django.db.models.Q—I’ll have another look in the morning. But either way avoids needing to create a generator or a whole new sequence.

15

This can still be condensed:

promoters = Profile.objects.filter(
    event_relationships__relationship_type='P',
    event_relationships__event=event
)
16

Wouldn’t that also fetch profiles that have different types of relationships for the same event, but who are also promoters for other events?

17

Also, do we need to explicitly compare with the value, 'P', instead of using the TextChoices class? I try to avoid magic numbers, or magic letters in this case, for the sake of clarity and consistency, but if there’s a reason to, it’d be good to know. Thanks.

18

That’s just me being lazy. Any Python valid value would work here. (The value is passed through to the SQL statement.)

19

It shouldn’t. It should only return those Profile objects matching both conditions. (Multiple conditions within a filter are joined by AND, not OR.)

You can examine the SQL statement being issued by printing the query attribute of the queryset to verify that the correct query is being generated.

20

I think that the point that you’re missing is the need to turn a QuerySet into a sequence, that is already done when you iterate over it, so there’s almost no use cases where you want to turn a QuerySet into a generator just for the sake of iterating over it after. When you do that, you lose all the good methods that QuerySet provides, thats why you’re missing them when you need it.

Well, you don’t need to actually do that in your model, although a lot of folks tend to do that, I don’t like this approach because when you look at a property, its accessed like an attribute, but it actually does more than attribute access, it does a database query, or many of them. The way I approach this problem depends on the actual project, and how things are separated. But one thing that I’ve done in the past, and it worked for me, was the concept of having selectors, a python file under the application directory that contains functions that returns querysets (or iterator, iterables), the main benefit is when you have actual important and crucial business logic that you don’t want to repeat, and mess up. But I only do that when I need to.
Most of the times, we as developers tend to try guess the future, but we’re really poor at it, and we think that something is going to be used across several parts of the system, actually are just twice or more. So in this case, my recommendation is to create a method on the same level where it’s being used, i.e views.py. If you ever want to use that same logic elsewhere, then you can move it to a custom objects Manager class or other approach.

One thing that I always regret doing, is letting templates know to much stuff about the models, when I do that is hard to refactor, or find references to code that is used by templates. But that’s just my feeling.