Model field for Form Field

Using Django Using the ORM
1

I have a class Event with a field label for which I defined a form SingleEventForm. This form redefines the model field label:

label = ModelMultipleChoiceField(queryset=Label.objects.filter(edge__target_node=self.request.user.node_id))

I want to store a list of ids in the label field of Event so that when I have a given id from Label I can retrieve events from the database where the list in the label field contains that id.

qs = Event.objects.filter(label__contains=id)

I’d like to know if this is possible in Django, and if so, model field I should use to store the list of ids.

Kind regards,

Johanna

2

You can do this using either the PostgreSQL ArrayField or a JsonField - but I strongly recommend against doing either.

Referential integrity is not enforced in either case.

Structurally, what you are describing is a Many-to-Many relationship between Event and Label, and so that’s how I would recommend it be modelled.

3

Hi Ken,

Thanks for your reply.

I know, however, the Label class belongs to an app called edges, and the Event class and other classes I want to have a label field, all belong to their own app. Making it Many-to-Many relationships would result in intermediary join tables between apps.

Since, the relationship between Label and the other classes having a label field will be very loose, i.e. retrieve a label, retrieve the events with that label’s id in the label list, and given that Events have a relatively short lifespan, I am not sure referential integrity will be an issue.

What do you think?

4

I always start from the “purist” perspective when modeling data in a relational database environment. I never recommend ignoring data integrity as a starting point.
I have generally found over the past 37 years that it’s easier to denormalize when necessary than it is to renormalize to resolve data issues.

I don’t see where that is a “problem”. It’s very common to do when linking models between Django apps. (For example, one common way to extend the User model in the auth app is to create a Profile model in your own app - creating a link between apps.)

1 Like
5

You convinced me :slight_smile:

I hope you don’t mind me asking, but I’ve two other many-to-many cases which I didn’t implement as such. Maybe you could give me your thoughts on them.

The first case is a table containing tags and a table containing nodes. Since tags don’t change, they just sort of tag nodes, like ‘web development’, ‘Django’, ‘Python’ etc. I store them in a table NodeTag with the following fields, where tag is an autocomplete on the tag table.

node = models.ForeignKey(Node, on_delete=models.CASCADE, related_name='node_tags', related_query_name='node_tag')
tag = models.CharField(max_length=64)
order_by = models.PositiveSmallIntegerField(choices=ordinal_choices(), default=1)

The second case is a table containing ads and a table containing spaces with a third table AdSpace storing the relationship between ads and spaces. AdSpace stores all the data related to that relationship.

Would you model these two cases as many-to-many relationships? Where would the ManyToManyField go in both cases?

6

Yes, I would model both as a ManyToManyField.

Since a M2M is symmetric across the join-table that gets created, it really doesn’t matter on which side you define it. As a result, I would tend to create the field in the model where most of the activity concerning that link would go.

Therefore, in your first case, I would most likely defined a tags field in Node as a ManyToManyField to Tag.

(Mentally, I tend to think of this operation as “assigning a tag to a node”, so that the operation is being performed on the Node and not the Tag - recognizing that physically, the operation isn’t being performed on either of them.)

I’m not sure I understand what the relationships are between ads, spaces and AdSpace to offer a firm suggestion there - but very superficially it seems to me that you’re assigning an ad to one or more space, and so my gut reaction would be to define that relationship in the model defining the space. But again, it really doesn’t matter much - you have equal access to that link from both sides.

7

Hi Ken,

Thanks for your advise, I spent the day adjusting my models accordingly.

I only read the reference on many-to-many fields in the Django documentation, today I followed the links and especially the topics guides were very helpful.

I work alone, hence I very much appreciate your sharing your thoughts.

8

Hi Ken,

I am struggling a bit with the querysets. I hope you can provide me with an example to get me started.

In models.py I have the following classes:

class Node(models.Model):
    full_name = models.CharField()

class Address(models.Model):
    node = models.ForeignKey(Node)
    locality = models.CharField()

class Set(models.Model):
    name = models.CharField()
    node = models.ForeignKey(Node)
    members = models.ManyToManyField(Node, through='Member')

class Member(models.Model):
    node = models.ForeignKey(Node)
    set = models.ForeignKey(Set)

In a class-based view I have the following view:

class MemberListView(PermissionRequiredMixin, ListView):
    permission_required = ('sets.view_set', 'sets.view_member',)
    context_object_name = 'member_list'
    template_name = 'sets/member_list.html'

    def get_queryset(self):
        queryset = models.Set.objects.filter(node_id=self.request.user.node_id)
        return queryset

This gives me all the sets created by a user c.q. node. What I want is a list with the full_name and locality of the nodes who are members of these sets. In SQL it would look something like this:

SELECT Set.id, Set.name, Node.id, Node.full_name, Address.locality
FROM Set, Member, Node, Address
JOIN Member ON Set.id = Member.set_id
JOIN Node ON Member.node_id = Node.id
JOIN Address ON Node.id = Address.node_id
WHERE Set.node_id = user.node_id

Kind regards,

Johanna

9

You’ve got two options, and it depends upon what you want your output to be.

The issue is that Address is a many-to-one relationship with Node. This means you need to decide whether you want one Node object being retrieved and then iterate over the related Address objects, or, you want to retrieve all of the Address objects and have the Node objects instantiated for each one (creating multiple copies of the Node objects.

Your choice between these two options depends upon what you’re going to do with the results of the query.

(And for clarity, you need to make this decision even if you currently only have 1 Address per Node because the existance of this relationship fundamentally affects what the query can return.)

Since this is a ListView, it would actually be more helpful to me if you described what your desired output will look like - what is the data you are presenting?

Side note - I’m also confused by the existance of both the node and members fields in Set that both relate to Node.

10

I want the output to be a list of nodes and display their full_name, locality and set_name;

Millstream, Bosham, SetA
Bodleain, Oxford, SetB
Dolphin & Anchor, Chichester, SetA

At the moment the relationship between Node and Address in practice is a one-to-one relationship. I implemented it as a one-to-many because I am not sure whether the one-to-one is the right choice.

The node field references the owner of the set, the members field references the nodes in the set. A node can own many sets, but a set is owned by one node. Sets can have many nodes and nodes can belong to many sets.

I hope I’ve provided you with sufficient information, to help me get my head around querysets in this case.

11

I’m still not sure I’m understanding what you’re trying to get to here - but that’s just because I don’t understand what you’re trying to model.

Let’s follow this along though and see where we end up.

You start out by saying you want a list of Nodes.

That starts us out with:
node_list = Nodes.objects.all() (or filtered as necessary - the difference is not relevent here)

So in your template, you want to iterate over these nodes and for each node you could have multiple addresses and multiple sets.

Does this mean you might want something like:

Node 1 full name, Address 1 locality, Set 1 name
Node 1 full name, Address 2 locality, Set 1 name
Node 1 full name, Address 3 locality, Set 1 name
Node 1 full name, Address 1 locality, Set 2 name
Node 1 full name, Address 2 locality, Set 2 name
Node 1 full name, Address 3 locality, Set 2 name
Node 1 full name, Address 1 locality, Set 3 name
Node 1 full name, Address 2 locality, Set 3 name
Node 1 full name, Address 3 locality, Set 3 name
Node 2 full name, Address 1 locality, Set 1 name
Node 2 full name, Address 2 locality, Set 1 name
Node 2 full name, Address 3 locality, Set 1 name
Node 2 full name, Address 1 locality, Set 2 name
Node 2 full name, Address 2 locality, Set 2 name
Node 2 full name, Address 3 locality, Set 2 name
Node 2 full name, Address 1 locality, Set 3 name
Node 2 full name, Address 2 locality, Set 3 name
Node 2 full name, Address 3 locality, Set 3 name

This is what the cardinality of your relationships imply, but it doesn’t seem to me like this is something you’d really want - and that’s why it’s raising so many questions in my head.

12

Thanks for bearing with me.

What I want is something like:

Node 1 full name, Address 1 locality, Set 1 name
Node 2 full name, Address 2 locality, Set 1 name
Node 3 full name, Address 3 locality, Set 2 name
Node 4 full name, Address 4 locality, Set 3 name 
Node 5 full name, Address 5 locality, Set 3 name

I don’t want to start out with nodes but with sets, if that’s possible.
If Node 0 owns Set 1, Set 2 and Set 3, then I think I start out with:

node_list = Set.objects.filter(node_id=0)

Then, I want the members of Set 1, Set 2 and Set 3, and to display them in a meaningful way, I want the member’s full_name and locality.

It is sort of a labeled network, a way to model this would be:

class Node(models.Model):
    full_name = models.CharField()

class Label(models.Model):
    node = models.ForeignKey(Node)
    name = models.CharField()

class Edge(models.Model):
    source_node = models.ForeignKey(Node)
    target_node = models.ForeignKey(Node)
    label = models.ForeignKey(Label)

Modelling it as a many-to-many relationship as a Set prevents me from repeating the node in class Label as source_node in class Edge, I think.

13

It’s possible - again, the difficulty here is with me trying to understand what you’re looking to produce.

That helps a lot.

So let’s start with:
set_list = Set.objects.filter(node_id=1)
(Standard Django table construction is not going to generate a pk of 0.)

So then, for each instance of set within set_list, you want to iterate over members and then within each instance of Node within members, you want to iterate over each instance of Address within address_set.

This means your template will do the following:
(Note, this is written in pythonish rather than the template language for clarity - I hope)

for a_set in set_list:
    for member in a_set.members:  # `member` is an object of type `Node`
        for address in member.address_set:  # Using the intrinsic related-object manager
            print(member.fullname, address.locality, a_set.name)
14

Hi Ken,

Yes, that’s exactly what I want. That said, how do I get my template to do exactly that, and what documentation or book should I dive into to get my head around Django’s way of building querysets.

It’s all about objects, isn’t it? Is there a resource you can recommend to learn more about this aspect of Django? Years ago, I learned OO-programming in Java, but that’s of little help here.

Looking forward to your reply,

15

It’s actually going to look quite close to this. Assuming you pass set_list in to the context by the same name, you’ll end up with something like this:

{% for a_set in set_list %}
  {% for member in a_set.members %}
    {% for address in member.address_set %}
      {{ member.fullname }} {{ address.locality }} {{ a_set.name }}
    {% endfor %}
  {% endfor %}
{% endfor %}

(You still need to work in whatever other html you want to use to create the page, but this shows how to render the data)

More than anything else, I believe it just takes time and practice. My experience has been that at some point it just starts to “click”.

There are some really good resources out there - you can check out GitHub - wsvincent/awesome-django: A curated list of awesome things related to Django to get some ideas of places to look. I tend to avoid making specific recommendations because “learning” is such an individual experience, and a resource that works for one individual may not work for another.

Yes, … and no.

Yes, a solid understanding of Python, Python syntax, and how Python classes work is extremely useful - I’d almost say critical once you get beyond the beginning levels of Django. There’s a lot of what Django does internally that relies upon some very sophisticated Python code.

And no, it’s not just about objects. It’s also about wrapping your head around the concepts of how those objects map to a relational database.
(Side note: There are a number of different ways that you can create an “Object-Relational Mapping” (ORM), Django’s way is just one of them.)

That depends upon how deeply you got into it. If you got far enough into it to work with a Java-based ORM such as hibernate, the concepts would transfer to a significant degree.

Fundamentally, an object is the combination of some data and the code that operates on that data. So understanding objects is an issue of understanding what attributes (data or methods) that are available to other objects to use. Everything else is just details…

16

This is an aspect of Django I don’t understand. What I think happens is:

set_list = Set.objects.filter(node_id=1)

Queries the database for all sets related to node 1

{% for address in member.address_set %}

Queries the database for all members of set 1

{% for address in member.address_set %}

Queries the database for each address of a each member.

Isn’t it inefficient to query the database for each set of members and each address. Isn’t it more efficient to build a queryset that passes all matching records into the context at once?

I got started with Django after Harvard’s CS50W course. I bought the three books William Vincent wrote, but somehow missed this GitHub resource, it’s very helpful.

Then Harvard’s CS50P will be time well spent.

Django is my first encounter with an ORM. Working with Java I didn’t get any further than the basic concepts and JSPs and Servlets.

Thanks for your extensive reply. I think it’s getting me to the point it starts to click, as you said.

17

It would be if it were possible. But because of your database relationships, you’re not querying for “objects” - you’re querying for related sets, and those don’t map directly to objects. (If it is a significant enough issue, you could create a database view with a corresponding Django model to access that view, but that creates other potential issues of its own.)

Now, as a matter of practical performance improvements, you can reduce the number of queries being made using the prefetch_related related clause in your query.
I did not mention this earlier because it is only a performance enhancement - there is no requirement it be used. (Some people have gotten the impression that using it is necessary in order to retrieve related objects - but it’s not.)

I’ll also point out that it’s not a panacea. Depending upon how your queries are constructed and how those querysets end up being used, it’s possible that using it makes things worse.
What happens is that using prefetch_related causes those related queries to be made immediately after the base query is executed and the results are cached. Certain operations can cause that cache to be ignored and requires the queryset to re-execute those queries. So instead of reducing the number of queries, it increases that number.

What you would want to do is track the number of queries being executed by a view to see if it helps. The django-debug-toolbar is a great tool to help with that.

18

After our discussion here, a lot of thoughts are going through my head. I reconsidered my models from an OO perspective instead of a relational perspective, which lead to some adjustments.

Looking at my project structure, I wonder whether it is possible to put my apps in a sub directory. Given the following structure:

project
|__ accounts
|__ app1
|__ app2
|__ app3
|__ config
|    |__ __init__.py
|    |__ settings.py
|    |__ urls.py
|    |__ wsgi.py
|__ core1
|__ core2
|__ core3
|__ ext1
|__ ext2
|__ ext3
|__ media
|__ static
|__ templates

Is it possible to put all the apps in a directory called apps like this:

project
|__ accounts
|__ apps
|    |__ app1
|    |__ app2
|    |__ app3
|    |__ core1
|    |__ core2
|    |__ core3
|    |__ ext1
|    |__ ext2
|    |__ ext3
|__ config
|    |__ __init__.py
|    |__ settings.py
|    |__ urls.py
|    |__ wsgi.py
|__ media
|__ static
|__ templates

If so, will this have consequences for Django’s naming conveniences.

19

Is it possible? I’m sure it is. Is it “seamless”, “transparent” or “easy”? Not that I can tell from the various threads here and elsewhere from people who have done it. There are a lot of side-effects you will need to account for by doing it.

Personally, I’ve never tried to do it. I’ve never seen a value in attempting it.