Indexing one table against a foreign key's property?

I’m somewhat new to databases, and completely new to ORMs. I’m trying to design an “insurance claims portal” server, and I’d like to make sure that I’m approaching this correctly.

What I’m worried about is the case where, say, user A has 100 total claims, and wants to load the first page of their claims (up to 25, chronologically newest, owned by that user). I want to make sure that this event, which should occur on every user’s landing page after login, doesn’t inadvertently read and inspect potentially ~25% of all claims in the database.

I’d also really like to avoid denormalizing the data by storing a “duplicate” .policyholder column on each Claim when that’s already something that’s accessible from the .contract.policyholder.

What kind of index do I need to create for this? Will the indexes Django gives me automatically be sufficient to make this work reasonably well, or is there some specific index I should consider adding?

class Claim(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)

    class State(models.IntegerChoices):
        AWAITING_INSURER = 1, _("Awaiting Insurer")
        AWAITING_CLIENT = 2, _("Awaiting Client")
        CLOSED = 0, _("Closed")

    state = models.IntegerField(choices=State)

    # TODO: verify whether we need to add an index
    # to make Claim.objects.filter(contract__policyholder=user) efficient
    #
    # I suspect/hope it will be at least TOLERABLY efficient as-is:
    #
    #  * Django will(??) automatically create an index
    #    to efficiently look up Claim by Contract
    #    because of the contract ForeignKey field
    #
    #  * Looking up Contract by policyholder should also be efficient
    #    since that property has db_index=True, too?

    type = models.ForeignKey("policies.Policy", models.RESTRICT)
    contract = models.ForeignKey("policies.Contract", models.RESTRICT)

    class Meta:
        ordering = ["-created_at"]
class Contract(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)

    # the customer that "owns" the policy
    policyholder = models.ForeignKey(get_user_model(), db_index=True, on_delete=models.SET_NULL, null=True)

    endorsements = models.ManyToManyField("Policy", related_name="used_by")

    is_rider_on = models.ForeignKey("self", null=True, blank=True, related_name="rider_set", on_delete=models.CASCADE)

    effective_from = models.DateTimeField()
    effective_to = models.DateTimeField()

    class Meta:
        ordering = ["-created_at"]

Welcome @James_E_A !

This would be handled by the query that you write to populate the page.

You don’t.

A standard Foreign Key field creates an index automatically. And since that FK references the primary key of the related table, it has an index as well.

Additionally, Django automatically provides reference functions for related models. In this case, for any instance of a Contract, the set of Claim related to that instance is accessible through the Related object manager.

Using the models you’ve posted, if you have an instance of Contract named contract, then the set of Claim related to contract would be contract.claim_set.all()

(If you have worked your way through the official Django tutorial, you have already seen this in action in the Polls application.)

Side note: There is never a need to specify editable=False on a primary key.

As quoted in the docs for primary_key:

The primary key field is read-only. If you change the value of the primary key on an existing object and then save it, a new object will be created alongside the old one.

The ORM will not allow you to change the primary key of a row.

But I don’t exactly want to query “all the claims related to a particular contract”.

I want to query “all the claims related to any contract related to a particular user, in chronological order of the claims’ creation date.” How can I ensure that this query remains efficient to iterate over, even for a user who’s filed too many claims and/or owns too many contracts to fit into memory all at once?

I did that to ensure that the primary key wouldn’t have a field in the “admin” interface.

I have another table where I do want the default behavior, including the “record gets copied if saved with a different primary key” behavior, but I don’t want that behavior available on this table, where I believe it’d just be a footgun. Isn’t that a good reason to set editable=False?

Fundamentally, this distinction doesn’t matter - the indexes spanning these relationships exist, and it’s the database’s responsibility to handle it.

The query itself doesn’t change - how you manage that query is potentially a different issue, and depends upon what you mean by “iterate over” and what you consider “too many claims” to be.

For example, if you’re just processing data from within the query (summarizing it), you could use the aggregation methods to summarize the data within the query. Or, if you’re looking at rendering it to be displayed as web pages, you can use Pagination to fetch a subset for each page.

But in either case, the basic query would remain the same.

1 Like

My inclination would be to remove it from the fields made available in the ModelAdmin class.

That’s a valid opinion, just not one that I share. In this situation, my preference is to reflect that in the ModelAdmin class, not the Model.

Hi everyone.
Say. Where is this found?

claim_set

I (the first time) see it and i didn’t even see it to the doc. What is it?

With models.ForeignKey, Django will by default create a reverse property named like _set.

You can adjust this with the related_name parameter.

It’s mentioned in the docs at: https://docs.djangoproject.com/en/5.2/topics/db/queries/#backwards-related-objects

1 Like

It’s also explained in the tutorial part 2, and used in part 3 and part 4.

If you have not yet worked your way through the tutorial, I suggest you do so.

1 Like