Help with caching related objects

Using Django Using the ORM
1

Hi everyone,

Recently I have been tasked to investigate caching solutions for some of our queries at work and I think I found a roadblock that I am hoping someone here can shed some light at.

Consider this scenario:

I have two models: MyModelA and MyModelB.

class MyModelB(models.Model):
   ...
   model_a = models.ForeignKey("MyModelA", models.CASCADE)
   ...

class MyModelA(models.Model):
   ...

Our system access MyModelA a lot, sometimes by itself (MyModelA.objects.get() (or filter())), sometimes via other models as in the MyModelB case. This obviously leads to thousands of calls that translate to SELECT * from MyModelA where id = X in the database.

My first instinct was to implement a CachedQuerySet class and override get so to cache MyModelA using the low-level cache api. So far, so good. The amount of calls to SELECT * from MyModelA where id = X was reduced, but it is still high.

I did a little digging and find out that accessing a child object of MyModelA will indeed also generate the SELECT calls. And guess what? There are a few places within my codebase where we do:

my_model_a = my_model_b.model_a

From here, I am not sure what would be the best way to go. I did think about a couple of options:

  • Should I write a method get_model_a inside MyModelB which implements the same caching logic that I used for my CachedQuerySet? I know it would work BUT I would have to potentially write similar methods in other models.

  • Should I try to override getattr? (It seems a bit too much tbh, as if I was forcing something into Django and Python)

Or, alternatively, has anyone here faced a similar problem and knows a better or more appropriate solution?

Thanks in advance,
Adriano

2

For this exact situation:

I believe it would be benefical for you to add a select_related stament to your queryset, this would reduce 1 query, since the MyModelA would be joined with MyModelB on 1 query. And this would also work for the cache as well, if you cache a model instance that has already fetched the MyModelA instance, the instance would also be pickled and available when you get that from the cache.

1 Like
3

Hey @leandrodesouzadev
Thanks for that! You are right, I did a bit of more digging and the select_related will defs help in this case.
I will have to run more tests to see if that will get rid of the remaining SELECT calls or if something else needs to be done :slight_smile: