Cache invalidation/renewal options

I feel that Django is lacking some possibilities regarding cache renewal and indvalidation especially regarding {% cache %} templatetag.

There is ticket #5815 and I also created a ticket #35524 (Add possibility to renew {% cache %} templatetag) – Django for my use case regarding renewal of cache in {% cache %} templatetag.

What are the possibilities. Is there any chance that Django would implement some guidelines that would help dealing with such situations?

I think if we could solve the per-view invalidation issue in #5815, that would be a big step forward here.

IIRC correctly the issue is not being able to easily recreate the cache key for a given path, so there’s no simple API for clearing.

With that in place it should be simple enough (no?) to recache items, and do that in a background worker with a bit of stagger to avoid
stampedes :thinking:

When I am thinking about it now, I am not fully convinced, that per-cache invalidation is general enough solution.
What if I am having gallery of items and I want to invalidate cache for all pages containing some item?

Maybe there could be something like caching tags/identifiers that would be set with the key. Then path could be one of the identifiers.

Also for my use-case, anything regarding invalidation would probably not work. If I invalidate the cache the requests would hammer my DB.
The only solution I can use in the background worker (or on page with special parameter or in randomized case) is to renew the cache with new value.

But at the same time, my use case can be solve by the simple modification of the {% cache %} templatetag I suggested at #35524 (Add possibility to renew {% cache %} templatetag) – Django.

Maybe the solution to both cases could be just to implement function (possibly also decorator) renew_cache(fn) which would get a function fn (a view or anything else) as parameter and would execute the fn with modified caching backend which would treat all keys as expired and thus renew their values.

Not sure how it is possible and/or if it should be integral part of Django.

How would something like this work? Cache backends usually support a key and a value. Putting tags/identifiers into the value is no option since you would then have to iterate all keys to find items with a certain identifier. Encoding it into the key is not really an option either, depending on the backend they don’t support prefix queries etc…