django-valkey: a featureful backend for valkey cache system

Show & Tell
1

django-valkey is a backend that i’ve been working on for the past few days.
it was initially a fork of the wonderful django-redis project.
since the fork:

  1. added two new compressors (bz2 and brotli),
  2. added customization ability for compressors,
  3. reworked the documentation and deployed on readthedocs,
  4. moved some codes around to be more clean/efficient

i’m also thinking about adding an async client, theres already a branch (but no commits yet)
and providing access to cluster clients that underlyig valkey-py library provides.

so this is probably gonna stay a hobby project unless there is a breakthrough in the valkey world, but if anyone has aby feedback/ideas or wants to contribute to a cache project please feel welcome to start an issue

4 Likes
2

I think that breakthrough is coming with AWS & GCP supporting valkey now.

I’m just looking at switching from a redis instance to a valkey cluster.

What I’m not finding is much information on how to use a cache cluster. So at a real basic example, we’ve got a middleware that checks it can use cache.

        # Check if redis cache is alive by setting an arbitrary key
        try:
            caches["redis"].get("HealthCheckMiddleware.readiness")
        except Exception as e:
            logger.exception(e)
            return HttpResponseServerError("cache: cannot connect to cache.")

With a cache cluster this fails. Running with a local docker compose cluster I’m seeing;

Cannot connect to redis://cache:6379/2: SELECT is not allowed in cluster mode.

If you could, perhaps some example usage for more advanced usage would be good for people. Beyond the basic set/get commands that people would be familiar with working with a single cache instance.

(and for the record, I’ve kept the “redis” alias but this is using valkey underneath)

3

hi there

there is an experimental cluster backend available at django_valkey.cluster_cache.cache.ClusterValkeyCache

and it’s client is django_valkey.cluster_cache.client.DefaultClusterClient

but it’s not officially released.
at the moment i’m struggling with a broken CI, if anyone is intereseted in helping there is a CI branch you could look.

until the CI is fixed and i can make sure the package is healthy, i can’t just push all this to stable version.

also we have documentations, yes cluster is not included cause it’s not meant to be used really but here’s a sample setup, note that client is optional and you don’t need to specify it

but if you see the docs lacking please do notify me

1 Like
4

Thanks @amirreza8002 - Since posting I learned that redis-py only got cluster support last year and django-redis still doesn’t have it as I understand, so appreciate the position you’re in with clusters now better than I did when I posted the above!

I’ll switch to that backend and see if I can take a look at your CI

5

hi there!

the main branch just got a ci update today.
the normal tests are running fine now, tho i had to let go of testing unix sockets on github actions for now until i find what was wrong with it , till then, socket connection will only be tested locally.

i’ll be making some big changes to cluster branch, since that branch was made some time ago and is not updated.
if you want to work on it i suggest waiting a bit until i take care of rebasing and merging and such.

2 Likes