Changing SESSION_ENGINE from cache to cached_db

Using Django Deployment
1

Hi all,

I’ve encountered a situation where I need to migrate my cache to another instance. The difficult part is keeping sessions alive during the transition. While I wait on the provider to get back to me about transitioning the data, I explored backing it up myself to the database via cached_db. From what I can tell, this transition while maintaining sessions isn’t supported by Django out of the box.

Has anyone else attempted this and what was your experience?

2

Can’t say I’ve done such a migration, but the migration should be feasible though. The data should be copyable into the database table, since it’s mainly a pickle blob and a little metadata.

If you pull it off, I’d be interested to read a bit how.

Re-reading the docs’ “cache vs cached_db”, I wish it was more firm about using cached_db. It’s rarely a good idea to forgo session persistence, as you’re experiencing!

3

Thanks Adam. The slight deference to cache is likely why it was picked. I have a solution, but I’m waiting on deploying it because if I got it wrong, I’m breaking a lot of sessions.

Since the key_salt property has some name mangling, I ended up reimplementing the cached_db.SessionStore so that the key prefix would match and the hashing/signing would work properly.

I added three functions:


from django.conf import settings
from django.contrib.sessions.backends import cache, db
from django.core.cache import caches
from django.db import router, transaction


class SessionStore(db.SessionStore, cache.SessionStore):
    ...
    #
    def save_to_db(self, cache_key, session_key):
        """Helper function to forcibly store cache data to the database."""
        try:
            data = self._cache.get(cache_key)
        except Exception:
            pass
        else:
            obj = self.model(
                session_key=session_key,
                session_data=self.encode(data),
                expire_date=self.get_expiry_date(),
            )
            using = router.db_for_write(self.model, instance=obj)
            with transaction.atomic(using=using):
                obj.save(using=using)

def get_all_cache_keys():
    pattern = caches["default"].make_key(
        SessionStore.cache_key_prefix + "*", version=None
    )
    for key in caches["default"].master_client.scan_iter(match=pattern):
        key = key.decode("utf-8")
        yield key.replace(":1:django", "django"), key.split(
            SessionStore.cache_key_prefix
        )[1]


def save_to_db():
    for cache_key, session_key in get_all_cache_keys():
        store = SessionStore(session_key=session_key)
        store.save_to_db(cache_key, session_key)

The deployment flow is:

  1. Deploy updated SessionStore
  2. Run save_to_db() in a shell.
  3. Confirm the data has been saved via SessionStore.get_model_class().objects.count() > 0
  4. Switch the cache in the settings.
  5. Either wait 2 weeks, then remove the cache_db logic from SessionStore, or more likely run more code that moves the data from the db into the cache.
4

I’m curious, what cacheing backend engine are you using? For example, if you’re using redis, could you set up a redis replica and let it duplicate the data to the new host?

5

I’m using Redis Labs cloud offering. I opened a ticket with them to see if they have any other options that I couldn’t find.

6

Nice work, and good luck!

Redis replication may indeed work, and at least Redis persists to disk so your sessions weren’t at so much risk.

I made a ticket about the docs: #33797 (Reword session docs on prioritizing "cache" vs "cached_db") – Django . Input and PR’s welcome.

7

Well this is a bit embarrassing. The passive-active replication / Replica-Of option is clearly on the edit database portion of the site. I simply missed it and used the wrong search terms to find it.