Django postgres migration causing authentication errors

Using Django Mystery Errors
1

I’m encountering an authentication issue with a Django multi-tenant application using JWT tokens.

The Issue:

  • Trying to restore a postgres pg_dump from a prod to local environment using pg_restore
  • Database restores work fine within the same environment (prod→prod or local→local)
  • When restoring prod database to local, users exist in the correct tenant schema (‘wwm’)
  • However, authentication attempts return 401 Unauthorized for /api/users/token/
  • Token refresh attempts return Bad Request for /api/users/token/refresh/
  • When I hit an authenticated endpoint after the pg_restore the backend always returns “{
    “detail”: “User not found”,
    “code”: “user_not_found”
    }”

Setup:

  • Django with SimpleJWT for authentication
  • PostgreSQL 16 in Docker containers
  • The data is persisted in volumes hence on the host OS
  • Both the dev and prod environment are docker compose networks with the only difference being that volumes are used to persist the postgres data in between docker-compose up and downs.
  • The host OS and local OS are different (Linux and MacOS respectively)
  • Custom user model (users_baseuser)
  • DRF
  • Multi-tenant setup using schema-based tenancy and a custom middleware for tenant selection

Database Investigation:

  • Users are present in the tenant schema after restore
  • Query SET search_path TO wwm; SELECT * FROM users_baseuser; shows users exist

Current Backup/Restore Process:

# Backup
docker exec $CONTAINER pg_dump -U $DB_USER -Fc -d $DB_NAME > "$BACKUP_FILE"

# Restore
docker exec -i $CONTAINER pg_restore -U $DB_USER -Fc -d $DB_NAME -c --if-exists

I have all containers running when I run the restore command, this doesn’t cause any errors when done (prod→prod or local→local).

Has anyone encountered similar issues with authentication after database restores in different environments? This has kept me busy for a couple of days and I’m at a loss as to what it could be seeing as the backup and restore works perfectly within the same environment.

Any insights would be appreciated!

2

What JWT token library are you using?

Is that library using the Django SECRET_KEY setting as part of its digital signature?

If so, then my guess would be that the authentication is failing because of different SECRET_KEY settings being used - you would want to regenerate the tokens when moving data between environments.

4

Im using auth_classes = [“rest_framework_simplejwt.authentication.JWTAuthentication”], even when using the same secret key the error persists. Further my intuition was that re-submitting credentials for an account in another environment should still work to generate a new token?

5

There are multiple ways of handling tokens - that why I needed to ask the questions, to rule out issues like this.

6

Yes of course, I was more so just double checking my own understanding about if the copying over of stale tokens from another environment could prevent the creation of new ones.

1 Like
7

I believe you are correct. If you’re issuing standard credentials (username, password) and the system is generating a new token with every passing of those credentials, then that should allow for the creation of a new token. Additionally, the transfer of the SECRET_KEY between the environments should also prevent this from being a problem.

However, this is about as far as my knowledge of that particular library goes. I hope someone else with more knowledge is able to jump in to help.

8

Finally found the culprit. In case anyone stumbles across a similar issue it’s to do with multi-tenancy. I was using a custom middleware to resolve tenants from the hostname, my public tenant had a table that mapped hostnames to tenants. Copying across this table caused the bug as my local environment was running on localhost and hence the hostname ended with *.localhost vs *.my-actual-domain. This threw off the dict resolving tenants from request hostnames and my default public tenant has no user table data – hence auth broke. Thanks @KenWhitesell for you’re time, very much appreciated.