Trying to debug DB issues in flaky frontend tests

Using Django Mystery Errors
1

Im running a larger Django app that has a test suite that runs both on Debian Bookworm and Trixie. The tests use the test_server fixture and sqlite. We’re currently experimenting with HTMX based views, which we test using playwright.

A few weeks ago the test covering our new HTMX view started occasionally failing, but only ever on Trixie. The only relevant packages provided by the Debian Trixie host are sqlite 3.46 and python 3.13. Everything else comes in via pip and pyproject.toml in the same versions as on Bookworm.

The test does not fail reliably and when it does fail, it doesn’t always fail with the same error. The only commonality is that it always fails while trying to get the current session, in order to authenticate the request user (the tested view has an auth decorator).

The error messages include

  • django.contrib.auth.models.User.MultipleObjectsReturned: get() returned more than one User -- it returned 2!
  • django/db/models/sql/compiler.py stepping out with List index out of range when trying to load the session cache
  • django.db.utils.InterfaceError: bad parameter or other API misuse when trying to get the session cache.

They mostly end up with AttributeError: 'SessionStore' object has no attribute '_session_cache' but there can be different issues causing the retrieval of the session cache to fail.

The way I see it, there is something going wrong with the database here. I am using tox and pytest, the database is reset for every test, the test suite has about two hundred other tests, all of which run fine.

So, all I know is there is an issue that only occurs when I use playwright to test a frontend that loads content using HTMX and only when I do so on Debian Trixie. When it happens, it’s always connected to Django wanting to retrieve session data and the database failing to provide them in some way.

I suspect a good next step would be to get more information on what that database does during the tests, or what errors it runs into, but I’m not sure how to get that info.

I’m aware this is a horribly convoluted issue, but I’m kind of at my wits end with it.

Would be very grateful for any hint or tips on what I could do to get more info to somehow get to the bottom of this.

2

Yeah, flaky tests are tricky to debug.

The ideal way to go about it would be (obviously) to have the test fail regularly. Have you tried refactoring your tests into more modular, to test less of the code and see if you can get it to reliably fail then?

One thing you can also try is rerunning the pytest tests multiple times on failure, from what I remember the syntax is something along the lines of @pytest.mark.flaky(reruns=3). You might need the pytest-rerunfailures package installed.

You can also enable playwright tracing to try to get more details on why the test fails when it does and what actions were made to cause the test to fail.

Finally, if you suspect something is really going wrong with the database, you can add some SQL monitoring, maybe a middleware? This way, you can look at the queries on passing and failing tests to see the differences and what’s really happening in the background.

While none of these are fixes, they are ways you can go about further debugging the flaky tests. I’m not sure anyone can give you an exact answer on this without additional data. Good luck. :slight_smile:

1 Like
3

Flaky tests can be tricky. Sometimes they can be caused by db-pollution.
Anthony (creator of several tools on the python ecosytem) has a very good video on how to debug them.

2 Likes