Session lost when using django.contrib.sessions.backends.file

Using Django Mystery Errors
1

Hello there,

I have an intermittent issue with losing session data when using “django.contrib.sessions.backends.file” as the session engine (Python 3.11.6 and Django 4.2.8 on Windows 10).

Consider the following view:

from django.http import HttpResponse
import time
# Create your views here.
def index(request):
    counter_sess_key = "counter"
    # init
    if counter_sess_key not in request.session:
        request.session[counter_sess_key] = 0
    # increment
    request.session[counter_sess_key] += 1
    # sleep
    time.sleep(0.100)
    # get
    counter_val = request.session[counter_sess_key]
    # display
    return HttpResponse(
        f"Hello, world. You're at the polls index. Counter = {counter_val}")

And the following additional configuration in settings.py:

SESSION_ENGINE = "django.contrib.sessions.backends.file"
SESSION_FILE_PATH = BASE_DIR / 'sess'
SESSION_SAVE_EVERY_REQUEST = True

Using Firefox and holding Ctrl+R will generate a lot of GET requests the majority of which will end up with a “Broken pipe from…” message on the web server console. However, at times, the counter will reset to 0 (this happens before it reaches 200 usually on my system).

The use case here is when using multiple Ajax requests that arrive very close one after another.

This issue seems to exist when:

  • Running the development server as follows: “python manage.py runserver”
  • Also when running Apache 2.4.58 with mod_wsgi 4.9.2 and Django on Windows 10

The issue does not seem to exist when:

  • Running the development server: “python manage.py runserver --nothreading”
  • Using the default database session storage

Any ideas, suggestions or workarounds?
Many thanks!

2

Hi, you’re probably facing a race condition problem with session file writing not being atomic in windows (see django/django/contrib/sessions/backends/file.py at main · django/django · GitHub and related ticket #9084 (File-based session does not store any data on Windows) – Django).

Using the development server with --nothreading, the problem does not occur as only one request is being processed at a time, so no read/write concurrency occurs on session file. But in other cases, you may have a request reading session file while the other one is saving, at the moment shutil.copy2 opened and truncated the session file for writing but didn’t write to it yet, leading the reading request to see an empty session file and so, create a new session

3

Yes, that’s what I think so too.
I’ve created this test to highlight the issue a bit better:

import requests
from concurrent.futures import as_completed
from requests_futures.sessions import FuturesSession

def run_parallel_requests(url, max_requests = 30):
   with FuturesSession() as rsess:
      # run one first
      print(list(as_completed([rsess.get(url)]))[0].result()._content)
      # and then run the rest
      futures = [rsess.get(url) for i in range(max_requests-1)]
      for future in as_completed(futures):
         response = future.result()
         print(response._content)

url = "http://localhost:8000/mysess"
max_requests = 30
run_parallel_requests(url, max_requests)

Running with “django.contrib.sessions.backends.file” and --nothreading (works as expected):

>>> run_parallel_requests(url, max_requests)
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 4"
b"Hello, world. You're at the polls index. Counter = 5"
b"Hello, world. You're at the polls index. Counter = 6"
b"Hello, world. You're at the polls index. Counter = 7"
b"Hello, world. You're at the polls index. Counter = 8"
b"Hello, world. You're at the polls index. Counter = 9"
b"Hello, world. You're at the polls index. Counter = 10"
b"Hello, world. You're at the polls index. Counter = 11"
b"Hello, world. You're at the polls index. Counter = 12"
b"Hello, world. You're at the polls index. Counter = 13"
b"Hello, world. You're at the polls index. Counter = 14"
b"Hello, world. You're at the polls index. Counter = 15"
b"Hello, world. You're at the polls index. Counter = 16"
b"Hello, world. You're at the polls index. Counter = 17"
b"Hello, world. You're at the polls index. Counter = 18"
b"Hello, world. You're at the polls index. Counter = 19"
b"Hello, world. You're at the polls index. Counter = 20"
b"Hello, world. You're at the polls index. Counter = 21"
b"Hello, world. You're at the polls index. Counter = 22"
b"Hello, world. You're at the polls index. Counter = 23"
b"Hello, world. You're at the polls index. Counter = 24"
b"Hello, world. You're at the polls index. Counter = 25"
b"Hello, world. You're at the polls index. Counter = 26"
b"Hello, world. You're at the polls index. Counter = 27"
b"Hello, world. You're at the polls index. Counter = 28"
b"Hello, world. You're at the polls index. Counter = 29"
b"Hello, world. You're at the polls index. Counter = 30"

And without --nothreading (does not work as expected):

>>> run_parallel_requests(url, max_requests)
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 3"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 1"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 2"
b"Hello, world. You're at the polls index. Counter = 2"

I am going to try it on Linux shortly.