I’ve had a case where a user has created duplicate database entries via a form. In this case matches in a board game tournament.
Is there a known good way to prevent this? I have a review page now, with the fields stored as session data, so by the time the user clicks the Create button, the form data has been validated. So perhaps I should disable the button via JavaScript.
But is there any way to prevent this server-side? Could it be the case that two instances of gunicorn serviced the form submission? In which case, I have no idea how to prevent the two processes from creating two entries in the database. There’s always the possibility that the same user actually wants multiple entries with the same data.
Soon, I’d like to implement creating all the matches in a tournament in one go via a CSV. In that case, duplicates would be extremely undesirable.
Here is my code:
The view above this one stores the form data to session storage. This function pulls that data out of session storage and uses it to create the match. Is the pop atomic?
Create database constraints. Validate that the submission doesn’t create a duplicate entry. Provide the option for a dialog box asking for confirmation in case that a duplicate entry would be created.
I don’t have any field that needs to be unique. It’s completely valid to have multiple matches with the same title, players, and settings. This might help others but not this situation.
I hesitate to kick this back to the user. Even then, how do I know it’s a duplicate if I have a race condition? If I check for uniqueness, it’s unique, and then I create the entry, and by then it’s not unique, then it’s already too late.
BTW, as far as I know, only this user has had this problem, and I can’t reproduce it locally. However, this same user has had this problem multiple times, both before and after I added the review/confirmation page.
Yes, but what about the combination of fields? You can define a constraint such that individually, “players”, “title”, and “settings” can all appear multiple times, but you can only have one combination of (“player”, “title”, “settings”). If multiple instances of that combination are valid, then what is the problem?
Someone needs to make a decision whether or not a duplicate is a problem. If it’s not a problem, then there was no need for this post. If there is a problem, then someone needs to decide whether or not it’s valid.
Fundamentally, the bigger problem is storing it in the session rather than in the database. Two different people are always going to have separate sessions. One person working with two different browsers is also going to have separate sessions. You can avoid a lot of these issues by having the entries created in the database immediately, with a status field of some kind to indicate whether or not it has been reviewed and approved.
I see that my statement was ambiguous. I don’t have any field or combination of fields that require uniqueness.
What I’m trying to avoid is someone clicking the Create button and getting two matches created. It would be valid for them to go back to the match creation form and input identical data into the fields and create an additional match.
I’m guessing you didn’t mean it this way, but I think “there was no need for this post” could be seen as rude.
My understanding is that session data is stored in the database (using the sessions.session model). It is per-session, so the same user will generally have per-browser session data. I think that’s fine.
If there’s a better way to pass the data to the next view without creating a match object, then I am definitely interested.
It’s difficult to know how to program a solution without knowing the mechanism of the problem. I haven’t been able to reproduce locally, so I am not sure how to debug. Maybe it’s because I’m running a single process locally, so there’s no way for this to happen. My local instance is usually sufficient to test, but there are differences. On my local machine, I’m using sqlite3, in-memory message passing, 1 daphne, etc. Deployed, I’m using PostgreSQL, Redis, multiple gunicorns + 1 daphne, etc.
If two processes are grabbing the same session data and creating two objects (I don’t know how else I’d be getting more than one object.), then which process is the one issuing the redirect response that the user’s browser sees?
If there’s a race condition, then I’m not sure that both processes would be able to reliably detect a duplicate. In which case, I can’t reliably communicate to the browser to ask whether this was intentional.
I also prefer not to have a new Match field that I now have to use as an additional filter on every query of Match objects.
If they’re stored in the database, then there is no race condition. The first one has submitted the data and the second encounters the duplication.
A custom manager or manager queryset addresses that issue.
Django provides these type of features precisely to allow a developer to avoid problems like this. It seems to me to be suboptimal to implement a solution instead of taking advantage of the features provided.
Have you tried double-clicking the submit button? Some users habitually double-click every button. In forms, that results in duplicate submissions a few milliseconds apart.
You can use client-side JavaScript to help prevent this (disable the submit button during the first submit so the second click is ignored). Or you can include a hidden field in the form specifically to detect duplicate submissions server side (a nonce, or maybe the timestamp the form was served). Or both.
Of course. I think I would need multiple processes serving those clicks to reproduce locally. I’ve started looking into being able to run multiple processes locally.
Yes, as noted before, I will try preventing this in JavaScript. I was partly wondering if this has been encountered by others when using Django, and whether there is a known solution.
Is it a known problem that double-clicking a form submit button double-submits the form? Yes, and that’s not unique to Django, and a popular known solution is to debounce form submission in JavaScript. (I think you probably already know this, but just stating it here for the record.)
I’m not aware of any known bugs with Django or gunicorn somehow processing a single browser request twice. That doesn’t mean there aren’t any, or that something like a misconfigured load balancer might cause that. But the far more likely explanation is the user is double-clicking the submit button. (It’s definitely a known issue that some users tend to double-click everything.)
To debounce server-side, you’ll need to persist some de-duplicating criteria somewhere. Since you’ve said that the same user should be able to create multiple instances of the same match data, I’d think something like a created timestamp could work. The same match data with timestamps within a short threshold is a bounce to be ignored, further apart is intent to create a new match.
It may help to know that request.session.pop('match_options', None) does not immediately persist the session, but merely marks it modified. It gets saved to the database later in the session middleware, shortly before the response is returned. (I think this explains how your confirm_create_match() view could create a duplicate match if running concurrently when a user double-submits the form.)
