I’ve had a case where a user has created duplicate database entries via a form. In this case matches in a board game tournament.
Is there a known good way to prevent this? I have a review page now, with the fields stored as session data, so by the time the user clicks the Create button, the form data has been validated. So perhaps I should disable the button via JavaScript.
But is there any way to prevent this server-side? Could it be the case that two instances of gunicorn serviced the form submission? In which case, I have no idea how to prevent the two processes from creating two entries in the database. There’s always the possibility that the same user actually wants multiple entries with the same data.
Soon, I’d like to implement creating all the matches in a tournament in one go via a CSV. In that case, duplicates would be extremely undesirable.
Here is my code:
The view above this one stores the form data to session storage. This function pulls that data out of session storage and uses it to create the match. Is the pop atomic?
Create database constraints. Validate that the submission doesn’t create a duplicate entry. Provide the option for a dialog box asking for confirmation in case that a duplicate entry would be created.
I don’t have any field that needs to be unique. It’s completely valid to have multiple matches with the same title, players, and settings. This might help others but not this situation.
I hesitate to kick this back to the user. Even then, how do I know it’s a duplicate if I have a race condition? If I check for uniqueness, it’s unique, and then I create the entry, and by then it’s not unique, then it’s already too late.
BTW, as far as I know, only this user has had this problem, and I can’t reproduce it locally. However, this same user has had this problem multiple times, both before and after I added the review/confirmation page.
Yes, but what about the combination of fields? You can define a constraint such that individually, “players”, “title”, and “settings” can all appear multiple times, but you can only have one combination of (“player”, “title”, “settings”). If multiple instances of that combination are valid, then what is the problem?
Someone needs to make a decision whether or not a duplicate is a problem. If it’s not a problem, then there was no need for this post. If there is a problem, then someone needs to decide whether or not it’s valid.
Fundamentally, the bigger problem is storing it in the session rather than in the database. Two different people are always going to have separate sessions. One person working with two different browsers is also going to have separate sessions. You can avoid a lot of these issues by having the entries created in the database immediately, with a status field of some kind to indicate whether or not it has been reviewed and approved.
I see that my statement was ambiguous. I don’t have any field or combination of fields that require uniqueness.
What I’m trying to avoid is someone clicking the Create button and getting two matches created. It would be valid for them to go back to the match creation form and input identical data into the fields and create an additional match.
I’m guessing you didn’t mean it this way, but I think “there was no need for this post” could be seen as rude.
My understanding is that session data is stored in the database (using the sessions.session model). It is per-session, so the same user will generally have per-browser session data. I think that’s fine.
If there’s a better way to pass the data to the next view without creating a match object, then I am definitely interested.
It’s difficult to know how to program a solution without knowing the mechanism of the problem. I haven’t been able to reproduce locally, so I am not sure how to debug. Maybe it’s because I’m running a single process locally, so there’s no way for this to happen. My local instance is usually sufficient to test, but there are differences. On my local machine, I’m using sqlite3, in-memory message passing, 1 daphne, etc. Deployed, I’m using PostgreSQL, Redis, multiple gunicorns + 1 daphne, etc.
If two processes are grabbing the same session data and creating two objects (I don’t know how else I’d be getting more than one object.), then which process is the one issuing the redirect response that the user’s browser sees?
If there’s a race condition, then I’m not sure that both processes would be able to reliably detect a duplicate. In which case, I can’t reliably communicate to the browser to ask whether this was intentional.
I also prefer not to have a new Match field that I now have to use as an additional filter on every query of Match objects.
If they’re stored in the database, then there is no race condition. The first one has submitted the data and the second encounters the duplication.
A custom manager or manager queryset addresses that issue.
Django provides these type of features precisely to allow a developer to avoid problems like this. It seems to me to be suboptimal to implement a solution instead of taking advantage of the features provided.
Have you tried double-clicking the submit button? Some users habitually double-click every button. In forms, that results in duplicate submissions a few milliseconds apart.
You can use client-side JavaScript to help prevent this (disable the submit button during the first submit so the second click is ignored). Or you can include a hidden field in the form specifically to detect duplicate submissions server side (a nonce, or maybe the timestamp the form was served). Or both.
Of course. I think I would need multiple processes serving those clicks to reproduce locally. I’ve started looking into being able to run multiple processes locally.
Yes, as noted before, I will try preventing this in JavaScript. I was partly wondering if this has been encountered by others when using Django, and whether there is a known solution.
Is it a known problem that double-clicking a form submit button double-submits the form? Yes, and that’s not unique to Django, and a popular known solution is to debounce form submission in JavaScript. (I think you probably already know this, but just stating it here for the record.)
I’m not aware of any known bugs with Django or gunicorn somehow processing a single browser request twice. That doesn’t mean there aren’t any, or that something like a misconfigured load balancer might cause that. But the far more likely explanation is the user is double-clicking the submit button. (It’s definitely a known issue that some users tend to double-click everything.)
To debounce server-side, you’ll need to persist some de-duplicating criteria somewhere. Since you’ve said that the same user should be able to create multiple instances of the same match data, I’d think something like a created timestamp could work. The same match data with timestamps within a short threshold is a bounce to be ignored, further apart is intent to create a new match.
It may help to know that request.session.pop('match_options', None) does not immediately persist the session, but merely marks it modified. It gets saved to the database later in the session middleware, shortly before the response is returned. (I think this explains how your confirm_create_match() view could create a duplicate match if running concurrently when a user double-submits the form.)
I had actually seen this before but only skimmed it earlier. Thanks for pointing me back there. I’ve finally found a good time to read through it.
Blog post tl;dr: Hash the form contents and prevent processing duplicate identical contents back-to-back. [Let me know if I’m mischaracterizing the contents of the blog post.]
Unfortunately, I don’t see that this works for me. The same user might want to create multiple matches with the same title (which can be empty), the same players, and the same settings by going back to the creation form and filling it out the same way. If they were prevented from doing this, then I would have a bug. I could potentially use the information to present an “Are you sure?” dialogue, but I don’t want that to be part of my UI.
I would either need a timer or a check that the same user loaded the creation-form view again before allowing the same data.
For my particular situation, especially since I have a separate review page, I think using JavaScript to disable the submission button is the best solution, along with a message informing the user that they should be redirected “soon”, and if they continue to see this message, then they should check to see if their match was created before trying to resubmit. Normally the redirect is nearly instantaneous, but might not be once I implement creating a set of matches all at once for a tournament.
I don’t see that this works for me. The same user might want to create multiple matches with the same title (which can be empty), the same players, and the same settings by going back to the creation form and filling it out the same way.
You need to make a decision here: You have chosen to define a VALID user action and an INVALID user action in exactly the same way.
Using the browser-back button is an extremely difficult case to satisfy, as a page will then be loaded from the browser’s cache. I.e. no way to inject a unique serial number known by the server.
I think - going back to Ken’s suggestion - you need to really ask the user.
Regarding the blog post’s method that hashes the fields - You can use a time-based cache and say “if the user re-submits the same form within 1 minute, then prompt them” - or as you do, use the same time-based guard to block the user from resubmitting within that grace period.
At the end of the day, there are some different methods to avoid double form submissions, and it’s very problem-specific which to choose and how to combine them.
If there is no reliable way to distinguish the two cases server-side, then that answers my question.
From the user’s perspective, these cases are quite different:
A) I fill out a form to create a match and submit the form. One match gets created and my browser redirects me to it. I navigate back to the match creation form, fill it out again, and submit. A second match gets created.
B) I fill out a form to create a match and submit the form. Two matches get created and my browser redirect me to one of them. I have no clue that a second match was created unless I check the match listings.
My solution, at least for now, is to disable the submit button when pressed, and append a message letting the user know that the form was submitted and they should be redirected. If they aren’t, then they should check whether the match was created before trying to create again. Here is the code:
