Case-insensitive email uniqueness with Django Simple JWT — avoiding full table scan

I’m using a custom User model with Django’s built-in auth system alongside djangorestframework-simplejwt on PostgreSQL. I’ve run into a case-sensitivity issue with email uniqueness.

Authenticating Registering with test@test.com succeeds. Authenticating Registering again with Test@test.com also succeeds since djangorestframework-simplejwt uses Django’s authenticate() creating a duplicate. Most modern email providers treat these as identical, so this needs to be enforced at the application level.
P.S.: I meant Registering not authenticating, My bad.

What I’ve tried:

Using __iexact for validation — but it causes up to 13 seconds of delay because it generates a UPPER(email) = UPPER(value) query which seq scans the table.

I then added a functional index thinking it would fix it:

constraints = [
    models.UniqueConstraint(Lower("email"), name="unique_lower_email"),
]

Still a Seq Scan. The problem is the index is on LOWER but __iexact generates UPPER — they don’t match, so the index is never used.


Solutions I found:

A) Normalize on save

Store all emails as lowercase. Drop the functional constraint, use a plain unique=True, and normalize in save():

def save(self, *args, **kwargs):
    self.email = self.email.lower()
    super().save(*args, **kwargs)

Queries become a simple filter(email=input.lower()) — hits the btree index, no footguns, still this worries me for the sake of the [RFC 5321 Section 2.4](RFC 5321 - Simple Mail Transfer Protocol.

B) Fix the index to match __iexact

Switch the constraint to Upper to match what __iexact generates:

constraints = [
    models.UniqueConstraint(Upper("email"), name="unique_upper_email"),
]

This makes __iexact hit the index. The downside: every lookup must use __iexact — a plain filter(email=value) silently bypasses it and duplicates can slip through or make it filter(email=value.upper()) where it feels more or less like A but with extra hops.

I would really appreciate some insights into this.

Is this true if the email address in the database is lowercase? as in, if the DB entry is lowercase (test@test.com) email address, does both test@test.com and Test@test.com succeed auth wise against the same DB entry?

if the answer is yes, then I would recommend fixing the auth so that it acts as expected by RFC 5321.

if the answer is no, and as you pointed out in RFC 5321 - Simple Mail Transfer Protocol, SMTP email address’ specifically the local part, “is to be treated as case sensitive.” So with that in mind, any user supplying an email as part of their JWT will always be the same user (with the case sensitivity acting the same as having a separate value), regardless of the case of the local part of the email address. Therefore I’d recommend no change.

So, I modified question. I did not mean authenticating I meant more like registering, if I registered with test@test.com it would register an instance, and I did the second with Test@test.com it would do another instance.

While this is according to the RFC 5321, it raises another problem regarding registering with the same email multiple accounts since major providers like Google lowercase local part of the mail.

how is it another problem? if the registering within django is case sensitive and auth is also case sensitive (yeah I know you said registering, but auth must also be), then it is the equivalent of having seperate values. Emails localpart@domain.tld and Localpart@domain.tld are not the same value, they are different values (look at an ascii table, you will see that the chars are not the same). This is clearly defined in RFC 5321 to which I would advise against doing anything different than a standard.

The fact that email providers normalise the local part of the email address is not relevant.

my advice still remains, “as long as registering and authentication are case sensitive, then there is nothing I would be changing” this includes normalising prior to saving in db as part of registering.

Why is this a problem? As you noted, those are technically two different email addresses. Although Gmail and several other providers ignore case in the local-part, how do you know that test.com has the same rules?

If you’re trying to prevent the same person from intentionally creating multiple accounts, there are a lot of domain-specific rules you’ll need to figure out. For example, all of these email addresses refer to the same Gmail account:

example@gmail.com
Example@gmail.com
example+123@gmail.com
ex.ample@gmail.com
e.x.a.m.p.l.e+test@gmail.com
example@googlemail.com

Many email services support plus-aliasing or similar features, but the exact details depend on the provider.

If you’re trying to prevent unintentional entry of capitalized email addresses on mobile devices, that’s better handled client side. Make sure your HTML input field has type="email" (or autocapitalize="none" autocorrect="off" spellcheck="false" and possibly autocomplete="username" depending on how you’re using it).

Either way, I’d agree with @jon-nfc’s advice to avoid trying to normalize email addresses.

While you are correct technically it raises a problem during login the user must write down the email in the case sensitivity that he was written during Registration and I cannot insure that [even me during my registration or login into any services sometimes write my email in uppercase lowercase or some mixed cases].

While both your ( @jon-nfc & @medmunds ) suggestions are technically correct, I don’t know if it’s a good idea to implement?

This is no different than supplying an incorrect value.

In all honesty, I don’t see why you are so caught up in what you stated as a problem in OP. There is no problem at all and nothing to change. That is on the proviso that both registration and authentication are case sensitive.

If you really want to treat your CustomUser.email field as case-insensitive for comparison purposes, one possibility is specifying a case-insensitive db_collation for that model field. That should make __exact lookups and unique enforcement ignore case. It should also be case-preserving, so whatever capitalization the user enters when registering will be used when sending email (which I gather was your concern when you referenced RFC 5321).

Collations are database engine specific. For PostgreSQL you may need to ensure a CI collation exists with a CreateCollation migration. Also, note that Postgres non-deterministic collations don’t support partial matching, so queries like email__icontains and email__startswith will raise errors.

I’d argue that it’s quite different, but you are correct in that I am too caught up with it, I will probably stick with lowering values for now, and later we shall see if it needs any further modification.

@medmunds & @jon-nfc Thanks to both of you for taking the time to share your thoughts! I really appreciate both of your insights.