Adding a per-installation salt to password hashing for (also known as a pepper or password peppering)

In order to ensure we’re following the latest security best-practices in Django, we should look to published guidelines as a reference.

NIST Special Publication 800-63B states:

[passwords MUST be hashed securely which we already do]

In addition, verifiers SHOULD perform an additional iteration of a key derivation function using a salt value that is secret and known only to the verifier.

This is also known as password peppering. I propose adding a new password hasher that uses a pepper value from a setting. Updates to the new peppered hashing scheme will happen automatically as users log in, through the existing Django mechanism.

Implementation notes:

The setting for pepper should be distinct from SECRET_KEY, because SECRET_KEY can be rotated and the pepper value can not (all existing password hashes would be invalidated). Alternatively, if rotating pepper values is desired, it could be combined with a identification scheme where the pepper version or hash is stored with the hashed password. The correct pepper value would be looked up based on its identifier, and upgraded to the latest version on login.

A default securely-randomly generated pepper value can be added as part of the project template. Doing it this way is fine because the pepper accidentally becoming non-secret can’t make the existing password mechanism less secure. It merely makes it equivalent to the existing scheme. Users would be expected to override the default value on a per-installation basis using their existing secret management system.

I propose making the peppered password hasher the new default because it is more secure and has no downside. Updates to the new hashing scheme will happen automatically as users log in through the existing Django mechanism.

A nice implementation of this hasher was posted in a ticket, but it was closed due to lack of mailing list discussion, so I’m opening this for discussion. I believe this feature should be implemented in Django core due to the subtlety of implementing cryptographic features. Third-party packages can’t be relied on to implement it correctly and securely.

1 Like

Welcome @gavinwahl !

Please explain, how is this different from what Django already does?

Quoting from How Django stores passwords:

The password attribute of a User object is a string in this format:

<algorithm>$<iterations>$<salt>$<hash>

Those are the components used for storing a User’s password, separated by the dollar-sign character and consist of: the hashing algorithm, the number of algorithm iterations (work factor), the random salt, and the resulting password hash.

The salt being used is randomly generated for each user, which would be far better than a common salt defined at the project level.

The per installation salt would be in addition to that and would add extra security if only the database is hacked and not the app servers as well (where that salt would be stored)

I’ve written an [incomplete, untested] implementation of a PepperedPBKDF2PasswordHasher that supports pepper rotation here:

Technically, this still doesn’t comply with the intent of SP800.

That “second hash” needs to be in a separate location from the primary hash - either a separate physical device or separate OS (e.g. virtual machine) such that the originating system cannot access the salt value itself.

Retrieving that second salt must not be possible without the second system being compromized.

I believe this complies to the intent and letter. Here’s the full excerpt:

In addition, verifiers SHOULD perform an additional iteration of a key derivation function using a salt value that is secret and known only to the verifier. This salt value, if used, SHALL be generated by an approved random bit generator [SP 800-90Ar1] and provide at least the minimum security strength specified in the latest revision of SP 800-131A (112 bits as of the date of this publication). The secret salt value SHALL be stored separately from the hashed memorized secrets (e.g., in a specialized device like a hardware security module). With this additional iteration, brute-force attacks on the hashed memorized secrets are impractical as long as the secret salt value remains secret.

  • We are doing the SHOULD pepper
  • We are doing the SHALL secure random generation
  • The SHALL be stored separately is up to the deployment’s secret management system. Even storing the pepper in the settings file satisfies the requirement if the Django deployment and database are on different machines (which is typical).
  • The “e.g., in a specialized device like a hardware security module” is non-normative and up to the deployment’s secret management system. Deployments that don’t have an HSM will still benefit.

I interpret that the intent of the section is to make brute-forcing passwords from a database dump alone impossible. This achieves that intent. Database dumps are often copied to, and subsequently leaked from, developer workstations or staging environments that don’t have access to the deployment secrets.

Um your next paragraph exactly confirms that it does comply with SP800.

There is no second hash, it is all about a second salt that goes into the derivation of the hash. And the proposed scheme by @gavinwahl does exactly that. One salt is in the database with the hash and changes per user and the other is on the application servers and is constant for all users (and as such separated (eg virtual machine)).

And yes if the database and the app are on the same host then the gains are less (but still there if the database is exposed by accident etc…).

Yes, that’s bad wording on my part - I should have written “second salt” instead of “hash” in this context. (In the case of a true HSM, it is a second hashing - you pass the already hashed password to the HSM, and it returns your re-hashed password.)

Anyway, after a brief discussion with a co-worker on this, we’ve submitted a request for clarification on this. I’ll report whatever response we receive.

1 Like

FWIW, the draft fourth revision of SP 800-63B adds a SHOULD recommendation covering both storage and use of the pepper:

… The [pepper] secret key value SHALL be stored separately from the hashed passwords. It SHOULD be stored and used within a hardware-protected area, such as a hardware security module or trusted execution environment (TEE). …

(But of course that’s SHOULD, not SHALL or MUST.)

shrug if we are going to be nitpicky you can also call it second hashing without a HSM, after all that is what the multiple iterations kinda do in the first place. Nevertheless, while toy HSMs like yubikey HSM etc exist which are kinda afforable (probably around 1000$/€), there is basically no chance that most Django deployments will have access to a network HSM or similar to use for their password hashing :smiley:

What do you want clarification on? No matter how you spin it, putting a second static salt in (even if not within a HSM) is a massive improvement worthwhile on it’s own no matter if the SP wants/requires it.

Using something like a TPM instead of an HSM is an interesting idea but probably out of scope for Django. If one wants to use their TPM/HSM for extra security they can probably just write their own password hasher doing that as opposed to Django providing an API to hook into that (though I guess even that might be doable with the default just reading the value from settings and doing the hashing on it’s own – but not sure if worth the complexity).

That’s great, I thought it was strange that the use of an HSM was left ambiguous and now is an explicit SHOULD, especially since most deployments don’t have access to one.

Like @apollo13 says, satisfying the SHOULD pepper is an improvement even without the SHOULD use an HSM.

Uff, who did flag the initial posts from @gavinwahl and why? They show up as

This post was flagged by the community and is temporarily hidden.

for me.

It was the automatic scanner because of the multiple links to SP800 - I’ve restored those posts. (Multiple links to the same third-party site that isn’t on the defined “approved list” identifies the post as possible spam. The NIST site is not on the approved list.)

This makes sense and the threat model (DB is compromised but pepper is not) is realistic, in both offline scenarios like backup leak, and online scenarios like SQL injection.

The setting for pepper should be distinct from SECRET_KEY, because SECRET_KEY can be rotated and the pepper value can not (all existing password hashes would be invalidated). Alternatively, if rotating pepper values is desired, it could be combined with a identification scheme where the pepper version or hash is stored with the hashed password. The correct pepper value would be looked up based on its identifier, and upgraded to the latest version on login.

IMO, for the common case it very tempting to use SECRET_KEY and SECRET_KEY_FALLBACKS anyway. When rotating SECRET_KEY one should keep the old one in SECRET_KEY_FALLBACKS so I don’t see it as a big difference compared to PASSWORD_PEPPERS.

There are cases where people might not want to use SECRET_KEY as the pepper. But I think we should eventually aim toward making pepper use the default for new projects, and IMO PASSWORD_PEPPERS is too much cognitive & operational overhead to be the default, while SECRET_KEY should be mostly transparent.

One very important to note here is that makes passwords non-portable across systems or environments. This is arguably a feature, but there are definitely places which do this (sometimes legitimately, e.g. loading a dump from a staging environment to a dev environment), so it should be noted in any documentation.

The rotation schedule of SECRET_KEY and the password pepper is different. SECRET_KEY_FALLBACKS only needs to contain secret keys for as long as sessions last, or however long you expect django.core.signing signatures to be valid.

The PASSWORD_PEPPERS values can essentially NEVER be removed, because passwords hashed with that pepper may still be stored in the database indefinitely.

My opinion is that despite this, if using a pepper is to become a default, it is better to reuse SECRET_KEY than to add another secret setting. This means entries must not be removed from SECRET_KEY_FALLBACKS. If someone doesn’t want this binding between SECRET_KEY and passwords, they should use a custom password hashing backend which uses a different setting. This should be made reasonably easy (for example, move the fetching of the peppers to a method that can be overridden in a subclass).

I do not think using the pepper will become the default (at least not at first). And there are reasons where you would want to remove keys from SECRET_KEY_FALLBACKS if due to an (security)issue you have to invalidate all data signed by those keys. But you most certainly don’t want to invalidate all passwords then. Also you could regularly rotate the secret keys (which is a good idea) but that is much harder with the pepper and you really don’t want to try all keys in the list to figure out your pepper is the oldest one or not even in the list.

Cheers,
Florian

Well hopefully we can aim for that.

This is definitely true. I just think that we should cater for the 99% case. But I understand the reservation.

Rotating the pepper is probably a good idea as well.

Regarding the efficiency concern, @gavinwahl PR cleverly stores the hash of the pepper in the password string, so the correct pepper is retrieved O(1).

This is not desirable. If your SECRET_KEY is compromised, you should immediately regenerate it and not add it to SECRET_KEY_FALLBACKS. If your PASSWORD_PEPPER is compromised, you should rotate it but must keep it as a fallback.

If we were to reuse the same setting, it would not be possible to do the right thing for a SECRET_KEY compromise. That seems unacceptable to me.