Hi everyone. I’d like to propose publishing PEP 740 provenance (digital attestations) alongside Django’s wheels and sdists on PyPI. This doesn’t require any code changes—just a small update to the GitHub Actions release pipeline: enable Trusted Publishing, use pypa/gh-action-pypi-publish@release/v1, and add a brief step in the release guide to verify that provenance appears via the Integrity API. As an alternative, we can also consider the uv toolchain (uv build / uv publish); once attestation support there is fully settled, it could replace the above with little friction. For reference, Flask already exposes provenance on PyPI. Thanks for reading.
Yeah, this would be good. I switched to trusted publishing for my open source projects and it’s nice to be able to provide more guarantees to users.
Would you be able to make a draft pull request with the changes you think are needed to use the action? That could help define how much further work would be needed, especially important since this would likely require the fellows to complete the task.
you are assuming that releases get published via github actions I do not think we have a release pipeline, do we?
Yes, this will certainly need a ticket and probably some discussion around whether we even want to move the release process to github in the first place.
@apollo13 Thanks, that makes sense — I was implicitly assuming a GitHub Actions–based release. To move this forward, I’ll open a ticket comparing two options
It seems a bit previous to open a ticket. That will only be closed pending a discussion here.
Q: is there a list of PyPI Trusted Publishers available anywhere? As far as I can see, is it only GitHub? It’s not clear to me that it’s actually possible to become a Trusted Publisher unless you’re a public CI host of some renown. It looks like these PyPI advances are tied to GitHub, which is a shame if so.
I would sceptical about moving the release process to GitHub Actions. I understand the convenience if you’re using actions already, but it strikes me as less secure. (A breach via GitHub seems significantly more likely than capture of one of the small number tokens on releaser machines, not withstanding that the actual token used by the GHA to make the release is short lived.)
At the least we’d need an audit of permissions around the django/django repo. We have a very small number of known project managers on PyPI whereas the Django GitHub org has all number of members and teams, with not the cleanest permissions breakdown.
GH have also recently stopped taking contributions to their Actions components, citing a pending AI rewrite. We should see how it roles out before tying ourselves to that particular mast. “AI” and “secure” not being often seen together currently.
As an aside, I think we should seriously be considering moving off of GitHub, not committing ourselves more. The hosting is fine. The free CI is fine. Everything else is increasing problematic.
Do you have a source for that, I’m interested ! Couldn’t find anything on my own.
The Warehouse (PyPI) project is a true gem, one of the only sites (as far as I know) of that scale that’s open source and really relying on community contributions. And of course, contributions are not as frequent as one would hope. The team behind it, which I used to be close to, are awesome people. I’d be very surprised that anyone coming with a serious proposal for implementing other sources would find anything else but a warm welcome.
Having said that, I wonder what exactly we’d want to have. I wonder what the security model of trusted publishing is: do we just need the package maintainer to say what the provenance of the package is and PyPI just checks that, or do we need PyPI to actively trust the provider explicitly ?
The idea is that it’s more secure, as Trusted Publishing uses short-lived tokens that automatically expire after 15 minutes:
Good idea. I suggest the zizmor tool to help with this. It can also be added to pre-commit.
GH have also recently stopped taking contributions to their Actions components, citing a pending AI rewrite.
It is concerning to see they’re no longer taking contributions, but I’m not sure they’ve explicitly cited an AI rewrite? (But won’t be surprised if this happens.) The GitHub Next thing seems more like an experimental thing than tied to any Actions roadmap.
Yes, but it seems the contrast is with long-lived tokens uploaded to CI (where, sure, short lived tokens win) vs similar kept locally (where we don’t have the additional risk of/to the platform itself, which is direct breaches there overlapping with permissions issues and so on). More or less, I have more faith in the security of (and our ability to audit) the Fellows’ laptops than I do in GitHub’s platform.
I linked the security model above. One of the requirements of becoming a provider is essentially that PyPI trusts them:
Reliability & notability: The effort necessary to integrate with a new Trusted Publisher is not exceptional, but not trivial either. In the interest of making the best use of PyPI’s finite resources, we only plan to support platforms that have a reasonable level of usage among PyPI users for publishing. Additionally, we have high standards for overall reliability and security in the operation of a supported Identity Provider: in practice, this means that a home-grown or personal use IdP will not be eligible.
I don’t want to burst everyone’s bubble, but Django’s release process is manual, and there is virtually no automation. The Fellows/Releasers generate a checklist and the process changes based on the nature of the release (security, bugfix, major, etc).
So while UV might help with a step or two, this change can’t happen because there is no CI step or GitHub Action to modify to see how it might work.
Side-note: This could be easier, but historical pushback has made this process more complicated than necessary. Maybe that could be PR’ed or DEP’ed or Forum Posted for review and consideration?
I’m not answering to every point made here, but I do want to add a few clarifications from the POV of a Fellow:
I had to close the ticket as wontfix, given the current documented triage procedure and everything that was already discussed. Additionally, since this is a New Feature, I would prefer it to go through the new feature process via the relevant GitHub project.
Regarding the tokens and the security/trustworthiness of a release: yes, I agree that Fellows’ laptops are easier to audit, but I don’t agree that they are inherently more secure. For the Fellows, there is no baseline for:
allowed operating system
OS updates/versions
disk encryption
physical security
backup procedures (specially for GPG keys)
general security hygiene, etc.
Personally, I think that holding a GPG key that can sign releases and potentially affect thousands of services using Django is a serious, non-trivial responsibility. I do my best to secure it (disk encrypted, modern OS fully up to date, laptop always with me when traveling even if I don’t need it, etc.), but I cannot agree that relying on people is more secure, human error is a major factor!
To increase awareness, here are some useful references for how a Django release is performed:
This definitely isn’t a criticism for those minor mistakes. I’ve found the automated publishing workflows a huge weight off my mind for the packages that I build/release (nothing particularly important), and the digital attestations are the icing on the cake. No tokens need to be added as secrets thanks to OIDC.
We certainly do, but for the sake of documentation I will say:
I do not reallyy trust a fellow’s laptop. This is not because I do not trust the people using them but because of their position. Due to the position they are in it is way more likely for them to get contacted by new people, try out “new” code etc… making them an ideal target for a targeted attack. Personally I think our release process is kinda yolo and includes way to many manual steps.
I also trust Github less and less (especially since Microsoft and the Storm-0558 debacle). That said, I think that if Github is compromised there might be bigger fishes to fry than Django, so we might have going that for us.
So what options do we have? Independent of whether any attestation might be a good idea or not, the first steps imo are reproducible builds. We might even have them without knowing it (or via slight adjustments only) since all in all we are just packing up some files from a known revision in a tar/zip and we mostly just need to fix timestamps (we don’t have to worry about compiled code etc). This way it is possible to verify the built release by multiple people before publishing. This makes a compromise of an individual machine even less likely/useful. The next step would be to build the release in CI as well providing another verifier for the reproducible build.
Once we got all that we can consider how we upload to PyPI. More thoughts on that in another post I guess
This gets to the heart of the issue, and I’m glad this has been brought up, because a lot of people get confused about who or what is being “trusted” in “Trusted Publishing”.
Trusted Publishing does not mean “end users can trust the package”. It means the package author can trust a third-party CI/CD system to upload to PyPI on the author’s behalf, in a way that mitigates the impact if the upload credential is accidentally exposed/leaked.
Notably, Trusted Publishing does not carry any sort of enhanced guarantee that the package is free of vulnerabilities or vetted by security experts.
So if you’re a package author who’s not already using CI/CD to publish releases (such as Django, which does not use CI/CD to publish releases), Trusted Publishing offers you nothing, because you don’t have the problem Trusted Publishing solves.
Similarly, the “attestation” on a package published by Trusted Publishing is just a way to verify that it came out the end of a specific pipeline on a specific CI/CD provider. Nothing more, nothing less. It isn’t a package “signature” in the sense people often ask for (Django stopped doing those, I’m fairly certain, and PyPI doesn’t support them anymore anyway), it isn’t an SBOM or any of those other fancy things, just “yup, this came from there”.
I do not think we have a release pipeline, do we?
