Setting Django's version statically (#35838)

I’ve opened this: #35828 (Switch to statically declaring version number in pyproject.toml) – Django

And I suspect it might lead to a lot of controversy, so I’m pre-emptively starting a forum thread for it :slight_smile:

Speaking as an ex-release-manager, the current complex dynamic-version-number setup exists for the reason given in the ticket: it used to be the Python packaging ecosystem just didn’t have the tooling to easily refer to your own version number without setting it as some sort of module-level attribute, and then people developed ways of dynamically reading that attribute into the packaging metadata to avoid having to re-declare the version number in multiple places.

Today, that is not a problem. There’s standard-library API for retrieving the version number of any installed package, and it’s available in every Python version the current main branch of Django supports. Which means any time code needs to know the installed Django version number, it can call importlib.metadata.version("Django").

While there is ongoing and highly contentious discussion in the Python packaging world about whether the whole concept of the __version__ attribute should be deprecated (and personally I think it should be, and will be removing it from my own personal packages), I think Django could, if we want to, keep it around for a few releases as a compatibility measure by having django.__version__ (and django.utils.version.get_version()) just use importlib.metadata.version() to read the statically-declared version from pyproject.toml.

The ticket mentions a couple use cases that are potentially affected. I’ll elaborate a bit more on them here, with my own opinions:

  • Vendoring or otherwise “installing” Django without actually installing it would break. In this situation, attempting to read the version number from importlib.metadata.version() will raise an exception. I personally am OK with this, because I don’t think this should be a supported way of “installing” Django (and it’s not currently an option covered by the install docs).
  • Editable installs would work, but changing the version number in pyproject.toml after performing the editable install would not cause the reported version to auto-update – re-running pip install -e would be necessary to pick up the changed version. I think this is acceptable; needing to touch the version number in an editable install is probably a pretty rare use case to begin with, and it can be documented that if you do need to do that, you also need to pip install -e again afterward.
  • Linux distributions, and other non-Python packagers, would need to ensure they patch in-code references to Django’s version number in order to return the correct information based on how they store package metadata, since importlib.metadata.version() would not know how to read their distro-specific metadata stores. Again I think this is acceptable since they probably have to solve this anyway as a prerequisite of offering packaged Python libraries (and it also potentially helps simplify their jobs by moving toward a world where every Python package declares its metadata in a single standard static way, without requiring arbitrary package-backend toolchains and code execution to retrieve dynamic version numbers from any of multiple places via any of multiple mechanisms).

So, obviously, I think it’s a good idea. I also think it’s a place where Django can serve as an example of using the newer and simpler standardized tooling the packaging ecosystem now offers.

What do other folks think?

Agreed on importlib.metatdata. As for defining the version itself – do we actually need to? All modern build backends allow reading it from git itself.

If the version number is to be set from the git revision, how would that work for an editable install that wasn’t done from a git checkout?

Not sure, but is that something we need to support?

It’s worth noting that use of the django.VERSION tuple in comparisons is also widespread and would also need to be maintained. (This is the sort of thing that folks ignore until it breaks and then complain about when the deprecation period ends, so we should try to plan for that.)

@ubernostrum Beyond the API docs links, is there a place you’d point at to get an overview of the debate in Python about. __version__? My initial gut-response here is +0 — but that’s likely because I don’t know enough about it to see that/how it matters. It would be good to read more. Thanks.

If you want to subject yourself to reading a contentious packaging thread, be my guest:

1 Like

I don’t know enough to have an opinion on the above and I’m new to releasing and my understanding might be wrong. However, I’m worried that this might result in us releasing different versions of Django in multiple ways.

For other folks who might not be aware, in every release we have two commits updating VERSION - one during the release (see step 5) and one post release (see step 1).

If I have understood, these steps won’t go away and instead be something we need to set in the pyproject.toml

We recently had some changes in our pyproject.toml (Migrated setuptools configuration to pyproject.toml. by claudep · Pull Request #17806 · django/django · GitHub) implemented in 5.1 which impacted some steps in issuing a release

I’m worried we might end up having to do a security release with 3 slightly different processes to release depending on the version being released (4.2 vs 5.1 vs 5.2).

Having an agreement that changes to issuing the release be a backport-able change would be nice.

In general, I’m very open to anything that reduces steps or simplifies issuing a release but just concerned how to manage this practically

So with my recommendation you wouldn’t need any extra commits at all. The version will solely be determined by the git tag you are on. This should make it easier and less error prone imo.

1 Like

Thank you @apollo13 – do you have a link handy that explains your proposal in more detail? While I can search for related topics, I’d like to understand and know that we are agreeing (or not) on the same thing.

Specifically, I’m not sure what would be the Django version reported in main when pip installed with -e.

Mhm, don’t have any links at hand but setuptools-scm is an example of this: GitHub - pypa/setuptools-scm: the blessed package to manage your versions by scm tags

Basically the version is only written into the distribution metadata and doesn’t exist in any file (it is accessible via importlib.metadata then though).

The generated package basically looks like this: dist/oversight-0.1.1.dev31+gff49a06.d20241010-py3-none-any.whl This tells you that it is 31 commits after 0.1.1 with a git revision of ff49a06 and a builddate of 20241010. Whether this is actually the version reported for -e I am not sure, will try once at home. But this would be the version you see when you build at an abritrary commit and then install that package.

As a maintainer of a few third party libraries and dozens of websites using Django I like django.VERSION because it can automatically be fixed/removed by django-upgrade’s versioned blocks fixer:

The same thing could certainly be achieved by using importlib, LooseVersion and friends. I worry a bit that these cases will be more error prone to write than if django.VERSION < (5,) and possibly also harder to detect in an automatic fixer.

No strong reasons to avoid changing or simplifying the process here, but maybe something to consider.

1 Like

I do not think that django.VERSION would go away, but it would be populated via the package metadata instead.

1 Like

The version reported would match the version at the time you ran the last pip install -e ., so in my case this is currently 5.2.dev20240823065001. Updating my main and rerunning pip install -e . would result in 5.2.dev20241010111846 (note the difference in the dates). Would this be a blocker? Plenty of packages seem to work fine like this :slight_smile:

@apollo13 It would (if I followed you) mean needing to reinstalling your development checkout once per major release, which I can imagine forgetting until it bit me, and then spending quite a while head scratching. That might be an issue.

Yep. That was fun :pleading_face:

Two points from the thread stood out.

Similar to @matthiask’s point django.VERSION is easy to remember, whereas I’d have to go and lookup the new one. (But as per @apollo13 no reason for VERSION to go away, just the source of truth to change.)

Then, it looks like there may be some agreement emerging. An armistice of sorts. Usually I’d expect Django to wait until the waters settle to move (as we did with pyproject.toml &co). I’m not really vested in this, but one change is better than two.

(It looks like __version__ will likely be allowed to remain; maybe we should change for other reasons. Not sure I’m convinced. Still probably +0)

@carltongibson Yes, you did follow correctly. Also not just once per major release but also when you are switching branches. That said how often is the version really important (well unless you are doing version checks in 3rd party software)? I honestly have not once looked at django.VERSION when developing on Django itself.

But if I am not mistaken (didn’t check, but I don’t see why it would behave differently) we would have the same problem when writing the version statically into pyproject.toml, a pip install -e would take it from there, put it into the metadata and then not ever touch it again. Or to put it differently: As soon as we no longer write the version into Django’s python source files editable installs will get the wrong version.

True, but one thing missing from the ticket is that this function returns a string rather than a parsed tuple. It makes it much less useful for comparison as you then need to parse the version, which means importing packaging. So, another argument for keeping django.VERSION around. Django would presumably have its own parsing code, to avoid the dependency…

Funnily enough, I just finished another feature there for removing test skip decorators based on django.VERSION: Strip skip decorators for old Django versions · Issue #364 · adamchainz/django-upgrade · GitHub.

Yeah, maybe we can wait til this (TLDR!) thread resolves, at least.

1 Like

I think we can live with the dependency, if it comes to that, or just temporarily vendor an implementation of the parsing function while VERSION undergoes deprecation.

To be clear, the lack of consensus there is on whether and how to update the packaging guide’s recommendations specifically around the existence and continued use of __version__ as a “standard” attribute and the pile of “dynamically set your version everywhere else from a single attribute declaration” hacks people have come up with ovr the years. Which led to one person interpreting a PEP rejection as evidence that __version__ is now deprecated, and someone else writing what seems to be a literal spite PEP in response trying to make __version__ mandatory for all importable modules.

I’m not sure that derails us or is even a thing that will ever reasonably resolve, based on my experience with the packaging community. If we say we’re going to wait for them to put an official recommendation in the docs, we should be prepared to wait an extremely long time.

When updating packages to support new versions of Django, this kind of thing is important. Essentially you branch on django.VERSION. (There may be other ways, likely there are, but I think that’s how most — certainly many — do it.)

Having to remember to pip install -e .… — well that looks like a pain. I could easily see it tripping up lots of folks across the ecosystem. (Likely, on an ongoing basis.)

Currently the version exists in a single place, in django/__init__.py. All that’s needed is to bump the VERSION tuple and the rest is done. For example, @nessita releasing 5.1.2 recently.

TBH I’m not seeing the advantage of moving it. Still only defined in one place. But running code to resolve it every startup vs just every time the package is built. And with this potential issue for third party package maintainers.

Other than favouring static data in pyproject.toml, which I’m failing (yet) to see as an end in itself, what’s the supposed benefit of the move here? I’m still missing that :thinking: (On my own projects I’m using flit to package, which uses all the modern tools, but still recommends having the version dynamically determined from the single code-based version. That seems totally fine. I’ve not thought once to change it. What’s the reason to do so?)

Thanks. :pray: