How to properly update Django version and all the project packages?

Hello everyone,

so we have a Django web app deployed for some time and now we want to update from Django 2.2 to 3.1 and eventually to 3.2.

The actual Django update went smoothly, I ran the tests with deprecation warnings, fixed those, then upgraded Django to 3.1 and see that crispy forms and server side analytics broke, so I updated those packages to new versions and everything runs fine.

What concerns me is that some of the packages we are depending on jumped (in a few cases) multiple major versions. So I think it makes sense to update those packages as well.

I found the Pur utility which bumps all the packages to newest version, but this seems a bit crazy, so I tried to do it by hand. Problem is that for example when I upgrade package X, this needs Celery 5, so I update Celery to 5.0 but then package Y complains that Celery needs to be in the 3.0-4.0 range for this package to work…

Is there any standard workflow I should use to upgrade? Is it even necessary to update all the packages? Many thanks.

Below is our requirements.txt file just to illustrate the state of packages in the project:

amqp==1.4.9
anyjson==0.3.3
beautifulsoup4==4.8.1
billiard==3.3.0.23
celery==3.1.26.post2
certifi==2019.6.16
chardet==3.0.4
defusedxml==0.6.0
Django==3.1
django-appconf==1.0.3
django-celery==3.3.1
django-compressor==2.4
django-crispy-forms==1.10
django-google-analytics-app==5.0.0
django-post-office==3.4.1
idna==2.8
jsonfield==3.1.0
kombu==3.0.37
oauthlib==3.0.2
Pillow==8.0.1
PyJWT==1.7.1
python3-openid==3.1.0
pytz==2019.1
rcssmin==1.0.6
requests==2.22.0
requests-oauthlib==1.2.0
rjsmin==1.1.0
sentry-sdk==0.18.0
six==1.12.0
social-auth-app-django==3.1.0
social-auth-core==3.2.0
soupsieve==1.9.5
sqlparse==0.3.0
structlog==19.2.0
urllib3==1.25.3

Actually, according to the docs, pur does not update the packages. All it does is update your requirements.txt file, giving you a chance to visually inspect it and look for potential problems before you perform any updates.

You can then try installing that set of packages in a different venv to see whether or not you’re going to have any problems with that particular blend.

Yes, you are right, I did not articulate this correctly. It indeed updates the requirements.txt file.

Is there maybe a way to check if the package versions in the requirements.txt are compatible? Because just trying to update bunch of them and then seeing what broke does not seem like wise solution. + poor PyCharm has to to a whole bunch of indexing after each try :smiley:

See pip check.

1 Like

Looks like that could be it. Thanks! Will report the results

I don’t know if I am using the pip check wrong but it never reported any issues, even after trying to instal new reqs which showed this error:

 Could not find a version that satisfies the requirement social-auth-app-django==4.0.3 (from -r     requirements.txt (line 30)) (from versions: 0.0.1, 0.1.0, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 2.0.0, 2.1.0, 3.0.0, 3.1.0, 3.3.0, 3.4.0, 4.0.0)
No matching distribution found for social-auth-app-django==4.0.3 (from -r requirements.txt (line 30))

So I guess manual approach starting with the “top level” packages is the only option?

No, check is doing what it’s supposed to do - it’s confirming that there are no conflicting requirements. However, I’m not sure how you ended up with a setting in your requirements file for social-auth-app-django==4.0.3, because pypi shows the most current released version as 4.0.0 and their github repo doesn’t show anything different being available.
(Was that pup that generated that entry? If so, then the issue is with pup, because the requirements.txt file you posted above refers to version 3.1.0.)

Yea, I think this is user error :smiley: I went with the same version as for the social-core-app and turned out the version of django flavor is indeed 4.0.0.

These experiments led me to utility pipdeptree which creates nice dependency graph. So I first plan to remove packages we are no longer using (I tried PyCharm analysis before but that did not found unused packages) and I am probably going to kick out django-google-analytics-app which is responsible for a lot of extra dependencies.

I recommend moving to the pip-compile tool in pip-tools. This allows you to declare your top-level dependencies in a separate file called requirements.in that is then “compiled” to your requirements.txt. The format lets you see which packages depend on each other.

You can use pipdeptree to build your requirements.in file, then use pip-compile to control your requirements.txt from then on.

1 Like