Problem with save() and related models

Using Django
1

I have a problem, that I have resolved, with different results I get with a project between using it manage.py runserver and via WSGI and Apache.

The problematic piece of code is (I’ll post the models if they are needed, but I think the general schematic is obvious enough):

At this point there are no database records (except the account record).

views.py
			…
			user = get_user_model().objects.create_user(username=username, email=email, password=password)
			…
			user.profile.account = account
			user.profile.member = member
			user.profile.member.account = account
			user.profile.member.save()
			user.save()

When running this with runserver this works exactly as I expect, the member, profile and user records are created, and they are linked to the account (profile is created automagically because it is a one-to-one relationship with user).

However, when I run it with WSGI and Apache, although there is no error in the saving and response, data is missing in the HTML display that originates in the member record. When I check the database tables I see that everything has saved as expected, except the user.profile.member_id field is empty (this is not in the model, it is inserted into the database by implication of user.profile.member relationship).

So, to resolve this problem:

views.py
			…
			user = get_user_model().objects.create_user(username=username, email=email, password=password)
			…
			user.profile.account = account
			user.profile.member = member
			user.profile.member.account = account
			user.profile.member.save()
			'''new line'''
			user.profile.member_id = user.profile.member.id
			user.save()

So, I’d like to know whether the initial way I did this is correct, or it’s just a fluke that it worked and the “fix” is the correct way.

I’m thinking that perhaps there is some difference in the Django or Python version between the different behaviours. Although I believe they are the same, I cannot work out a way to test that (ie some output on the template or the terminal/Apache error log that confirms the versions).

Thanks

2

Are you using the same databases in both cases (e.g. PostgreSQL)? Or is one using PostgreSQL and the other Sqlite?

(I am far more inclined to believe that it’s a difference in database handling between the two than any difference in the runtime environment.)

3

no, it’s the same database, mysql - in fact it’s exactly the same database host and tables. for the avoidance of doubt, both examples are running on the same machine.

sorry about the lowercase, shift key seems not to work with the browser.

i suppose what i meant earlier about checking the environments is: is there a django equivalent to phpinfo - ie that displays in the browser so you can be sure of the environment.

4

That’s really interesting. I wouldn’t have thought that your first version would work on anything other than Sqlite.

Where and how do you create member? (You show creating user, but not member.)

To see information about your environment, you could install django-debug-toolbar. It has a panel available to display a lot of detail. You might also want to log all SQL statements being executed, along with checking your server logs to see if there are any errors recorded.

5

Sorry, the member creation is immediately after the user creation:

       member = Member()

Surely the Django code is an abstraction of whatever database you’re running, so shouldn’t behave differently? Anyway, as above, same database, same host…

I thought that doing this:

user.profile.member = member

Told Django what it needed to know about the data that populated the objects for the models, and thus the member_id field should be set in that way.

If you’re indicating that I need to do this manually, ie as my fix, then it’s a bit odd because other instances of the same kind seem to assign automagically.

I’ll check the MySQL logs, but I think that it is unlikely this is the cause, as the database doesn’t change between deployments - and the profile record is created/written appropriately apart from the member_id field.

If it were a database write problem, then I’d expect the entire write to fail, not just one field.

I’ll try django-debug-toolbar to check the environments.

(Shift key working again!)

6

At the root of this, Django converts your ORM requests into SQL statements. It’s the SQL that is sent to the database. If the database being used doesn’t completely adhere to the SQL standard for relational databases, behavior will be different. So for example, if your database doesn’t fully validate the type of data being entered into a field, you could end up with the wrong type of data in that field. (Yes, this does occur.)

At that point in time, the user.profile object contains an object reference to the member object. However, the member object, having not been saved yet, does not have an ID from the database. It’s not until you do the member.save() that the ID field gets populated.
Now, whether this causes the member field of the profile object to get updated with that ID is the open question. My guess would be that it doesn’t, which is why it doesn’t surprise me that your first version fails in a production environment.

I would also guess that if you replaced these lines:

user.profile.account = account
user.profile.member = member
user.profile.member.account = account
user.profile.member.save()

with this:

member = Member(account=account)
member.save()
user.profile.account = account
user.profile.member = member

that it would also work

7

After some testing, I could see that the problem appeared to be with Django, in that user.profile.member_id was populated following user.save() in the runserver environment, but not in the Apache environment. With django-debug-toolbar installed I could also see that the SQL was as expected in the runserver environment.

The big revelation was that, somehow the runserver environment is running Django 3.2 and the Apache environment is running Django 2.2. So, I expect that this is the cause of the problem… just need to figure out how that is happening, since the debug info seems to indicate the same Python environment is being used for both environments…

Fixed a typo.

8

After some not inconsiderable exertion the solution seems to be to put this at the top of wsgi.py.

import sys
sys.path.insert(0,'/path/to/env/lib/python3.8/site-packages')

Not sure why this became such an issue with this project.

Anyway, now all environments (Apache and runserver) are serving Django 3.2 and the save() bug went away too.

9

Actually, depending upon how you’re running your application within Apache, this could lead to instabilities within your system. If you’re using mod_wsgi, this is definitely a bad idea. (If you’re not using mod_wsgi, if say you’re connecting to it running in uwsgi, it’s still not a great idea - there are better ways of handling it.)

10

I am using mod_wsgi, there seem to be many examples of people advising similar techniques, eg sys.path.append(), but that didn’t work for me – the modules found were mostly in the system Python, rather than the envs.

If there are other fixes then I am open to further investigation.

11

See the mod_wsgi docs for your configuration options. (e.g. See WSGIPythonPath)

But take note of this paragraph from the WSGIPythonHome directive:

Note that the Python installation being referred to using this directive must be the same major/minor version of Python that mod_wsgi was compiled for. If you want to use a different version of major/minor version of Python than currently used, you must recompile mod_wsgi against the alternate version of Python.

This means you can set up a virtual environment for mod_wsgi, but you need to ensure that the version of Python being used is the same version as that used to compile the mod_wsgi module.

12

Well, I only wish it was as easy as that. The relevant httpd.conf already has those settings, but they did not work in this case. To be clear, the system Python and the env Pythons are identical, and the mod_wsgi has been compiled against the system version. The issue is not different versions of Python, but that wsgi seemed to prefer the system Python modules over the env Python modules. In some cases this meant that the env modules were loaded, in others the system modules – but it wasn’t as straightforward as what was the earlier version…

As previously, a great deal of time has been expended coming to this solution, there might be another solution, but python-home and python-path aren’t it.

13

Yea, I feel your pain here.

It was this type of issue that lead us away from mod_wsgi to running our code under uwsgi. Running multiple versions and venvs is actually possible that way. (And it also made for an easy transition to nginx when we decided to go that route, too.) I know I haven’t had to configure an apache server for at least 7 years - may be closer to 8 or 9.