Django command behaving differently when run from termin vs unit test

Hi,
We’ve been using Django since 1.3, but have recently come across a strange bug (behaviour ?) when running one of our commands.
This has only started since upgrading from ubuntu 18 to 20, and python 3.6 to 3.8

We’re currently using Django 3.0, with MySQL 8, running Python 3.8 on Ubuntu 20

Our command basically does:

> pull data
> format data
> _insert_my_data(format_data)
> save to "my-file.csv" in system /tmp dir

and then:

from django.db import connections

connection = get_connection(connections, 'account-id')

def _insert_my_data(self, formatted_data):
...
> write formatted_data to "my-file.csv" in system /tmp dir
...
with connection.cursor() as cursor:
    my_query = f'''
    LOAD DATA LOCAL INFILE "{file_name}"
    REPLACE INTO TABLE
    FIELDS TERMINATED BY ","
    OPTIONALLY ENCLOSED BY '"'
    LINES TERMINATED BY '\\n'
    '''
    cursor.execute(my_query)

When i run this command from the terminal:sudo python3 manage.py update_my_stuff it all works.
And has been working this way for years.

I have a test command, that calls this function and passes a similar array of “pull data” that is formatted correctly.

Since upgrading I now get this error when I run the test NOT when I run the command as normal:

django.db.utils.OperationalError: (2068, 'LOAD DATA LOCAL INFILE file request rejected due to restrictions on access.')

The command, and test command, run on same DB server - where we’ve got “local-infile=1” already set.

I’ve been fighting this for a few days, and have no idea what it could be.

Is the test run with different user/permissions? And so the sql lib cant access “/tmp” ? Im not even too sure how I could test that.
I’m really at a loss, any ideas or pointers would be greatly appreciated.

This means you’re running that command as root.

How are you running the test? If you’re not running the test as root, then it may be a permissions issue.

See the security requirements at MySQL :: MySQL 8.0 Reference Manual :: 13.2.7 LOAD DATA Statement

Hey Ken,
Cheers for messaging back. To be clear I’m running the test as: sudo python3 manage.py test my_app.tests.my_test_file.SpecificTestUnit
And just for my sake, that should have same permissions right?

Yes it should. You’d be executing it as root in that case.

Just for completeness, I noticed the mis-matched braces and quotes combination above.

Is this a copy/paste error? If that’s what you have in your actual code, that’s an error.

Hey Ken,
Good spot, I’ve written a psuedo version of the code and that is a lil-typo in the comment (I will edit it now).

From what I’ve described, this behaviour doesnt add up right ?
The command run from cmd line, and the test calling the same func in that command, should behave the same?
Django doesnt use a custom test/temp user to run the tests on the DB ? And I assume when the DB is made it obeys the same “global variables” set on the DB (in this case, for example, “local_infile=ON”)

Not that I’m aware of.

I would also make that assumption - but in this situation I’d try to verify that.

Unfortunately, we’re now well outside my areas of direct knowledge - hopefully one of the more knowledgeable people here can jump in.

@adamchainz - any thoughts?