Get datetime now, not at server initialisation

Using Django
#1

I have a function that uses the current date to set up a file path for uploaded images:

models.py

from sorl.thumbnail import ImageField
from datetime import datetime, timedelta, timezone
…

def upload_path():
	dtnow = datetime.now(timezone.utc)
	dtstr = '{:%Y/%m/%d/%H/%M/%S}'.format(dtnow)
	print('dtstr: ')
	print(dtstr)
	dtstr = 'images/items/' + dtstr
	return dtstr

class Image(models.Model):
	item = models.ForeignKey(Item, on_delete=models.CASCADE)
	#file = ImageField(_('image'), upload_to='images')
	file = ImageField(_('image'), upload_to=upload_path())

The problem is that when the Image class calls ‘upload_path’ the datetime portion of this is always the runserver initialisation time, rather than the time when ‘upload_path’ is called/used.

How to fix this?

Thanks

#2

I think this happens because the function is defined with parenthesis.
This way, when you execute runserver Django loads the models and execute your function.

Try removing the parenthesis, letting only the function name.

#3

Thanks, I tried:

def upload_path():
	dtnow = datetime.now(timezone.utc)
	dtstr = '{:%Y/%m/%d/%H/%M/%S}'.format(dtnow)
	print('dtstr: ')
	print(dtstr)
	dtstr = 'images/items/' + dtstr
	return dtstr
		

class Image(models.Model):
	item = models.ForeignKey(Item, on_delete=models.CASCADE)
	#file = ImageField(_('image'), upload_to='images')
	file = ImageField(_('image'), upload_to=upload_path)

But then I get an error in the browser:

Request Method: 	POST
Request URL: 	http://testserver3.indx.co.uk:8001/edit/36/
Django Version: 	2.2.16
Exception Type: 	TypeError
Exception Value: 	

upload_path() takes 0 positional arguments but 2 were given

Exception Location: 	/usr/local/envs/superseller/lib/python3.8/site-packages/django/db/models/fields/files.py in generate_filename, line 303

files.py has this on line 303

        """
…
301     """
302     if callable(self.upload_to):
303         filename = self.upload_to(instance, filename)
        else:
            dirname = datetime.datetime.now().strftime(self.upload_to)
            filename = posixpath.join(dirname, filename)
        return self.storage.generate_filename(filename)
#4

It seems you need to add those 2 arguments instance and filename to your custom function, because it’s what Django expects.

The example is pretty simple.

#5

Thanks, that was very helpful. So it turns out that the easiest way to do this is just to change the file line in the class to:

    file = ImageField(_('image'), upload_to='images/items/%Y/%m/%d/%H/%M/%S')

There are a few things that I do not understand, however, after playing around with some code for a while.

If I just pass a string to file it understands that this is a partial path, and it has to stick filename on the end. eg:

file = ImageField(_('image'), upload_to='images/items')

Likewise with the version above with the date formatting.

This is also the case when I used my initial upload_path function, calling it with upload_to=upload_path() – but this did not suit the task, because upload_path loaded at initialisation of the server.

So, given that upload_path returns a string, I don’t understand why I also have to concatenate the filename:

def upload_path(instance, filename):
	dtnow = datetime.now(timezone.utc)
	dtstr = '{:%Y/%m/%d}'.format(dtnow)
	dtstr = 'images/items/' + dtstr + '/' + filename
	return dtstr

In this case, if I do not concatenate filename then the uploaded file gets named with the last element of the date formatting and without a file extension, ie potentially like this:

images/items/2020/10/28/20/59/55

rather than:

images/items/2020/10/28/20/59/55/image.jpg

The other thing I don’t understand is how instance and filename are passed to the function upload_path. I would expect to do something like:

file = ImageField(_('image'), upload_to=upload_path(instance, filename))

But if I do that I get:

NameError: name 'instance' is not defined

Sorry for so many more questions!

#6

instance and filename needs to be defined in the definition of the upload_path function. In the ImageField you only need to declare the function name.

Sorry but I don’t see what’s your other question :sweat_smile:

#7

Basically I’m wondering why upload_to sometimes seems to take a string, and sometimes does not, and how the function upload_path knows the values of instance and filename when there seems to be no mechanism that passes those values.

#8

This is the way Django generates filenames for FileField and ImageField.

If you define a function, it’s called with the instance and filename arguments.

If you pass a string, eg. "/images/%y/%m/%d" it’ll be formatted like '/images/20/10/29'.

#9

I understand that the function wants/expects the two inputs. What I don’t understand is how it is getting those inputs - is it just implied because the function is defined that way?

For example, here’s a simple Python function (from https://www.w3schools.com/python/python_functions.asp):

def my_function(fname):
  print(fname + " Refsnes")

my_function("Emil")

We can see that the argument “Emil” is passed as the parameter fname to the function my_function. I don’t seee that process when instance and filename are passed to upload_path.

Thanks

#10

It’s in the generate_filename method posted above - specifically, this line:

#11

Thank you for taking the time with this, which is obviously a naive question, however I am still not getting the answer (I think) I need. I think part of the problem is that I am not being clear enough in my question, because I have mixed-up several questions in non-obvious ways, so I will simplify it to the core question, which is: how do arguments get passed to a function if no parameters are passed?

Here is my example:

def MY_UPLOAD_PATH(instance, filename):
	dtnow = datetime.now(timezone.utc)
	dtstr = '{:%Y/%m/%d}'.format(dtnow)
	dtstr = 'images/items/' + dtstr + '/' + filename
	return dtstr

class Image(models.Model):
	item = models.ForeignKey(Item, on_delete=models.CASCADE)
	file = ImageField(_('image'), upload_to=MY_UPLOAD_PATH)

In the above example, no arguments are passed to MY_UPLOAD_PATH, so it is not clear to me how MY_UPLOAD_PATH forms a file path with the filename – since filename is needed.

That is to say that this code is not used:

file = ImageField(_('image'), upload_to=MY_UPLOAD_PATH(instance, filename))

The only explanation that I can think of is that instance and filename are some species of global variable, that my function understands are available. Or, because upload_to is being set to equal MY_UPLOAD_PATH the arguments are passed there!?

Feel free to point me to some basic programming primer!

#12

( :bulb: Now I understand the question…)

Let’s start with a basic function:

def my_sum(a, b):
    return a+b

The statements above define a “thing”, usually called either “function” or “method”. This “thing” can also be generally referred to as a “callable”, which means that Python can call it.

We have assigned the name “my_sum” to this thing, but it’s important to remember that the name “my_sum” is a reference to that thing.

So, where am I heading with this?

If I write the statement:
a_total = my_sum(2, 3)

Python identifies that I’m trying to execute that thing we’ve defined earlier (because of the parameters being identified within the parenthesis), and so it calls the function and returns the result.

But, if I execute the following statement:
another_sum = my_sum
I’m not trying to execute the function identified by “my_sum”. Instead, I’m locating the thing that my_sum is referring to, and defining that the name “another_sum” is referring to the same “thing”.

So what has happened is that another_sum is not the return value of the function being executed, it’s another reference to the function itself. That means I can now execute:
a_total = another_sum(2, 3)
and get the same result.

So coming back to your question above, your statement:

is not trying to execute the function identified by MY_UPLOAD_PATH. What you’re doing is passing the reference to the function as a parameter. This means that the variable named “upload_to” can be used to execute your function - and that’s what the highlighted line of code is doing:

Again, your function isn’t being called when you define the ImageField. You’re passing the reference to that function so that the ImageField object can execute it when it’s needed - and when it’s needed it has the instance and filename parameters to pass to it.

(If this still isn’t resonating with you, I suggest you try the “my_sum” example above in the Python interpreter and play with it a bit until to “get” what’s happening. This is a very important concept to be comfortable with when working with Django. There are many cases where you’re going to be passing the names of functions and classes as parameters rather than instances of those items, and it’s a critical issue to understand the difference between the two.)

1 Like
#13

Thank you for the explanation, and for the time you have spent on it. I think I understand the theory:

if I have a function:

def function_one(x, y, z):
        return x + y + z

I can assign that function to a variable:

thesum = function_one

And I can assign the same function to another variable:

theothersum = function_one

While at the same time I can also assign other values, like strings, integers, etc to those variables

thesum = 'hello world'
theothersum = 1

Or if I use parentheses and some parameters I get the output from the function:

thesum = function_one(x, y, z)

And, assuming x, y and z were integers type(thesum) would output <class 'int'>.

Which is what is relevant to generate_filename

I suppose what you can also do (amongst other things) is have another function function_two, that you can also assign to thesum or theothersum. So effectively you have the same function able to be referenced by different names, different functions referenced by the same name, the vars just representing fixed values and combinations thereof.

From there I think I am struggling to see what might be the practical purpose. I can possibly see that assigning the function to a variable might mean that you do not repeat code that “sorts” the output according to what is received, as generate_filename does.

I think I am also struggling to understand why you’d want a function that is set at initialisation time, and then not callable/updateable on demand. For example:

>>> from datetime import datetime
>>> from datetime import timezone
>>> def the_time():
...     return datetime.now(timezone.utc)
... 
>>> print(the_time)
<function the_time at 0x63ef60>
>>> type(the_time)
<class 'function'>
>>> print(the_time())
2020-11-01 21:56:36.867347+00:00
>>> print(the_time())
2020-11-01 21:56:39.237433+00:00
>>>

So, here, each time I call the_time() I get an updated time string.

#14

Yes, you’re starting to get the idea. In the general case, variables are names referencing objects. The names are not the objects, they’re names for the objects. That means you can have multiple names referring to the same object, or, you can reassign a name to refer to a different object. (I’m just rephrasing what you wrote above, kinda trying to summarize it in different words.)

You’ve identified one - you provide a function that takes two parameters and returns a result. You don’t need to change the code that is going to use your function, you’re just passing the function to it. It’s an extremely useful pattern for people who are writing software for public consumption. These “function parameters” create extension points for customizing libraries. You’ll find multiple locations within Django where you pass a function as a parameter for just these purposes.

But then you kinda lost me

You then followed that with an example where the function returns different values each time, so I’m not sure what you’re referring to here.

Making one guess from one angle, I’ll try to answer this by saying that objects within Django don’t live forever. For example, every time you handle a request, a new instance of your Form is created - and so needs to be created / initialized for that request. If you’re using Class Based Views, new instances of those views are created for each request.

#15

In my first post there’s a code example

rom sorl.thumbnail import ImageField
from datetime import datetime, timedelta, timezone
…

def upload_path():
	dtnow = datetime.now(timezone.utc)
	dtstr = '{:%Y/%m/%d/%H/%M/%S}'.format(dtnow)
	print('dtstr: ')
	print(dtstr)
	dtstr = 'images/items/' + dtstr
	return dtstr

class Image(models.Model):
	item = models.ForeignKey(Item, on_delete=models.CASCADE)
	#file = ImageField(_('image'), upload_to='images')
	file = ImageField(_('image'), upload_to=upload_path())

So that when ipload_to=upload_path() the datestring elements that are returned are those related to the datetime at server initialisation, rather than the datetime the the image was actually uploaded.

This was the part I was indicating that I did not undestand why you’d want a function that did that.

#16

You generally don’t. As I look back on this thread, it seems to me that that was the cause of your original issue.

However, regardless of how frequently you may or may not want to do this, it is syntactically valid. The docs for upload_to point out that you can supply either a string or a callable. Since your function returned a string, it was proper to use it in that location.

Looking at it from a different angle, and it’s a bit of a stretch, but one reason why you might want to call the function at that point is if you had some configuration setting that would make the path more dynamic based upon the instance of the server being run. In other words, you want to set the path to something that isn’t going to change while the system is up and running, but is dependent upon some piece of information not necessarily available at the time the code is being developed and tested. (I’m not sure I would ever choose to do it that way, but I couldn’t make a case against anyone who did make that choice.)

#17

I think we can agree that there is a possibility that someone might want to lock in a value at server initialisation. However, I think that is more appropriately done in a settings file.

The point I was trying to make is that it seems to be a bit odd to have a function that, when executed, outputs the datetime at server initialisation, rather than the datetime when called.

Thanks for you input on this problem.

#18

Thank you for helping me.
I have removed prints cause these are exactly printing every refresh of server