DateTimeField validation and microseconds weirdness.

Hello kind folx, I have some questions regarding the use and validation of DateTime fields:
I have a model Alias with two DateFimeFields : start and end. They should be saved with microsecond precision and complex business logic (not included in question) around it. End field should be able to be None. I am not concerned about forms, just about using ORM queries.

0. Why does the DateTimeField save 7 ints in microseconds if 1 millisecond is just 1000 microseconds ?
I couldn’t find an answer to this question, and therefore my approach was to give errors if the user enters more than 999 microseconds or not None:

class AliasManager(models.Manager):
    # get_aliases, etc.

class Alias(models.Model):
    start = models.DateTimeField()
    end = models.DateTimeField(blank=True, null=True)
    objects = AliasManager()

    def clean(self):
         # making sure microseconds are less than 1000 or not none

    def save(self, *args, **kwargs):
        self.full_clean() # supposed to run my clean()
        return super().save(*args, **kwargs)
  1. From the docs it seems like these form and field validators, are only needed in forms but it’s the place in the docs where to_python is mentioned; It seems like it’s called not only through forms but also somewhere within the pre_save or save methods ? I’m guessing this since before trying model validation I tried this:
def validate_microseconds(dt):
    if dt.microsecond > 999:
        raise ValidationError('microsecond value should be less than 1000')
    else:
        return dt

class AliasManager(models.Manager):
    # ...

class Alias(models.Model):
    start = models.DateTimeField(validators=[validate_microseconds])
    end = models.DateTimeField(blank=True, null=True, 
            validators=[validate_microseconds])
    def clean(self):
            # TypeError: strptime() argument 1 must be str, not datetime.datetime
        if datetime.datetime.strptime(self.start, "%Y-%m-%d %H:%M:%S.%f").microsecond >1000:
            raise ValidationError(
                {'start': "Microsecond value should be less than 1000"})
        if self.end is not None or datetime.datetime.strptime(self.end, "%Y-%m-%d %H:%M:%S.%f").microsecond >1000:
            raise ValidationError(
                {'end': "Microsecond value should be None or less than 1000"})

        # i suppose this is the same as above:
        validate_microseconds(datetime.datetime.strptime(self.start, "%Y-%m-%d %H:%M:%S.%f")) # TypeError: strptime() argument 1 must be str, not datetime.datetime
        validate_microseconds(datetime.datetime.strptime(self.end, "%Y-%m-%d %H:%M:%S.%f")) # TypeError: strptime() argument 1 must be str, not datetime.datetime

    def save(self, *args, **kwargs):
        self.full_clean()
        return super().save(*args, **kwargs)

And then it gives me typeErrors, which means that my string got converted to datetime object already:

>>> Alias.objects.create(atart="2020-02-01 00:00:00.00",end=None)
TypeEerror: strptime() argument 1 must be str, not datetime.datetime 

But now something strange happens:

>>>Alias.objects.create(start="2020-02-01 00:00:00.1000", end=None)
django.core.exceptions.ValidationError: {'start': ['microsecond value should be less than 1000', 'Microsecond value should be less than 1000']}

Which means it was the string that passed the strptime and got converted to DateTime object. WTF ?!

  1. So instead of “form and field validation”, I try to implement custom model validation :
class Alias(models.Model):
    start = models.DateTimeField()
    end = models.DateTimeField(blank=True, null=True)

    def clean(self):

            ## AttributeError: 'str' object has no attribute 'microsecond'
        if int(self.start.microsecond) >1000: # 200 turns to 2000000 !!!
            print("self.start.microsecond is: ", self.start.microsecond)
            raise ValidationError(
                {'start': "Microsecond value should be less than 1000"})
        if self.end is not None or int(self.end.microsecond) >1000:
            raise ValidationError(
                {'end': "Microsecond value should less than 1000 or end should be None"})

            # this gives error as well: 
        validate_microseconds(self.start) # AttributeError: 'str' object has no attribute 'microsecond'
        validate_microseconds(self.end) # AttributeError: 'str' object has no attribute 'microsecond'
       
            # even this: 
        validate_microseconds(datetime.datetime.isoformat(self.start)) # TypeError: fromisoformat: argument must be str
        validate_microseconds(datetime.datetime.isoformat(self.end)) # TypeError: fromisoformat: argument must be str

And as I’ve put in the comments, this happens:

Alias.objects.create(start="2020-02-01 00:00:00.00", end=None)
AttributeError: 'NoneType' object has no attribute 'microsecond'

yet…

Alias.objects.create(start="2020-02-01 00:00:00.100000", end=None)
django.core.exceptions.ValidationError: {'start': ['Microsecond value should be less than 1000']}
  1. My custom str method returns “+00:00” in the end of these fields, is that a timezone ?
def __str__(self):
        return 'start:%s,end:%s' % (self.start, self.end)
>>> Alias.objects.first()
<Alias: useful-object,start:2021-05-27 05:20:45.280373+00:00,end:None>
>>> Alias.objects.first().start
datetime.datetime(2021, 5, 27, 5, 20, 45, 280373, tzinfo=<UTC>)
  1. how do I set “from the beginning of time” and “until the end of time” for optional arguments ? :slight_smile:
class AliasManager(models.Manager):
    def get_aliases(target, since="1000,1,1", until="3000,1,1"): # here
        Alias.objects.filter(start__gte=datetime(since),
                            end__lte=datetime(until))
  1. If I want end field to be either datetime or None, is this even a correct way to do it?:
    end = models.DateTimeField(blank=True, null=True)

I think you’re misunderstanding the nature of the microsecond value here.

That fractional portion of the time is the number of microseconds within the current second. If you limit the value to three decimal places, you’re restricting it to millisecond accuracy.

e.g. 2020-02-01 00:00:00.1000 means 1000 a tenth of a second after midnight, with an implied precision of a tenth of a millisecond.

The time 2020-02-01 00:00:00.123456 means 123,456 microseconds after midnight at the implied microsecond accuracy.

Truncating that to 2020-02-01 00:00:00.123 represents 123 milliseconds after midnight.

One microsecond after midnight would be 2020-02-01 00:00:00.000001

One millisecond after midnight (with a full implied precision) would be 2020-02-01 00:00:00.001000

Technically, it’s the UTC offset. “-05:00” either represents US Eastern Standard Time or US Central Daylight Time - two different timezones.

There are a couple different ways. Using boundary values is one common method when nulls aren’t allowed in the field.

The blank=True has no semantic meaning in a datetime field. (Not sure that’s 100% correct across all databases supported by Django.) The null=True is all you need.

Sir, Your response is greatly appreciated !