Use a weak reference for file field memory optimization

Django Internals
1

Ticket

pr

I wanted to bring this discussion to the forum. If the problem is not clear, check out the steps to repro in an example project.

In the pr, I am adding a weak kwarg available for a FileField. I show the threshold where amount of data on an instance caused by the FieldFile reference cycle would make a difference in system memory utilization. In my opinion, having half a gig of garbage from a 10mb model instance reference cycle is significant.

I think there are 2 things left to discuss:

  1. Is the built in exception good enough?
  2. Should we consider making weak=True the default?
  1. Is the built in exception good enough?

Yes, I think it is. I have an example traceback from the test with the assertion catching removed

======================================================================
ERROR: test_weak_file_field_save_out_of_scope (model_fields.test_filefield.FileFieldTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/massover/Personal/django/tests/model_fields/test_filefield.py", line 204, in test_weak_file_field_save_out_of_scope
    weakfile.save("new-name", new)
  File "/Users/massover/Personal/django/django/db/models/fields/files.py", line 97, in save
    setattr(self.instance, self.field.attname, self.name)
ReferenceError: weakly-referenced object no longer exists
  1. Should we consider making weak=True the default?

I don’t think we need to go down this path. First, there hasn’t been a single other comment on the ticket where someone has noticed this is similarly affecting them. Second, some footguns exist in the orm (eg. querying for nested relations without prefetching). If properly documented, the weak=True option is available if people notice memory issues caused by the cyclic reference.

If we decide on (1) and (2), I can go ahead and update the pr with documentation.