Hi all - I’m trying to use the ‘natural_keys’ settings for serialization.
I thought I was pretty clever defining ‘natural_key_fields’ as a tuple to be listed in each child model with one generic implementation of “natural_key” for any model and “get_by_natural_key” for models.Manager. All models in the project inherit from classes with these two additions (my own BaseModel and DiscreteManager). But I have run aground setting natural_key.dependencies, which is a list of strings defining the entities that should be serialized before the current model. There are two parts:
First, all the real children of BaseModel don’t take well to being given ‘natural_key.dependencies = []’ in the class definition, because natural_key is not defined, despite the inheritance meaning it is there when called. I’m not sure if inherited code is ‘there’ in the same way as explicit code; perhaps the children just have pointers to the parent code. If so, how can I set natural_key.dependencies for child classes where I don’t want to make an explicit ‘def’ of the natural_key function?
Second, my motivation for trying to DRY the natural_key functions was that our team develops and trials apps with pretty high turnover. The apps are distinctive UI experiments for different domains, and may come to me developed by people on a minimal django installation rather than built to fit the whole site. I wanted to minimise the amount of revision required to enroll an app as part of the overall site - if I could just swap in BaseModel for anywhere someone has used models.Model, it would save us all some work and keep the standalone and integrated versions of the code closer together. Is there a way I could define a function to get_natural_key_dependencies that looks for non-nullable foreign keys (and one-to-ones etc) and assembles the precedence list of the schema graph?
Third, this is all to avoid problems with primary key clashes on dumpdata and loaddata. Dumping with primary keys and even natural-primary has worked well, but if the foreign_key fields are still referring to the ‘id’ field, its not so helpful.
There are entities with relationships to contenttype, and others with relationships to Group from the auth app, which means excluding contenttypes or auth is not really feasible while retaining db integrity. I’d really like to be able to load fixtures for testing, but I am not succeeding.
There are two fundamentally different ways to define a Model class hierarchy, and it depends upon whether or not you set abstract = True in the base class Meta.
However, without abstract = True, your Base class actually defines a separate and distinct model from the derived class, using a one-to-one relationship defined in the derived class to create the association between the two classes.
If you’re using an abstract base class, I know of no reason why a natural key definition in the base class wouldn’t work. (We might be doing that in one of our systems, but I’m not sure right off-hand.)
Also, when you use dumpdata to export the data, natural-primary will create the entries in the fixture with the natural keys, and the natural-foreign parameter will create foreign key references using the natural keys. (It works great, we use it constantly.)
Thanks Ken. Yes its an abstract base class, so there’s no table.
class BaseModelWithHistory(models.Model):
natural_key_fields = ('id',) # default
class Meta:
abstract = True
def natural_key(self):
fieldlist = [getattr(self, fieldname) for fieldname in self.natural_key_fields]
return tuple(fieldlist)
natural_key.dependencies = ['projects.Project', 'projects.Task']
Then in the Manager:
def get_by_natural_key(self, *args):
z = zip(self.model.natural_key_fields, *args)
result = self.get_queryset()
for pair in z:
q = models.Q(pair)
result = result.filter(q) # should be single entry in queryset
result = result.get(q) # unpack from qs to object
return result
Child models in this app are Project and Task which also have relationships to Group and Contenttype. Child models in other apps may be dependent on Task or Project. e.g.
class Question(BaseModel):
question_text = models.TextField()
task = models.ForeignKey(Task, on_delete=models.CASCADE)
# can't do this -> natural_key.dependencies = ['projects.Task', ]
The first problem arises when I want to define natural_key.dependencies specifically for Question in the class definition. This is the normal case (most models are in this category), and there are e.g. Answer models that need to be serialized after the Questions. I am probably being unreasonable.
Ok, I think I’m following you here. Unfortunately, I’m not at a computer at the moment where I can try this out, but everything you’ve said sounds reasonable to me.
There is one tiny little thing though. When I look at the docs for the natural key dependencies, I see where they specify the name using the lower case reference name and not the capitalized class name. Have you tried specifying it as ‘projects.task’?
Oh yes, that’s right. It didn’t miraculously start working - there are other problems, but thanks for spotting that. I’m in Australia and am about to stop for now; last hour or so I’ve been working through ways to trace the dependencies. If I knew the framework deeply enough, I’m sure what I need is in there. Thanks, Ken
Andrew
I’ve had a little bit of time this morning to chase this down some (I’m east-coast US), and I can see (in general) where the problem lies.
I’ve traced it far enough to realize that defining a model involves some seriously intense metaprogramming at the class level. The way a model class is built is highly customized - and quite honestly, very opaque to me. Generally, I’d say that the normal expectations for Python class inheritance don’t apply to Django Models.
All this is just to say that I’m still convinced it’s possible, but I now seriously doubt it’s going to be easy.
Update on this:
I seem to have fixed the specific hurdle here by placing my universal ‘natural_key’ definition inside init_subclass() definition. This allowed me to define natural_key_dependencies as a list on the subclass and assign that to natural_key.dependencies during init_subclass. Child classes can define a tuple of natural_key fields and a list of app.model dependencies, but with defaults in the base model it isn’t essential.
It’s a long way from my initial question about automatically finding the dependencies, but with a sensible default dependencies in the BaseModel class, I think this is going to work.