Dynamic model relations

Using Django Using the ORM
1

Im building an SAAS with a model Project.
Each Project can have Events, but the event-details (Model fields) to be stored are vastly different between each Project.

To handle the diversity of Event-fields, ive come to believe i need to create a new Model for each project, maybe named ProjectnameEvent.
But doing so, how would i be able to import and query the correct EventModel in the views, forms and so on?

Alternatively, would it be a better idea (and possible?) to make a model BaseEvent with project_id, and then extend this like ProjectnameEvent(BaseEvent): …project_specific_fields…

Searching the forums ive come to realize this might be a case for GenericForeignKey, but even then, im unsure of how to import/query the correct models im the view/forms mm.

As you can undoubtably tell, i am fumbling in the dark for a solution, so hopefully you can point me in the right direction.
Thank you - Martin

2

Hi

It’s certainly possible to use dynamic tables, with Django - But most of the time I’ve seen this done its for multiple tenant applications. I am aware of one (closed-source) application I’ve met in my ‘travels’ , which does has a very similar problem and pattern to what you suggest here, and in that case the ORM is not used at all, and raw SQL queries are produced for the equivalent of event details in your case.

To do it with the ORM you would want something like this:

 class EventDetailBase(models.Model):

	class Meta:

		abstract = True

		

	event = models.ForeignKey(Event)



then:

@lru_cache

def get_details_model(project):

 	fields = {}

	for fld in project.detail_fields:

	  fields[fld.name] = fld.class(**fld.attrs)

	return type(f"{event.slug}_details",EventDetailBase, fields)

Inside the loop fld.name contains a string name for the field, fld.class will contain the class eg, models.CharField, and the fld.attrs will contian a dict of the kwargs used to create the field, you can store or derive these in whatever way makes sense for your app. I seem to recall there are some timing issues, so you might need to find a way to run get_details_model for each Project before django.setup().

Alternatively, while often considered a anti-pattern I’ve actually had good success with the EAV model (https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model), in similar use cases.

GenericForiegnKey is allows references (ie foreign keys) into ‘any’ table known to the ORM, you’ll only need that if you need foriegn keys into your details tables, which isn’t a common pattern. But you do need to be able to reliably name the tables with a static (‘app_name’, ‘model_name’) tuple . Admittedly this last isn’t a high bar.

Hopefully that gives you a few more jumping off points for you own thoughts.

Cheers.

3

I’d avoid GFKs unless you end up with no other way of doing this. And for this basic situation, I don’t see where it’s necessary.

You can take advantage of a lot of the flexibility within Python by the appropriate use of the hasattr and getattr methods to dynamically access attributes within objects.

In this particular case, you may use code something along lines like this. (Note: This is not complete / functional code, I’m just hoping to present enough of an idea to get you started.)

# models.py
class EventTypeOne(Model):
    project = ForeignKey('Project', related_name="event_type_one")
    ...

class EventTypeTwo(Model):
    project = ForeignKey('Project', related_name="event_type_two")
    ...

class EventTypeThree(Model):
    project = ForeignKey('Project', related_name="event_type_three")
    ...

class Project(Model):
   # String containing the related name for the Events of this project
    project_type = CharField(...)  

    # Retrieve the related Events
    def get_related_events(self):
        related_name_manager = getattr(self, self.project_type)
        return related_name_manager.all()

So through the judicious use of naming conventions and the introspective functions, you can write very generic code to handle this.

Using a class hierarchy can also work, where the EventType models inherit from Project.

4

Thank you for your detailed response!
My project is indeed tenant-based.
Ive been looking into your suggestion of EAV-models the entire weekend, and it might just be what i will go for!

5

Thanks KenWitesell,
i will skip the GFKs for now.
Your suggestion looks simple, and i like simple.
I think i will have to test-build both an EAV system as suggested by rgammans and the system you suggest, in order to test which fits my project the best.
Thank you for pushing me ahead.

6

Ive tried to implement your method @KenWhitesell, but i have a hard time figuring out how id go about using the specific EventTypeN model when including it in forms and so on?

I also have come across a method which might also work, please chime in if you see something that could wreck havoc down the line:

class Project(models.Model):
    event_table_name    = models.CharField(max_length=100, null=True, blank=True) 

class EventTypeTwo(models.Model):
    event_date      = models.DateField()

Now if i do Project.event_table_name = “EventTypeTwo”, i could do my views like:

from django.apps import apps
def index(request):
    project= Project.objects.get(id=request.session['project']['id'])
    EventModel = apps.get_model('events', project.event_table_name)
    events = EventModel.objects.all()

And i would be able to include the EventModel in forms and so on.

Any thoughts regarding this method?

7

What is this Inclusion model and how does it relate to Project?

What is this Study model and how does it relate to Project?

That’s fine if you want all EventTypeTwo. But what if you have two different Project that are both related to EventTypeTwo? That’s the purpose of the get_related_events model method.
For any instance of Project, it returns a queryset containing the proper related event type.

For example , if you have an instance of Project named project, and project.project_type is EventTypeTwo, then project.get_related_events() is the query you’re looking for. It returns the same queryset that you would get if you were to code EventTypeTwo.objects.filter(project=project). You don’t need that extra code in index. (And, if the only entries in EventTypeTwo relate to project, then it’s the same result as EventTypeTwo.objects.all().)

8

Apologies, i changed the class name from study to project in this thread to make it more generic, ive edited it now.
Inclusion is not relevant for the discussion, sorry for the confusion.

I see where you are going regarding if projects share the EventType model.
This however will not be relevant as each Project will have its own EventTypeModel.

But how would i get an instance of the specific EventType model? Like in a detail or update -view?

9

You can iterate over the queryset (e.g. ListView) or apply a filter to the queryset (Detail or Update).

You don’t show any additional fields on the EventTypeTwo model, so I can’t show the specifics, but this is no different than any other queryset.

10

Ah, i think i get it now - Thanks for being patient!

And i hope i dont bother you now, but as each project have its own EventType, do you have a suggestion as to how in the view i can instantiate the correct form?

#forms.py:

class EventTypeTwoForm(forms.ModelForm):
    class Meta:
        model = EventTypeTwo 

class EventTypeNForm(forms.ModelForm):
    class Meta:
        model = EventTypeN

#Views:

def event_update(request, event_id): 
    project = Project.objects.get(id=request.session['project']['id'])
    events = project.get_related_events()
    event = events.get(id=event_id)

    form = [FORM?](request.POST or None, instance=event)

If the current project uses EventTypeSix model for instance, how would i make my event_update view instantiate precisely EventTypeSixForm?

11

Same type of naming convention using the getattr method.

For example, using your names, and assuming you’ve done something like import myapp.forms it could be:
form = getattr(myapp.forms, f'{project.event_table_name}Form')(request.POST)
(and you can use something similar to this for templates.)

The key to making this all work easily is to define and enforce the appropriate naming conventions for the entities (forms, templates, etc) such that you can use getattr to retrieve the proper entity.

There are other ways of doing this to account for special cases, but this is the basic idea that you can build upon.

Side note: You can combine two steps above to be:
event = project.get_related_events().get(id=event_id)
They don’t need to exist as separate statements.
You could also modify that method to allow for a filter to be supplied and applied within that method, or you could even create additional model methods. You don’t need to do all this work in the views.

12

Holy moly, it works.

Thank you Ken!