Short relations.... They end when updating Organisation

My relation instances last untill I update the Organisation instance it refers to. I have checked a lot and it always occurs around my organisation = organisation_form.save(). Before that the relation exists and after that the relation is gone. Note I use PolymorphicModel

I have created:
class BaseModel(PolymorphicModel)

Class Context(BaseModel)Entity(Context)

The relation model:

class Relation(BaseModel):
    organisational_entity = models.ForeignKey(
        OrganisationalEntity,
        on_delete = models.CASCADE)

    person = models.ForeignKey( 
        Person, 
        on_delete = models.CASCADE, 
    )

   relation_specification = models.CharField(
            max_length=200,
            blank=True,
            null=True, 
            help_text = \_("The relation between a person and an organisation. This can be a client, employee, customer, member (of a club), board member and mutch more.")
        )

    start_date = models.DateField(\_('start date'), null=True, blank=True)
    end_date = models.DateField(\_('end date'), null=True, blank=True)

    def get_absolute_url(self):
        return reverse("relation-detail", kwargs={"pk": self.pk})

    def get_display_text(self):
        return self.relation_specification

    def __str__(self):
        return self.relation_specification + str(self.organisation) + str(self.person)

    def save(self, \*args, \*\*kwargs):
        self.clean()

        self.relation_specification = self.relation_specification.lower()
        self.relation_specification = self.relation_specification.strip()
        self.relation_specification = re.sub(r'\\s+', ' ', self.relation_specification)

        super().save(*args, **kwargs)

         return super(Relation, self).save(*args, **kwargs)

    class Meta:
        ordering = \['relation_specification', '-start_date', 'end_date'\]

The

class OrganisationalEntity(Entity):
    relations = models.ManyToManyField(
        Person, 
        through="Relation",
        blank=True,
    )

and

The Organisation model:

class Organisation(OrganisationalEntity):
    SUBJECT_ROOT_KEY = "organisations"

    name = models.CharField(_('organisation name'), max_length=100)
    description = models.TextField(_('description'), blank=True )

   pass

and
class Department(OrganisationalEntity):

The def edit_organisation(request, pk=None): Is a beast of a function. I dont know what is needed here so…

  • I create all forms and formsets,
  • Check if they are empty or valid (I ignore empty forms except the main organisation form)
  • I save the organisation (organisation = organisation_form.save())
  • else create the empty form
  • I create the context

I dont know how to check if on_delete = models.CASCADE could be the cause but then again… the organisation exists before and after save with the same ID.

The possible error location is at the save in the middle of:

def edit_organisation(request, pk=None):
    organisation = get_object_or_404(Organisation, pk=pk) if pk else None

    if request.method == "POST":
        organisation_form = OrganisationModelForm(
            request.POST, 
            instance=organisation
        )

        # --- Subformsets ---
        phonenumber_form_set = PhoneNumberFormSet(
            request.POST,
            queryset=organisation.telephoneNumbers.none() if organisation else PhoneNumber.objects.none(),
            form_kwargs={"required_fields": False}
        )
        address_form_set = AddressFormSet(
            request.POST,
            queryset=organisation.addresses.none() if organisation else Address.objects.none(),
            form_kwargs={"required_fields": False}
        )
        email_form_set = EmailFormSet(
            request.POST,
            queryset=organisation.emails.none() if organisation else Email.objects.none(),
            form_kwargs={"required_fields": False}
        )
        website_form_set = WebsiteFormSet(
            request.POST,
            queryset=organisation.websites.none() if organisation else Website.objects.none(),
            form_kwargs={"required_fields": False}
        )
        
        relation_form_set = RelationFormSet(
            request.POST,
            queryset=organisation.relation_set.all() if organisation else Relation.objects.none(),
            form_kwargs={
                "required_fields": False
            }
        )

        def form_ok(form):
            return (not form.has_changed()) or form.is_valid()

        def formset_ok(formset):
            for form in formset:
                if form.has_changed() and not form.is_valid():
                    return False
            return True

        all_valid = (
            form_ok(organisation_form) and
            formset_ok(phonenumber_form_set) and
            formset_ok(address_form_set) and
            formset_ok(email_form_set) and
            formset_ok(website_form_set) and
            formset_ok(relation_form_set)
        )

        if all_valid:
            if organisation is None and not organisation_form.has_changed():
                messages.warning(request, _("Edit organisation: An empty organisation form cannot be saved"))
            else:
                organisation = organisation_form.save()
                if hasattr(organisation_form, "save_m2m"):
                    organisation_form.save_m2m()

                if (
                    "add_related_subject" in organisation_form.cleaned_data and
                    organisation_form.cleaned_data["add_related_subject"]
                ):
                    root = get_or_create_subject(
                        name=Organisation.SUBJECT_ROOT_KEY,
                        parent=None,
                        system_subject=True
                    )

                    subject = get_or_create_subject(
                        name=organisation.name,
                        parent=root,
                        system_subject=False
                    ) 

                    organisation.subjects.add(subject)

                def save_and_link(formset, related_manager):
                    for form in formset:
                        if form.has_changed() and form.is_valid():
                            instance = form.save()
                            print("org save form(set) ",instance)
                            related_manager.add(instance)

                save_and_link(phonenumber_form_set, organisation.telephoneNumbers)
                save_and_link(address_form_set, organisation.addresses)
                save_and_link(email_form_set, organisation.emails)
                save_and_link(website_form_set, organisation.websites)

                # --- Save Relations ---
                for form in relation_form_set:
                    if form.has_changed() and form.is_valid():
                        relation_spec = form.cleaned_data.get("relation_specification")
                        person = form.cleaned_data.get("person")

                        Relation.objects.create(
                            person=person,
                            organisational_entity=organisation,
                            relation_specification=relation_spec
                        )

                return redirect("organisation-list")
        else:
            messages.warning(request, _("Edit Organisation1: Some fields are not valid. Please correct the errors below."))

    else: # GET: instantiate forms
        organisation_form = OrganisationModelForm(instance=organisation)

        phonenumber_form_set = PhoneNumberFormSet(
            request.POST if request.method == "POST" else None,
            queryset=PhoneNumber.objects.none(),  # only allow adding new addresses
            form_kwargs={"required_fields": False}
        )
        address_form_set = AddressFormSet(
            request.POST if request.method == "POST" else None,
            queryset=Address.objects.none(),  # only allow adding new addresses
            form_kwargs={"required_fields": False}
        )

        email_form_set = EmailFormSet(
            request.POST if request.method == "POST" else None,
            queryset=Email.objects.none(),  # only allow adding new emails
            form_kwargs={"required_fields": False}
        )

        website_form_set = WebsiteFormSet(
            request.POST if request.method == "POST" else None,
            queryset=Website.objects.none(),  # only allow adding new websites
            form_kwargs={"required_fields": False}
        )

        relation_form_set = RelationFormSet(
            queryset=Relation.objects.none(),  # <-- always empty
            #queryset=organisation.relation_set.all() if organisation else Relation.objects.none(),
            form_kwargs={
                "required_fields": False
            }
        )

    context = {
        "title": _("Create organisation") if organisation is None else _("Update ") + organisation.name,
        "form": organisation_form,
        "phonenumber_form_set": phonenumber_form_set,
        "address_form_set": address_form_set,
        "email_form_set": email_form_set,
        "website_form_set": website_form_set,
        "relation_form_set": relation_form_set,

        "error_sources": [
        {
            "label": _("Organisation"),
            "form": organisation_form,
        },
        {
            "label": _("Phone numbers"),
            "formset": phonenumber_form_set,
        },
        {
            "label": _("Addresses"),
            "formset": address_form_set,
        },
        {
            "label": _("Emails"),
            "formset": email_form_set,
        },
        {
            "label": _("Websites (url's)"),
            "formset": website_form_set,
        },
        {
            "label": _("Relations"),
            "formset": relation_form_set,
        }, ],
    }

    return render(request, "MyNetwork/organisation_form.html", context)

I have omitted all print statements I used to check for the cause. I can add them again with the terminal results to show when I lose my relation.

Can anybody tell me what I can do to find the cause of this? I prefer a long lasting relation :wink:

If I had to try and diagnose this, I’d start by tracing all SQL statements being issued, and adding some print statements at likely spots worth investigating. I’d also consider running this in the debugger to more easily examine variables at those key locations.

(To shorten the size of the logs, I’d probably also get rid of all fields that I could to narrow this down to a smaller size - for example, does this problem still occur if you get rid of the phone numbers, addresses, websites, and emails? That may reduce the number of queries being issued, making it easier to find what you’re looking for without having to wade through a bunch of other stuff.)

The site is still under construction. DEBUG = True is active.

How can I trace the SQL statements?

I stilll have a lot of print statements before and after the save() But in the end… the location wherer it occurs in the code is clear.

Creating a smaller codebase by removing all non essential code like the formsets except the Relation is a good idea.

Already found something before removing the extra code.
I added:

                from django.db import connection
                from django.test.utils import CaptureQueriesContext

                with CaptureQueriesContext(connection) as queries:
                    organisation = organisation_form.save()

                print(f"Executed {len(queries)} queries")

                for i, query in enumerate(queries.captured_queries, start=1):
                    print(f"\nQuery {i}")
                    print(f"Time: {query['time']}s")
                    print(query["sql"])

And now I see:

Query 26
Time: 0.015s
DELETE FROM "MyNetwork_relation" WHERE "MyNetwork_relation"."basemodel_ptr_id" IN (677)

Query 27
Time: 0.000s
DELETE FROM "MyContext_basemodel" WHERE "MyContext_basemodel"."id" IN (677)

The ID of the deleted model instance was 677.

I will remove the extra code and check the other 28 query’s Any tips how to procede?

(back after dinner :slight_smile: )

See the logging configurations at Logging | Django documentation | Django

And by running under the debugger, I mean using pdb to execute your code interactively. (The specific means for doing so depend upon what IDE you are using, For VSCode, see Python debugging in VS Code. I also found this quite helpful - https://www.youtube.com/watch?v=y_dCT9TQtyQ)

The idea here is to set a breakpoint shortly before the area of interest, then single-step your way through the code to see what’s happening. (Warning, the ORM internals can be quite confusing - at least they are to me.) You can also examine objects in detail to see what’s being changed.

(I’ve never used Django Polymorphic - I have no idea how it may be affecting things.)