many to many relationship only getting saved when using django rest's api view

Using Django Mystery Errors
1

I am building a chat application that has rooms in it.

#The room model in models.py
class Room(models.Model):
    name = models.CharField(max_length=150)
    members = models.ManyToManyField(
        get_user_model(), blank=True, editable=False, related_name="member_of"
    )
    limit = models.PositiveSmallIntegerField("Member limit")
    creator = models.ForeignKey(
        get_user_model(), on_delete=models.CASCADE, related_name="creator_of"
    )
    bans = models.ManyToManyField(
        get_user_model(), blank=True, related_name="banned_from"
    )
    invites = models.ManyToManyField(
        get_user_model(), blank=True, related_name="invited_to"
    )
    admins = models.ManyToManyField(
        get_user_model(), blank=True, related_name="admin_of"
    )
    code = models.CharField(max_length=36, blank=True, null=True, unique=True, editable=False)
    expire_date = models.DateTimeField(null=True,blank=True, editable=False)
    welcome_text = models.TextField("Welcome message", blank=True)
    has_password = models.BooleanField(default=False)
    password = models.CharField(max_length=32, blank=True)
    is_private = models.BooleanField(default=False)
    can_invite = models.BooleanField(default=True)
    can_admins_invite = models.BooleanField(default=True)
    can_upload = models.BooleanField(default=True)
    can_admins_upload = models.BooleanField(default=True)

    @property
    def member_count(self):
        return self.members.count()

    class Meta:
        constraints = [
            models.UniqueConstraint(fields=['name','creator'],name="name and creator unique")
        ]

    def __str__(self):
        return self.name

    def clean(self):
        if self.limit < 1 or self.limit > 100 or self.limit is None:
            raise ValidationError("Limit must be between 1 and 100.")
        if (self.has_password == False and self.password != "") or (
            self.has_password == True and self.password == ""
        ):
            raise ValidationError("Incorrect value for the room's password.")

The main focus here are the creator and admins fields.
I want to add the creator to the list of admins after the room is created.

#signals.py
@receiver(models.signals.post_save,sender=Room, dispatch_uid="generate code and add creator",weak=False)
def roomPostSave(sender,instance:Room,**kwargs):
    print("hello")
    if instance.code is None:
        instance.admins.add(instance.creator)
        instance.code=uuid.uuid4()
        instance.expire_date=now()+datetime.timedelta(days=3)
        instance.save()

#apps.py
class ClicApiConfig(AppConfig):
    default_auto_field = 'django.db.models.BigAutoField'
    name = 'clic_api'
    def ready(self):
        from . import signals

The code field is a uniquely generated field which has it’s own purposes for my own app that I will not get to right now. Since we want this operation to be done only after the object’s creation and not after it’s update we first check if the code field is None or not and if it is (meaning that the object is getting created for the first time) we launch the code below (the instance.save() will cause the receiver to run again but it will do nothing because of the condition)

Now here is the problem, when I create a new object using django rest’s api_view decorator with a post method and with my view, everything works fine.

@api_view(["GET","POST"])
@permission_classes([IsAuthenticated])
def rooms(request):
    if request.method=="GET":
        paginator=PageNumberPagination()
        paginator.page_size=10
        serializer=RoomSerializerR(paginator.paginate_queryset(Room.objects.filter(is_private=False),request),many=True)
        return paginator.get_paginated_response(serializer.data)
    elif request.method=="POST":
        serializer=RoomSerializer(data=request.data,context={"request":request})
        if serializer.is_valid():
            print(request.user)
            serializer.save(creator=request.user)
            return Response(serializer.data)
        else:
            return Response(serializer.errors,status=status.HTTP_400_BAD_REQUEST)

#serializers.py
class RoomSerializer(ModelSerializer):
    creator=serializers.HiddenField(default=serializers.CurrentUserDefault())
    member_count=serializers.IntegerField(read_only=True)
    def validate(self,data):
        if self.context['request'].method=="PATCH":
            if "limit" in data:
                if data['limit'] < 1 or data['limit'] > 100:
                    raise serializers.ValidationError("Limit must be between 1 and 100.")
            if "has_password" in data:
                if data["has_password"]==False:
                    data["password"]=""
                if data['has_password']==True:
                    if "password" in data:
                        if data["password"]=="":
                            raise serializers.ValidationError("Incorrect value for the room's password.")
                    else:
                        raise serializers.ValidationError("Password not provided.")
        else:
            if data['limit'] < 1 or data['limit'] > 100:
                raise serializers.ValidationError("Limit must be between 1 and 100.")
            if "has_password" in data:
                if data["has_password"]==False:
                    data["password"]=""
                if data['has_password']==True:
                    if "password" in data:
                        if data["password"]=="":
                            raise serializers.ValidationError("Incorrect value for the room's password.")
                    else:
                        raise serializers.ValidationError("Password not provided.")
        return data

    class Meta:
        model=Room
        fields='__all__'
        validators=[UniqueTogetherValidator(queryset=Room.objects.all(),fields=['name','creator'])]

But when I give a post request using python’s requests module, the receiver gets executed and does add the user to the admin’s field which can be seen by printing the admins.all() of the instance before and after adding to it in the receiver, but nothing gets saved in the database (other fields including code and expire date have no problem)
Logging the sql statements shows that some unwanted behavior happens after the object is updated via the signal, which doesn’t happen using the api view.

logging highlights

api view post
(0.000) SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."expire_date" > '2023-08-05 21:44:45.776922' AND "django_session"."session_key" = 'jnxltx78ip1koa5wk1511yqcx38ej80p') LIMIT 21; args=('2023-08-05 21:44:45.776922', 'jnxltx78ip1koa5wk1511yqcx38ej80p'); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" WHERE "clic_api_user"."id" = 5 LIMIT 21; args=(5,); alias=default
(0.016) SELECT 1 AS "a" FROM "clic_api_room" WHERE ("clic_api_room"."creator_id" = 5 AND "clic_api_room"."name" = '1') LIMIT 1; args=(1, 5, '1'); alias=default
user1
(0.078) INSERT INTO "clic_api_room" ("name", "limit", "creator_id", "code", "expire_date", "welcome_text", "has_password", "password", "is_private", "can_invite", "can_admins_invite", "can_upload", "can_admins_upload") VALUES ('1', 2, 5, NULL, NULL, '', 0, '', 0, 1, 1, 1, 1) RETURNING "clic_api_room"."id"; args=('1', 2, 
5, None, None, '', False, '', False, True, True, True, True); alias=default
hello
(0.000) BEGIN; args=None; alias=default
(0.000) INSERT OR IGNORE INTO "clic_api_room_admins" ("room_id", "user_id") VALUES (115, 5); args=(115, 5); alias=default
(0.172) COMMIT; args=None; alias=default
(0.140) UPDATE "clic_api_room" SET "name" = '1', "limit" = 2, "creator_id" = 5, "code" = 'cf6f74f4-8fd6-4873-ac8b-f40179524a6c', "expire_date" = '2023-08-08 21:44:46.249118', "welcome_text" = '', "has_password" = 0, "password" = '', "is_private" = 0, "can_invite" = 1, "can_admins_invite" = 1, "can_upload" = 1, "can_admins_upload" = 1 WHERE "clic_api_room"."id" = 115; args=('1', 2, 5, 'cf6f74f4-8fd6-4873-ac8b-f40179524a6c', '2023-08-08 21:44:46.249118', '', False, '', False, True, True, True, True, 115); alias=default
hello
(0.000) SELECT COUNT(*) AS "__count" FROM "clic_api_user" INNER JOIN "clic_api_room_members" ON ("clic_api_user"."id" = "clic_api_room_members"."user_id") WHERE 
"clic_api_room_members"."room_id" = 115; args=(115,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_members" ON ("clic_api_user"."id" = "clic_api_room_members"."user_id") WHERE "clic_api_room_members"."room_id" = 115; args=(115,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_bans" ON ("clic_api_user"."id" = "clic_api_room_bans"."user_id") WHERE "clic_api_room_bans"."room_id" = 115; args=(115,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_invites" ON ("clic_api_user"."id" = "clic_api_room_invites"."user_id") WHERE "clic_api_room_invites"."room_id" = 115; args=(115,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_admins" ON ("clic_api_user"."id" = "clic_api_room_admins"."user_id") WHERE "clic_api_room_admins"."room_id" = 
115; args=(115,); alias=default


requests module post
(0.015) SELECT "authtoken_token"."key", "authtoken_token"."user_id", "authtoken_token"."created", "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", 
"clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "authtoken_token" INNER JOIN "clic_api_user" ON ("authtoken_token"."user_id" = "clic_api_user"."id") WHERE "authtoken_token"."key" = '38feae5e26ccb6bbb2d2f78dd269c036cbfbc052' LIMIT 21; args=('38feae5e26ccb6bbb2d2f78dd269c036cbfbc052',); alias=default
(0.000) SELECT 1 AS "a" FROM "clic_api_room" WHERE ("clic_api_room"."creator_id" = 5 AND "clic_api_room"."name" = '2') LIMIT 1; args=(1, 5, '2'); alias=default  
user1
(0.000) INSERT INTO "clic_api_room" ("name", "limit", "creator_id", "code", "expire_date", "welcome_text", "has_password", "password", "is_private", "can_invite", "can_admins_invite", "can_upload", "can_admins_upload") VALUES ('2', 2, 5, NULL, NULL, '', 0, '', 0, 0, 0, 0, 0) RETURNING "clic_api_room"."id"; args=('2', 2, 
5, None, None, '', False, '', False, False, False, False, False); alias=default
hello
(0.000) BEGIN; args=None; alias=default
(0.000) INSERT OR IGNORE INTO "clic_api_room_admins" ("room_id", "user_id") VALUES (116, 5); args=(116, 5); alias=default
(0.156) COMMIT; args=None; alias=default
(0.156) UPDATE "clic_api_room" SET "name" = '2', "limit" = 2, "creator_id" = 5, "code" = '2733a0ca-9e13-4e11-8e10-0f790cd29c4b', "expire_date" = '2023-08-08 21:49:51.006153', "welcome_text" = '', "has_password" = 0, "password" = '', "is_private" = 0, "can_invite" = 0, "can_admins_invite" = 0, "can_upload" = 0, "can_admins_upload" = 0 WHERE "clic_api_room"."id" = 116; args=('2', 2, 5, '2733a0ca-9e13-4e11-8e10-0f790cd29c4b', '2023-08-08 21:49:51.006153', '', False, '', False, False, False, False, False, 116); alias=default
hello
(0.000) BEGIN; args=None; alias=default
(0.000) SELECT "clic_api_user"."id" FROM "clic_api_user" INNER JOIN "clic_api_room_bans" ON ("clic_api_user"."id" = "clic_api_room_bans"."user_id") WHERE "clic_api_room_bans"."room_id" = 116; args=(116,); alias=default
(0.000) COMMIT; args=None; alias=default
(0.000) BEGIN; args=None; alias=default
(0.000) SELECT "clic_api_user"."id" FROM "clic_api_user" INNER JOIN "clic_api_room_invites" ON ("clic_api_user"."id" = "clic_api_room_invites"."user_id") WHERE "clic_api_room_invites"."room_id" = 116; args=(116,); alias=default
(0.000) COMMIT; args=None; alias=default
(0.000) BEGIN; args=None; alias=default
(0.000) SELECT "clic_api_user"."id" FROM "clic_api_user" INNER JOIN "clic_api_room_admins" ON ("clic_api_user"."id" = "clic_api_room_admins"."user_id") WHERE "clic_api_room_admins"."room_id" = 116; args=(116,); alias=default
WHAT IS THIS
(0.000) DELETE FROM "clic_api_room_admins" WHERE ("clic_api_room_admins"."room_id" = 116 AND "clic_api_room_admins"."user_id" IN (5)); args=(116, 5); alias=default
(0.109) COMMIT; args=None; alias=default
(0.000) SELECT COUNT(*) AS "__count" FROM "clic_api_user" INNER JOIN "clic_api_room_members" ON ("clic_api_user"."id" = "clic_api_room_members"."user_id") WHERE 
"clic_api_room_members"."room_id" = 116; args=(116,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_members" ON ("clic_api_user"."id" = "clic_api_room_members"."user_id") WHERE "clic_api_room_members"."room_id" = 116; args=(116,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_bans" ON ("clic_api_user"."id" = "clic_api_room_bans"."user_id") WHERE "clic_api_room_bans"."room_id" = 116; args=(116,); alias=default
(0.000) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_invites" ON ("clic_api_user"."id" = "clic_api_room_invites"."user_id") WHERE "clic_api_room_invites"."room_id" = 116; args=(116,); alias=default
(0.015) SELECT "clic_api_user"."id", "clic_api_user"."password", "clic_api_user"."last_login", "clic_api_user"."is_superuser", "clic_api_user"."username", "clic_api_user"."first_name", "clic_api_user"."last_name", "clic_api_user"."email", "clic_api_user"."is_staff", "clic_api_user"."is_active", "clic_api_user"."date_joined" FROM "clic_api_user" INNER JOIN "clic_api_room_admins" ON ("clic_api_user"."id" = "clic_api_room_admins"."user_id") WHERE "clic_api_room_admins"."room_id" = 
116; args=(116,); alias=default

I have no such code that removes the creator from the user in my project, and even if I did there would be no way for it to get executed by me, the only thing diffrent here is that one request is made from the api view using session based auth and the other from a script file using the requests module with token based auth.
And yes there is no problem with the token the user does get authed correctly and the creator field will save correctly, look at the log.

here is the client code

                res = requests.post(BASE_URL+"rooms/",{
                    "name":name,
                    "limit":limit,
                    "welcome_text":welcome_text,
                    "has_password":has_password,
                    "password":password,
                    "is_private":is_private,
                    "can_invite":can_invite,
                    "can_admins_invite":can_admins_invite,
                    "can_upload":can_upload,
                    "can_admins_upload":can_admins_upload
                },headers=HEADERS)

#HEADERS is just the header for token auth HEADERS={"Authorization": f"Token {TOKEN}"}

Edit: creating rooms using objects.create in the shell works fine as well

2

The behaviour of DRF when saving the model is:

  • save the room to database (this triggers the signal adding the admin)
  • if some “many” relation fields were present in the received data, set the many2many relations

Here is what I think is happening:

  • in the first case, data are received as Json, and the admins, members, bans and invites fields are not present in Json, so the set of many2many relations does not happen

  • in the requests case, you use form-data input: in this case, all fields are considered as present in received data (with an empty value for admins, members, bans and invites) because it is not possible to distinguish between an unset field (key not present or set to null in json representation) and a field set with an empty value (empty string or empty list). So here, DRF considers the admins field is received with an empty value and so it sets it with an empty value, hence the delete because you updated it in the post_save receiver, but at the same time, you asked (from the DRF point of view) to set it with an empty value.

In your requests use case, using json={…} instead of data={…} (data is the second argument of requests.post) should solve the actual problem.

However, if you intend to receive form-data inputs, you should handle the default admins creation in a different manner: e.g. by overriding the create method of the serializer, or by always ensuring request’s user is present in the admins field of validated_data (by overriding the validate method). Using signals should be done as a last resort, e.g. when you need to connect to another app you don’t have control on, which is not the case here

1 Like
3

Thank you so much, you have no idea how many hours I spent banging my head over this XD
I had a similar problem with the built in admin panel which I solved by making the admin panel ignore those fields, should’ve though it was the same for this one but just didn’t know it was using form data, now I’ve set it to requests.post(…,json={data}) as you have said and everything is fine.

I will think about other ways of doing this instead of signals.