DRF Serializer issue

Using Django
1

Hi!

I have two querysets which query the same model.

queryset1 = Toy.objects.filter(...)
queryset2 = Toy.objects.exclude(...)

And I have ToySerializer(ModelSerializer) for Toy model.

And I want to create Response with these two queryset results.
So I just did

return Response(ToySerializer(queryset1).data + ToySerializer(queryset2).data)

It combines data from two querysets but the problem is count is not calculated correctly.

{
  "count": 38,
  "next": "http://0.0.0.0:8001/api/toy/?favorites=true&limit=25&offset=25",
  "previous": null,
  "results": [
    {
      "id": 15,
      ...
      ...
    }
   ]

Total count is 40 but that 38 is count of queryset2.
I want to get correct count of sum of two querysets.

I think it can be resolved if I can concatenate data of two querysets on serializer level.
Or is there any other method to achieve this?

Any help would be appreciated

@KenWhitesell, would you please help me on this?

2

First thought popping into my head would be to create a single queryset containing both sets.
e.g. queryset = (Toy.objects.filter(...) | Toy.objects.exclude(...)).distinct() My current understanding is that this would create a union of the two querysets as opposed to concatenating them, which I believe should treat this as a single query.

Note: I’m not in a position at the moment to actually try this - I could very easily be 100% wrong. I’d verify this by looking at the generated queries.

3

@KenWhitesell , thank you for your kind response.
That’s what I first tried but the problem is ordering is broken with that approach.
I have those two querysets because I want to put the values from queryset1 at the top of the list.
Those are favorite toys.
Besides, that, it has many other filters.
That’s why I am trying to concatenate data on serializer level.
The data looks good but only that count matters.

4

Original queryset includes many filters and ordered.
queryset1 is all favorite toys filtered from that original queryset.
queryset2 = original queryset - queryset1.

My goal is to put favorite toys at the top of API response and to make sure this does not break any other filters.

5

Two ideas:

  1. Post-process the queryset to correct the count before returning it.
  2. Annotate the favorites queryset with something like a 1, everything else with a 2, then sort using that arbitrary field.
    The number of filters shouldn’t really matter - a queryset is a queryset and can be union-ed with any other compatible queryset.
6

Also see the union, intersection and difference methods on querysets.
That may provide a clearer / cleaner implementation for what you’re looking to achieve.

7

There are many filters which actually order the items by many different values.
So keeping the order is critical.
The endpoint has a lot of filters, more than 10, ordering by distance based on given lat and lon, and so on.
So I don’t think #2 can work.
And if I sort the queryset by any other field after they are joined, the items of queryset will not be at top of the list which breaks the initial goal.

Could you please explain me more about #1?

8

I don’t understand how the number of filters is going to make any difference at all. A queryset is a queryset, regardless of what you do to it, provided the functions you perform on it continue to return a queryset. There’s nothing you’ve posted that changes the nature of my suggestion, perhaps only the specific of the implementation.

Regarding your question, you’re directly returning the Response object created from concatenating the data field from two querysets. Break that down into three steps.

  • Assign that concatenation to a variable.
  • Count the number of entries and replace the Count element
  • Return the Response object created from your modified variable.
1 Like
9

@KenWhitesell
I think I’ve been stupid the whole day.
Actually, the response object is just an OrderedDict and I know how many items are from queryset1.
So I just need to add that count of queryset1 to the count.
That’s all. Ha!~ :confused:

Thank you for your help.
Your last hint overhauled my brain. :smiley:

10

But another problem is pagination in that case. :thinking:
It would be great if there’s a way to join two queryset without breaking order of two querysets.
So let’s say queryset1.union(queryset2)
I want objects from queryset1 go to top of the list and then objects from queryset2 with their preserved orders.
That’s my problem