Querying Object

Using Django
1

Hello, I am experiencing a slow response whenever I request from the server. Is my code below the right approach to get all my models in a single request? For some instance, when the data is already large, requesting from the server take some quite time.

def production_report_filter(request):
    employees = Employee.objects.all()
    employee_serializer = EmployeeSerializer(employees, many=True)

    country = Country.objects.all()
    country_serializer = CountrySerializer(country, many=True)

    sites = Site.objects.all()
    site_serializer = SiteSerializer(sites, many=True)

    departments = Department.objects.all()
    department_serializer = DepartmentSerializer(departments, many=True)

    sections = Section.objects.all()
    section_serializer = SectionSerializer(sections, many=True)

    positions = Position.objects.all()
    position_serializer = PositionSerializer(positions, many=True)

    clients = Client.objects.all()
    client_serializer = ClientSerializer(clients, many=True)

    projects = Project.objects.all()
    project_serializer = ProjectSerializer(projects, many=True)

    return Response({
        'employees': employee_serializer.data,
        'country': country_serializer.data,
        'sites': site_serializer.data,
        'departments': department_serializer.data,
        'sections': section_serializer.data,
        'positions': position_serializer.data,
        'clients': client_serializer.data,
        'projects': project_serializer.data,
    }, status=status.HTTP_200_OK)
2

Hi Clark,

There are two concerns here.

The first would be if any of your serializers suffer from a N+1 issue. If you’re unfamiliar, this would mean that the serializer would take an instance it’s passed for example, the ClientSerializer takes a Client instance. And let’s say Client had a ForeignKey to Foo which has the property spam. If you expose a spam field on ClientSerializer with a SerializerMethodField using return instance.foo.spam, the Django ORM will have to go fetch that Foo instance. Now since you’re passing multiple clients into the serializer, you would have N queries where N is the number of Client instances plus 1 (to fetch all the clients). The way you avoid this is through the use of select_related and prefetch_related.

The second is a little more straightforward, but more complicated to solve. If you have a lot of data, it’s going to take longer to fetch it from the database and then serialize it into JSON. As you create more data, this view will continue to perform worse. The first solution that comes to mind is to split up the API request into multiple. Those that are known to be fixed and small dataset permanently, such as Country (if I’m making valid assumptions). could be aggregated into a view similar to what you have here. Then the other models that are likely to grow, such as Clients, Employees and Projects would have their own views and potentially even pagination built into them.

Another way to consider things that may help is to consider how your application would work if you have 10x the number of instances as you do now. For example, a select element with of 20 projects in a dropdown is manageable. But a select with 200 options is a bit unwieldy to manage as an end user. A potentially better user experience with 200 options is an autocomplete UI component that uses Ajax to search for any matching instances and limiting the results to some fixed result to force the user to keep searching.

Hopefully that helps answer some of your questions. If you’re simply looking to understand why this view is running slowly, you can use the Django Debug Toolbar which has instrumentation to help understand what your application is doing.

-Tim

1 Like
3

Thank you for your very informative answer, it really help me a lot!