I currently have a large number of objects, each containing multiple foreign keys. When displaying the list, I use pagination, but I wonder if prefetching is applied when pagination is in effect.
I’m considering fetching only the primary keys using values_list('pk') during pagination and then retrieving only the objects that need to be displayed on the current page.
Can someone tell me if this approach is unnecessary?
It is, yes. The original queryset will be paginated, and only the relevant related instances will be pre-fetched.
Fetching only the primary keys, then the entire object will actually hurt performance, since the DB will still need to find all the rows and do the filtering. Letting the paginator do its job is definitely the right way to go, and much less surprising for anyone else looking at the code.
As with everything performance related, your best bet is to always collect numbers for yourself. However, I doubt the performance gains from writing something yourself will be worth the additional effort.
Are you saying that when paginating, I should pass the PK list and avoid re-requesting only the objects corresponding to the current page?
No, not at all.
Use the paginator exactly how the paginator tells you to use it - it’s aware of how to prefetch a paginated QuerySet.
I think you’re falling into the common pit of premature optimisation.
Sorry, but English is not my native language. I don’t know exactly what you mean.
The answer to my question should be yes or no, so what is your answer?
Use the pagination how the docs tell you. Don’t fetch the list of PKs separately.