Post Reponse is too slow

Using Django
1

Hello,
i’m new to django framework, and i’m building a search engine bot based on NLP & machine learning,

everyting i made is working but i have issues with the response is too slow, and i don’t know why

Here is my code

class SemanticViewSet(APIView):
    
    def post(self, request, format=None):
        return Response(ResponseBuilder.build(request.data))


class ResponseBuilder:
    
    @staticmethod
    def build( requestParams ):
        params = {
            "start-date" : requestParams.get("startDate"),
            "end-date" : requestParams.get("endDate"),
            "category" : requestParams.get("category"),
            "keyword" : requestParams.get("keyword"),
            "sort" : requestParams.get("sort"),
            "author" : requestParams.get("author"),
            "text" : requestParams.get("text"),
            "dataType" : requestParams.get("dataType"),  
        }
        
        fetchedData = {}

        requestQuery = {
            'params' : params
        }
                
        if params["dataType"] == "article":

            requestQuery["query"] = {
                "articles" : ConcreteDataListProvider.getArticles()
            }
            fetchedData["articles"] = ArticleRepository.fetch(requestQuery)

        elif params["dataType"] == "journal":

            requestQuery["query"] = {
                "newspapers"  : ConcreteDataListProvider.getNewspapers(),
                "articles"  : ConcreteDataListProvider.getArticles(),
            }
            fetchedData["newspapers"] = NewspaperRepository.fetch(requestQuery)

        elif params["dataType"] == "0":
            
            requestQuery["query"] = {
                "newspapers"  : ConcreteDataListProvider.getNewspapers(),
                "articles"  : ConcreteDataListProvider.getArticles(),
            }
            fetchedData["newspapers"] = NewspaperRepository.fetch(requestQuery)

            fetchedData["articles"] = ArticleRepository.fetch({
                'query' : requestQuery["query"],
                'params' : params
            })
        
        if "text" in params and params["text"] != "":
            fetchedData["semantic"] = SemanticDataExtactor.gatherDataFromModels({
                "query" : requestQuery["query"],
                "termToSearch" : params["text"]
            })

            fetchedData["params"] = params

            fetchedSemanticData = SemanticDataExtactor.arrangeData(fetchedData)

            if  len(fetchedSemanticData) > 0:
                fetchedData = fetchedSemanticData
 
        return DataDecorator.mergeData(fetchedData)
2

There seem to be a number of places in this view where calls are made to external functions. Probably the first step in diagnosing this would be to identify how long each step requires to determine where the delays are.

You could add “time markers” to your log at each step within the view to show how your code is progressing.

Also it might help to clarify what you mean by “too slow” - are you talking about responses in terms of seconds, minutes, or hours?

1 Like
3

it tooks 15 seconds to load data, and the size of data is 5mo

4

Your next step then is to determine where that time is being spent in your view. One of the easiest ways to do it would be to add log statements (or print statements) that print the current time before and after each of the major procedural steps within the view. This will help you identify what the long-running components are.
However, if you’re uploading 5 mb as a post submission, I wouldn’t be too surprised to find out that a chunk of the time is being spent in the upload / decoding of the data. (Many possible factors involved there.)

1 Like
5

no, the size of the response is 5 mo, if there is posisbility to arrange a quick call to explain more.

6

No, I don’t need any further details - you need to determine where the time is being spent in the view.

1 Like
7

i’m will do it , thanks for the idea