How to forward all incoming request from Django API to another API?

Hi, I am working with an open source labeling tool written in Django (GitHub - HumanSignal/label-studio: Label Studio is a multi-type data labeling and annotation tool with standardized output format)
I want to forward all request from client to this Django API to my own API to process some logic, then return it back to Django API.
I am able to send request from my API to Django, but don’t know how to send from Django to my API.
I have tried using middleware, this is my code:

class ForwardToAPIMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        self.fastapi_base_url = "http://localhost:8000"

    def __call__(self, request):
        headers = dict(request.headers)
        print(headers)
        if "Allowed" in headers.keys():
            pass
        else:
            # Modify the request URL
            fastapi_url = self.fastapi_base_url + request.path

            try:
                # Forward the request to FastAPI
                response = requests.post(fastapi_url, data=request.body.decode('utf-8'))
                return HttpResponse(response.content, status=response.status_code)
            except Exception as e:
                # Return an error response if there's an issue forwarding the request
                return HttpResponse(str(e), status=500)

but it return error:

Traceback (most recent call last):
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/middleware.py", line 175, in __call__
    return f(*args, **kwargs)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/utils/deprecation.py", line 119, in __call__
    response = self.process_response(request, response)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/middleware/common.py", line 107, in process_response
    if response.status_code == 404 and self.should_redirect_with_slash(request):
AttributeError: 'NoneType' object has no attribute 'status_code'

I have searched gg, asked chatgpt for days, but can’t find anything helpful

You have a logic path in your middleware that allows “None” to be returned. (Specifically the case "Allowed" in header.keys())

Your middleware is directly returning an HttpResponse and not passing the request through the rest of the middleware / view stack. (Notice how in the examples, the middleware returns the value returned by self.get_response(request), it does not directly return an HttpResponse)

Finally, where does your middleware reside in the middleware chain? Is it possible that it needs to be somewhere else in the list?

Side note: If you want all requests from the client to be processed by your API, I’d suggest you make your API the target and allow your code to call the Django API. Architecturally, I don’t see any benefit by adding an extra round-trip for the request.

1 Like

Side note: If you want all requests from the client to be processed by your API, I’d suggest you make your API the target and allow your code to call the Django API. Architecturally, I don’t see any benefit by adding an extra round-trip for the request.

Yes, I also find this design does not make any sense, but since I am not allowed to do some stuffs in Django, I have to write another API and redirect request to it.
Also, I can’t make clients to target my API instead of Django, so forwarding requests is my only choice (or I think so)

You have a logic path in your middleware that allows “None” to be returned. (Specifically the case "Allowed" in header.keys())

The “Allowed” field in headers is added after request returned from my other API, to mark that request is the returned one, not coming from clients, so the request will be process in next step of Django API, not forward again to my API. If I don’t check for that field, 2 APIs will send requests back and forth for infinity. But yea, I think I am missing the code to send it down to next step, and I don’t know how to do it either, can you pls guide me?

Your middleware is directly returning an HttpResponse and not passing the request through the rest of the middleware / view stack. (Notice how in the examples, the middleware returns the value returned by self.get_response(request), it does not directly return an HttpResponse)

Yea I don’t know how to process next step, what should I do after receive the request back from other API?

Finally, where does your middleware reside in the middleware chain? Is it possible that it needs to be somewhere else in the list?

Currently I put it at the bottom of middleware list

Then you want to pass the request on to self.get_response(request). But do you really need to do this? Does the API you’re calling turn around and call you back? Or does it just return a response?

You don’t get the request back from the API. The API returns a response. After you do whatever you need to do with that response, then you can call self.get_response(request) to pass the original request you have received on to the next step.

So I’m a little confused by the flow of events here.

Is what you’re trying do to this: (option 1)

Browser Django API
Sends request
Forwards request to API
Processes request
Returns response
Processes request
Returns response

Or is it this (option 2):

Browser Django API
Sends request 1
Forwards request 1
Processes request 1
Makes request 2 back to Django
Processes request 2 from API
Returns response 2 to API
Returns response to Django
Returns response 1 to browser

This latter case is what you appear to be trying to handle with your handling of the Allowed header, when I think you’re really trying to do the first.

Yes, my other API will call back to Django, this is the code I used to do so (I am not 100% sure if it correct, but it worked):

@app.middleware("http")
async def proxy_to_django(request: Request, call_next):
    headers = dict(request.headers)

    if permission_check(headers):
        # add new field in headers to distinguish the return request
        headers["allowed"] = "True"

        # Get the path and query parameters from the request
        path = request.url.path
        query_params = request.url.query

        # Construct the URL to the Django endpoint
        django_url = f"{django_base_url}{path}"
        print(django_url)

        # async with httpx.AsyncClient() as client:
        #     # Forward the request to the Django application
        #     response = await client.request(
        #         request.method, django_url, params=query_params, headers=request.headers, data=await request.body()
        #     )

        response = requests.request(method=request.method, url=django_url, params=query_params, headers=headers, data=await request.body())

as you can see, I have tried both async and sync method

I don’t understand why you would be doing that. Your view (it doesn’t need to be middleware) can process the request it receives from Django and then return a response (perhaps JSON). Your Django middleware can then receive that response and do whatever it would have done with the second request, with the data received in the response.

You’re adding a whole lot of overhead this way.

1 Like

Yea, I just realized after read your comment, now I edited my Django middleware to this:

    def __call__(self, request):
        # request to my other API to check
        response = requests.request("GET", self.fastapi_base_url + "/fastapi-endpoint")
       
        if response.status_code != 200:
            # if failed, drop the request
            return HttpResponse("Denied", status=401)
        else:
            # if success, continue with the original request from client
            self.get_response(request)

and instead of using middleware in my other API, I just use a normal endpoint like so:

@app.get("/fastapi-endpoint")
async def fastapi_endpoint():
    return {"message": "Processed by FastAPI"}

its a lot simpler but it still have this error:

Traceback (most recent call last):
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/sentry_sdk/integrations/django/middleware.py", line 175, in __call__
    return f(*args, **kwargs)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/utils/deprecation.py", line 119, in __call__
    response = self.process_response(request, response)
  File "/home/ab/Desktop/git-projects/label-studio/venv/lib/python3.8/site-packages/django/middleware/common.py", line 107, in process_response
    if response.status_code == 404 and self.should_redirect_with_slash(request):
AttributeError: 'NoneType' object has no attribute 'status_code'

where did I do it wrong?

You’re still returning None from your middleware. You need to return the value you get from self.get_response(...)

1 Like

oh, right. I thought self.get_response(request) automatically return the value, turn out I have to do it myself. Now it works okay, thank you a lot.