Best way to serve multiple external APIs?

Using Django Forms & APIs
1

Hello! I’m quite new to django and I’m facing an issue.

I’m working on a web-based app that uses Django in the backend. We are trying to connect the app to several external APIs to fetch environmental data. They way this is currently done is like this: for each external API a new Django app is created and in each app views.py three django views are coded:

def get_stations(request) -> JSONResponse:
"""Fetch stations as points (lat and long) that can be plotted on the frontend"""

def get_parameters(request) -> JSONResponse:
"""Fetch the parameters available by station (this view is called after clicking on a station)"""

def get_timeseries(request) -> JSON Response:
"""Fetch the events from a specific parameter (called after clicking on a parameter)"""

Each of this views calls different functions that request the data from the associated API and then return a normalized response. For instance, the get_parameters view for the station ATK_01 of the API Example would return a List of parameters that look like this:

[{"ID": "PCP00123", "Name": "Precipitation", "Location_ID": "ATK_01", "API_source": "EXAMPLE", "Units": "mm"}, {"ID": "SNOW0001", "Name": "Snow Depth", "Location_ID": "ATK_01", "API_source": "EXAMPLE", "Units": "mm"}]

And the urls.py of each app looks like this:

from django.urls import path
from .views import get_parameters, get_locations, get_timeseries,

urlpatterns = [
               path("locations/", get_locations),
               path("parameters/", get_parameters),
               path("timeseries/", get_timeseries)
]

Now, this feels redundant. Also, I end up with lots of non-relevant files each time I start a new app (admin.py, models.py, tests.py, etc).
My question is what would be the smartest way to organize the application for a task like this? Also, how could I ensure that my responses are homogenous and well formated, other than typing the dictionary keys manually?
Thank you in advance.

2

Welcome @mclovin !

This really is a sub-optimal thing to do. There is no benefit to creating multiple apps for this.

The better way to handle this would be to make the service a parameter within the url, giving you a set of definitions similar to:

urlpatterns = [
               path("<str:service>/locations/", get_locations),
               path("<str:service>/parameters/", get_parameters),
               path("<str:service>/timeseries/", get_timeseries)
]

… and then having your views handle these parameters appropriately. (You don’t provide enough details about these views for us to be able to offer more specific suggestions.)

If these different services are from different providers such that there’s no commonality among them, you don’t really have a choice.

3

First of all, thank you very very much for answering.
Regarding your second point, a view would look like this:

def get_locations(request):
    locations = fetch_locations()
    locations_list = normalize_locations(locations=locations)
    response = {"response": locations_list}
    return JsonResponse(response, encoder=DjangoJSONEncoder)

locations_list is just a List[Dict[str]] which items look like this

{"ID": id, "Longitude": lon, "Latitude": lat, "Name": name, "Source": source}

So all normalized locations contain those keys. It is a similar procedure for the get_parameters view.

Another thing that I’m struggiling is that the external APIs should contain certain properties such as

hasForecasts: bool
hasDates: bool

This later because we also want to plot them and the behaviour changes based on certain attributes. I’m thinking about an external .py file containing the API metadata (see below). Do you think it makes sense?

@dataclass
class Api:
    name: str 
    hasForecasts: bool 
    hasDates: bool

# Here I define my API properties
API_X = Api("X", True, True)
4

To be honest here I’m not following what you’re trying to accomplish with that.

But then I tend to look at such things from a “data” perspective. If you’re looking to create a mapping from the JSON-that-you-get-from-an-external-API to the JSON-that-you-need-to-send-to-the-browser, there are a couple different cases to handle:

  • External API returns the data with the same key and data type as what you need to send: It can be copied through directly.
  • External API returns the data with the same key, but the data is in a different format: Reformat the data and return it
  • External API returns the data with a different key: Assign the value to the new key, with any reformatting of the data as necessary.
  • External API does not return that data: ???

Assuming each of the external APIs are consistent, it’s these differences that I would encapsulate within the transformation functions called by the views.