Insert or update object on get request with data from external API

I have a stock broker model:

# model.py
from django.db import models
from market.models import Market


class Broker(models.Model):
    choices_mode = [("live", "live"), ("paper", "paper")]

    name = models.CharField(max_length=120, blank=False, null=False)
    mode = models.CharField(max_length=5, choices=choices_mode, default="paper", blank=False, null=False)
    market_data_api_key = models.CharField(max_length=4096, blank=True, null=True)
    trading_api_key = models.CharField(max_length=4096, blank=True, null=True)
    markets = models.ManyToManyField(Market, blank=True, related_name="broker")

    class Meta:
        ordering = ["name"]

    def __str__(self):
        return self.name

Now I want to create Account and retrieve my account details of this broker. What I have so far:

# model.py
from django.db import models

from broker.models import Broker


class Account(models.Model):
    broker = models.ForeignKey(Broker, on_delete=models.CASCADE)
    datetime = models.DateTimeField(auto_now=True)

    # data from API
    firstname = models.CharField(max_length=120, blank=False, null=False)
    lastname = models.CharField(max_length=120, blank=False, null=False)
    # live, paper
    mode = models.CharField(max_length=5, blank=False, null=False)
    balance = models.BigIntegerField(blank=False, null=False)
    cash_to_invest = models.BigIntegerField(blank=False, null=False)
    cash_to_withdraw = models.BigIntegerField(blank=False, null=False)
    amount_bought_intraday = models.BigIntegerField(blank=False, null=False)
    amount_sold_intraday = models.BigIntegerField(blank=False, null=False)
    amount_open_orders = models.BigIntegerField(blank=False, null=False)
    amount_open_withdrawals = models.BigIntegerField(blank=False, null=False)
    amount_estimate_taxes = models.BigIntegerField(blank=False, null=False)

    def __str__(self):
        return self.lastname
# view.py
from datetime import datetime
from django.contrib.auth.mixins import LoginRequiredMixin
from django.urls import reverse_lazy
from django.views.generic import ListView

from .models import Account


class AccountListView(LoginRequiredMixin, ListView):
    model = Account
    template_name = "account/list.html"

    login_url = reverse_lazy("base:login")

    class Meta:
        ordering = ["broker"]

Goal:

Open the browser http://127.0.0.1:8000/account/list/ and get my account data from the Lemon.Markets API. My API key is saved under the broker model.

I Think (but I don’t know! :slight_smile: ) that I need to add something like this under the class AccountListView:

    def get(self, request):
        # if datetime.now() - self.datetime >= 60 seconds:
            # use requests to obtain my account data from the API
            # save my data into the database, maybe use post()
            # return the data, so the view gets rendered

Edit:
I want my account (or accounts) to get inserted into the database automatically when I click on the account view on my website. When opening this site after 60+ seconds the data shall be updated.

Do you have some hints or solutions for me? :slight_smile:

<opinion> Instead of overriding get, I’d probably override get_queryset instead. </opinion>

I’m not clear I understand your data models, but that probably isn’t critical at this point.

Beyond that, your basic idea is correct. But there’s no need to tie in any other methods. The requests library runs synchronously - which is going to cause a problem if the api returning the data is going to take more than the timeout period for your connection to complete.

1 Like

I haven’t heard about get_queryset yet, but I will take a look at it! Edit: I found this good explanation: python - When to use get, get_queryset, get_context_data in Django? - Stack Overflow

Regarding my models: As a Django learner I thought it would be a good idea to store my broker account data in the database (as a cache), so I don’t need to query the API every x seconds (and only 60 queries per minute are allowed!). At first my account data is empty in the database. When I open /account/list/ the first time a request will be send to the API and the data (and a datetime information) will be stored in the database and shown on my website. If I open /account/list/ once again the datetime information will be compared to datetime.now() to check if the information is older than 60 seconds - if yes then the API gets queried once again.

About the synchronous part: I already thought about using async, but for the moment I want to get it running as that’s already hard enough… :smiley: So yeah, I will try async afterwards as it also sounds like a good idea to me! :slight_smile:

1 Like

So it sounds like to me that you’re only expecting there to be one instance of this model? (That’s where my question lies.) If so, then a ListView probably isn’t the right view for this, since there’s only one row.

Or are you getting back results for multiple Account? If so, then that raises the question of whether you need to call your API for each account, or if you’re making one request to get data for all of them at the same time.

This leads me to the comment about async - I actually wasn’t going in that direction. If you’re only making one request to get all the data, async isn’t going to do anything for you. If you have to make multiple requests, async might help - but only if the target environment doesn’t object to multiple concurrent requests.

Regarding the CBVs, in addition to the docs, I recommend people become familiar with the Classy Class-Based Views site along with the CBV diagrams page.
(I never consider SO to be a valid source of information without external confirmation. I always caution people to take information obtained from there with a pound of salt.)

1 Like

There will be multiple accounts with multiple brokers and APIs. I will use async after getting the logic done.

I still don’t know how to approach here. I took a look into CCBV and I saw, that the ListView has no post function. Generally speaking: I want to create, update and list my accounts, but I only need a ListView and therefore only created a list route.

Any idea how to proceed?

How are you going to update from a ListView? (What is the UI/UX that you’re trying to provide?)

I simply go to example.org/account/list/ and let the system do the magic :smiley: As there’s no manual way to change the account information I did not see any other way than just use a ListView. But I’m open to any other real-life example! Later (far later…) I will also need to update one specific account from within a strategy function.

Ignore the “Django-parts” of this for the moment, focus only on what the user will see and do.

What do you want the interface to look like? How is that going to behave from the user’s perspective?

(There is no “magic” here - this is all work that needs to be done.)

The user will see a table with all accounts and their information. It’s used as an overview (e.g. funds). The user will do nothing but viewing the information. These information are used in the backend for trading strategies. Therefore I also need to get the account information updated before sending a buy/sell request via API to the broker.

From a user perspective: list will show the current (updated) accounts.

Now I’m confused.

Saying:

[emphasis added]

does not agree with:

So are you talking about multiple pages here? If so, how many and what is the function of each page?

Mock-ups or images of what you’re trying to create would be helpful.

After scratching my head even harder I finally understand how to use classes and Django. The first brick was get_queryset, as @KenWhitesell already wrote. Here’s my *synchronous` solution:

broker.model.py:

from django.db import models
from market.models import Market


class Broker(models.Model):
    choices_mode = [("live", "live"), ("paper", "paper")]
    choices_state = [("enabled", "enabled"), ("disabled", "disabled")]

    name = models.CharField(max_length=120, blank=False, null=False)
    mode = models.CharField(max_length=5, choices=choices_mode, default="paper", blank=False, null=False)
    state = models.CharField(max_length=8, choices=choices_state, default="disabled", blank=False, null=False)
    market_data_api_key = models.CharField(max_length=4096, blank=True, null=True)
    trading_api_key = models.CharField(max_length=4096, blank=True, null=True)

    markets = models.ManyToManyField(Market, blank=True, related_name="broker")

    class Meta:
        ordering = ["name"]

    def __str__(self):
        return self.name

account.model.py:

from django.db import models

from broker.models import Broker


class Account(models.Model):
    broker = models.OneToOneField(Broker, related_name="account", on_delete=models.CASCADE)
    datetime = models.DateTimeField(auto_now=True)

    # data from API
    firstname = models.CharField(max_length=120, blank=False, null=False)
    lastname = models.CharField(max_length=120, blank=False, null=False)
    # live, paper
    mode = models.CharField(max_length=5, blank=False, null=False)
    # TODO: Make all field types decimal: https://docs.djangoproject.com/en/4.1/ref/models/fields/#decimalfield
    balance = models.BigIntegerField(blank=False, null=False)
    cash_to_invest = models.BigIntegerField(blank=False, null=False)
    cash_to_withdraw = models.BigIntegerField(blank=False, null=False)
    amount_bought_intraday = models.BigIntegerField(blank=False, null=False)
    amount_sold_intraday = models.BigIntegerField(blank=False, null=False)
    amount_open_orders = models.BigIntegerField(blank=False, null=False)
    amount_open_withdrawals = models.BigIntegerField(blank=False, null=False)
    amount_estimate_taxes = models.BigIntegerField(blank=False, null=False)

    def __str__(self):
        return f"{self.broker}, {self.broker.mode}, {self.lastname}, {self.firstname}, {self.datetime}"

account.views.py:

from datetime import datetime
from django.contrib.auth.mixins import LoginRequiredMixin
from django.core.exceptions import ImproperlyConfigured
from django.db.models import QuerySet
from django.urls import reverse_lazy
from django.views.generic import ListView

from .models import Account
from broker.models import Broker
from power_tool.requests import get_request


class AccountListView(LoginRequiredMixin, ListView):
    model = Account
    template_name = "account/list.html"

    login_url = reverse_lazy("base:login")

    class Meta:
        ordering = ["broker"]

    def _create_account(self, broker):
        request = get_request(broker.mode, broker.trading_api_key)
        if request.status_code == 200:
            results = request.json()["results"]
            Account.objects.create(
                broker=broker,
                firstname=results["firstname"] if results["firstname"] is not None else "not defined",
                lastname=results["lastname"] if results["lastname"] is not None else "not defined",
                mode=broker.mode,
                balance=results["balance"],
                cash_to_invest=results["cash_to_invest"],
                cash_to_withdraw=results["cash_to_withdraw"],
                amount_bought_intraday=results["amount_bought_intraday"],
                amount_sold_intraday=results["amount_sold_intraday"],
                amount_open_orders=results["amount_open_orders"],
                amount_open_withdrawals=results["amount_open_withdrawals"],
                amount_estimate_taxes=results["amount_estimate_taxes"],
            )

    def _update_account(self, broker):
        request = get_request(broker.mode, broker.trading_api_key)
        if request.status_code == 200:
            results = request.json()["results"]
            broker.account.firstname = results["firstname"] if results["firstname"] is not None else "not defined"
            broker.account.lastname = results["lastname"] if results["lastname"] is not None else "not defined"
            broker.account.mode = broker.account.broker.mode
            broker.account.balance = results["balance"]
            broker.account.cash_to_invest = results["cash_to_invest"]
            broker.account.cash_to_withdraw = results["cash_to_withdraw"]
            broker.account.amount_bought_intraday = results["amount_bought_intraday"]
            broker.account.amount_sold_intraday = results["amount_sold_intraday"]
            broker.account.amount_open_orders = results["amount_open_orders"]
            broker.account.amount_open_withdrawals = results["amount_open_withdrawals"]
            broker.account.amount_estimate_taxes = results["amount_estimate_taxes"]
            broker.account.save()

    def get_queryset(self):
        """
        Return the list of items for this view.
        The return value must be an iterable and may be an instance of
        `QuerySet` in which case `QuerySet` specific behavior will be enabled.
        """

        # Update or create accounts
        broker_list = Broker.objects.filter(state__exact="enabled")
        for broker in broker_list:
            # Update existing accounts
            if hasattr(broker, "account"):
                # TODO: Use logging
                # print(
                #     f"account.datetime: {account.datetime}",
                #     f"datetime.now: {datetime.now().astimezone()}",
                #     f"datetime difference: {datetime.now().astimezone().timestamp() - account.datetime.timestamp()} s",
                #     sep="\n",
                # )

                # Check if >=60 seconds have been elapsed before updating an account
                if datetime.now().astimezone().timestamp() - broker.account.datetime.timestamp() >= 60:
                    self._update_account(broker)

            # Create new accounts
            else:
                self._create_account(broker)

        # Delete accounts if broker is disabled
        Account.objects.filter(broker__state__exact="disabled").delete()

        # Get currently configured accounts
        if self.queryset is not None:
            queryset = self.queryset
            if isinstance(queryset, QuerySet):
                queryset = queryset.all()
        elif self.model is not None:
            queryset = self.model._default_manager.all()
        else:
            raise ImproperlyConfigured(
                "%(cls)s is missing a QuerySet. Define "
                "%(cls)s.model, %(cls)s.queryset, or override "
                "%(cls)s.get_queryset()." % {"cls": self.__class__.__name__}
            )
        ordering = self.get_ordering()
        if ordering:
            if isinstance(ordering, str):
                ordering = (ordering,)
            queryset = queryset.order_by(*ordering)

        return queryset

power_tool.requests.py: (helper functions)

import requests


def _get_domain(mode):
    return (
        "https://paper-trading.lemon.markets/v1/account"
        if mode == "paper"
        else "https://trading.lemon.markets/v1/account"
    )


def get_request(mode, trading_api_key):
    domain = _get_domain(mode)

    return requests.get(
        domain,
        headers={"Authorization": f"Bearer {trading_api_key}"},
    )

I will check if I understand how async works in Python… :smiley: