Best practice to insert data automatically from a pipeline

Hi Django community,

(I am new to this forum, I tried my best to formulate my question)

Context: I study bioinformatics and for my minor ‘webapplication development’ my project group and I have to built a webapplication using Django. I am responsible to create models and write code that reads, parses and inserts data (using MySQL) into these models. The data comes from a pipeline, using Snakemake, and it has to be inserted automatically into these models. I wrote this code in that does the trick using dummy data (see code below for an example).

def insert_data_into_database():
"""location entiteit"""
with open(snakemake.output.a, 'r') as tsv:
    for line in csv.reader(tsv, delimiter='\t'):
        with open('C:\\Users\\Aron\\PycharmProjects\\bpexi\\bpexi\\lysteria\\%s' % (line[2]), 'rb') as image:
            binary_data =

My problem is: I have to read, parse and insert a lot of different tab seperated files, so becomes cluttered with my code. This feels kind of counter intuitive.

My question is: What are good practices to insert data, from a pipeline, automatically into my models? I tried to create another file into my app directory, but that does not seem to work due to the fact I can not import my models for some reason (see code and error below).


from .models import Sample


ModuleNotFoundError: No module named '__main__.models'; '__main__' is not a package



from models import Sample


django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

I think you probably want to wrap your code in a custom management command: .

1 Like

I made a command that inserts all the data into my models.

Thank you so much!

I am in a similar situation. Would you be kind and let me know what was your final approach? Did you make a custom script and schedule it in task scheduler? So when it executes, it sends data to your models?

Hi, I solved my problem with this tutorial:

The tutorial explains how to make a script that stores data in your models.