Initial database population using csv

Using Django
1

I am trying to create a management function to populate database tables with initial data from csv files. There are 9 csv files, all with varying number of columns. I have managed to get almost to the end apart from the following issue. When I get the model field names (with the expection of id which is auto populated):

all_fields = [f.name for f in table_object._meta.fields][:1]

This causes an issue were any foreign key field name is missing _id in the model foreign_key_field_name. This causes an issue with:

_, created = table_object.objects.get_or_create(**insert_dict)

which is expecting the database table name foreign_key_field_name_id

The question is, how do I know when to add an _id to the end of a field name? Is there a way of knowing if a field is a foreign key and therefore Django will have auto populated an _id for the database column name? Or can I get the database field names? Or is there a better way of doing this?!

   class Command(BaseCommand):
    help = 'Populates Static tables from csv files'

    def handle(self, *args, **kwargs):
        for table_object in object_list:

            table_name = table_object._meta.db_table
            all_fields = [f.name for f in table_object._meta.fields][1:]
            print(all_fields)

            path = BASE_DIR_LOCAL / f"data/database_tables/{table_name}.csv"

            with open(path) as f:
                reader = csv.reader(f)
                next(reader, None)  # skip the header
                for row in reader:

                    insert_dict = {}
                    row_num = 0
                    # For cvs files with varying column numbers
                    for field in all_fields:
                        row_num = row_num + 1
                        insert_dict[field] = row[row_num]

                    _, created = table_object.objects.get_or_create(**insert_dict)
                    print(created)
2

To ignore the auto id, it’s safer to use if isinstance(f, AutoField) to check for it, rather than slicing the list of fields like [1:]. If anything changes with auto fields or models it might not appear first in the list of fields any more.

To solve the problem witth field names, maybe you want field.attname rather than field.name ? You should also be able to use isinstance to check for FK’s and OneToOneFields.

3

Hi Adam, brilliant changing the name to attname sorted the core issue. I also changed the id removal logic as you suggested. Thanks again.