Conversion and import of a pre-existing database

Using Django
1

Hi,

I’m working on my first Django project. Already read the tutorial but following along it to be sure not to miss anything.

I’ve been asked to rebuild a pre-existing website. It’s basic in functionality: various users create complete descriptions of heritage items in the admin area, and the public page display the description, mostly in text format, sometimes with images too. The final user can navigate the items on the public page.

The data model contains many different classes and variables - it’s well structured and doesn’t need to change.

The data is stored into a MariaDB database. The new website will use PostgreSQL, so I need to convert the MariaDB database and import it into PostgreSQL. I’m aware of inspectdb but I decided to manually write models.py from the ground to get acquainted both with the data and with Django models.

What I’d like to know is:

  • Should I work on the conversion and import of the MariaDB database into PosgreSQL without thinking about Django at this stage?

  • I read that Django will add “_id” at the end of the field names. Should I worry about that? Or should I leave in PgSQL the same field names there was in MariaDB? How should I name my variables in models.py then?

  • Any other advice will be very welcome :slight_smile:

Thanks a lot for your help,

Jeremy

(edit: the source database is MariaDB, not MSSQL! My bad)

2

I wouldn’t. I’d start from building these as Django models. You’ll have a lot of tools available to more easily manage this conversion. It’s a lot easier to start from the perspective of “this is going to be a Django database, let’s migrate data into it” than “let’s build Django around this legacy schema”.

Additionally, it’ll help you identify some of the requirements that Django puts on your database structure.

For example, Django assumes every table has a single-column primary key. If the existing table doesn’t have a primary key, Django’s default is to create a column named “id” for an autoincrement field to use as the primary key.

This isn’t a completely accurate statement. That suffix is only added to the column name of a ForeignKey field by default.

Tools like inspectdb, dumpdata and loaddata are very much your friends for this type of project. Using inspectdb is a great starting point, but it’s not a 100% solution. Even after using it, you’re going to need to manually inspect all the results to ensure the new models are going to work. If the schema is at all complex, with many tangled foreign key relationships, something like SchemaSpy can be helpful trying to untangle it.

1 Like
3

Hi Ken,

A late message to thank you for your very detailed answer.

For now, due to external constraints, I’ll have to keep the original db as it is and just build a page to visualize the data. I build my models.py as if it was from scratch, except that:

  • For every class I added a Meta subclass with managed = False and db_table = 'original_table_name';
  • For every variable I added a db_column='original_column_name'.

This way it allows me to interact with the existent database. Once the project will be OK to fully migrate to Django, including the forms part, I’ll rework on the models.py to recreate a fully Django-compliant database. Then we’ll migrate the data and get rid of the old MariaDB one.

I hope I’m on the right track this way! :slight_smile:

4

Inspectdb creates the models with both the managed and db_table settings.

We have projects referencing existing tables in multiple databases. We create separate database connections to access the original data. We then add a prefix to the beginning of each model class to identify that a table comes from a non-default database, and have a router configured to route all access to any model with one of the prefixes to the appropriate database.

1 Like