Reading data from .sql file and load to django application database.

Using Django Using the ORM
1

I have backup file (data.sql), and I need to read some of its data and transform and load them to my django app using ORM. I wanted to do all process in django project, to be able to use with command like manage.py load_data. But I have no idea how to read directly from .sql file. How can I read all data from that file ?

2

If those are SQL statements in that file (“SELECT”, “INSERT”, “DELETE”, “COPY”, etc), you’re going to need to write something to parse those statements to figure out what you might want to do with them. The ORM itself is not going to provide you with any facilities to directly assist you with that. About the closest that it’s likely to get would be for you to process those statements as RawSQL. (But that’s not going to help you with doing any data transformations / filtering.)

It may actually be easier if you were to create another database, load that data into that database, create a set of Models using inspectdb, and then use the ORM to copy from one to the other.

3

Yes, they are SQL statements in that file, actually I am not going to use ORM for that, I just need to read all data from that file, and I can use them to load into my existing db. The problem is reading the data, I wanted to do it without creating database.

4

Yes you can do that. But it’s going to take a lot more time and effort than just creating a database temporarily to work from. Parsing arbitrary SQL statements is not a trivial task.

5

I’ve read data using pgadmin4 and going to export it as csv file then load into django db, is it good solution ? And also Is there a possibility that all the data may not back up? Because when restored I noticed that some data lost.

6

Not as a general rule. There are too many edge-cases of types of data in databases that may create invalid or improper lines in your CSV.

Your best solution is to create models for the originating database, and use the ORM to create your functionality to move data from the original database to your new database.

Your second best solution would be to take the SQL backup and load it into a database temporarily, and work with it from there.

Every other solution creates potential issues that may need to be addressed regarding data types and formats.