I have a local project and I want to deploy it. The server is ready and the code files are uploaded. Locally I use mysqlite, for deployment I have a mysql database.
When I use dumpdata > db.json and upload the file, it complains about the file not being UTF8. When I locally convert the file into UTF8, it loses half its size and when I upload that, it will not be loaded either.
When I check out the file, I can see that it breaks off right in the middle of it. As far I can tell, the model type that is not UTF8 compatible is a django cms page. The file breaks off where the body-property opens. So I figure that there is something there that cannot be implicitly converted into UTF8.
So how do I use dumpdata properly? I want all the data from the local database to be transfered to the new database which is entirely empty. Is there a way to force dumpdata to create a UTF8 file? Is there a different way without using Django to convert the data from the sqlite db into the mysql db?
I don’t have an immediate suggestion regarding the dumpdata / loaddata issue, but you could also dump your database as SQL statements. That might make it easier to identify / fix specific data causing issues.
sqlite3 db.sqlite3 .dump
Assuming the data is clean, you could then run those sql statements to load your new database.
It wasn’t clean. 
So, I tried the SQL dump and when I tried to reimport, it told me that there something wrong at line 1. I opened the dump file, it looked normal… So instead of trying to fix that, I did something else. I uploaded my mysqlite-db onto the server. I told the installation to use this db. I did “dumpdata” on the server, aka in a linux environment, where the file does not get changed to UTF16. I then reconfigured the installation to use the MySQL-db and did “loaddata”.
After learning that you should omit auth.permissions and contenttypes to avoid an IntegrityError, I managed to transfer my data from one db to another. Yay. 