Hey,
I’m migrating an application from an old DB system to django. I’ve imported a bunch of legacy data. I used a script to convert the table schemas from the old system into Django models, as it had a lots of tables. Then I used another script to write the content of the old db into Django fixtures, which I then imported. That all worked fine, the data is served in the views etc.
However, I now noticed that some ForeignKey in the old system were not properly identified as FK and were simply put in as Char/Int fields into the djando models. So I updated a field from IntegerField to ForeignKey, but that being legacy data I had the following issue:
django.db.utils.IntegrityError: The row in table 'pickings_detpick' with primary key '17170' has an invalid foreign key: pickings_detpick.fk_ent_id contains a value '30223' that does not have a corresponding value in pickings_entpick.id.
So basically, there are a number of (unsure how many yet) foreign key references that point to objects that actually do not exist in the source data (for whatever reason, I don’t know the hold system much).
Is there a way that I can simply tell Django to ignore those & set them as null or something? Or do I actually have to comb thru the data & fix them?
There is the db_constraint property on the ForeignKey field that you could use to do what you want (get the FK abilities in Django without the DB constraint). However, I would recommend cleaning up that data and using proper ForeignKey constraints to enforce integrity. Otherwise eventually you’ll be bit by this.
That’s exactly the hack that I need actually. I could migrate & test my application at least. Of course there’s the issue that you don’t want to run that production. So between now & prod that’ll have to be fixed.
It’s legacy data - I can’t control it, other than set invalid references to nulls. Or some dummy pk in the foreign that would act as a catch-all for those invalid references. I dunno if it happened because they used some weird on_delete do nothing instead of setting to null or cascade. It’s an old system (dBase) so maybe the functionalities were crap. Or maybe they didn’t enforce the constraints, manually edited stuff & messed things around. At any rate not really my problem - as long as it runs, and as long as new data will have integrity enforce, I’ll be happy.
At any rate not really my problem - as long as it runs, and as long as new data will have integrity enforce, I’ll be happy.
Sorry, I think I missed something and want to clear it up. How are you planning on having data integrity enforced for new data moving forward?
I expressed myself poorly - I’ll eventually have to clean it up. E.g. before that moves in production. In the meantime, I just want to be able to test my dev app with the actual data that will populate it. To that, db_constraints=False will do just fine.
To clean it up, I’ll have to see at the source what would hte cause of that issue is, and what they want to do with it (set to null, or perhaps they actually can/want to fix references, etc.).
I’m guessing I could add clauses to my scripts that produced the fixtures in the first place, to look for missing references in the data, and do whatever with it. Or use RunSQL for the migrations.
Gotcha, thank you for the additional explanation!