I am new using Django, and am working toward a my first web app using Django.
As a brief review, for this project, we got the fresh product inventory data on a daily base from external sources, data cleansing and save into a usable format, the simple Django app that I build is for the user to be able to search for specific product using product id and display all the records related with this specific product. The users do not need to add new product inventory to the data model. I build the Product data model, and I can initiate this model with the product data using the fixture functionality. And as the search result, the products records can be displayed correctly.
Now before deploying this app to web, I found that I am facing a bottleneck: since the product inventory data is refreshed daily, I need to update the data in the model using the new product inventory data daily, I do not need to keep the old data, I can manually update the daily data using fixture during the development, but after deploying this app to web, how to do the daily update of data automatically? Maybe there is serious flow in my design from the beginning, what should I do to correct this design in order to accomplish my goal? Or is this doable or feasible in Django app? I appreciate any comments or any suggestion. Thanks a lot
To some degree, how you want to do this will depend upon how you’re able to retrieve and process the updated data.
If this is something that can be fully automated, you can create your own custom django-admin command. You can then schedule this to be run at the proper time every day - perhaps using cron.
Hi Ken,
Thanks a lot for your suggestion and your direction.
To clarify, when you say ‘how you’re able to retrieve and process the updated data’, are you talking about the daily product inventory that I received? if that is the case, I already have a fully automated process scheduled to produce the .json data file in a format that is ready for consumption by the command ‘python manage.py loaddata sample.json’. In my development, after I got the data file generated by the automation process, I have to run this command manually to update data in the sqlite3 tables.
So will the ’ custom django-admin command’ be able to automate the data loading part? Please forgive my ignorance due to my lack of knowledge, If that is the case, I will spend time with the custom django-admin command’,
Thanks again for your reply
Ok, you’re a lot farther along than what I had gathered from your previous post.
If a straight loaddata sample.json is going to work for you, then that is the command you want to schedule to run every day. You don’t even need to write a custom django-admin command.
I was assuming that you didn’t already have this part:
I already have a fully automated process scheduled to produce the .json data file in a format that is ready for consumption …
That is what I was thinking you would want to put in the custom command. But since you already have it, there’s nothing more you need to do with it. Schedule them through cron (assuming a unix-ish-based environment) and you’re all set.