I’m attempting to move the data from a WordPress website to a Django application’s database. I am having trouble coming up with a solution because there aren’t many useful resources available on the subject.
I have a script that I downloaded from here, which seems to offer some direction on how to proceed but lacks detailed instructions.
I have a few inquiries about this subject.
I’m positive I need to export each of my WordPress articles as an xml or csv file one at a time. To accomplish this, I’ll need to write a “script”. to do the task. Do I have to host this xml or csv file elsewhere in order for the software that would be performing the migration to access it, or can I utilize this xml file locally?
I was hoping that using the django import export package would be very helpful in accomplishing this, but after exporting the csv file from Wordpress and attempting to import it into Django using the import export package I did notice that the name of the fields in the wordpress model and the one in django model needs to match for it to work. So is there any other package out there that you can specify what fields to populate with some certain fields e.g Populate Django Name field with the Title field in the csv file or would all this need to done using some kind of custom script?
As I need to resolve this problem immediately, any assistance would be greatly appreciated.
In part this depends upon how close the original WordPress database matches what you’re looking to use in Django.
In general, when we’re migrating data from a different system into Django, we do the following:
Create a definition in DATABASES for the original database
Create an app with a name representing the original system
Use inspectdb to create models from the original database for that new app
Fix and adjust those created models as necessary.
Create a set of Django manage commands to create the data from the original database for the new Django system.
By building models for the existing data, we can then do more sophisticated translations than just “copy data from table ‘A’ to table ‘B’”. We can write code to do processing on individual rows, manage foreign key and many-to-many relationships, restructure data, etc.
Whether you need to do something as comprehensive as this depends upon what might need to be done to the existing data to make it work with the new database.
There is a generic class of products known as “ETL” - “Extract, Transform, Load”, that are designed to facilitate operations like this. They’re ok in the more basic cases, but once you get beyond the simpler situation you can quickly encounter limitations that prevent you from doing what needs to be done.
Thanks for your response, i have been going through your answer for some days and it have really given me more clue on how to go about doing the migration.
How about in the case where i only have access to only the csv files of the model exports and not the database.
For example, i have csv exports for Users, Vendors, Products, Category and Orders. So far i have tried importing the Users Data to my Django User models using Django manage commands that you told me. So far it worked well, i successfully imported all the Users to my User Model, created Vendors Objects based on the roles of the users (customer or wcfm_vendor).
I am trying to import the Products from the csv but there is a little issue. Thecsv file for the product contain the images of the products but in a link. See example below
How do i import this images along side other data , NOTE: Other data like the title, price, vendor (foreignkey) are importing successfully, but i do not know how to go about importing the images.
Is it a must that i do these to successfully import the data that are needed in the django system? or, the way i am doing it using django manage command, is that okay?
Do i need to download this images manually to my media folder, this would be a really hard work are there are thousands of products on the original site.
It’s sub-optimal, but assuming you don’t have any data format issues, it generally can be made to work.
The most likely area for concern with trying to use them are ensuring foreign key relationships are properly maintained.
You’d need to somehow create the FileField objects. I don’t know any trivial way of doing it.
I know someone here did a project a couple years ago to import a large number of files into their system. (How to deal with a previous model to display existing images) There might be some useful information in that thread for you.
No, I was just pointing out our process. None of that is required, just that we’ve learned that it’s the best process for us.
I have been looking at the post, i can’t seem to get much information about importing images from a csv file to django model ImageField or maybe i am the one that missing the important part of the question.
I’m kinda stuck now, i was hoping i can get some more resources to help me achieve this.
You’re not "importing images from a csv file, because the csv file doesn’t contain those images. You need to get the files and then write a script to create the FileField instances from those files.
Thanks again for your response, my apologies if i am complicating the question.
I have gone through File Uploads | Django documentation | Django. it shows how to work with uploading files using a form. (that what i can make of it)
This is what the document section said:
When Django handles a file upload, the file data ends up placed in request.FILES (for more on the request object see the documentation for request and response objects). This document explains how files are stored on disk and in memory, and how to customize the default behavior.
I guess i am the one not understanding what to do here but is this what i am supposed to pay attention too (This document explains how files are stored on disk and in memory, and how to customize the default behavior.) with emphasis on …disk and in memory?
Hello, this is Gulsha Negi.
Well, there are several steps involved in migrating data from a WordPress database to a Django database. These steps include exporting the data from the WordPress database, transforming the data so that it conforms to the schema of the Django database, and importing the transformed data into the Django database. The cycle can be robotized utilizing different devices and libraries, for example, the WordPress REST programming interface, Django ORM, and outside movement instruments. During the migration process, data integrity, security, and compatibility issues should be carefully considered.
This was how I imported products (or any other item) from WordPress to Django.
Export the item from WordPress to a csv file.
Create a folder called csv in your root directory and add the products.csv file into that csv folder
In your terminal, run: python manage.py import-products
import-products.py
from django.core.management.base import BaseCommand
from django.utils.text import slugify
from django.core.files import File
from userauths.models import User
from vendor.models import Vendor
from store.models import Product, Category
from userauths.models import User
import urllib
import urllib.request
import pandas as pd
import shortuuid
class Command(BaseCommand):
def add_arguments(self, parser):
pass
def handle(self, *args, **options):
# Delete every other product from the database if necessary
# Product.objects.all().delete()
# Get the path to the CSV file
df = pd.read_csv('csv/products.csv')
for id, vendor_id, title, image, total_sales, post_content, status, stock_qty, price, category in zip(df.ID, df.post_author, df.post_title, df.images, df.total_sales, df.post_content, df.post_status, df.stock, df.regular_price, df.product_cat):
# Check if the product with the same 'id' already exists in the database
product_check = Product.objects.filter(id=id).first()
if product_check:
self.stdout.write(self.style.ERROR(f"Product {id} already exists"))
else:
# Fetch the 'Vendor' and 'User' objects based on 'vendor_id'
vendor = Vendor.objects.get(id=vendor_id)
user = User.objects.get(id=vendor_id)
# Create a new 'Product' object
product = Product(
id=id,
vendor=vendor,
user=user,
title=title,
stock_qty=stock_qty,
price=price or 0,
orders=total_sales,
description=post_content,
status=("published" if status == "publish" else "draft")
)
product.save()
# Generate a unique slug for the product
uuid_key = shortuuid.uuid()
uniqueid = uuid_key[:4]
product.slug = slugify(product.title) + "-" + str(uniqueid.lower())
# Save the image URL to the database and download the image
image_url = image
image_name = image_url.split('/')[-1]
response = urllib.request.urlopen(image_url)
image_file = File(response)
product.image.save(image_name, image_file)
product.save()
# Add categories to the product
category_list = category.split(",")
for c in category_list:
print("Category =============", c)
category_ = Category.objects.get(title=c)
product.category.add(category_)
product.save()
self.stdout.write(self.style.SUCCESS(f"Product {id} created"))
