Importing folders (and sub-folders) of Markdown posts into an SQLite query

I’m learning Python and Django to migrate my website from Eleventy. I have a folder of posts with subfolders that are broken down by year, and then by month. I’m reading up on importing files and folders into an SQL database as well as learning how to set up Django models.

It’s definitely a learning curve for me, plus I only started learning HTML and CSS to make my own website about a year ago, so I’m writing this as a bit of a sanity check to see if I’m understanding so far what I have to do. (My Eleventy knowledge isn’t great, I had to pretty much rely on the Discord support to help me get the site’s plumbing built because the documentation wasn’t great for a beginner.)

Right now I have two basic, tentative models built for my articles: one is for the articles themselves, and one is for the tags. These articles will go on my homepage, and they can be sorted or filtered by tags or through archives by year or year-and-month.

I keep seeing examples for importing files with Python by looking through all the data files in the folders and then importing all that data into a CSV file — is it ideal or recommended to import all the posts’ text into a CSV (or JSON) file and then importing that into the SQLite database, or should I import the posts’ data straight into the database?

If using a CSV file is recommended, since I have a JSON feed for my posts, should I replace CSV with JSON?

Naturally, let me know if anything I wrote is confusing or you need more details. I’ve been wanting to dabble with Python for a while, and migrating my site seems like a great way to do so, even if I’m diving off the deep end a bit. I’m happy to learn!

Actually, it’s not.

First, you should be clear that Django is not a Content Management System (CMS). It’s a tool that you could use to build a CMS. But there’s a lot of work that you would need to go through before you’re at the point where you can begin to even think about migrating your site to a bare Django platform.

If learning Python and Django are your ultimate goal, then you first need to learn Python, with an emphasis on classes, modules, and the Python Method Resolution Order (MRO). Once you’re comfortable with that, then I believe the best places to start in Django are the Official Django Tutorial and the Django Girls Tutorial . Once you’ve worked your way through them, look for additional resources on the Awesome Django page in the “Educational” section.

First, you should be clear that Django is not a Content Management System (CMS). It’s a tool that you could use to build a CMS.

Right, I’m not viewing it as a pre-built CMS, I’m trying to figure out how to build a system to scan the folders for files and import them to the database.

But I’m also not clear on what you mean by needing to go through a lot of work before I can even think about migrating my site to a Django platform? So far I’ve got my template built, I have my CSS hooked up to it, I switched the templating language from Django to Jinja, and I’ve got a media folder hooked up and can display images. I’m not doubting there’s a learning curve with this as I had to do quite a bit of reading to wrap my head around how to set up my Django project to get to this point, but that is what I have so far.

For my database, I’m trying to understand what I need to read up about with Django and Python to import Markdown files into the database properly, besides the basic Python tutorials on reading files and folders.

I do have the Django and Django Girls tutorials bookmarked, among others, but the Awesome Django page is new to me, thanks!

What you haven’t described here are the views and urls necessary to render your templates and provide navigation on your site, nor the knowledge of the ORM to access the data from the database.

Yes, you can short-cut a lot of things along the way - but the problem with taking this approach is the risk of learning the wrong things. It’s too easy to draw incorrect conclusions from what you do learn, making things tougher for yourself in the longer run.

If you really want to learn Django, start from the beginning. Ensure you understand the fundamentals of models, views, templates, urls, and the ORM, and how they work together to produce pages that are integrated with a logical template structure, and navigable among them.