Best way to have local static files and S3 media coexist

Using Django
1

Hello everyone!

I’m new to Django and to web development in general, please pardon me if my question seems trivial. :pray:

So, I started a project where I have a limited number of static images.
My current objective is to make several series of images available (let’s call them “campaigns”, where each campaign can consist of easily thousands of images). The solution I was thinking about was the following :

  • storing all my (few) assets within the project
  • while storing the media on S3 buckets (one for each “campaign”). The images would be served with a simple reference to their URL on S3.

:arrow_right: Question 1: does this architecture make sense?
:arrow_right: Question2: if so, how to do this exactly? If not so, what is the alternative?

I’ve completed multiple tutorials related to this, however all of them seem to assume you simply want to keep every single static file in S3. I have read the documentation and failed to see how I should go about my problem. I’ve dug up several questions from StackOverflow and the likes, but couldn’t find anything I felt was any help.

2

There probably are multiple ways of doing this - one way that I might use is to take advantage of the feature of S3 that allows it to directly serve files from a container. I would set up a table in my Django environment that maps the image information to the name in which it exists out on S3. Then, in my template, I can generate an img tag with the src being the url of the file in that container.

This would allow me to manage / store those images in S3 without committing all my static resources out there.

If the amount of your resources are small compared to the number of images, then I’d be most concerned about the images - in terms of traffic and load, is sounds like your resources would be negligible.

Ken

1 Like
3

I would suggest using django-storages with the S3Boto3Storage (https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html).

I believe you can use the DEFAULT_FILE_STORAGE to handle your media (i.e. all your campaign images) and skip STATICFILES_STORAGE to use a local storage scheme to run for your static files with collectstatic.

4

A big thank you to both of you for these answers. I’m still (very much) struggling with the code, if anyone wants to contribute please go ahead.

Anyway, I’ll let you know what it looks like in the end. :slightly_smiling_face:

5

It’s not really practical for me to contribute code in the absence of the context of the overall application, but if you post the particular sections of code that are giving you problems, I’d be more than happy to review them.

1 Like
6

Thanks very much Ken, and sorry for the ridiculous delay.
I finally decided to go all in and put all of my assets (both my static assets and the ones related to the campaigns) into S3 for now, for 2 reasons:

  1. It was easier than trying to diffentiate
  2. I understood that Heroku doesn’t have a great files system. So this way, I don’t need it.

I’ll probably need to improve this later, but for now (now = dev & debugging, no deployment yet) it seems to be working fine.

Thanks again for your input!