Migrating media files to an external server

Hello,

I have a Django app with a lot of media files stored locally through the MEDIA_ROOT setting. Basically the APP keeps control of the media files uploaded by the users to ensure its traceability.

Now I need to migrate all these files to a dedicated storage server (say IP 10.10.10.25) while keeping the Django APP in the original machine (say IP 10.10.10.5).

I wonder what would it be the best way to do this migration?? How should I configure the MEDIA_ROOT and the NGINX server to store and serve the media files to/from the new machine??

kind regards!!

Welcome @mizamae !

You’ve actually got three different issues here -

  • Moving the files
  • Retrieving uploaded files
  • Allowing for the upload of new files

Note: Selecting an option for these is likely to involve non-technical as well as technical considerations regarding the configuration and management of that new storage server.

The first, moving the files, is rather easy - assuming you’re going to keep the same directory structure relative to MEDIA_ROOT. Copy the files over and you’re done.

The second, retrieving the files, can be done a couple of different ways.

One of the easiest would be to set up nginx on the storage server (.25), and configure your MEDIA_URL to request files directly from .25

Or, configure your media location in the nginx on the original machine (.5) to proxy the requests through to .25. (Doesn’t require that MEDIA_URL be changed.

Or, set up your media directory on .25 as an NFS mount, and configure the nginx media location on .5 to access those files through the mount point.

(There are more options beyond this, but they get increasingly more complicated, and really only useful under specific circumstances.)

Setting up the media storage location as an NFS mount on .25 would allow new files being uploaded to be placed there.

Or, you could implement the ftp or sftp from django-storages to allow the uploads to .5 to be sent to .25.

Or, you could create a “file-store” API on .25 to allow for files to be sent to it directly.

Thanks a lot!! It is very much appreciated!!

If I understand correctly, you have focused on retrieving the existing files, but I would also need to upload new files (directly to .25).

The NFS mount seems to be quite straightforward but will check with IT regarding permissions and all those things…

Regarding the “django-storages”, I had a look at it before writting this issue and seems to solve the 2-way travel of the files as well (through FTP protocol) but I have not clear if I should also modify NGINX in any manner (maybe the /media/ directive should be redirected to the FTP server?)

Anyhow, loads of thanks for your unvaluable support Ken

That’s covered by my last three lines (after the parenthetical comment).

Yep, that’s part of the “non-technical” issues I mentioned.

Maybe.

You can run nginx on .25 and have nginx on .5 proxy to it, or change your MEDIA_URL to retrieve media files from .25 directly. But if you use an nfs mount, then your configuration might not need to be changed.

One item I haven’t looked at is whether the django-storages will make a distinction between your staticfiles and media files - and that’s an issue you’ll also have to consider.

1 Like

Hi
Using an NFS mount on the new server (.25) is a simple way to handle both old and new files. Another option is using django-storages with FTP to manage uploads and downloads.