@qvisty with Heroku the application will need to have the media files uploaded to a third party service such as S3, Azure or Google Cloud. A library to help with this is django-storages. It also supports uploading your static files if you want to. However, it may be easier if you use whitenoise for the static files.
I ended up getting it working using this:
https://docs.djangoproject.com/en/4.0/ref/urls/#static
I seems that I have to do this extra thing to get it going in debug mode.
I am looking forward to learning about serving media in production. I am planning on reading up on WhiteNoise. 
Thank you all for your help.