URLResolver failing to find pattern match for admin POST request - cPanel/Apache

[alstr]

I have a problem with the Django site deployed on Namecheap shared hosting running cPanel/Apache.

Recently a problem that started out as relatively infrequent has become a regular occurance, without any change on my side.

When submitting a POST request in an admin form that has an ImageField, a 404 response is returned and I see the handler404 view. This does not happen locally or with my deployments on Heroku or Google Cloud Platform so I am saying it is a specific issue with this server setup. GET requests to the same server work perfectly fine, however, as do forms without images.

Through much testing with a broken admin change view, I have located the source of the problem to the URLResolver in django.urls.resolvers. I’ve added a few logging calls and have noticed that a match is not being found due to differences in the URL structure:

The URLResolver actually first matches the URL:

PATH: /myadmin/myapp/mymodel/1/change/
MATCHED PATTERN: ^/

PATH: myadmin/myapp/mymodel/1/change/
MATCHED PATTERN: myadmin/

PATH: myapp/mymodel/1/change/
MATCHED PATTERN: myapp/mymodel/

ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=[], namespaces=[], route=<path:object_id>/change/)

It then tries to process it again, this time without the myadmin/ prefix, so it obviously doesn’t match anything and nothing is updated.

PATH: /myapp/mymodel/1/change/
MATCHED PATTERN: ^/

PATH: myapp/mymodel/1/change/
FAILED PATTERN: myadmin/

...


Not Found: /myadmin/myapp/mymodel/1/change/

I don’t know why the path is handled differently this second time around (or indeed why it is handled twice). There are no path issues when I run it locally.

Putting aside the fact that this is not the best place to host a Django site (which I’m aware of and can’t really change at the moment), what might be a way of resolving this?

To experiment, I’ve tried:

• Editing _get_response(self, request) in django.core.handlers.base to do resolver.resolve(request.path) rather than resolver.resolve(request.path_info). This now allows the model to be updated, unlike before, but a 404 is still returned.
• Looking to see if httpd.conf can be edited to set a WSGIScriptAlias, but shared hosting doesn’t allow that.

This seems to be an issue that others are having with the same setup:

django.contrib.admin.options.add_view 404 with image field

I’ve moved my app to my server for production. Some of my models work as expected, i can upload them via admin, see them etc. The ones that return django.contrib.admin.options.add_view 404 error, contain image fields. I’ve tested the MEDIA_URL and it returns an image as expected. The MEDIA_ROOT is set to what i need it to be. I’ve looked around the internet for 2 days now, all solutions i have found are regarding development. On development, everything is fine. Can someone help to point me …

django on cpanel returnt 404 uploading file

Hello, I am a little new deploying a project in django, this is a personal one for my master’s thesis, I am having problems uploading files from django in a namecheap share hosting. if anyone has a guide or something i’m not doing well in the apache configuration please i would be grateful for the help I leave the link to stack overflow but if you want to read part of what is there I will leave it down here when I upload a file either by admin or by a view it responds 404 settings DEBUG …

I am a new user so can’t post additional links but there is one on StackOverflow with
URL /questions/57344774/django-admin-returns-404-on-post-200-on-get.

None of these have any solutions.

Any ideas would be appreciated. The project URLs are defined as follows:

# Import custom AdminSite that just adds a couple of views.
from myapp.admin import admin_site


urlpatterns = [
    path('', include('myapp.urls')),
    re_path(r'^myadmin/', admin_site.urls),
    re_path(r'^ckeditor/', include('ckeditor_uploader.urls')),
    re_path(r'^maintenance-mode/', include('maintenance_mode.urls')),
    path('sitemap.xml', index, {'sitemaps': sitemaps}),
    path('sitemap-<section>.xml', sitemap, {'sitemaps': sitemaps},
         name='django.contrib.sitemaps.views.sitemap')
]

[practiceryan]

I’m also with Namecheap shared hosting. I have had to temporarily remove option for user image uploads due to this issue.

[KenWhitesell]

Couple comments about your urlpatterns -

• the empty path pointing to myapp.urls doesn’t seem like it’s going to match anything other than an empty path, which means there wouldn’t be any mapping to the urls in myapp.urls.
  (Notice that none of the examples in the URL dispatcher docs point an empty path to an included set of urls.)
  Should your first entry be: path('myapp/', include('myapp.urls')),?
  (Also see the paragraph on Including other URLConfs to see how that functions.

• I don’t understand why you would want to specify a regex path for myadmin, ckeditor, or maintenance-mode when none of those entries are regular expressions. I’m guessing it’s not hurting anything, but it does seem unnecessary.

Ken

[alstr]

Hi @KenWhitesell. Thanks for the reply.

Regarding the first point, I want to serve the app views from the base URL (i.e. /). I understand your comments but if I serve it from myapp/ I’m not sure whether this server setup would allow me to create an alias to map it to something else. Unless it’s possible to rewrite the URL through .htaccess but I imagine it would be rewritten before it reaches Django and therefore not work. Anyway, I’ve never had a problem with this on Heroku or Google Cloud Platform.

Regarding the second point, it is more force of habit when writing out the example for the post. The actual urls.py doesn’t use re_path for the admin URLs but I can confirm it doesn’t make a tangible difference.

[KenWhitesell]

In that particular case, I would just define my myapp URLs in my root URL conf - I wouldn’t include them. (I can’t address server-specific behavior of this - I’ve only ever deployed Django in environments where I have complete server-level control.)

[alstr]

Thanks for that. I’ve just run a test by changing it to path('test/', include('myapp.urls')) and it made no difference to the admin 404 error. In any case the admin path is myadmin/ so I wouldn’t have thought the myapp path being blank would have caused a problem.

[KenWhitesell]

Admittedly, it was a real WAG.
My thought was that that since urlpatterns are checked sequentially, that that formulation of a path was somehow interfering with the matching of urls behind it in the list. (Wasn’t the first time I’ve been wrong, won’t be my last. ) Thanks for taking the time to check.

[alstr]

Thanks for any comments, always happy to try things.

I’ve edited the URLResolver just to make sure it’s processing the correct URL, for experimentation only. I’ve done some more logging and I’m noticing something weird.

Inside _get_response(self, request) in django.core.handlers.base.py, I added the following logging statements:

logging.info('TRYING TO RESOLVE')
resolver_match = resolver.resolve(request.path_info)
logging.info('THIS IS THE RESOLVER MATCH: %s' % (resolver_match,))

On GET (and when run locally), the second log statement appears:

TRYING TO RESOLVE
...
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=[], namespaces=[], route=<path:object_id>/change/)
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=[], namespaces=[], route=<path:object_id>/change/)
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=['admin'], namespaces=['mycooladmin'], route=myapp/mymodel/<path:object_id>/change/)
THIS IS THE RESOLVER MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=['admin'], namespaces=['mycooladmin'], route=myadmin/myapp/mymodel/<path:object_id>/change/)

…but not on POST:

TRYING TO RESOLVE
...
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=[], namespaces=[], route=<path:object_id>/change/)
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=[], namespaces=[], route=<path:object_id>/change/)
SUB MATCH: ResolverMatch(func=django.contrib.admin.options.change_view, args=(), kwargs={'object_id': '1'}, url_name=myapp_mymodel_change, app_names=['admin'], namespaces=['mycooladmin'], route=myapp/mymodel/<path:object_id>/change/)
Not Found: /myadmin/myapp/mymodel/1/change/

The sub match is being printed but I don’t know what’s happening after the ResolverMatch is being returned by resolve(self, path) in django.urls.resolvers.py:

if sub_match:
    logging.info('SUB MATCH: %s' % (sub_match,))
    # Merge captured arguments in match with submatch
    sub_match_dict = {**kwargs, **self.default_kwargs}
    # Update the sub_match_dict with the kwargs from the sub_match.
    sub_match_dict.update(sub_match.kwargs)
    # If there are *any* named groups, ignore all non-named groups.
    # Otherwise, pass all non-named arguments as positional arguments.
    sub_match_args = sub_match.args
    if not sub_match_dict:
        sub_match_args = args + sub_match.args
    current_route = '' if isinstance(pattern, URLPattern) else str(pattern.pattern)
    return ResolverMatch(
        sub_match.func,
        sub_match_args,
        sub_match_dict,
        sub_match.url_name,
        [self.app_name] + sub_match.app_names,
        [self.namespace] + sub_match.namespaces,
        self._join_route(current_route, sub_match.route),
    )

[KenWhitesell]

This is quite, ummm, “interesting”, to say the least. I’m not even sure how I would go about diagnosing it at this point. Clearly, there’s something environmental going on - but I’m drawing blanks trying to think of how I could identify what.

Lacking any real visibility into the environment, I’d probably try installing django-debug-toolbar - just to give me some visibility into what the server’s seeing through the requests.

But, if I’m reading this correctly from their website, given that you have no control over the environment - what version of Django is being used, what verson of mod_wsgi they’re running, contents of the .conf or .htaccess files - even if you find why, you may not be able to resolve the problem.

[alstr]

Thanks @KenWhitesell, I am generally thinking along the same lines.

As I said initially, I’m aware that shared hosting is not the best place to deploy a Django site, but right now it’s hard to move elsewhere due to cost limitations on the side of the organisation running the website, and the fact that it seemed to work fine previously (and still largely does). Given that there seemed to be a fairly large number of people using this setup when I went ahead with this provider, I thought it’d be okay, but only recently have I noticed these kind of problems.

I think I’ve debugged this just about all I can do and I’ve hit a brick wall. I don’t know why things are happening the way they are and I can’t find any way to gain greater clarity with this server setup.

I’m hoping someone… anyone… will come up with a wonderful solution - it’s not just me with this problem after all - but if nothing comes up soon I think the only compromise will be to make some kind of Ajax instant-file uploader, that uploads the file outside of the form (i.e. before submission), and then just populates a URL form field with the path of the file. Something to that effect. Hopefully that would work. Not ideal, but that’s where I’m at, and the only other solution is to migrate the site elsewhere.

[alstr]

I have not solved this problem, but I have developed a workaround.

1. Edit _get_response(self, request) in django.core.handlers.base. Change resolver_match = resolver.resolve(request.path_info) to resolver_match = resolver.resolve(request.path).

2. Add this middleware (adjust to your exact needs):

    class ImageField404Middleware:
        def __init__(self, get_response):
            self.get_response = get_response
   
        def __call__(self, request):
            response = self.get_response(request)
   
            if (request.method == 'POST' and request.user.is_superuser and response.status_code == 302
                    and request.get_full_path().startswith('/pathtoadmin/')):
                post_messages = get_messages(request)
                for message in post_messages:
                    if ('was added successfully' in message.message or 'was changed successfully' in message.message
                            and message.level == message_levels.SUCCESS):
                        messages.success(request, message.message)
                        redirect_url = request.get_full_path()
                        if '_addanother' in request.POST:
                            redirect_url = re.sub(r'[^/]*/[^/]*/$', 'add/', redirect_url)
                        elif '_save' in request.POST:
                            redirect_url = re.sub(r'[^/]*/[^/]*/$', '', redirect_url)
                        return HttpResponseRedirect(redirect_url)
   
            return response
   

I have tested and this works with admin forms with ImageFields. Uploaded an image without issue.

Editing base.py isn’t ideal by any means but this is the least invasive solution I have found (indeed, the only thing that works). I don’t think using request.path instead of request.path_info will make a difference in my case, but other projects may not be able to do that.

It’s a horrible workaround, but it’s all I’ve got at the moment. Any thoughts in the absence of other solutions?

[KenWhitesell]

That’s an amazing catch! I don’t have any specific thoughts on it other than I tend to be very pragmatic about such things. If it works for you, and you’re satisfied with it (if not happy), then it’s a great solution.

Ken

[shoronts]

Thanks a lot brother. It works for admin. But I have this problem for user site. When I upload pics then I saw this errors:

App 87338 output: /opt/passenger-5.3.7-4.el6.cloudlinux/src/helper-scripts/wsgi-loader.py:26: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module’s documentation for alternative uses
App 87338 output: import sys, os, re, imp, threading, signal, traceback, socket, select, struct, logging, errno
App 87338 output: [ pid=87338, time=2020-08-05 19:40:37,324 ]: Not Found: /profile/profile/

I am not an expert on djnago. I just started it and have made a website something similar to facebook. Please help me to solve this problem.

[alstr]

I’ve not experienced this issue on the user-facing site but I don’t allow image/file uploads so that might be why.

You may be able to make some adjustments to the middleware I posted above to make it work for the user-facing site. It would probably involve inspecting the request for your user-facing POST URL and then performing a redirect to the correct destination URL. It will be a bit of trial and error unfortunately until a proper fix comes to light.

[alstr]

I have some more information that you may find useful.

I encountered the problem on the user-facing side recently after integrating some new functionality.

It seems there may have been multiple issues, but I did the following to resolve it:

• I had a form that was performing a HttpResponseRedirect on success, but it was 404ing because the URL prefix was getting duplicated for some reason, e.g. /mypath/mypath/page/. I’m not sure if the problem was the reverse function or something else, but I checked the path for the duplicate prefix and removed it before performing the redirect, which solved the 404.
• If you have any IP rules set up in cPanel, if you are missing a 403.shtml inside your public_html server directory the server throws an error seemingly before the request even gets to Django.
• I had a form with a WYSIWYG editor that allowed video embedding. This produced a 403 error on submit, but only when a video was embedded. I presumed this was getting blocked by the server for potentially malicious HTML content in the form field. I spoke to hosting support and found out:

As I have checked, the issue is triggered by ModSecurity Apache. ModSecurity is an Apache module which works as a web application firewall. It blocks known exploits and provides protection from a range of attacks against web applications.

• They made a change and it worked fine after that. So far all is working okay user and admin side.

I hope this helps.

[alstr]

Small amendment for anyone interested in the future:

class ImageField404Middleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)

        if (request.method == 'POST' and request.user.is_superuser and response.status_code == 302
                and request.get_full_path().startswith('/pathtoadmin/')):
            post_messages = get_messages(request)
            for message in post_messages:
                if ('was added successfully' in message.message or 'was changed successfully' in message.message
                        and message.level == message_levels.SUCCESS):
                    messages.success(request, message.message)
                    redirect_url = request.get_full_path()
                    if '_addanother' in request.POST:
                        redirect_url = re.sub(r'[^/]*/[^/]*/$', 'add/', redirect_url)
                    elif '_save' in request.POST:
                        redirect_url = re.sub(r'[^/]*/[^/]*/$', '', redirect_url)
                    elif '_continue' in request.POST:
                        redirect_url_search = re.search(r'((?<=href=)[^>]*)', message.message)    
                        if redirect_url_search:                                                  
                            redirect_url = redirect_url_search.group(0)                                                  
                            redirect_url = re.sub(r'[\\"]*', '', redirect_url).replace('/pathtoadmin/pathtoadmin/', '/pathtoadmin/')
                    return HttpResponseRedirect(redirect_url)

        return response