arabic slug show 404

Using Django Mystery Errors
1

Hi

if i use Arabic slug in localhost that works just fine, but on the server in production any page with Arabic slug well show 404 error, In the same time English slug works.

models.py

    slug = models.SlugField(blank=True, editable=True, allow_unicode=True, null=False, unique=True )

urls.py

from django.urls import path, re_path

from blogs import views

app_name = "blogs"
...
re_path(r'^(?P<slug>[-a-zA-Z0-9 _ءاأإآؤئبتثجحخدذرزسشصضطظعغفقكلمنهويةى٠١٢٣٤٥٦٧٨٩]+)/$', views.BlogDetailView.as_view(),
            name='blog_detail'),
    path('<str:slug>/', views.BlogDetailView.as_view(), name='blog_detail'),

views.py

class BlogDetailView(DetailView):
    model = Blog
    template_name = 'blogs/blogDetails.html'

    def post(self, request, *args, **kwargs):
        year = self.request.POST.get('year_choice', None)
        instance = self.get_object()
        if not year:
            return redirect('blogs:blog_detail', slug=instance.slug)
        return redirect('reports:blog_report', slug=instance.slug, year=year)

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        instance = self.get_object()
        year = self.kwargs.get('year')
        reports = Report.objects.filter(blog=instance)
        years = Year.objects.filter(name__gte=str(instance.since), report__in=reports)
        context['years'] = years
        context['year'] = year
        return context

settings.py

ALLOW_UNICODE_SLUGS = True

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '',
        'PASSWORD': '',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'OPTIONS'   : {
              'init_command'  : "SET sql_mode='STRICT_TRANS_TABLES'",
              'charset'       : 'utf8mb4',
          }
    }
}
2

have you checked you nginx/apache/lighttpd/whatever setup on production? If it works on local/dev and not on production, it looks unlikely it is a django issue. Your safest bet would be to investigate the “odd one out” :slight_smile:

3

i am with a shared hosting provider with this info:

cPanel Version 106.0 (build 11)
Apache Version 2.4.54
PHP Version 7.4.33
MySQL Version 10.5.18-MariaDB-cll-lve
Architecture x86_64
Operating System linux
Perl Version 5.26.3
Kernel Version 4.18.0-372.9.1.1.lve.el8.x86_64
Python v3.9.12
LiteSpeed Web Server

also i can reach the terminal as USER ( no root )

4

well then check your apache configuration options and what you can do with it?

5

You might also want to try writing the slug that you are seeing in a log file to see how it’s being presented to you. It’s possible that the slug is being url-encoded and that you may need to decode it before using it.

6

here is the line from the error log,

[22/Dec/2022 15:39:42] WARNING [django.request:241] Not Found: /blogs/%D8%A8/

i used one character “ب” to ease the debug.

7

That definitely looks like the slug is urlencoded. Try decoding it before using it in the view. (You could override get_object to modify self.kwargs['slug'] before calling super. There are other approaches you could take as well.)

1 Like
8

browsers will send utf-8 characters in encoded form.

so éééé will be sent as %C3%A9%C3%A9%C3%A9%C3%A9

As Ken suggest, you could try to override you get_object method. and do smth similar to this:

from urllib.parse import unquote
unquote('%C3%A9%C3%A9%C3%A9%C3%A9')
# -> 'éééé'
unquote('%D8%A8')
# -> 'ب'

hopefully that works :slight_smile:

9

Many thanks @KenWhitesell @plopidou :rose:

10

Some how i can’t solve it

English slug work, but not the Arabic one.

views.py

class BlogDetailView(DetailView):
    model = Blog
    template_name = 'blogs/blogDetails.html'
    # unquote_slug = unquote(Blog.slug)

    def post(self, request, *args, **kwargs):
        year = self.request.POST.get('year_choice', None)
        instance = self.get_object()
        if not year:
            return redirect('blogs:blog_detail', slug=instance.slug)
        return redirect('reports:blog_report', slug=instance.slug, year=year)

    def get_context_data(self, **kwargs):
        unquote_slug = unquote(self.kwargs.get('slug'))
        print('unquote_slug', unquote_slug)
        context = super().get_context_data(**kwargs)
        
        instance = Blog.objects.filter(slug=unquote_slug)[:1].get()
        
        year = self.kwargs.get('year')
        reports = Report.objects.filter(blog=instance)
        years = Year.objects.filter(name__gte=str(instance.since), report__in=reports)
        context['years'] = years
        context['year'] = year
        return context
11

Your current code isn’t doing anything because you’re creating a variable named instance in your get_context_data method and not doing anything with it. (You’re not adding it to the context.)

If this Blog object is the primary object to be displayed in your view, then this code to do the retrieval of the object for the view belongs in a get_object method, not get_context_data.

12

i did try for days now, but i can’t finger this out.
why English slug works, But the Arabic don’t work. (on deployment server)

but on localhost they both works just fine.

the deployment server error log now is 758.9.kb

class BlogDetailView(DetailView):
    model = Blog
    template_name = 'Blogs/BlogDetails.html'

    def post(self, request, *args, **kwargs):
        year = self.request.POST.get('year_choice', None)
        instance = self.get_object()
        if not year:
            return redirect('Blogs:Blog_detail', slug=instance.slug)
        return redirect('reports:Blog_report', slug=instance.slug, year=year)

    def get_context_data(self, **kwargs):
        context = {}
        instance = Blog.objects.filter(slug=unquote(self.kwargs.get('slug')))[:1].get()
        print(context)
        print('Slug', self.kwargs.get('slug'))
        print('After quote: ', unquote(self.kwargs.get('slug')))
        year = self.kwargs.get('year')
        reports = Report.objects.filter(Blog=instance)
        years = Year.objects.filter(name__gte=str(instance.since), report__in=reports)
        context['instance'] = instance
        context['years'] = years
        context['year'] = year
        print('Year: ', year)
        print('Years:', years)
        return context
13

I would suggest at this point to look at the databases (your development environment and your deployment environment) and verify that all the settings related to encoding schemes and character sets are the same. You can then compare the data in the tables to each other to verify that what’s in the database is the same in both places. (I don’t know enough about MySQL to provide any specifics about what those settings might be.)

1 Like
14

it’s the same identical db.sqlite3 on both servers.

15

That’s not what you’re showing that you’re using here:

16

this is my setting now


# Database
# https://docs.djangoproject.com/en/4.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }

}
17

i may try add
'charset' : 'utf8mb4',

18

And that’s also your settings in your production system?

19

both environments

# Database
# https://docs.djangoproject.com/en/4.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }

}
20

I see in your most recent code posting that you still have this code in get_context_data instead of get_object as recommended previously. Since get_object is called before get_context_data, it may be the method that is throwing the 404, and not your get_context_data.