call to `render` throws TypeError "expected str, bytes or os.PathLike object, not PosixPath"

Using Django Mystery Errors
1

Line 148 of this views.py file (permalink) is throwing the error:

TypeError at /run_make/manual_ingest
expected str, bytes or os.PathLike object, not PosixPath

Here are the project’s settings.py (permalink).

The code used to work. I’ve read online that the problem results from a breaking change in newer versions of Django. Those sources suggested that the problem was probably that I was setting some db path to be posix, and I should cast those paths to strings. I tried wrapping everything that seemed reasonable in the str() function – every path defined in settings.py, and the second argument to render() – to no avail.

Here are the later details of the error message:

Request Method:     GET
Request URL:        http://localhost:8000/microsim/run_make/manual_ingest
Django Version:     4.1
Exception Type:     TypeError
Exception Value:    expected str, bytes or os.PathLike object, not PosixPath
Exception Location: /usr/lib/python3.9/posixpath.py, line 76, in join
Raised during:      run_make.views.views.manual_ingest
Python Executable:  /usr/bin/python3
Python Version:     3.9.2
Server time:        Thu, 30 Mar 2023 16:49:49 -0500
Python Path:        ['/mnt/django',
                     '/mnt/apache2',
                     '/opt/conda/lib/python3.9/site-packages',
                     '/usr/lib/python39.zip',
                     '/usr/lib/python3.9',
                     '/usr/lib/python3.9/lib-dynload',
                     '/usr/local/lib/python3.9/dist-packages',
                     '/usr/lib/python3/dist-packages']
2

Please include the full traceback for the error.

3

So the root issue here is that somewhere along the line, you’re passing a path created with the os.path functions as a path to a Django function.

This problem actually starts from tax.co.web/settings.py at e889902be700c0c928eb4a1b3a669a8de616843d · ofiscal/tax.co.web · GitHub

While this specifically isn’t a problem here, it becomes a potential issue for other functions accessing that setting.

You’ll really want to clean up this entire project - I’m noticing many calls to os.path, that for consistency should all be replaced.

4

Yessir, here’s the full traceback:

Traceback Switch to copy-and-paste view
/opt/conda/lib/python3.9/site-packages/django/core/handlers/exception.py, line 55, in inner
                response = get_response(request) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/core/handlers/base.py, line 197, in _get_response
                response = wrapped_callback(request, *callback_args, **callback_kwargs) …
Local vars
/mnt/django/run_make/views/views.py, line 148, in manual_ingest
    return render ( …
Local vars
/opt/conda/lib/python3.9/site-packages/django/shortcuts.py, line 24, in render
    content = loader.render_to_string(template_name, context, request, using=using) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loader.py, line 61, in render_to_string
        template = get_template(template_name, using=using) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loader.py, line 15, in get_template
            return engine.get_template(template_name) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/backends/django.py, line 34, in get_template
            return Template(self.engine.get_template(template_name), self) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/engine.py, line 175, in get_template
        template, origin = self.find_template(template_name) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/engine.py, line 157, in find_template
                template = loader.get_template(name, skip=skip) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loaders/cached.py, line 57, in get_template
            template = super().get_template(template_name, skip) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loaders/base.py, line 17, in get_template
        for origin in self.get_template_sources(template_name): …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loaders/cached.py, line 70, in get_template_sources
            yield from loader.get_template_sources(template_name) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/template/loaders/filesystem.py, line 35, in get_template_sources
                name = safe_join(template_dir, template_name) …
Local vars
/opt/conda/lib/python3.9/site-packages/django/utils/_os.py, line 17, in safe_join
    final_path = abspath(join(base, *paths)) …
Local vars
/usr/lib/python3.9/posixpath.py, line 76, in join
    a = os.fspath(a) …
Local vars
5

Thanks, Ken! So … what should they be replaced with? I see the “django.urls.path” function (or is it a class, and path.call is the function?), which sounds like a reasonable candidate, but it requires a “view” argument, but how could I define something like settings.BASE_DIR around any particular view when every view will need it?

I also see os.path.dirname, which unless I’m wrong is used by Django’s template settings.py file. But os.path and os.path.join both return strings, so I don’t see why that would behave any differently.

If I don’t want my paths to be represented as strings, what do I want them to be?

6

The os.path might return a string, but it is a “PosixPath” object, and that’s the root cause of the error.

The error message is telling you basically what you need to do. You need to convert your code to use the pathlib module for all work involving file paths when that work intersects with Django.

7

Did the Django spec change so that pathlib objects are what you need now, but strings used to work? If so, can you recommend a project online compatible with the latest Django specs that I can ape?

Why I ask (perhaps not interesting)

The example projects I’m finding online look similar to mine. In particular, they include the same settings.py boilerplate, which defines BASE_DIR as a simple string:

BASE_DIR = os . path . dirname (
    os . path . dirname (
        os . path . abspath (  __file__ ) ) )

I tried replacing what seem like all relevant uses of os.path with pathlib equivalents but I still get the error “expected str, bytes or os.PathLike object, not PosixPath”.

8

Strings still work. But os.path isn’t a string.
Again, from the error:

Exception Value: expected str, bytes or os.PathLike object, not PosixPath

If you really want to track this down, you could run this in a debugger to figure out exactly which object is being referenced here causing the error.

What complicates this is that you don’t have anything more precise from the traceback than the error is coming from this code:

   return render (
      request,
      "run_make/manual_tax_tables.html",
      { "advanced_specs_form" : TaxConfigForm (),
        "income_taxes" : marginal_rate_floor_taxes,
        "vat_rate_groups"             : lib . get_VAT_rate_groups (),
        "consumable_groups_by_coicop" : lib . get_consumable_groups_by_coicop (),
        "consumable_groups_other"     : lib . get_consumable_groups_other (),
        "explainiers"                 : lib . get_csv (
          "/mnt/tax_co/config/vat/grouped/dicc_non_coicop.csv" ),
       } )

The problem could be in any of these functions being called in the dict.

It might be useful to call those functions and set variables external to the render call, and then construct the dict from those variables. It might help narrow down where the root issue lies.

9

I’ll try what you’re suggesting, Ken. Thanks!

But I am confused when you say that os.path is a PosixPath even if it returns a string. Python uses eager evaluation, so shouldn’t any call like os.path.join("a","b") return a string like "a/b"?

The only uses of os.path that I see are calls to join, dirname. split and exists. The first three all return strings, and the last one is a boolean – so none should evaluate to a PosixPath:

In [3]: type( os.path.join("") )
Out[3]: str

In [4]: type( os.path.dirname("") )
Out[4]: str

In [5]: type( os.path.split("a/b") )
Out[5]: tuple

In [7]: type( os.path.split("a/b")[1] )
Out[7]: str

In [9]: type(os.path.exists(""))
Out[9]: bool
``
10

I need to look at this from a very pragmatic perspective here, because I don’t have the detailed knowledge of either Django or your application to do anything else.

You’re getting an error message that is clearly telling you that somewhere within that function call, Django is encountering a PosixPath (or perhaps, the posixpath module itself) where that’s not allowed. I can’t, from my own knowledge identify the specific cause of that error.

About your question, if you print(os.path), you’ll see it’s a reference to posixpath. That’s about as far as I can take this without having a more precise identification of the source of the error.