UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 79: invalid start byte

Using Django Getting Started
1

Hey there!

I am new to coding and trying to build an app to learn. However I am having this error when ı wish to connect a remote postgres database. I am sure that ı am not using any character that doesnt fit utf-8 but somehow having this error. I check the similar problems here, Stackoverflow, youtube etc. but could’nt make sense. I would much appreciate anyhelp and explanation. I do understand it is a char_set issiue but ı checked a lot if there is a mistake but as far as ı can see no. I even put settings.py to chatgpt if there is a mistake for that regard but nope

Thank you very much!

C:\Users\cetin\OneDrive\Masaüstü\ecom\ecom\ecom\settings.py changed, reloading.
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1520.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1075, in _bootstrap_inner
    self.run()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1520.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\utils\autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\core\management\commands\runserver.py", line 137, in inner_run
    self.check_migrations()
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\core\management\base.py", line 581, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\migrations\executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\migrations\loader.py", line 58, in __init__
    self.build_graph()
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\migrations\loader.py", line 235, in build_graph
    self.applied_migrations = recorder.applied_migrations()
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\migrations\recorder.py", line 89, in applied_migrations
    if self.has_table():
       ^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\migrations\recorder.py", line 63, in has_table
    with self.connection.cursor() as cursor:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\backends\base\base.py", line 320, in cursor
    return self._cursor()
           ^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\backends\base\base.py", line 296, in _cursor
    self.ensure_connection()
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\backends\base\base.py", line 279, in ensure_connection
    self.connect()
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\backends\base\base.py", line 256, in connect
    self.connection = self.get_new_connection(conn_params)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\django\db\backends\postgresql\base.py", line 350, in get_new_connection
    connection = self.Database.connect(**conn_params)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cetin\OneDrive\Masaüstü\ecom\virt\Lib\site-packages\psycopg2\__init__.py", line 122, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 79: invalid start byte

settings.py

DATABASES = {
    'default': {
        #'ENGINE': 'django.db.backends.sqlite3',
        #'NAME': BASE_DIR / 'db.sqlite3',
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'railway',
        'USER': 'postgres',
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': 'postgres.railway.internal',
        'PORT': '5432',
    }
}
2

Wild guess - The byte 0xf6 is usually represented as ö - is that character in your DB_PASSWORD?

3

I am using the password provided by railway(literally copied and pasted it" and nope sir letter “ö” it doesn’t exist in DB_PASSWORD

4

Which command are you running that has produced this traceback?

What do you get as output from a manage.py check command?

Do you have that character anywhere within your settings.py, models.py or apps.py files?

5 
(virt) PS C:\Users\cetin\OneDrive\Masaüstü\ecom\ecom> python manage.py check
System check identified no issues (0 silenced).

manage.py runserver and manage.py migrate
both giving out the same error

7

Do you have that character anywhere within your settings.py, models.py or apps.py files?

Do you have it in any migration files?

Anywhere within the .py files in your project?

Do you have any filenames in your project with that character in the filename?

(I’ve got to believe that character is coming from somewhere within your project…)

8

It is gonna be a good search for me then sir! I will check them and get back. Otherwise ı will push to github and share

9

Here is the update sir! I went through every file that ends with .py and .html but ı could not find the letter ‘‘ö’’ I have no idea where does it refering that letter and why? ı tried my best to check DB_PASSWORD also no whitespace or anything in between but nope still the same. be it in file names or inside the files ı could’nt find! I hope ı am not blind or losing it but yep that is the update.

I add the githup repo for the code here. ı just simply can’t figure it out where, how, why? By all means be my guest! I have been going through again and again!

11

Which “dotenv” library are you using? It’s not listed in your requirements.txt file, and there are multiple packages with similar names.

12

ı am keeping track of requirements.txt myself sir. I am really sorry after all those ı forgot to update that. I apologize
Latest pip freeze outcome is this;

asgiref==3.8.1
Django==5.1
pillow==10.4.0
psycopg2==2.9.9
python-dotenv==1.0.1
sqlparse==0.5.1
typing_extensions==4.12.2
tzdata==2024.1

13

I really appreciate the help by the way sir thank you!

14

Mark the post that provided assistance to you as a solution, in order to help the community :slight_smile:

15

Should ı do it even it is not solved? Because ı couldn’t solve it yet

16

Oh no! You could mark a post as solution if your problem is solved, I guess.

1 Like
17

I have spent almost all day to check for this with many sources but couldn’t figure it out :frowning:

18

With the exception of your .env file, I am able to run your project as checked-in without any issues. I would suggest trying it by removing the load_dotenv() call and replacing the reference to the environment variable for the database password with the password itself.
(This is just a testing / diagnostic step.)

19

Same error arises when ı remove load_dotenv() and add directly PASSWORD.
I just added DB_PASSWORD here as well. I will create a new database ı just don’t want to have this same issiue coming up. I will delete this spesific DB service after a while because ı do not wish to violate railway’s service rules somehow someway.

my .env has only this environment variable(no space no quotation marks" I now go create a new DB and see how that goes. I just copy pased my .env file’s content.

DB_PASSWORD=kUadVrXzdlvEFUhDepPBATMyVZAlfIhi

20

So in other words, you removed the line load_dotenv(), and replaced this:

With this:

        'PASSWORD': 'kUadVrXzdlvEFUhDepPBATMyVZAlfIhi',

And you’re still getting the same error?

21

yes sir exactly. However, ı go back to db which django comes with it just works.

22

I see that you are using Windows. Have you tried using Python’s utf-8 mode?