In which file should I write my custom authentication?

Using Django The Admin
1

https://docs.djangoproject.com/en/5.0/topics/auth/customizing/

Do I need to write the authorization customization described in this document in models.py file, admin.py file or in another file?

2

If you’re talking about a custom authentication backend, you can put it in (almost) any file you like - wherever it makes the most sense to you. It can be in its own file or with some other code.

You telling Django where to find it by specifying its location in the AUTHENTICATION_BACKENDS setting.