Django 5 watchaman and pywatchman problem with ignore_dirs

Using Django
1

I’ve followed docs django-admin and manage.py | Django documentation | Django to speed up my development environment, I’ve installed facebook watchman in my django docker container as well pywatchman in my requirements.txt.

So everything partially seem to work cause when I startup django I can see message “Watching for file changes with WatchamReloader”.

In my django root folder i have created a watchman config file:
.watchmanconfig

{
"ignore_dirs": [".git", ".idea", "venv", "migrations", "/usr/local/lib/python3.12"]
}

For some reason this config doesn’t have an effect as it still watches for /usr/local/lib/python3.12 … as you can see below:

python manage.py runserver 0.0.0.0:8000

Watchman version (4, 9, 0)
Watching for file changes with WatchmanReloader
Waiting for apps ready_event.
Performing system checks…

Apps ready_event triggered. Sending autoreload_started signal.
Watching dir /usr/local/lib/python3.12/site-packages/django_extensions/templates with glob /.
Watching dir /usr/local/lib/python3.12/site-packages/django_json_widget/templates with glob **/.
Watching dir /usr/local/lib/python3.12/site-packages/debug_toolbar/templates with glob /.
Watching dir /usr/local/lib/python3.12/site-packages/rest_framework/templates with glob **/.
Watching dir /app/apps/accounts/templates with glob /.
Watching dir /usr/local/lib/python3.12/site-packages/django_filters/templates with glob **/.
Watching dir /usr/local/lib/python3.12/site-packages/drf_spectacular/templates with glob /.
Watching dir /app/apps/upload/templates with glob **/.
Watching dir /usr/local/lib/python3.12/site-packages/mptt/templates with glob /.
Watching dir /app/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/rest_framework/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/corsheaders/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/csp/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/django_json_widget/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/django_rest_passwordreset/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/django_filters/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/storages/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/mptt/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/django_extensions/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/taggit/locale with glob **/.mo.
Watching dir /app/apps/common/locale with glob /.mo.
Watching dir /app/apps/authentication/locale with glob **/.mo.
Watching dir /app/apps/accounts/locale with glob /.mo.
Watching dir /app/apps/profiles/locale with glob **/.mo.
Watching dir /app/apps/orders/locale with glob /.mo.
Watching dir /app/apps/payments/locale with glob **/.mo.
Watching dir /app/apps/inventory/locale with glob /.mo.
Watching dir /app/apps/subscriptions/locale with glob **/.mo.
Watching dir /app/apps/asd/locale with glob /.mo.
Watching dir /app/apps/upload/locale with glob **/.mo.
Watching dir /app/apps/posts/locale with glob /.mo.
Watching dir /app/apps/contact/locale with glob **/.mo.
Watching dir /app/apps/comments/locale with glob /.mo.
Watching dir /usr/local/lib/python3.12/site-packages/debug_toolbar/locale with glob **/.mo.
Watching dir /usr/local/lib/python3.12/site-packages/drf_spectacular/locale with glob /.mo.
Watching dir /app/core/projects/asd/locale with glob **/.mo.
System check identified no issues (0 silenced).
(0.006)
SELECT
c.relname,
CASE
WHEN c.relispartition THEN ‘p’
WHEN c.relkind IN (‘m’, ‘v’) THEN ‘v’
ELSE ‘t’
END,
obj_description(c.oid, ‘pg_class’)
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN (‘f’, ‘m’, ‘p’, ‘r’, ‘v’)
AND n.nspname NOT IN (‘pg_catalog’, ‘pg_toast’)
AND pg_catalog.pg_table_is_visible(c.oid)
; args=None; alias=default
(0.005) SELECT “django_migrations”.“id”, “django_migrations”.“app”, “django_migrations”.“name”, “django_migrations”.“applied” FROM “django_migrations”; args=(); alias=default
Watching 2019 files
Found common roots: (PosixPath(‘/usr/local/lib/python3.12’), PosixPath(‘/app’))
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_extensions/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/django_extensions/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_extensions/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_json_widget/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/django_json_widget/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_json_widget/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/debug_toolbar/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/debug_toolbar/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/debug_toolbar/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/rest_framework/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/rest_framework/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/rest_framework/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘apps/accounts/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Issuing watchman subscription glob:/app/apps/accounts/templates, for root /app. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:1:2’, ‘dedup_results’: True, ‘relative_root’: ‘apps/accounts/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_filters/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/django_filters/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_filters/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/drf_spectacular/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/drf_spectacular/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/drf_spectacular/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘apps/upload/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Issuing watchman subscription glob:/app/apps/upload/templates, for root /app. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:1:2’, ‘dedup_results’: True, ‘relative_root’: ‘apps/upload/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/mptt/templates’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/mptt/templates, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*', ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/mptt/templates’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Issuing watchman subscription glob-parent-locale:/app, for root /app. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:1:2’, ‘dedup_results’: True}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/rest_framework/locale’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/rest_framework/locale, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/rest_framework/locale’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/corsheaders’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob-parent-locale:/usr/local/lib/python3.12/site-packages/corsheaders, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/corsheaders’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/csp’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob-parent-locale:/usr/local/lib/python3.12/site-packages/csp, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/csp’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_json_widget’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob-parent-locale:/usr/local/lib/python3.12/site-packages/django_json_widget, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_json_widget’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_rest_passwordreset’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob-parent-locale:/usr/local/lib/python3.12/site-packages/django_rest_passwordreset, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_rest_passwordreset’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_filters/locale’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/django_filters/locale, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_filters/locale’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/storages’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob-parent-locale:/usr/local/lib/python3.12/site-packages/storages, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/storages’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/mptt/locale’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/mptt/locale, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '/.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/mptt/locale’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/django_extensions/locale’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/django_extensions/locale, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, '**/.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/django_extensions/locale’}

You have 1 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): django_rest_passwordreset.
Run ‘python manage.py migrate’ to apply them.
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘site-packages/taggit/locale’, ‘version’: ‘4.9.0’, ‘watch’: ‘/usr/local/lib/python3.12’}
Issuing watchman subscription glob:/usr/local/lib/python3.12/site-packages/taggit/locale, for root /usr/local/lib/python3.12. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, ‘/*.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:2:2’, ‘dedup_results’: True, ‘relative_root’: ‘site-packages/taggit/locale’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘apps/common’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Issuing watchman subscription glob-parent-locale:/app/apps/common, for root /app. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale//.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:1:2’, ‘dedup_results’: True, ‘relative_root’: ‘apps/common’}
Watchman watch-project result: {‘watcher’: ‘inotify’, ‘relative_path’: ‘apps/authentication’, ‘version’: ‘4.9.0’, ‘watch’: ‘/app’}
Issuing watchman subscription glob-parent-locale:/app/apps/authentication, for root /app. Query: {‘expression’: [‘allof’, [‘anyof’, [‘type’, ‘f’], [‘type’, ‘l’]], [‘anyof’, [‘match’, 'locale/**/.mo’, ‘wholename’]]], ‘fields’: [‘name’], ‘since’: ‘c:1717869585:62:1:2’, ‘dedup_results’: True, ‘relative_root’: ‘apps/authentication’}

I’ve even tried to remove things directly from inside of the container but for some reason django still watches all these files:

watchman watch-del-all
watchman watch-project .
watchman watch-del /usr/local/lib/python3.12watchman watch-del-all

What am I doing wrong here ?

2

Bump. Having the same issue and it’s pretty annoying as the website casually refreshes for no good reason.

I’ve also tried setting a global watchman config /etc/watchman.json with no luck. As soon as runserver runs, the watch becomes:

{
    "version": "20240414.112832.0",
    "roots": [
        "/usr/lib/python3.12",
        "/home/user/project/path"
    ]
}
3

Found a solution, but it’s not very clean, and requires root access.

As the watchman docs say:

Watchman looks for configuration files in two places:

  • The global configuration file /etc/watchman.json
  • The root specific configuration file .watchmanconfig

This is important, because /usr/lib/python3.12 is a root, and the config file needs to exists there. You can see that with watchman watch-list, while runserver is running.

So, I created a .watchmanconfig in /usr/lib/python3.12. At first, I added site-packages in ignore_dirs, then more dirs kept triggering watchman. So I just added all dirs in /usr/lib/python3.12. But still… watchman triggered again by files in python3.12/. I couldn’t bother spending more time, so I just added everything with the following:

#!/bin/bash

list=$(find /usr/lib/python3.12 -maxdepth 1 -printf '\t\t"%f",\n')

cat <<EOF | sudo tee /usr/lib/python3.12/.watchmanconfig
{
	"ignore_dirs": [
${list::-1}
	]
}
EOF

Despite the option being called ignore_dirs, it works for files too.

After you create the config file, you have to stop runserver and run watchman watch-del-all.

UPDATE: Unfortunately, this doesn’t do much. It fixes watchman unwanted triggers, but causes infinite page reloading in combination with django-browser-reload… not sure why or how.