Error when installing requirements for django tests - fatal error: 'libmemcached/memcached.h' file not found #include <libmemcached/memcached.h>

Environment: macOS Catalina 10.15.4
Python Version: 3.8.2 installed through pyenv

TL;DR solution:

  • You need to install libmemcached - brew install libmemcached if you are using homebrew. If not, first install homebrew using instructions on https://brew.sh and then execute brew install libmemcached

Exact error observed and Solution used is as posted on this StackOverflow question. Posting this here for ease of finding for new contributors (like myself :smiley:)

3 Likes

The contributing docs on running the tests ( Unit tests | Django documentation | Django ) say:

If you encounter an error during the installation, your system might be missing a dependency for one or more of the Python packages. Consult the failing package’s documentation or search the Web with the error message that you encounter.

I wonder if we should document these? I guess the problem is there are 17 main dependencies at the moment and that crossed with the various operating systems makes a lot of potential commands to run.

Hey @adamchainz, thanks for sharing context and link to documentation.

I agree, calling out every single command to run to install all the dependencies is probably not a maintainable approach, however I like the idea of just documenting these dependencies in the running-all-the-tests section so new contributors do not find it overwhelming when they see an unknown/unrelated issue pop up and do not feel discouraged in the newly started journey of contributing to Django.

This also aligns with the thought, calling out commands to run might require a lot of regular updates to the documentation since the commands might change or internal dependencies for packages might change as the dependencies themselves evolve however the fact that they are a dependency for Django tests to run, is under Django’s control hence just calling out the names of dependencies won’t be a too big off a task to achieve and maintain.

Would love to hear more thoughts on this.

1 Like

If you can make a documentation patch that’s not too big, I’d like to see it. I know at least the memcached headers thing has been the same for years since memcached is very stable.

I had a similar problem when I was following the PostGIS documentation for Windows. It’d definitely be nice to have a clear idea of what the dependencies for each system are, not the exact commands or the steps to take.

I’ll start working on a documentation patch. Let me know what the exact dependencies for macOS are and I’ll add them.

export LIBMEMCACHED=/usr/local

This one works for me! I am also new to Django open source!

Thanks @adamchainz I searched for libmemcached on Django Trac to ensure no one else is already working or has worked in the past and found this comment by Tim Graham:

I’m not a mac OS user so I can’t say for sure what the issue is here. What does “it always show problems in installing pylibmc and libmemcached” mean? My guess is that the equivalent of libmemcached-dev (Debian package) isn’t installed. (​related StackOverflow). I think it’s not Django’s job to document the requirements of third-party packages. I’d rather enhance ​pylibmc’s install instructions if needed.

This was then again raised in May 2018 where Tim again mentioned that we won’t be documenting this as a dependency.

The issue has been raised before and the consensus is that it’s not Django’s job to document all the possibilities for various operating systems.

Now 2 years after that, I (and seems some others too) observed it again and we discussed through this forum.

I think given the recurrence of this issue (several times over half a decade), we should rethink our stance and call out them at least in the Troubleshooting section of the contributing guide. This only reduces the hurdle for new contributors.

In the spirit of this, I have created a patch here Doc'd troubleshooting steps for failures when test requirements by ProProgrammer ¡ Pull Request #12660 ¡ django/django ¡ GitHub

FYI with homebrew now located in /opt/homebrew I had to let pip know where to find it:

LIBMEMCACHED=/opt/homebrew pip install pylibmc
1 Like