[Error] Max retries exceeded with url:

Using Django Deployment
1

Hello.

I’m trying to create a form, containing a dropdown which values would issue from an api call.

I have 2 apps :

  • An app to manage Quotes
  • An app to manage the master data from the Authors (I want to expose the data from this app to others apps or services)

On the Author app, I create an API endpoint with the REST framework, and a call to this endpoint (using the “requets” module).

I can get the data with a view and show it in a template.

But…

If I create a form using this api call, used by a view, I get an error when starting django : “Max retries exceeded with url:”

Here is the code :

In Authors app :

serializers.py :

from .models import Author
from rest_framework import serializers


class AuthorListSerializer(serializers.ModelSerializer):
    class Meta:
        model = Author
        fields = ['id', 'name', 'surname']

In Quotes (citatio) app :
forms.py :

from django import forms
from .models import Quote, Author, Source
from MyProject.api_calls import authorlist


class TestForm(forms.Form):
    liste_auteurs = authorlist()
    choix = forms.ChoiceField(choices=liste_auteurs, widget=forms.ChoiceField)

views.py :

from django.shortcuts import render
from django.http import HttpResponseRedirect, HttpResponse, JsonResponse
from django.shortcuts import render, redirect
from django.urls import reverse, reverse_lazy
from django.views import generic, decorators
from django.contrib.auth.models import User, Group
[...]
def test1(request):
    if request.method == 'POST':
        form = TestForm(request.POST)
        if form.is_valid():
            form.save()
            return HttpResponseRedirect(reverse('citatio/quote_list.html'))
    else:
        form = testform()

    return render(request, 'citatio/liste_auteurs.html', {'form': form })

I create a specific file, at the root of the projet, to have all the api calls :

api_calls.py :

import requests

def authorlist():
    response = requests.get('http://0.0.0.0:8000/authors/').json()
    return response

I tried a solution, adding this to api_calls (but it doesn’t work) :

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry


session = requests.Session()
retry = Retry(connect=3, backoff_factor=0.5)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

I’m not sure what’s happening… and can’t find a solution.

Any idea ? Thank you :slight_smile:

2

Please post your urls.py files (both your root file and your urls.py file for your authors app) and the complete traceback message being generated in the console.

3

root urls.py :

urlpatterns = [
    path('citatio/', include('citatio.urls')),
    path('obras/', include('obras.urls')),
    path('authors/', include('authors.urls')),
    path('admin/', admin.site.urls),
    path('accounts/', include('django.contrib.auth.urls')),
    path('api-auth/', include('rest_framework.urls')),
]

authors urls.py :

from django.urls import include, path
from authors import views
from rest_framework import routers

app_name = 'authors'
urlpatterns = [
    path('', views.author_list),
          ]

Console traceback :

jbal@OCTO-JBAL ~/P/MyProject> docker-compose up
Starting myproject_db_1 ... done
Starting myproject_web_1 ... done
Attaching to myproject_db_1, myproject_web_1
db_1   | 
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   | 
db_1   | 2021-08-26 13:00:17.271 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2021-08-26 13:00:17.272 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2021-08-26 13:00:17.272 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2021-08-26 13:00:17.279 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2021-08-26 13:00:17.288 UTC [26] LOG:  database system was shut down at 2021-08-26 13:00:14 UTC
db_1   | 2021-08-26 13:00:17.300 UTC [1] LOG:  database system is ready to accept connections
web_1  | Watching for file changes with StatReloader
web_1  | Performing system checks...
web_1  | 
web_1  | Exception in thread django-main-thread:
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
web_1  |     conn = connection.create_connection(
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
web_1  |     raise err
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
web_1  |     sock.connect(sa)
web_1  | ConnectionRefusedError: [Errno 111] Connection refused
web_1  | 
web_1  | During handling of the above exception, another exception occurred:
web_1  | 
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
web_1  |     httplib_response = self._make_request(
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
web_1  |     conn.request(method, url, **httplib_request_kw)
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
web_1  |     super(HTTPConnection, self).request(method, url, body=body, headers=headers)
web_1  |   File "/usr/local/lib/python3.9/http/client.py", line 1255, in request
web_1  |     self._send_request(method, url, body, headers, encode_chunked)
web_1  |   File "/usr/local/lib/python3.9/http/client.py", line 1301, in _send_request
web_1  |     self.endheaders(body, encode_chunked=encode_chunked)
web_1  |   File "/usr/local/lib/python3.9/http/client.py", line 1250, in endheaders
web_1  |     self._send_output(message_body, encode_chunked=encode_chunked)
web_1  |   File "/usr/local/lib/python3.9/http/client.py", line 1010, in _send_output
web_1  |     self.send(msg)
web_1  |   File "/usr/local/lib/python3.9/http/client.py", line 950, in send
web_1  |     self.connect()
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
web_1  |     conn = self._new_conn()
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
web_1  |     raise NewConnectionError(
web_1  | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd7a51f4730>: Failed to establish a new connection: [Errno 111] Connection refused
web_1  | 
web_1  | During handling of the above exception, another exception occurred:
web_1  | 
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
web_1  |     resp = conn.urlopen(
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
web_1  |     retries = retries.increment(
web_1  |   File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
web_1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
web_1  | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='0.0.0.0', port=8000): Max retries exceeded with url: /authors/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd7a51f4730>: Failed to establish a new connection: [Errno 111] Connection refused'))
web_1  | 
web_1  | During handling of the above exception, another exception occurred:
web_1  | 
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.9/threading.py", line 954, in _bootstrap_inner
web_1  |     self.run()
web_1  |   File "/usr/local/lib/python3.9/threading.py", line 892, in run
web_1  |     self._target(*self._args, **self._kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
web_1  |     fn(*args, **kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 118, in inner_run
web_1  |     self.check(display_num_errors=True)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 419, in check
web_1  |     all_issues = checks.run_checks(
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
web_1  |     new_errors = check(app_configs=app_configs, databases=databases)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/core/checks/urls.py", line 13, in check_url_config
web_1  |     return check_resolver(resolver)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/core/checks/urls.py", line 23, in check_resolver
web_1  |     return check_method()
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/urls/resolvers.py", line 412, in check
web_1  |     for pattern in self.url_patterns:
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
web_1  |     res = instance.__dict__[self.name] = self.func(instance)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/urls/resolvers.py", line 598, in url_patterns
web_1  |     patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
web_1  |     res = instance.__dict__[self.name] = self.func(instance)
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/urls/resolvers.py", line 591, in urlconf_module
web_1  |     return import_module(self.urlconf_name)
web_1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
web_1  |     return _bootstrap._gcd_import(name[level:], package, level)
web_1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
web_1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
web_1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
web_1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
web_1  |   File "<frozen importlib._bootstrap_external>", line 790, in exec_module
web_1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
web_1  |   File "/code/MyProject/urls.py", line 26, in <module>
web_1  |     path('citatio/', include('citatio.urls')),
web_1  |   File "/usr/local/lib/python3.9/site-packages/django/urls/conf.py", line 34, in include
web_1  |     urlconf_module = import_module(urlconf_module)
web_1  |   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
web_1  |     return _bootstrap._gcd_import(name[level:], package, level)
web_1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
web_1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
web_1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
web_1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
web_1  |   File "<frozen importlib._bootstrap_external>", line 790, in exec_module
web_1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
web_1  |   File "/code/citatio/urls.py", line 2, in <module>
web_1  |     from . import views
web_1  |   File "/code/citatio/views.py", line 8, in <module>
web_1  |     from .forms import QuoteForm, DeleteQuotesForm, TestForm
web_1  |   File "/code/citatio/forms.py", line 6, in <module>
web_1  |     class TestForm(forms.Form):
web_1  |   File "/code/citatio/forms.py", line 7, in TestForm
web_1  |     liste_auteurs = authorlist()
web_1  |   File "/code/MyProject/api_calls.py", line 15, in authorlist
web_1  |     response = requests.get('http://0.0.0.0:8000/authors/').json()
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 75, in get
web_1  |     return request('get', url, params=params, **kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
web_1  |     return session.request(method=method, url=url, **kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
web_1  |     resp = self.send(prep, **send_kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
web_1  |     r = adapter.send(request, **kwargs)
web_1  |   File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
web_1  |     raise ConnectionError(e, request=request)
web_1  | requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=8000): Max retries exceeded with url: /authors/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd7a51f4730>: Failed to establish a new connection: [Errno 111] Connection refused'))
4

Oops, I should have noticed this right away. 0.0.0.0 is not a valid address to attempt to make a connection to. You need to specify the address (or hostname) of the host you’re trying to connect to.

5

Well… don’t know what is a valid address or hostname. I tried with localhost and 127.0.0.1 but it still doesn’t work. Maybe it’s not more valid… ?

6

I see where you’re doing this in a docker-compose bundle. See Networking in Compose | Docker Documentation as the best place to start.

7

Hmmm… I think, it shouldn’t work at all if it was a pure network issue.

I tested it using it in directly a view (e.g. not using a form), and it works :

views.py :

def test1(request):
    response = authorlist()
    return render(request, 'citatio/liste_auteurs.html', {'response' : response })

Am I wrong ? :slight_smile:

8

It’s a network issue in so far as you’re trying to issue a network request to an invalid address.

Accessing the view directly involves no network communications at all, since both the caller and called functions exist in the same process.

9

So, I think I understood, that it didn’t work because I try to make a call to an API with a url/IP that is internal to the container : the call tries to reach a local IP (localhost, 0.0.0.0), outside the container (where I should make a call to a regular IP).

To be honest, I’m not clearly sure to understand why the same API works with the internal IP when I call it directly from a view…

Btw, I tried something with the my container configuration, but I’m not sure of me : I created a specific network for the container (specific subnet and gateway). I’ve two image (web and db), in this network, with a specific IP.

When I run docker compose, I can ping these adresses, but there is no open port, so the http service is not reachable. I think I’m missing something (like binding loop/ports to the external adresses, so that the web service is reachable)…

Here is my docker-compose.yml :

version: "3.9"

services:
  db:
    image: 'postgres:13'
    container_name: django_db
    networks:
      front:
        ipv4_address: 172.16.238.11
    ports:
      - '5432'
    volumes:
      - ./data/db:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=rialp
      - POSTGRES_HOST_AUTH_METHOD=trust
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=
  web:
    build: .
    container_name: django_web
    networks:
      front:
        ipv4_address: 172.16.238.10
    command: python manage.py migrate
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - '8000:8000'
    depends_on:
      - db
    links:
      - db:db
networks:
  front:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 172.16.238.0/24
        gateway: 172.16.238.1
10

Localhost is not, repeat, not 0.0.0.0. 0.0.0.0 is not a valid IP address.

When you use 0.0.0.0 in your runserver command, that has special meaning. It means to listen to every IP address bound to your system.

Localhost is 127.0.0.1

I’m having a hard time see where the confusion might be coming from here. When you’re calling a function, you’re not using the API. You’re calling a function. The two methods of invocation are completely different. The API invocation involves creating a network connection to your server, passing a URL which is then mapped to a view.

What do you mean “there is no open port”? You’re running runserver (which I hope is not how you’re planning to run this “for real” - runserver is an extremely bad choice) with port 8000 exposed. (The “ports” directive.)

11

Hi, firts create a network in docker then in you docker-compose file set this external network like this:

version: ‘3’

services:
apidga:
build: .
command: uvicorn app:app --host 0.0.0.0 --reload
volumes:
- .:/api
ports:
- 8000:8000
networks:
- redmapa1

networks:
redmapa1:
external: true

in the second docker-compose file do the same:

version: “3”

services:
mapa1: #imagen llamada web
build: .
command: gunicorn --bind 0.0.0.0:5000 app.init:app
ports:
- “5000:5000”
volumes:
- .:/app
env_file:
- .env
networks:
- redmapa1

networks:
redmapa1:
external: true

to call the firts container from the second set the base url endpoint like this:
.env file
BASE_URL=“http://apidga:8000”

Hope helps you!