docker / docker/docker-py

Connection errors

Open
#2,075 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
1.7k
Avg merge
13d 8h
Merged PRs (30d)
2

Description

I am using docker-py to run a test suite using docker containers. Everything works fine when I run a subset of the tests, but when I try to run the full suite I begin to get ConnectionErrors being thrown by docker-py when trying to communicate to the docker host. They are thrown from various sources, from reading logs to querying exec status. The example below is from querying the status of an exec job.

Is it possible docker-py is not closing socket connections properly and is exhausting the docker host resources? Or is it just not handling the EINPROGRESS socket status properly?

The errors occur in docker-py 3.3 and 3.4.

docker==3.4.0
docker-pycreds==0.3.0
Python 2.7.10
Client:
 Version:      18.05.0-ce
 API version:  1.32 (downgraded from 1.37)
 Go version:   go1.10.2
 Git commit:   f150324
 Built:        unknown-buildtime
 OS/Arch:      darwin/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      17.09.1-ce
  API version:  1.32 (minimum version 1.12)
  Go version:   go1.8.3
  Git commit:   19e2cf6
  Built:        Thu Dec  7 22:28:28 2017
  OS/Arch:      linux/amd64
  Experimental: false
Traceback (most recent call last):
  File "venv/lib/python2.7/site-packages/docker/api/exec_api.py", line 98, in exec_inspect
    res = self._get(self._url("/exec/{0}/json", exec_id))
  File "venv/lib/python2.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "venv/lib/python2.7/site-packages/docker/api/client.py", line 198, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "venv/lib/python2.7/site-packages/requests/sessions.py", line 525, in get
    return self.request('GET', url, **kwargs)
  File "venv/lib/python2.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "venv/lib/python2.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "venv/lib/python2.7/site-packages/requests/adapters.py", line 513, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='192.168.99.100', port=2376): Max retries exceeded with url: /v1.35/exec/19f91b02b6eb5e2a3d0ad82789473da8b186faf1da8dcfa34dbc226b72663df0/json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x10d810710>: Failed to establish a new connection: [Errno 36] Operation now in progress',))

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the full test suite with docker-py 3.3 or 3.4 and review the connection path shown in docker/api/exec_api.py, docker/api/client.py, and requests.sessions. Determine whether repeated operations exhaust connections or mishandle EINPROGRESS, then verify that the recurring ConnectionErrors are resolved with a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.