docker / docker/docker-py

containers.run() hangs without output when image is missing (+/- invalid mount)

Open
#2,257 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
1.7k
Avg merge
13d 8h
Merged PRs (30d)
2

Description

Hi all,

First of all thanks for the continued work on this SDK -- it's been very useful for making quick work of some local e2e tests. I ran into this problem while a colleague was trying to run some tests that "ran on my machine".

Issue

TL;DR - If you do not have the image you want to run pulled (for example minio/minio), and you have a configuration error (such as an invalid mount specified), the containers.run command will hang (due to one or more silent failures) -- the container will never be pulled, the program will not continue past or error, and the underlying mount problem won't cause an error either -- to the end user it just looks like a hang with no output.

The first progress we were able to make was installing the minio/minio image with the docker CLI directly -- this got rid of the missing image problem (maybe docker-py doesn't fetch missing automatically, I thought this was docker run standard behavior), and exposed an invalid mount configuration issue. The mount configuration issue has to do with how OSX temp directories are created (once the error was visible it was easy to fix):

E       docker.errors.APIError: 502 Server Error: Bad Gateway ("b'Mounts denied: \r\nThe path /var/folders/wj/x6b1l_7s1yg977rxrrjpsc8h0000gn/T/project-9dk0ewdx\r\nis not shared from OS X and is not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'")

this issue is due to namespacing on OSX, and the interaction with python's tempfile functionality:

import tempfile

# Create tmp local directory for minio to use                                                                                                                                                                                                                                 
d = tempfile.mkdtemp(prefix="e2e-minio") # /var/folders/wj/x6b1l_7s1yg977rxrrjpsc8h0000gn/T/project-minio9dk0ewdx

The fact that this folder can't be used by the spawned docker container is more of a gotcha more than a bug (maybe both of these issues are), so I'm not sure exactly what should be done (maybe it's just adding more documentation).

The main issue is the missing container appearing as a hang (which might have been exacerbated by the invalid mount issue), with no output (error or otherwise) returned to the user.

Reproduction

Steps:

  1. Install docker (native) 18.02 (latest as of 02/16) on OSX
  2. Create a python project that uses tmpfile.mkdtemp
  3. Create a client with docker.from_env()
  4. Run docker_client.containers.run w/ a mounted directory that is the temp directory

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported containers.run reproduction using a missing minio/minio image and an invalid mounted temporary directory on macOS. Compare the behavior when the image is absent and when Docker rejects the mount. Done means the command does not silently hang and the user receives useful progress or error output for these failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.