docker / docker/docker-py

Memory and socket leak of docker daemon when using with rootless docker

Open
#3,251 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
1.7k
Avg merge
13d 8h
Merged PRs (30d)
2

Description

When using rootless docker every time a client is created with docker.from_env() a UNIX-STREAM socket belonging to the docker daemon is also created. Over time this can lead to hundreds or thousands of sockets and an increased memory usage of the rootless docker process. This happens even when no containers are running.

I noticed this behaviour on our Debian 12 servers, both with the Debian provided version of this library and the latest version installed with pip in a virtual environment.

client.version() output:

{
   "Platform":{
      "Name":"Docker Engine - Community"
   },
   "Components":[
      {
         "Name":"Engine",
         "Version":"26.0.0",
         "Details":{
            "ApiVersion":"1.45",
            "Arch":"amd64",
            "BuildTime":"2024-03-20T15:18:01.000000000+00:00",
            "Experimental":"false",
            "GitCommit":"8b79278",
            "GoVersion":"go1.21.8",
            "KernelVersion":"6.1.0-18-amd64",
            "MinAPIVersion":"1.24",
            "Os":"linux"
         }
      },
      {
         "Name":"containerd",
         "Version":"1.6.31",
         "Details":{
            "GitCommit":"e377cd56a71523140ca6ae87e30244719194a521"
         }
      },
      {
         "Name":"runc",
         "Version":"1.1.12",
         "Details":{
            "GitCommit":"v1.1.12-0-g51d5e94"
         }
      },
      {
         "Name":"docker-init",
         "Version":"0.19.0",
         "Details":{
            "GitCommit":"de40ad0"
         }
      },
      {
         "Name":"rootlesskit",
         "Version":"2.0.2",
         "Details":{
            "ApiVersion":"1.1.1",
            "NetworkDriver":"slirp4netns",
            "PortDriver":"builtin",
            "StateDir":"/run/user/2001/dockerd-rootless"
         }
      },
      {
         "Name":"slirp4netns",
         "Version":"1.2.0",
         "Details":{
            "GitCommit":"656041d45cfca7a4176f6b7eed9e4fe6c11e8383"
         }
      }
   ],
   "Version":"26.0.0",
   "ApiVersion":"1.45",
   "MinAPIVersion":"1.24",
   "GitCommit":"8b79278",
   "GoVersion":"go1.21.8",
   "Os":"linux",
   "Arch":"amd64",
   "KernelVersion":"6.1.0-18-amd64",
   "BuildTime":"2024-03-20T15:18:01.000000000+00:00"
}

How to check the number of sockets belonging to the docker daemon:

lsof -p [PID of docker daemon] | grep UNIX-STREAM | wc -l

Testscript to produce a large amounts of sockets:

import docker

while True:
    client = docker.from_env()

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the docker.from_env() entry point and reproduce the issue with the provided loop on a rootless Docker setup. Use lsof -p [PID of docker daemon] | grep UNIX-STREAM | wc -l to observe socket counts; done means repeated client creation no longer leaves growing daemon sockets or memory usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
api, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.