docker / docker/cli

tcp_fin_timeout is not utilized from host kernel

Aperta
#4,100 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

area/networking kind/bug status/0-triage
Lingua principale
Go
Stelle
6.1k
Fork
2.2k
Merge medio
1g 15h
PR unite (30g)
43

Descrizione

Description

/proc/sys/net/ipv4/tcp_fin_timeout is not utilized when application connections within a docker container are sending a FIN_WAIT2.

The default should be a 60 seconds timeout. The file is completely missing inside a Docker container.

Also setting the timout by hand with sysctl does have no effect and returns a File oder directory not found

In our case where we have a small KVM-Qemu inside a Docker container with a TCP service.
Every client who disconnects will be CLOSED in KVM also on the host systen BUT NOT within Docker.

If we do a netstat -tunp we get a list of all disconnected clients still in FIN_WAIT2 state for example:

snip....
tcp        0      0 192.168.2.6:443         <client_ipv4>:47464      FIN_WAIT2   7/qemu-system-x86_6
tcp        0      0 192.168.2.6:443         <client_ipv4>:37036      FIN_WAIT2   7/qemu-system-x86_6
tcp        0      0 192.168.2.6:443         <client_ipv4>:50331      FIN_WAIT2   7/qemu-system-x86_6
tcp        0      0 192.168.2.6:443         <client_ipv4>:4387      FIN_WAIT2   7/qemu-system-x86_6
tcp        0      0 192.168.2.6:443         <client_ipv4>:27624      FIN_WAIT2   7/qemu-system-x86_6

These entries will never be cleared and at some point the Container crashes due to max open file descriptors.

Reproduce
  1. Create a Docker container with QEMU-KVM
  2. Create a service within the KVM guest (inside of Docker) such as a Webservice or any other service
  3. Forward ports from Host to Docker to KVM (for example: docker run ... -p 8080:80 ...)
  4. Use Userforward of KVM inside docker to redirect port 80 to KVM (for example: -net nic,model=virtio -net user,hostfwd=tcp::80-:80
  5. Connect to Docker by port 8080 and close the connection.
Expected behavior

Connections should be closed or terminated and cleared from Kernel FD table.

docker version
Client: Docker Engine - Community
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:30:24 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.9
  Git commit:       42c8b31
  Built:            Thu Dec 15 22:28:33 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.15
  GitCommit:        5b842e528e99d4d4c1686467debf2bd4b88ecd86
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
  compose: Docker Compose (Docker Inc., v2.14.1)
  scan: Docker Scan (Docker Inc., v0.23.0)

Server:
 Containers: 13
  Running: 11
  Paused: 0
  Stopped: 2
 Images: 637
 Server Version: 20.10.22
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b842e528e99d4d4c1686467debf2bd4b88ecd86
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.80.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 251.6GiB
 Name: <hostname>
 ID: CRUH:W64O:O4N2:UY65:JSPP:DERI:4SVO:MLGK:ZHNA:MWGM:RHQ4:EBFD
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
Additional Info

Also with Docker v23.0.1 on CentOS 7, the result is the same.

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia riproducendo la configurazione QEMU-in-Docker con il port forwarding di docker run e il networking utente di KVM descritti, quindi ispeziona /proc/sys/net/ipv4/tcp_fin_timeout e le voci FIN_WAIT2 riportate da netstat. Confronta il comportamento del container e dell'host e determina se Docker CLI o un altro componente del runtime gestisce l'impostazione mancante; il lavoro è completato quando le connessioni vengono liberate senza esaurire i descrittori di file.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
docker, linux
Ambito
infrastructure, networking
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.