tcp_fin_timeout is not utilized from host kernel
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 2.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Description
/proc/sys/net/ipv4/tcp_fin_timeout is not utilized when application connections within a docker container are sending a FIN_WAIT2.
The default should be a 60 seconds timeout. The file is completely missing inside a Docker container.
Also setting the timout by hand with sysctl does have no effect and returns a File oder directory not found
In our case where we have a small KVM-Qemu inside a Docker container with a TCP service.
Every client who disconnects will be CLOSED in KVM also on the host systen BUT NOT within Docker.
If we do a netstat -tunp we get a list of all disconnected clients still in FIN_WAIT2 state for example:
snip....
tcp 0 0 192.168.2.6:443 <client_ipv4>:47464 FIN_WAIT2 7/qemu-system-x86_6
tcp 0 0 192.168.2.6:443 <client_ipv4>:37036 FIN_WAIT2 7/qemu-system-x86_6
tcp 0 0 192.168.2.6:443 <client_ipv4>:50331 FIN_WAIT2 7/qemu-system-x86_6
tcp 0 0 192.168.2.6:443 <client_ipv4>:4387 FIN_WAIT2 7/qemu-system-x86_6
tcp 0 0 192.168.2.6:443 <client_ipv4>:27624 FIN_WAIT2 7/qemu-system-x86_6
These entries will never be cleared and at some point the Container crashes due to max open file descriptors.
Reproduce
- Create a Docker container with QEMU-KVM
- Create a service within the KVM guest (inside of Docker) such as a Webservice or any other service
- Forward ports from Host to Docker to KVM (for example:
docker run ... -p 8080:80 ...) - Use Userforward of KVM inside docker to redirect port 80 to KVM (for example:
-net nic,model=virtio -net user,hostfwd=tcp::80-:80 - Connect to Docker by port 8080 and close the connection.
Expected behavior
Connections should be closed or terminated and cleared from Kernel FD table.
docker version
Client: Docker Engine - Community
Version: 20.10.22
API version: 1.41
Go version: go1.18.9
Git commit: 3a2c30b
Built: Thu Dec 15 22:30:24 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.22
API version: 1.41 (minimum version 1.12)
Go version: go1.18.9
Git commit: 42c8b31
Built: Thu Dec 15 22:28:33 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.15
GitCommit: 5b842e528e99d4d4c1686467debf2bd4b88ecd86
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
compose: Docker Compose (Docker Inc., v2.14.1)
scan: Docker Scan (Docker Inc., v0.23.0)
Server:
Containers: 13
Running: 11
Paused: 0
Stopped: 2
Images: 637
Server Version: 20.10.22
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 5b842e528e99d4d4c1686467debf2bd4b88ecd86
runc version: v1.1.4-0-g5fd4c4d
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-1160.80.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 251.6GiB
Name: <hostname>
ID: CRUH:W64O:O4N2:UY65:JSPP:DERI:4SVO:MLGK:ZHNA:MWGM:RHQ4:EBFD
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional Info
Also with Docker v23.0.1 on CentOS 7, the result is the same.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the QEMU-in-Docker setup with the described docker run port forwarding and KVM user networking, then inspect /proc/sys/net/ipv4/tcp_fin_timeout and the FIN_WAIT2 entries reported by netstat. Compare the container and host behavior and determine whether the Docker CLI or another runtime component owns the missing setting; done means connections are cleared without exhausting file descriptors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, linux
- Domain
- infrastructure, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100