iptables locking issues if --wait is not available
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 875
- PR merge metrics
- No merged PRs in 30d
Description
If an iptables version is used that does not support or expose (in case a wrapper script is used arround iptables) the --wait flag issues occur if multiple containers are started at the same time. As only one iptables instance can run at any time, parallel calls to iptables fail and the rules for the dns resolver are not added to the container (in case a docker network is used).
Symptom:
Containers can not access other containers or the internet due to the DNS resolution failing.
Steps to reproduce:
- Use an iptables wrapper or version without --wait
- Create a docker network
- Add multiple containers to this network that are restarted on system start
- Restart docker or the system to start all containers at once
Cause for this issue:
Docker will detect that the xtables lock (iptables/iptables.go:121) is not supported and use its internal locking mechanism (bestEffortLock) instead (iptables/iptables.go:537).
The issue is that this lock is a mutex and can only provide locking within the same process. This becomes an issue if docker networks are used. In this case a new resolver process for each container is spawned via "reexec" to add the required iptables rules into the container network namespace (resolver_unix.go:29). If multiple containers start in parallel, multiple of these resolver setup processes are spawned which each have their own independent bestEffortLock and thus try to execute iptables in parallel to each other - and fail while attempting to do this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with iptables/iptables.go at the xtables-lock detection and bestEffortLock references, then inspect resolver_unix.go where reexec processes add iptables rules. Reproduce concurrent container startup with an iptables wrapper or version lacking --wait. Done means parallel resolver setup no longer causes iptables calls to fail and the DNS rules are added to each container network.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100