moby / moby/moby

Docker Engine does not start on Ubuntu 22.04 due to iptables issues

Open
#47,998 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/0-triage
Dominant language
Go
Stars
72.1k
Forks
19.2k
Avg merge
1d 18h
Merged PRs (30d)
164

Description

Description

Some time ago I updated, I did a usual system update and after that, docker refuses to start.

For some reason, It can no longer recognize an existing bridge

Reproduce

No longer recalls the process for when it started, beside the update process.

See log output of dockerd -D below.

Expected behavior

Docker can recognize it's own network bridge

docker version
lient: Docker Engine - Community
Version:           26.1.4
API version:       1.45
Go version:        go1.21.11
Git commit:        5650f9b
Built:             Wed Jun  5 11:28:57 2024
OS/Arch:           linux/amd64
Context:           default
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
docker info
spirillen@ns3109441:/opt/youtrack$ sudo docker info
Client: Docker Engine - Community
Version:    26.1.4
Context:    default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version:  v0.14.1
Path:     /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version:  v2.27.1
Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
Additional Info
sudo dockerd -D
INFO[2024-06-15T23:57:34.767730977Z] Starting up                                  
DEBU[2024-06-15T23:57:34.768194411Z] Listener created for HTTP on unix (/var/run/docker.sock) 
INFO[2024-06-15T23:57:34.768542499Z] detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf 
DEBU[2024-06-15T23:57:34.776403422Z] Golang's threads limit set to 227070         
DEBU[2024-06-15T23:57:34.776755420Z] metrics API listening on /var/run/docker/metrics.sock 
DEBU[2024-06-15T23:57:34.778426179Z] Using default logging driver json-file       
DEBU[2024-06-15T23:57:34.778552109Z] No quota support for local volumes in /var/lib/docker/volumes: Filesystem does not support, or has not enabled quotas 
DEBU[2024-06-15T23:57:34.778591605Z] processing event stream                       module=libcontainerd namespace=plugins.moby
DEBU[2024-06-15T23:57:34.779130321Z] [graphdriver] priority list: [overlay2 fuse-overlayfs btrfs zfs vfs] 
DEBU[2024-06-15T23:57:34.780850286Z] successfully detected metacopy status         storage-driver=overlay2 usingMetacopy=false
DEBU[2024-06-15T23:57:34.781471362Z] backingFs=extfs, projectQuotaSupported=false, usingMetacopy=false, indexOff="index=off,", userxattr=""  storage-driver=overlay2
INFO[2024-06-15T23:57:34.781482190Z] [graphdriver] using prior storage driver: overlay2 
DEBU[2024-06-15T23:57:34.781505658Z] Initialized graph driver overlay2            
DEBU[2024-06-15T23:57:34.783086519Z] Max Concurrent Downloads: 3                  
DEBU[2024-06-15T23:57:34.783095638Z] Max Concurrent Uploads: 5                    
DEBU[2024-06-15T23:57:34.783101850Z] Max Download Attempts: 5                     
INFO[2024-06-15T23:57:34.783141148Z] Loading containers: start.                   
DEBU[2024-06-15T23:57:34.783256993Z] processing event stream                       module=libcontainerd namespace=moby
DEBU[2024-06-15T23:57:34.784104380Z] loaded container                              container=1662630e8421b54e88b2ef7f9b98bf3f2d150df51541087f66e278a236528c36 paused=false running=false
DEBU[2024-06-15T23:57:34.784251840Z] loaded container                              container=e33f2d7ab6a0abaa0cb54864d1a12a342abdbc722ab23135ff9ddda92ea2caec paused=false running=false
DEBU[2024-06-15T23:57:34.786170635Z] restoring container                           container=e33f2d7ab6a0abaa0cb54864d1a12a342abdbc722ab23135ff9ddda92ea2caec paused=false restarting=false running=false
DEBU[2024-06-15T23:57:34.786231696Z] restoring container                           container=1662630e8421b54e88b2ef7f9b98bf3f2d150df51541087f66e278a236528c36 paused=false restarting=false running=false
DEBU[2024-06-15T23:57:34.786566903Z] done restoring container                      container=1662630e8421b54e88b2ef7f9b98bf3f2d150df51541087f66e278a236528c36 paused=false restarting=false running=false
DEBU[2024-06-15T23:57:34.786571549Z] done restoring container                      container=e33f2d7ab6a0abaa0cb54864d1a12a342abdbc722ab23135ff9ddda92ea2caec paused=false restarting=false running=false
DEBU[2024-06-15T23:57:34.786618014Z] Option DefaultDriver: bridge                 
DEBU[2024-06-15T23:57:34.786628015Z] Option DefaultNetwork: bridge                
DEBU[2024-06-15T23:57:34.786634061Z] Network Control Plane MTU: 1500              
DEBU[2024-06-15T23:57:34.790506740Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-ISOLATION] 
DEBU[2024-06-15T23:57:34.791482356Z] /usr/sbin/iptables, [--wait -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-06-15T23:57:34.792590639Z] /usr/sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER] 
DEBU[2024-06-15T23:57:34.860065928Z] /usr/sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-06-15T23:57:34.863664393Z] /usr/sbin/iptables, [--wait -t nat -D PREROUTING] 
DEBU[2024-06-15T23:57:34.866563470Z] /usr/sbin/iptables, [--wait -t nat -D OUTPUT] 
DEBU[2024-06-15T23:57:34.869517897Z] /usr/sbin/iptables, [--wait -t nat -F DOCKER] 
DEBU[2024-06-15T23:57:34.872344247Z] /usr/sbin/iptables, [--wait -t nat -X DOCKER] 
DEBU[2024-06-15T23:57:34.927978576Z] /usr/sbin/iptables, [--wait -t filter -F DOCKER] 
DEBU[2024-06-15T23:57:34.931083160Z] /usr/sbin/iptables, [--wait -t filter -X DOCKER] 
DEBU[2024-06-15T23:57:34.972093544Z] /usr/sbin/iptables, [--wait -t filter -F DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:34.975175666Z] /usr/sbin/iptables, [--wait -t filter -X DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:35.000584908Z] /usr/sbin/iptables, [--wait -t filter -F DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.003653833Z] /usr/sbin/iptables, [--wait -t filter -X DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.048085258Z] /usr/sbin/iptables, [--wait -t filter -F DOCKER-ISOLATION] 
DEBU[2024-06-15T23:57:35.051367725Z] /usr/sbin/iptables, [--wait -t filter -X DOCKER-ISOLATION] 
DEBU[2024-06-15T23:57:35.054220664Z] /usr/sbin/iptables, [--wait -t nat -n -L DOCKER] 
DEBU[2024-06-15T23:57:35.057336392Z] /usr/sbin/iptables, [--wait -t nat -N DOCKER] 
DEBU[2024-06-15T23:57:35.060270482Z] /usr/sbin/iptables, [--wait -t filter -n -L DOCKER] 
DEBU[2024-06-15T23:57:35.063481984Z] /usr/sbin/iptables, [--wait -t filter -n -L DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:35.066387736Z] /usr/sbin/iptables, [--wait -t filter -n -L DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.069227042Z] /usr/sbin/iptables, [--wait -t filter -N DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.071750338Z] /usr/sbin/iptables, [--wait -t filter -C DOCKER-ISOLATION-STAGE-1 -j RETURN] 
DEBU[2024-06-15T23:57:35.074452054Z] /usr/sbin/iptables, [--wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN] 
DEBU[2024-06-15T23:57:35.077104858Z] /usr/sbin/iptables, [--wait -t filter -C DOCKER-ISOLATION-STAGE-2 -j RETURN] 
DEBU[2024-06-15T23:57:35.079750459Z] /usr/sbin/iptables, [--wait -A DOCKER-ISOLATION-STAGE-2 -j RETURN] 
DEBU[2024-06-15T23:57:35.089751283Z] /usr/sbin/iptables, [--wait -t nat -C POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE] 
DEBU[2024-06-15T23:57:35.092301438Z] /usr/sbin/iptables, [--wait -t nat -C DOCKER -i docker0 -j RETURN] 
DEBU[2024-06-15T23:57:35.094664827Z] /usr/sbin/iptables, [--wait -t nat -I DOCKER -i docker0 -j RETURN] 
DEBU[2024-06-15T23:57:35.096962010Z] /usr/sbin/iptables, [--wait -t nat -C POSTROUTING -m addrtype --src-type LOCAL -o docker0 -j MASQUERADE] 
DEBU[2024-06-15T23:57:35.099585576Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -i docker0 -o docker0 -j DROP] 
DEBU[2024-06-15T23:57:35.102275865Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -i docker0 -o docker0 -j ACCEPT] 
DEBU[2024-06-15T23:57:35.105037766Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -i docker0 ! -o docker0 -j ACCEPT] 
DEBU[2024-06-15T23:57:35.107862423Z] /usr/sbin/iptables, [--wait -t nat -C PREROUTING -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-06-15T23:57:35.110394839Z] /usr/sbin/iptables, [--wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-06-15T23:57:35.112868810Z] /usr/sbin/iptables, [--wait -t nat -C OUTPUT -m addrtype --dst-type LOCAL -j DOCKER ! --dst 127.0.0.0/8] 
DEBU[2024-06-15T23:57:35.115336321Z] /usr/sbin/iptables, [--wait -t nat -A OUTPUT -m addrtype --dst-type LOCAL -j DOCKER ! --dst 127.0.0.0/8] 
DEBU[2024-06-15T23:57:35.117785762Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -o docker0 -j DOCKER] 
DEBU[2024-06-15T23:57:35.120480292Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -o docker0 -j DOCKER] 
DEBU[2024-06-15T23:57:35.123125047Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT] 
DEBU[2024-06-15T23:57:35.125695880Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT] 
DEBU[2024-06-15T23:57:35.128444317Z] /usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:35.130626628Z] /usr/sbin/iptables, [--wait -D FORWARD -j DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:35.132818126Z] /usr/sbin/iptables, [--wait -I FORWARD -j DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-06-15T23:57:35.134864438Z] /usr/sbin/iptables, [--wait -t filter -C DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.137199849Z] /usr/sbin/iptables, [--wait -t filter -I DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-06-15T23:57:35.139340347Z] /usr/sbin/iptables, [--wait -t filter -C DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP] 
DEBU[2024-06-15T23:57:35.141610144Z] /usr/sbin/iptables, [--wait -t filter -I DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP] 
DEBU[2024-06-15T23:57:35.143929117Z] Network (ca495b7) restored                   
INFO[2024-06-15T23:57:35.148785913Z] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address 
DEBU[2024-06-15T23:57:35.148842271Z] Allocating IPv4 pools for network bridge (e3c15033ce78fef18e55e1572fa250de9d5c7d623debd25517210cc9f37dc3cf) 
DEBU[2024-06-15T23:57:35.148858879Z] RequestPool(LocalDefault, 172.17.0.0/16, , _, false)
DEBU[2024-06-15T23:57:35.148902196Z] RequestAddress(LocalDefault/172.17.0.0/16, 172.17.0.1, map[RequestAddressType:com.docker.network.gateway])
DEBU[2024-06-15T23:57:35.148939925Z] Request address PoolID:172.17.0.0/16 Bits: 65536, Unselected: 65534, Sequence: (0x80000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:0 Serial:false PrefAddress:172.17.0.1
DEBU[2024-06-15T23:57:35.149063108Z] releasing IPv4 pools from network bridge (e3c15033ce78fef18e55e1572fa250de9d5c7d623debd25517210cc9f37dc3cf)
DEBU[2024-06-15T23:57:35.149084436Z] ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.1)
DEBU[2024-06-15T23:57:35.149111509Z] Released address Address:172.17.0.1 Sequence:Bits: 65536, Unselected: 65534, Sequence: (0x80000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:0
DEBU[2024-06-15T23:57:35.149129027Z] ReleasePool(LocalDefault/172.17.0.0/16)
DEBU[2024-06-15T23:57:35.149159549Z] daemon configured with a 15 seconds minimum shutdown timeout
DEBU[2024-06-15T23:57:35.149183618Z] start clean shutdown of all containers with a 15 seconds timeout...
DEBU[2024-06-15T23:57:35.150120897Z] Unix socket /var/run/docker/libnetwork/e671e8f88020.sock was closed. The external key listener will stop.
DEBU[2024-06-15T23:57:35.150628199Z] Cleaning up old mountid : start.
INFO[2024-06-15T23:57:35.150685373Z] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
DEBU[2024-06-15T23:57:35.151002424Z] Cleaning up old mountid : done.
failed to start daemon: Error initializing network controller: error creating default "bridge" network: cannot create network e3c15033ce78fef18e55e1572fa250de9d5c7d623debd25517210cc9f37dc3cf (docker0): conflicts with network ca495b73f0d5180761e35cfab4e58571671bf1bb1eb1c66e681b7fbac426d96f (docker0): networks have same bridge name
INFO[2024-06-15T23:57:35.151346244Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided dockerd -D output and investigate Docker Engine startup on Ubuntu 22.04 after a system update. The issue needs a reproducible trigger for the iptables-related bridge failure; done means the daemon starts and recognizes its existing bridge without shutting down.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, linux, ubuntu
Domain
backend, infrastructure, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.