panic in findLBEndpointSandbox
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 875
- PR merge metrics
- No merged PRs in 30d
Description
Found this panic in the journalctl logs while investigating other container restarts. (During restart some running containers were terminated, which was why I was investigating).
Nov 08 08:34:25 host-000009 dockerd[11306]: time="2018-11-08T08:34:25.766619649Z" level=error msg="fatal task error" error="task: non-zero exit (1)" module=node/agent/taskmanager node.id=uh2gr37arkq5hxc6aqge2lf9l service.id=i5zpbufandth734h0krljnf6g task.id=5oz8aynoku5n0poe2s6454p8v
Nov 08 08:34:35 host-000009 dockerd[11306]: panic: runtime error: invalid memory address or nil pointer dereference
Nov 08 08:34:35 host-000009 dockerd[11306]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x557e4cd7ce40]
Nov 08 08:34:35 host-000009 dockerd[11306]: goroutine 623 [running]:
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*endpoint).Name(0x0, 0x0, 0x0)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/endpoint.go:286 +0x30
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*network).findLBEndpointSandbox(0xc420cfa000, 0x0, 0xc42084e000, 0x30, 0x7f16ab6cf6c8)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/service_linux.go:72 +0x16b
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*network).rmLBBackend(0xc420cfa000, 0xc42219cf80, 0x10, 0x10, 0xc421fe8dc0, 0x101)Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/service_linux.go:179 +0x5a
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).rmServiceBinding(0xc420194100, 0xc421f24920, 0x20, 0xc421f24940, 0x19, 0xc420d64ba0, 0x19, 0xc4216f5780, 0x40, 0xc420c40200, ...)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/service_common.go:377 +0xc2a
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).handleEpTableEvent(0xc420194100, 0x557e4ddec860, 0xc4231a2230)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/agent.go:951 +0x1398
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).(github.com/docker/docker/vendor/github.com/docker/libnetwork.handleEpTableEvent)-fm(0x557e4ddec860, 0xc4231a2230)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/agent.go:320 +0x40
Nov 08 08:34:35 host-000009 dockerd[11306]: github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).handleTableEvents(0xc420194100, 0xc421c71fe0, 0xc4202d4830)
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/agent.go:818 +0x53
Nov 08 08:34:35 host-000009 dockerd[11306]: created by github.com/docker/docker/vendor/github.com/docker/libnetwork.(*controller).agentInit
Nov 08 08:34:35 host-000009 dockerd[11306]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/agent.go:320 +0x524
Nov 08 08:34:35 host-000009 systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Nov 08 08:34:35 host-000009 systemd[1]: docker.service: Unit entered failed state.
Nov 08 08:34:35 host-000009 systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 08 08:34:35 host-000009 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Nov 08 08:34:35 host-000009 systemd[1]: Stopped Docker Application Container Engine.
Nov 08 08:34:35 host-000009 systemd[1]: Starting Docker Application Container Engine...
Docker Version 18.06.1:
$ docker version
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:24:56 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:23:21 2018
OS/Arch: linux/amd64
Experimental: false
From docker info:
Kernel Version: 4.15.0-1025-azure
Operating System: Ubuntu 16.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.67GiB
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with vendor/github.com/docker/libnetwork/service_linux.go at findLBEndpointSandbox and inspect vendor/github.com/docker/libnetwork/endpoint.go around line 286, using the supplied stack trace as the entry point. Trace how the nil endpoint reaches Name, then verify with relevant libnetwork tests that the reported restart path no longer panics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go, linux, ubuntu
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100