moby / moby/moby

[BUG] ALL containers which use NFS volumes do not start after reboot

Open
#47,153 27 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/systemd area/volumes kind/question status/0-triage version/25.0
Dominant language
Go
Stars
72.1k
Forks
19.2k
Avg merge
1d 17h
Merged PRs (30d)
164

Description

Description

This is the ported Bug-Report from: https://github.com/docker/compose/issues/11354

I am currently facing an issue with my Intel NUC running Debian SID that has persisted for about a year. Despite trying various solutions, I have been unable to resolve it satisfactorily.

My configuration is as follows:

NUC ==(NFS - docker-compose volume)==> SYNO

I run numerous containers within my docker-compose stack, all of which are set to restart with the restart unless-stopped policy. However, upon system reboot, all containers with NFS volumes mapped fail to start automatically. They remain inactive unless manually initiated. Interestingly, initiating a manual start or restart at any other time works seamlessly, and everything functions as expected.

I anticipate that all containers should initiate during a system reboot, and suspect there may be an underlying hidden race condition that eludes my detection.

Also it seems this is the very same issue as the issues mentioned here:

  1. LINK1
  2. LINK2
  3. LINK3
  4. LINK4
Reproduce
  1. use docker-compose (or docker cli as proven in the old Bug-Report https://github.com/docker/compose/issues/11354#issuecomment-1902235822)
  2. configure any container
  3. configure a NFS Volume like this:
volumes:

  share:
    name: share
    driver_opts:
      type: "nfs"
      o: "addr=192.168.178.2,nfsvers=4"
      device: ":/volume1/NFS_SHARE/"
  1. use the named NFS-Volume in the configured container
  2. do the restart test (docker restart container_name) after restart (docker ps)
  3. do the reboot test (reboot) after reboots (docker ps)
Expected behavior

There shall not be any race-condition and all containers (also the ones having NFS-based volumes mounted to) shall restart.

docker version
Client: Docker Engine - Community
 Version:           25.0.0
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        e758fe5
 Built:             Thu Jan 18 17:09:59 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.0
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       615dfdf
  Built:            Thu Jan 18 17:09:59 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.27
  GitCommit:        a1496014c916f9e62104b33d1bb5bd03b0858e59
 runc:
  Version:          1.1.11
  GitCommit:        v1.1.11-0-g4bccb38
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
docker info
Client: Docker Engine - Community
 Version:    25.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 5
 Server Version: 25.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: a1496014c916f9e62104b33d1bb5bd03b0858e59
 runc version: v1.1.11-0-g4bccb38
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.11-amd64
 Operating System: Debian GNU/Linux trixie/sid
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.88GiB
 Name: h0tmann
 ID: 670157fc-30a9-4aa4-807c-4fa28aec7ec7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
Additional Info
  1. LINK1
  2. LINK2
  3. LINK3
  4. LINK4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by reproducing the reboot test with the supplied NFS volume configuration, then compare docker ps after reboot with a manual docker restart; done means containers using NFS volumes restart automatically after reboot without the reported race condition.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.