CLI fails to deploy valid stacks due to cross-platform network mixup
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 2.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Description
In some cases trying to deploy a valid stack file to a Linux swarm fails, but only when it is being deployed from Windows. The Docker CLI code in question appears to have confused the set of built-in (i.e. not user-defined) networks for the target with those for the host platform.
Steps to reproduce the issue:
- Have a docker swarm running on Linux.
- Create a compose file as follows.
- Try deploying the compose file from Linux, with a command line like
docker swarm deploy -c test.yaml test. (This works just fine). - On Windows, set up a context targeting the same Linux swarm.
- Try to deploy the same stack on windows targeting the same linux swarm with a command line like
docker -c swarm stack deploy -c test.yml test.
Describe the results you received:
The Linux-based deploy works.
The Windows-based deployment fails with the following message:
network "bridge" is declared as external, but it is not in the right scope: "local" instead of "swarm"
Describe the results you expected:
I would expect that deploying the exact same compose stack to the same swarm cluster would work the same
from both windows and Linux (expect possibly with respect to obvious things like any use of windows style paths, files having windows line endings, etc).
Presumptive cause
The code responsible for this error message attempts to check if the specified network is a user network, or
an always present system network. It uses the following code to do so:
Unfortunately, container.NetworkMode(networkName).IsUserDefined() seems returns information that would apply to a build of dockerd that for the same platform as the cli was built for. Thus for Windows a CLI build it only considers as built-in (i.e. non-user defined) networks those that are built in a Windows build of dockerd (for running Windows containers). That has a different set of built-in networks than Linux builds of dockerd. This even includes the WSL2 mode of Docker Desktop for Windows, which uses the normal Linux names for its built-in networks. For example windows builds of the CLI consider nat to be non-user defined, but considers host and bridge to be used defined, while Linux builds of the cli are the opposite.
I expect this issue may impact all 4 places in the cli code where container.NetworkMode(...).IsUserDefined() is used, but I've not looked into the other 3 cases that closely.
Additional information you deem important (e.g. issue happens only occasionally):
I am using the docker desktop build of the cli, (which is wrapped in compose-cli), but I'm fairly certain a clean windows build of this repo would exhibit the same behavior (I don't have a windows go environment set up to test building). I am using an ssh context, but I suspect a TCP based context for the swarm would behave the same.
The version and info outputs below are from windows while targeting the swarm context.
Output of docker version:
Client: Docker Engine - Community
Cloud integration: 1.0.1
Version: 19.03.13
API version: 1.40
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:00:27 2020
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.13
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:01:06 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.7
GitCommit: 8fba4e9a7d01810a393d5d25a3621dc101981175
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
Output of docker info:
docker info output
``` Client: Debug Mode: false Plugins: scan: Docker Scan (Docker Inc., v0.3.4)Server:
Containers: 6
Running: 5
Paused: 0
Stopped: 1
Images: 5
Server Version: 19.03.13
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: [REDACTED]
Is Manager: true
ClusterID: [REDACTED]
Managers: 3
Nodes: 3
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: [REDACTED]
Manager Addresses:
[REDACTED]:2377
[REDACTED]:2377
[REDACTED]:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-122-generic
Operating System: Ubuntu 18.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 15.67GiB
Name: caslinkdocker01
ID: [redacted]
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
</details>
**Additional environment details (AWS, VirtualBox, physical, etc.):**
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in cli/command/stack/swarm/deploy_composefile.go at the linked network-scope check, then inspect the other three uses of container.NetworkMode(...).IsUserDefined() mentioned in the issue. Reproduce deployment from a Windows CLI to a Linux swarm and verify that the same valid stack deploys successfully from both platforms without the incorrect external-network scope error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- cli, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100