Docker(swarm) container failing to cummincate for some port in different subnet,but works in same subnet
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 875
- PR merge metrics
- No merged PRs in 30d
Description
While deploying docker etcd container on nodes on the different subnet in different datacentre.getting the below-mentioned error.
2019-02-21 16:52:42.506314 I | raft: b8b747c74aaea686 is starting a new election at term 928
2019-02-21 16:52:42.506344 I | raft: b8b747c74aaea686 became candidate at term 929
2019-02-21 16:52:42.506353 I | raft: b8b747c74aaea686 received MsgVoteResp from b8b747c74aaea686 at term 929
2019-02-21 16:52:42.506361 I | raft: b8b747c74aaea686 [logterm: 1, index: 3] sent MsgVote request to b3504381e8ba3cb at term 929
2019-02-21 16:52:42.506367 I | raft: b8b747c74aaea686 [logterm: 1, index: 3] sent MsgVote request to f572fdfc5cb68406 at term 929
2019-02-21 16:52:43.158372 W | rafthttp: health check for peer b3504381e8ba3cb could not connect: dial tcp 10.0.2.81:2380: i/o timeout
2019-02-21 16:52:43.159658 W | rafthttp: health check for peer f572fdfc5cb68406 could not connect: dial tcp 10.0.2.83:2380: i/o timeout
docker version:
[user-docker@f1cloud2201 ~]$ docker version
Client:
Version: 17.12.0-ce
API version: 1.35
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:10:14 2017
OS/Arch: linux/amd64
Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:12:46 2017
OS/Arch: linux/amd64
Experimental: true
Followed all the pre-requieste of docker swarm port constraints.
telnet and nc also working
etcdc compose file:
version: '3'
services:
etcd01:
image: quay.io/coreos/etcd
ports:
- 2379
- 4001
- 2380
networks:
- dbs1
volumes:
- etcd01:/etcd_data
deploy:
placement:
constraints:
- node.labels.type20 == etcd01
replicas: 1
command:
- /usr/local/bin/etcd
- -name
- etcd01
- --data-dir
- /etcd_data
- -advertise-client-urls
- http://etcd01:2379,http://etcd01:4001,http://127.0.0.1:2379
- -listen-client-urls
- http://0.0.0.0:2379,http://0.0.0.0:4001
- -initial-advertise-peer-urls
- http://etcd01:2380
- -listen-peer-urls
- http://0.0.0.0:2380
- -initial-cluster
- etcd01=http://etcd01:2380,etcd02=http://etcd02:2380,etcd03=http://etcd03:2380
etcd02:
image: quay.io/coreos/etcd
ports:
- 2379
- 4001
- 2380
volumes:
- etcd02:/etcd_data
networks:
- dbs1
deploy:
placement:
constraints:
- node.labels.type20 == etcd02
replicas: 1
command:
- /usr/local/bin/etcd
- -name
- etcd02
- --data-dir
- /etcd_data
- -advertise-client-urls
- http://etcd02:2379,http://etcd02:4001,http://127.0.0.1:2379
- -listen-client-urls
- http://0.0.0.0:2379,http://0.0.0.0:4001
- -initial-advertise-peer-urls
- http://etcd02:2380
- -listen-peer-urls
- http://0.0.0.0:2380
- -initial-cluster
- etcd01=http://etcd01:2380,etcd02=http://etcd02:2380,etcd03=http://etcd03:2380
etcd03:
image: quay.io/coreos/etcd
ports:
- 2379
- 4001
- 2380
volumes:
- etcd03:/etcd_data
networks:
- dbs1
deploy:
placement:
constraints:
- node.labels.type20 == etcd03
replicas: 1
command:
- /usr/local/bin/etcd
- -name
- etcd03
- --data-dir
- /etcd_data
- -advertise-client-urls
- http://etcd03:2379,http://etcd03:4001,http://127.0.0.1:2379
- -listen-client-urls
- http://0.0.0.0:2379,http://0.0.0.0:4001
- -initial-advertise-peer-urls
- http://etcd03:2380
- -listen-peer-urls
- http://0.0.0.0:2380
- -initial-cluster
- etcd01=http://etcd01:2380,etcd02=http://etcd02:2380,etcd03=http://etcd03:2380
volumes:
etcd01:
etcd02:
etcd03:
networks:
dbs1:
external: true
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided etcd Compose configuration and the Docker Swarm overlay network, focusing on peer traffic to port 2380 between nodes in different subnets. Reproduce the timeout and compare it with the stated telnet and nc results; done should mean identifying a confirmed networking cause and documenting a reproducible resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100