moby / moby/libnetwork

Unable to route traffic through containers when using overlay networks

Open
#1,348 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
2.2k
Forks
875
PR merge metrics
No merged PRs in 30d

Description

Hi,
Update:
I refined the issue even further. the issue is with containers on the same host. if i setup containers in another host trying to route traffic through containers on the other host it works. Please advise

I used links to run a gateway container that masquerade all traffic to the outside world and the rest of my containers are routing their traffic by setting the gateway container as default route.

Today i tried to migrate to overlay network to allow inter-host traffic routing. Even the containers on the same hosts fail to route the traffic.
just to be clear, When using links, routing traffic works great, when using overlay network even on the same machine routing fail.

Pinging the gateway works and i get ping back, but when trying to route through that container it fails.,If the client-container is running for more than 10 minutes, i can see that packets are starting to work..

If the Gateway server is using the overlay network it can't seem to be successful in routing traffic even if the client containers don't use the overlay network and try to use links instead.

Any idea what can cause this issue? maybe a workaround? Are there any options i can customize the overlay network with ? ( for example the bridge network has an option to allow masquerading, inter-container-connectivity etc )

docker info:
Containers: 24
Running: 23
Paused: 0
Stopped: 1
Images: 247
Server Version: 1.11.2
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 292
Dirperm1 Supported: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host overlay
Kernel Version: 3.13.0-86-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.38 GiB
Name: muse
ID: KFJC:7RRJ:SZAO:VAPZ:4XNF:ORLC:3MJF:B42G:ICO6:N5IR:EJJ3:7QGV
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Cluster store: consul://consoldns:8500
Cluster advertise: serverip:54543

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported setup with Docker 1.11.2 on Ubuntu 14.04.4, comparing linked containers with overlay-network containers on the same host. Check the gateway, default route, and masquerading behavior described in the issue; done means client-container traffic routes successfully through the gateway without the reported delay.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.