moby / moby/swarmkit

Ingress Network limited to 128 ports?

Open
#2,958 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.7k
Forks
676
Avg merge
4d 9h
Merged PRs (30d)
6

Description

It seems that there is a limitation either with the ingress network, or the docker_gwbridge network, as its seemingly impossible to expose more than 128 ports externally via ingress on a swarm cluster. It does not matter how many nodes comprise the cluster; this issue occurs on a single node swarm cluster or multi-node.

To see this for yourself, on a empty swarm cluster, simply run:
docker service create -p 80 nginx
129 times..

128 of them will succeed to provision and expose their ports 30000 - 30128, however on reaching the 129th, the task will set pending forever. If i kill one of the previously deployed services, the one that is pending will instantly deploy.

Note that whilst this limitation exists for the ingress network, it does NOT apply to the standard bridge network. On the same cluster, i was able to deploy 300 containers, each exposing a port via the default bridge network.

Is there system tuning that is required to exceed 128 concurrently exposed ingress ports?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the limit on an empty swarm cluster with repeated docker service create -p 80 nginx commands, then trace ingress network and docker_gwbridge port allocation through swarmkit's task scheduling path. Compare the behavior with services using the standard bridge network; done means identifying why the 129th ingress task remains pending and documenting or fixing the limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.