moby / moby/libnetwork

[Bridge] internal network rules drop ipv4 broadcast (255.255.255.255) pakets

Open
#2,260 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
2.2k
Forks
875
No merged PRs in 30d

Description

Description

The FORWARD rules created by a bridged "--internal" network do not allow broadcast pakets to travel from one container to another. It works with non --internal bridged networks. The difference is displayed below.

For a non-internal, working setup the FORWARD rules look like:

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 4215 2796K DOCKER-USER  all  --  any    any     anywhere             anywhere
 4215 2796K DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere
 2320 2535K ACCEPT     all  --  any    br-5c4f0dd6fb2f  anywhere             anywhere             ctstate RELATED,ESTABLISHED
  457 89588 DOCKER     all  --  any    br-5c4f0dd6fb2f  anywhere             anywhere
 1438  172K ACCEPT     all  --  br-5c4f0dd6fb2f !br-5c4f0dd6fb2f  anywhere             anywhere
  457 89588 ACCEPT     all  --  br-5c4f0dd6fb2f br-5c4f0dd6fb2f  anywhere             anywhere
27940   41M ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    docker0  anywhere             anywhere
14354  816K ACCEPT     all  --  docker0 !docker0  anywhere             anywhere
    0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere

Chain OUTPUT (policy ACCEPT 145 packets, 16952 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
 1438  172K DOCKER-ISOLATION-STAGE-2  all  --  br-5c4f0dd6fb2f !br-5c4f0dd6fb2f  anywhere             anywhere
14354  816K DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere
 574K  361M RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  any    br-5c4f0dd6fb2f  anywhere             anywhere
    0     0 DROP       all  --  any    docker0  anywhere             anywhere
 243K   60M RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
 575K  361M RETURN     all  --  any    any     anywhere             anywhere

A container to container broadcast paket traverses the rules in the following way:

  • DOCKER-USER is empty, returned
  • DOCKER-ISOLATION-STAGE-1, rule 0 and 1 do not apply, returned
  • FORWARD, rule 6 accepts the paket because it is going from to br-5c4f0dd6fb2f (container x to y)

For an internal bridge the rules are set up as follows:

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-USER  all  --  any    any     anywhere             anywhere
    0     0 DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere
    0     0 ACCEPT     all  --  br-b87238754207 br-b87238754207  anywhere             anywhere
27940   41M ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    docker0  anywhere             anywhere
14354  816K ACCEPT     all  --  docker0 !docker0  anywhere             anywhere
    0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere

Chain OUTPUT (policy ACCEPT 43 packets, 6120 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  any    br-b87238754207 !10.1.1.0/24          anywhere
    0     0 DROP       all  --  br-b87238754207 any     anywhere            !10.1.1.0/24
14354  816K DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere
 574K  361M RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  any    docker0  anywhere             anywhere
 243K   60M RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
 575K  361M RETURN     all  --  any    any     anywhere             anywhere

A container to container broadcast paket traverses the rules in the following way:

  • DOCKER-USER is empty, returned
  • DOCKER-ISOLATION-STAGE-1, rule 1 and 2 will drop the paket as its destination ip is 255.255.255.255

The matching rule dump looks like:

# Generated by iptables-save v1.6.2 on Sun Aug 26 12:17:45 2018
*filter
:INPUT ACCEPT [621:59447]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [253:36005]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -i br-b87238754207 -o br-b87238754207 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s 10.1.1.0/24 -o br-b87238754207 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 10.1.1.0/24 -i br-b87238754207 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
Steps to reproduce this issue

Set up two container, connected by an internal bridge, one with a process broadcasting pakets to 255.255.255.255, the other one receiving pakets. The receiver will not receive broadcast pakets.

Expected behavior

Broadcast pakets can traverse container boundaries.

Additional info

Prepending the rule:

iptables -I DOCKER-ISOLATION-STAGE-1 -i br-b87238754207 -o br-b87238754207 -j ACCEPT

results in:

*filter
:INPUT ACCEPT [260:24354]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [15:2280]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -i br-b87238754207 -o br-b87238754207 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-b87238754207 -o br-b87238754207 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s 10.1.1.0/24 -o br-b87238754207 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 10.1.1.0/24 -i br-b87238754207 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT

which fixes the problem.

I am using the following docker versions:

Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.10.1
 Git commit:   9508e08017
 Built:        Thu May 31 07:52:56 2018
 OS/Arch:      linux/arm64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.10.1
  Git commit:   v18.03.1-ce
  Built:        Thu May 31 07:51:27 2018
  OS/Arch:      linux/arm64
  Experimental: false

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how internal bridge FORWARD and DOCKER-ISOLATION-STAGE-1 rules are generated, using the provided iptables-save output as the expected comparison. Reproduce the two-container broadcast setup and verify that IPv4 broadcasts to 255.255.255.255 cross the internal bridge without weakening isolation for traffic outside 10.1.1.0/24.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.