nestybox / nestybox/sysbox

Network communication failed when running Docker swarm inside a Sysbox system container

Open
#250 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Shell
Stars
3.9k
Forks
230
Avg merge
7h 48m
Merged PRs (30d)
3

Description

Hi

Given the test suite, it's possible to launch swarm services in a docker swarm inside a sysbox system container but services on the same networks can't communicate with each other:

I initialize a swarm cluster with the help of this test file:

Then I created a network and two services:

docker exec $mgr sh -c "docker network create --attachable=true -d overlay innet"
docker exec $mgr sh -c "docker service create --replicas 1 --network innet --name firstservice alpine ping docker.com"
docker exec $mgr sh -c "docker service create --replicas 1 --network innet --name secondservice alpine ping docker.com"

Then I connect to the firstservice replica and try to ping secondservice:

ping: bad address 'secondservice'

In manager's log:

time="2021-03-30T19:19:01.030773398Z" level=info msg="shim containerd-shim started" address=/containerd-shim/27a7d45d81c1bb86758304c6f770bc730fa970d03cc47daa673f7626a409deae.sock debug=false pid=11675
time="2021-03-30T19:19:01Z" level=warning msg="file does not exist: /proc/sys/net/ipv6/conf/all/disable_ipv6 : stat /proc/sys/net/ipv6/conf/all/disable_ipv6: no such file or directory Has IPv6 been disabled in this node's kernel?"
time="2021-03-30T19:19:05.966424391Z" level=info msg="NetworkDB stats d288bd289303(248d2ba3518c) - netID:8mnt6217s6bfua4qb9fk2keaj leaving:false netPeers:2 entries:4 Queue qLen:0 netMsg/s:0"
time="2021-03-30T19:19:05.966560293Z" level=info msg="NetworkDB stats d288bd289303(248d2ba3518c) - netID:uau5lrwi4lxmf52tycwtist4r leaving:false netPeers:2 entries:8 Queue qLen:0 netMsg/s:0"
time="2021-03-30T19:19:08.268646522Z" level=info msg="Container 510866142c71261d7dc4eea060c0a50e61b58afc34c28225452ded40d6c1f64e failed to exit within 10 seconds of signal 15 - using the force"
time="2021-03-30T19:19:08.431420306Z" level=info msg="shim reaped" id=510866142c71261d7dc4eea060c0a50e61b58afc34c28225452ded40d6c1f64e
time="2021-03-30T19:19:08.441486528Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2021-03-30T19:19:08.442291287Z" level=warning msg="rmServiceBinding 0c932c01ce0b750e93491916c6494b28b7762560a111299c691229492a5e8711 possible transient state ok:false entries:0 set:false "
time="2021-03-30T19:19:08Z" level=error msg="set up rule failed, [-t mangle -D INPUT -d 10.0.1.2/32 -j MARK --set-mark 256]:  (iptables failed: iptables --wait -t mangle -D INPUT -d 10.0.1.2/32 -j MARK --set-mark 256: iptables: No chain/target/match by that name.\n (exit status 1))"
time="2021-03-30T19:19:08.558023941Z" level=error msg="Failed to delete firewall mark rule in sbox lb_uau5 (lb-inne): reexec failed: exit status 8"
time="2021-03-30T19:19:08.558253463Z" level=error msg="Failed add IP alias 10.0.1.2 to network uau5lrwi4lxmf52tycwtist4r LB endpoint interface eth0: cannot assign requested address"

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/dind/swarm.bats and reproduce the Docker Swarm setup described in the issue, including the attachable overlay network and two services. Inspect the manager logs and the relevant networking behavior around service-name resolution and firewall rules. Done means services on the same overlay network can communicate by service name inside a Sysbox system container.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
devops, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.