docker: firewall: interdocker communication broken when using internal networks

What happened:

When creating containers connected through an internal network, the communication between these containers is not working.

What you expected to happen:

It shouldn't matter whether to use an internal-only network or a network with web access. The communication between containers attached to the network should work.

How to reproduce it (as minimally and precisely as possible):

docker network create --internal test
docker run --network test --name nginx -d nginx
docker run --network test curlimages/curl nginx

Failed to connect to nginx port 80 after 1 ms: Host is unreachable

But when using non internal network, everything works as expected and the curl image gets the default nginx page.

Anything else we need to know?:

We encountered that docker didn't add the interface to the docker firewalld group. By adding that interface to the "docker" or "internal" zone, the communication works again. Furthermore the outer-communication to the web is still blocked, which is expected when using "internal" networks.

It seems to be related to https://github.com/firewalld/firewalld/issues/844. But I don't know if its more a firewalld or a docker problem, so I opened 2 bug reports. See https://github.com/firewalld/firewalld/issues/887

Environment:

• Firewalld Version (if Fedora based dnf info firewalld or commit hash if developing from git git log -n1 --format=format:"%H"): 1.0.1 (2.fc35)
• Firewalld Backend (cat /etc/firewalld/firewalld.conf | grep FirewallBackend): nftables
• OS (e.g: cat /etc/os-release): Fedora Server 35
• Docker version: 20.10.10

Reproduce the failure with the listed docker network and container commands, then compare it with a non-internal network. Inspect how the network interface is assigned to firewalld zones, using the linked firewalld issues as context. Done means containers on an internal network communicate while their external web access remains blocked.