containerd / containerd/nerdctl

Extra_hosts records are overwitten by some accident

Open
#2,560 2 comments 1 reaction 0 assignees View on GitHub
kind/unconfirmed-bug-claim
Dominant language
Go
Stars
10.4k
Forks
826
Avg merge
1d 23h
Merged PRs (30d)
44

Description

### Description

I have chosen a **nerdctl compose** in order to provision containers without docker. The configuration has **extra_hosts:** parameters to define static hosts. It is expected that all the hosts I pass over the configuration is turned out into **--add-host** arguments in nerdctl to create containers according to the compose configuration.
In practice, running a --debug argument to nerdctl, I can see that containers is created as expected with every host I provide in compose configuration, also I see that meta.json file is created with defined and expected number of hosts, but hosts file itself doesn't contain all of them.
I noticed that sometimes ExtraHosts records are overwitten.
I described steps how it can be reproduced.
To be honest, suddenly I found the second bug. It happens when the compose configurations are similar and are located in the same directory. And I suppose both bugs are related to each other. Otherwise, I've been using containers with same hostname but in different compose configuration files.

### Steps to reproduce the issue

Start from scratch. We don't have any containers run on the host:
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
```

Check a directory where network configuraton for default namespace is saved:
```
[root@k8s-cp-node1 default]# ls -l /var/lib/nerdctl/1935db59/etchosts/default
total 0
```

I have decided to create two similar compose configurations with a container in both of them with an identical configuration.

Prepare the first container with following configuration:
```
[root@k8s-cp-node1 default]# cat /srv/test/docker-compose.yml.1
networks:
testnetwork:
external: true
name: "testnet"

services:
k8scpapi:
container_name: "test-cluster1-k8s-api-0-test1"
hostname: "k8s-api-0"
image: "alpine:latest"
pull_policy: "never"
restart: "unless-stopped"
read_only: true
runtime: "runc"
labels:
app: "host-main"
extra_hosts:
- "k8s-api-1:192.168.0.12"
- "etcd-0:192.168.0.11"
- "etcd-1:192.168.0.12"
networks:
testnetwork:
ipv4_address: "192.168.128.4"
privileged: false
user: "root"
command:
- "sh"
- "-c"
- "sleep inf"
```

Prepare the second container with following configuration:
```
[root@k8s-cp-node1 default]# cat /srv/test/docker-compose.yml.2
networks:
testnetwork:
external: true
name: "testnet"

services:
k8scpapi:
container_name: "test-cluster1-k8s-api-0-test2"
hostname: "k8s-api-0"
image: "alpine:latest"
pull_policy: "never"
restart: "unless-stopped"
read_only: true
runtime: "runc"
labels:
app: "host-main"
extra_hosts:
- "k8s-api-1:192.168.0.12"
- "etcd-0:192.168.0.11"
- "etcd-1:192.168.0.12"
networks:
testnetwork:
ipv4_address: "192.168.128.5"
privileged: false
user: "root"
command:
- "sh"
- "-c"
- "sleep inf"
```

For now we still don't have any container running:
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
```

Start the first container from nerdctl compose:
```
[root@k8s-cp-node1 default]# nerdctl compose -f /srv/test/docker-compose.yml.1 up -d
INFO[0000] Creating network test_default
INFO[0000] Ensuring image alpine:latest
INFO[0000] Creating container test-cluster1-k8s-api-0-test1
```

Now we have one container running:
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b2f2d032f4f6 docker.io/library/alpine:latest "sh -c sleep inf" 10 seconds ago Up test-cluster1-k8s-api-0-test1
```

Current configuration of the first container looks like the following. As you can notice we have **ExtraHosts** records in the hosts file (**it is fine for now**):
```
[root@k8s-cp-node1 default]# cat /var/lib/nerdctl/1935db59/etchosts/default/b2f2d032f4f6b3724bf49ff4a9d714850af9f7d887dfef0c83e842bba0038be5/hosts
#
127.0.0.1 localhost localhost.localdomain
::1 localhost localhost.localdomain
192.168.0.12 etcd-1
192.168.0.12 k8s-api-1
192.168.0.11 etcd-0
192.168.128.4 k8s-api-0 k8s-api-0.testnet test-cluster1-k8s-api-0-test1 test-cluster1-k8s-api-0-test1.testnet
#
```

Also we have meta.json here that is properly defined as expected:
```
[root@k8s-cp-node1 default]# cat /var/lib/nerdctl/1935db59/etchosts/default/b2f2d032f4f6b3724bf49ff4a9d714850af9f7d887dfef0c83e842bba0038be5/meta.json
{"Namespace":"default","ID":"b2f2d032f4f6b3724bf49ff4a9d714850af9f7d887dfef0c83e842bba0038be5","Networks":{"testnet":{"cniVersion":"1.0.0","interfaces":[{"name":"br-test","mac":"82:7c:9d:cb:9e:94"},{"name":"veth5251e54a","mac":"8e:b1:2b:4a:5c:28"},{"name":"eth0","mac":"fe:c7:a3:45:11:b3","sandbox":"/proc/43578/ns/net"}],"ips":[{"interface":2,"address":"192.168.128.4/24","gateway":"192.168.128.1"}],"routes":[{"dst":"0.0.0.0/0"}],"dns":{}}},"Hostname":"k8s-api-0","ExtraHosts":{"etcd-0":"192.168.0.11","etcd-1":"192.168.0.12","k8s-api-1":"192.168.0.12"},"Name":"test-cluster1-k8s-api-0-test1"}

```

Now it is a time to start the second container from nerdctl compose (**from a different compose-file**):
```
[root@k8s-cp-node1 default]# nerdctl compose -f /srv/test/docker-compose.yml.2 up -d
INFO[0000] Ensuring image alpine:latest
INFO[0000] Creating container test-cluster1-k8s-api-0-test2
```

And now we have two conntainer running (**it is expected**)
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2a4b4c251668 docker.io/library/alpine:latest "sh -c sleep inf" 16 seconds ago Up test-cluster1-k8s-api-0-test2
b2f2d032f4f6 docker.io/library/alpine:latest "sh -c sleep inf" About a minute ago Up test-cluster1-k8s-api-0-test1
```

Current configuration of the second container looks fine as we had with the first container. As you can notice we have also **ExtraHosts** records in the hosts file (**it is also fine for now**):
```
[root@k8s-cp-node1 default]# cat /var/lib/nerdctl/1935db59/etchosts/default/2a4b4c251668a1266fae04c9bd60570320f2e207e811b7696b575f2d5195f512/hosts
#
127.0.0.1 localhost localhost.localdomain
::1 localhost localhost.localdomain
192.168.0.12 k8s-api-1
192.168.0.11 etcd-0
192.168.0.12 etcd-1
192.168.128.5 k8s-api-0 k8s-api-0.testnet test-cluster1-k8s-api-0-test2 test-cluster1-k8s-api-0-test2.testnet
192.168.128.4 k8s-api-0 k8s-api-0.testnet test-cluster1-k8s-api-0-test1 test-cluster1-k8s-api-0-test1.testnet
#
```

Also we have meta.json here that is properly defined for the second container:
```
[root@k8s-cp-node1 default]# cat /var/lib/nerdctl/1935db59/etchosts/default/2a4b4c251668a1266fae04c9bd60570320f2e207e811b7696b575f2d5195f512/meta.json
{"Namespace":"default","ID":"2a4b4c251668a1266fae04c9bd60570320f2e207e811b7696b575f2d5195f512","Networks":{"testnet":{"cniVersion":"1.0.0","interfaces":[{"name":"br-test","mac":"82:7c:9d:cb:9e:94"},{"name":"veth13aa28bb","mac":"ea:ac:62:ed:ca:c7"},{"name":"eth0","mac":"de:0f:06:2b:34:cc","sandbox":"/proc/43698/ns/net"}],"ips":[{"interface":2,"address":"192.168.128.5/24","gateway":"192.168.128.1"}],"routes":[{"dst":"0.0.0.0/0"}],"dns":{}}},"Hostname":"k8s-api-0","ExtraHosts":{"etcd-0":"192.168.0.11","etcd-1":"192.168.0.12","k8s-api-1":"192.168.0.12"},"Name":"test-cluster1-k8s-api-0-test2"}

```
Let's check again a number of containers:
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2a4b4c251668 docker.io/library/alpine:latest "sh -c sleep inf" About a minute ago Up test-cluster1-k8s-api-0-test2
b2f2d032f4f6 docker.io/library/alpine:latest "sh -c sleep inf" 2 minutes ago Up test-cluster1-k8s-api-0-test1
```

And it is a time to check for bugs (**This is a first bug I found**). Hosts file was being overwritten and now it doesn't contain any **ExtraHost** record already after the second container was created:
```
[root@k8s-cp-node1 default]# cat /var/lib/nerdctl/1935db59/etchosts/default/b2f2d032f4f6b3724bf49ff4a9d714850af9f7d887dfef0c83e842bba0038be5/hosts
#
127.0.0.1 localhost localhost.localdomain
::1 localhost localhost.localdomain
192.168.128.5 k8s-api-0 k8s-api-0.testnet test-cluster1-k8s-api-0-test2 test-cluster1-k8s-api-0-test2.testnet
192.168.128.4 k8s-api-0 k8s-api-0.testnet test-cluster1-k8s-api-0-test1 test-cluster1-k8s-api-0-test1.testnet
#
```

If we decide to remove one of the containers from the host by any of the compose configuration we will remove all of them together (**I suppose it is a second bug**):
```
[root@k8s-cp-node1 default]# nerdctl compose -f /srv/test/docker-compose.yml.1 down
INFO[0000] Removing container test-cluster1-k8s-api-0-test1
INFO[0000] Removing container test-cluster1-k8s-api-0-test2
INFO[0000] Removing network test_default

```

To check if both containers were being removed we run this command to be aware of it:
```
[root@k8s-cp-node1 default]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
```

Up to this moment we don't have both containers we created on this host:
```
[root@k8s-cp-node1 default]# ls -l /var/lib/nerdctl/1935db59/etchosts/default
total 0
```

### Describe the results you received and expected

1. Compose configuration has to be independent and should not be removed with other similar ones.
2. ExtraHosts should be overwritten once created.

### What version of nerdctl are you using?

Nerdctl: v1.6.0
Containerd: v1.7.6

### Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

### Host information

Client:
Namespace: default
Debug Mode: false

Server:
Server Version: v1.7.6
Storage Driver: stargz
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Log: fluentd journald json-file syslog
Storage: native overlayfs stargz
Security Options:
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.5-200.fc38.x86_64
Operating System: Fedora Linux 38 (Cloud Edition)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 946.9MiB
Name: k8s-node1.domain.com
ID: b29f998c-8ca8-4065-9f08-26eb44dc19cd

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.