hashicorp / hashicorp/nomad

Nomad does not honor interface names with a colon in the name (:)

Open
#19,554 2 comments 0 reactions 1 assignee Claimed by @mismithhisler View on GitHub
stage/accepted theme/client theme/networking type/bug
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

### Nomad version
```
Nomad v1.7.2
BuildDate 2023-12-13T19:59:42Z
Revision 64e3dca9274b493e38a49fda3a70fd31d0485b91
```

### Operating system and Environment details
```
Distributor ID: Debian
Description: Debian GNU/Linux 11 (bullseye)
Release: 11
Codename: bullseye
```

### Issue
https://developer.hashicorp.com/nomad/docs/configuration/client#network_interface describes that adding `network_interface` to the `host_network` block will use the specified network interface. Setting a job to point to the specified `host_network` name fails with the message:

```
Scheduler dry-run:
- WARNING: Failed to place all allocations.
Task Group "space" (failed to place 1 allocation):
* Class "prod": 1 nodes excluded by filter
* Constraint "missing host network \"projects\" for port \"vcsssh\"": 1 nodes excluded by filter
```

when performing a job plan.

The following is my `nomad.hcl` config:

```hcl
data_dir = "/opt/nomad/data"
bind_addr = "192.168.5.2"
datacenter = "primary"
log_file = "/var/log/nomad/nomad.log"
log_rotate_max_files = 5

server {
enabled = true
bootstrap_expect = 1
encrypt = "lulno"

search {
fuzzy_enabled = true
limit_query = 200
limit_results = 1000
min_term_length = 5
}
}

client {
enabled = true
servers = ["192.168.5.2"]
node_class = "prod"

host_network "projects" {
cidr = "172.16.10.10/32"
interface = "enp1s0:10"
}

host_network "public" {
cidr = "172.16.10.2/32"
reserved_ports = "22"
interface = "enp1s0"
}

host_network "private" {
cidr = "192.168.5.2/32"
reserved_ports = "22"
}
}

consul {
address = "192.168.5.2:8500"
}

telemetry {
collection_interval = "1s"
disable_hostname = false
prometheus_metrics = true
publish_allocation_metrics = true
publish_node_metrics = true
}

ui {
enabled = true

consul {
ui_url = "http://172.16.10.2:8500/ui"
}
}

acl {
enabled = true
token_ttl = "30s"
policy_ttl = "60s"
}
```

The `/etc/network/interfaces` file:

```
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp1s0
iface enp1s0 inet static
address 172.16.10.2
netmask 255.255.255.0
gateway 172.16.10.1
up ip addr add 192.168.5.2/32 dev enp1s0 label enp1s0:1
down ip addr del 192.168.5.2/32 dev enp1s0 label enp1s0:1
dns-search home.lan
dns-nameservers 1.1.1.1

auto enp1s0:10
allow-hotplug enp1s0:10
iface enp1s0:10 inet static
address 172.16.10.10
netmask 255.255.255.0
gateway 172.16.10.1
dns-search home.lan
dns-nameservers 172.16.10.2

# This is an autoconfigured IPv6 interface
iface enp1s0 inet6 auto
```
Convention in Debian is to name virtual interfaces with `:`. This works just fine, as seen in `ip a`:

```
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether a8:a1:59:9a:35:df brd ff:ff:ff:ff:ff:ff
inet 172.16.10.2/24 brd 172.16.10.255 scope global enp1s0
valid_lft forever preferred_lft forever
inet 192.168.5.2/32 scope global enp1s0:1
valid_lft forever preferred_lft forever
inet6 2601:681:b80:8546:aaa1:59ff:fe9a:35df/64 scope global dynamic mngtmpaddr
valid_lft 77368sec preferred_lft 77368sec
inet6 fe80::aaa1:59ff:fe9a:35df/64 scope link
valid_lft forever preferred_lft forever
3: enp2s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a8:a1:59:9a:3a:69 brd ff:ff:ff:ff:ff:ff
4: docker0: mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:b9:84:26:a8 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:b9ff:fe84:26a8/64 scope link
valid_lft forever preferred_lft forever
[brevity]
39: enp1s0:10@enp1s0: mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether a8:a1:59:9a:35:df brd ff:ff:ff:ff:ff:ff
inet 172.16.10.10/24 brd 172.16.10.255 scope global enp1s0:10
valid_lft forever preferred_lft forever
inet6 fe80::aaa1:59ff:fe9a:35df/64 scope link
valid_lft forever preferred_lft forever
[...]
```

However, nomad doesn't seem to like these interface names. The documentation does not describe that nomad expects certain characters. Changing my interface name to `enp1s0.10` instead of `:10` fixed it all and nomad no longer complains about the missing host network.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.