hashicorp / hashicorp/nomad

QEMU ports stanza parsing fail

Open
#9,715 1 comment 0 reactions 0 assignees View on GitHub
theme/docs theme/driver/qemu
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

### Nomad version
Output from `nomad version`

```
Nomad v1.0.1 (c9c68aa55a7275f22d2338f2df53e67ebfcb9238)
```

### Operating system and Environment details

`inxi -F` result

```
System: Host: CentOS-78-64-minimal Kernel: 3.10.0-1160.6.1.el7.x86_64 x86_64 bits: 64 Console: tty 1
Distro: CentOS Linux release 7.9.2009 (Core)
Machine: Type: Desktop System: Hetzner product: N/A v: 1.0 serial: N/A
Mobo: ASRockRack model: B450D4U-V1L serial: M80-D2000304229 UEFI [Legacy]: American Megatrends v: L1.02U
date: 05/11/2020
CPU: Info: 6-Core model: AMD Ryzen 5 3600 bits: 64 type: MT MCP L2 cache: 3072 KiB
Speed: 3600 MHz min/max: 2200/3600 MHz Core speeds (MHz): 1: 3600 2: 3600 3: 3600 4: 3600 5: 3600 6: 3600 7: 3600
8: 3600 9: 3600 10: 3600 11: 3600 12: 3600
Graphics: Device-1: ASPEED Graphics Family driver: N/A
Display: server: No display server data found. Headless machine? tty: 489x68
Message: Unable to show advanced data. Required tool glxinfo missing.
Audio: Device-1: Advanced Micro Devices [AMD] Starship/Matisse HD Audio driver: snd_hda_intel
Sound Server: ALSA v: k3.10.0-1160.6.1.el7.x86_64
Network: Device-1: Intel I210 Gigabit Network driver: igb
IF: enp35s0 state: up speed: 1000 Mbps duplex: full mac: a8:a1:59:0f:2c:68
IF-ID-1: br-82d9a381839b state: up speed: N/A duplex: N/A mac: 02:42:b0:7d:36:dd
IF-ID-2: docker0 state: down mac: 02:42:91:c2:14:9b
IF-ID-3: veth00fffad state: up speed: 10000 Mbps duplex: full mac: 7e:45:6c:9a:de:73
IF-ID-4: veth174fa7c state: up speed: 10000 Mbps duplex: full mac: 4a:80:07:6a:94:ff
IF-ID-5: veth2792873 state: up speed: 10000 Mbps duplex: full mac: 7a:ba:57:bc:79:92
IF-ID-6: veth411273c state: up speed: 10000 Mbps duplex: full mac: 22:9f:bf:cc:15:94
IF-ID-7: veth41bd662 state: up speed: 10000 Mbps duplex: full mac: b6:5b:2d:69:65:f2
IF-ID-8: veth63de815 state: up speed: 10000 Mbps duplex: full mac: 12:ee:4f:6f:d2:58
IF-ID-9: veth6cd6b2b state: up speed: 10000 Mbps duplex: full mac: b2:4f:76:8c:4c:a0
IF-ID-10: veth73cd66d state: up speed: 10000 Mbps duplex: full mac: b2:44:7c:f4:99:d3
IF-ID-11: veth74ed125 state: up speed: 10000 Mbps duplex: full mac: 2e:97:71:45:14:88
IF-ID-12: veth84875d0 state: up speed: 10000 Mbps duplex: full mac: ba:26:f6:d5:67:03
IF-ID-13: veth9f1f25e state: up speed: 10000 Mbps duplex: full mac: 4a:9c:87:ff:9e:3c
IF-ID-14: vetha07d2b6 state: up speed: 10000 Mbps duplex: full mac: 92:cd:fd:b2:46:00
IF-ID-15: vetha715248 state: up speed: 10000 Mbps duplex: full mac: 62:cd:60:90:43:a5
IF-ID-16: vethac0f3b3 state: up speed: 10000 Mbps duplex: full mac: be:3c:48:dd:5d:da
IF-ID-17: vethc847fe1 state: up speed: 10000 Mbps duplex: full mac: ce:b8:08:a5:65:b1
IF-ID-18: vethcfe887f state: up speed: 10000 Mbps duplex: full mac: de:5f:46:4c:7d:f3
IF-ID-19: vethdb6dc1d state: up speed: 10000 Mbps duplex: full mac: fa:8f:9e:2c:15:54
IF-ID-20: vethe1b9d2b state: up speed: 10000 Mbps duplex: full mac: be:e6:df:86:14:da
IF-ID-21: vethe31a3d2 state: up speed: 10000 Mbps duplex: full mac: fe:d0:18:88:bc:e4
IF-ID-22: vetheb64941 state: up speed: 10000 Mbps duplex: full mac: 7a:41:78:38:9e:74
IF-ID-23: vethf06d03e state: up speed: 10000 Mbps duplex: full mac: 5a:18:68:21:81:62
IF-ID-24: vethf4e3a1c state: up speed: 10000 Mbps duplex: full mac: 5e:8d:06:cc:be:da
IF-ID-25: vethf7280db state: up speed: 10000 Mbps duplex: full mac: 1a:44:34:eb:1f:70
IF-ID-26: vethfe8ee58 state: up speed: 10000 Mbps duplex: full mac: a6:32:98:7a:db:2e
IF-ID-27: virbr0 state: down mac: 52:54:00:48:47:0a
IF-ID-28: virbr0-nic state: down mac: 52:54:00:48:47:0a
Drives: Local Storage: total: 953.88 GiB used: 305.33 GiB (32.0%)
ID-1: /dev/nvme0n1 vendor: Samsung model: MZVLB512HBJQ-00000 size: 476.94 GiB
ID-2: /dev/nvme1n1 vendor: Samsung model: MZVLB512HBJQ-00000 size: 476.94 GiB
RAID: Device-1: md0 type: mdraid status: active raid: mirror report: 2/2 UU Components: online: nvme0n1p1~c0 nvme1n1p1~c1
Device-2: md1 type: mdraid status: active raid: mirror report: 2/2 UU Components: online: nvme0n1p2~c0 nvme1n1p2~c1
Device-3: md2 type: mdraid status: active raid: mirror report: 2/2 UU Components: online: nvme0n1p3~c0 nvme1n1p3~c1
Partition: ID-1: / size: 436.34 GiB used: 152.43 GiB (34.9%) fs: ext4 dev: /dev/md2
ID-2: /boot size: 486.8 MiB used: 228.7 MiB (47.0%) fs: ext3 dev: /dev/md1
Swap: ID-1: swap-1 type: partition size: 31.97 GiB used: 247.8 MiB (0.8%) dev: /dev/md0
Sensors: Message: No ipmi sensors data was found.
Message: No sensors data was found. Is sensors configured?
Info: Processes: 462 Uptime: 36d 12h 59m Memory: 62.75 GiB used: 16.29 GiB (26.0%) Init: systemd runlevel: 3 Shell: Bash
inxi: 3.1.08
```

### Issue

Nomad job fails to start when there is `ports` stanza in QEMU driver config.
Docs screenshot:

![](https://i.imgur.com/gU9u9Xd.png)

Failture text ([screenshot](https://i.imgur.com/hNBGVD4.png)):
```
2 errors occurred: * failed to parse config: * Invalid label: No argument or block type is named "ports".
```

### Job HCL

```hcl
job "gitlab-runner" {
datacenters = ["dc1"]

priority = 80

constraint {
attribute = "${meta.client_id}"
value = "main_client"
}

group "gitlab-runner" {
count = 1

task "gitlab-runner" {
driver = "qemu"

config {
image_path = "/opt/kvm/images/GitlabRunner-disk1-nomad.img"
accelerator = "kvm"
graceful_shutdown = true
args = [
"-nodefaults",
"-nodefconfig",
"-smp", "2,sockets=2,cores=1,threads=1",
"-vnc", "0.0.0.0:1,password",
"-vga", "cirrus",
]

# port_map = {
# ssh = 22
# }
ports = ["ssh"] # Part from docs
}

resources {
cpu = 7200
memory = 6144

# Deprecated stanza, however it works with port_map
network {
port "ssh" {

}
}

}
}

restart {
attempts = 0
delay = "40s"
mode = "fail"
}

# That stanza don't work with port_map, unfortunately
# network {
# port "ssh" {
# static = 14000
# to = 22
# }
# }
}
}
```

### Reproduction steps

1. Deploy provided HCL job
2. Check status

### Comment

`nomad validate` says nothing about `ports` stanza, by the way.

Moreover, I wouldn't like to use `port_map` anymore, because it doesn't work with *group network block* (it works well with *task resources network* block, however it is deprecated now).


There is no `ports` in hclspec in [qemu driver.go](https://github.com/hashicorp/nomad/blob/9a36ebe2289ceed706bc5dc97472c4e697a9df13/drivers/qemu/driver.go#L97) (like in [docker driver config.go](https://github.com/hashicorp/nomad/blob/9a36ebe2289ceed706bc5dc97472c4e697a9df13/drivers/docker/config.go#L358))

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in drivers/qemu/driver.go at the hclspec referenced in the issue, then compare the QEMU configuration with drivers/docker/config.go. Reproduce the failure with the supplied HCL and check how nomad validate and deployment handle the ports stanza. Done means the documented configuration is accepted without the invalid-label error and the job can start.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.