hashicorp / hashicorp/nomad-driver-virt
net: Nomad service port forwarding from client host machine local to VM
- Dominant language
- Go
- Stars
- 32
- Forks
- 9
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 11
Description
The current iptables rules used, as per [libvirts forwarding incoming connections doc](https://wiki.libvirt.org/Networking.html#forwarding-incoming-connections), do not allow for connections coming from the local Nomad client host to VM's running on it via the client interface IP address.
```console
root@libvirt-0:/nomad-test# nomad service info jrasell-workstation-ssh
Job ID Address Tags Node ID Alloc ID
jrasell-vm 10.0.1.17:22982 [] a400db6e 6d0386a6
root@libvirt-0:/nomad-test# ssh 10.0.1.17 -p 22982
ssh: connect to host 10.0.1.17 port 22982: Connection refused
```
The virsh console command and using the libvirt DHCP assigned address work as expected and can be used as a workaround until this issue is fixed.
```console
root@libvirt-0:/nomad-test# virsh console vm-795c791f
Connected to domain 'vm-795c791f'
Escape character is ^] (Ctrl + ])
```
```console
root@libvirt-0:/nomad-test# virsh net-dhcp-leases default
Expiry Time MAC address Protocol IP address Hostname Client ID or DUID
--------------------------------------------------------------------------------------------------------------------------------------------------------
2024-09-27 10:14:20 52:54:00:0d:a6:7b ipv4 192.168.122.10/24 nomad-vm-795c791f ff:08:24:45:0e:00:02:00:00:ab:11:d5:ae:0a:38:57:5f:a4:96
root@libvirt-0:/nomad-test# ssh ubuntu@192.168.122.10
Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-193-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of Fri Sep 27 09:18:42 UTC 2024
System load: 0.02 Processes: 125
Usage of /: 70.5% of 1.96GB Users logged in: 0
Memory usage: 0% IPv4 address for enp0s5: 192.168.122.10
Swap usage: 0%
Expanded Security Maintenance for Applications is not enabled.
0 updates can be applied immediately.
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The list of available updates is more than a week old.
To check for new updates run: sudo apt update
To run a command as administrator (user "root"), use "sudo ".
See "man sudo_root" for details.
```
Kernel logs showing iptables logging entries for attempting the local client ssh connection:
```console
2024-09-27T09:31:05.852474+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=48746 DF PROTO=TCP SPT=59862 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:31:05.852486+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=59862 WINDOW=0 RES=0x00 ACK RST URGP=0
2024-09-27T09:31:06.896442+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=35825 DF PROTO=TCP SPT=59866 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:31:06.896454+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=59866 WINDOW=0 RES=0x00 ACK RST URGP=0
2024-09-27T09:32:35.909451+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=46814 DF PROTO=TCP SPT=53884 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:32:35.909462+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=53884 WINDOW=0 RES=0x00 ACK RST URGP=0
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure with `nomad service info`, the shown SSH command, and the kernel iptables logs; compare it with the working `virsh console` and libvirt DHCP address paths. Trace the driver’s service-port forwarding entry point, which is not identified in the issue, and verify that connections from the local client host to the VM service address succeed without regressing external forwarding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, linux
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100