CNI plugin hooks occasionally fail due to duplicate address allocation with system jobs
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Nomad version
```
$ nomad version
Nomad v1.1.3 (5e94e4c06cdc18916e9c0d5f0e8c2130d6cfd2ac)
```
### Operating system and Environment details
Void Linux x86_64-musl, CNI plugins 1.0.1, Docker 20.10.6
### Issue
Occasionally Nomad will try to assign the same IP to a task that's already been allocated. This feels like state is not being correctly reconciled when Nomad starts up on a node that rebooted.
### Reproduction steps
Launch some system tasks, reboot, nomad has some chance of not bringing back system tasks due to IP collisions.
#### Expected Result
Nomad should be able to survive a reboot and restore system services to a host.
#### Actual Result
Jobs fail to restore because duplicate IP addressing is not allowed.
### Nomad Server logs (if appropriate)
Sorry, logging is a system service and unavailable during this issue :/
### Nomad Client logs (if appropriate)
N/A
### Screenshot of a dead task:

Contributor guide
No contributing guide indexed for this repository
Research direction
Start at Nomad startup on a rebooted client and trace system-task restoration through the CNI plugin hooks; the report identifies duplicate IP allocation as the failure point. Reproduce with system tasks and inspect the available client or CNI evidence; done means system services restore without duplicate addresses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- infrastructure, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100