idea: eliminate the use of "pause" containers for docker networking
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
When a task group is using `"bridge"` networking mode, Nomad needs a network namespace shared by all tasks in the group. These tasks are not necessarily all using the same task driver (e.g. could be `exec` + `docker`, etc.)
Unfortunately once the Docker task driver gets involved this becomes tricky, because Docker explicitly does not support creating a container in a pre-defined network namespace (https://github.com/moby/moby/issues/7455). So unlike with the `exec` driver where we use normal Linux tooling to manage and join tasks to a shared network namespace, we are at the mercy of what Docker tooling enables.
The `docker network create` tool AFAICT has no support for creating a network namespace. Instead each container in Docker always gets created with its own network namespace, and then you can ["link"](https://docs.docker.com/network/links/) one container to another afterwords. This is where the `pause` container comes in. Because Docker ties the network namespace to a container (i.e. running process), Nomad needs a container that will not be stopped/replaced throughout the lifespan of the task group.
The problem of course, is that Bad Things can still happen to that pause container, and that is believed to be the source of most occurrences of https://github.com/hashicorp/nomad/issues/6385. While we can make improvements around cleaning up those orphaned resources, there's a lingering desire to just eliminate this whole class of problems outright.
I'm not sure what that means for users of the `docker` driver, unless we find a clever solution to start Docker containers in an existing network namespace, one that is created and managed by Nomad like we do for other task drivers. One idea would be to better invest in and promote the [podman](https://developer.hashicorp.com/nomad/plugins/drivers/podman) driver, which can also run Docker containers while being compatible with normal Linux tools and conventions.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by tracing Nomad's Docker-driver networking and pause-container lifecycle, then compare that behavior with the exec and Podman approaches described here; done would require a concrete, validated design that removes the pause-container dependency without breaking shared task-group networking.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- infrastructure, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100