google / google/gvisor

Running nvproxy containers without Docker

Open
#9,435 9 comments 0 reactions 0 assignees View on GitHub
area: gpu type: enhancement
Dominant language
Go
Stars
19.3k
Forks
2k
Avg merge
3d 5h
Merged PRs (30d)
264

Description

### Description

Opening this mostly to avoid spending too much time reverse engineering Docker, runc, nvidia-container-runtime and gVisor behaviors. 😄

Since we don't use Docker to run our containers, figuring out how runsc and nvidia-container-runtime hooks interact is a bit of a challenge. I've read through the nvproxy sandbox setup code, which got me the basic `nvidia-smi` tool to work with our runtime by brute-forcing the OCI config and the `--nvproxy` and `--nvproxy-docker` flags, but I've also managed to busy loop the gofer at one point somehow (server load over 3000) trying to just start one of our inferencing workloads.

Would there be any resources you would be willing to share about how this works under the hood? All the documentation I can find is mostly just "how to configure Docker/Kubernetes to expose GPUs", which doesn't go into those details.

### Is this feature related to a specific bug?

_No response_

### Do you have a specific solution in mind?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.