GoogleCloudPlatform / GoogleCloudPlatform/container-engine-accelerators

is there a solution to make all gpu deveices visible for a pod which not requests `nvidia.com/gpu`

Open
#239 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
252
Forks
184
Avg merge
1h 55m
Merged PRs (30d)
2

Description

when I use [NVIDIA/k8s-device-plugin](https://github.com/NVIDIA/k8s-device-plugin) in my k8s cluster
I set NVIDIA_VISIBLE_DEVICES=all in pod spec
```
apiVersion: v1
kind: Pod
metadata:
name: test
containers:
- args:
- -c
- top -b
command:
- /bin/sh
env:
- name: NVIDIA_VISIBLE_DEVICES
value: all
image: cuda:10.2-cudnn7-devel-ubuntu18.04
name: test
resources:
limits:
cpu: 150m
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
```
the devices.list under `/sys/fs/cgroup/devices/kubepods/burstable/podxxxxxx/xxxxxx/devices.list` has all gpu deveice on this node
![image](https://user-images.githubusercontent.com/29671725/172170987-82a11398-b0ce-454d-80d7-eb44a6849bbc.png)

I noticed that this GCE container-engine-accelerators doesn’t require using nvidia-docker. so NVIDIA_VISIBLE_DEVICES may doesn't work.
thus, is there a solution to make all gpu deveices visible for a pod which not requests `nvidia.com/gpu` ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.