intel / intel/llm-scaler

Need to set up proper permissions for `/dev/dri/{card,render}*`

Open
#370 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

**tldr** The Arc installer and/or README instructions should add the user to the `render` and `compute` groups so that things work without `sudo`, eg`sudo docker run ...` or `sudo xpu-smi ...` etc. should just be `docker run ...` or `xpu-smi ...`.

Additionally, there need to be better errors if `/dev/dri/{card,render}*` can't be accessed due to Unix permission issues.

**Expected behavior**

With everything properly configured, I expect to be able to serve models (and otherwise use my GPUs) without `sudo`. Ie,

docker run -td ...

followed by the like of

docker exec -it lsv-container bash
vllm serve ...

should be enough. Also, I expect

```bash
$ xpu-smi discovery
```

and such to work without `sudo`.

Also also, if the `/dev/dri/card*` or `/dev/dri/render*` device cannot be accessed due to file permission issues, I'd expect a clear error about that.

**Actual behavior**

The [instruction](https://github.com/intel/llm-scaler/tree/main/vllm#13-pulling-and-running-the-vllm-docker-container) say to start Docker with `sudo`

sudo docker run -td \
...

Based on general experience with Docker, this is confusing and counter to general Unix best practices. I assumed the `sudo` was there just for Docker stuff, relating to (not) being in the `docker` group. I added myself to `docker` as i standard practice, dropped the `sudo`, which led to very confusing errors and nothing working.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.