bigscience-workshop / bigscience-workshop/petals

Add "Podman" usage to the documentation

Open
#546 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

Hello,

At first, I'm very happy that this project exists. I could try Beluga2 thanks to the community who shares, like I do, small parts of GPU. That's very impressive!

As a Linux Fedora user, I use Podman instead of Docker. That works exactly the same as Docker in terms of performances.

The methods is to follow: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html

In short, for Fedora:

```bash
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf install nvidia-container-toolkit
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
# then edit /etc/nvidia-container-runtime/config.toml
# to replace:
# [nvidia-container-cli]
# #no-cgroups = false
# no-cgroups = true
# and
# [nvidia-container-runtime]
# #debug = "/var/log/nvidia-container-runtime.log"
# debug = "~/.local/nvidia-container-runtime.log"
```

Then, launching petals server is easy:

```bash
podman run -p 31330:31330 \
--ipc host \
--device nvidia.com/gpu=all \
--security-opt=label=disable \
--volume petals-cache:/cache \
--rm \
learningathome/petals:main \
python -m petals.cli.run_server --port 31330 petals-team/StableBeluga2
```

As you can see, the only differences are to set a security option and give the device names.

That works like a charm on my RTX 3070.

Maybe you can add it, or do you need me to create the page / part in the documentation ?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing documentation for launching a Petals server and compare its Docker instructions with the supplied Fedora and Podman commands. Use the documented `petals.cli.run_server` entry point as the reference; done means the documentation explains the required NVIDIA CDI setup, Podman options, and server command.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.