deepseek-ai / deepseek-ai/DeepSpec

launch_sglang_server.sh ignores CUDA_VISIBLE_DEVICES and always launches GPUs 0-7

Open
#36 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.1k
Forks
667
PR merge metrics
No merged PRs in 30d

Description

## Problem

`scripts/data/launch_sglang_server.sh` currently hardcodes `num_workers=8` and then launches workers by setting `CUDA_VISIBLE_DEVICES=${gpu_id}` for `gpu_id=0..7`.

This conflicts with the README guidance that users can reduce `CUDA_VISIBLE_DEVICES` for fewer GPUs. If a user only wants to run the data-generation servers on a subset such as GPUs `2,3`, the launcher still starts eight workers and overrides each child process to use GPUs `0` through `7`.

## Reproduction

I reproduced this by running the launcher with a stub `sglang` command that records the environment passed to each worker:

```bash
CUDA_VISIBLE_DEVICES=2,3 bash scripts/data/launch_sglang_server.sh
```

Observed worker launches:

```text
CUDA_VISIBLE_DEVICES=0 ... --port 30000
CUDA_VISIBLE_DEVICES=1 ... --port 30001
CUDA_VISIBLE_DEVICES=2 ... --port 30002
CUDA_VISIBLE_DEVICES=3 ... --port 30003
CUDA_VISIBLE_DEVICES=4 ... --port 30004
CUDA_VISIBLE_DEVICES=5 ... --port 30005
CUDA_VISIBLE_DEVICES=6 ... --port 30006
CUDA_VISIBLE_DEVICES=7 ... --port 30007
```

## Expected behavior

When `CUDA_VISIBLE_DEVICES=2,3` is set, the launcher should derive the worker GPU list from that value and start two workers, for example on ports `30000` and `30001`.

The existing eight-GPU default should remain unchanged when `CUDA_VISIBLE_DEVICES` is not set.

## Suggested fix

Parse `CUDA_VISIBLE_DEVICES` into a GPU list, derive the default `num_workers` from that list, keep the current `0,1,2,3,4,5,6,7` default when unset, and update the data-preparation docs so `prepare_data.sh` connects to the matching server ports.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.