THUDM / THUDM/slime

Maybe Inconsistent GPU Utilization Due to Hardcoded `num_gpus_per_actor` and `num_gpus`

Open
#379 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.5k
Forks
1.3k
Avg merge
5h 36m
Merged PRs (30d)
22

Description

https://github.com/THUDM/slime/blob/921e2656aa3e5f7ff9fefe7cccda962a8bacfc13/slime/ray/rollout.py#L215-L226
https://github.com/THUDM/slime/blob/921e2656aa3e5f7ff9fefe7cccda962a8bacfc13/slime/ray/placement_group.py#L112-L120

Description:
The current implementation hardcodes num_gpus_per_actor=0.4 and num_gpus=0.2 in the GPU resource allocation logic. This may lead to significant underutilization of GPU resources in multi-node/multi-GPU setups?

Example Scenario:

  • Cluster: 4 nodes × 8 GPUs = 32 GPUs total.
  • Actual utilization:
    • Rollout actors: 0.4 GPU/actor × 32 GPUs12.8 GPUs
    • SGLang engine: 0.2 GPU/engine × 4 engines0.8 GPUs
    • Total used: 13.6 GPUs (42.5% utilization)
Image

Code Snippets:

  1. Hardcoded values in RolloutRayActor and allocate_train_group:
# Rollout engine setup
rollout_engines.append(
    RolloutRayActor.options(
        num_cpus=num_cpus,
        num_gpus=num_gpus,  # Hardcoded to 0.2
        scheduling_strategy=PlacementGroupSchedulingStrategy(...)
    )
)

# Train group allocation
def allocate_train_group(...):
    return RayTrainGroup(
        ...,
        num_gpus_per_actor=0.4,  # Hardcoded value
    )
Question

Does this configuration lead to inefficient GPU utilization? Should we make num_gpus_per_actor=0.4 and num_gpus=0.2 configurable parameters instead of hardcoding them in the code?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read slime/ray/rollout.py at lines 215-226 and slime/ray/placement_group.py at lines 112-120, then trace how the hardcoded GPU values are used for rollout actors and train-group allocation. Confirm the intended allocation across the described multi-node setup; done should make the values configurable and demonstrate that the resulting resource usage matches the configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.