THUDM / THUDM/slime

[train_async.py] ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.

Open
#396 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.5k
Forks
1.3k
Avg merge
5h 36m
Merged PRs (30d)
22

Description

您好,很荣幸看到slime这么棒的工作。
我对slime框架的使用很感兴趣,目前正在尝试slime中关于自定义奖励函数的用法,实验环境为4*H100,模型为Qwen3-4B进行GRPO训练,其中设置
-- python3 train_async.py
--actor-num-nodes 1
--actor-num-gpus-per-node 2
--rollout-num-gpus 1
采用train_async.py 训推分离的模式,另外一张gpu用来本地部署模型作为奖励函数的api调用。

我已经做好前期的准备工作,并且调整合适的rollout_batch_size等参数可以运行起来,
但是当num_rollout训练完成保存模型后,日志打印报错信息如下:
(MegatronTrainRayActor pid=2121369) import pynvml # type: ignore[import]
Traceback (most recent call last):
File "/slime/train_async.py", line 108, in
train(args)
File "/slime/train_async.py", line 95, in train
rollout_data_curr_ref = ray.get(rollout_data_next_future)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/auto_init_hook.py, line 22, in auto_init_wrapper
return fn(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/client_mode_hook.py, line 104, in wrapper
return func(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/worker.py, line 2877, in get
raise ValueError(
ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.

Status message: Job entrypoint command failed with exit code 1, last available logs (truncated to 20,000 chars):
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/client_mode_hook.py, line 104, in wrapper
return func(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/worker.py, line 2877, in get
raise ValueError(
ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.

这个问题困扰了我很长时间,请教各位大佬有没有遇到过类似的问题?或者这个问题应该如何解决?
十分感谢

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in train_async.py at lines 95 and 108, then trace how rollout_data_next_future is produced before it is passed to ray.get. Reproduce the reported failure with the stated train_async.py configuration and inspect why the value is None; done means the training loop receives a valid object reference or reports the underlying failure clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.