[train_async.py] ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
您好,很荣幸看到slime这么棒的工作。
我对slime框架的使用很感兴趣,目前正在尝试slime中关于自定义奖励函数的用法,实验环境为4*H100,模型为Qwen3-4B进行GRPO训练,其中设置
-- python3 train_async.py
--actor-num-nodes 1
--actor-num-gpus-per-node 2
--rollout-num-gpus 1
采用train_async.py 训推分离的模式,另外一张gpu用来本地部署模型作为奖励函数的api调用。
我已经做好前期的准备工作,并且调整合适的rollout_batch_size等参数可以运行起来,
但是当num_rollout训练完成保存模型后,日志打印报错信息如下:
(MegatronTrainRayActor pid=2121369) import pynvml # type: ignore[import]
Traceback (most recent call last):
File "/slime/train_async.py", line 108, in
train(args)
File "/slime/train_async.py", line 95, in train
rollout_data_curr_ref = ray.get(rollout_data_next_future)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/auto_init_hook.py, line 22, in auto_init_wrapper
return fn(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/client_mode_hook.py, line 104, in wrapper
return func(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/worker.py, line 2877, in get
raise ValueError(
ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.
Status message: Job entrypoint command failed with exit code 1, last available logs (truncated to 20,000 chars):
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/client_mode_hook.py, line 104, in wrapper
return func(*args, **kwargs)
File "/root/micromamba/envs/slime/lib/python3.12/site-packages/ray/_private/worker.py, line 2877, in get
raise ValueError(
ValueError: Invalid type of object refs, <class 'NoneType'>, is given. 'object_refs' must either be an ObjectRef or a list of ObjectRefs.
这个问题困扰了我很长时间,请教各位大佬有没有遇到过类似的问题?或者这个问题应该如何解决?
十分感谢
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in train_async.py at lines 95 and 108, then trace how rollout_data_next_future is produced before it is passed to ray.get. Reproduce the reported failure with the stated train_async.py configuration and inspect why the value is None; done means the training loop receives a valid object reference or reports the underlying failure clearly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100