[Bug] Actor unavailable error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
Bug Description
Traceback (most recent call last):
2026-03-18 05:25:02
2855e530…vdumckda
File "/root/slime_siqi/train.py", line 100, in
2026-03-18 05:25:02
2855e530…vdumckda
train(args)
2026-03-18 05:25:02
2855e530…vdumckda
File "/root/slime_siqi/train.py", line 69, in train
2026-03-18 05:25:02
2855e530…vdumckda
rollout_data_ref = ray.get(rollout_manager.generate.remote(rollout_id))
2026-03-18 05:25:02
2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:02
2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
2026-03-18 05:25:02
2855e530…vdumckda
return fn(*args, **kwargs)
2026-03-18 05:25:02
2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03
2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
2026-03-18 05:25:03
2855e530…vdumckda
return func(*args, **kwargs)
2026-03-18 05:25:03
2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03
2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 2981, in get
2026-03-18 05:25:03
2855e530…vdumckda
values, debugger_breakpoint = worker.get_objects(
2026-03-18 05:25:03
2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03
2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 1014, in get_objects
2026-03-18 05:25:03
2855e530…vdumckda
raise value
2026-03-18 05:25:03
2855e530…vdumckda
ray.exceptions.ActorUnavailableError: The actor ed2ff6eb7e211cc420c657b002000000 is unavailable: The actor is temporarily unavailable: RpcError: RPC error: Socket closed rpc_code: 14. The task may or may not have been executed on the actor.
Steps to Reproduce
I ran qwen3-1.7b training on math dataset, in step 9, this error occurrs
Expected Behavior
The program shouldn't collapse
Actual Behavior
the program exit
Environment
- slime version:
- Python version:
- PyTorch version:
- CUDA/ROCm version:
- GPU type and count:
- OS:
- SGLang version (if relevant):
- Megatron-LM version (if relevant):
Logs
Additional Context
No response
Pre-submission Checklist
- I have read the CONTRIBUTING.md and understand the collaboration scope.
- I have read the documentation and my issue is not addressed there.
- I have searched for existing issues and this is not a duplicate.
- I have provided a minimal, reproducible example.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with train.py around lines 69 and 100, especially the ray.get call for rollout_manager.generate, and reproduce the qwen3-1.7b math training run through step 9. Trace why the actor becomes unavailable and document a verified outcome where the program no longer exits unexpectedly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100