UX/Unintended bug: sample.reward is None on aborted sample
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
Hi @zhuzilin, I think I found a subtle unintended UX issue. If a sample is aborted, sample.reward will be None because that is the default. This can easily be caused by using a --custom-generate-function-path where you might want to abort certain samples like in Search-R1 if a tool call fails.
However, aborting a sample is not safe as it will trigger an error when converting the list of rewards to a PyTorch Tensor in GRPO:
I suggest a minor patch like below or another way to handle None reward values (pending further testing if this works).
diff --git a/slime/rollout/sglang_rollout.py b/slime/rollout/sglang_rollout.py
index 51150f3..5b46358 100644
--- a/slime/rollout/sglang_rollout.py
+++ b/slime/rollout/sglang_rollout.py
@@ -139,6 +139,7 @@ async def generate_and_rm(args, sample: Sample, sampling_params: dict, evaluatio
# generate
async with state.semaphore:
if state.aborted:
+ sample.reward = 0.0
sample.status = Sample.Status.ABORTED
return sample
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in slime/rollout/sglang_rollout.py at generate_and_rm, then inspect slime/backends/utils/data.py around lines 78-88 where rewards are converted to a PyTorch tensor. Verify the aborted-sample path produces a tensor-compatible reward and that GRPO no longer errors when a sample is aborted, using the relevant existing test or reproduction path if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100