[Bug] shipped dynamic-sampling filters crash on fan-out groups (list[list[Sample]]) from multi-turn agent rollouts
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
Summary
The shipped dynamic-sampling filters assume groups are list[Sample], but multi-turn agent rollouts fan out into list[list[Sample]] — a shape that generate_and_rm_group itself documents and that every other consumer of the group handles explicitly. Enabling --dynamic-sampling-filter-path with any custom-generate function that returns a list of samples crashes the rollout within the first training step:
AttributeError: 'list' object has no attribute 'get_reward_value'
Evidence
- The shipped filter calls
sample.get_reward_value(args)for each element ofsamples, assumingsamples: list[Sample]:slime/rollout/filter_hub/dynamic_sampling_filters.py:9-11
generate_and_rm_groupdocuments both shapes (emphasis in the original docstring): "the group islist[Sample]for plain rollouts andlist[list[Sample]]for the fan-out case":slime/rollout/sglang_rollout.py:329-338
- The rollout loop itself defends against both shapes everywhere else — first-sample logging and final sorting both branch on
isinstance(group[0], list):slime/rollout/sglang_rollout.py:466-470, 494-497
- But
call_dynamic_filter(dynamic_filter, args, group)atsglang_rollout.py:469passes the raw group into the filter, which has no such branch.
Reproduction
Any rollout whose custom-generate function emits a list per trajectory (multi-turn agent fan-out, e.g. the pattern in examples/coding_agent_rl), with:
--dynamic-sampling-filter-path \
slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std_with_fallback
Minimal Python repro (no GPU):
import types
from slime.rollout.filter_hub.dynamic_sampling_filters import check_reward_nonzero_std
from slime.utils.types import Sample
args = types.SimpleNamespace(reward_key=None)
group = [[Sample(prompt="p", label="l", reward=0.2)], # fan-out: list[list[Sample]]
[Sample(prompt="p", label="l", reward=1.0)]]
check_reward_nonzero_std(args, group)
# AttributeError: 'list' object has no attribute 'get_reward_value'
Observed on slime main @ a3f50097, single-GPU H100, agentic GRPO run — reproduced at both N_SAMPLES=4 and N_SAMPLES=8.
Suggested fix
Handle the fan-out shape in the shipped filters, mirroring the pattern already used by the rollout loop:
def _entry_reward(args, entry):
# fan-out fragments of one trajectory share the trajectory reward
return entry[0].get_reward_value(args) if isinstance(entry, list) else entry.get_reward_value(args)
def check_reward_nonzero_std(args, samples: list, **kwargs):
rewards = [_entry_reward(args, s) for s in samples]
...
We shipped a fan-out-safe variant locally as a workaround and are happy to send a PR if the maintainers agree with the approach (fragments of one trajectory sharing entry[0]'s reward).
Related
- #2370 (same "agentic/multi-turn path is under-tested" family: final-step unconditional save)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with slime/rollout/filter_hub/dynamic_sampling_filters.py and compare its shipped filters with the fan-out shape documented in slime/rollout/sglang_rollout.py:329-338 and handled near lines 466-470 and 494-497. Run the minimal no-GPU reproduction from the issue, then exercise the filter through the dynamic-sampling path with both list[Sample] and list[list[Sample]] groups. Done means fan-out rollouts no longer raise AttributeError and plain rollouts continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100