Rollout dataset type implementation and validation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 349
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 318
Description
We should have explicit support in Gym for rollout datasets, datasets that are the result of running datasets through NeMo Gym agents. This would help efforts related to reward model training and SFT.
Specifically right now we have train, validation, and example dataset types. We should add a new rollout dataset type and then doing the corresponding validation similar to the train preparation. Rollout dataset validation validates that every row in the dataset is a valid instance of BaseVerifyResponse (link) which includes the responses create params, response, and reward.
Thanks to @HeyyyyyyG for the raise!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing train, validation, and example dataset type handling and the BaseVerifyResponse definition in nemo_gym/base_resources_server.py. Trace the train preparation validation path, then add the rollout type so each row is checked for response create parameters, response, and reward; done means valid rollout datasets are accepted and invalid rows are rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100