modelscope / modelscope/ms-swift
[Bug] Custom Dataset extra fields cause Tokenizer Error or are dropped in GRPO (v3.11.0)
@hjh0119 is already working on this.
Since Dec 15, 2025.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Description
In ms-swift==3.11.0, using a Custom Dataset with extra fields (e.g., metadata like behavior, history) for GRPO training fails. This setup worked perfectly in v3.10.2.
Tokenizer Error: If extra fields are defined in columns, ms-swift tries to tokenize raw dictionary keys instead of values, causing a ValueError.
Silent Drop: If extra fields are NOT defined in columns (to avoid Error 1), they are silently dropped by the Data Collator and are not passed to the Reward Function (ORM) via kwargs.
This makes it impossible to implement custom dataset in standard format.
To Reproduce
Environment: ms-swift==3.11.0
Dataset: A JSONL file with messages and an extra field (e.g., behavior).
jsonl
{"messages": [...], "behavior": "some instructions..."}
Run GRPO:
bash
swift rlhf --rlhf_type grpo --dataset my_dataset --external_plugins plugin.py ...
Observed Behavior (v3.11.0)
The training crashes immediately at the tokenization/padding step.
ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided ['messages', 'behavior', ...]
It seems the raw dictionary keys are being passed to tokenizer.pad() instead of tokenized input_ids.
Expected Behavior
Like in v3.10.2, extra fields should be preserved through the pipeline and passed to the Reward Function via kwargs.
The tokenizer should only process fields mapped to model inputs (e.g., messages/input_ids) and ignore or pass-through metadata fields without crashing.
Environment
ms-swift version: 3.11.0 (Issue present) / 3.10.2 (Works fine)
Python version: 3.11
Installation: pip install ms-swift
Task: RLHF (GRPO)
Additional Context
Downgrading to ms-swift==3.10.2 resolves the issue immediately without any code changes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.