LAION-AI / LAION-AI/Open-Assistant
Exception handling method when there is no pad token in tokenizer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 37.4k
- Forks
- 3.3k
- PR merge metrics
- No merged PRs in 30d
Description
In reinforcement learning, there is a part that converts the pad token of the sft tokenizer to the pad token of the rank tokenizer.
However, pad tokens do not exist in all tokenizers.
For example, the tokenizer of HuggingFace's "EleutherAI/pythia-70m-deduped" used for debugging does not have a pad token, so the following error occurs.
This is because the pad token does not exist in the tokenizer as shown below.
Therefore, exception handling is required for the tokenizer where the pad token does not exist for the code.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in model/model_training/trainer_rl.py at lines 77-83 and inspect how the SFT and rank tokenizers' pad tokens are converted. Reproduce the failure with Hugging Face's EleutherAI/pythia-70m-deduped tokenizer, then verify that tokenizers without a pad token no longer raise this error during reinforcement-learning setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100