deepseek-ai / deepseek-ai/DeepSeek-Coder
What is the correct padding side for train/eval of base model for FIM?
Open
- Dominant language
- Python
- Stars
- 24.3k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
I see in https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/finetune/finetune_deepseekcoder.py#L132 that the padding side for the instruct model is set to `right`.
However, the tokenizer's default is `left. And there are no other official examples I can find for a fine-tune on the base model.
So what should the padding side by for the base model during train and eval?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.