OptimalScale / OptimalScale/LMFlow
baichuan-inc_Baichuan2-7B-Chat can't training
Open
Nobody has claimed this yet.
pending
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 822
- PR merge metrics
- No merged PRs in 30d
Description
感觉是baicihuan2的tokenizer做了更新，可能需要适配一下
报了个这样的错误：
Tokenizer class BaichuanTokenizer does not exist or is not currently imported.
版本的话是用的最新的代码，v0.05。
感觉改一下tokenizer应该就可以。
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing training for baichuan-inc_Baichuan2-7B-Chat with the latest v0.05 code and inspect the tokenizer integration that raises “Tokenizer class BaichuanTokenizer does not exist or is not currently imported.” Identify the relevant tokenizer entry point from that failure; the work is done when the model’s tokenizer is recognized and training starts successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100