Support Baichuan2?
Open
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Baichuan2 is a generative model similar to llama, I find the following two differences:
-
- qkv merge as W_pack, so i change the file /ctranslate2/converters/transformers.py
- qkv merge as W_pack, so i change the file /ctranslate2/converters/transformers.py
-
- rotary changes to alibi, so i change the file /ctranslate2/converters/transformers.py
- rotary changes to alibi, so i change the file /ctranslate2/converters/transformers.py
But I found that the conversion only worked on simple sentences, complex sentences would not work.
Could you help me or is there a plan to support Baichuan2?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with /ctranslate2/converters/transformers.py and the linked Baichuan2 modeling_baichuan.py to compare W_pack and ALiBI handling. Reproduce the conversion on simple and complex sentences, then verify that Baichuan2 conversion and inference work beyond simple sentences.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100