Support for Phi3-Small, Medium, and Vision
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Does ctranslate2 to have plans to support the recently released small, medium, and vision. I've tried running them with transformers (on Windows) and can't get past a Triton and compiler not found kind of errors). I've not had the same issues with phi3-4k so it stands to reason that the phi3 converter in ctranslate2 might need to be modified to handle these models...even though they're in the same family.
ALSO, as far as I know ctranslate2 has never supported any vision models. Phi-3-Vision might be a good place to start or maybe you guys don't plan to support that general category due to time constraints.
Here are the models. Let me know if I can help in any way:
https://huggingface.co/microsoft/Phi-3-small-8k-instruct
https://huggingface.co/microsoft/Phi-3-small-128k-instruct
https://huggingface.co/microsoft/Phi-3-medium-4k-instruct
https://huggingface.co/microsoft/Phi-3-medium-128k-instruct
https://huggingface.co/microsoft/Phi-3-vision-128k-instruct
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing Phi-3 converter and the linked Phi-3 small, medium, and vision model descriptions. Determine which text-model variants and vision-model capabilities are in scope, then verify support against representative models; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100