[Feature] Support Idefics2 models
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
Extend lmdeploy supported multimodal models to include idefics2 architecture
### Related resources
- [Huggingface official documentation](https://huggingface.co/docs/transformers/main/en/model_doc/idefics2)
- [Huggingface Model Repo](https://huggingface.co/HuggingFaceM4/idefics2-8b)
- [Huggingface Trnasformers Implementation](https://github.com/huggingface/transformers/tree/main/src/transformers/models/idefics2)
### Additional context
_No response_
Contributor guide
Research direction
Start with the Hugging Face Idefics2 documentation, the HuggingFaceM4/idefics2-8b model repository, and the linked Transformers implementation to understand the architecture and expected inputs. Then locate the corresponding multimodal model entry points in lmdeploy; done means Idefics2 is supported by the toolkit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100