Optimum supports gemma3 4b multimodal
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🚀 The feature, motivation and pitch
Gemma3 4b (https://huggingface.co/google/gemma-3-4b-it) is a image-text to text model. ExecuTorch already have Llava supported but having 1 more image-text to text model supported gives us confidence on the export flow and runtime APIs.
Work needed:
- AOT export. Looking at optimum-executorch right now, it is not able to export a
Gemma3Modelwithlanguage_model,vision_tower,multimodal_projectorandlm_head. The existing example intest_modeling_gemma3works on a 2b model which doesn't have the multimodal portion. - Generalize Llava runner to be able to run Gemma3 once exported.
Alternatives
No response
Additional context
No response
RFC (Optional)
No response
cc @mergennachin @iseeyuan @lucylq @helunwencser @tarun292 @kimishpatel @jackzhxng
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/models/test_modeling_gemma3.py and compare its working 2b export with the Gemma3Model multimodal components listed in the issue. Then inspect the existing Llava runner and its runtime path. Done means AOT export works for Gemma3 4b and the generalized Llava runner can execute the exported image-text model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100