ml-explore / ml-explore/mlx-examples
Contributing SigLIP to `mlx-examples`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hello!
I'm new to the MLX ecosystem, and I came across the fact that there is a valid CLIP implementation available in the repository. Keeping in mind that SigLIP has the same structure as that of CLIP (as per the HF implementation available), I was wondering if there would be any interest for converting the SigLIP weights to a MLX-compatible format?
Many community modelling efforts to create MLLMs have been consistently choosing SigLIP over the original CLIP model due to better performance.
Happy to discuss and understand if there are any nuances I may have overlooked, or should keep in mind. Thank you so much!
(and thank you so much for MLX, such a life-saver for folks doing ML dev on Apple Silicon!)
Thoughts? @awni
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the repository's existing CLIP implementation and compare it with the linked Hugging Face SigLIP modeling implementation. Clarify the required weight-conversion scope and expected MLX-compatible output before starting; done should mean SigLIP weights can be converted and used in the repository, with validation identified during that investigation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100