huggingface / huggingface/candle

Unable to use any CLIP model with model_id

Open
#2,520 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21.1k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

I was unable to use [clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16) as there is no deserialization available for the CLIP Configuration. The only model that can be used is clip-vit-base-patch-32. In PR #2527 I have added the ability to get any model using the model_id.

https://github.com/huggingface/candle/blob/2f49e1b5349f4e56677ec0d3dc3fe98f9cbb87c7/candle-transformers/src/models/clip/mod.rs#L69-L75

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at candle-transformers/src/models/clip/mod.rs, particularly lines 69-75, and compare the current model loading path with PR #2527. Verify that CLIP configurations can be deserialized for model_id values beyond clip-vit-base-patch-32, including clip-vit-base-patch16.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.