huggingface / huggingface/diffusers

CLIP Score Evaluation without Pre-processing.

Aperta
#10,904 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

I am referring to [Evaluating Diffusion Models](https://huggingface.co/docs/diffusers/main/en/conceptual/evaluation), specifically the quantitative evaluation using CLIP score example.

We have images of shape (6, 512, 512, 3).

CLIP score is calculated using `"openai/clip-vit-base-patch16"`.

However, as far as I can tell, the images are not pre-processed to match the format that `"openai/clip-vit-base-patch16"` was trained on (e.g., images of size 224x224 pixels).

Should the images have been processed before or can we still reliably use the CLIP score with the images in their original format?

Please let me know if I have overlooked or am misunderstanding something. Thanks!

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with the quantitative CLIP score example in the Evaluating Diffusion Models documentation and inspect how the (6, 512, 512, 3) images are passed to openai/clip-vit-base-patch16. Compare that path with the model's expected image inputs. Done means the documentation clearly states whether preprocessing is applied or required, and the example is corrected if needed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
documentation, machine-learning
Tipo di issue
Documentazione
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.