allenai / allenai/molmo

Image resizing and point coordinates

Aperta
#52 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
933
Fork
96
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hi!

I am super interested in the pointing functionality but haven't seen anyone asked about this detail:

**When you resize the images in the preprocessor, do you also rescale the "_point coordinates_" accordingly?** It feels to be the right way but from the fact that the code handles the formatter before resizing the images:
first do data formatting
https://github.com/allenai/molmo/blob/793fa387edfd6fd0f5b21eb8e0a7620a1f3799e1/olmo/data/model_preprocessor.py#L836
then call multimodal processor
https://github.com/allenai/molmo/blob/793fa387edfd6fd0f5b21eb8e0a7620a1f3799e1/olmo/data/model_preprocessor.py#L841

(I might certainly miss some details!!), looks like the points' coordinates are kept to its original value and serialized into texts.
Can you share some more insights on this? Thanks a lot!

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.