huggingface / huggingface/nanoVLM

lusxvr/nanoVLM-222M image patching issue

Open
#174 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5k
Forks
509
PR merge metrics
No merged PRs in 30d

Description

running: python generate.py --hf_model lusxvr/nanoVLM-222M

python generate.py --hf_model lusxvr/nanoVLM-222M
Using device: cuda
Loading weights from: lusxvr/nanoVLM-222M
Traceback (most recent call last):
File "D:\_src\nanoVLM\generate.py", line 71, in
main()
File "D:\_src\nanoVLM\generate.py", line 55, in main
processed_image, splittedimage_count = image_processor(img)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gilad\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\transforms\transforms.py", line 95, in __call__
img = t(img)
^^^^^^
File "C:\Users\gilad\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gilad\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\_src\nanoVLM\data\custom_transforms.py", line 98, in forward
raise ValueError(f'Image size {(h,w)} not divisible by patch_size {self.p}')
ValueError: Image size (1024, 896) not divisible by patch_size 224

its seems that the dynamic resize the image to (1024, 896), but than the SplitImage cant handle this size,

help anyone

Contributor guide

No contributing guide indexed for this repository

Research direction

Run `python generate.py --hf_model lusxvr/nanoVLM-222M` to reproduce the failure, then inspect `data/custom_transforms.py` at line 98 and the call from `generate.py` line 55. Determine how the 1024x896 image reaches the patching transform; done means this input no longer raises the reported divisibility error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.