microsoft / microsoft/OmniParser
Florence Model stuck in forever loop
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 25.4k
- Forks
- 2.2k
- PR merge metrics
- No merged PRs in 30d
Description
line 110 of utils.py:
generated_ids = model.generate(input_ids=inputs["input_ids"],pixel_values=inputs["pixel_values"],max_new_tokens=100,num_beams=3, do_sample=False)
This line runs forever on macbook pro m2 pro.
paddleocr==2.9.1
paddlepaddle==3.0.0b1
numpy==1.26.4
I traced it into the library code. Florence2ForConditionalGeneration class _encode_image method is where it gets stuck. Specifically on this line within the method:
x = self.vision_tower.forward_features_unpool(pixel_values)
Can anyone help me out with this?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with line 110 of utils.py and reproduce the hang using the listed paddleocr, paddlepaddle, and numpy versions on a MacBook Pro M2 Pro. Trace model.generate into Florence2ForConditionalGeneration._encode_image, especially vision_tower.forward_features_unpool(pixel_values). Done means generation no longer remains in a forever loop under the reported setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100