HumanSignal / HumanSignal/label-studio-ml-backend
SAM Model Predict Bug
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 490
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 3
Description
I had followed https://github.com/heartexlabs/label-studio-ml-backend/tree/master/label_studio_ml/examples/segment_anything_model to deploy **SAM model**
But there are some bugs of **SAM** predictions:
### The coordinates of the mask will be offset from the target I expected, but in fact the shape of the mask is correct
Examples:
1. Here is a 2048*1024 image performance(**Offset**):

2. Here is a 640*426 image performance(**Correct**):

3. Here is a 640*480 image performance(**Offset**):

There are absolutely bugs on the process after prediction, and I think the key of question is the image padding
Here are the original images:



Contributor guide
No contributing guide indexed for this repository
Research direction
Start in label_studio_ml/examples/segment_anything_model and trace how SAM predictions are processed after inference, focusing on image padding and coordinate conversion. Reproduce the reported cases using the 2048×1024, 640×426, and 640×480 images; done means the predicted masks align with the target images without offsets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100