facebookresearch / facebookresearch/segment-anything
Is it possible to run ONNX decoder with multiple boxes?
- Dominant language
- Jupyter Notebook
- Stars
- 54.9k
- Forks
- 6.4k
- PR merge metrics
- No merged PRs in 30d
Description
I am experimenting with onnx decoder trying to run it with multiple boxes as:
```
point = index * 2;
point_coords[0, point, 0] = rect.Left
point_coords[0, point, 1] = rect.Top
point_coords[0, point + 1, 0] = rect.Right
point_coords[0, point + 1, 1] = rect.Bottom
point_labels[0, point] = 2
point_labels[0, point + 1] = 3
```
where **index** is index of a box, **rect** is the box by index in "post transform to longest side" space **point_coords** and **point_labels** are tensors which will be passed to inference session.
when running inference with single box everything works fine and generated masks are good, trying to add at least 1 additional box leads to masks being mostly black with some white artifacts in it.
Do ONNX decoder supports multi-box input at the moment?
If it does how do I need to put labels on boxes properly?
Contributor guide
Research direction
Review the ONNX decoder entry point and its point_coords and point_labels inputs, then compare the tensor shapes and label values used for one box with the multi-box construction shown. Confirm whether multiple boxes are supported and document the required encoding, or record that the decoder accepts only one box.
Written by the indexing model from the issue text.
Assessment
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100