facebookresearch / facebookresearch/sam3

Negative box prompt in sam3 tracker or sam3 video predictor

Open
#335 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11.7k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

Dear SAM3 Team,

Thank you for your great work.

I've started to follow since SAM2, and just started to take a look SAM3. Great to see it started to support "session" & multi-GPU support as well although they are not clearly mentioned in somewhere else.

I cross-checked between SAM2 and SAM3, since I would like to compare performance between two models on the same image with only visual prompts. Plus verify added value of SAM3 compared to SAM2.

It is quite clear that negative "Box" prompt is supported in SAM3 image processor (which I cannot find overlap with SAM2).

https://github.com/facebookresearch/sam3/blob/757bbb0206a0b68bee81b17d7eb4877177025b2f/sam3/model/sam3_image_processor.py#L128-L150

However, my main focus is on video, so I got confused in `SAM3_tracking_predictor`.

In the end, the tracker will use `add_new_points_or_box` function.

https://github.com/facebookresearch/sam3/blob/757bbb0206a0b68bee81b17d7eb4877177025b2f/sam3/model/sam3_tracking_predictor.py#L233-L237

As it did in SAM2, two points from bounding box with label (2,3) are fed into the model. I cannot see a negative box support in the current SAM3 tracker. Or did I miss somewhere?

Contributor guide

Open the contributing guide

Research direction

Compare the negative-box handling in sam3/model/sam3_image_processor.py at lines 128-150 with SAM3_tracking_predictor.py at lines 233-237 and its add_new_points_or_box entry point. Determine whether video tracking already supports negative boxes; done means resolving that discrepancy and verifying the behavior in the video predictor or tracker path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.