facebookresearch / facebookresearch/sam2

Refining masks after propagation?

Open
#546 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

I was wondering if anyone has had success with refining masks by adding additional points after running propagation?
In my attempts, it is not working well, if at all.

While the video_predictor_example notebook provides an example of how it should work, its difficult to see what its actually doing since the additional refining point is on such a small area and it gets almost fully covered by the star. But when I change the model used in the example to the tiny model, it's easier to see that nothing is really happening.

Putting a negative point on frame 150:

![Image](https://github.com/user-attachments/assets/04a3efe9-69a8-4c46-a7cf-a5b313b558c9)

After running propagation again, there is no effect on frame 150 from the negative prompt:

![Image](https://github.com/user-attachments/assets/84e23f8e-6971-4208-baad-d37c45375d82)

Contributor guide

Open the contributing guide

Research direction

Start with the video_predictor_example notebook and reproduce the reported case using the tiny model, adding a negative point on frame 150 before running propagation again. Compare the resulting mask with the example screenshots; done means the negative prompt visibly refines the mask on frame 150 and the behavior is explained or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.