facebookresearch / facebookresearch/sam2
Is there a way to have SAM2 track the same person across scenes when there are scene cuts?
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
Using Meta's SAM2 demo, the masks often swap onto a different person when there is a scene cut.
I know manually re-tagging the person in each scene is an option, but I wanted to explore if there's an automated solution available.
Thanks!
1. I tried using Meta's SAM2 demo at https://sam2.metademolab.com/ using a montage of Lebron
2. I was hoping for it to track Lebron across the entire video
3. I found it only did so in the first scene, and occasionally when Lebron was the only or main person in the shot
Below is an example:
https://github.com/user-attachments/assets/cfa028c6-838b-498f-b1ea-00ff1444b019
Contributor guide
Research direction
Start with the SAM2 demo at https://sam2.metademolab.com/ and reproduce the reported behavior using the linked Lebron montage and example attachment. The issue does not identify repository files, tests, or an implementation entry point; done would require a defined, validated approach for maintaining the same identity across scene cuts.
Written by the indexing model from the issue text.
Assessment
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100