facebookresearch / facebookresearch/sam2

Is there a way to have SAM2 track the same person across scenes when there are scene cuts?

Open
#334 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

Using Meta's SAM2 demo, the masks often swap onto a different person when there is a scene cut.

I know manually re-tagging the person in each scene is an option, but I wanted to explore if there's an automated solution available.

Thanks!

1. I tried using Meta's SAM2 demo at https://sam2.metademolab.com/ using a montage of Lebron
2. I was hoping for it to track Lebron across the entire video
3. I found it only did so in the first scene, and occasionally when Lebron was the only or main person in the shot

Below is an example:
https://github.com/user-attachments/assets/cfa028c6-838b-498f-b1ea-00ff1444b019

Contributor guide

Open the contributing guide

Research direction

Start with the SAM2 demo at https://sam2.metademolab.com/ and reproduce the reported behavior using the linked Lebron montage and example attachment. The issue does not identify repository files, tests, or an implementation entry point; done would require a defined, validated approach for maintaining the same identity across scene cuts.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.