facebookresearch / facebookresearch/sam2
is there any way to inference in a long time video
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
When doing inference in a video, you need to use
`inference_state = predictor.init_state()`
to initialize the state, which will load all frames of the video, and if the video is large, it will take up a lot of memory or even fail.
I don't know why it needs to load all frames, is there any way to use segmentation or other methods to do inference in a large video?
Contributor guide
Research direction
Start by reading the predictor.init_state() entry point and reproduce its memory use with a large video. Trace why all frames are loaded, then determine what supported large-video inference should look like and how completion would be verified without exhausting memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100