facebookresearch / facebookresearch/sam3
How implementing multi-frame and multi-category prompting (simultaneously!) in video segmentation.
- Dominant language
- Python
- Stars
- 11.7k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
I want to segment my video, and I want to do it using bounding box prompts. To ensure I don't miss anything, I need to input my bounding boxes in certain frames, for example, frames [1, 3, 5, 8, 10]. However, I found that I can only add a prompt to one frame index at a time and then propagate it. If I need prompts for multiple frames, I have to repeat this process. I want to add a certain number of bounding boxes to several frames throughout the entire video at once. How can I do this? Can anyone help me?
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating the video inference and prompt-propagation entry points, then trace how a single-frame bounding-box prompt is handled; done should mean bounding boxes can be supplied across the specified frames and categories in one workflow, with coverage verified on a representative video.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100