facebookresearch / facebookresearch/sam3

How implementing multi-frame and multi-category prompting (simultaneously!) in video segmentation.

Open
#345 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
11.7k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

I want to segment my video, and I want to do it using bounding box prompts. To ensure I don't miss anything, I need to input my bounding boxes in certain frames, for example, frames [1, 3, 5, 8, 10]. However, I found that I can only add a prompt to one frame index at a time and then propagate it. If I need prompts for multiple frames, I have to repeat this process. I want to add a certain number of bounding boxes to several frames throughout the entire video at once. How can I do this? Can anyone help me?

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating the video inference and prompt-propagation entry points, then trace how a single-frame bounding-box prompt is handled; done should mean bounding boxes can be supplied across the specified frames and categories in one workflow, with coverage verified on a representative video.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.