facebookresearch / facebookresearch/sam3

Multiple category text prompt inferencing on video

Open
#267 1 comment 21 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11.7k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

Is there a way to add multiple text prompts (i.e. ['dog', 'cat']) for frame 0 and have the model detect + track any instances of both of these categories.

My question is specifically for the video inferencing SAM3 (with tracking), and not the single image segmentation.

I saw that many were pointed towards the example with the single image model, but that is not what I mean.

Thanks!

Contributor guide

Open the contributing guide

Research direction

Start with the SAM3 video inference and tracking entry point, then compare it with the existing single-image example referenced in the issue. Trace how text prompts are supplied for frame 0; done means determining whether multiple categories can be supported and documenting or implementing the required video behavior with coverage for both requested categories.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.