facebookresearch / facebookresearch/sam2

Expected FPS (or per-frame inference time) difference between video and streaming settings?

Open
#593 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

I want to apply SAM2 in real-time for tracking 50 objects ideally. The performance in terms of robustness is fantastic with even tiny model for my application, however, I noticed the inference time almost increases linearly with the number of objects. If the speed difference between video and streaming settings is expected to be small, then I might think of upgrading my hardwares and then tracking less objects for my proof-of-concept.

With that, I have the following questions:
1. Regarding the speed measurement in https://github.com/facebookresearch/sam2?tab=readme-ov-file#sam-21-checkpoints, how many objects did you track?
2. How much difference on FPS (or per-frame inference time) would it be expected between video and streaming settings?
3. For your speed measurement, are they done under the compilation mode? If not, what's the expected improvement on FPS (or per-frame inference time) with compilation on?

I appreciate any comments and thanks in advance!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.