facebookresearch / facebookresearch/sam3

For anyone struggling with v3.1

Open
#554 2 comments 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11.7k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

Hey everyone!

There seems to be a variety of issues (#550, #544, #542, #526, #506, etc.) with the v3.1 update, mostly around device usage (cpu vs. gpu), memory usage or even just loading the model.

For anyone having issues with the v3.1 model, I have a repo ([MuggledSAM](https://github.com/heyoeyo/muggled_sam/tree/main)) that can run it (along with v1, v2 and v3.0 models) with support for cpu use, float16 on GPU and without any of the memory buildup problems. It might run on Mac as well, though it's untested. If you already installed SAM3, you can probably re-use the same env.

It also supports running at reduced resolutions, which works especially well with the v3.1 multiplexing features. For example, here's a screen capture (from running [this script](https://github.com/heyoeyo/muggled_sam/blob/main/simple_examples/video_segmentation_multiplexed.py) with the [SAM3 dance video](https://github.com/facebookresearch/sam3/tree/main/assets/videos/0001)) running at a reduced resolution of 336px, which manages to track 4 objects at 50fps on a 3090:

https://github.com/user-attachments/assets/870e1ecc-7764-4e69-8e4b-04196f2310a0

The model API is different from the original SAM implementation but there's a folder of [example scripts](https://github.com/heyoeyo/muggled_sam/tree/main/simple_examples) for how to use the model for different tasks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.