facebookresearch / facebookresearch/sam2

Sam2 in 🤗 Transformers

Open
#682 4 comments 5 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

Hello!

I'm an ML engineer at Hugging Face, and we're (finally) implementing Sam2 in Transformers! The PR is [here](https://github.com/huggingface/transformers/pull/32317), and should be almost ready to go.

Seeing as the checkpoints on the hub (such as [this one](https://huggingface.co/facebook/sam2.1-hiera-tiny) for 2.1 tiny) don't contain safetensors files or config files that would conflict with transformers checkpoints, I was wondering if you would be open to host the transformers checkpoints on the existing repo, alongside the existing weights and yaml configs?

It wouldn't change anything for the current users of this repo, but it would enable users wanting to use the transformers implementation to load it via

```python
from transformers import AutoModel

sam2_model = AutoModel.from_pretrained("facebook/sam2.1-hiera-tiny")
```

Happy to give more information if needed, and looking forward to your answer!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.