facebookresearch / facebookresearch/segment-anything

Use ViT-S as encoder

Open
#167 3 comments 5 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

Hi.

In the paper, you start training the model with the `ViT-B` model, and this is the smaller available model. Did you run experiments with smaller models, like [ViT-S](https://arxiv.org/pdf/2204.07141.pdf)? If not, do you think that a smaller model can be capable of providing good enough embeddings?

The reason I'm asking is that I noticed that running the encoder in the browser requires a significant amount of time and memory. For fully edge deployments having small and fast encoders may be beneficial.

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files, tests, or entry points. Begin by reviewing the paper’s ViT-B setup and the linked ViT-S reference, then identify how this repository selects encoder checkpoints. Done means establishing whether ViT-S provides suitable embeddings with acceptable browser time and memory, and documenting the result.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.