facebookresearch / facebookresearch/segment-anything

Text prompt embedding size

Open
#19 3 comments 3 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

Thank you for releasing the model.
The paper mentions that text prompts are encoded using a pretrained `ViT-L/14@336px` CLIP model. CLIP embedding from this model are of size 768 while SegmentAnything prompt embedding is of size 256. Are there any extra steps for converting CLIP embeddings before feeding into the model?

Contributor guide

Open the contributing guide

Research direction

Start with the paper's description of text prompts, the pretrained ViT-L/14@336px CLIP model, and SegmentAnything's 256-dimensional prompt embeddings. Determine whether the 768-dimensional CLIP output is transformed before inference; done means documenting the conversion steps or confirming that maintainer clarification is required.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.