Instance clustering using sam

Hey, I have many photos containing for example oranges and apples and I want to get all the mask of the oranges out of the photos.

I need to cluster the masks embeddings to 3 clusters: apples, oranges and noise

I tried grid prompting all the images and then cluster the embeddings but that didn't help.

Has anyone succeed in something similar?

Embedding I tried:
1.Roi pooling of masks to image embeddings
2. Masks tokens
3. Entering the resulted masks to the prompt encoder and adding them to the image embeddings

The issue names no files, tests, or entry points. Start by reviewing the repository’s example notebooks and the reported ROI-pooling, mask-token, and prompt-encoder embedding approaches; define a reproducible clustering evaluation for apple, orange, and noise masks before deciding whether a code change is warranted.

Written by the indexing model from the issue text.