facebookresearch / facebookresearch/segment-anything

Question on training on images with multiple labels

Open
#486 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

Hi,

First of all thank you for your fantastic work and very detailed paper, I really enjoyed reading it.
I couldn't comprehend one thing from training procedure APPENDIX though and I would really appreciate clarifying it.
When there is an image with multiple labels/segments, how was it fed into the model? Here is the original procedure as I understand it (based on point prompts):
1. We sample a point uniformly from ground-truth mask.
2. We feed an image with that point to SAM and predict a mask.
3. Next we sample additional points in consecutive iterations from a region of error: a) either we label it as foreground in case the region is missing or b) we label it as background in case too much region was segmented.
4. We run prediction one more time on same image, with additional points and previous prediction mask as prompts.

Now, what happens when we have multiple labels(objects for segmentation) on one image? Did you repeat whole procedure for each label on every image, which means running even more predictions on the same image in the same training epoch? Or you sample single label for segmentation on each epoch (one label per image per epoch)? Or maybe there is some whole approach on this step of training, that I missed when digging into paper?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.