facebookresearch / facebookresearch/segment-anything
Question on training on images with multiple labels
- Dominant language
- Jupyter Notebook
- Stars
- 54.9k
- Forks
- 6.4k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
First of all thank you for your fantastic work and very detailed paper, I really enjoyed reading it.
I couldn't comprehend one thing from training procedure APPENDIX though and I would really appreciate clarifying it.
When there is an image with multiple labels/segments, how was it fed into the model? Here is the original procedure as I understand it (based on point prompts):
1. We sample a point uniformly from ground-truth mask.
2. We feed an image with that point to SAM and predict a mask.
3. Next we sample additional points in consecutive iterations from a region of error: a) either we label it as foreground in case the region is missing or b) we label it as background in case too much region was segmented.
4. We run prediction one more time on same image, with additional points and previous prediction mask as prompts.
Now, what happens when we have multiple labels(objects for segmentation) on one image? Did you repeat whole procedure for each label on every image, which means running even more predictions on the same image in the same training epoch? Or you sample single label for segmentation on each epoch (one label per image per epoch)? Or maybe there is some whole approach on this step of training, that I missed when digging into paper?
Contributor guide
Assessment
This issue has not been assessed yet.