facebookresearch / facebookresearch/segment-anything
Fine-tune SAM on challenging cell segmentation images
- Dominant language
- Jupyter Notebook
- Stars
- 54.9k
- Forks
- 6.4k
- PR merge metrics
- No merged PRs in 30d
Description
Recently, there was a slew of tutorials and research papers available for ```SAM``` with which you can fine-tune SAM for pretty much any task. But, I was recently trying to fine-tune the decider of SAM on my cell segmentation **(binary classification)** problem and it gave me an error. Most tutorials are made in mind for datasets where the segmentation part can be easily encapsulated in a bounding box but my image can't be done as that. As there are maybe 30 different cells in one image and it is quite difficult to place bounding box around those cells.
The problem is not training SAM as SAM easily trains on my images rather it it is the moment of plotting the predicted masks by the model. I have tried experimenting with a few code and wrote some functions as well but they all fails. That's why, I am turning to the community if they could provide some support/guidance on this issue. It'll greatly help me.
**References**
* https://medium.com/@rekalantar/segment-anything-model-sam-for-medical-image-segmentation-9344ba57f2ca
* https://github.com/NielsRogge/Transformers-Tutorials/blob/master/SAM/Run_inference_with_MedSAM_using_HuggingFace_Transformers.ipynb
* https://github.com/NielsRogge/Transformers-Tutorials/blob/master/SAM/Fine_tune_SAM_(segment_anything)_on_a_custom_dataset.ipynb
**Image in my Dataset (Mask)**

**Image Copyright belongs to me and my team**
Contributor guide
Research direction
Start with the referenced SAM fine-tuning and MedSAM inference notebooks, then reproduce the failure at the predicted-mask plotting step using the multi-cell image described here. The issue does not include a traceback, repository file, or exact expected output, so first isolate those details; done means identifying a reproducible plotting failure and documenting the expected handling of the cell masks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100