facebookresearch / facebookresearch/segment-anything
The selection area has obvious jagged problems
- Dominant language
- Jupyter Notebook
- Stars
- 54.9k
- Forks
- 6.4k
- PR merge metrics
- No merged PRs in 30d
Description
When I deploy the back-end Python code locally, the selection of areas after uploading an image is quite jagged. the python code is same as the demo:
Initialize the predictor:
checkpoint = "sam_vit_h_4b8939.pth"
model_type = "vit_h"
sam = sam_model_registry[model_type](checkpoint=checkpoint)
sam.to(device='cuda')
predictor = SamPredictor(sam)
Set the new image and export the embedding:
image = cv2.imread('src/assets/dogs.jpg')
predictor.set_image(image)
image_embedding = predictor.get_image_embedding().cpu().numpy()
np.save("dogs_embedding.npy", image_embedding)
But if you submit the image to https://model-zoo.metademolab.com/predictions/segment _ Everything _ box _ model, return the NPY file to the front end, and then go to select the area, it won't be jagged. Why is that?
Contributor guide
Research direction
Start by reproducing the local Python pipeline with src/assets/dogs.jpg, sam_vit_h_4b8939.pth, vit_h, and the generated dogs_embedding.npy. Compare that result with the NPY returned by the model-zoo segment-everything box model and trace predictor.set_image, get_image_embedding, and the front-end selection flow. Done means identifying and documenting the cause of the jagged selection or providing a verified correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, opencv, python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100