facebookresearch / facebookresearch/segment-anything

The selection area has obvious jagged problems

Open
#710 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

When I deploy the back-end Python code locally, the selection of areas after uploading an image is quite jagged. the python code is same as the demo:

Initialize the predictor:

checkpoint = "sam_vit_h_4b8939.pth"
model_type = "vit_h"
sam = sam_model_registry[model_type](checkpoint=checkpoint)
sam.to(device='cuda')
predictor = SamPredictor(sam)
Set the new image and export the embedding:

image = cv2.imread('src/assets/dogs.jpg')
predictor.set_image(image)
image_embedding = predictor.get_image_embedding().cpu().numpy()
np.save("dogs_embedding.npy", image_embedding)

But if you submit the image to https://model-zoo.metademolab.com/predictions/segment _ Everything _ box _ model, return the NPY file to the front end, and then go to select the area, it won't be jagged. Why is that?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the local Python pipeline with src/assets/dogs.jpg, sam_vit_h_4b8939.pth, vit_h, and the generated dogs_embedding.npy. Compare that result with the NPY returned by the model-zoo segment-everything box model and trace predictor.set_image, get_image_embedding, and the front-end selection flow. Done means identifying and documenting the cause of the jagged selection or providing a verified correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, opencv, python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.