facebookresearch / facebookresearch/segment-anything

Why do the results of the same image differ?

Open
#185 38 comments 36 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

Hi, I try to find all the objects in the image automatically. I used below code.
``` python
import numpy as np
import torch
import matplotlib.pyplot as plt
import cv2
import glob
def show_anns(anns,save_path):
if len(anns) == 0:
print(save_path)
return
sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True)
ax = plt.gca()
ax.set_autoscale_on(False)
polygons = []
color = []
for ann in sorted_anns:
m = ann['segmentation']
img = np.ones((m.shape[0], m.shape[1], 3))
color_mask = np.random.random((1, 3)).tolist()[0]
for i in range(3):
img[:,:,i] = color_mask[i]
ax.imshow(np.dstack((img, m*0.35)))
plt.savefig(save_path)

import sys
sys.path.append("..")
from segment_anything import sam_model_registry, SamAutomaticMaskGenerator, SamPredictor

sam_checkpoint = "../sam_vit_h_4b8939.pth"
model_type = "vit_h"

device = "cuda"

sam = sam_model_registry[model_type](checkpoint=sam_checkpoint)
sam.to(device=device)

mask_generator = SamAutomaticMaskGenerator(sam)

files = glob.glob(fr"./*.jpg")
idx = 0
for file in files:
image = cv2.imread(file)

image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
print(f"image.shape:{image.shape}")
plt.clf()
plt.subplot(1,2,1)
plt.imshow(image)

plt.subplot(1,2,2)
plt.imshow(image)
masks = mask_generator.generate(image)
print(fr"masks:{len(masks)}")
show_anns(masks,fr"{idx}.png")
idx += 1

```
However, I got this result:
![1](https://user-images.githubusercontent.com/118299117/231519894-54d971e7-f6ee-4546-9867-b08b14b05976.png)

But the demo's effect is very good, you can see the below result
![1681316389260](https://user-images.githubusercontent.com/118299117/231520545-cab861b0-b505-461f-b2cd-f72b5d0024cd.png)

My biggest question is why the online effect is so good!!!
Have you used any other methods?

Contributor guide

Open the contributing guide

Research direction

The relevant entry point is SamAutomaticMaskGenerator.generate in the provided Python script; compare its inputs and output mask count with the demo workflow. Start by checking the loaded sam_vit_h_4b8939.pth checkpoint, image conversion, and visualization in show_anns. Done means explaining the observed difference from the repository's documented demo path or identifying a reproducible mismatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
matplotlib, numpy, opencv, python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.