facebookresearch / facebookresearch/sam3
Local Single Image Inference Speed is Far From Stated 30ms
- Dominant language
- Python
- Stars
- 11.7k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
GPU: RTX5070
Got processing time through the example script with example image, and the time taken seems to be 0.36s around there, which is far from the number stated in the paper, is this common?
Want to gather some info of GPU and real processing time for reference.
### Code
--------
```
import time
import torch
#################################### For Image ####################################
import matplotlib.pyplot as plt
from PIL import Image
from sam3.model_builder import build_sam3_image_model
from sam3.model.sam3_image_processor import Sam3Processor
from sam3.visualization_utils import plot_results
# Load the model
print(f"=== Loading Model ===")
load_start = time.time()
model = build_sam3_image_model()
load_time = time.time() - load_start
processor = Sam3Processor(model)
print(f"=== Model Loaded, using time: {load_time} seconds ===")
# Set concepts
concepts = [
"child",
"window",
"door"
]
all_results = {}
# Load an image
IMAGE_PATH = "./assets/images/test_image.jpg"
# IMAGE_PATH = "./assets/images/multiplayer.jpg"
print(f"=== Loading Image ===")
image = Image.open(IMAGE_PATH)
# loop through
for concept in concepts:
print(f"正在分割: {concept}")
set_image_start = time.time()
# 重新设置状态以避免干扰
inference_state = processor.set_image(image)
set_image_time= time.time() - set_image_start
# 执行分割
process_start = time.time()
output = processor.set_text_prompt(state=inference_state, prompt=concept)
process_time = time.time() - process_start
total_time = set_image_time + process_time
all_results[concept] = {
"masks": output["masks"],
"boxes": output["boxes"],
"scores": output["scores"],
"time": process_time,
}
print(f"找到 {len(output['masks'])} 个 {concept} 实例")
print(f"花费时间: {set_image_time}(set_image) + {process_time}(process_promt) = {total_time}s")
def show_multi_concept_results(image, results):
fig, axes = plt.subplots(2, 3, figsize=(18, 12))
axes = axes.flatten()
# 显示原图
axes[0].imshow(image)
axes[0].set_title("Original")
axes[0].axis('off')
# 显示各个概念的分割结果
for i, (concept, result) in enumerate(results.items()):
if i >= 5: # 最多显示5个概念
break
ax = axes[i+1]
ax.imshow(image)
masks = result["masks"]
# Check if any masks were actually found
if masks is not None and len(masks) > 0:
# Overlay each found mask
for mask in masks:
# FIX: Move tensor to CPU and convert to numpy for matplotlib
# We use .squeeze() to remove extra dimensions (e.g., 1xHxW -> HxW)
mask_np = mask.detach().cpu().squeeze().numpy()
ax.imshow(mask_np, alpha=0.4, cmap='viridis')
ax.set_title(f'{concept}: {len(masks)} found')
else:
# Informative title if nothing was detected
ax.set_title(f'{concept}: None detected')
ax.axis('off')
# Hide any unused subplots
for j in range(len(results) + 1, len(axes)):
axes[j].axis('off')
plt.tight_layout()
plt.show()
show_multi_concept_results(image, all_results)
```
### LOG
---------
```
(.sam3_venv) PS C:\Users\ellennhuang\sam3> & C:/.sam3_venv/Scripts/python.exe c:/Users/ellennhuang/sam3/test.py
=== Loading Model ===
=== Model Loaded, using time: 9.696309566497803 seconds ===
=== Loading Image ===
正在分割: child
找到 6 个 child 实例
花费时间: 0.42317867279052734(set_image) + 0.2846834659576416(process_promt) = 0.707862138748169s
正在分割: window
找到 1 个 window 实例
花费时间: 0.029998779296875(set_image) + 0.33757710456848145(process_promt) = 0.36757588386535645s
正在分割: door
找到 2 个 door 实例
花费时间: 0.025504112243652344(set_image) + 0.3391883373260498(process_promt) = 0.36469244956970215s
```
Thank you!
Contributor guide
Research direction
Start with the example script shown in the issue, especially test.py, build_sam3_image_model, Sam3Processor.set_image, and set_text_prompt. Reproduce the timings on the reported RTX5070 while separating image setup from prompt processing, then compare the results with the paper's stated 30ms conditions. Done means documenting whether the discrepancy is expected and identifying the relevant benchmark conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100