facebookresearch / facebookresearch/sam2

inference is too slow

Open
#387 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

"I want to ask why the propagation in video is so slow? I'm just processing a 17-second video, but it takes nearly an hour and a half. Can you help me figure out what is causing this?
my GPU is 3060Ti 12GB
![c4cbf07c2ff30a2f24d5deb9122ab29](https://github.com/user-attachments/assets/a4437343-6a31-41fb-8aa7-cb1a03058e6d)

my code is below:
video_segments = {} # video_segments contains the per-frame segmentation results
with tqdm(total=len(tracks["ball"]),desc="SAM2 ball init") as pbar:
for frame_index, track_item in enumerate(tracks["ball"]):
if isinstance(track_item, dict) and 'bbox' in track_item:
bbox = track_item['bbox']
box = np.array(bbox, dtype=np.float32)
frame_idx, object_ids, masks = predictor.add_new_points_or_box(
inference_state=state,
frame_idx=frame_index,
obj_id=ann_obj_ball_id,
box=box
)
pbar.update(1)

with tqdm(total=len(tracks["net"]),desc="SAM2 net init") as pbar:
for frame_index, track_item in enumerate(tracks["net"]):
if isinstance(track_item, dict) and 'bbox' in track_item:
bbox = track_item['bbox']
box = np.array(bbox, dtype=np.float32)
frame_idx, object_ids, masks = predictor.add_new_points_or_box(
inference_state=state,
frame_idx=frame_index,
obj_id=ann_obj_net_id,
box=box
)
pbar.update(1)

for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
video_segments[frame_idx] = {
out_obj_id: (masks[i] > 0.0).cpu().numpy()
for i, out_obj_id in enumerate(object_ids)
}

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.