facebookresearch / facebookresearch/sam2
mat1 and mat2 must have the same dtype, but got BFloat16 and Float
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
# 初始化推理状态
inference_state = sam2_model.video_predictor.init_state(video_path=input_video_path)
# 传入的是多帧的mask数组
for mask_object in mask_objects:
mask_byte = down_file.url_to_byte(mask_object.get("mask_url"))
image_data = io.BytesIO(mask_byte)
mask_image = Image.open(image_data).convert('L') # 转换为灰度图
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Lambda(lambda x: (x > 0.5).float())
])
mask = transform(mask_image)
mask = mask.squeeze(0)
sam2_model.video_predictor.add_new_mask(inference_state,
mask_object.get("frame_idx"),
mask_object.get("frame_idx"),
mask)
# 传播提示获取整个视频的masklet
video_segments = {}
for out_frame_idx, out_obj_ids, out_mask_logits in sam2_model.video_predictor.propagate_in_video(
inference_state):
video_segments[out_frame_idx] = {
out_obj_id: (out_mask_logits[i] > 0.0).cpu().numpy()
for i, out_obj_id in enumerate(out_obj_ids)
}
mask_objects的长度是1的适合能正常运行,大于1就报错
Contributor guide
Assessment
This issue has not been assessed yet.