Inference speed is significantly slower than SAM2 (5-6 FPS vs 30+ FPS on H200)

I am observing a significant performance gap between SAM3 and SAM2 during video propagation. On a single NVIDIA H200 (141GB), SAM3 achieves only 5-6 FPS for a 1080p video, while SAM2 can easily exceed 30 FPS on the same hardware and video.

I've noticed that model_builder.py and sam3_video_predictor.py do not explicitly expose precision settings (like fp16 or bf16), and some parts of video_base.py (e.g., in run_tracker_propagation) seem to cast tensors back to float32 before communication.

Read model_builder.py and sam3_video_predictor.py first, then trace run_tracker_propagation in video_base.py, paying attention to the reported float32 casts and unavailable precision settings. Profile the 1080p video path on an H200; done means identifying and addressing the relevant inference bottleneck and demonstrating a measured FPS improvement against the reported baseline.

Written by the indexing model from the issue text.