facebookresearch / facebookresearch/sam3
Feature request: batched & multi-GPU inference support for SAM3 Agent
- Dominant language
- Python
- Stars
- 11.7k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
### Feature request: batched + multi-GPU inference for SAM3 Agent
Hi, thanks for releasing SAM 3 and the Agent pipeline!
I’m currently using the Agent interface (e.g. `sam3/agent/inference.py` + `run_single_image_inference`) with a vLLM backend to run text-driven segmentation on a large image dataset on a multi-GPU server.
It would be very helpful to have:
1. **Batched Agent inference (single GPU)**
- A function like `run_batch_inference(...)` that takes a list of `(image_path, text_prompt)` pairs and runs the Agent in batch, reusing the same SAM3 model and LLM client.
- Output format could stay the same as the current per-image `*_pred.json` files.
2. **Multi-GPU inference recipe for the Agent**
- An example script (e.g. `examples/agent_multi_gpu.py`) using `torchrun` or similar to:
- Split a list of images across multiple GPUs.
- Run the Agent on each subset.
- Avoid collisions in `sam_out/` and `*_pred.json` outputs.
This would make it much easier to scale Agent-based experiments to 10k+ images on machines with 4–8 GPUs.
**Environment (example)**
- Linux, Python 3.x, PyTorch 2.x
- GPUs: multi-GPU server (e.g. 8 × A800)
- SAM3: latest from GitHub
- LLM backend: vLLM + Qwen/Qwen3-VL-32B-Instruct
If there is already a recommended way to do batched or multi-GPU Agent inference, pointers would be greatly appreciated. Thanks!
Contributor guide
Research direction
Start with sam3/agent/inference.py and run_single_image_inference to understand the current per-image Agent flow and output files. Then review the requested examples/agent_multi_gpu.py approach using torchrun, including image partitioning and output collision avoidance. Done means batched inference and a multi-GPU recipe work for the described image and prompt inputs while preserving the existing prediction output format.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100