Torchvision Object detection TPU Support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
❓ Torchvision object detection models with TPU.
My doubt lies somewhere between feature request and question. hence posting here.
PyTorch supports TPU through torch_xla. It makes it possible to train models over TPU.
I guess most torchvision classification models can be used with transfer learning/training over TPU.
For torchvision object detection models, do they support TPU?
Some operations such as NMS, rpn, roi_align do not support TPU and hence I get an error as follows.
I was trying Faster R-CNN resnet50 fpn model for object detection.
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py", line 70, in forward
proposals, proposal_losses = self.rpn(images, features, targets)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/rpn.py", line 493, in forward
boxes, scores = self.filter_proposals(proposals, objectness, images.image_sizes, num_anchors_per_level)
File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/rpn.py", line 416, in filter_proposals
keep = box_ops.batched_nms(boxes, scores, lvl, self.nms_thresh)
RuntimeError: Cannot access data pointer of Tensor that doesn't have storage
My doubts/concerns/feature request.
- Do torchvision object detection models support TPU training?
- Any Plans for TPU support in future releases for these models?
- Are these ops only CUDA native and GPU/CPU specific? Is there a work-around to train object detection / segmentation models with TPU?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing torchvision's Faster R-CNN path, especially detection/rpn.py and the box_ops.batched_nms call shown in the traceback. Check how torch_xla handles NMS, RPN, and roi_align, and whether the reported storage-pointer error has an existing workaround. Done would require a documented or implemented path for TPU training, including support for the affected operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100