Inefficient usage of torch.randn() in vision/model_factory.py
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 346
- PR merge metrics
- No merged PRs in 30d
Description
torch.randn() used here first allocates space for self.batch_size in the host memory and then copies it to the device memory, wasting host-device bandwidth.
As self.batch_size contains random values, we can directly create self.batch_size on the device to bypass this non-sense host-device data copy. The proposed patch is as follows.
-torch.randn((self.batch_size, 3, 224, 224)).to(self.device)
+torch.randn((self.batch_size, 3, 224, 224), device=self.device)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open torchbenchmark/util/framework/vision/model_factory.py and inspect the torch.randn() call at line 36. Update the allocation as proposed so the random tensor is created on self.device, then verify the affected vision benchmark still runs and avoids the unnecessary host-to-device copy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100