torch.inverse needs to be blacklisted
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 3
Description
SUMMARY
The method torch.inverse does not support fp16 arguments so it needs to be blacklisted.
DETAILS
Running the following
from apex import amp
import torch
class Foo(torch.nn.Module):
def __init__(self, dim):
super().__init__()
self.A = torch.nn.Parameter(torch.rand(dim, dim))
def forward(self, X):
AX = torch.matmul(self.A, X)
return torch.inverse(AX)
dim = 3
model = Foo(dim).cuda()
optimizer = torch.optim.AdamW(model.parameters())
model, optimizer = amp.initialize(model, optimizer, opt_level="O1", verbosity=False)
X = torch.rand(dim, dim).cuda()
Y = model(X)
Results in the error
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-4-46f64e772005> in <module>
1 X = torch.rand(dim, dim).cuda()
----> 2 Y = model(X)
3 Y
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
<ipython-input-2-d9366f5dfdeb> in forward(self, X)
6 def forward(self, X):
7 AX = torch.matmul(self.A, X)
----> 8 return torch.inverse(AX)
RuntimeError: "inverse_cuda" not implemented for 'Half'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file or test is named. Start by reproducing the provided Apex mixed-precision example and locating the operator blacklist handling for torch.inverse. Done means torch.inverse no longer receives fp16 arguments under the affected configuration, with a regression test covering the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100