Front-end conversion for nn.Linear() and rearrange() function generate wrong types
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 736
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 15
Description
I write a simple attention model:
```python
from torch import nn
from einops import rearrange
class Attention(nn.Module):
def __init__(self, dim = 32, heads = 3, dim_head = 8):
super().__init__()
inner_dim = dim_head * heads
self.heads = heads
self.scale = dim_head ** -0.5
self.norm = nn.LayerNorm(dim)
self.attend = nn.Softmax(dim = -1) # the last dim
self.to_qkv = nn.Linear(dim, inner_dim * 3, bias = False)
self.to_out = nn.Linear(inner_dim, dim, bias = False)
def forward(self, x):
x = self.norm(x)
q, k, v = self.to_qkv(x).chunk(3, dim = -1)
q, k, v = [einops.rearrange(t, 'n (h d) -> h n d', h=self.heads) for t in (q, k, v)]
dots = torch.matmul(q, k.transpose(-1, -2)) * self.scale
attn = self.attend(dots)
out = torch.matmul(attn, v)
out = rearrange(out, ' h n d -> n (h d)')
return self.to_out(out)
a = Attention(8, heads=1, dim_head = 2)
img2 = torch.ones(16, 8)
preds = a(img2)
module = torch_mlir.compile(a, img2, output_type="tosa",use_tracing=True,verbose=True)
print("TORCH OutputType\n", module.operation.get_asm(large_elements_limit=10))
```
torch_mlir.compile() goes wrong with following infomation:
/
Traceback (most recent call last):
File "/home/tianyi/Torchmlir/Models/vit-pytorch/OnlyAttention.py", line 66, in
module = torch_mlir.compile(a, img2, output_type="tosa",use_tracing=True,verbose=True)
File "/home/tianyi/anaconda3/envs/local-torch-mlir/lib/python3.10/site-packages/torch_mlir/__init__.py", line 391, in compile
run_pipeline_with_repro_report(
File "/home/tianyi/anaconda3/envs/local-torch-mlir/lib/python3.10/site-packages/torch_mlir/compiler_utils.py", line 75, in run_pipeline_with_repro_report
raise TorchMlirCompilerError(trimmed_message) from None
torch_mlir.compiler_utils.TorchMlirCompilerError: Lowering TorchScript IR -> Torch Backend IR failed with the following diagnostics:
python exception: Failure while executing pass pipeline:
error: "/home/tianyi/Torchmlir/Models/vit-pytorch/OnlyAttention.py":44:0: unsupported by backend contract: Unimplemented operator 'aten.chunk'
note: "/home/tianyi/Torchmlir/Models/vit-pytorch/OnlyAttention.py":44:0: see current operation: %21 = "torch.operator"(%20, %13, %14) {name = "aten.chunk"} : (!torch.tensor<[16,6],f32>, !torch.int, !torch.int) -> !torch.list
For Torch-MLIR developers, the error can be reproduced with:
$ torch-mlir-opt -pass-pipeline='builtin.module(torchscript-module-to-torch-backend-pipeline{backend-legal-ops=aten.flatten.using_ints,aten.native_layer_norm,aten.linear extra-library=})' /tmp/Attention.mlir
Add '-mlir-print-ir-after-all -mlir-disable-threading' to get the IR dump for debugging purpose.
/
Here I attach the /tmp/Attention.mlir.
The two wrong operations in Attention.mlir are :
1: torch.aten.linear generated from nn.Linear()
```mlir
func.func private @__torch__.torch.nn.modules.linear.Linear.forward(%arg0: !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear"> loc(unknown), %arg1: !torch.tensor loc(unknown)) -> !torch.tensor {
%9 = torch.tensor_static_info_cast %arg1 : !torch.tensor to !torch.tensor<[16,8],f32> loc(#loc)
%10 = torch.prim.GetAttr %arg0["weight"] : !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear"> -> !torch.tensor loc(#loc)
%none_0 = torch.constant.none loc(#loc)
%11 = torch.aten.linear %9, %10, %none_0 : !torch.tensor<[16,8],f32>, !torch.tensor, !torch.none -> !torch.tensor<[16,6],f32> loc(#loc2)
%12 = torch.tensor_static_info_cast %11 : !torch.tensor<[16,6],f32> to !torch.tensor loc(#loc2)
return %12 : !torch.tensor loc(#loc)
} loc(#loc)
```
2: torch.prim.NumToTensor.Scalar generated from einops.rearrange()
```mlir
......
%18:3 = torch.prim.ListUnpack %17 : !torch.list -> !torch.tensor<[16,2],f32>, !torch.tensor<[16,2],f32>, !torch.tensor<[16,2],f32> loc(#loc)
%int0 = torch.constant.int 0 loc(#loc5)
%19 = torch.aten.size.int %18#0, %int0 : !torch.tensor<[16,2],f32>, !torch.int -> !torch.int loc(#loc5)
%20 = torch.prim.NumToTensor.Scalar %19 : !torch.int -> !torch.tensor<[],si64> loc(#loc)
.....
```
How can I fix front end conversion for these two operations? Thanks!
[Attention.mlir.txt](https://github.com/llvm/torch-mlir/files/11454434/Attention.mlir.txt)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the supplied Attention.py reproducer and /tmp/Attention.mlir, then inspect the torch-mlir-opt backend pipeline shown in the error. Trace the aten.linear and torch.prim.NumToTensor.Scalar operations from the generated IR and compare them with the reported aten.chunk failure; done means the attention model converts without the incorrect operations or backend error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100