Lowering ops with mixed dtypes results in invalid TOSA IR
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 736
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 15
Description
Lowering an torch op that can mix/promote datatypes in its operands
like torch.aten.mul.Scalar %arg0, %float2.000000e00 : !torch.vtensor<[5],bf16>, !torch.float -> !torch.vtensor<[5],bf16>,
we generate invalid TOSA IR like "tosa.mul"(%arg0, %0) {shift = 0 : i32} : (tensor<5xbf16>, tensor<1xf32>) -> tensor<5xbf16>.
On TOSA level, ops generally need to have the same datatype in all operands, so we cannot mix f32 and bf16.
(On Linalg, this seems fine as the f32 constant operand is explicitly casted/turned into an bf16 constant)
Full reproducer
import torch
import torch_mlir
from torch_mlir_e2e_test.linalg_on_tensors_backends.refbackend import (
RefBackendLinalgOnTensorsBackend, )
from torch_mlir_e2e_test.tosa_backends.linalg_on_tensors import (
LinalgOnTensorsTosaBackend, )
class TestModule(torch.nn.Module):
def forward(self, x):
y = 2.0 * x
return y
args = torch.tensor([1.5, 0.5, -10, 255, 256], dtype=torch.bfloat16)
model = TestModule()
module = torch_mlir.compile(model, args, output_type="torch")
print("TORCH OutputType\n", module.operation.get_asm(large_elements_limit=10))
module = torch_mlir.compile(model, args, output_type="linalg-on-tensors")
print("LINALG_ON_TENSORS OutputType\n", module.operation.get_asm(large_elements_limit=10))
module = torch_mlir.compile(model, args, output_type="tosa")
print("TOSA OutputType\n", module.operation.get_asm(large_elements_limit=10))
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplied Python reproducer and compare the linalg-on-tensors and TOSA output from torch_mlir.compile, especially the aten.mul.Scalar lowering. Trace where the TOSA output is formed and verify completion by reproducing the case with operands that have compatible datatypes and valid TOSA IR.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100