llvm / llvm/torch-mlir

Lowering ops with mixed dtypes results in invalid TOSA IR

Open
#2,071 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
736
Avg merge
5d 22h
Merged PRs (30d)
15

Description

Lowering an torch op that can mix/promote datatypes in its operands
like torch.aten.mul.Scalar %arg0, %float2.000000e00 : !torch.vtensor<[5],bf16>, !torch.float -> !torch.vtensor<[5],bf16>,
we generate invalid TOSA IR like "tosa.mul"(%arg0, %0) {shift = 0 : i32} : (tensor<5xbf16>, tensor<1xf32>) -> tensor<5xbf16>.
On TOSA level, ops generally need to have the same datatype in all operands, so we cannot mix f32 and bf16.

(On Linalg, this seems fine as the f32 constant operand is explicitly casted/turned into an bf16 constant)

Full reproducer

import torch
import torch_mlir
from torch_mlir_e2e_test.linalg_on_tensors_backends.refbackend import (
    RefBackendLinalgOnTensorsBackend, )
from torch_mlir_e2e_test.tosa_backends.linalg_on_tensors import (
    LinalgOnTensorsTosaBackend, )

class TestModule(torch.nn.Module):
    def forward(self, x):
        y = 2.0 * x
        return y


args = torch.tensor([1.5, 0.5, -10, 255, 256], dtype=torch.bfloat16)
model = TestModule()

module = torch_mlir.compile(model, args, output_type="torch")
print("TORCH OutputType\n", module.operation.get_asm(large_elements_limit=10))

module = torch_mlir.compile(model, args, output_type="linalg-on-tensors")
print("LINALG_ON_TENSORS OutputType\n", module.operation.get_asm(large_elements_limit=10))

module = torch_mlir.compile(model, args, output_type="tosa")
print("TOSA OutputType\n", module.operation.get_asm(large_elements_limit=10))

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied Python reproducer and compare the linalg-on-tensors and TOSA output from torch_mlir.compile, especially the aten.mul.Scalar lowering. Trace where the TOSA output is formed and verify completion by reproducing the case with operands that have compatible datatypes and valid TOSA IR.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.