alibaba / alibaba/BladeDISC

[Stable Diffusion fine-tuning] constant tensor conversion failed

Open
#955 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
933
Forks
169
PR merge metrics
No merged PRs in 30d

Description

reproducing with https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README.md

with workaround #954, the script crashed, after debugging with gdb toolkit, I found torch-mlir conversion can not deal with constant value with tensor dtype.

``` text
#0 0x00007f5876f7defe in llvm::hashing::detail::fetch32 (p=0x0) at external/llvm-project/llvm/include/llvm/ADT/Hashing.h:157
#1 0x00007f5876f7e0ab in llvm::hashing::detail::hash_4to8_bytes (s=0x0, len=4, seed=18397679294719823053) at external/llvm-project/llvm/include/llvm/ADT/Hashing.h:204
#2 0x00007f5876f7e54e in llvm::hashing::detail::hash_short (s=0x0, length=4, seed=18397679294719823053) at external/llvm-project/llvm/include/llvm/ADT/Hashing.h:248
#3 0x00007f5878da6294 in llvm::hashing::detail::hash_combine_range_impl (first=0x0, last=0x4 )
at external/llvm-project/llvm/include/llvm/ADT/Hashing.h:457
#4 0x00007f5878da1d28 in llvm::hash_combine_range (first=0x0, last=0x4 ) at external/llvm-project/llvm/include/llvm/ADT/Hashing.h:484
#5 0x00007f5878d9d301 in llvm::hash_value (S=...) at external/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:606
#6 0x00007f5878d9939e in mlir::detail::DenseIntOrFPElementsAttrStorage::getKey (ty=..., data=..., isKnownSplat=false) at external/llvm-project/mlir/lib/IR/AttributeDetail.h:118
#7 0x00007f5878db2c3c in mlir::StorageUniquer::getKey&, bool&> ()
at external/llvm-project/mlir/include/mlir/Support/StorageUniquer.h:303
#8 0x00007f5878daf963 in mlir::StorageUniquer::get&, bool&>(llvm::function_ref, mlir::TypeID, mlir::ShapedType&, llvm::ArrayRef&, bool&) (this=0x6002f618, initFn=..., id=...) at external/llvm-project/mlir/include/mlir/Support/StorageUniquer.h:198
#9 0x00007f5878daaafa in mlir::detail::AttributeUniquer::getWithTypeID&, bool&> (ctx=0x7ffd6bbc7aa8, typeID=...)
at external/llvm-project/mlir/include/mlir/IR/AttributeSupport.h:204
#10 0x00007f5878da5a72 in mlir::detail::AttributeUniquer::get&, bool&> (ctx=0x7ffd6bbc7aa8)
at external/llvm-project/mlir/include/mlir/IR/AttributeSupport.h:177
#11 0x00007f5878da1570 in mlir::detail::StorageUserBase::get, bool> (ctx=0x7ffd6bbc7aa8) at external/llvm-project/mlir/include/mlir/IR/StorageUniquerSupport.h:154
#12 0x00007f5878d94b37 in mlir::DenseIntOrFPElementsAttr::getRaw (type=..., data=...) at external/llvm-project/mlir/lib/IR/BuiltinAttributes.cpp:1389
#13 0x00007f5878d94d75 in mlir::DenseIntOrFPElementsAttr::getRawIntOrFloat (type=..., data=..., dataEltSize=4, isInt=false, isSigned=true) at external/llvm-project/mlir/lib/IR/BuiltinAttributes.cpp:1423
#14 0x00007f5878d93d98 in mlir::DenseElementsAttr::getRawIntOrFloat (type=..., data=..., dataEltSize=4, isInt=false, isSigned=true) at external/llvm-project/mlir/lib/IR/BuiltinAttributes.cpp:1202
#15 0x00007f5877191f55 in mlir::DenseElementsAttr::get (type=..., values=...) at external/llvm-project/mlir/include/mlir/IR/BuiltinAttributes.h:117
#16 0x00007f5878ae67a1 in getDenseAttribute (shapedType=..., numElements=160, elements=0x0) at external/llvm-project/mlir/lib/CAPI/IR/BuiltinAttributes.cpp:521
#17 0x00007f5878ae5401 in mlirDenseElementsAttrFloatGet (shapedType=..., numElements=160, elements=0x0) at external/llvm-project/mlir/lib/CAPI/IR/BuiltinAttributes.cpp:568
#18 0x00007f587702c8bd in torch_mlir::convertTensorToMlirElementsAttr (tensor=..., loc=...)
at external/torch-mlir-importer/python/torch_mlir/dialects/torch/importer/jit_ir/csrc/torch_to_mlir_utils.cpp:366
```

The related IR:
``` text
[DEBUG mhlo_conversion.cpp:171] TorchMhlo input graph:
[DEBUG mhlo_conversion.cpp:171] graph(%arg0_1.1_ : Long(1, requires_grad=1, device=cuda:0)):
[DEBUG mhlo_conversion.cpp:171] %1 : int = prim::Constant[value=0]() # .4:5:35
[DEBUG mhlo_conversion.cpp:171] %2 : int = prim::Constant[value=9223372036854775807]() # .4:9:49
[DEBUG mhlo_conversion.cpp:171] %3 : int = prim::Constant[value=1]() # .4:10:50
[DEBUG mhlo_conversion.cpp:171] %slice_1.1 : Long(1, requires_grad=1, device=cuda:0) = aten::slice(%arg0_1.1_, %1, %1, %2, %3) # .4:9:14
[DEBUG mhlo_conversion.cpp:171] %5 : int = prim::Constant[value=1]() # .4:10:50
[DEBUG mhlo_conversion.cpp:171] %unsqueeze.1 : Long(*, *, requires_grad=1, device=cuda:0) = aten::unsqueeze(%slice_1.1, %5) # .4:10:16
[DEBUG mhlo_conversion.cpp:171] %7 : int = prim::Constant[value=6]() # .4:5:51
[DEBUG mhlo_conversion.cpp:171] %8 : bool = prim::Constant[value=0]() # .4:5:96
[DEBUG mhlo_conversion.cpp:171] %9 : NoneType = prim::Constant()
[DEBUG mhlo_conversion.cpp:171] %_to_copy.1 : Float(*, *, requires_grad=1, device=cuda:0) = aten::to(%unsqueeze.1, %7, %8, %8, %9) # .4:11:15
[DEBUG mhlo_conversion.cpp:171] %11 : Float(1, 160, strides=[160, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=]()
[DEBUG mhlo_conversion.cpp:171] %mul_1.1 : Float(*, *, device=cuda:0) = aten::mul(%_to_copy.1, %11) # .4:14:12
```

TorchBlade crashed on op:
``` text
[DEBUG mhlo_conversion.cpp:171] %11 : Float(1, 160, strides=[160, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=]()
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the crash using the Stable Diffusion example in examples/text_to_image/README.md and the workaround from issue #954. Start at torch_to_mlir_utils.cpp:366, following the constant tensor conversion into the MLIR C API call shown in the stack trace. Done means the prim::Constant tensor in the provided IR converts without crashing.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, pytorch
Domain
compilers, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.