Crash while compiling distilGPT2
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 736
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 15
Description
Hi, I'm getting an error while attempting to use torch.compile on the distilGPT2 model from hf. I'm using the following script to compile and run the model: https://gist.github.com/JakopinA/21c61452fce4b046cd58408c673d7117
Here is my crash backtrace:
```
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: ./build/bin/torch-mlir-opt -pass-pipeline=builtin.module(torchscript-module-to-torch-backend-pipeline) distilGPT2.mlir
#0 0x000056383786897a llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/lib/Support/Unix/Signals.inc:569:11
#1 0x0000563837868b2b PrintStackTraceSignalHandler(void*) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/lib/Support/Unix/Signals.inc:636:1
#2 0x0000563837867176 llvm::sys::RunSignalHandlers() /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/lib/Support/Signals.cpp:104:5
#3 0x0000563837869255 SignalHandler(int) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/lib/Support/Unix/Signals.inc:407:1
#4 0x00007f30a6667520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
#5 0x00007f30a66bba7c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
#6 0x00007f30a66bba7c __pthread_kill_internal ./nptl/pthread_kill.c:78:10
#7 0x00007f30a66bba7c pthread_kill ./nptl/pthread_kill.c:89:10
#8 0x00007f30a6667476 gsignal ./signal/../sysdeps/posix/raise.c:27:6
#9 0x00007f30a664d7f3 abort ./stdlib/abort.c:81:7
#10 0x00007f30a664d71b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#11 0x00007f30a665ee96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
#12 0x00005638331f9538 decltype(auto) llvm::dyn_cast(mlir::Type const&) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/include/llvm/Support/Casting.h:649:53
#13 0x00005638331f8e25 mlir::IntegerType mlir::Type::dyn_cast() const /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/../mlir/include/mlir/IR/Types.h:270:10
#14 0x00005638376afc18 mlir::Type::isInteger(unsigned int) const /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/IR/Types.cpp:33:20
#15 0x00005638332744cb (anonymous namespace)::TypeAnalysis::visitOperation(mlir::Operation*, llvm::ArrayRef const*>, llvm::ArrayRef*>) /home/alec/Nod/torch-mlir/lib/Dialect/Torch/Transforms/RefineTypes.cpp:950:9
#16 0x0000563833272dba mlir::dataflow::SparseDataFlowAnalysis>::visitOperationImpl(mlir::Operation*, llvm::ArrayRef, llvm::ArrayRef) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/../mlir/include/mlir/Analysis/DataFlow/SparseAnalysis.h:276:3
#17 0x00005638374a8eed mlir::dataflow::AbstractSparseDataFlowAnalysis::visitOperation(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp:125:1
#18 0x00005638374a86e7 mlir::dataflow::AbstractSparseDataFlowAnalysis::initializeRecursively(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp:55:25
#19 0x00005638374a8807 mlir::dataflow::AbstractSparseDataFlowAnalysis::initializeRecursively(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp:60:20
#20 0x00005638374a862d mlir::dataflow::AbstractSparseDataFlowAnalysis::initialize(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp:47:10
#21 0x00005638374865c6 mlir::DataFlowSolver::initializeAndRun(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Analysis/DataFlowFramework.cpp:70:25
#22 0x00005638332728cb (anonymous namespace)::RefineTypesPass::runOnOperation() /home/alec/Nod/torch-mlir/lib/Dialect/Torch/Transforms/RefineTypes.cpp:1686:23
#23 0x000056383743060a mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:466:21
#24 0x0000563837430c04 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:529:16
#25 0x0000563837436158 mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::$_14::operator()(mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::OpPMInfo&) const /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:749:36
#26 0x0000563837435dc9 mlir::LogicalResult mlir::failableParallelForEach<__gnu_cxx::__normal_iterator>>, mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::$_14&>(mlir::MLIRContext*, __gnu_cxx::__normal_iterator>>, __gnu_cxx::__normal_iterator>>, mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::$_14&) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/include/mlir/IR/Threading.h:46:18
#27 0x0000563837431ee3 mlir::LogicalResult mlir::failableParallelForEach>&, mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::$_14&>(mlir::MLIRContext*, std::vector>&, mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::$_14&) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/include/mlir/IR/Threading.h:92:10
#28 0x0000563837431783 mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:759:14
#29 0x00005638374308b7 mlir::detail::OpToOpPassAdaptor::runOnOperation(bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:650:5
#30 0x00005638374305fb mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:463:5
#31 0x0000563837430c04 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:529:16
#32 0x0000563837434ee1 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int)::$_6::operator()(mlir::OpPassManager&, mlir::Operation*) const /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:451:12
#33 0x0000563837434c62 mlir::LogicalResult llvm::function_ref::callback_fn(long, mlir::OpPassManager&, mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:45:12
#34 0x0000563833249031 llvm::function_ref::operator()(mlir::OpPassManager&, mlir::Operation*) const /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:68:12
#35 0x0000563833248d15 mlir::Pass::runPipeline(mlir::OpPassManager&, mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/../mlir/include/mlir/Pass/Pass.h:195:12
#36 0x0000563833246317 (anonymous namespace)::LowerToBackendContractPass::runOnOperation() /home/alec/Nod/torch-mlir/lib/Dialect/Torch/Transforms/LowerToBackendContract.cpp:235:18
#37 0x000056383743060a mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:466:21
#38 0x0000563837430c04 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:529:16
#39 0x000056383743259c mlir::PassManager::runPasses(mlir::Operation*, mlir::AnalysisManager) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:834:10
#40 0x00005638374324b2 mlir::PassManager::run(mlir::Operation*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Pass/Pass.cpp:814:60
#41 0x0000563832c028f2 performActions(llvm::raw_ostream&, bool, bool, llvm::SourceMgr&, mlir::MLIRContext*, llvm::function_ref, bool, bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:91:17
#42 0x0000563832c02559 processBuffer(llvm::raw_ostream&, std::unique_ptr>, bool, bool, bool, bool, bool, bool, llvm::function_ref, mlir::DialectRegistry&, llvm::ThreadPool*) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:139:12
#43 0x0000563832c02338 mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr>, llvm::function_ref, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool)::$_0::operator()(std::unique_ptr>, llvm::raw_ostream&) const /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:181:12
#44 0x0000563832c0222d mlir::LogicalResult llvm::function_ref>, llvm::raw_ostream&)>::callback_fn>, llvm::function_ref, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool)::$_0>(long, std::unique_ptr>, llvm::raw_ostream&) /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:45:12
#45 0x00005638376ef359 llvm::function_ref>, llvm::raw_ostream&)>::operator()(std::unique_ptr>, llvm::raw_ostream&) const /home/alec/Nod/torch-mlir/externals/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:68:12
#46 0x00005638376ee935 mlir::splitAndProcessBuffer(std::unique_ptr>, llvm::function_ref>, llvm::raw_ostream&)>, llvm::raw_ostream&, bool, bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Support/ToolUtilities.cpp:28:12
#47 0x0000563832c01283 mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr>, llvm::function_ref, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:186:10
#48 0x0000563832c0141f mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr>, mlir::PassPipelineCLParser const&, mlir::DialectRegistry&, bool, bool, bool, bool, bool, bool, bool, bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:209:10
#49 0x0000563832c02037 mlir::MlirOptMain(int, char**, llvm::StringRef, mlir::DialectRegistry&, bool) /home/alec/Nod/torch-mlir/externals/llvm-project/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:306:14
#50 0x0000563832bac247 main /home/alec/Nod/torch-mlir/tools/torch-mlir-opt/torch-mlir-opt.cpp:26:7
#51 0x00007f30a664ed90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#52 0x00007f30a664ee40 call_init ./csu/../csu/libc-start.c:128:20
#53 0x00007f30a664ee40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#54 0x0000563832bac105 _start (./build/bin/torch-mlir-opt+0x1b9105)
Aborted (core dumped)
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported torch-mlir-opt command using the distilGPT2 input from the linked gist, then inspect lib/Dialect/Torch/Transforms/RefineTypes.cpp around TypeAnalysis::visitOperation. Confirm the failing pipeline no longer aborts on this input and add a regression test if the repository's existing tests cover this pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100