llvm / llvm/llvm-project

[flang][MLIR] Hard crash when threading

Open
#218,401 2 comments 0 reactions 0 assignees View on GitHub
crash flang mlir
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

When Flang (or rather MLIR) cannot spawn new threads, the compiler crashes with a stack trace. While it is probably okay to throw an error, the hard crash is not expected.

Here is a simple reproducer script. It is very artificial in the sense that it doesn't allow *any* threads to be spawned, but disabling threading for MLIR works around the problem.
```bash
#!/usr/bin/env bash

echo "flang: $(command -v flang)"
flang --version | head -1
echo

baseline=$(ps -u "$(id -u)" -L --no-headers | wc -l)
echo "current threads/processes for this user: $baseline"
echo

cat <repro.f90
end
EOF

echo "=== crash: repro.f90 is just the single line 'end', default flags, RLIMIT_NPROC=$baseline (zero headroom) ==="
bash -c "ulimit -u $baseline && flang -c repro.f90 -o /tmp/repro.o" 2>&1
echo

echo "=== workaround: -mmlir -mlir-disable-threading avoids the crash under the same limit ==="
bash -c "ulimit -u $baseline && flang -mmlir -mlir-disable-threading -c repro.f90 -o /tmp/repro.o" \
&& echo "compiled OK: /tmp/repro.o"
```

```
flang: /home/hmenke/LLVM-23.1.0-rc3-Linux-X64/bin/flang
flang version 23.1.0-rc3 (https://github.com/llvm/llvm-project 7196f931f212fc7c406066b2628a0ff4ea0ee344)

current threads/processes for this user: 499

=== crash: repro.f90 is just the single line 'end', default flags, RLIMIT_NPROC=499 (zero headroom) ===
LLVM ERROR: pthread_create failed: Resource temporarily unavailable
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
Stack dump:
0. Program arguments: /home/hmenke/LLVM-23.1.0-rc3-Linux-X64/bin/flang-23 -fc1 -triple x86_64-unknown-linux-gnu -emit-obj -fcolor-diagnostics -mrelocation-model pic -pic-level 2 -pic-is-pie -target-cpu x86-64 -resource-dir /home/hmenke/LLVM-23.1.0-rc3-Linux-X64/lib/clang/23 -fintrinsic-modules-path /home/hmenke/LLVM-23.1.0-rc3-Linux-X64/lib/clang/23/finclude/flang/x86_64-unknown-linux-gnu -mframe-pointer=all -o /tmp/repro.o -x f95 repro.f90
1. Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0 flang-23 0x0000557bf3bd1bea llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 42
1 flang-23 0x0000557bf3bd257d
2 libc.so.6 0x0000147126c3ebf0
3 libc.so.6 0x0000147126c8c07c
4 libc.so.6 0x0000147126c3eb46 raise + 22
5 libc.so.6 0x0000147126c28833 abort + 211
6 flang-23 0x0000557bf3b861d1 llvm::report_fatal_error(llvm::Twine const&, bool) + 257
7 flang-23 0x0000557bf3bd352b
8 flang-23 0x0000557bf3bd34ae llvm::llvm_execute_on_thread_impl(void* (*)(void*), void*, std::optional) + 142
9 flang-23 0x0000557bf3d1d130
10 flang-23 0x0000557bf3d1b9a5 llvm::StdThreadPool::grow(int) + 261
11 flang-23 0x0000557bf3d1c994
12 flang-23 0x0000557bf3d1a5a1
13 flang-23 0x0000557bfa690ba0
14 flang-23 0x0000557bf3c26ef8
15 flang-23 0x0000557bfa48d520 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) + 1632
16 flang-23 0x0000557bfa491201 mlir::PassManager::run(mlir::Operation*) + 1409
17 flang-23 0x0000557bf3c1a5c2 Fortran::frontend::CodeGenAction::beginSourceFileAction() + 8306
18 flang-23 0x0000557bf3c12d03 Fortran::frontend::FrontendAction::beginSourceFile(Fortran::frontend::CompilerInstance&, Fortran::frontend::FrontendInputFile const&) + 355
19 flang-23 0x0000557bf3bfc532 Fortran::frontend::CompilerInstance::executeAction(Fortran::frontend::FrontendAction&) + 994
20 flang-23 0x0000557bf3c17d6e Fortran::frontend::executeCompilerInvocation(Fortran::frontend::CompilerInstance*) + 4126
21 flang-23 0x0000557bf29a4ca0 fc1_main(llvm::ArrayRef, char const*) + 2656
22 flang-23 0x0000557bf29a3658 main + 4392
23 libc.so.6 0x0000147126c295d0
24 libc.so.6 0x0000147126c29680 __libc_start_main + 128
25 flang-23 0x0000557bf28f4ed5 _start + 37
flang-23: error: unable to execute command: Aborted (core dumped)
flang-23: error: flang frontend command failed with exit code -2 (use -v to see invocation)
flang version 23.1.0-rc3 (https://github.com/llvm/llvm-project 7196f931f212fc7c406066b2628a0ff4ea0ee344)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/hmenke/LLVM-23.1.0-rc3-Linux-X64/bin
flang-23: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING CRASH REPRODUCER FILES TO THE BUG REPORT:
flang-23: note: diagnostic msg: /tmp/repro-99e0d3
flang-23: note: diagnostic msg: /tmp/repro-99e0d3.sh
flang-23: note: diagnostic msg:

********************

=== workaround: -mmlir -mlir-disable-threading avoids the crash under the same limit ===
compiled OK: /tmp/repro.o
```

Contributor guide

Open the contributing guide

Research direction

Run the supplied Bash reproducer with the RLIMIT_NPROC setting, then compare it with -mmlir -mlir-disable-threading. Start at llvm::StdThreadPool::grow and llvm_execute_on_thread_impl in the stack trace, followed through MLIR's OpToOpPassAdaptor; done means flang no longer hard-crashes when thread creation fails and reports a handled failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
fortran
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.