EnzymeAD / EnzymeAD/Enzyme-JAX

No proper stack trace for enzymexla-mlir-opt

Open
#2,185 0 comments 0 reactions 0 assignees View on GitHub
bazel
Dominant language
MLIR
Stars
131
Forks
53
Avg merge
1d 10h
Merged PRs (30d)
193

Description

When running into a crash while using enzymexla-mlir-opt the strack trace misses symbols (below is an example output).
I was suggested to create an issue on this.

I assume it has to do with bazel, also in my spmd mlir project where I switched from CMake to Bazel, there are no longer symbols in my stack trace.

I build with -c dbg.
I also tried with manually setting LLVMSymbolizer, I also tried building it with our bazel and specify the path to this very llvm-symbolizer. But nothing had an effect.
I also talked to other people using enzyme-jax, all report the same behavior.

However, when running the same command in gdb, one obtians a proper stack trace.

How I built enzymexlamlir-opt:
`bazel --output_base=/tmp/$USER/bazelOut build -c dbg --spawn_strategy=local @enzyme_jax//:enzymexlamlir-opt`

How to reproduce:
1. add assert(false) to the begin of a pass
2. run this pass with enzymexlamlir-opt on a dummy mlir file

Possible solution ideas:
1. Maybe some LLVM defines need to be set
2. Maybe some of the dependencies in the bazel build files are missing or only indirectly linked etc.

Example output:

```
$ spmd/bazel-bin/external/enzyme_jax/enzymexlamlir-opt -remove-duplicate-func-def spmd/externals/data-race-detection-benchmark-suite/MBB/level32/COLL/Correct-mpi_iallreduce-mpi_iallreduce-001.mlir
enzymexlamlir-opt: external/enzyme_jax/src/enzyme_ad/jax/Passes/RemoveDuplicateFuncDef.cpp:70: virtual void {anonymous}::RemoveDuplicateFuncDefPass::runOnOperation(): Assertion `false && "testing"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
Stack dump:
0. Program arguments: spmd/bazel-bin/external/enzyme_jax/enzymexlamlir-opt -remove-duplicate-func-def spmd/externals/data-race-detection-benchmark-suite/MBB/level32/COLL/Correct-mpi_iallreduce-mpi_iallreduce-001.mlir
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0 enzymexlamlir-opt 0x000000000f40b590
1 enzymexlamlir-opt 0x000000000f40b9ba
2 enzymexlamlir-opt 0x000000000f408df2
3 enzymexlamlir-opt 0x000000000f40b045
4 libc.so.6 0x000014c1a163ebf0
5 libc.so.6 0x000014c1a168bedc
6 libc.so.6 0x000014c1a163eb46 raise + 22
7 libc.so.6 0x000014c1a1628833 abort + 211
8 libc.so.6 0x000014c1a162875b
9 libc.so.6 0x000014c1a1637886
10 enzymexlamlir-opt 0x000000000165ebde
11 enzymexlamlir-opt 0x000000000a8a5a91
12 enzymexlamlir-opt 0x000000000a8ae8e6
13 enzymexlamlir-opt 0x0000000006131308
14 enzymexlamlir-opt 0x000000000a8b4e9d
15 enzymexlamlir-opt 0x000000000a8a6132
16 enzymexlamlir-opt 0x000000000a8a688c
17 enzymexlamlir-opt 0x000000000a8ad0c7
18 enzymexlamlir-opt 0x000000000a8ac8c0
19 enzymexlamlir-opt 0x00000000088a8a9d
20 enzymexlamlir-opt 0x00000000088a9586
21 enzymexlamlir-opt 0x00000000088a9cfb
22 enzymexlamlir-opt 0x00000000088ab2f8
23 enzymexlamlir-opt 0x000000000abe2ba5
24 enzymexlamlir-opt 0x000000000abe20c2
25 enzymexlamlir-opt 0x00000000088aa014
26 enzymexlamlir-opt 0x00000000088aa40e
27 enzymexlamlir-opt 0x00000000088aa69f
28 enzymexlamlir-opt 0x0000000000b90473
29 libc.so.6 0x000014c1a16295d0
30 libc.so.6 0x000014c1a1629680 __libc_start_main + 128
31 enzymexlamlir-opt 0x0000000000b29165
Aborted (core dumped)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.