avast / avast/retdec

"Error: Decompilation to LLVM IR failed" During Conditional branch optimization

Open
#552 12 comments 0 reactions 0 assignees View on GitHub
bug C-bin2llvmir P-run T-memory
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Command Line Output:
```
C:\[REDACTED]>py retdec-decompiler.py --no-memory-limit C:\[REDACTED]
##### Checking if file is a Mach-O Universal static library...

##### Checking if file is an archive...
RUN: C:\[REDACTED]\retdec-ar-extractor C:\[REDACTED] --arch-magic
Not an archive, going to the next step.

##### Gathering file information...
RUN: C:\[REDACTED]\retdec-fileinfo -c C:\[REDACTED] --similarity C:\[REDACTED] --no-hashes=all --crypto C:\[REDACTED]
Input file : C:\[REDACTED]
File format : ELF
File class : 32-bit
File type : DLL
Architecture : ARM
Endianness : Little endian
Detected tool : gold (1.11) (linker), .note section heuristic
Detected tool : GCC (4.9) (compiler), .comment section heuristic
Detected tool : GCC (4.8) (compiler), .comment section heuristic
Detected tool : GCC (4.4.3) (compiler), .comment section heuristic
Original language : C++

##### Trying to unpack C:\[REDACTED] into C:\[REDACTED]-unpacked.tmp by using generic unpacker...
RUN: C:\[REDACTED]\retdec-unpacker C:\[REDACTED] -o C:\[REDACTED]-unpacked.tmp
No matching plugins found for 'gold 1.11'
No matching plugins found for 'GCC 4.9'
No matching plugins found for 'GCC 4.8'
No matching plugins found for 'GCC 4.4.3'
##### Unpacking by using generic unpacker: nothing to do
##### 'upx' not available: nothing to do

##### Decompiling C:\[REDACTED] into C:\[REDACTED].c.backend.bc...
RUN: C:\[REDACTED]\retdec-bin2llvmir -provider-init -decoder -verify -main-detection -idioms-libgcc -inst-opt -register -cond-branch-opt -syscalls -stack -constants -param-return -local-vars -inst-opt -simple-types -generate-dsm -remove-asm-instrs -class-hierarchy -select-fncs -unreachable-funcs -inst-opt -value-protect -instcombine -tbaa -targetlibinfo -basicaa -domtree -simplifycfg -domtree -early-cse -lower-expect -targetlibinfo -tbaa -basicaa -globalopt -mem2reg -instcombine -simplifycfg -basiccg -domtree -early-cse -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -instcombine -scalar-evolution -loop-simplifycfg -loop-simplify -aa -loop-accesses -loop-load-elim -lcssa -indvars -loop-idiom -loop-deletion -memdep -gvn -memdep -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -dce -bdce -adce -die -simplifycfg -instcombine -strip-dead-prototypes -globaldce -constmerge -constprop -instnamer -domtree -instcombine -instcombine -tbaa -targetlibinfo -basicaa -domtree -simplifycfg -domtree -early-cse -lower-expect -targetlibinfo -tbaa -basicaa -globalopt -mem2reg -instcombine -simplifycfg -basiccg -domtree -early-cse -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -instcombine -scalar-evolution -loop-simplifycfg -loop-simplify -aa -loop-accesses -loop-load-elim -lcssa -indvars -loop-idiom -loop-deletion -memdep -gvn -memdep -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -dce -bdce -adce -die -simplifycfg -instcombine -strip-dead-prototypes -globaldce -constmerge -constprop -instnamer -domtree -instcombine -simple-types -stack-ptr-op-remove -inst-opt -idioms -global-to-local -dead-global-assign -instcombine -phi2seq -value-protect -disable-inlining -disable-simplify-libcalls -config-path C:\[REDACTED].c.json -o C:\[REDACTED].c.backend.bc
Running phase: Initialization ( 0.00s )
Running phase: LLVM ( 0.01s )
Running phase: Providers initialization ( 0.01s )
Running phase: Input binary to LLVM IR decoding ( 1.08s )
Running phase: LLVM ( 258.01s )
Running phase: Main function identification optimization ( 260.14s )
Running phase: Libgcc idioms optimization ( 260.17s )
Running phase: LLVM instruction optimization ( 261.18s )
Running phase: Assembly register optimization ( 262.11s )
Running phase: Conditional branch optimization ( 262.11s )
Wrote crash dump file "C:\[REDACTED]\AppData\Local\Temp\retdec-bin2llvmir.exe-2c8ddd.dmp"
0x00007FF9F17A9149 (0x00007FF6D97B3E77 0x000000EB1FF6E038 0x0000000000000100 0x00007FF9F52DD997), RaiseException() + 0x69 bytes(s)
0x00007FF9EEE33361 (0x000002E300000000 0x000000EB1FF6F200 0x0000000000000000 0x0000000000000000), _is_exception_typeof() + 0x1081 bytes(s)
0x00007FF9F5373BD6 (0x000002F69519B9B0 0x000002F69512A750 0x000002E476544B20 0x000002E4765298A0), RtlCaptureContext() + 0x566 bytes(s)
0x00007FF6D9025911 (0x000002E4764CDD40 0x000002F694F66D80 0x000002E4D8948CE0 0x000002E476544A80)
0x00007FF6D902A309 (0x000002E30CE934A8 0x000002E4739B8770 0x0000000000000000 0x000002F2F62742B0)
0x00007FF6D902B4D8 (0x000002F2F6298188 0x000002E464B6B2A0 0x000002E478349C98 0x000002E464B6AD60)
0x00007FF6D902BA5C (0x0000000000000000 0x0000000000000000 0x000002E3055B7FB0 0x000000EB1FF6F810)
0x00007FF6D900972F (0x00007FF6D9C87DC0 0x0000000000000000 0x0000000000000001 0x0000000000000000)
0x00007FF6D96F1F6F (0x000002E300000011 0x000002E305652300 0x0000000000000000 0x0000000000000000)
0x00007FF6D96F1660 (0x000002E306E73A10 0x000000EB1FF6FA50 0x000002E3055B63F0 0x000002E306E73A10)
0x00007FF6D8F67B2D (0x0000000000000404 0x00007FF6D9793339 0x00007FF9F244B570 0x0000000000000000)
0x00007FF6D8F6AEB2 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FF6D9793AFD (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FF9F25881F4 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FF9F533A251 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), RtlUserThreadStart() + 0x21 bytes(s)
Error: Decompilation to LLVM IR failed
```

My dump file:
[retdec-bin2llvmir.exe-2c8ddd.dmp.zip](https://github.com/avast/retdec/files/3092766/retdec-bin2llvmir.exe-2c8ddd.dmp.zip)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the retdec-bin2llvmir crash dump and the command output, focusing on the Conditional branch optimization phase for the reported 32-bit ARM ELF DLL. Reproduce the failure if the original input is available, then inspect the conditional branch optimization entry point and relevant tests. Done means the input completes this phase without the decompilation-to-LLVM-IR error and produces the requested output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.