avast / avast/retdec

Segmentation fault in llvmir2hll due to insufficient stack space

Open
#47 19 comments 0 reactions 0 assignees View on GitHub
bug C-llvmir2hll P-run T-memory
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Hi. Running retdec under Windows Server 2016, getting this:
```
...
-> converting function_7ca104() ( 2052.13s )
-> converting entry_point() ( 2052.13s )
Running phase: removing functions prefixed with [__decompiler_undefined_function_] ( 2053.03s )
decompile.sh: line 947: 5356 Segmentation fault $LLVMIR2HLL "${LLVMIR2HLL_PARAMS[@]}"
Error: Decompilation of file '/c/retdec/bin/myfile.c.backend.bc' failed
```
myfile.exe can be downloaded here: https://www.dropbox.com/s/jm7xnxtto3r4jxf/myfile.zip?dl=0

Please, help!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure on Windows Server 2016 with the supplied myfile.exe and inspect the llvmir2hll invocation in decompile.sh, especially the generated myfile.c.backend.bc. Determine what exhausts the available stack during conversion and verify that decompilation completes without the segmentation fault.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.