Some loops are erroneously turned into infinite loops
- Dominant language
- C++
- Stars
- 8.6k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
When this kind of source code (loop with a small body) is compiled in 32-bit with gcc, RetDec fails to properly decompile it. In fact, this source code :
~~~C
int myfunc(unsigned char* S) {
for(int i = 0; i < 256; i++)
S[i] = i;
printf("Supposed not to be removed.");
return 0;
}
~~~
becomes :
~~~C
int32_t myfunc(int32_t * a1) {
__x86_get_pc_thunk_ax(g2, 0);
char v1 = 0;
while (true) {
*(char *)((int32_t)v1 + (int32_t)a1) = v1;
v1++;
}
}
~~~
RetDec properly decompiles it when it's compiled in 64-bit though :
~~~C
int64_t myfunc(int64_t * a1) {
int32_t v1 = 0;
for (int64_t i = 0; i < 256; i++) {
*(char *)(i + (int64_t)a1) = (char)v1;
v1++;
}
printf("Supposed not to be removed.");
return 0;
}
~~~
Three passes seem to cause this issue : *Instruction Combine* (-instcombine), *Global Value Numbering* (-gvn) and *Early Cse* (-early-cse). In that case they tend to transform a "br \ \ \" into a "br false \ \", as my IR dumps show, resulting in a "while true". The code that follow is then considered unreachable and removed. When I disable these passes in RetDec, the decompilation outputs the expected C code that is just like the 64-bit example above.
I also tried with llvm-8 (on your new branch), doesn't work either.
This is probably due to an upstream problem.
However, this problem doesn't occur when the binary is compiled with clang.
Note : This is not gcc's fault since the generated assembly doesn't represent an infinite loop, and the binary is working as expected.
Thank you for your time !
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the 32-bit GCC case and inspect the IR around the named Instruction Combine, Global Value Numbering, and Early CSE passes. Done means the decompiler preserves the finite loop and does not remove the following printf and return; the payload names no source files or tests to inspect.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp
- Domain
- compilers, reverse-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100