avast / avast/retdec

Constant Decompiler Wrong

Open
#1,204 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Hello, I'm using retdec to lift a libssl binary from openssl project by
```
retdec-decompiler libssl.so
```

and I found that the constant value `0x2f8` in function `init_sig_algs` is translated in wrong way.

```asm
libssl.so[0x59480] <+0>: pushq %rbp
libssl.so[0x59481] <+1>: movq %rsp, %rbp
libssl.so[0x59484] <+4>: subq $0x10, %rsp
libssl.so[0x59488] <+8>: movq %rdi, -0x8(%rbp)
libssl.so[0x5948c] <+12>: movl %esi, -0xc(%rbp)
libssl.so[0x5948f] <+15>: movq -0x8(%rbp), %rax
libssl.so[0x59493] <+19>: movq 0xa8(%rax), %rax
libssl.so[0x5949a] <+26>: movq 0x2e8(%rax), %rdi
libssl.so[0x594a1] <+33>: leaq 0x40978(%rip), %rsi
libssl.so[0x594a8] <+40>: movl $0x47f, %edx
libssl.so[0x594ad] <+45>: callq 0x1c950
libssl.so[0x594b2] <+50>: movq -0x8(%rbp), %rax
libssl.so[0x594b6] <+54>: movq 0xa8(%rax), %rax
libssl.so[0x594bd] <+61>: movq $0x0, 0x2e8(%rax)
libssl.so[0x594c8] <+72>: movq -0x8(%rbp), %rax
libssl.so[0x594cc] <+76>: movq 0xa8(%rax), %rax
libssl.so[0x594d3] <+83>: movq $0x0, 0x2f8(%rax) ; **here**
libssl.so[0x594de] <+94>: movl $0x1, %eax
libssl.so[0x594e3] <+99>: addq $0x10, %rsp
libssl.so[0x594e7] <+103>: popq
libssl.so[0x594e8] <+104>: retq
```

The value 0x2f8 should be translated to 760, but I see strange value `0x510084000808890`
```c
int64_t init_sig_algs_cert(int64_t a1, int64_t a2) {
int64_t * v1 = (int64_t *)(a1 + 168); // 0x59403
int64_t v2 = *(int64_t *)(*v1 + (int64_t)&g300); // 0x5940a
int64_t v3; // 0x593f0
CRYPTO_free(v2, "ssl/statem/extensions.c", (int64_t)&g525, v3, v3, v3);
*(int64_t *)(*v1 + (int64_t)&g300) = 0;
*(int64_t *)(*v1 + (int64_t)&g309) = 0; // g309 = 0x510084000808890
return 1;
}
```
I observe the LLVM IR and find that the constant value in IR is already wrong:

```llvm
@global_var_2f8 = global i64 -4539487412014271340
%10 = load i64, i64* %1, align 8
%11 = add i64 %10, ptrtoint (i64* @global_var_2f8 to i64)
%12 = inttoptr i64 %11 to i64*
store i64 0, i64* %12, align 8
```

How can I fix that? Any suggestions will be appreciated!

[libssl.so.zip](https://github.com/avast/retdec/files/15343489/libssl.so.zip)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the report with retdec-decompiler on the supplied libssl.so.zip and inspect the LLVM IR around global_var_2f8. Compare the generated IR and C output with the assembly for init_sig_algs, especially the 0x2f8 memory offset; done means the offset is represented as 760 and the decompiled store targets the correct field.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.