avast / avast/retdec

x87: empty stack slot should not be infinity

Open
#890 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Subject: https://github.com/rfalke/decompiler-subjects/blob/master/from_holdec/i386_x87_fpu/ia32_elf/subject.exe
Output:
```
int32_t FXAM(int32_t a1) {
int32_t v1 = __asm_fxam((float80_t)(float64_t)(int64_t)a1); // 0x804b347
__asm_wait();
printf("FXAM(para): should=0x0400 is=0x%04x\n", v1 & 0x4500);
int32_t v2 = __asm_fxam(3.141592653589793116L); // 0x804b368
__asm_wait();
printf("FXAM(pi): should=0x0400 is=0x%04x\n", v2 & 0x4500);
int32_t v3 = __asm_fxam(0.0L); // 0x804b389
__asm_wait();
printf("FXAM(0.0): should=0x4000 is=0x%04x\n", v3 & 0x4500);
int32_t v4 = __asm_fxam(NAN); // 0x804b3ae
__asm_wait();
printf("FXAM(nan): should=0x0100 is=0x%04x\n", v4 & 0x4500);
int32_t v5 = __asm_fxam(INFINITY); // 0x804b3d3
__asm_wait();
printf("FXAM(inf): should=0x0500 is=0x%04x\n", v5 & 0x4500);
int32_t v6 = __asm_fxam(INFINITY); // 0x804b3f2
__asm_wait();
printf("FXAM(empty): should=0x4100 is=0x%04x\n", v6 & 0x4500);
return 0;
}
```
There are no values in the stack for the last fxam instruction. Therefore the call `int32_t v6 = __asm_fxam(INFINITY);` has the wrong argument.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked x87 subject executable and reproduce the generated FXAM output, focusing on the final empty-stack case. Compare the emitted argument and status classification for FXAM(empty) with the expected 0x4100 result; done when an empty stack slot is no longer emitted as INFINITY.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.