avast / avast/retdec

x87: fist and frndint doesn't respect rounding mode

Open
#884 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Subject: https://github.com/rfalke/decompiler-subjects/blob/master/from_holdec/i386_x87_fpu/ia32_elf/subject.exe
Output:
```
int32_t FIST(void) {
// 0x804a6e0
puts("FIST:\n RC=00: to nearest\n RC=01: down to -inf\n RC=10: up to +inf\n RC=11: to zero\n");
printf(" FISTT(%f): should=12 is=%d\n", 12.7, 12);
printf(" FIST(%f,RC=00): should=13 is=%d\n", 12.7, 12);
printf(" FIST(%f,RC=01): should=12 is=%d\n", 12.7, 12);
printf(" FIST(%f,RC=10): should=13 is=%d\n", 12.7, 12);
printf(" FIST(%f,RC=11): should=12 is=%d\n", 12.7, 12);
printf(" FISTT(%f): should=-7 is=%d\n", -7.1, -7);
printf(" FIST(%f,RC=00): should=-7 is=%d\n", -7.1, -7);
printf(" FIST(%f,RC=01): should=-8 is=%d\n", -7.1, -7);
printf(" FIST(%f,RC=10): should=-7 is=%d\n", -7.1, -7);
printf(" FIST(%f,RC=11): should=-7 is=%d\n", -7.1, -7);
return 0;
}
```

Some of these 12 and -7 should be other numbers.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the behavior with the subject executable linked in the issue and inspect the generated FIST output for each x87 rounding-control mode. Compare FIST and frndint results with the expected values shown in the report; done means the affected rounding modes produce the documented integer results for positive and negative inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.