x87 instructions do not produce indefinite result on invalid operand
- Dominant language
- C++
- Stars
- 8k
- Forks
- 351
- Avg merge
- 12h 31m
- Merged PRs (30d)
- 102
Description
On an operand that is in an unsupported format (Pseudo-NaN, Pseudo-Infinity, Unnormal), FEX's F80Fallback layer either fails to detect the invalid condition or never returns indefinite value (QNaN or packed-BCD).
Affects (at least):
- `F2XM1`, `FYL2X`, `FYL2XP1`, `FPATAN` , `FPTAN`, `FSIN`, `FCOS`, `FSINCOS`, `FBSTP`.
FADD/FSUB/FMUL and so on seem to be unaffected as they use Berkeley SoftFloat primitives.
Contributor guide
Research direction
Locate FEX's F80Fallback implementation and trace the handling of the listed x87 instructions for Pseudo-NaN, Pseudo-Infinity, and Unnormal operands. Compare the affected paths with the Berkeley SoftFloat primitives, then verify that each invalid case returns the required indefinite value: QNaN or packed-BCD.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100