FEX-Emu / FEX-Emu/FEX

Potential Decoding Failure for instructions prefixed with 0x66

Open
#5,671 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8k
Forks
351
Avg merge
12h 31m
Merged PRs (30d)
102

Description

FEX fails to decode certain instructions when they are prefixed with `0x66`.

For the affaceted instructions, the `0x66` prefix is not required and does not change their behavior. Real hardware seems to simply ignore the prefix and execute the instruction normally. However, FEX fails in the `Frontend` decoding phase.

This happens, e.g, because the lookup tables (like `H0F38TableOps`) do not have an entry for the prefixed version.

The following non-exhaustive list fails to decode with a `0x66` prefix:
- `movnti`
- `adox`
- `sha1msg1`

In contrast, instructions like `cvttss2si` and `movd` get decoded fine when prefixed with `0x66`.

Related issues: #5667 and #5670.

Contributor guide

Open the contributing guide

Research direction

Start in the Frontend decoding phase and inspect lookup tables such as H0F38TableOps, comparing prefixed and unprefixed entries. Reproduce the failure with the listed movnti, adox, and sha1msg1 instructions, then verify that a 0x66 prefix decodes them while preserving the behavior seen for unprefixed instructions.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.