DynamoRIO / DynamoRIO/dynamorio

DR decoder inflates xmm/ymm dest for some x86 gather instrs

Open
#7,887 0 comments 0 reactions 0 assignees View on GitHub
Component-IR OpSys-x86
Dominant language
C
Stars
3.2k
Forks
629
Avg merge
2d 15h
Merged PRs (30d)
31

Description

x86 gather instrs such as the following that have index element size > data element size are treated by the DR decoder in a way that's different from the x86 manual and other tools.

```
vpgatherqd/vgatherqps xmm0 {k1}, [xax + ymm1 * 4]
vpgatherqd/vgatherqps ymm0 {k1}, [xax + zmm1 * 4]
```

Here the xmm0 or ymm0 respectively is inflated to ymm0 or zmm0 size by DR.

E.g.,

```
INSTR_CREATE_vgatherqps_mask(
dc,
opnd_create_reg(DR_REG_XMM0),
opnd_create_reg(DR_REG_K1),
opnd_create_base_disp(DR_REG_RAX, DR_REG_YMM1, 4, 0, OPSZ_4));
```

disassembles into

```
vgatherqps {%k1} (%rax,%ymm1,4)[4byte] -> %xmm0 %k1
```

but if it is encoded and then decoded, we get:

```
vgatherqps {%k1} (%rax,%ymm1,4)[4byte] -> %ymm0 %k1
```

where the xmm0 is inflated to ymm0.

This is still functionally the same because such gathers (with index size > data size), do not write to the full ymm, but only its first half (which is the xmm) and zero out the rest.

vgatherqps uses OPSZ_16_vex32_evex64 which is 16B, 32B, or 64B based on the prefix (PREFIX_VEX_L, PREFIX_EVEX_LL). As I understand the ymm0 makes DR use 32B vector length for the instr. So the following then "inflates" the dest into a ymm too.

Xref
https://github.com/DynamoRIO/dynamorio/blob/af5c1cf24fed3354cdfea3c2d619b3c3bf8c3dfd/core/ir/x86/decode.c#L1545

https://github.com/DynamoRIO/dynamorio/blob/af5c1cf24fed3354cdfea3c2d619b3c3bf8c3dfd/core/ir/x86/decode.c#L294

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.