[xpu-alignment] XPU exponential_ can return exact zero for float16 at large finite lambda
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 107
Description
## Upstream source
- PyTorch PR: https://github.com/pytorch/pytorch/pull/195077 (`Fix exponential_ underflowing to exactly 0 for float16`, currently open and unmerged).
- The upstream change adds `c10::cast_no_underflow` and updates the CPU/CUDA kernels, but does not modify the XPU kernel in this repository.
## Observed XPU behavior
For a finite `lambda`, XPU `Tensor.exponential_()` can narrow a positive intermediate value to an exact zero when the output dtype is `float16`.
In the alignment workflow, with 24 million XPU samples per configuration:
```text
dtype lambda exact zeros minimum positive
float16 1 0 4.883e-04
float16 1024 0 4.768e-07
float16 16384 11655 5.960e-08
float16 65504 47262 5.960e-08
bfloat16 all above 0
float32 all above 0
```
An independent local run on an Intel Data Center GPU Max 1100 reproduced the issue. The original 24-million-sample test produced 11,931 zeros at `lambda=16384` and 46,840 zeros at `lambda=65504`. A reduced one-million-sample test also reproduced it across three fixed seeds, while `lambda=1024`, `bfloat16`, and `float32` remained unaffected.
No tested sample was negative or non-finite.
## Root cause
`ExponentialFunctor` in
`src/ATen/native/xpu/sycl/DistributionTemplates.h` calculates the sample in
`accscalar_t`. `DistributionElementwiseKernelFunctor` then writes that value to
a `scalar_t*` output:
```cpp
*out = transform_func_(static_cast((&rand.x)[i]));
```
For sufficiently large finite `lambda`, the positive accumulate-type result is
smaller than the minimum positive `float16` subnormal and the implicit
`accscalar_t -> scalar_t` conversion rounds it to exactly zero.
This is distinct from #4020, which tracks distribution statistical/moment test
failures rather than deterministic float16 narrowing at the XPU kernel output.
## Reproducer
```python
import torch
torch.xpu.manual_seed(0)
for lambd in (1024.0, 16384.0, 65504.0):
x = torch.empty(1_000_000, dtype=torch.float16, device="xpu")
x.exponential_(lambd)
torch.xpu.synchronize()
print(lambd, int((x == 0).sum().item()), x[x > 0].min().item())
```
Expected for each finite `lambda`: zero exact-zero samples. `lambda=inf` is a
separate valid boundary case and should continue to produce exact zero.
## Suggested fix
Use `c10::cast_no_underflow` at the XPU output-narrowing site after the helper
from PyTorch #195077 becomes available. If the XPU fix must land first, use a
minimal local guard at the same narrowing site and replace it with the common
helper later.
Regression coverage should use `lambda >= 16384` for the current XPU kernel;
`lambda=1024` did not reproduce the XPU failure. The fix should preserve the
random stream, unaffected dtypes, and `exponential_(inf) == 0`.
Contributor guide
Assessment
This issue has not been assessed yet.