torch.exp2 second derivative overflows to inf at 1024.0 although the float64 value is finite
- Dominant language
- Python
- Stars
- 103k
- Forks
- 29.5k
- PR merge metrics
- PR metrics pending
Description
### 🐛 Describe the bug
`torch.autograd` returns `inf` for the second derivative of `torch.exp2` at a `float64` input where the mathematically correct second derivative is finite and representable in `float64`.
The second derivative of `exp2(x)` is `(ln(2) ** 2) * 2^x`. For `x = 1024.0`, the true second derivative is approximately `8.637070847446594e+307`, which is below the maximum finite `float64` value. However, PyTorch returns `inf`.
This issue reproduces on CPU, so CUDA availability does not affect the result.
```python
import torch
torch.set_default_dtype(torch.float64)
x = torch.tensor(1024.0, requires_grad=True)
y = torch.exp2(x)
g1, = torch.autograd.grad(y, x, create_graph=True)
g2, = torch.autograd.grad(g1, x)
actual = g2
expected = torch.tensor(8.637070847446594e+307, dtype=torch.float64)
print("actual:")
print(actual)
print("expected:")
print(expected)
```
### Actual vs expected result
```text
actual:
tensor(inf)
expected:
tensor(8.6371e+307)
```
The discrepancy occurs in the second-order backward pass. Although the forward value `torch.exp2(1024.0)` overflows, the mathematical second derivative `(ln(2) ** 2) * 2^1024` is still finite in `float64`, so the second derivative should not be `inf`.
### Versions
```text
PyTorch version: 2.12.0+cu130
Is debug build: False
CUDA used to build PyTorch: 13.0
ROCM used to build PyTorch: N/A
OS: Ubuntu 24.04.3 LTS (x86_64)
Python version: 3.13.13 | packaged by Anaconda, Inc.
Python platform: Linux x86_64
Is CUDA available: False
GPU models:
GPU 0: NVIDIA RTX 6000 Ada Generation
GPU 1: NVIDIA RTX 6000 Ada Generation
Nvidia driver version: 570.211.01
CPU:
Model name: AMD Ryzen Threadripper PRO 7985WX 64-Cores
CPU(s): 128
Versions of relevant libraries:
[pip3] numpy==2.4.6
[pip3] torch==2.12.0
[pip3] triton==3.7.0
```
Contributor guide
Research direction
Start by running the provided CPU reproduction with float64 and inspect the torch.exp2 autograd implementation, focusing on the second-order backward pass. The issue names no source file or test path, so trace the operation from its Python entry point into the relevant implementation. Done means the second derivative at 1024.0 is finite and matches the expected value, with regression coverage for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100