NVIDIA / NVIDIA/TransformerEngine
Calling backward(retain_graph=True) multiple times with TE Layer does not work
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 831
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 65
Description
import torch
from transformer_engine.pytorch import Linear as TELinear, fp8_autocast
# m = torch.nn.Linear(16, 16).to("cuda") # This works
m = TELinear(16, 16)
x = torch.randn(16, 16, device='cuda')
with fp8_autocast(True):
o = m(x).sum()
o.backward(retain_graph=True)
# this fails with
# AssertionError: FP8 execution requires 2D input matrices with height divisible by 8 and width divisible by 16, but got tensor with dims=[0]
# looks like TELinear.backward mutates the context object such that it is not reusable.
o.backward()
This would be useful to support benchmarking just the backward pass.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided Python reproduction using transformer_engine.pytorch.Linear, fp8_autocast, and repeated backward(retain_graph=True). Inspect the TELinear backward path and its saved context, focusing on why the second backward receives a tensor with dims=[0]. Done means the reproduction completes both backward calls without the FP8 shape assertion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100