NVIDIA/TransformerEngine
View on GitHubA library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
- Stars
- 3.5k
- Forks
- 831
- Open beginner issues
- 0
- Indexed issues
- 163
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 65
- Dominant language
- Python
- License
- Apache-2.0
- Last GitHub push
- Sep 19, 2026
- Latest indexed
- Sep 19, 2026
- Contributing guide
- Contributing guide
- Code of conduct
- No code of conduct
- Beginner labels
- No beginner labels indexed
2 beginner-friendly issues open
Loading issues
-
attention
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
NVIDIA/TransformerEngine#3481 · 4 comments ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
NVIDIA/TransformerEngine#2189 · 7 comments · 5 reactions ·