NVIDIA / NVIDIA/TransformerEngine

[BUG] NVFP4 fails on RTX 5070 Laptop (SM120): CUDA Error: no kernel image is available for execution on the device

Open
#3,217 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.5k
Forks
831
Avg merge
3d 11h
Merged PRs (30d)
65

Description

Environment

  • GPU: NVIDIA GeForce RTX 5070 Laptop GPU
  • Compute Capability: SM120 (12.0)
  • CUDA Toolkit: 13.3
  • Python: 3.13
  • WSL2 Ubuntu 24.04
  • TransformerEngine built from source
  • Local commit: 6bf7790d
  • Upstream main: 9d92fa05

Problem

TransformerEngine builds successfully, but NVFP4 training fails immediately with:

RuntimeError:
/home/.../transformer_engine/common/recipe/current_scaling.cu:77

launch_amax_kernel

CUDA Error: no kernel image is available for execution on the device

The failure happens immediately after:

zero_amax_kernel<<<1,1,0,stream>>>(...);
NVTE_CHECK_CUDA(cudaGetLastError());

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the NVFP4 training failure in the reported CUDA 13.3, SM120 environment, then inspect transformer_engine/common/recipe/current_scaling.cu around line 77, including launch_amax_kernel and the zero_amax_kernel launch. Compare the failing kernel launch with the build's supported GPU architecture settings; done means NVFP4 training launches successfully on the RTX 5070 Laptop without the missing-kernel-image error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.