Indexing failed on a simple load.

Open
#5,200 9 comments 0 reactions 1 assignee View on GitHub

@naoyam is already working on this.

Since Oct 24, 2025.

Assessment

This issue has not been assessed yet.

Description

MoE Inference nvFP4
import torch
from nvfuser_direct import FusionDefinition, DataType
m = 1024
k = 1024 * 16

def nvfuser_fusion_id1(fd : FusionDefinition) -> None :
  T0 = fd.define_tensor(shape=[-1, -1], contiguity=[True, True], dtype=DataType.Float, is_cpu=False, stride_order=[1, 0])
  m_size = fd.ops.size(T0, 0);
  k_size = fd.ops.size(T0, 1);
  k_tile_size = fd.ops.div(k_size, 16);
  reshaped_mat1 = fd.ops.reshape(T0, [m, k//16, 16])
  scale1 = fd.ops.max(reshaped_mat1, 2)
  broadcast_scale1 = fd.ops.broadcast(scale1, [False, False, True])
  reshaped_scaled_mat1 = fd.ops.div(reshaped_mat1, broadcast_scale1)
  scaled_mat1 = fd.ops.reshape(reshaped_scaled_mat1, [m, k])
  #t_scaled_mat1 = fd.ops.permute(scaled_mat1, [1, 0])
  fp4_mat1 = fd.ops.cast(scaled_mat1, DataType.Float4_e2m1fn)
  fp4_scale1 = fd.ops.cast(scale1, DataType.Float4_e2m1fn)
  output = fd.ops.matmul(fp4_scale1, fp4_mat1)
  fd.add_output(output)

with FusionDefinition() as fd:
  nvfuser_fusion_id1(fd)

lookup = torch.randn(m, k, dtype=torch.float, device='cuda:0')
o = fd.execute([lookup])[0]
print(o)

Reverted to original example.

Indexing failed on this one. Note the the failure doesn't repro with id_model indexer.

The fusion looked pretty innocent
It's failing at the second segment on the load instruction.

Error from segmentation group 3:  INTERNAL ASSERT FAILED at /opt/pytorch/nvfuser/csrc/index_compute.cpp:1983, please report a bug with repro script to NVFuser at https://github.com/NVIDIA/Fuser/issues.
Expected index_map.find(alloc_dom[i]) != index_map.end() . Couldn't find allocation mapping for T14_l_float[iblockIdx.x123{( ceilDiv(( ceilDiv(256, blockDim.x) ), 2) )}, iblockIdx.y125{1024}, iUS126{1}, iS124{2}, ithreadIdx.x122{blockDim.x}, iS120{4}] ca_pos( 3 ) dim: 1 id: iS77{1024}, loops:  iblockIdx.x88{( ceilDiv(( ceilDiv(4096, blockDim.x) ), 2) )} iblockIdx.y90{1024} iUS91{1} iS124{2} ithreadIdx.x122{blockDim.x} iS120{4}
Exception raised from getNonGlobalConsumerStridedIndices at /opt/pytorch/nvfuser/csrc/index_compute.cpp:1983 (most recent call first):
frame #0: nvfuser::nvfCheckFail(char const*, char const*, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x110 (0xfb2f01f98650 in /opt/pytorch/nvfuser/nvfuser_direct/../build/libnvfuser_codegen.so)
Dominant language
C++
Stars
404
Forks
84
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/Fuser

All issues in NVIDIA/Fuser

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.